2024-04-24T06:20:06.368 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-24T06:20:06.368 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi/7671165 --name rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi --owner scheduled_rishabh@teuthology --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/fsx}} -- /home/teuthworker/archive/rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi/7671165/orig.config.yaml 2024-04-24T06:20:06.428 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T06:20:06.524 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi/7671165 branch: wip-rishabh-testing-20240416.193735-5 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/fsx}} email: null first_in_suite: false job_id: '7671165' kernel: client: branch: testing flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240416.193735-5 ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: a3c8a22bd5efe18ab230a8e2d072db8f381a0411 subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: a3c8a22bd5efe18ab230a8e2d072db8f381a0411 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240416.193735-5 sha1: a3c8a22bd5efe18ab230a8e2d072db8f381a0411 owner: scheduled_rishabh@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7954 sha1: a3c8a22bd5efe18ab230a8e2d072db8f381a0411 sleep_before_teardown: 0 subset: 364/512 suite: fs suite_branch: wip-rishabh-testing-20240416.193735-5 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a3c8a22bd5efe18ab230a8e2d072db8f381a0411/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a3c8a22bd5efe18ab230a8e2d072db8f381a0411 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi005.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKj7GNK3Wtmq8arxDItaws5Fl+9upi47CoNcM8UMyv5nceA/3uz7NO6nwS5IYipSqdXOoBlY8t46ktjkKGOy78I= smithi037.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFopbXrlRFxdN2HulQsNhehpuLGUo6Q2jl3w3Fvt0lxFADrcnB34mbgljrKTNngb6CEPVB4DXGLsF1NTEtdwAa8= smithi137.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJl4eVXNbwEo5ONmUM8Y0T1gP+7S6UE6oT+9urDJzP5PYsoEmXEhTzARVOlzvelG1eOvuEqctqarOnf0ZkgHpss= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - suites/fsx.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-24_05:22:11 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-24T06:20:06.525 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a3c8a22bd5efe18ab230a8e2d072db8f381a0411/qa; will attempt to use it 2024-04-24T06:20:06.525 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a3c8a22bd5efe18ab230a8e2d072db8f381a0411/qa/tasks 2024-04-24T06:20:06.526 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-24T06:20:06.527 INFO:teuthology.task.internal:Checking packages... 2024-04-24T06:20:06.551 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'a3c8a22bd5efe18ab230a8e2d072db8f381a0411' 2024-04-24T06:20:06.551 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-24T06:20:06.551 INFO:teuthology.packaging:ref: None 2024-04-24T06:20:06.552 INFO:teuthology.packaging:tag: None 2024-04-24T06:20:06.552 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240416.193735-5 2024-04-24T06:20:06.552 INFO:teuthology.packaging:sha1: a3c8a22bd5efe18ab230a8e2d072db8f381a0411 2024-04-24T06:20:06.552 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240416.193735-5 2024-04-24T06:20:06.771 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3124.ga3c8a22b 2024-04-24T06:20:06.772 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-24T06:20:06.779 INFO:teuthology.task.internal:no buildpackages task found 2024-04-24T06:20:06.779 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-24T06:20:06.786 INFO:teuthology.task.internal:Saving configuration 2024-04-24T06:20:06.807 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-24T06:20:06.813 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T06:20:06.838 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi005.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi/7671165', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 06:13:56.896466', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKj7GNK3Wtmq8arxDItaws5Fl+9upi47CoNcM8UMyv5nceA/3uz7NO6nwS5IYipSqdXOoBlY8t46ktjkKGOy78I='} 2024-04-24T06:20:06.861 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi037.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi/7671165', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 06:13:56.895443', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFopbXrlRFxdN2HulQsNhehpuLGUo6Q2jl3w3Fvt0lxFADrcnB34mbgljrKTNngb6CEPVB4DXGLsF1NTEtdwAa8='} 2024-04-24T06:20:06.881 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi137.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi/7671165', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 06:13:56.894206', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJl4eVXNbwEo5ONmUM8Y0T1gP+7S6UE6oT+9urDJzP5PYsoEmXEhTzARVOlzvelG1eOvuEqctqarOnf0ZkgHpss='} 2024-04-24T06:20:06.881 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-24T06:20:06.889 INFO:teuthology.task.internal:roles: ubuntu@smithi005.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-24T06:20:06.890 INFO:teuthology.task.internal:roles: ubuntu@smithi037.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-24T06:20:06.890 INFO:teuthology.task.internal:roles: ubuntu@smithi137.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-24T06:20:06.890 INFO:teuthology.run_tasks:Running task console_log... 2024-04-24T06:20:06.989 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f7290d9b160>, signals=[15]) 2024-04-24T06:20:06.989 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-24T06:20:06.995 INFO:teuthology.task.internal:Opening connections... 2024-04-24T06:20:06.995 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi005.front.sepia.ceph.com 2024-04-24T06:20:06.997 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T06:20:07.076 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi037.front.sepia.ceph.com 2024-04-24T06:20:07.077 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T06:20:07.149 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi137.front.sepia.ceph.com 2024-04-24T06:20:07.150 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T06:20:07.222 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-24T06:20:07.229 DEBUG:teuthology.orchestra.run.smithi005:> uname -m 2024-04-24T06:20:07.248 INFO:teuthology.orchestra.run.smithi005.stdout:x86_64 2024-04-24T06:20:07.248 DEBUG:teuthology.orchestra.run.smithi005:> cat /etc/os-release 2024-04-24T06:20:07.306 INFO:teuthology.orchestra.run.smithi005.stdout:NAME="CentOS Stream" 2024-04-24T06:20:07.306 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION="9" 2024-04-24T06:20:07.306 INFO:teuthology.orchestra.run.smithi005.stdout:ID="centos" 2024-04-24T06:20:07.306 INFO:teuthology.orchestra.run.smithi005.stdout:ID_LIKE="rhel fedora" 2024-04-24T06:20:07.306 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION_ID="9" 2024-04-24T06:20:07.306 INFO:teuthology.orchestra.run.smithi005.stdout:PLATFORM_ID="platform:el9" 2024-04-24T06:20:07.306 INFO:teuthology.orchestra.run.smithi005.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T06:20:07.306 INFO:teuthology.orchestra.run.smithi005.stdout:ANSI_COLOR="0;31" 2024-04-24T06:20:07.306 INFO:teuthology.orchestra.run.smithi005.stdout:LOGO="fedora-logo-icon" 2024-04-24T06:20:07.306 INFO:teuthology.orchestra.run.smithi005.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T06:20:07.306 INFO:teuthology.orchestra.run.smithi005.stdout:HOME_URL="https://centos.org/" 2024-04-24T06:20:07.306 INFO:teuthology.orchestra.run.smithi005.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T06:20:07.306 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T06:20:07.307 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T06:20:07.307 INFO:teuthology.lock.ops:Updating smithi005.front.sepia.ceph.com on lock server 2024-04-24T06:20:07.324 DEBUG:teuthology.orchestra.run.smithi037:> uname -m 2024-04-24T06:20:07.342 INFO:teuthology.orchestra.run.smithi037.stdout:x86_64 2024-04-24T06:20:07.342 DEBUG:teuthology.orchestra.run.smithi037:> cat /etc/os-release 2024-04-24T06:20:07.400 INFO:teuthology.orchestra.run.smithi037.stdout:NAME="CentOS Stream" 2024-04-24T06:20:07.401 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION="9" 2024-04-24T06:20:07.401 INFO:teuthology.orchestra.run.smithi037.stdout:ID="centos" 2024-04-24T06:20:07.401 INFO:teuthology.orchestra.run.smithi037.stdout:ID_LIKE="rhel fedora" 2024-04-24T06:20:07.401 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION_ID="9" 2024-04-24T06:20:07.401 INFO:teuthology.orchestra.run.smithi037.stdout:PLATFORM_ID="platform:el9" 2024-04-24T06:20:07.401 INFO:teuthology.orchestra.run.smithi037.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T06:20:07.401 INFO:teuthology.orchestra.run.smithi037.stdout:ANSI_COLOR="0;31" 2024-04-24T06:20:07.401 INFO:teuthology.orchestra.run.smithi037.stdout:LOGO="fedora-logo-icon" 2024-04-24T06:20:07.401 INFO:teuthology.orchestra.run.smithi037.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T06:20:07.401 INFO:teuthology.orchestra.run.smithi037.stdout:HOME_URL="https://centos.org/" 2024-04-24T06:20:07.401 INFO:teuthology.orchestra.run.smithi037.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T06:20:07.401 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T06:20:07.402 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T06:20:07.402 INFO:teuthology.lock.ops:Updating smithi037.front.sepia.ceph.com on lock server 2024-04-24T06:20:07.424 DEBUG:teuthology.orchestra.run.smithi137:> uname -m 2024-04-24T06:20:07.442 INFO:teuthology.orchestra.run.smithi137.stdout:x86_64 2024-04-24T06:20:07.442 DEBUG:teuthology.orchestra.run.smithi137:> cat /etc/os-release 2024-04-24T06:20:07.499 INFO:teuthology.orchestra.run.smithi137.stdout:NAME="CentOS Stream" 2024-04-24T06:20:07.499 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION="9" 2024-04-24T06:20:07.499 INFO:teuthology.orchestra.run.smithi137.stdout:ID="centos" 2024-04-24T06:20:07.499 INFO:teuthology.orchestra.run.smithi137.stdout:ID_LIKE="rhel fedora" 2024-04-24T06:20:07.500 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION_ID="9" 2024-04-24T06:20:07.500 INFO:teuthology.orchestra.run.smithi137.stdout:PLATFORM_ID="platform:el9" 2024-04-24T06:20:07.500 INFO:teuthology.orchestra.run.smithi137.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T06:20:07.500 INFO:teuthology.orchestra.run.smithi137.stdout:ANSI_COLOR="0;31" 2024-04-24T06:20:07.500 INFO:teuthology.orchestra.run.smithi137.stdout:LOGO="fedora-logo-icon" 2024-04-24T06:20:07.500 INFO:teuthology.orchestra.run.smithi137.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T06:20:07.500 INFO:teuthology.orchestra.run.smithi137.stdout:HOME_URL="https://centos.org/" 2024-04-24T06:20:07.500 INFO:teuthology.orchestra.run.smithi137.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T06:20:07.500 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T06:20:07.500 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T06:20:07.500 INFO:teuthology.lock.ops:Updating smithi137.front.sepia.ceph.com on lock server 2024-04-24T06:20:07.524 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-24T06:20:07.534 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-24T06:20:07.540 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-24T06:20:07.540 DEBUG:teuthology.orchestra.run.smithi005:> test '!' -e /home/ubuntu/cephtest 2024-04-24T06:20:07.543 DEBUG:teuthology.orchestra.run.smithi037:> test '!' -e /home/ubuntu/cephtest 2024-04-24T06:20:07.545 DEBUG:teuthology.orchestra.run.smithi137:> test '!' -e /home/ubuntu/cephtest 2024-04-24T06:20:07.561 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-24T06:20:07.567 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-24T06:20:07.567 DEBUG:teuthology.orchestra.run.smithi005:> test -z $(ls -A /var/lib/ceph) 2024-04-24T06:20:07.600 DEBUG:teuthology.orchestra.run.smithi037:> test -z $(ls -A /var/lib/ceph) 2024-04-24T06:20:07.602 DEBUG:teuthology.orchestra.run.smithi137:> test -z $(ls -A /var/lib/ceph) 2024-04-24T06:20:07.635 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-24T06:20:07.718 INFO:teuthology.run_tasks:Running task kernel... 2024-04-24T06:20:07.733 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-24T06:20:07.733 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-04-24T06:20:07.733 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-04-24T06:20:07.734 DEBUG:teuthology.orchestra.run.smithi005:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T06:20:07.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:20:07.751 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-24T06:20:08.004 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T06:20:08.004 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-24T06:20:08.004 DEBUG:teuthology.orchestra.run.smithi005:> uname -r 2024-04-24T06:20:08.022 INFO:teuthology.orchestra.run.smithi005.stdout:5.14.0-437.el9.x86_64 2024-04-24T06:20:08.023 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T06:20:08.023 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-24T06:20:08.023 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum upgrade -y linux-firmware 2024-04-24T06:20:16.305 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - BaseOS 1.2 MB/s | 8.0 MB 00:06 2024-04-24T06:20:19.547 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - AppStream 8.9 MB/s | 19 MB 00:02 2024-04-24T06:20:25.764 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 6.2 MB 00:02 2024-04-24T06:20:33.008 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - Extras packages 2.7 kB/s | 16 kB 00:06 2024-04-24T06:20:35.499 INFO:teuthology.orchestra.run.smithi005.stdout:Extra Packages for Enterprise Linux 8.9 MB/s | 21 MB 00:02 2024-04-24T06:20:40.682 INFO:teuthology.orchestra.run.smithi005.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-24T06:20:42.871 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-04-24T06:20:42.872 INFO:teuthology.orchestra.run.smithi005.stdout:Nothing to do. 2024-04-24T06:20:42.872 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-04-24T06:20:42.980 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-24T06:20:42.980 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T06:20:43.209 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-24T06:20:43.209 DEBUG:teuthology.orchestra.run.smithi005:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-24T06:20:43.212 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi005.front.sepia.ceph.com to complete... 2024-04-24T06:20:57.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24 06:20:57 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-24T06:20:57.796 INFO:teuthology.orchestra.run.smithi005.stderr:FINISHED --2024-04-24 06:20:57-- 2024-04-24T06:20:57.796 INFO:teuthology.orchestra.run.smithi005.stderr:Total wall clock time: 15s 2024-04-24T06:20:57.796 INFO:teuthology.orchestra.run.smithi005.stderr:Downloaded: 1 files, 341M in 14s (24.0 MB/s) 2024-04-24T06:20:57.798 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-24T06:20:57.798 DEBUG:teuthology.orchestra.run.smithi005:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-24T06:20:59.976 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying... ######################################## 2024-04-24T06:21:00.098 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing... ######################################## 2024-04-24T06:21:00.202 INFO:teuthology.orchestra.run.smithi005.stdout:Updating / installing... 2024-04-24T06:21:08.494 INFO:teuthology.orchestra.run.smithi005.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-24T06:21:48.809 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi005.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-24T06:21:48.809 DEBUG:teuthology.orchestra.run.smithi005:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-24T06:21:48.883 INFO:teuthology.orchestra.run.smithi005.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-24T06:21:48.883 INFO:teuthology.orchestra.run.smithi005.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-24T06:21:48.883 INFO:teuthology.orchestra.run.smithi005.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-24T06:21:48.883 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-24T06:21:48.884 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-24T06:21:48.884 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-24T06:21:48.884 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-24T06:21:48.884 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-24T06:21:48.884 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-24T06:21:48.884 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-24T06:21:48.884 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-24T06:21:48.884 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-24T06:21:48.884 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-24T06:21:48.884 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-24T06:21:48.884 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-24T06:21:48.884 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-24T06:21:48.884 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-24T06:21:48.885 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-24T06:21:48.885 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-24T06:21:48.885 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-24T06:21:48.885 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-24T06:21:48.885 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-24T06:21:48.885 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-24T06:21:48.885 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-24T06:21:48.885 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-24T06:21:48.885 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-24T06:21:48.885 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-24T06:21:48.885 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-24T06:21:48.885 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-24T06:21:48.885 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-24T06:21:48.886 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-24T06:21:48.886 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-24T06:21:48.886 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-24T06:21:48.886 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-24T06:21:48.886 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-24T06:21:48.886 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-24T06:21:48.886 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-24T06:21:48.886 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-24T06:21:48.886 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-24T06:21:48.886 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-24T06:21:48.886 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-24T06:21:48.886 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-24T06:21:48.887 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-24T06:21:48.887 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-24T06:21:48.887 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-24T06:21:48.887 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-24T06:21:48.887 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-24T06:21:48.887 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-24T06:21:48.887 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-24T06:21:48.887 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-24T06:21:48.887 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-24T06:21:48.887 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-24T06:21:48.887 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-24T06:21:48.887 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-24T06:21:48.887 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-24T06:21:48.888 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-24T06:21:48.888 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-24T06:21:48.888 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-24T06:21:48.888 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-24T06:21:48.888 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-24T06:21:48.888 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-24T06:21:48.888 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-24T06:21:48.888 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-24T06:21:48.888 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-24T06:21:48.888 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-24T06:21:48.888 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-24T06:21:48.888 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-24T06:21:48.888 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-24T06:21:48.888 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-24T06:21:48.888 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-24T06:21:48.889 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-24T06:21:48.889 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-24T06:21:48.889 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-24T06:21:48.889 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-24T06:21:48.889 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-24T06:21:48.889 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-24T06:21:48.889 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-24T06:21:48.889 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-24T06:21:48.889 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-24T06:21:48.889 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-24T06:21:48.889 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-24T06:21:48.889 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-24T06:21:48.889 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-24T06:21:48.889 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-24T06:21:48.890 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-24T06:21:48.890 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-24T06:21:48.890 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-24T06:21:48.890 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-24T06:21:48.890 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-24T06:21:48.890 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-24T06:21:48.890 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-24T06:21:48.890 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-24T06:21:48.890 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-24T06:21:48.890 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-24T06:21:48.890 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-24T06:21:48.890 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-24T06:21:48.890 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-24T06:21:48.891 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-24T06:21:48.891 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-24T06:21:48.891 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-24T06:21:48.891 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-24T06:21:48.891 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-24T06:21:48.891 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-24T06:21:48.891 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-24T06:21:48.891 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-24T06:21:48.891 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-24T06:21:48.891 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-24T06:21:48.891 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-24T06:21:48.891 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-24T06:21:48.891 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-24T06:21:48.891 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-24T06:21:48.892 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-24T06:21:48.892 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-24T06:21:48.892 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-24T06:21:48.892 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-24T06:21:48.892 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-24T06:21:48.892 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-24T06:21:48.892 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-24T06:21:48.892 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-24T06:21:48.892 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-24T06:21:48.892 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-24T06:21:48.892 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-24T06:21:48.892 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-24T06:21:48.892 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-24T06:21:48.892 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-24T06:21:48.893 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-24T06:21:48.893 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-24T06:21:48.893 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-24T06:21:48.893 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-24T06:21:48.893 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-24T06:21:48.893 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-24T06:21:48.893 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-24T06:21:48.893 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-24T06:21:48.893 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-24T06:21:48.893 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-24T06:21:48.893 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-24T06:21:48.893 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-24T06:21:48.893 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-24T06:21:48.893 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-24T06:21:48.893 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-24T06:21:48.894 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-24T06:21:48.894 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-24T06:21:48.894 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-24T06:21:48.894 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-24T06:21:48.894 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-24T06:21:48.894 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-24T06:21:48.894 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-24T06:21:48.894 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-24T06:21:48.894 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-24T06:21:48.894 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-24T06:21:48.894 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-24T06:21:48.894 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-24T06:21:48.894 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-24T06:21:48.895 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-24T06:21:48.895 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-24T06:21:48.895 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-24T06:21:48.895 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-24T06:21:48.895 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-24T06:21:48.895 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-24T06:21:48.895 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-24T06:21:48.895 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-24T06:21:48.895 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-24T06:21:48.895 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-24T06:21:48.895 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-24T06:21:48.895 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-24T06:21:48.895 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-24T06:21:48.895 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-24T06:21:48.895 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-24T06:21:48.896 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-24T06:21:48.896 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-24T06:21:48.896 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-24T06:21:48.896 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-24T06:21:48.896 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-24T06:21:48.896 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-24T06:21:48.896 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-24T06:21:48.896 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-24T06:21:48.896 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-24T06:21:48.896 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-24T06:21:48.896 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-24T06:21:48.896 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-24T06:21:48.896 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-24T06:21:48.896 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-24T06:21:48.896 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-24T06:21:48.897 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-24T06:21:48.897 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-24T06:21:48.897 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-24T06:21:48.897 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-24T06:21:48.897 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-24T06:21:48.897 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-24T06:21:48.897 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-24T06:21:48.897 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-24T06:21:48.897 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-24T06:21:48.897 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-24T06:21:48.897 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-24T06:21:48.897 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-24T06:21:48.897 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-24T06:21:48.897 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-24T06:21:48.897 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-24T06:21:48.898 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-24T06:21:48.898 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-24T06:21:48.898 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-24T06:21:48.898 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-24T06:21:48.898 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-24T06:21:48.898 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-24T06:21:48.898 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-24T06:21:48.898 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-24T06:21:48.898 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-24T06:21:48.898 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-24T06:21:48.898 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-24T06:21:48.898 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-24T06:21:48.898 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-24T06:21:48.899 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-24T06:21:48.899 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-24T06:21:48.899 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-24T06:21:48.899 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-24T06:21:48.899 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-24T06:21:48.899 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-24T06:21:48.899 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-24T06:21:48.899 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-24T06:21:48.899 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-24T06:21:48.899 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-24T06:21:48.899 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-24T06:21:48.899 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-24T06:21:48.899 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-24T06:21:48.899 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-24T06:21:48.900 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-24T06:21:48.900 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-24T06:21:48.900 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-24T06:21:48.900 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-24T06:21:48.900 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-24T06:21:48.900 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-24T06:21:48.900 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-24T06:21:48.900 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-24T06:21:48.900 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-24T06:21:48.900 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-24T06:21:48.900 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-24T06:21:48.900 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-24T06:21:48.900 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-24T06:21:48.900 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-24T06:21:48.901 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-24T06:21:48.901 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-24T06:21:48.901 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-24T06:21:48.901 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-24T06:21:48.901 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-24T06:21:48.901 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-24T06:21:48.901 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-24T06:21:48.901 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-24T06:21:48.901 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-24T06:21:48.901 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-24T06:21:48.901 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-24T06:21:48.901 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-24T06:21:48.901 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-24T06:21:48.901 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-24T06:21:48.902 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-24T06:21:48.902 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-24T06:21:48.902 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-24T06:21:48.902 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-24T06:21:48.902 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-24T06:21:48.902 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-24T06:21:48.902 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-24T06:21:48.902 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-24T06:21:48.902 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-24T06:21:48.902 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-24T06:21:48.902 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-24T06:21:48.902 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-24T06:21:48.902 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-24T06:21:48.903 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-24T06:21:48.903 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-24T06:21:48.903 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-24T06:21:48.903 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-24T06:21:48.903 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-24T06:21:48.903 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-24T06:21:48.903 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-24T06:21:48.903 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-24T06:21:48.903 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-24T06:21:48.903 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-24T06:21:48.903 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-24T06:21:48.903 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-24T06:21:48.903 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-24T06:21:48.903 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-24T06:21:48.904 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-24T06:21:48.904 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-24T06:21:48.904 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-24T06:21:48.904 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-24T06:21:48.904 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-24T06:21:48.904 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-24T06:21:48.904 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-24T06:21:48.904 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-24T06:21:48.904 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-24T06:21:48.904 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-24T06:21:48.904 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-24T06:21:48.904 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-24T06:21:48.904 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-24T06:21:48.904 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-24T06:21:48.904 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-24T06:21:48.905 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-24T06:21:48.905 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-24T06:21:48.905 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-24T06:21:48.905 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-24T06:21:48.905 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-24T06:21:48.905 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-24T06:21:48.905 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-24T06:21:48.905 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-24T06:21:48.905 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-24T06:21:48.905 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-24T06:21:48.905 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-24T06:21:48.905 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-24T06:21:48.905 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-24T06:21:48.905 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-24T06:21:48.905 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-24T06:21:48.906 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-24T06:21:48.906 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-24T06:21:48.906 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-24T06:21:48.906 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-24T06:21:48.906 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-24T06:21:48.906 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-24T06:21:48.906 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-24T06:21:48.906 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-24T06:21:48.906 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-24T06:21:48.906 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-24T06:21:48.906 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-24T06:21:48.906 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-24T06:21:48.906 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-24T06:21:48.906 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-24T06:21:48.906 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-24T06:21:48.907 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-24T06:21:48.907 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-24T06:21:48.907 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-24T06:21:48.907 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-24T06:21:48.907 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-24T06:21:48.907 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-24T06:21:48.907 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-24T06:21:48.907 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-24T06:21:48.907 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-24T06:21:48.907 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-24T06:21:48.907 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-24T06:21:48.907 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-24T06:21:48.907 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-24T06:21:48.908 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-24T06:21:48.908 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-24T06:21:48.908 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-24T06:21:48.908 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-24T06:21:48.908 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-24T06:21:48.908 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-24T06:21:48.908 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-24T06:21:48.908 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-24T06:21:48.908 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-24T06:21:48.908 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-24T06:21:48.908 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-24T06:21:48.908 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-24T06:21:48.908 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-24T06:21:48.908 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-24T06:21:48.909 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-24T06:21:48.909 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-24T06:21:48.909 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-24T06:21:48.909 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-24T06:21:48.909 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-24T06:21:48.909 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-24T06:21:48.909 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-24T06:21:48.909 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-24T06:21:48.909 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-24T06:21:48.909 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-24T06:21:48.909 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-24T06:21:48.909 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-24T06:21:48.909 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-24T06:21:48.910 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-24T06:21:48.910 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-24T06:21:48.910 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-24T06:21:48.910 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-24T06:21:48.910 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-24T06:21:48.910 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-24T06:21:48.910 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-24T06:21:48.910 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-24T06:21:48.910 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-24T06:21:48.910 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-24T06:21:48.910 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-24T06:21:48.910 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-24T06:21:48.910 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-24T06:21:48.910 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-24T06:21:48.911 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-24T06:21:48.911 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-24T06:21:48.911 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-24T06:21:48.911 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-24T06:21:48.911 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-24T06:21:48.911 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-24T06:21:48.911 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-24T06:21:48.911 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-24T06:21:48.911 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-24T06:21:48.911 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-24T06:21:48.911 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-24T06:21:48.911 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-24T06:21:48.911 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-24T06:21:48.911 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-24T06:21:48.911 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-24T06:21:48.912 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-24T06:21:48.912 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-24T06:21:48.912 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-24T06:21:48.912 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-24T06:21:48.912 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-24T06:21:48.912 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-24T06:21:48.912 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-24T06:21:48.912 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-24T06:21:48.912 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-24T06:21:48.912 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-24T06:21:48.912 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-24T06:21:48.912 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-24T06:21:48.912 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-24T06:21:48.912 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-24T06:21:48.912 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-24T06:21:48.913 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-24T06:21:48.913 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-24T06:21:48.913 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-24T06:21:48.913 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-24T06:21:48.913 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-24T06:21:48.913 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-24T06:21:48.913 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-24T06:21:48.913 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-24T06:21:48.913 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-24T06:21:48.913 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-24T06:21:48.913 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-24T06:21:48.913 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-24T06:21:48.913 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-24T06:21:48.913 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-24T06:21:48.914 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-24T06:21:48.914 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-24T06:21:48.914 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-24T06:21:48.914 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-24T06:21:48.914 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-24T06:21:48.914 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-24T06:21:48.914 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-24T06:21:48.914 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-24T06:21:48.914 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-24T06:21:48.914 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-24T06:21:48.914 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-24T06:21:48.914 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-24T06:21:48.914 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-24T06:21:48.914 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-24T06:21:48.914 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-24T06:21:48.915 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-24T06:21:48.915 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-24T06:21:48.915 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-24T06:21:48.915 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-24T06:21:48.915 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-24T06:21:48.915 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-24T06:21:48.915 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-24T06:21:48.915 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-24T06:21:48.915 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-24T06:21:48.915 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-24T06:21:48.915 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-24T06:21:48.915 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-24T06:21:48.915 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-24T06:21:48.915 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-24T06:21:48.916 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-24T06:21:48.916 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-24T06:21:48.916 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-24T06:21:48.916 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-24T06:21:48.916 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-24T06:21:48.916 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-24T06:21:48.916 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-24T06:21:48.916 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-24T06:21:48.916 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-24T06:21:48.916 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-24T06:21:48.916 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-24T06:21:48.916 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-24T06:21:48.916 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-24T06:21:48.916 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-24T06:21:48.917 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-24T06:21:48.917 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-24T06:21:48.917 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-24T06:21:48.917 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-24T06:21:48.917 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-24T06:21:48.917 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-24T06:21:48.917 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-24T06:21:48.917 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-24T06:21:48.917 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-24T06:21:48.917 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-24T06:21:48.917 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-24T06:21:48.917 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-24T06:21:48.917 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-24T06:21:48.917 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-24T06:21:48.917 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-24T06:21:48.918 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-24T06:21:48.918 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-24T06:21:48.918 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-24T06:21:48.918 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-24T06:21:48.918 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-24T06:21:48.918 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-24T06:21:48.918 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-24T06:21:48.918 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-24T06:21:48.918 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-24T06:21:48.918 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-24T06:21:48.918 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-24T06:21:48.918 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-24T06:21:48.918 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-24T06:21:48.919 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-24T06:21:48.919 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-24T06:21:48.919 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-24T06:21:48.919 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-24T06:21:48.919 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-24T06:21:48.919 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-24T06:21:48.919 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-24T06:21:48.919 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-24T06:21:48.919 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-24T06:21:48.919 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-24T06:21:48.919 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-24T06:21:48.919 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-24T06:21:48.919 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-24T06:21:48.920 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-24T06:21:48.920 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-24T06:21:48.920 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-24T06:21:48.920 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-24T06:21:48.920 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-24T06:21:48.920 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-24T06:21:48.920 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-24T06:21:48.920 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-24T06:21:48.920 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-24T06:21:48.920 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-24T06:21:48.920 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-24T06:21:48.920 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-24T06:21:48.920 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-24T06:21:48.920 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-24T06:21:48.920 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-24T06:21:48.921 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-24T06:21:48.921 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-24T06:21:48.921 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-24T06:21:48.921 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-24T06:21:48.921 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-24T06:21:48.921 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-24T06:21:48.921 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-24T06:21:48.921 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-24T06:21:48.921 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-24T06:21:48.921 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-24T06:21:48.921 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-24T06:21:48.921 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-24T06:21:48.921 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-24T06:21:48.921 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-24T06:21:48.922 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-24T06:21:48.922 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-24T06:21:48.922 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-24T06:21:48.922 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-24T06:21:48.922 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-24T06:21:48.922 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-24T06:21:48.922 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-24T06:21:48.922 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-24T06:21:48.922 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-24T06:21:48.922 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-24T06:21:48.922 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-24T06:21:48.922 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-24T06:21:48.922 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-24T06:21:48.922 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-24T06:21:48.923 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-24T06:21:48.923 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-24T06:21:48.923 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-24T06:21:48.923 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-24T06:21:48.923 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-24T06:21:48.923 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-24T06:21:48.923 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-24T06:21:48.923 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-24T06:21:48.923 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-24T06:21:48.923 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-24T06:21:48.923 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-24T06:21:48.923 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-24T06:21:48.923 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-24T06:21:48.923 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-24T06:21:48.923 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-24T06:21:48.924 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-24T06:21:48.924 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-24T06:21:48.924 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-24T06:21:48.924 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-24T06:21:48.924 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-24T06:21:48.924 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-24T06:21:48.924 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-24T06:21:48.924 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-24T06:21:48.924 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-24T06:21:48.924 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-24T06:21:48.924 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-24T06:21:48.924 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-24T06:21:48.924 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-24T06:21:48.924 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-24T06:21:48.924 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-24T06:21:48.925 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-24T06:21:48.925 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-24T06:21:48.925 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-24T06:21:48.925 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-24T06:21:48.925 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-24T06:21:48.925 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-24T06:21:48.925 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-24T06:21:48.925 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-24T06:21:48.925 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-24T06:21:48.925 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-24T06:21:48.925 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-24T06:21:48.925 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-24T06:21:48.925 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-24T06:21:48.925 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-24T06:21:48.926 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-24T06:21:48.926 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-24T06:21:48.926 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-24T06:21:48.926 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-24T06:21:48.926 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-24T06:21:48.926 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-24T06:21:48.926 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-24T06:21:48.926 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-24T06:21:48.926 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-24T06:21:48.926 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-24T06:21:48.926 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-24T06:21:48.926 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-24T06:21:48.926 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-24T06:21:48.927 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-24T06:21:48.927 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-24T06:21:48.927 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-24T06:21:48.927 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-24T06:21:48.927 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-24T06:21:48.927 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-24T06:21:48.927 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-24T06:21:48.927 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-24T06:21:48.927 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-24T06:21:48.927 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-24T06:21:48.927 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-24T06:21:48.927 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-24T06:21:48.927 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-24T06:21:48.927 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-24T06:21:48.927 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-24T06:21:48.928 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-24T06:21:48.928 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-24T06:21:48.928 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-24T06:21:48.928 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-24T06:21:48.928 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-24T06:21:48.928 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-24T06:21:48.928 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-24T06:21:48.928 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-24T06:21:48.928 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-24T06:21:48.928 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-24T06:21:48.928 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-24T06:21:48.928 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-24T06:21:48.928 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-24T06:21:48.928 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-24T06:21:48.929 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-24T06:21:48.929 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-24T06:21:48.929 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-24T06:21:48.929 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-24T06:21:48.929 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-24T06:21:48.929 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-24T06:21:48.929 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-24T06:21:48.929 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-24T06:21:48.929 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-24T06:21:48.929 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-24T06:21:48.929 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-24T06:21:48.929 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-24T06:21:48.929 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-24T06:21:48.929 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-24T06:21:48.929 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-24T06:21:48.930 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-24T06:21:48.930 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-24T06:21:48.930 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-24T06:21:48.930 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-24T06:21:48.930 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-24T06:21:48.930 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-24T06:21:48.930 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-24T06:21:48.930 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-24T06:21:48.930 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-24T06:21:48.930 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-24T06:21:48.930 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-24T06:21:48.930 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-24T06:21:48.930 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-24T06:21:48.931 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-24T06:21:48.931 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-24T06:21:48.931 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-24T06:21:48.931 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-24T06:21:48.931 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-24T06:21:48.931 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-24T06:21:48.931 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-24T06:21:48.931 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-24T06:21:48.931 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-24T06:21:48.931 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-24T06:21:48.931 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-24T06:21:48.931 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-24T06:21:48.931 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-24T06:21:48.931 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-24T06:21:48.931 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-24T06:21:48.932 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-24T06:21:48.932 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-24T06:21:48.932 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-24T06:21:48.932 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-24T06:21:48.932 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-24T06:21:48.932 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-24T06:21:48.932 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-24T06:21:48.932 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-24T06:21:48.932 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-24T06:21:48.932 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-24T06:21:48.932 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-24T06:21:48.932 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-24T06:21:48.932 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-24T06:21:48.933 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-24T06:21:48.933 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-24T06:21:48.933 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-24T06:21:48.933 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-24T06:21:48.933 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-24T06:21:48.933 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-24T06:21:48.933 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-24T06:21:48.933 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-24T06:21:48.933 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-24T06:21:48.933 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-24T06:21:48.933 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-24T06:21:48.933 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-24T06:21:48.933 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-24T06:21:48.933 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-24T06:21:48.934 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-24T06:21:48.934 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-24T06:21:48.934 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-24T06:21:48.934 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-24T06:21:48.934 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-24T06:21:48.934 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-24T06:21:48.934 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-24T06:21:48.934 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-24T06:21:48.934 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-24T06:21:48.934 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-24T06:21:48.934 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-24T06:21:48.934 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-24T06:21:48.934 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-24T06:21:48.934 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-24T06:21:48.935 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-24T06:21:48.935 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-24T06:21:48.935 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-24T06:21:48.935 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-24T06:21:48.935 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-24T06:21:48.935 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-24T06:21:48.935 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-24T06:21:48.935 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-24T06:21:48.935 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-24T06:21:48.935 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-24T06:21:48.935 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-24T06:21:48.935 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-24T06:21:48.935 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-24T06:21:48.935 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-24T06:21:48.935 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-24T06:21:48.935 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-24T06:21:48.936 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-24T06:21:48.936 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-24T06:21:48.936 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-24T06:21:48.936 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-24T06:21:48.936 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-24T06:21:48.936 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-24T06:21:48.936 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-24T06:21:48.936 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-24T06:21:48.936 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-24T06:21:48.936 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-24T06:21:48.936 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-24T06:21:48.936 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-24T06:21:48.936 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-24T06:21:48.936 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-24T06:21:48.937 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-24T06:21:48.937 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-24T06:21:48.937 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-24T06:21:48.937 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-24T06:21:48.937 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-24T06:21:48.937 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-24T06:21:48.937 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-24T06:21:48.937 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-24T06:21:48.937 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-24T06:21:48.937 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-24T06:21:48.937 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-24T06:21:48.937 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-24T06:21:48.937 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-24T06:21:48.937 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-24T06:21:48.937 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-24T06:21:48.938 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-24T06:21:48.938 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-24T06:21:48.938 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-24T06:21:48.938 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-24T06:21:48.938 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-24T06:21:48.938 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-24T06:21:48.938 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-24T06:21:48.938 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-24T06:21:48.938 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-24T06:21:48.938 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-24T06:21:48.938 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-24T06:21:48.938 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-24T06:21:48.938 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-24T06:21:48.938 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-24T06:21:48.939 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-24T06:21:48.939 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-24T06:21:48.939 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-24T06:21:48.939 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-24T06:21:48.939 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-24T06:21:48.939 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-24T06:21:48.939 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-24T06:21:48.939 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-24T06:21:48.939 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-24T06:21:48.939 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-24T06:21:48.939 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-24T06:21:48.939 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-24T06:21:48.939 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-24T06:21:48.939 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-24T06:21:48.939 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-24T06:21:48.940 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-24T06:21:48.940 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-24T06:21:48.940 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-24T06:21:48.940 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-24T06:21:48.940 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-24T06:21:48.940 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-24T06:21:48.940 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-24T06:21:48.940 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-24T06:21:48.940 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-24T06:21:48.940 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-24T06:21:48.940 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-24T06:21:48.940 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-24T06:21:48.940 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-24T06:21:48.940 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-24T06:21:48.941 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-24T06:21:48.941 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-24T06:21:48.941 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-24T06:21:48.941 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-24T06:21:48.941 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-24T06:21:48.941 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-24T06:21:48.941 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-24T06:21:48.941 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-24T06:21:48.941 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-24T06:21:48.941 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-24T06:21:48.941 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-24T06:21:48.941 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-24T06:21:48.941 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-24T06:21:48.941 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-24T06:21:48.941 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-24T06:21:48.942 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-24T06:21:48.942 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-24T06:21:48.942 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-24T06:21:48.942 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-24T06:21:48.942 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-24T06:21:48.942 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-24T06:21:48.942 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-24T06:21:48.942 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-24T06:21:48.942 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-24T06:21:48.942 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-24T06:21:48.942 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-24T06:21:48.942 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-24T06:21:48.942 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-24T06:21:48.943 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-24T06:21:48.943 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-24T06:21:48.943 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-24T06:21:48.943 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-24T06:21:48.943 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-24T06:21:48.943 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-24T06:21:48.943 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-24T06:21:48.943 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-24T06:21:48.943 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-24T06:21:48.943 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-24T06:21:48.943 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-24T06:21:48.943 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-24T06:21:48.943 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-24T06:21:48.943 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-24T06:21:48.943 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-24T06:21:48.944 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-24T06:21:48.944 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-24T06:21:48.944 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-24T06:21:48.944 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-24T06:21:48.944 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-24T06:21:48.944 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-24T06:21:48.944 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-24T06:21:48.944 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-24T06:21:48.944 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-24T06:21:48.944 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-24T06:21:48.944 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-24T06:21:48.944 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-24T06:21:48.944 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-24T06:21:48.944 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-24T06:21:48.945 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-24T06:21:48.945 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-24T06:21:48.945 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-24T06:21:48.945 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-24T06:21:48.945 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-24T06:21:48.945 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-24T06:21:48.945 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-24T06:21:48.945 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-24T06:21:48.945 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-24T06:21:48.945 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-24T06:21:48.945 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-24T06:21:48.945 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-24T06:21:48.945 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-24T06:21:48.945 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-24T06:21:48.945 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-24T06:21:48.946 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-24T06:21:48.946 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-24T06:21:48.946 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-24T06:21:48.946 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-24T06:21:48.946 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-24T06:21:48.946 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-24T06:21:48.946 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-24T06:21:48.946 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-24T06:21:48.946 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-24T06:21:48.946 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-24T06:21:48.946 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-24T06:21:48.946 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-24T06:21:48.946 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-24T06:21:48.946 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-24T06:21:48.947 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-24T06:21:48.947 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-24T06:21:48.947 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-24T06:21:48.947 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-24T06:21:48.947 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-24T06:21:48.947 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-24T06:21:48.947 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-24T06:21:48.947 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-24T06:21:48.947 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-24T06:21:48.947 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-24T06:21:48.947 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-24T06:21:48.947 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-24T06:21:48.947 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-24T06:21:48.947 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-24T06:21:48.947 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-24T06:21:48.948 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-24T06:21:48.948 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-24T06:21:48.948 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-24T06:21:48.948 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-24T06:21:48.948 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-24T06:21:48.948 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-24T06:21:48.948 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-24T06:21:48.948 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-24T06:21:48.948 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-24T06:21:48.948 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-24T06:21:48.948 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-24T06:21:48.948 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-24T06:21:48.948 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-24T06:21:48.948 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-24T06:21:48.949 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-24T06:21:48.949 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-24T06:21:48.949 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-24T06:21:48.949 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-24T06:21:48.949 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-24T06:21:48.949 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-24T06:21:48.949 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-24T06:21:48.949 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-24T06:21:48.949 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-24T06:21:48.949 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-24T06:21:48.949 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-24T06:21:48.949 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-24T06:21:48.949 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-24T06:21:48.949 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-24T06:21:48.950 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-24T06:21:48.950 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-24T06:21:48.950 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-24T06:21:48.950 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-24T06:21:48.950 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-24T06:21:48.950 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-24T06:21:48.950 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-24T06:21:48.950 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-24T06:21:48.950 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-24T06:21:48.950 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-24T06:21:48.950 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-24T06:21:48.950 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-24T06:21:48.950 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-24T06:21:48.950 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-24T06:21:48.951 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-24T06:21:48.951 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-24T06:21:48.951 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-24T06:21:48.951 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-24T06:21:48.951 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-24T06:21:48.951 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-24T06:21:48.951 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-24T06:21:48.951 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-24T06:21:48.951 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-24T06:21:48.951 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-24T06:21:48.951 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-24T06:21:48.951 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-24T06:21:48.951 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-24T06:21:48.951 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-24T06:21:48.952 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-24T06:21:48.952 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-24T06:21:48.952 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-24T06:21:48.952 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-24T06:21:48.952 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-24T06:21:48.952 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-24T06:21:48.952 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-24T06:21:48.952 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-24T06:21:48.952 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-24T06:21:48.952 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-24T06:21:48.952 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-24T06:21:48.952 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-24T06:21:48.952 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-24T06:21:48.952 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-24T06:21:48.952 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-24T06:21:48.953 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-24T06:21:48.953 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-24T06:21:48.953 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-24T06:21:48.953 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-24T06:21:48.953 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-24T06:21:48.953 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-24T06:21:48.953 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-24T06:21:48.953 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-24T06:21:48.953 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-24T06:21:48.953 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-24T06:21:48.953 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-24T06:21:48.953 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-24T06:21:48.953 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-24T06:21:48.953 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-24T06:21:48.954 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-24T06:21:48.954 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-24T06:21:48.954 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-24T06:21:48.954 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-24T06:21:48.954 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-24T06:21:48.954 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-24T06:21:48.954 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-24T06:21:48.954 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-24T06:21:48.954 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-24T06:21:48.954 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-24T06:21:48.954 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-24T06:21:48.954 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-24T06:21:48.954 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-24T06:21:48.954 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-24T06:21:48.955 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-24T06:21:48.955 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-24T06:21:48.955 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-24T06:21:48.955 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-24T06:21:48.955 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-24T06:21:48.955 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-24T06:21:48.955 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-24T06:21:48.955 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-24T06:21:48.955 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-24T06:21:48.955 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-24T06:21:48.955 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-24T06:21:48.955 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-24T06:21:48.955 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-24T06:21:48.955 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-24T06:21:48.955 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-24T06:21:48.956 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-24T06:21:48.956 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-24T06:21:48.956 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-24T06:21:48.956 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-24T06:21:48.956 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-24T06:21:48.956 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-24T06:21:48.956 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-24T06:21:48.956 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-24T06:21:48.956 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-24T06:21:48.956 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-24T06:21:48.956 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-24T06:21:48.956 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-24T06:21:48.956 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-24T06:21:48.957 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-24T06:21:48.957 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-24T06:21:48.957 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-24T06:21:48.957 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-24T06:21:48.957 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-24T06:21:48.957 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-24T06:21:48.957 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-24T06:21:48.957 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-24T06:21:48.957 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-24T06:21:48.957 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-24T06:21:48.957 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-24T06:21:48.957 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-24T06:21:48.957 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-24T06:21:48.957 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-24T06:21:48.957 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-24T06:21:48.958 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-24T06:21:48.958 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-24T06:21:48.958 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-24T06:21:48.958 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-24T06:21:48.958 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-24T06:21:48.958 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-24T06:21:48.958 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-24T06:21:48.958 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-24T06:21:48.958 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-24T06:21:48.958 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-24T06:21:48.958 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-24T06:21:48.958 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-24T06:21:48.958 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-24T06:21:48.958 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-24T06:21:48.959 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-24T06:21:48.959 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-24T06:21:48.959 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-24T06:21:48.959 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-24T06:21:48.959 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-24T06:21:48.959 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-24T06:21:48.959 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-24T06:21:48.959 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-24T06:21:48.959 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-24T06:21:48.959 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-24T06:21:48.959 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-24T06:21:48.959 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-24T06:21:48.959 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-24T06:21:48.959 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-24T06:21:48.959 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-24T06:21:48.960 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-24T06:21:48.960 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-24T06:21:48.960 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-24T06:21:48.960 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-24T06:21:48.960 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-24T06:21:48.960 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-24T06:21:48.960 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-24T06:21:48.960 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-24T06:21:48.960 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-24T06:21:48.960 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-24T06:21:48.960 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-24T06:21:48.960 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-24T06:21:48.960 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-24T06:21:48.960 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-24T06:21:48.961 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-24T06:21:48.961 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-24T06:21:48.961 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-24T06:21:48.961 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-24T06:21:48.961 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-24T06:21:48.961 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-24T06:21:48.961 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-24T06:21:48.961 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-24T06:21:48.961 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-24T06:21:48.961 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-24T06:21:48.961 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-24T06:21:48.961 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-24T06:21:48.961 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-24T06:21:48.961 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-24T06:21:48.961 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-24T06:21:48.962 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-24T06:21:48.962 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-24T06:21:48.962 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-24T06:21:48.962 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-24T06:21:48.962 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-24T06:21:48.962 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-24T06:21:48.962 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-24T06:21:48.962 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-24T06:21:48.962 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-24T06:21:48.962 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-24T06:21:48.962 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-24T06:21:48.962 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-24T06:21:48.962 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-24T06:21:48.962 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-24T06:21:48.963 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-24T06:21:48.963 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-24T06:21:48.963 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-24T06:21:48.963 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-24T06:21:48.963 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-24T06:21:48.963 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-24T06:21:48.963 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-24T06:21:48.963 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-24T06:21:48.963 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-24T06:21:48.963 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-24T06:21:48.963 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-24T06:21:48.963 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-24T06:21:48.963 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-24T06:21:48.963 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-24T06:21:48.964 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-24T06:21:48.964 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-24T06:21:48.964 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-24T06:21:48.964 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-24T06:21:48.964 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-24T06:21:48.964 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-24T06:21:48.964 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-24T06:21:48.964 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-24T06:21:48.964 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-24T06:21:48.964 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-24T06:21:48.964 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-24T06:21:48.964 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-24T06:21:48.964 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-24T06:21:48.964 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-24T06:21:48.964 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-24T06:21:48.965 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-24T06:21:48.965 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-24T06:21:48.965 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-24T06:21:48.965 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-24T06:21:48.965 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-24T06:21:48.965 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-24T06:21:48.965 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-24T06:21:48.965 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-24T06:21:48.965 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-24T06:21:48.965 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-24T06:21:48.965 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-24T06:21:48.965 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-24T06:21:48.965 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-24T06:21:48.965 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-24T06:21:48.966 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-24T06:21:48.966 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-24T06:21:48.966 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-24T06:21:48.966 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-24T06:21:48.966 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-24T06:21:48.966 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-24T06:21:48.966 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-24T06:21:48.966 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-24T06:21:48.966 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-24T06:21:48.966 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-24T06:21:48.966 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-24T06:21:48.966 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-24T06:21:48.966 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-24T06:21:48.966 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-24T06:21:48.967 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-24T06:21:48.967 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-24T06:21:48.967 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-24T06:21:48.967 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-24T06:21:48.967 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-24T06:21:48.967 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-24T06:21:48.967 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-24T06:21:48.967 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-24T06:21:48.967 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-24T06:21:48.967 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-24T06:21:48.967 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-24T06:21:48.967 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-24T06:21:48.967 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-24T06:21:48.967 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-24T06:21:48.967 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-24T06:21:48.968 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-24T06:21:48.968 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-24T06:21:48.968 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-24T06:21:48.968 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-24T06:21:48.968 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-24T06:21:48.968 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-24T06:21:48.968 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-24T06:21:48.968 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-24T06:21:48.968 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-24T06:21:48.968 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-24T06:21:48.968 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-24T06:21:48.968 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-24T06:21:48.968 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-24T06:21:48.968 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-24T06:21:48.968 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-24T06:21:48.969 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-24T06:21:48.969 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-24T06:21:48.969 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-24T06:21:48.969 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-24T06:21:48.969 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-24T06:21:48.969 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-24T06:21:48.969 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-24T06:21:48.969 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-24T06:21:48.969 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-24T06:21:48.969 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-24T06:21:48.969 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-24T06:21:48.969 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-24T06:21:48.969 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-24T06:21:48.969 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-24T06:21:48.969 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-24T06:21:48.970 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-24T06:21:48.970 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-24T06:21:48.970 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-24T06:21:48.970 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-24T06:21:48.970 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-24T06:21:48.970 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-24T06:21:48.970 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-24T06:21:48.970 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-24T06:21:48.970 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-24T06:21:48.970 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-24T06:21:48.970 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-24T06:21:48.970 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-24T06:21:48.970 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-24T06:21:48.970 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-24T06:21:48.971 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-24T06:21:48.971 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-24T06:21:48.971 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-24T06:21:48.971 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-24T06:21:48.971 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-24T06:21:48.971 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-24T06:21:48.971 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-24T06:21:48.971 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-24T06:21:48.971 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-24T06:21:48.971 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-24T06:21:48.971 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-24T06:21:48.971 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-24T06:21:48.971 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-24T06:21:48.971 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-24T06:21:48.971 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-24T06:21:48.972 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-24T06:21:48.972 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-24T06:21:48.972 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-24T06:21:48.972 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-24T06:21:48.972 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-24T06:21:48.972 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-24T06:21:48.972 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-24T06:21:48.972 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-24T06:21:48.972 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-24T06:21:48.972 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-24T06:21:48.972 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-24T06:21:48.972 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-24T06:21:48.972 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-24T06:21:48.972 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-24T06:21:48.972 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-24T06:21:48.973 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-24T06:21:48.973 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-24T06:21:48.973 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-24T06:21:48.973 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-24T06:21:48.973 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-24T06:21:48.973 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-24T06:21:48.973 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-24T06:21:48.973 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-24T06:21:48.973 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-24T06:21:48.973 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-24T06:21:48.973 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-24T06:21:48.973 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-24T06:21:48.973 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-24T06:21:48.973 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-24T06:21:48.973 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-24T06:21:48.974 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-24T06:21:48.974 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-24T06:21:48.974 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-24T06:21:48.974 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-24T06:21:48.974 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-24T06:21:48.974 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-24T06:21:48.974 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-24T06:21:48.974 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-24T06:21:48.974 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-24T06:21:48.974 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-24T06:21:48.974 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-24T06:21:48.974 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-24T06:21:48.974 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-24T06:21:48.974 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-24T06:21:48.975 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-24T06:21:48.975 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-24T06:21:48.975 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-24T06:21:48.975 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-24T06:21:48.975 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-24T06:21:48.975 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-24T06:21:48.975 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-24T06:21:48.975 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-24T06:21:48.975 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-24T06:21:48.975 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-24T06:21:48.975 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-24T06:21:48.975 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-24T06:21:48.975 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-24T06:21:48.975 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-24T06:21:48.976 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-24T06:21:48.976 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-24T06:21:48.976 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-24T06:21:48.976 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-24T06:21:48.976 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-24T06:21:48.976 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-24T06:21:48.976 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-24T06:21:48.976 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-24T06:21:48.976 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-24T06:21:48.976 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-24T06:21:48.976 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-24T06:21:48.976 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-24T06:21:48.976 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-24T06:21:48.976 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-24T06:21:48.977 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-24T06:21:48.977 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-24T06:21:48.977 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-24T06:21:48.977 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-24T06:21:48.977 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-24T06:21:48.977 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-24T06:21:48.977 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-24T06:21:48.977 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-24T06:21:48.977 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-24T06:21:48.977 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-24T06:21:48.977 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-24T06:21:48.977 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-24T06:21:48.977 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-24T06:21:48.977 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-24T06:21:48.977 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-24T06:21:48.978 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-24T06:21:48.978 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-24T06:21:48.978 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-24T06:21:48.978 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-24T06:21:48.978 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-24T06:21:48.978 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-24T06:21:48.978 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-24T06:21:48.978 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-24T06:21:48.978 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-24T06:21:48.978 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-24T06:21:48.978 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-24T06:21:48.978 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-24T06:21:48.978 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-24T06:21:48.978 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-24T06:21:48.979 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-24T06:21:48.979 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-24T06:21:48.979 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-24T06:21:48.979 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-24T06:21:48.979 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-24T06:21:48.979 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-24T06:21:48.979 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-24T06:21:48.979 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-24T06:21:48.979 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-24T06:21:48.979 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-24T06:21:48.979 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-24T06:21:48.979 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-24T06:21:48.979 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-24T06:21:48.979 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-24T06:21:48.980 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-24T06:21:48.980 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-24T06:21:48.980 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-24T06:21:48.980 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-24T06:21:48.980 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-24T06:21:48.980 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-24T06:21:48.980 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-24T06:21:48.980 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-24T06:21:48.980 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-24T06:21:48.980 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-24T06:21:48.980 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-24T06:21:48.981 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-24T06:21:48.981 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-24T06:21:48.981 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-24T06:21:48.981 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-24T06:21:48.981 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-24T06:21:48.981 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-24T06:21:48.982 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-24T06:21:48.982 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-24T06:21:48.982 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-24T06:21:48.982 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-24T06:21:48.982 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-24T06:21:48.982 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-24T06:21:48.982 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-24T06:21:48.982 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-24T06:21:48.982 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-24T06:21:48.982 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-24T06:21:48.982 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-24T06:21:48.982 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-24T06:21:48.982 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-24T06:21:48.982 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-24T06:21:48.983 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-24T06:21:48.983 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-24T06:21:48.983 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-24T06:21:48.983 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-24T06:21:48.983 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-24T06:21:48.983 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-24T06:21:48.983 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-24T06:21:48.983 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-24T06:21:48.983 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-24T06:21:48.983 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-24T06:21:48.983 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-24T06:21:48.983 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-24T06:21:48.983 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-24T06:21:48.983 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-24T06:21:48.983 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-24T06:21:48.984 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-24T06:21:48.984 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-24T06:21:48.984 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-24T06:21:48.984 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-24T06:21:48.984 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-24T06:21:48.984 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-24T06:21:48.984 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-24T06:21:48.984 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-24T06:21:48.984 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-24T06:21:48.984 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-24T06:21:48.984 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-24T06:21:48.984 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-24T06:21:48.984 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-24T06:21:48.984 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-24T06:21:48.985 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-24T06:21:48.985 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-24T06:21:48.985 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-24T06:21:48.985 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-24T06:21:48.985 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-24T06:21:48.985 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-24T06:21:48.985 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-24T06:21:48.985 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-24T06:21:48.985 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-24T06:21:48.985 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-24T06:21:48.985 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-24T06:21:48.985 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-24T06:21:48.985 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-24T06:21:48.985 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-24T06:21:48.986 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-24T06:21:48.986 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-24T06:21:48.986 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-24T06:21:48.986 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-24T06:21:48.986 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-24T06:21:48.986 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-24T06:21:48.986 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-24T06:21:48.986 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-24T06:21:48.986 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-24T06:21:48.986 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-24T06:21:48.986 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-24T06:21:48.986 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-24T06:21:48.986 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-24T06:21:48.987 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-24T06:21:48.987 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-24T06:21:48.987 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-24T06:21:48.987 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-24T06:21:48.987 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-24T06:21:48.987 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-24T06:21:48.987 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-24T06:21:48.987 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-24T06:21:48.987 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-24T06:21:48.987 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-24T06:21:48.987 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-24T06:21:48.987 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-24T06:21:48.987 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-24T06:21:48.987 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-24T06:21:48.988 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-24T06:21:48.988 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-24T06:21:48.988 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-24T06:21:48.988 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-24T06:21:48.988 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-24T06:21:48.988 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-24T06:21:48.988 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-24T06:21:48.988 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-24T06:21:48.988 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-24T06:21:48.988 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-24T06:21:48.988 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-24T06:21:48.988 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-24T06:21:48.988 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-24T06:21:48.988 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-24T06:21:48.988 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-24T06:21:48.989 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-24T06:21:48.989 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-24T06:21:48.989 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-24T06:21:48.989 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-24T06:21:48.989 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-24T06:21:48.989 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-24T06:21:48.989 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-24T06:21:48.989 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-24T06:21:48.989 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-24T06:21:48.989 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-24T06:21:48.989 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-24T06:21:48.989 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-24T06:21:48.989 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-24T06:21:48.989 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-24T06:21:48.990 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-24T06:21:48.990 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-24T06:21:48.990 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-24T06:21:48.990 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-24T06:21:48.990 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-24T06:21:48.990 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-24T06:21:48.990 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-24T06:21:48.990 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-24T06:21:48.990 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-24T06:21:48.990 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-24T06:21:48.990 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-24T06:21:48.990 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-24T06:21:48.990 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-24T06:21:48.990 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-24T06:21:48.990 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-24T06:21:48.991 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-24T06:21:48.991 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-24T06:21:48.991 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-24T06:21:48.991 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-24T06:21:48.991 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-24T06:21:48.991 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-24T06:21:48.991 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-24T06:21:48.991 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-24T06:21:48.991 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-24T06:21:48.991 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-24T06:21:48.991 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-24T06:21:48.991 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-24T06:21:48.991 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-24T06:21:48.991 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-24T06:21:48.992 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-24T06:21:48.992 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-24T06:21:48.992 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-24T06:21:48.992 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-24T06:21:48.992 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-24T06:21:48.992 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-24T06:21:48.992 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-24T06:21:48.992 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-24T06:21:48.992 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-24T06:21:48.992 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-24T06:21:48.992 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-24T06:21:48.992 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-24T06:21:48.992 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-24T06:21:48.992 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-24T06:21:48.993 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-24T06:21:48.993 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-24T06:21:48.993 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-24T06:21:48.993 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-24T06:21:48.993 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-24T06:21:48.993 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-24T06:21:48.993 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-24T06:21:48.993 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-24T06:21:48.993 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-24T06:21:48.993 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-24T06:21:48.993 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-24T06:21:48.993 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-24T06:21:48.993 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-24T06:21:48.993 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-24T06:21:48.994 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-24T06:21:48.994 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-24T06:21:48.994 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-24T06:21:48.994 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-24T06:21:48.994 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-24T06:21:48.994 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-24T06:21:48.994 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-24T06:21:48.994 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-24T06:21:48.994 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-24T06:21:48.994 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-24T06:21:48.994 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-24T06:21:48.994 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-24T06:21:48.994 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-24T06:21:48.994 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-24T06:21:48.995 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-24T06:21:48.995 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-24T06:21:48.995 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-24T06:21:48.995 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-24T06:21:48.995 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-24T06:21:48.995 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-24T06:21:48.995 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-24T06:21:48.995 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-24T06:21:48.995 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-24T06:21:48.995 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-24T06:21:48.995 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-24T06:21:48.995 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-24T06:21:48.995 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-24T06:21:48.995 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-24T06:21:48.996 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-24T06:21:48.996 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-24T06:21:48.996 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-24T06:21:48.996 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-24T06:21:48.996 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-24T06:21:48.996 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-24T06:21:48.996 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-24T06:21:48.996 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-24T06:21:48.996 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-24T06:21:48.996 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-24T06:21:48.996 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-24T06:21:48.996 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-24T06:21:48.997 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-24T06:21:48.997 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-24T06:21:48.997 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-24T06:21:48.997 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-24T06:21:48.997 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-24T06:21:48.997 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-24T06:21:48.997 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-24T06:21:48.997 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-24T06:21:48.997 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-24T06:21:48.997 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-24T06:21:48.997 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-24T06:21:48.997 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-24T06:21:48.997 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-24T06:21:48.997 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-24T06:21:48.998 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-24T06:21:48.998 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-24T06:21:48.998 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-24T06:21:48.998 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-24T06:21:48.998 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-24T06:21:48.998 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-24T06:21:48.998 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-24T06:21:48.998 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-24T06:21:48.998 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-24T06:21:48.998 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-24T06:21:48.998 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-24T06:21:48.998 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-24T06:21:48.998 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-24T06:21:48.998 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-24T06:21:48.999 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-24T06:21:48.999 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-24T06:21:48.999 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-24T06:21:48.999 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-24T06:21:48.999 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-24T06:21:48.999 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-24T06:21:48.999 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-24T06:21:48.999 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-24T06:21:48.999 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-24T06:21:48.999 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-24T06:21:48.999 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-24T06:21:48.999 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-24T06:21:48.999 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-24T06:21:48.999 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-24T06:21:49.000 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-24T06:21:49.000 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-24T06:21:49.000 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-24T06:21:49.000 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-24T06:21:49.000 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-24T06:21:49.000 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-24T06:21:49.000 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-24T06:21:49.000 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-24T06:21:49.000 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-24T06:21:49.000 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-24T06:21:49.000 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-24T06:21:49.000 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-24T06:21:49.000 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-24T06:21:49.000 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-24T06:21:49.000 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-24T06:21:49.001 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-24T06:21:49.001 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-24T06:21:49.001 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-24T06:21:49.001 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-24T06:21:49.001 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-24T06:21:49.001 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-24T06:21:49.001 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-24T06:21:49.001 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-24T06:21:49.001 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-24T06:21:49.001 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-24T06:21:49.001 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-24T06:21:49.001 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-24T06:21:49.001 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-24T06:21:49.002 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-24T06:21:49.002 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-24T06:21:49.002 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-24T06:21:49.002 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-24T06:21:49.002 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-24T06:21:49.002 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-24T06:21:49.002 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-24T06:21:49.002 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-24T06:21:49.002 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-24T06:21:49.002 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-24T06:21:49.002 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-24T06:21:49.002 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-24T06:21:49.003 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-24T06:21:49.003 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-24T06:21:49.003 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-24T06:21:49.003 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-24T06:21:49.003 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-24T06:21:49.003 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-24T06:21:49.003 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-24T06:21:49.003 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-24T06:21:49.003 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-24T06:21:49.003 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-24T06:21:49.003 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-24T06:21:49.003 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-24T06:21:49.004 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-24T06:21:49.004 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-24T06:21:49.004 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-24T06:21:49.004 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-24T06:21:49.004 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-24T06:21:49.004 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-24T06:21:49.004 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-24T06:21:49.004 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-24T06:21:49.004 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-24T06:21:49.004 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-24T06:21:49.004 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-24T06:21:49.004 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-24T06:21:49.004 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-24T06:21:49.005 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-24T06:21:49.005 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-24T06:21:49.005 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-24T06:21:49.005 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-24T06:21:49.005 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-24T06:21:49.005 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-24T06:21:49.005 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-24T06:21:49.005 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-24T06:21:49.005 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-24T06:21:49.005 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-24T06:21:49.005 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-24T06:21:49.005 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-24T06:21:49.005 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-24T06:21:49.005 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-24T06:21:49.005 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-24T06:21:49.006 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-24T06:21:49.006 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-24T06:21:49.006 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-24T06:21:49.006 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-24T06:21:49.006 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-24T06:21:49.006 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-24T06:21:49.006 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-24T06:21:49.006 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-24T06:21:49.006 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-24T06:21:49.006 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-24T06:21:49.006 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-24T06:21:49.006 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-24T06:21:49.006 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-24T06:21:49.006 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-24T06:21:49.007 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-24T06:21:49.007 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-24T06:21:49.007 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-24T06:21:49.007 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-24T06:21:49.007 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-24T06:21:49.007 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-24T06:21:49.007 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-24T06:21:49.007 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-24T06:21:49.007 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-24T06:21:49.007 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-24T06:21:49.007 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-24T06:21:49.007 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-24T06:21:49.007 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-24T06:21:49.007 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-24T06:21:49.007 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-24T06:21:49.008 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-24T06:21:49.008 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-24T06:21:49.008 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-24T06:21:49.008 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-24T06:21:49.008 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-24T06:21:49.008 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-24T06:21:49.008 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-24T06:21:49.008 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-24T06:21:49.008 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-24T06:21:49.008 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-24T06:21:49.008 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-24T06:21:49.008 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-24T06:21:49.008 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-24T06:21:49.008 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-24T06:21:49.009 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-24T06:21:49.009 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-24T06:21:49.009 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-24T06:21:49.009 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-24T06:21:49.009 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-24T06:21:49.009 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-24T06:21:49.009 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-24T06:21:49.009 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-24T06:21:49.009 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-24T06:21:49.009 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-24T06:21:49.009 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-24T06:21:49.009 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-24T06:21:49.009 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-24T06:21:49.009 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-24T06:21:49.010 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-24T06:21:49.010 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-24T06:21:49.010 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-24T06:21:49.010 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-24T06:21:49.010 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-24T06:21:49.010 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-24T06:21:49.010 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-24T06:21:49.010 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-24T06:21:49.010 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-24T06:21:49.010 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-24T06:21:49.010 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-24T06:21:49.010 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-24T06:21:49.010 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-24T06:21:49.010 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-24T06:21:49.010 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-24T06:21:49.011 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-24T06:21:49.011 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-24T06:21:49.011 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-24T06:21:49.011 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-24T06:21:49.011 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-24T06:21:49.011 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-24T06:21:49.011 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-24T06:21:49.011 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-24T06:21:49.011 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-24T06:21:49.011 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-24T06:21:49.011 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-24T06:21:49.011 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-24T06:21:49.012 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-24T06:21:49.012 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-24T06:21:49.012 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-24T06:21:49.012 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-24T06:21:49.012 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-24T06:21:49.012 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-24T06:21:49.012 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-24T06:21:49.012 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-24T06:21:49.012 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-24T06:21:49.012 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-24T06:21:49.012 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-24T06:21:49.012 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-24T06:21:49.012 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-24T06:21:49.012 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-24T06:21:49.012 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-24T06:21:49.013 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-24T06:21:49.013 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-24T06:21:49.013 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-24T06:21:49.013 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-24T06:21:49.013 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-24T06:21:49.013 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-24T06:21:49.013 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-24T06:21:49.013 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-24T06:21:49.013 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-24T06:21:49.013 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-24T06:21:49.013 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-24T06:21:49.013 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-24T06:21:49.013 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-24T06:21:49.013 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-24T06:21:49.014 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-24T06:21:49.014 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-24T06:21:49.014 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-24T06:21:49.014 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-24T06:21:49.014 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-24T06:21:49.014 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-24T06:21:49.014 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-24T06:21:49.014 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-24T06:21:49.014 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-24T06:21:49.014 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-24T06:21:49.014 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-24T06:21:49.014 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-24T06:21:49.014 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-24T06:21:49.014 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-24T06:21:49.015 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-24T06:21:49.015 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-24T06:21:49.015 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-24T06:21:49.015 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-24T06:21:49.015 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-24T06:21:49.015 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-24T06:21:49.015 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-24T06:21:49.015 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-24T06:21:49.015 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-24T06:21:49.015 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-24T06:21:49.015 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-24T06:21:49.015 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-24T06:21:49.015 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-24T06:21:49.015 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-24T06:21:49.015 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-24T06:21:49.016 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-24T06:21:49.016 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-24T06:21:49.016 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-24T06:21:49.016 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-24T06:21:49.016 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-24T06:21:49.016 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-24T06:21:49.016 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-24T06:21:49.016 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-24T06:21:49.016 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-24T06:21:49.016 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-24T06:21:49.016 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-24T06:21:49.016 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-24T06:21:49.016 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-24T06:21:49.016 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-24T06:21:49.017 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-24T06:21:49.017 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-24T06:21:49.017 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-24T06:21:49.017 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-24T06:21:49.017 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-24T06:21:49.017 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-24T06:21:49.017 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-24T06:21:49.017 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-24T06:21:49.017 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-24T06:21:49.017 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-24T06:21:49.017 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-24T06:21:49.017 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-24T06:21:49.017 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-24T06:21:49.017 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-24T06:21:49.017 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-24T06:21:49.018 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-24T06:21:49.018 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-24T06:21:49.018 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-24T06:21:49.018 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-24T06:21:49.018 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-24T06:21:49.018 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-24T06:21:49.018 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-24T06:21:49.018 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-24T06:21:49.018 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-24T06:21:49.018 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-24T06:21:49.018 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-24T06:21:49.018 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-24T06:21:49.018 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-24T06:21:49.018 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-24T06:21:49.019 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-24T06:21:49.019 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-24T06:21:49.019 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-24T06:21:49.019 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-24T06:21:49.019 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-24T06:21:49.019 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-24T06:21:49.019 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-24T06:21:49.019 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-24T06:21:49.019 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-24T06:21:49.019 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-24T06:21:49.019 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-24T06:21:49.019 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-24T06:21:49.019 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-24T06:21:49.019 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-24T06:21:49.019 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-24T06:21:49.020 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-24T06:21:49.020 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-24T06:21:49.020 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-24T06:21:49.020 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-24T06:21:49.020 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-24T06:21:49.020 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-24T06:21:49.020 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-24T06:21:49.020 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-24T06:21:49.020 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-24T06:21:49.020 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-24T06:21:49.020 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-24T06:21:49.020 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-24T06:21:49.020 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-24T06:21:49.020 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-24T06:21:49.021 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-24T06:21:49.021 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-24T06:21:49.021 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-24T06:21:49.021 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-24T06:21:49.021 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-24T06:21:49.021 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-24T06:21:49.021 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-24T06:21:49.021 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-24T06:21:49.021 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-24T06:21:49.021 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-24T06:21:49.021 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-24T06:21:49.021 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-24T06:21:49.021 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-24T06:21:49.021 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-24T06:21:49.021 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-24T06:21:49.026 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-24T06:21:49.026 DEBUG:teuthology.orchestra.run.smithi005:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-24T06:21:49.049 INFO:teuthology.orchestra.run.smithi005.stdout:postinstall scriptlet (using /bin/sh): 2024-04-24T06:21:49.049 INFO:teuthology.orchestra.run.smithi005.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-24T06:21:49.049 INFO:teuthology.orchestra.run.smithi005.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-24T06:21:49.049 INFO:teuthology.orchestra.run.smithi005.stdout:fi 2024-04-24T06:21:49.049 INFO:teuthology.orchestra.run.smithi005.stdout:for file in vmlinuz System.map config; do 2024-04-24T06:21:49.049 INFO:teuthology.orchestra.run.smithi005.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-24T06:21:49.050 INFO:teuthology.orchestra.run.smithi005.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-24T06:21:49.050 INFO:teuthology.orchestra.run.smithi005.stdout: fi 2024-04-24T06:21:49.050 INFO:teuthology.orchestra.run.smithi005.stdout:done 2024-04-24T06:21:49.050 INFO:teuthology.orchestra.run.smithi005.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-24T06:21:49.050 INFO:teuthology.orchestra.run.smithi005.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-24T06:21:49.050 INFO:teuthology.orchestra.run.smithi005.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-24T06:21:49.050 INFO:teuthology.orchestra.run.smithi005.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-24T06:21:49.050 INFO:teuthology.orchestra.run.smithi005.stdout:fi 2024-04-24T06:21:49.050 INFO:teuthology.orchestra.run.smithi005.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-24T06:21:49.050 INFO:teuthology.orchestra.run.smithi005.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-24T06:21:49.050 INFO:teuthology.orchestra.run.smithi005.stdout:fi 2024-04-24T06:21:49.050 INFO:teuthology.orchestra.run.smithi005.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-24T06:21:49.050 INFO:teuthology.orchestra.run.smithi005.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-24T06:21:49.050 INFO:teuthology.orchestra.run.smithi005.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-24T06:21:49.051 INFO:teuthology.orchestra.run.smithi005.stdout:fi 2024-04-24T06:21:49.051 DEBUG:teuthology.orchestra.run.smithi005:> sudo rpm -qi grub2-tools 2024-04-24T06:21:49.129 INFO:teuthology.orchestra.run.smithi005.stdout:Name : grub2-tools 2024-04-24T06:21:49.129 INFO:teuthology.orchestra.run.smithi005.stdout:Epoch : 1 2024-04-24T06:21:49.129 INFO:teuthology.orchestra.run.smithi005.stdout:Version : 2.06 2024-04-24T06:21:49.129 INFO:teuthology.orchestra.run.smithi005.stdout:Release : 68.el9 2024-04-24T06:21:49.129 INFO:teuthology.orchestra.run.smithi005.stdout:Architecture: x86_64 2024-04-24T06:21:49.129 INFO:teuthology.orchestra.run.smithi005.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-24T06:21:49.129 INFO:teuthology.orchestra.run.smithi005.stdout:Group : Unspecified 2024-04-24T06:21:49.129 INFO:teuthology.orchestra.run.smithi005.stdout:Size : 8264254 2024-04-24T06:21:49.129 INFO:teuthology.orchestra.run.smithi005.stdout:License : GPLv3+ 2024-04-24T06:21:49.130 INFO:teuthology.orchestra.run.smithi005.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-24T06:21:49.130 INFO:teuthology.orchestra.run.smithi005.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-24T06:21:49.130 INFO:teuthology.orchestra.run.smithi005.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-24T06:21:49.130 INFO:teuthology.orchestra.run.smithi005.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-24T06:21:49.130 INFO:teuthology.orchestra.run.smithi005.stdout:Packager : builder@centos.org 2024-04-24T06:21:49.130 INFO:teuthology.orchestra.run.smithi005.stdout:Vendor : CentOS 2024-04-24T06:21:49.130 INFO:teuthology.orchestra.run.smithi005.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-24T06:21:49.130 INFO:teuthology.orchestra.run.smithi005.stdout:Summary : Support tools for GRUB. 2024-04-24T06:21:49.130 INFO:teuthology.orchestra.run.smithi005.stdout:Description : 2024-04-24T06:21:49.130 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:21:49.130 INFO:teuthology.orchestra.run.smithi005.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-24T06:21:49.130 INFO:teuthology.orchestra.run.smithi005.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-24T06:21:49.130 INFO:teuthology.orchestra.run.smithi005.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-24T06:21:49.131 INFO:teuthology.orchestra.run.smithi005.stdout:hardware devices. 2024-04-24T06:21:49.131 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:21:49.131 INFO:teuthology.orchestra.run.smithi005.stdout:This subpackage provides tools for support of all platforms. 2024-04-24T06:21:49.131 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-24T06:21:49.132 INFO:teuthology.task.kernel:Updating grub on smithi005 to boot 6.9.0-rc3-g806ec36b2112 2024-04-24T06:21:49.132 DEBUG:teuthology.orchestra.run.smithi005:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-24T06:21:49.709 INFO:teuthology.orchestra.run.smithi005.stderr:Generating grub configuration file ... 2024-04-24T06:21:51.082 INFO:teuthology.orchestra.run.smithi005.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-24T06:21:51.123 INFO:teuthology.orchestra.run.smithi005.stderr:done 2024-04-24T06:21:51.125 DEBUG:teuthology.orchestra.run.smithi005:> mktemp 2024-04-24T06:21:51.146 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/tmp.9IelTBDQmA 2024-04-24T06:21:51.146 DEBUG:teuthology.orchestra.run.smithi005:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.9IelTBDQmA 2024-04-24T06:21:51.212 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0666 /tmp/tmp.9IelTBDQmA 2024-04-24T06:21:51.351 DEBUG:teuthology.orchestra.remote:smithi005:/tmp/tmp.9IelTBDQmA is 6KB 2024-04-24T06:21:51.400 DEBUG:teuthology.orchestra.run.smithi005:> rm -fr /tmp/tmp.9IelTBDQmA 2024-04-24T06:21:51.414 DEBUG:teuthology.orchestra.run.smithi005:> sudo /bin/ls /boot/loader/entries || true 2024-04-24T06:21:51.477 INFO:teuthology.orchestra.run.smithi005.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-24T06:21:51.477 INFO:teuthology.orchestra.run.smithi005.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-24T06:21:51.477 INFO:teuthology.orchestra.run.smithi005.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-24T06:21:51.477 INFO:teuthology.orchestra.run.smithi005.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-24T06:21:51.478 DEBUG:teuthology.orchestra.run.smithi005:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-24T06:21:51.659 DEBUG:teuthology.orchestra.run.smithi005:> sudo shutdown -r now 2024-04-24T06:22:21.662 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-24T06:22:21.663 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-04-24T06:22:21.664 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T06:23:21.666 DEBUG:teuthology.orchestra.remote:timed out 2024-04-24T06:23:30.668 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-04-24T06:23:30.669 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T06:23:31.103 DEBUG:teuthology.orchestra.run.smithi005:> true 2024-04-24T06:23:31.816 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-04-24T06:23:31.816 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-24T06:23:31.816 DEBUG:teuthology.orchestra.run.smithi005:> uname -r 2024-04-24T06:23:31.859 INFO:teuthology.orchestra.run.smithi005.stdout:6.9.0-rc3-g806ec36b2112 2024-04-24T06:23:31.860 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-24T06:23:31.860 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-24T06:23:31.860 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-24T06:23:32.862 DEBUG:teuthology.parallel:result is None 2024-04-24T06:23:32.863 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-24T06:23:32.870 INFO:teuthology.task.internal:Creating test directory... 2024-04-24T06:23:32.870 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T06:23:32.878 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T06:23:32.881 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T06:23:32.922 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-24T06:23:32.929 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-24T06:23:32.937 INFO:teuthology.task.internal:Creating archive directory... 2024-04-24T06:23:32.937 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T06:23:32.942 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T06:23:32.944 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T06:23:33.004 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-24T06:23:33.013 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-24T06:23:33.013 DEBUG:teuthology.orchestra.run.smithi037:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T06:23:33.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:23:33.031 DEBUG:teuthology.orchestra.run.smithi137:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T06:23:33.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:23:33.047 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T06:23:33.054 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T06:23:33.073 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T06:23:33.111 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T06:23:33.128 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T06:23:33.129 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T06:23:33.156 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T06:23:33.179 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T06:23:33.223 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T06:23:33.227 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-24T06:23:33.236 INFO:teuthology.task.internal:Configuring sudo... 2024-04-24T06:23:33.236 DEBUG:teuthology.orchestra.run.smithi005:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T06:23:33.242 DEBUG:teuthology.orchestra.run.smithi037:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T06:23:33.244 DEBUG:teuthology.orchestra.run.smithi137:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T06:23:33.312 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-24T06:23:33.321 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-24T06:23:33.322 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T06:23:33.326 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T06:23:33.328 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T06:23:33.371 DEBUG:teuthology.orchestra.run.smithi005:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T06:23:33.452 DEBUG:teuthology.orchestra.run.smithi005:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T06:23:33.540 DEBUG:teuthology.orchestra.run.smithi005:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T06:23:33.609 DEBUG:teuthology.orchestra.run.smithi005:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T06:23:33.690 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-24T06:23:33.691 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T06:23:33.771 DEBUG:teuthology.orchestra.run.smithi037:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T06:23:33.816 DEBUG:teuthology.orchestra.run.smithi037:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T06:23:33.870 DEBUG:teuthology.orchestra.run.smithi037:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T06:23:33.956 DEBUG:teuthology.orchestra.run.smithi037:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T06:23:33.999 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-24T06:23:33.999 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T06:23:34.067 DEBUG:teuthology.orchestra.run.smithi137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T06:23:34.112 DEBUG:teuthology.orchestra.run.smithi137:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T06:23:34.170 DEBUG:teuthology.orchestra.run.smithi137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T06:23:34.255 DEBUG:teuthology.orchestra.run.smithi137:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T06:23:34.295 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-24T06:23:34.295 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T06:23:34.361 DEBUG:teuthology.orchestra.run.smithi005:> sudo service rsyslog restart 2024-04-24T06:23:34.366 DEBUG:teuthology.orchestra.run.smithi037:> sudo service rsyslog restart 2024-04-24T06:23:34.368 DEBUG:teuthology.orchestra.run.smithi137:> sudo service rsyslog restart 2024-04-24T06:23:34.411 INFO:teuthology.orchestra.run.smithi037.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T06:23:34.438 INFO:teuthology.orchestra.run.smithi137.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T06:23:34.486 INFO:teuthology.orchestra.run.smithi005.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T06:23:34.911 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-24T06:23:34.920 INFO:teuthology.task.internal:Starting timer... 2024-04-24T06:23:34.920 INFO:teuthology.run_tasks:Running task pcp... 2024-04-24T06:23:34.931 INFO:teuthology.run_tasks:Running task selinux... 2024-04-24T06:23:34.940 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-24T06:23:34.940 DEBUG:teuthology.orchestra.run.smithi005:> sudo service auditd rotate 2024-04-24T06:23:35.059 INFO:teuthology.orchestra.run.smithi005.stdout:Rotating logs: 2024-04-24T06:23:35.064 DEBUG:teuthology.orchestra.run.smithi037:> sudo service auditd rotate 2024-04-24T06:23:35.132 INFO:teuthology.orchestra.run.smithi037.stdout:Rotating logs: 2024-04-24T06:23:35.134 DEBUG:teuthology.orchestra.run.smithi137:> sudo service auditd rotate 2024-04-24T06:23:35.197 INFO:teuthology.orchestra.run.smithi137.stdout:Rotating logs: 2024-04-24T06:23:35.200 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-24T06:23:35.200 DEBUG:teuthology.orchestra.run.smithi005:> /usr/sbin/getenforce 2024-04-24T06:23:35.250 INFO:teuthology.orchestra.run.smithi005.stdout:Permissive 2024-04-24T06:23:35.250 DEBUG:teuthology.orchestra.run.smithi037:> /usr/sbin/getenforce 2024-04-24T06:23:35.270 INFO:teuthology.orchestra.run.smithi037.stdout:Permissive 2024-04-24T06:23:35.270 DEBUG:teuthology.orchestra.run.smithi137:> /usr/sbin/getenforce 2024-04-24T06:23:35.297 INFO:teuthology.orchestra.run.smithi137.stdout:Permissive 2024-04-24T06:23:35.297 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi005.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi037.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi137.front.sepia.ceph.com': 'permissive'} 2024-04-24T06:23:35.298 DEBUG:teuthology.orchestra.run.smithi005:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T06:23:35.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:23:35.370 DEBUG:teuthology.orchestra.run.smithi037:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T06:23:35.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:23:35.398 DEBUG:teuthology.orchestra.run.smithi137:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T06:23:35.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:23:35.423 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-24T06:23:35.423 DEBUG:teuthology.orchestra.run.smithi005:> sudo /usr/sbin/setenforce permissive 2024-04-24T06:23:35.571 DEBUG:teuthology.orchestra.run.smithi037:> sudo /usr/sbin/setenforce permissive 2024-04-24T06:23:35.597 DEBUG:teuthology.orchestra.run.smithi137:> sudo /usr/sbin/setenforce permissive 2024-04-24T06:23:35.620 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-24T06:23:35.630 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-24T06:23:35.635 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-24T06:23:35.792 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-24T06:23:35.813 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-24T06:23:35.814 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi005.front.sepia.ceph.com,smithi037.front.sepia.ceph.com,smithi137.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-24T06:28:49.107 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi005.front.sepia.ceph.com'), Remote(name='ubuntu@smithi037.front.sepia.ceph.com'), Remote(name='ubuntu@smithi137.front.sepia.ceph.com')] 2024-04-24T06:28:49.109 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-04-24T06:28:49.110 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T06:28:49.235 DEBUG:teuthology.orchestra.run.smithi005:> true 2024-04-24T06:28:49.396 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-04-24T06:28:49.397 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-04-24T06:28:49.397 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T06:28:49.473 DEBUG:teuthology.orchestra.run.smithi037:> true 2024-04-24T06:28:49.556 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-04-24T06:28:49.557 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-04-24T06:28:49.557 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T06:28:49.633 DEBUG:teuthology.orchestra.run.smithi137:> true 2024-04-24T06:28:49.709 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-04-24T06:28:49.710 INFO:teuthology.run_tasks:Running task clock... 2024-04-24T06:28:49.722 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-24T06:28:49.722 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T06:28:49.722 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T06:28:49.729 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T06:28:49.729 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T06:28:49.732 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T06:28:49.732 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T06:28:49.762 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T06:28:49.777 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T06:28:49.779 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T06:28:49.793 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T06:28:49.802 INFO:teuthology.orchestra.run.smithi037.stderr:sudo: ntpd: command not found 2024-04-24T06:28:49.814 INFO:teuthology.orchestra.run.smithi037.stdout:506 Cannot talk to daemon 2024-04-24T06:28:49.816 INFO:teuthology.orchestra.run.smithi137.stderr:sudo: ntpd: command not found 2024-04-24T06:28:49.823 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T06:28:49.827 INFO:teuthology.orchestra.run.smithi137.stdout:506 Cannot talk to daemon 2024-04-24T06:28:49.829 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T06:28:49.840 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T06:28:49.843 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T06:28:49.853 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T06:28:49.868 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T06:28:49.901 INFO:teuthology.orchestra.run.smithi037.stderr:bash: line 1: ntpq: command not found 2024-04-24T06:28:49.905 INFO:teuthology.orchestra.run.smithi037.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T06:28:49.905 INFO:teuthology.orchestra.run.smithi037.stdout:=============================================================================== 2024-04-24T06:28:49.905 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T06:28:49.905 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T06:28:49.905 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T06:28:49.905 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T06:28:49.908 INFO:teuthology.orchestra.run.smithi137.stderr:bash: line 1: ntpq: command not found 2024-04-24T06:28:49.913 INFO:teuthology.orchestra.run.smithi137.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T06:28:49.913 INFO:teuthology.orchestra.run.smithi137.stdout:=============================================================================== 2024-04-24T06:28:49.913 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T06:28:49.913 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T06:28:49.913 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T06:28:49.913 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T06:28:49.940 INFO:teuthology.orchestra.run.smithi005.stderr:sudo: ntpd: command not found 2024-04-24T06:28:49.972 INFO:teuthology.orchestra.run.smithi005.stdout:506 Cannot talk to daemon 2024-04-24T06:28:50.016 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T06:28:50.060 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T06:28:50.207 INFO:teuthology.orchestra.run.smithi005.stderr:bash: line 1: ntpq: command not found 2024-04-24T06:28:50.218 INFO:teuthology.orchestra.run.smithi005.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T06:28:50.218 INFO:teuthology.orchestra.run.smithi005.stdout:=============================================================================== 2024-04-24T06:28:50.218 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T06:28:50.218 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T06:28:50.218 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T06:28:50.218 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T06:28:50.219 INFO:teuthology.run_tasks:Running task install... 2024-04-24T06:28:50.227 DEBUG:teuthology.task.install:project ceph 2024-04-24T06:28:50.228 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a3c8a22bd5efe18ab230a8e2d072db8f381a0411'}} 2024-04-24T06:28:50.228 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a3c8a22bd5efe18ab230a8e2d072db8f381a0411'} 2024-04-24T06:28:50.228 INFO:teuthology.task.install:Using flavor: default 2024-04-24T06:28:50.233 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-24T06:28:50.233 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-24T06:28:50.234 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a3c8a22bd5efe18ab230a8e2d072db8f381a0411', 'tag': None, 'wait_for_package': False} 2024-04-24T06:28:50.234 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a3c8a22bd5efe18ab230a8e2d072db8f381a0411 2024-04-24T06:28:50.236 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a3c8a22bd5efe18ab230a8e2d072db8f381a0411', 'tag': None, 'wait_for_package': False} 2024-04-24T06:28:50.236 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a3c8a22bd5efe18ab230a8e2d072db8f381a0411 2024-04-24T06:28:50.237 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a3c8a22bd5efe18ab230a8e2d072db8f381a0411', 'tag': None, 'wait_for_package': False} 2024-04-24T06:28:50.238 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a3c8a22bd5efe18ab230a8e2d072db8f381a0411 2024-04-24T06:28:50.413 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-5/a3c8a22bd5efe18ab230a8e2d072db8f381a0411/centos/9/flavors/default/ 2024-04-24T06:28:50.413 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3124.ga3c8a22b 2024-04-24T06:28:50.466 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-5/a3c8a22bd5efe18ab230a8e2d072db8f381a0411/centos/9/flavors/default/ 2024-04-24T06:28:50.467 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3124.ga3c8a22b 2024-04-24T06:28:50.472 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-5/a3c8a22bd5efe18ab230a8e2d072db8f381a0411/centos/9/flavors/default/ 2024-04-24T06:28:50.472 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3124.ga3c8a22b 2024-04-24T06:28:50.544 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-5/a3c8a22bd5efe18ab230a8e2d072db8f381a0411/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-5/a3c8a22bd5efe18ab230a8e2d072db8f381a0411/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-5/a3c8a22bd5efe18ab230a8e2d072db8f381a0411/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T06:28:50.544 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-24T06:28:50.545 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T06:28:50.576 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-5/a3c8a22bd5efe18ab230a8e2d072db8f381a0411/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-5/a3c8a22bd5efe18ab230a8e2d072db8f381a0411/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-5/a3c8a22bd5efe18ab230a8e2d072db8f381a0411/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T06:28:50.576 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-24T06:28:50.576 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T06:28:50.588 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-5/a3c8a22bd5efe18ab230a8e2d072db8f381a0411/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-5/a3c8a22bd5efe18ab230a8e2d072db8f381a0411/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-5/a3c8a22bd5efe18ab230a8e2d072db8f381a0411/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T06:28:50.588 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-24T06:28:50.588 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T06:28:50.606 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T06:28:50.606 DEBUG:teuthology.orchestra.run.smithi037:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a3c8a22bd5efe18ab230a8e2d072db8f381a0411/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T06:28:50.615 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T06:28:50.615 DEBUG:teuthology.orchestra.run.smithi137:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a3c8a22bd5efe18ab230a8e2d072db8f381a0411/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T06:28:50.638 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T06:28:50.638 DEBUG:teuthology.orchestra.run.smithi005:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a3c8a22bd5efe18ab230a8e2d072db8f381a0411/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T06:28:50.674 DEBUG:teuthology.orchestra.run.smithi037:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T06:28:50.681 DEBUG:teuthology.orchestra.run.smithi137:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T06:28:50.752 DEBUG:teuthology.orchestra.run.smithi037:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T06:28:50.757 DEBUG:teuthology.orchestra.run.smithi137:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T06:28:50.773 DEBUG:teuthology.orchestra.run.smithi005:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T06:28:50.818 INFO:teuthology.orchestra.run.smithi037.stdout:check_obsoletes = 1 2024-04-24T06:28:50.820 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum clean all 2024-04-24T06:28:50.821 INFO:teuthology.orchestra.run.smithi137.stdout:check_obsoletes = 1 2024-04-24T06:28:50.822 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum clean all 2024-04-24T06:28:50.894 DEBUG:teuthology.orchestra.run.smithi005:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T06:28:50.976 INFO:teuthology.orchestra.run.smithi005.stdout:check_obsoletes = 1 2024-04-24T06:28:50.980 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum clean all 2024-04-24T06:28:51.047 INFO:teuthology.orchestra.run.smithi137.stdout:48 files removed 2024-04-24T06:28:51.057 INFO:teuthology.orchestra.run.smithi037.stdout:48 files removed 2024-04-24T06:28:51.075 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T06:28:51.084 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T06:28:51.305 INFO:teuthology.orchestra.run.smithi005.stdout:48 files removed 2024-04-24T06:28:51.345 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T06:28:51.730 INFO:teuthology.orchestra.run.smithi137.stdout:ceph packages for x86_64 212 kB/s | 83 kB 00:00 2024-04-24T06:28:51.731 INFO:teuthology.orchestra.run.smithi037.stdout:ceph packages for x86_64 223 kB/s | 83 kB 00:00 2024-04-24T06:28:52.048 INFO:teuthology.orchestra.run.smithi137.stdout:ceph noarch packages 43 kB/s | 12 kB 00:00 2024-04-24T06:28:52.156 INFO:teuthology.orchestra.run.smithi037.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-04-24T06:28:52.209 INFO:teuthology.orchestra.run.smithi005.stdout:ceph packages for x86_64 201 kB/s | 83 kB 00:00 2024-04-24T06:28:52.448 INFO:teuthology.orchestra.run.smithi137.stdout:ceph source packages 5.2 kB/s | 1.9 kB 00:00 2024-04-24T06:28:52.507 INFO:teuthology.orchestra.run.smithi037.stdout:ceph source packages 6.0 kB/s | 1.9 kB 00:00 2024-04-24T06:28:52.810 INFO:teuthology.orchestra.run.smithi005.stdout:ceph noarch packages 23 kB/s | 12 kB 00:00 2024-04-24T06:28:53.065 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-04-24T06:28:53.277 INFO:teuthology.orchestra.run.smithi005.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-04-24T06:28:54.148 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - BaseOS 4.8 MB/s | 8.0 MB 00:01 2024-04-24T06:28:55.200 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - BaseOS 4.4 MB/s | 8.0 MB 00:01 2024-04-24T06:28:55.208 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - AppStream 18 MB/s | 19 MB 00:01 2024-04-24T06:28:59.158 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - AppStream 4.8 MB/s | 19 MB 00:03 2024-04-24T06:28:59.959 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - CRB 6.2 MB/s | 6.2 MB 00:01 2024-04-24T06:29:01.786 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - Extras packages 26 kB/s | 16 kB 00:00 2024-04-24T06:29:05.095 INFO:teuthology.orchestra.run.smithi037.stdout:Extra Packages for Enterprise Linux 6.6 MB/s | 21 MB 00:03 2024-04-24T06:29:10.353 INFO:teuthology.orchestra.run.smithi037.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-04-24T06:29:10.423 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - CRB 818 kB/s | 6.2 MB 00:07 2024-04-24T06:29:12.546 INFO:teuthology.orchestra.run.smithi037.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T06:29:12.547 INFO:teuthology.orchestra.run.smithi037.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T06:29:12.558 INFO:teuthology.orchestra.run.smithi037.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T06:29:12.559 INFO:teuthology.orchestra.run.smithi037.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T06:29:12.559 INFO:teuthology.orchestra.run.smithi037.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T06:29:12.560 INFO:teuthology.orchestra.run.smithi037.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T06:29:12.562 INFO:teuthology.orchestra.run.smithi037.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T06:29:12.564 INFO:teuthology.orchestra.run.smithi037.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T06:29:12.565 INFO:teuthology.orchestra.run.smithi037.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T06:29:12.629 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-04-24T06:29:12.654 INFO:teuthology.orchestra.run.smithi037.stdout:======================================================================================= 2024-04-24T06:29:12.654 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repository Size 2024-04-24T06:29:12.654 INFO:teuthology.orchestra.run.smithi037.stdout:======================================================================================= 2024-04-24T06:29:12.654 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-04-24T06:29:12.654 INFO:teuthology.orchestra.run.smithi037.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T06:29:12.654 INFO:teuthology.orchestra.run.smithi037.stdout: ceph x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 6.6 k 2024-04-24T06:29:12.654 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-base x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 5.4 M 2024-04-24T06:29:12.654 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-fuse x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 890 k 2024-04-24T06:29:12.655 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 145 k 2024-04-24T06:29:12.655 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 1.5 M 2024-04-24T06:29:12.655 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 142 k 2024-04-24T06:29:12.655 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 3.5 M 2024-04-24T06:29:12.655 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 7.4 M 2024-04-24T06:29:12.655 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-rook noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 49 k 2024-04-24T06:29:12.655 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-radosgw x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 11 M 2024-04-24T06:29:12.655 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-test x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 49 M 2024-04-24T06:29:12.655 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-volume noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 273 k 2024-04-24T06:29:12.655 INFO:teuthology.orchestra.run.smithi037.stdout: cephadm noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 762 k 2024-04-24T06:29:12.655 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-mirror x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 224 k 2024-04-24T06:29:12.655 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-shell x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 27 k 2024-04-24T06:29:12.655 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-top noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 18 k 2024-04-24T06:29:12.656 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T06:29:12.656 INFO:teuthology.orchestra.run.smithi037.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T06:29:12.656 INFO:teuthology.orchestra.run.smithi037.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T06:29:12.656 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs-devel x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 32 k 2024-04-24T06:29:12.656 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs2 x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 747 k 2024-04-24T06:29:12.656 INFO:teuthology.orchestra.run.smithi037.stdout: librados-devel x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 127 k 2024-04-24T06:29:12.656 INFO:teuthology.orchestra.run.smithi037.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T06:29:12.656 INFO:teuthology.orchestra.run.smithi037.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T06:29:12.656 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T06:29:12.656 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T06:29:12.656 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T06:29:12.656 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T06:29:12.657 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cephfs x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 162 k 2024-04-24T06:29:12.657 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rados x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 322 k 2024-04-24T06:29:12.657 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rbd x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 297 k 2024-04-24T06:29:12.657 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rgw x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 99 k 2024-04-24T06:29:12.657 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-fuse x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 85 k 2024-04-24T06:29:12.657 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-mirror x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 3.1 M 2024-04-24T06:29:12.657 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-nbd x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 170 k 2024-04-24T06:29:12.658 INFO:teuthology.orchestra.run.smithi037.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T06:29:12.658 INFO:teuthology.orchestra.run.smithi037.stdout:Upgrading: 2024-04-24T06:29:12.658 INFO:teuthology.orchestra.run.smithi037.stdout: librados2 x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 3.4 M 2024-04-24T06:29:12.658 INFO:teuthology.orchestra.run.smithi037.stdout: librbd1 x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 3.1 M 2024-04-24T06:29:12.658 INFO:teuthology.orchestra.run.smithi037.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T06:29:12.658 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T06:29:12.658 INFO:teuthology.orchestra.run.smithi037.stdout:Installing dependencies: 2024-04-24T06:29:12.658 INFO:teuthology.orchestra.run.smithi037.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T06:29:12.658 INFO:teuthology.orchestra.run.smithi037.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T06:29:12.658 INFO:teuthology.orchestra.run.smithi037.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T06:29:12.658 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-common x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 21 M 2024-04-24T06:29:12.658 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 34 k 2024-04-24T06:29:12.659 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mds x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 2.4 M 2024-04-24T06:29:12.659 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 249 k 2024-04-24T06:29:12.659 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mon x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 4.7 M 2024-04-24T06:29:12.659 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-osd x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 17 M 2024-04-24T06:29:12.659 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 16 k 2024-04-24T06:29:12.659 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-selinux x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 25 k 2024-04-24T06:29:12.659 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T06:29:12.659 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T06:29:12.660 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T06:29:12.660 INFO:teuthology.orchestra.run.smithi037.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T06:29:12.660 INFO:teuthology.orchestra.run.smithi037.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T06:29:12.660 INFO:teuthology.orchestra.run.smithi037.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T06:29:12.660 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T06:29:12.660 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T06:29:12.660 INFO:teuthology.orchestra.run.smithi037.stdout: libcephsqlite x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 163 k 2024-04-24T06:29:12.661 INFO:teuthology.orchestra.run.smithi037.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T06:29:12.661 INFO:teuthology.orchestra.run.smithi037.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T06:29:12.661 INFO:teuthology.orchestra.run.smithi037.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T06:29:12.661 INFO:teuthology.orchestra.run.smithi037.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T06:29:12.661 INFO:teuthology.orchestra.run.smithi037.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T06:29:12.661 INFO:teuthology.orchestra.run.smithi037.stdout: libradosstriper1 x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 501 k 2024-04-24T06:29:12.661 INFO:teuthology.orchestra.run.smithi037.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T06:29:12.662 INFO:teuthology.orchestra.run.smithi037.stdout: librgw2 x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 5.4 M 2024-04-24T06:29:12.662 INFO:teuthology.orchestra.run.smithi037.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T06:29:12.662 INFO:teuthology.orchestra.run.smithi037.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T06:29:12.662 INFO:teuthology.orchestra.run.smithi037.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T06:29:12.662 INFO:teuthology.orchestra.run.smithi037.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T06:29:12.662 INFO:teuthology.orchestra.run.smithi037.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T06:29:12.663 INFO:teuthology.orchestra.run.smithi037.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T06:29:12.663 INFO:teuthology.orchestra.run.smithi037.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T06:29:12.663 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T06:29:12.663 INFO:teuthology.orchestra.run.smithi037.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T06:29:12.663 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T06:29:12.663 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T06:29:12.663 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T06:29:12.663 INFO:teuthology.orchestra.run.smithi037.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T06:29:12.663 INFO:teuthology.orchestra.run.smithi037.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T06:29:12.663 INFO:teuthology.orchestra.run.smithi037.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T06:29:12.663 INFO:teuthology.orchestra.run.smithi037.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T06:29:12.663 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T06:29:12.663 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-argparse x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 45 k 2024-04-24T06:29:12.664 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-common x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 135 k 2024-04-24T06:29:12.664 INFO:teuthology.orchestra.run.smithi037.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T06:29:12.664 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T06:29:12.664 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T06:29:12.664 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T06:29:12.664 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T06:29:12.664 INFO:teuthology.orchestra.run.smithi037.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T06:29:12.664 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T06:29:12.664 INFO:teuthology.orchestra.run.smithi037.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T06:29:12.664 INFO:teuthology.orchestra.run.smithi037.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T06:29:12.664 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T06:29:12.664 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T06:29:12.664 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T06:29:12.664 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T06:29:12.665 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T06:29:12.665 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T06:29:12.665 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T06:29:12.665 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T06:29:12.665 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T06:29:12.665 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T06:29:12.665 INFO:teuthology.orchestra.run.smithi037.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T06:29:12.665 INFO:teuthology.orchestra.run.smithi037.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T06:29:12.665 INFO:teuthology.orchestra.run.smithi037.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T06:29:12.665 INFO:teuthology.orchestra.run.smithi037.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T06:29:12.665 INFO:teuthology.orchestra.run.smithi037.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T06:29:12.665 INFO:teuthology.orchestra.run.smithi037.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T06:29:12.665 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T06:29:12.665 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T06:29:12.665 INFO:teuthology.orchestra.run.smithi037.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T06:29:12.666 INFO:teuthology.orchestra.run.smithi037.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T06:29:12.666 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T06:29:12.666 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T06:29:12.666 INFO:teuthology.orchestra.run.smithi037.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T06:29:12.666 INFO:teuthology.orchestra.run.smithi037.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T06:29:12.666 INFO:teuthology.orchestra.run.smithi037.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T06:29:12.666 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T06:29:12.666 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T06:29:12.666 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T06:29:12.666 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T06:29:12.666 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T06:29:12.666 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T06:29:12.666 INFO:teuthology.orchestra.run.smithi037.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T06:29:12.666 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T06:29:12.667 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T06:29:12.667 INFO:teuthology.orchestra.run.smithi037.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T06:29:12.667 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T06:29:12.667 INFO:teuthology.orchestra.run.smithi037.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T06:29:12.667 INFO:teuthology.orchestra.run.smithi037.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T06:29:12.667 INFO:teuthology.orchestra.run.smithi037.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T06:29:12.667 INFO:teuthology.orchestra.run.smithi037.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T06:29:12.667 INFO:teuthology.orchestra.run.smithi037.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T06:29:12.667 INFO:teuthology.orchestra.run.smithi037.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T06:29:12.667 INFO:teuthology.orchestra.run.smithi037.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T06:29:12.667 INFO:teuthology.orchestra.run.smithi037.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T06:29:12.667 INFO:teuthology.orchestra.run.smithi037.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T06:29:12.667 INFO:teuthology.orchestra.run.smithi037.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T06:29:12.667 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T06:29:12.668 INFO:teuthology.orchestra.run.smithi037.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T06:29:12.668 INFO:teuthology.orchestra.run.smithi037.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T06:29:12.668 INFO:teuthology.orchestra.run.smithi037.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T06:29:12.668 INFO:teuthology.orchestra.run.smithi037.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T06:29:12.668 INFO:teuthology.orchestra.run.smithi037.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T06:29:12.668 INFO:teuthology.orchestra.run.smithi037.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T06:29:12.668 INFO:teuthology.orchestra.run.smithi037.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T06:29:12.668 INFO:teuthology.orchestra.run.smithi037.stdout:Installing weak dependencies: 2024-04-24T06:29:12.668 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T06:29:12.668 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-24T06:29:12.668 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-04-24T06:29:12.668 INFO:teuthology.orchestra.run.smithi037.stdout:======================================================================================= 2024-04-24T06:29:12.668 INFO:teuthology.orchestra.run.smithi037.stdout:Install 148 Packages 2024-04-24T06:29:12.668 INFO:teuthology.orchestra.run.smithi037.stdout:Upgrade 4 Packages 2024-04-24T06:29:12.668 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-24T06:29:12.669 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 222 M 2024-04-24T06:29:12.669 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-04-24T06:29:12.850 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - Extras packages 12 kB/s | 16 kB 00:01 2024-04-24T06:29:13.464 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 19 MB 00:17 2024-04-24T06:29:13.554 INFO:teuthology.orchestra.run.smithi037.stdout:(1/152): ceph-19.0.0-3124.ga3c8a22b.el9.x86_64. 25 kB/s | 6.6 kB 00:00 2024-04-24T06:29:13.887 INFO:teuthology.orchestra.run.smithi137.stdout:Extra Packages for Enterprise Linux 23 MB/s | 21 MB 00:00 2024-04-24T06:29:13.921 INFO:teuthology.orchestra.run.smithi037.stdout:(2/152): ceph-base-19.0.0-3124.ga3c8a22b.el9.x8 8.6 MB/s | 5.4 MB 00:00 2024-04-24T06:29:13.997 INFO:teuthology.orchestra.run.smithi037.stdout:(3/152): ceph-immutable-object-cache-19.0.0-312 1.9 MB/s | 145 kB 00:00 2024-04-24T06:29:14.239 INFO:teuthology.orchestra.run.smithi037.stdout:(4/152): ceph-mds-19.0.0-3124.ga3c8a22b.el9.x86 9.8 MB/s | 2.4 MB 00:00 2024-04-24T06:29:14.398 INFO:teuthology.orchestra.run.smithi037.stdout:(5/152): ceph-mgr-19.0.0-3124.ga3c8a22b.el9.x86 9.4 MB/s | 1.5 MB 00:00 2024-04-24T06:29:14.641 INFO:teuthology.orchestra.run.smithi037.stdout:(6/152): ceph-fuse-19.0.0-3124.ga3c8a22b.el9.x8 821 kB/s | 890 kB 00:01 2024-04-24T06:29:14.739 INFO:teuthology.orchestra.run.smithi037.stdout:(7/152): ceph-mon-19.0.0-3124.ga3c8a22b.el9.x86 14 MB/s | 4.7 MB 00:00 2024-04-24T06:29:15.123 INFO:teuthology.orchestra.run.smithi037.stdout:(8/152): ceph-common-19.0.0-3124.ga3c8a22b.el9. 12 MB/s | 21 MB 00:01 2024-04-24T06:29:15.165 INFO:teuthology.orchestra.run.smithi037.stdout:(9/152): ceph-selinux-19.0.0-3124.ga3c8a22b.el9 608 kB/s | 25 kB 00:00 2024-04-24T06:29:15.524 INFO:teuthology.orchestra.run.smithi037.stdout:(10/152): ceph-radosgw-19.0.0-3124.ga3c8a22b.el 14 MB/s | 11 MB 00:00 2024-04-24T06:29:15.591 INFO:teuthology.orchestra.run.smithi037.stdout:(11/152): cephfs-mirror-19.0.0-3124.ga3c8a22b.e 3.3 MB/s | 224 kB 00:00 2024-04-24T06:29:15.649 INFO:teuthology.orchestra.run.smithi037.stdout:(12/152): cephfs-shell-19.0.0-3124.ga3c8a22b.el 464 kB/s | 27 kB 00:00 2024-04-24T06:29:15.708 INFO:teuthology.orchestra.run.smithi037.stdout:(13/152): libcephfs-devel-19.0.0-3124.ga3c8a22b 544 kB/s | 32 kB 00:00 2024-04-24T06:29:15.808 INFO:teuthology.orchestra.run.smithi037.stdout:(14/152): libcephfs2-19.0.0-3124.ga3c8a22b.el9. 7.3 MB/s | 747 kB 00:00 2024-04-24T06:29:15.867 INFO:teuthology.orchestra.run.smithi037.stdout:(15/152): libcephsqlite-19.0.0-3124.ga3c8a22b.e 2.7 MB/s | 163 kB 00:00 2024-04-24T06:29:15.925 INFO:teuthology.orchestra.run.smithi037.stdout:(16/152): librados-devel-19.0.0-3124.ga3c8a22b. 2.1 MB/s | 127 kB 00:00 2024-04-24T06:29:16.017 INFO:teuthology.orchestra.run.smithi037.stdout:(17/152): libradosstriper1-19.0.0-3124.ga3c8a22 5.3 MB/s | 501 kB 00:00 2024-04-24T06:29:16.409 INFO:teuthology.orchestra.run.smithi037.stdout:(18/152): librgw2-19.0.0-3124.ga3c8a22b.el9.x86 14 MB/s | 5.4 MB 00:00 2024-04-24T06:29:16.485 INFO:teuthology.orchestra.run.smithi037.stdout:(19/152): python3-ceph-argparse-19.0.0-3124.ga3 604 kB/s | 45 kB 00:00 2024-04-24T06:29:16.552 INFO:teuthology.orchestra.run.smithi037.stdout:(20/152): python3-ceph-common-19.0.0-3124.ga3c8 2.0 MB/s | 135 kB 00:00 2024-04-24T06:29:16.618 INFO:teuthology.orchestra.run.smithi037.stdout:(21/152): python3-cephfs-19.0.0-3124.ga3c8a22b. 2.4 MB/s | 162 kB 00:00 2024-04-24T06:29:16.685 INFO:teuthology.orchestra.run.smithi037.stdout:(22/152): python3-rados-19.0.0-3124.ga3c8a22b.e 4.7 MB/s | 322 kB 00:00 2024-04-24T06:29:16.754 INFO:teuthology.orchestra.run.smithi037.stdout:(23/152): python3-rbd-19.0.0-3124.ga3c8a22b.el9 4.4 MB/s | 297 kB 00:00 2024-04-24T06:29:16.811 INFO:teuthology.orchestra.run.smithi037.stdout:(24/152): python3-rgw-19.0.0-3124.ga3c8a22b.el9 1.7 MB/s | 99 kB 00:00 2024-04-24T06:29:16.861 INFO:teuthology.orchestra.run.smithi037.stdout:(25/152): rbd-fuse-19.0.0-3124.ga3c8a22b.el9.x8 1.7 MB/s | 85 kB 00:00 2024-04-24T06:29:17.104 INFO:teuthology.orchestra.run.smithi037.stdout:(26/152): rbd-mirror-19.0.0-3124.ga3c8a22b.el9. 13 MB/s | 3.1 MB 00:00 2024-04-24T06:29:17.154 INFO:teuthology.orchestra.run.smithi037.stdout:(27/152): rbd-nbd-19.0.0-3124.ga3c8a22b.el9.x86 3.3 MB/s | 170 kB 00:00 2024-04-24T06:29:17.204 INFO:teuthology.orchestra.run.smithi037.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3124.g 686 kB/s | 34 kB 00:00 2024-04-24T06:29:17.262 INFO:teuthology.orchestra.run.smithi037.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3124.ga3c8a22 2.4 MB/s | 142 kB 00:00 2024-04-24T06:29:17.554 INFO:teuthology.orchestra.run.smithi037.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3124.ga3c8a 12 MB/s | 3.5 MB 00:00 2024-04-24T06:29:18.073 INFO:teuthology.orchestra.run.smithi037.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 14 MB/s | 7.4 MB 00:00 2024-04-24T06:29:18.149 INFO:teuthology.orchestra.run.smithi037.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3124.ga3 3.2 MB/s | 249 kB 00:00 2024-04-24T06:29:18.207 INFO:teuthology.orchestra.run.smithi037.stdout:(33/152): ceph-mgr-rook-19.0.0-3124.ga3c8a22b.e 847 kB/s | 49 kB 00:00 2024-04-24T06:29:18.266 INFO:teuthology.orchestra.run.smithi037.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3124.ga 283 kB/s | 16 kB 00:00 2024-04-24T06:29:18.341 INFO:teuthology.orchestra.run.smithi037.stdout:(35/152): ceph-volume-19.0.0-3124.ga3c8a22b.el9 3.6 MB/s | 273 kB 00:00 2024-04-24T06:29:18.433 INFO:teuthology.orchestra.run.smithi037.stdout:(36/152): cephadm-19.0.0-3124.ga3c8a22b.el9.noa 8.1 MB/s | 762 kB 00:00 2024-04-24T06:29:18.616 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - CRB 5.6 MB/s | 6.2 MB 00:01 2024-04-24T06:29:18.900 INFO:teuthology.orchestra.run.smithi037.stdout:(37/152): ceph-test-19.0.0-3124.ga3c8a22b.el9.x 13 MB/s | 49 MB 00:03 2024-04-24T06:29:18.925 INFO:teuthology.orchestra.run.smithi037.stdout:(38/152): cephfs-top-19.0.0-3124.ga3c8a22b.el9. 37 kB/s | 18 kB 00:00 2024-04-24T06:29:19.026 INFO:teuthology.orchestra.run.smithi137.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-24T06:29:19.076 INFO:teuthology.orchestra.run.smithi037.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 459 kB/s | 80 kB 00:00 2024-04-24T06:29:19.168 INFO:teuthology.orchestra.run.smithi037.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2024-04-24T06:29:19.209 INFO:teuthology.orchestra.run.smithi037.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.8 MB/s | 803 kB 00:00 2024-04-24T06:29:19.251 INFO:teuthology.orchestra.run.smithi037.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-04-24T06:29:19.310 INFO:teuthology.orchestra.run.smithi037.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-24T06:29:19.352 INFO:teuthology.orchestra.run.smithi037.stdout:(44/152): python3-cryptography-36.0.1-4.el9.x86 8.6 MB/s | 1.2 MB 00:00 2024-04-24T06:29:19.377 INFO:teuthology.orchestra.run.smithi037.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-04-24T06:29:19.402 INFO:teuthology.orchestra.run.smithi037.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-24T06:29:19.436 INFO:teuthology.orchestra.run.smithi037.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-04-24T06:29:20.153 INFO:teuthology.orchestra.run.smithi037.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 918 kB/s | 689 kB 00:00 2024-04-24T06:29:20.262 INFO:teuthology.orchestra.run.smithi037.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 988 kB/s | 107 kB 00:00 2024-04-24T06:29:20.295 INFO:teuthology.orchestra.run.smithi037.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 1.1 MB/s | 943 kB 00:00 2024-04-24T06:29:20.354 INFO:teuthology.orchestra.run.smithi037.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 234 kB/s | 21 kB 00:00 2024-04-24T06:29:20.404 INFO:teuthology.orchestra.run.smithi037.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-04-24T06:29:20.446 INFO:teuthology.orchestra.run.smithi037.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 355 kB/s | 33 kB 00:00 2024-04-24T06:29:20.573 INFO:teuthology.orchestra.run.smithi037.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 144 kB/s | 18 kB 00:00 2024-04-24T06:29:20.680 INFO:teuthology.orchestra.run.smithi037.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 104 kB/s | 11 kB 00:00 2024-04-24T06:29:20.780 INFO:teuthology.orchestra.run.smithi037.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-04-24T06:29:20.881 INFO:teuthology.orchestra.run.smithi037.stdout:(57/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.3 MB/s | 3.0 MB 00:00 2024-04-24T06:29:20.898 INFO:teuthology.orchestra.run.smithi037.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 387 kB/s | 45 kB 00:00 2024-04-24T06:29:21.090 INFO:teuthology.orchestra.run.smithi037.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 3.0 MB/s | 582 kB 00:00 2024-04-24T06:29:21.143 INFO:teuthology.orchestra.run.smithi137.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T06:29:21.144 INFO:teuthology.orchestra.run.smithi137.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T06:29:21.155 INFO:teuthology.orchestra.run.smithi137.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T06:29:21.156 INFO:teuthology.orchestra.run.smithi137.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T06:29:21.157 INFO:teuthology.orchestra.run.smithi137.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T06:29:21.158 INFO:teuthology.orchestra.run.smithi137.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T06:29:21.159 INFO:teuthology.orchestra.run.smithi137.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T06:29:21.161 INFO:teuthology.orchestra.run.smithi137.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T06:29:21.162 INFO:teuthology.orchestra.run.smithi137.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T06:29:21.190 INFO:teuthology.orchestra.run.smithi037.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 255 kB/s | 25 kB 00:00 2024-04-24T06:29:21.223 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-04-24T06:29:21.242 INFO:teuthology.orchestra.run.smithi137.stdout:======================================================================================= 2024-04-24T06:29:21.242 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-04-24T06:29:21.242 INFO:teuthology.orchestra.run.smithi137.stdout:======================================================================================= 2024-04-24T06:29:21.242 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-04-24T06:29:21.243 INFO:teuthology.orchestra.run.smithi137.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T06:29:21.243 INFO:teuthology.orchestra.run.smithi137.stdout: ceph x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 6.6 k 2024-04-24T06:29:21.243 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-base x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 5.4 M 2024-04-24T06:29:21.243 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-fuse x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 890 k 2024-04-24T06:29:21.243 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 145 k 2024-04-24T06:29:21.243 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 1.5 M 2024-04-24T06:29:21.243 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 142 k 2024-04-24T06:29:21.243 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 3.5 M 2024-04-24T06:29:21.243 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 7.4 M 2024-04-24T06:29:21.243 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-rook noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 49 k 2024-04-24T06:29:21.243 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-radosgw x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 11 M 2024-04-24T06:29:21.243 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-test x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 49 M 2024-04-24T06:29:21.243 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-volume noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 273 k 2024-04-24T06:29:21.244 INFO:teuthology.orchestra.run.smithi137.stdout: cephadm noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 762 k 2024-04-24T06:29:21.244 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-mirror x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 224 k 2024-04-24T06:29:21.244 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-shell x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 27 k 2024-04-24T06:29:21.244 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-top noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 18 k 2024-04-24T06:29:21.244 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T06:29:21.244 INFO:teuthology.orchestra.run.smithi137.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T06:29:21.244 INFO:teuthology.orchestra.run.smithi137.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T06:29:21.244 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs-devel x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 32 k 2024-04-24T06:29:21.244 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs2 x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 747 k 2024-04-24T06:29:21.244 INFO:teuthology.orchestra.run.smithi137.stdout: librados-devel x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 127 k 2024-04-24T06:29:21.244 INFO:teuthology.orchestra.run.smithi137.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T06:29:21.244 INFO:teuthology.orchestra.run.smithi137.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T06:29:21.244 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T06:29:21.245 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T06:29:21.245 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T06:29:21.245 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T06:29:21.245 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cephfs x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 162 k 2024-04-24T06:29:21.245 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rados x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 322 k 2024-04-24T06:29:21.245 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rbd x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 297 k 2024-04-24T06:29:21.245 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rgw x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 99 k 2024-04-24T06:29:21.245 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-fuse x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 85 k 2024-04-24T06:29:21.245 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-mirror x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 3.1 M 2024-04-24T06:29:21.245 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-nbd x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 170 k 2024-04-24T06:29:21.245 INFO:teuthology.orchestra.run.smithi137.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T06:29:21.245 INFO:teuthology.orchestra.run.smithi137.stdout:Upgrading: 2024-04-24T06:29:21.245 INFO:teuthology.orchestra.run.smithi137.stdout: librados2 x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 3.4 M 2024-04-24T06:29:21.246 INFO:teuthology.orchestra.run.smithi137.stdout: librbd1 x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 3.1 M 2024-04-24T06:29:21.246 INFO:teuthology.orchestra.run.smithi137.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T06:29:21.246 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T06:29:21.246 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-04-24T06:29:21.246 INFO:teuthology.orchestra.run.smithi137.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T06:29:21.246 INFO:teuthology.orchestra.run.smithi137.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T06:29:21.246 INFO:teuthology.orchestra.run.smithi137.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T06:29:21.246 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-common x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 21 M 2024-04-24T06:29:21.246 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 34 k 2024-04-24T06:29:21.246 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mds x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 2.4 M 2024-04-24T06:29:21.246 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 249 k 2024-04-24T06:29:21.246 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mon x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 4.7 M 2024-04-24T06:29:21.247 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-osd x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 17 M 2024-04-24T06:29:21.247 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 16 k 2024-04-24T06:29:21.247 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-selinux x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 25 k 2024-04-24T06:29:21.247 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T06:29:21.247 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T06:29:21.247 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T06:29:21.247 INFO:teuthology.orchestra.run.smithi137.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T06:29:21.247 INFO:teuthology.orchestra.run.smithi137.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T06:29:21.247 INFO:teuthology.orchestra.run.smithi137.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T06:29:21.247 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T06:29:21.247 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T06:29:21.247 INFO:teuthology.orchestra.run.smithi137.stdout: libcephsqlite x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 163 k 2024-04-24T06:29:21.248 INFO:teuthology.orchestra.run.smithi137.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T06:29:21.248 INFO:teuthology.orchestra.run.smithi137.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T06:29:21.248 INFO:teuthology.orchestra.run.smithi137.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T06:29:21.248 INFO:teuthology.orchestra.run.smithi137.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T06:29:21.248 INFO:teuthology.orchestra.run.smithi137.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T06:29:21.248 INFO:teuthology.orchestra.run.smithi137.stdout: libradosstriper1 x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 501 k 2024-04-24T06:29:21.248 INFO:teuthology.orchestra.run.smithi137.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T06:29:21.248 INFO:teuthology.orchestra.run.smithi137.stdout: librgw2 x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 5.4 M 2024-04-24T06:29:21.248 INFO:teuthology.orchestra.run.smithi137.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T06:29:21.248 INFO:teuthology.orchestra.run.smithi137.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T06:29:21.248 INFO:teuthology.orchestra.run.smithi137.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T06:29:21.249 INFO:teuthology.orchestra.run.smithi137.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T06:29:21.249 INFO:teuthology.orchestra.run.smithi137.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T06:29:21.249 INFO:teuthology.orchestra.run.smithi137.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T06:29:21.249 INFO:teuthology.orchestra.run.smithi137.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T06:29:21.249 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T06:29:21.249 INFO:teuthology.orchestra.run.smithi137.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T06:29:21.249 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T06:29:21.249 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T06:29:21.249 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T06:29:21.249 INFO:teuthology.orchestra.run.smithi137.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T06:29:21.249 INFO:teuthology.orchestra.run.smithi137.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T06:29:21.249 INFO:teuthology.orchestra.run.smithi137.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T06:29:21.250 INFO:teuthology.orchestra.run.smithi137.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T06:29:21.250 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T06:29:21.250 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-argparse x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 45 k 2024-04-24T06:29:21.250 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-common x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 135 k 2024-04-24T06:29:21.250 INFO:teuthology.orchestra.run.smithi137.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T06:29:21.250 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T06:29:21.250 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T06:29:21.250 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T06:29:21.250 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T06:29:21.250 INFO:teuthology.orchestra.run.smithi137.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T06:29:21.250 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T06:29:21.250 INFO:teuthology.orchestra.run.smithi137.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T06:29:21.250 INFO:teuthology.orchestra.run.smithi137.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T06:29:21.250 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T06:29:21.251 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T06:29:21.251 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T06:29:21.251 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T06:29:21.251 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T06:29:21.251 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T06:29:21.251 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T06:29:21.251 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T06:29:21.251 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T06:29:21.251 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T06:29:21.251 INFO:teuthology.orchestra.run.smithi137.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T06:29:21.251 INFO:teuthology.orchestra.run.smithi137.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T06:29:21.251 INFO:teuthology.orchestra.run.smithi137.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T06:29:21.252 INFO:teuthology.orchestra.run.smithi137.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T06:29:21.252 INFO:teuthology.orchestra.run.smithi137.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T06:29:21.252 INFO:teuthology.orchestra.run.smithi137.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T06:29:21.252 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T06:29:21.252 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T06:29:21.252 INFO:teuthology.orchestra.run.smithi137.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T06:29:21.252 INFO:teuthology.orchestra.run.smithi137.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T06:29:21.252 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T06:29:21.252 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T06:29:21.252 INFO:teuthology.orchestra.run.smithi137.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T06:29:21.252 INFO:teuthology.orchestra.run.smithi137.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T06:29:21.252 INFO:teuthology.orchestra.run.smithi137.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T06:29:21.252 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T06:29:21.252 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T06:29:21.253 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T06:29:21.253 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T06:29:21.253 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T06:29:21.253 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T06:29:21.253 INFO:teuthology.orchestra.run.smithi137.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T06:29:21.253 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T06:29:21.253 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T06:29:21.253 INFO:teuthology.orchestra.run.smithi137.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T06:29:21.253 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T06:29:21.253 INFO:teuthology.orchestra.run.smithi137.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T06:29:21.253 INFO:teuthology.orchestra.run.smithi137.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T06:29:21.253 INFO:teuthology.orchestra.run.smithi137.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T06:29:21.253 INFO:teuthology.orchestra.run.smithi137.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T06:29:21.253 INFO:teuthology.orchestra.run.smithi137.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T06:29:21.254 INFO:teuthology.orchestra.run.smithi137.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T06:29:21.254 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T06:29:21.254 INFO:teuthology.orchestra.run.smithi137.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T06:29:21.254 INFO:teuthology.orchestra.run.smithi137.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T06:29:21.254 INFO:teuthology.orchestra.run.smithi137.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T06:29:21.254 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T06:29:21.254 INFO:teuthology.orchestra.run.smithi137.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T06:29:21.254 INFO:teuthology.orchestra.run.smithi137.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T06:29:21.254 INFO:teuthology.orchestra.run.smithi137.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T06:29:21.254 INFO:teuthology.orchestra.run.smithi137.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T06:29:21.254 INFO:teuthology.orchestra.run.smithi137.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T06:29:21.254 INFO:teuthology.orchestra.run.smithi137.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T06:29:21.255 INFO:teuthology.orchestra.run.smithi137.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T06:29:21.255 INFO:teuthology.orchestra.run.smithi137.stdout:Installing weak dependencies: 2024-04-24T06:29:21.255 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T06:29:21.255 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-24T06:29:21.255 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-04-24T06:29:21.255 INFO:teuthology.orchestra.run.smithi137.stdout:======================================================================================= 2024-04-24T06:29:21.255 INFO:teuthology.orchestra.run.smithi137.stdout:Install 148 Packages 2024-04-24T06:29:21.255 INFO:teuthology.orchestra.run.smithi137.stdout:Upgrade 4 Packages 2024-04-24T06:29:21.255 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-24T06:29:21.256 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 222 M 2024-04-24T06:29:21.256 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-04-24T06:29:21.299 INFO:teuthology.orchestra.run.smithi037.stdout:(61/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 437 kB/s | 47 kB 00:00 2024-04-24T06:29:21.407 INFO:teuthology.orchestra.run.smithi037.stdout:(62/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-04-24T06:29:21.516 INFO:teuthology.orchestra.run.smithi037.stdout:(63/152): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-04-24T06:29:21.717 INFO:teuthology.orchestra.run.smithi037.stdout:(64/152): ceph-osd-19.0.0-3124.ga3c8a22b.el9.x8 2.4 MB/s | 17 MB 00:07 2024-04-24T06:29:21.742 INFO:teuthology.orchestra.run.smithi037.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 156 kB/s | 35 kB 00:00 2024-04-24T06:29:21.824 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - Extras packages 8.7 kB/s | 16 kB 00:01 2024-04-24T06:29:21.850 INFO:teuthology.orchestra.run.smithi037.stdout:(66/152): librdkafka-1.6.1-102.el9.x86_64.rpm 683 kB/s | 662 kB 00:00 2024-04-24T06:29:22.060 INFO:teuthology.orchestra.run.smithi137.stdout:(1/152): ceph-19.0.0-3124.ga3c8a22b.el9.x86_64. 39 kB/s | 6.6 kB 00:00 2024-04-24T06:29:22.302 INFO:teuthology.orchestra.run.smithi137.stdout:(2/152): ceph-fuse-19.0.0-3124.ga3c8a22b.el9.x8 3.6 MB/s | 890 kB 00:00 2024-04-24T06:29:22.369 INFO:teuthology.orchestra.run.smithi137.stdout:(3/152): ceph-immutable-object-cache-19.0.0-312 2.1 MB/s | 145 kB 00:00 2024-04-24T06:29:22.453 INFO:teuthology.orchestra.run.smithi137.stdout:(4/152): ceph-base-19.0.0-3124.ga3c8a22b.el9.x8 9.6 MB/s | 5.4 MB 00:00 2024-04-24T06:29:22.603 INFO:teuthology.orchestra.run.smithi137.stdout:(5/152): ceph-mgr-19.0.0-3124.ga3c8a22b.el9.x86 10 MB/s | 1.5 MB 00:00 2024-04-24T06:29:22.653 INFO:teuthology.orchestra.run.smithi137.stdout:(6/152): ceph-mds-19.0.0-3124.ga3c8a22b.el9.x86 8.3 MB/s | 2.4 MB 00:00 2024-04-24T06:29:22.929 INFO:teuthology.orchestra.run.smithi137.stdout:(7/152): ceph-mon-19.0.0-3124.ga3c8a22b.el9.x86 14 MB/s | 4.7 MB 00:00 2024-04-24T06:29:23.142 INFO:teuthology.orchestra.run.smithi037.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 3.3 MB/s | 4.7 MB 00:01 2024-04-24T06:29:23.259 INFO:teuthology.orchestra.run.smithi037.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 7.7 MB/s | 911 kB 00:00 2024-04-24T06:29:23.376 INFO:teuthology.orchestra.run.smithi037.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-04-24T06:29:23.754 INFO:teuthology.orchestra.run.smithi137.stdout:(8/152): ceph-radosgw-19.0.0-3124.ga3c8a22b.el9 13 MB/s | 11 MB 00:00 2024-04-24T06:29:23.820 INFO:teuthology.orchestra.run.smithi137.stdout:(9/152): ceph-selinux-19.0.0-3124.ga3c8a22b.el9 379 kB/s | 25 kB 00:00 2024-04-24T06:29:23.976 INFO:teuthology.orchestra.run.smithi037.stdout:(70/152): postgresql-13.14-1.el9.x86_64.rpm 765 kB/s | 1.6 MB 00:02 2024-04-24T06:29:24.276 INFO:teuthology.orchestra.run.smithi037.stdout:(71/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 1.7 MB/s | 4.4 MB 00:02 2024-04-24T06:29:24.468 INFO:teuthology.orchestra.run.smithi037.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 457 kB/s | 87 kB 00:00 2024-04-24T06:29:24.863 INFO:teuthology.orchestra.run.smithi137.stdout:(10/152): ceph-osd-19.0.0-3124.ga3c8a22b.el9.x8 7.6 MB/s | 17 MB 00:02 2024-04-24T06:29:24.946 INFO:teuthology.orchestra.run.smithi137.stdout:(11/152): cephfs-mirror-19.0.0-3124.ga3c8a22b.e 2.6 MB/s | 224 kB 00:00 2024-04-24T06:29:25.021 INFO:teuthology.orchestra.run.smithi137.stdout:(12/152): cephfs-shell-19.0.0-3124.ga3c8a22b.el 361 kB/s | 27 kB 00:00 2024-04-24T06:29:25.089 INFO:teuthology.orchestra.run.smithi137.stdout:(13/152): libcephfs-devel-19.0.0-3124.ga3c8a22b 475 kB/s | 32 kB 00:00 2024-04-24T06:29:25.264 INFO:teuthology.orchestra.run.smithi137.stdout:(14/152): libcephfs2-19.0.0-3124.ga3c8a22b.el9. 4.2 MB/s | 747 kB 00:00 2024-04-24T06:29:25.347 INFO:teuthology.orchestra.run.smithi137.stdout:(15/152): libcephsqlite-19.0.0-3124.ga3c8a22b.e 1.9 MB/s | 163 kB 00:00 2024-04-24T06:29:25.385 INFO:teuthology.orchestra.run.smithi037.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 733 kB/s | 1.0 MB 00:01 2024-04-24T06:29:25.439 INFO:teuthology.orchestra.run.smithi137.stdout:(16/152): librados-devel-19.0.0-3124.ga3c8a22b. 1.3 MB/s | 127 kB 00:00 2024-04-24T06:29:25.690 INFO:teuthology.orchestra.run.smithi137.stdout:(17/152): ceph-common-19.0.0-3124.ga3c8a22b.el9 5.6 MB/s | 21 MB 00:03 2024-04-24T06:29:25.723 INFO:teuthology.orchestra.run.smithi137.stdout:(18/152): libradosstriper1-19.0.0-3124.ga3c8a22 1.7 MB/s | 501 kB 00:00 2024-04-24T06:29:25.727 INFO:teuthology.orchestra.run.smithi037.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 715 kB/s | 244 kB 00:00 2024-04-24T06:29:25.765 INFO:teuthology.orchestra.run.smithi137.stdout:(19/152): python3-ceph-argparse-19.0.0-3124.ga3 1.1 MB/s | 45 kB 00:00 2024-04-24T06:29:25.857 INFO:teuthology.orchestra.run.smithi137.stdout:(20/152): python3-ceph-common-19.0.0-3124.ga3c8 1.4 MB/s | 135 kB 00:00 2024-04-24T06:29:25.949 INFO:teuthology.orchestra.run.smithi137.stdout:(21/152): python3-cephfs-19.0.0-3124.ga3c8a22b. 1.7 MB/s | 162 kB 00:00 2024-04-24T06:29:26.050 INFO:teuthology.orchestra.run.smithi137.stdout:(22/152): python3-rados-19.0.0-3124.ga3c8a22b.e 3.1 MB/s | 322 kB 00:00 2024-04-24T06:29:26.069 INFO:teuthology.orchestra.run.smithi037.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 726 kB/s | 248 kB 00:00 2024-04-24T06:29:26.150 INFO:teuthology.orchestra.run.smithi137.stdout:(23/152): python3-rbd-19.0.0-3124.ga3c8a22b.el9 2.9 MB/s | 297 kB 00:00 2024-04-24T06:29:26.177 INFO:teuthology.orchestra.run.smithi037.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 237 kB/s | 26 kB 00:00 2024-04-24T06:29:26.242 INFO:teuthology.orchestra.run.smithi137.stdout:(24/152): python3-rgw-19.0.0-3124.ga3c8a22b.el9 1.1 MB/s | 99 kB 00:00 2024-04-24T06:29:26.286 INFO:teuthology.orchestra.run.smithi037.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 177 kB/s | 19 kB 00:00 2024-04-24T06:29:26.342 INFO:teuthology.orchestra.run.smithi137.stdout:(25/152): rbd-fuse-19.0.0-3124.ga3c8a22b.el9.x8 852 kB/s | 85 kB 00:00 2024-04-24T06:29:26.434 INFO:teuthology.orchestra.run.smithi137.stdout:(26/152): librgw2-19.0.0-3124.ga3c8a22b.el9.x86 7.2 MB/s | 5.4 MB 00:00 2024-04-24T06:29:26.542 INFO:teuthology.orchestra.run.smithi137.stdout:(27/152): rbd-nbd-19.0.0-3124.ga3c8a22b.el9.x86 1.5 MB/s | 170 kB 00:00 2024-04-24T06:29:26.553 INFO:teuthology.orchestra.run.smithi037.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 646 kB/s | 172 kB 00:00 2024-04-24T06:29:26.661 INFO:teuthology.orchestra.run.smithi037.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 321 kB/s | 35 kB 00:00 2024-04-24T06:29:27.043 INFO:teuthology.orchestra.run.smithi137.stdout:(28/152): ceph-test-19.0.0-3124.ga3c8a22b.el9.x 15 MB/s | 49 MB 00:03 2024-04-24T06:29:27.068 INFO:teuthology.orchestra.run.smithi137.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3124.g 65 kB/s | 34 kB 00:00 2024-04-24T06:29:27.085 INFO:teuthology.orchestra.run.smithi137.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3124.ga3c8a22 3.3 MB/s | 142 kB 00:00 2024-04-24T06:29:27.144 INFO:teuthology.orchestra.run.smithi137.stdout:(31/152): rbd-mirror-19.0.0-3124.ga3c8a22b.el9. 3.9 MB/s | 3.1 MB 00:00 2024-04-24T06:29:27.286 INFO:teuthology.orchestra.run.smithi137.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3124.ga3 1.7 MB/s | 249 kB 00:00 2024-04-24T06:29:27.361 INFO:teuthology.orchestra.run.smithi137.stdout:(33/152): ceph-mgr-rook-19.0.0-3124.ga3c8a22b.e 659 kB/s | 49 kB 00:00 2024-04-24T06:29:27.436 INFO:teuthology.orchestra.run.smithi137.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3124.ga 219 kB/s | 16 kB 00:00 2024-04-24T06:29:27.561 INFO:teuthology.orchestra.run.smithi137.stdout:(35/152): ceph-mgr-dashboard-19.0.0-3124.ga3c8a 7.2 MB/s | 3.5 MB 00:00 2024-04-24T06:29:27.587 INFO:teuthology.orchestra.run.smithi137.stdout:(36/152): ceph-volume-19.0.0-3124.ga3c8a22b.el9 1.8 MB/s | 273 kB 00:00 2024-04-24T06:29:27.679 INFO:teuthology.orchestra.run.smithi137.stdout:(37/152): ceph-mgr-diskprediction-local-19.0.0- 12 MB/s | 7.4 MB 00:00 2024-04-24T06:29:27.704 INFO:teuthology.orchestra.run.smithi137.stdout:(38/152): cephfs-top-19.0.0-3124.ga3c8a22b.el9. 156 kB/s | 18 kB 00:00 2024-04-24T06:29:27.738 INFO:teuthology.orchestra.run.smithi137.stdout:(39/152): cephadm-19.0.0-3124.ga3c8a22b.el9.noa 4.2 MB/s | 762 kB 00:00 2024-04-24T06:29:27.846 INFO:teuthology.orchestra.run.smithi137.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 481 kB/s | 80 kB 00:00 2024-04-24T06:29:27.938 INFO:teuthology.orchestra.run.smithi137.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 976 kB/s | 195 kB 00:00 2024-04-24T06:29:27.953 INFO:teuthology.orchestra.run.smithi037.stdout:(80/152): postgresql-server-13.14-1.el9.x86_64. 1.3 MB/s | 5.8 MB 00:04 2024-04-24T06:29:27.963 INFO:teuthology.orchestra.run.smithi137.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-04-24T06:29:27.988 INFO:teuthology.orchestra.run.smithi137.stdout:(43/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.8 MB/s | 803 kB 00:00 2024-04-24T06:29:28.014 INFO:teuthology.orchestra.run.smithi137.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-04-24T06:29:28.047 INFO:teuthology.orchestra.run.smithi137.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-24T06:29:28.062 INFO:teuthology.orchestra.run.smithi037.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-04-24T06:29:28.073 INFO:teuthology.orchestra.run.smithi137.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-24T06:29:28.106 INFO:teuthology.orchestra.run.smithi137.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-04-24T06:29:28.165 INFO:teuthology.orchestra.run.smithi137.stdout:(48/152): python3-cryptography-36.0.1-4.el9.x86 5.4 MB/s | 1.2 MB 00:00 2024-04-24T06:29:28.170 INFO:teuthology.orchestra.run.smithi037.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-04-24T06:29:28.215 INFO:teuthology.orchestra.run.smithi137.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 4.7 MB/s | 689 kB 00:00 2024-04-24T06:29:28.241 INFO:teuthology.orchestra.run.smithi137.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 107 kB 00:00 2024-04-24T06:29:28.266 INFO:teuthology.orchestra.run.smithi137.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 429 kB/s | 21 kB 00:00 2024-04-24T06:29:28.279 INFO:teuthology.orchestra.run.smithi037.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 713 kB/s | 77 kB 00:00 2024-04-24T06:29:28.308 INFO:teuthology.orchestra.run.smithi137.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 4.6 MB/s | 943 kB 00:00 2024-04-24T06:29:28.333 INFO:teuthology.orchestra.run.smithi137.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 489 kB/s | 33 kB 00:00 2024-04-24T06:29:28.358 INFO:teuthology.orchestra.run.smithi137.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-04-24T06:29:28.383 INFO:teuthology.orchestra.run.smithi137.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 362 kB/s | 18 kB 00:00 2024-04-24T06:29:28.388 INFO:teuthology.orchestra.run.smithi037.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 390 kB/s | 42 kB 00:00 2024-04-24T06:29:28.409 INFO:teuthology.orchestra.run.smithi137.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-04-24T06:29:28.443 INFO:teuthology.orchestra.run.smithi137.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-24T06:29:28.468 INFO:teuthology.orchestra.run.smithi137.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 772 kB/s | 45 kB 00:00 2024-04-24T06:29:28.496 INFO:teuthology.orchestra.run.smithi037.stdout:(85/152): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-04-24T06:29:28.585 INFO:teuthology.orchestra.run.smithi137.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 4.9 MB/s | 582 kB 00:00 2024-04-24T06:29:28.605 INFO:teuthology.orchestra.run.smithi037.stdout:(86/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-04-24T06:29:28.610 INFO:teuthology.orchestra.run.smithi137.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 3.9 MB/s | 662 kB 00:00 2024-04-24T06:29:28.635 INFO:teuthology.orchestra.run.smithi137.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 506 kB/s | 25 kB 00:00 2024-04-24T06:29:28.660 INFO:teuthology.orchestra.run.smithi137.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-04-24T06:29:28.714 INFO:teuthology.orchestra.run.smithi037.stdout:(87/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-04-24T06:29:28.719 INFO:teuthology.orchestra.run.smithi137.stdout:(63/152): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-04-24T06:29:28.744 INFO:teuthology.orchestra.run.smithi137.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-04-24T06:29:28.778 INFO:teuthology.orchestra.run.smithi137.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 602 kB/s | 35 kB 00:00 2024-04-24T06:29:28.814 INFO:teuthology.orchestra.run.smithi037.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 508 kB/s | 51 kB 00:00 2024-04-24T06:29:28.854 INFO:teuthology.orchestra.run.smithi137.stdout:(66/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.5 MB/s | 3.0 MB 00:00 2024-04-24T06:29:28.914 INFO:teuthology.orchestra.run.smithi037.stdout:(89/152): python3-requests-oauthlib-1.3.0-12.el 537 kB/s | 54 kB 00:00 2024-04-24T06:29:29.113 INFO:teuthology.orchestra.run.smithi137.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 6.1 MB/s | 1.6 MB 00:00 2024-04-24T06:29:29.271 INFO:teuthology.orchestra.run.smithi137.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 5.6 MB/s | 911 kB 00:00 2024-04-24T06:29:29.338 INFO:teuthology.orchestra.run.smithi137.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-04-24T06:29:29.463 INFO:teuthology.orchestra.run.smithi137.stdout:(70/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 6.5 MB/s | 4.4 MB 00:00 2024-04-24T06:29:29.498 INFO:teuthology.orchestra.run.smithi037.stdout:(90/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-04-24T06:29:29.598 INFO:teuthology.orchestra.run.smithi037.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 417 kB/s | 42 kB 00:00 2024-04-24T06:29:29.621 INFO:teuthology.orchestra.run.smithi137.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 6.4 MB/s | 1.0 MB 00:00 2024-04-24T06:29:29.680 INFO:teuthology.orchestra.run.smithi137.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-24T06:29:29.698 INFO:teuthology.orchestra.run.smithi037.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 442 kB/s | 44 kB 00:00 2024-04-24T06:29:29.738 INFO:teuthology.orchestra.run.smithi137.stdout:(73/152): openblas-openmp-0.3.21-2.el9.x86_64.r 4.8 MB/s | 4.7 MB 00:00 2024-04-24T06:29:29.797 INFO:teuthology.orchestra.run.smithi137.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-04-24T06:29:29.858 INFO:teuthology.orchestra.run.smithi137.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-04-24T06:29:29.914 INFO:teuthology.orchestra.run.smithi137.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 440 kB/s | 26 kB 00:00 2024-04-24T06:29:29.932 INFO:teuthology.orchestra.run.smithi037.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 945 kB/s | 221 kB 00:00 2024-04-24T06:29:29.964 INFO:teuthology.orchestra.run.smithi137.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-04-24T06:29:30.023 INFO:teuthology.orchestra.run.smithi137.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-24T06:29:30.032 INFO:teuthology.orchestra.run.smithi037.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 360 kB/s | 36 kB 00:00 2024-04-24T06:29:30.073 INFO:teuthology.orchestra.run.smithi137.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 697 kB/s | 35 kB 00:00 2024-04-24T06:29:30.133 INFO:teuthology.orchestra.run.smithi037.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 473 kB/s | 47 kB 00:00 2024-04-24T06:29:30.240 INFO:teuthology.orchestra.run.smithi137.stdout:(80/152): postgresql-server-13.14-1.el9.x86_64. 6.4 MB/s | 5.8 MB 00:00 2024-04-24T06:29:30.241 INFO:teuthology.orchestra.run.smithi037.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.7 MB/s | 305 kB 00:00 2024-04-24T06:29:30.324 INFO:teuthology.orchestra.run.smithi137.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-04-24T06:29:30.342 INFO:teuthology.orchestra.run.smithi037.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 578 kB/s | 58 kB 00:00 2024-04-24T06:29:30.383 INFO:teuthology.orchestra.run.smithi137.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-04-24T06:29:30.433 INFO:teuthology.orchestra.run.smithi137.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-24T06:29:30.442 INFO:teuthology.orchestra.run.smithi037.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 939 kB/s | 94 kB 00:00 2024-04-24T06:29:30.483 INFO:teuthology.orchestra.run.smithi137.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 847 kB/s | 42 kB 00:00 2024-04-24T06:29:30.551 INFO:teuthology.orchestra.run.smithi037.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 587 kB/s | 64 kB 00:00 2024-04-24T06:29:30.567 INFO:teuthology.orchestra.run.smithi137.stdout:(85/152): python3-protobuf-3.14.0-13.el9.noarch 3.2 MB/s | 269 kB 00:00 2024-04-24T06:29:30.625 INFO:teuthology.orchestra.run.smithi137.stdout:(86/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-04-24T06:29:30.651 INFO:teuthology.orchestra.run.smithi037.stdout:(100/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 448 kB/s | 45 kB 00:00 2024-04-24T06:29:30.709 INFO:teuthology.orchestra.run.smithi137.stdout:(87/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.3 MB/s | 279 kB 00:00 2024-04-24T06:29:30.734 INFO:teuthology.orchestra.run.smithi037.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 268 kB/s | 22 kB 00:00 2024-04-24T06:29:30.785 INFO:teuthology.orchestra.run.smithi137.stdout:(88/152): python3-babel-2.9.1-2.el9.noarch.rpm 5.4 MB/s | 6.0 MB 00:01 2024-04-24T06:29:30.809 INFO:teuthology.orchestra.run.smithi137.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 509 kB/s | 51 kB 00:00 2024-04-24T06:29:30.835 INFO:teuthology.orchestra.run.smithi137.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-24T06:29:30.885 INFO:teuthology.orchestra.run.smithi137.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 833 kB/s | 42 kB 00:00 2024-04-24T06:29:30.944 INFO:teuthology.orchestra.run.smithi137.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 759 kB/s | 44 kB 00:00 2024-04-24T06:29:30.977 INFO:teuthology.orchestra.run.smithi037.stdout:(102/152): protobuf-compiler-3.14.0-13.el9.x86_ 3.5 MB/s | 863 kB 00:00 2024-04-24T06:29:31.035 INFO:teuthology.orchestra.run.smithi137.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.4 MB/s | 221 kB 00:00 2024-04-24T06:29:31.094 INFO:teuthology.orchestra.run.smithi137.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 615 kB/s | 36 kB 00:00 2024-04-24T06:29:31.135 INFO:teuthology.orchestra.run.smithi037.stdout:(103/152): python3-numpy-1.20.1-5.el9.x86_64.rp 1.1 MB/s | 5.1 MB 00:04 2024-04-24T06:29:31.152 INFO:teuthology.orchestra.run.smithi137.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 816 kB/s | 47 kB 00:00 2024-04-24T06:29:31.270 INFO:teuthology.orchestra.run.smithi137.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.6 MB/s | 305 kB 00:00 2024-04-24T06:29:31.319 INFO:teuthology.orchestra.run.smithi037.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.6 MB/s | 548 kB 00:00 2024-04-24T06:29:31.328 INFO:teuthology.orchestra.run.smithi137.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 994 kB/s | 58 kB 00:00 2024-04-24T06:29:31.361 INFO:teuthology.orchestra.run.smithi037.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.3 MB/s | 309 kB 00:00 2024-04-24T06:29:31.386 INFO:teuthology.orchestra.run.smithi037.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 290 kB/s | 19 kB 00:00 2024-04-24T06:29:31.387 INFO:teuthology.orchestra.run.smithi137.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-24T06:29:31.437 INFO:teuthology.orchestra.run.smithi137.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-24T06:29:31.444 INFO:teuthology.orchestra.run.smithi037.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 447 kB/s | 26 kB 00:00 2024-04-24T06:29:31.486 INFO:teuthology.orchestra.run.smithi037.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2024-04-24T06:29:31.487 INFO:teuthology.orchestra.run.smithi137.stdout:(100/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 897 kB/s | 45 kB 00:00 2024-04-24T06:29:31.529 INFO:teuthology.orchestra.run.smithi037.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-04-24T06:29:31.571 INFO:teuthology.orchestra.run.smithi137.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 268 kB/s | 22 kB 00:00 2024-04-24T06:29:31.587 INFO:teuthology.orchestra.run.smithi037.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-04-24T06:29:31.713 INFO:teuthology.orchestra.run.smithi137.stdout:(102/152): protobuf-compiler-3.14.0-13.el9.x86_ 6.0 MB/s | 863 kB 00:00 2024-04-24T06:29:31.771 INFO:teuthology.orchestra.run.smithi037.stdout:(111/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.5 MB/s | 838 kB 00:00 2024-04-24T06:29:31.896 INFO:teuthology.orchestra.run.smithi037.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-04-24T06:29:31.938 INFO:teuthology.orchestra.run.smithi037.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-04-24T06:29:31.980 INFO:teuthology.orchestra.run.smithi037.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 777 kB/s | 32 kB 00:00 2024-04-24T06:29:32.013 INFO:teuthology.orchestra.run.smithi137.stdout:(103/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.8 MB/s | 548 kB 00:00 2024-04-24T06:29:32.022 INFO:teuthology.orchestra.run.smithi037.stdout:(115/152): python3-certifi-2021.10.8-2.el9.noar 358 kB/s | 15 kB 00:00 2024-04-24T06:29:32.073 INFO:teuthology.orchestra.run.smithi037.stdout:(116/152): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-04-24T06:29:32.080 INFO:teuthology.orchestra.run.smithi137.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 4.5 MB/s | 309 kB 00:00 2024-04-24T06:29:32.130 INFO:teuthology.orchestra.run.smithi137.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 390 kB/s | 19 kB 00:00 2024-04-24T06:29:32.164 INFO:teuthology.orchestra.run.smithi037.stdout:(117/152): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-04-24T06:29:32.240 INFO:teuthology.orchestra.run.smithi037.stdout:(118/152): libarrow-9.0.0-10.el9.x86_64.rpm 5.0 MB/s | 4.4 MB 00:00 2024-04-24T06:29:32.282 INFO:teuthology.orchestra.run.smithi037.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-04-24T06:29:32.307 INFO:teuthology.orchestra.run.smithi037.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 527 kB/s | 35 kB 00:00 2024-04-24T06:29:32.331 INFO:teuthology.orchestra.run.smithi137.stdout:(106/152): libarrow-9.0.0-10.el9.x86_64.rpm 22 MB/s | 4.4 MB 00:00 2024-04-24T06:29:32.349 INFO:teuthology.orchestra.run.smithi037.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 3.0 MB/s | 206 kB 00:00 2024-04-24T06:29:32.416 INFO:teuthology.orchestra.run.smithi037.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-04-24T06:29:32.464 INFO:teuthology.orchestra.run.smithi137.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 195 kB/s | 26 kB 00:00 2024-04-24T06:29:32.466 INFO:teuthology.orchestra.run.smithi037.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-04-24T06:29:32.508 INFO:teuthology.orchestra.run.smithi037.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 425 kB/s | 18 kB 00:00 2024-04-24T06:29:32.541 INFO:teuthology.orchestra.run.smithi137.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 647 kB/s | 49 kB 00:00 2024-04-24T06:29:32.575 INFO:teuthology.orchestra.run.smithi037.stdout:(125/152): python3-jaraco-collections-3.0.0-8.e 348 kB/s | 23 kB 00:00 2024-04-24T06:29:32.600 INFO:teuthology.orchestra.run.smithi137.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-04-24T06:29:32.609 INFO:teuthology.orchestra.run.smithi037.stdout:(126/152): python3-jaraco-functools-3.5.0-2.el9 583 kB/s | 19 kB 00:00 2024-04-24T06:29:32.642 INFO:teuthology.orchestra.run.smithi037.stdout:(127/152): python3-jaraco-text-3.2.0-6.el9.noar 592 kB/s | 20 kB 00:00 2024-04-24T06:29:32.650 INFO:teuthology.orchestra.run.smithi137.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-04-24T06:29:32.734 INFO:teuthology.orchestra.run.smithi137.stdout:(111/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 9.8 MB/s | 838 kB 00:00 2024-04-24T06:29:32.801 INFO:teuthology.orchestra.run.smithi137.stdout:(112/152): python3-numpy-1.20.1-5.el9.x86_64.rp 1.9 MB/s | 5.1 MB 00:02 2024-04-24T06:29:32.826 INFO:teuthology.orchestra.run.smithi137.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 5.8 MB/s | 548 kB 00:00 2024-04-24T06:29:32.934 INFO:teuthology.orchestra.run.smithi037.stdout:(128/152): python3-babel-2.9.1-2.el9.noarch.rpm 721 kB/s | 6.0 MB 00:08 2024-04-24T06:29:32.984 INFO:teuthology.orchestra.run.smithi037.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 3.0 MB/s | 1.0 MB 00:00 2024-04-24T06:29:32.985 INFO:teuthology.orchestra.run.smithi137.stdout:(114/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.5 MB/s | 16 MB 00:02 2024-04-24T06:29:33.010 INFO:teuthology.orchestra.run.smithi137.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 175 kB/s | 32 kB 00:00 2024-04-24T06:29:33.060 INFO:teuthology.orchestra.run.smithi037.stdout:(130/152): python3-more-itertools-8.12.0-2.el9. 1.7 MB/s | 79 kB 00:00 2024-04-24T06:29:33.069 INFO:teuthology.orchestra.run.smithi137.stdout:(116/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-04-24T06:29:33.093 INFO:teuthology.orchestra.run.smithi037.stdout:(131/152): python3-natsort-7.1.1-5.el9.noarch.r 1.7 MB/s | 58 kB 00:00 2024-04-24T06:29:33.111 INFO:teuthology.orchestra.run.smithi137.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 120 kB/s | 15 kB 00:00 2024-04-24T06:29:33.127 INFO:teuthology.orchestra.run.smithi037.stdout:(132/152): python3-logutils-0.3.5-21.el9.noarch 241 kB/s | 46 kB 00:00 2024-04-24T06:29:33.136 INFO:teuthology.orchestra.run.smithi137.stdout:(118/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 130 kB/s | 43 kB 00:00 2024-04-24T06:29:33.161 INFO:teuthology.orchestra.run.smithi037.stdout:(133/152): python3-portend-3.1.0-2.el9.noarch.r 495 kB/s | 16 kB 00:00 2024-04-24T06:29:33.161 INFO:teuthology.orchestra.run.smithi137.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-04-24T06:29:33.186 INFO:teuthology.orchestra.run.smithi037.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-04-24T06:29:33.211 INFO:teuthology.orchestra.run.smithi037.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-04-24T06:29:33.212 INFO:teuthology.orchestra.run.smithi137.stdout:(120/152): python3-google-auth-2.29.0-1.el9.noa 4.0 MB/s | 206 kB 00:00 2024-04-24T06:29:33.236 INFO:teuthology.orchestra.run.smithi037.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-04-24T06:29:33.237 INFO:teuthology.orchestra.run.smithi137.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 351 kB/s | 35 kB 00:00 2024-04-24T06:29:33.253 INFO:teuthology.orchestra.run.smithi037.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 735 kB/s | 31 kB 00:00 2024-04-24T06:29:33.270 INFO:teuthology.orchestra.run.smithi137.stdout:(122/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.9 MB/s | 303 kB 00:00 2024-04-24T06:29:33.295 INFO:teuthology.orchestra.run.smithi037.stdout:(138/152): python3-grpcio-1.46.7-10.el9.x86_64. 2.1 MB/s | 2.0 MB 00:00 2024-04-24T06:29:33.312 INFO:teuthology.orchestra.run.smithi137.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 20 MB/s | 2.0 MB 00:00 2024-04-24T06:29:33.320 INFO:teuthology.orchestra.run.smithi037.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 886 kB/s | 59 kB 00:00 2024-04-24T06:29:33.338 INFO:teuthology.orchestra.run.smithi137.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 160 kB/s | 11 kB 00:00 2024-04-24T06:29:33.372 INFO:teuthology.orchestra.run.smithi137.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 302 kB/s | 18 kB 00:00 2024-04-24T06:29:33.375 INFO:teuthology.orchestra.run.smithi037.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 449 kB/s | 36 kB 00:00 2024-04-24T06:29:33.396 INFO:teuthology.orchestra.run.smithi037.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 678 kB/s | 51 kB 00:00 2024-04-24T06:29:33.397 INFO:teuthology.orchestra.run.smithi137.stdout:(126/152): python3-grpcio-tools-1.46.7-10.el9.x 906 kB/s | 144 kB 00:00 2024-04-24T06:29:33.421 INFO:teuthology.orchestra.run.smithi037.stdout:(142/152): python3-routes-2.5.1-5.el9.noarch.rp 1.0 MB/s | 188 kB 00:00 2024-04-24T06:29:33.422 INFO:teuthology.orchestra.run.smithi137.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 277 kB/s | 23 kB 00:00 2024-04-24T06:29:33.447 INFO:teuthology.orchestra.run.smithi037.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-04-24T06:29:33.447 INFO:teuthology.orchestra.run.smithi137.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2024-04-24T06:29:33.464 INFO:teuthology.orchestra.run.smithi137.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 293 kB/s | 20 kB 00:00 2024-04-24T06:29:33.489 INFO:teuthology.orchestra.run.smithi037.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.0 MB/s | 230 kB 00:00 2024-04-24T06:29:33.498 INFO:teuthology.orchestra.run.smithi137.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 921 kB/s | 46 kB 00:00 2024-04-24T06:29:33.514 INFO:teuthology.orchestra.run.smithi037.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 298 kB/s | 20 kB 00:00 2024-04-24T06:29:33.531 INFO:teuthology.orchestra.run.smithi137.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.1 MB/s | 79 kB 00:00 2024-04-24T06:29:33.539 INFO:teuthology.orchestra.run.smithi037.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.5 MB/s | 427 kB 00:00 2024-04-24T06:29:33.548 INFO:teuthology.orchestra.run.smithi137.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-04-24T06:29:33.581 INFO:teuthology.orchestra.run.smithi037.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-04-24T06:29:33.590 INFO:teuthology.orchestra.run.smithi137.stdout:(133/152): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-04-24T06:29:33.607 INFO:teuthology.orchestra.run.smithi137.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-04-24T06:29:33.641 INFO:teuthology.orchestra.run.smithi137.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-04-24T06:29:33.657 INFO:teuthology.orchestra.run.smithi137.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-04-24T06:29:33.683 INFO:teuthology.orchestra.run.smithi137.stdout:(137/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 976 kB/s | 90 kB 00:00 2024-04-24T06:29:33.708 INFO:teuthology.orchestra.run.smithi137.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 459 kB/s | 31 kB 00:00 2024-04-24T06:29:33.733 INFO:teuthology.orchestra.run.smithi137.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-04-24T06:29:33.759 INFO:teuthology.orchestra.run.smithi137.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 780 kB/s | 59 kB 00:00 2024-04-24T06:29:33.784 INFO:teuthology.orchestra.run.smithi137.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 472 kB/s | 36 kB 00:00 2024-04-24T06:29:33.810 INFO:teuthology.orchestra.run.smithi137.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 672 kB/s | 51 kB 00:00 2024-04-24T06:29:33.835 INFO:teuthology.orchestra.run.smithi137.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-24T06:29:33.840 INFO:teuthology.orchestra.run.smithi037.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 4.9 MB/s | 1.6 MB 00:00 2024-04-24T06:29:33.860 INFO:teuthology.orchestra.run.smithi137.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-24T06:29:33.885 INFO:teuthology.orchestra.run.smithi137.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-04-24T06:29:33.890 INFO:teuthology.orchestra.run.smithi037.stdout:(149/152): librbd1-19.0.0-3124.ga3c8a22b.el9.x8 10 MB/s | 3.1 MB 00:00 2024-04-24T06:29:33.910 INFO:teuthology.orchestra.run.smithi137.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 264 kB/s | 20 kB 00:00 2024-04-24T06:29:33.936 INFO:teuthology.orchestra.run.smithi137.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-24T06:29:33.978 INFO:teuthology.orchestra.run.smithi137.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-04-24T06:29:34.269 INFO:teuthology.orchestra.run.smithi137.stdout:(149/152): librbd1-19.0.0-3124.ga3c8a22b.el9.x8 9.4 MB/s | 3.1 MB 00:00 2024-04-24T06:29:34.337 INFO:teuthology.orchestra.run.smithi137.stdout:(150/152): librados2-19.0.0-3124.ga3c8a22b.el9. 8.0 MB/s | 3.4 MB 00:00 2024-04-24T06:29:34.378 INFO:teuthology.orchestra.run.smithi137.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 3.4 MB/s | 1.3 MB 00:00 2024-04-24T06:29:34.557 INFO:teuthology.orchestra.run.smithi037.stdout:(150/152): librados2-19.0.0-3124.ga3c8a22b.el9. 3.3 MB/s | 3.4 MB 00:01 2024-04-24T06:29:34.620 INFO:teuthology.orchestra.run.smithi137.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 6.8 MB/s | 2.4 MB 00:00 2024-04-24T06:29:34.623 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-04-24T06:29:34.623 INFO:teuthology.orchestra.run.smithi137.stdout:Total 17 MB/s | 222 MB 00:13 2024-04-24T06:29:34.665 INFO:teuthology.orchestra.run.smithi037.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 3.1 MB/s | 2.4 MB 00:00 2024-04-24T06:29:35.157 INFO:teuthology.orchestra.run.smithi037.stdout:(152/152): openssl-3.2.1-1.el9.x86_64.rpm 1.0 MB/s | 1.3 MB 00:01 2024-04-24T06:29:35.164 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-04-24T06:29:35.164 INFO:teuthology.orchestra.run.smithi037.stdout:Total 9.8 MB/s | 222 MB 00:22 2024-04-24T06:29:35.743 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-04-24T06:29:35.886 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-04-24T06:29:35.887 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-04-24T06:29:36.292 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-04-24T06:29:36.452 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-04-24T06:29:36.452 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-04-24T06:29:37.018 INFO:teuthology.orchestra.run.smithi005.stdout:Extra Packages for Enterprise Linux 1.4 MB/s | 21 MB 00:15 2024-04-24T06:29:37.640 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-04-24T06:29:37.641 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-04-24T06:29:38.334 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-04-24T06:29:38.335 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-04-24T06:29:43.151 INFO:teuthology.orchestra.run.smithi005.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2024-04-24T06:29:43.639 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-04-24T06:29:43.715 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T06:29:43.754 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T06:29:43.807 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T06:29:43.841 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T06:29:43.866 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T06:29:43.908 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T06:29:44.237 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T06:29:44.334 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : librados2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 8/156 2024-04-24T06:29:44.457 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librados2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 8/156 2024-04-24T06:29:44.494 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephfs2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 9/156 2024-04-24T06:29:44.568 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libcephfs2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 9/156 2024-04-24T06:29:44.606 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rados-2:19.0.0-3124.ga3c8a22b.el9.x86_64 10/156 2024-04-24T06:29:44.632 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-04-24T06:29:44.753 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T06:29:44.803 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T06:29:44.831 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephsqlite-2:19.0.0-3124.ga3c8a22b.el9.x86_64 13/156 2024-04-24T06:29:44.854 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T06:29:44.893 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T06:29:44.934 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3124.ga3c8a22b.el9.x86_64 13/156 2024-04-24T06:29:44.946 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T06:29:44.957 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libradosstriper1-2:19.0.0-3124.ga3c8a22b.el9.x86 14/156 2024-04-24T06:29:44.996 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T06:29:45.021 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T06:29:45.028 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3124.ga3c8a22b.el9.x86 14/156 2024-04-24T06:29:45.050 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T06:29:45.064 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T06:29:45.071 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T06:29:45.250 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T06:29:45.282 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T06:29:45.389 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T06:29:45.468 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T06:29:45.478 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T06:29:45.515 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T06:29:45.522 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : librados2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 8/156 2024-04-24T06:29:45.550 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T06:29:45.624 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T06:29:45.627 INFO:teuthology.orchestra.run.smithi005.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T06:29:45.629 INFO:teuthology.orchestra.run.smithi005.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T06:29:45.647 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librados2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 8/156 2024-04-24T06:29:45.650 INFO:teuthology.orchestra.run.smithi005.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T06:29:45.651 INFO:teuthology.orchestra.run.smithi005.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T06:29:45.653 INFO:teuthology.orchestra.run.smithi005.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T06:29:45.654 INFO:teuthology.orchestra.run.smithi005.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T06:29:45.657 INFO:teuthology.orchestra.run.smithi005.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T06:29:45.657 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T06:29:45.660 INFO:teuthology.orchestra.run.smithi005.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T06:29:45.662 INFO:teuthology.orchestra.run.smithi005.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T06:29:45.702 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T06:29:45.737 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-04-24T06:29:45.742 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T06:29:45.762 INFO:teuthology.orchestra.run.smithi005.stdout:======================================================================================= 2024-04-24T06:29:45.763 INFO:teuthology.orchestra.run.smithi005.stdout: Package Arch Version Repository Size 2024-04-24T06:29:45.763 INFO:teuthology.orchestra.run.smithi005.stdout:======================================================================================= 2024-04-24T06:29:45.763 INFO:teuthology.orchestra.run.smithi005.stdout:Installing: 2024-04-24T06:29:45.763 INFO:teuthology.orchestra.run.smithi005.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T06:29:45.763 INFO:teuthology.orchestra.run.smithi005.stdout: ceph x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 6.6 k 2024-04-24T06:29:45.763 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-base x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 5.4 M 2024-04-24T06:29:45.763 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-fuse x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 890 k 2024-04-24T06:29:45.763 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 145 k 2024-04-24T06:29:45.763 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 1.5 M 2024-04-24T06:29:45.763 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 142 k 2024-04-24T06:29:45.763 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 3.5 M 2024-04-24T06:29:45.763 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 7.4 M 2024-04-24T06:29:45.763 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-rook noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 49 k 2024-04-24T06:29:45.764 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-radosgw x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 11 M 2024-04-24T06:29:45.764 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-test x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 49 M 2024-04-24T06:29:45.764 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-volume noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 273 k 2024-04-24T06:29:45.764 INFO:teuthology.orchestra.run.smithi005.stdout: cephadm noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 762 k 2024-04-24T06:29:45.764 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-mirror x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 224 k 2024-04-24T06:29:45.764 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-shell x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 27 k 2024-04-24T06:29:45.764 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-top noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 18 k 2024-04-24T06:29:45.764 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T06:29:45.764 INFO:teuthology.orchestra.run.smithi005.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T06:29:45.764 INFO:teuthology.orchestra.run.smithi005.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T06:29:45.764 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs-devel x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 32 k 2024-04-24T06:29:45.764 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs2 x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 747 k 2024-04-24T06:29:45.764 INFO:teuthology.orchestra.run.smithi005.stdout: librados-devel x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 127 k 2024-04-24T06:29:45.765 INFO:teuthology.orchestra.run.smithi005.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T06:29:45.765 INFO:teuthology.orchestra.run.smithi005.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T06:29:45.765 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T06:29:45.765 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T06:29:45.765 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T06:29:45.765 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T06:29:45.765 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cephfs x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 162 k 2024-04-24T06:29:45.765 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rados x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 322 k 2024-04-24T06:29:45.765 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rbd x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 297 k 2024-04-24T06:29:45.765 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rgw x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 99 k 2024-04-24T06:29:45.765 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-fuse x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 85 k 2024-04-24T06:29:45.765 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-mirror x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 3.1 M 2024-04-24T06:29:45.765 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-nbd x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 170 k 2024-04-24T06:29:45.765 INFO:teuthology.orchestra.run.smithi005.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T06:29:45.766 INFO:teuthology.orchestra.run.smithi005.stdout:Upgrading: 2024-04-24T06:29:45.766 INFO:teuthology.orchestra.run.smithi005.stdout: librados2 x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 3.4 M 2024-04-24T06:29:45.766 INFO:teuthology.orchestra.run.smithi005.stdout: librbd1 x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 3.1 M 2024-04-24T06:29:45.766 INFO:teuthology.orchestra.run.smithi005.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T06:29:45.766 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T06:29:45.766 INFO:teuthology.orchestra.run.smithi005.stdout:Installing dependencies: 2024-04-24T06:29:45.766 INFO:teuthology.orchestra.run.smithi005.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T06:29:45.766 INFO:teuthology.orchestra.run.smithi005.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T06:29:45.766 INFO:teuthology.orchestra.run.smithi005.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T06:29:45.766 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-common x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 21 M 2024-04-24T06:29:45.766 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 34 k 2024-04-24T06:29:45.766 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mds x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 2.4 M 2024-04-24T06:29:45.766 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 249 k 2024-04-24T06:29:45.767 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mon x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 4.7 M 2024-04-24T06:29:45.767 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-osd x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 17 M 2024-04-24T06:29:45.767 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3124.ga3c8a22b.el9 ceph-noarch 16 k 2024-04-24T06:29:45.767 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-selinux x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 25 k 2024-04-24T06:29:45.767 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T06:29:45.767 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T06:29:45.767 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T06:29:45.767 INFO:teuthology.orchestra.run.smithi005.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T06:29:45.767 INFO:teuthology.orchestra.run.smithi005.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T06:29:45.767 INFO:teuthology.orchestra.run.smithi005.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T06:29:45.767 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T06:29:45.767 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T06:29:45.767 INFO:teuthology.orchestra.run.smithi005.stdout: libcephsqlite x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 163 k 2024-04-24T06:29:45.767 INFO:teuthology.orchestra.run.smithi005.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T06:29:45.768 INFO:teuthology.orchestra.run.smithi005.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T06:29:45.768 INFO:teuthology.orchestra.run.smithi005.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T06:29:45.768 INFO:teuthology.orchestra.run.smithi005.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T06:29:45.768 INFO:teuthology.orchestra.run.smithi005.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T06:29:45.768 INFO:teuthology.orchestra.run.smithi005.stdout: libradosstriper1 x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 501 k 2024-04-24T06:29:45.768 INFO:teuthology.orchestra.run.smithi005.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T06:29:45.768 INFO:teuthology.orchestra.run.smithi005.stdout: librgw2 x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 5.4 M 2024-04-24T06:29:45.768 INFO:teuthology.orchestra.run.smithi005.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T06:29:45.768 INFO:teuthology.orchestra.run.smithi005.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T06:29:45.768 INFO:teuthology.orchestra.run.smithi005.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T06:29:45.768 INFO:teuthology.orchestra.run.smithi005.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T06:29:45.768 INFO:teuthology.orchestra.run.smithi005.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T06:29:45.768 INFO:teuthology.orchestra.run.smithi005.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T06:29:45.768 INFO:teuthology.orchestra.run.smithi005.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T06:29:45.769 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T06:29:45.769 INFO:teuthology.orchestra.run.smithi005.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T06:29:45.769 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T06:29:45.769 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T06:29:45.769 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T06:29:45.769 INFO:teuthology.orchestra.run.smithi005.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T06:29:45.769 INFO:teuthology.orchestra.run.smithi005.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T06:29:45.769 INFO:teuthology.orchestra.run.smithi005.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T06:29:45.769 INFO:teuthology.orchestra.run.smithi005.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T06:29:45.769 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T06:29:45.769 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-argparse x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 45 k 2024-04-24T06:29:45.769 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-common x86_64 2:19.0.0-3124.ga3c8a22b.el9 ceph 135 k 2024-04-24T06:29:45.769 INFO:teuthology.orchestra.run.smithi005.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T06:29:45.769 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T06:29:45.770 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T06:29:45.770 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T06:29:45.770 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T06:29:45.770 INFO:teuthology.orchestra.run.smithi005.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T06:29:45.770 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T06:29:45.770 INFO:teuthology.orchestra.run.smithi005.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T06:29:45.770 INFO:teuthology.orchestra.run.smithi005.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T06:29:45.770 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T06:29:45.770 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T06:29:45.770 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T06:29:45.770 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T06:29:45.770 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T06:29:45.770 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T06:29:45.771 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T06:29:45.771 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T06:29:45.771 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T06:29:45.771 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T06:29:45.771 INFO:teuthology.orchestra.run.smithi005.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T06:29:45.771 INFO:teuthology.orchestra.run.smithi005.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T06:29:45.771 INFO:teuthology.orchestra.run.smithi005.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T06:29:45.771 INFO:teuthology.orchestra.run.smithi005.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T06:29:45.771 INFO:teuthology.orchestra.run.smithi005.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T06:29:45.771 INFO:teuthology.orchestra.run.smithi005.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T06:29:45.771 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T06:29:45.771 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T06:29:45.771 INFO:teuthology.orchestra.run.smithi005.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T06:29:45.771 INFO:teuthology.orchestra.run.smithi005.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T06:29:45.772 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T06:29:45.772 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T06:29:45.772 INFO:teuthology.orchestra.run.smithi005.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T06:29:45.772 INFO:teuthology.orchestra.run.smithi005.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T06:29:45.772 INFO:teuthology.orchestra.run.smithi005.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T06:29:45.772 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T06:29:45.772 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T06:29:45.772 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T06:29:45.772 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T06:29:45.772 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T06:29:45.772 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T06:29:45.772 INFO:teuthology.orchestra.run.smithi005.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T06:29:45.772 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T06:29:45.772 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T06:29:45.773 INFO:teuthology.orchestra.run.smithi005.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T06:29:45.773 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T06:29:45.773 INFO:teuthology.orchestra.run.smithi005.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T06:29:45.773 INFO:teuthology.orchestra.run.smithi005.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T06:29:45.773 INFO:teuthology.orchestra.run.smithi005.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T06:29:45.773 INFO:teuthology.orchestra.run.smithi005.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T06:29:45.773 INFO:teuthology.orchestra.run.smithi005.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T06:29:45.773 INFO:teuthology.orchestra.run.smithi005.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T06:29:45.773 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T06:29:45.773 INFO:teuthology.orchestra.run.smithi005.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T06:29:45.773 INFO:teuthology.orchestra.run.smithi005.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T06:29:45.773 INFO:teuthology.orchestra.run.smithi005.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T06:29:45.773 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T06:29:45.773 INFO:teuthology.orchestra.run.smithi005.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T06:29:45.774 INFO:teuthology.orchestra.run.smithi005.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T06:29:45.774 INFO:teuthology.orchestra.run.smithi005.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T06:29:45.774 INFO:teuthology.orchestra.run.smithi005.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T06:29:45.774 INFO:teuthology.orchestra.run.smithi005.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T06:29:45.774 INFO:teuthology.orchestra.run.smithi005.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T06:29:45.774 INFO:teuthology.orchestra.run.smithi005.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T06:29:45.774 INFO:teuthology.orchestra.run.smithi005.stdout:Installing weak dependencies: 2024-04-24T06:29:45.774 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T06:29:45.774 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:29:45.774 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2024-04-24T06:29:45.774 INFO:teuthology.orchestra.run.smithi005.stdout:======================================================================================= 2024-04-24T06:29:45.774 INFO:teuthology.orchestra.run.smithi005.stdout:Install 148 Packages 2024-04-24T06:29:45.774 INFO:teuthology.orchestra.run.smithi005.stdout:Upgrade 4 Packages 2024-04-24T06:29:45.774 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:29:45.775 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T06:29:45.775 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 222 M 2024-04-24T06:29:45.775 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2024-04-24T06:29:45.798 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephfs2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 9/156 2024-04-24T06:29:45.833 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T06:29:45.891 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libcephfs2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 9/156 2024-04-24T06:29:45.892 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T06:29:45.923 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T06:29:45.928 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rados-2:19.0.0-3124.ga3c8a22b.el9.x86_64 10/156 2024-04-24T06:29:45.967 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T06:29:46.044 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T06:29:46.077 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ceph-common-2:19.0.0-3124.ga3c8a22b.el9. 31/156 2024-04-24T06:29:46.090 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T06:29:46.121 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ceph-argparse-2:19.0.0-3124.ga3c8a22b.el 32/156 2024-04-24T06:29:46.142 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T06:29:46.149 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cephfs-2:19.0.0-3124.ga3c8a22b.el9.x86_6 33/156 2024-04-24T06:29:46.177 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephsqlite-2:19.0.0-3124.ga3c8a22b.el9.x86_64 13/156 2024-04-24T06:29:46.221 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T06:29:46.264 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T06:29:46.274 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3124.ga3c8a22b.el9.x86_64 13/156 2024-04-24T06:29:46.303 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libradosstriper1-2:19.0.0-3124.ga3c8a22b.el9.x86 14/156 2024-04-24T06:29:46.338 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T06:29:46.358 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3124.ga3c8a22b.el9.x86 14/156 2024-04-24T06:29:46.379 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T06:29:46.399 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T06:29:46.472 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T06:29:46.533 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T06:29:46.572 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T06:29:46.605 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T06:29:46.715 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T06:29:46.813 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T06:29:46.853 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T06:29:46.889 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T06:29:46.965 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T06:29:47.005 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T06:29:47.049 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T06:29:47.054 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T06:29:47.080 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T06:29:47.111 INFO:teuthology.orchestra.run.smithi005.stdout:(1/152): ceph-19.0.0-3124.ga3c8a22b.el9.x86_64. 20 kB/s | 6.6 kB 00:00 2024-04-24T06:29:47.111 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T06:29:47.134 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T06:29:47.165 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T06:29:47.202 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T06:29:47.207 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T06:29:47.246 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librados-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_6 43/156 2024-04-24T06:29:47.287 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T06:29:47.304 INFO:teuthology.orchestra.run.smithi005.stdout:(2/152): ceph-fuse-19.0.0-3124.ga3c8a22b.el9.x8 4.5 MB/s | 890 kB 00:00 2024-04-24T06:29:47.370 INFO:teuthology.orchestra.run.smithi005.stdout:(3/152): ceph-immutable-object-cache-19.0.0-312 2.2 MB/s | 145 kB 00:00 2024-04-24T06:29:47.383 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T06:29:47.411 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T06:29:47.445 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T06:29:47.455 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T06:29:47.507 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T06:29:47.534 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T06:29:47.561 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T06:29:47.565 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ceph-common-2:19.0.0-3124.ga3c8a22b.el9. 31/156 2024-04-24T06:29:47.609 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ceph-argparse-2:19.0.0-3124.ga3c8a22b.el 32/156 2024-04-24T06:29:47.625 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T06:29:47.637 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cephfs-2:19.0.0-3124.ga3c8a22b.el9.x86_6 33/156 2024-04-24T06:29:47.657 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T06:29:47.670 INFO:teuthology.orchestra.run.smithi005.stdout:(4/152): ceph-mds-19.0.0-3124.ga3c8a22b.el9.x86 7.9 MB/s | 2.4 MB 00:00 2024-04-24T06:29:47.694 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T06:29:47.711 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T06:29:47.740 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T06:29:47.761 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T06:29:47.776 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T06:29:47.796 INFO:teuthology.orchestra.run.smithi005.stdout:(5/152): ceph-base-19.0.0-3124.ga3c8a22b.el9.x8 5.3 MB/s | 5.4 MB 00:01 2024-04-24T06:29:47.819 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T06:29:47.839 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T06:29:47.870 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T06:29:47.887 INFO:teuthology.orchestra.run.smithi005.stdout:(6/152): ceph-mgr-19.0.0-3124.ga3c8a22b.el9.x86 6.9 MB/s | 1.5 MB 00:00 2024-04-24T06:29:47.987 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T06:29:48.003 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T06:29:48.040 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T06:29:48.059 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T06:29:48.089 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T06:29:48.130 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T06:29:48.223 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T06:29:48.256 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T06:29:48.304 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T06:29:48.321 INFO:teuthology.orchestra.run.smithi005.stdout:(7/152): ceph-mon-19.0.0-3124.ga3c8a22b.el9.x86 9.0 MB/s | 4.7 MB 00:00 2024-04-24T06:29:48.329 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T06:29:48.354 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T06:29:48.427 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T06:29:48.553 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T06:29:48.587 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T06:29:48.614 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T06:29:48.615 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T06:29:48.649 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T06:29:48.688 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T06:29:48.705 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T06:29:48.719 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T06:29:48.738 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T06:29:48.749 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T06:29:48.781 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T06:29:48.784 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T06:29:48.811 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librados-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_6 43/156 2024-04-24T06:29:48.817 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T06:29:48.850 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T06:29:48.853 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T06:29:48.885 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T06:29:48.971 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T06:29:49.015 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T06:29:49.017 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T06:29:49.046 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T06:29:49.071 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T06:29:49.093 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T06:29:49.129 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T06:29:49.189 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T06:29:49.203 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T06:29:49.239 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T06:29:49.247 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T06:29:49.276 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T06:29:49.324 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T06:29:49.366 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T06:29:49.411 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T06:29:49.452 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T06:29:49.596 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T06:29:49.641 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T06:29:49.646 INFO:teuthology.orchestra.run.smithi005.stdout:(8/152): ceph-osd-19.0.0-3124.ga3c8a22b.el9.x86 9.6 MB/s | 17 MB 00:01 2024-04-24T06:29:49.671 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T06:29:49.713 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T06:29:49.811 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T06:29:49.829 INFO:teuthology.orchestra.run.smithi005.stdout:(9/152): ceph-radosgw-19.0.0-3124.ga3c8a22b.el9 7.0 MB/s | 11 MB 00:01 2024-04-24T06:29:49.846 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T06:29:49.855 INFO:teuthology.orchestra.run.smithi005.stdout:(10/152): ceph-selinux-19.0.0-3124.ga3c8a22b.el 121 kB/s | 25 kB 00:00 2024-04-24T06:29:49.886 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T06:29:49.919 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T06:29:49.944 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T06:29:49.956 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T06:29:50.004 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T06:29:50.024 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T06:29:50.036 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T06:29:50.135 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T06:29:50.161 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T06:29:50.163 INFO:teuthology.orchestra.run.smithi005.stdout:(11/152): ceph-common-19.0.0-3124.ga3c8a22b.el9 6.3 MB/s | 21 MB 00:03 2024-04-24T06:29:50.178 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T06:29:50.193 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T06:29:50.197 INFO:teuthology.orchestra.run.smithi005.stdout:(12/152): cephfs-mirror-19.0.0-3124.ga3c8a22b.e 658 kB/s | 224 kB 00:00 2024-04-24T06:29:50.222 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T06:29:50.239 INFO:teuthology.orchestra.run.smithi005.stdout:(13/152): cephfs-shell-19.0.0-3124.ga3c8a22b.el 366 kB/s | 27 kB 00:00 2024-04-24T06:29:50.264 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T06:29:50.264 INFO:teuthology.orchestra.run.smithi005.stdout:(14/152): libcephfs-devel-19.0.0-3124.ga3c8a22b 478 kB/s | 32 kB 00:00 2024-04-24T06:29:50.314 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T06:29:50.315 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T06:29:50.323 INFO:teuthology.orchestra.run.smithi005.stdout:(15/152): libcephsqlite-19.0.0-3124.ga3c8a22b.e 2.8 MB/s | 163 kB 00:00 2024-04-24T06:29:50.374 INFO:teuthology.orchestra.run.smithi005.stdout:(16/152): libcephfs2-19.0.0-3124.ga3c8a22b.el9. 5.5 MB/s | 747 kB 00:00 2024-04-24T06:29:50.407 INFO:teuthology.orchestra.run.smithi005.stdout:(17/152): librados-devel-19.0.0-3124.ga3c8a22b. 1.5 MB/s | 127 kB 00:00 2024-04-24T06:29:50.479 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T06:29:50.490 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T06:29:50.490 INFO:teuthology.orchestra.run.smithi005.stdout:(18/152): libradosstriper1-19.0.0-3124.ga3c8a22 4.2 MB/s | 501 kB 00:00 2024-04-24T06:29:50.520 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T06:29:50.585 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T06:29:50.591 INFO:teuthology.orchestra.run.smithi005.stdout:(19/152): python3-ceph-argparse-19.0.0-3124.ga3 456 kB/s | 45 kB 00:00 2024-04-24T06:29:50.596 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T06:29:50.614 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T06:29:50.649 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T06:29:50.682 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T06:29:50.682 INFO:teuthology.orchestra.run.smithi005.stdout:(20/152): python3-ceph-common-19.0.0-3124.ga3c8 1.5 MB/s | 135 kB 00:00 2024-04-24T06:29:50.724 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T06:29:50.758 INFO:teuthology.orchestra.run.smithi005.stdout:(21/152): python3-cephfs-19.0.0-3124.ga3c8a22b. 2.1 MB/s | 162 kB 00:00 2024-04-24T06:29:50.758 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T06:29:50.849 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T06:29:50.850 INFO:teuthology.orchestra.run.smithi005.stdout:(22/152): python3-rados-19.0.0-3124.ga3c8a22b.e 3.5 MB/s | 322 kB 00:00 2024-04-24T06:29:50.897 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T06:29:50.928 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T06:29:50.968 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T06:29:50.983 INFO:teuthology.orchestra.run.smithi005.stdout:(23/152): librgw2-19.0.0-3124.ga3c8a22b.el9.x86 9.3 MB/s | 5.4 MB 00:00 2024-04-24T06:29:51.025 INFO:teuthology.orchestra.run.smithi005.stdout:(24/152): python3-rbd-19.0.0-3124.ga3c8a22b.el9 1.7 MB/s | 297 kB 00:00 2024-04-24T06:29:51.059 INFO:teuthology.orchestra.run.smithi005.stdout:(25/152): python3-rgw-19.0.0-3124.ga3c8a22b.el9 1.3 MB/s | 99 kB 00:00 2024-04-24T06:29:51.080 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T06:29:51.109 INFO:teuthology.orchestra.run.smithi005.stdout:(26/152): rbd-fuse-19.0.0-3124.ga3c8a22b.el9.x8 1.0 MB/s | 85 kB 00:00 2024-04-24T06:29:51.131 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T06:29:51.193 INFO:teuthology.orchestra.run.smithi005.stdout:(27/152): rbd-nbd-19.0.0-3124.ga3c8a22b.el9.x86 2.0 MB/s | 170 kB 00:00 2024-04-24T06:29:51.228 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T06:29:51.285 INFO:teuthology.orchestra.run.smithi005.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3124.g 377 kB/s | 34 kB 00:00 2024-04-24T06:29:51.341 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T06:29:51.402 INFO:teuthology.orchestra.run.smithi005.stdout:(29/152): rbd-mirror-19.0.0-3124.ga3c8a22b.el9. 9.0 MB/s | 3.1 MB 00:00 2024-04-24T06:29:51.435 INFO:teuthology.orchestra.run.smithi005.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3124.ga3c8a22 956 kB/s | 142 kB 00:00 2024-04-24T06:29:51.836 INFO:teuthology.orchestra.run.smithi005.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3124.ga3c8a 8.2 MB/s | 3.5 MB 00:00 2024-04-24T06:29:51.913 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T06:29:51.919 INFO:teuthology.orchestra.run.smithi005.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3124.ga3 3.0 MB/s | 249 kB 00:00 2024-04-24T06:29:51.969 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T06:29:51.986 INFO:teuthology.orchestra.run.smithi005.stdout:(33/152): ceph-mgr-rook-19.0.0-3124.ga3c8a22b.e 753 kB/s | 49 kB 00:00 2024-04-24T06:29:52.002 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T06:29:52.109 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T06:29:52.122 INFO:teuthology.orchestra.run.smithi005.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3124.ga 122 kB/s | 16 kB 00:00 2024-04-24T06:29:52.170 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T06:29:52.203 INFO:teuthology.orchestra.run.smithi005.stdout:(35/152): ceph-volume-19.0.0-3124.ga3c8a22b.el9 3.3 MB/s | 273 kB 00:00 2024-04-24T06:29:52.261 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T06:29:52.312 INFO:teuthology.orchestra.run.smithi005.stdout:(36/152): cephadm-19.0.0-3124.ga3c8a22b.el9.noa 6.9 MB/s | 762 kB 00:00 2024-04-24T06:29:52.312 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T06:29:52.437 INFO:teuthology.orchestra.run.smithi005.stdout:(37/152): ceph-mgr-diskprediction-local-19.0.0- 7.4 MB/s | 7.4 MB 00:01 2024-04-24T06:29:52.462 INFO:teuthology.orchestra.run.smithi005.stdout:(38/152): cephfs-top-19.0.0-3124.ga3c8a22b.el9. 122 kB/s | 18 kB 00:00 2024-04-24T06:29:52.498 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T06:29:52.527 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T06:29:52.537 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T06:29:52.596 INFO:teuthology.orchestra.run.smithi005.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 511 kB/s | 80 kB 00:00 2024-04-24T06:29:52.607 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T06:29:52.623 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T06:29:52.651 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T06:29:52.684 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T06:29:52.798 INFO:teuthology.orchestra.run.smithi005.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.4 MB/s | 803 kB 00:00 2024-04-24T06:29:52.821 INFO:teuthology.orchestra.run.smithi005.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 872 kB/s | 195 kB 00:00 2024-04-24T06:29:52.964 INFO:teuthology.orchestra.run.smithi005.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.5 MB/s | 253 kB 00:00 2024-04-24T06:29:52.995 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T06:29:53.022 INFO:teuthology.orchestra.run.smithi005.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-24T06:29:53.031 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : librbd1-2:19.0.0-3124.ga3c8a22b.el9.x86_64 99/156 2024-04-24T06:29:53.081 INFO:teuthology.orchestra.run.smithi005.stdout:(44/152): python3-cryptography-36.0.1-4.el9.x86 4.7 MB/s | 1.2 MB 00:00 2024-04-24T06:29:53.121 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librbd1-2:19.0.0-3124.ga3c8a22b.el9.x86_64 99/156 2024-04-24T06:29:53.152 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 100/156 2024-04-24T06:29:53.189 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T06:29:53.299 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T06:29:53.348 INFO:teuthology.orchestra.run.smithi005.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 416 kB/s | 135 kB 00:00 2024-04-24T06:29:53.382 INFO:teuthology.orchestra.run.smithi005.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 417 kB/s | 125 kB 00:00 2024-04-24T06:29:53.413 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T06:29:53.673 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T06:29:53.700 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librgw2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 103/156 2024-04-24T06:29:53.763 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librgw2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 103/156 2024-04-24T06:29:53.785 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 104/156 2024-04-24T06:29:53.865 INFO:teuthology.orchestra.run.smithi005.stdout:(47/152): ceph-test-19.0.0-3124.ga3c8a22b.el9.x 12 MB/s | 49 MB 00:04 2024-04-24T06:29:53.899 INFO:teuthology.orchestra.run.smithi005.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 390 kB/s | 215 kB 00:00 2024-04-24T06:29:54.034 INFO:teuthology.orchestra.run.smithi005.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 1.0 MB/s | 689 kB 00:00 2024-04-24T06:29:54.057 INFO:teuthology.orchestra.run.smithi005.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 679 kB/s | 107 kB 00:00 2024-04-24T06:29:54.083 INFO:teuthology.orchestra.run.smithi005.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 434 kB/s | 21 kB 00:00 2024-04-24T06:29:54.133 INFO:teuthology.orchestra.run.smithi005.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 668 kB/s | 33 kB 00:00 2024-04-24T06:29:54.175 INFO:teuthology.orchestra.run.smithi005.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-04-24T06:29:54.225 INFO:teuthology.orchestra.run.smithi005.stdout:(54/152): bison-3.7.4-5.el9.x86_64.rpm 2.6 MB/s | 943 kB 00:00 2024-04-24T06:29:54.251 INFO:teuthology.orchestra.run.smithi005.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 245 kB/s | 18 kB 00:00 2024-04-24T06:29:54.293 INFO:teuthology.orchestra.run.smithi005.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 169 kB/s | 11 kB 00:00 2024-04-24T06:29:54.326 INFO:teuthology.orchestra.run.smithi005.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-04-24T06:29:54.343 INFO:teuthology.orchestra.run.smithi005.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 917 kB/s | 45 kB 00:00 2024-04-24T06:29:54.410 INFO:teuthology.orchestra.run.smithi005.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 7.8 MB/s | 662 kB 00:00 2024-04-24T06:29:54.444 INFO:teuthology.orchestra.run.smithi005.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 5.7 MB/s | 582 kB 00:00 2024-04-24T06:29:54.469 INFO:teuthology.orchestra.run.smithi005.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 442 kB/s | 25 kB 00:00 2024-04-24T06:29:54.494 INFO:teuthology.orchestra.run.smithi005.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 961 kB/s | 47 kB 00:00 2024-04-24T06:29:54.528 INFO:teuthology.orchestra.run.smithi005.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.0 MB/s | 292 kB 00:00 2024-04-24T06:29:54.570 INFO:teuthology.orchestra.run.smithi005.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.5 MB/s | 188 kB 00:00 2024-04-24T06:29:54.595 INFO:teuthology.orchestra.run.smithi005.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 529 kB/s | 35 kB 00:00 2024-04-24T06:29:54.679 INFO:teuthology.orchestra.run.smithi005.stdout:(66/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.5 MB/s | 3.0 MB 00:00 2024-04-24T06:29:54.739 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T06:29:54.795 INFO:teuthology.orchestra.run.smithi005.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-04-24T06:29:54.814 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T06:29:54.843 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T06:29:54.875 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T06:29:54.896 INFO:teuthology.orchestra.run.smithi005.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 15 MB/s | 4.7 MB 00:00 2024-04-24T06:29:54.987 INFO:teuthology.orchestra.run.smithi005.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-04-24T06:29:55.021 INFO:teuthology.orchestra.run.smithi005.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.0 MB/s | 133 kB 00:00 2024-04-24T06:29:55.071 INFO:teuthology.orchestra.run.smithi005.stdout:(71/152): postgresql-contrib-13.14-1.el9.x86_64 3.2 MB/s | 911 kB 00:00 2024-04-24T06:29:55.155 INFO:teuthology.orchestra.run.smithi005.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 7.6 MB/s | 1.0 MB 00:00 2024-04-24T06:29:55.180 INFO:teuthology.orchestra.run.smithi005.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 817 kB/s | 87 kB 00:00 2024-04-24T06:29:55.196 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T06:29:55.238 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : librbd1-2:19.0.0-3124.ga3c8a22b.el9.x86_64 99/156 2024-04-24T06:29:55.256 INFO:teuthology.orchestra.run.smithi005.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.2 MB/s | 244 kB 00:00 2024-04-24T06:29:55.313 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librbd1-2:19.0.0-3124.ga3c8a22b.el9.x86_64 99/156 2024-04-24T06:29:55.314 INFO:teuthology.orchestra.run.smithi005.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-04-24T06:29:55.343 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 100/156 2024-04-24T06:29:55.380 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T06:29:55.431 INFO:teuthology.orchestra.run.smithi005.stdout:(76/152): postgresql-server-13.14-1.el9.x86_64. 13 MB/s | 5.8 MB 00:00 2024-04-24T06:29:55.457 INFO:teuthology.orchestra.run.smithi005.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 182 kB/s | 26 kB 00:00 2024-04-24T06:29:55.482 INFO:teuthology.orchestra.run.smithi005.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 390 kB/s | 19 kB 00:00 2024-04-24T06:29:55.515 INFO:teuthology.orchestra.run.smithi005.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-24T06:29:55.541 INFO:teuthology.orchestra.run.smithi005.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 604 kB/s | 35 kB 00:00 2024-04-24T06:29:55.608 INFO:teuthology.orchestra.run.smithi005.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-04-24T06:29:55.716 INFO:teuthology.orchestra.run.smithi005.stdout:(82/152): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-04-24T06:29:55.750 INFO:teuthology.orchestra.run.smithi005.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.5 MB/s | 222 kB 00:00 2024-04-24T06:29:55.775 INFO:teuthology.orchestra.run.smithi005.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-24T06:29:55.800 INFO:teuthology.orchestra.run.smithi005.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 859 kB/s | 42 kB 00:00 2024-04-24T06:29:55.834 INFO:teuthology.orchestra.run.smithi005.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 269 kB 00:00 2024-04-24T06:29:55.867 INFO:teuthology.orchestra.run.smithi005.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.4 MB/s | 159 kB 00:00 2024-04-24T06:29:55.886 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T06:29:55.932 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librgw2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 103/156 2024-04-24T06:29:55.933 INFO:teuthology.orchestra.run.smithi005.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-04-24T06:29:55.933 INFO:teuthology.orchestra.run.smithi005.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 878 kB/s | 51 kB 00:00 2024-04-24T06:29:55.969 INFO:teuthology.orchestra.run.smithi005.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 806 kB/s | 54 kB 00:00 2024-04-24T06:29:55.971 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librgw2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 103/156 2024-04-24T06:29:56.001 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 104/156 2024-04-24T06:29:56.019 INFO:teuthology.orchestra.run.smithi005.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 850 kB/s | 42 kB 00:00 2024-04-24T06:29:56.070 INFO:teuthology.orchestra.run.smithi005.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 901 kB/s | 44 kB 00:00 2024-04-24T06:29:56.178 INFO:teuthology.orchestra.run.smithi005.stdout:(93/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.7 MB/s | 5.1 MB 00:00 2024-04-24T06:29:56.211 INFO:teuthology.orchestra.run.smithi005.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.5 MB/s | 221 kB 00:00 2024-04-24T06:29:56.228 INFO:teuthology.orchestra.run.smithi005.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 736 kB/s | 36 kB 00:00 2024-04-24T06:29:56.254 INFO:teuthology.orchestra.run.smithi005.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-04-24T06:29:56.287 INFO:teuthology.orchestra.run.smithi005.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.2 MB/s | 305 kB 00:00 2024-04-24T06:29:56.304 INFO:teuthology.orchestra.run.smithi005.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-24T06:29:56.338 INFO:teuthology.orchestra.run.smithi005.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.9 MB/s | 94 kB 00:00 2024-04-24T06:29:56.348 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 105/156 2024-04-24T06:29:56.355 INFO:teuthology.orchestra.run.smithi005.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-04-24T06:29:56.382 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 105/156 2024-04-24T06:29:56.388 INFO:teuthology.orchestra.run.smithi005.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 911 kB/s | 45 kB 00:00 2024-04-24T06:29:56.447 INFO:teuthology.orchestra.run.smithi005.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 245 kB/s | 22 kB 00:00 2024-04-24T06:29:56.748 INFO:teuthology.orchestra.run.smithi005.stdout:(103/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-04-24T06:29:56.823 INFO:teuthology.orchestra.run.smithi005.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.9 MB/s | 863 kB 00:00 2024-04-24T06:29:56.929 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 105/156 2024-04-24T06:29:56.977 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-base-2:19.0.0-3124.ga3c8a22b.el9.x86_64 106/156 2024-04-24T06:29:57.067 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-base-2:19.0.0-3124.ga3c8a22b.el9.x86_64 106/156 2024-04-24T06:29:57.068 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T06:29:57.068 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T06:29:57.068 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-24T06:29:57.095 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-selinux-2:19.0.0-3124.ga3c8a22b.el9.x86_64 107/156 2024-04-24T06:29:57.124 INFO:teuthology.orchestra.run.smithi005.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 65 kB/s | 19 kB 00:00 2024-04-24T06:29:57.621 INFO:teuthology.orchestra.run.smithi005.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 354 kB/s | 309 kB 00:00 2024-04-24T06:29:57.740 INFO:teuthology.orchestra.run.smithi005.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 220 kB/s | 26 kB 00:00 2024-04-24T06:29:57.774 INFO:teuthology.orchestra.run.smithi005.stdout:(108/152): abseil-cpp-20211102.0-3.el9.x86_64.r 413 kB/s | 548 kB 00:01 2024-04-24T06:29:57.866 INFO:teuthology.orchestra.run.smithi005.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 390 kB/s | 49 kB 00:00 2024-04-24T06:29:57.891 INFO:teuthology.orchestra.run.smithi005.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 579 kB/s | 67 kB 00:00 2024-04-24T06:29:58.085 INFO:teuthology.orchestra.run.smithi005.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 700 kB/s | 151 kB 00:00 2024-04-24T06:29:58.370 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 105/156 2024-04-24T06:29:58.405 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 105/156 2024-04-24T06:29:58.850 INFO:teuthology.orchestra.run.smithi005.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 716 kB/s | 548 kB 00:00 2024-04-24T06:29:58.950 INFO:teuthology.orchestra.run.smithi005.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 792 kB/s | 838 kB 00:01 2024-04-24T06:29:58.978 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 105/156 2024-04-24T06:29:58.992 INFO:teuthology.orchestra.run.smithi005.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 307 kB/s | 43 kB 00:00 2024-04-24T06:29:59.017 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-base-2:19.0.0-3124.ga3c8a22b.el9.x86_64 106/156 2024-04-24T06:29:59.092 INFO:teuthology.orchestra.run.smithi005.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 228 kB/s | 32 kB 00:00 2024-04-24T06:29:59.098 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-base-2:19.0.0-3124.ga3c8a22b.el9.x86_64 106/156 2024-04-24T06:29:59.098 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T06:29:59.098 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T06:29:59.098 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-24T06:29:59.127 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-selinux-2:19.0.0-3124.ga3c8a22b.el9.x86_64 107/156 2024-04-24T06:29:59.134 INFO:teuthology.orchestra.run.smithi005.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 106 kB/s | 15 kB 00:00 2024-04-24T06:29:59.393 INFO:teuthology.orchestra.run.smithi005.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 574 kB/s | 172 kB 00:00 2024-04-24T06:29:59.651 INFO:teuthology.orchestra.run.smithi005.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 694 kB/s | 358 kB 00:00 2024-04-24T06:29:59.768 INFO:teuthology.orchestra.run.smithi005.stdout:(119/152): python3-colorama-0.4.4-12.el9.noarch 305 kB/s | 35 kB 00:00 2024-04-24T06:29:59.935 INFO:teuthology.orchestra.run.smithi005.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 560 kB/s | 303 kB 00:00 2024-04-24T06:30:00.077 INFO:teuthology.orchestra.run.smithi005.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 668 kB/s | 206 kB 00:00 2024-04-24T06:30:00.294 INFO:teuthology.orchestra.run.smithi005.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 669 kB/s | 144 kB 00:00 2024-04-24T06:30:00.436 INFO:teuthology.orchestra.run.smithi005.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 76 kB/s | 11 kB 00:00 2024-04-24T06:30:00.569 INFO:teuthology.orchestra.run.smithi005.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 133 kB/s | 18 kB 00:00 2024-04-24T06:30:00.703 INFO:teuthology.orchestra.run.smithi005.stdout:(125/152): python3-jaraco-collections-3.0.0-8.e 174 kB/s | 23 kB 00:00 2024-04-24T06:30:00.837 INFO:teuthology.orchestra.run.smithi005.stdout:(126/152): python3-jaraco-functools-3.5.0-2.el9 147 kB/s | 19 kB 00:00 2024-04-24T06:30:01.038 INFO:teuthology.orchestra.run.smithi005.stdout:(127/152): python3-jaraco-text-3.2.0-6.el9.noar 99 kB/s | 20 kB 00:00 2024-04-24T06:30:02.362 INFO:teuthology.orchestra.run.smithi005.stdout:(128/152): python3-kubernetes-26.1.0-2.el9.noar 790 kB/s | 1.0 MB 00:01 2024-04-24T06:30:02.529 INFO:teuthology.orchestra.run.smithi005.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 279 kB/s | 46 kB 00:00 2024-04-24T06:30:02.613 INFO:teuthology.orchestra.run.smithi005.stdout:(130/152): python3-grpcio-1.46.7-10.el9.x86_64. 780 kB/s | 2.0 MB 00:02 2024-04-24T06:30:02.680 INFO:teuthology.orchestra.run.smithi005.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 527 kB/s | 79 kB 00:00 2024-04-24T06:30:02.771 INFO:teuthology.orchestra.run.smithi005.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 366 kB/s | 58 kB 00:00 2024-04-24T06:30:02.955 INFO:teuthology.orchestra.run.smithi005.stdout:(133/152): libarrow-9.0.0-10.el9.x86_64.rpm 776 kB/s | 4.4 MB 00:05 2024-04-24T06:30:02.981 INFO:teuthology.orchestra.run.smithi005.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 79 kB/s | 16 kB 00:00 2024-04-24T06:30:03.139 INFO:teuthology.orchestra.run.smithi005.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 594 kB/s | 272 kB 00:00 2024-04-24T06:30:03.164 INFO:teuthology.orchestra.run.smithi005.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 432 kB/s | 90 kB 00:00 2024-04-24T06:30:03.190 INFO:teuthology.orchestra.run.smithi005.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 136 kB/s | 28 kB 00:00 2024-04-24T06:30:03.258 INFO:teuthology.orchestra.run.smithi005.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 264 kB/s | 31 kB 00:00 2024-04-24T06:30:03.307 INFO:teuthology.orchestra.run.smithi005.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 508 kB/s | 59 kB 00:00 2024-04-24T06:30:03.374 INFO:teuthology.orchestra.run.smithi005.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 308 kB/s | 36 kB 00:00 2024-04-24T06:30:03.440 INFO:teuthology.orchestra.run.smithi005.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 385 kB/s | 51 kB 00:00 2024-04-24T06:30:03.466 INFO:teuthology.orchestra.run.smithi005.stdout:(142/152): python3-routes-2.5.1-5.el9.noarch.rp 627 kB/s | 188 kB 00:00 2024-04-24T06:30:03.658 INFO:teuthology.orchestra.run.smithi005.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 414 kB/s | 90 kB 00:00 2024-04-24T06:30:03.692 INFO:teuthology.orchestra.run.smithi005.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 725 kB/s | 230 kB 00:00 2024-04-24T06:30:03.775 INFO:teuthology.orchestra.run.smithi005.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 172 kB/s | 20 kB 00:00 2024-04-24T06:30:03.984 INFO:teuthology.orchestra.run.smithi005.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 826 kB/s | 427 kB 00:00 2024-04-24T06:30:04.009 INFO:teuthology.orchestra.run.smithi005.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 605 kB/s | 192 kB 00:00 2024-04-24T06:30:04.334 INFO:teuthology.orchestra.run.smithi005.stdout:(148/152): librados2-19.0.0-3124.ga3c8a22b.el9. 9.7 MB/s | 3.4 MB 00:00 2024-04-24T06:30:04.501 INFO:teuthology.orchestra.run.smithi005.stdout:(149/152): librbd1-19.0.0-3124.ga3c8a22b.el9.x8 6.4 MB/s | 3.1 MB 00:00 2024-04-24T06:30:04.593 INFO:teuthology.orchestra.run.smithi005.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 5.2 MB/s | 1.3 MB 00:00 2024-04-24T06:30:04.785 INFO:teuthology.orchestra.run.smithi005.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 8.5 MB/s | 2.4 MB 00:00 2024-04-24T06:30:05.652 INFO:teuthology.orchestra.run.smithi005.stdout:(152/152): thrift-0.15.0-2.el9.x86_64.rpm 864 kB/s | 1.6 MB 00:01 2024-04-24T06:30:05.658 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-04-24T06:30:05.658 INFO:teuthology.orchestra.run.smithi005.stdout:Total 11 MB/s | 222 MB 00:19 2024-04-24T06:30:08.510 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2024-04-24T06:30:08.641 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3124.ga3c8a22b.el9.x86_64 107/156 2024-04-24T06:30:08.641 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /sys 2024-04-24T06:30:08.641 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /proc 2024-04-24T06:30:08.641 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /mnt 2024-04-24T06:30:08.641 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /var/tmp 2024-04-24T06:30:08.641 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /home 2024-04-24T06:30:08.641 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /root 2024-04-24T06:30:08.641 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /tmp 2024-04-24T06:30:08.641 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-24T06:30:08.773 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2024-04-24T06:30:08.773 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2024-04-24T06:30:08.815 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-volume-2:19.0.0-3124.ga3c8a22b.el9.noarch 108/156 2024-04-24T06:30:08.831 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-volume-2:19.0.0-3124.ga3c8a22b.el9.noarch 108/156 2024-04-24T06:30:08.831 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:30:08.831 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T06:30:08.831 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-24T06:30:10.310 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-osd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 109/156 2024-04-24T06:30:10.336 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-osd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 109/156 2024-04-24T06:30:10.336 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:30:10.336 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T06:30:10.336 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T06:30:10.336 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T06:30:10.336 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-24T06:30:10.561 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mds-2:19.0.0-3124.ga3c8a22b.el9.x86_64 110/156 2024-04-24T06:30:10.583 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mds-2:19.0.0-3124.ga3c8a22b.el9.x86_64 110/156 2024-04-24T06:30:10.583 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:30:10.583 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T06:30:10.583 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T06:30:10.584 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T06:30:10.584 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-24T06:30:11.020 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mon-2:19.0.0-3124.ga3c8a22b.el9.x86_64 111/156 2024-04-24T06:30:11.042 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mon-2:19.0.0-3124.ga3c8a22b.el9.x86_64 111/156 2024-04-24T06:30:11.042 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:30:11.042 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T06:30:11.042 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T06:30:11.043 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T06:30:11.043 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-24T06:30:11.143 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3124.ga3c8a22b.el9.x86_64 107/156 2024-04-24T06:30:11.144 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /sys 2024-04-24T06:30:11.144 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /proc 2024-04-24T06:30:11.144 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /mnt 2024-04-24T06:30:11.144 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /var/tmp 2024-04-24T06:30:11.144 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /home 2024-04-24T06:30:11.144 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /root 2024-04-24T06:30:11.144 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /tmp 2024-04-24T06:30:11.144 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-24T06:30:11.149 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T06:30:11.198 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T06:30:11.253 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T06:30:11.318 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-volume-2:19.0.0-3124.ga3c8a22b.el9.noarch 108/156 2024-04-24T06:30:11.335 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-volume-2:19.0.0-3124.ga3c8a22b.el9.noarch 108/156 2024-04-24T06:30:11.335 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:30:11.335 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T06:30:11.335 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-24T06:30:11.438 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T06:30:11.487 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T06:30:11.554 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2024-04-24T06:30:11.554 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2024-04-24T06:30:11.555 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T06:30:11.646 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T06:30:11.773 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T06:30:11.812 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T06:30:11.844 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T06:30:11.872 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T06:30:12.376 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: cephadm-2:19.0.0-3124.ga3c8a22b.el9.noarch 123/156 2024-04-24T06:30:12.404 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephadm-2:19.0.0-3124.ga3c8a22b.el9.noarch 123/156 2024-04-24T06:30:12.436 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3124.ga3c8a22b.e 124/156 2024-04-24T06:30:12.497 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3124.ga3c8a22b. 125/156 2024-04-24T06:30:12.527 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3124.ga3c8a22b.el9.noa 126/156 2024-04-24T06:30:12.868 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-osd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 109/156 2024-04-24T06:30:12.896 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-osd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 109/156 2024-04-24T06:30:12.896 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:30:12.896 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T06:30:12.896 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T06:30:12.896 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T06:30:12.896 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-24T06:30:13.135 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mds-2:19.0.0-3124.ga3c8a22b.el9.x86_64 110/156 2024-04-24T06:30:13.159 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mds-2:19.0.0-3124.ga3c8a22b.el9.x86_64 110/156 2024-04-24T06:30:13.160 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:30:13.160 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T06:30:13.160 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T06:30:13.160 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T06:30:13.160 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-24T06:30:13.553 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3124.ga3c8a22b.el9.noa 126/156 2024-04-24T06:30:13.587 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3124.ga3c8a22b.el9.n 127/156 2024-04-24T06:30:13.628 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mon-2:19.0.0-3124.ga3c8a22b.el9.x86_64 111/156 2024-04-24T06:30:13.651 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mon-2:19.0.0-3124.ga3c8a22b.el9.x86_64 111/156 2024-04-24T06:30:13.651 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:30:13.651 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T06:30:13.651 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T06:30:13.651 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T06:30:13.651 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-24T06:30:13.758 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T06:30:13.817 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T06:30:13.881 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T06:30:14.078 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T06:30:14.129 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T06:30:14.211 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T06:30:14.308 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T06:30:14.453 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T06:30:14.496 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T06:30:14.527 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T06:30:14.547 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3124.ga3c8a22b.el9.n 127/156 2024-04-24T06:30:14.555 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T06:30:14.579 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3124.ga3c 128/156 2024-04-24T06:30:14.679 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3124.ga3c 128/156 2024-04-24T06:30:14.852 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3124.ga3c8a22b.el 129/156 2024-04-24T06:30:14.881 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-2:19.0.0-3124.ga3c8a22b.el9.x86_64 130/156 2024-04-24T06:30:14.904 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3124.ga3c8a22b.el9.x86_64 130/156 2024-04-24T06:30:14.904 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:30:14.904 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T06:30:14.904 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T06:30:14.905 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T06:30:14.905 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-24T06:30:14.948 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-rook-2:19.0.0-3124.ga3c8a22b.el9.noarch 131/156 2024-04-24T06:30:14.960 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3124.ga3c8a22b.el9.noarch 131/156 2024-04-24T06:30:15.001 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: cephadm-2:19.0.0-3124.ga3c8a22b.el9.noarch 123/156 2024-04-24T06:30:15.029 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephadm-2:19.0.0-3124.ga3c8a22b.el9.noarch 123/156 2024-04-24T06:30:15.057 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-2:19.0.0-3124.ga3c8a22b.el9.x86_64 132/156 2024-04-24T06:30:15.070 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3124.ga3c8a22b.e 124/156 2024-04-24T06:30:15.113 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 133/156 2024-04-24T06:30:15.134 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3124.ga3c8a22b. 125/156 2024-04-24T06:30:15.160 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3124.ga3c8a22b.el9.noa 126/156 2024-04-24T06:30:16.042 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T06:30:16.076 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-radosgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 135/156 2024-04-24T06:30:16.098 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 135/156 2024-04-24T06:30:16.099 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:30:16.099 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T06:30:16.099 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T06:30:16.099 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T06:30:16.099 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-24T06:30:16.143 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3124.ga3c8a 136/156 2024-04-24T06:30:16.165 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3124.ga3c8a 136/156 2024-04-24T06:30:16.165 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:30:16.165 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T06:30:16.165 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-24T06:30:16.210 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephfs-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 137/156 2024-04-24T06:30:16.233 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 137/156 2024-04-24T06:30:16.233 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:30:16.233 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T06:30:16.233 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T06:30:16.233 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T06:30:16.233 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-24T06:30:16.264 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3124.ga3c8a22b.el9.noa 126/156 2024-04-24T06:30:16.295 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3124.ga3c8a22b.el9.n 127/156 2024-04-24T06:30:16.536 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 138/156 2024-04-24T06:30:16.559 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 138/156 2024-04-24T06:30:16.559 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:30:16.559 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T06:30:16.559 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T06:30:16.561 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T06:30:16.561 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-24T06:30:17.317 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3124.ga3c8a22b.el9.n 127/156 2024-04-24T06:30:17.345 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3124.ga3c 128/156 2024-04-24T06:30:17.455 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3124.ga3c 128/156 2024-04-24T06:30:17.616 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3124.ga3c8a22b.el 129/156 2024-04-24T06:30:17.639 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-2:19.0.0-3124.ga3c8a22b.el9.x86_64 130/156 2024-04-24T06:30:17.664 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3124.ga3c8a22b.el9.x86_64 130/156 2024-04-24T06:30:17.664 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:30:17.664 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T06:30:17.664 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T06:30:17.664 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T06:30:17.664 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-24T06:30:17.715 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-rook-2:19.0.0-3124.ga3c8a22b.el9.noarch 131/156 2024-04-24T06:30:17.728 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3124.ga3c8a22b.el9.noarch 131/156 2024-04-24T06:30:17.828 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-2:19.0.0-3124.ga3c8a22b.el9.x86_64 132/156 2024-04-24T06:30:17.881 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 133/156 2024-04-24T06:30:18.441 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2024-04-24T06:30:18.593 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T06:30:18.674 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T06:30:18.789 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T06:30:18.849 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T06:30:18.859 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T06:30:18.884 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-radosgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 135/156 2024-04-24T06:30:18.887 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T06:30:18.909 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 135/156 2024-04-24T06:30:18.909 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:30:18.909 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T06:30:18.909 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T06:30:18.909 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T06:30:18.909 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-24T06:30:18.952 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3124.ga3c8a 136/156 2024-04-24T06:30:18.975 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3124.ga3c8a 136/156 2024-04-24T06:30:18.975 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:30:18.976 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T06:30:18.976 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-24T06:30:19.019 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T06:30:19.027 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 137/156 2024-04-24T06:30:19.051 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 137/156 2024-04-24T06:30:19.051 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:30:19.051 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T06:30:19.051 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T06:30:19.051 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T06:30:19.052 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-24T06:30:19.369 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 138/156 2024-04-24T06:30:19.393 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 138/156 2024-04-24T06:30:19.393 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:30:19.393 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T06:30:19.393 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T06:30:19.393 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T06:30:19.393 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-24T06:30:19.461 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T06:30:19.576 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : librados2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 8/156 2024-04-24T06:30:19.782 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librados2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 8/156 2024-04-24T06:30:19.810 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephfs2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 9/156 2024-04-24T06:30:19.939 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libcephfs2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 9/156 2024-04-24T06:30:19.985 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rados-2:19.0.0-3124.ga3c8a22b.el9.x86_64 10/156 2024-04-24T06:30:20.417 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T06:30:20.485 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T06:30:20.515 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephsqlite-2:19.0.0-3124.ga3c8a22b.el9.x86_64 13/156 2024-04-24T06:30:20.688 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3124.ga3c8a22b.el9.x86_64 13/156 2024-04-24T06:30:20.726 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libradosstriper1-2:19.0.0-3124.ga3c8a22b.el9.x86 14/156 2024-04-24T06:30:20.821 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3124.ga3c8a22b.el9.x86 14/156 2024-04-24T06:30:20.850 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T06:30:20.926 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T06:30:21.140 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-test-2:19.0.0-3124.ga3c8a22b.el9.x86_64 139/156 2024-04-24T06:30:21.181 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 140/156 2024-04-24T06:30:21.230 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-nbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 141/156 2024-04-24T06:30:21.235 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T06:30:21.262 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T06:30:21.289 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T06:30:21.301 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T06:30:21.469 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephfs-shell-2:19.0.0-3124.ga3c8a22b.el9.x86_64 144/156 2024-04-24T06:30:21.503 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T06:30:21.527 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T06:30:21.557 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephfs-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_ 146/156 2024-04-24T06:30:21.630 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T06:30:21.668 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T06:30:21.715 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T06:30:21.837 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T06:30:21.897 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T06:30:21.961 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T06:30:22.010 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T06:30:22.050 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T06:30:22.144 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T06:30:22.171 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T06:30:22.180 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T06:30:22.200 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T06:30:22.242 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T06:30:22.277 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T06:30:22.332 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T06:30:22.443 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T06:30:22.488 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ceph-common-2:19.0.0-3124.ga3c8a22b.el9. 31/156 2024-04-24T06:30:22.542 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ceph-argparse-2:19.0.0-3124.ga3c8a22b.el 32/156 2024-04-24T06:30:22.607 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cephfs-2:19.0.0-3124.ga3c8a22b.el9.x86_6 33/156 2024-04-24T06:30:22.737 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T06:30:22.795 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T06:30:22.909 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T06:30:23.021 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephfs-top-2:19.0.0-3124.ga3c8a22b.el9.noarch 148/156 2024-04-24T06:30:23.093 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T06:30:23.105 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T06:30:23.133 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T06:30:23.176 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T06:30:23.214 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T06:30:23.247 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T06:30:23.247 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T06:30:23.331 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T06:30:23.364 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T06:30:23.364 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T06:30:23.455 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T06:30:23.528 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T06:30:23.858 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T06:30:23.976 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T06:30:24.017 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T06:30:24.064 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T06:30:24.113 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librados-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_6 43/156 2024-04-24T06:30:24.178 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T06:30:24.258 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-test-2:19.0.0-3124.ga3c8a22b.el9.x86_64 139/156 2024-04-24T06:30:24.299 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 140/156 2024-04-24T06:30:24.340 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-nbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 141/156 2024-04-24T06:30:24.371 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T06:30:24.398 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T06:30:24.440 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T06:30:24.496 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T06:30:24.579 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T06:30:24.602 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-shell-2:19.0.0-3124.ga3c8a22b.el9.x86_64 144/156 2024-04-24T06:30:24.653 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T06:30:24.679 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T06:30:24.682 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephfs-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_ 146/156 2024-04-24T06:30:24.745 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T06:30:24.786 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T06:30:24.859 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T06:30:24.901 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T06:30:24.965 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T06:30:25.031 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T06:30:25.243 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T06:30:25.299 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T06:30:25.323 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T06:30:25.363 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T06:30:25.385 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T06:30:25.487 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T06:30:25.659 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T06:30:25.818 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T06:30:25.861 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T06:30:25.914 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T06:30:25.951 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T06:30:25.984 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T06:30:26.189 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T06:30:26.289 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-top-2:19.0.0-3124.ga3c8a22b.el9.noarch 148/156 2024-04-24T06:30:26.383 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T06:30:26.385 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T06:30:26.424 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T06:30:26.428 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T06:30:26.463 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T06:30:26.510 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T06:30:26.522 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T06:30:26.556 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T06:30:26.556 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T06:30:26.569 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T06:30:26.627 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T06:30:26.640 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T06:30:26.669 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T06:30:26.681 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T06:30:26.681 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T06:30:26.715 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T06:30:26.765 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T06:30:26.770 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T06:30:26.840 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T06:30:26.854 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T06:30:26.904 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T06:30:27.047 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T06:30:27.118 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T06:30:27.162 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T06:30:27.235 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T06:30:27.395 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T06:30:27.467 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T06:30:28.453 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T06:30:28.516 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T06:30:28.566 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T06:30:28.684 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T06:30:28.752 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T06:30:28.862 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T06:30:28.909 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T06:30:29.153 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T06:30:29.189 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T06:30:29.295 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T06:30:29.350 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T06:30:29.350 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-2:19.0.0-3124.ga3c8a22b.el9.x86_64 1/156 2024-04-24T06:30:29.350 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-base-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2/156 2024-04-24T06:30:29.350 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 3/156 2024-04-24T06:30:29.350 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 4/156 2024-04-24T06:30:29.350 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3124.ga3c8a 5/156 2024-04-24T06:30:29.350 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mds-2:19.0.0-3124.ga3c8a22b.el9.x86_64 6/156 2024-04-24T06:30:29.350 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-2:19.0.0-3124.ga3c8a22b.el9.x86_64 7/156 2024-04-24T06:30:29.350 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mon-2:19.0.0-3124.ga3c8a22b.el9.x86_64 8/156 2024-04-24T06:30:29.351 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-osd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 9/156 2024-04-24T06:30:29.351 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-radosgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 10/156 2024-04-24T06:30:29.351 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-selinux-2:19.0.0-3124.ga3c8a22b.el9.x86_64 11/156 2024-04-24T06:30:29.351 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-test-2:19.0.0-3124.ga3c8a22b.el9.x86_64 12/156 2024-04-24T06:30:29.351 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephfs-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 13/156 2024-04-24T06:30:29.351 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephfs-shell-2:19.0.0-3124.ga3c8a22b.el9.x86_64 14/156 2024-04-24T06:30:29.351 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephfs-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_ 15/156 2024-04-24T06:30:29.351 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephfs2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 16/156 2024-04-24T06:30:29.351 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephsqlite-2:19.0.0-3124.ga3c8a22b.el9.x86_64 17/156 2024-04-24T06:30:29.351 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_6 18/156 2024-04-24T06:30:29.351 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libradosstriper1-2:19.0.0-3124.ga3c8a22b.el9.x86 19/156 2024-04-24T06:30:29.352 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librgw2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 20/156 2024-04-24T06:30:29.352 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3124.ga3c8a22b.el 21/156 2024-04-24T06:30:29.352 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ceph-common-2:19.0.0-3124.ga3c8a22b.el9. 22/156 2024-04-24T06:30:29.352 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cephfs-2:19.0.0-3124.ga3c8a22b.el9.x86_6 23/156 2024-04-24T06:30:29.352 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rados-2:19.0.0-3124.ga3c8a22b.el9.x86_64 24/156 2024-04-24T06:30:29.352 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 25/156 2024-04-24T06:30:29.352 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 26/156 2024-04-24T06:30:29.352 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 27/156 2024-04-24T06:30:29.352 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 28/156 2024-04-24T06:30:29.352 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-nbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 29/156 2024-04-24T06:30:29.353 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3124.ga3c8a22b. 30/156 2024-04-24T06:30:29.353 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3124.ga3c8a22b.el9.noa 31/156 2024-04-24T06:30:29.353 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3124.ga3c8a22b.el9.n 32/156 2024-04-24T06:30:29.353 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3124.ga3c 33/156 2024-04-24T06:30:29.353 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3124.ga3c8a22b.el 34/156 2024-04-24T06:30:29.353 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3124.ga3c8a22b.el9.noarch 35/156 2024-04-24T06:30:29.353 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3124.ga3c8a22b.e 36/156 2024-04-24T06:30:29.353 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-volume-2:19.0.0-3124.ga3c8a22b.el9.noarch 37/156 2024-04-24T06:30:29.353 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephadm-2:19.0.0-3124.ga3c8a22b.el9.noarch 38/156 2024-04-24T06:30:29.353 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephfs-top-2:19.0.0-3124.ga3c8a22b.el9.noarch 39/156 2024-04-24T06:30:29.353 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T06:30:29.353 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T06:30:29.354 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T06:30:29.354 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T06:30:29.354 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T06:30:29.354 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T06:30:29.354 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T06:30:29.354 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T06:30:29.354 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T06:30:29.354 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T06:30:29.354 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T06:30:29.355 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T06:30:29.355 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T06:30:29.355 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T06:30:29.355 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T06:30:29.355 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T06:30:29.355 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T06:30:29.355 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T06:30:29.355 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T06:30:29.355 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T06:30:29.355 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T06:30:29.355 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T06:30:29.355 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T06:30:29.356 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T06:30:29.356 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T06:30:29.356 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T06:30:29.356 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T06:30:29.356 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T06:30:29.356 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T06:30:29.356 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T06:30:29.356 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T06:30:29.357 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T06:30:29.357 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T06:30:29.357 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T06:30:29.357 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T06:30:29.357 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T06:30:29.357 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T06:30:29.357 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T06:30:29.357 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T06:30:29.357 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T06:30:29.357 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T06:30:29.357 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T06:30:29.358 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T06:30:29.358 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T06:30:29.358 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T06:30:29.358 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T06:30:29.358 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T06:30:29.358 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T06:30:29.358 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T06:30:29.358 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T06:30:29.358 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T06:30:29.358 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T06:30:29.358 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T06:30:29.358 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T06:30:29.359 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T06:30:29.359 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T06:30:29.359 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T06:30:29.359 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T06:30:29.359 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T06:30:29.359 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T06:30:29.359 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T06:30:29.359 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T06:30:29.359 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T06:30:29.359 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T06:30:29.360 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T06:30:29.360 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T06:30:29.360 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T06:30:29.360 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T06:30:29.360 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T06:30:29.360 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T06:30:29.360 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T06:30:29.361 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T06:30:29.361 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T06:30:29.361 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T06:30:29.361 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T06:30:29.361 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T06:30:29.361 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T06:30:29.361 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T06:30:29.361 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T06:30:29.362 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T06:30:29.362 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T06:30:29.362 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T06:30:29.362 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T06:30:29.362 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T06:30:29.362 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T06:30:29.362 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T06:30:29.362 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T06:30:29.362 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T06:30:29.362 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T06:30:29.362 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T06:30:29.362 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T06:30:29.363 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T06:30:29.365 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T06:30:29.365 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T06:30:29.366 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T06:30:29.366 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T06:30:29.366 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T06:30:29.366 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T06:30:29.366 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T06:30:29.366 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T06:30:29.366 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T06:30:29.366 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T06:30:29.366 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T06:30:29.367 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T06:30:29.367 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T06:30:29.367 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T06:30:29.367 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T06:30:29.367 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T06:30:29.367 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T06:30:29.367 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 149/156 2024-04-24T06:30:29.367 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T06:30:29.367 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librbd1-2:19.0.0-3124.ga3c8a22b.el9.x86_64 151/156 2024-04-24T06:30:29.367 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T06:30:29.367 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T06:30:29.367 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T06:30:29.368 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T06:30:30.150 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T06:30:30.325 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T06:30:31.457 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T06:30:31.457 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-24T06:30:31.457 INFO:teuthology.orchestra.run.smithi137.stdout:Upgraded: 2024-04-24T06:30:31.457 INFO:teuthology.orchestra.run.smithi137.stdout: librados2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:31.458 INFO:teuthology.orchestra.run.smithi137.stdout: librbd1-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:31.458 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T06:30:31.458 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T06:30:31.458 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-04-24T06:30:31.458 INFO:teuthology.orchestra.run.smithi137.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T06:30:31.458 INFO:teuthology.orchestra.run.smithi137.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T06:30:31.458 INFO:teuthology.orchestra.run.smithi137.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T06:30:31.458 INFO:teuthology.orchestra.run.smithi137.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T06:30:31.458 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:31.459 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-base-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:31.459 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:31.459 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:31.459 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-grafana-dashboards-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:30:31.459 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-immutable-object-cache-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:31.459 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mds-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:31.459 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:31.459 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-cephadm-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:30:31.459 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-dashboard-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:30:31.459 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:30:31.459 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-modules-core-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:30:31.459 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-rook-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:30:31.459 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mon-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:31.459 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-osd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:31.460 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-prometheus-alerts-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:30:31.460 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-radosgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:31.460 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-selinux-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:31.460 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-test-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:31.460 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-volume-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:30:31.460 INFO:teuthology.orchestra.run.smithi137.stdout: cephadm-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:30:31.460 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:31.460 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-shell-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:31.460 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-top-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:30:31.460 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T06:30:31.460 INFO:teuthology.orchestra.run.smithi137.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T06:30:31.460 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T06:30:31.460 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T06:30:31.460 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T06:30:31.461 INFO:teuthology.orchestra.run.smithi137.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T06:30:31.461 INFO:teuthology.orchestra.run.smithi137.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T06:30:31.461 INFO:teuthology.orchestra.run.smithi137.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T06:30:31.461 INFO:teuthology.orchestra.run.smithi137.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T06:30:31.461 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T06:30:31.461 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T06:30:31.461 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:31.461 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:31.461 INFO:teuthology.orchestra.run.smithi137.stdout: libcephsqlite-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:31.461 INFO:teuthology.orchestra.run.smithi137.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T06:30:31.461 INFO:teuthology.orchestra.run.smithi137.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T06:30:31.461 INFO:teuthology.orchestra.run.smithi137.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T06:30:31.462 INFO:teuthology.orchestra.run.smithi137.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T06:30:31.462 INFO:teuthology.orchestra.run.smithi137.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T06:30:31.462 INFO:teuthology.orchestra.run.smithi137.stdout: librados-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:31.462 INFO:teuthology.orchestra.run.smithi137.stdout: libradosstriper1-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:31.462 INFO:teuthology.orchestra.run.smithi137.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T06:30:31.462 INFO:teuthology.orchestra.run.smithi137.stdout: librgw2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:31.462 INFO:teuthology.orchestra.run.smithi137.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T06:30:31.462 INFO:teuthology.orchestra.run.smithi137.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T06:30:31.462 INFO:teuthology.orchestra.run.smithi137.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T06:30:31.462 INFO:teuthology.orchestra.run.smithi137.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T06:30:31.462 INFO:teuthology.orchestra.run.smithi137.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T06:30:31.462 INFO:teuthology.orchestra.run.smithi137.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T06:30:31.462 INFO:teuthology.orchestra.run.smithi137.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T06:30:31.463 INFO:teuthology.orchestra.run.smithi137.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T06:30:31.463 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T06:30:31.463 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T06:30:31.463 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T06:30:31.463 INFO:teuthology.orchestra.run.smithi137.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T06:30:31.463 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T06:30:31.463 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T06:30:31.463 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T06:30:31.463 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T06:30:31.463 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T06:30:31.463 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T06:30:31.463 INFO:teuthology.orchestra.run.smithi137.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T06:30:31.463 INFO:teuthology.orchestra.run.smithi137.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T06:30:31.463 INFO:teuthology.orchestra.run.smithi137.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T06:30:31.464 INFO:teuthology.orchestra.run.smithi137.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T06:30:31.464 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T06:30:31.464 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-argparse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:31.464 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:31.464 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cephfs-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:31.464 INFO:teuthology.orchestra.run.smithi137.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T06:30:31.464 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T06:30:31.464 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T06:30:31.464 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T06:30:31.464 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T06:30:31.464 INFO:teuthology.orchestra.run.smithi137.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T06:30:31.464 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T06:30:31.464 INFO:teuthology.orchestra.run.smithi137.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T06:30:31.465 INFO:teuthology.orchestra.run.smithi137.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T06:30:31.465 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T06:30:31.465 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T06:30:31.465 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T06:30:31.465 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T06:30:31.465 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T06:30:31.465 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T06:30:31.465 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T06:30:31.465 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T06:30:31.465 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T06:30:31.465 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T06:30:31.466 INFO:teuthology.orchestra.run.smithi137.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T06:30:31.466 INFO:teuthology.orchestra.run.smithi137.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T06:30:31.466 INFO:teuthology.orchestra.run.smithi137.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T06:30:31.466 INFO:teuthology.orchestra.run.smithi137.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T06:30:31.466 INFO:teuthology.orchestra.run.smithi137.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T06:30:31.466 INFO:teuthology.orchestra.run.smithi137.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T06:30:31.466 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T06:30:31.466 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T06:30:31.466 INFO:teuthology.orchestra.run.smithi137.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T06:30:31.466 INFO:teuthology.orchestra.run.smithi137.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T06:30:31.466 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T06:30:31.467 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T06:30:31.467 INFO:teuthology.orchestra.run.smithi137.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T06:30:31.467 INFO:teuthology.orchestra.run.smithi137.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T06:30:31.467 INFO:teuthology.orchestra.run.smithi137.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T06:30:31.467 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T06:30:31.467 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T06:30:31.467 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T06:30:31.467 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T06:30:31.467 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T06:30:31.467 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T06:30:31.467 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rados-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:31.467 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:31.467 INFO:teuthology.orchestra.run.smithi137.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T06:30:31.468 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T06:30:31.468 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T06:30:31.468 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:31.468 INFO:teuthology.orchestra.run.smithi137.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T06:30:31.468 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T06:30:31.468 INFO:teuthology.orchestra.run.smithi137.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T06:30:31.468 INFO:teuthology.orchestra.run.smithi137.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T06:30:31.468 INFO:teuthology.orchestra.run.smithi137.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T06:30:31.468 INFO:teuthology.orchestra.run.smithi137.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T06:30:31.468 INFO:teuthology.orchestra.run.smithi137.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T06:30:31.468 INFO:teuthology.orchestra.run.smithi137.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T06:30:31.468 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T06:30:31.469 INFO:teuthology.orchestra.run.smithi137.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T06:30:31.469 INFO:teuthology.orchestra.run.smithi137.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T06:30:31.469 INFO:teuthology.orchestra.run.smithi137.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T06:30:31.469 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T06:30:31.469 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T06:30:31.469 INFO:teuthology.orchestra.run.smithi137.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T06:30:31.469 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:31.469 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:31.469 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-nbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:31.469 INFO:teuthology.orchestra.run.smithi137.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T06:30:31.469 INFO:teuthology.orchestra.run.smithi137.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T06:30:31.469 INFO:teuthology.orchestra.run.smithi137.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T06:30:31.470 INFO:teuthology.orchestra.run.smithi137.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T06:30:31.470 INFO:teuthology.orchestra.run.smithi137.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T06:30:31.470 INFO:teuthology.orchestra.run.smithi137.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T06:30:31.470 INFO:teuthology.orchestra.run.smithi137.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T06:30:31.470 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-24T06:30:31.470 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-04-24T06:30:31.842 DEBUG:teuthology.parallel:result is None 2024-04-24T06:30:31.970 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T06:30:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T06:30:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T06:30:32.166 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T06:30:32.493 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T06:30:32.493 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-2:19.0.0-3124.ga3c8a22b.el9.x86_64 1/156 2024-04-24T06:30:32.493 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-base-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2/156 2024-04-24T06:30:32.493 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 3/156 2024-04-24T06:30:32.493 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 4/156 2024-04-24T06:30:32.493 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3124.ga3c8a 5/156 2024-04-24T06:30:32.493 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mds-2:19.0.0-3124.ga3c8a22b.el9.x86_64 6/156 2024-04-24T06:30:32.493 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-2:19.0.0-3124.ga3c8a22b.el9.x86_64 7/156 2024-04-24T06:30:32.493 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mon-2:19.0.0-3124.ga3c8a22b.el9.x86_64 8/156 2024-04-24T06:30:32.493 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-osd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 9/156 2024-04-24T06:30:32.494 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-radosgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 10/156 2024-04-24T06:30:32.494 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-selinux-2:19.0.0-3124.ga3c8a22b.el9.x86_64 11/156 2024-04-24T06:30:32.494 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-test-2:19.0.0-3124.ga3c8a22b.el9.x86_64 12/156 2024-04-24T06:30:32.494 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 13/156 2024-04-24T06:30:32.494 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-shell-2:19.0.0-3124.ga3c8a22b.el9.x86_64 14/156 2024-04-24T06:30:32.494 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephfs-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_ 15/156 2024-04-24T06:30:32.494 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephfs2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 16/156 2024-04-24T06:30:32.494 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephsqlite-2:19.0.0-3124.ga3c8a22b.el9.x86_64 17/156 2024-04-24T06:30:32.494 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_6 18/156 2024-04-24T06:30:32.494 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libradosstriper1-2:19.0.0-3124.ga3c8a22b.el9.x86 19/156 2024-04-24T06:30:32.494 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librgw2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 20/156 2024-04-24T06:30:32.494 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3124.ga3c8a22b.el 21/156 2024-04-24T06:30:32.494 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ceph-common-2:19.0.0-3124.ga3c8a22b.el9. 22/156 2024-04-24T06:30:32.495 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cephfs-2:19.0.0-3124.ga3c8a22b.el9.x86_6 23/156 2024-04-24T06:30:32.495 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rados-2:19.0.0-3124.ga3c8a22b.el9.x86_64 24/156 2024-04-24T06:30:32.495 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 25/156 2024-04-24T06:30:32.495 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 26/156 2024-04-24T06:30:32.496 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 27/156 2024-04-24T06:30:32.496 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 28/156 2024-04-24T06:30:32.496 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-nbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 29/156 2024-04-24T06:30:32.496 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3124.ga3c8a22b. 30/156 2024-04-24T06:30:32.496 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3124.ga3c8a22b.el9.noa 31/156 2024-04-24T06:30:32.496 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3124.ga3c8a22b.el9.n 32/156 2024-04-24T06:30:32.496 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3124.ga3c 33/156 2024-04-24T06:30:32.496 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3124.ga3c8a22b.el 34/156 2024-04-24T06:30:32.496 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3124.ga3c8a22b.el9.noarch 35/156 2024-04-24T06:30:32.496 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3124.ga3c8a22b.e 36/156 2024-04-24T06:30:32.497 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-volume-2:19.0.0-3124.ga3c8a22b.el9.noarch 37/156 2024-04-24T06:30:32.497 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephadm-2:19.0.0-3124.ga3c8a22b.el9.noarch 38/156 2024-04-24T06:30:32.497 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-top-2:19.0.0-3124.ga3c8a22b.el9.noarch 39/156 2024-04-24T06:30:32.497 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T06:30:32.497 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T06:30:32.497 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T06:30:32.497 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T06:30:32.497 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T06:30:32.497 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T06:30:32.497 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T06:30:32.497 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T06:30:32.497 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T06:30:32.497 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T06:30:32.497 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T06:30:32.497 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T06:30:32.498 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T06:30:32.498 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T06:30:32.498 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T06:30:32.498 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T06:30:32.498 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T06:30:32.498 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T06:30:32.498 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T06:30:32.498 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T06:30:32.498 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T06:30:32.498 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T06:30:32.498 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T06:30:32.498 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T06:30:32.498 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T06:30:32.499 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T06:30:32.499 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T06:30:32.499 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T06:30:32.499 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T06:30:32.499 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T06:30:32.499 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T06:30:32.499 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T06:30:32.499 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T06:30:32.499 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T06:30:32.499 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T06:30:32.499 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T06:30:32.499 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T06:30:32.499 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T06:30:32.499 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T06:30:32.499 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T06:30:32.500 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T06:30:32.500 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T06:30:32.500 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T06:30:32.500 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T06:30:32.500 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T06:30:32.500 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T06:30:32.500 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T06:30:32.500 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T06:30:32.500 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T06:30:32.500 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T06:30:32.500 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T06:30:32.500 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T06:30:32.500 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T06:30:32.500 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T06:30:32.501 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T06:30:32.501 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T06:30:32.501 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T06:30:32.501 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T06:30:32.501 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T06:30:32.501 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T06:30:32.501 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T06:30:32.501 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T06:30:32.501 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T06:30:32.501 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T06:30:32.501 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T06:30:32.501 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T06:30:32.501 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T06:30:32.501 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T06:30:32.502 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T06:30:32.502 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T06:30:32.502 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T06:30:32.502 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T06:30:32.502 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T06:30:32.502 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T06:30:32.502 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T06:30:32.502 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T06:30:32.502 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T06:30:32.502 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T06:30:32.502 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T06:30:32.502 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T06:30:32.502 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T06:30:32.502 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T06:30:32.503 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T06:30:32.503 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T06:30:32.503 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T06:30:32.503 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T06:30:32.503 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T06:30:32.503 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T06:30:32.503 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T06:30:32.503 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T06:30:32.503 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T06:30:32.503 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T06:30:32.503 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T06:30:32.503 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T06:30:32.503 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T06:30:32.503 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T06:30:32.503 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T06:30:32.504 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T06:30:32.504 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T06:30:32.504 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T06:30:32.504 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T06:30:32.504 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T06:30:32.504 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T06:30:32.504 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T06:30:32.504 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T06:30:32.504 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T06:30:32.504 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T06:30:32.504 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T06:30:32.504 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T06:30:32.504 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 149/156 2024-04-24T06:30:32.504 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T06:30:32.504 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librbd1-2:19.0.0-3124.ga3c8a22b.el9.x86_64 151/156 2024-04-24T06:30:32.505 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T06:30:32.505 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T06:30:32.505 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T06:30:32.505 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T06:30:32.570 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T06:30:32.625 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : librbd1-2:19.0.0-3124.ga3c8a22b.el9.x86_64 99/156 2024-04-24T06:30:32.783 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librbd1-2:19.0.0-3124.ga3c8a22b.el9.x86_64 99/156 2024-04-24T06:30:32.815 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 100/156 2024-04-24T06:30:32.874 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T06:30:33.461 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T06:30:33.532 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librgw2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 103/156 2024-04-24T06:30:33.638 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librgw2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 103/156 2024-04-24T06:30:33.671 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 104/156 2024-04-24T06:30:34.528 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T06:30:34.528 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-24T06:30:34.528 INFO:teuthology.orchestra.run.smithi037.stdout:Upgraded: 2024-04-24T06:30:34.528 INFO:teuthology.orchestra.run.smithi037.stdout: librados2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:34.528 INFO:teuthology.orchestra.run.smithi037.stdout: librbd1-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:34.529 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T06:30:34.529 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T06:30:34.529 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-04-24T06:30:34.529 INFO:teuthology.orchestra.run.smithi037.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T06:30:34.529 INFO:teuthology.orchestra.run.smithi037.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T06:30:34.529 INFO:teuthology.orchestra.run.smithi037.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T06:30:34.529 INFO:teuthology.orchestra.run.smithi037.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T06:30:34.529 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:34.529 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-base-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:34.529 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:34.529 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:34.530 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-grafana-dashboards-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:30:34.530 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-immutable-object-cache-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:34.530 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mds-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:34.530 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:34.530 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-cephadm-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:30:34.530 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-dashboard-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:30:34.530 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:30:34.530 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-modules-core-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:30:34.530 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-rook-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:30:34.530 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mon-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:34.530 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-osd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:34.530 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-prometheus-alerts-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:30:34.530 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-radosgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:34.531 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-selinux-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:34.531 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-test-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:34.531 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-volume-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:30:34.531 INFO:teuthology.orchestra.run.smithi037.stdout: cephadm-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:30:34.531 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:34.531 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-shell-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:34.531 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-top-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:30:34.531 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T06:30:34.531 INFO:teuthology.orchestra.run.smithi037.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T06:30:34.531 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T06:30:34.531 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T06:30:34.531 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T06:30:34.531 INFO:teuthology.orchestra.run.smithi037.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T06:30:34.531 INFO:teuthology.orchestra.run.smithi037.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T06:30:34.532 INFO:teuthology.orchestra.run.smithi037.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T06:30:34.532 INFO:teuthology.orchestra.run.smithi037.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T06:30:34.532 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T06:30:34.532 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T06:30:34.532 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:34.532 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:34.532 INFO:teuthology.orchestra.run.smithi037.stdout: libcephsqlite-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:34.532 INFO:teuthology.orchestra.run.smithi037.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T06:30:34.532 INFO:teuthology.orchestra.run.smithi037.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T06:30:34.532 INFO:teuthology.orchestra.run.smithi037.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T06:30:34.532 INFO:teuthology.orchestra.run.smithi037.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T06:30:34.532 INFO:teuthology.orchestra.run.smithi037.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T06:30:34.532 INFO:teuthology.orchestra.run.smithi037.stdout: librados-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:34.532 INFO:teuthology.orchestra.run.smithi037.stdout: libradosstriper1-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:34.533 INFO:teuthology.orchestra.run.smithi037.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T06:30:34.533 INFO:teuthology.orchestra.run.smithi037.stdout: librgw2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:34.533 INFO:teuthology.orchestra.run.smithi037.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T06:30:34.533 INFO:teuthology.orchestra.run.smithi037.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T06:30:34.533 INFO:teuthology.orchestra.run.smithi037.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T06:30:34.533 INFO:teuthology.orchestra.run.smithi037.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T06:30:34.533 INFO:teuthology.orchestra.run.smithi037.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T06:30:34.533 INFO:teuthology.orchestra.run.smithi037.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T06:30:34.533 INFO:teuthology.orchestra.run.smithi037.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T06:30:34.533 INFO:teuthology.orchestra.run.smithi037.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T06:30:34.533 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T06:30:34.533 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T06:30:34.533 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T06:30:34.533 INFO:teuthology.orchestra.run.smithi037.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T06:30:34.534 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T06:30:34.534 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T06:30:34.534 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T06:30:34.534 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T06:30:34.534 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T06:30:34.534 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T06:30:34.534 INFO:teuthology.orchestra.run.smithi037.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T06:30:34.534 INFO:teuthology.orchestra.run.smithi037.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T06:30:34.534 INFO:teuthology.orchestra.run.smithi037.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T06:30:34.534 INFO:teuthology.orchestra.run.smithi037.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T06:30:34.534 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T06:30:34.534 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-argparse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:34.534 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:34.535 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cephfs-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:34.535 INFO:teuthology.orchestra.run.smithi037.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T06:30:34.535 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T06:30:34.535 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T06:30:34.535 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T06:30:34.535 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T06:30:34.535 INFO:teuthology.orchestra.run.smithi037.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T06:30:34.535 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T06:30:34.535 INFO:teuthology.orchestra.run.smithi037.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T06:30:34.535 INFO:teuthology.orchestra.run.smithi037.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T06:30:34.535 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T06:30:34.535 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T06:30:34.535 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T06:30:34.536 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T06:30:34.536 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T06:30:34.536 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T06:30:34.536 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T06:30:34.536 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T06:30:34.536 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T06:30:34.536 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T06:30:34.536 INFO:teuthology.orchestra.run.smithi037.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T06:30:34.536 INFO:teuthology.orchestra.run.smithi037.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T06:30:34.536 INFO:teuthology.orchestra.run.smithi037.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T06:30:34.536 INFO:teuthology.orchestra.run.smithi037.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T06:30:34.536 INFO:teuthology.orchestra.run.smithi037.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T06:30:34.536 INFO:teuthology.orchestra.run.smithi037.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T06:30:34.536 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T06:30:34.537 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T06:30:34.537 INFO:teuthology.orchestra.run.smithi037.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T06:30:34.537 INFO:teuthology.orchestra.run.smithi037.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T06:30:34.537 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T06:30:34.537 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T06:30:34.537 INFO:teuthology.orchestra.run.smithi037.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T06:30:34.537 INFO:teuthology.orchestra.run.smithi037.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T06:30:34.537 INFO:teuthology.orchestra.run.smithi037.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T06:30:34.537 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T06:30:34.537 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T06:30:34.537 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T06:30:34.537 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T06:30:34.537 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T06:30:34.538 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T06:30:34.538 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rados-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:34.538 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:34.538 INFO:teuthology.orchestra.run.smithi037.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T06:30:34.538 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T06:30:34.538 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T06:30:34.538 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:34.538 INFO:teuthology.orchestra.run.smithi037.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T06:30:34.538 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T06:30:34.538 INFO:teuthology.orchestra.run.smithi037.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T06:30:34.538 INFO:teuthology.orchestra.run.smithi037.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T06:30:34.538 INFO:teuthology.orchestra.run.smithi037.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T06:30:34.539 INFO:teuthology.orchestra.run.smithi037.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T06:30:34.539 INFO:teuthology.orchestra.run.smithi037.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T06:30:34.539 INFO:teuthology.orchestra.run.smithi037.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T06:30:34.539 INFO:teuthology.orchestra.run.smithi037.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T06:30:34.539 INFO:teuthology.orchestra.run.smithi037.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T06:30:34.539 INFO:teuthology.orchestra.run.smithi037.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T06:30:34.539 INFO:teuthology.orchestra.run.smithi037.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T06:30:34.539 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T06:30:34.539 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T06:30:34.539 INFO:teuthology.orchestra.run.smithi037.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T06:30:34.539 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:34.539 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:34.539 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-nbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:30:34.539 INFO:teuthology.orchestra.run.smithi037.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T06:30:34.540 INFO:teuthology.orchestra.run.smithi037.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T06:30:34.540 INFO:teuthology.orchestra.run.smithi037.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T06:30:34.540 INFO:teuthology.orchestra.run.smithi037.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T06:30:34.540 INFO:teuthology.orchestra.run.smithi037.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T06:30:34.540 INFO:teuthology.orchestra.run.smithi037.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T06:30:34.540 INFO:teuthology.orchestra.run.smithi037.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T06:30:34.540 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-24T06:30:34.540 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-04-24T06:30:34.890 DEBUG:teuthology.parallel:result is None 2024-04-24T06:30:36.655 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 105/156 2024-04-24T06:30:36.700 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 105/156 2024-04-24T06:30:37.384 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 105/156 2024-04-24T06:30:37.429 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-base-2:19.0.0-3124.ga3c8a22b.el9.x86_64 106/156 2024-04-24T06:30:37.586 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-base-2:19.0.0-3124.ga3c8a22b.el9.x86_64 106/156 2024-04-24T06:30:37.586 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T06:30:37.586 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T06:30:37.586 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:30:37.638 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-selinux-2:19.0.0-3124.ga3c8a22b.el9.x86_64 107/156 2024-04-24T06:30:50.872 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3124.ga3c8a22b.el9.x86_64 107/156 2024-04-24T06:30:50.872 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /sys 2024-04-24T06:30:50.872 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /proc 2024-04-24T06:30:50.872 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /mnt 2024-04-24T06:30:50.873 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /var/tmp 2024-04-24T06:30:50.873 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /home 2024-04-24T06:30:50.873 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /root 2024-04-24T06:30:50.873 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /tmp 2024-04-24T06:30:50.873 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:30:51.118 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-volume-2:19.0.0-3124.ga3c8a22b.el9.noarch 108/156 2024-04-24T06:30:51.173 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-volume-2:19.0.0-3124.ga3c8a22b.el9.noarch 108/156 2024-04-24T06:30:51.173 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:30:51.173 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T06:30:51.173 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:30:52.861 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-osd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 109/156 2024-04-24T06:30:52.942 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-osd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 109/156 2024-04-24T06:30:52.942 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:30:52.942 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T06:30:52.943 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T06:30:52.943 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T06:30:52.943 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:30:53.211 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mds-2:19.0.0-3124.ga3c8a22b.el9.x86_64 110/156 2024-04-24T06:30:53.283 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mds-2:19.0.0-3124.ga3c8a22b.el9.x86_64 110/156 2024-04-24T06:30:53.283 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:30:53.283 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T06:30:53.283 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T06:30:53.283 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T06:30:53.283 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:30:53.795 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mon-2:19.0.0-3124.ga3c8a22b.el9.x86_64 111/156 2024-04-24T06:30:53.870 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mon-2:19.0.0-3124.ga3c8a22b.el9.x86_64 111/156 2024-04-24T06:30:53.870 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:30:53.870 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T06:30:53.870 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T06:30:53.871 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T06:30:53.871 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:30:54.005 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T06:30:54.068 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T06:30:54.144 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T06:30:54.399 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T06:30:54.459 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T06:30:54.549 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T06:30:54.668 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T06:30:54.842 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T06:30:54.891 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T06:30:54.931 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T06:30:54.965 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T06:30:55.637 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: cephadm-2:19.0.0-3124.ga3c8a22b.el9.noarch 123/156 2024-04-24T06:30:55.676 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephadm-2:19.0.0-3124.ga3c8a22b.el9.noarch 123/156 2024-04-24T06:30:55.734 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3124.ga3c8a22b.e 124/156 2024-04-24T06:30:55.846 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3124.ga3c8a22b. 125/156 2024-04-24T06:30:55.886 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3124.ga3c8a22b.el9.noa 126/156 2024-04-24T06:30:57.213 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3124.ga3c8a22b.el9.noa 126/156 2024-04-24T06:30:57.255 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3124.ga3c8a22b.el9.n 127/156 2024-04-24T06:30:58.419 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3124.ga3c8a22b.el9.n 127/156 2024-04-24T06:30:58.455 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3124.ga3c 128/156 2024-04-24T06:30:58.607 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3124.ga3c 128/156 2024-04-24T06:30:58.795 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3124.ga3c8a22b.el 129/156 2024-04-24T06:30:58.824 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-2:19.0.0-3124.ga3c8a22b.el9.x86_64 130/156 2024-04-24T06:30:58.897 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3124.ga3c8a22b.el9.x86_64 130/156 2024-04-24T06:30:58.897 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:30:58.897 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T06:30:58.898 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T06:30:58.898 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T06:30:58.898 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:30:58.949 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-rook-2:19.0.0-3124.ga3c8a22b.el9.noarch 131/156 2024-04-24T06:30:58.987 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3124.ga3c8a22b.el9.noarch 131/156 2024-04-24T06:30:59.118 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-2:19.0.0-3124.ga3c8a22b.el9.x86_64 132/156 2024-04-24T06:30:59.183 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 133/156 2024-04-24T06:31:00.249 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T06:31:00.285 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-radosgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 135/156 2024-04-24T06:31:00.356 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 135/156 2024-04-24T06:31:00.356 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:31:00.356 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T06:31:00.356 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T06:31:00.356 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T06:31:00.356 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:31:00.411 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3124.ga3c8a 136/156 2024-04-24T06:31:00.505 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3124.ga3c8a 136/156 2024-04-24T06:31:00.506 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:31:00.506 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T06:31:00.506 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:31:00.586 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 137/156 2024-04-24T06:31:00.668 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 137/156 2024-04-24T06:31:00.668 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:31:00.668 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T06:31:00.668 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T06:31:00.668 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T06:31:00.669 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:31:01.079 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 138/156 2024-04-24T06:31:01.148 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 138/156 2024-04-24T06:31:01.148 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T06:31:01.148 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T06:31:01.149 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T06:31:01.149 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T06:31:01.149 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:31:06.461 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-test-2:19.0.0-3124.ga3c8a22b.el9.x86_64 139/156 2024-04-24T06:31:06.517 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 140/156 2024-04-24T06:31:06.591 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-nbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 141/156 2024-04-24T06:31:06.637 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T06:31:06.685 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T06:31:07.030 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-shell-2:19.0.0-3124.ga3c8a22b.el9.x86_64 144/156 2024-04-24T06:31:07.081 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T06:31:07.109 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephfs-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_ 146/156 2024-04-24T06:31:08.133 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T06:31:08.182 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T06:31:08.238 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T06:31:09.638 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-top-2:19.0.0-3124.ga3c8a22b.el9.noarch 148/156 2024-04-24T06:31:09.745 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T06:31:09.832 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T06:31:10.003 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T06:31:10.044 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T06:31:10.044 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T06:31:10.196 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T06:31:10.251 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T06:31:10.252 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T06:31:10.399 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T06:31:10.537 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T06:31:10.840 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T06:31:10.840 INFO:teuthology.orchestra.run.smithi005.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-24T06:31:10.840 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:31:16.479 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-2:19.0.0-3124.ga3c8a22b.el9.x86_64 1/156 2024-04-24T06:31:16.479 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-base-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2/156 2024-04-24T06:31:16.479 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 3/156 2024-04-24T06:31:16.479 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 4/156 2024-04-24T06:31:16.479 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3124.ga3c8a 5/156 2024-04-24T06:31:16.480 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mds-2:19.0.0-3124.ga3c8a22b.el9.x86_64 6/156 2024-04-24T06:31:16.480 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-2:19.0.0-3124.ga3c8a22b.el9.x86_64 7/156 2024-04-24T06:31:16.480 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mon-2:19.0.0-3124.ga3c8a22b.el9.x86_64 8/156 2024-04-24T06:31:16.480 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-osd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 9/156 2024-04-24T06:31:16.480 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-radosgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 10/156 2024-04-24T06:31:16.480 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-selinux-2:19.0.0-3124.ga3c8a22b.el9.x86_64 11/156 2024-04-24T06:31:16.480 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-test-2:19.0.0-3124.ga3c8a22b.el9.x86_64 12/156 2024-04-24T06:31:16.480 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 13/156 2024-04-24T06:31:16.480 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-shell-2:19.0.0-3124.ga3c8a22b.el9.x86_64 14/156 2024-04-24T06:31:16.480 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephfs-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_ 15/156 2024-04-24T06:31:16.480 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephfs2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 16/156 2024-04-24T06:31:16.480 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephsqlite-2:19.0.0-3124.ga3c8a22b.el9.x86_64 17/156 2024-04-24T06:31:16.480 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_6 18/156 2024-04-24T06:31:16.481 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libradosstriper1-2:19.0.0-3124.ga3c8a22b.el9.x86 19/156 2024-04-24T06:31:16.481 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librgw2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 20/156 2024-04-24T06:31:16.481 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3124.ga3c8a22b.el 21/156 2024-04-24T06:31:16.481 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ceph-common-2:19.0.0-3124.ga3c8a22b.el9. 22/156 2024-04-24T06:31:16.481 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cephfs-2:19.0.0-3124.ga3c8a22b.el9.x86_6 23/156 2024-04-24T06:31:16.481 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rados-2:19.0.0-3124.ga3c8a22b.el9.x86_64 24/156 2024-04-24T06:31:16.481 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 25/156 2024-04-24T06:31:16.481 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 26/156 2024-04-24T06:31:16.481 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 27/156 2024-04-24T06:31:16.493 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 28/156 2024-04-24T06:31:16.493 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-nbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 29/156 2024-04-24T06:31:16.493 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3124.ga3c8a22b. 30/156 2024-04-24T06:31:16.493 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3124.ga3c8a22b.el9.noa 31/156 2024-04-24T06:31:16.494 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3124.ga3c8a22b.el9.n 32/156 2024-04-24T06:31:16.494 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3124.ga3c 33/156 2024-04-24T06:31:16.494 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3124.ga3c8a22b.el 34/156 2024-04-24T06:31:16.494 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3124.ga3c8a22b.el9.noarch 35/156 2024-04-24T06:31:16.494 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3124.ga3c8a22b.e 36/156 2024-04-24T06:31:16.494 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-volume-2:19.0.0-3124.ga3c8a22b.el9.noarch 37/156 2024-04-24T06:31:16.494 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephadm-2:19.0.0-3124.ga3c8a22b.el9.noarch 38/156 2024-04-24T06:31:16.494 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-top-2:19.0.0-3124.ga3c8a22b.el9.noarch 39/156 2024-04-24T06:31:16.494 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T06:31:16.494 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T06:31:16.494 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T06:31:16.494 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T06:31:16.495 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T06:31:16.495 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T06:31:16.495 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T06:31:16.495 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T06:31:16.495 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T06:31:16.495 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T06:31:16.495 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T06:31:16.495 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T06:31:16.495 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T06:31:16.495 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T06:31:16.495 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T06:31:16.495 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T06:31:16.495 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T06:31:16.495 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T06:31:16.496 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T06:31:16.496 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T06:31:16.496 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T06:31:16.496 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T06:31:16.496 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T06:31:16.496 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T06:31:16.496 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T06:31:16.496 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T06:31:16.496 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T06:31:16.496 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T06:31:16.496 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T06:31:16.496 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T06:31:16.496 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T06:31:16.497 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T06:31:16.497 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T06:31:16.497 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T06:31:16.497 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T06:31:16.497 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T06:31:16.497 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T06:31:16.497 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T06:31:16.497 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T06:31:16.497 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T06:31:16.497 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T06:31:16.497 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T06:31:16.497 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T06:31:16.497 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T06:31:16.497 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T06:31:16.498 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T06:31:16.498 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T06:31:16.498 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T06:31:16.498 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T06:31:16.498 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T06:31:16.498 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T06:31:16.498 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T06:31:16.498 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T06:31:16.498 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T06:31:16.498 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T06:31:16.498 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T06:31:16.498 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T06:31:16.498 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T06:31:16.498 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T06:31:16.499 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T06:31:16.499 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T06:31:16.499 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T06:31:16.499 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T06:31:16.499 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T06:31:16.499 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T06:31:16.499 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T06:31:16.499 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T06:31:16.499 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T06:31:16.499 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T06:31:16.499 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T06:31:16.499 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T06:31:16.499 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T06:31:16.499 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T06:31:16.500 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T06:31:16.500 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T06:31:16.500 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T06:31:16.500 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T06:31:16.500 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T06:31:16.500 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T06:31:16.500 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T06:31:16.500 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T06:31:16.500 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T06:31:16.500 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T06:31:16.500 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T06:31:16.500 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T06:31:16.500 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T06:31:16.500 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T06:31:16.501 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T06:31:16.501 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T06:31:16.501 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T06:31:16.501 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T06:31:16.501 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T06:31:16.501 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T06:31:16.501 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T06:31:16.501 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T06:31:16.501 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T06:31:16.501 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T06:31:16.501 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T06:31:16.501 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T06:31:16.501 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T06:31:16.502 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T06:31:16.502 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T06:31:16.502 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T06:31:16.502 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T06:31:16.502 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T06:31:16.502 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T06:31:16.502 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T06:31:16.502 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T06:31:16.502 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T06:31:16.502 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 149/156 2024-04-24T06:31:16.502 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T06:31:16.502 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librbd1-2:19.0.0-3124.ga3c8a22b.el9.x86_64 151/156 2024-04-24T06:31:16.502 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T06:31:16.502 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T06:31:16.503 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T06:31:16.503 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T06:31:18.882 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T06:31:18.883 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:31:18.883 INFO:teuthology.orchestra.run.smithi005.stdout:Upgraded: 2024-04-24T06:31:18.883 INFO:teuthology.orchestra.run.smithi005.stdout: librados2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:31:18.883 INFO:teuthology.orchestra.run.smithi005.stdout: librbd1-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:31:18.883 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T06:31:18.883 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T06:31:18.883 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2024-04-24T06:31:18.883 INFO:teuthology.orchestra.run.smithi005.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T06:31:18.883 INFO:teuthology.orchestra.run.smithi005.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T06:31:18.883 INFO:teuthology.orchestra.run.smithi005.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T06:31:18.884 INFO:teuthology.orchestra.run.smithi005.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T06:31:18.884 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:31:18.884 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-base-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:31:18.884 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:31:18.884 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:31:18.884 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-grafana-dashboards-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:31:18.884 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-immutable-object-cache-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:31:18.884 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mds-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:31:18.884 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:31:18.884 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-cephadm-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:31:18.884 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-dashboard-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:31:18.884 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:31:18.884 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-modules-core-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:31:18.885 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-rook-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:31:18.885 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mon-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:31:18.885 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-osd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:31:18.885 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-prometheus-alerts-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:31:18.885 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-radosgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:31:18.885 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-selinux-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:31:18.885 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-test-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:31:18.885 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-volume-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:31:18.885 INFO:teuthology.orchestra.run.smithi005.stdout: cephadm-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:31:18.885 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:31:18.885 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-shell-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:31:18.885 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-top-2:19.0.0-3124.ga3c8a22b.el9.noarch 2024-04-24T06:31:18.886 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T06:31:18.886 INFO:teuthology.orchestra.run.smithi005.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T06:31:18.886 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T06:31:18.886 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T06:31:18.886 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T06:31:18.886 INFO:teuthology.orchestra.run.smithi005.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T06:31:18.886 INFO:teuthology.orchestra.run.smithi005.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T06:31:18.886 INFO:teuthology.orchestra.run.smithi005.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T06:31:18.886 INFO:teuthology.orchestra.run.smithi005.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T06:31:18.886 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T06:31:18.886 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T06:31:18.886 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:31:18.886 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:31:18.887 INFO:teuthology.orchestra.run.smithi005.stdout: libcephsqlite-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:31:18.887 INFO:teuthology.orchestra.run.smithi005.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T06:31:18.887 INFO:teuthology.orchestra.run.smithi005.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T06:31:18.887 INFO:teuthology.orchestra.run.smithi005.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T06:31:18.887 INFO:teuthology.orchestra.run.smithi005.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T06:31:18.887 INFO:teuthology.orchestra.run.smithi005.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T06:31:18.887 INFO:teuthology.orchestra.run.smithi005.stdout: librados-devel-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:31:18.887 INFO:teuthology.orchestra.run.smithi005.stdout: libradosstriper1-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:31:18.887 INFO:teuthology.orchestra.run.smithi005.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T06:31:18.887 INFO:teuthology.orchestra.run.smithi005.stdout: librgw2-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:31:18.887 INFO:teuthology.orchestra.run.smithi005.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T06:31:18.887 INFO:teuthology.orchestra.run.smithi005.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T06:31:18.887 INFO:teuthology.orchestra.run.smithi005.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T06:31:18.888 INFO:teuthology.orchestra.run.smithi005.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T06:31:18.888 INFO:teuthology.orchestra.run.smithi005.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T06:31:18.888 INFO:teuthology.orchestra.run.smithi005.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T06:31:18.888 INFO:teuthology.orchestra.run.smithi005.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T06:31:18.888 INFO:teuthology.orchestra.run.smithi005.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T06:31:18.888 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T06:31:18.888 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T06:31:18.888 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T06:31:18.888 INFO:teuthology.orchestra.run.smithi005.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T06:31:18.888 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T06:31:18.888 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T06:31:18.888 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T06:31:18.889 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T06:31:18.889 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T06:31:18.889 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T06:31:18.889 INFO:teuthology.orchestra.run.smithi005.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T06:31:18.889 INFO:teuthology.orchestra.run.smithi005.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T06:31:18.889 INFO:teuthology.orchestra.run.smithi005.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T06:31:18.889 INFO:teuthology.orchestra.run.smithi005.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T06:31:18.889 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T06:31:18.889 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-argparse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:31:18.889 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-common-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:31:18.889 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cephfs-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:31:18.889 INFO:teuthology.orchestra.run.smithi005.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T06:31:18.889 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T06:31:18.889 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T06:31:18.890 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T06:31:18.890 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T06:31:18.890 INFO:teuthology.orchestra.run.smithi005.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T06:31:18.890 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T06:31:18.890 INFO:teuthology.orchestra.run.smithi005.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T06:31:18.890 INFO:teuthology.orchestra.run.smithi005.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T06:31:18.890 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T06:31:18.890 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T06:31:18.890 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T06:31:18.890 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T06:31:18.890 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T06:31:18.890 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T06:31:18.890 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T06:31:18.891 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T06:31:18.891 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T06:31:18.891 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T06:31:18.891 INFO:teuthology.orchestra.run.smithi005.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T06:31:18.891 INFO:teuthology.orchestra.run.smithi005.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T06:31:18.891 INFO:teuthology.orchestra.run.smithi005.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T06:31:18.891 INFO:teuthology.orchestra.run.smithi005.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T06:31:18.891 INFO:teuthology.orchestra.run.smithi005.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T06:31:18.891 INFO:teuthology.orchestra.run.smithi005.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T06:31:18.891 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T06:31:18.891 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T06:31:18.891 INFO:teuthology.orchestra.run.smithi005.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T06:31:18.891 INFO:teuthology.orchestra.run.smithi005.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T06:31:18.892 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T06:31:18.892 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T06:31:18.892 INFO:teuthology.orchestra.run.smithi005.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T06:31:18.892 INFO:teuthology.orchestra.run.smithi005.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T06:31:18.892 INFO:teuthology.orchestra.run.smithi005.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T06:31:18.892 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T06:31:18.892 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T06:31:18.892 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T06:31:18.892 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T06:31:18.892 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T06:31:18.892 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T06:31:18.892 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rados-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:31:18.893 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:31:18.893 INFO:teuthology.orchestra.run.smithi005.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T06:31:18.893 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T06:31:18.893 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T06:31:18.893 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rgw-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:31:18.893 INFO:teuthology.orchestra.run.smithi005.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T06:31:18.893 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T06:31:18.893 INFO:teuthology.orchestra.run.smithi005.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T06:31:18.893 INFO:teuthology.orchestra.run.smithi005.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T06:31:18.893 INFO:teuthology.orchestra.run.smithi005.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T06:31:18.893 INFO:teuthology.orchestra.run.smithi005.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T06:31:18.893 INFO:teuthology.orchestra.run.smithi005.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T06:31:18.894 INFO:teuthology.orchestra.run.smithi005.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T06:31:18.894 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T06:31:18.894 INFO:teuthology.orchestra.run.smithi005.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T06:31:18.894 INFO:teuthology.orchestra.run.smithi005.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T06:31:18.894 INFO:teuthology.orchestra.run.smithi005.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T06:31:18.894 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T06:31:18.894 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T06:31:18.894 INFO:teuthology.orchestra.run.smithi005.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T06:31:18.894 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-fuse-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:31:18.894 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-mirror-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:31:18.894 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-nbd-2:19.0.0-3124.ga3c8a22b.el9.x86_64 2024-04-24T06:31:18.894 INFO:teuthology.orchestra.run.smithi005.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T06:31:18.894 INFO:teuthology.orchestra.run.smithi005.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T06:31:18.895 INFO:teuthology.orchestra.run.smithi005.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T06:31:18.895 INFO:teuthology.orchestra.run.smithi005.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T06:31:18.895 INFO:teuthology.orchestra.run.smithi005.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T06:31:18.895 INFO:teuthology.orchestra.run.smithi005.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T06:31:18.895 INFO:teuthology.orchestra.run.smithi005.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T06:31:18.895 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:31:18.895 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-04-24T06:31:19.429 DEBUG:teuthology.parallel:result is None 2024-04-24T06:31:19.430 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a3c8a22bd5efe18ab230a8e2d072db8f381a0411 2024-04-24T06:31:19.579 DEBUG:teuthology.orchestra.run.smithi005:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T06:31:19.634 INFO:teuthology.orchestra.run.smithi005.stdout:19.0.0-3124.ga3c8a22b.el9 2024-04-24T06:31:19.634 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3124.ga3c8a22b.el9 2024-04-24T06:31:19.634 INFO:teuthology.task.install:The correct ceph version 19.0.0-3124.ga3c8a22b is installed. 2024-04-24T06:31:19.636 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a3c8a22bd5efe18ab230a8e2d072db8f381a0411 2024-04-24T06:31:19.863 DEBUG:teuthology.orchestra.run.smithi037:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T06:31:19.890 INFO:teuthology.orchestra.run.smithi037.stdout:19.0.0-3124.ga3c8a22b.el9 2024-04-24T06:31:19.890 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3124.ga3c8a22b.el9 2024-04-24T06:31:19.890 INFO:teuthology.task.install:The correct ceph version 19.0.0-3124.ga3c8a22b is installed. 2024-04-24T06:31:19.892 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a3c8a22bd5efe18ab230a8e2d072db8f381a0411 2024-04-24T06:31:20.035 DEBUG:teuthology.orchestra.run.smithi137:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T06:31:20.061 INFO:teuthology.orchestra.run.smithi137.stdout:19.0.0-3124.ga3c8a22b.el9 2024-04-24T06:31:20.061 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3124.ga3c8a22b.el9 2024-04-24T06:31:20.061 INFO:teuthology.task.install:The correct ceph version 19.0.0-3124.ga3c8a22b is installed. 2024-04-24T06:31:20.064 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-24T06:31:20.064 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-24T06:31:20.064 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T06:31:20.138 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-24T06:31:20.138 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T06:31:20.170 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-24T06:31:20.170 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T06:31:20.200 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-24T06:31:20.200 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-24T06:31:20.200 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T06:31:20.287 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T06:31:20.407 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-24T06:31:20.407 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T06:31:20.432 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T06:31:20.499 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-24T06:31:20.499 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T06:31:20.526 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T06:31:20.592 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-24T06:31:20.592 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-24T06:31:20.592 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T06:31:20.664 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T06:31:20.774 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-24T06:31:20.774 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T06:31:20.800 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T06:31:20.864 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-24T06:31:20.865 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T06:31:20.909 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T06:31:20.973 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-24T06:31:20.973 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-24T06:31:20.973 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T06:31:21.049 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T06:31:21.154 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-24T06:31:21.154 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T06:31:21.179 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T06:31:21.243 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-24T06:31:21.243 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T06:31:21.266 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T06:31:21.330 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-24T06:31:21.428 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'a3c8a22bd5efe18ab230a8e2d072db8f381a0411', 'subvols': {'create': 2}} 2024-04-24T06:31:21.428 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 2024-04-24T06:31:21.429 INFO:tasks.cephadm:Cluster fsid is 43e9bad0-0204-11ef-bc93-c7b262605968 2024-04-24T06:31:21.429 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-24T06:31:21.429 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.5', 'mon.b': '172.21.15.37', 'mon.c': '172.21.15.137'} 2024-04-24T06:31:21.429 INFO:tasks.cephadm:First mon is mon.a on smithi005 2024-04-24T06:31:21.429 INFO:tasks.cephadm:First mgr is x 2024-04-24T06:31:21.429 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-24T06:31:21.430 DEBUG:teuthology.orchestra.run.smithi005:> sudo hostname $(hostname -s) 2024-04-24T06:31:21.506 DEBUG:teuthology.orchestra.run.smithi037:> sudo hostname $(hostname -s) 2024-04-24T06:31:21.531 DEBUG:teuthology.orchestra.run.smithi137:> sudo hostname $(hostname -s) 2024-04-24T06:31:21.554 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-24T06:31:21.555 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a3c8a22bd5efe18ab230a8e2d072db8f381a0411 2024-04-24T06:31:21.696 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'a3c8a22bd5efe18ab230a8e2d072db8f381a0411', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78824/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3124-ga3c8a22b', 'node_name': '172.21.5.34+adami04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3124.ga3c8a22b'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-5/a3c8a22bd5efe18ab230a8e2d072db8f381a0411/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-23 20:15:08.469134', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240416.193735-5', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240416.193735-5/a3c8a22bd5efe18ab230a8e2d072db8f381a0411/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-04-24T06:31:21.836 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-rishabh-testing-20240416.193735-5, sha1 a3c8a22bd5efe18ab230a8e2d072db8f381a0411 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=a3c8a22bd5efe18ab230a8e2d072db8f381a0411 2024-04-24T06:31:21.838 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240416.193735-5/a3c8a22bd5efe18ab230a8e2d072db8f381a0411/centos/9/x86_64/flavors/default/cephadm 2024-04-24T06:31:21.838 DEBUG:teuthology.orchestra.run.smithi005:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240416.193735-5/a3c8a22bd5efe18ab230a8e2d072db8f381a0411/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-24T06:31:22.548 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-r--r--. 1 ubuntu ubuntu 781578 Apr 24 06:31 /home/ubuntu/cephtest/cephadm 2024-04-24T06:31:22.548 DEBUG:teuthology.orchestra.run.smithi037:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240416.193735-5/a3c8a22bd5efe18ab230a8e2d072db8f381a0411/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-24T06:31:22.983 INFO:teuthology.orchestra.run.smithi037.stdout:-rw-r--r--. 1 ubuntu ubuntu 781578 Apr 24 06:31 /home/ubuntu/cephtest/cephadm 2024-04-24T06:31:22.983 DEBUG:teuthology.orchestra.run.smithi137:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240416.193735-5/a3c8a22bd5efe18ab230a8e2d072db8f381a0411/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-24T06:31:23.480 INFO:teuthology.orchestra.run.smithi137.stdout:-rw-r--r--. 1 ubuntu ubuntu 781578 Apr 24 06:31 /home/ubuntu/cephtest/cephadm 2024-04-24T06:31:23.481 DEBUG:teuthology.orchestra.run.smithi005:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-24T06:31:23.528 DEBUG:teuthology.orchestra.run.smithi037:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-24T06:31:23.548 DEBUG:teuthology.orchestra.run.smithi137:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-24T06:31:23.584 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 on all hosts... 2024-04-24T06:31:23.584 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 pull 2024-04-24T06:31:23.590 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 pull 2024-04-24T06:31:23.593 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 pull 2024-04-24T06:31:25.064 INFO:teuthology.orchestra.run.smithi037.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411... 2024-04-24T06:31:25.162 INFO:teuthology.orchestra.run.smithi137.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411... 2024-04-24T06:31:25.804 INFO:teuthology.orchestra.run.smithi005.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411... 2024-04-24T06:32:01.734 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-24T06:32:01.735 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_version": "ceph version 19.0.0-3124-ga3c8a22b (a3c8a22bd5efe18ab230a8e2d072db8f381a0411) squid (dev)", 2024-04-24T06:32:01.735 INFO:teuthology.orchestra.run.smithi037.stdout: "image_id": "b5c3183062a6ce19f361681cc58de6e3ac1de1a54c80e6094ec5695237cd98d2", 2024-04-24T06:32:01.735 INFO:teuthology.orchestra.run.smithi037.stdout: "repo_digests": [ 2024-04-24T06:32:01.735 INFO:teuthology.orchestra.run.smithi037.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b" 2024-04-24T06:32:01.735 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-24T06:32:01.735 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-24T06:32:05.395 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-24T06:32:05.395 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_version": "ceph version 19.0.0-3124-ga3c8a22b (a3c8a22bd5efe18ab230a8e2d072db8f381a0411) squid (dev)", 2024-04-24T06:32:05.395 INFO:teuthology.orchestra.run.smithi137.stdout: "image_id": "b5c3183062a6ce19f361681cc58de6e3ac1de1a54c80e6094ec5695237cd98d2", 2024-04-24T06:32:05.395 INFO:teuthology.orchestra.run.smithi137.stdout: "repo_digests": [ 2024-04-24T06:32:05.395 INFO:teuthology.orchestra.run.smithi137.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b" 2024-04-24T06:32:05.395 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-24T06:32:05.395 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-24T06:32:14.161 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:32:14.161 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_version": "ceph version 19.0.0-3124-ga3c8a22b (a3c8a22bd5efe18ab230a8e2d072db8f381a0411) squid (dev)", 2024-04-24T06:32:14.161 INFO:teuthology.orchestra.run.smithi005.stdout: "image_id": "b5c3183062a6ce19f361681cc58de6e3ac1de1a54c80e6094ec5695237cd98d2", 2024-04-24T06:32:14.161 INFO:teuthology.orchestra.run.smithi005.stdout: "repo_digests": [ 2024-04-24T06:32:14.161 INFO:teuthology.orchestra.run.smithi005.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b" 2024-04-24T06:32:14.162 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-04-24T06:32:14.162 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:32:14.197 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /etc/ceph 2024-04-24T06:32:14.295 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /etc/ceph 2024-04-24T06:32:14.328 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /etc/ceph 2024-04-24T06:32:14.359 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 777 /etc/ceph 2024-04-24T06:32:14.437 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 777 /etc/ceph 2024-04-24T06:32:14.467 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 777 /etc/ceph 2024-04-24T06:32:14.494 INFO:tasks.cephadm:Writing seed config... 2024-04-24T06:32:14.495 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-24T06:32:14.495 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-24T06:32:14.495 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-24T06:32:14.495 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-24T06:32:14.495 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-24T06:32:14.496 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-24T06:32:14.496 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-24T06:32:14.496 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-24T06:32:14.496 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-24T06:32:14.496 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-24T06:32:14.496 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-24T06:32:14.496 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-24T06:32:14.496 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-24T06:32:14.496 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-24T06:32:14.496 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-24T06:32:14.496 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-24T06:32:14.496 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-24T06:32:14.496 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-24T06:32:14.496 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-24T06:32:14.497 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-24T06:32:14.497 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-24T06:32:14.497 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-24T06:32:14.497 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-24T06:32:14.497 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-24T06:32:14.497 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-24T06:32:14.497 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-24T06:32:14.497 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-24T06:32:14.497 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-24T06:32:14.497 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-24T06:32:14.497 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-24T06:32:14.497 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-24T06:32:14.497 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-24T06:32:14.497 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-24T06:32:14.497 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-24T06:32:14.498 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-24T06:32:14.498 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-24T06:32:14.498 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-24T06:32:14.498 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-24T06:32:14.498 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-24T06:32:14.498 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-24T06:32:14.498 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-24T06:32:14.498 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-24T06:32:14.498 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-24T06:32:14.498 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-24T06:32:14.498 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-24T06:32:14.498 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-24T06:32:14.498 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-24T06:32:14.498 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-24T06:32:14.499 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-24T06:32:14.499 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-24T06:32:14.499 DEBUG:teuthology.orchestra.run.smithi005:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-24T06:32:14.549 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 43e9bad0-0204-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-24T06:32:14.550 DEBUG:teuthology.orchestra.run.smithi005:mon.a> sudo journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mon.a.service 2024-04-24T06:32:14.597 DEBUG:teuthology.orchestra.run.smithi005:mgr.x> sudo journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mgr.x.service 2024-04-24T06:32:14.642 INFO:tasks.cephadm:Bootstrapping... 2024-04-24T06:32:14.642 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 -v bootstrap --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.5 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-24T06:32:14.921 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-04-24T06:32:14.921 INFO:teuthology.orchestra.run.smithi005.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411', '-v', 'bootstrap', '--fsid', '43e9bad0-0204-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.5', '--skip-admin-label'] 2024-04-24T06:32:14.921 INFO:teuthology.orchestra.run.smithi005.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-24T06:32:14.921 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying podman|docker is present... 2024-04-24T06:32:14.980 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-24T06:32:14.981 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying lvm2 is present... 2024-04-24T06:32:14.981 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying time synchronization is in place... 2024-04-24T06:32:14.997 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-24T06:32:14.997 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-24T06:32:15.011 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-24T06:32:15.011 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-04-24T06:32:15.028 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout enabled 2024-04-24T06:32:15.043 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout active 2024-04-24T06:32:15.043 INFO:teuthology.orchestra.run.smithi005.stdout:Unit chronyd.service is enabled and running 2024-04-24T06:32:15.043 INFO:teuthology.orchestra.run.smithi005.stdout:Repeating the final host check... 2024-04-24T06:32:15.101 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-24T06:32:15.101 INFO:teuthology.orchestra.run.smithi005.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-24T06:32:15.101 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl is present 2024-04-24T06:32:15.101 INFO:teuthology.orchestra.run.smithi005.stdout:lvcreate is present 2024-04-24T06:32:15.117 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-24T06:32:15.117 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-24T06:32:15.131 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-24T06:32:15.131 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-04-24T06:32:15.149 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout enabled 2024-04-24T06:32:15.171 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout active 2024-04-24T06:32:15.171 INFO:teuthology.orchestra.run.smithi005.stdout:Unit chronyd.service is enabled and running 2024-04-24T06:32:15.171 INFO:teuthology.orchestra.run.smithi005.stdout:Host looks OK 2024-04-24T06:32:15.171 INFO:teuthology.orchestra.run.smithi005.stdout:Cluster fsid: 43e9bad0-0204-11ef-bc93-c7b262605968 2024-04-24T06:32:15.172 INFO:teuthology.orchestra.run.smithi005.stdout:Acquiring lock 139937914207392 on /run/cephadm/43e9bad0-0204-11ef-bc93-c7b262605968.lock 2024-04-24T06:32:15.172 INFO:teuthology.orchestra.run.smithi005.stdout:Lock 139937914207392 acquired on /run/cephadm/43e9bad0-0204-11ef-bc93-c7b262605968.lock 2024-04-24T06:32:15.173 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying IP 172.21.15.5 port 3300 ... 2024-04-24T06:32:15.174 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying IP 172.21.15.5 port 6789 ... 2024-04-24T06:32:15.175 INFO:teuthology.orchestra.run.smithi005.stdout:Base mon IP(s) is [172.21.15.5:3300, 172.21.15.5:6789], mon addrv is [v2:172.21.15.5:3300,v1:172.21.15.5:6789] 2024-04-24T06:32:15.189 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.5 metric 100 2024-04-24T06:32:15.189 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.5 metric 100 2024-04-24T06:32:15.201 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-24T06:32:15.202 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-24T06:32:15.202 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-24T06:32:15.213 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-24T06:32:15.214 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-24T06:32:15.214 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-24T06:32:15.214 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-24T06:32:15.214 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-04-24T06:32:15.214 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-24T06:32:15.216 INFO:teuthology.orchestra.run.smithi005.stdout:Mon IP `172.21.15.5` is in CIDR network `172.21.0.0/20` 2024-04-24T06:32:15.216 INFO:teuthology.orchestra.run.smithi005.stdout:Mon IP `172.21.15.5` is in CIDR network `172.21.0.0/20` 2024-04-24T06:32:15.217 INFO:teuthology.orchestra.run.smithi005.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-24T06:32:15.220 INFO:teuthology.orchestra.run.smithi005.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-24T06:32:15.223 INFO:teuthology.orchestra.run.smithi005.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411... 2024-04-24T06:32:15.892 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stdout b5c3183062a6ce19f361681cc58de6e3ac1de1a54c80e6094ec5695237cd98d2 2024-04-24T06:32:15.892 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411... 2024-04-24T06:32:15.892 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Getting image source signatures 2024-04-24T06:32:15.892 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Copying blob sha256:7f0c7a46136f9bf8edb253eefd757e0b4d2d4241c1e983a048ea26e3350fec1e 2024-04-24T06:32:15.892 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-24T06:32:15.892 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Copying config sha256:b5c3183062a6ce19f361681cc58de6e3ac1de1a54c80e6094ec5695237cd98d2 2024-04-24T06:32:15.892 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-24T06:32:17.131 INFO:teuthology.orchestra.run.smithi005.stdout:ceph: stdout ceph version 19.0.0-3124-ga3c8a22b (a3c8a22bd5efe18ab230a8e2d072db8f381a0411) squid (dev) 2024-04-24T06:32:17.131 INFO:teuthology.orchestra.run.smithi005.stdout:Ceph version: ceph version 19.0.0-3124-ga3c8a22b (a3c8a22bd5efe18ab230a8e2d072db8f381a0411) squid (dev) 2024-04-24T06:32:17.131 INFO:teuthology.orchestra.run.smithi005.stdout:Extracting ceph user uid/gid from container image... 2024-04-24T06:32:18.240 INFO:teuthology.orchestra.run.smithi005.stdout:stat: stdout 167 167 2024-04-24T06:32:18.240 INFO:teuthology.orchestra.run.smithi005.stdout:Creating initial keys... 2024-04-24T06:32:19.393 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph-authtool: stdout AQBypyhmpCOxKxAAGVXVELPk07DqK7muAmR0JA== 2024-04-24T06:32:20.529 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph-authtool: stdout AQBzpyhmeNZrNhAAbZgKdUih2/SwJpjk7QLdGA== 2024-04-24T06:32:21.663 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph-authtool: stdout AQB0pyhmgR2AOhAAzPAwzjnm7j7Z20Rz2X3IUg== 2024-04-24T06:32:21.666 INFO:teuthology.orchestra.run.smithi005.stdout:Creating initial monmap... 2024-04-24T06:32:22.811 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-24T06:32:22.811 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-24T06:32:22.811 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 43e9bad0-0204-11ef-bc93-c7b262605968 2024-04-24T06:32:22.811 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-24T06:32:22.811 INFO:teuthology.orchestra.run.smithi005.stdout:monmaptool for a [v2:172.21.15.5:3300,v1:172.21.15.5:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-24T06:32:22.811 INFO:teuthology.orchestra.run.smithi005.stdout:setting min_mon_release = quincy 2024-04-24T06:32:22.811 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: set fsid to 43e9bad0-0204-11ef-bc93-c7b262605968 2024-04-24T06:32:22.811 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-24T06:32:22.812 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:32:22.812 INFO:teuthology.orchestra.run.smithi005.stdout:Creating mon... 2024-04-24T06:32:23.911 INFO:teuthology.orchestra.run.smithi005.stdout:create mon.a on 2024-04-24T06:32:24.321 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-24T06:32:24.570 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-24T06:32:24.867 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-43e9bad0-0204-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-43e9bad0-0204-11ef-bc93-c7b262605968.target. 2024-04-24T06:32:24.867 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-43e9bad0-0204-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-43e9bad0-0204-11ef-bc93-c7b262605968.target. 2024-04-24T06:32:25.248 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mon.a 2024-04-24T06:32:25.249 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to reset failed state of unit ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-24T06:32:25.541 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-43e9bad0-0204-11ef-bc93-c7b262605968.target.wants/ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-43e9bad0-0204-11ef-bc93-c7b262605968@.service. 2024-04-24T06:32:26.249 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:25 smithi005 podman[27410]: 2024-04-24 06:32:25.966604352 +0000 UTC m=+0.132737733 container create 0202765fff9bc094a11d236cbb2b455b2f8ddf0f88c637f0ef822ca69b1c6744 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0) 2024-04-24T06:32:26.249 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:26 smithi005 podman[27410]: 2024-04-24 06:32:26.245520682 +0000 UTC m=+0.411654067 container init 0202765fff9bc094a11d236cbb2b455b2f8ddf0f88c637f0ef822ca69b1c6744 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mon-a, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-24T06:32:26.347 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T06:32:26.347 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2024-04-24T06:32:26.362 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T06:32:26.362 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-04-24T06:32:26.362 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2024-04-24T06:32:26.362 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to enable service . firewalld.service is not available 2024-04-24T06:32:26.363 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mon to start... 2024-04-24T06:32:26.363 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mon... 2024-04-24T06:32:26.505 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:26 smithi005 podman[27410]: 2024-04-24 06:32:26.256864667 +0000 UTC m=+0.422998039 container start 0202765fff9bc094a11d236cbb2b455b2f8ddf0f88c637f0ef822ca69b1c6744 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T06:32:26.505 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:26 smithi005 bash[27410]: 0202765fff9bc094a11d236cbb2b455b2f8ddf0f88c637f0ef822ca69b1c6744 2024-04-24T06:32:26.505 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:26 smithi005 systemd[1]: Started Ceph mon.a for 43e9bad0-0204-11ef-bc93-c7b262605968. 2024-04-24T06:32:26.505 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:26 smithi005 ceph-mon[27428]: mkfs 43e9bad0-0204-11ef-bc93-c7b262605968 2024-04-24T06:32:26.505 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:26 smithi005 ceph-mon[27428]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-24T06:32:27.622 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:27 smithi005 ceph-mon[27428]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-24T06:32:27.622 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:27 smithi005 ceph-mon[27428]: monmap epoch 1 2024-04-24T06:32:27.622 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:27 smithi005 ceph-mon[27428]: fsid 43e9bad0-0204-11ef-bc93-c7b262605968 2024-04-24T06:32:27.622 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:27 smithi005 ceph-mon[27428]: last_changed 2024-04-24T06:32:22.137390+0000 2024-04-24T06:32:27.622 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:27 smithi005 ceph-mon[27428]: created 2024-04-24T06:32:22.137390+0000 2024-04-24T06:32:27.622 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:27 smithi005 ceph-mon[27428]: min_mon_release 19 (squid) 2024-04-24T06:32:27.622 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:27 smithi005 ceph-mon[27428]: election_strategy: 1 2024-04-24T06:32:27.622 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:27 smithi005 ceph-mon[27428]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-04-24T06:32:27.623 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:27 smithi005 ceph-mon[27428]: fsmap 2024-04-24T06:32:27.623 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:27 smithi005 ceph-mon[27428]: osdmap e1: 0 total, 0 up, 0 in 2024-04-24T06:32:27.623 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:27 smithi005 ceph-mon[27428]: mgrmap e1: no daemons active 2024-04-24T06:32:27.623 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:27 smithi005 ceph-mon[27428]: from='client.? 172.21.15.5:0/3973436706' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-24T06:32:27.637 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout cluster: 2024-04-24T06:32:27.638 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout id: 43e9bad0-0204-11ef-bc93-c7b262605968 2024-04-24T06:32:27.638 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-24T06:32:27.638 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-04-24T06:32:27.638 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout services: 2024-04-24T06:32:27.639 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.728987s) 2024-04-24T06:32:27.639 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-24T06:32:27.639 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-24T06:32:27.641 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-04-24T06:32:27.641 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout data: 2024-04-24T06:32:27.641 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-24T06:32:27.641 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-24T06:32:27.642 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-24T06:32:27.642 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout pgs: 2024-04-24T06:32:27.642 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-04-24T06:32:27.642 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:26.971+0000 7f2226600700 1 Processor -- start 2024-04-24T06:32:27.642 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:26.971+0000 7f2226600700 1 -- start start 2024-04-24T06:32:27.642 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:26.972+0000 7f2226600700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f222010a910 0x7f222010ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:27.642 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:26.972+0000 7f2226600700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f222010b300 con 0x7f222010a910 2024-04-24T06:32:27.642 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:26.973+0000 7f221f400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f222010a910 0x7f222010ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:27.642 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:26.973+0000 7f221f400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f222010a910 0x7f222010ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35558/0 (socket says 172.21.15.5:35558) 2024-04-24T06:32:27.642 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:26.973+0000 7f221f400700 1 -- 172.21.15.5:0/42993680 learned_addr learned my addr 172.21.15.5:0/42993680 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:32:27.642 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:26.974+0000 7f221f400700 1 -- 172.21.15.5:0/42993680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f222010bac0 con 0x7f222010a910 2024-04-24T06:32:27.642 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:26.975+0000 7f221f400700 1 --2- 172.21.15.5:0/42993680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f222010a910 0x7f222010ad30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f22100098a0 tx=0x7f2210009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88bd81ea8044e517 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:27.643 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:26.975+0000 7f221e000700 1 -- 172.21.15.5:0/42993680 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2210004160 con 0x7f222010a910 2024-04-24T06:32:27.643 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:26.975+0000 7f221e000700 1 -- 172.21.15.5:0/42993680 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2210004300 con 0x7f222010a910 2024-04-24T06:32:27.643 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:26.975+0000 7f221e000700 1 -- 172.21.15.5:0/42993680 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22100036a0 con 0x7f222010a910 2024-04-24T06:32:27.643 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:26.976+0000 7f2226600700 1 -- 172.21.15.5:0/42993680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f222010a910 msgr2=0x7f222010ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:27.643 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:26.976+0000 7f2226600700 1 --2- 172.21.15.5:0/42993680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f222010a910 0x7f222010ad30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f22100098a0 tx=0x7f2210009bb0 comp rx=0 tx=0).stop 2024-04-24T06:32:27.647 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:26.976+0000 7f2226600700 1 -- 172.21.15.5:0/42993680 shutdown_connections 2024-04-24T06:32:27.647 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:26.976+0000 7f2226600700 1 --2- 172.21.15.5:0/42993680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f222010a910 0x7f222010ad30 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:27.647 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:26.976+0000 7f2226600700 1 -- 172.21.15.5:0/42993680 >> 172.21.15.5:0/42993680 conn(0x7f222007de90 msgr2=0x7f22201084f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:27.647 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:26.976+0000 7f2226600700 1 -- 172.21.15.5:0/42993680 shutdown_connections 2024-04-24T06:32:27.647 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:26.976+0000 7f2226600700 1 -- 172.21.15.5:0/42993680 wait complete. 2024-04-24T06:32:27.647 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:26.977+0000 7f2226600700 1 Processor -- start 2024-04-24T06:32:27.647 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:26.977+0000 7f2226600700 1 -- start start 2024-04-24T06:32:27.647 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:26.978+0000 7f2226600700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f222010a910 0x7f222019dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:27.647 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:26.978+0000 7f2226600700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f222010bff0 con 0x7f222010a910 2024-04-24T06:32:27.647 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:26.978+0000 7f221f400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f222010a910 0x7f222019dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:27.647 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:26.978+0000 7f221f400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f222010a910 0x7f222019dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35562/0 (socket says 172.21.15.5:35562) 2024-04-24T06:32:27.648 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:26.978+0000 7f221f400700 1 -- 172.21.15.5:0/3973436706 learned_addr learned my addr 172.21.15.5:0/3973436706 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:32:27.648 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:26.979+0000 7f221f400700 1 -- 172.21.15.5:0/3973436706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f222019e520 con 0x7f222010a910 2024-04-24T06:32:27.648 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:26.979+0000 7f221f400700 1 --2- 172.21.15.5:0/3973436706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f222010a910 0x7f222019dfe0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2210000c00 tx=0x7f2210004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:27.648 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:26.979+0000 7f2217e00700 1 -- 172.21.15.5:0/3973436706 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2210003db0 con 0x7f222010a910 2024-04-24T06:32:27.648 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:26.979+0000 7f2217e00700 1 -- 172.21.15.5:0/3973436706 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2210011630 con 0x7f222010a910 2024-04-24T06:32:27.648 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:26.979+0000 7f2226600700 1 -- 172.21.15.5:0/3973436706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f222019e7b0 con 0x7f222010a910 2024-04-24T06:32:27.648 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:26.979+0000 7f2217e00700 1 -- 172.21.15.5:0/3973436706 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2210011950 con 0x7f222010a910 2024-04-24T06:32:27.648 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:26.979+0000 7f2226600700 1 -- 172.21.15.5:0/3973436706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22201a14a0 con 0x7f222010a910 2024-04-24T06:32:27.648 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:26.980+0000 7f2217e00700 1 -- 172.21.15.5:0/3973436706 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f221001a460 con 0x7f222010a910 2024-04-24T06:32:27.648 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:26.980+0000 7f2217e00700 1 -- 172.21.15.5:0/3973436706 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f221001b780 con 0x7f222010a910 2024-04-24T06:32:27.648 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:26.981+0000 7f2226600700 1 -- 172.21.15.5:0/3973436706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2220064150 con 0x7f222010a910 2024-04-24T06:32:27.648 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:26.984+0000 7f2217e00700 1 -- 172.21.15.5:0/3973436706 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f2210031020 con 0x7f222010a910 2024-04-24T06:32:27.648 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:27.067+0000 7f2226600700 1 -- 172.21.15.5:0/3973436706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f222002e230 con 0x7f222010a910 2024-04-24T06:32:27.648 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:27.067+0000 7f2217e00700 1 -- 172.21.15.5:0/3973436706 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f221001b9d0 con 0x7f222010a910 2024-04-24T06:32:27.649 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:27.069+0000 7f2226600700 1 -- 172.21.15.5:0/3973436706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f222010a910 msgr2=0x7f222019dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:27.649 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:27.069+0000 7f2226600700 1 --2- 172.21.15.5:0/3973436706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f222010a910 0x7f222019dfe0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2210000c00 tx=0x7f2210004750 comp rx=0 tx=0).stop 2024-04-24T06:32:27.649 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:27.069+0000 7f2226600700 1 -- 172.21.15.5:0/3973436706 shutdown_connections 2024-04-24T06:32:27.649 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:27.069+0000 7f2226600700 1 --2- 172.21.15.5:0/3973436706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f222010a910 0x7f222019dfe0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:27.649 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:27.069+0000 7f2226600700 1 -- 172.21.15.5:0/3973436706 >> 172.21.15.5:0/3973436706 conn(0x7f222007de90 msgr2=0x7f2220194c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:27.649 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:27.069+0000 7f2226600700 1 -- 172.21.15.5:0/3973436706 shutdown_connections 2024-04-24T06:32:27.649 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:27.069+0000 7f2226600700 1 -- 172.21.15.5:0/3973436706 wait complete. 2024-04-24T06:32:27.649 INFO:teuthology.orchestra.run.smithi005.stdout:mon is available 2024-04-24T06:32:27.649 INFO:teuthology.orchestra.run.smithi005.stdout:Assimilating anything we can from ceph.conf... 2024-04-24T06:32:28.683 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:28 smithi005 ceph-mon[27428]: from='client.? 172.21.15.5:0/1403621580' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-24T06:32:28.683 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:28 smithi005 ceph-mon[27428]: from='client.? 172.21.15.5:0/1403621580' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-24T06:32:28.971 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-04-24T06:32:28.972 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [global] 2024-04-24T06:32:28.972 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout fsid = 43e9bad0-0204-11ef-bc93-c7b262605968 2024-04-24T06:32:28.972 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.5:3300,v1:172.21.15.5:6789] 2024-04-24T06:32:28.972 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-24T06:32:28.972 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-24T06:32:28.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-24T06:32:28.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-24T06:32:28.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-04-24T06:32:28.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-24T06:32:28.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-24T06:32:28.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-04-24T06:32:28.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [osd] 2024-04-24T06:32:28.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-24T06:32:28.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-24T06:32:28.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-24T06:32:28.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-24T06:32:28.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-24T06:32:28.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-24T06:32:28.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.265+0000 7f7916000700 1 Processor -- start 2024-04-24T06:32:28.975 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.266+0000 7f7916000700 1 -- start start 2024-04-24T06:32:28.978 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.266+0000 7f7916000700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f791010f030 0x7f791010f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:28.978 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.266+0000 7f7916000700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f791010f990 con 0x7f791010f030 2024-04-24T06:32:28.978 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.267+0000 7f790f400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f791010f030 0x7f791010f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:28.978 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.267+0000 7f790f400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f791010f030 0x7f791010f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35574/0 (socket says 172.21.15.5:35574) 2024-04-24T06:32:28.978 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.267+0000 7f790f400700 1 -- 172.21.15.5:0/313977922 learned_addr learned my addr 172.21.15.5:0/313977922 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:32:28.978 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.267+0000 7f790f400700 1 -- 172.21.15.5:0/313977922 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f791010fb10 con 0x7f791010f030 2024-04-24T06:32:28.978 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.268+0000 7f790f400700 1 --2- 172.21.15.5:0/313977922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f791010f030 0x7f791010f450 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7900009e80 tx=0x7f790000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1206c524ad150d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:28.978 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.268+0000 7f790ea00700 1 -- 172.21.15.5:0/313977922 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7900017070 con 0x7f791010f030 2024-04-24T06:32:28.978 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.268+0000 7f790ea00700 1 -- 172.21.15.5:0/313977922 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7900003a20 con 0x7f791010f030 2024-04-24T06:32:28.979 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.268+0000 7f790ea00700 1 -- 172.21.15.5:0/313977922 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7900003d20 con 0x7f791010f030 2024-04-24T06:32:28.979 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.269+0000 7f7916000700 1 -- 172.21.15.5:0/313977922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f791010f030 msgr2=0x7f791010f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:28.979 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.269+0000 7f7916000700 1 --2- 172.21.15.5:0/313977922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f791010f030 0x7f791010f450 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7900009e80 tx=0x7f790000b330 comp rx=0 tx=0).stop 2024-04-24T06:32:28.979 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.269+0000 7f7916000700 1 -- 172.21.15.5:0/313977922 shutdown_connections 2024-04-24T06:32:28.979 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.269+0000 7f7916000700 1 --2- 172.21.15.5:0/313977922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f791010f030 0x7f791010f450 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:28.979 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.269+0000 7f7916000700 1 -- 172.21.15.5:0/313977922 >> 172.21.15.5:0/313977922 conn(0x7f791007de90 msgr2=0x7f79100802b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:28.979 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.269+0000 7f7916000700 1 -- 172.21.15.5:0/313977922 shutdown_connections 2024-04-24T06:32:28.979 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.269+0000 7f7916000700 1 -- 172.21.15.5:0/313977922 wait complete. 2024-04-24T06:32:28.979 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.271+0000 7f7916000700 1 Processor -- start 2024-04-24T06:32:28.979 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.271+0000 7f7916000700 1 -- start start 2024-04-24T06:32:28.979 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.271+0000 7f7916000700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f791010f030 0x7f79101a23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:28.979 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.271+0000 7f7916000700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7910110190 con 0x7f791010f030 2024-04-24T06:32:28.979 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.272+0000 7f790f400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f791010f030 0x7f79101a23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:28.979 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.272+0000 7f790f400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f791010f030 0x7f79101a23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35586/0 (socket says 172.21.15.5:35586) 2024-04-24T06:32:28.980 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.272+0000 7f790f400700 1 -- 172.21.15.5:0/1403621580 learned_addr learned my addr 172.21.15.5:0/1403621580 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:32:28.980 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.272+0000 7f790f400700 1 -- 172.21.15.5:0/1403621580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79101a28e0 con 0x7f791010f030 2024-04-24T06:32:28.980 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.272+0000 7f790f400700 1 --2- 172.21.15.5:0/1403621580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f791010f030 0x7f79101a23a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7910082430 tx=0x7f7900004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:28.980 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.273+0000 7f790cc00700 1 -- 172.21.15.5:0/1403621580 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7900022020 con 0x7f791010f030 2024-04-24T06:32:28.980 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.273+0000 7f790cc00700 1 -- 172.21.15.5:0/1403621580 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7900018040 con 0x7f791010f030 2024-04-24T06:32:28.980 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.273+0000 7f790cc00700 1 -- 172.21.15.5:0/1403621580 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7900017070 con 0x7f791010f030 2024-04-24T06:32:28.980 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.273+0000 7f7916000700 1 -- 172.21.15.5:0/1403621580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79101a2b70 con 0x7f791010f030 2024-04-24T06:32:28.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.273+0000 7f7916000700 1 -- 172.21.15.5:0/1403621580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79101a5860 con 0x7f791010f030 2024-04-24T06:32:28.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.273+0000 7f790cc00700 1 -- 172.21.15.5:0/1403621580 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7900025020 con 0x7f791010f030 2024-04-24T06:32:28.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.273+0000 7f790cc00700 1 -- 172.21.15.5:0/1403621580 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f790000f050 con 0x7f791010f030 2024-04-24T06:32:28.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.274+0000 7f7916000700 1 -- 172.21.15.5:0/1403621580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7910064150 con 0x7f791010f030 2024-04-24T06:32:28.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.277+0000 7f790cc00700 1 -- 172.21.15.5:0/1403621580 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f790002c030 con 0x7f791010f030 2024-04-24T06:32:28.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.356+0000 7f7916000700 1 -- 172.21.15.5:0/1403621580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f791002e230 con 0x7f791010f030 2024-04-24T06:32:28.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.364+0000 7f790cc00700 1 -- 172.21.15.5:0/1403621580 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+454 (secure 0 0 0) 0x7f790001e480 con 0x7f791010f030 2024-04-24T06:32:28.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.365+0000 7f7916000700 1 -- 172.21.15.5:0/1403621580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f791010f030 msgr2=0x7f79101a23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:28.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.365+0000 7f7916000700 1 --2- 172.21.15.5:0/1403621580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f791010f030 0x7f79101a23a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7910082430 tx=0x7f7900004ad0 comp rx=0 tx=0).stop 2024-04-24T06:32:28.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.365+0000 7f7916000700 1 -- 172.21.15.5:0/1403621580 shutdown_connections 2024-04-24T06:32:28.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.365+0000 7f7916000700 1 --2- 172.21.15.5:0/1403621580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f791010f030 0x7f79101a23a0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:28.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.365+0000 7f7916000700 1 -- 172.21.15.5:0/1403621580 >> 172.21.15.5:0/1403621580 conn(0x7f791007de90 msgr2=0x7f7910198f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:28.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.366+0000 7f7916000700 1 -- 172.21.15.5:0/1403621580 shutdown_connections 2024-04-24T06:32:28.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:28.366+0000 7f7916000700 1 -- 172.21.15.5:0/1403621580 wait complete. 2024-04-24T06:32:28.983 INFO:teuthology.orchestra.run.smithi005.stdout:Generating new minimal ceph.conf... 2024-04-24T06:32:29.965 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:29 smithi005 ceph-mon[27428]: from='client.? 172.21.15.5:0/424882760' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:32:30.211 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.586+0000 7f957f000700 1 Processor -- start 2024-04-24T06:32:30.211 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.586+0000 7f957f000700 1 -- start start 2024-04-24T06:32:30.211 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.587+0000 7f957f000700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f957810a740 0x7f957810ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:30.211 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.587+0000 7f957f000700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f957810b130 con 0x7f957810a740 2024-04-24T06:32:30.211 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.587+0000 7f957dc00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f957810a740 0x7f957810ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:30.211 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.587+0000 7f957dc00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f957810a740 0x7f957810ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35596/0 (socket says 172.21.15.5:35596) 2024-04-24T06:32:30.211 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.587+0000 7f957dc00700 1 -- 172.21.15.5:0/1173578632 learned_addr learned my addr 172.21.15.5:0/1173578632 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:32:30.211 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.588+0000 7f957dc00700 1 -- 172.21.15.5:0/1173578632 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f957810b8f0 con 0x7f957810a740 2024-04-24T06:32:30.211 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.588+0000 7f957dc00700 1 --2- 172.21.15.5:0/1173578632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f957810a740 0x7f957810ab60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f956c0098a0 tx=0x7f956c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf41a0c07e78112 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:30.211 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.589+0000 7f9577e00700 1 -- 172.21.15.5:0/1173578632 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f956c004160 con 0x7f957810a740 2024-04-24T06:32:30.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.589+0000 7f9577e00700 1 -- 172.21.15.5:0/1173578632 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f956c004300 con 0x7f957810a740 2024-04-24T06:32:30.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.589+0000 7f9577e00700 1 -- 172.21.15.5:0/1173578632 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f956c011b50 con 0x7f957810a740 2024-04-24T06:32:30.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.589+0000 7f957f000700 1 -- 172.21.15.5:0/1173578632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f957810a740 msgr2=0x7f957810ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:30.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.589+0000 7f957f000700 1 --2- 172.21.15.5:0/1173578632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f957810a740 0x7f957810ab60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f956c0098a0 tx=0x7f956c009bb0 comp rx=0 tx=0).stop 2024-04-24T06:32:30.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.589+0000 7f957f000700 1 -- 172.21.15.5:0/1173578632 shutdown_connections 2024-04-24T06:32:30.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.589+0000 7f957f000700 1 --2- 172.21.15.5:0/1173578632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f957810a740 0x7f957810ab60 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:30.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.590+0000 7f957f000700 1 -- 172.21.15.5:0/1173578632 >> 172.21.15.5:0/1173578632 conn(0x7f957807dbd0 msgr2=0x7f95781084f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:30.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.590+0000 7f957f000700 1 -- 172.21.15.5:0/1173578632 shutdown_connections 2024-04-24T06:32:30.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.590+0000 7f957f000700 1 -- 172.21.15.5:0/1173578632 wait complete. 2024-04-24T06:32:30.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.591+0000 7f957f000700 1 Processor -- start 2024-04-24T06:32:30.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.591+0000 7f957f000700 1 -- start start 2024-04-24T06:32:30.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.591+0000 7f957f000700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f957810a740 0x7f957819e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:30.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.591+0000 7f957f000700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f957810eca0 con 0x7f957810a740 2024-04-24T06:32:30.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.592+0000 7f957dc00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f957810a740 0x7f957819e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:30.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.592+0000 7f957dc00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f957810a740 0x7f957819e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35604/0 (socket says 172.21.15.5:35604) 2024-04-24T06:32:30.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.592+0000 7f957dc00700 1 -- 172.21.15.5:0/424882760 learned_addr learned my addr 172.21.15.5:0/424882760 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:32:30.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.592+0000 7f957dc00700 1 -- 172.21.15.5:0/424882760 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f957819ec20 con 0x7f957810a740 2024-04-24T06:32:30.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.592+0000 7f957dc00700 1 --2- 172.21.15.5:0/424882760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f957810a740 0x7f957819e6e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f956c011850 tx=0x7f956c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:30.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.593+0000 7f9576000700 1 -- 172.21.15.5:0/424882760 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f956c003af0 con 0x7f957810a740 2024-04-24T06:32:30.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.593+0000 7f9576000700 1 -- 172.21.15.5:0/424882760 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f956c020070 con 0x7f957810a740 2024-04-24T06:32:30.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.593+0000 7f957f000700 1 -- 172.21.15.5:0/424882760 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f957819eeb0 con 0x7f957810a740 2024-04-24T06:32:30.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.593+0000 7f9576000700 1 -- 172.21.15.5:0/424882760 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f956c0195a0 con 0x7f957810a740 2024-04-24T06:32:30.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.593+0000 7f957f000700 1 -- 172.21.15.5:0/424882760 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95781a1ba0 con 0x7f957810a740 2024-04-24T06:32:30.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.595+0000 7f9576000700 1 -- 172.21.15.5:0/424882760 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f956c019850 con 0x7f957810a740 2024-04-24T06:32:30.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.595+0000 7f9576000700 1 -- 172.21.15.5:0/424882760 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f956c011e00 con 0x7f957810a740 2024-04-24T06:32:30.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.595+0000 7f957f000700 1 -- 172.21.15.5:0/424882760 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9578069c50 con 0x7f957810a740 2024-04-24T06:32:30.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.598+0000 7f9576000700 1 -- 172.21.15.5:0/424882760 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f956c003c90 con 0x7f957810a740 2024-04-24T06:32:30.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.677+0000 7f957f000700 1 -- 172.21.15.5:0/424882760 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f95781a2150 con 0x7f957810a740 2024-04-24T06:32:30.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.677+0000 7f9576000700 1 -- 172.21.15.5:0/424882760 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+173 (secure 0 0 0) 0x7f956c019cd0 con 0x7f957810a740 2024-04-24T06:32:30.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.679+0000 7f957f000700 1 -- 172.21.15.5:0/424882760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f957810a740 msgr2=0x7f957819e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:30.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.679+0000 7f957f000700 1 --2- 172.21.15.5:0/424882760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f957810a740 0x7f957819e6e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f956c011850 tx=0x7f956c011c10 comp rx=0 tx=0).stop 2024-04-24T06:32:30.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.679+0000 7f957f000700 1 -- 172.21.15.5:0/424882760 shutdown_connections 2024-04-24T06:32:30.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.679+0000 7f957f000700 1 --2- 172.21.15.5:0/424882760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f957810a740 0x7f957819e6e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:30.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.679+0000 7f957f000700 1 -- 172.21.15.5:0/424882760 >> 172.21.15.5:0/424882760 conn(0x7f957807dbd0 msgr2=0x7f9578107cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:30.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.679+0000 7f957f000700 1 -- 172.21.15.5:0/424882760 shutdown_connections 2024-04-24T06:32:30.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:29.679+0000 7f957f000700 1 -- 172.21.15.5:0/424882760 wait complete. 2024-04-24T06:32:30.214 INFO:teuthology.orchestra.run.smithi005.stdout:Restarting the monitor... 2024-04-24T06:32:30.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:30 smithi005 systemd[1]: Stopping Ceph mon.a for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:32:30.931 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:30 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mon-a[27424]: 2024-04-24T06:32:30.670+0000 7f6577800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T06:32:30.931 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:30 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mon-a[27424]: 2024-04-24T06:32:30.670+0000 7f6577800700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-24T06:32:31.195 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:30 smithi005 podman[27670]: 2024-04-24 06:32:30.929762313 +0000 UTC m=+0.385991391 container died 0202765fff9bc094a11d236cbb2b455b2f8ddf0f88c637f0ef822ca69b1c6744 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T06:32:31.502 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:31 smithi005 podman[27670]: 2024-04-24 06:32:31.347274035 +0000 UTC m=+0.803503106 container cleanup 0202765fff9bc094a11d236cbb2b455b2f8ddf0f88c637f0ef822ca69b1c6744 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T06:32:31.502 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:31 smithi005 bash[27670]: ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mon-a 2024-04-24T06:32:31.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:31 smithi005 podman[27684]: 2024-04-24 06:32:31.500126712 +0000 UTC m=+0.555894979 container remove 0202765fff9bc094a11d236cbb2b455b2f8ddf0f88c637f0ef822ca69b1c6744 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.build-date=20240416, io.buildah.version=1.33.5) 2024-04-24T06:32:32.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:31 smithi005 systemd[1]: ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-24T06:32:32.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:31 smithi005 systemd[1]: Stopped Ceph mon.a for 43e9bad0-0204-11ef-bc93-c7b262605968. 2024-04-24T06:32:32.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:31 smithi005 systemd[1]: ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mon.a.service: Consumed 1.908s CPU time. 2024-04-24T06:32:32.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:31 smithi005 systemd[1]: Starting Ceph mon.a for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:32:32.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:32 smithi005 podman[27779]: 2024-04-24 06:32:32.288491598 +0000 UTC m=+0.037538572 image pull b5c3183062a6ce19f361681cc58de6e3ac1de1a54c80e6094ec5695237cd98d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 2024-04-24T06:32:37.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:37 smithi005 podman[27779]: 2024-04-24 06:32:37.425283601 +0000 UTC m=+5.174330598 container create a5810a88074d7113bae166a1dc2a8c1cbfafa4aa077ee7a5fe566495cd6d1c44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-24T06:32:40.621 INFO:teuthology.orchestra.run.smithi005.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-24T06:32:40.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 podman[27779]: 2024-04-24 06:32:40.527905288 +0000 UTC m=+8.276952303 container init a5810a88074d7113bae166a1dc2a8c1cbfafa4aa077ee7a5fe566495cd6d1c44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T06:32:40.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 podman[27779]: 2024-04-24 06:32:40.538445532 +0000 UTC m=+8.287492513 container start a5810a88074d7113bae166a1dc2a8c1cbfafa4aa077ee7a5fe566495cd6d1c44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416) 2024-04-24T06:32:40.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: set uid:gid to 167:167 (ceph:ceph) 2024-04-24T06:32:40.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: ceph version 19.0.0-3124-ga3c8a22b (a3c8a22bd5efe18ab230a8e2d072db8f381a0411) squid (dev), process ceph-mon, pid 2 2024-04-24T06:32:40.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: pidfile_write: ignore empty --pid-file 2024-04-24T06:32:40.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: load: jerasure load: lrc 2024-04-24T06:32:40.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: RocksDB version: 7.9.2 2024-04-24T06:32:40.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Git sha 0 2024-04-24T06:32:40.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Compile date 2024-04-23 20:58:29 2024-04-24T06:32:40.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: DB SUMMARY 2024-04-24T06:32:40.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: DB Session ID: 8MGF09YN6HT1AP92OR6S 2024-04-24T06:32:40.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: CURRENT file: CURRENT 2024-04-24T06:32:40.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: IDENTITY file: IDENTITY 2024-04-24T06:32:40.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-24T06:32:40.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-24T06:32:40.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105998 ; 2024-04-24T06:32:40.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.error_if_exists: 0 2024-04-24T06:32:40.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.create_if_missing: 0 2024-04-24T06:32:40.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.paranoid_checks: 1 2024-04-24T06:32:40.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-24T06:32:40.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-24T06:32:40.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-24T06:32:40.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.env: 0x5620b4e2bcc0 2024-04-24T06:32:40.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.fs: PosixFileSystem 2024-04-24T06:32:40.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.info_log: 0x5620cc4dfc60 2024-04-24T06:32:40.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.max_file_opening_threads: 16 2024-04-24T06:32:40.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.statistics: (nil) 2024-04-24T06:32:40.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.use_fsync: 0 2024-04-24T06:32:40.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.max_log_file_size: 0 2024-04-24T06:32:40.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-24T06:32:40.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-24T06:32:40.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.keep_log_file_num: 1000 2024-04-24T06:32:40.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.recycle_log_file_num: 0 2024-04-24T06:32:40.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.allow_fallocate: 1 2024-04-24T06:32:40.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.allow_mmap_reads: 0 2024-04-24T06:32:40.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.allow_mmap_writes: 0 2024-04-24T06:32:40.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.use_direct_reads: 0 2024-04-24T06:32:40.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-24T06:32:40.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.create_missing_column_families: 0 2024-04-24T06:32:40.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.db_log_dir: 2024-04-24T06:32:40.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.wal_dir: 2024-04-24T06:32:40.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-24T06:32:40.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-24T06:32:40.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-24T06:32:40.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-24T06:32:40.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-24T06:32:40.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-24T06:32:40.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.advise_random_on_open: 1 2024-04-24T06:32:40.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.db_write_buffer_size: 0 2024-04-24T06:32:40.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.write_buffer_manager: 0x5620cc4ae1e0 2024-04-24T06:32:40.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-24T06:32:40.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-24T06:32:40.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-24T06:32:40.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.rate_limiter: (nil) 2024-04-24T06:32:40.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-24T06:32:40.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.wal_recovery_mode: 2 2024-04-24T06:32:40.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.enable_thread_tracking: 0 2024-04-24T06:32:40.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.enable_pipelined_write: 0 2024-04-24T06:32:40.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.unordered_write: 0 2024-04-24T06:32:40.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-24T06:32:40.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-24T06:32:40.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-24T06:32:40.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-24T06:32:40.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.row_cache: None 2024-04-24T06:32:40.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.wal_filter: None 2024-04-24T06:32:40.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-24T06:32:40.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.allow_ingest_behind: 0 2024-04-24T06:32:40.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.two_write_queues: 0 2024-04-24T06:32:40.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.manual_wal_flush: 0 2024-04-24T06:32:40.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.wal_compression: 0 2024-04-24T06:32:40.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.atomic_flush: 0 2024-04-24T06:32:40.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-24T06:32:40.911 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-24T06:32:40.911 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-24T06:32:40.911 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.log_readahead_size: 0 2024-04-24T06:32:40.911 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-24T06:32:40.911 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.best_efforts_recovery: 0 2024-04-24T06:32:40.911 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-24T06:32:40.912 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-24T06:32:40.912 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.allow_data_in_errors: 0 2024-04-24T06:32:40.912 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.db_host_id: __hostname__ 2024-04-24T06:32:40.912 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-24T06:32:40.912 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.max_background_jobs: 2 2024-04-24T06:32:40.912 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.max_background_compactions: -1 2024-04-24T06:32:40.913 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.max_subcompactions: 1 2024-04-24T06:32:40.913 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-24T06:32:40.913 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-24T06:32:40.914 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-24T06:32:40.914 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.max_total_wal_size: 0 2024-04-24T06:32:40.914 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-24T06:32:40.914 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-24T06:32:40.914 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-24T06:32:40.914 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-24T06:32:40.914 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.max_open_files: -1 2024-04-24T06:32:40.914 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.bytes_per_sync: 0 2024-04-24T06:32:40.914 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-24T06:32:40.915 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-24T06:32:40.915 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.compaction_readahead_size: 0 2024-04-24T06:32:40.915 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.max_background_flushes: -1 2024-04-24T06:32:40.915 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Compression algorithms supported: 2024-04-24T06:32:40.915 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-24T06:32:40.915 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: kZSTD supported: 0 2024-04-24T06:32:40.915 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: kXpressCompression supported: 0 2024-04-24T06:32:40.915 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: kLZ4HCCompression supported: 1 2024-04-24T06:32:40.915 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: kZlibCompression supported: 1 2024-04-24T06:32:40.915 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: kSnappyCompression supported: 1 2024-04-24T06:32:40.915 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: kLZ4Compression supported: 1 2024-04-24T06:32:40.915 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: kBZip2Compression supported: 0 2024-04-24T06:32:40.915 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-24T06:32:40.916 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-24T06:32:40.916 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-24T06:32:40.916 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-24T06:32:40.916 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-24T06:32:40.916 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.merge_operator: 2024-04-24T06:32:40.916 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.compaction_filter: None 2024-04-24T06:32:40.916 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.compaction_filter_factory: None 2024-04-24T06:32:40.916 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.sst_partitioner_factory: None 2024-04-24T06:32:40.916 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-24T06:32:40.916 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-24T06:32:40.916 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5620cc4dfe20) 2024-04-24T06:32:40.916 INFO:journalctl@ceph.mon.a.smithi005.stdout: cache_index_and_filter_blocks: 1 2024-04-24T06:32:40.916 INFO:journalctl@ceph.mon.a.smithi005.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-24T06:32:40.916 INFO:journalctl@ceph.mon.a.smithi005.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-24T06:32:40.917 INFO:journalctl@ceph.mon.a.smithi005.stdout: pin_top_level_index_and_filter: 1 2024-04-24T06:32:40.917 INFO:journalctl@ceph.mon.a.smithi005.stdout: index_type: 0 2024-04-24T06:32:40.917 INFO:journalctl@ceph.mon.a.smithi005.stdout: data_block_index_type: 0 2024-04-24T06:32:40.917 INFO:journalctl@ceph.mon.a.smithi005.stdout: index_shortening: 1 2024-04-24T06:32:40.917 INFO:journalctl@ceph.mon.a.smithi005.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-24T06:32:40.917 INFO:journalctl@ceph.mon.a.smithi005.stdout: checksum: 4 2024-04-24T06:32:40.917 INFO:journalctl@ceph.mon.a.smithi005.stdout: no_block_cache: 0 2024-04-24T06:32:40.917 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache: 0x5620cc4e1090 2024-04-24T06:32:40.917 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache_name: BinnedLRUCache 2024-04-24T06:32:40.917 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache_options: 2024-04-24T06:32:40.917 INFO:journalctl@ceph.mon.a.smithi005.stdout: capacity : 536870912 2024-04-24T06:32:40.917 INFO:journalctl@ceph.mon.a.smithi005.stdout: num_shard_bits : 4 2024-04-24T06:32:40.917 INFO:journalctl@ceph.mon.a.smithi005.stdout: strict_capacity_limit : 0 2024-04-24T06:32:40.917 INFO:journalctl@ceph.mon.a.smithi005.stdout: high_pri_pool_ratio: 0.000 2024-04-24T06:32:40.918 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache_compressed: (nil) 2024-04-24T06:32:40.918 INFO:journalctl@ceph.mon.a.smithi005.stdout: persistent_cache: (nil) 2024-04-24T06:32:40.918 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_size: 4096 2024-04-24T06:32:40.918 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_size_deviation: 10 2024-04-24T06:32:40.918 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_restart_interval: 16 2024-04-24T06:32:40.918 INFO:journalctl@ceph.mon.a.smithi005.stdout: index_block_restart_interval: 1 2024-04-24T06:32:40.918 INFO:journalctl@ceph.mon.a.smithi005.stdout: metadata_block_size: 4096 2024-04-24T06:32:40.918 INFO:journalctl@ceph.mon.a.smithi005.stdout: partition_filters: 0 2024-04-24T06:32:40.918 INFO:journalctl@ceph.mon.a.smithi005.stdout: use_delta_encoding: 1 2024-04-24T06:32:40.918 INFO:journalctl@ceph.mon.a.smithi005.stdout: filter_policy: bloomfilter 2024-04-24T06:32:40.918 INFO:journalctl@ceph.mon.a.smithi005.stdout: whole_key_filtering: 1 2024-04-24T06:32:40.918 INFO:journalctl@ceph.mon.a.smithi005.stdout: verify_compression: 0 2024-04-24T06:32:40.918 INFO:journalctl@ceph.mon.a.smithi005.stdout: read_amp_bytes_per_bit: 0 2024-04-24T06:32:40.919 INFO:journalctl@ceph.mon.a.smithi005.stdout: format_version: 5 2024-04-24T06:32:40.919 INFO:journalctl@ceph.mon.a.smithi005.stdout: enable_index_compression: 1 2024-04-24T06:32:40.919 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_align: 0 2024-04-24T06:32:40.919 INFO:journalctl@ceph.mon.a.smithi005.stdout: max_auto_readahead_size: 262144 2024-04-24T06:32:40.919 INFO:journalctl@ceph.mon.a.smithi005.stdout: prepopulate_block_cache: 0 2024-04-24T06:32:40.919 INFO:journalctl@ceph.mon.a.smithi005.stdout: initial_auto_readahead_size: 8192 2024-04-24T06:32:40.919 INFO:journalctl@ceph.mon.a.smithi005.stdout: num_file_reads_for_auto_readahead: 2 2024-04-24T06:32:40.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.write_buffer_size: 33554432 2024-04-24T06:32:40.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.max_write_buffer_number: 2 2024-04-24T06:32:40.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.compression: NoCompression 2024-04-24T06:32:40.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.bottommost_compression: Disabled 2024-04-24T06:32:40.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.prefix_extractor: nullptr 2024-04-24T06:32:40.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-24T06:32:40.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.num_levels: 7 2024-04-24T06:32:40.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-24T06:32:40.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-24T06:32:40.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-24T06:32:40.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-24T06:32:40.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-24T06:32:40.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-24T06:32:40.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-24T06:32:40.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-24T06:32:40.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-24T06:32:40.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-24T06:32:40.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-24T06:32:40.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-24T06:32:40.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-24T06:32:40.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.compression_opts.level: 32767 2024-04-24T06:32:40.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.compression_opts.strategy: 0 2024-04-24T06:32:40.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-24T06:32:40.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-24T06:32:40.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-24T06:32:40.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-24T06:32:40.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.compression_opts.enabled: false 2024-04-24T06:32:40.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-24T06:32:40.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-24T06:32:40.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-24T06:32:40.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-24T06:32:40.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.target_file_size_base: 67108864 2024-04-24T06:32:40.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-24T06:32:40.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-24T06:32:40.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-24T06:32:40.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-24T06:32:40.922 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-24T06:32:40.922 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-24T06:32:40.922 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-24T06:32:40.922 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-24T06:32:40.922 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-24T06:32:40.922 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-24T06:32:40.922 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-24T06:32:40.922 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-24T06:32:40.922 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-24T06:32:40.922 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-24T06:32:40.922 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.arena_block_size: 1048576 2024-04-24T06:32:40.922 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-24T06:32:40.922 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-24T06:32:40.923 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.disable_auto_compactions: 0 2024-04-24T06:32:40.923 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-24T06:32:40.923 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-24T06:32:40.923 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-24T06:32:40.923 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-24T06:32:40.923 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-24T06:32:40.923 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-24T06:32:40.923 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-24T06:32:40.923 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-24T06:32:40.923 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-24T06:32:40.923 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-24T06:32:40.923 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-24T06:32:40.923 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.inplace_update_support: 0 2024-04-24T06:32:40.923 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-24T06:32:40.924 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-24T06:32:40.924 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-24T06:32:40.924 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-24T06:32:40.924 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.bloom_locality: 0 2024-04-24T06:32:40.924 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.max_successive_merges: 0 2024-04-24T06:32:40.924 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-24T06:32:40.924 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.paranoid_file_checks: 0 2024-04-24T06:32:40.924 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.force_consistency_checks: 1 2024-04-24T06:32:40.924 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.report_bg_io_stats: 0 2024-04-24T06:32:40.924 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.ttl: 2592000 2024-04-24T06:32:40.924 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-24T06:32:40.924 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-24T06:32:40.925 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-24T06:32:40.925 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.enable_blob_files: false 2024-04-24T06:32:40.925 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.min_blob_size: 0 2024-04-24T06:32:40.925 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.blob_file_size: 268435456 2024-04-24T06:32:40.925 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-24T06:32:40.925 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-24T06:32:40.926 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-24T06:32:40.926 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-24T06:32:40.926 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-24T06:32:40.926 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.blob_file_starting_level: 0 2024-04-24T06:32:40.926 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-24T06:32:40.926 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-24T06:32:40.926 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-24T06:32:40.926 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e4a854fc-1d31-4e9d-a04b-9055466ee452 2024-04-24T06:32:40.926 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713940360598636, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-24T06:32:40.926 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-24T06:32:40.926 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713940360600672, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 101022, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 330, "table_properties": {"data_size": 98911, "index_size": 303, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14409, "raw_average_key_size": 49, "raw_value_size": 91114, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 289, "num_filter_entries": 289, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713940360, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e4a854fc-1d31-4e9d-a04b-9055466ee452", "db_session_id": "8MGF09YN6HT1AP92OR6S", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-24T06:32:40.926 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713940360600881, "job": 1, "event": "recovery_finished"} 2024-04-24T06:32:40.927 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-24T06:32:40.927 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-24T06:32:40.927 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5620cc5ce000 2024-04-24T06:32:40.927 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: DB pointer 0x5620cc5b8000 2024-04-24T06:32:40.927 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-24T06:32:40.927 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-24T06:32:40.927 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** DB Stats ** 2024-04-24T06:32:40.927 INFO:journalctl@ceph.mon.a.smithi005.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T06:32:40.927 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-24T06:32:40.927 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T06:32:40.927 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T06:32:40.927 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-24T06:32:40.927 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T06:32:40.927 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T06:32:40.928 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-04-24T06:32:40.928 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** Compaction Stats [default] ** 2024-04-24T06:32:40.928 INFO:journalctl@ceph.mon.a.smithi005.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T06:32:40.928 INFO:journalctl@ceph.mon.a.smithi005.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-24T06:32:40.928 INFO:journalctl@ceph.mon.a.smithi005.stdout: L0 2/0 100.51 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 70.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T06:32:40.928 INFO:journalctl@ceph.mon.a.smithi005.stdout: Sum 2/0 100.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 70.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T06:32:40.928 INFO:journalctl@ceph.mon.a.smithi005.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 70.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T06:32:40.928 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-04-24T06:32:40.928 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** Compaction Stats [default] ** 2024-04-24T06:32:40.928 INFO:journalctl@ceph.mon.a.smithi005.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T06:32:40.928 INFO:journalctl@ceph.mon.a.smithi005.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T06:32:40.928 INFO:journalctl@ceph.mon.a.smithi005.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 70.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T06:32:40.928 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-04-24T06:32:40.928 INFO:journalctl@ceph.mon.a.smithi005.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-24T06:32:40.928 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-04-24T06:32:40.929 INFO:journalctl@ceph.mon.a.smithi005.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T06:32:40.929 INFO:journalctl@ceph.mon.a.smithi005.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-24T06:32:40.929 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-24T06:32:40.929 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-24T06:32:40.929 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-24T06:32:40.929 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-24T06:32:40.929 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative compaction: 0.00 GB write, 14.36 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T06:32:40.929 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval compaction: 0.00 GB write, 14.36 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T06:32:40.929 INFO:journalctl@ceph.mon.a.smithi005.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-24T06:32:40.929 INFO:journalctl@ceph.mon.a.smithi005.stdout: Block cache BinnedLRUCache@0x5620cc4e1090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-04-24T06:32:40.929 INFO:journalctl@ceph.mon.a.smithi005.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-24T06:32:40.929 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-04-24T06:32:40.929 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-24T06:32:40.929 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: starting mon.a rank 0 at public addrs [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] at bind addrs [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 43e9bad0-0204-11ef-bc93-c7b262605968 2024-04-24T06:32:40.930 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: mon.a@-1(???) e1 preinit fsid 43e9bad0-0204-11ef-bc93-c7b262605968 2024-04-24T06:32:40.930 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: mon.a@-1(???).mds e1 new map 2024-04-24T06:32:40.930 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: mon.a@-1(???).mds e1 print_map 2024-04-24T06:32:40.930 INFO:journalctl@ceph.mon.a.smithi005.stdout: e1 2024-04-24T06:32:40.930 INFO:journalctl@ceph.mon.a.smithi005.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T06:32:40.930 INFO:journalctl@ceph.mon.a.smithi005.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T06:32:40.930 INFO:journalctl@ceph.mon.a.smithi005.stdout: legacy client fscid: -1 2024-04-24T06:32:40.930 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-04-24T06:32:40.930 INFO:journalctl@ceph.mon.a.smithi005.stdout: No filesystems configured 2024-04-24T06:32:40.930 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-24T06:32:40.930 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T06:32:40.930 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T06:32:40.930 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T06:32:40.930 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-24T06:32:40.931 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 bash[27779]: a5810a88074d7113bae166a1dc2a8c1cbfafa4aa077ee7a5fe566495cd6d1c44 2024-04-24T06:32:40.931 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 systemd[1]: Started Ceph mon.a for 43e9bad0-0204-11ef-bc93-c7b262605968. 2024-04-24T06:32:40.931 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-24T06:32:40.931 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: monmap epoch 1 2024-04-24T06:32:40.931 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: fsid 43e9bad0-0204-11ef-bc93-c7b262605968 2024-04-24T06:32:40.931 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: last_changed 2024-04-24T06:32:22.137390+0000 2024-04-24T06:32:40.931 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: created 2024-04-24T06:32:22.137390+0000 2024-04-24T06:32:40.931 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: min_mon_release 19 (squid) 2024-04-24T06:32:40.931 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: election_strategy: 1 2024-04-24T06:32:40.931 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-04-24T06:32:40.931 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: fsmap 2024-04-24T06:32:40.931 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: osdmap e1: 0 total, 0 up, 0 in 2024-04-24T06:32:40.931 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:40 smithi005 ceph-mon[27798]: mgrmap e1: no daemons active 2024-04-24T06:32:41.981 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.196+0000 7faade400700 1 Processor -- start 2024-04-24T06:32:41.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.197+0000 7faade400700 1 -- start start 2024-04-24T06:32:41.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.197+0000 7faade400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faad8106d30 0x7faad8107150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:41.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.197+0000 7faade400700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faad807bed0 con 0x7faad8106d30 2024-04-24T06:32:41.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.198+0000 7faadd000700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faad8106d30 0x7faad8107150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:41.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.198+0000 7faadd000700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faad8106d30 0x7faad8107150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57620/0 (socket says 172.21.15.5:57620) 2024-04-24T06:32:41.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.198+0000 7faadd000700 1 -- 172.21.15.5:0/1697306789 learned_addr learned my addr 172.21.15.5:0/1697306789 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:32:41.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.199+0000 7faadd000700 1 -- 172.21.15.5:0/1697306789 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faad8107690 con 0x7faad8106d30 2024-04-24T06:32:41.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.199+0000 7faadd000700 1 --2- 172.21.15.5:0/1697306789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faad8106d30 0x7faad8107150 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7faacc0098a0 tx=0x7faacc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ff49f9bca866b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:41.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.199+0000 7faad7400700 1 -- 172.21.15.5:0/1697306789 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faacc004160 con 0x7faad8106d30 2024-04-24T06:32:41.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.199+0000 7faad7400700 1 -- 172.21.15.5:0/1697306789 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faacc004300 con 0x7faad8106d30 2024-04-24T06:32:41.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.200+0000 7faad7400700 1 -- 172.21.15.5:0/1697306789 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faacc011b50 con 0x7faad8106d30 2024-04-24T06:32:41.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.200+0000 7faade400700 1 -- 172.21.15.5:0/1697306789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faad8106d30 msgr2=0x7faad8107150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:41.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.200+0000 7faade400700 1 --2- 172.21.15.5:0/1697306789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faad8106d30 0x7faad8107150 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7faacc0098a0 tx=0x7faacc009bb0 comp rx=0 tx=0).stop 2024-04-24T06:32:41.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.200+0000 7faade400700 1 -- 172.21.15.5:0/1697306789 shutdown_connections 2024-04-24T06:32:41.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.200+0000 7faade400700 1 --2- 172.21.15.5:0/1697306789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faad8106d30 0x7faad8107150 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:41.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.200+0000 7faade400700 1 -- 172.21.15.5:0/1697306789 >> 172.21.15.5:0/1697306789 conn(0x7faad81029d0 msgr2=0x7faad8104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:41.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.200+0000 7faade400700 1 -- 172.21.15.5:0/1697306789 shutdown_connections 2024-04-24T06:32:41.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.200+0000 7faade400700 1 -- 172.21.15.5:0/1697306789 wait complete. 2024-04-24T06:32:41.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.202+0000 7faade400700 1 Processor -- start 2024-04-24T06:32:41.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.202+0000 7faade400700 1 -- start start 2024-04-24T06:32:41.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.202+0000 7faade400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faad8106d30 0x7faad819e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:41.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.202+0000 7faade400700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faad8108720 con 0x7faad8106d30 2024-04-24T06:32:41.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.202+0000 7faadd000700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faad8106d30 0x7faad819e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:41.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.202+0000 7faadd000700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faad8106d30 0x7faad819e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57622/0 (socket says 172.21.15.5:57622) 2024-04-24T06:32:41.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.202+0000 7faadd000700 1 -- 172.21.15.5:0/3045449421 learned_addr learned my addr 172.21.15.5:0/3045449421 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:32:41.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.203+0000 7faadd000700 1 -- 172.21.15.5:0/3045449421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faad819ec40 con 0x7faad8106d30 2024-04-24T06:32:41.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.203+0000 7faadd000700 1 --2- 172.21.15.5:0/3045449421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faad8106d30 0x7faad819e700 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7faacc009fb0 tx=0x7faacc011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:41.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.203+0000 7faad5600700 1 -- 172.21.15.5:0/3045449421 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faacc0038b0 con 0x7faad8106d30 2024-04-24T06:32:41.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.204+0000 7faade400700 1 -- 172.21.15.5:0/3045449421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faad819eed0 con 0x7faad8106d30 2024-04-24T06:32:41.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.204+0000 7faad5600700 1 -- 172.21.15.5:0/3045449421 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faacc011e50 con 0x7faad8106d30 2024-04-24T06:32:41.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.204+0000 7faad5600700 1 -- 172.21.15.5:0/3045449421 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faacc019420 con 0x7faad8106d30 2024-04-24T06:32:41.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.204+0000 7faade400700 1 -- 172.21.15.5:0/3045449421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faad81a1bc0 con 0x7faad8106d30 2024-04-24T06:32:41.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.204+0000 7faad5600700 1 -- 172.21.15.5:0/3045449421 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7faacc019ab0 con 0x7faad8106d30 2024-04-24T06:32:41.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.204+0000 7faad5600700 1 -- 172.21.15.5:0/3045449421 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7faacc024730 con 0x7faad8106d30 2024-04-24T06:32:41.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.205+0000 7faade400700 1 -- 172.21.15.5:0/3045449421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faad8069c50 con 0x7faad8106d30 2024-04-24T06:32:41.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.208+0000 7faad5600700 1 -- 172.21.15.5:0/3045449421 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7faacc043e50 con 0x7faad8106d30 2024-04-24T06:32:41.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.287+0000 7faade400700 1 -- 172.21.15.5:0/3045449421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7faad802e260 con 0x7faad8106d30 2024-04-24T06:32:41.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.291+0000 7faad5600700 1 -- 172.21.15.5:0/3045449421 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7faacc032030 con 0x7faad8106d30 2024-04-24T06:32:41.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.292+0000 7faade400700 1 -- 172.21.15.5:0/3045449421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faad8106d30 msgr2=0x7faad819e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:41.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.292+0000 7faade400700 1 --2- 172.21.15.5:0/3045449421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faad8106d30 0x7faad819e700 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7faacc009fb0 tx=0x7faacc011bf0 comp rx=0 tx=0).stop 2024-04-24T06:32:41.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.293+0000 7faade400700 1 -- 172.21.15.5:0/3045449421 shutdown_connections 2024-04-24T06:32:41.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.293+0000 7faade400700 1 --2- 172.21.15.5:0/3045449421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faad8106d30 0x7faad819e700 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:41.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.293+0000 7faade400700 1 -- 172.21.15.5:0/3045449421 >> 172.21.15.5:0/3045449421 conn(0x7faad81029d0 msgr2=0x7faad8195420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:41.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.293+0000 7faade400700 1 -- 172.21.15.5:0/3045449421 shutdown_connections 2024-04-24T06:32:41.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:41.293+0000 7faade400700 1 -- 172.21.15.5:0/3045449421 wait complete. 2024-04-24T06:32:41.987 INFO:teuthology.orchestra.run.smithi005.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-24T06:32:42.009 INFO:teuthology.orchestra.run.smithi005.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-24T06:32:42.009 INFO:teuthology.orchestra.run.smithi005.stdout:Creating mgr... 2024-04-24T06:32:42.010 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-24T06:32:42.011 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-24T06:32:42.461 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mgr.x 2024-04-24T06:32:42.461 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to reset failed state of unit ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-24T06:32:42.729 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-43e9bad0-0204-11ef-bc93-c7b262605968.target.wants/ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-43e9bad0-0204-11ef-bc93-c7b262605968@.service. 2024-04-24T06:32:42.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:42 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/3045449421' entity='client.admin' 2024-04-24T06:32:43.146 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:32:42 smithi005 systemd[1]: Starting Ceph mgr.x for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:32:43.463 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:32:43 smithi005 podman[28010]: 2024-04-24 06:32:43.143648901 +0000 UTC m=+0.125473796 container create 96ec027a3df56bded49d5e581d53c11a5c83dccfdebab4f2a4cc463b5da72e02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T06:32:43.464 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:32:43 smithi005 podman[28010]: 2024-04-24 06:32:43.054529678 +0000 UTC m=+0.036354547 image pull b5c3183062a6ce19f361681cc58de6e3ac1de1a54c80e6094ec5695237cd98d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 2024-04-24T06:32:43.464 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:32:43 smithi005 podman[28010]: 2024-04-24 06:32:43.37233048 +0000 UTC m=+0.354155392 container init 96ec027a3df56bded49d5e581d53c11a5c83dccfdebab4f2a4cc463b5da72e02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, ceph=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T06:32:43.464 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:32:43 smithi005 podman[28010]: 2024-04-24 06:32:43.382339887 +0000 UTC m=+0.364164761 container start 96ec027a3df56bded49d5e581d53c11a5c83dccfdebab4f2a4cc463b5da72e02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-24T06:32:43.498 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T06:32:43.498 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2024-04-24T06:32:43.515 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T06:32:43.515 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-04-24T06:32:43.515 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2024-04-24T06:32:43.515 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to enable service . firewalld.service is not available 2024-04-24T06:32:43.539 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T06:32:43.539 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2024-04-24T06:32:43.557 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T06:32:43.558 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-04-24T06:32:43.558 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2024-04-24T06:32:43.558 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-24T06:32:43.558 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr to start... 2024-04-24T06:32:43.558 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr... 2024-04-24T06:32:43.715 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:32:43 smithi005 bash[28010]: 96ec027a3df56bded49d5e581d53c11a5c83dccfdebab4f2a4cc463b5da72e02 2024-04-24T06:32:43.715 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:32:43 smithi005 systemd[1]: Started Ceph mgr.x for 43e9bad0-0204-11ef-bc93-c7b262605968. 2024-04-24T06:32:43.715 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:32:43 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:32:43.635+0000 7ff739d73200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T06:32:43.999 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:32:43 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:32:43.912+0000 7ff739d73200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T06:32:44.256 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:32:44 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:32:44.051+0000 7ff739d73200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T06:32:44.597 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:44 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/2574618740' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T06:32:44.598 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:32:44 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:32:44.511+0000 7ff739d73200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T06:32:44.904 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:32:44 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:32:44.640+0000 7ff739d73200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T06:32:44.915 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-04-24T06:32:44.925 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-04-24T06:32:44.925 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsid": "43e9bad0-0204-11ef-bc93-c7b262605968", 2024-04-24T06:32:44.925 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T06:32:44.925 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T06:32:44.925 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T06:32:44.925 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T06:32:44.926 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-24T06:32:44.926 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T06:32:44.926 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T06:32:44.926 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 0 2024-04-24T06:32:44.926 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-04-24T06:32:44.926 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T06:32:44.926 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "a" 2024-04-24T06:32:44.926 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-04-24T06:32:44.926 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-04-24T06:32:44.926 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T06:32:44.926 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T06:32:44.926 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-24T06:32:44.926 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T06:32:44.926 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-24T06:32:44.926 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T06:32:44.927 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T06:32:44.927 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T06:32:44.927 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T06:32:44.927 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T06:32:44.927 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T06:32:44.927 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T06:32:44.927 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T06:32:44.927 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-24T06:32:44.927 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T06:32:44.927 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T06:32:44.927 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T06:32:44.927 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T06:32:44.927 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T06:32:44.927 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T06:32:44.927 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T06:32:44.927 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T06:32:44.928 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T06:32:44.928 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-24T06:32:44.928 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T06:32:44.928 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T06:32:44.928 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T06:32:44.928 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T06:32:44.928 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-24T06:32:44.928 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T06:32:44.928 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-24T06:32:44.928 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T06:32:44.928 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T06:32:44.928 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T06:32:44.928 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T06:32:44.928 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T06:32:44.928 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-04-24T06:32:44.929 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T06:32:44.929 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-24T06:32:44.929 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T06:32:44.929 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T06:32:44.929 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T06:32:26.341159+0000", 2024-04-24T06:32:44.929 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T06:32:44.929 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-24T06:32:44.929 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T06:32:44.929 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-04-24T06:32:44.929 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.177+0000 7fafc7e00700 1 Processor -- start 2024-04-24T06:32:44.929 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.177+0000 7fafc7e00700 1 -- start start 2024-04-24T06:32:44.929 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.177+0000 7fafc7e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafc81073f0 0x7fafc8107810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:44.929 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.177+0000 7fafc7e00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fafc8107de0 con 0x7fafc81073f0 2024-04-24T06:32:44.929 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.178+0000 7fafc6a00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafc81073f0 0x7fafc8107810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:44.930 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.178+0000 7fafc6a00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafc81073f0 0x7fafc8107810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48748/0 (socket says 172.21.15.5:48748) 2024-04-24T06:32:44.930 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.178+0000 7fafc6a00700 1 -- 172.21.15.5:0/1715344083 learned_addr learned my addr 172.21.15.5:0/1715344083 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:32:44.930 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.178+0000 7fafc6a00700 1 -- 172.21.15.5:0/1715344083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc81085f0 con 0x7fafc81073f0 2024-04-24T06:32:44.930 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.179+0000 7fafc6a00700 1 --2- 172.21.15.5:0/1715344083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafc81073f0 0x7fafc8107810 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fafb800bb40 tx=0x7fafb800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c54c9ccd3c6fd71 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:44.930 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.179+0000 7fafc5600700 1 -- 172.21.15.5:0/1715344083 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafb800dbe0 con 0x7fafc81073f0 2024-04-24T06:32:44.930 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.179+0000 7fafc5600700 1 -- 172.21.15.5:0/1715344083 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fafb80044f0 con 0x7fafc81073f0 2024-04-24T06:32:44.930 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.179+0000 7fafc5600700 1 -- 172.21.15.5:0/1715344083 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafb8010470 con 0x7fafc81073f0 2024-04-24T06:32:44.930 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.180+0000 7fafc7e00700 1 -- 172.21.15.5:0/1715344083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafc81073f0 msgr2=0x7fafc8107810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:44.930 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.180+0000 7fafc7e00700 1 --2- 172.21.15.5:0/1715344083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafc81073f0 0x7fafc8107810 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fafb800bb40 tx=0x7fafb800be50 comp rx=0 tx=0).stop 2024-04-24T06:32:44.930 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.180+0000 7fafc7e00700 1 -- 172.21.15.5:0/1715344083 shutdown_connections 2024-04-24T06:32:44.930 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.180+0000 7fafc7e00700 1 --2- 172.21.15.5:0/1715344083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafc81073f0 0x7fafc8107810 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:44.930 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.180+0000 7fafc7e00700 1 -- 172.21.15.5:0/1715344083 >> 172.21.15.5:0/1715344083 conn(0x7fafc8102a10 msgr2=0x7fafc8104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:44.930 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.181+0000 7fafc7e00700 1 -- 172.21.15.5:0/1715344083 shutdown_connections 2024-04-24T06:32:44.930 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.181+0000 7fafc7e00700 1 -- 172.21.15.5:0/1715344083 wait complete. 2024-04-24T06:32:44.930 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.182+0000 7fafc7e00700 1 Processor -- start 2024-04-24T06:32:44.931 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.182+0000 7fafc7e00700 1 -- start start 2024-04-24T06:32:44.931 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.183+0000 7fafc7e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafc81073f0 0x7fafc819e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:44.931 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.183+0000 7fafc7e00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fafc807aff0 con 0x7fafc81073f0 2024-04-24T06:32:44.931 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.183+0000 7fafc6a00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafc81073f0 0x7fafc819e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:44.931 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.183+0000 7fafc6a00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafc81073f0 0x7fafc819e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48750/0 (socket says 172.21.15.5:48750) 2024-04-24T06:32:44.931 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.183+0000 7fafc6a00700 1 -- 172.21.15.5:0/2574618740 learned_addr learned my addr 172.21.15.5:0/2574618740 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:32:44.931 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.184+0000 7fafc6a00700 1 -- 172.21.15.5:0/2574618740 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc819ece0 con 0x7fafc81073f0 2024-04-24T06:32:44.931 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.184+0000 7fafc6a00700 1 --2- 172.21.15.5:0/2574618740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafc81073f0 0x7fafc819e7a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fafb800b340 tx=0x7fafb800dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:44.931 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.184+0000 7fafbf400700 1 -- 172.21.15.5:0/2574618740 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafb8010680 con 0x7fafc81073f0 2024-04-24T06:32:44.931 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.184+0000 7fafbf400700 1 -- 172.21.15.5:0/2574618740 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fafb801d070 con 0x7fafc81073f0 2024-04-24T06:32:44.931 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.184+0000 7fafc7e00700 1 -- 172.21.15.5:0/2574618740 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafc819ef70 con 0x7fafc81073f0 2024-04-24T06:32:44.931 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.184+0000 7fafbf400700 1 -- 172.21.15.5:0/2574618740 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafb8021af0 con 0x7fafc81073f0 2024-04-24T06:32:44.931 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.185+0000 7fafc7e00700 1 -- 172.21.15.5:0/2574618740 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafc81a1c60 con 0x7fafc81073f0 2024-04-24T06:32:44.931 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.185+0000 7fafbf400700 1 -- 172.21.15.5:0/2574618740 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fafb801f030 con 0x7fafc81073f0 2024-04-24T06:32:44.931 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.185+0000 7fafbf400700 1 -- 172.21.15.5:0/2574618740 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fafb8019430 con 0x7fafc81073f0 2024-04-24T06:32:44.932 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.186+0000 7fafc7e00700 1 -- 172.21.15.5:0/2574618740 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafc8069c50 con 0x7fafc81073f0 2024-04-24T06:32:44.932 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.189+0000 7fafbf400700 1 -- 172.21.15.5:0/2574618740 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fafb8015070 con 0x7fafc81073f0 2024-04-24T06:32:44.932 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.297+0000 7fafc7e00700 1 -- 172.21.15.5:0/2574618740 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fafc81938f0 con 0x7fafc81073f0 2024-04-24T06:32:44.932 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.298+0000 7fafbf400700 1 -- 172.21.15.5:0/2574618740 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fafb8021e20 con 0x7fafc81073f0 2024-04-24T06:32:44.932 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.299+0000 7fafc7e00700 1 -- 172.21.15.5:0/2574618740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafc81073f0 msgr2=0x7fafc819e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:44.932 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.299+0000 7fafc7e00700 1 --2- 172.21.15.5:0/2574618740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafc81073f0 0x7fafc819e7a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fafb800b340 tx=0x7fafb800dd90 comp rx=0 tx=0).stop 2024-04-24T06:32:44.932 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.299+0000 7fafc7e00700 1 -- 172.21.15.5:0/2574618740 shutdown_connections 2024-04-24T06:32:44.932 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.299+0000 7fafc7e00700 1 --2- 172.21.15.5:0/2574618740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafc81073f0 0x7fafc819e7a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:44.932 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.299+0000 7fafc7e00700 1 -- 172.21.15.5:0/2574618740 >> 172.21.15.5:0/2574618740 conn(0x7fafc8102a10 msgr2=0x7fafc81909b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:44.932 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.299+0000 7fafc7e00700 1 -- 172.21.15.5:0/2574618740 shutdown_connections 2024-04-24T06:32:44.932 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:44.299+0000 7fafc7e00700 1 -- 172.21.15.5:0/2574618740 wait complete. 2024-04-24T06:32:44.932 INFO:teuthology.orchestra.run.smithi005.stdout:mgr not available, waiting (1/15)... 2024-04-24T06:32:46.630 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:32:46 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:32:46.271+0000 7ff739d73200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T06:32:46.630 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:32:46 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:32:46.502+0000 7ff739d73200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T06:32:47.006 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:32:46 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:32:46.720+0000 7ff739d73200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T06:32:47.358 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:32:47 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:32:47.118+0000 7ff739d73200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T06:32:47.358 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:32:47 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:32:47.247+0000 7ff739d73200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T06:32:47.608 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:32:47 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:32:47.509+0000 7ff739d73200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T06:32:48.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:47 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1303672363' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T06:32:48.007 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:32:47 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:32:47.780+0000 7ff739d73200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T06:32:48.483 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-04-24T06:32:48.487 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-04-24T06:32:48.487 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsid": "43e9bad0-0204-11ef-bc93-c7b262605968", 2024-04-24T06:32:48.487 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T06:32:48.487 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T06:32:48.487 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T06:32:48.487 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T06:32:48.487 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-24T06:32:48.487 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T06:32:48.487 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T06:32:48.488 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 0 2024-04-24T06:32:48.488 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-04-24T06:32:48.488 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T06:32:48.488 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "a" 2024-04-24T06:32:48.488 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-04-24T06:32:48.488 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-04-24T06:32:48.488 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T06:32:48.488 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T06:32:48.488 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-24T06:32:48.488 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T06:32:48.488 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-24T06:32:48.488 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T06:32:48.489 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T06:32:48.489 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T06:32:48.489 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T06:32:48.489 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T06:32:48.497 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T06:32:48.497 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T06:32:48.497 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T06:32:48.498 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-24T06:32:48.498 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T06:32:48.498 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T06:32:48.498 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T06:32:48.498 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T06:32:48.498 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T06:32:48.498 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T06:32:48.498 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T06:32:48.498 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T06:32:48.498 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T06:32:48.498 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-24T06:32:48.498 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T06:32:48.499 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T06:32:48.499 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T06:32:48.499 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T06:32:48.499 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-24T06:32:48.499 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T06:32:48.499 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-24T06:32:48.499 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T06:32:48.499 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T06:32:48.499 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T06:32:48.499 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T06:32:48.499 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T06:32:48.499 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-04-24T06:32:48.499 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T06:32:48.499 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-24T06:32:48.500 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T06:32:48.500 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T06:32:48.500 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T06:32:26.341159+0000", 2024-04-24T06:32:48.500 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T06:32:48.500 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-24T06:32:48.500 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T06:32:48.500 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-04-24T06:32:48.500 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.516+0000 7f728ee00700 1 Processor -- start 2024-04-24T06:32:48.500 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.516+0000 7f728ee00700 1 -- start start 2024-04-24T06:32:48.500 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.516+0000 7f728ee00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f728807de70 0x7f728807c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:48.500 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.516+0000 7f728ee00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f728807c8f0 con 0x7f728807de70 2024-04-24T06:32:48.500 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.517+0000 7f7287e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f728807de70 0x7f728807c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:48.500 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.517+0000 7f7287e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f728807de70 0x7f728807c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48766/0 (socket says 172.21.15.5:48766) 2024-04-24T06:32:48.500 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.517+0000 7f7287e00700 1 -- 172.21.15.5:0/2851724416 learned_addr learned my addr 172.21.15.5:0/2851724416 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:32:48.500 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.518+0000 7f7287e00700 1 -- 172.21.15.5:0/2851724416 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f728807ca70 con 0x7f728807de70 2024-04-24T06:32:48.501 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.518+0000 7f7287e00700 1 --2- 172.21.15.5:0/2851724416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f728807de70 0x7f728807c320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f72780098a0 tx=0x7f7278009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d59431229203a9b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:48.501 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.518+0000 7f7286a00700 1 -- 172.21.15.5:0/2851724416 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7278004160 con 0x7f728807de70 2024-04-24T06:32:48.501 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.518+0000 7f7286a00700 1 -- 172.21.15.5:0/2851724416 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7278004300 con 0x7f728807de70 2024-04-24T06:32:48.501 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.518+0000 7f7286a00700 1 -- 172.21.15.5:0/2851724416 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7278011b50 con 0x7f728807de70 2024-04-24T06:32:48.501 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.519+0000 7f728ee00700 1 -- 172.21.15.5:0/2851724416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f728807de70 msgr2=0x7f728807c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:48.501 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.519+0000 7f728ee00700 1 --2- 172.21.15.5:0/2851724416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f728807de70 0x7f728807c320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f72780098a0 tx=0x7f7278009bb0 comp rx=0 tx=0).stop 2024-04-24T06:32:48.501 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.519+0000 7f728ee00700 1 -- 172.21.15.5:0/2851724416 shutdown_connections 2024-04-24T06:32:48.501 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.519+0000 7f728ee00700 1 --2- 172.21.15.5:0/2851724416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f728807de70 0x7f728807c320 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:48.501 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.519+0000 7f728ee00700 1 -- 172.21.15.5:0/2851724416 >> 172.21.15.5:0/2851724416 conn(0x7f7288103d40 msgr2=0x7f7288106160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:48.501 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.520+0000 7f728ee00700 1 -- 172.21.15.5:0/2851724416 shutdown_connections 2024-04-24T06:32:48.501 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.520+0000 7f728ee00700 1 -- 172.21.15.5:0/2851724416 wait complete. 2024-04-24T06:32:48.501 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.521+0000 7f728ee00700 1 Processor -- start 2024-04-24T06:32:48.501 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.521+0000 7f728ee00700 1 -- start start 2024-04-24T06:32:48.502 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.521+0000 7f728ee00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f728807de70 0x7f728819e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:48.502 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.521+0000 7f728ee00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f728810ecb0 con 0x7f728807de70 2024-04-24T06:32:48.502 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.521+0000 7f7287e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f728807de70 0x7f728819e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:48.502 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.521+0000 7f7287e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f728807de70 0x7f728819e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48768/0 (socket says 172.21.15.5:48768) 2024-04-24T06:32:48.502 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.521+0000 7f7287e00700 1 -- 172.21.15.5:0/1303672363 learned_addr learned my addr 172.21.15.5:0/1303672363 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:32:48.502 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.522+0000 7f7287e00700 1 -- 172.21.15.5:0/1303672363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f728819edb0 con 0x7f728807de70 2024-04-24T06:32:48.502 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.522+0000 7f7287e00700 1 --2- 172.21.15.5:0/1303672363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f728807de70 0x7f728819e870 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7278009fb0 tx=0x7f7278011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:48.502 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.522+0000 7f7284c00700 1 -- 172.21.15.5:0/1303672363 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f72780038b0 con 0x7f728807de70 2024-04-24T06:32:48.502 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.522+0000 7f7284c00700 1 -- 172.21.15.5:0/1303672363 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7278011e50 con 0x7f728807de70 2024-04-24T06:32:48.502 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.522+0000 7f728ee00700 1 -- 172.21.15.5:0/1303672363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f728819f040 con 0x7f728807de70 2024-04-24T06:32:48.502 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.522+0000 7f7284c00700 1 -- 172.21.15.5:0/1303672363 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7278019420 con 0x7f728807de70 2024-04-24T06:32:48.502 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.522+0000 7f728ee00700 1 -- 172.21.15.5:0/1303672363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72881a1d30 con 0x7f728807de70 2024-04-24T06:32:48.503 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.523+0000 7f7284c00700 1 -- 172.21.15.5:0/1303672363 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7278019a80 con 0x7f728807de70 2024-04-24T06:32:48.503 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.523+0000 7f7284c00700 1 -- 172.21.15.5:0/1303672363 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7278024930 con 0x7f728807de70 2024-04-24T06:32:48.503 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.524+0000 7f728ee00700 1 -- 172.21.15.5:0/1303672363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7288064150 con 0x7f728807de70 2024-04-24T06:32:48.503 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.526+0000 7f7284c00700 1 -- 172.21.15.5:0/1303672363 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f7278043e50 con 0x7f728807de70 2024-04-24T06:32:48.503 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.605+0000 7f728ee00700 1 -- 172.21.15.5:0/1303672363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f72881a2270 con 0x7f728807de70 2024-04-24T06:32:48.503 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.606+0000 7f7284c00700 1 -- 172.21.15.5:0/1303672363 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f7278032020 con 0x7f728807de70 2024-04-24T06:32:48.503 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.608+0000 7f728ee00700 1 -- 172.21.15.5:0/1303672363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f728807de70 msgr2=0x7f728819e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:48.503 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.608+0000 7f728ee00700 1 --2- 172.21.15.5:0/1303672363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f728807de70 0x7f728819e870 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7278009fb0 tx=0x7f7278011bf0 comp rx=0 tx=0).stop 2024-04-24T06:32:48.503 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.608+0000 7f728ee00700 1 -- 172.21.15.5:0/1303672363 shutdown_connections 2024-04-24T06:32:48.503 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.608+0000 7f728ee00700 1 --2- 172.21.15.5:0/1303672363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f728807de70 0x7f728819e870 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:48.503 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.608+0000 7f728ee00700 1 -- 172.21.15.5:0/1303672363 >> 172.21.15.5:0/1303672363 conn(0x7f7288103d40 msgr2=0x7f72881048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:48.503 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.608+0000 7f728ee00700 1 -- 172.21.15.5:0/1303672363 shutdown_connections 2024-04-24T06:32:48.503 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:47.608+0000 7f728ee00700 1 -- 172.21.15.5:0/1303672363 wait complete. 2024-04-24T06:32:48.503 INFO:teuthology.orchestra.run.smithi005.stdout:mgr not available, waiting (2/15)... 2024-04-24T06:32:48.756 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:32:48 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:32:48.603+0000 7ff739d73200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T06:32:49.506 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:32:49 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:32:49.169+0000 7ff739d73200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T06:32:49.506 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:32:49 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:32:49.299+0000 7ff739d73200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T06:32:50.006 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:32:49 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:32:49.570+0000 7ff739d73200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T06:32:50.506 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:32:50 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:32:50.233+0000 7ff739d73200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T06:32:50.506 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:32:50 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:32:50.357+0000 7ff739d73200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T06:32:50.944 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:32:50 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:32:50.692+0000 7ff739d73200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T06:32:50.944 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:32:50 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:32:50.879+0000 7ff739d73200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T06:32:51.194 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:32:51 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:32:51.010+0000 7ff739d73200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T06:32:51.194 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:32:51 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:32:51.167+0000 7ff739d73200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T06:32:51.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:51 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/3523935079' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T06:32:51.792 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:32:51 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:32:51.549+0000 7ff739d73200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T06:32:51.802 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-04-24T06:32:51.805 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-04-24T06:32:51.805 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsid": "43e9bad0-0204-11ef-bc93-c7b262605968", 2024-04-24T06:32:51.805 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T06:32:51.805 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T06:32:51.805 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T06:32:51.805 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T06:32:51.805 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-24T06:32:51.806 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T06:32:51.806 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T06:32:51.806 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 0 2024-04-24T06:32:51.806 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-04-24T06:32:51.806 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T06:32:51.806 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "a" 2024-04-24T06:32:51.806 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-04-24T06:32:51.806 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-04-24T06:32:51.806 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T06:32:51.806 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T06:32:51.806 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-24T06:32:51.806 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T06:32:51.806 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-24T06:32:51.807 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T06:32:51.807 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T06:32:51.807 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T06:32:51.807 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T06:32:51.815 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T06:32:51.815 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T06:32:51.815 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T06:32:51.815 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T06:32:51.815 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-24T06:32:51.815 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T06:32:51.815 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T06:32:51.815 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T06:32:51.815 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T06:32:51.815 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T06:32:51.815 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T06:32:51.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T06:32:51.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T06:32:51.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T06:32:51.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-24T06:32:51.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T06:32:51.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T06:32:51.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T06:32:51.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T06:32:51.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-24T06:32:51.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T06:32:51.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-24T06:32:51.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T06:32:51.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T06:32:51.817 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T06:32:51.817 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T06:32:51.817 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T06:32:51.817 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-04-24T06:32:51.817 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T06:32:51.817 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-24T06:32:51.817 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T06:32:51.817 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T06:32:51.817 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T06:32:26.341159+0000", 2024-04-24T06:32:51.817 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T06:32:51.817 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-24T06:32:51.817 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T06:32:51.817 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-04-24T06:32:51.818 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.097+0000 7f4e2b800700 1 Processor -- start 2024-04-24T06:32:51.818 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.097+0000 7f4e2b800700 1 -- start start 2024-04-24T06:32:51.818 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.097+0000 7f4e2b800700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e24108700 0x7f4e24108b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:51.818 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.097+0000 7f4e2b800700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4e241090f0 con 0x7f4e24108700 2024-04-24T06:32:51.818 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.098+0000 7f4e28e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e24108700 0x7f4e24108b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:51.818 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.098+0000 7f4e28e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e24108700 0x7f4e24108b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48774/0 (socket says 172.21.15.5:48774) 2024-04-24T06:32:51.818 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.098+0000 7f4e28e00700 1 -- 172.21.15.5:0/2909056234 learned_addr learned my addr 172.21.15.5:0/2909056234 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:32:51.818 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.098+0000 7f4e28e00700 1 -- 172.21.15.5:0/2909056234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e241098b0 con 0x7f4e24108700 2024-04-24T06:32:51.818 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.099+0000 7f4e28e00700 1 --2- 172.21.15.5:0/2909056234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e24108700 0x7f4e24108b20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4e180098a0 tx=0x7f4e18009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb393c70e87c18e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:51.818 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.099+0000 7f4e23400700 1 -- 172.21.15.5:0/2909056234 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e18004160 con 0x7f4e24108700 2024-04-24T06:32:51.818 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.099+0000 7f4e23400700 1 -- 172.21.15.5:0/2909056234 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4e18004300 con 0x7f4e24108700 2024-04-24T06:32:51.818 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.099+0000 7f4e23400700 1 -- 172.21.15.5:0/2909056234 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e18011b50 con 0x7f4e24108700 2024-04-24T06:32:51.819 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.100+0000 7f4e2b800700 1 -- 172.21.15.5:0/2909056234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e24108700 msgr2=0x7f4e24108b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:51.819 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.100+0000 7f4e2b800700 1 --2- 172.21.15.5:0/2909056234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e24108700 0x7f4e24108b20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4e180098a0 tx=0x7f4e18009bb0 comp rx=0 tx=0).stop 2024-04-24T06:32:51.819 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.100+0000 7f4e2b800700 1 -- 172.21.15.5:0/2909056234 shutdown_connections 2024-04-24T06:32:51.819 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.100+0000 7f4e2b800700 1 --2- 172.21.15.5:0/2909056234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e24108700 0x7f4e24108b20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:51.819 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.100+0000 7f4e2b800700 1 -- 172.21.15.5:0/2909056234 >> 172.21.15.5:0/2909056234 conn(0x7f4e24103d40 msgr2=0x7f4e24106160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:51.819 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.100+0000 7f4e2b800700 1 -- 172.21.15.5:0/2909056234 shutdown_connections 2024-04-24T06:32:51.819 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.100+0000 7f4e2b800700 1 -- 172.21.15.5:0/2909056234 wait complete. 2024-04-24T06:32:51.819 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.101+0000 7f4e2b800700 1 Processor -- start 2024-04-24T06:32:51.819 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.101+0000 7f4e2b800700 1 -- start start 2024-04-24T06:32:51.819 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.102+0000 7f4e2b800700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e24108700 0x7f4e2419c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:51.819 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.102+0000 7f4e2b800700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2410cc60 con 0x7f4e24108700 2024-04-24T06:32:51.819 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.102+0000 7f4e28e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e24108700 0x7f4e2419c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:51.819 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.102+0000 7f4e28e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e24108700 0x7f4e2419c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48776/0 (socket says 172.21.15.5:48776) 2024-04-24T06:32:51.820 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.102+0000 7f4e28e00700 1 -- 172.21.15.5:0/3523935079 learned_addr learned my addr 172.21.15.5:0/3523935079 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:32:51.820 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.102+0000 7f4e28e00700 1 -- 172.21.15.5:0/3523935079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e2419cc10 con 0x7f4e24108700 2024-04-24T06:32:51.820 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.103+0000 7f4e28e00700 1 --2- 172.21.15.5:0/3523935079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e24108700 0x7f4e2419c6d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4e18011850 tx=0x7f4e18011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:51.820 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.103+0000 7f4e21600700 1 -- 172.21.15.5:0/3523935079 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e18003af0 con 0x7f4e24108700 2024-04-24T06:32:51.820 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.103+0000 7f4e21600700 1 -- 172.21.15.5:0/3523935079 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4e18020070 con 0x7f4e24108700 2024-04-24T06:32:51.820 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.103+0000 7f4e2b800700 1 -- 172.21.15.5:0/3523935079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e2419cea0 con 0x7f4e24108700 2024-04-24T06:32:51.820 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.103+0000 7f4e21600700 1 -- 172.21.15.5:0/3523935079 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e180195a0 con 0x7f4e24108700 2024-04-24T06:32:51.820 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.103+0000 7f4e2b800700 1 -- 172.21.15.5:0/3523935079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e2419fb90 con 0x7f4e24108700 2024-04-24T06:32:51.820 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.104+0000 7f4e21600700 1 -- 172.21.15.5:0/3523935079 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4e18019be0 con 0x7f4e24108700 2024-04-24T06:32:51.820 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.104+0000 7f4e21600700 1 -- 172.21.15.5:0/3523935079 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4e18011e00 con 0x7f4e24108700 2024-04-24T06:32:51.820 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.104+0000 7f4e2b800700 1 -- 172.21.15.5:0/3523935079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e24064150 con 0x7f4e24108700 2024-04-24T06:32:51.820 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.107+0000 7f4e21600700 1 -- 172.21.15.5:0/3523935079 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f4e18003c90 con 0x7f4e24108700 2024-04-24T06:32:51.820 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.189+0000 7f4e2b800700 1 -- 172.21.15.5:0/3523935079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f4e2419fd60 con 0x7f4e24108700 2024-04-24T06:32:51.821 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.189+0000 7f4e21600700 1 -- 172.21.15.5:0/3523935079 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f4e18025460 con 0x7f4e24108700 2024-04-24T06:32:51.821 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.191+0000 7f4e2b800700 1 -- 172.21.15.5:0/3523935079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e24108700 msgr2=0x7f4e2419c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:51.821 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.191+0000 7f4e2b800700 1 --2- 172.21.15.5:0/3523935079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e24108700 0x7f4e2419c6d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4e18011850 tx=0x7f4e18011c10 comp rx=0 tx=0).stop 2024-04-24T06:32:51.821 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.191+0000 7f4e2b800700 1 -- 172.21.15.5:0/3523935079 shutdown_connections 2024-04-24T06:32:51.821 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.191+0000 7f4e2b800700 1 --2- 172.21.15.5:0/3523935079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e24108700 0x7f4e2419c6d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:51.821 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.191+0000 7f4e2b800700 1 -- 172.21.15.5:0/3523935079 >> 172.21.15.5:0/3523935079 conn(0x7f4e24103d40 msgr2=0x7f4e241048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:51.821 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.191+0000 7f4e2b800700 1 -- 172.21.15.5:0/3523935079 shutdown_connections 2024-04-24T06:32:51.821 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:51.191+0000 7f4e2b800700 1 -- 172.21.15.5:0/3523935079 wait complete. 2024-04-24T06:32:51.821 INFO:teuthology.orchestra.run.smithi005.stdout:mgr not available, waiting (3/15)... 2024-04-24T06:32:52.633 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:32:52 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:32:52.341+0000 7ff739d73200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T06:32:53.006 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:32:52 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:32:52.631+0000 7ff739d73200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T06:32:53.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:52 smithi005 ceph-mon[27798]: Activating manager daemon x 2024-04-24T06:32:53.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:52 smithi005 ceph-mon[27798]: mgrmap e2: x(active, starting, since 0.0120001s) 2024-04-24T06:32:53.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:52 smithi005 ceph-mon[27798]: from='mgr.14100 172.21.15.5:0/2276580851' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T06:32:53.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:52 smithi005 ceph-mon[27798]: from='mgr.14100 172.21.15.5:0/2276580851' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T06:32:53.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:52 smithi005 ceph-mon[27798]: from='mgr.14100 172.21.15.5:0/2276580851' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T06:32:53.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:52 smithi005 ceph-mon[27798]: from='mgr.14100 172.21.15.5:0/2276580851' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T06:32:53.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:52 smithi005 ceph-mon[27798]: from='mgr.14100 172.21.15.5:0/2276580851' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T06:32:53.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:52 smithi005 ceph-mon[27798]: from='mgr.14100 172.21.15.5:0/2276580851' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T06:32:53.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:52 smithi005 ceph-mon[27798]: from='mgr.14100 172.21.15.5:0/2276580851' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T06:32:53.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:52 smithi005 ceph-mon[27798]: from='mgr.14100 172.21.15.5:0/2276580851' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T06:32:53.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:52 smithi005 ceph-mon[27798]: Manager daemon x is now available 2024-04-24T06:32:53.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:52 smithi005 ceph-mon[27798]: from='mgr.14100 172.21.15.5:0/2276580851' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T06:32:53.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:52 smithi005 ceph-mon[27798]: from='mgr.14100 172.21.15.5:0/2276580851' entity='mgr.x' 2024-04-24T06:32:53.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:52 smithi005 ceph-mon[27798]: from='mgr.14100 172.21.15.5:0/2276580851' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T06:32:53.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:52 smithi005 ceph-mon[27798]: from='mgr.14100 172.21.15.5:0/2276580851' entity='mgr.x' 2024-04-24T06:32:53.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:52 smithi005 ceph-mon[27798]: from='mgr.14100 172.21.15.5:0/2276580851' entity='mgr.x' 2024-04-24T06:32:55.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:54 smithi005 ceph-mon[27798]: mgrmap e3: x(active, since 1.02494s) 2024-04-24T06:32:55.417 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-04-24T06:32:55.418 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-04-24T06:32:55.418 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsid": "43e9bad0-0204-11ef-bc93-c7b262605968", 2024-04-24T06:32:55.418 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T06:32:55.418 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T06:32:55.419 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T06:32:55.419 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T06:32:55.420 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-24T06:32:55.420 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T06:32:55.420 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T06:32:55.420 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 0 2024-04-24T06:32:55.421 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-04-24T06:32:55.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T06:32:55.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "a" 2024-04-24T06:32:55.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-04-24T06:32:55.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-04-24T06:32:55.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T06:32:55.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T06:32:55.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-24T06:32:55.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T06:32:55.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-24T06:32:55.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T06:32:55.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T06:32:55.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T06:32:55.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T06:32:55.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T06:32:55.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T06:32:55.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T06:32:55.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T06:32:55.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-24T06:32:55.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T06:32:55.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T06:32:55.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T06:32:55.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T06:32:55.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T06:32:55.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T06:32:55.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T06:32:55.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T06:32:55.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T06:32:55.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-24T06:32:55.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T06:32:55.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T06:32:55.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T06:32:55.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T06:32:55.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-24T06:32:55.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T06:32:55.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-24T06:32:55.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T06:32:55.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T06:32:55.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T06:32:55.434 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T06:32:55.434 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T06:32:55.434 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-04-24T06:32:55.434 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T06:32:55.434 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-24T06:32:55.434 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T06:32:55.434 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T06:32:55.434 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T06:32:26.341159+0000", 2024-04-24T06:32:55.434 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T06:32:55.434 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-04-24T06:32:55.434 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T06:32:55.434 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-04-24T06:32:55.434 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.407+0000 7fa5f2e00700 1 Processor -- start 2024-04-24T06:32:55.435 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.408+0000 7fa5f2e00700 1 -- start start 2024-04-24T06:32:55.435 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.408+0000 7fa5f2e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ec07deb0 0x7fa5ec07c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:55.435 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.408+0000 7fa5f2e00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ec07c930 con 0x7fa5ec07deb0 2024-04-24T06:32:55.435 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.408+0000 7fa5f1a00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ec07deb0 0x7fa5ec07c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:55.435 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.408+0000 7fa5f1a00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ec07deb0 0x7fa5ec07c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44786/0 (socket says 172.21.15.5:44786) 2024-04-24T06:32:55.435 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.409+0000 7fa5f1a00700 1 -- 172.21.15.5:0/3310153605 learned_addr learned my addr 172.21.15.5:0/3310153605 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:32:55.435 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.409+0000 7fa5f1a00700 1 -- 172.21.15.5:0/3310153605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5ec07cab0 con 0x7fa5ec07deb0 2024-04-24T06:32:55.435 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.409+0000 7fa5f1a00700 1 --2- 172.21.15.5:0/3310153605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ec07deb0 0x7fa5ec07c360 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa5e00098a0 tx=0x7fa5e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a3690007f763e2f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:55.435 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.410+0000 7fa5ebe00700 1 -- 172.21.15.5:0/3310153605 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5e0004160 con 0x7fa5ec07deb0 2024-04-24T06:32:55.435 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.410+0000 7fa5ebe00700 1 -- 172.21.15.5:0/3310153605 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa5e0004300 con 0x7fa5ec07deb0 2024-04-24T06:32:55.435 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.410+0000 7fa5ebe00700 1 -- 172.21.15.5:0/3310153605 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5e0011b50 con 0x7fa5ec07deb0 2024-04-24T06:32:55.435 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.410+0000 7fa5f2e00700 1 -- 172.21.15.5:0/3310153605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ec07deb0 msgr2=0x7fa5ec07c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:55.435 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.410+0000 7fa5f2e00700 1 --2- 172.21.15.5:0/3310153605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ec07deb0 0x7fa5ec07c360 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa5e00098a0 tx=0x7fa5e0009bb0 comp rx=0 tx=0).stop 2024-04-24T06:32:55.436 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.411+0000 7fa5f2e00700 1 -- 172.21.15.5:0/3310153605 shutdown_connections 2024-04-24T06:32:55.436 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.411+0000 7fa5f2e00700 1 --2- 172.21.15.5:0/3310153605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ec07deb0 0x7fa5ec07c360 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:55.436 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.411+0000 7fa5f2e00700 1 -- 172.21.15.5:0/3310153605 >> 172.21.15.5:0/3310153605 conn(0x7fa5ec103b50 msgr2=0x7fa5ec105f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:55.436 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.411+0000 7fa5f2e00700 1 -- 172.21.15.5:0/3310153605 shutdown_connections 2024-04-24T06:32:55.436 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.411+0000 7fa5f2e00700 1 -- 172.21.15.5:0/3310153605 wait complete. 2024-04-24T06:32:55.436 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.412+0000 7fa5f2e00700 1 Processor -- start 2024-04-24T06:32:55.436 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.412+0000 7fa5f2e00700 1 -- start start 2024-04-24T06:32:55.436 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.412+0000 7fa5f2e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ec07deb0 0x7fa5ec19e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:55.436 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.412+0000 7fa5f2e00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ec10eae0 con 0x7fa5ec07deb0 2024-04-24T06:32:55.436 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.412+0000 7fa5f1a00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ec07deb0 0x7fa5ec19e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:55.436 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.413+0000 7fa5f1a00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ec07deb0 0x7fa5ec19e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44790/0 (socket says 172.21.15.5:44790) 2024-04-24T06:32:55.436 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.413+0000 7fa5f1a00700 1 -- 172.21.15.5:0/2049257290 learned_addr learned my addr 172.21.15.5:0/2049257290 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:32:55.436 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.413+0000 7fa5f1a00700 1 -- 172.21.15.5:0/2049257290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5ec19ebf0 con 0x7fa5ec07deb0 2024-04-24T06:32:55.436 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.413+0000 7fa5f1a00700 1 --2- 172.21.15.5:0/2049257290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ec07deb0 0x7fa5ec19e6b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0011b60 tx=0x7fa5e0011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:55.436 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.414+0000 7fa5ea000700 1 -- 172.21.15.5:0/2049257290 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5e0003970 con 0x7fa5ec07deb0 2024-04-24T06:32:55.437 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.414+0000 7fa5ea000700 1 -- 172.21.15.5:0/2049257290 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa5e0003b10 con 0x7fa5ec07deb0 2024-04-24T06:32:55.437 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.414+0000 7fa5ea000700 1 -- 172.21.15.5:0/2049257290 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5e0024610 con 0x7fa5ec07deb0 2024-04-24T06:32:55.437 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.414+0000 7fa5f2e00700 1 -- 172.21.15.5:0/2049257290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5ec19ee80 con 0x7fa5ec07deb0 2024-04-24T06:32:55.437 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.414+0000 7fa5f2e00700 1 -- 172.21.15.5:0/2049257290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5ec1a1b70 con 0x7fa5ec07deb0 2024-04-24T06:32:55.437 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.415+0000 7fa5ea000700 1 -- 172.21.15.5:0/2049257290 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 3) ==== 46441+0+0 (secure 0 0 0) 0x7fa5e001a9f0 con 0x7fa5ec07deb0 2024-04-24T06:32:55.437 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.415+0000 7fa5f2e00700 1 -- 172.21.15.5:0/2049257290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5ec069c50 con 0x7fa5ec07deb0 2024-04-24T06:32:55.437 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.415+0000 7fa5ea000700 1 --2- 172.21.15.5:0/2049257290 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7fa5d4039490 0x7fa5d403b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:55.437 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.415+0000 7fa5ea000700 1 -- 172.21.15.5:0/2049257290 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa5e0011e00 con 0x7fa5ec07deb0 2024-04-24T06:32:55.437 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.416+0000 7fa5f1000700 1 --2- 172.21.15.5:0/2049257290 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7fa5d4039490 0x7fa5d403b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:55.437 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.416+0000 7fa5f1000700 1 --2- 172.21.15.5:0/2049257290 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7fa5d4039490 0x7fa5d403b950 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8006fd0 tx=0x7fa5d8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:55.437 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.422+0000 7fa5ea000700 1 -- 172.21.15.5:0/2049257290 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa5e0024c40 con 0x7fa5ec07deb0 2024-04-24T06:32:55.437 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.653+0000 7fa5ea000700 1 -- 172.21.15.5:0/2049257290 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fa5e000b3e0 con 0x7fa5ec07deb0 2024-04-24T06:32:55.437 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.746+0000 7fa5f2e00700 1 -- 172.21.15.5:0/2049257290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa5ec1a1d40 con 0x7fa5ec07deb0 2024-04-24T06:32:55.438 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.748+0000 7fa5ea000700 1 -- 172.21.15.5:0/2049257290 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fa5ec1a1d40 con 0x7fa5ec07deb0 2024-04-24T06:32:55.438 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.750+0000 7fa5f2e00700 1 -- 172.21.15.5:0/2049257290 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7fa5d4039490 msgr2=0x7fa5d403b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:55.438 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.750+0000 7fa5f2e00700 1 --2- 172.21.15.5:0/2049257290 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7fa5d4039490 0x7fa5d403b950 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8006fd0 tx=0x7fa5d8006e40 comp rx=0 tx=0).stop 2024-04-24T06:32:55.438 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.750+0000 7fa5f2e00700 1 -- 172.21.15.5:0/2049257290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ec07deb0 msgr2=0x7fa5ec19e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:55.438 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.750+0000 7fa5f2e00700 1 --2- 172.21.15.5:0/2049257290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ec07deb0 0x7fa5ec19e6b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0011b60 tx=0x7fa5e0011c40 comp rx=0 tx=0).stop 2024-04-24T06:32:55.438 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.751+0000 7fa5f2e00700 1 -- 172.21.15.5:0/2049257290 shutdown_connections 2024-04-24T06:32:55.438 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.751+0000 7fa5f2e00700 1 --2- 172.21.15.5:0/2049257290 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7fa5d4039490 0x7fa5d403b950 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:55.438 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.751+0000 7fa5f2e00700 1 --2- 172.21.15.5:0/2049257290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5ec07deb0 0x7fa5ec19e6b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:55.438 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.751+0000 7fa5f2e00700 1 -- 172.21.15.5:0/2049257290 >> 172.21.15.5:0/2049257290 conn(0x7fa5ec103b50 msgr2=0x7fa5ec104730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:55.438 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.751+0000 7fa5f2e00700 1 -- 172.21.15.5:0/2049257290 shutdown_connections 2024-04-24T06:32:55.438 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:54.751+0000 7fa5f2e00700 1 -- 172.21.15.5:0/2049257290 wait complete. 2024-04-24T06:32:55.438 INFO:teuthology.orchestra.run.smithi005.stdout:mgr is available 2024-04-24T06:32:55.950 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:55 smithi005 ceph-mon[27798]: mgrmap e4: x(active, since 2s) 2024-04-24T06:32:55.950 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:55 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/2049257290' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T06:32:56.945 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:56 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1724915819' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-24T06:32:56.957 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-04-24T06:32:56.958 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [global] 2024-04-24T06:32:56.958 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout fsid = 43e9bad0-0204-11ef-bc93-c7b262605968 2024-04-24T06:32:56.958 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-24T06:32:56.959 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-24T06:32:56.959 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-24T06:32:56.959 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-24T06:32:56.960 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-04-24T06:32:56.960 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-24T06:32:56.960 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-24T06:32:56.960 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-04-24T06:32:56.960 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [osd] 2024-04-24T06:32:56.960 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-24T06:32:56.961 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-24T06:32:56.961 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-24T06:32:56.961 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-24T06:32:56.961 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-24T06:32:56.962 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-24T06:32:56.962 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.026+0000 7fd864c00700 1 Processor -- start 2024-04-24T06:32:56.962 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.026+0000 7fd864c00700 1 -- start start 2024-04-24T06:32:56.963 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.026+0000 7fd864c00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd860110090 0x7fd8601104b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:56.963 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.026+0000 7fd864c00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd860110a80 con 0x7fd860110090 2024-04-24T06:32:56.963 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.027+0000 7fd85f400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd860110090 0x7fd8601104b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:56.963 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.027+0000 7fd85f400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd860110090 0x7fd8601104b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44806/0 (socket says 172.21.15.5:44806) 2024-04-24T06:32:56.964 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.027+0000 7fd85f400700 1 -- 172.21.15.5:0/365870327 learned_addr learned my addr 172.21.15.5:0/365870327 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:32:56.964 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.027+0000 7fd85f400700 1 -- 172.21.15.5:0/365870327 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd86010ecf0 con 0x7fd860110090 2024-04-24T06:32:56.964 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.028+0000 7fd85f400700 1 --2- 172.21.15.5:0/365870327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd860110090 0x7fd8601104b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd8500098a0 tx=0x7fd850009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ce20ee3e715080 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:56.964 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.028+0000 7fd85e000700 1 -- 172.21.15.5:0/365870327 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd850004160 con 0x7fd860110090 2024-04-24T06:32:56.964 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.028+0000 7fd85e000700 1 -- 172.21.15.5:0/365870327 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd850004300 con 0x7fd860110090 2024-04-24T06:32:56.965 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.028+0000 7fd85e000700 1 -- 172.21.15.5:0/365870327 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd850011b50 con 0x7fd860110090 2024-04-24T06:32:56.965 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.029+0000 7fd864c00700 1 -- 172.21.15.5:0/365870327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd860110090 msgr2=0x7fd8601104b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:56.965 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.029+0000 7fd864c00700 1 --2- 172.21.15.5:0/365870327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd860110090 0x7fd8601104b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd8500098a0 tx=0x7fd850009bb0 comp rx=0 tx=0).stop 2024-04-24T06:32:56.965 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.029+0000 7fd864c00700 1 -- 172.21.15.5:0/365870327 shutdown_connections 2024-04-24T06:32:56.966 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.029+0000 7fd864c00700 1 --2- 172.21.15.5:0/365870327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd860110090 0x7fd8601104b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:56.966 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.029+0000 7fd864c00700 1 -- 172.21.15.5:0/365870327 >> 172.21.15.5:0/365870327 conn(0x7fd86007dba0 msgr2=0x7fd86007ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:56.966 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.029+0000 7fd864c00700 1 -- 172.21.15.5:0/365870327 shutdown_connections 2024-04-24T06:32:56.966 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.029+0000 7fd864c00700 1 -- 172.21.15.5:0/365870327 wait complete. 2024-04-24T06:32:56.966 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.030+0000 7fd864c00700 1 Processor -- start 2024-04-24T06:32:56.966 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.030+0000 7fd864c00700 1 -- start start 2024-04-24T06:32:56.967 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.030+0000 7fd864c00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd860110090 0x7fd8601a2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:56.967 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.030+0000 7fd864c00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd8601130b0 con 0x7fd860110090 2024-04-24T06:32:56.967 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.031+0000 7fd85f400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd860110090 0x7fd8601a2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:56.967 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.031+0000 7fd85f400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd860110090 0x7fd8601a2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44820/0 (socket says 172.21.15.5:44820) 2024-04-24T06:32:56.968 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.031+0000 7fd85f400700 1 -- 172.21.15.5:0/1724915819 learned_addr learned my addr 172.21.15.5:0/1724915819 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:32:56.968 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.031+0000 7fd85f400700 1 -- 172.21.15.5:0/1724915819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8601a2fd0 con 0x7fd860110090 2024-04-24T06:32:56.968 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.032+0000 7fd85f400700 1 --2- 172.21.15.5:0/1724915819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd860110090 0x7fd8601a2a90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd850011b60 tx=0x7fd850011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:56.968 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.032+0000 7fd857e00700 1 -- 172.21.15.5:0/1724915819 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd850003970 con 0x7fd860110090 2024-04-24T06:32:56.969 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.032+0000 7fd857e00700 1 -- 172.21.15.5:0/1724915819 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd850003b10 con 0x7fd860110090 2024-04-24T06:32:56.969 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.032+0000 7fd857e00700 1 -- 172.21.15.5:0/1724915819 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd850024610 con 0x7fd860110090 2024-04-24T06:32:56.969 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.032+0000 7fd864c00700 1 -- 172.21.15.5:0/1724915819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8601a3260 con 0x7fd860110090 2024-04-24T06:32:56.970 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.032+0000 7fd864c00700 1 -- 172.21.15.5:0/1724915819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8601a5f50 con 0x7fd860110090 2024-04-24T06:32:56.970 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.033+0000 7fd857e00700 1 -- 172.21.15.5:0/1724915819 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fd85001a9f0 con 0x7fd860110090 2024-04-24T06:32:56.970 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.033+0000 7fd864c00700 1 -- 172.21.15.5:0/1724915819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd860069c50 con 0x7fd860110090 2024-04-24T06:32:56.970 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.034+0000 7fd857e00700 1 --2- 172.21.15.5:0/1724915819 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7fd8440395b0 0x7fd84403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:56.972 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.034+0000 7fd857e00700 1 -- 172.21.15.5:0/1724915819 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd850011e00 con 0x7fd860110090 2024-04-24T06:32:56.972 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.034+0000 7fd85ea00700 1 --2- 172.21.15.5:0/1724915819 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7fd8440395b0 0x7fd84403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:56.972 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.035+0000 7fd85ea00700 1 --2- 172.21.15.5:0/1724915819 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7fd8440395b0 0x7fd84403ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd84c006fd0 tx=0x7fd84c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:56.972 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.040+0000 7fd857e00700 1 -- 172.21.15.5:0/1724915819 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd85000b3e0 con 0x7fd860110090 2024-04-24T06:32:56.972 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.268+0000 7fd864c00700 1 -- 172.21.15.5:0/1724915819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fd8601a6120 con 0x7fd860110090 2024-04-24T06:32:56.972 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.269+0000 7fd857e00700 1 -- 172.21.15.5:0/1724915819 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+400 (secure 0 0 0) 0x7fd8601a6120 con 0x7fd860110090 2024-04-24T06:32:56.972 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.272+0000 7fd864c00700 1 -- 172.21.15.5:0/1724915819 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7fd8440395b0 msgr2=0x7fd84403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:56.972 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.272+0000 7fd864c00700 1 --2- 172.21.15.5:0/1724915819 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7fd8440395b0 0x7fd84403ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd84c006fd0 tx=0x7fd84c006e40 comp rx=0 tx=0).stop 2024-04-24T06:32:56.972 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.273+0000 7fd864c00700 1 -- 172.21.15.5:0/1724915819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd860110090 msgr2=0x7fd8601a2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:56.972 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.273+0000 7fd864c00700 1 --2- 172.21.15.5:0/1724915819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd860110090 0x7fd8601a2a90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd850011b60 tx=0x7fd850011c40 comp rx=0 tx=0).stop 2024-04-24T06:32:56.972 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.273+0000 7fd864c00700 1 -- 172.21.15.5:0/1724915819 shutdown_connections 2024-04-24T06:32:56.972 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.273+0000 7fd864c00700 1 --2- 172.21.15.5:0/1724915819 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7fd8440395b0 0x7fd84403ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:56.973 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.273+0000 7fd864c00700 1 --2- 172.21.15.5:0/1724915819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd860110090 0x7fd8601a2a90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:56.973 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.273+0000 7fd864c00700 1 -- 172.21.15.5:0/1724915819 >> 172.21.15.5:0/1724915819 conn(0x7fd86007dba0 msgr2=0x7fd86007e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:56.973 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.273+0000 7fd864c00700 1 -- 172.21.15.5:0/1724915819 shutdown_connections 2024-04-24T06:32:56.973 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:56.273+0000 7fd864c00700 1 -- 172.21.15.5:0/1724915819 wait complete. 2024-04-24T06:32:56.973 INFO:teuthology.orchestra.run.smithi005.stdout:Enabling cephadm module... 2024-04-24T06:32:58.196 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:57 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/2819881541' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-24T06:32:58.196 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:32:58 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: ignoring --setuser ceph since I am not root 2024-04-24T06:32:58.196 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:32:58 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: ignoring --setgroup ceph since I am not root 2024-04-24T06:32:58.196 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:32:58 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:32:58.193+0000 7fd4ead0d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T06:32:58.506 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:32:58 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:32:58.438+0000 7fd4ead0d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T06:32:58.670 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.556+0000 7ff559200700 1 Processor -- start 2024-04-24T06:32:58.670 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.557+0000 7ff559200700 1 -- start start 2024-04-24T06:32:58.670 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.557+0000 7ff559200700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff554108700 0x7ff554108b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:58.670 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.557+0000 7ff559200700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff5541090f0 con 0x7ff554108700 2024-04-24T06:32:58.671 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.558+0000 7ff551e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff554108700 0x7ff554108b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:58.671 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.558+0000 7ff551e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff554108700 0x7ff554108b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44828/0 (socket says 172.21.15.5:44828) 2024-04-24T06:32:58.671 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.558+0000 7ff551e00700 1 -- 172.21.15.5:0/3760171796 learned_addr learned my addr 172.21.15.5:0/3760171796 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:32:58.671 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.558+0000 7ff551e00700 1 -- 172.21.15.5:0/3760171796 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5541098b0 con 0x7ff554108700 2024-04-24T06:32:58.671 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.558+0000 7ff551e00700 1 --2- 172.21.15.5:0/3760171796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff554108700 0x7ff554108b20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff5440098a0 tx=0x7ff544009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=775ab8f9ece328f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:58.671 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.559+0000 7ff550a00700 1 -- 172.21.15.5:0/3760171796 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff544004160 con 0x7ff554108700 2024-04-24T06:32:58.671 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.559+0000 7ff550a00700 1 -- 172.21.15.5:0/3760171796 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff544004300 con 0x7ff554108700 2024-04-24T06:32:58.671 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.559+0000 7ff550a00700 1 -- 172.21.15.5:0/3760171796 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff544011b50 con 0x7ff554108700 2024-04-24T06:32:58.671 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.559+0000 7ff559200700 1 -- 172.21.15.5:0/3760171796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff554108700 msgr2=0x7ff554108b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:58.671 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.559+0000 7ff559200700 1 --2- 172.21.15.5:0/3760171796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff554108700 0x7ff554108b20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff5440098a0 tx=0x7ff544009bb0 comp rx=0 tx=0).stop 2024-04-24T06:32:58.671 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.560+0000 7ff559200700 1 -- 172.21.15.5:0/3760171796 shutdown_connections 2024-04-24T06:32:58.671 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.560+0000 7ff559200700 1 --2- 172.21.15.5:0/3760171796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff554108700 0x7ff554108b20 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:58.671 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.560+0000 7ff559200700 1 -- 172.21.15.5:0/3760171796 >> 172.21.15.5:0/3760171796 conn(0x7ff554103d40 msgr2=0x7ff554106160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:58.671 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.560+0000 7ff559200700 1 -- 172.21.15.5:0/3760171796 shutdown_connections 2024-04-24T06:32:58.672 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.560+0000 7ff559200700 1 -- 172.21.15.5:0/3760171796 wait complete. 2024-04-24T06:32:58.672 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.561+0000 7ff559200700 1 Processor -- start 2024-04-24T06:32:58.672 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.561+0000 7ff559200700 1 -- start start 2024-04-24T06:32:58.672 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.561+0000 7ff559200700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff554108700 0x7ff55419e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:58.672 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.561+0000 7ff559200700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff55410cc60 con 0x7ff554108700 2024-04-24T06:32:58.672 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.562+0000 7ff551e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff554108700 0x7ff55419e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:58.672 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.562+0000 7ff551e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff554108700 0x7ff55419e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44832/0 (socket says 172.21.15.5:44832) 2024-04-24T06:32:58.672 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.562+0000 7ff551e00700 1 -- 172.21.15.5:0/2819881541 learned_addr learned my addr 172.21.15.5:0/2819881541 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:32:58.672 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.562+0000 7ff551e00700 1 -- 172.21.15.5:0/2819881541 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff55419ee10 con 0x7ff554108700 2024-04-24T06:32:58.672 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.562+0000 7ff551e00700 1 --2- 172.21.15.5:0/2819881541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff554108700 0x7ff55419e8d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff544011850 tx=0x7ff544011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:58.672 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.563+0000 7ff54aa00700 1 -- 172.21.15.5:0/2819881541 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff544003af0 con 0x7ff554108700 2024-04-24T06:32:58.672 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.563+0000 7ff54aa00700 1 -- 172.21.15.5:0/2819881541 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff544020070 con 0x7ff554108700 2024-04-24T06:32:58.672 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.563+0000 7ff559200700 1 -- 172.21.15.5:0/2819881541 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff55419f0a0 con 0x7ff554108700 2024-04-24T06:32:58.673 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.563+0000 7ff54aa00700 1 -- 172.21.15.5:0/2819881541 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5440195a0 con 0x7ff554108700 2024-04-24T06:32:58.673 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.563+0000 7ff559200700 1 -- 172.21.15.5:0/2819881541 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5541a1d90 con 0x7ff554108700 2024-04-24T06:32:58.673 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.564+0000 7ff559200700 1 -- 172.21.15.5:0/2819881541 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff554064150 con 0x7ff554108700 2024-04-24T06:32:58.673 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.564+0000 7ff54aa00700 1 -- 172.21.15.5:0/2819881541 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7ff5440198a0 con 0x7ff554108700 2024-04-24T06:32:58.673 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.564+0000 7ff54aa00700 1 --2- 172.21.15.5:0/2819881541 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7ff538039560 0x7ff53803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:32:58.673 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.564+0000 7ff54aa00700 1 -- 172.21.15.5:0/2819881541 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff544011e00 con 0x7ff554108700 2024-04-24T06:32:58.673 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.565+0000 7ff551400700 1 --2- 172.21.15.5:0/2819881541 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7ff538039560 0x7ff53803ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:32:58.673 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.565+0000 7ff551400700 1 --2- 172.21.15.5:0/2819881541 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7ff538039560 0x7ff53803ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff540006fd0 tx=0x7ff540006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:32:58.673 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.570+0000 7ff54aa00700 1 -- 172.21.15.5:0/2819881541 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff54400e260 con 0x7ff554108700 2024-04-24T06:32:58.673 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.859+0000 7ff559200700 1 -- 172.21.15.5:0/2819881541 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7ff5541a1f60 con 0x7ff554108700 2024-04-24T06:32:58.673 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.913+0000 7ff54aa00700 1 -- 172.21.15.5:0/2819881541 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7ff5541a1f60 con 0x7ff554108700 2024-04-24T06:32:58.673 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.916+0000 7ff54aa00700 1 -- 172.21.15.5:0/2819881541 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7ff54401a430 con 0x7ff554108700 2024-04-24T06:32:58.673 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.918+0000 7ff559200700 1 -- 172.21.15.5:0/2819881541 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7ff538039560 msgr2=0x7ff53803ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:58.673 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.918+0000 7ff559200700 1 --2- 172.21.15.5:0/2819881541 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7ff538039560 0x7ff53803ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff540006fd0 tx=0x7ff540006e40 comp rx=0 tx=0).stop 2024-04-24T06:32:58.674 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.918+0000 7ff559200700 1 -- 172.21.15.5:0/2819881541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff554108700 msgr2=0x7ff55419e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:32:58.674 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.918+0000 7ff559200700 1 --2- 172.21.15.5:0/2819881541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff554108700 0x7ff55419e8d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff544011850 tx=0x7ff544011c10 comp rx=0 tx=0).stop 2024-04-24T06:32:58.674 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.919+0000 7ff559200700 1 -- 172.21.15.5:0/2819881541 shutdown_connections 2024-04-24T06:32:58.674 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.919+0000 7ff559200700 1 --2- 172.21.15.5:0/2819881541 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7ff538039560 0x7ff53803ba20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:58.674 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.919+0000 7ff559200700 1 --2- 172.21.15.5:0/2819881541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff554108700 0x7ff55419e8d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:32:58.674 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.919+0000 7ff559200700 1 -- 172.21.15.5:0/2819881541 >> 172.21.15.5:0/2819881541 conn(0x7ff554103d40 msgr2=0x7ff5541048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:32:58.674 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.919+0000 7ff559200700 1 -- 172.21.15.5:0/2819881541 shutdown_connections 2024-04-24T06:32:58.674 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:57.919+0000 7ff559200700 1 -- 172.21.15.5:0/2819881541 wait complete. 2024-04-24T06:32:58.924 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:32:58 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:32:58.570+0000 7fd4ead0d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T06:32:59.196 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:58 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/2819881541' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-24T06:32:59.196 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:58 smithi005 ceph-mon[27798]: mgrmap e5: x(active, since 5s) 2024-04-24T06:32:59.196 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:32:58 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:32:58.985+0000 7fd4ead0d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T06:32:59.196 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:32:59 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:32:59.104+0000 7fd4ead0d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T06:33:00.150 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:32:59 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/648304177' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-24T06:33:00.163 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-04-24T06:33:00.163 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-24T06:33:00.163 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-24T06:33:00.163 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-24T06:33:00.163 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-24T06:33:00.163 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-04-24T06:33:00.163 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.257+0000 7f8e7c400700 1 Processor -- start 2024-04-24T06:33:00.163 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.257+0000 7f8e7c400700 1 -- start start 2024-04-24T06:33:00.164 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.258+0000 7f8e7c400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e74108700 0x7f8e74108b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:00.164 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.258+0000 7f8e7c400700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8e741090f0 con 0x7f8e74108700 2024-04-24T06:33:00.164 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.258+0000 7f8e79a00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e74108700 0x7f8e74108b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:00.164 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.258+0000 7f8e79a00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e74108700 0x7f8e74108b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44866/0 (socket says 172.21.15.5:44866) 2024-04-24T06:33:00.164 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.258+0000 7f8e79a00700 1 -- 172.21.15.5:0/1244794436 learned_addr learned my addr 172.21.15.5:0/1244794436 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:33:00.164 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.259+0000 7f8e79a00700 1 -- 172.21.15.5:0/1244794436 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e741098b0 con 0x7f8e74108700 2024-04-24T06:33:00.164 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.259+0000 7f8e79a00700 1 --2- 172.21.15.5:0/1244794436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e74108700 0x7f8e74108b20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8e680098a0 tx=0x7f8e68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43049248b663c31 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:00.164 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.259+0000 7f8e73e00700 1 -- 172.21.15.5:0/1244794436 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e68004160 con 0x7f8e74108700 2024-04-24T06:33:00.164 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.260+0000 7f8e73e00700 1 -- 172.21.15.5:0/1244794436 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8e68004300 con 0x7f8e74108700 2024-04-24T06:33:00.164 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.260+0000 7f8e73e00700 1 -- 172.21.15.5:0/1244794436 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e68011b50 con 0x7f8e74108700 2024-04-24T06:33:00.164 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.260+0000 7f8e7c400700 1 -- 172.21.15.5:0/1244794436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e74108700 msgr2=0x7f8e74108b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:00.164 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.260+0000 7f8e7c400700 1 --2- 172.21.15.5:0/1244794436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e74108700 0x7f8e74108b20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8e680098a0 tx=0x7f8e68009bb0 comp rx=0 tx=0).stop 2024-04-24T06:33:00.165 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.261+0000 7f8e7c400700 1 -- 172.21.15.5:0/1244794436 shutdown_connections 2024-04-24T06:33:00.165 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.261+0000 7f8e7c400700 1 --2- 172.21.15.5:0/1244794436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e74108700 0x7f8e74108b20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:00.165 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.261+0000 7f8e7c400700 1 -- 172.21.15.5:0/1244794436 >> 172.21.15.5:0/1244794436 conn(0x7f8e74103d40 msgr2=0x7f8e74106160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:00.165 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.261+0000 7f8e7c400700 1 -- 172.21.15.5:0/1244794436 shutdown_connections 2024-04-24T06:33:00.165 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.261+0000 7f8e7c400700 1 -- 172.21.15.5:0/1244794436 wait complete. 2024-04-24T06:33:00.165 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.262+0000 7f8e7c400700 1 Processor -- start 2024-04-24T06:33:00.165 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.262+0000 7f8e7c400700 1 -- start start 2024-04-24T06:33:00.165 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.262+0000 7f8e7c400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e7419c8d0 0x7f8e7419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:00.165 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.262+0000 7f8e7c400700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7410cc60 con 0x7f8e7419c8d0 2024-04-24T06:33:00.165 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.263+0000 7f8e79a00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e7419c8d0 0x7f8e7419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:00.165 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.263+0000 7f8e79a00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e7419c8d0 0x7f8e7419ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44880/0 (socket says 172.21.15.5:44880) 2024-04-24T06:33:00.166 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.263+0000 7f8e79a00700 1 -- 172.21.15.5:0/648304177 learned_addr learned my addr 172.21.15.5:0/648304177 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:33:00.166 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.263+0000 7f8e79a00700 1 -- 172.21.15.5:0/648304177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e7419d230 con 0x7f8e7419c8d0 2024-04-24T06:33:00.166 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.264+0000 7f8e79a00700 1 --2- 172.21.15.5:0/648304177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e7419c8d0 0x7f8e7419ccf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8e68000c00 tx=0x7f8e680119e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:00.166 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.264+0000 7f8e72000700 1 -- 172.21.15.5:0/648304177 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e680038b0 con 0x7f8e7419c8d0 2024-04-24T06:33:00.166 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.264+0000 7f8e72000700 1 -- 172.21.15.5:0/648304177 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8e68003a50 con 0x7f8e7419c8d0 2024-04-24T06:33:00.166 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.264+0000 7f8e72000700 1 -- 172.21.15.5:0/648304177 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e6801b440 con 0x7f8e7419c8d0 2024-04-24T06:33:00.166 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.264+0000 7f8e7c400700 1 -- 172.21.15.5:0/648304177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e7419d4c0 con 0x7f8e7419c8d0 2024-04-24T06:33:00.166 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.264+0000 7f8e7c400700 1 -- 172.21.15.5:0/648304177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e7419ffa0 con 0x7f8e7419c8d0 2024-04-24T06:33:00.166 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.265+0000 7f8e67400700 1 -- 172.21.15.5:0/648304177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e74064150 con 0x7f8e7419c8d0 2024-04-24T06:33:00.167 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.266+0000 7f8e72000700 1 -- 172.21.15.5:0/648304177 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f8e6801a440 con 0x7f8e7419c8d0 2024-04-24T06:33:00.167 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.266+0000 7f8e72000700 1 --2- 172.21.15.5:0/648304177 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7f8e5c039600 0x7f8e5c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:00.167 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.266+0000 7f8e72000700 1 -- 172.21.15.5:0/648304177 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8e6804dad0 con 0x7f8e7419c8d0 2024-04-24T06:33:00.167 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.266+0000 7f8e79000700 1 -- 172.21.15.5:0/648304177 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7f8e5c039600 msgr2=0x7f8e5c03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2413496885 2024-04-24T06:33:00.167 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.266+0000 7f8e79000700 1 --2- 172.21.15.5:0/648304177 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7f8e5c039600 0x7f8e5c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T06:33:00.167 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.272+0000 7f8e72000700 1 -- 172.21.15.5:0/648304177 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8e6804d4f0 con 0x7f8e7419c8d0 2024-04-24T06:33:00.167 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.467+0000 7f8e79000700 1 -- 172.21.15.5:0/648304177 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7f8e5c039600 msgr2=0x7f8e5c03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2413496885 2024-04-24T06:33:00.168 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.467+0000 7f8e79000700 1 --2- 172.21.15.5:0/648304177 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7f8e5c039600 0x7f8e5c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T06:33:00.168 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.559+0000 7f8e67400700 1 -- 172.21.15.5:0/648304177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f8e74063700 con 0x7f8e7419c8d0 2024-04-24T06:33:00.168 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.560+0000 7f8e72000700 1 -- 172.21.15.5:0/648304177 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f8e68046e20 con 0x7f8e7419c8d0 2024-04-24T06:33:00.168 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.563+0000 7f8e67400700 1 -- 172.21.15.5:0/648304177 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7f8e5c039600 msgr2=0x7f8e5c03bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T06:33:00.168 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.563+0000 7f8e67400700 1 --2- 172.21.15.5:0/648304177 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7f8e5c039600 0x7f8e5c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:00.168 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.563+0000 7f8e67400700 1 -- 172.21.15.5:0/648304177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e7419c8d0 msgr2=0x7f8e7419ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:00.168 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.563+0000 7f8e67400700 1 --2- 172.21.15.5:0/648304177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e7419c8d0 0x7f8e7419ccf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8e68000c00 tx=0x7f8e680119e0 comp rx=0 tx=0).stop 2024-04-24T06:33:00.168 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.563+0000 7f8e67400700 1 -- 172.21.15.5:0/648304177 shutdown_connections 2024-04-24T06:33:00.168 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.563+0000 7f8e67400700 1 --2- 172.21.15.5:0/648304177 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7f8e5c039600 0x7f8e5c03bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:00.168 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.563+0000 7f8e67400700 1 --2- 172.21.15.5:0/648304177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e7419c8d0 0x7f8e7419ccf0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:00.168 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.563+0000 7f8e67400700 1 -- 172.21.15.5:0/648304177 >> 172.21.15.5:0/648304177 conn(0x7f8e74103d40 msgr2=0x7f8e74104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:00.168 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.563+0000 7f8e67400700 1 -- 172.21.15.5:0/648304177 shutdown_connections 2024-04-24T06:33:00.168 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:32:59.563+0000 7f8e67400700 1 -- 172.21.15.5:0/648304177 wait complete. 2024-04-24T06:33:00.169 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for the mgr to restart... 2024-04-24T06:33:00.169 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr epoch 5... 2024-04-24T06:33:01.006 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:00 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:00.722+0000 7fd4ead0d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T06:33:01.007 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:00 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:00.957+0000 7fd4ead0d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T06:33:01.506 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:01 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:01.177+0000 7fd4ead0d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T06:33:01.952 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:01 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:01.569+0000 7fd4ead0d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T06:33:01.953 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:01 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:01.694+0000 7fd4ead0d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T06:33:02.217 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:01 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:01.950+0000 7fd4ead0d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T06:33:02.217 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:02 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:02.215+0000 7fd4ead0d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T06:33:03.506 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:03 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:03.023+0000 7fd4ead0d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T06:33:03.995 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:03 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:03.590+0000 7fd4ead0d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T06:33:03.995 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:03 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:03.720+0000 7fd4ead0d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T06:33:03.995 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:03 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:03.993+0000 7fd4ead0d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T06:33:05.006 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:04 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:04.658+0000 7fd4ead0d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T06:33:05.006 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:04 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:04.785+0000 7fd4ead0d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T06:33:05.430 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:05 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:05.116+0000 7fd4ead0d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T06:33:05.430 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:05 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:05.301+0000 7fd4ead0d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T06:33:05.430 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:05 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:05.428+0000 7fd4ead0d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T06:33:05.756 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:05 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:05.582+0000 7fd4ead0d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T06:33:06.256 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:05 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:05.959+0000 7fd4ead0d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T06:33:07.006 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:06 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:06.744+0000 7fd4ead0d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T06:33:07.506 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:07 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:07.036+0000 7fd4ead0d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T06:33:07.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:07 smithi005 ceph-mon[27798]: Active manager daemon x restarted 2024-04-24T06:33:07.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:07 smithi005 ceph-mon[27798]: Activating manager daemon x 2024-04-24T06:33:07.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:07 smithi005 ceph-mon[27798]: osdmap e2: 0 total, 0 up, 0 in 2024-04-24T06:33:07.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:07 smithi005 ceph-mon[27798]: mgrmap e6: x(active, starting, since 0.00370392s) 2024-04-24T06:33:07.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:07 smithi005 ceph-mon[27798]: from='mgr.14120 172.21.15.5:0/2924182213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T06:33:07.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:07 smithi005 ceph-mon[27798]: from='mgr.14120 172.21.15.5:0/2924182213' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T06:33:07.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:07 smithi005 ceph-mon[27798]: from='mgr.14120 172.21.15.5:0/2924182213' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T06:33:07.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:07 smithi005 ceph-mon[27798]: from='mgr.14120 172.21.15.5:0/2924182213' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T06:33:07.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:07 smithi005 ceph-mon[27798]: from='mgr.14120 172.21.15.5:0/2924182213' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T06:33:07.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:07 smithi005 ceph-mon[27798]: Manager daemon x is now available 2024-04-24T06:33:08.402 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:08 smithi005 ceph-mon[27798]: Found migration_current of "None". Setting to last migration. 2024-04-24T06:33:08.402 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:08 smithi005 ceph-mon[27798]: from='mgr.14120 172.21.15.5:0/2924182213' entity='mgr.x' 2024-04-24T06:33:08.402 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:08 smithi005 ceph-mon[27798]: from='mgr.14120 172.21.15.5:0/2924182213' entity='mgr.x' 2024-04-24T06:33:08.402 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:08 smithi005 ceph-mon[27798]: from='mgr.14120 172.21.15.5:0/2924182213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:33:08.402 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:08 smithi005 ceph-mon[27798]: from='mgr.14120 172.21.15.5:0/2924182213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:33:08.402 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:08 smithi005 ceph-mon[27798]: from='mgr.14120 172.21.15.5:0/2924182213' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T06:33:08.403 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:08 smithi005 ceph-mon[27798]: from='mgr.14120 172.21.15.5:0/2924182213' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T06:33:08.403 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:08 smithi005 ceph-mon[27798]: from='mgr.14120 172.21.15.5:0/2924182213' entity='mgr.x' 2024-04-24T06:33:08.403 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:08 smithi005 ceph-mon[27798]: from='mgr.14120 172.21.15.5:0/2924182213' entity='mgr.x' 2024-04-24T06:33:08.403 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:08 smithi005 ceph-mon[27798]: [24/Apr/2024:06:33:08] ENGINE Bus STARTING 2024-04-24T06:33:08.403 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:08 smithi005 ceph-mon[27798]: mgrmap e7: x(active, since 1.00814s) 2024-04-24T06:33:08.734 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-04-24T06:33:08.735 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-24T06:33:08.735 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-24T06:33:08.735 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-04-24T06:33:08.735 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:00.747+0000 7f34f3e00700 1 Processor -- start 2024-04-24T06:33:08.735 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:00.747+0000 7f34f3e00700 1 -- start start 2024-04-24T06:33:08.735 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:00.748+0000 7f34f3e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34ec106f30 0x7f34ec109360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:08.735 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:00.748+0000 7f34f3e00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f34ec07bed0 con 0x7f34ec106f30 2024-04-24T06:33:08.735 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:00.748+0000 7f34f1400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34ec106f30 0x7f34ec109360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:08.735 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:00.748+0000 7f34f1400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34ec106f30 0x7f34ec109360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44882/0 (socket says 172.21.15.5:44882) 2024-04-24T06:33:08.735 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:00.748+0000 7f34f1400700 1 -- 172.21.15.5:0/2139188982 learned_addr learned my addr 172.21.15.5:0/2139188982 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:33:08.735 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:00.749+0000 7f34f1400700 1 -- 172.21.15.5:0/2139188982 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34ec1098a0 con 0x7f34ec106f30 2024-04-24T06:33:08.736 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:00.749+0000 7f34f1400700 1 --2- 172.21.15.5:0/2139188982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34ec106f30 0x7f34ec109360 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f34e00098a0 tx=0x7f34e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8730c09906e17d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:08.736 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:00.749+0000 7f34ebe00700 1 -- 172.21.15.5:0/2139188982 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f34e0004160 con 0x7f34ec106f30 2024-04-24T06:33:08.736 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:00.750+0000 7f34ebe00700 1 -- 172.21.15.5:0/2139188982 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f34e0004300 con 0x7f34ec106f30 2024-04-24T06:33:08.736 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:00.750+0000 7f34ebe00700 1 -- 172.21.15.5:0/2139188982 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f34e0011b50 con 0x7f34ec106f30 2024-04-24T06:33:08.736 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:00.750+0000 7f34f3e00700 1 -- 172.21.15.5:0/2139188982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34ec106f30 msgr2=0x7f34ec109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:08.736 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:00.750+0000 7f34f3e00700 1 --2- 172.21.15.5:0/2139188982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34ec106f30 0x7f34ec109360 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f34e00098a0 tx=0x7f34e0009bb0 comp rx=0 tx=0).stop 2024-04-24T06:33:08.736 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:00.751+0000 7f34f3e00700 1 -- 172.21.15.5:0/2139188982 shutdown_connections 2024-04-24T06:33:08.736 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:00.751+0000 7f34f3e00700 1 --2- 172.21.15.5:0/2139188982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34ec106f30 0x7f34ec109360 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:08.736 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:00.751+0000 7f34f3e00700 1 -- 172.21.15.5:0/2139188982 >> 172.21.15.5:0/2139188982 conn(0x7f34ec102c30 msgr2=0x7f34ec105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:08.736 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:00.751+0000 7f34f3e00700 1 -- 172.21.15.5:0/2139188982 shutdown_connections 2024-04-24T06:33:08.736 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:00.751+0000 7f34f3e00700 1 -- 172.21.15.5:0/2139188982 wait complete. 2024-04-24T06:33:08.736 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:00.752+0000 7f34f3e00700 1 Processor -- start 2024-04-24T06:33:08.737 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:00.752+0000 7f34f3e00700 1 -- start start 2024-04-24T06:33:08.737 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:00.752+0000 7f34f3e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34ec106f30 0x7f34ec1a2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:08.737 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:00.752+0000 7f34f3e00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f34ec10a930 con 0x7f34ec106f30 2024-04-24T06:33:08.737 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:00.752+0000 7f34f1400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34ec106f30 0x7f34ec1a2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:08.737 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:00.752+0000 7f34f1400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34ec106f30 0x7f34ec1a2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44898/0 (socket says 172.21.15.5:44898) 2024-04-24T06:33:08.737 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:00.752+0000 7f34f1400700 1 -- 172.21.15.5:0/1483777807 learned_addr learned my addr 172.21.15.5:0/1483777807 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:33:08.737 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:00.753+0000 7f34f1400700 1 -- 172.21.15.5:0/1483777807 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34ec1a31f0 con 0x7f34ec106f30 2024-04-24T06:33:08.737 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:00.753+0000 7f34f1400700 1 --2- 172.21.15.5:0/1483777807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34ec106f30 0x7f34ec1a2cb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f34e0011b60 tx=0x7f34e0011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:08.737 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:00.753+0000 7f34ea000700 1 -- 172.21.15.5:0/1483777807 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f34e0003970 con 0x7f34ec106f30 2024-04-24T06:33:08.737 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:00.753+0000 7f34ea000700 1 -- 172.21.15.5:0/1483777807 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f34e0003b10 con 0x7f34ec106f30 2024-04-24T06:33:08.737 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:00.753+0000 7f34ea000700 1 -- 172.21.15.5:0/1483777807 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f34e0024610 con 0x7f34ec106f30 2024-04-24T06:33:08.737 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:00.753+0000 7f34f3e00700 1 -- 172.21.15.5:0/1483777807 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34ec1a3480 con 0x7f34ec106f30 2024-04-24T06:33:08.737 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:00.754+0000 7f34f3e00700 1 -- 172.21.15.5:0/1483777807 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34ec1a6170 con 0x7f34ec106f30 2024-04-24T06:33:08.737 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:00.755+0000 7f34ea000700 1 -- 172.21.15.5:0/1483777807 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f34e001a430 con 0x7f34ec106f30 2024-04-24T06:33:08.737 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:00.755+0000 7f34ea000700 1 --2- 172.21.15.5:0/1483777807 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7f34d40395b0 0x7f34d403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:08.738 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:00.755+0000 7f34ea000700 1 -- 172.21.15.5:0/1483777807 --> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f34d403c180 con 0x7f34d40395b0 2024-04-24T06:33:08.738 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:00.755+0000 7f34ea000700 1 -- 172.21.15.5:0/1483777807 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f34e0011e00 con 0x7f34ec106f30 2024-04-24T06:33:08.738 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:00.755+0000 7f34f0a00700 1 -- 172.21.15.5:0/1483777807 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7f34d40395b0 msgr2=0x7f34d403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2413496885 2024-04-24T06:33:08.738 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:00.755+0000 7f34f0a00700 1 --2- 172.21.15.5:0/1483777807 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7f34d40395b0 0x7f34d403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T06:33:08.738 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:00.956+0000 7f34f0a00700 1 -- 172.21.15.5:0/1483777807 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7f34d40395b0 msgr2=0x7f34d403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2413496885 2024-04-24T06:33:08.738 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:00.956+0000 7f34f0a00700 1 --2- 172.21.15.5:0/1483777807 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7f34d40395b0 0x7f34d403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T06:33:08.738 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:01.357+0000 7f34f0a00700 1 -- 172.21.15.5:0/1483777807 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7f34d40395b0 msgr2=0x7f34d403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2413496885 2024-04-24T06:33:08.738 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:01.357+0000 7f34f0a00700 1 --2- 172.21.15.5:0/1483777807 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7f34d40395b0 0x7f34d403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-24T06:33:08.738 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:02.158+0000 7f34f0a00700 1 -- 172.21.15.5:0/1483777807 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7f34d40395b0 msgr2=0x7f34d403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2413496885 2024-04-24T06:33:08.738 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:02.158+0000 7f34f0a00700 1 --2- 172.21.15.5:0/1483777807 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7f34d40395b0 0x7f34d403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-24T06:33:08.738 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:03.761+0000 7f34f0a00700 1 -- 172.21.15.5:0/1483777807 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7f34d40395b0 msgr2=0x7f34d403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2413496885 2024-04-24T06:33:08.738 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:03.761+0000 7f34f0a00700 1 --2- 172.21.15.5:0/1483777807 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7f34d40395b0 0x7f34d403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-24T06:33:08.738 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:06.965+0000 7f34f0a00700 1 -- 172.21.15.5:0/1483777807 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7f34d40395b0 msgr2=0x7f34d403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2413496885 2024-04-24T06:33:08.738 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:06.965+0000 7f34f0a00700 1 --2- 172.21.15.5:0/1483777807 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7f34d40395b0 0x7f34d403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-04-24T06:33:08.739 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:07.043+0000 7f34ea000700 1 -- 172.21.15.5:0/1483777807 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f34e001a9f0 con 0x7f34ec106f30 2024-04-24T06:33:08.739 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:07.044+0000 7f34ea000700 1 -- 172.21.15.5:0/1483777807 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7f34d40395b0 msgr2=0x7f34d403ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T06:33:08.739 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:07.044+0000 7f34ea000700 1 --2- 172.21.15.5:0/1483777807 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7f34d40395b0 0x7f34d403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:08.739 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:08.045+0000 7f34ea000700 1 -- 172.21.15.5:0/1483777807 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f34e004f5b0 con 0x7f34ec106f30 2024-04-24T06:33:08.739 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:08.045+0000 7f34ea000700 1 --2- 172.21.15.5:0/1483777807 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f34d403cea0 0x7f34d403f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:08.739 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:08.045+0000 7f34ea000700 1 -- 172.21.15.5:0/1483777807 --> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f34e0046a30 con 0x7f34d403cea0 2024-04-24T06:33:08.739 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:08.046+0000 7f34f0a00700 1 --2- 172.21.15.5:0/1483777807 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f34d403cea0 0x7f34d403f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:08.739 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:08.047+0000 7f34f0a00700 1 --2- 172.21.15.5:0/1483777807 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f34d403cea0 0x7f34d403f290 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f34d8003a10 tx=0x7f34d8009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:08.739 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:08.048+0000 7f34ea000700 1 -- 172.21.15.5:0/1483777807 <== mgr.14120 v2:172.21.15.5:6800/2147029578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f34e0046a30 con 0x7f34d403cea0 2024-04-24T06:33:08.739 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:08.055+0000 7f34df400700 1 -- 172.21.15.5:0/1483777807 --> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f34c8002600 con 0x7f34d403cea0 2024-04-24T06:33:08.739 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:08.055+0000 7f34ea000700 1 -- 172.21.15.5:0/1483777807 <== mgr.14120 v2:172.21.15.5:6800/2147029578 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f34c8002600 con 0x7f34d403cea0 2024-04-24T06:33:08.739 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:08.056+0000 7f34df400700 1 -- 172.21.15.5:0/1483777807 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f34d403cea0 msgr2=0x7f34d403f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:08.740 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:08.056+0000 7f34df400700 1 --2- 172.21.15.5:0/1483777807 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f34d403cea0 0x7f34d403f290 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f34d8003a10 tx=0x7f34d8009290 comp rx=0 tx=0).stop 2024-04-24T06:33:08.740 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:08.056+0000 7f34df400700 1 -- 172.21.15.5:0/1483777807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34ec106f30 msgr2=0x7f34ec1a2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:08.740 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:08.056+0000 7f34df400700 1 --2- 172.21.15.5:0/1483777807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34ec106f30 0x7f34ec1a2cb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f34e0011b60 tx=0x7f34e0011c40 comp rx=0 tx=0).stop 2024-04-24T06:33:08.740 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:08.056+0000 7f34df400700 1 -- 172.21.15.5:0/1483777807 shutdown_connections 2024-04-24T06:33:08.740 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:08.056+0000 7f34df400700 1 --2- 172.21.15.5:0/1483777807 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f34d403cea0 0x7f34d403f290 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:08.740 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:08.056+0000 7f34df400700 1 --2- 172.21.15.5:0/1483777807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34ec106f30 0x7f34ec1a2cb0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:08.740 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:08.056+0000 7f34df400700 1 --2- 172.21.15.5:0/1483777807 >> [v2:172.21.15.5:6800/2413496885,v1:172.21.15.5:6801/2413496885] conn(0x7f34d40395b0 0x7f34d403ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:08.740 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:08.056+0000 7f34df400700 1 -- 172.21.15.5:0/1483777807 >> 172.21.15.5:0/1483777807 conn(0x7f34ec102c30 msgr2=0x7f34ec105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:08.740 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:08.056+0000 7f34df400700 1 -- 172.21.15.5:0/1483777807 shutdown_connections 2024-04-24T06:33:08.740 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:08.056+0000 7f34df400700 1 -- 172.21.15.5:0/1483777807 wait complete. 2024-04-24T06:33:08.740 INFO:teuthology.orchestra.run.smithi005.stdout:mgr epoch 5 is available 2024-04-24T06:33:08.740 INFO:teuthology.orchestra.run.smithi005.stdout:Setting orchestrator backend to cephadm... 2024-04-24T06:33:09.284 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:09 smithi005 ceph-mon[27798]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-24T06:33:09.285 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:09 smithi005 ceph-mon[27798]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-24T06:33:09.285 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:09 smithi005 ceph-mon[27798]: [24/Apr/2024:06:33:08] ENGINE Serving on https://172.21.15.5:7150 2024-04-24T06:33:09.285 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:09 smithi005 ceph-mon[27798]: [24/Apr/2024:06:33:08] ENGINE Serving on http://172.21.15.5:8765 2024-04-24T06:33:09.285 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:09 smithi005 ceph-mon[27798]: [24/Apr/2024:06:33:08] ENGINE Bus STARTED 2024-04-24T06:33:09.285 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:09 smithi005 ceph-mon[27798]: from='mgr.14120 172.21.15.5:0/2924182213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:33:10.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:10 smithi005 ceph-mon[27798]: mgrmap e8: x(active, since 2s) 2024-04-24T06:33:10.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:10 smithi005 ceph-mon[27798]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:33:10.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:10 smithi005 ceph-mon[27798]: from='mgr.14120 172.21.15.5:0/2924182213' entity='mgr.x' 2024-04-24T06:33:10.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:10 smithi005 ceph-mon[27798]: from='mgr.14120 172.21.15.5:0/2924182213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:33:10.321 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.356+0000 7ff560e00700 1 Processor -- start 2024-04-24T06:33:10.321 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.356+0000 7ff560e00700 1 -- start start 2024-04-24T06:33:10.321 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.356+0000 7ff560e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff55c108510 0x7ff55c108930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:10.321 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.356+0000 7ff560e00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff55c108f00 con 0x7ff55c108510 2024-04-24T06:33:10.321 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.357+0000 7ff55b400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff55c108510 0x7ff55c108930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:10.321 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.357+0000 7ff55b400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff55c108510 0x7ff55c108930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34362/0 (socket says 172.21.15.5:34362) 2024-04-24T06:33:10.321 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.357+0000 7ff55b400700 1 -- 172.21.15.5:0/3190557429 learned_addr learned my addr 172.21.15.5:0/3190557429 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:33:10.321 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.357+0000 7ff55b400700 1 -- 172.21.15.5:0/3190557429 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff55c1096c0 con 0x7ff55c108510 2024-04-24T06:33:10.321 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.358+0000 7ff55b400700 1 --2- 172.21.15.5:0/3190557429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff55c108510 0x7ff55c108930 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff54c009e80 tx=0x7ff54c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=532cc90b75d45bbd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:10.321 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.358+0000 7ff55aa00700 1 -- 172.21.15.5:0/3190557429 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff54c017070 con 0x7ff55c108510 2024-04-24T06:33:10.321 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.358+0000 7ff55aa00700 1 -- 172.21.15.5:0/3190557429 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff54c00b8c0 con 0x7ff55c108510 2024-04-24T06:33:10.322 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.358+0000 7ff55aa00700 1 -- 172.21.15.5:0/3190557429 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff54c0045c0 con 0x7ff55c108510 2024-04-24T06:33:10.322 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.359+0000 7ff560e00700 1 -- 172.21.15.5:0/3190557429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff55c108510 msgr2=0x7ff55c108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:10.322 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.359+0000 7ff560e00700 1 --2- 172.21.15.5:0/3190557429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff55c108510 0x7ff55c108930 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff54c009e80 tx=0x7ff54c00b330 comp rx=0 tx=0).stop 2024-04-24T06:33:10.322 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.359+0000 7ff560e00700 1 -- 172.21.15.5:0/3190557429 shutdown_connections 2024-04-24T06:33:10.322 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.359+0000 7ff560e00700 1 --2- 172.21.15.5:0/3190557429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff55c108510 0x7ff55c108930 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:10.322 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.359+0000 7ff560e00700 1 -- 172.21.15.5:0/3190557429 >> 172.21.15.5:0/3190557429 conn(0x7ff55c103b50 msgr2=0x7ff55c105f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:10.322 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.359+0000 7ff560e00700 1 -- 172.21.15.5:0/3190557429 shutdown_connections 2024-04-24T06:33:10.322 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.359+0000 7ff560e00700 1 -- 172.21.15.5:0/3190557429 wait complete. 2024-04-24T06:33:10.322 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.360+0000 7ff560e00700 1 Processor -- start 2024-04-24T06:33:10.322 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.360+0000 7ff560e00700 1 -- start start 2024-04-24T06:33:10.322 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.360+0000 7ff560e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff55c108510 0x7ff55c19e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:10.322 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.360+0000 7ff560e00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff55c10ca70 con 0x7ff55c108510 2024-04-24T06:33:10.322 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.361+0000 7ff55b400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff55c108510 0x7ff55c19e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:10.322 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.361+0000 7ff55b400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff55c108510 0x7ff55c19e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34374/0 (socket says 172.21.15.5:34374) 2024-04-24T06:33:10.323 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.361+0000 7ff55b400700 1 -- 172.21.15.5:0/843453474 learned_addr learned my addr 172.21.15.5:0/843453474 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:33:10.323 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.361+0000 7ff55b400700 1 -- 172.21.15.5:0/843453474 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff55c19ec20 con 0x7ff55c108510 2024-04-24T06:33:10.323 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.361+0000 7ff55b400700 1 --2- 172.21.15.5:0/843453474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff55c108510 0x7ff55c19e6e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff54c004210 tx=0x7ff54c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:10.323 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.362+0000 7ff558c00700 1 -- 172.21.15.5:0/843453474 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff54c020020 con 0x7ff55c108510 2024-04-24T06:33:10.323 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.362+0000 7ff558c00700 1 -- 172.21.15.5:0/843453474 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff54c003b50 con 0x7ff55c108510 2024-04-24T06:33:10.323 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.362+0000 7ff558c00700 1 -- 172.21.15.5:0/843453474 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff54c017040 con 0x7ff55c108510 2024-04-24T06:33:10.323 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.362+0000 7ff560e00700 1 -- 172.21.15.5:0/843453474 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff55c19eeb0 con 0x7ff55c108510 2024-04-24T06:33:10.323 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.362+0000 7ff560e00700 1 -- 172.21.15.5:0/843453474 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff55c1a1ba0 con 0x7ff55c108510 2024-04-24T06:33:10.324 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.363+0000 7ff558c00700 1 -- 172.21.15.5:0/843453474 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7ff54c023020 con 0x7ff55c108510 2024-04-24T06:33:10.324 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.363+0000 7ff560e00700 1 -- 172.21.15.5:0/843453474 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff55c069c50 con 0x7ff55c108510 2024-04-24T06:33:10.324 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.364+0000 7ff558c00700 1 --2- 172.21.15.5:0/843453474 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7ff540039600 0x7ff54003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:10.324 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.364+0000 7ff558c00700 1 -- 172.21.15.5:0/843453474 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff54c00f050 con 0x7ff55c108510 2024-04-24T06:33:10.325 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.364+0000 7ff552a00700 1 --2- 172.21.15.5:0/843453474 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7ff540039600 0x7ff54003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:10.325 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.365+0000 7ff552a00700 1 --2- 172.21.15.5:0/843453474 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7ff540039600 0x7ff54003bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff544006fd0 tx=0x7ff544006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:10.325 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.370+0000 7ff558c00700 1 -- 172.21.15.5:0/843453474 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff54c02a020 con 0x7ff55c108510 2024-04-24T06:33:10.325 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.616+0000 7ff560e00700 1 -- 172.21.15.5:0/843453474 --> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7ff55c104d10 con 0x7ff540039600 2024-04-24T06:33:10.325 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.626+0000 7ff558c00700 1 -- 172.21.15.5:0/843453474 <== mgr.14120 v2:172.21.15.5:6800/2147029578 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff55c104d10 con 0x7ff540039600 2024-04-24T06:33:10.325 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.629+0000 7ff560e00700 1 -- 172.21.15.5:0/843453474 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7ff540039600 msgr2=0x7ff54003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:10.326 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.629+0000 7ff560e00700 1 --2- 172.21.15.5:0/843453474 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7ff540039600 0x7ff54003bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff544006fd0 tx=0x7ff544006e40 comp rx=0 tx=0).stop 2024-04-24T06:33:10.326 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.629+0000 7ff560e00700 1 -- 172.21.15.5:0/843453474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff55c108510 msgr2=0x7ff55c19e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:10.327 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.629+0000 7ff560e00700 1 --2- 172.21.15.5:0/843453474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff55c108510 0x7ff55c19e6e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff54c004210 tx=0x7ff54c00bd50 comp rx=0 tx=0).stop 2024-04-24T06:33:10.327 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.629+0000 7ff560e00700 1 -- 172.21.15.5:0/843453474 shutdown_connections 2024-04-24T06:33:10.327 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.629+0000 7ff560e00700 1 --2- 172.21.15.5:0/843453474 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7ff540039600 0x7ff54003bac0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:10.327 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.629+0000 7ff560e00700 1 --2- 172.21.15.5:0/843453474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff55c108510 0x7ff55c19e6e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:10.327 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.629+0000 7ff560e00700 1 -- 172.21.15.5:0/843453474 >> 172.21.15.5:0/843453474 conn(0x7ff55c103b50 msgr2=0x7ff55c104600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:10.327 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.630+0000 7ff560e00700 1 -- 172.21.15.5:0/843453474 shutdown_connections 2024-04-24T06:33:10.327 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:09.630+0000 7ff560e00700 1 -- 172.21.15.5:0/843453474 wait complete. 2024-04-24T06:33:11.723 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-24T06:33:11.723 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:10.918+0000 7f3b11c00700 1 Processor -- start 2024-04-24T06:33:11.724 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:10.918+0000 7f3b11c00700 1 -- start start 2024-04-24T06:33:11.724 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:10.919+0000 7f3b11c00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b0c07de50 0x7f3b0c07c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:11.724 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:10.919+0000 7f3b11c00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0c07c840 con 0x7f3b0c07de50 2024-04-24T06:33:11.724 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:10.920+0000 7f3b0be00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b0c07de50 0x7f3b0c07c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:11.724 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:10.920+0000 7f3b0be00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b0c07de50 0x7f3b0c07c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34382/0 (socket says 172.21.15.5:34382) 2024-04-24T06:33:11.724 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:10.920+0000 7f3b0be00700 1 -- 172.21.15.5:0/832725809 learned_addr learned my addr 172.21.15.5:0/832725809 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:33:11.724 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:10.920+0000 7f3b0be00700 1 -- 172.21.15.5:0/832725809 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b0c07c9c0 con 0x7f3b0c07de50 2024-04-24T06:33:11.724 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:10.921+0000 7f3b0be00700 1 --2- 172.21.15.5:0/832725809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b0c07de50 0x7f3b0c07c300 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3afc0098a0 tx=0x7f3afc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d00bb777a24894ef server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:11.724 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:10.921+0000 7f3b0aa00700 1 -- 172.21.15.5:0/832725809 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3afc004160 con 0x7f3b0c07de50 2024-04-24T06:33:11.724 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:10.921+0000 7f3b0aa00700 1 -- 172.21.15.5:0/832725809 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3afc004300 con 0x7f3b0c07de50 2024-04-24T06:33:11.724 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:10.921+0000 7f3b0aa00700 1 -- 172.21.15.5:0/832725809 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3afc011b50 con 0x7f3b0c07de50 2024-04-24T06:33:11.724 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:10.922+0000 7f3b11c00700 1 -- 172.21.15.5:0/832725809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b0c07de50 msgr2=0x7f3b0c07c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:11.729 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:10.922+0000 7f3b11c00700 1 --2- 172.21.15.5:0/832725809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b0c07de50 0x7f3b0c07c300 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3afc0098a0 tx=0x7f3afc009bb0 comp rx=0 tx=0).stop 2024-04-24T06:33:11.729 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:10.922+0000 7f3b11c00700 1 -- 172.21.15.5:0/832725809 shutdown_connections 2024-04-24T06:33:11.729 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:10.922+0000 7f3b11c00700 1 --2- 172.21.15.5:0/832725809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b0c07de50 0x7f3b0c07c300 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:11.729 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:10.922+0000 7f3b11c00700 1 -- 172.21.15.5:0/832725809 >> 172.21.15.5:0/832725809 conn(0x7f3b0c103b50 msgr2=0x7f3b0c105f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:11.729 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:10.923+0000 7f3b11c00700 1 -- 172.21.15.5:0/832725809 shutdown_connections 2024-04-24T06:33:11.729 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:10.923+0000 7f3b11c00700 1 -- 172.21.15.5:0/832725809 wait complete. 2024-04-24T06:33:11.729 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:10.924+0000 7f3b11c00700 1 Processor -- start 2024-04-24T06:33:11.729 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:10.924+0000 7f3b11c00700 1 -- start start 2024-04-24T06:33:11.729 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:10.924+0000 7f3b11c00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b0c07de50 0x7f3b0c19e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:11.729 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:10.924+0000 7f3b11c00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0c10eae0 con 0x7f3b0c07de50 2024-04-24T06:33:11.730 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:10.925+0000 7f3b0be00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b0c07de50 0x7f3b0c19e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:11.730 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:10.925+0000 7f3b0be00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b0c07de50 0x7f3b0c19e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34394/0 (socket says 172.21.15.5:34394) 2024-04-24T06:33:11.730 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:10.925+0000 7f3b0be00700 1 -- 172.21.15.5:0/1499703924 learned_addr learned my addr 172.21.15.5:0/1499703924 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:33:11.730 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:10.925+0000 7f3b0be00700 1 -- 172.21.15.5:0/1499703924 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b0c19ec20 con 0x7f3b0c07de50 2024-04-24T06:33:11.730 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:10.925+0000 7f3b0be00700 1 --2- 172.21.15.5:0/1499703924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b0c07de50 0x7f3b0c19e6e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3afc011b60 tx=0x7f3afc011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:11.730 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:10.926+0000 7f3b08c00700 1 -- 172.21.15.5:0/1499703924 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3afc003970 con 0x7f3b0c07de50 2024-04-24T06:33:11.730 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:10.926+0000 7f3b08c00700 1 -- 172.21.15.5:0/1499703924 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3afc003b10 con 0x7f3b0c07de50 2024-04-24T06:33:11.730 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:10.926+0000 7f3b11c00700 1 -- 172.21.15.5:0/1499703924 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b0c19eeb0 con 0x7f3b0c07de50 2024-04-24T06:33:11.730 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:10.926+0000 7f3b08c00700 1 -- 172.21.15.5:0/1499703924 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3afc024610 con 0x7f3b0c07de50 2024-04-24T06:33:11.730 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:10.926+0000 7f3b11c00700 1 -- 172.21.15.5:0/1499703924 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b0c1a1ba0 con 0x7f3b0c07de50 2024-04-24T06:33:11.730 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:10.927+0000 7f3b08c00700 1 -- 172.21.15.5:0/1499703924 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f3afc01a430 con 0x7f3b0c07de50 2024-04-24T06:33:11.731 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:10.927+0000 7f3b11c00700 1 -- 172.21.15.5:0/1499703924 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b0c069c50 con 0x7f3b0c07de50 2024-04-24T06:33:11.731 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:10.928+0000 7f3b08c00700 1 --2- 172.21.15.5:0/1499703924 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f3af0039600 0x7f3af003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:11.731 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:10.928+0000 7f3b08c00700 1 -- 172.21.15.5:0/1499703924 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3afc011e00 con 0x7f3b0c07de50 2024-04-24T06:33:11.731 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:10.928+0000 7f3b0b400700 1 --2- 172.21.15.5:0/1499703924 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f3af0039600 0x7f3af003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:11.731 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:10.929+0000 7f3b0b400700 1 --2- 172.21.15.5:0/1499703924 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f3af0039600 0x7f3af003bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3af8006fd0 tx=0x7f3af8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:11.731 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:10.934+0000 7f3b08c00700 1 -- 172.21.15.5:0/1499703924 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3afc016260 con 0x7f3b0c07de50 2024-04-24T06:33:11.731 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:11.165+0000 7f3b11c00700 1 -- 172.21.15.5:0/1499703924 --> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f3b0c104e60 con 0x7f3af0039600 2024-04-24T06:33:11.731 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:11.167+0000 7f3b08c00700 1 -- 172.21.15.5:0/1499703924 <== mgr.14120 v2:172.21.15.5:6800/2147029578 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f3b0c104e60 con 0x7f3af0039600 2024-04-24T06:33:11.731 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:11.170+0000 7f3b11c00700 1 -- 172.21.15.5:0/1499703924 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f3af0039600 msgr2=0x7f3af003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:11.731 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:11.170+0000 7f3b11c00700 1 --2- 172.21.15.5:0/1499703924 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f3af0039600 0x7f3af003bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3af8006fd0 tx=0x7f3af8006e40 comp rx=0 tx=0).stop 2024-04-24T06:33:11.731 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:11.170+0000 7f3b11c00700 1 -- 172.21.15.5:0/1499703924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b0c07de50 msgr2=0x7f3b0c19e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:11.731 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:11.170+0000 7f3b11c00700 1 --2- 172.21.15.5:0/1499703924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b0c07de50 0x7f3b0c19e6e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3afc011b60 tx=0x7f3afc011c40 comp rx=0 tx=0).stop 2024-04-24T06:33:11.731 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:11.170+0000 7f3b11c00700 1 -- 172.21.15.5:0/1499703924 shutdown_connections 2024-04-24T06:33:11.732 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:11.170+0000 7f3b11c00700 1 --2- 172.21.15.5:0/1499703924 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f3af0039600 0x7f3af003bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:11.732 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:11.170+0000 7f3b11c00700 1 --2- 172.21.15.5:0/1499703924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b0c07de50 0x7f3b0c19e6e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:11.732 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:11.170+0000 7f3b11c00700 1 -- 172.21.15.5:0/1499703924 >> 172.21.15.5:0/1499703924 conn(0x7f3b0c103b50 msgr2=0x7f3b0c104750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:11.732 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:11.171+0000 7f3b11c00700 1 -- 172.21.15.5:0/1499703924 shutdown_connections 2024-04-24T06:33:11.732 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:11.171+0000 7f3b11c00700 1 -- 172.21.15.5:0/1499703924 wait complete. 2024-04-24T06:33:11.732 INFO:teuthology.orchestra.run.smithi005.stdout:Generating ssh key... 2024-04-24T06:33:12.197 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:12 smithi005 ceph-mon[27798]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:33:13.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:13 smithi005 ceph-mon[27798]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:33:13.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:13 smithi005 ceph-mon[27798]: Generating ssh key... 2024-04-24T06:33:13.507 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:13 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: Generating public/private rsa key pair. 2024-04-24T06:33:13.507 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:13 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: Your identification has been saved in /tmp/tmps2b3s7cz/key. 2024-04-24T06:33:13.507 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:13 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: Your public key has been saved in /tmp/tmps2b3s7cz/key.pub. 2024-04-24T06:33:13.507 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:13 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: The key fingerprint is: 2024-04-24T06:33:13.507 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:13 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: SHA256:VJy9JLB9LXSJcQkgys+3fCTf+9s42A4cNvTI3SRQquU ceph-43e9bad0-0204-11ef-bc93-c7b262605968 2024-04-24T06:33:13.507 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:13 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: The key's randomart image is: 2024-04-24T06:33:13.508 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:13 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: +---[RSA 3072]----+ 2024-04-24T06:33:13.508 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:13 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: | ooo+==+o | 2024-04-24T06:33:13.508 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:13 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: | . . =+.+*o | 2024-04-24T06:33:13.508 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:13 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: | o o .o*.o .| 2024-04-24T06:33:13.508 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:13 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: | + B.= + | 2024-04-24T06:33:13.508 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:13 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: | S + E o .| 2024-04-24T06:33:13.508 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:13 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: | o B + | 2024-04-24T06:33:13.508 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:13 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: | o =o. | 2024-04-24T06:33:13.508 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:13 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: | ...oo.| 2024-04-24T06:33:13.508 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:13 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: | .+++| 2024-04-24T06:33:13.508 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:13 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: +----[SHA256]-----+ 2024-04-24T06:33:13.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:12.363+0000 7ff902600700 1 Processor -- start 2024-04-24T06:33:13.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:12.363+0000 7ff902600700 1 -- start start 2024-04-24T06:33:13.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:12.364+0000 7ff902600700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8fc07e880 0x7ff8fc07eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:13.910 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:12.364+0000 7ff902600700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc07f270 con 0x7ff8fc07e880 2024-04-24T06:33:13.910 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:12.364+0000 7ff8fb400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8fc07e880 0x7ff8fc07eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:13.910 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:12.364+0000 7ff8fb400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8fc07e880 0x7ff8fc07eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34410/0 (socket says 172.21.15.5:34410) 2024-04-24T06:33:13.910 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:12.364+0000 7ff8fb400700 1 -- 172.21.15.5:0/3214601516 learned_addr learned my addr 172.21.15.5:0/3214601516 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:33:13.910 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:12.364+0000 7ff8fb400700 1 -- 172.21.15.5:0/3214601516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8fc07fa30 con 0x7ff8fc07e880 2024-04-24T06:33:13.910 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:12.365+0000 7ff8fb400700 1 --2- 172.21.15.5:0/3214601516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8fc07e880 0x7ff8fc07eca0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec0098a0 tx=0x7ff8ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0d142f631c3ffab server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:13.910 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:12.365+0000 7ff8fa000700 1 -- 172.21.15.5:0/3214601516 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8ec004160 con 0x7ff8fc07e880 2024-04-24T06:33:13.910 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:12.365+0000 7ff8fa000700 1 -- 172.21.15.5:0/3214601516 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff8ec004300 con 0x7ff8fc07e880 2024-04-24T06:33:13.910 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:12.365+0000 7ff8fa000700 1 -- 172.21.15.5:0/3214601516 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8ec011b50 con 0x7ff8fc07e880 2024-04-24T06:33:13.910 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:12.366+0000 7ff902600700 1 -- 172.21.15.5:0/3214601516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8fc07e880 msgr2=0x7ff8fc07eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:13.910 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:12.366+0000 7ff902600700 1 --2- 172.21.15.5:0/3214601516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8fc07e880 0x7ff8fc07eca0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec0098a0 tx=0x7ff8ec009bb0 comp rx=0 tx=0).stop 2024-04-24T06:33:13.910 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:12.366+0000 7ff902600700 1 -- 172.21.15.5:0/3214601516 shutdown_connections 2024-04-24T06:33:13.911 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:12.366+0000 7ff902600700 1 --2- 172.21.15.5:0/3214601516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8fc07e880 0x7ff8fc07eca0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:13.911 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:12.366+0000 7ff902600700 1 -- 172.21.15.5:0/3214601516 >> 172.21.15.5:0/3214601516 conn(0x7ff8fc07de90 msgr2=0x7ff8fc1084f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:13.911 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:12.366+0000 7ff902600700 1 -- 172.21.15.5:0/3214601516 shutdown_connections 2024-04-24T06:33:13.911 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:12.366+0000 7ff902600700 1 -- 172.21.15.5:0/3214601516 wait complete. 2024-04-24T06:33:13.911 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:12.367+0000 7ff902600700 1 Processor -- start 2024-04-24T06:33:13.911 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:12.367+0000 7ff902600700 1 -- start start 2024-04-24T06:33:13.911 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:12.367+0000 7ff902600700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8fc07e880 0x7ff8fc1a2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:13.911 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:12.368+0000 7ff902600700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc080df0 con 0x7ff8fc07e880 2024-04-24T06:33:13.911 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:12.368+0000 7ff8fb400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8fc07e880 0x7ff8fc1a2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:13.911 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:12.368+0000 7ff8fb400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8fc07e880 0x7ff8fc1a2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34424/0 (socket says 172.21.15.5:34424) 2024-04-24T06:33:13.911 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:12.368+0000 7ff8fb400700 1 -- 172.21.15.5:0/436800274 learned_addr learned my addr 172.21.15.5:0/436800274 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:33:13.911 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:12.368+0000 7ff8fb400700 1 -- 172.21.15.5:0/436800274 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8fc1a31d0 con 0x7ff8fc07e880 2024-04-24T06:33:13.912 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:12.369+0000 7ff8fb400700 1 --2- 172.21.15.5:0/436800274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8fc07e880 0x7ff8fc1a2c90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec009fb0 tx=0x7ff8ec011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:13.912 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:12.369+0000 7ff8f3e00700 1 -- 172.21.15.5:0/436800274 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8ec0038b0 con 0x7ff8fc07e880 2024-04-24T06:33:13.912 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:12.369+0000 7ff8f3e00700 1 -- 172.21.15.5:0/436800274 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff8ec011e50 con 0x7ff8fc07e880 2024-04-24T06:33:13.912 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:12.369+0000 7ff8f3e00700 1 -- 172.21.15.5:0/436800274 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8ec019420 con 0x7ff8fc07e880 2024-04-24T06:33:13.912 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:12.369+0000 7ff902600700 1 -- 172.21.15.5:0/436800274 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8fc1a3460 con 0x7ff8fc07e880 2024-04-24T06:33:13.912 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:12.369+0000 7ff902600700 1 -- 172.21.15.5:0/436800274 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8fc1a6150 con 0x7ff8fc07e880 2024-04-24T06:33:13.912 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:12.370+0000 7ff8f3e00700 1 -- 172.21.15.5:0/436800274 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7ff8ec003a50 con 0x7ff8fc07e880 2024-04-24T06:33:13.912 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:12.370+0000 7ff902600700 1 -- 172.21.15.5:0/436800274 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8fc064150 con 0x7ff8fc07e880 2024-04-24T06:33:13.912 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:12.371+0000 7ff8f3e00700 1 --2- 172.21.15.5:0/436800274 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7ff8e00395b0 0x7ff8e003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:13.912 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:12.371+0000 7ff8f3e00700 1 -- 172.21.15.5:0/436800274 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff8ec04da00 con 0x7ff8fc07e880 2024-04-24T06:33:13.912 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:12.371+0000 7ff8faa00700 1 --2- 172.21.15.5:0/436800274 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7ff8e00395b0 0x7ff8e003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:13.912 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:12.372+0000 7ff8faa00700 1 --2- 172.21.15.5:0/436800274 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7ff8e00395b0 0x7ff8e003ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8006fd0 tx=0x7ff8e8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:13.912 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:12.376+0000 7ff8f3e00700 1 -- 172.21.15.5:0/436800274 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff8ec04d350 con 0x7ff8fc07e880 2024-04-24T06:33:13.913 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:12.610+0000 7ff902600700 1 -- 172.21.15.5:0/436800274 --> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7ff8fc02e310 con 0x7ff8e00395b0 2024-04-24T06:33:13.913 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:13.234+0000 7ff8f3e00700 1 -- 172.21.15.5:0/436800274 <== mgr.14120 v2:172.21.15.5:6800/2147029578 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff8fc02e310 con 0x7ff8e00395b0 2024-04-24T06:33:13.913 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:13.237+0000 7ff902600700 1 -- 172.21.15.5:0/436800274 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7ff8e00395b0 msgr2=0x7ff8e003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:13.913 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:13.237+0000 7ff902600700 1 --2- 172.21.15.5:0/436800274 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7ff8e00395b0 0x7ff8e003ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8006fd0 tx=0x7ff8e8006e40 comp rx=0 tx=0).stop 2024-04-24T06:33:13.913 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:13.238+0000 7ff902600700 1 -- 172.21.15.5:0/436800274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8fc07e880 msgr2=0x7ff8fc1a2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:13.913 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:13.238+0000 7ff902600700 1 --2- 172.21.15.5:0/436800274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8fc07e880 0x7ff8fc1a2c90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec009fb0 tx=0x7ff8ec011bf0 comp rx=0 tx=0).stop 2024-04-24T06:33:13.913 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:13.238+0000 7ff902600700 1 -- 172.21.15.5:0/436800274 shutdown_connections 2024-04-24T06:33:13.913 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:13.238+0000 7ff902600700 1 --2- 172.21.15.5:0/436800274 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7ff8e00395b0 0x7ff8e003ba70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:13.913 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:13.238+0000 7ff902600700 1 --2- 172.21.15.5:0/436800274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8fc07e880 0x7ff8fc1a2c90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:13.913 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:13.238+0000 7ff902600700 1 -- 172.21.15.5:0/436800274 >> 172.21.15.5:0/436800274 conn(0x7ff8fc07de90 msgr2=0x7ff8fc199920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:13.913 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:13.238+0000 7ff902600700 1 -- 172.21.15.5:0/436800274 shutdown_connections 2024-04-24T06:33:13.913 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:13.238+0000 7ff902600700 1 -- 172.21.15.5:0/436800274 wait complete. 2024-04-24T06:33:14.469 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:14 smithi005 ceph-mon[27798]: from='mgr.14120 172.21.15.5:0/2924182213' entity='mgr.x' 2024-04-24T06:33:14.469 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:14 smithi005 ceph-mon[27798]: from='mgr.14120 172.21.15.5:0/2924182213' entity='mgr.x' 2024-04-24T06:33:15.457 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:15 smithi005 ceph-mon[27798]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:33:15.471 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDi/8pw/bZ6/8Xsd/BkMGyfongGq3MLjEHeDsDuvvYn5OxaYqmusQxO2CO/1RjA+7cvLgUeNrldrn3SPkPmW/AuXXIbTIaabhH5hfS+bIZMqO1hmL6IB9AnRYQLYQHWI31AFtNtAlTC0EmiyZDp+5VXoxjOi0U/l/gcz9YD6FAf1zi95rwpTkkqZxCat65Czdw3yDhVp8Mpxmy0tdjuCAsG40JPZ9PZ9ern6ls4WKG05ljg24HfmHD+ArPUJD+0IenjpCu5ZjeKhwTLY4dAozZPuHXvvjTw8ki6pb1wsZiAfgav0NMvVNrtdpRFmI3IhQjVY1dhVdMA6rDoPm0f35Z0VJQtnCyCgsIVAmS2hkncW6+kf3NZnQlrBZwhPmTfOHH2aG/G/e1+YlUbcLipUj+qpedk0yIFOl7dit5/+RziGSBXnq7OB5Nx2hePaXu1QmVmoNfFf2XK/YcomopaoUEm2L0+Ar6tfc65gaxcRvkfGu9FGELwI7WnysIFEvtzZ6c= ceph-43e9bad0-0204-11ef-bc93-c7b262605968 2024-04-24T06:33:15.473 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.467+0000 7f9032600700 1 Processor -- start 2024-04-24T06:33:15.473 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.467+0000 7f9032600700 1 -- start start 2024-04-24T06:33:15.473 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.468+0000 7f9032600700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f902c109170 0x7f902c10b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:15.473 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.468+0000 7f9032600700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f902c07bed0 con 0x7f902c109170 2024-04-24T06:33:15.473 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.468+0000 7f902b400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f902c109170 0x7f902c10b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:15.473 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.468+0000 7f902b400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f902c109170 0x7f902c10b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44960/0 (socket says 172.21.15.5:44960) 2024-04-24T06:33:15.476 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.468+0000 7f902b400700 1 -- 172.21.15.5:0/194629164 learned_addr learned my addr 172.21.15.5:0/194629164 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:33:15.476 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.469+0000 7f902b400700 1 -- 172.21.15.5:0/194629164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f902c10baa0 con 0x7f902c109170 2024-04-24T06:33:15.476 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.469+0000 7f902b400700 1 --2- 172.21.15.5:0/194629164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f902c109170 0x7f902c10b560 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f901c0098a0 tx=0x7f901c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a9ef6550a0da15 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:15.477 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.469+0000 7f902a000700 1 -- 172.21.15.5:0/194629164 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f901c004160 con 0x7f902c109170 2024-04-24T06:33:15.477 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.470+0000 7f902a000700 1 -- 172.21.15.5:0/194629164 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f901c004300 con 0x7f902c109170 2024-04-24T06:33:15.477 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.470+0000 7f902a000700 1 -- 172.21.15.5:0/194629164 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f901c011b50 con 0x7f902c109170 2024-04-24T06:33:15.477 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.470+0000 7f9032600700 1 -- 172.21.15.5:0/194629164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f902c109170 msgr2=0x7f902c10b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:15.477 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.470+0000 7f9032600700 1 --2- 172.21.15.5:0/194629164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f902c109170 0x7f902c10b560 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f901c0098a0 tx=0x7f901c009bb0 comp rx=0 tx=0).stop 2024-04-24T06:33:15.477 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.470+0000 7f9032600700 1 -- 172.21.15.5:0/194629164 shutdown_connections 2024-04-24T06:33:15.477 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.470+0000 7f9032600700 1 --2- 172.21.15.5:0/194629164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f902c109170 0x7f902c10b560 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:15.477 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.470+0000 7f9032600700 1 -- 172.21.15.5:0/194629164 >> 172.21.15.5:0/194629164 conn(0x7f902c102c00 msgr2=0x7f902c105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:15.477 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.470+0000 7f9032600700 1 -- 172.21.15.5:0/194629164 shutdown_connections 2024-04-24T06:33:15.477 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.471+0000 7f9032600700 1 -- 172.21.15.5:0/194629164 wait complete. 2024-04-24T06:33:15.477 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.472+0000 7f9032600700 1 Processor -- start 2024-04-24T06:33:15.477 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.472+0000 7f9032600700 1 -- start start 2024-04-24T06:33:15.478 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.472+0000 7f9032600700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f902c109170 0x7f902c19e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:15.478 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.472+0000 7f9032600700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f902c10cad0 con 0x7f902c109170 2024-04-24T06:33:15.478 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.472+0000 7f902b400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f902c109170 0x7f902c19e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:15.478 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.472+0000 7f902b400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f902c109170 0x7f902c19e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44974/0 (socket says 172.21.15.5:44974) 2024-04-24T06:33:15.478 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.473+0000 7f902b400700 1 -- 172.21.15.5:0/3471872772 learned_addr learned my addr 172.21.15.5:0/3471872772 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:33:15.478 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.473+0000 7f902b400700 1 -- 172.21.15.5:0/3471872772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f902c19ee20 con 0x7f902c109170 2024-04-24T06:33:15.478 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.473+0000 7f902b400700 1 --2- 172.21.15.5:0/3471872772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f902c109170 0x7f902c19e8e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f901c011b60 tx=0x7f901c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:15.478 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.473+0000 7f9023e00700 1 -- 172.21.15.5:0/3471872772 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f901c003970 con 0x7f902c109170 2024-04-24T06:33:15.478 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.474+0000 7f9023e00700 1 -- 172.21.15.5:0/3471872772 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f901c003b10 con 0x7f902c109170 2024-04-24T06:33:15.478 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.474+0000 7f9032600700 1 -- 172.21.15.5:0/3471872772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f902c19f0b0 con 0x7f902c109170 2024-04-24T06:33:15.481 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.474+0000 7f9023e00700 1 -- 172.21.15.5:0/3471872772 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f901c024610 con 0x7f902c109170 2024-04-24T06:33:15.481 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.474+0000 7f9032600700 1 -- 172.21.15.5:0/3471872772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f902c1a1da0 con 0x7f902c109170 2024-04-24T06:33:15.481 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.475+0000 7f9023e00700 1 -- 172.21.15.5:0/3471872772 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f901c01a430 con 0x7f902c109170 2024-04-24T06:33:15.481 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.475+0000 7f9032600700 1 -- 172.21.15.5:0/3471872772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f902c064150 con 0x7f902c109170 2024-04-24T06:33:15.481 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.475+0000 7f9023e00700 1 --2- 172.21.15.5:0/3471872772 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f9010039600 0x7f901003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:15.481 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.475+0000 7f9023e00700 1 -- 172.21.15.5:0/3471872772 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f901c011e00 con 0x7f902c109170 2024-04-24T06:33:15.481 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.476+0000 7f902aa00700 1 --2- 172.21.15.5:0/3471872772 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f9010039600 0x7f901003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:15.482 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.476+0000 7f902aa00700 1 --2- 172.21.15.5:0/3471872772 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f9010039600 0x7f901003bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9018006fd0 tx=0x7f9018006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:15.482 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.481+0000 7f9023e00700 1 -- 172.21.15.5:0/3471872772 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f901c016260 con 0x7f902c109170 2024-04-24T06:33:15.482 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.714+0000 7f9032600700 1 -- 172.21.15.5:0/3471872772 --> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f902c103da0 con 0x7f9010039600 2024-04-24T06:33:15.482 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.716+0000 7f9023e00700 1 -- 172.21.15.5:0/3471872772 <== mgr.14120 v2:172.21.15.5:6800/2147029578 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f902c103da0 con 0x7f9010039600 2024-04-24T06:33:15.482 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.719+0000 7f9032600700 1 -- 172.21.15.5:0/3471872772 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f9010039600 msgr2=0x7f901003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:15.482 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.719+0000 7f9032600700 1 --2- 172.21.15.5:0/3471872772 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f9010039600 0x7f901003bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9018006fd0 tx=0x7f9018006e40 comp rx=0 tx=0).stop 2024-04-24T06:33:15.482 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.719+0000 7f9032600700 1 -- 172.21.15.5:0/3471872772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f902c109170 msgr2=0x7f902c19e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:15.482 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.719+0000 7f9032600700 1 --2- 172.21.15.5:0/3471872772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f902c109170 0x7f902c19e8e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f901c011b60 tx=0x7f901c011c40 comp rx=0 tx=0).stop 2024-04-24T06:33:15.482 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.719+0000 7f9032600700 1 -- 172.21.15.5:0/3471872772 shutdown_connections 2024-04-24T06:33:15.482 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.719+0000 7f9032600700 1 --2- 172.21.15.5:0/3471872772 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f9010039600 0x7f901003bac0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:15.482 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.719+0000 7f9032600700 1 --2- 172.21.15.5:0/3471872772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f902c109170 0x7f902c19e8e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:15.483 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.719+0000 7f9032600700 1 -- 172.21.15.5:0/3471872772 >> 172.21.15.5:0/3471872772 conn(0x7f902c102c00 msgr2=0x7f902c103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:15.483 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.719+0000 7f9032600700 1 -- 172.21.15.5:0/3471872772 shutdown_connections 2024-04-24T06:33:15.483 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:14.719+0000 7f9032600700 1 -- 172.21.15.5:0/3471872772 wait complete. 2024-04-24T06:33:15.483 INFO:teuthology.orchestra.run.smithi005.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-24T06:33:15.483 INFO:teuthology.orchestra.run.smithi005.stdout:Adding key to root@localhost authorized_keys... 2024-04-24T06:33:15.488 INFO:teuthology.orchestra.run.smithi005.stdout:Adding host smithi005... 2024-04-24T06:33:17.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:17 smithi005 ceph-mon[27798]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi005", "addr": "172.21.15.5", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:33:18.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:18 smithi005 ceph-mon[27798]: Deploying cephadm binary to smithi005 2024-04-24T06:33:23.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Added host 'smithi005' with addr '172.21.15.5' 2024-04-24T06:33:23.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:16.065+0000 7fdf53a00700 1 Processor -- start 2024-04-24T06:33:23.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:16.066+0000 7fdf53a00700 1 -- start start 2024-04-24T06:33:23.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:16.066+0000 7fdf53a00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf4c07de90 0x7fdf4c07c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:23.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:16.066+0000 7fdf53a00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4c07c910 con 0x7fdf4c07de90 2024-04-24T06:33:23.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:16.067+0000 7fdf51000700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf4c07de90 0x7fdf4c07c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:23.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:16.067+0000 7fdf51000700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf4c07de90 0x7fdf4c07c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44986/0 (socket says 172.21.15.5:44986) 2024-04-24T06:33:23.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:16.067+0000 7fdf51000700 1 -- 172.21.15.5:0/737122034 learned_addr learned my addr 172.21.15.5:0/737122034 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:33:23.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:16.067+0000 7fdf51000700 1 -- 172.21.15.5:0/737122034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf4c07ca90 con 0x7fdf4c07de90 2024-04-24T06:33:23.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:16.068+0000 7fdf51000700 1 --2- 172.21.15.5:0/737122034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf4c07de90 0x7fdf4c07c340 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009e80 tx=0x7fdf3c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=260a2d7134329ffb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:23.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:16.068+0000 7fdf4be00700 1 -- 172.21.15.5:0/737122034 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf3c017070 con 0x7fdf4c07de90 2024-04-24T06:33:23.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:16.068+0000 7fdf4be00700 1 -- 172.21.15.5:0/737122034 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdf3c00b8c0 con 0x7fdf4c07de90 2024-04-24T06:33:23.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:16.068+0000 7fdf4be00700 1 -- 172.21.15.5:0/737122034 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf3c0045c0 con 0x7fdf4c07de90 2024-04-24T06:33:23.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:16.069+0000 7fdf53a00700 1 -- 172.21.15.5:0/737122034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf4c07de90 msgr2=0x7fdf4c07c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:23.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:16.069+0000 7fdf53a00700 1 --2- 172.21.15.5:0/737122034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf4c07de90 0x7fdf4c07c340 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009e80 tx=0x7fdf3c00b330 comp rx=0 tx=0).stop 2024-04-24T06:33:23.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:16.069+0000 7fdf53a00700 1 -- 172.21.15.5:0/737122034 shutdown_connections 2024-04-24T06:33:23.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:16.069+0000 7fdf53a00700 1 --2- 172.21.15.5:0/737122034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf4c07de90 0x7fdf4c07c340 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:23.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:16.069+0000 7fdf53a00700 1 -- 172.21.15.5:0/737122034 >> 172.21.15.5:0/737122034 conn(0x7fdf4c103d40 msgr2=0x7fdf4c106160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:23.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:16.069+0000 7fdf53a00700 1 -- 172.21.15.5:0/737122034 shutdown_connections 2024-04-24T06:33:23.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:16.069+0000 7fdf53a00700 1 -- 172.21.15.5:0/737122034 wait complete. 2024-04-24T06:33:23.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:16.070+0000 7fdf53a00700 1 Processor -- start 2024-04-24T06:33:23.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:16.070+0000 7fdf53a00700 1 -- start start 2024-04-24T06:33:23.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:16.070+0000 7fdf53a00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf4c07de90 0x7fdf4c19e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:23.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:16.071+0000 7fdf53a00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4c10ec90 con 0x7fdf4c07de90 2024-04-24T06:33:23.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:16.071+0000 7fdf51000700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf4c07de90 0x7fdf4c19e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:23.214 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:16.071+0000 7fdf51000700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf4c07de90 0x7fdf4c19e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44988/0 (socket says 172.21.15.5:44988) 2024-04-24T06:33:23.215 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:16.071+0000 7fdf51000700 1 -- 172.21.15.5:0/2667223913 learned_addr learned my addr 172.21.15.5:0/2667223913 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:33:23.215 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:16.071+0000 7fdf51000700 1 -- 172.21.15.5:0/2667223913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf4c19edb0 con 0x7fdf4c07de90 2024-04-24T06:33:23.215 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:16.071+0000 7fdf51000700 1 --2- 172.21.15.5:0/2667223913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf4c07de90 0x7fdf4c19e870 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009530 tx=0x7fdf3c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:23.215 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:16.072+0000 7fdf4a000700 1 -- 172.21.15.5:0/2667223913 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf3c020020 con 0x7fdf4c07de90 2024-04-24T06:33:23.215 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:16.072+0000 7fdf4a000700 1 -- 172.21.15.5:0/2667223913 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdf3c003b50 con 0x7fdf4c07de90 2024-04-24T06:33:23.215 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:16.072+0000 7fdf53a00700 1 -- 172.21.15.5:0/2667223913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf4c19f040 con 0x7fdf4c07de90 2024-04-24T06:33:23.215 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:16.072+0000 7fdf4a000700 1 -- 172.21.15.5:0/2667223913 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf3c017040 con 0x7fdf4c07de90 2024-04-24T06:33:23.215 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:16.072+0000 7fdf53a00700 1 -- 172.21.15.5:0/2667223913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf4c1a1d30 con 0x7fdf4c07de90 2024-04-24T06:33:23.215 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:16.073+0000 7fdf4a000700 1 -- 172.21.15.5:0/2667223913 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fdf3c023020 con 0x7fdf4c07de90 2024-04-24T06:33:23.215 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:16.073+0000 7fdf53a00700 1 -- 172.21.15.5:0/2667223913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf4c064150 con 0x7fdf4c07de90 2024-04-24T06:33:23.215 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:16.074+0000 7fdf4a000700 1 --2- 172.21.15.5:0/2667223913 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7fdf300395b0 0x7fdf3003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:23.215 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:16.074+0000 7fdf4a000700 1 -- 172.21.15.5:0/2667223913 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdf3c00f050 con 0x7fdf4c07de90 2024-04-24T06:33:23.215 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:16.074+0000 7fdf43e00700 1 --2- 172.21.15.5:0/2667223913 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7fdf300395b0 0x7fdf3003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:23.215 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:16.075+0000 7fdf43e00700 1 --2- 172.21.15.5:0/2667223913 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7fdf300395b0 0x7fdf3003ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdf38006fd0 tx=0x7fdf38006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:23.216 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:16.079+0000 7fdf4a000700 1 -- 172.21.15.5:0/2667223913 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdf3c02a020 con 0x7fdf4c07de90 2024-04-24T06:33:23.216 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:16.318+0000 7fdf53a00700 1 -- 172.21.15.5:0/2667223913 --> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi005", "addr": "172.21.15.5", "target": ["mon-mgr", ""]}) -- 0x7fdf4c104f80 con 0x7fdf300395b0 2024-04-24T06:33:23.216 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:22.534+0000 7fdf4a000700 1 -- 172.21.15.5:0/2667223913 <== mgr.14120 v2:172.21.15.5:6800/2147029578 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7fdf4c104f80 con 0x7fdf300395b0 2024-04-24T06:33:23.216 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:22.539+0000 7fdf53a00700 1 -- 172.21.15.5:0/2667223913 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7fdf300395b0 msgr2=0x7fdf3003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:23.216 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:22.539+0000 7fdf53a00700 1 --2- 172.21.15.5:0/2667223913 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7fdf300395b0 0x7fdf3003ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdf38006fd0 tx=0x7fdf38006e40 comp rx=0 tx=0).stop 2024-04-24T06:33:23.216 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:22.539+0000 7fdf53a00700 1 -- 172.21.15.5:0/2667223913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf4c07de90 msgr2=0x7fdf4c19e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:23.216 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:22.539+0000 7fdf53a00700 1 --2- 172.21.15.5:0/2667223913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf4c07de90 0x7fdf4c19e870 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009530 tx=0x7fdf3c00bd50 comp rx=0 tx=0).stop 2024-04-24T06:33:23.216 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:22.540+0000 7fdf53a00700 1 -- 172.21.15.5:0/2667223913 shutdown_connections 2024-04-24T06:33:23.216 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:22.540+0000 7fdf53a00700 1 --2- 172.21.15.5:0/2667223913 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7fdf300395b0 0x7fdf3003ba70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:23.216 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:22.540+0000 7fdf53a00700 1 --2- 172.21.15.5:0/2667223913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf4c07de90 0x7fdf4c19e870 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:23.216 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:22.540+0000 7fdf53a00700 1 -- 172.21.15.5:0/2667223913 >> 172.21.15.5:0/2667223913 conn(0x7fdf4c103d40 msgr2=0x7fdf4c104870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:23.216 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:22.540+0000 7fdf53a00700 1 -- 172.21.15.5:0/2667223913 shutdown_connections 2024-04-24T06:33:23.216 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:22.540+0000 7fdf53a00700 1 -- 172.21.15.5:0/2667223913 wait complete. 2024-04-24T06:33:23.216 INFO:teuthology.orchestra.run.smithi005.stdout:Deploying unmanaged mon service... 2024-04-24T06:33:23.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:23 smithi005 ceph-mon[27798]: from='mgr.14120 172.21.15.5:0/2924182213' entity='mgr.x' 2024-04-24T06:33:23.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:23 smithi005 ceph-mon[27798]: Added host smithi005 2024-04-24T06:33:23.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:23 smithi005 ceph-mon[27798]: from='mgr.14120 172.21.15.5:0/2924182213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:33:24.940 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-24T06:33:24.940 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:23.810+0000 7f3fee400700 1 Processor -- start 2024-04-24T06:33:24.940 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:23.810+0000 7f3fee400700 1 -- start start 2024-04-24T06:33:24.940 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:23.810+0000 7f3fee400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe810a8e0 0x7f3fe810ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:24.940 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:23.811+0000 7f3fee400700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe810b2d0 con 0x7f3fe810a8e0 2024-04-24T06:33:24.940 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:23.811+0000 7f3fe7400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe810a8e0 0x7f3fe810ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:24.940 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:23.811+0000 7f3fe7400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe810a8e0 0x7f3fe810ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44992/0 (socket says 172.21.15.5:44992) 2024-04-24T06:33:24.941 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:23.811+0000 7f3fe7400700 1 -- 172.21.15.5:0/3009341999 learned_addr learned my addr 172.21.15.5:0/3009341999 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:33:24.941 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:23.812+0000 7f3fe7400700 1 -- 172.21.15.5:0/3009341999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fe810ba90 con 0x7f3fe810a8e0 2024-04-24T06:33:24.941 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:23.812+0000 7f3fe7400700 1 --2- 172.21.15.5:0/3009341999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe810a8e0 0x7f3fe810ad00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3fd80098a0 tx=0x7f3fd8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=113a295abdf9d8a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:24.941 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:23.812+0000 7f3fe6000700 1 -- 172.21.15.5:0/3009341999 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3fd8004160 con 0x7f3fe810a8e0 2024-04-24T06:33:24.941 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:23.813+0000 7f3fe6000700 1 -- 172.21.15.5:0/3009341999 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3fd8004300 con 0x7f3fe810a8e0 2024-04-24T06:33:24.941 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:23.813+0000 7f3fe6000700 1 -- 172.21.15.5:0/3009341999 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3fd8011b50 con 0x7f3fe810a8e0 2024-04-24T06:33:24.941 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:23.813+0000 7f3fee400700 1 -- 172.21.15.5:0/3009341999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe810a8e0 msgr2=0x7f3fe810ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:24.941 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:23.813+0000 7f3fee400700 1 --2- 172.21.15.5:0/3009341999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe810a8e0 0x7f3fe810ad00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3fd80098a0 tx=0x7f3fd8009bb0 comp rx=0 tx=0).stop 2024-04-24T06:33:24.941 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:23.813+0000 7f3fee400700 1 -- 172.21.15.5:0/3009341999 shutdown_connections 2024-04-24T06:33:24.941 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:23.813+0000 7f3fee400700 1 --2- 172.21.15.5:0/3009341999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe810a8e0 0x7f3fe810ad00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:24.941 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:23.813+0000 7f3fee400700 1 -- 172.21.15.5:0/3009341999 >> 172.21.15.5:0/3009341999 conn(0x7f3fe807de60 msgr2=0x7f3fe81084f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:24.941 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:23.813+0000 7f3fee400700 1 -- 172.21.15.5:0/3009341999 shutdown_connections 2024-04-24T06:33:24.941 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:23.813+0000 7f3fee400700 1 -- 172.21.15.5:0/3009341999 wait complete. 2024-04-24T06:33:24.942 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:23.814+0000 7f3fee400700 1 Processor -- start 2024-04-24T06:33:24.942 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:23.815+0000 7f3fee400700 1 -- start start 2024-04-24T06:33:24.942 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:23.815+0000 7f3fee400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe810a8e0 0x7f3fe819e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:24.942 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:23.815+0000 7f3fee400700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe810ee40 con 0x7f3fe810a8e0 2024-04-24T06:33:24.942 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:23.815+0000 7f3fe7400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe810a8e0 0x7f3fe819e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:24.942 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:23.815+0000 7f3fe7400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe810a8e0 0x7f3fe819e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45006/0 (socket says 172.21.15.5:45006) 2024-04-24T06:33:24.942 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:23.815+0000 7f3fe7400700 1 -- 172.21.15.5:0/2610928728 learned_addr learned my addr 172.21.15.5:0/2610928728 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:33:24.942 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:23.816+0000 7f3fe7400700 1 -- 172.21.15.5:0/2610928728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fe819ed40 con 0x7f3fe810a8e0 2024-04-24T06:33:24.942 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:23.816+0000 7f3fe7400700 1 --2- 172.21.15.5:0/2610928728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe810a8e0 0x7f3fe819e800 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8011670 tx=0x7f3fd8011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:24.942 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:23.816+0000 7f3fdfe00700 1 -- 172.21.15.5:0/2610928728 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3fd80039c0 con 0x7f3fe810a8e0 2024-04-24T06:33:24.942 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:23.816+0000 7f3fdfe00700 1 -- 172.21.15.5:0/2610928728 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3fd8011e50 con 0x7f3fe810a8e0 2024-04-24T06:33:24.942 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:23.816+0000 7f3fdfe00700 1 -- 172.21.15.5:0/2610928728 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3fd801a5c0 con 0x7f3fe810a8e0 2024-04-24T06:33:24.943 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:23.816+0000 7f3fee400700 1 -- 172.21.15.5:0/2610928728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fe819efd0 con 0x7f3fe810a8e0 2024-04-24T06:33:24.943 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:23.817+0000 7f3fee400700 1 -- 172.21.15.5:0/2610928728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fe81a1cc0 con 0x7f3fe810a8e0 2024-04-24T06:33:24.943 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:23.818+0000 7f3fdfe00700 1 -- 172.21.15.5:0/2610928728 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f3fd8024430 con 0x7f3fe810a8e0 2024-04-24T06:33:24.943 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:23.818+0000 7f3fee400700 1 -- 172.21.15.5:0/2610928728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fe8064150 con 0x7f3fe810a8e0 2024-04-24T06:33:24.943 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:23.818+0000 7f3fdfe00700 1 --2- 172.21.15.5:0/2610928728 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f3fcc039610 0x7f3fcc03bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:24.943 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:23.818+0000 7f3fdfe00700 1 -- 172.21.15.5:0/2610928728 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3fd804db60 con 0x7f3fe810a8e0 2024-04-24T06:33:24.943 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:23.818+0000 7f3fe6a00700 1 --2- 172.21.15.5:0/2610928728 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f3fcc039610 0x7f3fcc03bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:24.944 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:23.819+0000 7f3fe6a00700 1 --2- 172.21.15.5:0/2610928728 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f3fcc039610 0x7f3fcc03bad0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4006fd0 tx=0x7f3fd4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:24.944 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:23.824+0000 7f3fdfe00700 1 -- 172.21.15.5:0/2610928728 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3fd802d3f0 con 0x7f3fe810a8e0 2024-04-24T06:33:24.944 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:24.070+0000 7f3fee400700 1 -- 172.21.15.5:0/2610928728 --> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f3fe81082f0 con 0x7f3fcc039610 2024-04-24T06:33:24.944 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:24.081+0000 7f3fdfe00700 1 -- 172.21.15.5:0/2610928728 <== mgr.14120 v2:172.21.15.5:6800/2147029578 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3fe81082f0 con 0x7f3fcc039610 2024-04-24T06:33:24.944 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:24.083+0000 7f3fee400700 1 -- 172.21.15.5:0/2610928728 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f3fcc039610 msgr2=0x7f3fcc03bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:24.944 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:24.083+0000 7f3fee400700 1 --2- 172.21.15.5:0/2610928728 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f3fcc039610 0x7f3fcc03bad0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4006fd0 tx=0x7f3fd4006e40 comp rx=0 tx=0).stop 2024-04-24T06:33:24.944 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:24.083+0000 7f3fee400700 1 -- 172.21.15.5:0/2610928728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe810a8e0 msgr2=0x7f3fe819e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:24.944 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:24.083+0000 7f3fee400700 1 --2- 172.21.15.5:0/2610928728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe810a8e0 0x7f3fe819e800 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8011670 tx=0x7f3fd8011b60 comp rx=0 tx=0).stop 2024-04-24T06:33:24.944 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:24.084+0000 7f3fee400700 1 -- 172.21.15.5:0/2610928728 shutdown_connections 2024-04-24T06:33:24.944 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:24.084+0000 7f3fee400700 1 --2- 172.21.15.5:0/2610928728 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f3fcc039610 0x7f3fcc03bad0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:24.944 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:24.084+0000 7f3fee400700 1 --2- 172.21.15.5:0/2610928728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe810a8e0 0x7f3fe819e800 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:24.944 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:24.084+0000 7f3fee400700 1 -- 172.21.15.5:0/2610928728 >> 172.21.15.5:0/2610928728 conn(0x7f3fe807de60 msgr2=0x7f3fe8107be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:24.944 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:24.084+0000 7f3fee400700 1 -- 172.21.15.5:0/2610928728 shutdown_connections 2024-04-24T06:33:24.945 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:24.084+0000 7f3fee400700 1 -- 172.21.15.5:0/2610928728 wait complete. 2024-04-24T06:33:24.945 INFO:teuthology.orchestra.run.smithi005.stdout:Deploying unmanaged mgr service... 2024-04-24T06:33:25.343 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:25 smithi005 ceph-mon[27798]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:33:25.343 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:25 smithi005 ceph-mon[27798]: Saving service mon spec with placement count:5 2024-04-24T06:33:25.343 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:25 smithi005 ceph-mon[27798]: from='mgr.14120 172.21.15.5:0/2924182213' entity='mgr.x' 2024-04-24T06:33:26.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-24T06:33:26.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.649+0000 7fe04b600700 1 Processor -- start 2024-04-24T06:33:26.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.649+0000 7fe04b600700 1 -- start start 2024-04-24T06:33:26.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.649+0000 7fe04b600700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe04410a900 0x7fe04410ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:26.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.650+0000 7fe04b600700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe04410b2f0 con 0x7fe04410a900 2024-04-24T06:33:26.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.650+0000 7fe048c00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe04410a900 0x7fe04410ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:26.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.650+0000 7fe048c00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe04410a900 0x7fe04410ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42206/0 (socket says 172.21.15.5:42206) 2024-04-24T06:33:26.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.650+0000 7fe048c00700 1 -- 172.21.15.5:0/275563392 learned_addr learned my addr 172.21.15.5:0/275563392 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:33:26.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.650+0000 7fe048c00700 1 -- 172.21.15.5:0/275563392 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe04410bab0 con 0x7fe04410a900 2024-04-24T06:33:26.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.651+0000 7fe048c00700 1 --2- 172.21.15.5:0/275563392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe04410a900 0x7fe04410ad20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe0340098a0 tx=0x7fe034009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50b56cc12b198c2c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:26.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.651+0000 7fe043e00700 1 -- 172.21.15.5:0/275563392 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe034004160 con 0x7fe04410a900 2024-04-24T06:33:26.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.651+0000 7fe043e00700 1 -- 172.21.15.5:0/275563392 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe034004300 con 0x7fe04410a900 2024-04-24T06:33:26.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.651+0000 7fe043e00700 1 -- 172.21.15.5:0/275563392 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe034011b50 con 0x7fe04410a900 2024-04-24T06:33:26.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.652+0000 7fe04b600700 1 -- 172.21.15.5:0/275563392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe04410a900 msgr2=0x7fe04410ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:26.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.652+0000 7fe04b600700 1 --2- 172.21.15.5:0/275563392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe04410a900 0x7fe04410ad20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe0340098a0 tx=0x7fe034009bb0 comp rx=0 tx=0).stop 2024-04-24T06:33:26.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.652+0000 7fe04b600700 1 -- 172.21.15.5:0/275563392 shutdown_connections 2024-04-24T06:33:26.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.652+0000 7fe04b600700 1 --2- 172.21.15.5:0/275563392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe04410a900 0x7fe04410ad20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:26.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.652+0000 7fe04b600700 1 -- 172.21.15.5:0/275563392 >> 172.21.15.5:0/275563392 conn(0x7fe04407de00 msgr2=0x7fe0441084f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:26.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.652+0000 7fe04b600700 1 -- 172.21.15.5:0/275563392 shutdown_connections 2024-04-24T06:33:26.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.652+0000 7fe04b600700 1 -- 172.21.15.5:0/275563392 wait complete. 2024-04-24T06:33:26.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.654+0000 7fe04b600700 1 Processor -- start 2024-04-24T06:33:26.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.654+0000 7fe04b600700 1 -- start start 2024-04-24T06:33:26.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.654+0000 7fe04b600700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe04410a900 0x7fe04419e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:26.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.654+0000 7fe04b600700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe04410ee60 con 0x7fe04410a900 2024-04-24T06:33:26.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.654+0000 7fe048c00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe04410a900 0x7fe04419e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:26.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.655+0000 7fe048c00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe04410a900 0x7fe04419e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42210/0 (socket says 172.21.15.5:42210) 2024-04-24T06:33:26.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.655+0000 7fe048c00700 1 -- 172.21.15.5:0/347446896 learned_addr learned my addr 172.21.15.5:0/347446896 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:33:26.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.655+0000 7fe048c00700 1 -- 172.21.15.5:0/347446896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe04419ed20 con 0x7fe04410a900 2024-04-24T06:33:26.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.655+0000 7fe048c00700 1 --2- 172.21.15.5:0/347446896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe04410a900 0x7fe04419e7e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe034000c00 tx=0x7fe034011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:26.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.656+0000 7fe042000700 1 -- 172.21.15.5:0/347446896 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe034003b20 con 0x7fe04410a900 2024-04-24T06:33:26.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.656+0000 7fe042000700 1 -- 172.21.15.5:0/347446896 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe034003cc0 con 0x7fe04410a900 2024-04-24T06:33:26.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.656+0000 7fe042000700 1 -- 172.21.15.5:0/347446896 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe034024910 con 0x7fe04410a900 2024-04-24T06:33:26.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.656+0000 7fe04b600700 1 -- 172.21.15.5:0/347446896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe04419efb0 con 0x7fe04410a900 2024-04-24T06:33:26.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.656+0000 7fe04b600700 1 -- 172.21.15.5:0/347446896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0441a1ca0 con 0x7fe04410a900 2024-04-24T06:33:26.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.657+0000 7fe042000700 1 -- 172.21.15.5:0/347446896 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fe034022020 con 0x7fe04410a900 2024-04-24T06:33:26.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.658+0000 7fe04b600700 1 -- 172.21.15.5:0/347446896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe044064150 con 0x7fe04410a900 2024-04-24T06:33:26.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.658+0000 7fe042000700 1 --2- 172.21.15.5:0/347446896 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7fe028039600 0x7fe02803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:26.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.658+0000 7fe042000700 1 -- 172.21.15.5:0/347446896 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe034011e00 con 0x7fe04410a900 2024-04-24T06:33:26.989 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.658+0000 7fe03be00700 1 --2- 172.21.15.5:0/347446896 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7fe028039600 0x7fe02803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:26.989 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.659+0000 7fe03be00700 1 --2- 172.21.15.5:0/347446896 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7fe028039600 0x7fe02803bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe02c006fd0 tx=0x7fe02c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:26.989 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.666+0000 7fe042000700 1 -- 172.21.15.5:0/347446896 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe03401ad80 con 0x7fe04410a900 2024-04-24T06:33:26.989 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.917+0000 7fe04b600700 1 -- 172.21.15.5:0/347446896 --> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fe0441082f0 con 0x7fe028039600 2024-04-24T06:33:26.989 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.925+0000 7fe042000700 1 -- 172.21.15.5:0/347446896 <== mgr.14120 v2:172.21.15.5:6800/2147029578 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe0441082f0 con 0x7fe028039600 2024-04-24T06:33:26.989 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.928+0000 7fe04b600700 1 -- 172.21.15.5:0/347446896 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7fe028039600 msgr2=0x7fe02803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:26.989 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.928+0000 7fe04b600700 1 --2- 172.21.15.5:0/347446896 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7fe028039600 0x7fe02803bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe02c006fd0 tx=0x7fe02c006e40 comp rx=0 tx=0).stop 2024-04-24T06:33:26.990 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.928+0000 7fe04b600700 1 -- 172.21.15.5:0/347446896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe04410a900 msgr2=0x7fe04419e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:26.990 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.928+0000 7fe04b600700 1 --2- 172.21.15.5:0/347446896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe04410a900 0x7fe04419e7e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe034000c00 tx=0x7fe034011c10 comp rx=0 tx=0).stop 2024-04-24T06:33:26.990 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.928+0000 7fe04b600700 1 -- 172.21.15.5:0/347446896 shutdown_connections 2024-04-24T06:33:26.990 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.928+0000 7fe04b600700 1 --2- 172.21.15.5:0/347446896 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7fe028039600 0x7fe02803bac0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:26.990 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.928+0000 7fe04b600700 1 --2- 172.21.15.5:0/347446896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe04410a900 0x7fe04419e7e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:26.990 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.928+0000 7fe04b600700 1 -- 172.21.15.5:0/347446896 >> 172.21.15.5:0/347446896 conn(0x7fe04407de00 msgr2=0x7fe044107be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:26.991 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.928+0000 7fe04b600700 1 -- 172.21.15.5:0/347446896 shutdown_connections 2024-04-24T06:33:26.991 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:25.928+0000 7fe04b600700 1 -- 172.21.15.5:0/347446896 wait complete. 2024-04-24T06:33:27.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:26 smithi005 ceph-mon[27798]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:33:27.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:26 smithi005 ceph-mon[27798]: Saving service mgr spec with placement count:2 2024-04-24T06:33:27.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:26 smithi005 ceph-mon[27798]: from='mgr.14120 172.21.15.5:0/2924182213' entity='mgr.x' 2024-04-24T06:33:28.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:28 smithi005 ceph-mon[27798]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:33:28.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:28 smithi005 ceph-mon[27798]: from='mgr.14120 172.21.15.5:0/2924182213' entity='mgr.x' 2024-04-24T06:33:28.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:28 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/2399573700' entity='client.admin' 2024-04-24T06:33:28.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:28 smithi005 ceph-mon[27798]: from='mgr.14120 172.21.15.5:0/2924182213' entity='mgr.x' 2024-04-24T06:33:28.907 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:27.947+0000 7f9952c00700 1 Processor -- start 2024-04-24T06:33:28.907 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:27.947+0000 7f9952c00700 1 -- start start 2024-04-24T06:33:28.907 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:27.948+0000 7f9952c00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f994c106f60 0x7f994c109390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:28.907 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:27.948+0000 7f9952c00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f994c07bed0 con 0x7f994c106f60 2024-04-24T06:33:28.907 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:27.949+0000 7f994be00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f994c106f60 0x7f994c109390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:28.907 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:27.949+0000 7f994be00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f994c106f60 0x7f994c109390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42220/0 (socket says 172.21.15.5:42220) 2024-04-24T06:33:28.907 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:27.949+0000 7f994be00700 1 -- 172.21.15.5:0/1896871457 learned_addr learned my addr 172.21.15.5:0/1896871457 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:33:28.908 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:27.949+0000 7f994be00700 1 -- 172.21.15.5:0/1896871457 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994c1098d0 con 0x7f994c106f60 2024-04-24T06:33:28.908 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:27.949+0000 7f994be00700 1 --2- 172.21.15.5:0/1896871457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f994c106f60 0x7f994c109390 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f993c0098a0 tx=0x7f993c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926d9a2ee9970054 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:28.908 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:27.950+0000 7f994aa00700 1 -- 172.21.15.5:0/1896871457 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f993c004160 con 0x7f994c106f60 2024-04-24T06:33:28.908 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:27.950+0000 7f994aa00700 1 -- 172.21.15.5:0/1896871457 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f993c004300 con 0x7f994c106f60 2024-04-24T06:33:28.908 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:27.950+0000 7f994aa00700 1 -- 172.21.15.5:0/1896871457 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f993c011b20 con 0x7f994c106f60 2024-04-24T06:33:28.908 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:27.950+0000 7f9952c00700 1 -- 172.21.15.5:0/1896871457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f994c106f60 msgr2=0x7f994c109390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:28.908 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:27.950+0000 7f9952c00700 1 --2- 172.21.15.5:0/1896871457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f994c106f60 0x7f994c109390 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f993c0098a0 tx=0x7f993c009bb0 comp rx=0 tx=0).stop 2024-04-24T06:33:28.908 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:27.951+0000 7f9952c00700 1 -- 172.21.15.5:0/1896871457 shutdown_connections 2024-04-24T06:33:28.908 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:27.951+0000 7f9952c00700 1 --2- 172.21.15.5:0/1896871457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f994c106f60 0x7f994c109390 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:28.908 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:27.951+0000 7f9952c00700 1 -- 172.21.15.5:0/1896871457 >> 172.21.15.5:0/1896871457 conn(0x7f994c102c00 msgr2=0x7f994c105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:28.908 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:27.951+0000 7f9952c00700 1 -- 172.21.15.5:0/1896871457 shutdown_connections 2024-04-24T06:33:28.908 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:27.951+0000 7f9952c00700 1 -- 172.21.15.5:0/1896871457 wait complete. 2024-04-24T06:33:28.908 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:27.952+0000 7f9952c00700 1 Processor -- start 2024-04-24T06:33:28.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:27.952+0000 7f9952c00700 1 -- start start 2024-04-24T06:33:28.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:27.952+0000 7f9952c00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f994c106f60 0x7f994c1a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:28.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:27.952+0000 7f9952c00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f994c10a9a0 con 0x7f994c106f60 2024-04-24T06:33:28.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:27.953+0000 7f994be00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f994c106f60 0x7f994c1a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:28.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:27.953+0000 7f994be00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f994c106f60 0x7f994c1a2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42236/0 (socket says 172.21.15.5:42236) 2024-04-24T06:33:28.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:27.953+0000 7f994be00700 1 -- 172.21.15.5:0/2399573700 learned_addr learned my addr 172.21.15.5:0/2399573700 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:33:28.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:27.953+0000 7f994be00700 1 -- 172.21.15.5:0/2399573700 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994c1a3190 con 0x7f994c106f60 2024-04-24T06:33:28.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:27.954+0000 7f994be00700 1 --2- 172.21.15.5:0/2399573700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f994c106f60 0x7f994c1a2c50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f993c011b60 tx=0x7f993c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:28.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:27.954+0000 7f9948c00700 1 -- 172.21.15.5:0/2399573700 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f993c003b60 con 0x7f994c106f60 2024-04-24T06:33:28.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:27.954+0000 7f9948c00700 1 -- 172.21.15.5:0/2399573700 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f993c01a910 con 0x7f994c106f60 2024-04-24T06:33:28.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:27.954+0000 7f9952c00700 1 -- 172.21.15.5:0/2399573700 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f994c1a3420 con 0x7f994c106f60 2024-04-24T06:33:28.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:27.954+0000 7f9948c00700 1 -- 172.21.15.5:0/2399573700 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f993c024930 con 0x7f994c106f60 2024-04-24T06:33:28.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:27.954+0000 7f9952c00700 1 -- 172.21.15.5:0/2399573700 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f994c1a3880 con 0x7f994c106f60 2024-04-24T06:33:28.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:27.955+0000 7f9952c00700 1 -- 172.21.15.5:0/2399573700 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f994c064150 con 0x7f994c106f60 2024-04-24T06:33:28.910 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:27.956+0000 7f9948c00700 1 -- 172.21.15.5:0/2399573700 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f993c022020 con 0x7f994c106f60 2024-04-24T06:33:28.910 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:27.956+0000 7f9948c00700 1 --2- 172.21.15.5:0/2399573700 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f9930039600 0x7f993003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:28.910 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:27.956+0000 7f9948c00700 1 -- 172.21.15.5:0/2399573700 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f993c04d960 con 0x7f994c106f60 2024-04-24T06:33:28.910 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:27.957+0000 7f994b400700 1 --2- 172.21.15.5:0/2399573700 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f9930039600 0x7f993003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:28.910 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:27.957+0000 7f994b400700 1 --2- 172.21.15.5:0/2399573700 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f9930039600 0x7f993003bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9938006fd0 tx=0x7f9938006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:28.910 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:27.962+0000 7f9948c00700 1 -- 172.21.15.5:0/2399573700 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f993c04d2b0 con 0x7f994c106f60 2024-04-24T06:33:28.910 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:28.205+0000 7f9952c00700 1 -- 172.21.15.5:0/2399573700 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f994c063700 con 0x7f994c106f60 2024-04-24T06:33:28.910 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:28.209+0000 7f9948c00700 1 -- 172.21.15.5:0/2399573700 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f993c01ab70 con 0x7f994c106f60 2024-04-24T06:33:28.910 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:28.212+0000 7f9952c00700 1 -- 172.21.15.5:0/2399573700 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f9930039600 msgr2=0x7f993003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:28.910 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:28.212+0000 7f9952c00700 1 --2- 172.21.15.5:0/2399573700 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f9930039600 0x7f993003bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9938006fd0 tx=0x7f9938006e40 comp rx=0 tx=0).stop 2024-04-24T06:33:28.910 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:28.212+0000 7f9952c00700 1 -- 172.21.15.5:0/2399573700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f994c106f60 msgr2=0x7f994c1a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:28.910 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:28.212+0000 7f9952c00700 1 --2- 172.21.15.5:0/2399573700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f994c106f60 0x7f994c1a2c50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f993c011b60 tx=0x7f993c011c40 comp rx=0 tx=0).stop 2024-04-24T06:33:28.910 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:28.212+0000 7f9952c00700 1 -- 172.21.15.5:0/2399573700 shutdown_connections 2024-04-24T06:33:28.911 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:28.212+0000 7f9952c00700 1 --2- 172.21.15.5:0/2399573700 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f9930039600 0x7f993003bac0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:28.911 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:28.212+0000 7f9952c00700 1 --2- 172.21.15.5:0/2399573700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f994c106f60 0x7f994c1a2c50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:28.911 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:28.212+0000 7f9952c00700 1 -- 172.21.15.5:0/2399573700 >> 172.21.15.5:0/2399573700 conn(0x7f994c102c00 msgr2=0x7f994c104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:28.911 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:28.213+0000 7f9952c00700 1 -- 172.21.15.5:0/2399573700 shutdown_connections 2024-04-24T06:33:28.911 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:28.213+0000 7f9952c00700 1 -- 172.21.15.5:0/2399573700 wait complete. 2024-04-24T06:33:30.815 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.457+0000 7f2cc0c00700 1 Processor -- start 2024-04-24T06:33:30.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.457+0000 7f2cc0c00700 1 -- start start 2024-04-24T06:33:30.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.458+0000 7f2cc0c00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cbc109190 0x7f2cbc10b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:30.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.458+0000 7f2cc0c00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2cbc07beb0 con 0x7f2cbc109190 2024-04-24T06:33:30.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.458+0000 7f2cb9e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cbc109190 0x7f2cbc10b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:30.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.458+0000 7f2cb9e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cbc109190 0x7f2cbc10b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42244/0 (socket says 172.21.15.5:42244) 2024-04-24T06:33:30.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.458+0000 7f2cb9e00700 1 -- 172.21.15.5:0/2812754464 learned_addr learned my addr 172.21.15.5:0/2812754464 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:33:30.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.459+0000 7f2cb9e00700 1 -- 172.21.15.5:0/2812754464 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cbc10bac0 con 0x7f2cbc109190 2024-04-24T06:33:30.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.459+0000 7f2cb9e00700 1 --2- 172.21.15.5:0/2812754464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cbc109190 0x7f2cbc10b580 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2cac0098a0 tx=0x7f2cac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=386404257b09207b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:30.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.459+0000 7f2cb9400700 1 -- 172.21.15.5:0/2812754464 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2cac004160 con 0x7f2cbc109190 2024-04-24T06:33:30.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.460+0000 7f2cb9400700 1 -- 172.21.15.5:0/2812754464 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cac004300 con 0x7f2cbc109190 2024-04-24T06:33:30.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.460+0000 7f2cb9400700 1 -- 172.21.15.5:0/2812754464 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2cac011b20 con 0x7f2cbc109190 2024-04-24T06:33:30.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.460+0000 7f2cc0c00700 1 -- 172.21.15.5:0/2812754464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cbc109190 msgr2=0x7f2cbc10b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:30.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.460+0000 7f2cc0c00700 1 --2- 172.21.15.5:0/2812754464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cbc109190 0x7f2cbc10b580 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2cac0098a0 tx=0x7f2cac009bb0 comp rx=0 tx=0).stop 2024-04-24T06:33:30.817 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.460+0000 7f2cc0c00700 1 -- 172.21.15.5:0/2812754464 shutdown_connections 2024-04-24T06:33:30.817 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.460+0000 7f2cc0c00700 1 --2- 172.21.15.5:0/2812754464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cbc109190 0x7f2cbc10b580 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:30.817 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.460+0000 7f2cc0c00700 1 -- 172.21.15.5:0/2812754464 >> 172.21.15.5:0/2812754464 conn(0x7f2cbc102c00 msgr2=0x7f2cbc105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:30.817 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.461+0000 7f2cc0c00700 1 -- 172.21.15.5:0/2812754464 shutdown_connections 2024-04-24T06:33:30.817 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.461+0000 7f2cc0c00700 1 -- 172.21.15.5:0/2812754464 wait complete. 2024-04-24T06:33:30.817 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.462+0000 7f2cc0c00700 1 Processor -- start 2024-04-24T06:33:30.817 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.462+0000 7f2cc0c00700 1 -- start start 2024-04-24T06:33:30.817 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.463+0000 7f2cc0c00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cbc109190 0x7f2cbc108a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:30.817 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.463+0000 7f2cc0c00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2cbc10cac0 con 0x7f2cbc109190 2024-04-24T06:33:30.817 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.463+0000 7f2cb9e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cbc109190 0x7f2cbc108a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:30.817 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.463+0000 7f2cb9e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cbc109190 0x7f2cbc108a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42258/0 (socket says 172.21.15.5:42258) 2024-04-24T06:33:30.817 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.463+0000 7f2cb9e00700 1 -- 172.21.15.5:0/1534547686 learned_addr learned my addr 172.21.15.5:0/1534547686 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:33:30.820 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.463+0000 7f2cb9e00700 1 -- 172.21.15.5:0/1534547686 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cbc108f50 con 0x7f2cbc109190 2024-04-24T06:33:30.821 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.464+0000 7f2cb9e00700 1 --2- 172.21.15.5:0/1534547686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cbc109190 0x7f2cbc108a10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2cac011b60 tx=0x7f2cac011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:30.821 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.464+0000 7f2cb2a00700 1 -- 172.21.15.5:0/1534547686 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2cac003b60 con 0x7f2cbc109190 2024-04-24T06:33:30.821 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.464+0000 7f2cb2a00700 1 -- 172.21.15.5:0/1534547686 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cac01a910 con 0x7f2cbc109190 2024-04-24T06:33:30.821 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.464+0000 7f2cb2a00700 1 -- 172.21.15.5:0/1534547686 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2cac024930 con 0x7f2cbc109190 2024-04-24T06:33:30.821 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.464+0000 7f2cc0c00700 1 -- 172.21.15.5:0/1534547686 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cbc107140 con 0x7f2cbc109190 2024-04-24T06:33:30.821 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.464+0000 7f2cc0c00700 1 -- 172.21.15.5:0/1534547686 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cbc107600 con 0x7f2cbc109190 2024-04-24T06:33:30.821 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.465+0000 7f2cc0c00700 1 -- 172.21.15.5:0/1534547686 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cbc064150 con 0x7f2cbc109190 2024-04-24T06:33:30.821 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.466+0000 7f2cb2a00700 1 -- 172.21.15.5:0/1534547686 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f2cac022020 con 0x7f2cbc109190 2024-04-24T06:33:30.821 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.467+0000 7f2cb2a00700 1 --2- 172.21.15.5:0/1534547686 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f2ca0039660 0x7f2ca003bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:30.821 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.467+0000 7f2cb2a00700 1 -- 172.21.15.5:0/1534547686 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2cac04dbc0 con 0x7f2cbc109190 2024-04-24T06:33:30.821 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.467+0000 7f2cb3e00700 1 --2- 172.21.15.5:0/1534547686 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f2ca0039660 0x7f2ca003bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:30.822 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.468+0000 7f2cb3e00700 1 --2- 172.21.15.5:0/1534547686 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f2ca0039660 0x7f2ca003bb20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4006fd0 tx=0x7f2ca4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:30.822 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.472+0000 7f2cb2a00700 1 -- 172.21.15.5:0/1534547686 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2cac029060 con 0x7f2cbc109190 2024-04-24T06:33:30.822 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.750+0000 7f2cc0c00700 1 -- 172.21.15.5:0/1534547686 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f2cbc107eb0 con 0x7f2cbc109190 2024-04-24T06:33:30.822 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.755+0000 7f2cb2a00700 1 -- 172.21.15.5:0/1534547686 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f2cac00b3e0 con 0x7f2cbc109190 2024-04-24T06:33:30.822 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.758+0000 7f2cc0c00700 1 -- 172.21.15.5:0/1534547686 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f2ca0039660 msgr2=0x7f2ca003bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:30.822 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.758+0000 7f2cc0c00700 1 --2- 172.21.15.5:0/1534547686 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f2ca0039660 0x7f2ca003bb20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4006fd0 tx=0x7f2ca4006e40 comp rx=0 tx=0).stop 2024-04-24T06:33:30.822 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.758+0000 7f2cc0c00700 1 -- 172.21.15.5:0/1534547686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cbc109190 msgr2=0x7f2cbc108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:30.822 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.758+0000 7f2cc0c00700 1 --2- 172.21.15.5:0/1534547686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cbc109190 0x7f2cbc108a10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2cac011b60 tx=0x7f2cac011c40 comp rx=0 tx=0).stop 2024-04-24T06:33:30.822 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.758+0000 7f2cc0c00700 1 -- 172.21.15.5:0/1534547686 shutdown_connections 2024-04-24T06:33:30.822 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.758+0000 7f2cc0c00700 1 --2- 172.21.15.5:0/1534547686 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f2ca0039660 0x7f2ca003bb20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:30.822 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.758+0000 7f2cc0c00700 1 --2- 172.21.15.5:0/1534547686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cbc109190 0x7f2cbc108a10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:30.822 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.758+0000 7f2cc0c00700 1 -- 172.21.15.5:0/1534547686 >> 172.21.15.5:0/1534547686 conn(0x7f2cbc102c00 msgr2=0x7f2cbc1036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:30.822 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.759+0000 7f2cc0c00700 1 -- 172.21.15.5:0/1534547686 shutdown_connections 2024-04-24T06:33:30.823 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:29.759+0000 7f2cc0c00700 1 -- 172.21.15.5:0/1534547686 wait complete. 2024-04-24T06:33:30.823 INFO:teuthology.orchestra.run.smithi005.stdout:Enabling the dashboard module... 2024-04-24T06:33:31.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:30 smithi005 ceph-mon[27798]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:33:31.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:30 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1534547686' entity='client.admin' 2024-04-24T06:33:32.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:31 smithi005 ceph-mon[27798]: from='mgr.14120 172.21.15.5:0/2924182213' entity='mgr.x' 2024-04-24T06:33:32.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:31 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1644092926' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-24T06:33:32.256 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:32 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: ignoring --setuser ceph since I am not root 2024-04-24T06:33:32.256 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:32 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: ignoring --setgroup ceph since I am not root 2024-04-24T06:33:32.256 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:32 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:32.254+0000 7f5a76c2a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T06:33:32.681 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.434+0000 7f98d4800700 1 Processor -- start 2024-04-24T06:33:32.681 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.434+0000 7f98d4800700 1 -- start start 2024-04-24T06:33:32.682 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.434+0000 7f98d4800700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98cc108700 0x7f98cc108b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:32.682 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.434+0000 7f98d4800700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc1090f0 con 0x7f98cc108700 2024-04-24T06:33:32.682 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.435+0000 7f98d1e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98cc108700 0x7f98cc108b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:32.682 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.435+0000 7f98d1e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98cc108700 0x7f98cc108b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42266/0 (socket says 172.21.15.5:42266) 2024-04-24T06:33:32.682 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.435+0000 7f98d1e00700 1 -- 172.21.15.5:0/1841280525 learned_addr learned my addr 172.21.15.5:0/1841280525 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:33:32.682 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.435+0000 7f98d1e00700 1 -- 172.21.15.5:0/1841280525 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98cc1098b0 con 0x7f98cc108700 2024-04-24T06:33:32.682 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.436+0000 7f98d1e00700 1 --2- 172.21.15.5:0/1841280525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98cc108700 0x7f98cc108b20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f98c00098a0 tx=0x7f98c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d78cedb36c7325e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:32.682 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.436+0000 7f98d0a00700 1 -- 172.21.15.5:0/1841280525 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98c0004160 con 0x7f98cc108700 2024-04-24T06:33:32.682 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.436+0000 7f98d0a00700 1 -- 172.21.15.5:0/1841280525 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98c0004300 con 0x7f98cc108700 2024-04-24T06:33:32.682 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.436+0000 7f98d0a00700 1 -- 172.21.15.5:0/1841280525 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98c0011b20 con 0x7f98cc108700 2024-04-24T06:33:32.682 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.437+0000 7f98d4800700 1 -- 172.21.15.5:0/1841280525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98cc108700 msgr2=0x7f98cc108b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:32.682 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.437+0000 7f98d4800700 1 --2- 172.21.15.5:0/1841280525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98cc108700 0x7f98cc108b20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f98c00098a0 tx=0x7f98c0009bb0 comp rx=0 tx=0).stop 2024-04-24T06:33:32.683 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.437+0000 7f98d4800700 1 -- 172.21.15.5:0/1841280525 shutdown_connections 2024-04-24T06:33:32.683 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.437+0000 7f98d4800700 1 --2- 172.21.15.5:0/1841280525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98cc108700 0x7f98cc108b20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:32.683 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.437+0000 7f98d4800700 1 -- 172.21.15.5:0/1841280525 >> 172.21.15.5:0/1841280525 conn(0x7f98cc103d40 msgr2=0x7f98cc106160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:32.683 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.437+0000 7f98d4800700 1 -- 172.21.15.5:0/1841280525 shutdown_connections 2024-04-24T06:33:32.683 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.437+0000 7f98d4800700 1 -- 172.21.15.5:0/1841280525 wait complete. 2024-04-24T06:33:32.683 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.438+0000 7f98d4800700 1 Processor -- start 2024-04-24T06:33:32.683 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.438+0000 7f98d4800700 1 -- start start 2024-04-24T06:33:32.683 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.439+0000 7f98d4800700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98cc108700 0x7f98cc19c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:32.683 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.439+0000 7f98d4800700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc10cc60 con 0x7f98cc108700 2024-04-24T06:33:32.683 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.439+0000 7f98d1e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98cc108700 0x7f98cc19c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:32.683 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.439+0000 7f98d1e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98cc108700 0x7f98cc19c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42280/0 (socket says 172.21.15.5:42280) 2024-04-24T06:33:32.684 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.439+0000 7f98d1e00700 1 -- 172.21.15.5:0/1644092926 learned_addr learned my addr 172.21.15.5:0/1644092926 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:33:32.684 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.439+0000 7f98d1e00700 1 -- 172.21.15.5:0/1644092926 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98cc19cb20 con 0x7f98cc108700 2024-04-24T06:33:32.684 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.440+0000 7f98d1e00700 1 --2- 172.21.15.5:0/1644092926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98cc108700 0x7f98cc19c5e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f98c00117e0 tx=0x7f98c0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:32.684 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.440+0000 7f98c6a00700 1 -- 172.21.15.5:0/1644092926 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98c0003bf0 con 0x7f98cc108700 2024-04-24T06:33:32.684 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.440+0000 7f98c6a00700 1 -- 172.21.15.5:0/1644092926 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98c001a910 con 0x7f98cc108700 2024-04-24T06:33:32.684 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.440+0000 7f98c6a00700 1 -- 172.21.15.5:0/1644092926 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98c0024970 con 0x7f98cc108700 2024-04-24T06:33:32.684 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.440+0000 7f98d4800700 1 -- 172.21.15.5:0/1644092926 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98cc19cdb0 con 0x7f98cc108700 2024-04-24T06:33:32.684 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.440+0000 7f98d4800700 1 -- 172.21.15.5:0/1644092926 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98cc19d210 con 0x7f98cc108700 2024-04-24T06:33:32.684 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.441+0000 7f98d4800700 1 -- 172.21.15.5:0/1644092926 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98cc064150 con 0x7f98cc108700 2024-04-24T06:33:32.684 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.442+0000 7f98c6a00700 1 -- 172.21.15.5:0/1644092926 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f98c0022020 con 0x7f98cc108700 2024-04-24T06:33:32.684 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.442+0000 7f98c6a00700 1 --2- 172.21.15.5:0/1644092926 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f98b4039660 0x7f98b403bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:32.684 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.442+0000 7f98c6a00700 1 -- 172.21.15.5:0/1644092926 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f98c004dad0 con 0x7f98cc108700 2024-04-24T06:33:32.684 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.443+0000 7f98d1400700 1 --2- 172.21.15.5:0/1644092926 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f98b4039660 0x7f98b403bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:32.685 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.444+0000 7f98d1400700 1 --2- 172.21.15.5:0/1644092926 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f98b4039660 0x7f98b403bb20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f98bc006fd0 tx=0x7f98bc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:32.685 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.449+0000 7f98c6a00700 1 -- 172.21.15.5:0/1644092926 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f98c001ab70 con 0x7f98cc108700 2024-04-24T06:33:32.685 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.756+0000 7f98d4800700 1 -- 172.21.15.5:0/1644092926 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f98cc19fbe0 con 0x7f98cc108700 2024-04-24T06:33:32.685 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.920+0000 7f98c6a00700 1 -- 172.21.15.5:0/1644092926 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f98c002d3f0 con 0x7f98cc108700 2024-04-24T06:33:32.685 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.922+0000 7f98c6a00700 1 -- 172.21.15.5:0/1644092926 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f98c001a430 con 0x7f98cc108700 2024-04-24T06:33:32.685 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.924+0000 7f98d4800700 1 -- 172.21.15.5:0/1644092926 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f98b4039660 msgr2=0x7f98b403bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:32.685 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.924+0000 7f98d4800700 1 --2- 172.21.15.5:0/1644092926 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f98b4039660 0x7f98b403bb20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f98bc006fd0 tx=0x7f98bc006e40 comp rx=0 tx=0).stop 2024-04-24T06:33:32.685 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.924+0000 7f98d4800700 1 -- 172.21.15.5:0/1644092926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98cc108700 msgr2=0x7f98cc19c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:32.685 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.924+0000 7f98d4800700 1 --2- 172.21.15.5:0/1644092926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98cc108700 0x7f98cc19c5e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f98c00117e0 tx=0x7f98c0011ba0 comp rx=0 tx=0).stop 2024-04-24T06:33:32.685 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.924+0000 7f98d4800700 1 -- 172.21.15.5:0/1644092926 shutdown_connections 2024-04-24T06:33:32.685 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.924+0000 7f98d4800700 1 --2- 172.21.15.5:0/1644092926 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f98b4039660 0x7f98b403bb20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:32.685 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.924+0000 7f98d4800700 1 --2- 172.21.15.5:0/1644092926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98cc108700 0x7f98cc19c5e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:32.685 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.924+0000 7f98d4800700 1 -- 172.21.15.5:0/1644092926 >> 172.21.15.5:0/1644092926 conn(0x7f98cc103d40 msgr2=0x7f98cc104830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:32.686 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.925+0000 7f98d4800700 1 -- 172.21.15.5:0/1644092926 shutdown_connections 2024-04-24T06:33:32.686 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:31.925+0000 7f98d4800700 1 -- 172.21.15.5:0/1644092926 wait complete. 2024-04-24T06:33:32.827 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:32 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:32.509+0000 7f5a76c2a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T06:33:32.828 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:32 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:32.644+0000 7f5a76c2a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T06:33:33.085 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:32 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1644092926' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-24T06:33:33.085 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:32 smithi005 ceph-mon[27798]: mgrmap e9: x(active, since 24s) 2024-04-24T06:33:33.085 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:33 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:33.072+0000 7f5a76c2a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T06:33:33.443 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:33 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:33.191+0000 7f5a76c2a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T06:33:34.208 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:33 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/4264423935' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-24T06:33:34.219 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-04-24T06:33:34.220 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-24T06:33:34.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-24T06:33:34.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-24T06:33:34.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-24T06:33:34.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-04-24T06:33:34.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.232+0000 7f27ac600700 1 Processor -- start 2024-04-24T06:33:34.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.232+0000 7f27ac600700 1 -- start start 2024-04-24T06:33:34.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.232+0000 7f27ac600700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27a407de90 0x7f27a407c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:34.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.232+0000 7f27ac600700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f27a407c910 con 0x7f27a407de90 2024-04-24T06:33:34.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.233+0000 7f27a9c00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27a407de90 0x7f27a407c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:34.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.233+0000 7f27a9c00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27a407de90 0x7f27a407c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42298/0 (socket says 172.21.15.5:42298) 2024-04-24T06:33:34.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.233+0000 7f27a9c00700 1 -- 172.21.15.5:0/1404521595 learned_addr learned my addr 172.21.15.5:0/1404521595 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:33:34.227 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.233+0000 7f27a9c00700 1 -- 172.21.15.5:0/1404521595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27a407ca90 con 0x7f27a407de90 2024-04-24T06:33:34.227 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.234+0000 7f27a9c00700 1 --2- 172.21.15.5:0/1404521595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27a407de90 0x7f27a407c340 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f27980098a0 tx=0x7f2798009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c688f1ff776f0e7e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:34.227 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.234+0000 7f27a3e00700 1 -- 172.21.15.5:0/1404521595 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2798004160 con 0x7f27a407de90 2024-04-24T06:33:34.227 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.234+0000 7f27a3e00700 1 -- 172.21.15.5:0/1404521595 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2798004300 con 0x7f27a407de90 2024-04-24T06:33:34.227 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.234+0000 7f27a3e00700 1 -- 172.21.15.5:0/1404521595 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2798011b20 con 0x7f27a407de90 2024-04-24T06:33:34.227 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.235+0000 7f27ac600700 1 -- 172.21.15.5:0/1404521595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27a407de90 msgr2=0x7f27a407c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:34.227 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.235+0000 7f27ac600700 1 --2- 172.21.15.5:0/1404521595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27a407de90 0x7f27a407c340 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f27980098a0 tx=0x7f2798009bb0 comp rx=0 tx=0).stop 2024-04-24T06:33:34.227 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.235+0000 7f27ac600700 1 -- 172.21.15.5:0/1404521595 shutdown_connections 2024-04-24T06:33:34.227 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.235+0000 7f27ac600700 1 --2- 172.21.15.5:0/1404521595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27a407de90 0x7f27a407c340 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:34.227 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.235+0000 7f27ac600700 1 -- 172.21.15.5:0/1404521595 >> 172.21.15.5:0/1404521595 conn(0x7f27a4103d40 msgr2=0x7f27a4106160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:34.227 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.235+0000 7f27ac600700 1 -- 172.21.15.5:0/1404521595 shutdown_connections 2024-04-24T06:33:34.227 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.235+0000 7f27ac600700 1 -- 172.21.15.5:0/1404521595 wait complete. 2024-04-24T06:33:34.227 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.236+0000 7f27ac600700 1 Processor -- start 2024-04-24T06:33:34.227 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.236+0000 7f27ac600700 1 -- start start 2024-04-24T06:33:34.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.237+0000 7f27ac600700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27a407de90 0x7f27a419e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:34.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.237+0000 7f27ac600700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f27a410ece0 con 0x7f27a407de90 2024-04-24T06:33:34.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.237+0000 7f27a9c00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27a407de90 0x7f27a419e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:34.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.237+0000 7f27a9c00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27a407de90 0x7f27a419e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42312/0 (socket says 172.21.15.5:42312) 2024-04-24T06:33:34.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.237+0000 7f27a9c00700 1 -- 172.21.15.5:0/4264423935 learned_addr learned my addr 172.21.15.5:0/4264423935 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:33:34.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.237+0000 7f27a9c00700 1 -- 172.21.15.5:0/4264423935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27a419edb0 con 0x7f27a407de90 2024-04-24T06:33:34.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.238+0000 7f27a9c00700 1 --2- 172.21.15.5:0/4264423935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27a407de90 0x7f27a419e870 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2798011b60 tx=0x7f2798011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:34.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.238+0000 7f27a2000700 1 -- 172.21.15.5:0/4264423935 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2798003b60 con 0x7f27a407de90 2024-04-24T06:33:34.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.238+0000 7f27a2000700 1 -- 172.21.15.5:0/4264423935 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f279801a910 con 0x7f27a407de90 2024-04-24T06:33:34.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.238+0000 7f27ac600700 1 -- 172.21.15.5:0/4264423935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27a419f040 con 0x7f27a407de90 2024-04-24T06:33:34.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.238+0000 7f27a2000700 1 -- 172.21.15.5:0/4264423935 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2798024930 con 0x7f27a407de90 2024-04-24T06:33:34.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.238+0000 7f27ac600700 1 -- 172.21.15.5:0/4264423935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27a419f4a0 con 0x7f27a407de90 2024-04-24T06:33:34.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.239+0000 7f27a2000700 1 -- 172.21.15.5:0/4264423935 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f2798022020 con 0x7f27a407de90 2024-04-24T06:33:34.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.240+0000 7f27ac600700 1 -- 172.21.15.5:0/4264423935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27a4064150 con 0x7f27a407de90 2024-04-24T06:33:34.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.240+0000 7f27a2000700 1 --2- 172.21.15.5:0/4264423935 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f278c039660 0x7f278c03bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:34.229 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.240+0000 7f27a2000700 1 -- 172.21.15.5:0/4264423935 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f279804d930 con 0x7f27a407de90 2024-04-24T06:33:34.229 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.240+0000 7f27a9200700 1 -- 172.21.15.5:0/4264423935 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f278c039660 msgr2=0x7f278c03bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2147029578 2024-04-24T06:33:34.229 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.240+0000 7f27a9200700 1 --2- 172.21.15.5:0/4264423935 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f278c039660 0x7f278c03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T06:33:34.229 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.246+0000 7f27a2000700 1 -- 172.21.15.5:0/4264423935 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f279801a4e0 con 0x7f27a407de90 2024-04-24T06:33:34.229 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.441+0000 7f27a9200700 1 -- 172.21.15.5:0/4264423935 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f278c039660 msgr2=0x7f278c03bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2147029578 2024-04-24T06:33:34.229 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.441+0000 7f27a9200700 1 --2- 172.21.15.5:0/4264423935 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f278c039660 0x7f278c03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T06:33:34.229 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.535+0000 7f27ac600700 1 -- 172.21.15.5:0/4264423935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f27a41a1e00 con 0x7f27a407de90 2024-04-24T06:33:34.229 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.536+0000 7f27a2000700 1 -- 172.21.15.5:0/4264423935 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f279802d3f0 con 0x7f27a407de90 2024-04-24T06:33:34.229 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.539+0000 7f27ac600700 1 -- 172.21.15.5:0/4264423935 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f278c039660 msgr2=0x7f278c03bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T06:33:34.229 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.539+0000 7f27ac600700 1 --2- 172.21.15.5:0/4264423935 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f278c039660 0x7f278c03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:34.229 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.539+0000 7f27ac600700 1 -- 172.21.15.5:0/4264423935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27a407de90 msgr2=0x7f27a419e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:34.229 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.539+0000 7f27ac600700 1 --2- 172.21.15.5:0/4264423935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27a407de90 0x7f27a419e870 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2798011b60 tx=0x7f2798011c40 comp rx=0 tx=0).stop 2024-04-24T06:33:34.229 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.539+0000 7f27ac600700 1 -- 172.21.15.5:0/4264423935 shutdown_connections 2024-04-24T06:33:34.229 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.539+0000 7f27ac600700 1 --2- 172.21.15.5:0/4264423935 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7f278c039660 0x7f278c03bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:34.230 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.539+0000 7f27ac600700 1 --2- 172.21.15.5:0/4264423935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27a407de90 0x7f27a419e870 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:34.230 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.539+0000 7f27ac600700 1 -- 172.21.15.5:0/4264423935 >> 172.21.15.5:0/4264423935 conn(0x7f27a4103d40 msgr2=0x7f27a4104870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:34.230 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.539+0000 7f27ac600700 1 -- 172.21.15.5:0/4264423935 shutdown_connections 2024-04-24T06:33:34.230 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:33.539+0000 7f27ac600700 1 -- 172.21.15.5:0/4264423935 wait complete. 2024-04-24T06:33:34.230 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for the mgr to restart... 2024-04-24T06:33:34.230 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr epoch 9... 2024-04-24T06:33:35.027 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:34 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:34.802+0000 7f5a76c2a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T06:33:35.429 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:35 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:35.037+0000 7f5a76c2a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T06:33:35.429 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:35 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:35.256+0000 7f5a76c2a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T06:33:35.756 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:35 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:35.654+0000 7f5a76c2a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T06:33:36.041 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:35 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:35.780+0000 7f5a76c2a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T06:33:36.305 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:36 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:36.039+0000 7f5a76c2a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T06:33:36.305 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:36 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:36.303+0000 7f5a76c2a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T06:33:37.506 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:37 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:37.114+0000 7f5a76c2a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T06:33:38.009 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:37 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:37.684+0000 7f5a76c2a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T06:33:38.009 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:37 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:37.815+0000 7f5a76c2a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T06:33:38.506 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:38 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:38.089+0000 7f5a76c2a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T06:33:39.223 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:38 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:38.756+0000 7f5a76c2a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T06:33:39.223 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:38 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:38.887+0000 7f5a76c2a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T06:33:39.223 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:39 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:39.221+0000 7f5a76c2a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T06:33:39.506 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:39 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:39.406+0000 7f5a76c2a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T06:33:40.006 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:39 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:39.538+0000 7f5a76c2a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T06:33:40.007 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:39 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:39.696+0000 7f5a76c2a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T06:33:40.506 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:40 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:40.072+0000 7f5a76c2a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T06:33:41.159 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:40 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:40.862+0000 7f5a76c2a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T06:33:41.506 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:33:41 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:33:41.157+0000 7f5a76c2a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T06:33:41.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:41 smithi005 ceph-mon[27798]: Active manager daemon x restarted 2024-04-24T06:33:41.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:41 smithi005 ceph-mon[27798]: Activating manager daemon x 2024-04-24T06:33:41.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:41 smithi005 ceph-mon[27798]: osdmap e3: 0 total, 0 up, 0 in 2024-04-24T06:33:41.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:41 smithi005 ceph-mon[27798]: mgrmap e10: x(active, starting, since 0.0079648s) 2024-04-24T06:33:41.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:41 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T06:33:41.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:41 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T06:33:41.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:41 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T06:33:41.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:41 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T06:33:41.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:41 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T06:33:41.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:41 smithi005 ceph-mon[27798]: Manager daemon x is now available 2024-04-24T06:33:42.509 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:42 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:33:42.509 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:42 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T06:33:42.509 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:42 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T06:33:42.509 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:42 smithi005 ceph-mon[27798]: [24/Apr/2024:06:33:41] ENGINE Bus STARTING 2024-04-24T06:33:42.510 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:42 smithi005 ceph-mon[27798]: [24/Apr/2024:06:33:42] ENGINE Serving on https://172.21.15.5:7150 2024-04-24T06:33:42.510 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:42 smithi005 ceph-mon[27798]: mgrmap e11: x(active, since 1.00849s) 2024-04-24T06:33:43.029 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-04-24T06:33:43.029 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-24T06:33:43.029 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-24T06:33:43.029 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-04-24T06:33:43.029 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:34.816+0000 7fac8b400700 1 Processor -- start 2024-04-24T06:33:43.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:34.816+0000 7fac8b400700 1 -- start start 2024-04-24T06:33:43.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:34.817+0000 7fac8b400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac84109220 0x7fac8410b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:43.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:34.817+0000 7fac8b400700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fac8407be30 con 0x7fac84109220 2024-04-24T06:33:43.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:34.817+0000 7fac88a00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac84109220 0x7fac8410b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:43.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:34.817+0000 7fac88a00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac84109220 0x7fac8410b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52476/0 (socket says 172.21.15.5:52476) 2024-04-24T06:33:43.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:34.817+0000 7fac88a00700 1 -- 172.21.15.5:0/1791904055 learned_addr learned my addr 172.21.15.5:0/1791904055 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:33:43.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:34.818+0000 7fac88a00700 1 -- 172.21.15.5:0/1791904055 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac8410bb50 con 0x7fac84109220 2024-04-24T06:33:43.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:34.818+0000 7fac88a00700 1 --2- 172.21.15.5:0/1791904055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac84109220 0x7fac8410b610 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fac780098a0 tx=0x7fac78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2389f4dfde2fcedb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:43.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:34.818+0000 7fac83400700 1 -- 172.21.15.5:0/1791904055 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac78004160 con 0x7fac84109220 2024-04-24T06:33:43.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:34.819+0000 7fac83400700 1 -- 172.21.15.5:0/1791904055 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac78004300 con 0x7fac84109220 2024-04-24T06:33:43.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:34.819+0000 7fac83400700 1 -- 172.21.15.5:0/1791904055 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac78011b20 con 0x7fac84109220 2024-04-24T06:33:43.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:34.819+0000 7fac8b400700 1 -- 172.21.15.5:0/1791904055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac84109220 msgr2=0x7fac8410b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:43.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:34.819+0000 7fac8b400700 1 --2- 172.21.15.5:0/1791904055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac84109220 0x7fac8410b610 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fac780098a0 tx=0x7fac78009bb0 comp rx=0 tx=0).stop 2024-04-24T06:33:43.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:34.819+0000 7fac8b400700 1 -- 172.21.15.5:0/1791904055 shutdown_connections 2024-04-24T06:33:43.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:34.819+0000 7fac8b400700 1 --2- 172.21.15.5:0/1791904055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac84109220 0x7fac8410b610 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:43.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:34.819+0000 7fac8b400700 1 -- 172.21.15.5:0/1791904055 >> 172.21.15.5:0/1791904055 conn(0x7fac84102c30 msgr2=0x7fac84105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:43.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:34.820+0000 7fac8b400700 1 -- 172.21.15.5:0/1791904055 shutdown_connections 2024-04-24T06:33:43.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:34.820+0000 7fac8b400700 1 -- 172.21.15.5:0/1791904055 wait complete. 2024-04-24T06:33:43.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:34.821+0000 7fac8b400700 1 Processor -- start 2024-04-24T06:33:43.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:34.821+0000 7fac8b400700 1 -- start start 2024-04-24T06:33:43.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:34.821+0000 7fac8b400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac84109220 0x7fac84108aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:43.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:34.821+0000 7fac8b400700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fac8410cb50 con 0x7fac84109220 2024-04-24T06:33:43.032 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:34.822+0000 7fac88a00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac84109220 0x7fac84108aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:43.032 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:34.822+0000 7fac88a00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac84109220 0x7fac84108aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52492/0 (socket says 172.21.15.5:52492) 2024-04-24T06:33:43.032 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:34.822+0000 7fac88a00700 1 -- 172.21.15.5:0/2359021907 learned_addr learned my addr 172.21.15.5:0/2359021907 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:33:43.032 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:34.822+0000 7fac88a00700 1 -- 172.21.15.5:0/2359021907 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac84108fe0 con 0x7fac84109220 2024-04-24T06:33:43.032 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:34.822+0000 7fac88a00700 1 --2- 172.21.15.5:0/2359021907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac84109220 0x7fac84108aa0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fac78011b60 tx=0x7fac78011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:43.032 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:34.823+0000 7fac81600700 1 -- 172.21.15.5:0/2359021907 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac78003b60 con 0x7fac84109220 2024-04-24T06:33:43.032 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:34.823+0000 7fac81600700 1 -- 172.21.15.5:0/2359021907 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac7801a910 con 0x7fac84109220 2024-04-24T06:33:43.032 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:34.823+0000 7fac81600700 1 -- 172.21.15.5:0/2359021907 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac78024930 con 0x7fac84109220 2024-04-24T06:33:43.032 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:34.823+0000 7fac8b400700 1 -- 172.21.15.5:0/2359021907 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac841071d0 con 0x7fac84109220 2024-04-24T06:33:43.032 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:34.823+0000 7fac8b400700 1 -- 172.21.15.5:0/2359021907 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac84107690 con 0x7fac84109220 2024-04-24T06:33:43.032 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:34.824+0000 7fac81600700 1 -- 172.21.15.5:0/2359021907 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fac78022020 con 0x7fac84109220 2024-04-24T06:33:43.032 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:34.825+0000 7fac81600700 1 --2- 172.21.15.5:0/2359021907 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7fac6c0396b0 0x7fac6c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:43.032 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:34.825+0000 7fac81600700 1 -- 172.21.15.5:0/2359021907 --> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fac6c03c280 con 0x7fac6c0396b0 2024-04-24T06:33:43.033 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:34.825+0000 7fac81600700 1 -- 172.21.15.5:0/2359021907 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fac7804d9c0 con 0x7fac84109220 2024-04-24T06:33:43.033 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:34.825+0000 7fac83e00700 1 -- 172.21.15.5:0/2359021907 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7fac6c0396b0 msgr2=0x7fac6c03bb70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2147029578 2024-04-24T06:33:43.033 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:34.825+0000 7fac83e00700 1 --2- 172.21.15.5:0/2359021907 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7fac6c0396b0 0x7fac6c03bb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T06:33:43.033 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:35.026+0000 7fac83e00700 1 -- 172.21.15.5:0/2359021907 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7fac6c0396b0 msgr2=0x7fac6c03bb70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2147029578 2024-04-24T06:33:43.033 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:35.026+0000 7fac83e00700 1 --2- 172.21.15.5:0/2359021907 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7fac6c0396b0 0x7fac6c03bb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T06:33:43.033 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:35.427+0000 7fac83e00700 1 -- 172.21.15.5:0/2359021907 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7fac6c0396b0 msgr2=0x7fac6c03bb70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2147029578 2024-04-24T06:33:43.033 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:35.427+0000 7fac83e00700 1 --2- 172.21.15.5:0/2359021907 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7fac6c0396b0 0x7fac6c03bb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-24T06:33:43.033 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:36.228+0000 7fac83e00700 1 -- 172.21.15.5:0/2359021907 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7fac6c0396b0 msgr2=0x7fac6c03bb70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2147029578 2024-04-24T06:33:43.033 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:36.228+0000 7fac83e00700 1 --2- 172.21.15.5:0/2359021907 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7fac6c0396b0 0x7fac6c03bb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-24T06:33:43.033 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:37.830+0000 7fac83e00700 1 -- 172.21.15.5:0/2359021907 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7fac6c0396b0 msgr2=0x7fac6c03bb70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2147029578 2024-04-24T06:33:43.033 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:37.830+0000 7fac83e00700 1 --2- 172.21.15.5:0/2359021907 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7fac6c0396b0 0x7fac6c03bb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-24T06:33:43.033 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:41.034+0000 7fac83e00700 1 -- 172.21.15.5:0/2359021907 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7fac6c0396b0 msgr2=0x7fac6c03bb70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2147029578 2024-04-24T06:33:43.034 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:41.034+0000 7fac83e00700 1 --2- 172.21.15.5:0/2359021907 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7fac6c0396b0 0x7fac6c03bb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-04-24T06:33:43.034 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:41.166+0000 7fac81600700 1 -- 172.21.15.5:0/2359021907 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7fac78024ad0 con 0x7fac84109220 2024-04-24T06:33:43.034 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:41.167+0000 7fac81600700 1 -- 172.21.15.5:0/2359021907 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7fac6c0396b0 msgr2=0x7fac6c03bb70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T06:33:43.034 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:41.167+0000 7fac81600700 1 --2- 172.21.15.5:0/2359021907 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7fac6c0396b0 0x7fac6c03bb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:43.034 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:42.166+0000 7fac81600700 1 -- 172.21.15.5:0/2359021907 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7fac7804f710 con 0x7fac84109220 2024-04-24T06:33:43.034 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:42.166+0000 7fac81600700 1 --2- 172.21.15.5:0/2359021907 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fac6c03cff0 0x7fac6c03f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:43.034 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:42.166+0000 7fac81600700 1 -- 172.21.15.5:0/2359021907 --> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fac7802ee50 con 0x7fac6c03cff0 2024-04-24T06:33:43.034 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:42.167+0000 7fac83e00700 1 --2- 172.21.15.5:0/2359021907 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fac6c03cff0 0x7fac6c03f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:43.034 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:42.168+0000 7fac83e00700 1 --2- 172.21.15.5:0/2359021907 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fac6c03cff0 0x7fac6c03f3e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fac70003a10 tx=0x7fac700092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:43.034 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:42.169+0000 7fac81600700 1 -- 172.21.15.5:0/2359021907 <== mgr.14152 v2:172.21.15.5:6800/4042936965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fac7802ee50 con 0x7fac6c03cff0 2024-04-24T06:33:43.034 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:42.176+0000 7fac8b400700 1 -- 172.21.15.5:0/2359021907 --> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fac84069c50 con 0x7fac6c03cff0 2024-04-24T06:33:43.035 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:42.177+0000 7fac81600700 1 -- 172.21.15.5:0/2359021907 <== mgr.14152 v2:172.21.15.5:6800/4042936965 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fac84069c50 con 0x7fac6c03cff0 2024-04-24T06:33:43.035 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:42.177+0000 7fac8b400700 1 -- 172.21.15.5:0/2359021907 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fac6c03cff0 msgr2=0x7fac6c03f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:43.035 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:42.177+0000 7fac8b400700 1 --2- 172.21.15.5:0/2359021907 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fac6c03cff0 0x7fac6c03f3e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fac70003a10 tx=0x7fac700092b0 comp rx=0 tx=0).stop 2024-04-24T06:33:43.035 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:42.177+0000 7fac8b400700 1 -- 172.21.15.5:0/2359021907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac84109220 msgr2=0x7fac84108aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:43.035 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:42.177+0000 7fac8b400700 1 --2- 172.21.15.5:0/2359021907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac84109220 0x7fac84108aa0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fac78011b60 tx=0x7fac78011c40 comp rx=0 tx=0).stop 2024-04-24T06:33:43.035 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:42.177+0000 7fac8b400700 1 -- 172.21.15.5:0/2359021907 shutdown_connections 2024-04-24T06:33:43.035 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:42.177+0000 7fac8b400700 1 --2- 172.21.15.5:0/2359021907 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fac6c03cff0 0x7fac6c03f3e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:43.035 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:42.177+0000 7fac8b400700 1 --2- 172.21.15.5:0/2359021907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac84109220 0x7fac84108aa0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:43.035 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:42.177+0000 7fac8b400700 1 --2- 172.21.15.5:0/2359021907 >> [v2:172.21.15.5:6800/2147029578,v1:172.21.15.5:6801/2147029578] conn(0x7fac6c0396b0 0x7fac6c03bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:43.035 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:42.177+0000 7fac8b400700 1 -- 172.21.15.5:0/2359021907 >> 172.21.15.5:0/2359021907 conn(0x7fac84102c30 msgr2=0x7fac841036e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:43.035 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:42.178+0000 7fac8b400700 1 -- 172.21.15.5:0/2359021907 shutdown_connections 2024-04-24T06:33:43.035 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:42.178+0000 7fac8b400700 1 -- 172.21.15.5:0/2359021907 wait complete. 2024-04-24T06:33:43.035 INFO:teuthology.orchestra.run.smithi005.stdout:mgr epoch 9 is available 2024-04-24T06:33:43.036 INFO:teuthology.orchestra.run.smithi005.stdout:Generating a dashboard self-signed certificate... 2024-04-24T06:33:43.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:43 smithi005 ceph-mon[27798]: [24/Apr/2024:06:33:42] ENGINE Serving on http://172.21.15.5:8765 2024-04-24T06:33:43.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:43 smithi005 ceph-mon[27798]: [24/Apr/2024:06:33:42] ENGINE Bus STARTED 2024-04-24T06:33:43.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:43 smithi005 ceph-mon[27798]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-24T06:33:43.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:43 smithi005 ceph-mon[27798]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-24T06:33:44.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:44 smithi005 ceph-mon[27798]: mgrmap e12: x(active, since 2s) 2024-04-24T06:33:44.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:44 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:33:45.443 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-24T06:33:45.443 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.093+0000 7fe4ffa00700 1 Processor -- start 2024-04-24T06:33:45.443 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.094+0000 7fe4ffa00700 1 -- start start 2024-04-24T06:33:45.443 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.094+0000 7fe4ffa00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4f8108700 0x7fe4f8108b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:45.443 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.094+0000 7fe4ffa00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f81090f0 con 0x7fe4f8108700 2024-04-24T06:33:45.443 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.095+0000 7fe4fd000700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4f8108700 0x7fe4f8108b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:45.443 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.095+0000 7fe4fd000700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4f8108700 0x7fe4f8108b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33834/0 (socket says 172.21.15.5:33834) 2024-04-24T06:33:45.443 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.095+0000 7fe4fd000700 1 -- 172.21.15.5:0/2386473008 learned_addr learned my addr 172.21.15.5:0/2386473008 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:33:45.448 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.095+0000 7fe4fd000700 1 -- 172.21.15.5:0/2386473008 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4f81098b0 con 0x7fe4f8108700 2024-04-24T06:33:45.448 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.096+0000 7fe4fd000700 1 --2- 172.21.15.5:0/2386473008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4f8108700 0x7fe4f8108b20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe4e80098a0 tx=0x7fe4e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=408264ab5f35e332 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:45.449 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.096+0000 7fe4f7e00700 1 -- 172.21.15.5:0/2386473008 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4e8004160 con 0x7fe4f8108700 2024-04-24T06:33:45.449 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.096+0000 7fe4f7e00700 1 -- 172.21.15.5:0/2386473008 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4e8004300 con 0x7fe4f8108700 2024-04-24T06:33:45.449 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.096+0000 7fe4f7e00700 1 -- 172.21.15.5:0/2386473008 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4e8011b20 con 0x7fe4f8108700 2024-04-24T06:33:45.449 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.097+0000 7fe4ffa00700 1 -- 172.21.15.5:0/2386473008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4f8108700 msgr2=0x7fe4f8108b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:45.449 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.097+0000 7fe4ffa00700 1 --2- 172.21.15.5:0/2386473008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4f8108700 0x7fe4f8108b20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe4e80098a0 tx=0x7fe4e8009bb0 comp rx=0 tx=0).stop 2024-04-24T06:33:45.449 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.097+0000 7fe4ffa00700 1 -- 172.21.15.5:0/2386473008 shutdown_connections 2024-04-24T06:33:45.449 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.097+0000 7fe4ffa00700 1 --2- 172.21.15.5:0/2386473008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4f8108700 0x7fe4f8108b20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:45.449 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.097+0000 7fe4ffa00700 1 -- 172.21.15.5:0/2386473008 >> 172.21.15.5:0/2386473008 conn(0x7fe4f8103d40 msgr2=0x7fe4f8106160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:45.449 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.097+0000 7fe4ffa00700 1 -- 172.21.15.5:0/2386473008 shutdown_connections 2024-04-24T06:33:45.449 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.097+0000 7fe4ffa00700 1 -- 172.21.15.5:0/2386473008 wait complete. 2024-04-24T06:33:45.449 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.098+0000 7fe4ffa00700 1 Processor -- start 2024-04-24T06:33:45.449 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.099+0000 7fe4ffa00700 1 -- start start 2024-04-24T06:33:45.450 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.099+0000 7fe4ffa00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4f8108700 0x7fe4f807db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:45.450 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.099+0000 7fe4ffa00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f810cc60 con 0x7fe4f8108700 2024-04-24T06:33:45.450 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.099+0000 7fe4fd000700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4f8108700 0x7fe4f807db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:45.450 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.099+0000 7fe4fd000700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4f8108700 0x7fe4f807db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33850/0 (socket says 172.21.15.5:33850) 2024-04-24T06:33:45.450 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.099+0000 7fe4fd000700 1 -- 172.21.15.5:0/1831318530 learned_addr learned my addr 172.21.15.5:0/1831318530 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:33:45.450 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.100+0000 7fe4fd000700 1 -- 172.21.15.5:0/1831318530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4f807e0d0 con 0x7fe4f8108700 2024-04-24T06:33:45.450 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.100+0000 7fe4fd000700 1 --2- 172.21.15.5:0/1831318530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4f8108700 0x7fe4f807db90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8000c00 tx=0x7fe4e8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:45.450 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.100+0000 7fe4f6000700 1 -- 172.21.15.5:0/1831318530 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4e8003b60 con 0x7fe4f8108700 2024-04-24T06:33:45.450 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.101+0000 7fe4f6000700 1 -- 172.21.15.5:0/1831318530 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4e801a910 con 0x7fe4f8108700 2024-04-24T06:33:45.450 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.101+0000 7fe4ffa00700 1 -- 172.21.15.5:0/1831318530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4f807c2c0 con 0x7fe4f8108700 2024-04-24T06:33:45.450 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.101+0000 7fe4f6000700 1 -- 172.21.15.5:0/1831318530 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4e8024930 con 0x7fe4f8108700 2024-04-24T06:33:45.450 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.101+0000 7fe4ffa00700 1 -- 172.21.15.5:0/1831318530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4f807c780 con 0x7fe4f8108700 2024-04-24T06:33:45.450 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.102+0000 7fe4ffa00700 1 -- 172.21.15.5:0/1831318530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4f8064150 con 0x7fe4f8108700 2024-04-24T06:33:45.451 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.102+0000 7fe4f6000700 1 -- 172.21.15.5:0/1831318530 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fe4e8022020 con 0x7fe4f8108700 2024-04-24T06:33:45.451 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.103+0000 7fe4f6000700 1 --2- 172.21.15.5:0/1831318530 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fe4dc039660 0x7fe4dc03bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:45.451 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.103+0000 7fe4f6000700 1 -- 172.21.15.5:0/1831318530 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe4e804dbb0 con 0x7fe4f8108700 2024-04-24T06:33:45.451 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.103+0000 7fe4efe00700 1 --2- 172.21.15.5:0/1831318530 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fe4dc039660 0x7fe4dc03bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:45.451 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.104+0000 7fe4efe00700 1 --2- 172.21.15.5:0/1831318530 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fe4dc039660 0x7fe4dc03bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0006fd0 tx=0x7fe4e0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:45.451 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.109+0000 7fe4f6000700 1 -- 172.21.15.5:0/1831318530 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe4e800be30 con 0x7fe4f8108700 2024-04-24T06:33:45.451 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.348+0000 7fe4ffa00700 1 -- 172.21.15.5:0/1831318530 --> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fe4f807d090 con 0x7fe4dc039660 2024-04-24T06:33:45.451 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.599+0000 7fe4f6000700 1 -- 172.21.15.5:0/1831318530 <== mgr.14152 v2:172.21.15.5:6800/4042936965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fe4f807d090 con 0x7fe4dc039660 2024-04-24T06:33:45.451 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.602+0000 7fe4ffa00700 1 -- 172.21.15.5:0/1831318530 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fe4dc039660 msgr2=0x7fe4dc03bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:45.451 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.602+0000 7fe4ffa00700 1 --2- 172.21.15.5:0/1831318530 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fe4dc039660 0x7fe4dc03bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0006fd0 tx=0x7fe4e0006e40 comp rx=0 tx=0).stop 2024-04-24T06:33:45.451 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.602+0000 7fe4ffa00700 1 -- 172.21.15.5:0/1831318530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4f8108700 msgr2=0x7fe4f807db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:45.451 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.602+0000 7fe4ffa00700 1 --2- 172.21.15.5:0/1831318530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4f8108700 0x7fe4f807db90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8000c00 tx=0x7fe4e8011c10 comp rx=0 tx=0).stop 2024-04-24T06:33:45.451 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.602+0000 7fe4ffa00700 1 -- 172.21.15.5:0/1831318530 shutdown_connections 2024-04-24T06:33:45.451 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.602+0000 7fe4ffa00700 1 --2- 172.21.15.5:0/1831318530 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fe4dc039660 0x7fe4dc03bb20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:45.452 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.602+0000 7fe4ffa00700 1 --2- 172.21.15.5:0/1831318530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4f8108700 0x7fe4f807db90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:45.452 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.602+0000 7fe4ffa00700 1 -- 172.21.15.5:0/1831318530 >> 172.21.15.5:0/1831318530 conn(0x7fe4f8103d40 msgr2=0x7fe4f8104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:45.452 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.603+0000 7fe4ffa00700 1 -- 172.21.15.5:0/1831318530 shutdown_connections 2024-04-24T06:33:45.452 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:44.603+0000 7fe4ffa00700 1 -- 172.21.15.5:0/1831318530 wait complete. 2024-04-24T06:33:45.452 INFO:teuthology.orchestra.run.smithi005.stdout:Creating initial admin user... 2024-04-24T06:33:45.935 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:45 smithi005 ceph-mon[27798]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:33:45.935 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:45 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:33:45.935 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:45 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:33:47.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:46 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:33:47.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:46 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:33:48.127 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:47 smithi005 ceph-mon[27798]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:33:48.610 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$48vHcIBqVNSbL2CsJxf/R.lZltRpWwg86o58SFNmQzpyZR8QEhpuy", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1713940426, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-24T06:33:48.611 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.099+0000 7f664c600700 1 Processor -- start 2024-04-24T06:33:48.611 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.100+0000 7f664c600700 1 -- start start 2024-04-24T06:33:48.611 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.100+0000 7f664c600700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6644108700 0x7f6644108b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:48.611 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.100+0000 7f664c600700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f66441090f0 con 0x7f6644108700 2024-04-24T06:33:48.611 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.100+0000 7f6649c00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6644108700 0x7f6644108b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:48.611 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.100+0000 7f6649c00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6644108700 0x7f6644108b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33852/0 (socket says 172.21.15.5:33852) 2024-04-24T06:33:48.612 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.101+0000 7f6649c00700 1 -- 172.21.15.5:0/2622387077 learned_addr learned my addr 172.21.15.5:0/2622387077 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:33:48.612 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.101+0000 7f6649c00700 1 -- 172.21.15.5:0/2622387077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66441098b0 con 0x7f6644108700 2024-04-24T06:33:48.612 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.101+0000 7f6649c00700 1 --2- 172.21.15.5:0/2622387077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6644108700 0x7f6644108b20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f66380098a0 tx=0x7f6638009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ce33d70025b9e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:48.615 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.101+0000 7f6643e00700 1 -- 172.21.15.5:0/2622387077 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6638004160 con 0x7f6644108700 2024-04-24T06:33:48.615 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.102+0000 7f6643e00700 1 -- 172.21.15.5:0/2622387077 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6638004300 con 0x7f6644108700 2024-04-24T06:33:48.615 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.102+0000 7f6643e00700 1 -- 172.21.15.5:0/2622387077 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6638011b20 con 0x7f6644108700 2024-04-24T06:33:48.615 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.102+0000 7f664c600700 1 -- 172.21.15.5:0/2622387077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6644108700 msgr2=0x7f6644108b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:48.615 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.102+0000 7f664c600700 1 --2- 172.21.15.5:0/2622387077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6644108700 0x7f6644108b20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f66380098a0 tx=0x7f6638009bb0 comp rx=0 tx=0).stop 2024-04-24T06:33:48.615 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.102+0000 7f664c600700 1 -- 172.21.15.5:0/2622387077 shutdown_connections 2024-04-24T06:33:48.615 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.102+0000 7f664c600700 1 --2- 172.21.15.5:0/2622387077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6644108700 0x7f6644108b20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:48.615 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.102+0000 7f664c600700 1 -- 172.21.15.5:0/2622387077 >> 172.21.15.5:0/2622387077 conn(0x7f6644103d40 msgr2=0x7f6644106160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:48.615 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.103+0000 7f664c600700 1 -- 172.21.15.5:0/2622387077 shutdown_connections 2024-04-24T06:33:48.615 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.103+0000 7f664c600700 1 -- 172.21.15.5:0/2622387077 wait complete. 2024-04-24T06:33:48.616 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.104+0000 7f664c600700 1 Processor -- start 2024-04-24T06:33:48.616 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.104+0000 7f664c600700 1 -- start start 2024-04-24T06:33:48.616 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.104+0000 7f664c600700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6644108700 0x7f664419e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:48.616 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.104+0000 7f664c600700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f664410cc60 con 0x7f6644108700 2024-04-24T06:33:48.616 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.105+0000 7f6649c00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6644108700 0x7f664419e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:48.616 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.105+0000 7f6649c00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6644108700 0x7f664419e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33866/0 (socket says 172.21.15.5:33866) 2024-04-24T06:33:48.616 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.105+0000 7f6649c00700 1 -- 172.21.15.5:0/1557925870 learned_addr learned my addr 172.21.15.5:0/1557925870 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:33:48.616 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.105+0000 7f6649c00700 1 -- 172.21.15.5:0/1557925870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f664419edb0 con 0x7f6644108700 2024-04-24T06:33:48.616 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.105+0000 7f6649c00700 1 --2- 172.21.15.5:0/1557925870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6644108700 0x7f664419e870 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6638000c00 tx=0x7f6638011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:48.616 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.106+0000 7f6642000700 1 -- 172.21.15.5:0/1557925870 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6638003b60 con 0x7f6644108700 2024-04-24T06:33:48.616 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.106+0000 7f6642000700 1 -- 172.21.15.5:0/1557925870 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f663801a910 con 0x7f6644108700 2024-04-24T06:33:48.616 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.106+0000 7f6642000700 1 -- 172.21.15.5:0/1557925870 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6638024930 con 0x7f6644108700 2024-04-24T06:33:48.616 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.106+0000 7f664c600700 1 -- 172.21.15.5:0/1557925870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f664419f040 con 0x7f6644108700 2024-04-24T06:33:48.616 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.106+0000 7f664c600700 1 -- 172.21.15.5:0/1557925870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f664419f4a0 con 0x7f6644108700 2024-04-24T06:33:48.616 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.107+0000 7f6642000700 1 -- 172.21.15.5:0/1557925870 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f6638022020 con 0x7f6644108700 2024-04-24T06:33:48.617 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.107+0000 7f664c600700 1 -- 172.21.15.5:0/1557925870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6644064150 con 0x7f6644108700 2024-04-24T06:33:48.617 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.108+0000 7f6642000700 1 --2- 172.21.15.5:0/1557925870 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f662c039660 0x7f662c03bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:48.619 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.108+0000 7f6642000700 1 -- 172.21.15.5:0/1557925870 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f663804d9e0 con 0x7f6644108700 2024-04-24T06:33:48.619 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.108+0000 7f6649200700 1 --2- 172.21.15.5:0/1557925870 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f662c039660 0x7f662c03bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:48.619 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.109+0000 7f6649200700 1 --2- 172.21.15.5:0/1557925870 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f662c039660 0x7f662c03bb20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6630006fd0 tx=0x7f6630006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:48.619 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.114+0000 7f6642000700 1 -- 172.21.15.5:0/1557925870 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f663800b3e0 con 0x7f6644108700 2024-04-24T06:33:48.619 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.435+0000 7f664c600700 1 -- 172.21.15.5:0/1557925870 --> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f66441a1eb0 con 0x7f662c039660 2024-04-24T06:33:48.619 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.692+0000 7f6642000700 1 -- 172.21.15.5:0/1557925870 <== mgr.14152 v2:172.21.15.5:6800/4042936965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f66441a1eb0 con 0x7f662c039660 2024-04-24T06:33:48.619 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.694+0000 7f664c600700 1 -- 172.21.15.5:0/1557925870 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f662c039660 msgr2=0x7f662c03bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:48.619 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.694+0000 7f664c600700 1 --2- 172.21.15.5:0/1557925870 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f662c039660 0x7f662c03bb20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6630006fd0 tx=0x7f6630006e40 comp rx=0 tx=0).stop 2024-04-24T06:33:48.619 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.694+0000 7f664c600700 1 -- 172.21.15.5:0/1557925870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6644108700 msgr2=0x7f664419e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:48.619 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.694+0000 7f664c600700 1 --2- 172.21.15.5:0/1557925870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6644108700 0x7f664419e870 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6638000c00 tx=0x7f6638011c10 comp rx=0 tx=0).stop 2024-04-24T06:33:48.619 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.695+0000 7f664c600700 1 -- 172.21.15.5:0/1557925870 shutdown_connections 2024-04-24T06:33:48.619 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.695+0000 7f664c600700 1 --2- 172.21.15.5:0/1557925870 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f662c039660 0x7f662c03bb20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:48.619 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.695+0000 7f664c600700 1 --2- 172.21.15.5:0/1557925870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6644108700 0x7f664419e870 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:48.620 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.695+0000 7f664c600700 1 -- 172.21.15.5:0/1557925870 >> 172.21.15.5:0/1557925870 conn(0x7f6644103d40 msgr2=0x7f6644104810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:48.620 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.695+0000 7f664c600700 1 -- 172.21.15.5:0/1557925870 shutdown_connections 2024-04-24T06:33:48.620 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:46.695+0000 7f664c600700 1 -- 172.21.15.5:0/1557925870 wait complete. 2024-04-24T06:33:48.620 INFO:teuthology.orchestra.run.smithi005.stdout:Fetching dashboard port number... 2024-04-24T06:33:49.149 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:48 smithi005 ceph-mon[27798]: mgrmap e13: x(active, since 6s) 2024-04-24T06:33:50.202 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:49 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1598856943' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-24T06:33:50.218 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 8443 2024-04-24T06:33:50.218 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.319+0000 7f1c3ba00700 1 Processor -- start 2024-04-24T06:33:50.218 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.319+0000 7f1c3ba00700 1 -- start start 2024-04-24T06:33:50.218 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.319+0000 7f1c3ba00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c341091f0 0x7f1c3410b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:50.218 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.319+0000 7f1c3ba00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1c3407be50 con 0x7f1c341091f0 2024-04-24T06:33:50.218 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.320+0000 7f1c39000700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c341091f0 0x7f1c3410b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:50.218 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.320+0000 7f1c39000700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c341091f0 0x7f1c3410b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33874/0 (socket says 172.21.15.5:33874) 2024-04-24T06:33:50.218 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.320+0000 7f1c39000700 1 -- 172.21.15.5:0/1526516814 learned_addr learned my addr 172.21.15.5:0/1526516814 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:33:50.219 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.320+0000 7f1c39000700 1 -- 172.21.15.5:0/1526516814 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c3410bb20 con 0x7f1c341091f0 2024-04-24T06:33:50.219 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.321+0000 7f1c39000700 1 --2- 172.21.15.5:0/1526516814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c341091f0 0x7f1c3410b5e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1c280098a0 tx=0x7f1c28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d214dda3685ebb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:50.219 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.321+0000 7f1c33400700 1 -- 172.21.15.5:0/1526516814 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c28004160 con 0x7f1c341091f0 2024-04-24T06:33:50.219 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.321+0000 7f1c33400700 1 -- 172.21.15.5:0/1526516814 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c28004300 con 0x7f1c341091f0 2024-04-24T06:33:50.219 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.321+0000 7f1c33400700 1 -- 172.21.15.5:0/1526516814 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c28011b20 con 0x7f1c341091f0 2024-04-24T06:33:50.219 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.322+0000 7f1c3ba00700 1 -- 172.21.15.5:0/1526516814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c341091f0 msgr2=0x7f1c3410b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:50.219 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.322+0000 7f1c3ba00700 1 --2- 172.21.15.5:0/1526516814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c341091f0 0x7f1c3410b5e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1c280098a0 tx=0x7f1c28009bb0 comp rx=0 tx=0).stop 2024-04-24T06:33:50.219 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.322+0000 7f1c3ba00700 1 -- 172.21.15.5:0/1526516814 shutdown_connections 2024-04-24T06:33:50.219 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.322+0000 7f1c3ba00700 1 --2- 172.21.15.5:0/1526516814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c341091f0 0x7f1c3410b5e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:50.219 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.322+0000 7f1c3ba00700 1 -- 172.21.15.5:0/1526516814 >> 172.21.15.5:0/1526516814 conn(0x7f1c34102c00 msgr2=0x7f1c34105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:50.219 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.322+0000 7f1c3ba00700 1 -- 172.21.15.5:0/1526516814 shutdown_connections 2024-04-24T06:33:50.219 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.322+0000 7f1c3ba00700 1 -- 172.21.15.5:0/1526516814 wait complete. 2024-04-24T06:33:50.219 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.323+0000 7f1c3ba00700 1 Processor -- start 2024-04-24T06:33:50.220 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.323+0000 7f1c3ba00700 1 -- start start 2024-04-24T06:33:50.220 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.324+0000 7f1c3ba00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c341091f0 0x7f1c3419e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:50.220 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.324+0000 7f1c3ba00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1c3410cb20 con 0x7f1c341091f0 2024-04-24T06:33:50.220 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.324+0000 7f1c39000700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c341091f0 0x7f1c3419e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:50.220 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.324+0000 7f1c39000700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c341091f0 0x7f1c3419e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33888/0 (socket says 172.21.15.5:33888) 2024-04-24T06:33:50.220 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.324+0000 7f1c39000700 1 -- 172.21.15.5:0/1598856943 learned_addr learned my addr 172.21.15.5:0/1598856943 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:33:50.220 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.324+0000 7f1c39000700 1 -- 172.21.15.5:0/1598856943 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c3419ede0 con 0x7f1c341091f0 2024-04-24T06:33:50.220 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.325+0000 7f1c39000700 1 --2- 172.21.15.5:0/1598856943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c341091f0 0x7f1c3419e8a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1c28011b60 tx=0x7f1c28011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:50.220 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.325+0000 7f1c31600700 1 -- 172.21.15.5:0/1598856943 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c28003b60 con 0x7f1c341091f0 2024-04-24T06:33:50.220 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.325+0000 7f1c31600700 1 -- 172.21.15.5:0/1598856943 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c2801a910 con 0x7f1c341091f0 2024-04-24T06:33:50.220 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.325+0000 7f1c3ba00700 1 -- 172.21.15.5:0/1598856943 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c3419f070 con 0x7f1c341091f0 2024-04-24T06:33:50.220 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.325+0000 7f1c31600700 1 -- 172.21.15.5:0/1598856943 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c28024930 con 0x7f1c341091f0 2024-04-24T06:33:50.220 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.325+0000 7f1c3ba00700 1 -- 172.21.15.5:0/1598856943 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c3419f4d0 con 0x7f1c341091f0 2024-04-24T06:33:50.221 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.326+0000 7f1c31600700 1 -- 172.21.15.5:0/1598856943 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7f1c28022020 con 0x7f1c341091f0 2024-04-24T06:33:50.221 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.326+0000 7f1c3ba00700 1 -- 172.21.15.5:0/1598856943 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c34064150 con 0x7f1c341091f0 2024-04-24T06:33:50.221 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.327+0000 7f1c31600700 1 --2- 172.21.15.5:0/1598856943 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f1c1c039700 0x7f1c1c03bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:50.221 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.327+0000 7f1c31600700 1 -- 172.21.15.5:0/1598856943 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1c2804db70 con 0x7f1c341091f0 2024-04-24T06:33:50.221 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.327+0000 7f1c33e00700 1 --2- 172.21.15.5:0/1598856943 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f1c1c039700 0x7f1c1c03bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:50.221 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.328+0000 7f1c33e00700 1 --2- 172.21.15.5:0/1598856943 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f1c1c039700 0x7f1c1c03bbc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1c20006fd0 tx=0x7f1c20006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:50.221 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.332+0000 7f1c31600700 1 -- 172.21.15.5:0/1598856943 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1c28016260 con 0x7f1c341091f0 2024-04-24T06:33:50.221 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.562+0000 7f1c3ba00700 1 -- 172.21.15.5:0/1598856943 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f1c341a1ea0 con 0x7f1c341091f0 2024-04-24T06:33:50.221 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.563+0000 7f1c31600700 1 -- 172.21.15.5:0/1598856943 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f1c2802ad90 con 0x7f1c341091f0 2024-04-24T06:33:50.221 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.566+0000 7f1c3ba00700 1 -- 172.21.15.5:0/1598856943 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f1c1c039700 msgr2=0x7f1c1c03bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:50.221 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.566+0000 7f1c3ba00700 1 --2- 172.21.15.5:0/1598856943 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f1c1c039700 0x7f1c1c03bbc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1c20006fd0 tx=0x7f1c20006e40 comp rx=0 tx=0).stop 2024-04-24T06:33:50.221 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.566+0000 7f1c3ba00700 1 -- 172.21.15.5:0/1598856943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c341091f0 msgr2=0x7f1c3419e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:50.221 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.566+0000 7f1c3ba00700 1 --2- 172.21.15.5:0/1598856943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c341091f0 0x7f1c3419e8a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1c28011b60 tx=0x7f1c28011c40 comp rx=0 tx=0).stop 2024-04-24T06:33:50.222 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.566+0000 7f1c3ba00700 1 -- 172.21.15.5:0/1598856943 shutdown_connections 2024-04-24T06:33:50.222 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.566+0000 7f1c3ba00700 1 --2- 172.21.15.5:0/1598856943 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f1c1c039700 0x7f1c1c03bbc0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:50.222 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.566+0000 7f1c3ba00700 1 --2- 172.21.15.5:0/1598856943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c341091f0 0x7f1c3419e8a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:50.222 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.566+0000 7f1c3ba00700 1 -- 172.21.15.5:0/1598856943 >> 172.21.15.5:0/1598856943 conn(0x7f1c34102c00 msgr2=0x7f1c34103620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:50.222 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.567+0000 7f1c3ba00700 1 -- 172.21.15.5:0/1598856943 shutdown_connections 2024-04-24T06:33:50.222 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:49.567+0000 7f1c3ba00700 1 -- 172.21.15.5:0/1598856943 wait complete. 2024-04-24T06:33:50.250 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T06:33:50.250 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2024-04-24T06:33:50.277 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T06:33:50.278 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-04-24T06:33:50.278 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2024-04-24T06:33:50.278 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-24T06:33:50.286 INFO:teuthology.orchestra.run.smithi005.stdout:Ceph Dashboard is now available at: 2024-04-24T06:33:50.286 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:33:50.286 INFO:teuthology.orchestra.run.smithi005.stdout: URL: https://smithi005.front.sepia.ceph.com:8443/ 2024-04-24T06:33:50.286 INFO:teuthology.orchestra.run.smithi005.stdout: User: admin 2024-04-24T06:33:50.286 INFO:teuthology.orchestra.run.smithi005.stdout: Password: 068cdp0iq6 2024-04-24T06:33:50.286 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:33:50.287 INFO:teuthology.orchestra.run.smithi005.stdout:Saving cluster configuration to /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config directory 2024-04-24T06:33:50.289 INFO:teuthology.orchestra.run.smithi005.stdout:Enabling autotune for osd_memory_target 2024-04-24T06:33:51.840 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:50.910+0000 7ff46e200700 1 Processor -- start 2024-04-24T06:33:51.840 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:50.910+0000 7ff46e200700 1 -- start start 2024-04-24T06:33:51.840 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:50.910+0000 7ff46e200700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff468108510 0x7ff468108930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:51.840 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:50.910+0000 7ff46e200700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff468108f00 con 0x7ff468108510 2024-04-24T06:33:51.841 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:50.911+0000 7ff46ce00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff468108510 0x7ff468108930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:51.841 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:50.911+0000 7ff46ce00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff468108510 0x7ff468108930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33894/0 (socket says 172.21.15.5:33894) 2024-04-24T06:33:51.841 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:50.911+0000 7ff46ce00700 1 -- 172.21.15.5:0/3748383885 learned_addr learned my addr 172.21.15.5:0/3748383885 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:33:51.841 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:50.911+0000 7ff46ce00700 1 -- 172.21.15.5:0/3748383885 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4681096c0 con 0x7ff468108510 2024-04-24T06:33:51.841 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:50.912+0000 7ff46ce00700 1 --2- 172.21.15.5:0/3748383885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff468108510 0x7ff468108930 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff45c0098a0 tx=0x7ff45c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2862dff4452f4541 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:51.841 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:50.912+0000 7ff467400700 1 -- 172.21.15.5:0/3748383885 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff45c004160 con 0x7ff468108510 2024-04-24T06:33:51.841 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:50.912+0000 7ff467400700 1 -- 172.21.15.5:0/3748383885 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff45c004300 con 0x7ff468108510 2024-04-24T06:33:51.841 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:50.912+0000 7ff467400700 1 -- 172.21.15.5:0/3748383885 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff45c011b20 con 0x7ff468108510 2024-04-24T06:33:51.841 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:50.913+0000 7ff46e200700 1 -- 172.21.15.5:0/3748383885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff468108510 msgr2=0x7ff468108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:51.841 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:50.913+0000 7ff46e200700 1 --2- 172.21.15.5:0/3748383885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff468108510 0x7ff468108930 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff45c0098a0 tx=0x7ff45c009bb0 comp rx=0 tx=0).stop 2024-04-24T06:33:51.841 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:50.913+0000 7ff46e200700 1 -- 172.21.15.5:0/3748383885 shutdown_connections 2024-04-24T06:33:51.841 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:50.913+0000 7ff46e200700 1 --2- 172.21.15.5:0/3748383885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff468108510 0x7ff468108930 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:51.842 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:50.913+0000 7ff46e200700 1 -- 172.21.15.5:0/3748383885 >> 172.21.15.5:0/3748383885 conn(0x7ff468103b50 msgr2=0x7ff468105f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:51.842 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:50.913+0000 7ff46e200700 1 -- 172.21.15.5:0/3748383885 shutdown_connections 2024-04-24T06:33:51.842 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:50.913+0000 7ff46e200700 1 -- 172.21.15.5:0/3748383885 wait complete. 2024-04-24T06:33:51.842 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:50.914+0000 7ff46e200700 1 Processor -- start 2024-04-24T06:33:51.842 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:50.915+0000 7ff46e200700 1 -- start start 2024-04-24T06:33:51.842 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:50.915+0000 7ff46e200700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff468108510 0x7ff46819e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:51.842 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:50.915+0000 7ff46e200700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff46810ca70 con 0x7ff468108510 2024-04-24T06:33:51.842 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:50.915+0000 7ff46ce00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff468108510 0x7ff46819e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:51.842 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:50.915+0000 7ff46ce00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff468108510 0x7ff46819e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33898/0 (socket says 172.21.15.5:33898) 2024-04-24T06:33:51.842 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:50.915+0000 7ff46ce00700 1 -- 172.21.15.5:0/2391596902 learned_addr learned my addr 172.21.15.5:0/2391596902 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:33:51.842 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:50.916+0000 7ff46ce00700 1 -- 172.21.15.5:0/2391596902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff46819ebd0 con 0x7ff468108510 2024-04-24T06:33:51.842 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:50.916+0000 7ff46ce00700 1 --2- 172.21.15.5:0/2391596902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff468108510 0x7ff46819e690 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff45c0116a0 tx=0x7ff45c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:51.842 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:50.916+0000 7ff465600700 1 -- 172.21.15.5:0/2391596902 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff45c003a40 con 0x7ff468108510 2024-04-24T06:33:51.843 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:50.916+0000 7ff465600700 1 -- 172.21.15.5:0/2391596902 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff45c003be0 con 0x7ff468108510 2024-04-24T06:33:51.843 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:50.916+0000 7ff465600700 1 -- 172.21.15.5:0/2391596902 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff45c01a720 con 0x7ff468108510 2024-04-24T06:33:51.843 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:50.916+0000 7ff46e200700 1 -- 172.21.15.5:0/2391596902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff46819ee60 con 0x7ff468108510 2024-04-24T06:33:51.843 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:50.917+0000 7ff46e200700 1 -- 172.21.15.5:0/2391596902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff46819f2c0 con 0x7ff468108510 2024-04-24T06:33:51.843 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:50.918+0000 7ff465600700 1 -- 172.21.15.5:0/2391596902 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7ff45c024aa0 con 0x7ff468108510 2024-04-24T06:33:51.843 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:50.918+0000 7ff46e200700 1 -- 172.21.15.5:0/2391596902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff468069c50 con 0x7ff468108510 2024-04-24T06:33:51.843 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:50.918+0000 7ff465600700 1 --2- 172.21.15.5:0/2391596902 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ff4500396b0 0x7ff45003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:51.843 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:50.918+0000 7ff465600700 1 -- 172.21.15.5:0/2391596902 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff45c04dec0 con 0x7ff468108510 2024-04-24T06:33:51.843 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:50.918+0000 7ff467e00700 1 --2- 172.21.15.5:0/2391596902 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ff4500396b0 0x7ff45003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:51.843 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:50.919+0000 7ff467e00700 1 --2- 172.21.15.5:0/2391596902 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ff4500396b0 0x7ff45003bb70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff454006fd0 tx=0x7ff454006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:51.843 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:50.924+0000 7ff465600700 1 -- 172.21.15.5:0/2391596902 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff45c011e20 con 0x7ff468108510 2024-04-24T06:33:51.843 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:51.155+0000 7ff46e200700 1 -- 172.21.15.5:0/2391596902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7ff4681a1c90 con 0x7ff468108510 2024-04-24T06:33:51.843 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:51.156+0000 7ff465600700 1 -- 172.21.15.5:0/2391596902 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7ff45c02d770 con 0x7ff468108510 2024-04-24T06:33:51.843 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:51.158+0000 7ff46e200700 1 -- 172.21.15.5:0/2391596902 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ff4500396b0 msgr2=0x7ff45003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:51.844 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:51.158+0000 7ff46e200700 1 --2- 172.21.15.5:0/2391596902 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ff4500396b0 0x7ff45003bb70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff454006fd0 tx=0x7ff454006e40 comp rx=0 tx=0).stop 2024-04-24T06:33:51.844 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:51.158+0000 7ff46e200700 1 -- 172.21.15.5:0/2391596902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff468108510 msgr2=0x7ff46819e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:51.844 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:51.158+0000 7ff46e200700 1 --2- 172.21.15.5:0/2391596902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff468108510 0x7ff46819e690 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff45c0116a0 tx=0x7ff45c011ba0 comp rx=0 tx=0).stop 2024-04-24T06:33:51.844 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:51.159+0000 7ff46e200700 1 -- 172.21.15.5:0/2391596902 shutdown_connections 2024-04-24T06:33:51.844 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:51.159+0000 7ff46e200700 1 --2- 172.21.15.5:0/2391596902 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ff4500396b0 0x7ff45003bb70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:51.844 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:51.159+0000 7ff46e200700 1 --2- 172.21.15.5:0/2391596902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff468108510 0x7ff46819e690 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:51.844 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:51.159+0000 7ff46e200700 1 -- 172.21.15.5:0/2391596902 >> 172.21.15.5:0/2391596902 conn(0x7ff468103b50 msgr2=0x7ff468104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:51.844 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:51.159+0000 7ff46e200700 1 -- 172.21.15.5:0/2391596902 shutdown_connections 2024-04-24T06:33:51.844 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:51.159+0000 7ff46e200700 1 -- 172.21.15.5:0/2391596902 wait complete. 2024-04-24T06:33:53.526 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.513+0000 7f4d65200700 1 Processor -- start 2024-04-24T06:33:53.526 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.513+0000 7f4d65200700 1 -- start start 2024-04-24T06:33:53.526 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.513+0000 7f4d65200700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d6010a730 0x7f4d6010ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:53.526 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.513+0000 7f4d65200700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6010b120 con 0x7f4d6010a730 2024-04-24T06:33:53.526 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.514+0000 7f4d5f400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d6010a730 0x7f4d6010ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:53.527 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.514+0000 7f4d5f400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d6010a730 0x7f4d6010ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33904/0 (socket says 172.21.15.5:33904) 2024-04-24T06:33:53.527 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.514+0000 7f4d5f400700 1 -- 172.21.15.5:0/1968406478 learned_addr learned my addr 172.21.15.5:0/1968406478 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:33:53.527 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.514+0000 7f4d5f400700 1 -- 172.21.15.5:0/1968406478 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d6010b8d0 con 0x7f4d6010a730 2024-04-24T06:33:53.531 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.515+0000 7f4d5f400700 1 --2- 172.21.15.5:0/1968406478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d6010a730 0x7f4d6010ab50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4d500098a0 tx=0x7f4d50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c48cd4a8e20d94 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:53.532 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.515+0000 7f4d5e000700 1 -- 172.21.15.5:0/1968406478 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d50004160 con 0x7f4d6010a730 2024-04-24T06:33:53.533 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.515+0000 7f4d5e000700 1 -- 172.21.15.5:0/1968406478 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d50004300 con 0x7f4d6010a730 2024-04-24T06:33:53.533 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.515+0000 7f4d5e000700 1 -- 172.21.15.5:0/1968406478 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d50011b20 con 0x7f4d6010a730 2024-04-24T06:33:53.533 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.516+0000 7f4d65200700 1 -- 172.21.15.5:0/1968406478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d6010a730 msgr2=0x7f4d6010ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:53.533 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.516+0000 7f4d65200700 1 --2- 172.21.15.5:0/1968406478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d6010a730 0x7f4d6010ab50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4d500098a0 tx=0x7f4d50009bb0 comp rx=0 tx=0).stop 2024-04-24T06:33:53.533 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.516+0000 7f4d65200700 1 -- 172.21.15.5:0/1968406478 shutdown_connections 2024-04-24T06:33:53.533 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.516+0000 7f4d65200700 1 --2- 172.21.15.5:0/1968406478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d6010a730 0x7f4d6010ab50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:53.533 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.516+0000 7f4d65200700 1 -- 172.21.15.5:0/1968406478 >> 172.21.15.5:0/1968406478 conn(0x7f4d6007de30 msgr2=0x7f4d601084f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:53.533 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.516+0000 7f4d65200700 1 -- 172.21.15.5:0/1968406478 shutdown_connections 2024-04-24T06:33:53.533 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.516+0000 7f4d65200700 1 -- 172.21.15.5:0/1968406478 wait complete. 2024-04-24T06:33:53.534 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.517+0000 7f4d65200700 1 Processor -- start 2024-04-24T06:33:53.534 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.517+0000 7f4d65200700 1 -- start start 2024-04-24T06:33:53.534 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.517+0000 7f4d65200700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d6010a730 0x7f4d6019e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:53.534 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.517+0000 7f4d65200700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6010ec80 con 0x7f4d6010a730 2024-04-24T06:33:53.534 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.518+0000 7f4d5f400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d6010a730 0x7f4d6019e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:53.534 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.518+0000 7f4d5f400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d6010a730 0x7f4d6019e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33916/0 (socket says 172.21.15.5:33916) 2024-04-24T06:33:53.534 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.518+0000 7f4d5f400700 1 -- 172.21.15.5:0/1092575134 learned_addr learned my addr 172.21.15.5:0/1092575134 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:33:53.535 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.518+0000 7f4d5f400700 1 -- 172.21.15.5:0/1092575134 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d6019eba0 con 0x7f4d6010a730 2024-04-24T06:33:53.535 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.519+0000 7f4d5f400700 1 --2- 172.21.15.5:0/1092575134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d6010a730 0x7f4d6019e660 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4d50000c00 tx=0x7f4d50011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:53.535 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.519+0000 7f4d57e00700 1 -- 172.21.15.5:0/1092575134 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d50003b60 con 0x7f4d6010a730 2024-04-24T06:33:53.535 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.519+0000 7f4d57e00700 1 -- 172.21.15.5:0/1092575134 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d5001a910 con 0x7f4d6010a730 2024-04-24T06:33:53.535 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.519+0000 7f4d57e00700 1 -- 172.21.15.5:0/1092575134 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d50024930 con 0x7f4d6010a730 2024-04-24T06:33:53.535 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.519+0000 7f4d65200700 1 -- 172.21.15.5:0/1092575134 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d6019ee30 con 0x7f4d6010a730 2024-04-24T06:33:53.535 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.519+0000 7f4d65200700 1 -- 172.21.15.5:0/1092575134 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d6019f290 con 0x7f4d6010a730 2024-04-24T06:33:53.535 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.520+0000 7f4d57e00700 1 -- 172.21.15.5:0/1092575134 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7f4d50022020 con 0x7f4d6010a730 2024-04-24T06:33:53.535 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.520+0000 7f4d65200700 1 -- 172.21.15.5:0/1092575134 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d60069c50 con 0x7f4d6010a730 2024-04-24T06:33:53.535 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.521+0000 7f4d57e00700 1 --2- 172.21.15.5:0/1092575134 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f4d44039700 0x7f4d4403bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:53.535 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.521+0000 7f4d57e00700 1 -- 172.21.15.5:0/1092575134 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4d5004da40 con 0x7f4d6010a730 2024-04-24T06:33:53.535 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.521+0000 7f4d5ea00700 1 --2- 172.21.15.5:0/1092575134 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f4d44039700 0x7f4d4403bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:53.535 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.522+0000 7f4d5ea00700 1 --2- 172.21.15.5:0/1092575134 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f4d44039700 0x7f4d4403bbc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c006fd0 tx=0x7f4d4c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:53.536 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.526+0000 7f4d57e00700 1 -- 172.21.15.5:0/1092575134 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4d5004fe20 con 0x7f4d6010a730 2024-04-24T06:33:53.536 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.871+0000 7f4d65200700 1 -- 172.21.15.5:0/1092575134 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f4d601a1c60 con 0x7f4d6010a730 2024-04-24T06:33:53.536 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.876+0000 7f4d57e00700 1 -- 172.21.15.5:0/1092575134 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f4d5002a2a0 con 0x7f4d6010a730 2024-04-24T06:33:53.536 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.879+0000 7f4d65200700 1 -- 172.21.15.5:0/1092575134 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f4d44039700 msgr2=0x7f4d4403bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:53.536 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.879+0000 7f4d65200700 1 --2- 172.21.15.5:0/1092575134 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f4d44039700 0x7f4d4403bbc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c006fd0 tx=0x7f4d4c006e40 comp rx=0 tx=0).stop 2024-04-24T06:33:53.536 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.879+0000 7f4d65200700 1 -- 172.21.15.5:0/1092575134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d6010a730 msgr2=0x7f4d6019e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:53.536 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.879+0000 7f4d65200700 1 --2- 172.21.15.5:0/1092575134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d6010a730 0x7f4d6019e660 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4d50000c00 tx=0x7f4d50011c10 comp rx=0 tx=0).stop 2024-04-24T06:33:53.536 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.880+0000 7f4d65200700 1 -- 172.21.15.5:0/1092575134 shutdown_connections 2024-04-24T06:33:53.536 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.880+0000 7f4d65200700 1 --2- 172.21.15.5:0/1092575134 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f4d44039700 0x7f4d4403bbc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:53.536 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.880+0000 7f4d65200700 1 --2- 172.21.15.5:0/1092575134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d6010a730 0x7f4d6019e660 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:53.537 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.880+0000 7f4d65200700 1 -- 172.21.15.5:0/1092575134 >> 172.21.15.5:0/1092575134 conn(0x7f4d6007de30 msgr2=0x7f4d60107c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:53.537 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.880+0000 7f4d65200700 1 -- 172.21.15.5:0/1092575134 shutdown_connections 2024-04-24T06:33:53.537 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-04-24T06:33:52.880+0000 7f4d65200700 1 -- 172.21.15.5:0/1092575134 wait complete. 2024-04-24T06:33:53.537 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-24T06:33:53.538 INFO:teuthology.orchestra.run.smithi005.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-24T06:33:53.538 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:33:53.538 INFO:teuthology.orchestra.run.smithi005.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-24T06:33:53.538 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:33:53.538 INFO:teuthology.orchestra.run.smithi005.stdout:Or, if you are only running a single cluster on this host: 2024-04-24T06:33:53.538 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:33:53.538 INFO:teuthology.orchestra.run.smithi005.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-24T06:33:53.538 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:33:53.538 INFO:teuthology.orchestra.run.smithi005.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-24T06:33:53.538 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:33:53.538 INFO:teuthology.orchestra.run.smithi005.stdout: ceph telemetry on 2024-04-24T06:33:53.538 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:33:53.538 INFO:teuthology.orchestra.run.smithi005.stdout:For more information see: 2024-04-24T06:33:53.538 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:33:53.539 INFO:teuthology.orchestra.run.smithi005.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-24T06:33:53.539 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:33:53.539 INFO:teuthology.orchestra.run.smithi005.stdout:Bootstrap complete. 2024-04-24T06:33:53.638 INFO:tasks.cephadm:Fetching config... 2024-04-24T06:33:53.638 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-24T06:33:53.638 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-24T06:33:53.686 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-24T06:33:53.687 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-24T06:33:53.687 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-24T06:33:53.769 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-24T06:33:53.769 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-24T06:33:53.769 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-24T06:33:53.896 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-24T06:33:53.896 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-24T06:33:53.896 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-24T06:33:53.946 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-24T06:33:53.946 DEBUG:teuthology.orchestra.run.smithi005:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDi/8pw/bZ6/8Xsd/BkMGyfongGq3MLjEHeDsDuvvYn5OxaYqmusQxO2CO/1RjA+7cvLgUeNrldrn3SPkPmW/AuXXIbTIaabhH5hfS+bIZMqO1hmL6IB9AnRYQLYQHWI31AFtNtAlTC0EmiyZDp+5VXoxjOi0U/l/gcz9YD6FAf1zi95rwpTkkqZxCat65Czdw3yDhVp8Mpxmy0tdjuCAsG40JPZ9PZ9ern6ls4WKG05ljg24HfmHD+ArPUJD+0IenjpCu5ZjeKhwTLY4dAozZPuHXvvjTw8ki6pb1wsZiAfgav0NMvVNrtdpRFmI3IhQjVY1dhVdMA6rDoPm0f35Z0VJQtnCyCgsIVAmS2hkncW6+kf3NZnQlrBZwhPmTfOHH2aG/G/e1+YlUbcLipUj+qpedk0yIFOl7dit5/+RziGSBXnq7OB5Nx2hePaXu1QmVmoNfFf2XK/YcomopaoUEm2L0+Ar6tfc65gaxcRvkfGu9FGELwI7WnysIFEvtzZ6c= ceph-43e9bad0-0204-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-24T06:33:54.064 INFO:teuthology.orchestra.run.smithi005.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDi/8pw/bZ6/8Xsd/BkMGyfongGq3MLjEHeDsDuvvYn5OxaYqmusQxO2CO/1RjA+7cvLgUeNrldrn3SPkPmW/AuXXIbTIaabhH5hfS+bIZMqO1hmL6IB9AnRYQLYQHWI31AFtNtAlTC0EmiyZDp+5VXoxjOi0U/l/gcz9YD6FAf1zi95rwpTkkqZxCat65Czdw3yDhVp8Mpxmy0tdjuCAsG40JPZ9PZ9ern6ls4WKG05ljg24HfmHD+ArPUJD+0IenjpCu5ZjeKhwTLY4dAozZPuHXvvjTw8ki6pb1wsZiAfgav0NMvVNrtdpRFmI3IhQjVY1dhVdMA6rDoPm0f35Z0VJQtnCyCgsIVAmS2hkncW6+kf3NZnQlrBZwhPmTfOHH2aG/G/e1+YlUbcLipUj+qpedk0yIFOl7dit5/+RziGSBXnq7OB5Nx2hePaXu1QmVmoNfFf2XK/YcomopaoUEm2L0+Ar6tfc65gaxcRvkfGu9FGELwI7WnysIFEvtzZ6c= ceph-43e9bad0-0204-11ef-bc93-c7b262605968 2024-04-24T06:33:54.093 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:53 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1092575134' entity='client.admin' 2024-04-24T06:33:54.095 DEBUG:teuthology.orchestra.run.smithi037:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDi/8pw/bZ6/8Xsd/BkMGyfongGq3MLjEHeDsDuvvYn5OxaYqmusQxO2CO/1RjA+7cvLgUeNrldrn3SPkPmW/AuXXIbTIaabhH5hfS+bIZMqO1hmL6IB9AnRYQLYQHWI31AFtNtAlTC0EmiyZDp+5VXoxjOi0U/l/gcz9YD6FAf1zi95rwpTkkqZxCat65Czdw3yDhVp8Mpxmy0tdjuCAsG40JPZ9PZ9ern6ls4WKG05ljg24HfmHD+ArPUJD+0IenjpCu5ZjeKhwTLY4dAozZPuHXvvjTw8ki6pb1wsZiAfgav0NMvVNrtdpRFmI3IhQjVY1dhVdMA6rDoPm0f35Z0VJQtnCyCgsIVAmS2hkncW6+kf3NZnQlrBZwhPmTfOHH2aG/G/e1+YlUbcLipUj+qpedk0yIFOl7dit5/+RziGSBXnq7OB5Nx2hePaXu1QmVmoNfFf2XK/YcomopaoUEm2L0+Ar6tfc65gaxcRvkfGu9FGELwI7WnysIFEvtzZ6c= ceph-43e9bad0-0204-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-24T06:33:54.141 INFO:teuthology.orchestra.run.smithi037.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDi/8pw/bZ6/8Xsd/BkMGyfongGq3MLjEHeDsDuvvYn5OxaYqmusQxO2CO/1RjA+7cvLgUeNrldrn3SPkPmW/AuXXIbTIaabhH5hfS+bIZMqO1hmL6IB9AnRYQLYQHWI31AFtNtAlTC0EmiyZDp+5VXoxjOi0U/l/gcz9YD6FAf1zi95rwpTkkqZxCat65Czdw3yDhVp8Mpxmy0tdjuCAsG40JPZ9PZ9ern6ls4WKG05ljg24HfmHD+ArPUJD+0IenjpCu5ZjeKhwTLY4dAozZPuHXvvjTw8ki6pb1wsZiAfgav0NMvVNrtdpRFmI3IhQjVY1dhVdMA6rDoPm0f35Z0VJQtnCyCgsIVAmS2hkncW6+kf3NZnQlrBZwhPmTfOHH2aG/G/e1+YlUbcLipUj+qpedk0yIFOl7dit5/+RziGSBXnq7OB5Nx2hePaXu1QmVmoNfFf2XK/YcomopaoUEm2L0+Ar6tfc65gaxcRvkfGu9FGELwI7WnysIFEvtzZ6c= ceph-43e9bad0-0204-11ef-bc93-c7b262605968 2024-04-24T06:33:54.151 DEBUG:teuthology.orchestra.run.smithi137:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDi/8pw/bZ6/8Xsd/BkMGyfongGq3MLjEHeDsDuvvYn5OxaYqmusQxO2CO/1RjA+7cvLgUeNrldrn3SPkPmW/AuXXIbTIaabhH5hfS+bIZMqO1hmL6IB9AnRYQLYQHWI31AFtNtAlTC0EmiyZDp+5VXoxjOi0U/l/gcz9YD6FAf1zi95rwpTkkqZxCat65Czdw3yDhVp8Mpxmy0tdjuCAsG40JPZ9PZ9ern6ls4WKG05ljg24HfmHD+ArPUJD+0IenjpCu5ZjeKhwTLY4dAozZPuHXvvjTw8ki6pb1wsZiAfgav0NMvVNrtdpRFmI3IhQjVY1dhVdMA6rDoPm0f35Z0VJQtnCyCgsIVAmS2hkncW6+kf3NZnQlrBZwhPmTfOHH2aG/G/e1+YlUbcLipUj+qpedk0yIFOl7dit5/+RziGSBXnq7OB5Nx2hePaXu1QmVmoNfFf2XK/YcomopaoUEm2L0+Ar6tfc65gaxcRvkfGu9FGELwI7WnysIFEvtzZ6c= ceph-43e9bad0-0204-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-24T06:33:54.189 INFO:teuthology.orchestra.run.smithi137.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDi/8pw/bZ6/8Xsd/BkMGyfongGq3MLjEHeDsDuvvYn5OxaYqmusQxO2CO/1RjA+7cvLgUeNrldrn3SPkPmW/AuXXIbTIaabhH5hfS+bIZMqO1hmL6IB9AnRYQLYQHWI31AFtNtAlTC0EmiyZDp+5VXoxjOi0U/l/gcz9YD6FAf1zi95rwpTkkqZxCat65Czdw3yDhVp8Mpxmy0tdjuCAsG40JPZ9PZ9ern6ls4WKG05ljg24HfmHD+ArPUJD+0IenjpCu5ZjeKhwTLY4dAozZPuHXvvjTw8ki6pb1wsZiAfgav0NMvVNrtdpRFmI3IhQjVY1dhVdMA6rDoPm0f35Z0VJQtnCyCgsIVAmS2hkncW6+kf3NZnQlrBZwhPmTfOHH2aG/G/e1+YlUbcLipUj+qpedk0yIFOl7dit5/+RziGSBXnq7OB5Nx2hePaXu1QmVmoNfFf2XK/YcomopaoUEm2L0+Ar6tfc65gaxcRvkfGu9FGELwI7WnysIFEvtzZ6c= ceph-43e9bad0-0204-11ef-bc93-c7b262605968 2024-04-24T06:33:54.201 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-24T06:33:54.638 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:33:56.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:33:56.922+0000 7f5411000700 1 -- 172.21.15.5:0/264723305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f540c1047a0 msgr2=0x7f540c104bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:56.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:33:56.922+0000 7f5411000700 1 --2- 172.21.15.5:0/264723305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f540c1047a0 0x7f540c104bc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f53fc009910 tx=0x7f53fc009c20 comp rx=0 tx=0).stop 2024-04-24T06:33:56.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:33:56.922+0000 7f5411000700 1 -- 172.21.15.5:0/264723305 shutdown_connections 2024-04-24T06:33:56.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:33:56.922+0000 7f5411000700 1 --2- 172.21.15.5:0/264723305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f540c1047a0 0x7f540c104bc0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:56.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:33:56.922+0000 7f5411000700 1 -- 172.21.15.5:0/264723305 >> 172.21.15.5:0/264723305 conn(0x7f540c0ffd20 msgr2=0x7f540c102180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:56.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:33:56.922+0000 7f5411000700 1 -- 172.21.15.5:0/264723305 shutdown_connections 2024-04-24T06:33:56.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:33:56.922+0000 7f5411000700 1 -- 172.21.15.5:0/264723305 wait complete. 2024-04-24T06:33:56.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:33:56.924+0000 7f5411000700 1 Processor -- start 2024-04-24T06:33:56.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:33:56.924+0000 7f5411000700 1 -- start start 2024-04-24T06:33:56.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:33:56.924+0000 7f5411000700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f540c1047a0 0x7f540c199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:56.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:33:56.924+0000 7f5411000700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f540c10a830 con 0x7f540c1047a0 2024-04-24T06:33:56.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:33:56.924+0000 7f5409e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f540c1047a0 0x7f540c199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:56.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:33:56.925+0000 7f5409e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f540c1047a0 0x7f540c199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39928/0 (socket says 172.21.15.5:39928) 2024-04-24T06:33:56.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:33:56.925+0000 7f5409e00700 1 -- 172.21.15.5:0/1408423738 learned_addr learned my addr 172.21.15.5:0/1408423738 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:33:56.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:33:56.925+0000 7f5409e00700 1 -- 172.21.15.5:0/1408423738 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f540c19a3e0 con 0x7f540c1047a0 2024-04-24T06:33:56.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:33:56.925+0000 7f5409e00700 1 --2- 172.21.15.5:0/1408423738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f540c1047a0 0x7f540c199ea0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f53fc005850 tx=0x7f53fc005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:56.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:33:56.925+0000 7f5402a00700 1 -- 172.21.15.5:0/1408423738 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53fc026070 con 0x7f540c1047a0 2024-04-24T06:33:56.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:33:56.926+0000 7f5402a00700 1 -- 172.21.15.5:0/1408423738 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53fc0044a0 con 0x7f540c1047a0 2024-04-24T06:33:56.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:33:56.926+0000 7f5411000700 1 -- 172.21.15.5:0/1408423738 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f540c19a670 con 0x7f540c1047a0 2024-04-24T06:33:56.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:33:56.926+0000 7f5402a00700 1 -- 172.21.15.5:0/1408423738 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53fc003ac0 con 0x7f540c1047a0 2024-04-24T06:33:56.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:33:56.926+0000 7f5411000700 1 -- 172.21.15.5:0/1408423738 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f540c19d360 con 0x7f540c1047a0 2024-04-24T06:33:56.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:33:56.927+0000 7f5411000700 1 -- 172.21.15.5:0/1408423738 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f540c10afb0 con 0x7f540c1047a0 2024-04-24T06:33:56.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:33:56.927+0000 7f5402a00700 1 -- 172.21.15.5:0/1408423738 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7f53fc029070 con 0x7f540c1047a0 2024-04-24T06:33:56.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:33:56.927+0000 7f5402a00700 1 --2- 172.21.15.5:0/1408423738 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f53f00396b0 0x7f53f003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:33:56.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:33:56.928+0000 7f5402a00700 1 -- 172.21.15.5:0/1408423738 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f53fc01e070 con 0x7f540c1047a0 2024-04-24T06:33:56.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:33:56.928+0000 7f5409400700 1 --2- 172.21.15.5:0/1408423738 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f53f00396b0 0x7f53f003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:33:56.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:33:56.928+0000 7f5409400700 1 --2- 172.21.15.5:0/1408423738 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f53f00396b0 0x7f53f003bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f53f8006fd0 tx=0x7f53f8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:33:56.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:33:56.933+0000 7f5402a00700 1 -- 172.21.15.5:0/1408423738 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f53fc013070 con 0x7f540c1047a0 2024-04-24T06:33:57.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:33:57.166+0000 7f5411000700 1 -- 172.21.15.5:0/1408423738 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f540c02e0f0 con 0x7f540c1047a0 2024-04-24T06:33:57.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:33:57.170+0000 7f5402a00700 1 -- 172.21.15.5:0/1408423738 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f53fc0282c0 con 0x7f540c1047a0 2024-04-24T06:33:57.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:33:57.174+0000 7f5411000700 1 -- 172.21.15.5:0/1408423738 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f53f00396b0 msgr2=0x7f53f003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:57.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:33:57.174+0000 7f5411000700 1 --2- 172.21.15.5:0/1408423738 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f53f00396b0 0x7f53f003bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f53f8006fd0 tx=0x7f53f8006e40 comp rx=0 tx=0).stop 2024-04-24T06:33:57.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:33:57.174+0000 7f5411000700 1 -- 172.21.15.5:0/1408423738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f540c1047a0 msgr2=0x7f540c199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:33:57.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:33:57.174+0000 7f5411000700 1 --2- 172.21.15.5:0/1408423738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f540c1047a0 0x7f540c199ea0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f53fc005850 tx=0x7f53fc005030 comp rx=0 tx=0).stop 2024-04-24T06:33:57.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:33:57.174+0000 7f5411000700 1 -- 172.21.15.5:0/1408423738 shutdown_connections 2024-04-24T06:33:57.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:33:57.174+0000 7f5411000700 1 --2- 172.21.15.5:0/1408423738 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f53f00396b0 0x7f53f003bb70 secure :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f53f8006fd0 tx=0x7f53f8006e40 comp rx=0 tx=0).stop 2024-04-24T06:33:57.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:33:57.174+0000 7f5411000700 1 --2- 172.21.15.5:0/1408423738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f540c1047a0 0x7f540c199ea0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:33:57.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:33:57.174+0000 7f5411000700 1 -- 172.21.15.5:0/1408423738 >> 172.21.15.5:0/1408423738 conn(0x7f540c0ffd20 msgr2=0x7f540c1007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:33:57.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:33:57.174+0000 7f5411000700 1 -- 172.21.15.5:0/1408423738 shutdown_connections 2024-04-24T06:33:57.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:33:57.174+0000 7f5411000700 1 -- 172.21.15.5:0/1408423738 wait complete. 2024-04-24T06:33:57.915 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-24T06:33:57.916 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-24T06:33:58.350 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:33:58.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:58 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1408423738' entity='client.admin' 2024-04-24T06:33:58.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:58 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:33:58.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:58 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:33:58.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:58 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi005", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:33:58.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:58 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:33:58.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:58 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:33:58.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:33:58 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:00.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:00.331+0000 7f90fda00700 1 -- 172.21.15.5:0/3352207335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90f8108d00 msgr2=0x7f90f8109120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:00.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:00.331+0000 7f90fda00700 1 --2- 172.21.15.5:0/3352207335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90f8108d00 0x7f90f8109120 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f90e8009910 tx=0x7f90e8009c20 comp rx=0 tx=0).stop 2024-04-24T06:34:00.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:00.331+0000 7f90fda00700 1 -- 172.21.15.5:0/3352207335 shutdown_connections 2024-04-24T06:34:00.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:00.331+0000 7f90fda00700 1 --2- 172.21.15.5:0/3352207335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90f8108d00 0x7f90f8109120 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:00.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:00.331+0000 7f90fda00700 1 -- 172.21.15.5:0/3352207335 >> 172.21.15.5:0/3352207335 conn(0x7f90f8077be0 msgr2=0x7f90f807a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:00.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:00.331+0000 7f90fda00700 1 -- 172.21.15.5:0/3352207335 shutdown_connections 2024-04-24T06:34:00.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:00.331+0000 7f90fda00700 1 -- 172.21.15.5:0/3352207335 wait complete. 2024-04-24T06:34:00.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:00.332+0000 7f90fda00700 1 Processor -- start 2024-04-24T06:34:00.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:00.333+0000 7f90fda00700 1 -- start start 2024-04-24T06:34:00.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:00.333+0000 7f90fda00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90f8108d00 0x7f90f819e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:00.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:00.333+0000 7f90fda00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f90f810ec20 con 0x7f90f8108d00 2024-04-24T06:34:00.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:00.333+0000 7f90f6a00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90f8108d00 0x7f90f819e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:00.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:00.333+0000 7f90f6a00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90f8108d00 0x7f90f819e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39952/0 (socket says 172.21.15.5:39952) 2024-04-24T06:34:00.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:00.333+0000 7f90f6a00700 1 -- 172.21.15.5:0/2968651574 learned_addr learned my addr 172.21.15.5:0/2968651574 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:34:00.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:00.334+0000 7f90f6a00700 1 -- 172.21.15.5:0/2968651574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90f819e7d0 con 0x7f90f8108d00 2024-04-24T06:34:00.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:00.334+0000 7f90f6a00700 1 --2- 172.21.15.5:0/2968651574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90f8108d00 0x7f90f819e290 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f90e8005850 tx=0x7f90e8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:00.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:00.334+0000 7f90ef400700 1 -- 172.21.15.5:0/2968651574 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90e8026070 con 0x7f90f8108d00 2024-04-24T06:34:00.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:00.334+0000 7f90ef400700 1 -- 172.21.15.5:0/2968651574 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90e80044a0 con 0x7f90f8108d00 2024-04-24T06:34:00.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:00.334+0000 7f90fda00700 1 -- 172.21.15.5:0/2968651574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90f819ea60 con 0x7f90f8108d00 2024-04-24T06:34:00.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:00.335+0000 7f90ef400700 1 -- 172.21.15.5:0/2968651574 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90e8003ab0 con 0x7f90f8108d00 2024-04-24T06:34:00.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:00.335+0000 7f90fda00700 1 -- 172.21.15.5:0/2968651574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90f81a1750 con 0x7f90f8108d00 2024-04-24T06:34:00.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:00.336+0000 7f90fda00700 1 -- 172.21.15.5:0/2968651574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90f810f3a0 con 0x7f90f8108d00 2024-04-24T06:34:00.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:00.336+0000 7f90ef400700 1 -- 172.21.15.5:0/2968651574 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7f90e8029070 con 0x7f90f8108d00 2024-04-24T06:34:00.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:00.336+0000 7f90ef400700 1 --2- 172.21.15.5:0/2968651574 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f90dc039650 0x7f90dc03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:00.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:00.336+0000 7f90ef400700 1 -- 172.21.15.5:0/2968651574 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f90e801e070 con 0x7f90f8108d00 2024-04-24T06:34:00.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:00.337+0000 7f90f6000700 1 --2- 172.21.15.5:0/2968651574 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f90dc039650 0x7f90dc03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:00.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:00.337+0000 7f90f6000700 1 --2- 172.21.15.5:0/2968651574 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f90dc039650 0x7f90dc03bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f90e4006fd0 tx=0x7f90e4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:00.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:00.342+0000 7f90ef400700 1 -- 172.21.15.5:0/2968651574 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f90e8013070 con 0x7f90f8108d00 2024-04-24T06:34:00.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:00.580+0000 7f90fda00700 1 -- 172.21.15.5:0/2968651574 --> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f90f802e0f0 con 0x7f90dc039650 2024-04-24T06:34:00.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:00.587+0000 7f90ef400700 1 -- 172.21.15.5:0/2968651574 <== mgr.14152 v2:172.21.15.5:6800/4042936965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f90f802e0f0 con 0x7f90dc039650 2024-04-24T06:34:00.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:00.590+0000 7f90fda00700 1 -- 172.21.15.5:0/2968651574 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f90dc039650 msgr2=0x7f90dc03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:00.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:00.590+0000 7f90fda00700 1 --2- 172.21.15.5:0/2968651574 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f90dc039650 0x7f90dc03bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f90e4006fd0 tx=0x7f90e4006e40 comp rx=0 tx=0).stop 2024-04-24T06:34:00.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:00.590+0000 7f90fda00700 1 -- 172.21.15.5:0/2968651574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90f8108d00 msgr2=0x7f90f819e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:00.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:00.590+0000 7f90fda00700 1 --2- 172.21.15.5:0/2968651574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90f8108d00 0x7f90f819e290 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f90e8005850 tx=0x7f90e8005030 comp rx=0 tx=0).stop 2024-04-24T06:34:00.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:00.590+0000 7f90fda00700 1 -- 172.21.15.5:0/2968651574 shutdown_connections 2024-04-24T06:34:00.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:00.590+0000 7f90fda00700 1 --2- 172.21.15.5:0/2968651574 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f90dc039650 0x7f90dc03bb10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:00.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:00.590+0000 7f90fda00700 1 --2- 172.21.15.5:0/2968651574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90f8108d00 0x7f90f819e290 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:00.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:00.590+0000 7f90fda00700 1 -- 172.21.15.5:0/2968651574 >> 172.21.15.5:0/2968651574 conn(0x7f90f8077be0 msgr2=0x7f90f8078730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:00.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:00.590+0000 7f90fda00700 1 -- 172.21.15.5:0/2968651574 shutdown_connections 2024-04-24T06:34:00.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:00.590+0000 7f90fda00700 1 -- 172.21.15.5:0/2968651574 wait complete. 2024-04-24T06:34:01.358 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi037 2024-04-24T06:34:01.358 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-24T06:34:01.358 DEBUG:teuthology.orchestra.run.smithi037:> dd of=/etc/ceph/ceph.conf 2024-04-24T06:34:01.378 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-24T06:34:01.378 DEBUG:teuthology.orchestra.run.smithi037:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T06:34:01.436 INFO:tasks.cephadm:Adding host smithi037 to orchestrator... 2024-04-24T06:34:01.437 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph orch host add smithi037 2024-04-24T06:34:01.858 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:34:02.005 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:01 smithi005 ceph-mon[27798]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:34:02.005 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:01 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:02.005 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:01 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:34:02.005 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:01 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:34:02.005 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:01 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:34:02.005 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:01 smithi005 ceph-mon[27798]: Updating smithi005:/etc/ceph/ceph.conf 2024-04-24T06:34:03.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:02 smithi005 ceph-mon[27798]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:03.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:02 smithi005 ceph-mon[27798]: Updating smithi005:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T06:34:03.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:02 smithi005 ceph-mon[27798]: Updating smithi005:/etc/ceph/ceph.client.admin.keyring 2024-04-24T06:34:03.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:03.848+0000 7f6e32a00700 1 -- 172.21.15.5:0/2783450519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e2c1085d0 msgr2=0x7f6e2c1089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:03.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:03.848+0000 7f6e32a00700 1 --2- 172.21.15.5:0/2783450519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e2c1085d0 0x7f6e2c1089b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c00b0a0 tx=0x7f6e1c00b3b0 comp rx=0 tx=0).stop 2024-04-24T06:34:03.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:03.848+0000 7f6e32a00700 1 -- 172.21.15.5:0/2783450519 shutdown_connections 2024-04-24T06:34:03.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:03.848+0000 7f6e32a00700 1 --2- 172.21.15.5:0/2783450519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e2c1085d0 0x7f6e2c1089b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:03.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:03.848+0000 7f6e32a00700 1 -- 172.21.15.5:0/2783450519 >> 172.21.15.5:0/2783450519 conn(0x7f6e2c0fe030 msgr2=0x7f6e2c100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:03.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:03.848+0000 7f6e32a00700 1 -- 172.21.15.5:0/2783450519 shutdown_connections 2024-04-24T06:34:03.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:03.848+0000 7f6e32a00700 1 -- 172.21.15.5:0/2783450519 wait complete. 2024-04-24T06:34:03.852 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:03 smithi005 ceph-mon[27798]: Updating smithi005:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T06:34:03.852 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:03 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:03.852 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:03 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:03.852 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:03 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:03.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:03.849+0000 7f6e32a00700 1 Processor -- start 2024-04-24T06:34:03.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:03.849+0000 7f6e32a00700 1 -- start start 2024-04-24T06:34:03.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:03.850+0000 7f6e32a00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e2c1085d0 0x7f6e2c197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:03.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:03.850+0000 7f6e32a00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c10cb90 con 0x7f6e2c1085d0 2024-04-24T06:34:03.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:03.850+0000 7f6e2be00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e2c1085d0 0x7f6e2c197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:03.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:03.850+0000 7f6e2be00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e2c1085d0 0x7f6e2c197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39978/0 (socket says 172.21.15.5:39978) 2024-04-24T06:34:03.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:03.850+0000 7f6e2be00700 1 -- 172.21.15.5:0/2758432464 learned_addr learned my addr 172.21.15.5:0/2758432464 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:34:03.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:03.851+0000 7f6e2be00700 1 -- 172.21.15.5:0/2758432464 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e2c1984f0 con 0x7f6e2c1085d0 2024-04-24T06:34:03.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:03.851+0000 7f6e2be00700 1 --2- 172.21.15.5:0/2758432464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e2c1085d0 0x7f6e2c197fb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c000f80 tx=0x7f6e1c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:03.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:03.851+0000 7f6e29600700 1 -- 172.21.15.5:0/2758432464 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e1c017040 con 0x7f6e2c1085d0 2024-04-24T06:34:03.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:03.851+0000 7f6e32a00700 1 -- 172.21.15.5:0/2758432464 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e2c198780 con 0x7f6e2c1085d0 2024-04-24T06:34:03.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:03.851+0000 7f6e29600700 1 -- 172.21.15.5:0/2758432464 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e1c022070 con 0x7f6e2c1085d0 2024-04-24T06:34:03.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:03.851+0000 7f6e29600700 1 -- 172.21.15.5:0/2758432464 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e1c01d440 con 0x7f6e2c1085d0 2024-04-24T06:34:03.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:03.851+0000 7f6e32a00700 1 -- 172.21.15.5:0/2758432464 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e2c198be0 con 0x7f6e2c1085d0 2024-04-24T06:34:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:03.853+0000 7f6e29600700 1 -- 172.21.15.5:0/2758432464 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7f6e1c004000 con 0x7f6e2c1085d0 2024-04-24T06:34:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:03.853+0000 7f6e32a00700 1 -- 172.21.15.5:0/2758432464 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e2c103c40 con 0x7f6e2c1085d0 2024-04-24T06:34:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:03.853+0000 7f6e29600700 1 --2- 172.21.15.5:0/2758432464 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f6e10039700 0x7f6e1003bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:03.853+0000 7f6e29600700 1 -- 172.21.15.5:0/2758432464 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6e1c00f050 con 0x7f6e2c1085d0 2024-04-24T06:34:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:03.853+0000 7f6e23400700 1 --2- 172.21.15.5:0/2758432464 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f6e10039700 0x7f6e1003bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:03.854+0000 7f6e23400700 1 --2- 172.21.15.5:0/2758432464 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f6e10039700 0x7f6e1003bbc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6e14006fd0 tx=0x7f6e14006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:03.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:03.859+0000 7f6e29600700 1 -- 172.21.15.5:0/2758432464 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6e1c007e20 con 0x7f6e2c1085d0 2024-04-24T06:34:04.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:04.095+0000 7f6e32a00700 1 -- 172.21.15.5:0/2758432464 --> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi037", "target": ["mon-mgr", ""]}) -- 0x7f6e2c199380 con 0x7f6e10039700 2024-04-24T06:34:05.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:04 smithi005 ceph-mon[27798]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:06.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:05 smithi005 ceph-mon[27798]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi037", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:34:06.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:05 smithi005 ceph-mon[27798]: Deploying cephadm binary to smithi037 2024-04-24T06:34:07.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:06 smithi005 ceph-mon[27798]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:09.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:08 smithi005 ceph-mon[27798]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:09.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:09.015+0000 7f6e29600700 1 -- 172.21.15.5:0/2758432464 <== mgr.14152 v2:172.21.15.5:6800/4042936965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f6e2c199380 con 0x7f6e10039700 2024-04-24T06:34:09.017 INFO:teuthology.orchestra.run.smithi005.stdout:Added host 'smithi037' with addr '172.21.15.37' 2024-04-24T06:34:09.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:09.019+0000 7f6e32a00700 1 -- 172.21.15.5:0/2758432464 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f6e10039700 msgr2=0x7f6e1003bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:09.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:09.019+0000 7f6e32a00700 1 --2- 172.21.15.5:0/2758432464 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f6e10039700 0x7f6e1003bbc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6e14006fd0 tx=0x7f6e14006e40 comp rx=0 tx=0).stop 2024-04-24T06:34:09.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:09.019+0000 7f6e32a00700 1 -- 172.21.15.5:0/2758432464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e2c1085d0 msgr2=0x7f6e2c197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:09.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:09.019+0000 7f6e32a00700 1 --2- 172.21.15.5:0/2758432464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e2c1085d0 0x7f6e2c197fb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c000f80 tx=0x7f6e1c003ce0 comp rx=0 tx=0).stop 2024-04-24T06:34:09.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:09.019+0000 7f6e32a00700 1 -- 172.21.15.5:0/2758432464 shutdown_connections 2024-04-24T06:34:09.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:09.019+0000 7f6e32a00700 1 --2- 172.21.15.5:0/2758432464 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f6e10039700 0x7f6e1003bbc0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:09.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:09.019+0000 7f6e32a00700 1 --2- 172.21.15.5:0/2758432464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e2c1085d0 0x7f6e2c197fb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:09.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:09.019+0000 7f6e32a00700 1 -- 172.21.15.5:0/2758432464 >> 172.21.15.5:0/2758432464 conn(0x7f6e2c0fe030 msgr2=0x7f6e2c0fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:09.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:09.020+0000 7f6e32a00700 1 -- 172.21.15.5:0/2758432464 shutdown_connections 2024-04-24T06:34:09.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:09.020+0000 7f6e32a00700 1 -- 172.21.15.5:0/2758432464 wait complete. 2024-04-24T06:34:09.783 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-24T06:34:10.247 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:34:10.398 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:10 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:10.398 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:10 smithi005 ceph-mon[27798]: Added host smithi037 2024-04-24T06:34:10.398 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:10 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:34:10.398 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:10 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:11.427 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:11 smithi005 ceph-mon[27798]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:11.427 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:11 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:12.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:12.196+0000 7f1a2a200700 1 -- 172.21.15.5:0/609100243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a241036c0 msgr2=0x7f1a24103aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:12.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:12.196+0000 7f1a2a200700 1 --2- 172.21.15.5:0/609100243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a241036c0 0x7f1a24103aa0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1a14009910 tx=0x7f1a14009c20 comp rx=0 tx=0).stop 2024-04-24T06:34:12.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:12.196+0000 7f1a2a200700 1 -- 172.21.15.5:0/609100243 shutdown_connections 2024-04-24T06:34:12.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:12.196+0000 7f1a2a200700 1 --2- 172.21.15.5:0/609100243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a241036c0 0x7f1a24103aa0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:12.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:12.196+0000 7f1a2a200700 1 -- 172.21.15.5:0/609100243 >> 172.21.15.5:0/609100243 conn(0x7f1a240fcef0 msgr2=0x7f1a240ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:12.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:12.196+0000 7f1a2a200700 1 -- 172.21.15.5:0/609100243 shutdown_connections 2024-04-24T06:34:12.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:12.196+0000 7f1a2a200700 1 -- 172.21.15.5:0/609100243 wait complete. 2024-04-24T06:34:12.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:12.198+0000 7f1a2a200700 1 Processor -- start 2024-04-24T06:34:12.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:12.198+0000 7f1a2a200700 1 -- start start 2024-04-24T06:34:12.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:12.198+0000 7f1a2a200700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a241036c0 0x7f1a241980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:12.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:12.198+0000 7f1a2a200700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2410cb90 con 0x7f1a241036c0 2024-04-24T06:34:12.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:12.199+0000 7f1a23400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a241036c0 0x7f1a241980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:12.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:12.199+0000 7f1a23400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a241036c0 0x7f1a241980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47726/0 (socket says 172.21.15.5:47726) 2024-04-24T06:34:12.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:12.199+0000 7f1a23400700 1 -- 172.21.15.5:0/3924432091 learned_addr learned my addr 172.21.15.5:0/3924432091 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:34:12.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:12.199+0000 7f1a23400700 1 -- 172.21.15.5:0/3924432091 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a24198600 con 0x7f1a241036c0 2024-04-24T06:34:12.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:12.200+0000 7f1a23400700 1 --2- 172.21.15.5:0/3924432091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a241036c0 0x7f1a241980c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1a14006010 tx=0x7f1a14005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:12.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:12.200+0000 7f1a1be00700 1 -- 172.21.15.5:0/3924432091 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a14026070 con 0x7f1a241036c0 2024-04-24T06:34:12.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:12.200+0000 7f1a1be00700 1 -- 172.21.15.5:0/3924432091 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a1400bc30 con 0x7f1a241036c0 2024-04-24T06:34:12.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:12.200+0000 7f1a1be00700 1 -- 172.21.15.5:0/3924432091 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a14003ab0 con 0x7f1a241036c0 2024-04-24T06:34:12.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:12.200+0000 7f1a2a200700 1 -- 172.21.15.5:0/3924432091 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a24198890 con 0x7f1a241036c0 2024-04-24T06:34:12.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:12.200+0000 7f1a2a200700 1 -- 172.21.15.5:0/3924432091 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a24198cf0 con 0x7f1a241036c0 2024-04-24T06:34:12.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:12.201+0000 7f1a1be00700 1 -- 172.21.15.5:0/3924432091 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7f1a14029070 con 0x7f1a241036c0 2024-04-24T06:34:12.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:12.201+0000 7f1a2a200700 1 -- 172.21.15.5:0/3924432091 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a2406b1e0 con 0x7f1a241036c0 2024-04-24T06:34:12.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:12.202+0000 7f1a1be00700 1 --2- 172.21.15.5:0/3924432091 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f1a080396b0 0x7f1a0803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:12.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:12.202+0000 7f1a1be00700 1 -- 172.21.15.5:0/3924432091 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1a1401e070 con 0x7f1a241036c0 2024-04-24T06:34:12.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:12.202+0000 7f1a22a00700 1 --2- 172.21.15.5:0/3924432091 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f1a080396b0 0x7f1a0803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:12.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:12.203+0000 7f1a22a00700 1 --2- 172.21.15.5:0/3924432091 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f1a080396b0 0x7f1a0803bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1a10006fd0 tx=0x7f1a10006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:12.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:12.208+0000 7f1a1be00700 1 -- 172.21.15.5:0/3924432091 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1a14013070 con 0x7f1a241036c0 2024-04-24T06:34:12.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:12.446+0000 7f1a2a200700 1 -- 172.21.15.5:0/3924432091 --> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1a24199490 con 0x7f1a080396b0 2024-04-24T06:34:12.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:12.449+0000 7f1a1be00700 1 -- 172.21.15.5:0/3924432091 <== mgr.14152 v2:172.21.15.5:6800/4042936965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+158 (secure 0 0 0) 0x7f1a24199490 con 0x7f1a080396b0 2024-04-24T06:34:12.450 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:34:12.450 INFO:teuthology.orchestra.run.smithi005.stdout:[{"addr": "172.21.15.5", "hostname": "smithi005", "labels": [], "status": ""}, {"addr": "172.21.15.37", "hostname": "smithi037", "labels": [], "status": ""}] 2024-04-24T06:34:12.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:12.452+0000 7f1a2a200700 1 -- 172.21.15.5:0/3924432091 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f1a080396b0 msgr2=0x7f1a0803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:12.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:12.452+0000 7f1a2a200700 1 --2- 172.21.15.5:0/3924432091 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f1a080396b0 0x7f1a0803bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1a10006fd0 tx=0x7f1a10006e40 comp rx=0 tx=0).stop 2024-04-24T06:34:12.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:12.452+0000 7f1a2a200700 1 -- 172.21.15.5:0/3924432091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a241036c0 msgr2=0x7f1a241980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:12.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:12.452+0000 7f1a2a200700 1 --2- 172.21.15.5:0/3924432091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a241036c0 0x7f1a241980c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1a14006010 tx=0x7f1a14005050 comp rx=0 tx=0).stop 2024-04-24T06:34:12.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:12.452+0000 7f1a2a200700 1 -- 172.21.15.5:0/3924432091 shutdown_connections 2024-04-24T06:34:12.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:12.452+0000 7f1a2a200700 1 --2- 172.21.15.5:0/3924432091 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f1a080396b0 0x7f1a0803bb70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:12.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:12.452+0000 7f1a2a200700 1 --2- 172.21.15.5:0/3924432091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a241036c0 0x7f1a241980c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:12.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:12.452+0000 7f1a2a200700 1 -- 172.21.15.5:0/3924432091 >> 172.21.15.5:0/3924432091 conn(0x7f1a240fcef0 msgr2=0x7f1a24104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:12.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:12.452+0000 7f1a2a200700 1 -- 172.21.15.5:0/3924432091 shutdown_connections 2024-04-24T06:34:12.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:12.453+0000 7f1a2a200700 1 -- 172.21.15.5:0/3924432091 wait complete. 2024-04-24T06:34:13.120 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:12 smithi005 ceph-mon[27798]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:13.120 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:12 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:13.179 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi137 2024-04-24T06:34:13.180 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-24T06:34:13.180 DEBUG:teuthology.orchestra.run.smithi137:> dd of=/etc/ceph/ceph.conf 2024-04-24T06:34:13.199 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-24T06:34:13.199 DEBUG:teuthology.orchestra.run.smithi137:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T06:34:13.256 INFO:tasks.cephadm:Adding host smithi137 to orchestrator... 2024-04-24T06:34:13.256 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph orch host add smithi137 2024-04-24T06:34:13.704 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:34:14.097 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:13 smithi005 ceph-mon[27798]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T06:34:14.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:14 smithi005 ceph-mon[27798]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:15.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:15.754+0000 7f5162000700 1 -- 172.21.15.5:0/850176071 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5154005590 con 0x7f51641025a0 2024-04-24T06:34:15.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:15.755+0000 7f5163e00700 1 -- 172.21.15.5:0/850176071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51641025a0 msgr2=0x7f5164102980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:15.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:15.755+0000 7f5163e00700 1 --2- 172.21.15.5:0/850176071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51641025a0 0x7f5164102980 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5154009910 tx=0x7f5154009c20 comp rx=0 tx=0).stop 2024-04-24T06:34:15.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:15.755+0000 7f5163e00700 1 -- 172.21.15.5:0/850176071 shutdown_connections 2024-04-24T06:34:15.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:15.755+0000 7f5163e00700 1 --2- 172.21.15.5:0/850176071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51641025a0 0x7f5164102980 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:15.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:15.755+0000 7f5163e00700 1 -- 172.21.15.5:0/850176071 >> 172.21.15.5:0/850176071 conn(0x7f51640fde50 msgr2=0x7f5164100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:15.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:15.755+0000 7f5163e00700 1 -- 172.21.15.5:0/850176071 shutdown_connections 2024-04-24T06:34:15.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:15.755+0000 7f5163e00700 1 -- 172.21.15.5:0/850176071 wait complete. 2024-04-24T06:34:15.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:15.757+0000 7f5163e00700 1 Processor -- start 2024-04-24T06:34:15.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:15.757+0000 7f5163e00700 1 -- start start 2024-04-24T06:34:15.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:15.757+0000 7f5163e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f516419a2a0 0x7f516419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:15.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:15.757+0000 7f5163e00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f516410c970 con 0x7f516419a2a0 2024-04-24T06:34:15.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:15.758+0000 7f5162a00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f516419a2a0 0x7f516419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:15.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:15.758+0000 7f5162a00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f516419a2a0 0x7f516419a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51108/0 (socket says 172.21.15.5:51108) 2024-04-24T06:34:15.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:15.758+0000 7f5162a00700 1 -- 172.21.15.5:0/488752802 learned_addr learned my addr 172.21.15.5:0/488752802 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:34:15.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:15.758+0000 7f5162a00700 1 -- 172.21.15.5:0/488752802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f516419abc0 con 0x7f516419a2a0 2024-04-24T06:34:15.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:15.758+0000 7f5162a00700 1 --2- 172.21.15.5:0/488752802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f516419a2a0 0x7f516419a680 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f515400b5c0 tx=0x7f5154005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:15.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:15.759+0000 7f515be00700 1 -- 172.21.15.5:0/488752802 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5154026070 con 0x7f516419a2a0 2024-04-24T06:34:15.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:15.759+0000 7f515be00700 1 -- 172.21.15.5:0/488752802 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f515400b6f0 con 0x7f516419a2a0 2024-04-24T06:34:15.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:15.759+0000 7f515be00700 1 -- 172.21.15.5:0/488752802 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51540037f0 con 0x7f516419a2a0 2024-04-24T06:34:15.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:15.759+0000 7f5163e00700 1 -- 172.21.15.5:0/488752802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f516419aeb0 con 0x7f516419a2a0 2024-04-24T06:34:15.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:15.759+0000 7f5163e00700 1 -- 172.21.15.5:0/488752802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f516419ed40 con 0x7f516419a2a0 2024-04-24T06:34:15.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:15.760+0000 7f515be00700 1 -- 172.21.15.5:0/488752802 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7f5154029070 con 0x7f516419a2a0 2024-04-24T06:34:15.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:15.760+0000 7f5158c00700 1 -- 172.21.15.5:0/488752802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f513c005330 con 0x7f516419a2a0 2024-04-24T06:34:15.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:15.760+0000 7f515be00700 1 --2- 172.21.15.5:0/488752802 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f51480396b0 0x7f514803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:15.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:15.761+0000 7f515be00700 1 -- 172.21.15.5:0/488752802 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f515401e070 con 0x7f516419a2a0 2024-04-24T06:34:15.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:15.761+0000 7f515a000700 1 --2- 172.21.15.5:0/488752802 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f51480396b0 0x7f514803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:15.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:15.762+0000 7f515a000700 1 --2- 172.21.15.5:0/488752802 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f51480396b0 0x7f514803bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f514c006fd0 tx=0x7f514c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:15.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:15.766+0000 7f515be00700 1 -- 172.21.15.5:0/488752802 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5154013070 con 0x7f516419a2a0 2024-04-24T06:34:16.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:16.060+0000 7f5158c00700 1 -- 172.21.15.5:0/488752802 --> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi137", "target": ["mon-mgr", ""]}) -- 0x7f513c000bc0 con 0x7f51480396b0 2024-04-24T06:34:16.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:16 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:16.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:16 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:16.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:16 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:16.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:16 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:16.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:16 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi037", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:34:16.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:16 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:34:16.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:16 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:34:16.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:16 smithi005 ceph-mon[27798]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-24T06:34:16.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:16 smithi005 ceph-mon[27798]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:16.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:16 smithi005 ceph-mon[27798]: Updating smithi037:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T06:34:16.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:16 smithi005 ceph-mon[27798]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-04-24T06:34:16.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:16 smithi005 ceph-mon[27798]: Updating smithi037:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T06:34:16.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:16 smithi005 ceph-mon[27798]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi137", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:34:17.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:17 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:17.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:17 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:17.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:17 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:17.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:17 smithi005 ceph-mon[27798]: Deploying cephadm binary to smithi137 2024-04-24T06:34:18.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:18 smithi005 ceph-mon[27798]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:20.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:20 smithi005 ceph-mon[27798]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:20.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:20.894+0000 7f515be00700 1 -- 172.21.15.5:0/488752802 <== mgr.14152 v2:172.21.15.5:6800/4042936965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f513c000bc0 con 0x7f51480396b0 2024-04-24T06:34:20.896 INFO:teuthology.orchestra.run.smithi005.stdout:Added host 'smithi137' with addr '172.21.15.137' 2024-04-24T06:34:20.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:20.898+0000 7f5158c00700 1 -- 172.21.15.5:0/488752802 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f51480396b0 msgr2=0x7f514803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:20.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:20.898+0000 7f5158c00700 1 --2- 172.21.15.5:0/488752802 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f51480396b0 0x7f514803bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f514c006fd0 tx=0x7f514c006e40 comp rx=0 tx=0).stop 2024-04-24T06:34:20.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:20.898+0000 7f5158c00700 1 -- 172.21.15.5:0/488752802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f516419a2a0 msgr2=0x7f516419a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:20.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:20.898+0000 7f5158c00700 1 --2- 172.21.15.5:0/488752802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f516419a2a0 0x7f516419a680 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f515400b5c0 tx=0x7f5154005030 comp rx=0 tx=0).stop 2024-04-24T06:34:20.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:20.898+0000 7f5158c00700 1 -- 172.21.15.5:0/488752802 shutdown_connections 2024-04-24T06:34:20.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:20.898+0000 7f5158c00700 1 --2- 172.21.15.5:0/488752802 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f51480396b0 0x7f514803bb70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:20.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:20.898+0000 7f5158c00700 1 --2- 172.21.15.5:0/488752802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f516419a2a0 0x7f516419a680 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:20.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:20.898+0000 7f5158c00700 1 -- 172.21.15.5:0/488752802 >> 172.21.15.5:0/488752802 conn(0x7f51640fde50 msgr2=0x7f51640ff6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:20.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:20.899+0000 7f5158c00700 1 -- 172.21.15.5:0/488752802 shutdown_connections 2024-04-24T06:34:20.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:20.899+0000 7f5158c00700 1 -- 172.21.15.5:0/488752802 wait complete. 2024-04-24T06:34:21.670 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-24T06:34:22.141 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:34:22.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:21 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:22.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:21 smithi005 ceph-mon[27798]: Added host smithi137 2024-04-24T06:34:22.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:21 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:34:22.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:21 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:23.301 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:23 smithi005 ceph-mon[27798]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:23.301 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:23 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:24.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:24.190+0000 7efe0f200700 1 -- 172.21.15.5:0/4117398789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe10074ec0 msgr2=0x7efe100752a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:24.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:24.190+0000 7efe0f200700 1 --2- 172.21.15.5:0/4117398789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe10074ec0 0x7efe100752a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7efe00009910 tx=0x7efe00009c20 comp rx=0 tx=0).stop 2024-04-24T06:34:24.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:24.190+0000 7efe0f200700 1 -- 172.21.15.5:0/4117398789 shutdown_connections 2024-04-24T06:34:24.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:24.190+0000 7efe0f200700 1 --2- 172.21.15.5:0/4117398789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe10074ec0 0x7efe100752a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:24.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:24.190+0000 7efe0f200700 1 -- 172.21.15.5:0/4117398789 >> 172.21.15.5:0/4117398789 conn(0x7efe100fde50 msgr2=0x7efe10100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:24.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:24.190+0000 7efe0f200700 1 -- 172.21.15.5:0/4117398789 shutdown_connections 2024-04-24T06:34:24.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:24.190+0000 7efe0f200700 1 -- 172.21.15.5:0/4117398789 wait complete. 2024-04-24T06:34:24.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:24.192+0000 7efe0f200700 1 Processor -- start 2024-04-24T06:34:24.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:24.192+0000 7efe0f200700 1 -- start start 2024-04-24T06:34:24.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:24.192+0000 7efe0f200700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe10074ec0 0x7efe1019a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:24.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:24.192+0000 7efe0f200700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efe1010ebe0 con 0x7efe10074ec0 2024-04-24T06:34:24.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:24.193+0000 7efe0de00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe10074ec0 0x7efe1019a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:24.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:24.193+0000 7efe0de00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe10074ec0 0x7efe1019a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57894/0 (socket says 172.21.15.5:57894) 2024-04-24T06:34:24.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:24.193+0000 7efe0de00700 1 -- 172.21.15.5:0/599795261 learned_addr learned my addr 172.21.15.5:0/599795261 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:34:24.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:24.193+0000 7efe0de00700 1 -- 172.21.15.5:0/599795261 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe1019a600 con 0x7efe10074ec0 2024-04-24T06:34:24.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:24.193+0000 7efe0de00700 1 --2- 172.21.15.5:0/599795261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe10074ec0 0x7efe1019a0c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efe00006010 tx=0x7efe00005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:24.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:24.194+0000 7efe06a00700 1 -- 172.21.15.5:0/599795261 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe00026070 con 0x7efe10074ec0 2024-04-24T06:34:24.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:24.194+0000 7efe06a00700 1 -- 172.21.15.5:0/599795261 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe0000bc30 con 0x7efe10074ec0 2024-04-24T06:34:24.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:24.194+0000 7efe06a00700 1 -- 172.21.15.5:0/599795261 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe00003ab0 con 0x7efe10074ec0 2024-04-24T06:34:24.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:24.194+0000 7efe0f200700 1 -- 172.21.15.5:0/599795261 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe1019a890 con 0x7efe10074ec0 2024-04-24T06:34:24.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:24.194+0000 7efe0f200700 1 -- 172.21.15.5:0/599795261 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe1019acf0 con 0x7efe10074ec0 2024-04-24T06:34:24.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:24.195+0000 7efe0f200700 1 -- 172.21.15.5:0/599795261 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe10076570 con 0x7efe10074ec0 2024-04-24T06:34:24.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:24.195+0000 7efe06a00700 1 -- 172.21.15.5:0/599795261 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7efe00029070 con 0x7efe10074ec0 2024-04-24T06:34:24.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:24.196+0000 7efe06a00700 1 --2- 172.21.15.5:0/599795261 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7efdf4039700 0x7efdf403bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:24.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:24.196+0000 7efe06a00700 1 -- 172.21.15.5:0/599795261 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7efe0001e070 con 0x7efe10074ec0 2024-04-24T06:34:24.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:24.196+0000 7efe0d400700 1 --2- 172.21.15.5:0/599795261 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7efdf4039700 0x7efdf403bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:24.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:24.197+0000 7efe0d400700 1 --2- 172.21.15.5:0/599795261 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7efdf4039700 0x7efdf403bbc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7efdfc006fd0 tx=0x7efdfc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:24.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:24.202+0000 7efe06a00700 1 -- 172.21.15.5:0/599795261 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efe00013070 con 0x7efe10074ec0 2024-04-24T06:34:24.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:24.437+0000 7efe0f200700 1 -- 172.21.15.5:0/599795261 --> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efe1019b490 con 0x7efdf4039700 2024-04-24T06:34:24.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:24.439+0000 7efe06a00700 1 -- 172.21.15.5:0/599795261 <== mgr.14152 v2:172.21.15.5:6800/4042936965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+238 (secure 0 0 0) 0x7efe1019b490 con 0x7efdf4039700 2024-04-24T06:34:24.441 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:34:24.441 INFO:teuthology.orchestra.run.smithi005.stdout:[{"addr": "172.21.15.5", "hostname": "smithi005", "labels": [], "status": ""}, {"addr": "172.21.15.37", "hostname": "smithi037", "labels": [], "status": ""}, {"addr": "172.21.15.137", "hostname": "smithi137", "labels": [], "status": ""}] 2024-04-24T06:34:24.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:24.442+0000 7efe0f200700 1 -- 172.21.15.5:0/599795261 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7efdf4039700 msgr2=0x7efdf403bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:24.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:24.442+0000 7efe0f200700 1 --2- 172.21.15.5:0/599795261 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7efdf4039700 0x7efdf403bbc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7efdfc006fd0 tx=0x7efdfc006e40 comp rx=0 tx=0).stop 2024-04-24T06:34:24.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:24.442+0000 7efe0f200700 1 -- 172.21.15.5:0/599795261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe10074ec0 msgr2=0x7efe1019a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:24.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:24.442+0000 7efe0f200700 1 --2- 172.21.15.5:0/599795261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe10074ec0 0x7efe1019a0c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efe00006010 tx=0x7efe00005050 comp rx=0 tx=0).stop 2024-04-24T06:34:24.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:24.443+0000 7efe0f200700 1 -- 172.21.15.5:0/599795261 shutdown_connections 2024-04-24T06:34:24.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:24.443+0000 7efe0f200700 1 --2- 172.21.15.5:0/599795261 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7efdf4039700 0x7efdf403bbc0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:24.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:24.443+0000 7efe0f200700 1 --2- 172.21.15.5:0/599795261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe10074ec0 0x7efe1019a0c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:24.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:24.443+0000 7efe0f200700 1 -- 172.21.15.5:0/599795261 >> 172.21.15.5:0/599795261 conn(0x7efe100fde50 msgr2=0x7efe10106640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:24.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:24.443+0000 7efe0f200700 1 -- 172.21.15.5:0/599795261 shutdown_connections 2024-04-24T06:34:24.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:24.443+0000 7efe0f200700 1 -- 172.21.15.5:0/599795261 wait complete. 2024-04-24T06:34:24.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:24 smithi005 ceph-mon[27798]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:24.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:24 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:25.148 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-24T06:34:25.148 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-24T06:34:25.601 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:34:25.923 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:25 smithi005 ceph-mon[27798]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T06:34:26.851 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:26 smithi005 ceph-mon[27798]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:27.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:27.628+0000 7f39eea00700 1 -- 172.21.15.5:0/1876570004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39e8101490 msgr2=0x7f39e8101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:27.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:27.628+0000 7f39eea00700 1 --2- 172.21.15.5:0/1876570004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39e8101490 0x7f39e8101870 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f39d8009910 tx=0x7f39d8009c20 comp rx=0 tx=0).stop 2024-04-24T06:34:27.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:27.628+0000 7f39eea00700 1 -- 172.21.15.5:0/1876570004 shutdown_connections 2024-04-24T06:34:27.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:27.628+0000 7f39eea00700 1 --2- 172.21.15.5:0/1876570004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39e8101490 0x7f39e8101870 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:27.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:27.628+0000 7f39eea00700 1 -- 172.21.15.5:0/1876570004 >> 172.21.15.5:0/1876570004 conn(0x7f39e80fcef0 msgr2=0x7f39e80ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:27.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:27.628+0000 7f39eea00700 1 -- 172.21.15.5:0/1876570004 shutdown_connections 2024-04-24T06:34:27.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:27.628+0000 7f39eea00700 1 -- 172.21.15.5:0/1876570004 wait complete. 2024-04-24T06:34:27.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:27.630+0000 7f39eea00700 1 Processor -- start 2024-04-24T06:34:27.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:27.630+0000 7f39eea00700 1 -- start start 2024-04-24T06:34:27.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:27.630+0000 7f39eea00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39e8101490 0x7f39e819e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:27.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:27.630+0000 7f39eea00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39e8113190 con 0x7f39e8101490 2024-04-24T06:34:27.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:27.631+0000 7f39e7e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39e8101490 0x7f39e819e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:27.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:27.631+0000 7f39e7e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39e8101490 0x7f39e819e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57910/0 (socket says 172.21.15.5:57910) 2024-04-24T06:34:27.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:27.631+0000 7f39e7e00700 1 -- 172.21.15.5:0/1385169739 learned_addr learned my addr 172.21.15.5:0/1385169739 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:34:27.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:27.631+0000 7f39e7e00700 1 -- 172.21.15.5:0/1385169739 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e819ebc0 con 0x7f39e8101490 2024-04-24T06:34:27.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:27.632+0000 7f39e7e00700 1 --2- 172.21.15.5:0/1385169739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39e8101490 0x7f39e819e680 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f39d8006010 tx=0x7f39d8005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:27.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:27.632+0000 7f39e4c00700 1 -- 172.21.15.5:0/1385169739 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39d8026070 con 0x7f39e8101490 2024-04-24T06:34:27.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:27.632+0000 7f39e4c00700 1 -- 172.21.15.5:0/1385169739 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39d800bc30 con 0x7f39e8101490 2024-04-24T06:34:27.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:27.632+0000 7f39eea00700 1 -- 172.21.15.5:0/1385169739 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39e819ee50 con 0x7f39e8101490 2024-04-24T06:34:27.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:27.632+0000 7f39e4c00700 1 -- 172.21.15.5:0/1385169739 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39d8003ab0 con 0x7f39e8101490 2024-04-24T06:34:27.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:27.632+0000 7f39eea00700 1 -- 172.21.15.5:0/1385169739 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39e819f2b0 con 0x7f39e8101490 2024-04-24T06:34:27.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:27.633+0000 7f39eea00700 1 -- 172.21.15.5:0/1385169739 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39e806b080 con 0x7f39e8101490 2024-04-24T06:34:27.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:27.633+0000 7f39e4c00700 1 -- 172.21.15.5:0/1385169739 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7f39d8029070 con 0x7f39e8101490 2024-04-24T06:34:27.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:27.634+0000 7f39e4c00700 1 --2- 172.21.15.5:0/1385169739 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f39cc0396b0 0x7f39cc03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:27.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:27.634+0000 7f39e4c00700 1 -- 172.21.15.5:0/1385169739 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f39d801e070 con 0x7f39e8101490 2024-04-24T06:34:27.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:27.634+0000 7f39e7400700 1 --2- 172.21.15.5:0/1385169739 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f39cc0396b0 0x7f39cc03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:27.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:27.635+0000 7f39e7400700 1 --2- 172.21.15.5:0/1385169739 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f39cc0396b0 0x7f39cc03bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f39d4006fd0 tx=0x7f39d4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:27.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:27.639+0000 7f39e4c00700 1 -- 172.21.15.5:0/1385169739 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f39d8013070 con 0x7f39e8101490 2024-04-24T06:34:27.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:27 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:27.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:27 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:27.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:27 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:27.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:27 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:27.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:27 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi137", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:34:27.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:27 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:34:27.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:27 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:34:27.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:27 smithi005 ceph-mon[27798]: Updating smithi137:/etc/ceph/ceph.conf 2024-04-24T06:34:27.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:27 smithi005 ceph-mon[27798]: Updating smithi137:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T06:34:27.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:27.883+0000 7f39eea00700 1 -- 172.21.15.5:0/1385169739 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f39e819f9e0 con 0x7f39e8101490 2024-04-24T06:34:28.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:28.724+0000 7f39e4c00700 1 -- 172.21.15.5:0/1385169739 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f39d8017440 con 0x7f39e8101490 2024-04-24T06:34:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:28.729+0000 7f39eea00700 1 -- 172.21.15.5:0/1385169739 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f39cc0396b0 msgr2=0x7f39cc03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:28.729+0000 7f39eea00700 1 --2- 172.21.15.5:0/1385169739 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f39cc0396b0 0x7f39cc03bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f39d4006fd0 tx=0x7f39d4006e40 comp rx=0 tx=0).stop 2024-04-24T06:34:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:28.729+0000 7f39eea00700 1 -- 172.21.15.5:0/1385169739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39e8101490 msgr2=0x7f39e819e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:28.729+0000 7f39eea00700 1 --2- 172.21.15.5:0/1385169739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39e8101490 0x7f39e819e680 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f39d8006010 tx=0x7f39d8005050 comp rx=0 tx=0).stop 2024-04-24T06:34:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:28.729+0000 7f39eea00700 1 -- 172.21.15.5:0/1385169739 shutdown_connections 2024-04-24T06:34:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:28.729+0000 7f39eea00700 1 --2- 172.21.15.5:0/1385169739 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f39cc0396b0 0x7f39cc03bb70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:28.729+0000 7f39eea00700 1 --2- 172.21.15.5:0/1385169739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39e8101490 0x7f39e819e680 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:28.729+0000 7f39eea00700 1 -- 172.21.15.5:0/1385169739 >> 172.21.15.5:0/1385169739 conn(0x7f39e80fcef0 msgr2=0x7f39e80fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:28.729+0000 7f39eea00700 1 -- 172.21.15.5:0/1385169739 shutdown_connections 2024-04-24T06:34:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:28.729+0000 7f39eea00700 1 -- 172.21.15.5:0/1385169739 wait complete. 2024-04-24T06:34:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:adjusted tunables profile to default 2024-04-24T06:34:29.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:28 smithi005 ceph-mon[27798]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:29.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:28 smithi005 ceph-mon[27798]: Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-04-24T06:34:29.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:28 smithi005 ceph-mon[27798]: Updating smithi137:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T06:34:29.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:28 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1385169739' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-24T06:34:29.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:28 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:29.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:28 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:29.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:28 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:29.472 INFO:tasks.cephadm:Adding mon.a on smithi005 2024-04-24T06:34:29.472 INFO:tasks.cephadm:Adding mon.b on smithi037 2024-04-24T06:34:29.472 INFO:tasks.cephadm:Adding mon.c on smithi137 2024-04-24T06:34:29.473 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi005:172.21.15.5=a;smithi037:172.21.15.37=b;smithi137:172.21.15.137=c' 2024-04-24T06:34:29.710 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T06:34:30.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:29 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1385169739' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-24T06:34:30.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:29 smithi005 ceph-mon[27798]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T06:34:31.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:30 smithi005 ceph-mon[27798]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:31.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:31.155+0000 7ff8dad22700 1 -- 172.21.15.137:0/220252064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8d41085d0 msgr2=0x7ff8d41089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:31.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:31.155+0000 7ff8dad22700 1 --2- 172.21.15.137:0/220252064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8d41085d0 0x7ff8d41089b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8009910 tx=0x7ff8c8009c20 comp rx=0 tx=0).stop 2024-04-24T06:34:31.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:31.155+0000 7ff8dad22700 1 -- 172.21.15.137:0/220252064 shutdown_connections 2024-04-24T06:34:31.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:31.155+0000 7ff8dad22700 1 --2- 172.21.15.137:0/220252064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8d41085d0 0x7ff8d41089b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:31.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:31.155+0000 7ff8dad22700 1 -- 172.21.15.137:0/220252064 >> 172.21.15.137:0/220252064 conn(0x7ff8d40fe030 msgr2=0x7ff8d4100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:31.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:31.155+0000 7ff8dad22700 1 -- 172.21.15.137:0/220252064 shutdown_connections 2024-04-24T06:34:31.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:31.155+0000 7ff8dad22700 1 -- 172.21.15.137:0/220252064 wait complete. 2024-04-24T06:34:31.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:31.156+0000 7ff8dad22700 1 Processor -- start 2024-04-24T06:34:31.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:31.156+0000 7ff8dad22700 1 -- start start 2024-04-24T06:34:31.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:31.156+0000 7ff8dad22700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8d41085d0 0x7ff8d419a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:31.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:31.156+0000 7ff8dad22700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d410cb90 con 0x7ff8d41085d0 2024-04-24T06:34:31.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:31.157+0000 7ff8d8abe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8d41085d0 0x7ff8d419a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:31.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:31.157+0000 7ff8d8abe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8d41085d0 0x7ff8d419a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.137:54946/0 (socket says 172.21.15.137:54946) 2024-04-24T06:34:31.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:31.157+0000 7ff8d8abe700 1 -- 172.21.15.137:0/457128338 learned_addr learned my addr 172.21.15.137:0/457128338 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-04-24T06:34:31.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:31.157+0000 7ff8d8abe700 1 -- 172.21.15.137:0/457128338 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8d419a6c0 con 0x7ff8d41085d0 2024-04-24T06:34:31.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:31.158+0000 7ff8d8abe700 1 --2- 172.21.15.137:0/457128338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8d41085d0 0x7ff8d419a180 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8005850 tx=0x7ff8c8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:31.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:31.158+0000 7ff8d1ffb700 1 -- 172.21.15.137:0/457128338 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8c8026070 con 0x7ff8d41085d0 2024-04-24T06:34:31.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:31.158+0000 7ff8dad22700 1 -- 172.21.15.137:0/457128338 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8d419a950 con 0x7ff8d41085d0 2024-04-24T06:34:31.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:31.158+0000 7ff8dad22700 1 -- 172.21.15.137:0/457128338 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8d419ea50 con 0x7ff8d41085d0 2024-04-24T06:34:31.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:31.158+0000 7ff8d1ffb700 1 -- 172.21.15.137:0/457128338 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8c800bc30 con 0x7ff8d41085d0 2024-04-24T06:34:31.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:31.158+0000 7ff8d1ffb700 1 -- 172.21.15.137:0/457128338 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8c80039b0 con 0x7ff8d41085d0 2024-04-24T06:34:31.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:31.159+0000 7ff8dad22700 1 -- 172.21.15.137:0/457128338 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8b8005330 con 0x7ff8d41085d0 2024-04-24T06:34:31.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:31.160+0000 7ff8d1ffb700 1 -- 172.21.15.137:0/457128338 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7ff8c8029070 con 0x7ff8d41085d0 2024-04-24T06:34:31.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:31.160+0000 7ff8d1ffb700 1 --2- 172.21.15.137:0/457128338 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ff8c4039650 0x7ff8c403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:31.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:31.160+0000 7ff8d1ffb700 1 -- 172.21.15.137:0/457128338 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff8c801e070 con 0x7ff8d41085d0 2024-04-24T06:34:31.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:31.161+0000 7ff8d3fff700 1 --2- 172.21.15.137:0/457128338 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ff8c4039650 0x7ff8c403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:31.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:31.162+0000 7ff8d3fff700 1 --2- 172.21.15.137:0/457128338 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ff8c4039650 0x7ff8c403bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0006fd0 tx=0x7ff8c0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:31.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:31.176+0000 7ff8d1ffb700 1 -- 172.21.15.137:0/457128338 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff8c8013070 con 0x7ff8d41085d0 2024-04-24T06:34:31.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:31.383+0000 7ff8dad22700 1 -- 172.21.15.137:0/457128338 --> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi005:172.21.15.5=a;smithi037:172.21.15.37=b;smithi137:172.21.15.137=c", "target": ["mon-mgr", ""]}) -- 0x7ff8b8000c70 con 0x7ff8c4039650 2024-04-24T06:34:31.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:31.398+0000 7ff8d1ffb700 1 -- 172.21.15.137:0/457128338 <== mgr.14152 v2:172.21.15.5:6800/4042936965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff8b8000c70 con 0x7ff8c4039650 2024-04-24T06:34:31.399 INFO:teuthology.orchestra.run.smithi137.stdout:Scheduled mon update... 2024-04-24T06:34:31.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:31.400+0000 7ff8dad22700 1 -- 172.21.15.137:0/457128338 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ff8c4039650 msgr2=0x7ff8c403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:31.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:31.400+0000 7ff8dad22700 1 --2- 172.21.15.137:0/457128338 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ff8c4039650 0x7ff8c403bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0006fd0 tx=0x7ff8c0006e40 comp rx=0 tx=0).stop 2024-04-24T06:34:31.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:31.400+0000 7ff8dad22700 1 -- 172.21.15.137:0/457128338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8d41085d0 msgr2=0x7ff8d419a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:31.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:31.400+0000 7ff8dad22700 1 --2- 172.21.15.137:0/457128338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8d41085d0 0x7ff8d419a180 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8005850 tx=0x7ff8c8005030 comp rx=0 tx=0).stop 2024-04-24T06:34:31.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:31.400+0000 7ff8dad22700 1 -- 172.21.15.137:0/457128338 shutdown_connections 2024-04-24T06:34:31.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:31.400+0000 7ff8dad22700 1 --2- 172.21.15.137:0/457128338 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ff8c4039650 0x7ff8c403bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:31.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:31.400+0000 7ff8dad22700 1 --2- 172.21.15.137:0/457128338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8d41085d0 0x7ff8d419a180 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:31.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:31.400+0000 7ff8dad22700 1 -- 172.21.15.137:0/457128338 >> 172.21.15.137:0/457128338 conn(0x7ff8d40fe030 msgr2=0x7ff8d4076710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:31.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:31.400+0000 7ff8dad22700 1 -- 172.21.15.137:0/457128338 shutdown_connections 2024-04-24T06:34:31.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:31.400+0000 7ff8dad22700 1 -- 172.21.15.137:0/457128338 wait complete. 2024-04-24T06:34:32.145 DEBUG:teuthology.orchestra.run.smithi037:mon.b> sudo journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mon.b.service 2024-04-24T06:34:32.148 DEBUG:teuthology.orchestra.run.smithi137:mon.c> sudo journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mon.c.service 2024-04-24T06:34:32.150 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-24T06:34:32.150 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T06:34:32.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:32 smithi005 ceph-mon[27798]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:32.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:32 smithi005 ceph-mon[27798]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi005:172.21.15.5=a;smithi037:172.21.15.37=b;smithi137:172.21.15.137=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:34:32.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:32 smithi005 ceph-mon[27798]: Saving service mon spec with placement smithi005:172.21.15.5=a;smithi037:172.21.15.37=b;smithi137:172.21.15.137=c;count:3 2024-04-24T06:34:32.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:32 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:32.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:32 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:34:32.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:32 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:34:32.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:32 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:34:32.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:32 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:32.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:32 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T06:34:32.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:32 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:34:32.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:32 smithi005 ceph-mon[27798]: Deploying daemon mon.c on smithi137 2024-04-24T06:34:33.636 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T06:34:34.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:34 smithi005 ceph-mon[27798]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:35.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:35.751+0000 7f4b0d667700 1 -- 172.21.15.137:0/2700556244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b081085d0 msgr2=0x7f4b081089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:35.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:35.751+0000 7f4b0d667700 1 --2- 172.21.15.137:0/2700556244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b081085d0 0x7f4b081089b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4af8009910 tx=0x7f4af8009c20 comp rx=0 tx=0).stop 2024-04-24T06:34:35.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:35.751+0000 7f4b0d667700 1 -- 172.21.15.137:0/2700556244 shutdown_connections 2024-04-24T06:34:35.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:35.751+0000 7f4b0d667700 1 --2- 172.21.15.137:0/2700556244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b081085d0 0x7f4b081089b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:35.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:35.751+0000 7f4b0d667700 1 -- 172.21.15.137:0/2700556244 >> 172.21.15.137:0/2700556244 conn(0x7f4b080fe030 msgr2=0x7f4b08100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:35.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:35.751+0000 7f4b0d667700 1 -- 172.21.15.137:0/2700556244 shutdown_connections 2024-04-24T06:34:35.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:35.751+0000 7f4b0d667700 1 -- 172.21.15.137:0/2700556244 wait complete. 2024-04-24T06:34:35.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:35.751+0000 7f4b0d667700 1 Processor -- start 2024-04-24T06:34:35.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:35.751+0000 7f4b0d667700 1 -- start start 2024-04-24T06:34:35.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:35.752+0000 7f4b0d667700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b081085d0 0x7f4b08197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:35.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:35.752+0000 7f4b0d667700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0810cb90 con 0x7f4b081085d0 2024-04-24T06:34:35.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:35.752+0000 7f4b06ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b081085d0 0x7f4b08197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:35.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:35.752+0000 7f4b06ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b081085d0 0x7f4b08197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.137:54958/0 (socket says 172.21.15.137:54958) 2024-04-24T06:34:35.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:35.752+0000 7f4b06ffd700 1 -- 172.21.15.137:0/3835759932 learned_addr learned my addr 172.21.15.137:0/3835759932 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-04-24T06:34:35.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:35.753+0000 7f4b06ffd700 1 -- 172.21.15.137:0/3835759932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b08198510 con 0x7f4b081085d0 2024-04-24T06:34:35.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:35.753+0000 7f4b06ffd700 1 --2- 172.21.15.137:0/3835759932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b081085d0 0x7f4b08197fd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4af8006010 tx=0x7f4af8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:35.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:35.754+0000 7f4b04ff9700 1 -- 172.21.15.137:0/3835759932 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4af8026070 con 0x7f4b081085d0 2024-04-24T06:34:35.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:35.754+0000 7f4b0d667700 1 -- 172.21.15.137:0/3835759932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b081987a0 con 0x7f4b081085d0 2024-04-24T06:34:35.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:35.754+0000 7f4b0d667700 1 -- 172.21.15.137:0/3835759932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b0819c8a0 con 0x7f4b081085d0 2024-04-24T06:34:35.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:35.754+0000 7f4b04ff9700 1 -- 172.21.15.137:0/3835759932 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4af800bc30 con 0x7f4b081085d0 2024-04-24T06:34:35.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:35.754+0000 7f4b04ff9700 1 -- 172.21.15.137:0/3835759932 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4af80047e0 con 0x7f4b081085d0 2024-04-24T06:34:35.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:35.754+0000 7f4b0d667700 1 -- 172.21.15.137:0/3835759932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ae8005330 con 0x7f4b081085d0 2024-04-24T06:34:35.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:35.755+0000 7f4b04ff9700 1 -- 172.21.15.137:0/3835759932 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7f4af8029070 con 0x7f4b081085d0 2024-04-24T06:34:35.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:35.755+0000 7f4b04ff9700 1 --2- 172.21.15.137:0/3835759932 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f4af4039650 0x7f4af403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:35.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:35.755+0000 7f4b04ff9700 1 -- 172.21.15.137:0/3835759932 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4af801e070 con 0x7f4b081085d0 2024-04-24T06:34:35.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:35.756+0000 7f4afe7fc700 1 --2- 172.21.15.137:0/3835759932 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f4af4039650 0x7f4af403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:35.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:35.757+0000 7f4afe7fc700 1 --2- 172.21.15.137:0/3835759932 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f4af4039650 0x7f4af403bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4af0006fd0 tx=0x7f4af0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:35.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:35.771+0000 7f4b04ff9700 1 -- 172.21.15.137:0/3835759932 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4af8013070 con 0x7f4b081085d0 2024-04-24T06:34:36.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:36.060+0000 7f4b0d667700 1 -- 172.21.15.137:0/3835759932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f4ae80058f0 con 0x7f4b081085d0 2024-04-24T06:34:36.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:36.061+0000 7f4b04ff9700 1 -- 172.21.15.137:0/3835759932 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+741 (secure 0 0 0) 0x7f4af800bde0 con 0x7f4b081085d0 2024-04-24T06:34:36.062 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-24T06:34:36.062 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":1,"fsid":"43e9bad0-0204-11ef-bc93-c7b262605968","modified":"2024-04-24T06:32:22.137390Z","created":"2024-04-24T06:32:22.137390Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:3300","nonce":0},{"type":"v1","addr":"172.21.15.5:6789","nonce":0}]},"addr":"172.21.15.5:6789/0","public_addr":"172.21.15.5:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T06:34:36.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:36.063+0000 7f4b0d667700 1 -- 172.21.15.137:0/3835759932 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f4af4039650 msgr2=0x7f4af403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:36.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:36.063+0000 7f4b0d667700 1 --2- 172.21.15.137:0/3835759932 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f4af4039650 0x7f4af403bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4af0006fd0 tx=0x7f4af0006e40 comp rx=0 tx=0).stop 2024-04-24T06:34:36.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:36.063+0000 7f4b0d667700 1 -- 172.21.15.137:0/3835759932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b081085d0 msgr2=0x7f4b08197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:36.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:36.063+0000 7f4b0d667700 1 --2- 172.21.15.137:0/3835759932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b081085d0 0x7f4b08197fd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4af8006010 tx=0x7f4af8005030 comp rx=0 tx=0).stop 2024-04-24T06:34:36.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:36.063+0000 7f4b0d667700 1 -- 172.21.15.137:0/3835759932 shutdown_connections 2024-04-24T06:34:36.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:36.063+0000 7f4b0d667700 1 --2- 172.21.15.137:0/3835759932 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f4af4039650 0x7f4af403bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:36.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:36.063+0000 7f4b0d667700 1 --2- 172.21.15.137:0/3835759932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b081085d0 0x7f4b08197fd0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:36.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:36.063+0000 7f4b0d667700 1 -- 172.21.15.137:0/3835759932 >> 172.21.15.137:0/3835759932 conn(0x7f4b080fe030 msgr2=0x7f4b080ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:36.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:36.063+0000 7f4b0d667700 1 -- 172.21.15.137:0/3835759932 shutdown_connections 2024-04-24T06:34:36.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:36.063+0000 7f4b0d667700 1 -- 172.21.15.137:0/3835759932 wait complete. 2024-04-24T06:34:36.065 INFO:teuthology.orchestra.run.smithi137.stderr:dumped monmap epoch 1 2024-04-24T06:34:36.150 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: ceph version 19.0.0-3124-ga3c8a22b (a3c8a22bd5efe18ab230a8e2d072db8f381a0411) squid (dev), process ceph-mon, pid 2 2024-04-24T06:34:36.150 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: pidfile_write: ignore empty --pid-file 2024-04-24T06:34:36.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: load: jerasure load: lrc 2024-04-24T06:34:36.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: RocksDB version: 7.9.2 2024-04-24T06:34:36.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Git sha 0 2024-04-24T06:34:36.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Compile date 2024-04-23 20:58:29 2024-04-24T06:34:36.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: DB SUMMARY 2024-04-24T06:34:36.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: DB Session ID: F2CSL9DS1YOCYN7AMO9E 2024-04-24T06:34:36.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: CURRENT file: CURRENT 2024-04-24T06:34:36.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: IDENTITY file: IDENTITY 2024-04-24T06:34:36.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-24T06:34:36.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-24T06:34:36.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-24T06:34:36.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.error_if_exists: 0 2024-04-24T06:34:36.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.create_if_missing: 0 2024-04-24T06:34:36.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.paranoid_checks: 1 2024-04-24T06:34:36.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-24T06:34:36.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-24T06:34:36.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-24T06:34:36.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.env: 0x56307dee4cc0 2024-04-24T06:34:36.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.fs: PosixFileSystem 2024-04-24T06:34:36.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.info_log: 0x563080399c20 2024-04-24T06:34:36.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.max_file_opening_threads: 16 2024-04-24T06:34:36.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.statistics: (nil) 2024-04-24T06:34:36.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.use_fsync: 0 2024-04-24T06:34:36.449 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.max_log_file_size: 0 2024-04-24T06:34:36.449 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-24T06:34:36.449 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-24T06:34:36.449 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.keep_log_file_num: 1000 2024-04-24T06:34:36.449 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.recycle_log_file_num: 0 2024-04-24T06:34:36.449 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.allow_fallocate: 1 2024-04-24T06:34:36.449 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.allow_mmap_reads: 0 2024-04-24T06:34:36.449 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.allow_mmap_writes: 0 2024-04-24T06:34:36.449 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.use_direct_reads: 0 2024-04-24T06:34:36.449 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-24T06:34:36.449 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.create_missing_column_families: 0 2024-04-24T06:34:36.450 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.db_log_dir: 2024-04-24T06:34:36.450 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.wal_dir: 2024-04-24T06:34:36.450 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-24T06:34:36.450 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-24T06:34:36.450 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-24T06:34:36.450 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-24T06:34:36.450 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-24T06:34:36.450 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-24T06:34:36.450 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.advise_random_on_open: 1 2024-04-24T06:34:36.450 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.db_write_buffer_size: 0 2024-04-24T06:34:36.450 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.write_buffer_manager: 0x5630803681e0 2024-04-24T06:34:36.450 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-24T06:34:36.451 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-24T06:34:36.451 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-24T06:34:36.451 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.rate_limiter: (nil) 2024-04-24T06:34:36.451 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-24T06:34:36.451 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.wal_recovery_mode: 2 2024-04-24T06:34:36.451 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.enable_thread_tracking: 0 2024-04-24T06:34:36.451 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.enable_pipelined_write: 0 2024-04-24T06:34:36.451 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.unordered_write: 0 2024-04-24T06:34:36.451 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-24T06:34:36.451 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-24T06:34:36.451 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-24T06:34:36.451 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-24T06:34:36.451 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.row_cache: None 2024-04-24T06:34:36.451 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.wal_filter: None 2024-04-24T06:34:36.452 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-24T06:34:36.452 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.allow_ingest_behind: 0 2024-04-24T06:34:36.452 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.two_write_queues: 0 2024-04-24T06:34:36.452 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.manual_wal_flush: 0 2024-04-24T06:34:36.452 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.wal_compression: 0 2024-04-24T06:34:36.452 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.atomic_flush: 0 2024-04-24T06:34:36.452 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-24T06:34:36.452 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-24T06:34:36.452 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-24T06:34:36.452 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.log_readahead_size: 0 2024-04-24T06:34:36.452 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-24T06:34:36.452 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.best_efforts_recovery: 0 2024-04-24T06:34:36.452 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-24T06:34:36.452 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-24T06:34:36.452 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.allow_data_in_errors: 0 2024-04-24T06:34:36.453 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.db_host_id: __hostname__ 2024-04-24T06:34:36.453 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-24T06:34:36.453 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.max_background_jobs: 2 2024-04-24T06:34:36.453 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.max_background_compactions: -1 2024-04-24T06:34:36.453 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.max_subcompactions: 1 2024-04-24T06:34:36.454 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-24T06:34:36.455 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-24T06:34:36.455 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-24T06:34:36.455 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.max_total_wal_size: 0 2024-04-24T06:34:36.455 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-24T06:34:36.455 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-24T06:34:36.455 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-24T06:34:36.455 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-24T06:34:36.455 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.max_open_files: -1 2024-04-24T06:34:36.455 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.bytes_per_sync: 0 2024-04-24T06:34:36.455 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-24T06:34:36.455 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-24T06:34:36.455 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.compaction_readahead_size: 0 2024-04-24T06:34:36.455 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.max_background_flushes: -1 2024-04-24T06:34:36.456 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Compression algorithms supported: 2024-04-24T06:34:36.456 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-24T06:34:36.456 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: kZSTD supported: 0 2024-04-24T06:34:36.456 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: kXpressCompression supported: 0 2024-04-24T06:34:36.456 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: kLZ4HCCompression supported: 1 2024-04-24T06:34:36.456 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: kZlibCompression supported: 1 2024-04-24T06:34:36.456 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: kSnappyCompression supported: 1 2024-04-24T06:34:36.456 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: kLZ4Compression supported: 1 2024-04-24T06:34:36.456 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: kBZip2Compression supported: 0 2024-04-24T06:34:36.456 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-24T06:34:36.456 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-24T06:34:36.456 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-24T06:34:36.456 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-24T06:34:36.456 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-24T06:34:36.457 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.merge_operator: 2024-04-24T06:34:36.457 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.compaction_filter: None 2024-04-24T06:34:36.457 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.compaction_filter_factory: None 2024-04-24T06:34:36.457 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.sst_partitioner_factory: None 2024-04-24T06:34:36.457 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-24T06:34:36.457 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-24T06:34:36.457 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x563080399de0) 2024-04-24T06:34:36.457 INFO:journalctl@ceph.mon.c.smithi137.stdout: cache_index_and_filter_blocks: 1 2024-04-24T06:34:36.457 INFO:journalctl@ceph.mon.c.smithi137.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-24T06:34:36.457 INFO:journalctl@ceph.mon.c.smithi137.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-24T06:34:36.457 INFO:journalctl@ceph.mon.c.smithi137.stdout: pin_top_level_index_and_filter: 1 2024-04-24T06:34:36.457 INFO:journalctl@ceph.mon.c.smithi137.stdout: index_type: 0 2024-04-24T06:34:36.457 INFO:journalctl@ceph.mon.c.smithi137.stdout: data_block_index_type: 0 2024-04-24T06:34:36.458 INFO:journalctl@ceph.mon.c.smithi137.stdout: index_shortening: 1 2024-04-24T06:34:36.458 INFO:journalctl@ceph.mon.c.smithi137.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-24T06:34:36.458 INFO:journalctl@ceph.mon.c.smithi137.stdout: checksum: 4 2024-04-24T06:34:36.458 INFO:journalctl@ceph.mon.c.smithi137.stdout: no_block_cache: 0 2024-04-24T06:34:36.458 INFO:journalctl@ceph.mon.c.smithi137.stdout: block_cache: 0x56308039b090 2024-04-24T06:34:36.458 INFO:journalctl@ceph.mon.c.smithi137.stdout: block_cache_name: BinnedLRUCache 2024-04-24T06:34:36.458 INFO:journalctl@ceph.mon.c.smithi137.stdout: block_cache_options: 2024-04-24T06:34:36.458 INFO:journalctl@ceph.mon.c.smithi137.stdout: capacity : 536870912 2024-04-24T06:34:36.458 INFO:journalctl@ceph.mon.c.smithi137.stdout: num_shard_bits : 4 2024-04-24T06:34:36.458 INFO:journalctl@ceph.mon.c.smithi137.stdout: strict_capacity_limit : 0 2024-04-24T06:34:36.458 INFO:journalctl@ceph.mon.c.smithi137.stdout: high_pri_pool_ratio: 0.000 2024-04-24T06:34:36.458 INFO:journalctl@ceph.mon.c.smithi137.stdout: block_cache_compressed: (nil) 2024-04-24T06:34:36.458 INFO:journalctl@ceph.mon.c.smithi137.stdout: persistent_cache: (nil) 2024-04-24T06:34:36.459 INFO:journalctl@ceph.mon.c.smithi137.stdout: block_size: 4096 2024-04-24T06:34:36.459 INFO:journalctl@ceph.mon.c.smithi137.stdout: block_size_deviation: 10 2024-04-24T06:34:36.459 INFO:journalctl@ceph.mon.c.smithi137.stdout: block_restart_interval: 16 2024-04-24T06:34:36.459 INFO:journalctl@ceph.mon.c.smithi137.stdout: index_block_restart_interval: 1 2024-04-24T06:34:36.459 INFO:journalctl@ceph.mon.c.smithi137.stdout: metadata_block_size: 4096 2024-04-24T06:34:36.459 INFO:journalctl@ceph.mon.c.smithi137.stdout: partition_filters: 0 2024-04-24T06:34:36.459 INFO:journalctl@ceph.mon.c.smithi137.stdout: use_delta_encoding: 1 2024-04-24T06:34:36.459 INFO:journalctl@ceph.mon.c.smithi137.stdout: filter_policy: bloomfilter 2024-04-24T06:34:36.459 INFO:journalctl@ceph.mon.c.smithi137.stdout: whole_key_filtering: 1 2024-04-24T06:34:36.459 INFO:journalctl@ceph.mon.c.smithi137.stdout: verify_compression: 0 2024-04-24T06:34:36.459 INFO:journalctl@ceph.mon.c.smithi137.stdout: read_amp_bytes_per_bit: 0 2024-04-24T06:34:36.459 INFO:journalctl@ceph.mon.c.smithi137.stdout: format_version: 5 2024-04-24T06:34:36.460 INFO:journalctl@ceph.mon.c.smithi137.stdout: enable_index_compression: 1 2024-04-24T06:34:36.460 INFO:journalctl@ceph.mon.c.smithi137.stdout: block_align: 0 2024-04-24T06:34:36.460 INFO:journalctl@ceph.mon.c.smithi137.stdout: max_auto_readahead_size: 262144 2024-04-24T06:34:36.460 INFO:journalctl@ceph.mon.c.smithi137.stdout: prepopulate_block_cache: 0 2024-04-24T06:34:36.460 INFO:journalctl@ceph.mon.c.smithi137.stdout: initial_auto_readahead_size: 8192 2024-04-24T06:34:36.460 INFO:journalctl@ceph.mon.c.smithi137.stdout: num_file_reads_for_auto_readahead: 2 2024-04-24T06:34:36.460 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.write_buffer_size: 33554432 2024-04-24T06:34:36.460 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.max_write_buffer_number: 2 2024-04-24T06:34:36.460 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.compression: NoCompression 2024-04-24T06:34:36.460 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.bottommost_compression: Disabled 2024-04-24T06:34:36.460 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.prefix_extractor: nullptr 2024-04-24T06:34:36.460 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-24T06:34:36.460 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.num_levels: 7 2024-04-24T06:34:36.460 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-24T06:34:36.461 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-24T06:34:36.461 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-24T06:34:36.461 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-24T06:34:36.461 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-24T06:34:36.461 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-24T06:34:36.461 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-24T06:34:36.461 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-24T06:34:36.461 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-24T06:34:36.461 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-24T06:34:36.461 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-24T06:34:36.461 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-24T06:34:36.461 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-24T06:34:36.461 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.compression_opts.level: 32767 2024-04-24T06:34:36.461 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.compression_opts.strategy: 0 2024-04-24T06:34:36.462 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-24T06:34:36.462 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-24T06:34:36.462 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-24T06:34:36.462 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-24T06:34:36.462 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.compression_opts.enabled: false 2024-04-24T06:34:36.462 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-24T06:34:36.462 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-24T06:34:36.462 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-24T06:34:36.462 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-24T06:34:36.462 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.target_file_size_base: 67108864 2024-04-24T06:34:36.462 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-24T06:34:36.462 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-24T06:34:36.462 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-24T06:34:36.462 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-24T06:34:36.463 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-24T06:34:36.463 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-24T06:34:36.463 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-24T06:34:36.463 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-24T06:34:36.463 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-24T06:34:36.463 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-24T06:34:36.463 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-24T06:34:36.463 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-24T06:34:36.463 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-24T06:34:36.463 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-24T06:34:36.463 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.arena_block_size: 1048576 2024-04-24T06:34:36.463 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-24T06:34:36.464 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-24T06:34:36.464 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.disable_auto_compactions: 0 2024-04-24T06:34:36.464 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-24T06:34:36.464 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-24T06:34:36.464 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-24T06:34:36.464 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-24T06:34:36.464 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-24T06:34:36.464 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-24T06:34:36.464 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-24T06:34:36.464 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-24T06:34:36.464 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-24T06:34:36.465 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-24T06:34:36.465 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-24T06:34:36.465 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.inplace_update_support: 0 2024-04-24T06:34:36.465 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-24T06:34:36.465 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-24T06:34:36.465 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-24T06:34:36.465 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-24T06:34:36.465 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.bloom_locality: 0 2024-04-24T06:34:36.465 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.max_successive_merges: 0 2024-04-24T06:34:36.465 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-24T06:34:36.465 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.paranoid_file_checks: 0 2024-04-24T06:34:36.465 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.force_consistency_checks: 1 2024-04-24T06:34:36.465 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.report_bg_io_stats: 0 2024-04-24T06:34:36.465 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.ttl: 2592000 2024-04-24T06:34:36.466 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-24T06:34:36.466 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-24T06:34:36.466 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-24T06:34:36.466 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.enable_blob_files: false 2024-04-24T06:34:36.466 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.min_blob_size: 0 2024-04-24T06:34:36.466 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.blob_file_size: 268435456 2024-04-24T06:34:36.466 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-24T06:34:36.466 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-24T06:34:36.466 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-24T06:34:36.466 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-24T06:34:36.466 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-24T06:34:36.466 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.blob_file_starting_level: 0 2024-04-24T06:34:36.466 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-24T06:34:36.466 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-24T06:34:36.467 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-24T06:34:36.467 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 9abeaba3-ea33-4bec-b6af-bda221fa39f6 2024-04-24T06:34:36.467 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713940476165902, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-24T06:34:36.467 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-24T06:34:36.467 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713940476166308, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713940476, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "9abeaba3-ea33-4bec-b6af-bda221fa39f6", "db_session_id": "F2CSL9DS1YOCYN7AMO9E", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-24T06:34:36.467 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713940476166388, "job": 1, "event": "recovery_finished"} 2024-04-24T06:34:36.467 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-24T06:34:36.467 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-24T06:34:36.467 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x563080488000 2024-04-24T06:34:36.467 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: DB pointer 0x563080472000 2024-04-24T06:34:36.467 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-24T06:34:36.467 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-24T06:34:36.467 INFO:journalctl@ceph.mon.c.smithi137.stdout: ** DB Stats ** 2024-04-24T06:34:36.468 INFO:journalctl@ceph.mon.c.smithi137.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T06:34:36.468 INFO:journalctl@ceph.mon.c.smithi137.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-24T06:34:36.468 INFO:journalctl@ceph.mon.c.smithi137.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T06:34:36.468 INFO:journalctl@ceph.mon.c.smithi137.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T06:34:36.468 INFO:journalctl@ceph.mon.c.smithi137.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-24T06:34:36.468 INFO:journalctl@ceph.mon.c.smithi137.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T06:34:36.468 INFO:journalctl@ceph.mon.c.smithi137.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T06:34:36.468 INFO:journalctl@ceph.mon.c.smithi137.stdout: 2024-04-24T06:34:36.468 INFO:journalctl@ceph.mon.c.smithi137.stdout: ** Compaction Stats [default] ** 2024-04-24T06:34:36.468 INFO:journalctl@ceph.mon.c.smithi137.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T06:34:36.468 INFO:journalctl@ceph.mon.c.smithi137.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-24T06:34:36.468 INFO:journalctl@ceph.mon.c.smithi137.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-24T06:34:36.468 INFO:journalctl@ceph.mon.c.smithi137.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-24T06:34:36.468 INFO:journalctl@ceph.mon.c.smithi137.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-24T06:34:36.468 INFO:journalctl@ceph.mon.c.smithi137.stdout: 2024-04-24T06:34:36.469 INFO:journalctl@ceph.mon.c.smithi137.stdout: ** Compaction Stats [default] ** 2024-04-24T06:34:36.469 INFO:journalctl@ceph.mon.c.smithi137.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T06:34:36.469 INFO:journalctl@ceph.mon.c.smithi137.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T06:34:36.469 INFO:journalctl@ceph.mon.c.smithi137.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-24T06:34:36.469 INFO:journalctl@ceph.mon.c.smithi137.stdout: 2024-04-24T06:34:36.469 INFO:journalctl@ceph.mon.c.smithi137.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-24T06:34:36.469 INFO:journalctl@ceph.mon.c.smithi137.stdout: 2024-04-24T06:34:36.470 INFO:journalctl@ceph.mon.c.smithi137.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T06:34:36.470 INFO:journalctl@ceph.mon.c.smithi137.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-24T06:34:36.470 INFO:journalctl@ceph.mon.c.smithi137.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-24T06:34:36.470 INFO:journalctl@ceph.mon.c.smithi137.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-24T06:34:36.470 INFO:journalctl@ceph.mon.c.smithi137.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-24T06:34:36.470 INFO:journalctl@ceph.mon.c.smithi137.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-24T06:34:36.471 INFO:journalctl@ceph.mon.c.smithi137.stdout: Cumulative compaction: 0.00 GB write, 0.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T06:34:36.471 INFO:journalctl@ceph.mon.c.smithi137.stdout: Interval compaction: 0.00 GB write, 0.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T06:34:36.471 INFO:journalctl@ceph.mon.c.smithi137.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-24T06:34:36.471 INFO:journalctl@ceph.mon.c.smithi137.stdout: Block cache BinnedLRUCache@0x56308039b090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-04-24T06:34:36.471 INFO:journalctl@ceph.mon.c.smithi137.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-24T06:34:36.471 INFO:journalctl@ceph.mon.c.smithi137.stdout: 2024-04-24T06:34:36.471 INFO:journalctl@ceph.mon.c.smithi137.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-24T06:34:36.471 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-24T06:34:36.471 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: using public_addr v2:172.21.15.137:0/0 -> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] 2024-04-24T06:34:36.471 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: starting mon.c rank -1 at public addrs [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] at bind addrs [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 43e9bad0-0204-11ef-bc93-c7b262605968 2024-04-24T06:34:36.471 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: mon.c@-1(???) e0 preinit fsid 43e9bad0-0204-11ef-bc93-c7b262605968 2024-04-24T06:34:36.471 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: mon.c@-1(synchronizing).mds e1 new map 2024-04-24T06:34:36.471 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-24T06:34:36.471 INFO:journalctl@ceph.mon.c.smithi137.stdout: e1 2024-04-24T06:34:36.472 INFO:journalctl@ceph.mon.c.smithi137.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T06:34:36.472 INFO:journalctl@ceph.mon.c.smithi137.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T06:34:36.472 INFO:journalctl@ceph.mon.c.smithi137.stdout: legacy client fscid: -1 2024-04-24T06:34:36.472 INFO:journalctl@ceph.mon.c.smithi137.stdout: 2024-04-24T06:34:36.472 INFO:journalctl@ceph.mon.c.smithi137.stdout: No filesystems configured 2024-04-24T06:34:36.472 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-24T06:34:36.472 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-24T06:34:36.472 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-24T06:34:36.472 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-24T06:34:36.472 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-24T06:34:36.472 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-24T06:34:36.472 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-24T06:34:36.472 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T06:34:36.472 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T06:34:36.473 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T06:34:36.473 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:36.473 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:36.473 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:36.473 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: Added host smithi037 2024-04-24T06:34:36.473 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:34:36.473 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:36.473 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:36.473 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:36.473 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:36.473 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:36.473 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T06:34:36.473 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:36.473 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:36.474 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:36.474 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:36.474 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:36.474 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi037", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:34:36.474 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:34:36.474 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:34:36.474 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-24T06:34:36.474 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:36.474 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: Updating smithi037:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T06:34:36.474 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-04-24T06:34:36.474 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: Updating smithi037:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T06:34:36.474 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi137", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:34:36.474 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:36.474 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:36.475 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:36.475 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: Deploying cephadm binary to smithi137 2024-04-24T06:34:36.475 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:36.475 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:36.475 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:36.475 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: Added host smithi137 2024-04-24T06:34:36.475 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:34:36.475 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:36.475 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:36.475 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:36.475 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:36.475 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:36.475 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T06:34:36.475 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:36.475 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:36.476 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:36.476 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:36.476 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:36.476 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi137", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:34:36.476 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:34:36.476 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:34:36.476 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: Updating smithi137:/etc/ceph/ceph.conf 2024-04-24T06:34:36.476 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: Updating smithi137:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T06:34:36.476 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:36.476 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-04-24T06:34:36.476 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: Updating smithi137:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T06:34:36.476 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/1385169739' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-24T06:34:36.476 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:36.476 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:36.476 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:36.477 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/1385169739' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-24T06:34:36.477 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T06:34:36.477 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:36.477 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:36.477 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi005:172.21.15.5=a;smithi037:172.21.15.37=b;smithi137:172.21.15.137=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:34:36.477 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: Saving service mon spec with placement smithi005:172.21.15.5=a;smithi037:172.21.15.37=b;smithi137:172.21.15.137=c;count:3 2024-04-24T06:34:36.477 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:36.477 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:34:36.477 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:34:36.477 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:34:36.477 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:36.477 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T06:34:36.477 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:34:36.477 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: Deploying daemon mon.c on smithi137 2024-04-24T06:34:36.477 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:36.478 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 ceph-mon[33017]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-24T06:34:36.478 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 bash[32985]: 15c232db705d9b32bb64c22cae9e32439bbd1e994337c7498b9c14abbf619bd5 2024-04-24T06:34:36.478 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:36 smithi137 systemd[1]: Started Ceph mon.c for 43e9bad0-0204-11ef-bc93-c7b262605968. 2024-04-24T06:34:37.796 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-24T06:34:37.797 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T06:34:38.043 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T06:34:41.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:41 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T06:34:41.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:41 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T06:34:41.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:41 smithi137 ceph-mon[33017]: mon.a calling monitor election 2024-04-24T06:34:41.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:41 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T06:34:41.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:41 smithi137 ceph-mon[33017]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:41.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:41 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T06:34:41.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:41 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T06:34:41.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:41 smithi137 ceph-mon[33017]: mon.c calling monitor election 2024-04-24T06:34:41.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:41 smithi137 ceph-mon[33017]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:41.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:41 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T06:34:41.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:41 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T06:34:41.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:41 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T06:34:41.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:41 smithi137 ceph-mon[33017]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T06:34:41.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:41 smithi137 ceph-mon[33017]: monmap epoch 2 2024-04-24T06:34:41.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:41 smithi137 ceph-mon[33017]: fsid 43e9bad0-0204-11ef-bc93-c7b262605968 2024-04-24T06:34:41.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:41 smithi137 ceph-mon[33017]: last_changed 2024-04-24T06:34:36.198113+0000 2024-04-24T06:34:41.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:41 smithi137 ceph-mon[33017]: created 2024-04-24T06:32:22.137390+0000 2024-04-24T06:34:41.698 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:41 smithi137 ceph-mon[33017]: min_mon_release 19 (squid) 2024-04-24T06:34:41.698 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:41 smithi137 ceph-mon[33017]: election_strategy: 1 2024-04-24T06:34:41.698 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:41 smithi137 ceph-mon[33017]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-04-24T06:34:41.698 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:41 smithi137 ceph-mon[33017]: 1: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.c 2024-04-24T06:34:41.698 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:41 smithi137 ceph-mon[33017]: fsmap 2024-04-24T06:34:41.698 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:41 smithi137 ceph-mon[33017]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T06:34:41.698 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:41 smithi137 ceph-mon[33017]: mgrmap e13: x(active, since 60s) 2024-04-24T06:34:41.698 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:41 smithi137 ceph-mon[33017]: overall HEALTH_OK 2024-04-24T06:34:41.698 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:41 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:41.698 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:41 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:41.698 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:41 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:41.698 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:41 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T06:34:41.698 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:41 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:34:41.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:41 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T06:34:41.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:41 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T06:34:41.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:41 smithi005 ceph-mon[27798]: mon.a calling monitor election 2024-04-24T06:34:41.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:41 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T06:34:41.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:41 smithi005 ceph-mon[27798]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:41.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:41 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T06:34:41.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:41 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T06:34:41.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:41 smithi005 ceph-mon[27798]: mon.c calling monitor election 2024-04-24T06:34:41.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:41 smithi005 ceph-mon[27798]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:41.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:41 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T06:34:41.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:41 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T06:34:41.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:41 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T06:34:41.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:41 smithi005 ceph-mon[27798]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T06:34:41.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:41 smithi005 ceph-mon[27798]: monmap epoch 2 2024-04-24T06:34:41.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:41 smithi005 ceph-mon[27798]: fsid 43e9bad0-0204-11ef-bc93-c7b262605968 2024-04-24T06:34:41.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:41 smithi005 ceph-mon[27798]: last_changed 2024-04-24T06:34:36.198113+0000 2024-04-24T06:34:41.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:41 smithi005 ceph-mon[27798]: created 2024-04-24T06:32:22.137390+0000 2024-04-24T06:34:41.760 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:41 smithi005 ceph-mon[27798]: min_mon_release 19 (squid) 2024-04-24T06:34:41.760 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:41 smithi005 ceph-mon[27798]: election_strategy: 1 2024-04-24T06:34:41.760 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:41 smithi005 ceph-mon[27798]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-04-24T06:34:41.760 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:41 smithi005 ceph-mon[27798]: 1: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.c 2024-04-24T06:34:41.760 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:41 smithi005 ceph-mon[27798]: fsmap 2024-04-24T06:34:41.760 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:41 smithi005 ceph-mon[27798]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T06:34:41.760 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:41 smithi005 ceph-mon[27798]: mgrmap e13: x(active, since 60s) 2024-04-24T06:34:41.760 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:41 smithi005 ceph-mon[27798]: overall HEALTH_OK 2024-04-24T06:34:41.760 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:41 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:41.760 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:41 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:41.760 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:41 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:41.760 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:41 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T06:34:41.760 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:41 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:34:42.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.370+0000 7f43463c2700 1 -- 172.21.15.137:0/4228746231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43401015a0 msgr2=0x7f4328005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:42.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.370+0000 7f43463c2700 1 --2- 172.21.15.137:0/4228746231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43401015a0 0x7f4328005610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4330005fa0 tx=0x7f433000ffc0 comp rx=0 tx=0).stop 2024-04-24T06:34:42.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.370+0000 7f43463c2700 1 -- 172.21.15.137:0/4228746231 shutdown_connections 2024-04-24T06:34:42.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.370+0000 7f43463c2700 1 --2- 172.21.15.137:0/4228746231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43401015a0 0x7f4328005610 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:42.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.370+0000 7f43463c2700 1 -- 172.21.15.137:0/4228746231 >> 172.21.15.137:0/4228746231 conn(0x7f43400fd000 msgr2=0x7f43400ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:42.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.370+0000 7f43463c2700 1 -- 172.21.15.137:0/4228746231 shutdown_connections 2024-04-24T06:34:42.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.370+0000 7f43463c2700 1 -- 172.21.15.137:0/4228746231 wait complete. 2024-04-24T06:34:42.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.371+0000 7f43463c2700 1 Processor -- start 2024-04-24T06:34:42.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.372+0000 7f43463c2700 1 -- start start 2024-04-24T06:34:42.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.372+0000 7f43463c2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43401015a0 0x7f434019e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:42.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.372+0000 7f43463c2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f434019ebf0 0x7f43401a2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:42.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.372+0000 7f43463c2700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4340110ca0 con 0x7f43401015a0 2024-04-24T06:34:42.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.372+0000 7f43463c2700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4340110b20 con 0x7f434019ebf0 2024-04-24T06:34:42.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.373+0000 7f433f7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f434019ebf0 0x7f43401a2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:42.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.373+0000 7f433f7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f434019ebf0 0x7f43401a2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49276/0 (socket says 172.21.15.137:49276) 2024-04-24T06:34:42.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.373+0000 7f433f7fe700 1 -- 172.21.15.137:0/2718933891 learned_addr learned my addr 172.21.15.137:0/2718933891 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-04-24T06:34:42.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.373+0000 7f433f7fe700 1 -- 172.21.15.137:0/2718933891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f434019ebf0 msgr2=0x7f43401a2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-24T06:34:42.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.373+0000 7f433f7fe700 1 -- 172.21.15.137:0/2718933891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f434019ebf0 msgr2=0x7f43401a2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T06:34:42.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.373+0000 7f433f7fe700 1 --2- 172.21.15.137:0/2718933891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f434019ebf0 0x7f43401a2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T06:34:42.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.373+0000 7f433f7fe700 1 --2- 172.21.15.137:0/2718933891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f434019ebf0 0x7f43401a2f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T06:34:42.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.373+0000 7f433ffff700 1 --2- 172.21.15.137:0/2718933891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43401015a0 0x7f434019e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:42.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.374+0000 7f433ffff700 1 -- 172.21.15.137:0/2718933891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f434019ebf0 msgr2=0x7f43401a2f80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T06:34:42.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.374+0000 7f433ffff700 1 --2- 172.21.15.137:0/2718933891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f434019ebf0 0x7f43401a2f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:42.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.374+0000 7f433ffff700 1 -- 172.21.15.137:0/2718933891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f434019f230 con 0x7f43401015a0 2024-04-24T06:34:42.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.374+0000 7f433ffff700 1 --2- 172.21.15.137:0/2718933891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43401015a0 0x7f434019e6b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4330000c00 tx=0x7f4330004d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:42.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.374+0000 7f433d7fa700 1 -- 172.21.15.137:0/2718933891 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f43300053b0 con 0x7f43401015a0 2024-04-24T06:34:42.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.375+0000 7f43463c2700 1 -- 172.21.15.137:0/2718933891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43401a34c0 con 0x7f43401015a0 2024-04-24T06:34:42.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.375+0000 7f433d7fa700 1 -- 172.21.15.137:0/2718933891 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4330005550 con 0x7f43401015a0 2024-04-24T06:34:42.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.375+0000 7f433d7fa700 1 -- 172.21.15.137:0/2718933891 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f4330011670 con 0x7f43401015a0 2024-04-24T06:34:42.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.375+0000 7f43463c2700 1 -- 172.21.15.137:0/2718933891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43401a3a20 con 0x7f43401015a0 2024-04-24T06:34:42.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.376+0000 7f43463c2700 1 -- 172.21.15.137:0/2718933891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f434006b5e0 con 0x7f43401015a0 2024-04-24T06:34:42.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.376+0000 7f433d7fa700 1 -- 172.21.15.137:0/2718933891 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7f4330004030 con 0x7f43401015a0 2024-04-24T06:34:42.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.376+0000 7f433d7fa700 1 --2- 172.21.15.137:0/2718933891 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f432c039410 0x7f432c03b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:42.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.377+0000 7f433d7fa700 1 -- 172.21.15.137:0/2718933891 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f433004ef10 con 0x7f43401015a0 2024-04-24T06:34:42.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.377+0000 7f433f7fe700 1 --2- 172.21.15.137:0/2718933891 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f432c039410 0x7f432c03b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:42.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.378+0000 7f433f7fe700 1 --2- 172.21.15.137:0/2718933891 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f432c039410 0x7f432c03b8d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4334009180 tx=0x7f4334008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:42.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.391+0000 7f433d7fa700 1 -- 172.21.15.137:0/2718933891 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4330022070 con 0x7f43401015a0 2024-04-24T06:34:42.663 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:42 smithi005 ceph-mon[27798]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:42.663 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:42 smithi005 ceph-mon[27798]: Deploying daemon mon.b on smithi037 2024-04-24T06:34:42.663 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:42 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T06:34:42.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.672+0000 7f43463c2700 1 -- 172.21.15.137:0/2718933891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f434019faa0 con 0x7f43401015a0 2024-04-24T06:34:42.674 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:42 smithi137 ceph-mon[33017]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:42.674 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:42 smithi137 ceph-mon[33017]: Deploying daemon mon.b on smithi037 2024-04-24T06:34:42.674 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:42 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T06:34:42.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.674+0000 7f433d7fa700 1 -- 172.21.15.137:0/2718933891 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7f434019faa0 con 0x7f43401015a0 2024-04-24T06:34:42.675 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-24T06:34:42.675 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":2,"fsid":"43e9bad0-0204-11ef-bc93-c7b262605968","modified":"2024-04-24T06:34:36.198113Z","created":"2024-04-24T06:32:22.137390Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:3300","nonce":0},{"type":"v1","addr":"172.21.15.5:6789","nonce":0}]},"addr":"172.21.15.5:6789/0","public_addr":"172.21.15.5:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:3300","nonce":0},{"type":"v1","addr":"172.21.15.137:6789","nonce":0}]},"addr":"172.21.15.137:6789/0","public_addr":"172.21.15.137:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-24T06:34:42.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.676+0000 7f43463c2700 1 -- 172.21.15.137:0/2718933891 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f432c039410 msgr2=0x7f432c03b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:42.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.676+0000 7f43463c2700 1 --2- 172.21.15.137:0/2718933891 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f432c039410 0x7f432c03b8d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4334009180 tx=0x7f4334008040 comp rx=0 tx=0).stop 2024-04-24T06:34:42.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.676+0000 7f43463c2700 1 -- 172.21.15.137:0/2718933891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43401015a0 msgr2=0x7f434019e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:42.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.676+0000 7f43463c2700 1 --2- 172.21.15.137:0/2718933891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43401015a0 0x7f434019e6b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4330000c00 tx=0x7f4330004d60 comp rx=0 tx=0).stop 2024-04-24T06:34:42.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.676+0000 7f43463c2700 1 -- 172.21.15.137:0/2718933891 shutdown_connections 2024-04-24T06:34:42.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.676+0000 7f43463c2700 1 --2- 172.21.15.137:0/2718933891 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f432c039410 0x7f432c03b8d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:42.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.676+0000 7f43463c2700 1 --2- 172.21.15.137:0/2718933891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43401015a0 0x7f434019e6b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:42.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.676+0000 7f43463c2700 1 --2- 172.21.15.137:0/2718933891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f434019ebf0 0x7f43401a2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:42.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.676+0000 7f43463c2700 1 -- 172.21.15.137:0/2718933891 >> 172.21.15.137:0/2718933891 conn(0x7f43400fd000 msgr2=0x7f43400fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:42.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.676+0000 7f43463c2700 1 -- 172.21.15.137:0/2718933891 shutdown_connections 2024-04-24T06:34:42.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:42.676+0000 7f43463c2700 1 -- 172.21.15.137:0/2718933891 wait complete. 2024-04-24T06:34:42.679 INFO:teuthology.orchestra.run.smithi137.stderr:dumped monmap epoch 2 2024-04-24T06:34:43.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:43 smithi005 ceph-mon[27798]: from='client.? 172.21.15.137:0/2718933891' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T06:34:43.507 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:34:43 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:34:43.203+0000 7f5a33200700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-24T06:34:43.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:43 smithi137 ceph-mon[33017]: from='client.? 172.21.15.137:0/2718933891' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T06:34:44.251 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-24T06:34:44.251 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T06:34:44.497 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T06:34:44.619 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:44 smithi137 ceph-mon[33017]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:44.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:44 smithi005 ceph-mon[27798]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:45.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 podman[32844]: 2024-04-24 06:34:45.132123126 +0000 UTC m=+0.018603020 image pull b5c3183062a6ce19f361681cc58de6e3ac1de1a54c80e6094ec5695237cd98d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b 2024-04-24T06:34:45.570 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 podman[32844]: 2024-04-24 06:34:45.394351373 +0000 UTC m=+0.280831247 container init 68e4b241a9f40057a2400d4e4e92af04fb8330239ae7776805f34b5edf0a0a6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mon-b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-24T06:34:45.570 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 podman[32844]: 2024-04-24 06:34:45.397867665 +0000 UTC m=+0.284347537 container start 68e4b241a9f40057a2400d4e4e92af04fb8330239ae7776805f34b5edf0a0a6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mon-b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22) 2024-04-24T06:34:45.570 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: set uid:gid to 167:167 (ceph:ceph) 2024-04-24T06:34:45.570 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: ceph version 19.0.0-3124-ga3c8a22b (a3c8a22bd5efe18ab230a8e2d072db8f381a0411) squid (dev), process ceph-mon, pid 2 2024-04-24T06:34:45.570 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: pidfile_write: ignore empty --pid-file 2024-04-24T06:34:45.570 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: load: jerasure load: lrc 2024-04-24T06:34:45.570 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: RocksDB version: 7.9.2 2024-04-24T06:34:45.570 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Git sha 0 2024-04-24T06:34:45.570 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Compile date 2024-04-23 20:58:29 2024-04-24T06:34:45.571 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: DB SUMMARY 2024-04-24T06:34:45.571 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: DB Session ID: RW9WY43NV6PWS827NIHJ 2024-04-24T06:34:45.571 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: CURRENT file: CURRENT 2024-04-24T06:34:45.571 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: IDENTITY file: IDENTITY 2024-04-24T06:34:45.571 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-24T06:34:45.571 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-04-24T06:34:45.571 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-04-24T06:34:45.571 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.error_if_exists: 0 2024-04-24T06:34:45.571 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.create_if_missing: 0 2024-04-24T06:34:45.571 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.paranoid_checks: 1 2024-04-24T06:34:45.571 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-24T06:34:45.571 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-24T06:34:45.571 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-24T06:34:45.572 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.env: 0x55b490b09cc0 2024-04-24T06:34:45.572 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.fs: PosixFileSystem 2024-04-24T06:34:45.572 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.info_log: 0x55b492b03c20 2024-04-24T06:34:45.572 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.max_file_opening_threads: 16 2024-04-24T06:34:45.572 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.statistics: (nil) 2024-04-24T06:34:45.572 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.use_fsync: 0 2024-04-24T06:34:45.572 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.max_log_file_size: 0 2024-04-24T06:34:45.572 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-24T06:34:45.572 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-24T06:34:45.572 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.keep_log_file_num: 1000 2024-04-24T06:34:45.572 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.recycle_log_file_num: 0 2024-04-24T06:34:45.572 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.allow_fallocate: 1 2024-04-24T06:34:45.572 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.allow_mmap_reads: 0 2024-04-24T06:34:45.572 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.allow_mmap_writes: 0 2024-04-24T06:34:45.573 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.use_direct_reads: 0 2024-04-24T06:34:45.573 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-24T06:34:45.573 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.create_missing_column_families: 0 2024-04-24T06:34:45.573 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.db_log_dir: 2024-04-24T06:34:45.573 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.wal_dir: 2024-04-24T06:34:45.573 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-24T06:34:45.573 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-24T06:34:45.573 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-24T06:34:45.573 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-24T06:34:45.573 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-24T06:34:45.573 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-24T06:34:45.573 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.advise_random_on_open: 1 2024-04-24T06:34:45.573 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.db_write_buffer_size: 0 2024-04-24T06:34:45.573 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.write_buffer_manager: 0x55b492ad21e0 2024-04-24T06:34:45.573 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-24T06:34:45.574 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-24T06:34:45.574 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-24T06:34:45.574 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.rate_limiter: (nil) 2024-04-24T06:34:45.574 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-24T06:34:45.574 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.wal_recovery_mode: 2 2024-04-24T06:34:45.574 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.enable_thread_tracking: 0 2024-04-24T06:34:45.574 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.enable_pipelined_write: 0 2024-04-24T06:34:45.574 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.unordered_write: 0 2024-04-24T06:34:45.574 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-24T06:34:45.574 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-24T06:34:45.574 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-24T06:34:45.574 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-24T06:34:45.574 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.row_cache: None 2024-04-24T06:34:45.574 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.wal_filter: None 2024-04-24T06:34:45.576 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-24T06:34:45.576 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.allow_ingest_behind: 0 2024-04-24T06:34:45.577 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.two_write_queues: 0 2024-04-24T06:34:45.577 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.manual_wal_flush: 0 2024-04-24T06:34:45.577 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.wal_compression: 0 2024-04-24T06:34:45.577 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.atomic_flush: 0 2024-04-24T06:34:45.577 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-24T06:34:45.577 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-24T06:34:45.577 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-24T06:34:45.577 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.log_readahead_size: 0 2024-04-24T06:34:45.577 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-24T06:34:45.577 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.best_efforts_recovery: 0 2024-04-24T06:34:45.577 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-24T06:34:45.577 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-24T06:34:45.577 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.allow_data_in_errors: 0 2024-04-24T06:34:45.577 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.db_host_id: __hostname__ 2024-04-24T06:34:45.578 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-24T06:34:45.578 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.max_background_jobs: 2 2024-04-24T06:34:45.578 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.max_background_compactions: -1 2024-04-24T06:34:45.578 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.max_subcompactions: 1 2024-04-24T06:34:45.578 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-24T06:34:45.578 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-24T06:34:45.578 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-24T06:34:45.578 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.max_total_wal_size: 0 2024-04-24T06:34:45.578 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-24T06:34:45.578 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-24T06:34:45.578 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-24T06:34:45.578 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-24T06:34:45.578 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.max_open_files: -1 2024-04-24T06:34:45.579 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.bytes_per_sync: 0 2024-04-24T06:34:45.579 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-24T06:34:45.579 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-24T06:34:45.579 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.compaction_readahead_size: 0 2024-04-24T06:34:45.579 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.max_background_flushes: -1 2024-04-24T06:34:45.579 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Compression algorithms supported: 2024-04-24T06:34:45.579 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-24T06:34:45.579 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: kZSTD supported: 0 2024-04-24T06:34:45.579 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: kXpressCompression supported: 0 2024-04-24T06:34:45.579 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: kLZ4HCCompression supported: 1 2024-04-24T06:34:45.579 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: kZlibCompression supported: 1 2024-04-24T06:34:45.579 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: kSnappyCompression supported: 1 2024-04-24T06:34:45.579 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: kLZ4Compression supported: 1 2024-04-24T06:34:45.579 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: kBZip2Compression supported: 0 2024-04-24T06:34:45.579 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-24T06:34:45.580 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-24T06:34:45.580 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-04-24T06:34:45.580 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-24T06:34:45.580 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-24T06:34:45.580 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.merge_operator: 2024-04-24T06:34:45.580 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.compaction_filter: None 2024-04-24T06:34:45.580 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.compaction_filter_factory: None 2024-04-24T06:34:45.580 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.sst_partitioner_factory: None 2024-04-24T06:34:45.580 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-24T06:34:45.580 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-24T06:34:45.580 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b492b03de0) 2024-04-24T06:34:45.580 INFO:journalctl@ceph.mon.b.smithi037.stdout: cache_index_and_filter_blocks: 1 2024-04-24T06:34:45.580 INFO:journalctl@ceph.mon.b.smithi037.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-24T06:34:45.580 INFO:journalctl@ceph.mon.b.smithi037.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-24T06:34:45.581 INFO:journalctl@ceph.mon.b.smithi037.stdout: pin_top_level_index_and_filter: 1 2024-04-24T06:34:45.581 INFO:journalctl@ceph.mon.b.smithi037.stdout: index_type: 0 2024-04-24T06:34:45.581 INFO:journalctl@ceph.mon.b.smithi037.stdout: data_block_index_type: 0 2024-04-24T06:34:45.581 INFO:journalctl@ceph.mon.b.smithi037.stdout: index_shortening: 1 2024-04-24T06:34:45.581 INFO:journalctl@ceph.mon.b.smithi037.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-24T06:34:45.581 INFO:journalctl@ceph.mon.b.smithi037.stdout: checksum: 4 2024-04-24T06:34:45.581 INFO:journalctl@ceph.mon.b.smithi037.stdout: no_block_cache: 0 2024-04-24T06:34:45.581 INFO:journalctl@ceph.mon.b.smithi037.stdout: block_cache: 0x55b492b05090 2024-04-24T06:34:45.581 INFO:journalctl@ceph.mon.b.smithi037.stdout: block_cache_name: BinnedLRUCache 2024-04-24T06:34:45.581 INFO:journalctl@ceph.mon.b.smithi037.stdout: block_cache_options: 2024-04-24T06:34:45.581 INFO:journalctl@ceph.mon.b.smithi037.stdout: capacity : 536870912 2024-04-24T06:34:45.581 INFO:journalctl@ceph.mon.b.smithi037.stdout: num_shard_bits : 4 2024-04-24T06:34:45.581 INFO:journalctl@ceph.mon.b.smithi037.stdout: strict_capacity_limit : 0 2024-04-24T06:34:45.581 INFO:journalctl@ceph.mon.b.smithi037.stdout: high_pri_pool_ratio: 0.000 2024-04-24T06:34:45.581 INFO:journalctl@ceph.mon.b.smithi037.stdout: block_cache_compressed: (nil) 2024-04-24T06:34:45.582 INFO:journalctl@ceph.mon.b.smithi037.stdout: persistent_cache: (nil) 2024-04-24T06:34:45.582 INFO:journalctl@ceph.mon.b.smithi037.stdout: block_size: 4096 2024-04-24T06:34:45.582 INFO:journalctl@ceph.mon.b.smithi037.stdout: block_size_deviation: 10 2024-04-24T06:34:45.582 INFO:journalctl@ceph.mon.b.smithi037.stdout: block_restart_interval: 16 2024-04-24T06:34:45.582 INFO:journalctl@ceph.mon.b.smithi037.stdout: index_block_restart_interval: 1 2024-04-24T06:34:45.582 INFO:journalctl@ceph.mon.b.smithi037.stdout: metadata_block_size: 4096 2024-04-24T06:34:45.582 INFO:journalctl@ceph.mon.b.smithi037.stdout: partition_filters: 0 2024-04-24T06:34:45.582 INFO:journalctl@ceph.mon.b.smithi037.stdout: use_delta_encoding: 1 2024-04-24T06:34:45.582 INFO:journalctl@ceph.mon.b.smithi037.stdout: filter_policy: bloomfilter 2024-04-24T06:34:45.582 INFO:journalctl@ceph.mon.b.smithi037.stdout: whole_key_filtering: 1 2024-04-24T06:34:45.582 INFO:journalctl@ceph.mon.b.smithi037.stdout: verify_compression: 0 2024-04-24T06:34:45.582 INFO:journalctl@ceph.mon.b.smithi037.stdout: read_amp_bytes_per_bit: 0 2024-04-24T06:34:45.582 INFO:journalctl@ceph.mon.b.smithi037.stdout: format_version: 5 2024-04-24T06:34:45.582 INFO:journalctl@ceph.mon.b.smithi037.stdout: enable_index_compression: 1 2024-04-24T06:34:45.583 INFO:journalctl@ceph.mon.b.smithi037.stdout: block_align: 0 2024-04-24T06:34:45.583 INFO:journalctl@ceph.mon.b.smithi037.stdout: max_auto_readahead_size: 262144 2024-04-24T06:34:45.583 INFO:journalctl@ceph.mon.b.smithi037.stdout: prepopulate_block_cache: 0 2024-04-24T06:34:45.583 INFO:journalctl@ceph.mon.b.smithi037.stdout: initial_auto_readahead_size: 8192 2024-04-24T06:34:45.583 INFO:journalctl@ceph.mon.b.smithi037.stdout: num_file_reads_for_auto_readahead: 2 2024-04-24T06:34:45.583 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.write_buffer_size: 33554432 2024-04-24T06:34:45.583 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.max_write_buffer_number: 2 2024-04-24T06:34:45.583 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.compression: NoCompression 2024-04-24T06:34:45.583 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.bottommost_compression: Disabled 2024-04-24T06:34:45.583 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.prefix_extractor: nullptr 2024-04-24T06:34:45.583 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-24T06:34:45.583 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.num_levels: 7 2024-04-24T06:34:45.583 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-24T06:34:45.583 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-24T06:34:45.584 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-24T06:34:45.584 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-24T06:34:45.584 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-24T06:34:45.584 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-24T06:34:45.584 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-24T06:34:45.584 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-24T06:34:45.584 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-24T06:34:45.584 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-24T06:34:45.584 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-24T06:34:45.584 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-24T06:34:45.584 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-24T06:34:45.584 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.compression_opts.level: 32767 2024-04-24T06:34:45.584 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.compression_opts.strategy: 0 2024-04-24T06:34:45.584 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-24T06:34:45.585 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-24T06:34:45.585 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-24T06:34:45.585 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-24T06:34:45.585 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.compression_opts.enabled: false 2024-04-24T06:34:45.585 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-24T06:34:45.585 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-24T06:34:45.585 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-24T06:34:45.585 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-24T06:34:45.585 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.target_file_size_base: 67108864 2024-04-24T06:34:45.585 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-24T06:34:45.585 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-24T06:34:45.585 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-24T06:34:45.585 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-24T06:34:45.585 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-24T06:34:45.586 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-24T06:34:45.586 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-24T06:34:45.586 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-24T06:34:45.586 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-24T06:34:45.586 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-24T06:34:45.586 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-24T06:34:45.586 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-24T06:34:45.586 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-24T06:34:45.586 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-24T06:34:45.586 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.arena_block_size: 1048576 2024-04-24T06:34:45.586 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-24T06:34:45.586 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-24T06:34:45.586 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.disable_auto_compactions: 0 2024-04-24T06:34:45.586 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-24T06:34:45.587 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-24T06:34:45.587 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-24T06:34:45.587 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-24T06:34:45.587 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-24T06:34:45.587 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-24T06:34:45.587 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-24T06:34:45.587 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-24T06:34:45.587 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-24T06:34:45.587 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-24T06:34:45.587 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-24T06:34:45.587 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.inplace_update_support: 0 2024-04-24T06:34:45.587 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-24T06:34:45.587 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-24T06:34:45.587 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-24T06:34:45.588 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-24T06:34:45.588 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.bloom_locality: 0 2024-04-24T06:34:45.588 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.max_successive_merges: 0 2024-04-24T06:34:45.588 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-24T06:34:45.588 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.paranoid_file_checks: 0 2024-04-24T06:34:45.588 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.force_consistency_checks: 1 2024-04-24T06:34:45.588 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.report_bg_io_stats: 0 2024-04-24T06:34:45.588 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.ttl: 2592000 2024-04-24T06:34:45.588 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-24T06:34:45.588 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-24T06:34:45.588 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-24T06:34:45.588 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.enable_blob_files: false 2024-04-24T06:34:45.588 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.min_blob_size: 0 2024-04-24T06:34:45.588 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.blob_file_size: 268435456 2024-04-24T06:34:45.588 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-24T06:34:45.589 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-24T06:34:45.589 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-24T06:34:45.589 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-24T06:34:45.589 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-24T06:34:45.589 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.blob_file_starting_level: 0 2024-04-24T06:34:45.589 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-24T06:34:45.589 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-24T06:34:45.589 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-24T06:34:45.589 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 75450fc5-a831-4e59-9b99-94b4aff8da26 2024-04-24T06:34:45.589 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713940485428675, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-24T06:34:45.589 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-24T06:34:45.589 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713940485429222, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713940485, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "75450fc5-a831-4e59-9b99-94b4aff8da26", "db_session_id": "RW9WY43NV6PWS827NIHJ", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-24T06:34:45.589 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713940485429325, "job": 1, "event": "recovery_finished"} 2024-04-24T06:34:45.589 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-24T06:34:45.590 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-24T06:34:45.590 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55b492bf2000 2024-04-24T06:34:45.590 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: DB pointer 0x55b492bdc000 2024-04-24T06:34:45.590 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-24T06:34:45.590 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-24T06:34:45.590 INFO:journalctl@ceph.mon.b.smithi037.stdout: ** DB Stats ** 2024-04-24T06:34:45.590 INFO:journalctl@ceph.mon.b.smithi037.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T06:34:45.590 INFO:journalctl@ceph.mon.b.smithi037.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-24T06:34:45.590 INFO:journalctl@ceph.mon.b.smithi037.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T06:34:45.590 INFO:journalctl@ceph.mon.b.smithi037.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T06:34:45.590 INFO:journalctl@ceph.mon.b.smithi037.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-24T06:34:45.590 INFO:journalctl@ceph.mon.b.smithi037.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T06:34:45.590 INFO:journalctl@ceph.mon.b.smithi037.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T06:34:45.591 INFO:journalctl@ceph.mon.b.smithi037.stdout: 2024-04-24T06:34:45.591 INFO:journalctl@ceph.mon.b.smithi037.stdout: ** Compaction Stats [default] ** 2024-04-24T06:34:45.591 INFO:journalctl@ceph.mon.b.smithi037.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T06:34:45.591 INFO:journalctl@ceph.mon.b.smithi037.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-24T06:34:45.591 INFO:journalctl@ceph.mon.b.smithi037.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T06:34:45.591 INFO:journalctl@ceph.mon.b.smithi037.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T06:34:45.591 INFO:journalctl@ceph.mon.b.smithi037.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T06:34:45.591 INFO:journalctl@ceph.mon.b.smithi037.stdout: 2024-04-24T06:34:45.591 INFO:journalctl@ceph.mon.b.smithi037.stdout: ** Compaction Stats [default] ** 2024-04-24T06:34:45.591 INFO:journalctl@ceph.mon.b.smithi037.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T06:34:45.591 INFO:journalctl@ceph.mon.b.smithi037.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T06:34:45.591 INFO:journalctl@ceph.mon.b.smithi037.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T06:34:45.591 INFO:journalctl@ceph.mon.b.smithi037.stdout: 2024-04-24T06:34:45.591 INFO:journalctl@ceph.mon.b.smithi037.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-24T06:34:45.591 INFO:journalctl@ceph.mon.b.smithi037.stdout: 2024-04-24T06:34:45.592 INFO:journalctl@ceph.mon.b.smithi037.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T06:34:45.592 INFO:journalctl@ceph.mon.b.smithi037.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-24T06:34:45.592 INFO:journalctl@ceph.mon.b.smithi037.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-24T06:34:45.592 INFO:journalctl@ceph.mon.b.smithi037.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-24T06:34:45.592 INFO:journalctl@ceph.mon.b.smithi037.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-24T06:34:45.592 INFO:journalctl@ceph.mon.b.smithi037.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-24T06:34:45.592 INFO:journalctl@ceph.mon.b.smithi037.stdout: Cumulative compaction: 0.00 GB write, 0.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T06:34:45.592 INFO:journalctl@ceph.mon.b.smithi037.stdout: Interval compaction: 0.00 GB write, 0.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T06:34:45.592 INFO:journalctl@ceph.mon.b.smithi037.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-24T06:34:45.592 INFO:journalctl@ceph.mon.b.smithi037.stdout: Block cache BinnedLRUCache@0x55b492b05090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-04-24T06:34:45.592 INFO:journalctl@ceph.mon.b.smithi037.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-24T06:34:45.592 INFO:journalctl@ceph.mon.b.smithi037.stdout: 2024-04-24T06:34:45.592 INFO:journalctl@ceph.mon.b.smithi037.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-24T06:34:45.592 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-04-24T06:34:45.593 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: using public_addr v2:172.21.15.37:0/0 -> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] 2024-04-24T06:34:45.593 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: starting mon.b rank -1 at public addrs [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] at bind addrs [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 43e9bad0-0204-11ef-bc93-c7b262605968 2024-04-24T06:34:45.593 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: mon.b@-1(???) e0 preinit fsid 43e9bad0-0204-11ef-bc93-c7b262605968 2024-04-24T06:34:45.593 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: mon.b@-1(synchronizing).mds e1 new map 2024-04-24T06:34:45.593 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: mon.b@-1(synchronizing).mds e1 print_map 2024-04-24T06:34:45.593 INFO:journalctl@ceph.mon.b.smithi037.stdout: e1 2024-04-24T06:34:45.593 INFO:journalctl@ceph.mon.b.smithi037.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T06:34:45.593 INFO:journalctl@ceph.mon.b.smithi037.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T06:34:45.593 INFO:journalctl@ceph.mon.b.smithi037.stdout: legacy client fscid: -1 2024-04-24T06:34:45.593 INFO:journalctl@ceph.mon.b.smithi037.stdout: 2024-04-24T06:34:45.593 INFO:journalctl@ceph.mon.b.smithi037.stdout: No filesystems configured 2024-04-24T06:34:45.593 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-24T06:34:45.593 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-24T06:34:45.593 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-24T06:34:45.593 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-24T06:34:45.594 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-24T06:34:45.594 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-24T06:34:45.594 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-24T06:34:45.594 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T06:34:45.594 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T06:34:45.594 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T06:34:45.594 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:45.594 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:45.594 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:45.594 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: Added host smithi037 2024-04-24T06:34:45.594 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:34:45.594 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:45.594 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:45.595 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:45.595 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:45.595 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:45.595 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T06:34:45.595 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:45.595 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:45.595 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:45.595 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:45.595 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:45.595 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi037", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:34:45.595 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:34:45.595 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:34:45.595 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-24T06:34:45.595 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:45.595 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: Updating smithi037:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T06:34:45.596 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-04-24T06:34:45.596 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: Updating smithi037:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T06:34:45.596 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi137", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:34:45.596 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:45.596 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:45.596 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:45.596 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: Deploying cephadm binary to smithi137 2024-04-24T06:34:45.596 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:45.596 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:45.596 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:45.596 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: Added host smithi137 2024-04-24T06:34:45.596 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:34:45.596 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:45.596 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:45.597 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:45.597 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:45.597 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:45.597 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T06:34:45.598 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:45.598 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:45.598 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:45.598 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:45.598 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:45.598 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi137", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:34:45.598 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:34:45.598 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:34:45.598 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: Updating smithi137:/etc/ceph/ceph.conf 2024-04-24T06:34:45.598 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: Updating smithi137:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T06:34:45.598 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:45.598 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-04-24T06:34:45.598 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: Updating smithi137:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T06:34:45.599 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/1385169739' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-24T06:34:45.599 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:45.599 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:45.599 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:45.599 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/1385169739' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-24T06:34:45.599 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T06:34:45.599 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:45.599 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:45.599 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi005:172.21.15.5=a;smithi037:172.21.15.37=b;smithi137:172.21.15.137=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:34:45.599 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: Saving service mon spec with placement smithi005:172.21.15.5=a;smithi037:172.21.15.37=b;smithi137:172.21.15.137=c;count:3 2024-04-24T06:34:45.599 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:45.599 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:34:45.599 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:34:45.599 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:34:45.600 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:45.600 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T06:34:45.600 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:34:45.600 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: Deploying daemon mon.c on smithi137 2024-04-24T06:34:45.601 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:45.601 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T06:34:45.601 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T06:34:45.601 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: mon.a calling monitor election 2024-04-24T06:34:45.601 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T06:34:45.601 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:45.601 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T06:34:45.601 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T06:34:45.601 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: mon.c calling monitor election 2024-04-24T06:34:45.601 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:45.601 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T06:34:45.601 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T06:34:45.601 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T06:34:45.601 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T06:34:45.602 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: monmap epoch 2 2024-04-24T06:34:45.602 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: fsid 43e9bad0-0204-11ef-bc93-c7b262605968 2024-04-24T06:34:45.602 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: last_changed 2024-04-24T06:34:36.198113+0000 2024-04-24T06:34:45.602 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: created 2024-04-24T06:32:22.137390+0000 2024-04-24T06:34:45.602 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: min_mon_release 19 (squid) 2024-04-24T06:34:45.602 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: election_strategy: 1 2024-04-24T06:34:45.602 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-04-24T06:34:45.602 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: 1: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.c 2024-04-24T06:34:45.602 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: fsmap 2024-04-24T06:34:45.602 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T06:34:45.602 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: mgrmap e13: x(active, since 60s) 2024-04-24T06:34:45.602 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: overall HEALTH_OK 2024-04-24T06:34:45.602 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:45.602 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:45.603 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:45.603 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T06:34:45.603 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:34:45.603 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:45.603 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: Deploying daemon mon.b on smithi037 2024-04-24T06:34:45.603 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T06:34:45.603 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: from='client.? 172.21.15.137:0/2718933891' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T06:34:45.603 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:45.603 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 ceph-mon[32864]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-24T06:34:45.603 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 bash[32844]: 68e4b241a9f40057a2400d4e4e92af04fb8330239ae7776805f34b5edf0a0a6a 2024-04-24T06:34:45.603 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:45 smithi037 systemd[1]: Started Ceph mon.b for 43e9bad0-0204-11ef-bc93-c7b262605968. 2024-04-24T06:34:50.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:50 smithi137 ceph-mon[33017]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:50.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:50 smithi137 ceph-mon[33017]: mon.c calling monitor election 2024-04-24T06:34:50.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:50 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T06:34:50.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:50 smithi137 ceph-mon[33017]: mon.a calling monitor election 2024-04-24T06:34:50.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:50 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T06:34:50.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:50 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T06:34:50.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:50 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T06:34:50.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:50 smithi137 ceph-mon[33017]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:50.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:50 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T06:34:50.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:50 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T06:34:50.948 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:50 smithi137 ceph-mon[33017]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:50.948 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:50 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T06:34:50.948 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:50 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T06:34:50.948 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:50 smithi137 ceph-mon[33017]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T06:34:50.948 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:50 smithi137 ceph-mon[33017]: monmap epoch 3 2024-04-24T06:34:50.948 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:50 smithi137 ceph-mon[33017]: fsid 43e9bad0-0204-11ef-bc93-c7b262605968 2024-04-24T06:34:50.948 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:50 smithi137 ceph-mon[33017]: last_changed 2024-04-24T06:34:45.459941+0000 2024-04-24T06:34:50.948 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:50 smithi137 ceph-mon[33017]: created 2024-04-24T06:32:22.137390+0000 2024-04-24T06:34:50.948 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:50 smithi137 ceph-mon[33017]: min_mon_release 19 (squid) 2024-04-24T06:34:50.948 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:50 smithi137 ceph-mon[33017]: election_strategy: 1 2024-04-24T06:34:50.948 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:50 smithi137 ceph-mon[33017]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-04-24T06:34:50.948 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:50 smithi137 ceph-mon[33017]: 1: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.c 2024-04-24T06:34:50.948 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:50 smithi137 ceph-mon[33017]: 2: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.b 2024-04-24T06:34:50.948 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:50 smithi137 ceph-mon[33017]: fsmap 2024-04-24T06:34:50.948 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:50 smithi137 ceph-mon[33017]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T06:34:50.949 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:50 smithi137 ceph-mon[33017]: mgrmap e13: x(active, since 69s) 2024-04-24T06:34:50.949 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:50 smithi137 ceph-mon[33017]: overall HEALTH_OK 2024-04-24T06:34:50.949 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:50 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:50.949 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:50 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:50.949 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:50 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:50.949 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:50 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:51.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:50 smithi005 ceph-mon[27798]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:51.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:50 smithi005 ceph-mon[27798]: mon.c calling monitor election 2024-04-24T06:34:51.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:50 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T06:34:51.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:50 smithi005 ceph-mon[27798]: mon.a calling monitor election 2024-04-24T06:34:51.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:50 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T06:34:51.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:50 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T06:34:51.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:50 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T06:34:51.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:50 smithi005 ceph-mon[27798]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:51.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:50 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T06:34:51.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:50 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T06:34:51.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:50 smithi005 ceph-mon[27798]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:51.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:50 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T06:34:51.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:50 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T06:34:51.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:50 smithi005 ceph-mon[27798]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T06:34:51.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:50 smithi005 ceph-mon[27798]: monmap epoch 3 2024-04-24T06:34:51.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:50 smithi005 ceph-mon[27798]: fsid 43e9bad0-0204-11ef-bc93-c7b262605968 2024-04-24T06:34:51.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:50 smithi005 ceph-mon[27798]: last_changed 2024-04-24T06:34:45.459941+0000 2024-04-24T06:34:51.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:50 smithi005 ceph-mon[27798]: created 2024-04-24T06:32:22.137390+0000 2024-04-24T06:34:51.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:50 smithi005 ceph-mon[27798]: min_mon_release 19 (squid) 2024-04-24T06:34:51.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:50 smithi005 ceph-mon[27798]: election_strategy: 1 2024-04-24T06:34:51.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:50 smithi005 ceph-mon[27798]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-04-24T06:34:51.009 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:50 smithi005 ceph-mon[27798]: 1: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.c 2024-04-24T06:34:51.009 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:50 smithi005 ceph-mon[27798]: 2: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.b 2024-04-24T06:34:51.009 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:50 smithi005 ceph-mon[27798]: fsmap 2024-04-24T06:34:51.009 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:50 smithi005 ceph-mon[27798]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T06:34:51.009 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:50 smithi005 ceph-mon[27798]: mgrmap e13: x(active, since 69s) 2024-04-24T06:34:51.009 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:50 smithi005 ceph-mon[27798]: overall HEALTH_OK 2024-04-24T06:34:51.009 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:50 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:51.009 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:50 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:51.009 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:50 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:51.009 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:50 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:51.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.880+0000 7f45bc5c8700 1 -- 172.21.15.137:0/1564849936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f459c0062a0 msgr2=0x7f459c008710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:51.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.880+0000 7f45bc5c8700 1 --2- 172.21.15.137:0/1564849936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f459c0062a0 0x7f459c008710 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f45b41036d0 tx=0x7f45ac009e50 comp rx=0 tx=0).stop 2024-04-24T06:34:51.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.880+0000 7f45bc5c8700 1 -- 172.21.15.137:0/1564849936 shutdown_connections 2024-04-24T06:34:51.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.880+0000 7f45bc5c8700 1 --2- 172.21.15.137:0/1564849936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f459c0062a0 0x7f459c008710 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:51.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.880+0000 7f45bc5c8700 1 -- 172.21.15.137:0/1564849936 >> 172.21.15.137:0/1564849936 conn(0x7f45b40fe100 msgr2=0x7f45b4100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:51.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.880+0000 7f45bc5c8700 1 -- 172.21.15.137:0/1564849936 shutdown_connections 2024-04-24T06:34:51.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.880+0000 7f45bc5c8700 1 -- 172.21.15.137:0/1564849936 wait complete. 2024-04-24T06:34:51.884 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:51 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:34:51.884 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:51 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:51.884 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:51 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T06:34:51.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.881+0000 7f45bc5c8700 1 Processor -- start 2024-04-24T06:34:51.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.881+0000 7f45bc5c8700 1 -- start start 2024-04-24T06:34:51.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.881+0000 7f45bc5c8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f459c0062a0 0x7f45b4074df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:51.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.881+0000 7f45bc5c8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45b4075330 0x7f45b40757b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:51.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.881+0000 7f45bc5c8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45b41a43d0 0x7f45b41a67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:51.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.881+0000 7f45bc5c8700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f45b410a660 con 0x7f45b41a43d0 2024-04-24T06:34:51.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.881+0000 7f45bc5c8700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f45b410a4e0 con 0x7f459c0062a0 2024-04-24T06:34:51.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.881+0000 7f45bc5c8700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f45b410a7e0 con 0x7f45b4075330 2024-04-24T06:34:51.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.881+0000 7f45b9b63700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45b4075330 0x7f45b40757b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:51.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.881+0000 7f45b9b63700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45b4075330 0x7f45b40757b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58228/0 (socket says 172.21.15.137:58228) 2024-04-24T06:34:51.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.881+0000 7f45b9b63700 1 -- 172.21.15.137:0/3889825525 learned_addr learned my addr 172.21.15.137:0/3889825525 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-04-24T06:34:51.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.882+0000 7f45ba364700 1 --2- 172.21.15.137:0/3889825525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f459c0062a0 0x7f45b4074df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:51.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.882+0000 7f45ba364700 1 -- 172.21.15.137:0/3889825525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f459c0062a0 msgr2=0x7f45b4074df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-24T06:34:51.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.882+0000 7f45ba364700 1 -- 172.21.15.137:0/3889825525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f459c0062a0 msgr2=0x7f45b4074df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T06:34:51.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.882+0000 7f45b9b63700 1 -- 172.21.15.137:0/3889825525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f459c0062a0 msgr2=0x7f45b4074df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:51.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.882+0000 7f45ba364700 1 --2- 172.21.15.137:0/3889825525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f459c0062a0 0x7f45b4074df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T06:34:51.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.882+0000 7f45ba364700 1 --2- 172.21.15.137:0/3889825525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f459c0062a0 0x7f45b4074df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T06:34:51.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.882+0000 7f45b9b63700 1 --2- 172.21.15.137:0/3889825525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f459c0062a0 0x7f45b4074df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:51.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.882+0000 7f45b9b63700 1 -- 172.21.15.137:0/3889825525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45b41a43d0 msgr2=0x7f45b41a67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:51.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.882+0000 7f45bab65700 1 --2- 172.21.15.137:0/3889825525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45b41a43d0 0x7f45b41a67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:51.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.882+0000 7f45b9b63700 1 --2- 172.21.15.137:0/3889825525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45b41a43d0 0x7f45b41a67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:51.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.882+0000 7f45b9b63700 1 -- 172.21.15.137:0/3889825525 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b41a6ef0 con 0x7f45b4075330 2024-04-24T06:34:51.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.882+0000 7f45b9b63700 1 --2- 172.21.15.137:0/3889825525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45b4075330 0x7f45b40757b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f45a800b780 tx=0x7f45a800bb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:51.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.882+0000 7f45b37fe700 1 -- 172.21.15.137:0/3889825525 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a8004230 con 0x7f45b4075330 2024-04-24T06:34:51.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.882+0000 7f45bc5c8700 1 -- 172.21.15.137:0/3889825525 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45b41a71e0 con 0x7f45b4075330 2024-04-24T06:34:51.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.882+0000 7f45b37fe700 1 -- 172.21.15.137:0/3889825525 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45a80043d0 con 0x7f45b4075330 2024-04-24T06:34:51.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.882+0000 7f45b37fe700 1 -- 172.21.15.137:0/3889825525 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a8011c60 con 0x7f45b4075330 2024-04-24T06:34:51.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.882+0000 7f45bc5c8700 1 -- 172.21.15.137:0/3889825525 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45b41a7720 con 0x7f45b4075330 2024-04-24T06:34:51.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.883+0000 7f45b37fe700 1 -- 172.21.15.137:0/3889825525 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7f45a801a400 con 0x7f45b4075330 2024-04-24T06:34:51.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.883+0000 7f45bc5c8700 1 -- 172.21.15.137:0/3889825525 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45b4064150 con 0x7f45b4075330 2024-04-24T06:34:51.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.883+0000 7f45b37fe700 1 --2- 172.21.15.137:0/3889825525 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f45a0039ae0 0x7f45a003bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:51.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.883+0000 7f45b37fe700 1 -- 172.21.15.137:0/3889825525 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f45a804c4d0 con 0x7f45b4075330 2024-04-24T06:34:51.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.884+0000 7f45ba364700 1 --2- 172.21.15.137:0/3889825525 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f45a0039ae0 0x7f45a003bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:51.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.884+0000 7f45ba364700 1 --2- 172.21.15.137:0/3889825525 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f45a0039ae0 0x7f45a003bfa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f45b41036d0 tx=0x7f45ac009960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:51.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:51.889+0000 7f45b37fe700 1 -- 172.21.15.137:0/3889825525 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f45a8020500 con 0x7f45b4075330 2024-04-24T06:34:52.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:51 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:34:52.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:51 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:52.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:51 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T06:34:52.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:52.178+0000 7f45bc5c8700 1 -- 172.21.15.137:0/3889825525 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f45b41a7cb0 con 0x7f45b4075330 2024-04-24T06:34:52.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:52.179+0000 7f45b37fe700 1 -- 172.21.15.137:0/3889825525 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1275 (secure 0 0 0) 0x7f45a80202a0 con 0x7f45b4075330 2024-04-24T06:34:52.182 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-24T06:34:52.182 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":3,"fsid":"43e9bad0-0204-11ef-bc93-c7b262605968","modified":"2024-04-24T06:34:45.459941Z","created":"2024-04-24T06:32:22.137390Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:3300","nonce":0},{"type":"v1","addr":"172.21.15.5:6789","nonce":0}]},"addr":"172.21.15.5:6789/0","public_addr":"172.21.15.5:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:3300","nonce":0},{"type":"v1","addr":"172.21.15.137:6789","nonce":0}]},"addr":"172.21.15.137:6789/0","public_addr":"172.21.15.137:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-24T06:34:52.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:52.181+0000 7f45bc5c8700 1 -- 172.21.15.137:0/3889825525 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f45a0039ae0 msgr2=0x7f45a003bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:52.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:52.181+0000 7f45bc5c8700 1 --2- 172.21.15.137:0/3889825525 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f45a0039ae0 0x7f45a003bfa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f45b41036d0 tx=0x7f45ac009960 comp rx=0 tx=0).stop 2024-04-24T06:34:52.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:52.181+0000 7f45bc5c8700 1 -- 172.21.15.137:0/3889825525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45b4075330 msgr2=0x7f45b40757b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:52.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:52.181+0000 7f45bc5c8700 1 --2- 172.21.15.137:0/3889825525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45b4075330 0x7f45b40757b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f45a800b780 tx=0x7f45a800bb40 comp rx=0 tx=0).stop 2024-04-24T06:34:52.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:52.181+0000 7f45bc5c8700 1 -- 172.21.15.137:0/3889825525 shutdown_connections 2024-04-24T06:34:52.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:52.181+0000 7f45bc5c8700 1 --2- 172.21.15.137:0/3889825525 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f45a0039ae0 0x7f45a003bfa0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:52.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:52.181+0000 7f45bc5c8700 1 --2- 172.21.15.137:0/3889825525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45b41a43d0 0x7f45b41a67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:52.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:52.181+0000 7f45bc5c8700 1 --2- 172.21.15.137:0/3889825525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f459c0062a0 0x7f45b4074df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:52.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:52.181+0000 7f45bc5c8700 1 --2- 172.21.15.137:0/3889825525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45b4075330 0x7f45b40757b0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:52.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:52.181+0000 7f45bc5c8700 1 -- 172.21.15.137:0/3889825525 >> 172.21.15.137:0/3889825525 conn(0x7f45b40fe100 msgr2=0x7f45b40fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:52.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:52.181+0000 7f45bc5c8700 1 -- 172.21.15.137:0/3889825525 shutdown_connections 2024-04-24T06:34:52.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:52.181+0000 7f45bc5c8700 1 -- 172.21.15.137:0/3889825525 wait complete. 2024-04-24T06:34:52.185 INFO:teuthology.orchestra.run.smithi137.stderr:dumped monmap epoch 3 2024-04-24T06:34:52.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:52 smithi037 ceph-mon[32864]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:52.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:52 smithi037 ceph-mon[32864]: mon.c calling monitor election 2024-04-24T06:34:52.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:52 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T06:34:52.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:52 smithi037 ceph-mon[32864]: mon.a calling monitor election 2024-04-24T06:34:52.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:52 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T06:34:52.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:52 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T06:34:52.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:52 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T06:34:52.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:52 smithi037 ceph-mon[32864]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:52.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:52 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T06:34:52.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:52 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T06:34:52.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:52 smithi037 ceph-mon[32864]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:52.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:52 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T06:34:52.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:52 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T06:34:52.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:52 smithi037 ceph-mon[32864]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T06:34:52.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:52 smithi037 ceph-mon[32864]: monmap epoch 3 2024-04-24T06:34:52.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:52 smithi037 ceph-mon[32864]: fsid 43e9bad0-0204-11ef-bc93-c7b262605968 2024-04-24T06:34:52.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:52 smithi037 ceph-mon[32864]: last_changed 2024-04-24T06:34:45.459941+0000 2024-04-24T06:34:52.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:52 smithi037 ceph-mon[32864]: created 2024-04-24T06:32:22.137390+0000 2024-04-24T06:34:52.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:52 smithi037 ceph-mon[32864]: min_mon_release 19 (squid) 2024-04-24T06:34:52.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:52 smithi037 ceph-mon[32864]: election_strategy: 1 2024-04-24T06:34:52.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:52 smithi037 ceph-mon[32864]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-04-24T06:34:52.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:52 smithi037 ceph-mon[32864]: 1: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.c 2024-04-24T06:34:52.771 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:52 smithi037 ceph-mon[32864]: 2: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.b 2024-04-24T06:34:52.771 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:52 smithi037 ceph-mon[32864]: fsmap 2024-04-24T06:34:52.771 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:52 smithi037 ceph-mon[32864]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T06:34:52.771 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:52 smithi037 ceph-mon[32864]: mgrmap e13: x(active, since 69s) 2024-04-24T06:34:52.771 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:52 smithi037 ceph-mon[32864]: overall HEALTH_OK 2024-04-24T06:34:52.771 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:52 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:52.771 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:52 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:52.771 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:52 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:52.771 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:52 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:52.771 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:52 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:34:52.771 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:52 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:52.771 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:52 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T06:34:52.873 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-24T06:34:52.873 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-24T06:34:53.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:53 smithi005 ceph-mon[27798]: mon.b calling monitor election 2024-04-24T06:34:53.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:53 smithi005 ceph-mon[27798]: Updating smithi005:/etc/ceph/ceph.conf 2024-04-24T06:34:53.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:53 smithi005 ceph-mon[27798]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-24T06:34:53.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:53 smithi005 ceph-mon[27798]: Updating smithi137:/etc/ceph/ceph.conf 2024-04-24T06:34:53.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:53 smithi005 ceph-mon[27798]: mon.b calling monitor election 2024-04-24T06:34:53.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:53 smithi005 ceph-mon[27798]: mon.c calling monitor election 2024-04-24T06:34:53.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:53 smithi005 ceph-mon[27798]: mon.a calling monitor election 2024-04-24T06:34:53.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:53 smithi005 ceph-mon[27798]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-24T06:34:53.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:53 smithi005 ceph-mon[27798]: monmap epoch 3 2024-04-24T06:34:53.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:53 smithi005 ceph-mon[27798]: fsid 43e9bad0-0204-11ef-bc93-c7b262605968 2024-04-24T06:34:53.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:53 smithi005 ceph-mon[27798]: last_changed 2024-04-24T06:34:45.459941+0000 2024-04-24T06:34:53.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:53 smithi005 ceph-mon[27798]: created 2024-04-24T06:32:22.137390+0000 2024-04-24T06:34:53.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:53 smithi005 ceph-mon[27798]: min_mon_release 19 (squid) 2024-04-24T06:34:53.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:53 smithi005 ceph-mon[27798]: election_strategy: 1 2024-04-24T06:34:53.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:53 smithi005 ceph-mon[27798]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-04-24T06:34:53.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:53 smithi005 ceph-mon[27798]: 1: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.c 2024-04-24T06:34:53.258 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:53 smithi005 ceph-mon[27798]: 2: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.b 2024-04-24T06:34:53.258 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:53 smithi005 ceph-mon[27798]: fsmap 2024-04-24T06:34:53.258 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:53 smithi005 ceph-mon[27798]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T06:34:53.258 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:53 smithi005 ceph-mon[27798]: mgrmap e13: x(active, since 71s) 2024-04-24T06:34:53.258 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:53 smithi005 ceph-mon[27798]: overall HEALTH_OK 2024-04-24T06:34:53.258 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:53 smithi005 ceph-mon[27798]: Updating smithi037:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T06:34:53.258 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:53 smithi005 ceph-mon[27798]: Updating smithi137:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T06:34:53.258 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:53 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:53.258 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:53 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:53.258 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:53 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:53.258 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:53 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:53.258 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:53 smithi005 ceph-mon[27798]: Updating smithi005:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T06:34:53.299 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:34:53.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:53 smithi137 ceph-mon[33017]: mon.b calling monitor election 2024-04-24T06:34:53.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:53 smithi137 ceph-mon[33017]: Updating smithi005:/etc/ceph/ceph.conf 2024-04-24T06:34:53.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:53 smithi137 ceph-mon[33017]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-24T06:34:53.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:53 smithi137 ceph-mon[33017]: Updating smithi137:/etc/ceph/ceph.conf 2024-04-24T06:34:53.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:53 smithi137 ceph-mon[33017]: mon.b calling monitor election 2024-04-24T06:34:53.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:53 smithi137 ceph-mon[33017]: mon.c calling monitor election 2024-04-24T06:34:53.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:53 smithi137 ceph-mon[33017]: mon.a calling monitor election 2024-04-24T06:34:53.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:53 smithi137 ceph-mon[33017]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-24T06:34:53.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:53 smithi137 ceph-mon[33017]: monmap epoch 3 2024-04-24T06:34:53.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:53 smithi137 ceph-mon[33017]: fsid 43e9bad0-0204-11ef-bc93-c7b262605968 2024-04-24T06:34:53.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:53 smithi137 ceph-mon[33017]: last_changed 2024-04-24T06:34:45.459941+0000 2024-04-24T06:34:53.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:53 smithi137 ceph-mon[33017]: created 2024-04-24T06:32:22.137390+0000 2024-04-24T06:34:53.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:53 smithi137 ceph-mon[33017]: min_mon_release 19 (squid) 2024-04-24T06:34:53.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:53 smithi137 ceph-mon[33017]: election_strategy: 1 2024-04-24T06:34:53.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:53 smithi137 ceph-mon[33017]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-04-24T06:34:53.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:53 smithi137 ceph-mon[33017]: 1: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.c 2024-04-24T06:34:53.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:53 smithi137 ceph-mon[33017]: 2: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.b 2024-04-24T06:34:53.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:53 smithi137 ceph-mon[33017]: fsmap 2024-04-24T06:34:53.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:53 smithi137 ceph-mon[33017]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T06:34:53.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:53 smithi137 ceph-mon[33017]: mgrmap e13: x(active, since 71s) 2024-04-24T06:34:53.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:53 smithi137 ceph-mon[33017]: overall HEALTH_OK 2024-04-24T06:34:53.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:53 smithi137 ceph-mon[33017]: Updating smithi037:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T06:34:53.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:53 smithi137 ceph-mon[33017]: Updating smithi137:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T06:34:53.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:53 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:53.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:53 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:53.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:53 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:53.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:53 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:53.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:53 smithi137 ceph-mon[33017]: Updating smithi005:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T06:34:53.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:53 smithi037 ceph-mon[32864]: mon.b calling monitor election 2024-04-24T06:34:53.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:53 smithi037 ceph-mon[32864]: Updating smithi005:/etc/ceph/ceph.conf 2024-04-24T06:34:53.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:53 smithi037 ceph-mon[32864]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-24T06:34:53.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:53 smithi037 ceph-mon[32864]: Updating smithi137:/etc/ceph/ceph.conf 2024-04-24T06:34:53.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:53 smithi037 ceph-mon[32864]: mon.b calling monitor election 2024-04-24T06:34:53.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:53 smithi037 ceph-mon[32864]: mon.c calling monitor election 2024-04-24T06:34:53.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:53 smithi037 ceph-mon[32864]: mon.a calling monitor election 2024-04-24T06:34:53.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:53 smithi037 ceph-mon[32864]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-24T06:34:53.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:53 smithi037 ceph-mon[32864]: monmap epoch 3 2024-04-24T06:34:53.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:53 smithi037 ceph-mon[32864]: fsid 43e9bad0-0204-11ef-bc93-c7b262605968 2024-04-24T06:34:53.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:53 smithi037 ceph-mon[32864]: last_changed 2024-04-24T06:34:45.459941+0000 2024-04-24T06:34:53.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:53 smithi037 ceph-mon[32864]: created 2024-04-24T06:32:22.137390+0000 2024-04-24T06:34:53.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:53 smithi037 ceph-mon[32864]: min_mon_release 19 (squid) 2024-04-24T06:34:53.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:53 smithi037 ceph-mon[32864]: election_strategy: 1 2024-04-24T06:34:53.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:53 smithi037 ceph-mon[32864]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-04-24T06:34:53.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:53 smithi037 ceph-mon[32864]: 1: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.c 2024-04-24T06:34:53.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:53 smithi037 ceph-mon[32864]: 2: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.b 2024-04-24T06:34:53.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:53 smithi037 ceph-mon[32864]: fsmap 2024-04-24T06:34:53.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:53 smithi037 ceph-mon[32864]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T06:34:53.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:53 smithi037 ceph-mon[32864]: mgrmap e13: x(active, since 71s) 2024-04-24T06:34:53.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:53 smithi037 ceph-mon[32864]: overall HEALTH_OK 2024-04-24T06:34:53.521 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:53 smithi037 ceph-mon[32864]: Updating smithi037:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T06:34:53.521 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:53 smithi037 ceph-mon[32864]: Updating smithi137:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T06:34:53.521 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:53 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:53.521 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:53 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:53.521 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:53 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:53.521 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:53 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:53.521 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:53 smithi037 ceph-mon[32864]: Updating smithi005:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T06:34:54.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:54 smithi137 ceph-mon[33017]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:54.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:54 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T06:34:54.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:54 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:54.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:54 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:54.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:54 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:54.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:54 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:54.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:54 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:54.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:54 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:54.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:54 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:54.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:54 smithi137 ceph-mon[33017]: Reconfiguring mon.a (unknown last config time)... 2024-04-24T06:34:54.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:54 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T06:34:54.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:54 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T06:34:54.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:54 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:34:54.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:54 smithi137 ceph-mon[33017]: Reconfiguring daemon mon.a on smithi005 2024-04-24T06:34:54.455 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:54 smithi005 ceph-mon[27798]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:54.455 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:54 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T06:34:54.455 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:54 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:54.456 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:54 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:54.456 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:54 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:54.456 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:54 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:54.456 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:54 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:54.456 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:54 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:54.456 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:54 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:54.456 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:54 smithi005 ceph-mon[27798]: Reconfiguring mon.a (unknown last config time)... 2024-04-24T06:34:54.456 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:54 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T06:34:54.456 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:54 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T06:34:54.456 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:54 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:34:54.456 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:54 smithi005 ceph-mon[27798]: Reconfiguring daemon mon.a on smithi005 2024-04-24T06:34:54.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:54 smithi037 ceph-mon[32864]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:54.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:54 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T06:34:54.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:54 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:54.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:54 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:54.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:54 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:54.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:54 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:54.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:54 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:54.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:54 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:54.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:54 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:54.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:54 smithi037 ceph-mon[32864]: Reconfiguring mon.a (unknown last config time)... 2024-04-24T06:34:54.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:54 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T06:34:54.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:54 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T06:34:54.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:54 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:34:54.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:54 smithi037 ceph-mon[32864]: Reconfiguring daemon mon.a on smithi005 2024-04-24T06:34:55.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.445+0000 7f15d5c00700 1 -- 172.21.15.5:0/3204335403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15d0074f20 msgr2=0x7f15d0075300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:55.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.445+0000 7f15d5c00700 1 --2- 172.21.15.5:0/3204335403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15d0074f20 0x7f15d0075300 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f15c0009910 tx=0x7f15c0009c20 comp rx=0 tx=0).stop 2024-04-24T06:34:55.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.445+0000 7f15d5c00700 1 -- 172.21.15.5:0/3204335403 shutdown_connections 2024-04-24T06:34:55.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.445+0000 7f15d5c00700 1 --2- 172.21.15.5:0/3204335403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15d0074f20 0x7f15d0075300 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:55.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.445+0000 7f15d5c00700 1 -- 172.21.15.5:0/3204335403 >> 172.21.15.5:0/3204335403 conn(0x7f15d00fe010 msgr2=0x7f15d0100430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:55.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.445+0000 7f15d5c00700 1 -- 172.21.15.5:0/3204335403 shutdown_connections 2024-04-24T06:34:55.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.446+0000 7f15d5c00700 1 -- 172.21.15.5:0/3204335403 wait complete. 2024-04-24T06:34:55.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.447+0000 7f15d5c00700 1 Processor -- start 2024-04-24T06:34:55.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.447+0000 7f15d5c00700 1 -- start start 2024-04-24T06:34:55.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.447+0000 7f15d5c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15d0074f20 0x7f15d019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:55.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.447+0000 7f15d5c00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15d019aa10 0x7f15d019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:55.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.447+0000 7f15d5c00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15d019f2e0 0x7f15d019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:55.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.447+0000 7f15d5c00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f15d010ed60 con 0x7f15d019aa10 2024-04-24T06:34:55.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.447+0000 7f15d5c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f15d010ebe0 con 0x7f15d0074f20 2024-04-24T06:34:55.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.447+0000 7f15d5c00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f15d010eee0 con 0x7f15d019f2e0 2024-04-24T06:34:55.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.448+0000 7f15ce000700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15d019aa10 0x7f15d019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:55.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.448+0000 7f15cea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15d0074f20 0x7f15d019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:55.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.448+0000 7f15ce000700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15d019aa10 0x7f15d019eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35746/0 (socket says 172.21.15.5:35746) 2024-04-24T06:34:55.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.448+0000 7f15ce000700 1 -- 172.21.15.5:0/200909981 learned_addr learned my addr 172.21.15.5:0/200909981 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:34:55.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.448+0000 7f15cf400700 1 --2- 172.21.15.5:0/200909981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15d019f2e0 0x7f15d019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:55.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.448+0000 7f15ce000700 1 -- 172.21.15.5:0/200909981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15d019f2e0 msgr2=0x7f15d019f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:55.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.448+0000 7f15ce000700 1 --2- 172.21.15.5:0/200909981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15d019f2e0 0x7f15d019f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:55.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.448+0000 7f15ce000700 1 -- 172.21.15.5:0/200909981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15d0074f20 msgr2=0x7f15d019a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:55.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.448+0000 7f15ce000700 1 --2- 172.21.15.5:0/200909981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15d0074f20 0x7f15d019a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:55.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.448+0000 7f15ce000700 1 -- 172.21.15.5:0/200909981 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d01a7050 con 0x7f15d019aa10 2024-04-24T06:34:55.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.449+0000 7f15ce000700 1 --2- 172.21.15.5:0/200909981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15d019aa10 0x7f15d019eda0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f15bc00b780 tx=0x7f15bc00ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:55.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.449+0000 7f15c7400700 1 -- 172.21.15.5:0/200909981 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15bc016070 con 0x7f15d019aa10 2024-04-24T06:34:55.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.449+0000 7f15c7400700 1 -- 172.21.15.5:0/200909981 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15bc0036a0 con 0x7f15d019aa10 2024-04-24T06:34:55.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.449+0000 7f15c7400700 1 -- 172.21.15.5:0/200909981 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15bc011af0 con 0x7f15d019aa10 2024-04-24T06:34:55.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.449+0000 7f15d5c00700 1 -- 172.21.15.5:0/200909981 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15d01a7280 con 0x7f15d019aa10 2024-04-24T06:34:55.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.449+0000 7f15d5c00700 1 -- 172.21.15.5:0/200909981 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15d01a77c0 con 0x7f15d019aa10 2024-04-24T06:34:55.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.452+0000 7f15c7400700 1 -- 172.21.15.5:0/200909981 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7f15bc019070 con 0x7f15d019aa10 2024-04-24T06:34:55.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.452+0000 7f15c7400700 1 --2- 172.21.15.5:0/200909981 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f15b4039ae0 0x7f15b403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:55.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.452+0000 7f15c7400700 1 -- 172.21.15.5:0/200909981 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f15bc04b4c0 con 0x7f15d019aa10 2024-04-24T06:34:55.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.452+0000 7f15d5c00700 1 -- 172.21.15.5:0/200909981 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15d0064150 con 0x7f15d019aa10 2024-04-24T06:34:55.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.453+0000 7f15cea00700 1 --2- 172.21.15.5:0/200909981 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f15b4039ae0 0x7f15b403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:55.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.453+0000 7f15cea00700 1 --2- 172.21.15.5:0/200909981 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f15b4039ae0 0x7f15b403bfa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f15c0009750 tx=0x7f15c000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:55.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.459+0000 7f15c7400700 1 -- 172.21.15.5:0/200909981 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f15bc039e20 con 0x7f15d019aa10 2024-04-24T06:34:55.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.695+0000 7f15d5c00700 1 -- 172.21.15.5:0/200909981 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f15d01a7ad0 con 0x7f15d019aa10 2024-04-24T06:34:55.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.696+0000 7f15c7400700 1 -- 172.21.15.5:0/200909981 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+271 (secure 0 0 0) 0x7f15bc010380 con 0x7f15d019aa10 2024-04-24T06:34:55.697 INFO:teuthology.orchestra.run.smithi005.stdout:# minimal ceph.conf for 43e9bad0-0204-11ef-bc93-c7b262605968 2024-04-24T06:34:55.697 INFO:teuthology.orchestra.run.smithi005.stdout:[global] 2024-04-24T06:34:55.698 INFO:teuthology.orchestra.run.smithi005.stdout: fsid = 43e9bad0-0204-11ef-bc93-c7b262605968 2024-04-24T06:34:55.698 INFO:teuthology.orchestra.run.smithi005.stdout: mon_host = [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] 2024-04-24T06:34:55.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.699+0000 7f15d5c00700 1 -- 172.21.15.5:0/200909981 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f15b4039ae0 msgr2=0x7f15b403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:55.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.699+0000 7f15d5c00700 1 --2- 172.21.15.5:0/200909981 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f15b4039ae0 0x7f15b403bfa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f15c0009750 tx=0x7f15c000b540 comp rx=0 tx=0).stop 2024-04-24T06:34:55.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.699+0000 7f15d5c00700 1 -- 172.21.15.5:0/200909981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15d019aa10 msgr2=0x7f15d019eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:55.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.699+0000 7f15d5c00700 1 --2- 172.21.15.5:0/200909981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15d019aa10 0x7f15d019eda0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f15bc00b780 tx=0x7f15bc00ba90 comp rx=0 tx=0).stop 2024-04-24T06:34:55.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.699+0000 7f15d5c00700 1 -- 172.21.15.5:0/200909981 shutdown_connections 2024-04-24T06:34:55.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.699+0000 7f15d5c00700 1 --2- 172.21.15.5:0/200909981 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f15b4039ae0 0x7f15b403bfa0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:55.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.699+0000 7f15d5c00700 1 --2- 172.21.15.5:0/200909981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15d019f2e0 0x7f15d019f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:55.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.699+0000 7f15d5c00700 1 --2- 172.21.15.5:0/200909981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15d0074f20 0x7f15d019a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:55.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.699+0000 7f15d5c00700 1 --2- 172.21.15.5:0/200909981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15d019aa10 0x7f15d019eda0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:55.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.699+0000 7f15d5c00700 1 -- 172.21.15.5:0/200909981 >> 172.21.15.5:0/200909981 conn(0x7f15d00fe010 msgr2=0x7f15d01068e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:55.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.699+0000 7f15d5c00700 1 -- 172.21.15.5:0/200909981 shutdown_connections 2024-04-24T06:34:55.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:34:55.699+0000 7f15d5c00700 1 -- 172.21.15.5:0/200909981 wait complete. 2024-04-24T06:34:56.440 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:56 smithi005 ceph-mon[27798]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:56.441 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:56 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/200909981' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:34:56.490 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-24T06:34:56.491 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-24T06:34:56.491 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/ceph/ceph.conf 2024-04-24T06:34:56.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:56 smithi037 ceph-mon[32864]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:56.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:56 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/200909981' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:34:56.528 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:56 smithi137 ceph-mon[33017]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:56.528 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:56 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/200909981' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:34:56.573 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-24T06:34:56.573 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T06:34:56.694 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-24T06:34:56.694 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/ceph/ceph.conf 2024-04-24T06:34:56.724 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-24T06:34:56.724 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T06:34:56.793 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-24T06:34:56.793 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/ceph/ceph.conf 2024-04-24T06:34:56.821 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-24T06:34:56.821 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T06:34:56.885 INFO:tasks.cephadm:Adding mgr.x on smithi005 2024-04-24T06:34:56.885 INFO:tasks.cephadm:Adding mgr.y on smithi037 2024-04-24T06:34:56.885 INFO:tasks.cephadm:Adding mgr.z on smithi137 2024-04-24T06:34:56.885 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi005=x;smithi037=y;smithi137=z' 2024-04-24T06:34:57.159 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T06:34:58.377 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:58 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:58.377 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:58 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:58.378 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:58 smithi137 ceph-mon[33017]: Reconfiguring mon.b (monmap changed)... 2024-04-24T06:34:58.378 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:58 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T06:34:58.378 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:58 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T06:34:58.378 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:58 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:34:58.378 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:58 smithi137 ceph-mon[33017]: Reconfiguring daemon mon.b on smithi037 2024-04-24T06:34:58.378 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:58 smithi137 ceph-mon[33017]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:58.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:58 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:58.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:58 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:58.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:58 smithi005 ceph-mon[27798]: Reconfiguring mon.b (monmap changed)... 2024-04-24T06:34:58.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:58 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T06:34:58.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:58 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T06:34:58.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:58 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:34:58.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:58 smithi005 ceph-mon[27798]: Reconfiguring daemon mon.b on smithi037 2024-04-24T06:34:58.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:58 smithi005 ceph-mon[27798]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:58.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:58 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:58.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:58 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:58.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:58 smithi037 ceph-mon[32864]: Reconfiguring mon.b (monmap changed)... 2024-04-24T06:34:58.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:58 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T06:34:58.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:58 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T06:34:58.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:58 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:34:58.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:58 smithi037 ceph-mon[32864]: Reconfiguring daemon mon.b on smithi037 2024-04-24T06:34:58.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:58 smithi037 ceph-mon[32864]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:34:58.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.653+0000 7ff1c044d700 1 -- 172.21.15.137:0/4145563178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1b8100900 msgr2=0x7ff1b8100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:58.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.653+0000 7ff1c044d700 1 --2- 172.21.15.137:0/4145563178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1b8100900 0x7ff1b8100d20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4009910 tx=0x7ff1b4009c20 comp rx=0 tx=0).stop 2024-04-24T06:34:58.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.653+0000 7ff1c044d700 1 -- 172.21.15.137:0/4145563178 shutdown_connections 2024-04-24T06:34:58.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.653+0000 7ff1c044d700 1 --2- 172.21.15.137:0/4145563178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1b8100900 0x7ff1b8100d20 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.653+0000 7ff1c044d700 1 -- 172.21.15.137:0/4145563178 >> 172.21.15.137:0/4145563178 conn(0x7ff1b80fc260 msgr2=0x7ff1b80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:58.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.653+0000 7ff1c044d700 1 -- 172.21.15.137:0/4145563178 shutdown_connections 2024-04-24T06:34:58.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.653+0000 7ff1c044d700 1 -- 172.21.15.137:0/4145563178 wait complete. 2024-04-24T06:34:58.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.653+0000 7ff1c044d700 1 Processor -- start 2024-04-24T06:34:58.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.653+0000 7ff1c044d700 1 -- start start 2024-04-24T06:34:58.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.654+0000 7ff1c044d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1b8100900 0x7ff1b8103240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.654+0000 7ff1c044d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1b8101890 0x7ff1b8101d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.654+0000 7ff1c044d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1b8102250 0x7ff1b81a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.654+0000 7ff1c044d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b8108680 con 0x7ff1b8100900 2024-04-24T06:34:58.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.654+0000 7ff1c044d700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b8108500 con 0x7ff1b8102250 2024-04-24T06:34:58.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.654+0000 7ff1c044d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b8108800 con 0x7ff1b8101890 2024-04-24T06:34:58.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.654+0000 7ff1bd9e8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1b8101890 0x7ff1b8101d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.654+0000 7ff1bd9e8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1b8101890 0x7ff1b8101d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58240/0 (socket says 172.21.15.137:58240) 2024-04-24T06:34:58.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.654+0000 7ff1bd9e8700 1 -- 172.21.15.137:0/984569825 learned_addr learned my addr 172.21.15.137:0/984569825 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-04-24T06:34:58.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.654+0000 7ff1be9ea700 1 --2- 172.21.15.137:0/984569825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1b8102250 0x7ff1b81a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.654+0000 7ff1bd9e8700 1 -- 172.21.15.137:0/984569825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1b8102250 msgr2=0x7ff1b81a1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:58.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.654+0000 7ff1bd9e8700 1 --2- 172.21.15.137:0/984569825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1b8102250 0x7ff1b81a1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.654+0000 7ff1bd9e8700 1 -- 172.21.15.137:0/984569825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1b8100900 msgr2=0x7ff1b8103240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:58.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.654+0000 7ff1bd9e8700 1 --2- 172.21.15.137:0/984569825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1b8100900 0x7ff1b8103240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.654+0000 7ff1bd9e8700 1 -- 172.21.15.137:0/984569825 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b81a2500 con 0x7ff1b8101890 2024-04-24T06:34:58.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.654+0000 7ff1bd9e8700 1 --2- 172.21.15.137:0/984569825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1b8101890 0x7ff1b8101d10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff1a800b780 tx=0x7ff1a800ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:58.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.655+0000 7ff1af7fe700 1 -- 172.21.15.137:0/984569825 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8004230 con 0x7ff1b8101890 2024-04-24T06:34:58.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.655+0000 7ff1c044d700 1 -- 172.21.15.137:0/984569825 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1b81a27f0 con 0x7ff1b8101890 2024-04-24T06:34:58.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.655+0000 7ff1af7fe700 1 -- 172.21.15.137:0/984569825 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1a80043d0 con 0x7ff1b8101890 2024-04-24T06:34:58.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.655+0000 7ff1af7fe700 1 -- 172.21.15.137:0/984569825 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8011c90 con 0x7ff1b8101890 2024-04-24T06:34:58.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.655+0000 7ff1c044d700 1 -- 172.21.15.137:0/984569825 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1b81a2db0 con 0x7ff1b8101890 2024-04-24T06:34:58.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.656+0000 7ff1af7fe700 1 -- 172.21.15.137:0/984569825 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7ff1a801a400 con 0x7ff1b8101890 2024-04-24T06:34:58.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.656+0000 7ff1c044d700 1 -- 172.21.15.137:0/984569825 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1b8064150 con 0x7ff1b8101890 2024-04-24T06:34:58.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.656+0000 7ff1af7fe700 1 --2- 172.21.15.137:0/984569825 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ff1a4039ae0 0x7ff1a403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:34:58.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.656+0000 7ff1af7fe700 1 -- 172.21.15.137:0/984569825 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff1a804c610 con 0x7ff1b8101890 2024-04-24T06:34:58.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.657+0000 7ff1be1e9700 1 --2- 172.21.15.137:0/984569825 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ff1a4039ae0 0x7ff1a403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:34:58.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.658+0000 7ff1be1e9700 1 --2- 172.21.15.137:0/984569825 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ff1a4039ae0 0x7ff1a403bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4005330 tx=0x7ff1b4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:34:58.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.661+0000 7ff1af7fe700 1 -- 172.21.15.137:0/984569825 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff1a8020500 con 0x7ff1b8101890 2024-04-24T06:34:58.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.881+0000 7ff1c044d700 1 -- 172.21.15.137:0/984569825 --> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi037=y;smithi137=z", "target": ["mon-mgr", ""]}) -- 0x7ff1b8062fc0 con 0x7ff1a4039ae0 2024-04-24T06:34:58.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.895+0000 7ff1af7fe700 1 -- 172.21.15.137:0/984569825 <== mgr.14152 v2:172.21.15.5:6800/4042936965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff1b8062fc0 con 0x7ff1a4039ae0 2024-04-24T06:34:58.898 INFO:teuthology.orchestra.run.smithi137.stdout:Scheduled mgr update... 2024-04-24T06:34:58.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.897+0000 7ff1c044d700 1 -- 172.21.15.137:0/984569825 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ff1a4039ae0 msgr2=0x7ff1a403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:58.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.897+0000 7ff1c044d700 1 --2- 172.21.15.137:0/984569825 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ff1a4039ae0 0x7ff1a403bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4005330 tx=0x7ff1b4005fb0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.897+0000 7ff1c044d700 1 -- 172.21.15.137:0/984569825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1b8101890 msgr2=0x7ff1b8101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:34:58.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.897+0000 7ff1c044d700 1 --2- 172.21.15.137:0/984569825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1b8101890 0x7ff1b8101d10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff1a800b780 tx=0x7ff1a800ba90 comp rx=0 tx=0).stop 2024-04-24T06:34:58.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.897+0000 7ff1c044d700 1 -- 172.21.15.137:0/984569825 shutdown_connections 2024-04-24T06:34:58.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.897+0000 7ff1c044d700 1 --2- 172.21.15.137:0/984569825 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ff1a4039ae0 0x7ff1a403bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.897+0000 7ff1c044d700 1 --2- 172.21.15.137:0/984569825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1b8102250 0x7ff1b81a1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.897+0000 7ff1c044d700 1 --2- 172.21.15.137:0/984569825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1b8100900 0x7ff1b8103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.897+0000 7ff1c044d700 1 --2- 172.21.15.137:0/984569825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1b8101890 0x7ff1b8101d10 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:34:58.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.898+0000 7ff1c044d700 1 -- 172.21.15.137:0/984569825 >> 172.21.15.137:0/984569825 conn(0x7ff1b80fc260 msgr2=0x7ff1b80fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:34:58.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.898+0000 7ff1c044d700 1 -- 172.21.15.137:0/984569825 shutdown_connections 2024-04-24T06:34:58.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:34:58.898+0000 7ff1c044d700 1 -- 172.21.15.137:0/984569825 wait complete. 2024-04-24T06:34:59.443 DEBUG:teuthology.orchestra.run.smithi037:mgr.y> sudo journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mgr.y.service 2024-04-24T06:34:59.446 DEBUG:teuthology.orchestra.run.smithi137:mgr.z> sudo journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mgr.z.service 2024-04-24T06:34:59.449 INFO:tasks.cephadm:Deploying OSDs... 2024-04-24T06:34:59.449 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-24T06:34:59.449 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T06:34:59.510 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T06:34:59.510 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_1 2024-04-24T06:34:59.568 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T06:34:59.569 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T06:34:59.569 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 6h/6d Inode: 627 Links: 1 2024-04-24T06:34:59.569 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T06:34:59.569 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T06:34:59.569 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-04-24 06:33:55.189723688 +0000 2024-04-24T06:34:59.569 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-04-24 06:28:21.321268907 +0000 2024-04-24T06:34:59.569 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-04-24 06:28:21.321268907 +0000 2024-04-24T06:34:59.569 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-04-24 06:28:21.321268907 +0000 2024-04-24T06:34:59.569 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T06:34:59.656 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-04-24T06:34:59.656 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-04-24T06:34:59.657 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000290328 s, 1.8 MB/s 2024-04-24T06:34:59.660 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T06:34:59.717 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_2 2024-04-24T06:34:59.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:59 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:59.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:59 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:59.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:59 smithi037 ceph-mon[32864]: Reconfiguring mon.c (monmap changed)... 2024-04-24T06:34:59.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:59 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T06:34:59.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:59 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T06:34:59.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:59 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:34:59.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:59 smithi037 ceph-mon[32864]: Reconfiguring daemon mon.c on smithi137 2024-04-24T06:34:59.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:59 smithi037 ceph-mon[32864]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi037=y;smithi137=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:34:59.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:59 smithi037 ceph-mon[32864]: Saving service mgr spec with placement smithi005=x;smithi037=y;smithi137=z;count:3 2024-04-24T06:34:59.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:34:59 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:59.771 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T06:34:59.771 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T06:34:59.771 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 6h/6d Inode: 644 Links: 1 2024-04-24T06:34:59.771 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T06:34:59.771 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T06:34:59.771 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-04-24 06:33:55.246722532 +0000 2024-04-24T06:34:59.771 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-04-24 06:28:21.939256393 +0000 2024-04-24T06:34:59.771 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-04-24 06:28:21.939256393 +0000 2024-04-24T06:34:59.771 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-04-24 06:28:21.939256393 +0000 2024-04-24T06:34:59.772 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T06:34:59.829 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:59 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:59.829 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:59 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:59.830 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:59 smithi137 ceph-mon[33017]: Reconfiguring mon.c (monmap changed)... 2024-04-24T06:34:59.830 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:59 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T06:34:59.830 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:59 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T06:34:59.830 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:59 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:34:59.830 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:59 smithi137 ceph-mon[33017]: Reconfiguring daemon mon.c on smithi137 2024-04-24T06:34:59.830 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:59 smithi137 ceph-mon[33017]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi037=y;smithi137=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:34:59.830 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:59 smithi137 ceph-mon[33017]: Saving service mgr spec with placement smithi005=x;smithi037=y;smithi137=z;count:3 2024-04-24T06:34:59.830 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:34:59 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:34:59.854 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-04-24T06:34:59.854 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-04-24T06:34:59.854 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000279838 s, 1.8 MB/s 2024-04-24T06:34:59.858 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T06:34:59.916 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_3 2024-04-24T06:34:59.964 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T06:34:59.965 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T06:34:59.965 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 6h/6d Inode: 666 Links: 1 2024-04-24T06:34:59.965 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T06:34:59.965 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T06:34:59.965 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-04-24 06:33:55.307721296 +0000 2024-04-24T06:34:59.965 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-04-24 06:28:22.543244162 +0000 2024-04-24T06:34:59.965 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-04-24 06:28:22.543244162 +0000 2024-04-24T06:34:59.965 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-04-24 06:28:22.543244162 +0000 2024-04-24T06:34:59.965 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T06:35:00.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:59 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:00.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:59 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:00.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:59 smithi005 ceph-mon[27798]: Reconfiguring mon.c (monmap changed)... 2024-04-24T06:35:00.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:59 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T06:35:00.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:59 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T06:35:00.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:59 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:35:00.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:59 smithi005 ceph-mon[27798]: Reconfiguring daemon mon.c on smithi137 2024-04-24T06:35:00.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:59 smithi005 ceph-mon[27798]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi037=y;smithi137=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:35:00.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:59 smithi005 ceph-mon[27798]: Saving service mgr spec with placement smithi005=x;smithi037=y;smithi137=z;count:3 2024-04-24T06:35:00.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:34:59 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-04-24T06:35:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-04-24T06:35:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000281689 s, 1.8 MB/s 2024-04-24T06:35:00.051 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T06:35:00.155 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_4 2024-04-24T06:35:00.209 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T06:35:00.209 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T06:35:00.209 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 6h/6d Inode: 684 Links: 1 2024-04-24T06:35:00.209 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T06:35:00.209 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T06:35:00.209 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-04-24 06:33:55.376719897 +0000 2024-04-24T06:35:00.209 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-04-24 06:28:23.185231162 +0000 2024-04-24T06:35:00.209 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-04-24 06:28:23.185231162 +0000 2024-04-24T06:35:00.210 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-04-24 06:28:23.185231162 +0000 2024-04-24T06:35:00.210 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T06:35:00.286 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-04-24T06:35:00.286 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-04-24T06:35:00.286 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.00028959 s, 1.8 MB/s 2024-04-24T06:35:00.289 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T06:35:00.346 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-24T06:35:00.346 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T06:35:00.366 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T06:35:00.367 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_1 2024-04-24T06:35:00.426 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T06:35:00.426 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T06:35:00.426 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-24T06:35:00.426 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T06:35:00.426 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T06:35:00.426 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-04-24 06:34:44.920782378 +0000 2024-04-24T06:35:00.426 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-04-24 06:27:13.747498302 +0000 2024-04-24T06:35:00.426 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-04-24 06:27:13.747498302 +0000 2024-04-24T06:35:00.426 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-04-24 06:27:13.747498302 +0000 2024-04-24T06:35:00.427 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T06:35:00.494 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-24T06:35:00.494 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-24T06:35:00.494 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000234623 s, 2.2 MB/s 2024-04-24T06:35:00.495 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T06:35:00.555 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_2 2024-04-24T06:35:00.612 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T06:35:00.613 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T06:35:00.613 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-24T06:35:00.613 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T06:35:00.613 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T06:35:00.613 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-04-24 06:34:44.920782378 +0000 2024-04-24T06:35:00.613 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-04-24 06:27:14.097493502 +0000 2024-04-24T06:35:00.613 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-04-24 06:27:14.097493502 +0000 2024-04-24T06:35:00.613 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-04-24 06:27:14.097493502 +0000 2024-04-24T06:35:00.613 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T06:35:00.680 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-24T06:35:00.680 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-24T06:35:00.680 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000206733 s, 2.5 MB/s 2024-04-24T06:35:00.681 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T06:35:00.742 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_3 2024-04-24T06:35:00.799 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T06:35:00.799 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T06:35:00.799 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-24T06:35:00.799 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T06:35:00.799 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T06:35:00.799 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-04-24 06:34:44.920782378 +0000 2024-04-24T06:35:00.799 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-04-24 06:27:14.453488619 +0000 2024-04-24T06:35:00.800 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-04-24 06:27:14.453488619 +0000 2024-04-24T06:35:00.800 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-04-24 06:27:14.453488619 +0000 2024-04-24T06:35:00.800 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T06:35:00.864 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-24T06:35:00.864 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-24T06:35:00.864 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000195609 s, 2.6 MB/s 2024-04-24T06:35:00.865 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T06:35:00.922 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_4 2024-04-24T06:35:00.979 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T06:35:00.979 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T06:35:00.980 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-24T06:35:00.980 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T06:35:00.980 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T06:35:00.980 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-04-24 06:34:44.921782365 +0000 2024-04-24T06:35:00.980 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-04-24 06:27:14.809483735 +0000 2024-04-24T06:35:00.980 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-04-24 06:27:14.809483735 +0000 2024-04-24T06:35:00.980 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-04-24 06:27:14.809483735 +0000 2024-04-24T06:35:00.980 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T06:35:01.045 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-24T06:35:01.045 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-24T06:35:01.046 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000197097 s, 2.6 MB/s 2024-04-24T06:35:01.047 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T06:35:01.104 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-24T06:35:01.104 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T06:35:01.120 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T06:35:01.120 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_1 2024-04-24T06:35:01.151 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:00 smithi137 ceph-mon[33017]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:01.151 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:00 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:01.151 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:00 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:01.152 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:00 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:35:01.152 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:00 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:35:01.152 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:00 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:35:01.152 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:00 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:01.152 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:00 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T06:35:01.152 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:00 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T06:35:01.152 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:00 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T06:35:01.152 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:00 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:35:01.166 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T06:35:01.166 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T06:35:01.166 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-24T06:35:01.167 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T06:35:01.167 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T06:35:01.167 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-04-24 06:34:35.267301497 +0000 2024-04-24T06:35:01.167 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-04-24 06:27:20.782832926 +0000 2024-04-24T06:35:01.167 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-04-24 06:27:20.782832926 +0000 2024-04-24T06:35:01.167 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-04-24 06:27:20.782832926 +0000 2024-04-24T06:35:01.167 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T06:35:01.230 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-04-24T06:35:01.230 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-04-24T06:35:01.230 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000242989 s, 2.1 MB/s 2024-04-24T06:35:01.231 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T06:35:01.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:00 smithi005 ceph-mon[27798]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:01.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:00 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:01.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:00 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:01.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:00 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:35:01.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:00 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:35:01.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:00 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:35:01.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:00 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:01.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:00 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T06:35:01.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:00 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T06:35:01.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:00 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T06:35:01.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:00 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:35:01.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:00 smithi037 ceph-mon[32864]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:01.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:00 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:01.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:00 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:01.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:00 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:35:01.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:00 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:35:01.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:00 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:35:01.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:00 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:01.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:00 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T06:35:01.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:00 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T06:35:01.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:00 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T06:35:01.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:00 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:35:01.286 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_2 2024-04-24T06:35:01.340 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T06:35:01.340 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T06:35:01.340 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-24T06:35:01.340 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T06:35:01.340 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T06:35:01.341 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-04-24 06:34:35.268301478 +0000 2024-04-24T06:35:01.341 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-04-24 06:27:21.132826404 +0000 2024-04-24T06:35:01.341 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-04-24 06:27:21.132826404 +0000 2024-04-24T06:35:01.341 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-04-24 06:27:21.132826404 +0000 2024-04-24T06:35:01.341 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T06:35:01.402 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-04-24T06:35:01.402 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-04-24T06:35:01.402 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000172251 s, 3.0 MB/s 2024-04-24T06:35:01.403 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T06:35:01.458 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_3 2024-04-24T06:35:01.514 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T06:35:01.514 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T06:35:01.514 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-24T06:35:01.515 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T06:35:01.515 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T06:35:01.515 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-04-24 06:34:35.268301478 +0000 2024-04-24T06:35:01.515 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-04-24 06:27:21.468820142 +0000 2024-04-24T06:35:01.515 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-04-24 06:27:21.468820142 +0000 2024-04-24T06:35:01.515 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-04-24 06:27:21.468820142 +0000 2024-04-24T06:35:01.515 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T06:35:01.579 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-04-24T06:35:01.579 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-04-24T06:35:01.579 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000208476 s, 2.5 MB/s 2024-04-24T06:35:01.580 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T06:35:01.635 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_4 2024-04-24T06:35:01.689 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T06:35:01.689 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T06:35:01.689 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-24T06:35:01.690 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T06:35:01.690 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T06:35:01.690 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-04-24 06:34:35.269301460 +0000 2024-04-24T06:35:01.690 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-04-24 06:27:21.806813843 +0000 2024-04-24T06:35:01.690 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-04-24 06:27:21.806813843 +0000 2024-04-24T06:35:01.690 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-04-24 06:27:21.806813843 +0000 2024-04-24T06:35:01.690 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T06:35:01.752 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-04-24T06:35:01.752 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-04-24T06:35:01.752 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000192417 s, 2.7 MB/s 2024-04-24T06:35:01.753 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T06:35:01.810 INFO:tasks.cephadm:Deploying osd.0 on smithi005 with /dev/vg_nvme/lv_4... 2024-04-24T06:35:01.810 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-24T06:35:02.171 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:01 smithi037 ceph-mon[32864]: Deploying daemon mgr.z on smithi137 2024-04-24T06:35:02.174 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:01 smithi137 ceph-mon[33017]: Deploying daemon mgr.z on smithi137 2024-04-24T06:35:02.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:01 smithi005 ceph-mon[27798]: Deploying daemon mgr.z on smithi137 2024-04-24T06:35:02.278 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:35:02.446 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:35:02 smithi137 systemd[1]: Starting Ceph mgr.z for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:35:02.838 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:35:02 smithi137 podman[34870]: 2024-04-24 06:35:02.458056529 +0000 UTC m=+0.017839441 image pull b5c3183062a6ce19f361681cc58de6e3ac1de1a54c80e6094ec5695237cd98d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b 2024-04-24T06:35:02.838 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:35:02 smithi137 podman[34870]: 2024-04-24 06:35:02.57393347 +0000 UTC m=+0.133716384 container create 1b2a03d1f584c2d2fdeb021d51c5faabd6730240e0e1d7a991189a3f2ff8e9e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-24T06:35:03.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:02 smithi005 ceph-mon[27798]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:03.093 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:02 smithi137 ceph-mon[33017]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:03.094 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:35:02 smithi137 podman[34870]: 2024-04-24 06:35:02.934449572 +0000 UTC m=+0.494232483 container init 1b2a03d1f584c2d2fdeb021d51c5faabd6730240e0e1d7a991189a3f2ff8e9e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.license=GPLv2) 2024-04-24T06:35:03.094 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:35:02 smithi137 podman[34870]: 2024-04-24 06:35:02.93695471 +0000 UTC m=+0.496737621 container start 1b2a03d1f584c2d2fdeb021d51c5faabd6730240e0e1d7a991189a3f2ff8e9e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0) 2024-04-24T06:35:03.094 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:35:02 smithi137 ceph-mgr[34889]: -- 172.21.15.137:0/3918017587 <== mon.1 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b70f16c5a0 con 0x55b70f140800 2024-04-24T06:35:03.094 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:35:03 smithi137 bash[34870]: 1b2a03d1f584c2d2fdeb021d51c5faabd6730240e0e1d7a991189a3f2ff8e9e5 2024-04-24T06:35:03.094 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:35:03 smithi137 systemd[1]: Started Ceph mgr.z for 43e9bad0-0204-11ef-bc93-c7b262605968. 2024-04-24T06:35:03.094 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:35:03 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:35:03.092+0000 7f6a0745c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T06:35:03.186 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:02 smithi037 ceph-mon[32864]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:03.373 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:35:03 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:35:03.273+0000 7f6a0745c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T06:35:03.373 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:35:03 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:35:03.371+0000 7f6a0745c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T06:35:03.946 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:35:03 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:35:03.673+0000 7f6a0745c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T06:35:03.946 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:35:03 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:35:03.757+0000 7f6a0745c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T06:35:04.123 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:04 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:04.123 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:04 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:04.123 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:04 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:04.123 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:04 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T06:35:04.123 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:04 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T06:35:04.124 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:04 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T06:35:04.124 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:04 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:35:04.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:04 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:04.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:04 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:04.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:04 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:04.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:04 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T06:35:04.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:04 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T06:35:04.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:04 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T06:35:04.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:04 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:35:04.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:04 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:04.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:04 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:04.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:04 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:04.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:04 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T06:35:04.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:04 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T06:35:04.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:04 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T06:35:04.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:04 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:35:05.045 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-24T06:35:05.045 INFO:teuthology.orchestra.run.smithi005.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-24T06:35:05.045 INFO:teuthology.orchestra.run.smithi005.stderr: stderr: 10+0 records in 2024-04-24T06:35:05.045 INFO:teuthology.orchestra.run.smithi005.stderr:10+0 records out 2024-04-24T06:35:05.045 INFO:teuthology.orchestra.run.smithi005.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0374419 s, 280 MB/s 2024-04-24T06:35:05.045 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping successful for: 2024-04-24T06:35:05.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:05 smithi137 ceph-mon[33017]: Deploying daemon mgr.y on smithi037 2024-04-24T06:35:05.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:05 smithi137 ceph-mon[33017]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:05.197 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:35:04 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:35:04.934+0000 7f6a0745c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T06:35:05.197 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:35:05 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:35:05.098+0000 7f6a0745c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T06:35:05.344 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:05 smithi037 ceph-mon[32864]: Deploying daemon mgr.y on smithi037 2024-04-24T06:35:05.344 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:05 smithi037 ceph-mon[32864]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:05.345 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:35:05 smithi037 systemd[1]: Starting Ceph mgr.y for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:35:05.450 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:05 smithi005 ceph-mon[27798]: Deploying daemon mgr.y on smithi037 2024-04-24T06:35:05.451 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:05 smithi005 ceph-mon[27798]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:05.547 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:35:05 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:35:05.266+0000 7f6a0745c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T06:35:05.547 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:35:05 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:35:05.545+0000 7f6a0745c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T06:35:05.656 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:35:05 smithi037 podman[34244]: 2024-04-24 06:35:05.246628413 +0000 UTC m=+0.018113440 image pull b5c3183062a6ce19f361681cc58de6e3ac1de1a54c80e6094ec5695237cd98d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b 2024-04-24T06:35:05.657 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:35:05 smithi037 podman[34244]: 2024-04-24 06:35:05.344233261 +0000 UTC m=+0.115718282 container create 9a3b3e8361ba8db73b3abfdf55057c37b36ccc670d8421b7344c1b87744b7370 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-24T06:35:05.657 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:35:05 smithi037 podman[34244]: 2024-04-24 06:35:05.557133472 +0000 UTC m=+0.328618487 container init 9a3b3e8361ba8db73b3abfdf55057c37b36ccc670d8421b7344c1b87744b7370 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T06:35:05.657 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:35:05 smithi037 podman[34244]: 2024-04-24 06:35:05.560214073 +0000 UTC m=+0.331699086 container start 9a3b3e8361ba8db73b3abfdf55057c37b36ccc670d8421b7344c1b87744b7370 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T06:35:05.657 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:35:05 smithi037 ceph-mgr[34263]: -- 172.21.15.37:0/562492706 <== mon.2 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dd1ba985a0 con 0x55dd1ba6cc00 2024-04-24T06:35:05.657 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:35:05 smithi037 bash[34244]: 9a3b3e8361ba8db73b3abfdf55057c37b36ccc670d8421b7344c1b87744b7370 2024-04-24T06:35:05.823 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi005:vg_nvme/lv_4 2024-04-24T06:35:05.838 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:35:05 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:35:05.635+0000 7f6a0745c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T06:35:05.838 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:35:05 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:35:05.837+0000 7f6a0745c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T06:35:05.915 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:35:05 smithi037 systemd[1]: Started Ceph mgr.y for 43e9bad0-0204-11ef-bc93-c7b262605968. 2024-04-24T06:35:05.916 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:35:05 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:35:05.721+0000 7f97fefe2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T06:35:05.916 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:35:05 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:35:05.914+0000 7f97fefe2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T06:35:06.196 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:35:06 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:35:06.026+0000 7f6a0745c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T06:35:06.256 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:35:06.268 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:35:06 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:35:06.021+0000 7f97fefe2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T06:35:06.735 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:35:06 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:35:06.353+0000 7f97fefe2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T06:35:06.735 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:35:06 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:35:06.443+0000 7f97fefe2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T06:35:06.946 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:35:06 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:35:06.660+0000 7f6a0745c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T06:35:06.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:06 smithi137 ceph-mon[33017]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:06.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:06 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:06.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:06 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:06.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:06 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:06.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:06 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:06.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:06 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:35:06.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:06 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:07.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:06 smithi005 ceph-mon[27798]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:07.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:06 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:07.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:06 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:07.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:06 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:07.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:06 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:07.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:06 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:35:07.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:06 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:07.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:06 smithi037 ceph-mon[32864]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:07.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:06 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:07.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:06 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:07.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:06 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:07.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:06 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:07.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:06 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:35:07.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:06 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:07.402 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:35:07 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:35:07.092+0000 7f6a0745c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T06:35:07.402 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:35:07 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:35:07.188+0000 7f6a0745c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T06:35:07.402 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:35:07 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:35:07.400+0000 7f6a0745c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T06:35:08.126 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:35:07 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:35:07.773+0000 7f97fefe2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T06:35:08.127 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:35:07 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:35:07.949+0000 7f97fefe2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T06:35:08.127 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:35:08 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:35:08.125+0000 7f97fefe2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T06:35:08.196 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:35:07 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:35:07.927+0000 7f6a0745c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T06:35:08.196 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:35:08 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:35:08.018+0000 7f6a0745c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T06:35:08.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.366+0000 7effef000700 1 -- 172.21.15.5:0/3393998619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7effe8108840 msgr2=0x7effe810acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:08.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.366+0000 7effef000700 1 --2- 172.21.15.5:0/3393998619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7effe8108840 0x7effe810acb0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7effe0009990 tx=0x7effe0009ca0 comp rx=0 tx=0).stop 2024-04-24T06:35:08.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.366+0000 7effef000700 1 -- 172.21.15.5:0/3393998619 shutdown_connections 2024-04-24T06:35:08.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.366+0000 7effef000700 1 --2- 172.21.15.5:0/3393998619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7effe8108840 0x7effe810acb0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:08.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.366+0000 7effef000700 1 --2- 172.21.15.5:0/3393998619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effe81035e0 0x7effe81059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:08.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.366+0000 7effef000700 1 --2- 172.21.15.5:0/3393998619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effe8105f10 0x7effe8108300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:08.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.366+0000 7effef000700 1 -- 172.21.15.5:0/3393998619 >> 172.21.15.5:0/3393998619 conn(0x7effe80fcff0 msgr2=0x7effe80ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:08.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.366+0000 7effef000700 1 -- 172.21.15.5:0/3393998619 shutdown_connections 2024-04-24T06:35:08.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.366+0000 7effef000700 1 -- 172.21.15.5:0/3393998619 wait complete. 2024-04-24T06:35:08.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.367+0000 7effef000700 1 Processor -- start 2024-04-24T06:35:08.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.368+0000 7effef000700 1 -- start start 2024-04-24T06:35:08.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.368+0000 7effef000700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effe81035e0 0x7effe8198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:08.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.368+0000 7effef000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effe8105f10 0x7effe81992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:08.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.368+0000 7effef000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7effe8108840 0x7effe81a0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:08.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.368+0000 7effef000700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7effe810e750 con 0x7effe81035e0 2024-04-24T06:35:08.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.368+0000 7effef000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7effe810e5d0 con 0x7effe8105f10 2024-04-24T06:35:08.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.368+0000 7effef000700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7effe810e8d0 con 0x7effe8108840 2024-04-24T06:35:08.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.368+0000 7effe7e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effe81035e0 0x7effe8198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:08.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.368+0000 7effe7400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effe8105f10 0x7effe81992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:08.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.368+0000 7effe7400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effe8105f10 0x7effe81992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:39584/0 (socket says 172.21.15.5:39584) 2024-04-24T06:35:08.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.368+0000 7effe7e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effe81035e0 0x7effe8198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49700/0 (socket says 172.21.15.5:49700) 2024-04-24T06:35:08.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.368+0000 7effe7400700 1 -- 172.21.15.5:0/3280887254 learned_addr learned my addr 172.21.15.5:0/3280887254 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:35:08.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.368+0000 7effed000700 1 --2- 172.21.15.5:0/3280887254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7effe8108840 0x7effe81a0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:08.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.369+0000 7effe7400700 1 -- 172.21.15.5:0/3280887254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7effe8108840 msgr2=0x7effe81a0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:08.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.369+0000 7effe7400700 1 --2- 172.21.15.5:0/3280887254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7effe8108840 0x7effe81a0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:08.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.369+0000 7effe7400700 1 -- 172.21.15.5:0/3280887254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effe81035e0 msgr2=0x7effe8198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:08.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.369+0000 7effe7400700 1 --2- 172.21.15.5:0/3280887254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effe81035e0 0x7effe8198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:08.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.369+0000 7effe7400700 1 -- 172.21.15.5:0/3280887254 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe81013d0 con 0x7effe8105f10 2024-04-24T06:35:08.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.369+0000 7effe7400700 1 --2- 172.21.15.5:0/3280887254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effe8105f10 0x7effe81992f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7effd400e9a0 tx=0x7effd400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:08.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.369+0000 7effe4c00700 1 -- 172.21.15.5:0/3280887254 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd400ca10 con 0x7effe8105f10 2024-04-24T06:35:08.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.369+0000 7effe4c00700 1 -- 172.21.15.5:0/3280887254 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effd400cbb0 con 0x7effe8105f10 2024-04-24T06:35:08.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.369+0000 7effef000700 1 -- 172.21.15.5:0/3280887254 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effe81016c0 con 0x7effe8105f10 2024-04-24T06:35:08.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.370+0000 7effe4c00700 1 -- 172.21.15.5:0/3280887254 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd4018680 con 0x7effe8105f10 2024-04-24T06:35:08.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.370+0000 7effef000700 1 -- 172.21.15.5:0/3280887254 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effe8101c00 con 0x7effe8105f10 2024-04-24T06:35:08.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.371+0000 7effef000700 1 -- 172.21.15.5:0/3280887254 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effe80697e0 con 0x7effe8105f10 2024-04-24T06:35:08.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.371+0000 7effe4c00700 1 -- 172.21.15.5:0/3280887254 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7effd4018820 con 0x7effe8105f10 2024-04-24T06:35:08.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.371+0000 7effe4c00700 1 --2- 172.21.15.5:0/3280887254 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7effcc039890 0x7effcc03bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:08.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.371+0000 7effe4c00700 1 -- 172.21.15.5:0/3280887254 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7effd404ccd0 con 0x7effe8105f10 2024-04-24T06:35:08.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.372+0000 7effe7e00700 1 --2- 172.21.15.5:0/3280887254 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7effcc039890 0x7effcc03bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:08.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.372+0000 7effe7e00700 1 --2- 172.21.15.5:0/3280887254 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7effcc039890 0x7effcc03bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7effe819a290 tx=0x7effd800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:08.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.382+0000 7effe4c00700 1 -- 172.21.15.5:0/3280887254 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7effd40071c0 con 0x7effe8105f10 2024-04-24T06:35:08.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:08 smithi005 ceph-mon[27798]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:08.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:08 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:08.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:08 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:08.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:08 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:08.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:08 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:08.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:08 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:35:08.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:08 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:35:08.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:08 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:08.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:08 smithi005 ceph-mon[27798]: Reconfiguring mgr.x (unknown last config time)... 2024-04-24T06:35:08.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:08 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T06:35:08.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:08 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T06:35:08.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:08 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:35:08.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:08 smithi005 ceph-mon[27798]: Reconfiguring daemon mgr.x on smithi005 2024-04-24T06:35:08.600 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:08 smithi137 ceph-mon[33017]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:08.600 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:08 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:08.600 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:08 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:08.600 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:08 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:08.600 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:08 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:08.600 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:08 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:35:08.600 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:08 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:35:08.601 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:08 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:08.601 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:08 smithi137 ceph-mon[33017]: Reconfiguring mgr.x (unknown last config time)... 2024-04-24T06:35:08.601 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:08 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T06:35:08.601 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:08 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T06:35:08.601 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:08 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:35:08.601 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:08 smithi137 ceph-mon[33017]: Reconfiguring daemon mgr.x on smithi005 2024-04-24T06:35:08.601 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:35:08 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:35:08.258+0000 7f6a0745c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T06:35:08.601 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:35:08 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:35:08.389+0000 7f6a0745c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T06:35:08.601 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:35:08 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:35:08.484+0000 7f6a0745c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T06:35:08.601 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:35:08 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:35:08.598+0000 7f6a0745c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T06:35:08.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:08.637+0000 7effef000700 1 -- 172.21.15.5:0/3280887254 --> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7effe80fe6d0 con 0x7effcc039890 2024-04-24T06:35:08.735 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:35:08 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:35:08.429+0000 7f97fefe2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T06:35:08.735 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:35:08 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:35:08.524+0000 7f97fefe2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T06:35:08.735 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:35:08 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:35:08.734+0000 7f97fefe2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T06:35:08.736 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:08 smithi037 ceph-mon[32864]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:08.736 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:08 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:08.736 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:08 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:08.736 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:08 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:08.736 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:08 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:08.736 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:08 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:35:08.736 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:08 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:35:08.736 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:08 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:08.736 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:08 smithi037 ceph-mon[32864]: Reconfiguring mgr.x (unknown last config time)... 2024-04-24T06:35:08.736 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:08 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T06:35:08.736 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:08 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T06:35:08.737 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:08 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:35:08.737 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:08 smithi037 ceph-mon[32864]: Reconfiguring daemon mgr.x on smithi005 2024-04-24T06:35:09.018 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:35:08 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:35:08.933+0000 7f97fefe2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T06:35:09.196 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:35:08 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:35:08.931+0000 7f6a0745c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T06:35:09.645 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:09 smithi005 ceph-mon[27798]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:35:09.645 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:09 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T06:35:09.645 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:09 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T06:35:09.645 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:09 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:35:09.696 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:35:09 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:35:09.528+0000 7f6a0745c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T06:35:09.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:09 smithi137 ceph-mon[33017]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:35:09.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:09 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T06:35:09.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:09 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T06:35:09.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:09 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:35:09.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:09 smithi037 ceph-mon[32864]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:35:09.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:09 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T06:35:09.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:09 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T06:35:09.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:09 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:35:09.769 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:35:09 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:35:09.561+0000 7f97fefe2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T06:35:10.196 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:35:09 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:35:09.741+0000 7f6a0745c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T06:35:10.268 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:35:10 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:35:10.001+0000 7f97fefe2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T06:35:10.269 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:35:10 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:35:10.101+0000 7f97fefe2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T06:35:10.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:10.412+0000 7effe4c00700 1 -- 172.21.15.5:0/3280887254 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 14) ==== 92601+0+0 (secure 0 0 0) 0x7effd40178f0 con 0x7effe8105f10 2024-04-24T06:35:10.666 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:10 smithi005 ceph-mon[27798]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:10.666 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:10 smithi005 ceph-mon[27798]: Standby manager daemon z started 2024-04-24T06:35:10.666 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:10 smithi005 ceph-mon[27798]: from='mgr.? 172.21.15.137:0/3899501177' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T06:35:10.666 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:10 smithi005 ceph-mon[27798]: from='mgr.? 172.21.15.137:0/3899501177' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T06:35:10.666 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:10 smithi005 ceph-mon[27798]: from='mgr.? 172.21.15.137:0/3899501177' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T06:35:10.666 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:10 smithi005 ceph-mon[27798]: from='mgr.? 172.21.15.137:0/3899501177' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T06:35:10.666 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:10 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:10.666 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:10 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:10.667 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:10 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:35:10.667 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:10 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:35:10.667 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:10 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:35:10.667 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:10 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:10.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:10 smithi137 ceph-mon[33017]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:10.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:10 smithi137 ceph-mon[33017]: Standby manager daemon z started 2024-04-24T06:35:10.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:10 smithi137 ceph-mon[33017]: from='mgr.? 172.21.15.137:0/3899501177' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T06:35:10.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:10 smithi137 ceph-mon[33017]: from='mgr.? 172.21.15.137:0/3899501177' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T06:35:10.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:10 smithi137 ceph-mon[33017]: from='mgr.? 172.21.15.137:0/3899501177' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T06:35:10.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:10 smithi137 ceph-mon[33017]: from='mgr.? 172.21.15.137:0/3899501177' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T06:35:10.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:10 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:10.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:10 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:10.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:10 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:35:10.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:10 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:35:10.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:10 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:35:10.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:10 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:10.768 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:35:10 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:35:10.310+0000 7f97fefe2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T06:35:10.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:10 smithi037 ceph-mon[32864]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:10.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:10 smithi037 ceph-mon[32864]: Standby manager daemon z started 2024-04-24T06:35:10.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:10 smithi037 ceph-mon[32864]: from='mgr.? 172.21.15.137:0/3899501177' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T06:35:10.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:10 smithi037 ceph-mon[32864]: from='mgr.? 172.21.15.137:0/3899501177' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T06:35:10.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:10 smithi037 ceph-mon[32864]: from='mgr.? 172.21.15.137:0/3899501177' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T06:35:10.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:10 smithi037 ceph-mon[32864]: from='mgr.? 172.21.15.137:0/3899501177' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T06:35:10.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:10 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:10.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:10 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:10.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:10 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:35:10.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:10 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:35:10.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:10 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:35:10.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:10 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:11.206 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:35:10 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:35:10.856+0000 7f97fefe2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T06:35:11.206 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:35:10 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:35:10.950+0000 7f97fefe2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T06:35:11.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:11 smithi037 ceph-mon[32864]: mgrmap e14: x(active, since 89s), standbys: z 2024-04-24T06:35:11.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:11 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T06:35:11.519 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:35:11 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:35:11.204+0000 7f97fefe2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T06:35:11.519 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:35:11 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:35:11.342+0000 7f97fefe2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T06:35:11.519 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:35:11 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:35:11.443+0000 7f97fefe2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T06:35:11.547 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:11 smithi005 ceph-mon[27798]: mgrmap e14: x(active, since 89s), standbys: z 2024-04-24T06:35:11.548 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:11 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T06:35:11.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:11 smithi137 ceph-mon[33017]: mgrmap e14: x(active, since 89s), standbys: z 2024-04-24T06:35:11.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:11 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T06:35:11.891 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:35:11 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:35:11.562+0000 7f97fefe2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T06:35:11.891 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:35:11 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:35:11.889+0000 7f97fefe2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T06:35:12.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:12 smithi137 ceph-mon[33017]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:12.729 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:12 smithi005 ceph-mon[27798]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:12.744 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:12 smithi037 ceph-mon[32864]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:12.744 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:35:12 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:35:12.519+0000 7f97fefe2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T06:35:12.744 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:35:12 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:35:12.743+0000 7f97fefe2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T06:35:13.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:13.426+0000 7effe4c00700 1 -- 172.21.15.5:0/3280887254 <== mon.2 v2:172.21.15.37:3300/0 8 ==== mgrmap(e 15) ==== 138589+0+0 (secure 0 0 0) 0x7effd4025850 con 0x7effe8105f10 2024-04-24T06:35:13.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:13 smithi137 ceph-mon[33017]: Standby manager daemon y started 2024-04-24T06:35:13.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:13 smithi137 ceph-mon[33017]: from='mgr.? 172.21.15.37:0/2970245939' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T06:35:13.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:13 smithi137 ceph-mon[33017]: from='mgr.? 172.21.15.37:0/2970245939' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T06:35:13.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:13 smithi137 ceph-mon[33017]: from='mgr.? 172.21.15.37:0/2970245939' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T06:35:13.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:13 smithi137 ceph-mon[33017]: from='mgr.? 172.21.15.37:0/2970245939' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T06:35:13.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:13 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/2665897696' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c2e974c-745c-4dd5-8c3d-8e8da2eb1ce0"}]: dispatch 2024-04-24T06:35:13.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:13 smithi137 ceph-mon[33017]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c2e974c-745c-4dd5-8c3d-8e8da2eb1ce0"}]: dispatch 2024-04-24T06:35:13.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:13 smithi137 ceph-mon[33017]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c2e974c-745c-4dd5-8c3d-8e8da2eb1ce0"}]': finished 2024-04-24T06:35:13.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:13 smithi137 ceph-mon[33017]: osdmap e5: 1 total, 0 up, 1 in 2024-04-24T06:35:13.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:13 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T06:35:13.727 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:13 smithi005 ceph-mon[27798]: Standby manager daemon y started 2024-04-24T06:35:13.727 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:13 smithi005 ceph-mon[27798]: from='mgr.? 172.21.15.37:0/2970245939' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T06:35:13.728 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:13 smithi005 ceph-mon[27798]: from='mgr.? 172.21.15.37:0/2970245939' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T06:35:13.728 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:13 smithi005 ceph-mon[27798]: from='mgr.? 172.21.15.37:0/2970245939' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T06:35:13.728 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:13 smithi005 ceph-mon[27798]: from='mgr.? 172.21.15.37:0/2970245939' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T06:35:13.728 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:13 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/2665897696' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c2e974c-745c-4dd5-8c3d-8e8da2eb1ce0"}]: dispatch 2024-04-24T06:35:13.728 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:13 smithi005 ceph-mon[27798]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c2e974c-745c-4dd5-8c3d-8e8da2eb1ce0"}]: dispatch 2024-04-24T06:35:13.728 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:13 smithi005 ceph-mon[27798]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c2e974c-745c-4dd5-8c3d-8e8da2eb1ce0"}]': finished 2024-04-24T06:35:13.728 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:13 smithi005 ceph-mon[27798]: osdmap e5: 1 total, 0 up, 1 in 2024-04-24T06:35:13.728 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:13 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T06:35:13.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:13 smithi037 ceph-mon[32864]: Standby manager daemon y started 2024-04-24T06:35:13.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:13 smithi037 ceph-mon[32864]: from='mgr.? 172.21.15.37:0/2970245939' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T06:35:13.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:13 smithi037 ceph-mon[32864]: from='mgr.? 172.21.15.37:0/2970245939' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T06:35:13.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:13 smithi037 ceph-mon[32864]: from='mgr.? 172.21.15.37:0/2970245939' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T06:35:13.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:13 smithi037 ceph-mon[32864]: from='mgr.? 172.21.15.37:0/2970245939' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T06:35:13.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:13 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/2665897696' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c2e974c-745c-4dd5-8c3d-8e8da2eb1ce0"}]: dispatch 2024-04-24T06:35:13.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:13 smithi037 ceph-mon[32864]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c2e974c-745c-4dd5-8c3d-8e8da2eb1ce0"}]: dispatch 2024-04-24T06:35:13.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:13 smithi037 ceph-mon[32864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c2e974c-745c-4dd5-8c3d-8e8da2eb1ce0"}]': finished 2024-04-24T06:35:13.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:13 smithi037 ceph-mon[32864]: osdmap e5: 1 total, 0 up, 1 in 2024-04-24T06:35:13.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:13 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T06:35:14.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:14 smithi137 ceph-mon[33017]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:14.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:14 smithi137 ceph-mon[33017]: mgrmap e15: x(active, since 92s), standbys: y, z 2024-04-24T06:35:14.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:14 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T06:35:14.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:14 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/4084597215' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T06:35:14.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:14 smithi005 ceph-mon[27798]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:14.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:14 smithi005 ceph-mon[27798]: mgrmap e15: x(active, since 92s), standbys: y, z 2024-04-24T06:35:14.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:14 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T06:35:14.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:14 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/4084597215' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T06:35:14.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:14 smithi037 ceph-mon[32864]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:14.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:14 smithi037 ceph-mon[32864]: mgrmap e15: x(active, since 92s), standbys: y, z 2024-04-24T06:35:14.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:14 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T06:35:14.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:14 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/4084597215' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T06:35:16.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:16 smithi005 ceph-mon[27798]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:16.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:16 smithi037 ceph-mon[32864]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:16.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:16 smithi137 ceph-mon[33017]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:18.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:18 smithi005 ceph-mon[27798]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:18.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:18 smithi037 ceph-mon[32864]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:18.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:18 smithi137 ceph-mon[33017]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:20.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:20 smithi005 ceph-mon[27798]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:20.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:20 smithi037 ceph-mon[32864]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:20.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:20 smithi137 ceph-mon[33017]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:22.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:22 smithi005 ceph-mon[27798]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:22.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:22 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T06:35:22.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:22 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:35:22.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:22 smithi005 ceph-mon[27798]: Deploying daemon osd.0 on smithi005 2024-04-24T06:35:22.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:22 smithi037 ceph-mon[32864]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:22.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:22 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T06:35:22.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:22 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:35:22.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:22 smithi037 ceph-mon[32864]: Deploying daemon osd.0 on smithi005 2024-04-24T06:35:22.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:22 smithi137 ceph-mon[33017]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:22.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:22 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T06:35:22.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:22 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:35:22.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:22 smithi137 ceph-mon[33017]: Deploying daemon osd.0 on smithi005 2024-04-24T06:35:24.748 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:24 smithi137 ceph-mon[33017]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:24.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:24 smithi005 ceph-mon[27798]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:24.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:24 smithi037 ceph-mon[32864]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:26.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:26 smithi005 ceph-mon[27798]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:26.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:26 smithi037 ceph-mon[32864]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:26.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:26 smithi137 ceph-mon[33017]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:28.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:28 smithi005 ceph-mon[27798]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:28.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:28 smithi037 ceph-mon[32864]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:28.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:28 smithi137 ceph-mon[33017]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:30.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:30 smithi005 ceph-mon[27798]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:30.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:30 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:30.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:30 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:30.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:30 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:35:30.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:30 smithi037 ceph-mon[32864]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:30.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:30 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:30.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:30 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:30.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:30 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:35:30.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:30 smithi137 ceph-mon[33017]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:30.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:30 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:30.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:30 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:30.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:30 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:35:32.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:32 smithi005 ceph-mon[27798]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:32.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:32 smithi037 ceph-mon[32864]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:32.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:32 smithi137 ceph-mon[33017]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:35.102 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:34 smithi005 ceph-mon[27798]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:35.102 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:34 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:35.102 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:34 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:35.102 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:34 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:35:35.102 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:34 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:35:35.102 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:34 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:35.102 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:34 smithi005 ceph-mon[27798]: from='osd.0 [v2:172.21.15.5:6802/2155858515,v1:172.21.15.5:6803/2155858515]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T06:35:35.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:34 smithi137 ceph-mon[33017]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:35.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:34 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:35.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:34 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:35.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:34 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:35:35.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:34 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:35:35.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:34 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:35.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:34 smithi137 ceph-mon[33017]: from='osd.0 [v2:172.21.15.5:6802/2155858515,v1:172.21.15.5:6803/2155858515]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T06:35:35.232 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 0 on host 'smithi005' 2024-04-24T06:35:35.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:35.230+0000 7effe4c00700 1 -- 172.21.15.5:0/3280887254 <== mgr.14152 v2:172.21.15.5:6800/4042936965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7effe80fe6d0 con 0x7effcc039890 2024-04-24T06:35:35.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:35.234+0000 7effef000700 1 -- 172.21.15.5:0/3280887254 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7effcc039890 msgr2=0x7effcc03bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:35.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:35.234+0000 7effef000700 1 --2- 172.21.15.5:0/3280887254 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7effcc039890 0x7effcc03bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7effe819a290 tx=0x7effd800b040 comp rx=0 tx=0).stop 2024-04-24T06:35:35.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:35.234+0000 7effef000700 1 -- 172.21.15.5:0/3280887254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effe8105f10 msgr2=0x7effe81992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:35.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:35.234+0000 7effef000700 1 --2- 172.21.15.5:0/3280887254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effe8105f10 0x7effe81992f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7effd400e9a0 tx=0x7effd400ecb0 comp rx=0 tx=0).stop 2024-04-24T06:35:35.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:35.234+0000 7effef000700 1 -- 172.21.15.5:0/3280887254 shutdown_connections 2024-04-24T06:35:35.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:35.234+0000 7effef000700 1 --2- 172.21.15.5:0/3280887254 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7effcc039890 0x7effcc03bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:35.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:35.234+0000 7effef000700 1 --2- 172.21.15.5:0/3280887254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7effe8108840 0x7effe81a0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:35.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:35.234+0000 7effef000700 1 --2- 172.21.15.5:0/3280887254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effe81035e0 0x7effe8198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:35.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:35.234+0000 7effef000700 1 --2- 172.21.15.5:0/3280887254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effe8105f10 0x7effe81992f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:35.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:35.234+0000 7effef000700 1 -- 172.21.15.5:0/3280887254 >> 172.21.15.5:0/3280887254 conn(0x7effe80fcff0 msgr2=0x7effe8103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:35.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:35.234+0000 7effef000700 1 -- 172.21.15.5:0/3280887254 shutdown_connections 2024-04-24T06:35:35.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:35:35.234+0000 7effef000700 1 -- 172.21.15.5:0/3280887254 wait complete. 2024-04-24T06:35:35.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:34 smithi037 ceph-mon[32864]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:35.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:34 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:35.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:34 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:35.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:34 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:35:35.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:34 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:35:35.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:34 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:35.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:34 smithi037 ceph-mon[32864]: from='osd.0 [v2:172.21.15.5:6802/2155858515,v1:172.21.15.5:6803/2155858515]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T06:35:35.931 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:35 smithi005 ceph-mon[27798]: from='osd.0 [v2:172.21.15.5:6802/2155858515,v1:172.21.15.5:6803/2155858515]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T06:35:35.931 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:35 smithi005 ceph-mon[27798]: osdmap e6: 1 total, 0 up, 1 in 2024-04-24T06:35:35.931 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:35 smithi005 ceph-mon[27798]: from='osd.0 [v2:172.21.15.5:6802/2155858515,v1:172.21.15.5:6803/2155858515]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-04-24T06:35:35.931 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:35 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T06:35:35.931 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:35 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:35.931 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:35 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:35:35.931 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:35 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:35.997 DEBUG:teuthology.orchestra.run.smithi005:osd.0> sudo journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@osd.0.service 2024-04-24T06:35:36.002 INFO:tasks.cephadm:Deploying osd.1 on smithi037 with /dev/vg_nvme/lv_4... 2024-04-24T06:35:36.002 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-24T06:35:36.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:35 smithi137 ceph-mon[33017]: from='osd.0 [v2:172.21.15.5:6802/2155858515,v1:172.21.15.5:6803/2155858515]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T06:35:36.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:35 smithi137 ceph-mon[33017]: osdmap e6: 1 total, 0 up, 1 in 2024-04-24T06:35:36.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:35 smithi137 ceph-mon[33017]: from='osd.0 [v2:172.21.15.5:6802/2155858515,v1:172.21.15.5:6803/2155858515]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-04-24T06:35:36.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:35 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T06:35:36.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:35 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:36.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:35 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:35:36.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:35 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:36.256 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T06:35:36.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:35 smithi037 ceph-mon[32864]: from='osd.0 [v2:172.21.15.5:6802/2155858515,v1:172.21.15.5:6803/2155858515]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T06:35:36.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:35 smithi037 ceph-mon[32864]: osdmap e6: 1 total, 0 up, 1 in 2024-04-24T06:35:36.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:35 smithi037 ceph-mon[32864]: from='osd.0 [v2:172.21.15.5:6802/2155858515,v1:172.21.15.5:6803/2155858515]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-04-24T06:35:36.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:35 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T06:35:36.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:35 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:36.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:35 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:35:36.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:35 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:36.943 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:36 smithi005 ceph-mon[27798]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:36.943 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:36 smithi005 ceph-mon[27798]: from='osd.0 [v2:172.21.15.5:6802/2155858515,v1:172.21.15.5:6803/2155858515]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-04-24T06:35:36.943 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:36 smithi005 ceph-mon[27798]: osdmap e7: 1 total, 0 up, 1 in 2024-04-24T06:35:36.943 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:36 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T06:35:36.943 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:36 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T06:35:36.944 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 24 06:35:36 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-0[39877]: 2024-04-24T06:35:36.918+0000 7f9fa6e00700 -1 osd.0 0 waiting for initial osdmap 2024-04-24T06:35:36.944 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 24 06:35:36 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-0[39877]: 2024-04-24T06:35:36.940+0000 7f9f9cc00700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T06:35:37.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:36 smithi037 ceph-mon[32864]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:37.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:36 smithi037 ceph-mon[32864]: from='osd.0 [v2:172.21.15.5:6802/2155858515,v1:172.21.15.5:6803/2155858515]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-04-24T06:35:37.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:36 smithi037 ceph-mon[32864]: osdmap e7: 1 total, 0 up, 1 in 2024-04-24T06:35:37.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:36 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T06:35:37.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:36 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T06:35:37.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:36 smithi137 ceph-mon[33017]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:37.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:36 smithi137 ceph-mon[33017]: from='osd.0 [v2:172.21.15.5:6802/2155858515,v1:172.21.15.5:6803/2155858515]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-04-24T06:35:37.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:36 smithi137 ceph-mon[33017]: osdmap e7: 1 total, 0 up, 1 in 2024-04-24T06:35:37.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T06:35:37.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T06:35:38.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:37 smithi005 ceph-mon[27798]: purged_snaps scrub starts 2024-04-24T06:35:38.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:37 smithi005 ceph-mon[27798]: purged_snaps scrub ok 2024-04-24T06:35:38.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:37 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T06:35:38.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:37 smithi005 ceph-mon[27798]: from='osd.0 [v2:172.21.15.5:6802/2155858515,v1:172.21.15.5:6803/2155858515]' entity='osd.0' 2024-04-24T06:35:38.167 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-24T06:35:38.168 INFO:teuthology.orchestra.run.smithi037.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-24T06:35:38.168 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10+0 records in 2024-04-24T06:35:38.168 INFO:teuthology.orchestra.run.smithi037.stderr:10+0 records out 2024-04-24T06:35:38.168 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112343 s, 933 MB/s 2024-04-24T06:35:38.168 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping successful for: 2024-04-24T06:35:38.169 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:37 smithi037 ceph-mon[32864]: purged_snaps scrub starts 2024-04-24T06:35:38.169 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:37 smithi037 ceph-mon[32864]: purged_snaps scrub ok 2024-04-24T06:35:38.169 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:37 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T06:35:38.169 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:37 smithi037 ceph-mon[32864]: from='osd.0 [v2:172.21.15.5:6802/2155858515,v1:172.21.15.5:6803/2155858515]' entity='osd.0' 2024-04-24T06:35:38.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:37 smithi137 ceph-mon[33017]: purged_snaps scrub starts 2024-04-24T06:35:38.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:37 smithi137 ceph-mon[33017]: purged_snaps scrub ok 2024-04-24T06:35:38.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:37 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T06:35:38.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:37 smithi137 ceph-mon[33017]: from='osd.0 [v2:172.21.15.5:6802/2155858515,v1:172.21.15.5:6803/2155858515]' entity='osd.0' 2024-04-24T06:35:38.762 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi037:vg_nvme/lv_4 2024-04-24T06:35:39.007 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T06:35:39.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:38 smithi037 ceph-mon[32864]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:39.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:38 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T06:35:39.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:38 smithi037 ceph-mon[32864]: osd.0 [v2:172.21.15.5:6802/2155858515,v1:172.21.15.5:6803/2155858515] boot 2024-04-24T06:35:39.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:38 smithi037 ceph-mon[32864]: osdmap e8: 1 total, 1 up, 1 in 2024-04-24T06:35:39.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:38 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T06:35:39.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:38 smithi137 ceph-mon[33017]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:39.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:38 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T06:35:39.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:38 smithi137 ceph-mon[33017]: osd.0 [v2:172.21.15.5:6802/2155858515,v1:172.21.15.5:6803/2155858515] boot 2024-04-24T06:35:39.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:38 smithi137 ceph-mon[33017]: osdmap e8: 1 total, 1 up, 1 in 2024-04-24T06:35:39.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:38 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T06:35:39.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:38 smithi005 ceph-mon[27798]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T06:35:39.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:38 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T06:35:39.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:38 smithi005 ceph-mon[27798]: osd.0 [v2:172.21.15.5:6802/2155858515,v1:172.21.15.5:6803/2155858515] boot 2024-04-24T06:35:39.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:38 smithi005 ceph-mon[27798]: osdmap e8: 1 total, 1 up, 1 in 2024-04-24T06:35:39.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:38 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T06:35:40.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:39 smithi037 ceph-mon[32864]: osdmap e9: 1 total, 1 up, 1 in 2024-04-24T06:35:40.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:39 smithi137 ceph-mon[33017]: osdmap e9: 1 total, 1 up, 1 in 2024-04-24T06:35:40.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:39 smithi005 ceph-mon[27798]: osdmap e9: 1 total, 1 up, 1 in 2024-04-24T06:35:40.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:40.423+0000 7f3fdecf3700 1 -- 172.21.15.37:0/4173438684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fd8102c70 msgr2=0x7f3fd81030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:40.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:40.423+0000 7f3fdecf3700 1 --2- 172.21.15.37:0/4173438684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fd8102c70 0x7f3fd81030d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3fc40099c0 tx=0x7f3fc4009cd0 comp rx=0 tx=0).stop 2024-04-24T06:35:40.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:40.423+0000 7f3fdecf3700 1 -- 172.21.15.37:0/4173438684 shutdown_connections 2024-04-24T06:35:40.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:40.423+0000 7f3fdecf3700 1 --2- 172.21.15.37:0/4173438684 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fd8103610 0x7f3fd8109ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:40.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:40.423+0000 7f3fdecf3700 1 --2- 172.21.15.37:0/4173438684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fd8101a70 0x7f3fd8101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:40.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:40.423+0000 7f3fdecf3700 1 --2- 172.21.15.37:0/4173438684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fd8102c70 0x7f3fd81030d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:40.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:40.423+0000 7f3fdecf3700 1 -- 172.21.15.37:0/4173438684 >> 172.21.15.37:0/4173438684 conn(0x7f3fd80fd050 msgr2=0x7f3fd80ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:40.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:40.423+0000 7f3fdecf3700 1 -- 172.21.15.37:0/4173438684 shutdown_connections 2024-04-24T06:35:40.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:40.423+0000 7f3fdecf3700 1 -- 172.21.15.37:0/4173438684 wait complete. 2024-04-24T06:35:40.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:40.423+0000 7f3fdecf3700 1 Processor -- start 2024-04-24T06:35:40.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:40.423+0000 7f3fdecf3700 1 -- start start 2024-04-24T06:35:40.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:40.423+0000 7f3fdecf3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fd8101a70 0x7f3fd810c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:40.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:40.423+0000 7f3fdecf3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fd8102c70 0x7f3fd810aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:40.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:40.423+0000 7f3fdecf3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fd8103610 0x7f3fd810afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:40.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:40.424+0000 7f3fdecf3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd810e7d0 con 0x7f3fd8103610 2024-04-24T06:35:40.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:40.424+0000 7f3fdecf3700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd810e650 con 0x7f3fd8102c70 2024-04-24T06:35:40.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:40.424+0000 7f3fdecf3700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd810e950 con 0x7f3fd8101a70 2024-04-24T06:35:40.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:40.424+0000 7f3fd7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fd8102c70 0x7f3fd810aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:40.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:40.424+0000 7f3fd7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fd8102c70 0x7f3fd810aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49392/0 (socket says 172.21.15.37:49392) 2024-04-24T06:35:40.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:40.424+0000 7f3fd7fff700 1 -- 172.21.15.37:0/3140940035 learned_addr learned my addr 172.21.15.37:0/3140940035 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-24T06:35:40.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:40.424+0000 7f3fdca8f700 1 --2- 172.21.15.37:0/3140940035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fd8101a70 0x7f3fd810c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:40.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:40.424+0000 7f3fd7fff700 1 -- 172.21.15.37:0/3140940035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fd8101a70 msgr2=0x7f3fd810c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:40.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:40.424+0000 7f3fd7fff700 1 --2- 172.21.15.37:0/3140940035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fd8101a70 0x7f3fd810c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:40.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:40.424+0000 7f3fdd290700 1 --2- 172.21.15.37:0/3140940035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fd8103610 0x7f3fd810afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:40.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:40.424+0000 7f3fd7fff700 1 -- 172.21.15.37:0/3140940035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fd8103610 msgr2=0x7f3fd810afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:40.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:40.424+0000 7f3fd7fff700 1 --2- 172.21.15.37:0/3140940035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fd8103610 0x7f3fd810afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:40.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:40.424+0000 7f3fd7fff700 1 -- 172.21.15.37:0/3140940035 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fd810b880 con 0x7f3fd8102c70 2024-04-24T06:35:40.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:40.424+0000 7f3fd7fff700 1 --2- 172.21.15.37:0/3140940035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fd8102c70 0x7f3fd810aa90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4000c00 tx=0x7f3fc40057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:40.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:40.424+0000 7f3fd5ffb700 1 -- 172.21.15.37:0/3140940035 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc4027070 con 0x7f3fd8102c70 2024-04-24T06:35:40.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:40.424+0000 7f3fd5ffb700 1 -- 172.21.15.37:0/3140940035 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fc4004150 con 0x7f3fd8102c70 2024-04-24T06:35:40.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:40.424+0000 7f3fdecf3700 1 -- 172.21.15.37:0/3140940035 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fd81a2eb0 con 0x7f3fd8102c70 2024-04-24T06:35:40.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:40.424+0000 7f3fd5ffb700 1 -- 172.21.15.37:0/3140940035 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc400f460 con 0x7f3fd8102c70 2024-04-24T06:35:40.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:40.425+0000 7f3fdecf3700 1 -- 172.21.15.37:0/3140940035 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fd81a3320 con 0x7f3fd8102c70 2024-04-24T06:35:40.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:40.425+0000 7f3fdecf3700 1 -- 172.21.15.37:0/3140940035 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fd80697e0 con 0x7f3fd8102c70 2024-04-24T06:35:40.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:40.427+0000 7f3fd5ffb700 1 -- 172.21.15.37:0/3140940035 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 15) ==== 138589+0+0 (secure 0 0 0) 0x7f3fc402a070 con 0x7f3fd8102c70 2024-04-24T06:35:40.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:40.427+0000 7f3fd5ffb700 1 --2- 172.21.15.37:0/3140940035 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f3fc80a3c30 0x7f3fc80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:35:40.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:40.427+0000 7f3fd5ffb700 1 -- 172.21.15.37:0/3140940035 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f3fc40cfa30 con 0x7f3fd8102c70 2024-04-24T06:35:40.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:40.428+0000 7f3fdca8f700 1 --2- 172.21.15.37:0/3140940035 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f3fc80a3c30 0x7f3fc80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:35:40.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:40.429+0000 7f3fdca8f700 1 --2- 172.21.15.37:0/3140940035 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f3fc80a3c30 0x7f3fc80a60f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8102ad0 tx=0x7f3fcc009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:35:40.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:40.431+0000 7f3fd5ffb700 1 -- 172.21.15.37:0/3140940035 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3fc4013070 con 0x7f3fd8102c70 2024-04-24T06:35:40.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:40.650+0000 7f3fdecf3700 1 -- 172.21.15.37:0/3140940035 --> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f3fd8062f30 con 0x7f3fc80a3c30 2024-04-24T06:35:41.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:40 smithi005 ceph-mon[27798]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-24T06:35:41.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:40 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T06:35:41.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:40 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T06:35:41.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:40 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:35:41.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:40 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:41.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:40 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:41.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:40 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:35:41.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:40 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:41.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:40 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:35:41.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:40 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:35:41.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:40 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:41.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:40 smithi137 ceph-mon[33017]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-24T06:35:41.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:40 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T06:35:41.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:40 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T06:35:41.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:40 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:35:41.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:40 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:41.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:40 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:41.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:40 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:35:41.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:40 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:41.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:40 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:35:41.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:40 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:35:41.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:40 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:41.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:40 smithi037 ceph-mon[32864]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-24T06:35:41.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:40 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T06:35:41.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:40 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T06:35:41.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:40 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:35:41.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:40 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:41.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:40 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:41.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:40 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:35:41.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:40 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:41.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:40 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:35:41.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:40 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:35:41.270 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:40 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:42.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:41 smithi005 ceph-mon[27798]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:35:42.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:41 smithi005 ceph-mon[27798]: Detected new or changed devices on smithi005 2024-04-24T06:35:42.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:41 smithi005 ceph-mon[27798]: Adjusting osd_memory_target on smithi005 to 17094M 2024-04-24T06:35:42.271 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:41 smithi037 ceph-mon[32864]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:35:42.271 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:41 smithi037 ceph-mon[32864]: Detected new or changed devices on smithi005 2024-04-24T06:35:42.271 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:41 smithi037 ceph-mon[32864]: Adjusting osd_memory_target on smithi005 to 17094M 2024-04-24T06:35:42.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:41 smithi137 ceph-mon[33017]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:35:42.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:41 smithi137 ceph-mon[33017]: Detected new or changed devices on smithi005 2024-04-24T06:35:42.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:41 smithi137 ceph-mon[33017]: Adjusting osd_memory_target on smithi005 to 17094M 2024-04-24T06:35:43.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:42 smithi005 ceph-mon[27798]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-24T06:35:43.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:42 smithi037 ceph-mon[32864]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-24T06:35:43.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:42 smithi137 ceph-mon[33017]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-24T06:35:44.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:43 smithi005 ceph-mon[27798]: from='client.? 172.21.15.37:0/2127951270' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50467744-db20-4721-bf2d-1c5db938297b"}]: dispatch 2024-04-24T06:35:44.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:43 smithi005 ceph-mon[27798]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50467744-db20-4721-bf2d-1c5db938297b"}]: dispatch 2024-04-24T06:35:44.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:43 smithi005 ceph-mon[27798]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50467744-db20-4721-bf2d-1c5db938297b"}]': finished 2024-04-24T06:35:44.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:43 smithi005 ceph-mon[27798]: osdmap e10: 2 total, 1 up, 2 in 2024-04-24T06:35:44.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:43 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T06:35:44.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:43 smithi005 ceph-mon[27798]: from='client.? 172.21.15.37:0/2738553363' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T06:35:44.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:43 smithi037 ceph-mon[32864]: from='client.? 172.21.15.37:0/2127951270' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50467744-db20-4721-bf2d-1c5db938297b"}]: dispatch 2024-04-24T06:35:44.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:43 smithi037 ceph-mon[32864]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50467744-db20-4721-bf2d-1c5db938297b"}]: dispatch 2024-04-24T06:35:44.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:43 smithi037 ceph-mon[32864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50467744-db20-4721-bf2d-1c5db938297b"}]': finished 2024-04-24T06:35:44.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:43 smithi037 ceph-mon[32864]: osdmap e10: 2 total, 1 up, 2 in 2024-04-24T06:35:44.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:43 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T06:35:44.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:43 smithi037 ceph-mon[32864]: from='client.? 172.21.15.37:0/2738553363' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T06:35:44.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:43 smithi137 ceph-mon[33017]: from='client.? 172.21.15.37:0/2127951270' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50467744-db20-4721-bf2d-1c5db938297b"}]: dispatch 2024-04-24T06:35:44.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:43 smithi137 ceph-mon[33017]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50467744-db20-4721-bf2d-1c5db938297b"}]: dispatch 2024-04-24T06:35:44.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:43 smithi137 ceph-mon[33017]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50467744-db20-4721-bf2d-1c5db938297b"}]': finished 2024-04-24T06:35:44.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:43 smithi137 ceph-mon[33017]: osdmap e10: 2 total, 1 up, 2 in 2024-04-24T06:35:44.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:43 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T06:35:44.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:43 smithi137 ceph-mon[33017]: from='client.? 172.21.15.37:0/2738553363' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T06:35:45.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:44 smithi005 ceph-mon[27798]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-24T06:35:45.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:44 smithi037 ceph-mon[32864]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-24T06:35:45.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:44 smithi137 ceph-mon[33017]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-24T06:35:47.222 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:46 smithi037 ceph-mon[32864]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-24T06:35:47.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:46 smithi005 ceph-mon[27798]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-24T06:35:47.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:46 smithi137 ceph-mon[33017]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-24T06:35:49.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:48 smithi005 ceph-mon[27798]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-24T06:35:49.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:48 smithi037 ceph-mon[32864]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-24T06:35:49.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:48 smithi137 ceph-mon[33017]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-24T06:35:51.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:50 smithi005 ceph-mon[27798]: pgmap v64: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-24T06:35:51.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:50 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T06:35:51.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:50 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:35:51.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:50 smithi037 ceph-mon[32864]: pgmap v64: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-24T06:35:51.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:50 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T06:35:51.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:50 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:35:51.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:50 smithi137 ceph-mon[33017]: pgmap v64: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-24T06:35:51.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:50 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T06:35:51.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:50 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:35:52.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:51 smithi005 ceph-mon[27798]: Deploying daemon osd.1 on smithi037 2024-04-24T06:35:52.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:51 smithi037 ceph-mon[32864]: Deploying daemon osd.1 on smithi037 2024-04-24T06:35:52.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:51 smithi137 ceph-mon[33017]: Deploying daemon osd.1 on smithi037 2024-04-24T06:35:53.243 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:52 smithi037 ceph-mon[32864]: pgmap v65: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-24T06:35:53.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:52 smithi005 ceph-mon[27798]: pgmap v65: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-24T06:35:53.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:52 smithi137 ceph-mon[33017]: pgmap v65: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-24T06:35:55.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:55 smithi005 ceph-mon[27798]: pgmap v66: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-24T06:35:55.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:55 smithi037 ceph-mon[32864]: pgmap v66: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-24T06:35:55.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:55 smithi137 ceph-mon[33017]: pgmap v66: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-24T06:35:56.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:56 smithi137 ceph-mon[33017]: pgmap v67: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-24T06:35:56.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:56 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:56.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:56 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:56.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:56 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:35:57.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:56 smithi005 ceph-mon[27798]: pgmap v67: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-24T06:35:57.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:56 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:57.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:56 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:57.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:56 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:35:57.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:56 smithi037 ceph-mon[32864]: pgmap v67: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-24T06:35:57.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:56 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:57.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:56 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:57.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:56 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:35:58.829 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:58 smithi037 ceph-mon[32864]: pgmap v68: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-24T06:35:58.829 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:58 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:58.829 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:58 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:58.829 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:58 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:35:58.829 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:58 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:35:58.829 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:58 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:58.887 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 1 on host 'smithi037' 2024-04-24T06:35:58.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:58.885+0000 7f3fd5ffb700 1 -- 172.21.15.37:0/3140940035 <== mgr.14152 v2:172.21.15.5:6800/4042936965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3fd8062f30 con 0x7f3fc80a3c30 2024-04-24T06:35:58.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:58.887+0000 7f3fdecf3700 1 -- 172.21.15.37:0/3140940035 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f3fc80a3c30 msgr2=0x7f3fc80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:58.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:58.887+0000 7f3fdecf3700 1 --2- 172.21.15.37:0/3140940035 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f3fc80a3c30 0x7f3fc80a60f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8102ad0 tx=0x7f3fcc009190 comp rx=0 tx=0).stop 2024-04-24T06:35:58.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:58.887+0000 7f3fdecf3700 1 -- 172.21.15.37:0/3140940035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fd8102c70 msgr2=0x7f3fd810aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:35:58.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:58.887+0000 7f3fdecf3700 1 --2- 172.21.15.37:0/3140940035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fd8102c70 0x7f3fd810aa90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4000c00 tx=0x7f3fc40057c0 comp rx=0 tx=0).stop 2024-04-24T06:35:58.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:58.887+0000 7f3fdecf3700 1 -- 172.21.15.37:0/3140940035 shutdown_connections 2024-04-24T06:35:58.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:58.887+0000 7f3fdecf3700 1 --2- 172.21.15.37:0/3140940035 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f3fc80a3c30 0x7f3fc80a60f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:58.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:58.887+0000 7f3fdecf3700 1 --2- 172.21.15.37:0/3140940035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fd8103610 0x7f3fd810afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:58.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:58.887+0000 7f3fdecf3700 1 --2- 172.21.15.37:0/3140940035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fd8101a70 0x7f3fd810c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:58.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:58.887+0000 7f3fdecf3700 1 --2- 172.21.15.37:0/3140940035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fd8102c70 0x7f3fd810aa90 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:35:58.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:58.887+0000 7f3fdecf3700 1 -- 172.21.15.37:0/3140940035 >> 172.21.15.37:0/3140940035 conn(0x7f3fd80fd050 msgr2=0x7f3fd8105f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:35:58.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:58.888+0000 7f3fdecf3700 1 -- 172.21.15.37:0/3140940035 shutdown_connections 2024-04-24T06:35:58.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:35:58.888+0000 7f3fdecf3700 1 -- 172.21.15.37:0/3140940035 wait complete. 2024-04-24T06:35:58.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:58 smithi137 ceph-mon[33017]: pgmap v68: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-24T06:35:58.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:58 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:58.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:58 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:58.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:58 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:35:58.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:58 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:35:58.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:58 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:59.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:58 smithi005 ceph-mon[27798]: pgmap v68: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-24T06:35:59.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:58 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:59.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:58 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:59.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:58 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:35:59.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:58 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:35:59.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:58 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:35:59.479 DEBUG:teuthology.orchestra.run.smithi037:osd.1> sudo journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@osd.1.service 2024-04-24T06:35:59.481 INFO:tasks.cephadm:Deploying osd.2 on smithi137 with /dev/vg_nvme/lv_4... 2024-04-24T06:35:59.481 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-24T06:35:59.702 INFO:journalctl@ceph.osd.1.smithi037.stdout:Apr 24 06:35:59 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-1[37455]: 2024-04-24T06:35:59.530+0000 7fecc8cb17c0 -1 osd.1 0 log_to_monitors true 2024-04-24T06:35:59.723 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T06:36:00.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:59 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:00.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:59 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:36:00.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:59 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:00.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:59 smithi037 ceph-mon[32864]: from='osd.1 [v2:172.21.15.37:6800/1302362650,v1:172.21.15.37:6801/1302362650]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T06:36:00.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:35:59 smithi037 ceph-mon[32864]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T06:36:00.170 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:59 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:00.171 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:59 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:36:00.171 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:59 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:00.171 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:59 smithi137 ceph-mon[33017]: from='osd.1 [v2:172.21.15.37:6800/1302362650,v1:172.21.15.37:6801/1302362650]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T06:36:00.171 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:35:59 smithi137 ceph-mon[33017]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T06:36:00.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:59 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:00.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:59 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:36:00.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:59 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:00.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:59 smithi005 ceph-mon[27798]: from='osd.1 [v2:172.21.15.37:6800/1302362650,v1:172.21.15.37:6801/1302362650]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T06:36:00.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:35:59 smithi005 ceph-mon[27798]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T06:36:01.008 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:00 smithi137 ceph-mon[33017]: pgmap v69: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-24T06:36:01.008 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:00 smithi137 ceph-mon[33017]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T06:36:01.008 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:00 smithi137 ceph-mon[33017]: from='osd.1 [v2:172.21.15.37:6800/1302362650,v1:172.21.15.37:6801/1302362650]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-24T06:36:01.008 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:00 smithi137 ceph-mon[33017]: osdmap e11: 2 total, 1 up, 2 in 2024-04-24T06:36:01.008 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:00 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T06:36:01.008 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:00 smithi137 ceph-mon[33017]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-24T06:36:01.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:00 smithi037 ceph-mon[32864]: pgmap v69: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-24T06:36:01.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:00 smithi037 ceph-mon[32864]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T06:36:01.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:00 smithi037 ceph-mon[32864]: from='osd.1 [v2:172.21.15.37:6800/1302362650,v1:172.21.15.37:6801/1302362650]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-24T06:36:01.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:00 smithi037 ceph-mon[32864]: osdmap e11: 2 total, 1 up, 2 in 2024-04-24T06:36:01.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:00 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T06:36:01.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:00 smithi037 ceph-mon[32864]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-24T06:36:01.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:00 smithi005 ceph-mon[27798]: pgmap v69: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-24T06:36:01.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:00 smithi005 ceph-mon[27798]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T06:36:01.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:00 smithi005 ceph-mon[27798]: from='osd.1 [v2:172.21.15.37:6800/1302362650,v1:172.21.15.37:6801/1302362650]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-24T06:36:01.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:00 smithi005 ceph-mon[27798]: osdmap e11: 2 total, 1 up, 2 in 2024-04-24T06:36:01.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:00 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T06:36:01.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:00 smithi005 ceph-mon[27798]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-24T06:36:01.553 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-24T06:36:01.554 INFO:teuthology.orchestra.run.smithi137.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-24T06:36:01.554 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10+0 records in 2024-04-24T06:36:01.554 INFO:teuthology.orchestra.run.smithi137.stderr:10+0 records out 2024-04-24T06:36:01.554 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111937 s, 937 MB/s 2024-04-24T06:36:01.554 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping successful for: 2024-04-24T06:36:02.019 INFO:journalctl@ceph.osd.1.smithi037.stdout:Apr 24 06:36:01 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-1[37455]: 2024-04-24T06:36:01.815+0000 7fecbeb17700 -1 osd.1 0 waiting for initial osdmap 2024-04-24T06:36:02.019 INFO:journalctl@ceph.osd.1.smithi037.stdout:Apr 24 06:36:01 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-1[37455]: 2024-04-24T06:36:01.905+0000 7fecb6101700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T06:36:02.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:01 smithi037 ceph-mon[32864]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-24T06:36:02.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:01 smithi037 ceph-mon[32864]: osdmap e12: 2 total, 1 up, 2 in 2024-04-24T06:36:02.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:01 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T06:36:02.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:01 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T06:36:02.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:01 smithi037 ceph-mon[32864]: from='osd.1 ' entity='osd.1' 2024-04-24T06:36:02.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:01 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T06:36:02.132 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:01 smithi137 ceph-mon[33017]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-24T06:36:02.132 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:01 smithi137 ceph-mon[33017]: osdmap e12: 2 total, 1 up, 2 in 2024-04-24T06:36:02.132 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:01 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T06:36:02.132 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:01 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T06:36:02.133 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:01 smithi137 ceph-mon[33017]: from='osd.1 ' entity='osd.1' 2024-04-24T06:36:02.133 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:01 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T06:36:02.157 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi137:vg_nvme/lv_4 2024-04-24T06:36:02.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:01 smithi005 ceph-mon[27798]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-24T06:36:02.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:01 smithi005 ceph-mon[27798]: osdmap e12: 2 total, 1 up, 2 in 2024-04-24T06:36:02.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:01 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T06:36:02.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:01 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T06:36:02.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:01 smithi005 ceph-mon[27798]: from='osd.1 ' entity='osd.1' 2024-04-24T06:36:02.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:01 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T06:36:02.404 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T06:36:03.177 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:02 smithi137 ceph-mon[33017]: purged_snaps scrub starts 2024-04-24T06:36:03.178 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:02 smithi137 ceph-mon[33017]: purged_snaps scrub ok 2024-04-24T06:36:03.178 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:02 smithi137 ceph-mon[33017]: pgmap v72: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-24T06:36:03.178 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:02 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:03.178 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:02 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:03.178 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:02 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:36:03.178 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:02 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:03.178 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:02 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:36:03.178 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:02 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:36:03.178 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:02 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:03.178 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:02 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T06:36:03.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:02 smithi005 ceph-mon[27798]: purged_snaps scrub starts 2024-04-24T06:36:03.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:02 smithi005 ceph-mon[27798]: purged_snaps scrub ok 2024-04-24T06:36:03.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:02 smithi005 ceph-mon[27798]: pgmap v72: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-24T06:36:03.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:02 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:03.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:02 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:03.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:02 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:36:03.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:02 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:03.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:02 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:36:03.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:02 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:36:03.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:02 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:03.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:02 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T06:36:03.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:02 smithi037 ceph-mon[32864]: purged_snaps scrub starts 2024-04-24T06:36:03.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:02 smithi037 ceph-mon[32864]: purged_snaps scrub ok 2024-04-24T06:36:03.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:02 smithi037 ceph-mon[32864]: pgmap v72: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-24T06:36:03.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:02 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:03.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:02 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:03.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:02 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:36:03.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:02 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:03.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:02 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:36:03.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:02 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:36:03.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:02 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:03.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:02 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T06:36:03.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:03.978+0000 7f11819ac700 1 -- 172.21.15.137:0/3776542327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f117c103600 msgr2=0x7f117c1059f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:03.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:03.978+0000 7f11819ac700 1 --2- 172.21.15.137:0/3776542327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f117c103600 0x7f117c1059f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f116c009990 tx=0x7f116c009ca0 comp rx=0 tx=0).stop 2024-04-24T06:36:03.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:03.978+0000 7f11819ac700 1 -- 172.21.15.137:0/3776542327 shutdown_connections 2024-04-24T06:36:03.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:03.978+0000 7f11819ac700 1 --2- 172.21.15.137:0/3776542327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f117c108860 0x7f117c10acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:03.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:03.978+0000 7f11819ac700 1 --2- 172.21.15.137:0/3776542327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f117c103600 0x7f117c1059f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:03.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:03.978+0000 7f11819ac700 1 --2- 172.21.15.137:0/3776542327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f117c105f30 0x7f117c108320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:03.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:03.978+0000 7f11819ac700 1 -- 172.21.15.137:0/3776542327 >> 172.21.15.137:0/3776542327 conn(0x7f117c0fd010 msgr2=0x7f117c0ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:03.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:03.978+0000 7f11819ac700 1 -- 172.21.15.137:0/3776542327 shutdown_connections 2024-04-24T06:36:03.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:03.978+0000 7f11819ac700 1 -- 172.21.15.137:0/3776542327 wait complete. 2024-04-24T06:36:03.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:03.978+0000 7f11819ac700 1 Processor -- start 2024-04-24T06:36:03.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:03.979+0000 7f11819ac700 1 -- start start 2024-04-24T06:36:03.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:03.979+0000 7f11819ac700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f117c103600 0x7f117c198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:03.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:03.979+0000 7f11819ac700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f117c105f30 0x7f117c199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:03.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:03.979+0000 7f11819ac700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f117c108860 0x7f117c1a0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:03.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:03.979+0000 7f11819ac700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f117c10e7c0 con 0x7f117c108860 2024-04-24T06:36:03.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:03.979+0000 7f11819ac700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f117c10e640 con 0x7f117c103600 2024-04-24T06:36:03.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:03.979+0000 7f11819ac700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f117c10e940 con 0x7f117c105f30 2024-04-24T06:36:03.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:03.979+0000 7f117a7fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f117c105f30 0x7f117c199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:03.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:03.979+0000 7f117a7fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f117c105f30 0x7f117c199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48818/0 (socket says 172.21.15.137:48818) 2024-04-24T06:36:03.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:03.979+0000 7f117a7fc700 1 -- 172.21.15.137:0/566457029 learned_addr learned my addr 172.21.15.137:0/566457029 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-04-24T06:36:03.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:03.979+0000 7f117affd700 1 --2- 172.21.15.137:0/566457029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f117c103600 0x7f117c198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:03.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:03.979+0000 7f117a7fc700 1 -- 172.21.15.137:0/566457029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f117c103600 msgr2=0x7f117c198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:03.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:03.979+0000 7f117a7fc700 1 --2- 172.21.15.137:0/566457029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f117c103600 0x7f117c198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:03.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:03.979+0000 7f117a7fc700 1 -- 172.21.15.137:0/566457029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f117c108860 msgr2=0x7f117c1a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:03.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:03.979+0000 7f117a7fc700 1 --2- 172.21.15.137:0/566457029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f117c108860 0x7f117c1a0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:03.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:03.979+0000 7f117a7fc700 1 -- 172.21.15.137:0/566457029 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f117c1a09a0 con 0x7f117c105f30 2024-04-24T06:36:03.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:03.979+0000 7f117a7fc700 1 --2- 172.21.15.137:0/566457029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f117c105f30 0x7f117c199280 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f117000d9b0 tx=0x7f117000dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:03.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:03.979+0000 7f11809aa700 1 -- 172.21.15.137:0/566457029 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1170016070 con 0x7f117c105f30 2024-04-24T06:36:03.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:03.980+0000 7f11809aa700 1 -- 172.21.15.137:0/566457029 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1170004d10 con 0x7f117c105f30 2024-04-24T06:36:03.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:03.980+0000 7f11819ac700 1 -- 172.21.15.137:0/566457029 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f117c1a0c90 con 0x7f117c105f30 2024-04-24T06:36:03.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:03.980+0000 7f11809aa700 1 -- 172.21.15.137:0/566457029 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1170005630 con 0x7f117c105f30 2024-04-24T06:36:03.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:03.980+0000 7f11819ac700 1 -- 172.21.15.137:0/566457029 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f117c1a1180 con 0x7f117c105f30 2024-04-24T06:36:03.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:03.980+0000 7f11819ac700 1 -- 172.21.15.137:0/566457029 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f117c0697e0 con 0x7f117c105f30 2024-04-24T06:36:03.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:03.982+0000 7f11809aa700 1 -- 172.21.15.137:0/566457029 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 15) ==== 138589+0+0 (secure 0 0 0) 0x7f1170004750 con 0x7f117c105f30 2024-04-24T06:36:03.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:03.982+0000 7f11809aa700 1 --2- 172.21.15.137:0/566457029 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f11640a3bb0 0x7f11640a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:03.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:03.982+0000 7f11809aa700 1 -- 172.21.15.137:0/566457029 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f11700cea20 con 0x7f117c105f30 2024-04-24T06:36:03.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:03.983+0000 7f117affd700 1 --2- 172.21.15.137:0/566457029 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f11640a3bb0 0x7f11640a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:03.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:03.984+0000 7f117affd700 1 --2- 172.21.15.137:0/566457029 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f11640a3bb0 0x7f11640a6070 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f116c00b5c0 tx=0x7f116c003960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:03.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:03.987+0000 7f11809aa700 1 -- 172.21.15.137:0/566457029 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f117001f020 con 0x7f117c105f30 2024-04-24T06:36:04.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:04.193+0000 7f11819ac700 1 -- 172.21.15.137:0/566457029 --> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f117c062f30 con 0x7f11640a3bb0 2024-04-24T06:36:04.194 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:03 smithi137 ceph-mon[33017]: Detected new or changed devices on smithi037 2024-04-24T06:36:04.194 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:03 smithi137 ceph-mon[33017]: Adjusting osd_memory_target on smithi037 to 17093M 2024-04-24T06:36:04.195 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:03 smithi137 ceph-mon[33017]: osd.1 [v2:172.21.15.37:6800/1302362650,v1:172.21.15.37:6801/1302362650] boot 2024-04-24T06:36:04.195 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:03 smithi137 ceph-mon[33017]: osdmap e13: 2 total, 2 up, 2 in 2024-04-24T06:36:04.195 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:03 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T06:36:04.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:03 smithi005 ceph-mon[27798]: Detected new or changed devices on smithi037 2024-04-24T06:36:04.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:03 smithi005 ceph-mon[27798]: Adjusting osd_memory_target on smithi037 to 17093M 2024-04-24T06:36:04.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:03 smithi005 ceph-mon[27798]: osd.1 [v2:172.21.15.37:6800/1302362650,v1:172.21.15.37:6801/1302362650] boot 2024-04-24T06:36:04.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:03 smithi005 ceph-mon[27798]: osdmap e13: 2 total, 2 up, 2 in 2024-04-24T06:36:04.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:03 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T06:36:04.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:03 smithi037 ceph-mon[32864]: Detected new or changed devices on smithi037 2024-04-24T06:36:04.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:03 smithi037 ceph-mon[32864]: Adjusting osd_memory_target on smithi037 to 17093M 2024-04-24T06:36:04.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:03 smithi037 ceph-mon[32864]: osd.1 [v2:172.21.15.37:6800/1302362650,v1:172.21.15.37:6801/1302362650] boot 2024-04-24T06:36:04.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:03 smithi037 ceph-mon[32864]: osdmap e13: 2 total, 2 up, 2 in 2024-04-24T06:36:04.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:03 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T06:36:05.125 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:04 smithi137 ceph-mon[33017]: pgmap v74: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-24T06:36:05.126 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:04 smithi137 ceph-mon[33017]: osdmap e14: 2 total, 2 up, 2 in 2024-04-24T06:36:05.126 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:04 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T06:36:05.126 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:04 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T06:36:05.126 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:04 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:36:05.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:04 smithi005 ceph-mon[27798]: pgmap v74: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-24T06:36:05.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:04 smithi005 ceph-mon[27798]: osdmap e14: 2 total, 2 up, 2 in 2024-04-24T06:36:05.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:04 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T06:36:05.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:04 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T06:36:05.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:04 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:36:05.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:04 smithi037 ceph-mon[32864]: pgmap v74: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-24T06:36:05.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:04 smithi037 ceph-mon[32864]: osdmap e14: 2 total, 2 up, 2 in 2024-04-24T06:36:05.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:04 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T06:36:05.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:04 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T06:36:05.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:04 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:36:05.988 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:05 smithi137 ceph-mon[33017]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:36:06.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:05 smithi005 ceph-mon[27798]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:36:06.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:05 smithi037 ceph-mon[32864]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:36:07.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:06 smithi137 ceph-mon[33017]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-24T06:36:07.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:06 smithi137 ceph-mon[33017]: from='client.? 172.21.15.137:0/4275077684' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e190b15-d04c-4288-8d48-ecf6664392fe"}]: dispatch 2024-04-24T06:36:07.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:06 smithi137 ceph-mon[33017]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e190b15-d04c-4288-8d48-ecf6664392fe"}]: dispatch 2024-04-24T06:36:07.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:06 smithi137 ceph-mon[33017]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e190b15-d04c-4288-8d48-ecf6664392fe"}]': finished 2024-04-24T06:36:07.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:06 smithi137 ceph-mon[33017]: osdmap e15: 3 total, 2 up, 3 in 2024-04-24T06:36:07.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:06 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T06:36:07.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:06 smithi005 ceph-mon[27798]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-24T06:36:07.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:06 smithi005 ceph-mon[27798]: from='client.? 172.21.15.137:0/4275077684' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e190b15-d04c-4288-8d48-ecf6664392fe"}]: dispatch 2024-04-24T06:36:07.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:06 smithi005 ceph-mon[27798]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e190b15-d04c-4288-8d48-ecf6664392fe"}]: dispatch 2024-04-24T06:36:07.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:06 smithi005 ceph-mon[27798]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e190b15-d04c-4288-8d48-ecf6664392fe"}]': finished 2024-04-24T06:36:07.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:06 smithi005 ceph-mon[27798]: osdmap e15: 3 total, 2 up, 3 in 2024-04-24T06:36:07.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:06 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T06:36:07.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:06 smithi037 ceph-mon[32864]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-24T06:36:07.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:06 smithi037 ceph-mon[32864]: from='client.? 172.21.15.137:0/4275077684' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e190b15-d04c-4288-8d48-ecf6664392fe"}]: dispatch 2024-04-24T06:36:07.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:06 smithi037 ceph-mon[32864]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e190b15-d04c-4288-8d48-ecf6664392fe"}]: dispatch 2024-04-24T06:36:07.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:06 smithi037 ceph-mon[32864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e190b15-d04c-4288-8d48-ecf6664392fe"}]': finished 2024-04-24T06:36:07.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:06 smithi037 ceph-mon[32864]: osdmap e15: 3 total, 2 up, 3 in 2024-04-24T06:36:07.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:06 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T06:36:08.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:07 smithi005 ceph-mon[27798]: from='client.? 172.21.15.137:0/381495394' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T06:36:08.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:07 smithi037 ceph-mon[32864]: from='client.? 172.21.15.137:0/381495394' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T06:36:08.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:07 smithi137 ceph-mon[33017]: from='client.? 172.21.15.137:0/381495394' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T06:36:09.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:08 smithi005 ceph-mon[27798]: pgmap v78: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-24T06:36:09.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:08 smithi037 ceph-mon[32864]: pgmap v78: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-24T06:36:09.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:08 smithi137 ceph-mon[33017]: pgmap v78: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-24T06:36:11.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:10 smithi005 ceph-mon[27798]: pgmap v79: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-24T06:36:11.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:10 smithi037 ceph-mon[32864]: pgmap v79: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-24T06:36:11.278 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:10 smithi137 ceph-mon[33017]: pgmap v79: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-24T06:36:13.004 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:12 smithi137 ceph-mon[33017]: pgmap v80: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-24T06:36:13.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:12 smithi005 ceph-mon[27798]: pgmap v80: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-24T06:36:13.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:12 smithi037 ceph-mon[32864]: pgmap v80: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-24T06:36:15.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:14 smithi005 ceph-mon[27798]: pgmap v81: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-24T06:36:15.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:14 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T06:36:15.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:14 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:36:15.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:14 smithi005 ceph-mon[27798]: Deploying daemon osd.2 on smithi137 2024-04-24T06:36:15.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:14 smithi037 ceph-mon[32864]: pgmap v81: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-24T06:36:15.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:14 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T06:36:15.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:14 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:36:15.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:14 smithi037 ceph-mon[32864]: Deploying daemon osd.2 on smithi137 2024-04-24T06:36:15.321 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:14 smithi137 ceph-mon[33017]: pgmap v81: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-24T06:36:15.321 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:14 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T06:36:15.321 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:14 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:36:15.321 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:14 smithi137 ceph-mon[33017]: Deploying daemon osd.2 on smithi137 2024-04-24T06:36:17.141 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:16 smithi137 ceph-mon[33017]: pgmap v82: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-24T06:36:17.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:16 smithi005 ceph-mon[27798]: pgmap v82: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-24T06:36:17.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:16 smithi037 ceph-mon[32864]: pgmap v82: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-24T06:36:19.100 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:18 smithi137 ceph-mon[33017]: pgmap v83: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-24T06:36:19.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:18 smithi005 ceph-mon[27798]: pgmap v83: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-24T06:36:19.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:18 smithi037 ceph-mon[32864]: pgmap v83: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-24T06:36:20.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:20 smithi137 ceph-mon[33017]: pgmap v84: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-24T06:36:20.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:20 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:20.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:20 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:20.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:20 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:36:20.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:20 smithi005 ceph-mon[27798]: pgmap v84: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-24T06:36:20.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:20 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:20.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:20 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:20.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:20 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:36:20.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:20 smithi037 ceph-mon[32864]: pgmap v84: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-24T06:36:20.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:20 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:20.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:20 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:20.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:20 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:36:22.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:22.375+0000 7f11809aa700 1 -- 172.21.15.137:0/566457029 <== mgr.14152 v2:172.21.15.5:6800/4042936965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f117c062f30 con 0x7f11640a3bb0 2024-04-24T06:36:22.377 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 2 on host 'smithi137' 2024-04-24T06:36:22.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:22.377+0000 7f11819ac700 1 -- 172.21.15.137:0/566457029 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f11640a3bb0 msgr2=0x7f11640a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:22.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:22.377+0000 7f11819ac700 1 --2- 172.21.15.137:0/566457029 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f11640a3bb0 0x7f11640a6070 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f116c00b5c0 tx=0x7f116c003960 comp rx=0 tx=0).stop 2024-04-24T06:36:22.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:22.377+0000 7f11819ac700 1 -- 172.21.15.137:0/566457029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f117c105f30 msgr2=0x7f117c199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:22.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:22.377+0000 7f11819ac700 1 --2- 172.21.15.137:0/566457029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f117c105f30 0x7f117c199280 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f117000d9b0 tx=0x7f117000dcc0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:22.378+0000 7f11819ac700 1 -- 172.21.15.137:0/566457029 shutdown_connections 2024-04-24T06:36:22.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:22.378+0000 7f11819ac700 1 --2- 172.21.15.137:0/566457029 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f11640a3bb0 0x7f11640a6070 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:22.378+0000 7f11819ac700 1 --2- 172.21.15.137:0/566457029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f117c108860 0x7f117c1a0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:22.378+0000 7f11819ac700 1 --2- 172.21.15.137:0/566457029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f117c103600 0x7f117c198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:22.378+0000 7f11819ac700 1 --2- 172.21.15.137:0/566457029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f117c105f30 0x7f117c199280 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:22.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:22.378+0000 7f11819ac700 1 -- 172.21.15.137:0/566457029 >> 172.21.15.137:0/566457029 conn(0x7f117c0fd010 msgr2=0x7f117c103f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:22.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:22.378+0000 7f11819ac700 1 -- 172.21.15.137:0/566457029 shutdown_connections 2024-04-24T06:36:22.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:36:22.378+0000 7f11819ac700 1 -- 172.21.15.137:0/566457029 wait complete. 2024-04-24T06:36:22.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:22 smithi137 ceph-mon[33017]: pgmap v85: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-24T06:36:22.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:22 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:22.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:22 smithi005 ceph-mon[27798]: pgmap v85: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-24T06:36:22.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:22 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:22.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:22 smithi037 ceph-mon[32864]: pgmap v85: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-24T06:36:22.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:22 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:23.143 DEBUG:teuthology.orchestra.run.smithi137:osd.2> sudo journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@osd.2.service 2024-04-24T06:36:23.145 INFO:tasks.cephadm:Deploying osd.3 on smithi005 with /dev/vg_nvme/lv_3... 2024-04-24T06:36:23.146 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-24T06:36:23.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:23 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:23.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:23 smithi137 ceph-mon[33017]: from='osd.2 [v2:172.21.15.137:6800/2885027256,v1:172.21.15.137:6801/2885027256]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T06:36:23.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:23 smithi137 ceph-mon[33017]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T06:36:23.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:23 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:23.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:23 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:23.447 INFO:journalctl@ceph.osd.2.smithi137.stdout:Apr 24 06:36:23 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-2[38168]: 2024-04-24T06:36:23.217+0000 7f72c7bd67c0 -1 osd.2 0 log_to_monitors true 2024-04-24T06:36:23.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:23 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:23.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:23 smithi005 ceph-mon[27798]: from='osd.2 [v2:172.21.15.137:6800/2885027256,v1:172.21.15.137:6801/2885027256]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T06:36:23.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:23 smithi005 ceph-mon[27798]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T06:36:23.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:23 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:23.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:23 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:23.598 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:36:23.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:23 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:23.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:23 smithi037 ceph-mon[32864]: from='osd.2 [v2:172.21.15.137:6800/2885027256,v1:172.21.15.137:6801/2885027256]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T06:36:23.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:23 smithi037 ceph-mon[32864]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T06:36:23.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:23 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:23.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:23 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:24.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:24 smithi137 ceph-mon[33017]: pgmap v86: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-24T06:36:24.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:24 smithi137 ceph-mon[33017]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T06:36:24.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:24 smithi137 ceph-mon[33017]: from='osd.2 [v2:172.21.15.137:6800/2885027256,v1:172.21.15.137:6801/2885027256]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-24T06:36:24.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:24 smithi137 ceph-mon[33017]: osdmap e16: 3 total, 2 up, 3 in 2024-04-24T06:36:24.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:24 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T06:36:24.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:24 smithi137 ceph-mon[33017]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-24T06:36:24.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:24 smithi005 ceph-mon[27798]: pgmap v86: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-24T06:36:24.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:24 smithi005 ceph-mon[27798]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T06:36:24.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:24 smithi005 ceph-mon[27798]: from='osd.2 [v2:172.21.15.137:6800/2885027256,v1:172.21.15.137:6801/2885027256]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-24T06:36:24.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:24 smithi005 ceph-mon[27798]: osdmap e16: 3 total, 2 up, 3 in 2024-04-24T06:36:24.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:24 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T06:36:24.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:24 smithi005 ceph-mon[27798]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-24T06:36:24.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:24 smithi037 ceph-mon[32864]: pgmap v86: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-24T06:36:24.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:24 smithi037 ceph-mon[32864]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T06:36:24.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:24 smithi037 ceph-mon[32864]: from='osd.2 [v2:172.21.15.137:6800/2885027256,v1:172.21.15.137:6801/2885027256]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-24T06:36:24.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:24 smithi037 ceph-mon[32864]: osdmap e16: 3 total, 2 up, 3 in 2024-04-24T06:36:24.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:24 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T06:36:24.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:24 smithi037 ceph-mon[32864]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-24T06:36:25.419 INFO:journalctl@ceph.osd.2.smithi137.stdout:Apr 24 06:36:25 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-2[38168]: 2024-04-24T06:36:25.278+0000 7f72bda3c700 -1 osd.2 0 waiting for initial osdmap 2024-04-24T06:36:25.419 INFO:journalctl@ceph.osd.2.smithi137.stdout:Apr 24 06:36:25 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-2[38168]: 2024-04-24T06:36:25.418+0000 7f72b5026700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T06:36:25.420 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:25 smithi137 ceph-mon[33017]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-04-24T06:36:25.420 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:25 smithi137 ceph-mon[33017]: osdmap e17: 3 total, 2 up, 3 in 2024-04-24T06:36:25.420 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:25 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T06:36:25.420 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:25 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T06:36:25.420 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:25 smithi137 ceph-mon[33017]: from='osd.2 ' entity='osd.2' 2024-04-24T06:36:25.425 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:25 smithi005 ceph-mon[27798]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-04-24T06:36:25.425 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:25 smithi005 ceph-mon[27798]: osdmap e17: 3 total, 2 up, 3 in 2024-04-24T06:36:25.425 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:25 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T06:36:25.425 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:25 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T06:36:25.425 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:25 smithi005 ceph-mon[27798]: from='osd.2 ' entity='osd.2' 2024-04-24T06:36:25.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:25 smithi037 ceph-mon[32864]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-04-24T06:36:25.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:25 smithi037 ceph-mon[32864]: osdmap e17: 3 total, 2 up, 3 in 2024-04-24T06:36:25.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:25 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T06:36:25.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:25 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T06:36:25.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:25 smithi037 ceph-mon[32864]: from='osd.2 ' entity='osd.2' 2024-04-24T06:36:26.418 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-24T06:36:26.418 INFO:teuthology.orchestra.run.smithi005.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-24T06:36:26.418 INFO:teuthology.orchestra.run.smithi005.stderr: stderr: 10+0 records in 2024-04-24T06:36:26.418 INFO:teuthology.orchestra.run.smithi005.stderr:10+0 records out 2024-04-24T06:36:26.418 INFO:teuthology.orchestra.run.smithi005.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0352101 s, 298 MB/s 2024-04-24T06:36:26.419 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping successful for: 2024-04-24T06:36:26.616 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:26 smithi137 ceph-mon[33017]: purged_snaps scrub starts 2024-04-24T06:36:26.616 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:26 smithi137 ceph-mon[33017]: purged_snaps scrub ok 2024-04-24T06:36:26.616 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:26 smithi137 ceph-mon[33017]: pgmap v89: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-24T06:36:26.616 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:26 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T06:36:26.616 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:26 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T06:36:26.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:26 smithi005 ceph-mon[27798]: purged_snaps scrub starts 2024-04-24T06:36:26.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:26 smithi005 ceph-mon[27798]: purged_snaps scrub ok 2024-04-24T06:36:26.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:26 smithi005 ceph-mon[27798]: pgmap v89: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-24T06:36:26.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:26 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T06:36:26.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:26 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T06:36:26.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:26 smithi037 ceph-mon[32864]: purged_snaps scrub starts 2024-04-24T06:36:26.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:26 smithi037 ceph-mon[32864]: purged_snaps scrub ok 2024-04-24T06:36:26.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:26 smithi037 ceph-mon[32864]: pgmap v89: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-24T06:36:26.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:26 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T06:36:26.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:26 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T06:36:27.247 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi005:vg_nvme/lv_3 2024-04-24T06:36:27.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:27 smithi005 ceph-mon[27798]: osd.2 [v2:172.21.15.137:6800/2885027256,v1:172.21.15.137:6801/2885027256] boot 2024-04-24T06:36:27.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:27 smithi005 ceph-mon[27798]: osdmap e18: 3 total, 3 up, 3 in 2024-04-24T06:36:27.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:27 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T06:36:27.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:27 smithi005 ceph-mon[27798]: Detected new or changed devices on smithi137 2024-04-24T06:36:27.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:27 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:27.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:27 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:27.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:27 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:36:27.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:27 smithi005 ceph-mon[27798]: Adjusting osd_memory_target on smithi137 to 17088M 2024-04-24T06:36:27.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:27 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:27.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:27 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:36:27.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:27 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:36:27.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:27 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:27.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:27 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:36:27.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:27 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:36:27.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:27 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:36:27.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:27 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:27.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:27 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-24T06:36:27.685 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:36:27.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:27 smithi137 ceph-mon[33017]: osd.2 [v2:172.21.15.137:6800/2885027256,v1:172.21.15.137:6801/2885027256] boot 2024-04-24T06:36:27.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:27 smithi137 ceph-mon[33017]: osdmap e18: 3 total, 3 up, 3 in 2024-04-24T06:36:27.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:27 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T06:36:27.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:27 smithi137 ceph-mon[33017]: Detected new or changed devices on smithi137 2024-04-24T06:36:27.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:27 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:27.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:27 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:27.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:27 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:36:27.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:27 smithi137 ceph-mon[33017]: Adjusting osd_memory_target on smithi137 to 17088M 2024-04-24T06:36:27.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:27 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:27.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:27 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:36:27.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:27 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:36:27.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:27 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:27.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:27 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:36:27.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:27 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:36:27.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:27 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:36:27.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:27 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:27.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:27 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-24T06:36:27.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:27 smithi037 ceph-mon[32864]: osd.2 [v2:172.21.15.137:6800/2885027256,v1:172.21.15.137:6801/2885027256] boot 2024-04-24T06:36:27.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:27 smithi037 ceph-mon[32864]: osdmap e18: 3 total, 3 up, 3 in 2024-04-24T06:36:27.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:27 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T06:36:27.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:27 smithi037 ceph-mon[32864]: Detected new or changed devices on smithi137 2024-04-24T06:36:27.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:27 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:27.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:27 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:27.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:27 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:36:27.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:27 smithi037 ceph-mon[32864]: Adjusting osd_memory_target on smithi137 to 17088M 2024-04-24T06:36:27.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:27 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:27.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:27 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:36:27.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:27 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:36:27.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:27 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:27.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:27 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:36:27.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:27 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:36:27.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:27 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:36:27.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:27 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:27.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:27 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-24T06:36:29.153 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:28 smithi005 ceph-mon[27798]: pgmap v91: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-24T06:36:29.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:28 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-24T06:36:29.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:28 smithi005 ceph-mon[27798]: osdmap e19: 3 total, 3 up, 3 in 2024-04-24T06:36:29.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:28 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T06:36:29.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:28 smithi037 ceph-mon[32864]: pgmap v91: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-24T06:36:29.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:28 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-24T06:36:29.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:28 smithi037 ceph-mon[32864]: osdmap e19: 3 total, 3 up, 3 in 2024-04-24T06:36:29.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:28 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T06:36:29.362 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:28 smithi137 ceph-mon[33017]: pgmap v91: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-24T06:36:29.362 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:28 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-24T06:36:29.362 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:28 smithi137 ceph-mon[33017]: osdmap e19: 3 total, 3 up, 3 in 2024-04-24T06:36:29.362 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:28 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T06:36:29.408 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 24 06:36:29 smithi005 sudo[44069]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T06:36:29.408 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 24 06:36:29 smithi005 sudo[44069]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T06:36:29.408 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 24 06:36:29 smithi005 sudo[44069]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T06:36:29.408 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 24 06:36:29 smithi005 sudo[44069]: pam_unix(sudo:session): session closed for user root 2024-04-24T06:36:29.408 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 24 06:36:29 smithi005 sudo[44072]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T06:36:29.408 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 24 06:36:29 smithi005 sudo[44072]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T06:36:29.409 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 24 06:36:29 smithi005 sudo[44072]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T06:36:29.409 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 24 06:36:29 smithi005 sudo[44072]: pam_unix(sudo:session): session closed for user root 2024-04-24T06:36:29.580 INFO:journalctl@ceph.osd.1.smithi037.stdout:Apr 24 06:36:29 smithi037 sudo[41310]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T06:36:29.581 INFO:journalctl@ceph.osd.1.smithi037.stdout:Apr 24 06:36:29 smithi037 sudo[41310]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T06:36:29.581 INFO:journalctl@ceph.osd.1.smithi037.stdout:Apr 24 06:36:29 smithi037 sudo[41310]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T06:36:29.581 INFO:journalctl@ceph.osd.1.smithi037.stdout:Apr 24 06:36:29 smithi037 sudo[41310]: pam_unix(sudo:session): session closed for user root 2024-04-24T06:36:29.581 INFO:journalctl@ceph.osd.1.smithi037.stdout:Apr 24 06:36:29 smithi037 sudo[41313]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T06:36:29.581 INFO:journalctl@ceph.osd.1.smithi037.stdout:Apr 24 06:36:29 smithi037 sudo[41313]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T06:36:29.581 INFO:journalctl@ceph.osd.1.smithi037.stdout:Apr 24 06:36:29 smithi037 sudo[41313]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T06:36:29.581 INFO:journalctl@ceph.osd.1.smithi037.stdout:Apr 24 06:36:29 smithi037 sudo[41313]: pam_unix(sudo:session): session closed for user root 2024-04-24T06:36:29.690 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:29 smithi005 sudo[44080]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T06:36:29.690 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:29 smithi005 sudo[44080]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T06:36:29.690 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:29 smithi005 sudo[44080]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T06:36:29.690 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:29 smithi005 sudo[44080]: pam_unix(sudo:session): session closed for user root 2024-04-24T06:36:29.690 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:29 smithi005 sudo[44083]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T06:36:29.690 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:29 smithi005 sudo[44083]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T06:36:29.690 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:29 smithi005 sudo[44083]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T06:36:29.691 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:29 smithi005 sudo[44083]: pam_unix(sudo:session): session closed for user root 2024-04-24T06:36:29.696 INFO:journalctl@ceph.osd.2.smithi137.stdout:Apr 24 06:36:29 smithi137 sudo[42238]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T06:36:29.696 INFO:journalctl@ceph.osd.2.smithi137.stdout:Apr 24 06:36:29 smithi137 sudo[42238]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T06:36:29.696 INFO:journalctl@ceph.osd.2.smithi137.stdout:Apr 24 06:36:29 smithi137 sudo[42238]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T06:36:29.696 INFO:journalctl@ceph.osd.2.smithi137.stdout:Apr 24 06:36:29 smithi137 sudo[42238]: pam_unix(sudo:session): session closed for user root 2024-04-24T06:36:29.696 INFO:journalctl@ceph.osd.2.smithi137.stdout:Apr 24 06:36:29 smithi137 sudo[42241]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T06:36:29.696 INFO:journalctl@ceph.osd.2.smithi137.stdout:Apr 24 06:36:29 smithi137 sudo[42241]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T06:36:29.696 INFO:journalctl@ceph.osd.2.smithi137.stdout:Apr 24 06:36:29 smithi137 sudo[42241]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T06:36:29.696 INFO:journalctl@ceph.osd.2.smithi137.stdout:Apr 24 06:36:29 smithi137 sudo[42241]: pam_unix(sudo:session): session closed for user root 2024-04-24T06:36:29.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:29 smithi137 sudo[42244]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T06:36:29.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:29 smithi137 sudo[42244]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T06:36:29.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:29 smithi137 sudo[42244]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T06:36:29.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:29 smithi137 sudo[42244]: pam_unix(sudo:session): session closed for user root 2024-04-24T06:36:29.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:29 smithi137 sudo[42247]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T06:36:29.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:29 smithi137 sudo[42247]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T06:36:29.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:29 smithi137 sudo[42247]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T06:36:29.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:29 smithi137 sudo[42247]: pam_unix(sudo:session): session closed for user root 2024-04-24T06:36:29.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:29.772+0000 7f3f3a200700 1 -- 172.21.15.5:0/3418748971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f34104610 msgr2=0x7f3f3410aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:29.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:29.772+0000 7f3f3a200700 1 --2- 172.21.15.5:0/3418748971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f34104610 0x7f3f3410aec0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c00b0a0 tx=0x7f3f2c00b3b0 comp rx=0 tx=0).stop 2024-04-24T06:36:29.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:29.772+0000 7f3f3a200700 1 -- 172.21.15.5:0/3418748971 shutdown_connections 2024-04-24T06:36:29.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:29.772+0000 7f3f3a200700 1 --2- 172.21.15.5:0/3418748971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f34104610 0x7f3f3410aec0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:29.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:29.772+0000 7f3f3a200700 1 --2- 172.21.15.5:0/3418748971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f34102a70 0x7f3f34102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:29.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:29.772+0000 7f3f3a200700 1 --2- 172.21.15.5:0/3418748971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f34103c70 0x7f3f341040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:29.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:29.772+0000 7f3f3a200700 1 -- 172.21.15.5:0/3418748971 >> 172.21.15.5:0/3418748971 conn(0x7f3f340fe030 msgr2=0x7f3f34100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:29.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:29.772+0000 7f3f3a200700 1 -- 172.21.15.5:0/3418748971 shutdown_connections 2024-04-24T06:36:29.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:29.772+0000 7f3f3a200700 1 -- 172.21.15.5:0/3418748971 wait complete. 2024-04-24T06:36:29.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:29.773+0000 7f3f3a200700 1 Processor -- start 2024-04-24T06:36:29.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:29.774+0000 7f3f3a200700 1 -- start start 2024-04-24T06:36:29.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:29.774+0000 7f3f3a200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f34102a70 0x7f3f3419ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:29.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:29.774+0000 7f3f3a200700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f34103c70 0x7f3f3419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:29.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:29.774+0000 7f3f3a200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f34104610 0x7f3f341a23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:29.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:29.774+0000 7f3f3a200700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3410e640 con 0x7f3f34103c70 2024-04-24T06:36:29.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:29.774+0000 7f3f3a200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3410e4c0 con 0x7f3f34104610 2024-04-24T06:36:29.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:29.774+0000 7f3f3a200700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3410e7c0 con 0x7f3f34102a70 2024-04-24T06:36:29.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:29.774+0000 7f3f39800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f34104610 0x7f3f341a23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:29.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:29.774+0000 7f3f2be00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f34103c70 0x7f3f3419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:29.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:29.774+0000 7f3f38e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f34102a70 0x7f3f3419ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:29.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:29.774+0000 7f3f39800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f34104610 0x7f3f341a23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:43104/0 (socket says 172.21.15.5:43104) 2024-04-24T06:36:29.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:29.774+0000 7f3f39800700 1 -- 172.21.15.5:0/912110972 learned_addr learned my addr 172.21.15.5:0/912110972 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:36:29.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:29.775+0000 7f3f39800700 1 -- 172.21.15.5:0/912110972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f34102a70 msgr2=0x7f3f3419ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:29.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:29.775+0000 7f3f39800700 1 --2- 172.21.15.5:0/912110972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f34102a70 0x7f3f3419ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:29.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:29.775+0000 7f3f39800700 1 -- 172.21.15.5:0/912110972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f34103c70 msgr2=0x7f3f3419b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:29.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:29.775+0000 7f3f39800700 1 --2- 172.21.15.5:0/912110972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f34103c70 0x7f3f3419b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:29.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:29.775+0000 7f3f39800700 1 -- 172.21.15.5:0/912110972 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f341a2a90 con 0x7f3f34104610 2024-04-24T06:36:29.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:29.775+0000 7f3f39800700 1 --2- 172.21.15.5:0/912110972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f34104610 0x7f3f341a23f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c00b780 tx=0x7f3f2c004f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:29.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:29.776+0000 7f3f32000700 1 -- 172.21.15.5:0/912110972 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c021070 con 0x7f3f34104610 2024-04-24T06:36:29.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:29.776+0000 7f3f32000700 1 -- 172.21.15.5:0/912110972 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f2c007b80 con 0x7f3f34104610 2024-04-24T06:36:29.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:29.776+0000 7f3f32000700 1 -- 172.21.15.5:0/912110972 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c0052a0 con 0x7f3f34104610 2024-04-24T06:36:29.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:29.776+0000 7f3f3a200700 1 -- 172.21.15.5:0/912110972 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f341a2d20 con 0x7f3f34104610 2024-04-24T06:36:29.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:29.776+0000 7f3f3a200700 1 -- 172.21.15.5:0/912110972 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f341a31c0 con 0x7f3f34104610 2024-04-24T06:36:29.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:29.777+0000 7f3f3a200700 1 -- 172.21.15.5:0/912110972 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f340697e0 con 0x7f3f34104610 2024-04-24T06:36:29.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:29.779+0000 7f3f32000700 1 -- 172.21.15.5:0/912110972 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 15) ==== 138589+0+0 (secure 0 0 0) 0x7f3f2c024070 con 0x7f3f34104610 2024-04-24T06:36:29.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:29.780+0000 7f3f32000700 1 --2- 172.21.15.5:0/912110972 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f3f180a3c30 0x7f3f180a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:36:29.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:29.781+0000 7f3f32000700 1 -- 172.21.15.5:0/912110972 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3237+0+0 (secure 0 0 0) 0x7f3f2c00f020 con 0x7f3f34104610 2024-04-24T06:36:29.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:29.781+0000 7f3f38e00700 1 --2- 172.21.15.5:0/912110972 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f3f180a3c30 0x7f3f180a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:36:29.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:29.781+0000 7f3f38e00700 1 --2- 172.21.15.5:0/912110972 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f3f180a3c30 0x7f3f180a60f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3f34103ad0 tx=0x7f3f24009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:36:29.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:29.783+0000 7f3f32000700 1 -- 172.21.15.5:0/912110972 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3f2c02b020 con 0x7f3f34104610 2024-04-24T06:36:29.987 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:29 smithi037 sudo[41316]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T06:36:29.987 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:29 smithi037 sudo[41316]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T06:36:29.987 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:29 smithi037 sudo[41316]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T06:36:29.987 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:29 smithi037 sudo[41316]: pam_unix(sudo:session): session closed for user root 2024-04-24T06:36:29.987 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:29 smithi037 sudo[41319]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T06:36:29.988 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:29 smithi037 sudo[41319]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T06:36:29.988 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:29 smithi037 sudo[41319]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T06:36:29.988 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:29 smithi037 sudo[41319]: pam_unix(sudo:session): session closed for user root 2024-04-24T06:36:30.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:30.025+0000 7f3f3a200700 1 -- 172.21.15.5:0/912110972 --> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f3f34076440 con 0x7f3f180a3c30 2024-04-24T06:36:30.258 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:29 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-24T06:36:30.258 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:29 smithi005 ceph-mon[27798]: osdmap e20: 3 total, 3 up, 3 in 2024-04-24T06:36:30.259 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:29 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:30.259 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:29 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:30.259 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:29 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:30.259 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:29 smithi005 ceph-mon[27798]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T06:36:30.259 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:29 smithi005 ceph-mon[27798]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T06:36:30.259 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:29 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T06:36:30.259 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:29 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T06:36:30.259 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:29 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T06:36:30.259 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:29 smithi005 ceph-mon[27798]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T06:36:30.259 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:29 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T06:36:30.259 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:29 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T06:36:30.259 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:29 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T06:36:30.260 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:29 smithi005 ceph-mon[27798]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T06:36:30.260 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:29 smithi005 ceph-mon[27798]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T06:36:30.260 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:29 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T06:36:30.260 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:29 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T06:36:30.260 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:29 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T06:36:30.260 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:29 smithi005 ceph-mon[27798]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T06:36:30.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:30.266+0000 7f3f32000700 1 -- 172.21.15.5:0/912110972 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f3f2c09a670 con 0x7f3f34104610 2024-04-24T06:36:30.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:29 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-24T06:36:30.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:29 smithi037 ceph-mon[32864]: osdmap e20: 3 total, 3 up, 3 in 2024-04-24T06:36:30.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:29 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:30.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:29 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:30.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:29 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:30.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:29 smithi037 ceph-mon[32864]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T06:36:30.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:29 smithi037 ceph-mon[32864]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T06:36:30.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:29 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T06:36:30.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:29 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T06:36:30.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:29 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T06:36:30.270 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:29 smithi037 ceph-mon[32864]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T06:36:30.270 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:29 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T06:36:30.270 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:29 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T06:36:30.270 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:29 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T06:36:30.270 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:29 smithi037 ceph-mon[32864]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T06:36:30.270 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:29 smithi037 ceph-mon[32864]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T06:36:30.270 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:29 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T06:36:30.270 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:29 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T06:36:30.270 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:29 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T06:36:30.270 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:29 smithi037 ceph-mon[32864]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T06:36:30.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:29 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-24T06:36:30.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:29 smithi137 ceph-mon[33017]: osdmap e20: 3 total, 3 up, 3 in 2024-04-24T06:36:30.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:29 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:30.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:29 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:30.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:29 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:30.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:29 smithi137 ceph-mon[33017]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T06:36:30.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:29 smithi137 ceph-mon[33017]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T06:36:30.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:29 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T06:36:30.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:29 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T06:36:30.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:29 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T06:36:30.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:29 smithi137 ceph-mon[33017]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T06:36:30.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:29 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T06:36:30.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:29 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T06:36:30.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:29 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T06:36:30.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:29 smithi137 ceph-mon[33017]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T06:36:30.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:29 smithi137 ceph-mon[33017]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T06:36:30.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:29 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T06:36:30.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:29 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T06:36:30.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:29 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T06:36:30.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:29 smithi137 ceph-mon[33017]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T06:36:31.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:30 smithi005 ceph-mon[27798]: pgmap v94: 1 pgs: 1 creating+peering; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:31.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:30 smithi005 ceph-mon[27798]: osdmap e21: 3 total, 3 up, 3 in 2024-04-24T06:36:31.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:30 smithi005 ceph-mon[27798]: from='client.24200 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:36:31.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:30 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T06:36:31.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:30 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T06:36:31.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:30 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:36:31.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:30 smithi005 ceph-mon[27798]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-04-24T06:36:31.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:30 smithi037 ceph-mon[32864]: pgmap v94: 1 pgs: 1 creating+peering; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:31.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:30 smithi037 ceph-mon[32864]: osdmap e21: 3 total, 3 up, 3 in 2024-04-24T06:36:31.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:30 smithi037 ceph-mon[32864]: from='client.24200 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:36:31.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:30 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T06:36:31.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:30 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T06:36:31.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:30 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:36:31.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:30 smithi037 ceph-mon[32864]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-04-24T06:36:31.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:30 smithi137 ceph-mon[33017]: pgmap v94: 1 pgs: 1 creating+peering; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:31.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:30 smithi137 ceph-mon[33017]: osdmap e21: 3 total, 3 up, 3 in 2024-04-24T06:36:31.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:30 smithi137 ceph-mon[33017]: from='client.24200 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:36:31.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:30 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T06:36:31.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:30 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T06:36:31.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:30 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:36:31.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:30 smithi137 ceph-mon[33017]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-04-24T06:36:33.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:32 smithi005 ceph-mon[27798]: pgmap v96: 1 pgs: 1 creating+peering; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:33.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:32 smithi037 ceph-mon[32864]: pgmap v96: 1 pgs: 1 creating+peering; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:33.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:32 smithi137 ceph-mon[33017]: pgmap v96: 1 pgs: 1 creating+peering; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:34.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:33 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/634548881' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e05592ba-d96a-472a-9f63-09a535d04dbb"}]: dispatch 2024-04-24T06:36:34.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:33 smithi005 ceph-mon[27798]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e05592ba-d96a-472a-9f63-09a535d04dbb"}]: dispatch 2024-04-24T06:36:34.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:33 smithi005 ceph-mon[27798]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e05592ba-d96a-472a-9f63-09a535d04dbb"}]': finished 2024-04-24T06:36:34.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:33 smithi005 ceph-mon[27798]: osdmap e22: 4 total, 3 up, 4 in 2024-04-24T06:36:34.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:33 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T06:36:34.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:33 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/634548881' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e05592ba-d96a-472a-9f63-09a535d04dbb"}]: dispatch 2024-04-24T06:36:34.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:33 smithi037 ceph-mon[32864]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e05592ba-d96a-472a-9f63-09a535d04dbb"}]: dispatch 2024-04-24T06:36:34.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:33 smithi037 ceph-mon[32864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e05592ba-d96a-472a-9f63-09a535d04dbb"}]': finished 2024-04-24T06:36:34.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:33 smithi037 ceph-mon[32864]: osdmap e22: 4 total, 3 up, 4 in 2024-04-24T06:36:34.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:33 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T06:36:34.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:34 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/634548881' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e05592ba-d96a-472a-9f63-09a535d04dbb"}]: dispatch 2024-04-24T06:36:34.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:34 smithi137 ceph-mon[33017]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e05592ba-d96a-472a-9f63-09a535d04dbb"}]: dispatch 2024-04-24T06:36:34.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:34 smithi137 ceph-mon[33017]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e05592ba-d96a-472a-9f63-09a535d04dbb"}]': finished 2024-04-24T06:36:34.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:34 smithi137 ceph-mon[33017]: osdmap e22: 4 total, 3 up, 4 in 2024-04-24T06:36:34.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:34 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T06:36:35.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:35 smithi005 ceph-mon[27798]: pgmap v97: 1 pgs: 1 creating+peering; 0 B data, 101 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:35.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:35 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/3759259343' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T06:36:35.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:35 smithi037 ceph-mon[32864]: pgmap v97: 1 pgs: 1 creating+peering; 0 B data, 101 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:35.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:35 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/3759259343' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T06:36:35.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:35 smithi137 ceph-mon[33017]: pgmap v97: 1 pgs: 1 creating+peering; 0 B data, 101 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:35.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:35 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/3759259343' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T06:36:37.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:37 smithi137 ceph-mon[33017]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 104 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:37.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:37 smithi037 ceph-mon[32864]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 104 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:37.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:37 smithi005 ceph-mon[27798]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 104 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:39.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:39 smithi005 ceph-mon[27798]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 104 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:39.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:39 smithi037 ceph-mon[32864]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 104 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:39.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:39 smithi137 ceph-mon[33017]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 104 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:41.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:41 smithi037 ceph-mon[32864]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:41.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:41 smithi005 ceph-mon[27798]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:41.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:41 smithi137 ceph-mon[33017]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:43.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:43 smithi005 ceph-mon[27798]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:43.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:43 smithi137 ceph-mon[33017]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:43.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:43 smithi037 ceph-mon[32864]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:44.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:44 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T06:36:44.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:44 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:36:44.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:44 smithi037 ceph-mon[32864]: Deploying daemon osd.3 on smithi005 2024-04-24T06:36:44.319 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:44 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T06:36:44.319 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:44 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:36:44.319 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:44 smithi005 ceph-mon[27798]: Deploying daemon osd.3 on smithi005 2024-04-24T06:36:44.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:44 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T06:36:44.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:44 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:36:44.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:44 smithi137 ceph-mon[33017]: Deploying daemon osd.3 on smithi005 2024-04-24T06:36:45.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:45 smithi005 ceph-mon[27798]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:45.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:45 smithi137 ceph-mon[33017]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:45.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:45 smithi037 ceph-mon[32864]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:47.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:47 smithi005 ceph-mon[27798]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:47.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:47 smithi137 ceph-mon[33017]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:47.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:47 smithi037 ceph-mon[32864]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:49.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:49 smithi137 ceph-mon[33017]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:49.510 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:49 smithi005 ceph-mon[27798]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:49.522 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:49 smithi037 ceph-mon[32864]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:51.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:51 smithi137 ceph-mon[33017]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:51.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:51 smithi005 ceph-mon[27798]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:51.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:51 smithi037 ceph-mon[32864]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:52.510 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:52 smithi005 ceph-mon[27798]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:52.510 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:52 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:52.510 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:52 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:52.510 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:52 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:36:52.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:52 smithi137 ceph-mon[33017]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:52.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:52 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:52.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:52 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:52.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:52 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:36:52.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:52 smithi037 ceph-mon[32864]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:52.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:52 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:52.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:52 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:52.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:52 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:36:54.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:54 smithi137 ceph-mon[33017]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:54.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:54 smithi005 ceph-mon[27798]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:54.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:54 smithi037 ceph-mon[32864]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:55.872 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 3 on host 'smithi005' 2024-04-24T06:36:55.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:55.869+0000 7f3f32000700 1 -- 172.21.15.5:0/912110972 <== mgr.14152 v2:172.21.15.5:6800/4042936965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3f34076440 con 0x7f3f180a3c30 2024-04-24T06:36:55.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:55.873+0000 7f3f3a200700 1 -- 172.21.15.5:0/912110972 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f3f180a3c30 msgr2=0x7f3f180a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:55.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:55.873+0000 7f3f3a200700 1 --2- 172.21.15.5:0/912110972 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f3f180a3c30 0x7f3f180a60f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3f34103ad0 tx=0x7f3f24009260 comp rx=0 tx=0).stop 2024-04-24T06:36:55.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:55.873+0000 7f3f3a200700 1 -- 172.21.15.5:0/912110972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f34104610 msgr2=0x7f3f341a23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:36:55.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:55.873+0000 7f3f3a200700 1 --2- 172.21.15.5:0/912110972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f34104610 0x7f3f341a23f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c00b780 tx=0x7f3f2c004f40 comp rx=0 tx=0).stop 2024-04-24T06:36:55.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:55.873+0000 7f3f3a200700 1 -- 172.21.15.5:0/912110972 shutdown_connections 2024-04-24T06:36:55.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:55.873+0000 7f3f3a200700 1 --2- 172.21.15.5:0/912110972 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f3f180a3c30 0x7f3f180a60f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:55.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:55.873+0000 7f3f3a200700 1 --2- 172.21.15.5:0/912110972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f34104610 0x7f3f341a23f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:55.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:55.873+0000 7f3f3a200700 1 --2- 172.21.15.5:0/912110972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f34102a70 0x7f3f3419ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:55.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:55.873+0000 7f3f3a200700 1 --2- 172.21.15.5:0/912110972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f34103c70 0x7f3f3419b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:36:55.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:55.873+0000 7f3f3a200700 1 -- 172.21.15.5:0/912110972 >> 172.21.15.5:0/912110972 conn(0x7f3f340fe030 msgr2=0x7f3f34074d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:36:55.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:55.873+0000 7f3f3a200700 1 -- 172.21.15.5:0/912110972 shutdown_connections 2024-04-24T06:36:55.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:36:55.874+0000 7f3f3a200700 1 -- 172.21.15.5:0/912110972 wait complete. 2024-04-24T06:36:56.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:55 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:56.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:55 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:56.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:55 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:36:56.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:55 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:36:56.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:55 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:56.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:55 smithi005 ceph-mon[27798]: from='osd.3 [v2:172.21.15.5:6810/3701747260,v1:172.21.15.5:6811/3701747260]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T06:36:56.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:55 smithi005 ceph-mon[27798]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T06:36:56.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:55 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:56.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:55 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:56.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:55 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:36:56.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:55 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:36:56.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:55 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:56.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:55 smithi037 ceph-mon[32864]: from='osd.3 [v2:172.21.15.5:6810/3701747260,v1:172.21.15.5:6811/3701747260]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T06:36:56.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:55 smithi037 ceph-mon[32864]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T06:36:56.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:55 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:56.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:55 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:56.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:55 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:36:56.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:55 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:36:56.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:55 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:56.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:55 smithi137 ceph-mon[33017]: from='osd.3 [v2:172.21.15.5:6810/3701747260,v1:172.21.15.5:6811/3701747260]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T06:36:56.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:55 smithi137 ceph-mon[33017]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T06:36:56.681 DEBUG:teuthology.orchestra.run.smithi005:osd.3> sudo journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@osd.3.service 2024-04-24T06:36:56.685 INFO:tasks.cephadm:Deploying osd.4 on smithi037 with /dev/vg_nvme/lv_3... 2024-04-24T06:36:56.686 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-24T06:36:56.944 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T06:36:56.947 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:56 smithi005 ceph-mon[27798]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:56.947 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:56 smithi005 ceph-mon[27798]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T06:36:56.947 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:56 smithi005 ceph-mon[27798]: from='osd.3 [v2:172.21.15.5:6810/3701747260,v1:172.21.15.5:6811/3701747260]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-04-24T06:36:56.947 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:56 smithi005 ceph-mon[27798]: osdmap e23: 4 total, 3 up, 4 in 2024-04-24T06:36:56.947 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:56 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T06:36:56.947 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:56 smithi005 ceph-mon[27798]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-04-24T06:36:56.948 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:56 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:56.948 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:56 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:56.948 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:56 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:36:56.948 INFO:journalctl@ceph.osd.3.smithi005.stdout:Apr 24 06:36:56 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-3[46366]: 2024-04-24T06:36:56.773+0000 7f89e1000700 -1 osd.3 0 waiting for initial osdmap 2024-04-24T06:36:56.948 INFO:journalctl@ceph.osd.3.smithi005.stdout:Apr 24 06:36:56 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-3[46366]: 2024-04-24T06:36:56.781+0000 7f89dbe00700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T06:36:57.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:56 smithi037 ceph-mon[32864]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:57.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:56 smithi037 ceph-mon[32864]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T06:36:57.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:56 smithi037 ceph-mon[32864]: from='osd.3 [v2:172.21.15.5:6810/3701747260,v1:172.21.15.5:6811/3701747260]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-04-24T06:36:57.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:56 smithi037 ceph-mon[32864]: osdmap e23: 4 total, 3 up, 4 in 2024-04-24T06:36:57.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:56 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T06:36:57.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:56 smithi037 ceph-mon[32864]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-04-24T06:36:57.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:56 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:57.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:56 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:57.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:56 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:36:57.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:56 smithi137 ceph-mon[33017]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:57.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:56 smithi137 ceph-mon[33017]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T06:36:57.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:56 smithi137 ceph-mon[33017]: from='osd.3 [v2:172.21.15.5:6810/3701747260,v1:172.21.15.5:6811/3701747260]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-04-24T06:36:57.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:56 smithi137 ceph-mon[33017]: osdmap e23: 4 total, 3 up, 4 in 2024-04-24T06:36:57.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:56 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T06:36:57.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:56 smithi137 ceph-mon[33017]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-04-24T06:36:57.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:56 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:57.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:56 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:36:57.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:56 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:36:58.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:57 smithi005 ceph-mon[27798]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-04-24T06:36:58.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:57 smithi005 ceph-mon[27798]: osdmap e24: 4 total, 3 up, 4 in 2024-04-24T06:36:58.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:57 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T06:36:58.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:57 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T06:36:58.127 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:57 smithi037 ceph-mon[32864]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-04-24T06:36:58.127 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:57 smithi037 ceph-mon[32864]: osdmap e24: 4 total, 3 up, 4 in 2024-04-24T06:36:58.127 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:57 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T06:36:58.127 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:57 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T06:36:58.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:57 smithi137 ceph-mon[33017]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-04-24T06:36:58.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:57 smithi137 ceph-mon[33017]: osdmap e24: 4 total, 3 up, 4 in 2024-04-24T06:36:58.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:57 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T06:36:58.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:57 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T06:36:58.864 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-24T06:36:58.864 INFO:teuthology.orchestra.run.smithi037.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-24T06:36:58.864 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10+0 records in 2024-04-24T06:36:58.864 INFO:teuthology.orchestra.run.smithi037.stderr:10+0 records out 2024-04-24T06:36:58.864 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112764 s, 930 MB/s 2024-04-24T06:36:58.864 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping successful for: 2024-04-24T06:36:59.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:58 smithi005 ceph-mon[27798]: purged_snaps scrub starts 2024-04-24T06:36:59.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:58 smithi005 ceph-mon[27798]: purged_snaps scrub ok 2024-04-24T06:36:59.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:58 smithi005 ceph-mon[27798]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:59.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:58 smithi005 ceph-mon[27798]: osd.3 [v2:172.21.15.5:6810/3701747260,v1:172.21.15.5:6811/3701747260] boot 2024-04-24T06:36:59.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:58 smithi005 ceph-mon[27798]: osdmap e25: 4 total, 4 up, 4 in 2024-04-24T06:36:59.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:58 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T06:36:59.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:36:58 smithi005 ceph-mon[27798]: osdmap e26: 4 total, 4 up, 4 in 2024-04-24T06:36:59.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:58 smithi137 ceph-mon[33017]: purged_snaps scrub starts 2024-04-24T06:36:59.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:58 smithi137 ceph-mon[33017]: purged_snaps scrub ok 2024-04-24T06:36:59.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:58 smithi137 ceph-mon[33017]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:59.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:58 smithi137 ceph-mon[33017]: osd.3 [v2:172.21.15.5:6810/3701747260,v1:172.21.15.5:6811/3701747260] boot 2024-04-24T06:36:59.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:58 smithi137 ceph-mon[33017]: osdmap e25: 4 total, 4 up, 4 in 2024-04-24T06:36:59.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:58 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T06:36:59.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:36:58 smithi137 ceph-mon[33017]: osdmap e26: 4 total, 4 up, 4 in 2024-04-24T06:36:59.220 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:58 smithi037 ceph-mon[32864]: purged_snaps scrub starts 2024-04-24T06:36:59.220 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:58 smithi037 ceph-mon[32864]: purged_snaps scrub ok 2024-04-24T06:36:59.220 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:58 smithi037 ceph-mon[32864]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-24T06:36:59.220 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:58 smithi037 ceph-mon[32864]: osd.3 [v2:172.21.15.5:6810/3701747260,v1:172.21.15.5:6811/3701747260] boot 2024-04-24T06:36:59.220 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:58 smithi037 ceph-mon[32864]: osdmap e25: 4 total, 4 up, 4 in 2024-04-24T06:36:59.220 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:58 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T06:36:59.220 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:36:58 smithi037 ceph-mon[32864]: osdmap e26: 4 total, 4 up, 4 in 2024-04-24T06:36:59.535 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi037:vg_nvme/lv_3 2024-04-24T06:36:59.789 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T06:37:00.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:00 smithi005 ceph-mon[27798]: pgmap v115: 1 pgs: 1 remapped+peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-04-24T06:37:00.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:00 smithi005 ceph-mon[27798]: osdmap e27: 4 total, 4 up, 4 in 2024-04-24T06:37:00.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:00 smithi005 ceph-mon[27798]: osdmap e28: 4 total, 4 up, 4 in 2024-04-24T06:37:00.927 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:00 smithi037 ceph-mon[32864]: pgmap v115: 1 pgs: 1 remapped+peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-04-24T06:37:00.927 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:00 smithi037 ceph-mon[32864]: osdmap e27: 4 total, 4 up, 4 in 2024-04-24T06:37:00.927 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:00 smithi037 ceph-mon[32864]: osdmap e28: 4 total, 4 up, 4 in 2024-04-24T06:37:01.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:00 smithi137 ceph-mon[33017]: pgmap v115: 1 pgs: 1 remapped+peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-04-24T06:37:01.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:00 smithi137 ceph-mon[33017]: osdmap e27: 4 total, 4 up, 4 in 2024-04-24T06:37:01.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:00 smithi137 ceph-mon[33017]: osdmap e28: 4 total, 4 up, 4 in 2024-04-24T06:37:01.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:01.411+0000 7f31885d0700 1 -- 172.21.15.37:0/1340669724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3180102770 msgr2=0x7f3180102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:01.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:01.411+0000 7f31885d0700 1 --2- 172.21.15.37:0/1340669724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3180102770 0x7f3180102bf0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3178009990 tx=0x7f3178009ca0 comp rx=0 tx=0).stop 2024-04-24T06:37:01.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:01.411+0000 7f31885d0700 1 -- 172.21.15.37:0/1340669724 shutdown_connections 2024-04-24T06:37:01.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:01.411+0000 7f31885d0700 1 --2- 172.21.15.37:0/1340669724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3180102770 0x7f3180102bf0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:01.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:01.411+0000 7f31885d0700 1 --2- 172.21.15.37:0/1340669724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3180101450 0x7f3180101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:01.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:01.411+0000 7f31885d0700 1 --2- 172.21.15.37:0/1340669724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3180101db0 0x7f3180102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:01.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:01.411+0000 7f31885d0700 1 -- 172.21.15.37:0/1340669724 >> 172.21.15.37:0/1340669724 conn(0x7f31800fcff0 msgr2=0x7f31800ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:01.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:01.411+0000 7f31885d0700 1 -- 172.21.15.37:0/1340669724 shutdown_connections 2024-04-24T06:37:01.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:01.411+0000 7f31885d0700 1 -- 172.21.15.37:0/1340669724 wait complete. 2024-04-24T06:37:01.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:01.412+0000 7f31885d0700 1 Processor -- start 2024-04-24T06:37:01.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:01.412+0000 7f31885d0700 1 -- start start 2024-04-24T06:37:01.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:01.412+0000 7f31885d0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3180101450 0x7f318019b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:01.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:01.412+0000 7f31885d0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3180101db0 0x7f318019b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:01.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:01.412+0000 7f31885d0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3180102770 0x7f31801a2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:01.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:01.412+0000 7f31885d0700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f31801109f0 con 0x7f3180101450 2024-04-24T06:37:01.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:01.412+0000 7f31885d0700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3180110870 con 0x7f3180101db0 2024-04-24T06:37:01.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:01.412+0000 7f31885d0700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3180110b70 con 0x7f3180102770 2024-04-24T06:37:01.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:01.412+0000 7f3185b6b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3180101db0 0x7f318019b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:01.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:01.412+0000 7f3185b6b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3180101db0 0x7f318019b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36774/0 (socket says 172.21.15.37:36774) 2024-04-24T06:37:01.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:01.412+0000 7f3185b6b700 1 -- 172.21.15.37:0/3305650951 learned_addr learned my addr 172.21.15.37:0/3305650951 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-24T06:37:01.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:01.412+0000 7f3185b6b700 1 -- 172.21.15.37:0/3305650951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3180102770 msgr2=0x7f31801a2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:01.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:01.412+0000 7f3185b6b700 1 --2- 172.21.15.37:0/3305650951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3180102770 0x7f31801a2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:01.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:01.412+0000 7f3185b6b700 1 -- 172.21.15.37:0/3305650951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3180101450 msgr2=0x7f318019b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:01.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:01.412+0000 7f3185b6b700 1 --2- 172.21.15.37:0/3305650951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3180101450 0x7f318019b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:01.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:01.412+0000 7f3185b6b700 1 -- 172.21.15.37:0/3305650951 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31801a2cf0 con 0x7f3180101db0 2024-04-24T06:37:01.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:01.413+0000 7f3185b6b700 1 --2- 172.21.15.37:0/3305650951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3180101db0 0x7f318019b5d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f317000b530 tx=0x7f317000b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:01.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:01.413+0000 7f31777fe700 1 -- 172.21.15.37:0/3305650951 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3170015020 con 0x7f3180101db0 2024-04-24T06:37:01.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:01.413+0000 7f31777fe700 1 -- 172.21.15.37:0/3305650951 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3170011d10 con 0x7f3180101db0 2024-04-24T06:37:01.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:01.413+0000 7f31885d0700 1 -- 172.21.15.37:0/3305650951 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31801a2fe0 con 0x7f3180101db0 2024-04-24T06:37:01.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:01.413+0000 7f31777fe700 1 -- 172.21.15.37:0/3305650951 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317000f2e0 con 0x7f3180101db0 2024-04-24T06:37:01.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:01.413+0000 7f31885d0700 1 -- 172.21.15.37:0/3305650951 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31801a34c0 con 0x7f3180101db0 2024-04-24T06:37:01.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:01.414+0000 7f31885d0700 1 -- 172.21.15.37:0/3305650951 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31800697e0 con 0x7f3180101db0 2024-04-24T06:37:01.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:01.415+0000 7f31777fe700 1 -- 172.21.15.37:0/3305650951 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f3170018020 con 0x7f3180101db0 2024-04-24T06:37:01.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:01.415+0000 7f31777fe700 1 --2- 172.21.15.37:0/3305650951 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f316c0a3c40 0x7f316c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:01.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:01.416+0000 7f31777fe700 1 -- 172.21.15.37:0/3305650951 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3669+0+0 (secure 0 0 0) 0x7f31700cea00 con 0x7f3180101db0 2024-04-24T06:37:01.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:01.416+0000 7f318636c700 1 --2- 172.21.15.37:0/3305650951 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f316c0a3c40 0x7f316c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:01.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:01.417+0000 7f318636c700 1 --2- 172.21.15.37:0/3305650951 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f316c0a3c40 0x7f316c0a6100 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f317c00ab90 tx=0x7f317c009250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:01.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:01.420+0000 7f31777fe700 1 -- 172.21.15.37:0/3305650951 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f317001f020 con 0x7f3180101db0 2024-04-24T06:37:01.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:01.639+0000 7f31885d0700 1 -- 172.21.15.37:0/3305650951 --> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f3180062f30 con 0x7f316c0a3c40 2024-04-24T06:37:02.649 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:02 smithi037 ceph-mon[32864]: pgmap v118: 1 pgs: 1 remapped+peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-04-24T06:37:02.649 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:02 smithi037 ceph-mon[32864]: Detected new or changed devices on smithi005 2024-04-24T06:37:02.649 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:02 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:02.650 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:02 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:02.650 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:02 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:37:02.650 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:02 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:37:02.650 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:02 smithi037 ceph-mon[32864]: Adjusting osd_memory_target on smithi005 to 8547M 2024-04-24T06:37:02.650 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:02 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:02.650 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:02 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:37:02.650 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:02 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:37:02.650 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:02 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:02.650 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:02 smithi037 ceph-mon[32864]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:37:02.650 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:02 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T06:37:02.650 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:02 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T06:37:02.650 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:02 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:37:02.650 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:02 smithi037 ceph-mon[32864]: osdmap e29: 4 total, 4 up, 4 in 2024-04-24T06:37:02.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:02 smithi137 ceph-mon[33017]: pgmap v118: 1 pgs: 1 remapped+peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-04-24T06:37:02.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:02 smithi137 ceph-mon[33017]: Detected new or changed devices on smithi005 2024-04-24T06:37:02.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:02 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:02.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:02 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:02.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:02 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:37:02.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:02 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:37:02.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:02 smithi137 ceph-mon[33017]: Adjusting osd_memory_target on smithi005 to 8547M 2024-04-24T06:37:02.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:02 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:02.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:02 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:37:02.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:02 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:37:02.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:02 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:02.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:02 smithi137 ceph-mon[33017]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:37:02.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:02 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T06:37:02.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:02 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T06:37:02.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:02 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:37:02.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:02 smithi137 ceph-mon[33017]: osdmap e29: 4 total, 4 up, 4 in 2024-04-24T06:37:02.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:02 smithi005 ceph-mon[27798]: pgmap v118: 1 pgs: 1 remapped+peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-04-24T06:37:02.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:02 smithi005 ceph-mon[27798]: Detected new or changed devices on smithi005 2024-04-24T06:37:02.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:02 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:02.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:02 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:02.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:02 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:37:02.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:02 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:37:02.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:02 smithi005 ceph-mon[27798]: Adjusting osd_memory_target on smithi005 to 8547M 2024-04-24T06:37:02.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:02 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:02.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:02 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:37:02.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:02 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:37:02.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:02 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:02.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:02 smithi005 ceph-mon[27798]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:37:02.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:02 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T06:37:02.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:02 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T06:37:02.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:02 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:37:02.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:02 smithi005 ceph-mon[27798]: osdmap e29: 4 total, 4 up, 4 in 2024-04-24T06:37:04.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:04 smithi137 ceph-mon[33017]: pgmap v120: 1 pgs: 1 remapped+peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-04-24T06:37:04.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:04 smithi005 ceph-mon[27798]: pgmap v120: 1 pgs: 1 remapped+peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-04-24T06:37:04.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:04 smithi037 ceph-mon[32864]: pgmap v120: 1 pgs: 1 remapped+peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-04-24T06:37:05.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:05 smithi137 ceph-mon[33017]: from='client.? 172.21.15.37:0/2578025696' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3f36e51-7a3b-4dd9-af4b-a10558830c4c"}]: dispatch 2024-04-24T06:37:05.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:05 smithi137 ceph-mon[33017]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3f36e51-7a3b-4dd9-af4b-a10558830c4c"}]: dispatch 2024-04-24T06:37:05.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:05 smithi137 ceph-mon[33017]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3f36e51-7a3b-4dd9-af4b-a10558830c4c"}]': finished 2024-04-24T06:37:05.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:05 smithi137 ceph-mon[33017]: osdmap e30: 5 total, 4 up, 5 in 2024-04-24T06:37:05.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:05 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T06:37:05.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:05 smithi137 ceph-mon[33017]: from='client.? 172.21.15.37:0/1037185910' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T06:37:05.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:05 smithi005 ceph-mon[27798]: from='client.? 172.21.15.37:0/2578025696' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3f36e51-7a3b-4dd9-af4b-a10558830c4c"}]: dispatch 2024-04-24T06:37:05.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:05 smithi005 ceph-mon[27798]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3f36e51-7a3b-4dd9-af4b-a10558830c4c"}]: dispatch 2024-04-24T06:37:05.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:05 smithi005 ceph-mon[27798]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3f36e51-7a3b-4dd9-af4b-a10558830c4c"}]': finished 2024-04-24T06:37:05.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:05 smithi005 ceph-mon[27798]: osdmap e30: 5 total, 4 up, 5 in 2024-04-24T06:37:05.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:05 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T06:37:05.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:05 smithi005 ceph-mon[27798]: from='client.? 172.21.15.37:0/1037185910' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T06:37:05.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:05 smithi037 ceph-mon[32864]: from='client.? 172.21.15.37:0/2578025696' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3f36e51-7a3b-4dd9-af4b-a10558830c4c"}]: dispatch 2024-04-24T06:37:05.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:05 smithi037 ceph-mon[32864]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3f36e51-7a3b-4dd9-af4b-a10558830c4c"}]: dispatch 2024-04-24T06:37:05.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:05 smithi037 ceph-mon[32864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3f36e51-7a3b-4dd9-af4b-a10558830c4c"}]': finished 2024-04-24T06:37:05.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:05 smithi037 ceph-mon[32864]: osdmap e30: 5 total, 4 up, 5 in 2024-04-24T06:37:05.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:05 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T06:37:05.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:05 smithi037 ceph-mon[32864]: from='client.? 172.21.15.37:0/1037185910' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T06:37:06.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:06 smithi005 ceph-mon[27798]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-04-24T06:37:06.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:06 smithi037 ceph-mon[32864]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-04-24T06:37:06.913 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:06 smithi137 ceph-mon[33017]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-04-24T06:37:08.727 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:08 smithi037 ceph-mon[32864]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-04-24T06:37:08.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:08 smithi005 ceph-mon[27798]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-04-24T06:37:08.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:08 smithi137 ceph-mon[33017]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-04-24T06:37:10.752 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:10 smithi037 ceph-mon[32864]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T06:37:10.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:10 smithi005 ceph-mon[27798]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T06:37:10.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:10 smithi137 ceph-mon[33017]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T06:37:12.742 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:12 smithi037 ceph-mon[32864]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-24T06:37:12.742 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:12 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T06:37:12.743 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:12 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:37:12.743 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:12 smithi037 ceph-mon[32864]: Deploying daemon osd.4 on smithi037 2024-04-24T06:37:12.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:12 smithi005 ceph-mon[27798]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-24T06:37:12.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:12 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T06:37:12.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:12 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:37:12.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:12 smithi005 ceph-mon[27798]: Deploying daemon osd.4 on smithi037 2024-04-24T06:37:12.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:12 smithi137 ceph-mon[33017]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-24T06:37:12.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:12 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T06:37:12.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:12 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:37:12.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:12 smithi137 ceph-mon[33017]: Deploying daemon osd.4 on smithi037 2024-04-24T06:37:14.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:14 smithi005 ceph-mon[27798]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T06:37:14.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:14 smithi037 ceph-mon[32864]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T06:37:14.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:14 smithi137 ceph-mon[33017]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T06:37:16.618 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:16 smithi037 ceph-mon[32864]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-24T06:37:16.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:16 smithi137 ceph-mon[33017]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-24T06:37:17.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:16 smithi005 ceph-mon[27798]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-24T06:37:17.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:17 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:37:17.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:17 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:17.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:17 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:17.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:17 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:37:17.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:17 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:17.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:17 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:18.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:17 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:37:18.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:17 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:18.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:17 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:18.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:18 smithi037 ceph-mon[32864]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-24T06:37:18.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:18 smithi137 ceph-mon[33017]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-24T06:37:19.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:18 smithi005 ceph-mon[27798]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-24T06:37:20.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:20.827+0000 7f31777fe700 1 -- 172.21.15.37:0/3305650951 <== mgr.14152 v2:172.21.15.5:6800/4042936965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3180062f30 con 0x7f316c0a3c40 2024-04-24T06:37:20.828 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 4 on host 'smithi037' 2024-04-24T06:37:20.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:20.829+0000 7f31885d0700 1 -- 172.21.15.37:0/3305650951 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f316c0a3c40 msgr2=0x7f316c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:20.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:20.829+0000 7f31885d0700 1 --2- 172.21.15.37:0/3305650951 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f316c0a3c40 0x7f316c0a6100 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f317c00ab90 tx=0x7f317c009250 comp rx=0 tx=0).stop 2024-04-24T06:37:20.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:20.829+0000 7f31885d0700 1 -- 172.21.15.37:0/3305650951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3180101db0 msgr2=0x7f318019b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:20.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:20.829+0000 7f31885d0700 1 --2- 172.21.15.37:0/3305650951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3180101db0 0x7f318019b5d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f317000b530 tx=0x7f317000b8f0 comp rx=0 tx=0).stop 2024-04-24T06:37:20.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:20.829+0000 7f31885d0700 1 -- 172.21.15.37:0/3305650951 shutdown_connections 2024-04-24T06:37:20.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:20.829+0000 7f31885d0700 1 --2- 172.21.15.37:0/3305650951 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f316c0a3c40 0x7f316c0a6100 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:20.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:20.829+0000 7f31885d0700 1 --2- 172.21.15.37:0/3305650951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3180102770 0x7f31801a2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:20.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:20.829+0000 7f31885d0700 1 --2- 172.21.15.37:0/3305650951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3180101450 0x7f318019b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:20.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:20.829+0000 7f31885d0700 1 --2- 172.21.15.37:0/3305650951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3180101db0 0x7f318019b5d0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:20.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:20.829+0000 7f31885d0700 1 -- 172.21.15.37:0/3305650951 >> 172.21.15.37:0/3305650951 conn(0x7f31800fcff0 msgr2=0x7f31800feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:20.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:20.829+0000 7f31885d0700 1 -- 172.21.15.37:0/3305650951 shutdown_connections 2024-04-24T06:37:20.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:37:20.829+0000 7f31885d0700 1 -- 172.21.15.37:0/3305650951 wait complete. 2024-04-24T06:37:21.417 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:21 smithi037 ceph-mon[32864]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-24T06:37:21.417 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:21 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:21.417 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:21 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:21.418 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:21 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:37:21.418 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:21 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:37:21.418 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:21 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:21.418 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:21 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:21.418 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:21 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:21.418 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:21 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:37:21.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:21 smithi137 ceph-mon[33017]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-24T06:37:21.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:21 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:21.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:21 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:21.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:21 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:37:21.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:21 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:37:21.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:21 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:21.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:21 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:21.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:21 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:21.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:21 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:37:21.448 DEBUG:teuthology.orchestra.run.smithi037:osd.4> sudo journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@osd.4.service 2024-04-24T06:37:21.450 INFO:tasks.cephadm:Deploying osd.5 on smithi137 with /dev/vg_nvme/lv_3... 2024-04-24T06:37:21.450 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-24T06:37:21.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:21 smithi005 ceph-mon[27798]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-24T06:37:21.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:21 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:21.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:21 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:21.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:21 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:37:21.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:21 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:37:21.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:21 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:21.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:21 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:21.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:21 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:21.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:21 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:37:21.693 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T06:37:22.111 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:22 smithi137 ceph-mon[33017]: from='osd.4 [v2:172.21.15.37:6808/759253844,v1:172.21.15.37:6809/759253844]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T06:37:22.111 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:22 smithi137 ceph-mon[33017]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T06:37:22.417 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:22 smithi037 ceph-mon[32864]: from='osd.4 [v2:172.21.15.37:6808/759253844,v1:172.21.15.37:6809/759253844]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T06:37:22.417 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:22 smithi037 ceph-mon[32864]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T06:37:22.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:22 smithi005 ceph-mon[27798]: from='osd.4 [v2:172.21.15.37:6808/759253844,v1:172.21.15.37:6809/759253844]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T06:37:22.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:22 smithi005 ceph-mon[27798]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T06:37:23.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:23 smithi137 ceph-mon[33017]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-24T06:37:23.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:23 smithi137 ceph-mon[33017]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T06:37:23.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:23 smithi137 ceph-mon[33017]: from='osd.4 [v2:172.21.15.37:6808/759253844,v1:172.21.15.37:6809/759253844]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-24T06:37:23.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:23 smithi137 ceph-mon[33017]: osdmap e31: 5 total, 4 up, 5 in 2024-04-24T06:37:23.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:23 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T06:37:23.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:23 smithi137 ceph-mon[33017]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-24T06:37:23.452 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-24T06:37:23.452 INFO:teuthology.orchestra.run.smithi137.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-24T06:37:23.452 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10+0 records in 2024-04-24T06:37:23.452 INFO:teuthology.orchestra.run.smithi137.stderr:10+0 records out 2024-04-24T06:37:23.452 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113172 s, 927 MB/s 2024-04-24T06:37:23.452 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping successful for: 2024-04-24T06:37:23.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:23 smithi005 ceph-mon[27798]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-24T06:37:23.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:23 smithi005 ceph-mon[27798]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T06:37:23.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:23 smithi005 ceph-mon[27798]: from='osd.4 [v2:172.21.15.37:6808/759253844,v1:172.21.15.37:6809/759253844]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-24T06:37:23.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:23 smithi005 ceph-mon[27798]: osdmap e31: 5 total, 4 up, 5 in 2024-04-24T06:37:23.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:23 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T06:37:23.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:23 smithi005 ceph-mon[27798]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-24T06:37:23.508 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:23 smithi037 ceph-mon[32864]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-24T06:37:23.508 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:23 smithi037 ceph-mon[32864]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T06:37:23.508 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:23 smithi037 ceph-mon[32864]: from='osd.4 [v2:172.21.15.37:6808/759253844,v1:172.21.15.37:6809/759253844]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-24T06:37:23.508 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:23 smithi037 ceph-mon[32864]: osdmap e31: 5 total, 4 up, 5 in 2024-04-24T06:37:23.508 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:23 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T06:37:23.508 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:23 smithi037 ceph-mon[32864]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-24T06:37:23.508 INFO:journalctl@ceph.osd.4.smithi037.stdout:Apr 24 06:37:23 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-4[44150]: 2024-04-24T06:37:23.045+0000 7f7c753c6700 -1 osd.4 0 waiting for initial osdmap 2024-04-24T06:37:23.508 INFO:journalctl@ceph.osd.4.smithi037.stdout:Apr 24 06:37:23 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-4[44150]: 2024-04-24T06:37:23.051+0000 7f7c711b9700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T06:37:24.051 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi137:vg_nvme/lv_3 2024-04-24T06:37:24.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:24 smithi037 ceph-mon[32864]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-24T06:37:24.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:24 smithi037 ceph-mon[32864]: osdmap e32: 5 total, 4 up, 5 in 2024-04-24T06:37:24.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:24 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T06:37:24.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:24 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T06:37:24.292 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T06:37:24.334 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:24 smithi137 ceph-mon[33017]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-24T06:37:24.334 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:24 smithi137 ceph-mon[33017]: osdmap e32: 5 total, 4 up, 5 in 2024-04-24T06:37:24.334 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:24 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T06:37:24.334 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:24 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T06:37:24.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:24 smithi005 ceph-mon[27798]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-24T06:37:24.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:24 smithi005 ceph-mon[27798]: osdmap e32: 5 total, 4 up, 5 in 2024-04-24T06:37:24.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:24 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T06:37:24.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:24 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T06:37:25.407 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:25 smithi137 ceph-mon[33017]: purged_snaps scrub starts 2024-04-24T06:37:25.407 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:25 smithi137 ceph-mon[33017]: purged_snaps scrub ok 2024-04-24T06:37:25.407 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:25 smithi137 ceph-mon[33017]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-24T06:37:25.407 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:25 smithi137 ceph-mon[33017]: osd.4 [v2:172.21.15.37:6808/759253844,v1:172.21.15.37:6809/759253844] boot 2024-04-24T06:37:25.408 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:25 smithi137 ceph-mon[33017]: osdmap e33: 5 total, 5 up, 5 in 2024-04-24T06:37:25.408 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:25 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T06:37:25.408 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:25 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:25.408 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:25 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:25.408 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:25 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:37:25.408 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:25 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:37:25.408 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:25 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:25.408 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:25 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:37:25.408 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:25 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:37:25.408 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:25 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:25.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:25 smithi005 ceph-mon[27798]: purged_snaps scrub starts 2024-04-24T06:37:25.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:25 smithi005 ceph-mon[27798]: purged_snaps scrub ok 2024-04-24T06:37:25.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:25 smithi005 ceph-mon[27798]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-24T06:37:25.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:25 smithi005 ceph-mon[27798]: osd.4 [v2:172.21.15.37:6808/759253844,v1:172.21.15.37:6809/759253844] boot 2024-04-24T06:37:25.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:25 smithi005 ceph-mon[27798]: osdmap e33: 5 total, 5 up, 5 in 2024-04-24T06:37:25.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:25 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T06:37:25.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:25 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:25.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:25 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:25.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:25 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:37:25.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:25 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:37:25.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:25 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:25.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:25 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:37:25.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:25 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:37:25.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:25 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:25.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:25 smithi037 ceph-mon[32864]: purged_snaps scrub starts 2024-04-24T06:37:25.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:25 smithi037 ceph-mon[32864]: purged_snaps scrub ok 2024-04-24T06:37:25.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:25 smithi037 ceph-mon[32864]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-24T06:37:25.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:25 smithi037 ceph-mon[32864]: osd.4 [v2:172.21.15.37:6808/759253844,v1:172.21.15.37:6809/759253844] boot 2024-04-24T06:37:25.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:25 smithi037 ceph-mon[32864]: osdmap e33: 5 total, 5 up, 5 in 2024-04-24T06:37:25.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:25 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T06:37:25.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:25 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:25.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:25 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:25.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:25 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:37:25.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:25 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:37:25.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:25 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:25.521 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:25 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:37:25.521 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:25 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:37:25.521 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:25 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:25.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:25.740+0000 7f61c88e4700 1 -- 172.21.15.137:0/4202102097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61c4101a40 msgr2=0x7f61c4101e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:25.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:25.740+0000 7f61c88e4700 1 --2- 172.21.15.137:0/4202102097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61c4101a40 0x7f61c4101e60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f61b400b0a0 tx=0x7f61b400b3b0 comp rx=0 tx=0).stop 2024-04-24T06:37:25.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:25.740+0000 7f61c88e4700 1 -- 172.21.15.137:0/4202102097 shutdown_connections 2024-04-24T06:37:25.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:25.740+0000 7f61c88e4700 1 --2- 172.21.15.137:0/4202102097 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61c41035e0 0x7f61c4109e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:25.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:25.740+0000 7f61c88e4700 1 --2- 172.21.15.137:0/4202102097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61c4101a40 0x7f61c4101e60 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:25.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:25.740+0000 7f61c88e4700 1 --2- 172.21.15.137:0/4202102097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61c4102c40 0x7f61c41030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:25.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:25.740+0000 7f61c88e4700 1 -- 172.21.15.137:0/4202102097 >> 172.21.15.137:0/4202102097 conn(0x7f61c40fcfe0 msgr2=0x7f61c40ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:25.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:25.740+0000 7f61c88e4700 1 -- 172.21.15.137:0/4202102097 shutdown_connections 2024-04-24T06:37:25.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:25.740+0000 7f61c88e4700 1 -- 172.21.15.137:0/4202102097 wait complete. 2024-04-24T06:37:25.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:25.741+0000 7f61c88e4700 1 Processor -- start 2024-04-24T06:37:25.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:25.741+0000 7f61c88e4700 1 -- start start 2024-04-24T06:37:25.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:25.741+0000 7f61c88e4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61c4101a40 0x7f61c4198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:25.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:25.741+0000 7f61c88e4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61c4102c40 0x7f61c4199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:25.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:25.741+0000 7f61c88e4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61c41035e0 0x7f61c41a0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:25.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:25.741+0000 7f61c88e4700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f61c410e800 con 0x7f61c41035e0 2024-04-24T06:37:25.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:25.741+0000 7f61c88e4700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f61c410e680 con 0x7f61c4102c40 2024-04-24T06:37:25.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:25.741+0000 7f61c88e4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f61c410e980 con 0x7f61c4101a40 2024-04-24T06:37:25.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:25.741+0000 7f61c259c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61c4101a40 0x7f61c4198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:25.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:25.741+0000 7f61c259c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61c4101a40 0x7f61c4198ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39912/0 (socket says 172.21.15.137:39912) 2024-04-24T06:37:25.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:25.741+0000 7f61c259c700 1 -- 172.21.15.137:0/1215540037 learned_addr learned my addr 172.21.15.137:0/1215540037 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-04-24T06:37:25.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:25.741+0000 7f61c1d9b700 1 --2- 172.21.15.137:0/1215540037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61c4102c40 0x7f61c4199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:25.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:25.741+0000 7f61c259c700 1 -- 172.21.15.137:0/1215540037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61c4102c40 msgr2=0x7f61c4199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:25.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:25.741+0000 7f61c259c700 1 --2- 172.21.15.137:0/1215540037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61c4102c40 0x7f61c4199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:25.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:25.741+0000 7f61c259c700 1 -- 172.21.15.137:0/1215540037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61c41035e0 msgr2=0x7f61c41a0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:25.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:25.741+0000 7f61c259c700 1 --2- 172.21.15.137:0/1215540037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61c41035e0 0x7f61c41a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:25.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:25.741+0000 7f61c259c700 1 -- 172.21.15.137:0/1215540037 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c41a0b30 con 0x7f61c4101a40 2024-04-24T06:37:25.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:25.741+0000 7f61c1d9b700 1 --2- 172.21.15.137:0/1215540037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61c4102c40 0x7f61c4199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:37:25.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:25.742+0000 7f61c259c700 1 --2- 172.21.15.137:0/1215540037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61c4101a40 0x7f61c4198ed0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f61b4006070 tx=0x7f61b4007820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:25.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:25.742+0000 7f61bb7fe700 1 -- 172.21.15.137:0/1215540037 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b4021030 con 0x7f61c4101a40 2024-04-24T06:37:25.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:25.742+0000 7f61bb7fe700 1 -- 172.21.15.137:0/1215540037 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61b4007cd0 con 0x7f61c4101a40 2024-04-24T06:37:25.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:25.742+0000 7f61c88e4700 1 -- 172.21.15.137:0/1215540037 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61c41a0dc0 con 0x7f61c4101a40 2024-04-24T06:37:25.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:25.742+0000 7f61bb7fe700 1 -- 172.21.15.137:0/1215540037 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b4005040 con 0x7f61c4101a40 2024-04-24T06:37:25.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:25.742+0000 7f61c88e4700 1 -- 172.21.15.137:0/1215540037 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61c41a1240 con 0x7f61c4101a40 2024-04-24T06:37:25.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:25.743+0000 7f61c88e4700 1 -- 172.21.15.137:0/1215540037 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61c402dfe0 con 0x7f61c4101a40 2024-04-24T06:37:25.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:25.744+0000 7f61bb7fe700 1 -- 172.21.15.137:0/1215540037 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f61b4024070 con 0x7f61c4101a40 2024-04-24T06:37:25.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:25.744+0000 7f61bb7fe700 1 --2- 172.21.15.137:0/1215540037 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f61b00a3c40 0x7f61b00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:25.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:25.745+0000 7f61bb7fe700 1 -- 172.21.15.137:0/1215540037 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7f61b400f020 con 0x7f61c4101a40 2024-04-24T06:37:25.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:25.745+0000 7f61c1d9b700 1 --2- 172.21.15.137:0/1215540037 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f61b00a3c40 0x7f61b00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:25.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:25.746+0000 7f61c1d9b700 1 --2- 172.21.15.137:0/1215540037 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f61b00a3c40 0x7f61b00a6100 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f61ac0099d0 tx=0x7f61ac008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:25.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:25.748+0000 7f61bb7fe700 1 -- 172.21.15.137:0/1215540037 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f61b4009cb0 con 0x7f61c4101a40 2024-04-24T06:37:25.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:25.955+0000 7f61c88e4700 1 -- 172.21.15.137:0/1215540037 --> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f61c4062f30 con 0x7f61b00a3c40 2024-04-24T06:37:26.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:26 smithi005 ceph-mon[27798]: Detected new or changed devices on smithi037 2024-04-24T06:37:26.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:26 smithi005 ceph-mon[27798]: Adjusting osd_memory_target on smithi037 to 8546M 2024-04-24T06:37:26.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:26 smithi005 ceph-mon[27798]: osdmap e34: 5 total, 5 up, 5 in 2024-04-24T06:37:26.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:26 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T06:37:26.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:26 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T06:37:26.359 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:26 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:37:26.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:26 smithi137 ceph-mon[33017]: Detected new or changed devices on smithi037 2024-04-24T06:37:26.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:26 smithi137 ceph-mon[33017]: Adjusting osd_memory_target on smithi037 to 8546M 2024-04-24T06:37:26.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:26 smithi137 ceph-mon[33017]: osdmap e34: 5 total, 5 up, 5 in 2024-04-24T06:37:26.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:26 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T06:37:26.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:26 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T06:37:26.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:26 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:37:26.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:26 smithi037 ceph-mon[32864]: Detected new or changed devices on smithi037 2024-04-24T06:37:26.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:26 smithi037 ceph-mon[32864]: Adjusting osd_memory_target on smithi037 to 8546M 2024-04-24T06:37:26.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:26 smithi037 ceph-mon[32864]: osdmap e34: 5 total, 5 up, 5 in 2024-04-24T06:37:26.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:26 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T06:37:26.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:26 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T06:37:26.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:26 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:37:27.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:27 smithi137 ceph-mon[33017]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T06:37:27.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:27 smithi137 ceph-mon[33017]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:37:27.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:27 smithi005 ceph-mon[27798]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T06:37:27.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:27 smithi005 ceph-mon[27798]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:37:27.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:27 smithi037 ceph-mon[32864]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T06:37:27.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:27 smithi037 ceph-mon[32864]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:37:29.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:29 smithi137 ceph-mon[33017]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T06:37:29.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:29 smithi137 ceph-mon[33017]: from='client.? 172.21.15.137:0/603853158' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97a0500c-cbb8-4ff5-b476-8634b26b3821"}]: dispatch 2024-04-24T06:37:29.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:29 smithi137 ceph-mon[33017]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97a0500c-cbb8-4ff5-b476-8634b26b3821"}]: dispatch 2024-04-24T06:37:29.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:29 smithi137 ceph-mon[33017]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "97a0500c-cbb8-4ff5-b476-8634b26b3821"}]': finished 2024-04-24T06:37:29.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:29 smithi137 ceph-mon[33017]: osdmap e35: 6 total, 5 up, 6 in 2024-04-24T06:37:29.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:29 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T06:37:29.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:29 smithi005 ceph-mon[27798]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T06:37:29.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:29 smithi005 ceph-mon[27798]: from='client.? 172.21.15.137:0/603853158' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97a0500c-cbb8-4ff5-b476-8634b26b3821"}]: dispatch 2024-04-24T06:37:29.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:29 smithi005 ceph-mon[27798]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97a0500c-cbb8-4ff5-b476-8634b26b3821"}]: dispatch 2024-04-24T06:37:29.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:29 smithi005 ceph-mon[27798]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "97a0500c-cbb8-4ff5-b476-8634b26b3821"}]': finished 2024-04-24T06:37:29.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:29 smithi005 ceph-mon[27798]: osdmap e35: 6 total, 5 up, 6 in 2024-04-24T06:37:29.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:29 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T06:37:29.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:29 smithi037 ceph-mon[32864]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T06:37:29.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:29 smithi037 ceph-mon[32864]: from='client.? 172.21.15.137:0/603853158' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97a0500c-cbb8-4ff5-b476-8634b26b3821"}]: dispatch 2024-04-24T06:37:29.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:29 smithi037 ceph-mon[32864]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97a0500c-cbb8-4ff5-b476-8634b26b3821"}]: dispatch 2024-04-24T06:37:29.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:29 smithi037 ceph-mon[32864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "97a0500c-cbb8-4ff5-b476-8634b26b3821"}]': finished 2024-04-24T06:37:29.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:29 smithi037 ceph-mon[32864]: osdmap e35: 6 total, 5 up, 6 in 2024-04-24T06:37:29.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:29 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T06:37:30.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:30 smithi137 ceph-mon[33017]: from='client.? 172.21.15.137:0/450475152' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T06:37:30.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:30 smithi005 ceph-mon[27798]: from='client.? 172.21.15.137:0/450475152' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T06:37:30.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:30 smithi037 ceph-mon[32864]: from='client.? 172.21.15.137:0/450475152' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T06:37:31.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:31 smithi137 ceph-mon[33017]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T06:37:31.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:31 smithi005 ceph-mon[27798]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T06:37:31.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:31 smithi037 ceph-mon[32864]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T06:37:33.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:33 smithi137 ceph-mon[33017]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T06:37:33.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:33 smithi005 ceph-mon[27798]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T06:37:33.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:33 smithi037 ceph-mon[32864]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T06:37:35.238 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:35 smithi137 ceph-mon[33017]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T06:37:35.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:35 smithi005 ceph-mon[27798]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T06:37:35.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:35 smithi037 ceph-mon[32864]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T06:37:36.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T06:37:36.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:37:36.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:36 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T06:37:36.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:36 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:37:36.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:36 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T06:37:36.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:36 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:37:37.441 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:37 smithi137 ceph-mon[33017]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T06:37:37.441 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:37 smithi137 ceph-mon[33017]: Deploying daemon osd.5 on smithi137 2024-04-24T06:37:37.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:37 smithi005 ceph-mon[27798]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T06:37:37.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:37 smithi005 ceph-mon[27798]: Deploying daemon osd.5 on smithi137 2024-04-24T06:37:37.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:37 smithi037 ceph-mon[32864]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T06:37:37.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:37 smithi037 ceph-mon[32864]: Deploying daemon osd.5 on smithi137 2024-04-24T06:37:38.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:38 smithi137 ceph-mon[33017]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T06:37:38.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:38 smithi005 ceph-mon[27798]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T06:37:38.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:38 smithi037 ceph-mon[32864]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T06:37:40.445 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:40 smithi137 ceph-mon[33017]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T06:37:40.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:40 smithi005 ceph-mon[27798]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T06:37:40.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:40 smithi037 ceph-mon[32864]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T06:37:41.689 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:41 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:37:41.689 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:41 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:41.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:41 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:37:41.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:41 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:41.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:41 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:37:41.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:41 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:42.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:42 smithi137 ceph-mon[33017]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T06:37:42.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:42 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:42.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:42 smithi005 ceph-mon[27798]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T06:37:42.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:42 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:42.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:42 smithi037 ceph-mon[32864]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T06:37:42.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:42 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:44.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:44 smithi037 ceph-mon[32864]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T06:37:44.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:44 smithi005 ceph-mon[27798]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T06:37:44.573 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:44 smithi137 ceph-mon[33017]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T06:37:44.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:44.640+0000 7f61bb7fe700 1 -- 172.21.15.137:0/1215540037 <== mgr.14152 v2:172.21.15.5:6800/4042936965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f61c4062f30 con 0x7f61b00a3c40 2024-04-24T06:37:44.642 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 5 on host 'smithi137' 2024-04-24T06:37:44.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:44.642+0000 7f61c88e4700 1 -- 172.21.15.137:0/1215540037 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f61b00a3c40 msgr2=0x7f61b00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:44.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:44.642+0000 7f61c88e4700 1 --2- 172.21.15.137:0/1215540037 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f61b00a3c40 0x7f61b00a6100 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f61ac0099d0 tx=0x7f61ac008040 comp rx=0 tx=0).stop 2024-04-24T06:37:44.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:44.642+0000 7f61c88e4700 1 -- 172.21.15.137:0/1215540037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61c4101a40 msgr2=0x7f61c4198ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:44.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:44.642+0000 7f61c88e4700 1 --2- 172.21.15.137:0/1215540037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61c4101a40 0x7f61c4198ed0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f61b4006070 tx=0x7f61b4007820 comp rx=0 tx=0).stop 2024-04-24T06:37:44.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:44.643+0000 7f61c88e4700 1 -- 172.21.15.137:0/1215540037 shutdown_connections 2024-04-24T06:37:44.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:44.643+0000 7f61c88e4700 1 --2- 172.21.15.137:0/1215540037 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f61b00a3c40 0x7f61b00a6100 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:44.643+0000 7f61c88e4700 1 --2- 172.21.15.137:0/1215540037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61c41035e0 0x7f61c41a0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:44.643+0000 7f61c88e4700 1 --2- 172.21.15.137:0/1215540037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61c4101a40 0x7f61c4198ed0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:44.643+0000 7f61c88e4700 1 --2- 172.21.15.137:0/1215540037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61c4102c40 0x7f61c4199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:44.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:44.643+0000 7f61c88e4700 1 -- 172.21.15.137:0/1215540037 >> 172.21.15.137:0/1215540037 conn(0x7f61c40fcfe0 msgr2=0x7f61c4106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:44.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:44.643+0000 7f61c88e4700 1 -- 172.21.15.137:0/1215540037 shutdown_connections 2024-04-24T06:37:44.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:37:44.643+0000 7f61c88e4700 1 -- 172.21.15.137:0/1215540037 wait complete. 2024-04-24T06:37:45.266 DEBUG:teuthology.orchestra.run.smithi137:osd.5> sudo journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@osd.5.service 2024-04-24T06:37:45.269 INFO:tasks.cephadm:Deploying osd.6 on smithi005 with /dev/vg_nvme/lv_2... 2024-04-24T06:37:45.269 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-24T06:37:45.697 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:37:45.891 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:45 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:45.891 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:45 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:45.891 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:45 smithi005 ceph-mon[27798]: from='osd.5 [v2:172.21.15.137:6808/3771921560,v1:172.21.15.137:6809/3771921560]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T06:37:45.891 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:45 smithi005 ceph-mon[27798]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T06:37:45.891 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:45 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:45.891 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:45 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:45.937 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:45 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:45.937 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:45 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:45.937 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:45 smithi137 ceph-mon[33017]: from='osd.5 [v2:172.21.15.137:6808/3771921560,v1:172.21.15.137:6809/3771921560]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T06:37:45.937 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:45 smithi137 ceph-mon[33017]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T06:37:45.937 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:45 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:45.938 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:45 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:46.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:46.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:46.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:45 smithi037 ceph-mon[32864]: from='osd.5 [v2:172.21.15.137:6808/3771921560,v1:172.21.15.137:6809/3771921560]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T06:37:46.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:45 smithi037 ceph-mon[32864]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T06:37:46.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:46.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:46.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:46 smithi005 ceph-mon[27798]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T06:37:46.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:46 smithi005 ceph-mon[27798]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T06:37:46.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:46 smithi005 ceph-mon[27798]: osdmap e36: 6 total, 5 up, 6 in 2024-04-24T06:37:46.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:46 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T06:37:46.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:46 smithi005 ceph-mon[27798]: from='osd.5 [v2:172.21.15.137:6808/3771921560,v1:172.21.15.137:6809/3771921560]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-24T06:37:46.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:46 smithi005 ceph-mon[27798]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-24T06:37:46.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:46 smithi137 ceph-mon[33017]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T06:37:46.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:46 smithi137 ceph-mon[33017]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T06:37:46.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:46 smithi137 ceph-mon[33017]: osdmap e36: 6 total, 5 up, 6 in 2024-04-24T06:37:46.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:46 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T06:37:46.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:46 smithi137 ceph-mon[33017]: from='osd.5 [v2:172.21.15.137:6808/3771921560,v1:172.21.15.137:6809/3771921560]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-24T06:37:46.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:46 smithi137 ceph-mon[33017]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-24T06:37:47.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:46 smithi037 ceph-mon[32864]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T06:37:47.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:46 smithi037 ceph-mon[32864]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T06:37:47.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:46 smithi037 ceph-mon[32864]: osdmap e36: 6 total, 5 up, 6 in 2024-04-24T06:37:47.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:46 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T06:37:47.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:46 smithi037 ceph-mon[32864]: from='osd.5 [v2:172.21.15.137:6808/3771921560,v1:172.21.15.137:6809/3771921560]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-24T06:37:47.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:46 smithi037 ceph-mon[32864]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-24T06:37:47.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:47 smithi005 ceph-mon[27798]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-04-24T06:37:47.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:47 smithi005 ceph-mon[27798]: osdmap e37: 6 total, 5 up, 6 in 2024-04-24T06:37:47.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:47 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T06:37:47.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:47 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T06:37:47.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:47 smithi005 ceph-mon[27798]: from='osd.5 ' entity='osd.5' 2024-04-24T06:37:47.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:47 smithi005 ceph-mon[27798]: osdmap e38: 6 total, 5 up, 6 in 2024-04-24T06:37:47.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:47 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T06:37:47.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:47 smithi137 ceph-mon[33017]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-04-24T06:37:47.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:47 smithi137 ceph-mon[33017]: osdmap e37: 6 total, 5 up, 6 in 2024-04-24T06:37:47.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:47 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T06:37:47.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:47 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T06:37:47.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:47 smithi137 ceph-mon[33017]: from='osd.5 ' entity='osd.5' 2024-04-24T06:37:47.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:47 smithi137 ceph-mon[33017]: osdmap e38: 6 total, 5 up, 6 in 2024-04-24T06:37:47.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:47 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T06:37:47.947 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:37:47 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5[45127]: 2024-04-24T06:37:47.546+0000 7f0a380f3700 -1 osd.5 0 waiting for initial osdmap 2024-04-24T06:37:47.947 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:37:47 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5[45127]: 2024-04-24T06:37:47.706+0000 7f0a2fede700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T06:37:48.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:47 smithi037 ceph-mon[32864]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-04-24T06:37:48.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:47 smithi037 ceph-mon[32864]: osdmap e37: 6 total, 5 up, 6 in 2024-04-24T06:37:48.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:47 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T06:37:48.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:47 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T06:37:48.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:47 smithi037 ceph-mon[32864]: from='osd.5 ' entity='osd.5' 2024-04-24T06:37:48.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:47 smithi037 ceph-mon[32864]: osdmap e38: 6 total, 5 up, 6 in 2024-04-24T06:37:48.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:47 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T06:37:48.705 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-24T06:37:48.705 INFO:teuthology.orchestra.run.smithi005.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-24T06:37:48.705 INFO:teuthology.orchestra.run.smithi005.stderr: stderr: 10+0 records in 2024-04-24T06:37:48.705 INFO:teuthology.orchestra.run.smithi005.stderr:10+0 records out 2024-04-24T06:37:48.705 INFO:teuthology.orchestra.run.smithi005.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.03526 s, 297 MB/s 2024-04-24T06:37:48.705 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping successful for: 2024-04-24T06:37:48.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:48 smithi137 ceph-mon[33017]: purged_snaps scrub starts 2024-04-24T06:37:48.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:48 smithi137 ceph-mon[33017]: purged_snaps scrub ok 2024-04-24T06:37:48.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:48 smithi137 ceph-mon[33017]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T06:37:48.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:48 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T06:37:48.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:48 smithi137 ceph-mon[33017]: osd.5 [v2:172.21.15.137:6808/3771921560,v1:172.21.15.137:6809/3771921560] boot 2024-04-24T06:37:48.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:48 smithi137 ceph-mon[33017]: osdmap e39: 6 total, 6 up, 6 in 2024-04-24T06:37:48.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:48 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T06:37:49.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:48 smithi005 ceph-mon[27798]: purged_snaps scrub starts 2024-04-24T06:37:49.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:48 smithi005 ceph-mon[27798]: purged_snaps scrub ok 2024-04-24T06:37:49.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:48 smithi005 ceph-mon[27798]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T06:37:49.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:48 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T06:37:49.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:48 smithi005 ceph-mon[27798]: osd.5 [v2:172.21.15.137:6808/3771921560,v1:172.21.15.137:6809/3771921560] boot 2024-04-24T06:37:49.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:48 smithi005 ceph-mon[27798]: osdmap e39: 6 total, 6 up, 6 in 2024-04-24T06:37:49.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:48 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T06:37:49.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:48 smithi037 ceph-mon[32864]: purged_snaps scrub starts 2024-04-24T06:37:49.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:48 smithi037 ceph-mon[32864]: purged_snaps scrub ok 2024-04-24T06:37:49.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:48 smithi037 ceph-mon[32864]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T06:37:49.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:48 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T06:37:49.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:48 smithi037 ceph-mon[32864]: osd.5 [v2:172.21.15.137:6808/3771921560,v1:172.21.15.137:6809/3771921560] boot 2024-04-24T06:37:49.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:48 smithi037 ceph-mon[32864]: osdmap e39: 6 total, 6 up, 6 in 2024-04-24T06:37:49.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:48 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T06:37:49.531 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi005:vg_nvme/lv_2 2024-04-24T06:37:49.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:49 smithi005 ceph-mon[27798]: Detected new or changed devices on smithi137 2024-04-24T06:37:49.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:49 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:49.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:49 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:49.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:49 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:37:49.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:49 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:37:49.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:49 smithi005 ceph-mon[27798]: Adjusting osd_memory_target on smithi137 to 8544M 2024-04-24T06:37:49.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:49 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:49.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:49 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:37:49.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:49 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:37:49.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:49 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:49.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:49 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:37:49.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:49 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:37:49.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:49 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:37:49.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:49 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:49.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:49 smithi005 ceph-mon[27798]: osdmap e40: 6 total, 6 up, 6 in 2024-04-24T06:37:49.997 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:37:50.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:49 smithi037 ceph-mon[32864]: Detected new or changed devices on smithi137 2024-04-24T06:37:50.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:49 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:50.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:49 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:50.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:49 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:37:50.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:49 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:37:50.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:49 smithi037 ceph-mon[32864]: Adjusting osd_memory_target on smithi137 to 8544M 2024-04-24T06:37:50.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:49 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:50.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:49 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:37:50.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:49 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:37:50.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:49 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:50.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:49 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:37:50.020 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:49 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:37:50.020 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:49 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:37:50.020 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:49 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:50.020 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:49 smithi037 ceph-mon[32864]: osdmap e40: 6 total, 6 up, 6 in 2024-04-24T06:37:50.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:49 smithi137 ceph-mon[33017]: Detected new or changed devices on smithi137 2024-04-24T06:37:50.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:49 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:50.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:49 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:50.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:49 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:37:50.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:49 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:37:50.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:49 smithi137 ceph-mon[33017]: Adjusting osd_memory_target on smithi137 to 8544M 2024-04-24T06:37:50.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:49 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:50.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:49 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:37:50.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:49 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:37:50.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:49 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:50.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:49 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:37:50.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:49 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:37:50.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:49 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:37:50.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:49 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:37:50.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:49 smithi137 ceph-mon[33017]: osdmap e40: 6 total, 6 up, 6 in 2024-04-24T06:37:51.176 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:51 smithi005 ceph-mon[27798]: pgmap v153: 1 pgs: 1 remapped+peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-24T06:37:51.176 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:51 smithi005 ceph-mon[27798]: osdmap e41: 6 total, 6 up, 6 in 2024-04-24T06:37:51.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:51 smithi037 ceph-mon[32864]: pgmap v153: 1 pgs: 1 remapped+peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-24T06:37:51.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:51 smithi037 ceph-mon[32864]: osdmap e41: 6 total, 6 up, 6 in 2024-04-24T06:37:51.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:51 smithi137 ceph-mon[33017]: pgmap v153: 1 pgs: 1 remapped+peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-24T06:37:51.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:51 smithi137 ceph-mon[33017]: osdmap e41: 6 total, 6 up, 6 in 2024-04-24T06:37:52.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:37:52.019+0000 7ff674400700 1 -- 172.21.15.5:0/1308398951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff66c106460 msgr2=0x7ff66c106880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:52.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:37:52.019+0000 7ff674400700 1 --2- 172.21.15.5:0/1308398951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff66c106460 0x7ff66c106880 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff6600099c0 tx=0x7ff660009cd0 comp rx=0 tx=0).stop 2024-04-24T06:37:52.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:37:52.020+0000 7ff674400700 1 -- 172.21.15.5:0/1308398951 shutdown_connections 2024-04-24T06:37:52.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:37:52.020+0000 7ff674400700 1 --2- 172.21.15.5:0/1308398951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff66c105340 0x7ff66c10d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:52.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:37:52.020+0000 7ff674400700 1 --2- 172.21.15.5:0/1308398951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff66c106460 0x7ff66c106880 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:52.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:37:52.020+0000 7ff674400700 1 --2- 172.21.15.5:0/1308398951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff66c1049a0 0x7ff66c104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:52.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:37:52.020+0000 7ff674400700 1 -- 172.21.15.5:0/1308398951 >> 172.21.15.5:0/1308398951 conn(0x7ff66c100340 msgr2=0x7ff66c102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:37:52.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:37:52.020+0000 7ff674400700 1 -- 172.21.15.5:0/1308398951 shutdown_connections 2024-04-24T06:37:52.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:37:52.020+0000 7ff674400700 1 -- 172.21.15.5:0/1308398951 wait complete. 2024-04-24T06:37:52.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:37:52.021+0000 7ff674400700 1 Processor -- start 2024-04-24T06:37:52.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:37:52.022+0000 7ff674400700 1 -- start start 2024-04-24T06:37:52.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:37:52.022+0000 7ff674400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff66c1049a0 0x7ff66c19b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:52.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:37:52.022+0000 7ff674400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff66c105340 0x7ff66c19b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:52.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:37:52.022+0000 7ff674400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff66c106460 0x7ff66c1a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:52.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:37:52.022+0000 7ff674400700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c110a90 con 0x7ff66c1049a0 2024-04-24T06:37:52.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:37:52.022+0000 7ff674400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c110910 con 0x7ff66c105340 2024-04-24T06:37:52.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:37:52.022+0000 7ff674400700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c110c10 con 0x7ff66c106460 2024-04-24T06:37:52.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:37:52.022+0000 7ff671a00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff66c1049a0 0x7ff66c19b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:52.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:37:52.022+0000 7ff671000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff66c105340 0x7ff66c19b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:52.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:37:52.022+0000 7ff671a00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff66c1049a0 0x7ff66c19b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49862/0 (socket says 172.21.15.5:49862) 2024-04-24T06:37:52.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:37:52.022+0000 7ff671a00700 1 -- 172.21.15.5:0/374506486 learned_addr learned my addr 172.21.15.5:0/374506486 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:37:52.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:37:52.023+0000 7ff672400700 1 --2- 172.21.15.5:0/374506486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff66c106460 0x7ff66c1a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:52.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:37:52.023+0000 7ff671000700 1 -- 172.21.15.5:0/374506486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff66c106460 msgr2=0x7ff66c1a2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:52.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:37:52.023+0000 7ff671000700 1 --2- 172.21.15.5:0/374506486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff66c106460 0x7ff66c1a2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:52.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:37:52.023+0000 7ff671000700 1 -- 172.21.15.5:0/374506486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff66c1049a0 msgr2=0x7ff66c19b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:37:52.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:37:52.023+0000 7ff671000700 1 --2- 172.21.15.5:0/374506486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff66c1049a0 0x7ff66c19b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:37:52.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:37:52.023+0000 7ff671000700 1 -- 172.21.15.5:0/374506486 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff66c1a2cb0 con 0x7ff66c105340 2024-04-24T06:37:52.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:37:52.023+0000 7ff671000700 1 --2- 172.21.15.5:0/374506486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff66c105340 0x7ff66c19b590 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff65800e9f0 tx=0x7ff65800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:52.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:37:52.024+0000 7ff66a000700 1 -- 172.21.15.5:0/374506486 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65800c9c0 con 0x7ff66c105340 2024-04-24T06:37:52.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:37:52.024+0000 7ff66a000700 1 -- 172.21.15.5:0/374506486 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff65800cb60 con 0x7ff66c105340 2024-04-24T06:37:52.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:37:52.024+0000 7ff66a000700 1 -- 172.21.15.5:0/374506486 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff658018680 con 0x7ff66c105340 2024-04-24T06:37:52.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:37:52.024+0000 7ff674400700 1 -- 172.21.15.5:0/374506486 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff66c1a2fa0 con 0x7ff66c105340 2024-04-24T06:37:52.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:37:52.024+0000 7ff674400700 1 -- 172.21.15.5:0/374506486 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff66c1a33a0 con 0x7ff66c105340 2024-04-24T06:37:52.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:37:52.025+0000 7ff674400700 1 -- 172.21.15.5:0/374506486 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff66c0697e0 con 0x7ff66c105340 2024-04-24T06:37:52.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:37:52.027+0000 7ff66a000700 1 -- 172.21.15.5:0/374506486 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7ff658010430 con 0x7ff66c105340 2024-04-24T06:37:52.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:37:52.029+0000 7ff66a000700 1 --2- 172.21.15.5:0/374506486 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ff6540a3c40 0x7ff6540a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:37:52.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:37:52.029+0000 7ff66a000700 1 -- 172.21.15.5:0/374506486 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4533+0+0 (secure 0 0 0) 0x7ff6580cf600 con 0x7ff66c105340 2024-04-24T06:37:52.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:37:52.029+0000 7ff671a00700 1 --2- 172.21.15.5:0/374506486 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ff6540a3c40 0x7ff6540a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:37:52.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:37:52.030+0000 7ff671a00700 1 --2- 172.21.15.5:0/374506486 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ff6540a3c40 0x7ff6540a6100 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff6600038c0 tx=0x7ff660005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:37:52.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:37:52.033+0000 7ff66a000700 1 -- 172.21.15.5:0/374506486 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff65809ce00 con 0x7ff66c105340 2024-04-24T06:37:52.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:37:52.282+0000 7ff674400700 1 -- 172.21.15.5:0/374506486 --> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7ff66c062f30 con 0x7ff6540a3c40 2024-04-24T06:37:53.279 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:53 smithi005 ceph-mon[27798]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-24T06:37:53.280 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:53 smithi005 ceph-mon[27798]: osdmap e42: 6 total, 6 up, 6 in 2024-04-24T06:37:53.280 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:53 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T06:37:53.280 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:53 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T06:37:53.280 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:53 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:37:53.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:53 smithi137 ceph-mon[33017]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-24T06:37:53.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:53 smithi137 ceph-mon[33017]: osdmap e42: 6 total, 6 up, 6 in 2024-04-24T06:37:53.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:53 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T06:37:53.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:53 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T06:37:53.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:53 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:37:53.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:53 smithi037 ceph-mon[32864]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-24T06:37:53.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:53 smithi037 ceph-mon[32864]: osdmap e42: 6 total, 6 up, 6 in 2024-04-24T06:37:53.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:53 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T06:37:53.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:53 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T06:37:53.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:53 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:37:54.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:54 smithi137 ceph-mon[33017]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:37:54.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:54 smithi137 ceph-mon[33017]: osdmap e43: 6 total, 6 up, 6 in 2024-04-24T06:37:54.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:54 smithi005 ceph-mon[27798]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:37:54.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:54 smithi005 ceph-mon[27798]: osdmap e43: 6 total, 6 up, 6 in 2024-04-24T06:37:54.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:54 smithi037 ceph-mon[32864]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:37:54.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:54 smithi037 ceph-mon[32864]: osdmap e43: 6 total, 6 up, 6 in 2024-04-24T06:37:55.103 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:55 smithi005 ceph-mon[27798]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-24T06:37:55.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:55 smithi137 ceph-mon[33017]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-24T06:37:55.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:55 smithi037 ceph-mon[32864]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-24T06:37:57.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:57 smithi137 ceph-mon[33017]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-04-24T06:37:57.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:57 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/2281900165' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33a33d62-bbb3-4ed9-834e-19ae93a97670"}]: dispatch 2024-04-24T06:37:57.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:57 smithi137 ceph-mon[33017]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33a33d62-bbb3-4ed9-834e-19ae93a97670"}]: dispatch 2024-04-24T06:37:57.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:57 smithi137 ceph-mon[33017]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33a33d62-bbb3-4ed9-834e-19ae93a97670"}]': finished 2024-04-24T06:37:57.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:57 smithi137 ceph-mon[33017]: osdmap e44: 7 total, 6 up, 7 in 2024-04-24T06:37:57.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:57 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T06:37:57.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:57 smithi005 ceph-mon[27798]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-04-24T06:37:57.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:57 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/2281900165' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33a33d62-bbb3-4ed9-834e-19ae93a97670"}]: dispatch 2024-04-24T06:37:57.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:57 smithi005 ceph-mon[27798]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33a33d62-bbb3-4ed9-834e-19ae93a97670"}]: dispatch 2024-04-24T06:37:57.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:57 smithi005 ceph-mon[27798]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33a33d62-bbb3-4ed9-834e-19ae93a97670"}]': finished 2024-04-24T06:37:57.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:57 smithi005 ceph-mon[27798]: osdmap e44: 7 total, 6 up, 7 in 2024-04-24T06:37:57.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:57 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T06:37:57.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:57 smithi037 ceph-mon[32864]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-04-24T06:37:57.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:57 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/2281900165' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33a33d62-bbb3-4ed9-834e-19ae93a97670"}]: dispatch 2024-04-24T06:37:57.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:57 smithi037 ceph-mon[32864]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33a33d62-bbb3-4ed9-834e-19ae93a97670"}]: dispatch 2024-04-24T06:37:57.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:57 smithi037 ceph-mon[32864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33a33d62-bbb3-4ed9-834e-19ae93a97670"}]': finished 2024-04-24T06:37:57.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:57 smithi037 ceph-mon[32864]: osdmap e44: 7 total, 6 up, 7 in 2024-04-24T06:37:57.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:57 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T06:37:58.442 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:58 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/1796152032' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T06:37:58.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:58 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1796152032' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T06:37:58.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:58 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/1796152032' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T06:37:59.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:37:59 smithi137 ceph-mon[33017]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T06:37:59.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:37:59 smithi005 ceph-mon[27798]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T06:37:59.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:37:59 smithi037 ceph-mon[32864]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T06:38:01.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:01 smithi137 ceph-mon[33017]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-04-24T06:38:01.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:01 smithi005 ceph-mon[27798]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-04-24T06:38:01.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:01 smithi037 ceph-mon[32864]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-04-24T06:38:03.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:03 smithi005 ceph-mon[27798]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-24T06:38:03.321 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:03 smithi037 ceph-mon[32864]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-24T06:38:03.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:03 smithi137 ceph-mon[33017]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-24T06:38:05.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:05 smithi005 ceph-mon[27798]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T06:38:05.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:05 smithi137 ceph-mon[33017]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T06:38:05.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:05 smithi037 ceph-mon[32864]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T06:38:06.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:06 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-24T06:38:06.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:06 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:38:06.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:06 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-24T06:38:06.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:06 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:38:06.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:06 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-24T06:38:06.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:06 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:38:07.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:07 smithi137 ceph-mon[33017]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T06:38:07.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:07 smithi137 ceph-mon[33017]: Deploying daemon osd.6 on smithi005 2024-04-24T06:38:07.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:07 smithi005 ceph-mon[27798]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T06:38:07.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:07 smithi005 ceph-mon[27798]: Deploying daemon osd.6 on smithi005 2024-04-24T06:38:07.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:07 smithi037 ceph-mon[32864]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T06:38:07.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:07 smithi037 ceph-mon[32864]: Deploying daemon osd.6 on smithi005 2024-04-24T06:38:09.403 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:09 smithi005 ceph-mon[27798]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T06:38:09.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:09 smithi137 ceph-mon[33017]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T06:38:09.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:09 smithi037 ceph-mon[32864]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T06:38:11.237 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:11 smithi005 ceph-mon[27798]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T06:38:11.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:11 smithi137 ceph-mon[33017]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T06:38:11.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:11 smithi037 ceph-mon[32864]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T06:38:13.439 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:13 smithi005 ceph-mon[27798]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T06:38:13.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:13 smithi137 ceph-mon[33017]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T06:38:13.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:13 smithi037 ceph-mon[32864]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T06:38:15.432 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:15 smithi005 ceph-mon[27798]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T06:38:15.432 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:15 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:15.432 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:15 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:38:15.432 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:15 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:15.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:15 smithi137 ceph-mon[33017]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T06:38:15.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:15 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:15.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:15 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:38:15.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:15 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:15.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:15 smithi037 ceph-mon[32864]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T06:38:15.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:15 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:15.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:15 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:38:15.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:15 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:17.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:17 smithi005 ceph-mon[27798]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T06:38:17.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:17 smithi137 ceph-mon[33017]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T06:38:17.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:17 smithi037 ceph-mon[32864]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T06:38:19.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:19 smithi005 ceph-mon[27798]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T06:38:19.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:19 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:19.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:19 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:19.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:19 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:38:19.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:19 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:38:19.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:19 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:19.157 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:19 smithi005 ceph-mon[27798]: from='osd.6 [v2:172.21.15.5:6818/2064636084,v1:172.21.15.5:6819/2064636084]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-24T06:38:19.282 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 6 on host 'smithi005' 2024-04-24T06:38:19.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:38:19.280+0000 7ff66a000700 1 -- 172.21.15.5:0/374506486 <== mgr.14152 v2:172.21.15.5:6800/4042936965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff66c062f30 con 0x7ff6540a3c40 2024-04-24T06:38:19.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:38:19.284+0000 7ff674400700 1 -- 172.21.15.5:0/374506486 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ff6540a3c40 msgr2=0x7ff6540a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:19.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:38:19.284+0000 7ff674400700 1 --2- 172.21.15.5:0/374506486 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ff6540a3c40 0x7ff6540a6100 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff6600038c0 tx=0x7ff660005fb0 comp rx=0 tx=0).stop 2024-04-24T06:38:19.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:38:19.284+0000 7ff674400700 1 -- 172.21.15.5:0/374506486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff66c105340 msgr2=0x7ff66c19b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:19.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:38:19.284+0000 7ff674400700 1 --2- 172.21.15.5:0/374506486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff66c105340 0x7ff66c19b590 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff65800e9f0 tx=0x7ff65800ed00 comp rx=0 tx=0).stop 2024-04-24T06:38:19.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:38:19.284+0000 7ff674400700 1 -- 172.21.15.5:0/374506486 shutdown_connections 2024-04-24T06:38:19.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:38:19.284+0000 7ff674400700 1 --2- 172.21.15.5:0/374506486 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ff6540a3c40 0x7ff6540a6100 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:19.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:38:19.284+0000 7ff674400700 1 --2- 172.21.15.5:0/374506486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff66c106460 0x7ff66c1a2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:19.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:38:19.284+0000 7ff674400700 1 --2- 172.21.15.5:0/374506486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff66c1049a0 0x7ff66c19b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:19.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:38:19.284+0000 7ff674400700 1 --2- 172.21.15.5:0/374506486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff66c105340 0x7ff66c19b590 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:19.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:38:19.284+0000 7ff674400700 1 -- 172.21.15.5:0/374506486 >> 172.21.15.5:0/374506486 conn(0x7ff66c100340 msgr2=0x7ff66c109220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:19.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:38:19.284+0000 7ff674400700 1 -- 172.21.15.5:0/374506486 shutdown_connections 2024-04-24T06:38:19.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:38:19.284+0000 7ff674400700 1 -- 172.21.15.5:0/374506486 wait complete. 2024-04-24T06:38:19.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:19 smithi137 ceph-mon[33017]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T06:38:19.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:19 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:19.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:19 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:19.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:19 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:38:19.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:19 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:38:19.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:19 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:19.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:19 smithi137 ceph-mon[33017]: from='osd.6 [v2:172.21.15.5:6818/2064636084,v1:172.21.15.5:6819/2064636084]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-24T06:38:19.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:19 smithi037 ceph-mon[32864]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T06:38:19.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:19 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:19.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:19 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:19.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:19 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:38:19.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:19 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:38:19.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:19 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:19.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:19 smithi037 ceph-mon[32864]: from='osd.6 [v2:172.21.15.5:6818/2064636084,v1:172.21.15.5:6819/2064636084]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-24T06:38:20.106 DEBUG:teuthology.orchestra.run.smithi005:osd.6> sudo journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@osd.6.service 2024-04-24T06:38:20.114 INFO:tasks.cephadm:Deploying osd.7 on smithi037 with /dev/vg_nvme/lv_2... 2024-04-24T06:38:20.114 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-24T06:38:20.369 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T06:38:20.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:20 smithi137 ceph-mon[33017]: from='osd.6 [v2:172.21.15.5:6818/2064636084,v1:172.21.15.5:6819/2064636084]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-24T06:38:20.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:20 smithi137 ceph-mon[33017]: osdmap e45: 7 total, 6 up, 7 in 2024-04-24T06:38:20.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:20 smithi137 ceph-mon[33017]: from='osd.6 [v2:172.21.15.5:6818/2064636084,v1:172.21.15.5:6819/2064636084]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-04-24T06:38:20.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:20 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T06:38:20.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:20 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:20.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:20 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:20.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:20 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:38:20.491 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:20 smithi037 ceph-mon[32864]: from='osd.6 [v2:172.21.15.5:6818/2064636084,v1:172.21.15.5:6819/2064636084]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-24T06:38:20.492 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:20 smithi037 ceph-mon[32864]: osdmap e45: 7 total, 6 up, 7 in 2024-04-24T06:38:20.492 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:20 smithi037 ceph-mon[32864]: from='osd.6 [v2:172.21.15.5:6818/2064636084,v1:172.21.15.5:6819/2064636084]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-04-24T06:38:20.492 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:20 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T06:38:20.492 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:20 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:20.492 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:20 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:20.492 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:20 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:38:20.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:20 smithi005 ceph-mon[27798]: from='osd.6 [v2:172.21.15.5:6818/2064636084,v1:172.21.15.5:6819/2064636084]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-24T06:38:20.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:20 smithi005 ceph-mon[27798]: osdmap e45: 7 total, 6 up, 7 in 2024-04-24T06:38:20.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:20 smithi005 ceph-mon[27798]: from='osd.6 [v2:172.21.15.5:6818/2064636084,v1:172.21.15.5:6819/2064636084]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-04-24T06:38:20.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:20 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T06:38:20.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:20 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:20.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:20 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:20.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:20 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:38:21.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:21 smithi137 ceph-mon[33017]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T06:38:21.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:21 smithi137 ceph-mon[33017]: from='osd.6 [v2:172.21.15.5:6818/2064636084,v1:172.21.15.5:6819/2064636084]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-04-24T06:38:21.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:21 smithi137 ceph-mon[33017]: osdmap e46: 7 total, 6 up, 7 in 2024-04-24T06:38:21.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:21 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T06:38:21.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:21 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T06:38:21.449 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:21 smithi005 ceph-mon[27798]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T06:38:21.449 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:21 smithi005 ceph-mon[27798]: from='osd.6 [v2:172.21.15.5:6818/2064636084,v1:172.21.15.5:6819/2064636084]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-04-24T06:38:21.449 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:21 smithi005 ceph-mon[27798]: osdmap e46: 7 total, 6 up, 7 in 2024-04-24T06:38:21.449 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:21 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T06:38:21.449 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:21 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T06:38:21.514 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:21 smithi037 ceph-mon[32864]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T06:38:21.515 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:21 smithi037 ceph-mon[32864]: from='osd.6 [v2:172.21.15.5:6818/2064636084,v1:172.21.15.5:6819/2064636084]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-04-24T06:38:21.515 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:21 smithi037 ceph-mon[32864]: osdmap e46: 7 total, 6 up, 7 in 2024-04-24T06:38:21.515 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:21 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T06:38:21.515 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:21 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T06:38:22.204 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-24T06:38:22.205 INFO:teuthology.orchestra.run.smithi037.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-24T06:38:22.205 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10+0 records in 2024-04-24T06:38:22.205 INFO:teuthology.orchestra.run.smithi037.stderr:10+0 records out 2024-04-24T06:38:22.205 INFO:teuthology.orchestra.run.smithi037.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112338 s, 933 MB/s 2024-04-24T06:38:22.205 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping successful for: 2024-04-24T06:38:22.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:22 smithi137 ceph-mon[33017]: purged_snaps scrub starts 2024-04-24T06:38:22.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:22 smithi137 ceph-mon[33017]: purged_snaps scrub ok 2024-04-24T06:38:22.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:22 smithi137 ceph-mon[33017]: osd.6 [v2:172.21.15.5:6818/2064636084,v1:172.21.15.5:6819/2064636084] boot 2024-04-24T06:38:22.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:22 smithi137 ceph-mon[33017]: osdmap e47: 7 total, 7 up, 7 in 2024-04-24T06:38:22.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:22 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T06:38:22.462 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:22 smithi005 ceph-mon[27798]: purged_snaps scrub starts 2024-04-24T06:38:22.462 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:22 smithi005 ceph-mon[27798]: purged_snaps scrub ok 2024-04-24T06:38:22.462 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:22 smithi005 ceph-mon[27798]: osd.6 [v2:172.21.15.5:6818/2064636084,v1:172.21.15.5:6819/2064636084] boot 2024-04-24T06:38:22.462 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:22 smithi005 ceph-mon[27798]: osdmap e47: 7 total, 7 up, 7 in 2024-04-24T06:38:22.462 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:22 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T06:38:22.513 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:22 smithi037 ceph-mon[32864]: purged_snaps scrub starts 2024-04-24T06:38:22.514 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:22 smithi037 ceph-mon[32864]: purged_snaps scrub ok 2024-04-24T06:38:22.514 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:22 smithi037 ceph-mon[32864]: osd.6 [v2:172.21.15.5:6818/2064636084,v1:172.21.15.5:6819/2064636084] boot 2024-04-24T06:38:22.514 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:22 smithi037 ceph-mon[32864]: osdmap e47: 7 total, 7 up, 7 in 2024-04-24T06:38:22.514 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:22 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T06:38:22.860 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi037:vg_nvme/lv_2 2024-04-24T06:38:23.109 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T06:38:23.410 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:23 smithi037 ceph-mon[32864]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:23.410 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:23 smithi037 ceph-mon[32864]: osdmap e48: 7 total, 7 up, 7 in 2024-04-24T06:38:23.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:23 smithi137 ceph-mon[33017]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:23.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:23 smithi137 ceph-mon[33017]: osdmap e48: 7 total, 7 up, 7 in 2024-04-24T06:38:23.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:23 smithi005 ceph-mon[27798]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:23.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:23 smithi005 ceph-mon[27798]: osdmap e48: 7 total, 7 up, 7 in 2024-04-24T06:38:24.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:24 smithi037 ceph-mon[32864]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:24.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:24 smithi137 ceph-mon[33017]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:24.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:24 smithi005 ceph-mon[27798]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:24.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:24.513+0000 7fb2b0ba1700 1 -- 172.21.15.37:0/2297294651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2ac102b70 msgr2=0x7fb2ac102f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:24.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:24.513+0000 7fb2b0ba1700 1 --2- 172.21.15.37:0/2297294651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2ac102b70 0x7fb2ac102f90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb29c0099c0 tx=0x7fb29c009cd0 comp rx=0 tx=0).stop 2024-04-24T06:38:24.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:24.513+0000 7fb2b0ba1700 1 -- 172.21.15.37:0/2297294651 shutdown_connections 2024-04-24T06:38:24.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:24.513+0000 7fb2b0ba1700 1 --2- 172.21.15.37:0/2297294651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2ac104710 0x7fb2ac10afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:24.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:24.513+0000 7fb2b0ba1700 1 --2- 172.21.15.37:0/2297294651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2ac102b70 0x7fb2ac102f90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:24.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:24.513+0000 7fb2b0ba1700 1 --2- 172.21.15.37:0/2297294651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2ac103d70 0x7fb2ac1041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:24.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:24.513+0000 7fb2b0ba1700 1 -- 172.21.15.37:0/2297294651 >> 172.21.15.37:0/2297294651 conn(0x7fb2ac0fe110 msgr2=0x7fb2ac100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:24.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:24.514+0000 7fb2b0ba1700 1 -- 172.21.15.37:0/2297294651 shutdown_connections 2024-04-24T06:38:24.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:24.514+0000 7fb2b0ba1700 1 -- 172.21.15.37:0/2297294651 wait complete. 2024-04-24T06:38:24.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:24.514+0000 7fb2b0ba1700 1 Processor -- start 2024-04-24T06:38:24.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:24.514+0000 7fb2b0ba1700 1 -- start start 2024-04-24T06:38:24.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:24.514+0000 7fb2b0ba1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2ac102b70 0x7fb2ac198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:24.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:24.514+0000 7fb2b0ba1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2ac103d70 0x7fb2ac199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:24.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:24.515+0000 7fb2b0ba1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2ac104710 0x7fb2ac1a04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:24.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:24.515+0000 7fb2b0ba1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ac10e830 con 0x7fb2ac104710 2024-04-24T06:38:24.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:24.515+0000 7fb2b0ba1700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ac10e6b0 con 0x7fb2ac103d70 2024-04-24T06:38:24.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:24.515+0000 7fb2b0ba1700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ac10e9b0 con 0x7fb2ac102b70 2024-04-24T06:38:24.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:24.515+0000 7fb2a9d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2ac103d70 0x7fb2ac199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:24.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:24.515+0000 7fb2a9d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2ac103d70 0x7fb2ac199430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38334/0 (socket says 172.21.15.37:38334) 2024-04-24T06:38:24.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:24.515+0000 7fb2a9d9b700 1 -- 172.21.15.37:0/2984418575 learned_addr learned my addr 172.21.15.37:0/2984418575 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-24T06:38:24.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:24.515+0000 7fb2aa59c700 1 --2- 172.21.15.37:0/2984418575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2ac102b70 0x7fb2ac198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:24.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:24.515+0000 7fb2aad9d700 1 --2- 172.21.15.37:0/2984418575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2ac104710 0x7fb2ac1a04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:24.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:24.515+0000 7fb2a9d9b700 1 -- 172.21.15.37:0/2984418575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2ac102b70 msgr2=0x7fb2ac198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:24.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:24.515+0000 7fb2a9d9b700 1 --2- 172.21.15.37:0/2984418575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2ac102b70 0x7fb2ac198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:24.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:24.515+0000 7fb2a9d9b700 1 -- 172.21.15.37:0/2984418575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2ac104710 msgr2=0x7fb2ac1a04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:24.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:24.515+0000 7fb2a9d9b700 1 --2- 172.21.15.37:0/2984418575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2ac104710 0x7fb2ac1a04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:24.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:24.515+0000 7fb2a9d9b700 1 -- 172.21.15.37:0/2984418575 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2ac1a0b50 con 0x7fb2ac103d70 2024-04-24T06:38:24.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:24.515+0000 7fb2a9d9b700 1 --2- 172.21.15.37:0/2984418575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2ac103d70 0x7fb2ac199430 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000b530 tx=0x7fb2a000b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:24.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:24.515+0000 7fb29b7fe700 1 -- 172.21.15.37:0/2984418575 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0015020 con 0x7fb2ac103d70 2024-04-24T06:38:24.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:24.515+0000 7fb29b7fe700 1 -- 172.21.15.37:0/2984418575 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2a0011d10 con 0x7fb2ac103d70 2024-04-24T06:38:24.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:24.515+0000 7fb2b0ba1700 1 -- 172.21.15.37:0/2984418575 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2ac1a0e40 con 0x7fb2ac103d70 2024-04-24T06:38:24.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:24.515+0000 7fb29b7fe700 1 -- 172.21.15.37:0/2984418575 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a000f2e0 con 0x7fb2ac103d70 2024-04-24T06:38:24.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:24.515+0000 7fb2b0ba1700 1 -- 172.21.15.37:0/2984418575 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2ac1a1370 con 0x7fb2ac103d70 2024-04-24T06:38:24.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:24.516+0000 7fb2b0ba1700 1 -- 172.21.15.37:0/2984418575 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2ac0697e0 con 0x7fb2ac103d70 2024-04-24T06:38:24.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:24.517+0000 7fb29b7fe700 1 -- 172.21.15.37:0/2984418575 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fb2a0018020 con 0x7fb2ac103d70 2024-04-24T06:38:24.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:24.518+0000 7fb29b7fe700 1 --2- 172.21.15.37:0/2984418575 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fb2940a3c40 0x7fb2940a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:24.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:24.518+0000 7fb29b7fe700 1 -- 172.21.15.37:0/2984418575 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7fb2a00ceec0 con 0x7fb2ac103d70 2024-04-24T06:38:24.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:24.518+0000 7fb2aa59c700 1 --2- 172.21.15.37:0/2984418575 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fb2940a3c40 0x7fb2940a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:24.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:24.519+0000 7fb2aa59c700 1 --2- 172.21.15.37:0/2984418575 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fb2940a3c40 0x7fb2940a6100 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb29c003890 tx=0x7fb29c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:24.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:24.523+0000 7fb29b7fe700 1 -- 172.21.15.37:0/2984418575 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb2a001f020 con 0x7fb2ac103d70 2024-04-24T06:38:24.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:24.739+0000 7fb2b0ba1700 1 -- 172.21.15.37:0/2984418575 --> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fb2ac062f30 con 0x7fb2940a3c40 2024-04-24T06:38:25.415 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:25 smithi005 ceph-mon[27798]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:38:25.415 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:25 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T06:38:25.415 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:25 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T06:38:25.415 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:25 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:38:25.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:25 smithi137 ceph-mon[33017]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:38:25.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:25 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T06:38:25.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:25 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T06:38:25.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:25 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:38:25.502 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:25 smithi037 ceph-mon[32864]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:38:25.503 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:25 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T06:38:25.503 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:25 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T06:38:25.503 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:25 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:38:27.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:26 smithi137 ceph-mon[33017]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:27.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:26 smithi137 ceph-mon[33017]: Detected new or changed devices on smithi005 2024-04-24T06:38:27.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:26 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:27.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:26 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:27.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:26 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:38:27.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:26 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:38:27.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:26 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:38:27.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:26 smithi137 ceph-mon[33017]: Adjusting osd_memory_target on smithi005 to 5698M 2024-04-24T06:38:27.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:26 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:27.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:26 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:38:27.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:26 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:38:27.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:26 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:27.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:26 smithi005 ceph-mon[27798]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:27.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:26 smithi005 ceph-mon[27798]: Detected new or changed devices on smithi005 2024-04-24T06:38:27.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:26 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:27.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:26 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:27.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:26 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:38:27.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:26 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:38:27.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:26 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:38:27.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:26 smithi005 ceph-mon[27798]: Adjusting osd_memory_target on smithi005 to 5698M 2024-04-24T06:38:27.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:26 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:27.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:26 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:38:27.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:26 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:38:27.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:26 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:27.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:26 smithi037 ceph-mon[32864]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:27.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:26 smithi037 ceph-mon[32864]: Detected new or changed devices on smithi005 2024-04-24T06:38:27.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:26 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:27.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:26 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:27.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:26 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:38:27.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:26 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:38:27.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:26 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:38:27.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:26 smithi037 ceph-mon[32864]: Adjusting osd_memory_target on smithi005 to 5698M 2024-04-24T06:38:27.270 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:26 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:27.270 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:26 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:38:27.270 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:26 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:38:27.270 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:26 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:28.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:27 smithi137 ceph-mon[33017]: from='client.? 172.21.15.37:0/2610565759' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0f4e036-4717-4514-bb59-ab11fbaab063"}]: dispatch 2024-04-24T06:38:28.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:27 smithi137 ceph-mon[33017]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0f4e036-4717-4514-bb59-ab11fbaab063"}]: dispatch 2024-04-24T06:38:28.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:27 smithi137 ceph-mon[33017]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0f4e036-4717-4514-bb59-ab11fbaab063"}]': finished 2024-04-24T06:38:28.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:27 smithi137 ceph-mon[33017]: osdmap e49: 8 total, 7 up, 8 in 2024-04-24T06:38:28.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:27 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T06:38:28.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:27 smithi005 ceph-mon[27798]: from='client.? 172.21.15.37:0/2610565759' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0f4e036-4717-4514-bb59-ab11fbaab063"}]: dispatch 2024-04-24T06:38:28.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:27 smithi005 ceph-mon[27798]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0f4e036-4717-4514-bb59-ab11fbaab063"}]: dispatch 2024-04-24T06:38:28.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:27 smithi005 ceph-mon[27798]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0f4e036-4717-4514-bb59-ab11fbaab063"}]': finished 2024-04-24T06:38:28.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:27 smithi005 ceph-mon[27798]: osdmap e49: 8 total, 7 up, 8 in 2024-04-24T06:38:28.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:27 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T06:38:28.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:27 smithi037 ceph-mon[32864]: from='client.? 172.21.15.37:0/2610565759' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0f4e036-4717-4514-bb59-ab11fbaab063"}]: dispatch 2024-04-24T06:38:28.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:27 smithi037 ceph-mon[32864]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0f4e036-4717-4514-bb59-ab11fbaab063"}]: dispatch 2024-04-24T06:38:28.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:27 smithi037 ceph-mon[32864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0f4e036-4717-4514-bb59-ab11fbaab063"}]': finished 2024-04-24T06:38:28.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:27 smithi037 ceph-mon[32864]: osdmap e49: 8 total, 7 up, 8 in 2024-04-24T06:38:28.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:27 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T06:38:29.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:28 smithi137 ceph-mon[33017]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:29.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:28 smithi137 ceph-mon[33017]: from='client.? 172.21.15.37:0/3714822394' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T06:38:29.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:28 smithi005 ceph-mon[27798]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:29.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:28 smithi005 ceph-mon[27798]: from='client.? 172.21.15.37:0/3714822394' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T06:38:29.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:28 smithi037 ceph-mon[32864]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:29.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:28 smithi037 ceph-mon[32864]: from='client.? 172.21.15.37:0/3714822394' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T06:38:31.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:30 smithi005 ceph-mon[27798]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:31.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:30 smithi037 ceph-mon[32864]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:31.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:30 smithi137 ceph-mon[33017]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:33.175 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:32 smithi037 ceph-mon[32864]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:33.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:32 smithi005 ceph-mon[27798]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:33.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:32 smithi137 ceph-mon[33017]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:35.236 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:34 smithi037 ceph-mon[32864]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:35.258 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:34 smithi005 ceph-mon[27798]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:35.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:34 smithi137 ceph-mon[33017]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:36.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:35 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-24T06:38:36.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:35 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:38:36.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:35 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-24T06:38:36.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:35 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:38:36.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:35 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-24T06:38:36.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:35 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:38:37.193 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:36 smithi037 ceph-mon[32864]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:37.194 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:36 smithi037 ceph-mon[32864]: Deploying daemon osd.7 on smithi037 2024-04-24T06:38:37.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:36 smithi005 ceph-mon[27798]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:37.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:36 smithi005 ceph-mon[27798]: Deploying daemon osd.7 on smithi037 2024-04-24T06:38:37.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:36 smithi137 ceph-mon[33017]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:37.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:36 smithi137 ceph-mon[33017]: Deploying daemon osd.7 on smithi037 2024-04-24T06:38:39.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:38 smithi005 ceph-mon[27798]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:39.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:38 smithi037 ceph-mon[32864]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:39.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:38 smithi137 ceph-mon[33017]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:41.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:40 smithi005 ceph-mon[27798]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:41.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:40 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:41.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:40 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:41.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:40 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:38:41.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:40 smithi037 ceph-mon[32864]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:41.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:40 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:41.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:40 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:41.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:40 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:38:41.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:40 smithi137 ceph-mon[33017]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:41.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:40 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:41.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:40 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:41.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:40 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:38:43.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:42 smithi005 ceph-mon[27798]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:43.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:42 smithi037 ceph-mon[32864]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:43.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:42 smithi137 ceph-mon[33017]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:44.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:44.083+0000 7fb29b7fe700 1 -- 172.21.15.37:0/2984418575 <== mgr.14152 v2:172.21.15.5:6800/4042936965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb2ac062f30 con 0x7fb2940a3c40 2024-04-24T06:38:44.085 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 7 on host 'smithi037' 2024-04-24T06:38:44.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:44.086+0000 7fb2b0ba1700 1 -- 172.21.15.37:0/2984418575 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fb2940a3c40 msgr2=0x7fb2940a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:44.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:44.086+0000 7fb2b0ba1700 1 --2- 172.21.15.37:0/2984418575 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fb2940a3c40 0x7fb2940a6100 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb29c003890 tx=0x7fb29c005fb0 comp rx=0 tx=0).stop 2024-04-24T06:38:44.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:44.086+0000 7fb2b0ba1700 1 -- 172.21.15.37:0/2984418575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2ac103d70 msgr2=0x7fb2ac199430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:44.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:44.086+0000 7fb2b0ba1700 1 --2- 172.21.15.37:0/2984418575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2ac103d70 0x7fb2ac199430 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000b530 tx=0x7fb2a000b840 comp rx=0 tx=0).stop 2024-04-24T06:38:44.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:44.086+0000 7fb2b0ba1700 1 -- 172.21.15.37:0/2984418575 shutdown_connections 2024-04-24T06:38:44.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:44.086+0000 7fb2b0ba1700 1 --2- 172.21.15.37:0/2984418575 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fb2940a3c40 0x7fb2940a6100 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:44.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:44.086+0000 7fb2b0ba1700 1 --2- 172.21.15.37:0/2984418575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2ac104710 0x7fb2ac1a04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:44.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:44.086+0000 7fb2b0ba1700 1 --2- 172.21.15.37:0/2984418575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2ac102b70 0x7fb2ac198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:44.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:44.086+0000 7fb2b0ba1700 1 --2- 172.21.15.37:0/2984418575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2ac103d70 0x7fb2ac199430 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:44.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:44.086+0000 7fb2b0ba1700 1 -- 172.21.15.37:0/2984418575 >> 172.21.15.37:0/2984418575 conn(0x7fb2ac0fe110 msgr2=0x7fb2ac107160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:44.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:44.086+0000 7fb2b0ba1700 1 -- 172.21.15.37:0/2984418575 shutdown_connections 2024-04-24T06:38:44.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:38:44.086+0000 7fb2b0ba1700 1 -- 172.21.15.37:0/2984418575 wait complete. 2024-04-24T06:38:44.705 DEBUG:teuthology.orchestra.run.smithi037:osd.7> sudo journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@osd.7.service 2024-04-24T06:38:44.707 INFO:tasks.cephadm:Deploying osd.8 on smithi137 with /dev/vg_nvme/lv_2... 2024-04-24T06:38:44.707 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-24T06:38:44.954 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T06:38:45.271 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:45 smithi037 ceph-mon[32864]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:45.271 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:45.271 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:45.271 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:45.271 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:45.271 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:45 smithi037 ceph-mon[32864]: from='osd.7 [v2:172.21.15.37:6816/2952750659,v1:172.21.15.37:6817/2952750659]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T06:38:45.271 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:45 smithi037 ceph-mon[32864]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T06:38:45.303 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:45 smithi137 ceph-mon[33017]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:45.303 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:45 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:45.303 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:45 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:45.303 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:45 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:45.303 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:45 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:45.303 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:45 smithi137 ceph-mon[33017]: from='osd.7 [v2:172.21.15.37:6816/2952750659,v1:172.21.15.37:6817/2952750659]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T06:38:45.303 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:45 smithi137 ceph-mon[33017]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T06:38:45.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:45 smithi005 ceph-mon[27798]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:45.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:45 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:45.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:45 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:45.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:45 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:45.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:45 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:45.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:45 smithi005 ceph-mon[27798]: from='osd.7 [v2:172.21.15.37:6816/2952750659,v1:172.21.15.37:6817/2952750659]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T06:38:45.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:45 smithi005 ceph-mon[27798]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T06:38:46.384 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:46 smithi137 ceph-mon[33017]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-24T06:38:46.384 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:46 smithi137 ceph-mon[33017]: from='osd.7 [v2:172.21.15.37:6816/2952750659,v1:172.21.15.37:6817/2952750659]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-24T06:38:46.384 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:46 smithi137 ceph-mon[33017]: osdmap e50: 8 total, 7 up, 8 in 2024-04-24T06:38:46.384 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:46 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T06:38:46.384 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:46 smithi137 ceph-mon[33017]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-24T06:38:46.456 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:46 smithi005 ceph-mon[27798]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-24T06:38:46.457 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:46 smithi005 ceph-mon[27798]: from='osd.7 [v2:172.21.15.37:6816/2952750659,v1:172.21.15.37:6817/2952750659]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-24T06:38:46.457 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:46 smithi005 ceph-mon[27798]: osdmap e50: 8 total, 7 up, 8 in 2024-04-24T06:38:46.457 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:46 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T06:38:46.457 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:46 smithi005 ceph-mon[27798]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-24T06:38:46.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:46 smithi037 ceph-mon[32864]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-24T06:38:46.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:46 smithi037 ceph-mon[32864]: from='osd.7 [v2:172.21.15.37:6816/2952750659,v1:172.21.15.37:6817/2952750659]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-24T06:38:46.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:46 smithi037 ceph-mon[32864]: osdmap e50: 8 total, 7 up, 8 in 2024-04-24T06:38:46.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:46 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T06:38:46.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:46 smithi037 ceph-mon[32864]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-24T06:38:46.519 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:38:46 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:38:46.117+0000 7f993b71d700 -1 osd.7 0 waiting for initial osdmap 2024-04-24T06:38:46.520 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:38:46 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:38:46.128+0000 7f9937510700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T06:38:47.044 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-24T06:38:47.044 INFO:teuthology.orchestra.run.smithi137.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-24T06:38:47.045 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10+0 records in 2024-04-24T06:38:47.045 INFO:teuthology.orchestra.run.smithi137.stderr:10+0 records out 2024-04-24T06:38:47.045 INFO:teuthology.orchestra.run.smithi137.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112939 s, 928 MB/s 2024-04-24T06:38:47.045 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping successful for: 2024-04-24T06:38:47.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:47 smithi037 ceph-mon[32864]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:47.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:47 smithi037 ceph-mon[32864]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-24T06:38:47.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:47 smithi037 ceph-mon[32864]: osdmap e51: 8 total, 7 up, 8 in 2024-04-24T06:38:47.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:47 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T06:38:47.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:47 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T06:38:47.346 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:47 smithi137 ceph-mon[33017]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:47.346 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:47 smithi137 ceph-mon[33017]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-24T06:38:47.347 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:47 smithi137 ceph-mon[33017]: osdmap e51: 8 total, 7 up, 8 in 2024-04-24T06:38:47.347 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:47 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T06:38:47.347 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:47 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T06:38:47.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:47 smithi005 ceph-mon[27798]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:47.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:47 smithi005 ceph-mon[27798]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-24T06:38:47.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:47 smithi005 ceph-mon[27798]: osdmap e51: 8 total, 7 up, 8 in 2024-04-24T06:38:47.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:47 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T06:38:47.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:47 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T06:38:47.668 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi137:vg_nvme/lv_2 2024-04-24T06:38:47.941 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T06:38:48.315 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:48 smithi137 ceph-mon[33017]: purged_snaps scrub starts 2024-04-24T06:38:48.316 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:48 smithi137 ceph-mon[33017]: purged_snaps scrub ok 2024-04-24T06:38:48.316 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:48 smithi137 ceph-mon[33017]: osd.7 [v2:172.21.15.37:6816/2952750659,v1:172.21.15.37:6817/2952750659] boot 2024-04-24T06:38:48.316 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:48 smithi137 ceph-mon[33017]: osdmap e52: 8 total, 8 up, 8 in 2024-04-24T06:38:48.316 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:48 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T06:38:48.316 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:48 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:48.316 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:48 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:48.316 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:48 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:38:48.316 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:48 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:38:48.316 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:48 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:38:48.316 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:48 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:48.316 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:48 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:38:48.316 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:48 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:38:48.317 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:48 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:48.317 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:48 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:38:48.317 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:48 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:38:48.317 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:48 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:38:48.317 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:48 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:48.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:48 smithi005 ceph-mon[27798]: purged_snaps scrub starts 2024-04-24T06:38:48.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:48 smithi005 ceph-mon[27798]: purged_snaps scrub ok 2024-04-24T06:38:48.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:48 smithi005 ceph-mon[27798]: osd.7 [v2:172.21.15.37:6816/2952750659,v1:172.21.15.37:6817/2952750659] boot 2024-04-24T06:38:48.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:48 smithi005 ceph-mon[27798]: osdmap e52: 8 total, 8 up, 8 in 2024-04-24T06:38:48.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:48 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T06:38:48.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:48 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:48.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:48 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:48.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:48 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:38:48.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:48 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:38:48.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:48 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:38:48.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:48 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:48.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:48 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:38:48.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:48 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:38:48.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:48 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:48.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:48 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:38:48.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:48 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:38:48.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:48 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:38:48.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:48 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:48.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:48 smithi037 ceph-mon[32864]: purged_snaps scrub starts 2024-04-24T06:38:48.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:48 smithi037 ceph-mon[32864]: purged_snaps scrub ok 2024-04-24T06:38:48.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:48 smithi037 ceph-mon[32864]: osd.7 [v2:172.21.15.37:6816/2952750659,v1:172.21.15.37:6817/2952750659] boot 2024-04-24T06:38:48.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:48 smithi037 ceph-mon[32864]: osdmap e52: 8 total, 8 up, 8 in 2024-04-24T06:38:48.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:48 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T06:38:48.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:48 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:48.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:48 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:48.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:48 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:38:48.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:48 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:38:48.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:48 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:38:48.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:48 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:48.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:48 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:38:48.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:48 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:38:48.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:48 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:48.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:48 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:38:48.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:48 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:38:48.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:48 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:38:48.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:48 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:38:49.344 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:49 smithi137 ceph-mon[33017]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:49.345 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:49 smithi137 ceph-mon[33017]: Detected new or changed devices on smithi037 2024-04-24T06:38:49.345 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:49 smithi137 ceph-mon[33017]: Adjusting osd_memory_target on smithi037 to 5697M 2024-04-24T06:38:49.345 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:49 smithi137 ceph-mon[33017]: osdmap e53: 8 total, 8 up, 8 in 2024-04-24T06:38:49.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:49 smithi005 ceph-mon[27798]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:49.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:49 smithi005 ceph-mon[27798]: Detected new or changed devices on smithi037 2024-04-24T06:38:49.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:49 smithi005 ceph-mon[27798]: Adjusting osd_memory_target on smithi037 to 5697M 2024-04-24T06:38:49.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:49 smithi005 ceph-mon[27798]: osdmap e53: 8 total, 8 up, 8 in 2024-04-24T06:38:49.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:49 smithi037 ceph-mon[32864]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-24T06:38:49.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:49 smithi037 ceph-mon[32864]: Detected new or changed devices on smithi037 2024-04-24T06:38:49.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:49 smithi037 ceph-mon[32864]: Adjusting osd_memory_target on smithi037 to 5697M 2024-04-24T06:38:49.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:49 smithi037 ceph-mon[32864]: osdmap e53: 8 total, 8 up, 8 in 2024-04-24T06:38:49.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:38:49.563+0000 7fd2aaa8e700 1 -- 172.21.15.137:0/4012984805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2a4102c70 msgr2=0x7fd2a41030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:49.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:38:49.563+0000 7fd2aaa8e700 1 --2- 172.21.15.137:0/4012984805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2a4102c70 0x7fd2a41030d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd290009990 tx=0x7fd290009ca0 comp rx=0 tx=0).stop 2024-04-24T06:38:49.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:38:49.563+0000 7fd2aaa8e700 1 -- 172.21.15.137:0/4012984805 shutdown_connections 2024-04-24T06:38:49.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:38:49.563+0000 7fd2aaa8e700 1 --2- 172.21.15.137:0/4012984805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2a4103610 0x7fd2a4109ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:49.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:38:49.563+0000 7fd2aaa8e700 1 --2- 172.21.15.137:0/4012984805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2a4101a70 0x7fd2a4101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:49.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:38:49.563+0000 7fd2aaa8e700 1 --2- 172.21.15.137:0/4012984805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2a4102c70 0x7fd2a41030d0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:49.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:38:49.563+0000 7fd2aaa8e700 1 -- 172.21.15.137:0/4012984805 >> 172.21.15.137:0/4012984805 conn(0x7fd2a40fd050 msgr2=0x7fd2a40ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:38:49.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:38:49.563+0000 7fd2aaa8e700 1 -- 172.21.15.137:0/4012984805 shutdown_connections 2024-04-24T06:38:49.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:38:49.563+0000 7fd2aaa8e700 1 -- 172.21.15.137:0/4012984805 wait complete. 2024-04-24T06:38:49.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:38:49.564+0000 7fd2aaa8e700 1 Processor -- start 2024-04-24T06:38:49.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:38:49.564+0000 7fd2aaa8e700 1 -- start start 2024-04-24T06:38:49.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:38:49.564+0000 7fd2aaa8e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2a4101a70 0x7fd2a406bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:49.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:38:49.564+0000 7fd2aaa8e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2a4102c70 0x7fd2a419dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:49.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:38:49.564+0000 7fd2aaa8e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2a4103610 0x7fd2a41a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:49.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:38:49.564+0000 7fd2aaa8e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a410d750 con 0x7fd2a4101a70 2024-04-24T06:38:49.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:38:49.564+0000 7fd2aaa8e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a410d5d0 con 0x7fd2a4102c70 2024-04-24T06:38:49.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:38:49.564+0000 7fd2aaa8e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a410d8d0 con 0x7fd2a4103610 2024-04-24T06:38:49.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:38:49.564+0000 7fd2a902b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2a4103610 0x7fd2a41a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:49.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:38:49.564+0000 7fd2a902b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2a4103610 0x7fd2a41a0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33152/0 (socket says 172.21.15.137:33152) 2024-04-24T06:38:49.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:38:49.564+0000 7fd2a902b700 1 -- 172.21.15.137:0/2650293429 learned_addr learned my addr 172.21.15.137:0/2650293429 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-04-24T06:38:49.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:38:49.565+0000 7fd2a3fff700 1 --2- 172.21.15.137:0/2650293429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2a4102c70 0x7fd2a419dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:49.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:38:49.565+0000 7fd2a902b700 1 -- 172.21.15.137:0/2650293429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2a4102c70 msgr2=0x7fd2a419dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:49.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:38:49.565+0000 7fd2a902b700 1 --2- 172.21.15.137:0/2650293429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2a4102c70 0x7fd2a419dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:49.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:38:49.565+0000 7fd2a902b700 1 -- 172.21.15.137:0/2650293429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2a4101a70 msgr2=0x7fd2a406bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:38:49.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:38:49.565+0000 7fd2a902b700 1 --2- 172.21.15.137:0/2650293429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2a4101a70 0x7fd2a406bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:38:49.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:38:49.565+0000 7fd2a902b700 1 -- 172.21.15.137:0/2650293429 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a41a0b50 con 0x7fd2a4103610 2024-04-24T06:38:49.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:38:49.565+0000 7fd2a3fff700 1 --2- 172.21.15.137:0/2650293429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2a4102c70 0x7fd2a419dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:38:49.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:38:49.565+0000 7fd2a902b700 1 --2- 172.21.15.137:0/2650293429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2a4103610 0x7fd2a41a0280 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd29c00b970 tx=0x7fd29c00bd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:49.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:38:49.565+0000 7fd2a1ffb700 1 -- 172.21.15.137:0/2650293429 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29c019070 con 0x7fd2a4103610 2024-04-24T06:38:49.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:38:49.565+0000 7fd2a1ffb700 1 -- 172.21.15.137:0/2650293429 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd29c014440 con 0x7fd2a4103610 2024-04-24T06:38:49.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:38:49.565+0000 7fd2aaa8e700 1 -- 172.21.15.137:0/2650293429 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2a41a0e40 con 0x7fd2a4103610 2024-04-24T06:38:49.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:38:49.565+0000 7fd2a1ffb700 1 -- 172.21.15.137:0/2650293429 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29c0075d0 con 0x7fd2a4103610 2024-04-24T06:38:49.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:38:49.565+0000 7fd2aaa8e700 1 -- 172.21.15.137:0/2650293429 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2a41a1380 con 0x7fd2a4103610 2024-04-24T06:38:49.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:38:49.566+0000 7fd2aaa8e700 1 -- 172.21.15.137:0/2650293429 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2a40697e0 con 0x7fd2a4103610 2024-04-24T06:38:49.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:38:49.567+0000 7fd2a1ffb700 1 -- 172.21.15.137:0/2650293429 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fd29c01c070 con 0x7fd2a4103610 2024-04-24T06:38:49.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:38:49.568+0000 7fd2a1ffb700 1 --2- 172.21.15.137:0/2650293429 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fd2940a3c40 0x7fd2940a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:38:49.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:38:49.568+0000 7fd2a1ffb700 1 -- 172.21.15.137:0/2650293429 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5426+0+0 (secure 0 0 0) 0x7fd29c0cecd0 con 0x7fd2a4103610 2024-04-24T06:38:49.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:38:49.568+0000 7fd2a882a700 1 --2- 172.21.15.137:0/2650293429 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fd2940a3c40 0x7fd2940a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:38:49.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:38:49.569+0000 7fd2a882a700 1 --2- 172.21.15.137:0/2650293429 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fd2940a3c40 0x7fd2940a6100 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4102ad0 tx=0x7fd298006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:38:49.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:38:49.572+0000 7fd2a1ffb700 1 -- 172.21.15.137:0/2650293429 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd29c09c1f0 con 0x7fd2a4103610 2024-04-24T06:38:49.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:38:49.782+0000 7fd2aaa8e700 1 -- 172.21.15.137:0/2650293429 --> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fd2a4062f30 con 0x7fd2940a3c40 2024-04-24T06:38:50.422 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:50 smithi137 ceph-mon[33017]: osdmap e54: 8 total, 8 up, 8 in 2024-04-24T06:38:50.422 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:50 smithi137 ceph-mon[33017]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-24T06:38:50.422 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:50 smithi137 ceph-mon[33017]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:38:50.422 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:50 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T06:38:50.422 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:50 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T06:38:50.423 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:50 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:38:50.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:50 smithi005 ceph-mon[27798]: osdmap e54: 8 total, 8 up, 8 in 2024-04-24T06:38:50.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:50 smithi005 ceph-mon[27798]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-24T06:38:50.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:50 smithi005 ceph-mon[27798]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:38:50.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:50 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T06:38:50.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:50 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T06:38:50.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:50 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:38:50.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:50 smithi037 ceph-mon[32864]: osdmap e54: 8 total, 8 up, 8 in 2024-04-24T06:38:50.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:50 smithi037 ceph-mon[32864]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-24T06:38:50.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:50 smithi037 ceph-mon[32864]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:38:50.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:50 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T06:38:50.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:50 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T06:38:50.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:50 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:38:51.428 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:51 smithi137 ceph-mon[33017]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-04-24T06:38:51.428 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:51 smithi137 ceph-mon[33017]: osdmap e55: 8 total, 8 up, 8 in 2024-04-24T06:38:51.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:51 smithi005 ceph-mon[27798]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-04-24T06:38:51.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:51 smithi005 ceph-mon[27798]: osdmap e55: 8 total, 8 up, 8 in 2024-04-24T06:38:51.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:51 smithi037 ceph-mon[32864]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-04-24T06:38:51.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:51 smithi037 ceph-mon[32864]: osdmap e55: 8 total, 8 up, 8 in 2024-04-24T06:38:52.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:52 smithi137 ceph-mon[33017]: osdmap e56: 8 total, 8 up, 8 in 2024-04-24T06:38:52.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:52 smithi137 ceph-mon[33017]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-24T06:38:52.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:52 smithi005 ceph-mon[27798]: osdmap e56: 8 total, 8 up, 8 in 2024-04-24T06:38:52.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:52 smithi005 ceph-mon[27798]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-24T06:38:52.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:52 smithi037 ceph-mon[32864]: osdmap e56: 8 total, 8 up, 8 in 2024-04-24T06:38:52.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:52 smithi037 ceph-mon[32864]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-24T06:38:53.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:53 smithi005 ceph-mon[27798]: from='client.? 172.21.15.137:0/297957029' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5391fd87-44ea-440f-80da-c54d0327c04c"}]: dispatch 2024-04-24T06:38:53.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:53 smithi005 ceph-mon[27798]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5391fd87-44ea-440f-80da-c54d0327c04c"}]: dispatch 2024-04-24T06:38:53.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:53 smithi005 ceph-mon[27798]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5391fd87-44ea-440f-80da-c54d0327c04c"}]': finished 2024-04-24T06:38:53.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:53 smithi005 ceph-mon[27798]: osdmap e57: 9 total, 8 up, 9 in 2024-04-24T06:38:53.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:53 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T06:38:53.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:53 smithi005 ceph-mon[27798]: from='client.? 172.21.15.137:0/4061405121' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T06:38:53.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:53 smithi037 ceph-mon[32864]: from='client.? 172.21.15.137:0/297957029' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5391fd87-44ea-440f-80da-c54d0327c04c"}]: dispatch 2024-04-24T06:38:53.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:53 smithi037 ceph-mon[32864]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5391fd87-44ea-440f-80da-c54d0327c04c"}]: dispatch 2024-04-24T06:38:53.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:53 smithi037 ceph-mon[32864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5391fd87-44ea-440f-80da-c54d0327c04c"}]': finished 2024-04-24T06:38:53.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:53 smithi037 ceph-mon[32864]: osdmap e57: 9 total, 8 up, 9 in 2024-04-24T06:38:53.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:53 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T06:38:53.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:53 smithi037 ceph-mon[32864]: from='client.? 172.21.15.137:0/4061405121' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T06:38:53.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:53 smithi137 ceph-mon[33017]: from='client.? 172.21.15.137:0/297957029' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5391fd87-44ea-440f-80da-c54d0327c04c"}]: dispatch 2024-04-24T06:38:53.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:53 smithi137 ceph-mon[33017]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5391fd87-44ea-440f-80da-c54d0327c04c"}]: dispatch 2024-04-24T06:38:53.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:53 smithi137 ceph-mon[33017]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5391fd87-44ea-440f-80da-c54d0327c04c"}]': finished 2024-04-24T06:38:53.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:53 smithi137 ceph-mon[33017]: osdmap e57: 9 total, 8 up, 9 in 2024-04-24T06:38:53.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:53 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T06:38:53.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:53 smithi137 ceph-mon[33017]: from='client.? 172.21.15.137:0/4061405121' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T06:38:54.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:54 smithi005 ceph-mon[27798]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-24T06:38:54.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:54 smithi037 ceph-mon[32864]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-24T06:38:54.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:54 smithi137 ceph-mon[33017]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-24T06:38:55.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:55 smithi005 ceph-mon[27798]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-04-24T06:38:55.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:55 smithi005 ceph-mon[27798]: Cluster is now healthy 2024-04-24T06:38:55.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:55 smithi037 ceph-mon[32864]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-04-24T06:38:55.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:55 smithi037 ceph-mon[32864]: Cluster is now healthy 2024-04-24T06:38:55.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:55 smithi137 ceph-mon[33017]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-04-24T06:38:55.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:55 smithi137 ceph-mon[33017]: Cluster is now healthy 2024-04-24T06:38:56.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:56 smithi005 ceph-mon[27798]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-24T06:38:56.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:56 smithi037 ceph-mon[32864]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-24T06:38:56.571 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:56 smithi137 ceph-mon[33017]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-04-24T06:38:58.609 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:38:58 smithi137 ceph-mon[33017]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-24T06:38:58.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:38:58 smithi005 ceph-mon[27798]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-24T06:38:58.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:38:58 smithi037 ceph-mon[32864]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-24T06:39:00.630 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:00 smithi137 ceph-mon[33017]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-24T06:39:00.630 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:00 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-24T06:39:00.630 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:00 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:39:00.630 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:00 smithi137 ceph-mon[33017]: Deploying daemon osd.8 on smithi137 2024-04-24T06:39:00.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:00 smithi005 ceph-mon[27798]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-24T06:39:00.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:00 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-24T06:39:00.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:00 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:39:00.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:00 smithi005 ceph-mon[27798]: Deploying daemon osd.8 on smithi137 2024-04-24T06:39:00.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:00 smithi037 ceph-mon[32864]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-24T06:39:00.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:00 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-24T06:39:00.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:00 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:39:00.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:00 smithi037 ceph-mon[32864]: Deploying daemon osd.8 on smithi137 2024-04-24T06:39:02.606 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:02 smithi137 ceph-mon[33017]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-24T06:39:02.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:02 smithi005 ceph-mon[27798]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-24T06:39:02.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:02 smithi037 ceph-mon[32864]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-24T06:39:04.584 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:04 smithi037 ceph-mon[32864]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-24T06:39:04.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:04 smithi137 ceph-mon[33017]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-24T06:39:04.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:04 smithi005 ceph-mon[27798]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-24T06:39:06.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:06 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:06.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:06 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:06.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:06 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:39:06.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:06 smithi137 ceph-mon[33017]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-24T06:39:06.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:06 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:06.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:06 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:06.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:06 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:39:06.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:06 smithi005 ceph-mon[27798]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-24T06:39:06.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:06 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:06.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:06 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:06.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:06 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:39:06.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:06 smithi037 ceph-mon[32864]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-24T06:39:08.634 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:08 smithi137 ceph-mon[33017]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-24T06:39:08.634 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:08 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:08.634 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:08 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:08.634 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:08 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:39:08.634 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:08 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:39:08.634 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:08 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:08.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:08 smithi005 ceph-mon[27798]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-24T06:39:08.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:08 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:08.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:08 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:08.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:08 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:39:08.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:08 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:39:08.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:08 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:08.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:08 smithi037 ceph-mon[32864]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-24T06:39:08.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:08 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:08.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:08 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:08.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:08 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:39:08.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:08 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:39:08.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:08 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:08.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:39:08.969+0000 7fd2a1ffb700 1 -- 172.21.15.137:0/2650293429 <== mgr.14152 v2:172.21.15.5:6800/4042936965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd2a4062f30 con 0x7fd2940a3c40 2024-04-24T06:39:08.971 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 8 on host 'smithi137' 2024-04-24T06:39:08.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:39:08.972+0000 7fd2aaa8e700 1 -- 172.21.15.137:0/2650293429 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fd2940a3c40 msgr2=0x7fd2940a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:08.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:39:08.972+0000 7fd2aaa8e700 1 --2- 172.21.15.137:0/2650293429 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fd2940a3c40 0x7fd2940a6100 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4102ad0 tx=0x7fd298006d20 comp rx=0 tx=0).stop 2024-04-24T06:39:08.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:39:08.972+0000 7fd2aaa8e700 1 -- 172.21.15.137:0/2650293429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2a4103610 msgr2=0x7fd2a41a0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:08.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:39:08.972+0000 7fd2aaa8e700 1 --2- 172.21.15.137:0/2650293429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2a4103610 0x7fd2a41a0280 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd29c00b970 tx=0x7fd29c00bd30 comp rx=0 tx=0).stop 2024-04-24T06:39:08.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:39:08.972+0000 7fd2aaa8e700 1 -- 172.21.15.137:0/2650293429 shutdown_connections 2024-04-24T06:39:08.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:39:08.972+0000 7fd2aaa8e700 1 --2- 172.21.15.137:0/2650293429 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fd2940a3c40 0x7fd2940a6100 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:08.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:39:08.972+0000 7fd2aaa8e700 1 --2- 172.21.15.137:0/2650293429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2a4103610 0x7fd2a41a0280 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:08.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:39:08.972+0000 7fd2aaa8e700 1 --2- 172.21.15.137:0/2650293429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2a4101a70 0x7fd2a406bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:08.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:39:08.972+0000 7fd2aaa8e700 1 --2- 172.21.15.137:0/2650293429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2a4102c70 0x7fd2a419dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:08.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:39:08.972+0000 7fd2aaa8e700 1 -- 172.21.15.137:0/2650293429 >> 172.21.15.137:0/2650293429 conn(0x7fd2a40fd050 msgr2=0x7fd2a4106040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:08.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:39:08.972+0000 7fd2aaa8e700 1 -- 172.21.15.137:0/2650293429 shutdown_connections 2024-04-24T06:39:08.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:39:08.972+0000 7fd2aaa8e700 1 -- 172.21.15.137:0/2650293429 wait complete. 2024-04-24T06:39:09.573 DEBUG:teuthology.orchestra.run.smithi137:osd.8> sudo journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@osd.8.service 2024-04-24T06:39:09.575 INFO:tasks.cephadm:Deploying osd.9 on smithi005 with /dev/vg_nvme/lv_1... 2024-04-24T06:39:09.575 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-24T06:39:10.039 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:39:10.116 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:09 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:10.116 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:09 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:10.116 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:09 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:39:10.116 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:09 smithi137 ceph-mon[33017]: from='osd.8 [v2:172.21.15.137:6816/29724945,v1:172.21.15.137:6817/29724945]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T06:39:10.116 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:09 smithi137 ceph-mon[33017]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T06:39:10.195 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:09 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:10.197 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:09 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:10.197 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:09 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:39:10.197 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:09 smithi005 ceph-mon[27798]: from='osd.8 [v2:172.21.15.137:6816/29724945,v1:172.21.15.137:6817/29724945]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T06:39:10.197 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:09 smithi005 ceph-mon[27798]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T06:39:10.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:09 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:10.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:09 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:10.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:09 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:39:10.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:09 smithi037 ceph-mon[32864]: from='osd.8 [v2:172.21.15.137:6816/29724945,v1:172.21.15.137:6817/29724945]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T06:39:10.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:09 smithi037 ceph-mon[32864]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T06:39:11.244 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:10 smithi005 ceph-mon[27798]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-24T06:39:11.245 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:10 smithi005 ceph-mon[27798]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-24T06:39:11.245 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:10 smithi005 ceph-mon[27798]: from='osd.8 [v2:172.21.15.137:6816/29724945,v1:172.21.15.137:6817/29724945]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-24T06:39:11.245 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:10 smithi005 ceph-mon[27798]: osdmap e58: 9 total, 8 up, 9 in 2024-04-24T06:39:11.245 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:10 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T06:39:11.245 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:10 smithi005 ceph-mon[27798]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-24T06:39:11.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:10 smithi037 ceph-mon[32864]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-24T06:39:11.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:10 smithi037 ceph-mon[32864]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-24T06:39:11.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:10 smithi037 ceph-mon[32864]: from='osd.8 [v2:172.21.15.137:6816/29724945,v1:172.21.15.137:6817/29724945]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-24T06:39:11.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:10 smithi037 ceph-mon[32864]: osdmap e58: 9 total, 8 up, 9 in 2024-04-24T06:39:11.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:10 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T06:39:11.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:10 smithi037 ceph-mon[32864]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-24T06:39:11.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:10 smithi137 ceph-mon[33017]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-24T06:39:11.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:10 smithi137 ceph-mon[33017]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-24T06:39:11.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:10 smithi137 ceph-mon[33017]: from='osd.8 [v2:172.21.15.137:6816/29724945,v1:172.21.15.137:6817/29724945]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-24T06:39:11.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:10 smithi137 ceph-mon[33017]: osdmap e58: 9 total, 8 up, 9 in 2024-04-24T06:39:11.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:10 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T06:39:11.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:10 smithi137 ceph-mon[33017]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-24T06:39:11.447 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:39:10 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:39:10.963+0000 7f7e5bfb9700 -1 osd.8 0 waiting for initial osdmap 2024-04-24T06:39:11.447 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:39:10 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:39:10.978+0000 7f7e57dac700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T06:39:12.067 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:11 smithi137 ceph-mon[33017]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-04-24T06:39:12.067 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:11 smithi137 ceph-mon[33017]: osdmap e59: 9 total, 8 up, 9 in 2024-04-24T06:39:12.067 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:11 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T06:39:12.068 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:11 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T06:39:12.068 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:11 smithi137 ceph-mon[33017]: osd.8 [v2:172.21.15.137:6816/29724945,v1:172.21.15.137:6817/29724945] boot 2024-04-24T06:39:12.068 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:11 smithi137 ceph-mon[33017]: osdmap e60: 9 total, 9 up, 9 in 2024-04-24T06:39:12.068 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:11 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T06:39:12.243 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:11 smithi005 ceph-mon[27798]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-04-24T06:39:12.243 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:11 smithi005 ceph-mon[27798]: osdmap e59: 9 total, 8 up, 9 in 2024-04-24T06:39:12.243 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:11 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T06:39:12.244 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:11 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T06:39:12.244 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:11 smithi005 ceph-mon[27798]: osd.8 [v2:172.21.15.137:6816/29724945,v1:172.21.15.137:6817/29724945] boot 2024-04-24T06:39:12.244 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:11 smithi005 ceph-mon[27798]: osdmap e60: 9 total, 9 up, 9 in 2024-04-24T06:39:12.244 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:11 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T06:39:12.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:11 smithi037 ceph-mon[32864]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-04-24T06:39:12.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:11 smithi037 ceph-mon[32864]: osdmap e59: 9 total, 8 up, 9 in 2024-04-24T06:39:12.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:11 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T06:39:12.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:11 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T06:39:12.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:11 smithi037 ceph-mon[32864]: osd.8 [v2:172.21.15.137:6816/29724945,v1:172.21.15.137:6817/29724945] boot 2024-04-24T06:39:12.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:11 smithi037 ceph-mon[32864]: osdmap e60: 9 total, 9 up, 9 in 2024-04-24T06:39:12.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:11 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T06:39:13.019 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-24T06:39:13.019 INFO:teuthology.orchestra.run.smithi005.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-24T06:39:13.020 INFO:teuthology.orchestra.run.smithi005.stderr: stderr: 10+0 records in 2024-04-24T06:39:13.020 INFO:teuthology.orchestra.run.smithi005.stderr:10+0 records out 2024-04-24T06:39:13.020 INFO:teuthology.orchestra.run.smithi005.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0328163 s, 320 MB/s 2024-04-24T06:39:13.020 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping successful for: 2024-04-24T06:39:13.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:12 smithi005 ceph-mon[27798]: purged_snaps scrub starts 2024-04-24T06:39:13.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:12 smithi005 ceph-mon[27798]: purged_snaps scrub ok 2024-04-24T06:39:13.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:12 smithi005 ceph-mon[27798]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-24T06:39:13.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:12 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:13.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:12 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:13.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:12 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:39:13.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:12 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:39:13.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:12 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:39:13.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:12 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:13.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:12 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:39:13.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:12 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:39:13.258 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:12 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:13.258 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:12 smithi005 ceph-mon[27798]: osdmap e61: 9 total, 9 up, 9 in 2024-04-24T06:39:13.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:12 smithi037 ceph-mon[32864]: purged_snaps scrub starts 2024-04-24T06:39:13.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:12 smithi037 ceph-mon[32864]: purged_snaps scrub ok 2024-04-24T06:39:13.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:12 smithi037 ceph-mon[32864]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-24T06:39:13.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:12 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:13.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:12 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:13.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:12 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:39:13.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:12 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:39:13.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:12 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:39:13.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:12 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:13.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:12 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:39:13.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:12 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:39:13.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:12 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:13.270 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:12 smithi037 ceph-mon[32864]: osdmap e61: 9 total, 9 up, 9 in 2024-04-24T06:39:13.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:12 smithi137 ceph-mon[33017]: purged_snaps scrub starts 2024-04-24T06:39:13.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:12 smithi137 ceph-mon[33017]: purged_snaps scrub ok 2024-04-24T06:39:13.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:12 smithi137 ceph-mon[33017]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-04-24T06:39:13.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:12 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:13.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:12 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:13.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:12 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:39:13.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:12 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:39:13.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:12 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:39:13.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:12 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:13.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:12 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:39:13.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:12 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:39:13.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:12 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:13.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:12 smithi137 ceph-mon[33017]: osdmap e61: 9 total, 9 up, 9 in 2024-04-24T06:39:13.898 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi005:vg_nvme/lv_1 2024-04-24T06:39:14.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:13 smithi005 ceph-mon[27798]: Detected new or changed devices on smithi137 2024-04-24T06:39:14.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:13 smithi005 ceph-mon[27798]: Adjusting osd_memory_target on smithi137 to 5696M 2024-04-24T06:39:14.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:13 smithi005 ceph-mon[27798]: osdmap e62: 9 total, 9 up, 9 in 2024-04-24T06:39:14.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:13 smithi037 ceph-mon[32864]: Detected new or changed devices on smithi137 2024-04-24T06:39:14.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:13 smithi037 ceph-mon[32864]: Adjusting osd_memory_target on smithi137 to 5696M 2024-04-24T06:39:14.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:13 smithi037 ceph-mon[32864]: osdmap e62: 9 total, 9 up, 9 in 2024-04-24T06:39:14.353 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:39:14.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:13 smithi137 ceph-mon[33017]: Detected new or changed devices on smithi137 2024-04-24T06:39:14.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:13 smithi137 ceph-mon[33017]: Adjusting osd_memory_target on smithi137 to 5696M 2024-04-24T06:39:14.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:13 smithi137 ceph-mon[33017]: osdmap e62: 9 total, 9 up, 9 in 2024-04-24T06:39:15.242 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:15 smithi005 ceph-mon[27798]: pgmap v217: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T06:39:15.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:15 smithi037 ceph-mon[32864]: pgmap v217: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T06:39:15.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:15 smithi137 ceph-mon[33017]: pgmap v217: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T06:39:16.087 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:16 smithi005 ceph-mon[27798]: osdmap e63: 9 total, 9 up, 9 in 2024-04-24T06:39:16.087 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:16 smithi005 ceph-mon[27798]: osdmap e64: 9 total, 9 up, 9 in 2024-04-24T06:39:16.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:16 smithi037 ceph-mon[32864]: osdmap e63: 9 total, 9 up, 9 in 2024-04-24T06:39:16.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:16 smithi037 ceph-mon[32864]: osdmap e64: 9 total, 9 up, 9 in 2024-04-24T06:39:16.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:16 smithi137 ceph-mon[33017]: osdmap e63: 9 total, 9 up, 9 in 2024-04-24T06:39:16.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:16 smithi137 ceph-mon[33017]: osdmap e64: 9 total, 9 up, 9 in 2024-04-24T06:39:16.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.583+0000 7f2fdd800700 1 -- 172.21.15.5:0/2411135011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2fd8106940 msgr2=0x7f2fd810d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:16.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.583+0000 7f2fdd800700 1 --2- 172.21.15.5:0/2411135011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2fd8106940 0x7f2fd810d1f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0009fc0 tx=0x7f2fd000b330 comp rx=0 tx=0).stop 2024-04-24T06:39:16.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.584+0000 7f2fdd800700 1 -- 172.21.15.5:0/2411135011 shutdown_connections 2024-04-24T06:39:16.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.584+0000 7f2fdd800700 1 --2- 172.21.15.5:0/2411135011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2fd8106940 0x7f2fd810d1f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:16.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.584+0000 7f2fdd800700 1 --2- 172.21.15.5:0/2411135011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fd8104da0 0x7f2fd81051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:16.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.584+0000 7f2fdd800700 1 --2- 172.21.15.5:0/2411135011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fd8105fa0 0x7f2fd8106400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:16.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.584+0000 7f2fdd800700 1 -- 172.21.15.5:0/2411135011 >> 172.21.15.5:0/2411135011 conn(0x7f2fd8100340 msgr2=0x7f2fd8102780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:16.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.584+0000 7f2fdd800700 1 -- 172.21.15.5:0/2411135011 shutdown_connections 2024-04-24T06:39:16.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.584+0000 7f2fdd800700 1 -- 172.21.15.5:0/2411135011 wait complete. 2024-04-24T06:39:16.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.585+0000 7f2fdd800700 1 Processor -- start 2024-04-24T06:39:16.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.585+0000 7f2fdd800700 1 -- start start 2024-04-24T06:39:16.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.585+0000 7f2fdd800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2fd8104da0 0x7f2fd819aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:16.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.585+0000 7f2fdd800700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fd8105fa0 0x7f2fd819b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:16.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.585+0000 7f2fdd800700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fd8106940 0x7f2fd81a2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:16.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.585+0000 7f2fdd800700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd8110970 con 0x7f2fd8106940 2024-04-24T06:39:16.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.585+0000 7f2fdd800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd81107f0 con 0x7f2fd8104da0 2024-04-24T06:39:16.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.585+0000 7f2fdd800700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd8110af0 con 0x7f2fd8105fa0 2024-04-24T06:39:16.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.586+0000 7f2fd7400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fd8106940 0x7f2fd81a2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:16.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.586+0000 7f2fcfe00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fd8105fa0 0x7f2fd819b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:16.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.586+0000 7f2fd6a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2fd8104da0 0x7f2fd819aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:16.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.586+0000 7f2fcfe00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fd8105fa0 0x7f2fd819b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:44986/0 (socket says 172.21.15.5:44986) 2024-04-24T06:39:16.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.586+0000 7f2fcfe00700 1 -- 172.21.15.5:0/3860182197 learned_addr learned my addr 172.21.15.5:0/3860182197 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:39:16.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.586+0000 7f2fd6a00700 1 -- 172.21.15.5:0/3860182197 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fd8105fa0 msgr2=0x7f2fd819b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:16.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.586+0000 7f2fd6a00700 1 --2- 172.21.15.5:0/3860182197 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fd8105fa0 0x7f2fd819b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:16.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.586+0000 7f2fd6a00700 1 -- 172.21.15.5:0/3860182197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fd8106940 msgr2=0x7f2fd81a2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:16.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.586+0000 7f2fd6a00700 1 --2- 172.21.15.5:0/3860182197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fd8106940 0x7f2fd81a2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:16.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.586+0000 7f2fd6a00700 1 -- 172.21.15.5:0/3860182197 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd81a2b20 con 0x7f2fd8104da0 2024-04-24T06:39:16.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.586+0000 7f2fd7400700 1 --2- 172.21.15.5:0/3860182197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fd8106940 0x7f2fd81a2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:39:16.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.587+0000 7f2fd6a00700 1 --2- 172.21.15.5:0/3860182197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2fd8104da0 0x7f2fd819aec0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2fc800e9c0 tx=0x7f2fc800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:16.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.587+0000 7f2fcf400700 1 -- 172.21.15.5:0/3860182197 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc800cb70 con 0x7f2fd8104da0 2024-04-24T06:39:16.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.587+0000 7f2fcf400700 1 -- 172.21.15.5:0/3860182197 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fc800ee60 con 0x7f2fd8104da0 2024-04-24T06:39:16.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.587+0000 7f2fcf400700 1 -- 172.21.15.5:0/3860182197 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc8018640 con 0x7f2fd8104da0 2024-04-24T06:39:16.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.587+0000 7f2fdd800700 1 -- 172.21.15.5:0/3860182197 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fd81a2e10 con 0x7f2fd8104da0 2024-04-24T06:39:16.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.587+0000 7f2fdd800700 1 -- 172.21.15.5:0/3860182197 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fd81a3210 con 0x7f2fd8104da0 2024-04-24T06:39:16.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.588+0000 7f2fdd800700 1 -- 172.21.15.5:0/3860182197 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fd802dfe0 con 0x7f2fd8104da0 2024-04-24T06:39:16.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.590+0000 7f2fcf400700 1 -- 172.21.15.5:0/3860182197 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f2fc801f020 con 0x7f2fd8104da0 2024-04-24T06:39:16.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.592+0000 7f2fcf400700 1 --2- 172.21.15.5:0/3860182197 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f2fbc0a3cc0 0x7f2fbc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:16.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.592+0000 7f2fcf400700 1 -- 172.21.15.5:0/3860182197 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5829+0+0 (secure 0 0 0) 0x7f2fc80cfab0 con 0x7f2fd8104da0 2024-04-24T06:39:16.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.592+0000 7f2fcfe00700 1 --2- 172.21.15.5:0/3860182197 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f2fbc0a3cc0 0x7f2fbc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:16.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.592+0000 7f2fcfe00700 1 --2- 172.21.15.5:0/3860182197 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f2fbc0a3cc0 0x7f2fbc0a6180 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0005950 tx=0x7f2fc0009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:16.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.594+0000 7f2fcf400700 1 -- 172.21.15.5:0/3860182197 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2fc809cda0 con 0x7f2fd8104da0 2024-04-24T06:39:16.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:16.838+0000 7f2fdd800700 1 -- 172.21.15.5:0/3860182197 --> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f2fd8063970 con 0x7f2fbc0a3cc0 2024-04-24T06:39:17.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:17 smithi005 ceph-mon[27798]: pgmap v220: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T06:39:17.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:17 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T06:39:17.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:17 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T06:39:17.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:17 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:39:17.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:17 smithi037 ceph-mon[32864]: pgmap v220: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T06:39:17.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:17 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T06:39:17.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:17 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T06:39:17.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:17 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:39:17.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:17 smithi137 ceph-mon[33017]: pgmap v220: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T06:39:17.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:17 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T06:39:17.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:17 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T06:39:17.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:17 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:39:18.136 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:18 smithi005 ceph-mon[27798]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:39:18.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:18 smithi137 ceph-mon[33017]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:39:18.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:18 smithi037 ceph-mon[32864]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:39:19.229 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:19 smithi005 ceph-mon[27798]: pgmap v222: 1 pgs: 1 remapped+peering; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-04-24T06:39:19.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:19 smithi137 ceph-mon[33017]: pgmap v222: 1 pgs: 1 remapped+peering; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-04-24T06:39:19.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:19 smithi037 ceph-mon[32864]: pgmap v222: 1 pgs: 1 remapped+peering; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail 2024-04-24T06:39:21.442 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:21 smithi005 ceph-mon[27798]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T06:39:21.442 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:21 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/2290667384' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d94c9dbc-f43d-4237-ac7f-47a6a5ede1c4"}]: dispatch 2024-04-24T06:39:21.442 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:21 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/2290667384' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d94c9dbc-f43d-4237-ac7f-47a6a5ede1c4"}]': finished 2024-04-24T06:39:21.442 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:21 smithi005 ceph-mon[27798]: osdmap e65: 10 total, 9 up, 10 in 2024-04-24T06:39:21.442 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:21 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T06:39:21.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:21 smithi137 ceph-mon[33017]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T06:39:21.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:21 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/2290667384' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d94c9dbc-f43d-4237-ac7f-47a6a5ede1c4"}]: dispatch 2024-04-24T06:39:21.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:21 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/2290667384' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d94c9dbc-f43d-4237-ac7f-47a6a5ede1c4"}]': finished 2024-04-24T06:39:21.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:21 smithi137 ceph-mon[33017]: osdmap e65: 10 total, 9 up, 10 in 2024-04-24T06:39:21.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:21 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T06:39:21.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:21 smithi037 ceph-mon[32864]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T06:39:21.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:21 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/2290667384' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d94c9dbc-f43d-4237-ac7f-47a6a5ede1c4"}]: dispatch 2024-04-24T06:39:21.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:21 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/2290667384' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d94c9dbc-f43d-4237-ac7f-47a6a5ede1c4"}]': finished 2024-04-24T06:39:21.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:21 smithi037 ceph-mon[32864]: osdmap e65: 10 total, 9 up, 10 in 2024-04-24T06:39:21.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:21 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T06:39:22.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:22 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/1268149264' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T06:39:22.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:22 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1268149264' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T06:39:22.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:22 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/1268149264' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T06:39:23.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:23 smithi137 ceph-mon[33017]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-24T06:39:23.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:23 smithi005 ceph-mon[27798]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-24T06:39:23.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:23 smithi037 ceph-mon[32864]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-24T06:39:25.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:25 smithi137 ceph-mon[33017]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T06:39:25.488 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:25 smithi005 ceph-mon[27798]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T06:39:25.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:25 smithi037 ceph-mon[32864]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T06:39:27.331 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:27 smithi005 ceph-mon[27798]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-24T06:39:27.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:27 smithi137 ceph-mon[33017]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-24T06:39:27.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:27 smithi037 ceph-mon[32864]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-24T06:39:29.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:29 smithi005 ceph-mon[27798]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T06:39:29.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:29 smithi137 ceph-mon[33017]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T06:39:29.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:29 smithi037 ceph-mon[32864]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T06:39:31.409 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:31 smithi005 ceph-mon[27798]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T06:39:31.410 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:31 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-24T06:39:31.410 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:31 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:39:31.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:31 smithi137 ceph-mon[33017]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T06:39:31.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:31 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-24T06:39:31.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:31 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:39:31.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:31 smithi037 ceph-mon[32864]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T06:39:31.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:31 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-24T06:39:31.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:31 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:39:32.390 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:32 smithi005 ceph-mon[27798]: Deploying daemon osd.9 on smithi005 2024-04-24T06:39:32.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:32 smithi137 ceph-mon[33017]: Deploying daemon osd.9 on smithi005 2024-04-24T06:39:32.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:32 smithi037 ceph-mon[32864]: Deploying daemon osd.9 on smithi005 2024-04-24T06:39:33.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:33 smithi137 ceph-mon[33017]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T06:39:33.489 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:33 smithi005 ceph-mon[27798]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T06:39:33.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:33 smithi037 ceph-mon[32864]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T06:39:35.337 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:35 smithi037 ceph-mon[32864]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T06:39:35.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:35 smithi005 ceph-mon[27798]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T06:39:35.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:35 smithi137 ceph-mon[33017]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T06:39:37.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:37 smithi137 ceph-mon[33017]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T06:39:37.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:37 smithi005 ceph-mon[27798]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T06:39:37.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:37 smithi037 ceph-mon[32864]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T06:39:39.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:39 smithi137 ceph-mon[33017]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T06:39:39.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:39 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:39.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:39 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:39.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:39 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:39:39.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:39 smithi005 ceph-mon[27798]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T06:39:39.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:39 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:39.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:39 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:39.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:39 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:39:39.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:39 smithi037 ceph-mon[32864]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T06:39:39.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:39 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:39.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:39 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:39.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:39 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:39:41.176 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:41 smithi005 ceph-mon[27798]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T06:39:41.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:41 smithi137 ceph-mon[33017]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T06:39:41.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:41 smithi037 ceph-mon[32864]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T06:39:43.431 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:43 smithi005 ceph-mon[27798]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T06:39:43.431 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:43 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:43.431 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:43 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:43.431 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:43 smithi005 ceph-mon[27798]: from='osd.9 [v2:172.21.15.5:6826/2257102107,v1:172.21.15.5:6827/2257102107]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-24T06:39:43.431 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:43 smithi005 ceph-mon[27798]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-24T06:39:43.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:43 smithi137 ceph-mon[33017]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T06:39:43.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:43 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:43.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:43 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:43.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:43 smithi137 ceph-mon[33017]: from='osd.9 [v2:172.21.15.5:6826/2257102107,v1:172.21.15.5:6827/2257102107]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-24T06:39:43.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:43 smithi137 ceph-mon[33017]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-24T06:39:43.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:43 smithi037 ceph-mon[32864]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T06:39:43.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:43 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:43.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:43 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:43.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:43 smithi037 ceph-mon[32864]: from='osd.9 [v2:172.21.15.5:6826/2257102107,v1:172.21.15.5:6827/2257102107]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-24T06:39:43.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:43 smithi037 ceph-mon[32864]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-24T06:39:44.005 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 9 on host 'smithi005' 2024-04-24T06:39:44.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:44.002+0000 7f2fcf400700 1 -- 172.21.15.5:0/3860182197 <== mgr.14152 v2:172.21.15.5:6800/4042936965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2fd8063970 con 0x7f2fbc0a3cc0 2024-04-24T06:39:44.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:44.006+0000 7f2fdd800700 1 -- 172.21.15.5:0/3860182197 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f2fbc0a3cc0 msgr2=0x7f2fbc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:44.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:44.006+0000 7f2fdd800700 1 --2- 172.21.15.5:0/3860182197 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f2fbc0a3cc0 0x7f2fbc0a6180 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0005950 tx=0x7f2fc0009310 comp rx=0 tx=0).stop 2024-04-24T06:39:44.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:44.007+0000 7f2fdd800700 1 -- 172.21.15.5:0/3860182197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2fd8104da0 msgr2=0x7f2fd819aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:44.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:44.007+0000 7f2fdd800700 1 --2- 172.21.15.5:0/3860182197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2fd8104da0 0x7f2fd819aec0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2fc800e9c0 tx=0x7f2fc800ecd0 comp rx=0 tx=0).stop 2024-04-24T06:39:44.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:44.007+0000 7f2fdd800700 1 -- 172.21.15.5:0/3860182197 shutdown_connections 2024-04-24T06:39:44.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:44.007+0000 7f2fdd800700 1 --2- 172.21.15.5:0/3860182197 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f2fbc0a3cc0 0x7f2fbc0a6180 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:44.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:44.007+0000 7f2fdd800700 1 --2- 172.21.15.5:0/3860182197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fd8106940 0x7f2fd81a2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:44.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:44.007+0000 7f2fdd800700 1 --2- 172.21.15.5:0/3860182197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2fd8104da0 0x7f2fd819aec0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:44.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:44.007+0000 7f2fdd800700 1 --2- 172.21.15.5:0/3860182197 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fd8105fa0 0x7f2fd819b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:44.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:44.007+0000 7f2fdd800700 1 -- 172.21.15.5:0/3860182197 >> 172.21.15.5:0/3860182197 conn(0x7f2fd8100340 msgr2=0x7f2fd81091d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:44.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:44.007+0000 7f2fdd800700 1 -- 172.21.15.5:0/3860182197 shutdown_connections 2024-04-24T06:39:44.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:39:44.007+0000 7f2fdd800700 1 -- 172.21.15.5:0/3860182197 wait complete. 2024-04-24T06:39:44.780 DEBUG:teuthology.orchestra.run.smithi005:osd.9> sudo journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@osd.9.service 2024-04-24T06:39:44.788 INFO:tasks.cephadm:Deploying osd.10 on smithi037 with /dev/vg_nvme/lv_1... 2024-04-24T06:39:44.788 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-24T06:39:44.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:44 smithi137 ceph-mon[33017]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T06:39:44.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:44 smithi137 ceph-mon[33017]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-24T06:39:44.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:44 smithi137 ceph-mon[33017]: osdmap e66: 10 total, 9 up, 10 in 2024-04-24T06:39:44.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:44 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T06:39:44.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:44 smithi137 ceph-mon[33017]: from='osd.9 [v2:172.21.15.5:6826/2257102107,v1:172.21.15.5:6827/2257102107]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-04-24T06:39:44.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:44 smithi137 ceph-mon[33017]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-04-24T06:39:44.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:44 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:39:44.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:44 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:39:44.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:44 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:44.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:44 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:44.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:44 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:44.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:44 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:39:45.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:44 smithi005 ceph-mon[27798]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T06:39:45.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:44 smithi005 ceph-mon[27798]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-24T06:39:45.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:44 smithi005 ceph-mon[27798]: osdmap e66: 10 total, 9 up, 10 in 2024-04-24T06:39:45.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:44 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T06:39:45.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:44 smithi005 ceph-mon[27798]: from='osd.9 [v2:172.21.15.5:6826/2257102107,v1:172.21.15.5:6827/2257102107]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-04-24T06:39:45.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:44 smithi005 ceph-mon[27798]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-04-24T06:39:45.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:44 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:39:45.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:44 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:39:45.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:44 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:45.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:44 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:45.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:44 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:45.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:44 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:39:45.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:44 smithi037 ceph-mon[32864]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T06:39:45.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:44 smithi037 ceph-mon[32864]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-24T06:39:45.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:44 smithi037 ceph-mon[32864]: osdmap e66: 10 total, 9 up, 10 in 2024-04-24T06:39:45.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:44 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T06:39:45.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:44 smithi037 ceph-mon[32864]: from='osd.9 [v2:172.21.15.5:6826/2257102107,v1:172.21.15.5:6827/2257102107]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-04-24T06:39:45.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:44 smithi037 ceph-mon[32864]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-04-24T06:39:45.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:44 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:39:45.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:44 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:39:45.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:44 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:45.020 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:44 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:45.020 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:44 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:45.020 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:44 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:39:45.043 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T06:39:45.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:45 smithi037 ceph-mon[32864]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-04-24T06:39:45.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:45 smithi037 ceph-mon[32864]: osdmap e67: 10 total, 9 up, 10 in 2024-04-24T06:39:45.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T06:39:45.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:45 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T06:39:45.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:45 smithi037 ceph-mon[32864]: from='osd.9 ' entity='osd.9' 2024-04-24T06:39:45.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:45 smithi137 ceph-mon[33017]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-04-24T06:39:45.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:45 smithi137 ceph-mon[33017]: osdmap e67: 10 total, 9 up, 10 in 2024-04-24T06:39:45.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:45 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T06:39:45.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:45 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T06:39:45.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:45 smithi137 ceph-mon[33017]: from='osd.9 ' entity='osd.9' 2024-04-24T06:39:46.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:45 smithi005 ceph-mon[27798]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-04-24T06:39:46.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:45 smithi005 ceph-mon[27798]: osdmap e67: 10 total, 9 up, 10 in 2024-04-24T06:39:46.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:45 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T06:39:46.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:45 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T06:39:46.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:45 smithi005 ceph-mon[27798]: from='osd.9 ' entity='osd.9' 2024-04-24T06:39:46.007 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:39:45 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:39:45.671+0000 7f711de00700 -1 osd.9 0 waiting for initial osdmap 2024-04-24T06:39:46.007 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:39:45 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:39:45.716+0000 7f7113c00700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T06:39:46.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:46 smithi005 ceph-mon[27798]: purged_snaps scrub starts 2024-04-24T06:39:46.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:46 smithi005 ceph-mon[27798]: purged_snaps scrub ok 2024-04-24T06:39:46.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:46 smithi005 ceph-mon[27798]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T06:39:46.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:46 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T06:39:47.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:46 smithi037 ceph-mon[32864]: purged_snaps scrub starts 2024-04-24T06:39:47.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:46 smithi037 ceph-mon[32864]: purged_snaps scrub ok 2024-04-24T06:39:47.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:46 smithi037 ceph-mon[32864]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T06:39:47.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:46 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T06:39:47.141 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-24T06:39:47.141 INFO:teuthology.orchestra.run.smithi037.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-24T06:39:47.141 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10+0 records in 2024-04-24T06:39:47.141 INFO:teuthology.orchestra.run.smithi037.stderr:10+0 records out 2024-04-24T06:39:47.141 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0131413 s, 798 MB/s 2024-04-24T06:39:47.141 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping successful for: 2024-04-24T06:39:47.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:46 smithi137 ceph-mon[33017]: purged_snaps scrub starts 2024-04-24T06:39:47.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:46 smithi137 ceph-mon[33017]: purged_snaps scrub ok 2024-04-24T06:39:47.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:46 smithi137 ceph-mon[33017]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T06:39:47.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:46 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T06:39:47.750 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi037:vg_nvme/lv_1 2024-04-24T06:39:47.999 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T06:39:48.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:47 smithi005 ceph-mon[27798]: osd.9 [v2:172.21.15.5:6826/2257102107,v1:172.21.15.5:6827/2257102107] boot 2024-04-24T06:39:48.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:47 smithi005 ceph-mon[27798]: osdmap e68: 10 total, 10 up, 10 in 2024-04-24T06:39:48.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:47 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T06:39:48.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:47 smithi037 ceph-mon[32864]: osd.9 [v2:172.21.15.5:6826/2257102107,v1:172.21.15.5:6827/2257102107] boot 2024-04-24T06:39:48.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:47 smithi037 ceph-mon[32864]: osdmap e68: 10 total, 10 up, 10 in 2024-04-24T06:39:48.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:47 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T06:39:48.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:47 smithi137 ceph-mon[33017]: osd.9 [v2:172.21.15.5:6826/2257102107,v1:172.21.15.5:6827/2257102107] boot 2024-04-24T06:39:48.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:47 smithi137 ceph-mon[33017]: osdmap e68: 10 total, 10 up, 10 in 2024-04-24T06:39:48.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:47 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T06:39:49.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:48 smithi005 ceph-mon[27798]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T06:39:49.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:48 smithi005 ceph-mon[27798]: osdmap e69: 10 total, 10 up, 10 in 2024-04-24T06:39:49.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:48 smithi037 ceph-mon[32864]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T06:39:49.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:48 smithi037 ceph-mon[32864]: osdmap e69: 10 total, 10 up, 10 in 2024-04-24T06:39:49.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:48 smithi137 ceph-mon[33017]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T06:39:49.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:48 smithi137 ceph-mon[33017]: osdmap e69: 10 total, 10 up, 10 in 2024-04-24T06:39:49.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:39:49.522+0000 7f494eb1b700 1 -- 172.21.15.37:0/112934399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4948102c70 msgr2=0x7f49481030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:49.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:39:49.522+0000 7f494eb1b700 1 --2- 172.21.15.37:0/112934399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4948102c70 0x7f49481030d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f49340099c0 tx=0x7f4934009cd0 comp rx=0 tx=0).stop 2024-04-24T06:39:49.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:39:49.522+0000 7f494eb1b700 1 -- 172.21.15.37:0/112934399 shutdown_connections 2024-04-24T06:39:49.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:39:49.522+0000 7f494eb1b700 1 --2- 172.21.15.37:0/112934399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4948103610 0x7f4948109ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:49.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:39:49.522+0000 7f494eb1b700 1 --2- 172.21.15.37:0/112934399 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4948101a70 0x7f4948101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:49.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:39:49.522+0000 7f494eb1b700 1 --2- 172.21.15.37:0/112934399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4948102c70 0x7f49481030d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:49.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:39:49.522+0000 7f494eb1b700 1 -- 172.21.15.37:0/112934399 >> 172.21.15.37:0/112934399 conn(0x7f49480fd010 msgr2=0x7f49480ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:39:49.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:39:49.522+0000 7f494eb1b700 1 -- 172.21.15.37:0/112934399 shutdown_connections 2024-04-24T06:39:49.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:39:49.522+0000 7f494eb1b700 1 -- 172.21.15.37:0/112934399 wait complete. 2024-04-24T06:39:49.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:39:49.522+0000 7f494eb1b700 1 Processor -- start 2024-04-24T06:39:49.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:39:49.522+0000 7f494eb1b700 1 -- start start 2024-04-24T06:39:49.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:39:49.523+0000 7f494eb1b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4948101a70 0x7f4948197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:49.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:39:49.523+0000 7f494eb1b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4948102c70 0x7f4948198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:49.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:39:49.523+0000 7f494eb1b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4948103610 0x7f494819f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:49.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:39:49.523+0000 7f494eb1b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f494810d700 con 0x7f4948101a70 2024-04-24T06:39:49.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:39:49.523+0000 7f494eb1b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f494810d580 con 0x7f4948103610 2024-04-24T06:39:49.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:39:49.523+0000 7f494eb1b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f494810d880 con 0x7f4948102c70 2024-04-24T06:39:49.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:39:49.523+0000 7f494d0b8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4948103610 0x7f494819f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:49.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:39:49.523+0000 7f494d0b8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4948103610 0x7f494819f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59164/0 (socket says 172.21.15.37:59164) 2024-04-24T06:39:49.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:39:49.523+0000 7f494d0b8700 1 -- 172.21.15.37:0/354916401 learned_addr learned my addr 172.21.15.37:0/354916401 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-24T06:39:49.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:39:49.523+0000 7f4947fff700 1 --2- 172.21.15.37:0/354916401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4948102c70 0x7f4948198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:49.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:39:49.523+0000 7f494d0b8700 1 -- 172.21.15.37:0/354916401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4948102c70 msgr2=0x7f4948198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:49.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:39:49.523+0000 7f494d0b8700 1 --2- 172.21.15.37:0/354916401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4948102c70 0x7f4948198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:49.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:39:49.523+0000 7f494d0b8700 1 -- 172.21.15.37:0/354916401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4948101a70 msgr2=0x7f4948197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:39:49.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:39:49.523+0000 7f494d0b8700 1 --2- 172.21.15.37:0/354916401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4948101a70 0x7f4948197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:39:49.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:39:49.523+0000 7f494d0b8700 1 -- 172.21.15.37:0/354916401 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f494819f990 con 0x7f4948103610 2024-04-24T06:39:49.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:39:49.523+0000 7f494d0b8700 1 --2- 172.21.15.37:0/354916401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4948103610 0x7f494819f2f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f494000db90 tx=0x7f494000dea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:49.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:39:49.523+0000 7f4945ffb700 1 -- 172.21.15.37:0/354916401 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494000e750 con 0x7f4948103610 2024-04-24T06:39:49.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:39:49.524+0000 7f4945ffb700 1 -- 172.21.15.37:0/354916401 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f494000edd0 con 0x7f4948103610 2024-04-24T06:39:49.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:39:49.524+0000 7f494eb1b700 1 -- 172.21.15.37:0/354916401 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f494806b220 con 0x7f4948103610 2024-04-24T06:39:49.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:39:49.524+0000 7f4945ffb700 1 -- 172.21.15.37:0/354916401 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49400163f0 con 0x7f4948103610 2024-04-24T06:39:49.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:39:49.524+0000 7f494eb1b700 1 -- 172.21.15.37:0/354916401 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f494806b7e0 con 0x7f4948103610 2024-04-24T06:39:49.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:39:49.525+0000 7f494eb1b700 1 -- 172.21.15.37:0/354916401 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49480697e0 con 0x7f4948103610 2024-04-24T06:39:49.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:39:49.526+0000 7f4945ffb700 1 -- 172.21.15.37:0/354916401 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f4940009580 con 0x7f4948103610 2024-04-24T06:39:49.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:39:49.526+0000 7f4945ffb700 1 --2- 172.21.15.37:0/354916401 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f49380a3cc0 0x7f49380a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:39:49.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:39:49.526+0000 7f4945ffb700 1 -- 172.21.15.37:0/354916401 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7f49400d1110 con 0x7f4948103610 2024-04-24T06:39:49.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:39:49.527+0000 7f494c8b7700 1 --2- 172.21.15.37:0/354916401 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f49380a3cc0 0x7f49380a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:39:49.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:39:49.528+0000 7f494c8b7700 1 --2- 172.21.15.37:0/354916401 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f49380a3cc0 0x7f49380a6180 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4948102ad0 tx=0x7f493c005ca0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:39:49.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:39:49.531+0000 7f4945ffb700 1 -- 172.21.15.37:0/354916401 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f494009e2f0 con 0x7f4948103610 2024-04-24T06:39:49.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:39:49.748+0000 7f494eb1b700 1 -- 172.21.15.37:0/354916401 --> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f494806ba40 con 0x7f49380a3cc0 2024-04-24T06:39:51.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:50 smithi005 ceph-mon[27798]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T06:39:51.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:50 smithi005 ceph-mon[27798]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:39:51.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:50 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T06:39:51.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:50 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T06:39:51.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:50 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:39:51.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:50 smithi005 ceph-mon[27798]: Detected new or changed devices on smithi005 2024-04-24T06:39:51.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:50 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:51.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:50 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:51.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:50 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:39:51.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:50 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:39:51.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:50 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:39:51.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:50 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:39:51.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:50 smithi005 ceph-mon[27798]: Adjusting osd_memory_target on smithi005 to 4273M 2024-04-24T06:39:51.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:50 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:51.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:50 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:39:51.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:50 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:39:51.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:50 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:51.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:50 smithi037 ceph-mon[32864]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T06:39:51.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:50 smithi037 ceph-mon[32864]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:39:51.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:50 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T06:39:51.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:50 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T06:39:51.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:50 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:39:51.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:50 smithi037 ceph-mon[32864]: Detected new or changed devices on smithi005 2024-04-24T06:39:51.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:50 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:51.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:50 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:51.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:50 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:39:51.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:50 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:39:51.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:50 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:39:51.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:50 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:39:51.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:50 smithi037 ceph-mon[32864]: Adjusting osd_memory_target on smithi005 to 4273M 2024-04-24T06:39:51.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:50 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:51.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:50 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:39:51.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:50 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:39:51.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:50 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:51.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:50 smithi137 ceph-mon[33017]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T06:39:51.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:50 smithi137 ceph-mon[33017]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:39:51.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:50 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T06:39:51.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:50 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T06:39:51.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:50 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:39:51.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:50 smithi137 ceph-mon[33017]: Detected new or changed devices on smithi005 2024-04-24T06:39:51.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:50 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:51.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:50 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:51.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:50 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:39:51.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:50 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:39:51.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:50 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:39:51.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:50 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:39:51.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:50 smithi137 ceph-mon[33017]: Adjusting osd_memory_target on smithi005 to 4273M 2024-04-24T06:39:51.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:50 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:51.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:50 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:39:51.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:50 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:39:51.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:50 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:39:53.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:52 smithi005 ceph-mon[27798]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T06:39:53.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:52 smithi005 ceph-mon[27798]: from='client.? 172.21.15.37:0/3055190232' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2455332-2a5e-44b1-85d3-8e47da399f9e"}]: dispatch 2024-04-24T06:39:53.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:52 smithi005 ceph-mon[27798]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2455332-2a5e-44b1-85d3-8e47da399f9e"}]: dispatch 2024-04-24T06:39:53.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:52 smithi005 ceph-mon[27798]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2455332-2a5e-44b1-85d3-8e47da399f9e"}]': finished 2024-04-24T06:39:53.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:52 smithi005 ceph-mon[27798]: osdmap e70: 11 total, 10 up, 11 in 2024-04-24T06:39:53.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:52 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T06:39:53.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:52 smithi037 ceph-mon[32864]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T06:39:53.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:52 smithi037 ceph-mon[32864]: from='client.? 172.21.15.37:0/3055190232' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2455332-2a5e-44b1-85d3-8e47da399f9e"}]: dispatch 2024-04-24T06:39:53.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:52 smithi037 ceph-mon[32864]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2455332-2a5e-44b1-85d3-8e47da399f9e"}]: dispatch 2024-04-24T06:39:53.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:52 smithi037 ceph-mon[32864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2455332-2a5e-44b1-85d3-8e47da399f9e"}]': finished 2024-04-24T06:39:53.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:52 smithi037 ceph-mon[32864]: osdmap e70: 11 total, 10 up, 11 in 2024-04-24T06:39:53.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:52 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T06:39:53.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:52 smithi137 ceph-mon[33017]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T06:39:53.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:52 smithi137 ceph-mon[33017]: from='client.? 172.21.15.37:0/3055190232' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2455332-2a5e-44b1-85d3-8e47da399f9e"}]: dispatch 2024-04-24T06:39:53.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:52 smithi137 ceph-mon[33017]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2455332-2a5e-44b1-85d3-8e47da399f9e"}]: dispatch 2024-04-24T06:39:53.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:52 smithi137 ceph-mon[33017]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2455332-2a5e-44b1-85d3-8e47da399f9e"}]': finished 2024-04-24T06:39:53.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:52 smithi137 ceph-mon[33017]: osdmap e70: 11 total, 10 up, 11 in 2024-04-24T06:39:53.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:52 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T06:39:54.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:53 smithi005 ceph-mon[27798]: from='client.? 172.21.15.37:0/775881145' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T06:39:54.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:53 smithi037 ceph-mon[32864]: from='client.? 172.21.15.37:0/775881145' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T06:39:54.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:53 smithi137 ceph-mon[33017]: from='client.? 172.21.15.37:0/775881145' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T06:39:55.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:54 smithi005 ceph-mon[27798]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T06:39:55.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:54 smithi037 ceph-mon[32864]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T06:39:55.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:54 smithi137 ceph-mon[33017]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T06:39:57.009 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:56 smithi005 ceph-mon[27798]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T06:39:57.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:56 smithi037 ceph-mon[32864]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T06:39:57.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:56 smithi137 ceph-mon[33017]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T06:39:58.758 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:39:58 smithi037 ceph-mon[32864]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T06:39:58.998 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:39:58 smithi137 ceph-mon[33017]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T06:39:59.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:39:58 smithi005 ceph-mon[27798]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T06:40:01.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:00 smithi005 ceph-mon[27798]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T06:40:01.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:00 smithi005 ceph-mon[27798]: overall HEALTH_OK 2024-04-24T06:40:01.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:00 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-24T06:40:01.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:00 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:40:01.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:00 smithi037 ceph-mon[32864]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T06:40:01.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:00 smithi037 ceph-mon[32864]: overall HEALTH_OK 2024-04-24T06:40:01.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:00 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-24T06:40:01.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:00 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:40:01.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:00 smithi137 ceph-mon[33017]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T06:40:01.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:00 smithi137 ceph-mon[33017]: overall HEALTH_OK 2024-04-24T06:40:01.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:00 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-24T06:40:01.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:00 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:40:02.003 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:01 smithi037 ceph-mon[32864]: Deploying daemon osd.10 on smithi037 2024-04-24T06:40:02.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:01 smithi137 ceph-mon[33017]: Deploying daemon osd.10 on smithi037 2024-04-24T06:40:02.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:01 smithi005 ceph-mon[27798]: Deploying daemon osd.10 on smithi037 2024-04-24T06:40:03.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:02 smithi037 ceph-mon[32864]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T06:40:03.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:02 smithi137 ceph-mon[33017]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T06:40:03.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:02 smithi005 ceph-mon[27798]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T06:40:05.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:04 smithi037 ceph-mon[32864]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T06:40:05.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:04 smithi137 ceph-mon[33017]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T06:40:05.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:04 smithi005 ceph-mon[27798]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T06:40:07.126 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:06 smithi037 ceph-mon[32864]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T06:40:07.126 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:06 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:07.126 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:06 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:07.126 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:06 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:40:07.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:06 smithi137 ceph-mon[33017]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T06:40:07.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:06 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:07.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:06 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:07.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:06 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:40:07.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:06 smithi005 ceph-mon[27798]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T06:40:07.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:06 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:07.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:06 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:07.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:06 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:40:09.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:08 smithi137 ceph-mon[33017]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T06:40:09.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:08 smithi005 ceph-mon[27798]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T06:40:09.259 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:08 smithi037 ceph-mon[32864]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T06:40:09.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:40:09.648+0000 7f4945ffb700 1 -- 172.21.15.37:0/354916401 <== mgr.14152 v2:172.21.15.5:6800/4042936965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f494806ba40 con 0x7f49380a3cc0 2024-04-24T06:40:09.650 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 10 on host 'smithi037' 2024-04-24T06:40:09.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:40:09.650+0000 7f494eb1b700 1 -- 172.21.15.37:0/354916401 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f49380a3cc0 msgr2=0x7f49380a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:09.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:40:09.650+0000 7f494eb1b700 1 --2- 172.21.15.37:0/354916401 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f49380a3cc0 0x7f49380a6180 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4948102ad0 tx=0x7f493c005ca0 comp rx=0 tx=0).stop 2024-04-24T06:40:09.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:40:09.651+0000 7f494eb1b700 1 -- 172.21.15.37:0/354916401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4948103610 msgr2=0x7f494819f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:09.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:40:09.651+0000 7f494eb1b700 1 --2- 172.21.15.37:0/354916401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4948103610 0x7f494819f2f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f494000db90 tx=0x7f494000dea0 comp rx=0 tx=0).stop 2024-04-24T06:40:09.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:40:09.651+0000 7f494eb1b700 1 -- 172.21.15.37:0/354916401 shutdown_connections 2024-04-24T06:40:09.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:40:09.651+0000 7f494eb1b700 1 --2- 172.21.15.37:0/354916401 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f49380a3cc0 0x7f49380a6180 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:09.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:40:09.651+0000 7f494eb1b700 1 --2- 172.21.15.37:0/354916401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4948103610 0x7f494819f2f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:09.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:40:09.651+0000 7f494eb1b700 1 --2- 172.21.15.37:0/354916401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4948101a70 0x7f4948197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:09.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:40:09.651+0000 7f494eb1b700 1 --2- 172.21.15.37:0/354916401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4948102c70 0x7f4948198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:09.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:40:09.651+0000 7f494eb1b700 1 -- 172.21.15.37:0/354916401 >> 172.21.15.37:0/354916401 conn(0x7f49480fd010 msgr2=0x7f4948105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:09.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:40:09.651+0000 7f494eb1b700 1 -- 172.21.15.37:0/354916401 shutdown_connections 2024-04-24T06:40:09.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-24T06:40:09.651+0000 7f494eb1b700 1 -- 172.21.15.37:0/354916401 wait complete. 2024-04-24T06:40:10.005 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:09 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:10.005 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:09 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:10.005 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:09 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:40:10.005 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:09 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:40:10.005 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:09 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:10.005 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:09 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:10.005 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:09 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:40:10.005 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:09 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:10.005 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:09 smithi037 ceph-mon[32864]: from='osd.10 [v2:172.21.15.37:6824/2451570637,v1:172.21.15.37:6825/2451570637]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T06:40:10.005 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:09 smithi037 ceph-mon[32864]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T06:40:10.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:09 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:10.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:09 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:10.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:09 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:40:10.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:09 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:40:10.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:09 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:10.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:09 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:10.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:09 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:40:10.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:09 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:10.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:09 smithi137 ceph-mon[33017]: from='osd.10 [v2:172.21.15.37:6824/2451570637,v1:172.21.15.37:6825/2451570637]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T06:40:10.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:09 smithi137 ceph-mon[33017]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T06:40:10.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:09 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:10.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:09 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:10.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:09 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:40:10.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:09 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:40:10.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:09 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:10.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:09 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:10.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:09 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:40:10.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:09 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:10.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:09 smithi005 ceph-mon[27798]: from='osd.10 [v2:172.21.15.37:6824/2451570637,v1:172.21.15.37:6825/2451570637]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T06:40:10.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:09 smithi005 ceph-mon[27798]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T06:40:10.290 DEBUG:teuthology.orchestra.run.smithi037:osd.10> sudo journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@osd.10.service 2024-04-24T06:40:10.293 INFO:tasks.cephadm:Deploying osd.11 on smithi137 with /dev/vg_nvme/lv_1... 2024-04-24T06:40:10.293 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-24T06:40:10.542 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T06:40:10.925 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:10 smithi137 ceph-mon[33017]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T06:40:10.925 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:10 smithi137 ceph-mon[33017]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-24T06:40:10.925 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:10 smithi137 ceph-mon[33017]: from='osd.10 [v2:172.21.15.37:6824/2451570637,v1:172.21.15.37:6825/2451570637]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-24T06:40:10.926 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:10 smithi137 ceph-mon[33017]: osdmap e71: 11 total, 10 up, 11 in 2024-04-24T06:40:10.926 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:10 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T06:40:10.926 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:10 smithi137 ceph-mon[33017]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-24T06:40:11.120 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:10 smithi037 ceph-mon[32864]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T06:40:11.120 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:10 smithi037 ceph-mon[32864]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-24T06:40:11.120 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:10 smithi037 ceph-mon[32864]: from='osd.10 [v2:172.21.15.37:6824/2451570637,v1:172.21.15.37:6825/2451570637]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-24T06:40:11.120 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:10 smithi037 ceph-mon[32864]: osdmap e71: 11 total, 10 up, 11 in 2024-04-24T06:40:11.120 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:10 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T06:40:11.120 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:10 smithi037 ceph-mon[32864]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-24T06:40:11.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:10 smithi005 ceph-mon[27798]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T06:40:11.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:10 smithi005 ceph-mon[27798]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-24T06:40:11.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:10 smithi005 ceph-mon[27798]: from='osd.10 [v2:172.21.15.37:6824/2451570637,v1:172.21.15.37:6825/2451570637]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-24T06:40:11.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:10 smithi005 ceph-mon[27798]: osdmap e71: 11 total, 10 up, 11 in 2024-04-24T06:40:11.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:10 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T06:40:11.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:10 smithi005 ceph-mon[27798]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-24T06:40:12.019 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:40:11 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:40:11.646+0000 7fdbfe70c700 -1 osd.10 0 waiting for initial osdmap 2024-04-24T06:40:12.019 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:40:11 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:40:11.660+0000 7fdbfa4ff700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T06:40:12.524 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-24T06:40:12.524 INFO:teuthology.orchestra.run.smithi137.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-24T06:40:12.524 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10+0 records in 2024-04-24T06:40:12.524 INFO:teuthology.orchestra.run.smithi137.stderr:10+0 records out 2024-04-24T06:40:12.524 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113826 s, 921 MB/s 2024-04-24T06:40:12.524 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping successful for: 2024-04-24T06:40:12.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:12 smithi037 ceph-mon[32864]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T06:40:12.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:12 smithi037 ceph-mon[32864]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-24T06:40:12.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:12 smithi037 ceph-mon[32864]: osdmap e72: 11 total, 10 up, 11 in 2024-04-24T06:40:12.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:12 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T06:40:12.843 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:12 smithi137 ceph-mon[33017]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T06:40:12.844 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:12 smithi137 ceph-mon[33017]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-24T06:40:12.844 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:12 smithi137 ceph-mon[33017]: osdmap e72: 11 total, 10 up, 11 in 2024-04-24T06:40:12.844 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:12 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T06:40:13.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:12 smithi005 ceph-mon[27798]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T06:40:13.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:12 smithi005 ceph-mon[27798]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-24T06:40:13.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:12 smithi005 ceph-mon[27798]: osdmap e72: 11 total, 10 up, 11 in 2024-04-24T06:40:13.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:12 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T06:40:13.163 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi137:vg_nvme/lv_1 2024-04-24T06:40:13.416 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T06:40:13.702 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:13 smithi137 ceph-mon[33017]: purged_snaps scrub starts 2024-04-24T06:40:13.702 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:13 smithi137 ceph-mon[33017]: purged_snaps scrub ok 2024-04-24T06:40:13.702 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:13 smithi137 ceph-mon[33017]: osd.10 [v2:172.21.15.37:6824/2451570637,v1:172.21.15.37:6825/2451570637] boot 2024-04-24T06:40:13.703 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:13 smithi137 ceph-mon[33017]: osdmap e73: 11 total, 11 up, 11 in 2024-04-24T06:40:13.703 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:13 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T06:40:13.703 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:13 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:13.703 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:13 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:13.703 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:13 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:40:13.703 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:13 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:40:13.703 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:13 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:40:13.703 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:13 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:40:13.703 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:13 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:13.703 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:13 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:40:13.703 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:13 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:40:13.703 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:13 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:14.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:13 smithi005 ceph-mon[27798]: purged_snaps scrub starts 2024-04-24T06:40:14.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:13 smithi005 ceph-mon[27798]: purged_snaps scrub ok 2024-04-24T06:40:14.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:13 smithi005 ceph-mon[27798]: osd.10 [v2:172.21.15.37:6824/2451570637,v1:172.21.15.37:6825/2451570637] boot 2024-04-24T06:40:14.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:13 smithi005 ceph-mon[27798]: osdmap e73: 11 total, 11 up, 11 in 2024-04-24T06:40:14.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:13 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T06:40:14.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:13 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:14.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:13 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:14.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:13 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:40:14.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:13 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:40:14.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:13 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:40:14.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:13 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:40:14.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:13 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:14.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:13 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:40:14.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:13 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:40:14.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:13 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:14.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:13 smithi037 ceph-mon[32864]: purged_snaps scrub starts 2024-04-24T06:40:14.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:13 smithi037 ceph-mon[32864]: purged_snaps scrub ok 2024-04-24T06:40:14.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:13 smithi037 ceph-mon[32864]: osd.10 [v2:172.21.15.37:6824/2451570637,v1:172.21.15.37:6825/2451570637] boot 2024-04-24T06:40:14.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:13 smithi037 ceph-mon[32864]: osdmap e73: 11 total, 11 up, 11 in 2024-04-24T06:40:14.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:13 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T06:40:14.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:13 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:14.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:13 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:14.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:13 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:40:14.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:13 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:40:14.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:13 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:40:14.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:13 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:40:14.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:13 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:14.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:13 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:40:14.020 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:13 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:40:14.020 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:13 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:14.937 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:14 smithi137 ceph-mon[33017]: Detected new or changed devices on smithi037 2024-04-24T06:40:14.937 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:14 smithi137 ceph-mon[33017]: Adjusting osd_memory_target on smithi037 to 4273M 2024-04-24T06:40:14.937 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:14 smithi137 ceph-mon[33017]: pgmap v259: 1 pgs: 1 remapped+peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-24T06:40:14.937 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:14 smithi137 ceph-mon[33017]: osdmap e74: 11 total, 11 up, 11 in 2024-04-24T06:40:14.937 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:14 smithi137 ceph-mon[33017]: osdmap e75: 11 total, 11 up, 11 in 2024-04-24T06:40:15.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:14 smithi005 ceph-mon[27798]: Detected new or changed devices on smithi037 2024-04-24T06:40:15.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:14 smithi005 ceph-mon[27798]: Adjusting osd_memory_target on smithi037 to 4273M 2024-04-24T06:40:15.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:14 smithi005 ceph-mon[27798]: pgmap v259: 1 pgs: 1 remapped+peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-24T06:40:15.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:14 smithi005 ceph-mon[27798]: osdmap e74: 11 total, 11 up, 11 in 2024-04-24T06:40:15.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:14 smithi005 ceph-mon[27798]: osdmap e75: 11 total, 11 up, 11 in 2024-04-24T06:40:15.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:14 smithi037 ceph-mon[32864]: Detected new or changed devices on smithi037 2024-04-24T06:40:15.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:14 smithi037 ceph-mon[32864]: Adjusting osd_memory_target on smithi037 to 4273M 2024-04-24T06:40:15.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:14 smithi037 ceph-mon[32864]: pgmap v259: 1 pgs: 1 remapped+peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-24T06:40:15.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:14 smithi037 ceph-mon[32864]: osdmap e74: 11 total, 11 up, 11 in 2024-04-24T06:40:15.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:14 smithi037 ceph-mon[32864]: osdmap e75: 11 total, 11 up, 11 in 2024-04-24T06:40:15.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:15.038+0000 7f69ffac8700 1 -- 172.21.15.137:0/3704941023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69f8102c60 msgr2=0x7f69f81030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:15.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:15.038+0000 7f69ffac8700 1 --2- 172.21.15.137:0/3704941023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69f8102c60 0x7f69f81030c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f69f4009990 tx=0x7f69f4009ca0 comp rx=0 tx=0).stop 2024-04-24T06:40:15.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:15.038+0000 7f69ffac8700 1 -- 172.21.15.137:0/3704941023 shutdown_connections 2024-04-24T06:40:15.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:15.038+0000 7f69ffac8700 1 --2- 172.21.15.137:0/3704941023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69f8103600 0x7f69f8109eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:15.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:15.038+0000 7f69ffac8700 1 --2- 172.21.15.137:0/3704941023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69f8101a60 0x7f69f8101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:15.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:15.038+0000 7f69ffac8700 1 --2- 172.21.15.137:0/3704941023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69f8102c60 0x7f69f81030c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:15.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:15.038+0000 7f69ffac8700 1 -- 172.21.15.137:0/3704941023 >> 172.21.15.137:0/3704941023 conn(0x7f69f80fd020 msgr2=0x7f69f80ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:15.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:15.038+0000 7f69ffac8700 1 -- 172.21.15.137:0/3704941023 shutdown_connections 2024-04-24T06:40:15.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:15.038+0000 7f69ffac8700 1 -- 172.21.15.137:0/3704941023 wait complete. 2024-04-24T06:40:15.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:15.039+0000 7f69ffac8700 1 Processor -- start 2024-04-24T06:40:15.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:15.039+0000 7f69ffac8700 1 -- start start 2024-04-24T06:40:15.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:15.039+0000 7f69ffac8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69f8101a60 0x7f69f8105b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:15.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:15.039+0000 7f69ffac8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69f8102c60 0x7f69f8104190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:15.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:15.039+0000 7f69ffac8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69f8103600 0x7f69f81046d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:15.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:15.039+0000 7f69ffac8700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f69f810d710 con 0x7f69f8102c60 2024-04-24T06:40:15.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:15.039+0000 7f69ffac8700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f69f810d590 con 0x7f69f8101a60 2024-04-24T06:40:15.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:15.039+0000 7f69ffac8700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f69f810d890 con 0x7f69f8103600 2024-04-24T06:40:15.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:15.039+0000 7f69fe065700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69f8103600 0x7f69f81046d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:15.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:15.039+0000 7f69fe065700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69f8103600 0x7f69f81046d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33822/0 (socket says 172.21.15.137:33822) 2024-04-24T06:40:15.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:15.039+0000 7f69fe065700 1 -- 172.21.15.137:0/661050996 learned_addr learned my addr 172.21.15.137:0/661050996 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-04-24T06:40:15.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:15.039+0000 7f69fd864700 1 --2- 172.21.15.137:0/661050996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69f8101a60 0x7f69f8105b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:15.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:15.039+0000 7f69fe065700 1 -- 172.21.15.137:0/661050996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69f8101a60 msgr2=0x7f69f8105b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:15.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:15.039+0000 7f69fe065700 1 --2- 172.21.15.137:0/661050996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69f8101a60 0x7f69f8105b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:15.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:15.039+0000 7f69fe065700 1 -- 172.21.15.137:0/661050996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69f8102c60 msgr2=0x7f69f8104190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:15.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:15.039+0000 7f69fe065700 1 --2- 172.21.15.137:0/661050996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69f8102c60 0x7f69f8104190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:15.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:15.039+0000 7f69fe065700 1 -- 172.21.15.137:0/661050996 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69f8104f80 con 0x7f69f8103600 2024-04-24T06:40:15.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:15.040+0000 7f69fe065700 1 --2- 172.21.15.137:0/661050996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69f8103600 0x7f69f81046d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f69f000b970 tx=0x7f69f000bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:15.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:15.040+0000 7f69eeffd700 1 -- 172.21.15.137:0/661050996 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f0019070 con 0x7f69f8103600 2024-04-24T06:40:15.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:15.040+0000 7f69eeffd700 1 -- 172.21.15.137:0/661050996 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69f000ce10 con 0x7f69f8103600 2024-04-24T06:40:15.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:15.040+0000 7f69ffac8700 1 -- 172.21.15.137:0/661050996 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69f81a2fa0 con 0x7f69f8103600 2024-04-24T06:40:15.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:15.040+0000 7f69eeffd700 1 -- 172.21.15.137:0/661050996 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f0007370 con 0x7f69f8103600 2024-04-24T06:40:15.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:15.040+0000 7f69ffac8700 1 -- 172.21.15.137:0/661050996 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69f81a34e0 con 0x7f69f8103600 2024-04-24T06:40:15.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:15.041+0000 7f69ffac8700 1 -- 172.21.15.137:0/661050996 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69f80697e0 con 0x7f69f8103600 2024-04-24T06:40:15.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:15.042+0000 7f69eeffd700 1 -- 172.21.15.137:0/661050996 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f69f001c070 con 0x7f69f8103600 2024-04-24T06:40:15.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:15.042+0000 7f69eeffd700 1 --2- 172.21.15.137:0/661050996 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f69e40a3c40 0x7f69e40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:15.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:15.043+0000 7f69eeffd700 1 -- 172.21.15.137:0/661050996 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6723+0+0 (secure 0 0 0) 0x7f69f00cf150 con 0x7f69f8103600 2024-04-24T06:40:15.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:15.043+0000 7f69fd864700 1 --2- 172.21.15.137:0/661050996 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f69e40a3c40 0x7f69e40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:15.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:15.044+0000 7f69fd864700 1 --2- 172.21.15.137:0/661050996 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f69e40a3c40 0x7f69e40a6100 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f69f8102ac0 tx=0x7f69e800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:15.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:15.047+0000 7f69eeffd700 1 -- 172.21.15.137:0/661050996 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f69f009c160 con 0x7f69f8103600 2024-04-24T06:40:15.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:15.254+0000 7f69ffac8700 1 -- 172.21.15.137:0/661050996 --> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f69f806b950 con 0x7f69e40a3c40 2024-04-24T06:40:15.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:15 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T06:40:15.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:15 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T06:40:15.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:15 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:40:16.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:15 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T06:40:16.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:15 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T06:40:16.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:15 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:40:16.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:15 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T06:40:16.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:15 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T06:40:16.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:15 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:40:16.942 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:16 smithi005 ceph-mon[27798]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:40:16.942 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:16 smithi005 ceph-mon[27798]: pgmap v262: 1 pgs: 1 remapped+peering; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-24T06:40:16.942 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:16 smithi005 ceph-mon[27798]: osdmap e76: 11 total, 11 up, 11 in 2024-04-24T06:40:16.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:16 smithi137 ceph-mon[33017]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:40:16.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:16 smithi137 ceph-mon[33017]: pgmap v262: 1 pgs: 1 remapped+peering; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-24T06:40:16.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:16 smithi137 ceph-mon[33017]: osdmap e76: 11 total, 11 up, 11 in 2024-04-24T06:40:17.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:16 smithi037 ceph-mon[32864]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:40:17.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:16 smithi037 ceph-mon[32864]: pgmap v262: 1 pgs: 1 remapped+peering; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-24T06:40:17.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:16 smithi037 ceph-mon[32864]: osdmap e76: 11 total, 11 up, 11 in 2024-04-24T06:40:17.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:17 smithi137 ceph-mon[33017]: osdmap e77: 11 total, 11 up, 11 in 2024-04-24T06:40:18.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:17 smithi005 ceph-mon[27798]: osdmap e77: 11 total, 11 up, 11 in 2024-04-24T06:40:18.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:17 smithi037 ceph-mon[32864]: osdmap e77: 11 total, 11 up, 11 in 2024-04-24T06:40:19.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:18 smithi005 ceph-mon[27798]: pgmap v265: 1 pgs: 1 remapped+peering; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-24T06:40:19.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:18 smithi005 ceph-mon[27798]: from='client.? 172.21.15.137:0/1855883874' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49d9396b-704a-4cef-8210-75d30c28e924"}]: dispatch 2024-04-24T06:40:19.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:18 smithi005 ceph-mon[27798]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49d9396b-704a-4cef-8210-75d30c28e924"}]: dispatch 2024-04-24T06:40:19.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:18 smithi005 ceph-mon[27798]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49d9396b-704a-4cef-8210-75d30c28e924"}]': finished 2024-04-24T06:40:19.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:18 smithi005 ceph-mon[27798]: osdmap e78: 12 total, 11 up, 12 in 2024-04-24T06:40:19.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:18 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T06:40:19.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:18 smithi005 ceph-mon[27798]: from='client.? 172.21.15.137:0/259701127' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T06:40:19.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:18 smithi037 ceph-mon[32864]: pgmap v265: 1 pgs: 1 remapped+peering; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-24T06:40:19.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:18 smithi037 ceph-mon[32864]: from='client.? 172.21.15.137:0/1855883874' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49d9396b-704a-4cef-8210-75d30c28e924"}]: dispatch 2024-04-24T06:40:19.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:18 smithi037 ceph-mon[32864]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49d9396b-704a-4cef-8210-75d30c28e924"}]: dispatch 2024-04-24T06:40:19.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:18 smithi037 ceph-mon[32864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49d9396b-704a-4cef-8210-75d30c28e924"}]': finished 2024-04-24T06:40:19.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:18 smithi037 ceph-mon[32864]: osdmap e78: 12 total, 11 up, 12 in 2024-04-24T06:40:19.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:18 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T06:40:19.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:18 smithi037 ceph-mon[32864]: from='client.? 172.21.15.137:0/259701127' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T06:40:19.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:18 smithi137 ceph-mon[33017]: pgmap v265: 1 pgs: 1 remapped+peering; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-24T06:40:19.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:18 smithi137 ceph-mon[33017]: from='client.? 172.21.15.137:0/1855883874' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49d9396b-704a-4cef-8210-75d30c28e924"}]: dispatch 2024-04-24T06:40:19.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:18 smithi137 ceph-mon[33017]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49d9396b-704a-4cef-8210-75d30c28e924"}]: dispatch 2024-04-24T06:40:19.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:18 smithi137 ceph-mon[33017]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49d9396b-704a-4cef-8210-75d30c28e924"}]': finished 2024-04-24T06:40:19.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:18 smithi137 ceph-mon[33017]: osdmap e78: 12 total, 11 up, 12 in 2024-04-24T06:40:19.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:18 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T06:40:19.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:18 smithi137 ceph-mon[33017]: from='client.? 172.21.15.137:0/259701127' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T06:40:21.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:20 smithi005 ceph-mon[27798]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-04-24T06:40:21.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:20 smithi037 ceph-mon[32864]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-04-24T06:40:21.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:20 smithi137 ceph-mon[33017]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-04-24T06:40:22.997 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:22 smithi137 ceph-mon[33017]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T06:40:23.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:22 smithi005 ceph-mon[27798]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T06:40:23.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:22 smithi037 ceph-mon[32864]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T06:40:24.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:24 smithi137 ceph-mon[33017]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-24T06:40:25.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:24 smithi005 ceph-mon[27798]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-24T06:40:25.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:24 smithi037 ceph-mon[32864]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-24T06:40:26.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:25 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-24T06:40:26.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:25 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:40:26.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:25 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-24T06:40:26.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:25 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:40:26.033 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:25 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-24T06:40:26.034 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:25 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:40:26.942 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:26 smithi137 ceph-mon[33017]: Deploying daemon osd.11 on smithi137 2024-04-24T06:40:26.942 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:26 smithi137 ceph-mon[33017]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-24T06:40:27.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:26 smithi005 ceph-mon[27798]: Deploying daemon osd.11 on smithi137 2024-04-24T06:40:27.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:26 smithi005 ceph-mon[27798]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-24T06:40:27.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:26 smithi037 ceph-mon[32864]: Deploying daemon osd.11 on smithi137 2024-04-24T06:40:27.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:26 smithi037 ceph-mon[32864]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-24T06:40:28.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:28 smithi137 ceph-mon[33017]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T06:40:29.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:28 smithi005 ceph-mon[27798]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T06:40:29.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:28 smithi037 ceph-mon[32864]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T06:40:30.884 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:30 smithi137 ceph-mon[33017]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-04-24T06:40:30.884 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:30 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:30.885 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:30 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:40:30.885 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:30 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:31.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:30 smithi005 ceph-mon[27798]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-04-24T06:40:31.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:30 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:31.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:30 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:40:31.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:30 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:31.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:30 smithi037 ceph-mon[32864]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-04-24T06:40:31.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:30 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:31.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:30 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:40:31.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:30 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:33.135 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:32 smithi137 ceph-mon[33017]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-24T06:40:33.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:32 smithi005 ceph-mon[27798]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-24T06:40:33.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:32 smithi037 ceph-mon[32864]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-24T06:40:33.792 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 11 on host 'smithi137' 2024-04-24T06:40:33.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:33.790+0000 7f69eeffd700 1 -- 172.21.15.137:0/661050996 <== mgr.14152 v2:172.21.15.5:6800/4042936965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f69f806b950 con 0x7f69e40a3c40 2024-04-24T06:40:33.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:33.793+0000 7f69ffac8700 1 -- 172.21.15.137:0/661050996 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f69e40a3c40 msgr2=0x7f69e40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:33.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:33.793+0000 7f69ffac8700 1 --2- 172.21.15.137:0/661050996 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f69e40a3c40 0x7f69e40a6100 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f69f8102ac0 tx=0x7f69e800b410 comp rx=0 tx=0).stop 2024-04-24T06:40:33.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:33.793+0000 7f69ffac8700 1 -- 172.21.15.137:0/661050996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69f8103600 msgr2=0x7f69f81046d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:33.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:33.793+0000 7f69ffac8700 1 --2- 172.21.15.137:0/661050996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69f8103600 0x7f69f81046d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f69f000b970 tx=0x7f69f000bc80 comp rx=0 tx=0).stop 2024-04-24T06:40:33.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:33.793+0000 7f69ffac8700 1 -- 172.21.15.137:0/661050996 shutdown_connections 2024-04-24T06:40:33.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:33.793+0000 7f69ffac8700 1 --2- 172.21.15.137:0/661050996 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f69e40a3c40 0x7f69e40a6100 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:33.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:33.793+0000 7f69ffac8700 1 --2- 172.21.15.137:0/661050996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69f8103600 0x7f69f81046d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:33.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:33.793+0000 7f69ffac8700 1 --2- 172.21.15.137:0/661050996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69f8101a60 0x7f69f8105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:33.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:33.793+0000 7f69ffac8700 1 --2- 172.21.15.137:0/661050996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69f8102c60 0x7f69f8104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:33.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:33.793+0000 7f69ffac8700 1 -- 172.21.15.137:0/661050996 >> 172.21.15.137:0/661050996 conn(0x7f69f80fd020 msgr2=0x7f69f806b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:33.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:33.793+0000 7f69ffac8700 1 -- 172.21.15.137:0/661050996 shutdown_connections 2024-04-24T06:40:33.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:33.793+0000 7f69ffac8700 1 -- 172.21.15.137:0/661050996 wait complete. 2024-04-24T06:40:34.401 DEBUG:teuthology.orchestra.run.smithi137:osd.11> sudo journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@osd.11.service 2024-04-24T06:40:34.403 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-24T06:40:34.404 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-24T06:40:34.881 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:40:35.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:34 smithi005 ceph-mon[27798]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-24T06:40:35.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:34 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:35.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:34 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:35.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:34 smithi005 ceph-mon[27798]: from='osd.11 [v2:172.21.15.137:6824/90600596,v1:172.21.15.137:6825/90600596]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T06:40:35.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:34 smithi005 ceph-mon[27798]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T06:40:35.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:34 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:35.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:34 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:35.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:34 smithi037 ceph-mon[32864]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-24T06:40:35.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:34 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:35.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:34 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:35.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:34 smithi037 ceph-mon[32864]: from='osd.11 [v2:172.21.15.137:6824/90600596,v1:172.21.15.137:6825/90600596]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T06:40:35.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:34 smithi037 ceph-mon[32864]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T06:40:35.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:34 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:35.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:34 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:35.106 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:34 smithi137 ceph-mon[33017]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-24T06:40:35.106 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:34 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:35.106 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:34 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:35.106 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:34 smithi137 ceph-mon[33017]: from='osd.11 [v2:172.21.15.137:6824/90600596,v1:172.21.15.137:6825/90600596]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T06:40:35.107 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:34 smithi137 ceph-mon[33017]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T06:40:35.107 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:34 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:35.107 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:34 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:36.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:35 smithi005 ceph-mon[27798]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-24T06:40:36.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:35 smithi005 ceph-mon[27798]: osdmap e79: 12 total, 11 up, 12 in 2024-04-24T06:40:36.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:35 smithi005 ceph-mon[27798]: from='osd.11 [v2:172.21.15.137:6824/90600596,v1:172.21.15.137:6825/90600596]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-24T06:40:36.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:35 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T06:40:36.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:35 smithi005 ceph-mon[27798]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-24T06:40:36.095 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:35 smithi137 ceph-mon[33017]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-24T06:40:36.095 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:35 smithi137 ceph-mon[33017]: osdmap e79: 12 total, 11 up, 12 in 2024-04-24T06:40:36.095 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:35 smithi137 ceph-mon[33017]: from='osd.11 [v2:172.21.15.137:6824/90600596,v1:172.21.15.137:6825/90600596]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-24T06:40:36.096 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:35 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T06:40:36.096 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:35 smithi137 ceph-mon[33017]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-24T06:40:36.096 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:40:35 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:40:35.787+0000 7f3478eb4700 -1 osd.11 0 waiting for initial osdmap 2024-04-24T06:40:36.096 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:40:35 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:40:35.806+0000 7f3474ca7700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T06:40:36.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:35 smithi037 ceph-mon[32864]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-24T06:40:36.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:35 smithi037 ceph-mon[32864]: osdmap e79: 12 total, 11 up, 12 in 2024-04-24T06:40:36.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:35 smithi037 ceph-mon[32864]: from='osd.11 [v2:172.21.15.137:6824/90600596,v1:172.21.15.137:6825/90600596]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-24T06:40:36.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:35 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T06:40:36.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:35 smithi037 ceph-mon[32864]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-24T06:40:37.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:36 smithi005 ceph-mon[27798]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-24T06:40:37.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:36 smithi005 ceph-mon[27798]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-04-24T06:40:37.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:36 smithi005 ceph-mon[27798]: osdmap e80: 12 total, 11 up, 12 in 2024-04-24T06:40:37.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:36 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T06:40:37.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:36 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T06:40:37.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:36 smithi137 ceph-mon[33017]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-24T06:40:37.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:36 smithi137 ceph-mon[33017]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-04-24T06:40:37.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:36 smithi137 ceph-mon[33017]: osdmap e80: 12 total, 11 up, 12 in 2024-04-24T06:40:37.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T06:40:37.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T06:40:37.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:36 smithi037 ceph-mon[32864]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-24T06:40:37.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:36 smithi037 ceph-mon[32864]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-04-24T06:40:37.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:36 smithi037 ceph-mon[32864]: osdmap e80: 12 total, 11 up, 12 in 2024-04-24T06:40:37.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:36 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T06:40:37.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:36 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T06:40:37.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.284+0000 7fc909000700 1 -- 172.21.15.5:0/1932217733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc90410a390 msgr2=0x7fc90410a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:37.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.284+0000 7fc909000700 1 --2- 172.21.15.5:0/1932217733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc90410a390 0x7fc90410a7f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc8f00099c0 tx=0x7fc8f0009cd0 comp rx=0 tx=0).stop 2024-04-24T06:40:37.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.284+0000 7fc909000700 1 -- 172.21.15.5:0/1932217733 shutdown_connections 2024-04-24T06:40:37.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.284+0000 7fc909000700 1 --2- 172.21.15.5:0/1932217733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc90410ad30 0x7fc9041115e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:37.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.284+0000 7fc909000700 1 --2- 172.21.15.5:0/1932217733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc904109190 0x7fc9041095b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:37.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.284+0000 7fc909000700 1 --2- 172.21.15.5:0/1932217733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc90410a390 0x7fc90410a7f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:37.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.284+0000 7fc909000700 1 -- 172.21.15.5:0/1932217733 >> 172.21.15.5:0/1932217733 conn(0x7fc904078160 msgr2=0x7fc90407a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:37.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.284+0000 7fc909000700 1 -- 172.21.15.5:0/1932217733 shutdown_connections 2024-04-24T06:40:37.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.284+0000 7fc909000700 1 -- 172.21.15.5:0/1932217733 wait complete. 2024-04-24T06:40:37.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.285+0000 7fc909000700 1 Processor -- start 2024-04-24T06:40:37.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.286+0000 7fc909000700 1 -- start start 2024-04-24T06:40:37.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.286+0000 7fc909000700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc904109190 0x7fc90419f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:37.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.286+0000 7fc909000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc90410a390 0x7fc90419f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:37.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.286+0000 7fc909000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc90410ad30 0x7fc9041a6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:37.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.286+0000 7fc909000700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc904114d50 con 0x7fc904109190 2024-04-24T06:40:37.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.286+0000 7fc909000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc904114bd0 con 0x7fc90410a390 2024-04-24T06:40:37.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.286+0000 7fc909000700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc904114ed0 con 0x7fc90410ad30 2024-04-24T06:40:37.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.286+0000 7fc901e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc904109190 0x7fc90419f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:37.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.286+0000 7fc901e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc904109190 0x7fc90419f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40878/0 (socket says 172.21.15.5:40878) 2024-04-24T06:40:37.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.286+0000 7fc901e00700 1 -- 172.21.15.5:0/494937171 learned_addr learned my addr 172.21.15.5:0/494937171 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:40:37.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.286+0000 7fc901400700 1 --2- 172.21.15.5:0/494937171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc90410a390 0x7fc90419f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:37.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.286+0000 7fc902800700 1 --2- 172.21.15.5:0/494937171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc90410ad30 0x7fc9041a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:37.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.287+0000 7fc901e00700 1 -- 172.21.15.5:0/494937171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc90410ad30 msgr2=0x7fc9041a6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:37.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.287+0000 7fc901e00700 1 --2- 172.21.15.5:0/494937171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc90410ad30 0x7fc9041a6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:37.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.287+0000 7fc901e00700 1 -- 172.21.15.5:0/494937171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc90410a390 msgr2=0x7fc90419f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:37.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.287+0000 7fc901e00700 1 --2- 172.21.15.5:0/494937171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc90410a390 0x7fc90419f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:37.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.287+0000 7fc901e00700 1 -- 172.21.15.5:0/494937171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9041a6fc0 con 0x7fc904109190 2024-04-24T06:40:37.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.287+0000 7fc902800700 1 --2- 172.21.15.5:0/494937171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc90410ad30 0x7fc9041a6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:40:37.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.287+0000 7fc901400700 1 --2- 172.21.15.5:0/494937171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc90410a390 0x7fc90419f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:40:37.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.287+0000 7fc901e00700 1 --2- 172.21.15.5:0/494937171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc904109190 0x7fc90419f360 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400ea80 tx=0x7fc8f400ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:37.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.287+0000 7fc8faa00700 1 -- 172.21.15.5:0/494937171 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f400cbd0 con 0x7fc904109190 2024-04-24T06:40:37.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.288+0000 7fc8faa00700 1 -- 172.21.15.5:0/494937171 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8f400cd70 con 0x7fc904109190 2024-04-24T06:40:37.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.288+0000 7fc909000700 1 -- 172.21.15.5:0/494937171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9041a72b0 con 0x7fc904109190 2024-04-24T06:40:37.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.288+0000 7fc8faa00700 1 -- 172.21.15.5:0/494937171 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4018780 con 0x7fc904109190 2024-04-24T06:40:37.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.288+0000 7fc909000700 1 -- 172.21.15.5:0/494937171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9041a77f0 con 0x7fc904109190 2024-04-24T06:40:37.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.289+0000 7fc909000700 1 -- 172.21.15.5:0/494937171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9040697e0 con 0x7fc904109190 2024-04-24T06:40:37.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.293+0000 7fc8faa00700 1 -- 172.21.15.5:0/494937171 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fc8f401f020 con 0x7fc904109190 2024-04-24T06:40:37.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.294+0000 7fc8faa00700 1 --2- 172.21.15.5:0/494937171 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fc8e80a3c40 0x7fc8e80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:37.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.294+0000 7fc8faa00700 1 -- 172.21.15.5:0/494937171 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fc8f40d0770 con 0x7fc904109190 2024-04-24T06:40:37.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.294+0000 7fc901400700 1 --2- 172.21.15.5:0/494937171 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fc8e80a3c40 0x7fc8e80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:37.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.295+0000 7fc901400700 1 --2- 172.21.15.5:0/494937171 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fc8e80a3c40 0x7fc8e80a6100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0000c00 tx=0x7fc8f0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:37.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.297+0000 7fc8faa00700 1 -- 172.21.15.5:0/494937171 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc8f4026070 con 0x7fc904109190 2024-04-24T06:40:37.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.532+0000 7fc909000700 1 -- 172.21.15.5:0/494937171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fc90402dfe0 con 0x7fc904109190 2024-04-24T06:40:37.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.533+0000 7fc8faa00700 1 -- 172.21.15.5:0/494937171 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7fc8f409d550 con 0x7fc904109190 2024-04-24T06:40:37.534 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:40:37.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.535+0000 7fc909000700 1 -- 172.21.15.5:0/494937171 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fc8e80a3c40 msgr2=0x7fc8e80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:37.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.535+0000 7fc909000700 1 --2- 172.21.15.5:0/494937171 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fc8e80a3c40 0x7fc8e80a6100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0000c00 tx=0x7fc8f0005fb0 comp rx=0 tx=0).stop 2024-04-24T06:40:37.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.535+0000 7fc909000700 1 -- 172.21.15.5:0/494937171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc904109190 msgr2=0x7fc90419f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:37.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.535+0000 7fc909000700 1 --2- 172.21.15.5:0/494937171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc904109190 0x7fc90419f360 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400ea80 tx=0x7fc8f400ed90 comp rx=0 tx=0).stop 2024-04-24T06:40:37.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.536+0000 7fc909000700 1 -- 172.21.15.5:0/494937171 shutdown_connections 2024-04-24T06:40:37.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.536+0000 7fc909000700 1 --2- 172.21.15.5:0/494937171 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fc8e80a3c40 0x7fc8e80a6100 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:37.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.536+0000 7fc909000700 1 --2- 172.21.15.5:0/494937171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc90410ad30 0x7fc9041a6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:37.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.536+0000 7fc909000700 1 --2- 172.21.15.5:0/494937171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc904109190 0x7fc90419f360 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:37.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.536+0000 7fc909000700 1 --2- 172.21.15.5:0/494937171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc90410a390 0x7fc90419f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:37.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.536+0000 7fc909000700 1 -- 172.21.15.5:0/494937171 >> 172.21.15.5:0/494937171 conn(0x7fc904078160 msgr2=0x7fc90410d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:37.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.536+0000 7fc909000700 1 -- 172.21.15.5:0/494937171 shutdown_connections 2024-04-24T06:40:37.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:37.536+0000 7fc909000700 1 -- 172.21.15.5:0/494937171 wait complete. 2024-04-24T06:40:38.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:37 smithi137 ceph-mon[33017]: purged_snaps scrub starts 2024-04-24T06:40:38.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:37 smithi137 ceph-mon[33017]: purged_snaps scrub ok 2024-04-24T06:40:38.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:37 smithi137 ceph-mon[33017]: osd.11 [v2:172.21.15.137:6824/90600596,v1:172.21.15.137:6825/90600596] boot 2024-04-24T06:40:38.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:37 smithi137 ceph-mon[33017]: osdmap e81: 12 total, 12 up, 12 in 2024-04-24T06:40:38.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:37 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T06:40:38.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:37 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/494937171' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T06:40:38.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:37 smithi005 ceph-mon[27798]: purged_snaps scrub starts 2024-04-24T06:40:38.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:37 smithi005 ceph-mon[27798]: purged_snaps scrub ok 2024-04-24T06:40:38.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:37 smithi005 ceph-mon[27798]: osd.11 [v2:172.21.15.137:6824/90600596,v1:172.21.15.137:6825/90600596] boot 2024-04-24T06:40:38.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:37 smithi005 ceph-mon[27798]: osdmap e81: 12 total, 12 up, 12 in 2024-04-24T06:40:38.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:37 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T06:40:38.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:37 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/494937171' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T06:40:38.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:37 smithi037 ceph-mon[32864]: purged_snaps scrub starts 2024-04-24T06:40:38.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:37 smithi037 ceph-mon[32864]: purged_snaps scrub ok 2024-04-24T06:40:38.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:37 smithi037 ceph-mon[32864]: osd.11 [v2:172.21.15.137:6824/90600596,v1:172.21.15.137:6825/90600596] boot 2024-04-24T06:40:38.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:37 smithi037 ceph-mon[32864]: osdmap e81: 12 total, 12 up, 12 in 2024-04-24T06:40:38.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:37 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T06:40:38.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:37 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/494937171' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T06:40:38.392 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1713940836,"num_in_osds":12,"osd_in_since":1713940817,"num_remapped_pgs":0} 2024-04-24T06:40:38.393 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-24T06:40:38.870 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:40:39.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:38 smithi005 ceph-mon[27798]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-24T06:40:39.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:38 smithi005 ceph-mon[27798]: osdmap e82: 12 total, 12 up, 12 in 2024-04-24T06:40:39.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:38 smithi005 ceph-mon[27798]: Detected new or changed devices on smithi137 2024-04-24T06:40:39.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:38 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:39.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:38 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:39.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:38 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:40:39.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:38 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:40:39.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:38 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:40:39.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:38 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:40:39.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:38 smithi005 ceph-mon[27798]: Adjusting osd_memory_target on smithi137 to 4272M 2024-04-24T06:40:39.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:38 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:39.149 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:38 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:40:39.149 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:38 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:40:39.149 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:38 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:39.149 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:38 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:40:39.149 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:38 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:40:39.149 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:38 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:40:39.149 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:38 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:39.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:38 smithi137 ceph-mon[33017]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-24T06:40:39.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:38 smithi137 ceph-mon[33017]: osdmap e82: 12 total, 12 up, 12 in 2024-04-24T06:40:39.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:38 smithi137 ceph-mon[33017]: Detected new or changed devices on smithi137 2024-04-24T06:40:39.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:38 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:39.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:38 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:39.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:38 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:40:39.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:38 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:40:39.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:38 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:40:39.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:38 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:40:39.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:38 smithi137 ceph-mon[33017]: Adjusting osd_memory_target on smithi137 to 4272M 2024-04-24T06:40:39.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:38 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:39.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:38 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:40:39.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:38 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:40:39.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:38 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:39.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:38 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:40:39.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:38 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:40:39.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:38 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:40:39.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:38 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:39.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:38 smithi037 ceph-mon[32864]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-24T06:40:39.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:38 smithi037 ceph-mon[32864]: osdmap e82: 12 total, 12 up, 12 in 2024-04-24T06:40:39.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:38 smithi037 ceph-mon[32864]: Detected new or changed devices on smithi137 2024-04-24T06:40:39.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:38 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:39.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:38 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:39.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:38 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:40:39.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:38 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:40:39.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:38 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:40:39.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:38 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:40:39.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:38 smithi037 ceph-mon[32864]: Adjusting osd_memory_target on smithi137 to 4272M 2024-04-24T06:40:39.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:38 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:39.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:38 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:40:39.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:38 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:40:39.270 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:38 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:39.270 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:38 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:40:39.270 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:38 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:40:39.270 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:38 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:40:39.270 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:38 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:40.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:40.908+0000 7f855b000700 1 -- 172.21.15.5:0/831707748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8554103500 msgr2=0x7f855410f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:40.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:40.908+0000 7f855b000700 1 --2- 172.21.15.5:0/831707748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8554103500 0x7f855410f960 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f854c00b0a0 tx=0x7f854c00b3b0 comp rx=0 tx=0).stop 2024-04-24T06:40:40.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:40.909+0000 7f855b000700 1 -- 172.21.15.5:0/831707748 shutdown_connections 2024-04-24T06:40:40.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:40.909+0000 7f855b000700 1 --2- 172.21.15.5:0/831707748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8554103500 0x7f855410f960 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:40.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:40.909+0000 7f855b000700 1 --2- 172.21.15.5:0/831707748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8554108b90 0x7f8554108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:40.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:40.909+0000 7f855b000700 1 --2- 172.21.15.5:0/831707748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8554102b40 0x7f8554102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:40.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:40.909+0000 7f855b000700 1 -- 172.21.15.5:0/831707748 >> 172.21.15.5:0/831707748 conn(0x7f8554079c50 msgr2=0x7f855407a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:40.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:40.909+0000 7f855b000700 1 -- 172.21.15.5:0/831707748 shutdown_connections 2024-04-24T06:40:40.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:40.909+0000 7f855b000700 1 -- 172.21.15.5:0/831707748 wait complete. 2024-04-24T06:40:40.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:40.910+0000 7f855b000700 1 Processor -- start 2024-04-24T06:40:40.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:40.910+0000 7f855b000700 1 -- start start 2024-04-24T06:40:40.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:40.911+0000 7f855b000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8554102b40 0x7f855419b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:40.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:40.911+0000 7f855b000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8554103500 0x7f855419ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:40.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:40.911+0000 7f855b000700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8554108b90 0x7f855419fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:40.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:40.911+0000 7f855b000700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8554112e00 con 0x7f8554108b90 2024-04-24T06:40:40.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:40.911+0000 7f855b000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8554112c80 con 0x7f8554103500 2024-04-24T06:40:40.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:40.911+0000 7f855b000700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8554112f80 con 0x7f8554102b40 2024-04-24T06:40:40.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:40.911+0000 7f8559000700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8554108b90 0x7f855419fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:40.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:40.911+0000 7f8553e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8554102b40 0x7f855419b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:40.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:40.911+0000 7f8559000700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8554108b90 0x7f855419fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40882/0 (socket says 172.21.15.5:40882) 2024-04-24T06:40:40.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:40.911+0000 7f8559000700 1 -- 172.21.15.5:0/2360508886 learned_addr learned my addr 172.21.15.5:0/2360508886 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:40:40.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:40.911+0000 7f8553400700 1 --2- 172.21.15.5:0/2360508886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8554103500 0x7f855419ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:40.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:40.912+0000 7f8553e00700 1 -- 172.21.15.5:0/2360508886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8554103500 msgr2=0x7f855419ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:40.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:40.912+0000 7f8553e00700 1 --2- 172.21.15.5:0/2360508886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8554103500 0x7f855419ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:40.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:40.912+0000 7f8553e00700 1 -- 172.21.15.5:0/2360508886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8554108b90 msgr2=0x7f855419fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:40.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:40.912+0000 7f8553e00700 1 --2- 172.21.15.5:0/2360508886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8554108b90 0x7f855419fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:40.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:40.912+0000 7f8553e00700 1 -- 172.21.15.5:0/2360508886 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85541a0520 con 0x7f8554102b40 2024-04-24T06:40:40.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:40.912+0000 7f8559000700 1 --2- 172.21.15.5:0/2360508886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8554108b90 0x7f855419fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:40:40.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:40.912+0000 7f8553e00700 1 --2- 172.21.15.5:0/2360508886 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8554102b40 0x7f855419b510 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f854400d950 tx=0x7f854400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:40.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:40.912+0000 7f8550c00700 1 -- 172.21.15.5:0/2360508886 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8544016070 con 0x7f8554102b40 2024-04-24T06:40:40.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:40.912+0000 7f8550c00700 1 -- 172.21.15.5:0/2360508886 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f854400fd70 con 0x7f8554102b40 2024-04-24T06:40:40.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:40.912+0000 7f855b000700 1 -- 172.21.15.5:0/2360508886 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85541a0810 con 0x7f8554102b40 2024-04-24T06:40:40.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:40.912+0000 7f8550c00700 1 -- 172.21.15.5:0/2360508886 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854400e3b0 con 0x7f8554102b40 2024-04-24T06:40:40.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:40.913+0000 7f855b000700 1 -- 172.21.15.5:0/2360508886 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85541a8090 con 0x7f8554102b40 2024-04-24T06:40:40.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:40.914+0000 7f855b000700 1 -- 172.21.15.5:0/2360508886 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85540697e0 con 0x7f8554102b40 2024-04-24T06:40:40.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:40.915+0000 7f8550c00700 1 -- 172.21.15.5:0/2360508886 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f8544019070 con 0x7f8554102b40 2024-04-24T06:40:40.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:40.917+0000 7f8550c00700 1 --2- 172.21.15.5:0/2360508886 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f85380a3c40 0x7f85380a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:40.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:40.917+0000 7f8550c00700 1 -- 172.21.15.5:0/2360508886 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f85440cf980 con 0x7f8554102b40 2024-04-24T06:40:40.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:40.917+0000 7f8553400700 1 --2- 172.21.15.5:0/2360508886 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f85380a3c40 0x7f85380a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:40.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:40.918+0000 7f8553400700 1 --2- 172.21.15.5:0/2360508886 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f85380a3c40 0x7f85380a6100 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f855407a5d0 tx=0x7f8540009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:40.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:40.921+0000 7f8550c00700 1 -- 172.21.15.5:0/2360508886 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8544010460 con 0x7f8554102b40 2024-04-24T06:40:40.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:40 smithi137 ceph-mon[33017]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:40:41.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:40 smithi005 ceph-mon[27798]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:40:41.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:41.154+0000 7f855b000700 1 -- 172.21.15.5:0/2360508886 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f855419c870 con 0x7f8554102b40 2024-04-24T06:40:41.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:41.156+0000 7f8550c00700 1 -- 172.21.15.5:0/2360508886 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18325 (secure 0 0 0) 0x7f854409c760 con 0x7f8554102b40 2024-04-24T06:40:41.158 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:40:41.158 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":82,"fsid":"43e9bad0-0204-11ef-bc93-c7b262605968","created":"2024-04-24T06:32:26.340555+0000","modified":"2024-04-24T06:40:37.808668+0000","last_up_change":"2024-04-24T06:40:36.790934+0000","last_in_change":"2024-04-24T06:40:17.803257+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:36:27.320342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c2e974c-745c-4dd5-8c3d-8e8da2eb1ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6803","nonce":2155858515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6805","nonce":2155858515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6809","nonce":2155858515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6807","nonce":2155858515}]},"public_addr":"172.21.15.5:6803/2155858515","cluster_addr":"172.21.15.5:6805/2155858515","heartbeat_back_addr":"172.21.15.5:6809/2155858515","heartbeat_front_addr":"172.21.15.5:6807/2155858515","state":["exists","up"]},{"osd":1,"uuid":"50467744-db20-4721-bf2d-1c5db938297b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6801","nonce":1302362650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6803","nonce":1302362650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6807","nonce":1302362650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6805","nonce":1302362650}]},"public_addr":"172.21.15.37:6801/1302362650","cluster_addr":"172.21.15.37:6803/1302362650","heartbeat_back_addr":"172.21.15.37:6807/1302362650","heartbeat_front_addr":"172.21.15.37:6805/1302362650","state":["exists","up"]},{"osd":2,"uuid":"0e190b15-d04c-4288-8d48-ecf6664392fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6801","nonce":2885027256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6803","nonce":2885027256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6807","nonce":2885027256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6805","nonce":2885027256}]},"public_addr":"172.21.15.137:6801/2885027256","cluster_addr":"172.21.15.137:6803/2885027256","heartbeat_back_addr":"172.21.15.137:6807/2885027256","heartbeat_front_addr":"172.21.15.137:6805/2885027256","state":["exists","up"]},{"osd":3,"uuid":"e05592ba-d96a-472a-9f63-09a535d04dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6811","nonce":3701747260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6813","nonce":3701747260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6817","nonce":3701747260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6815","nonce":3701747260}]},"public_addr":"172.21.15.5:6811/3701747260","cluster_addr":"172.21.15.5:6813/3701747260","heartbeat_back_addr":"172.21.15.5:6817/3701747260","heartbeat_front_addr":"172.21.15.5:6815/3701747260","state":["exists","up"]},{"osd":4,"uuid":"c3f36e51-7a3b-4dd9-af4b-a10558830c4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6809","nonce":759253844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6811","nonce":759253844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6815","nonce":759253844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6813","nonce":759253844}]},"public_addr":"172.21.15.37:6809/759253844","cluster_addr":"172.21.15.37:6811/759253844","heartbeat_back_addr":"172.21.15.37:6815/759253844","heartbeat_front_addr":"172.21.15.37:6813/759253844","state":["exists","up"]},{"osd":5,"uuid":"97a0500c-cbb8-4ff5-b476-8634b26b3821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6809","nonce":3771921560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6811","nonce":3771921560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6815","nonce":3771921560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6813","nonce":3771921560}]},"public_addr":"172.21.15.137:6809/3771921560","cluster_addr":"172.21.15.137:6811/3771921560","heartbeat_back_addr":"172.21.15.137:6815/3771921560","heartbeat_front_addr":"172.21.15.137:6813/3771921560","state":["exists","up"]},{"osd":6,"uuid":"33a33d62-bbb3-4ed9-834e-19ae93a97670","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6819","nonce":2064636084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6821","nonce":2064636084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6825","nonce":2064636084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6823","nonce":2064636084}]},"public_addr":"172.21.15.5:6819/2064636084","cluster_addr":"172.21.15.5:6821/2064636084","heartbeat_back_addr":"172.21.15.5:6825/2064636084","heartbeat_front_addr":"172.21.15.5:6823/2064636084","state":["exists","up"]},{"osd":7,"uuid":"c0f4e036-4717-4514-bb59-ab11fbaab063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6817","nonce":2952750659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6819","nonce":2952750659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6823","nonce":2952750659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6821","nonce":2952750659}]},"public_addr":"172.21.15.37:6817/2952750659","cluster_addr":"172.21.15.37:6819/2952750659","heartbeat_back_addr":"172.21.15.37:6823/2952750659","heartbeat_front_addr":"172.21.15.37:6821/2952750659","state":["exists","up"]},{"osd":8,"uuid":"5391fd87-44ea-440f-80da-c54d0327c04c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6817","nonce":29724945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6819","nonce":29724945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6823","nonce":29724945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6821","nonce":29724945}]},"public_addr":"172.21.15.137:6817/29724945","cluster_addr":"172.21.15.137:6819/29724945","heartbeat_back_addr":"172.21.15.137:6823/29724945","heartbeat_front_addr":"172.21.15.137:6821/29724945","state":["exists","up"]},{"osd":9,"uuid":"d94c9dbc-f43d-4237-ac7f-47a6a5ede1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6827","nonce":2257102107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6829","nonce":2257102107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6833","nonce":2257102107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6831","nonce":2257102107}]},"public_addr":"172.21.15.5:6827/2257102107","cluster_addr":"172.21.15.5:6829/2257102107","heartbeat_back_addr":"172.21.15.5:6833/2257102107","heartbeat_front_addr":"172.21.15.5:6831/2257102107","state":["exists","up"]},{"osd":10,"uuid":"e2455332-2a5e-44b1-85d3-8e47da399f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6825","nonce":2451570637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6827","nonce":2451570637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6831","nonce":2451570637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6829","nonce":2451570637}]},"public_addr":"172.21.15.37:6825/2451570637","cluster_addr":"172.21.15.37:6827/2451570637","heartbeat_back_addr":"172.21.15.37:6831/2451570637","heartbeat_front_addr":"172.21.15.37:6829/2451570637","state":["exists","up"]},{"osd":11,"uuid":"49d9396b-704a-4cef-8210-75d30c28e924","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6825","nonce":90600596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6827","nonce":90600596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6831","nonce":90600596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6829","nonce":90600596}]},"public_addr":"172.21.15.137:6825/90600596","cluster_addr":"172.21.15.137:6827/90600596","heartbeat_back_addr":"172.21.15.137:6831/90600596","heartbeat_front_addr":"172.21.15.137:6829/90600596","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:35:35.591619+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:00.568817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:24.195643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:56.451724+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:22.109293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:46.161749+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:19.278305+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:45.620256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:10.041165+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:43.813697+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:10.736026+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:35.384377+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/23307164":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6801/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:0/230785941":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/2940023597":"2024-04-25T06:33:07.040369+0000","172.21.15.5:6801/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3886613904":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/4199165437":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3459593499":"2024-04-25T06:33:41.161852+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:40:41.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:41.159+0000 7f855b000700 1 -- 172.21.15.5:0/2360508886 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f85380a3c40 msgr2=0x7f85380a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:41.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:41.159+0000 7f855b000700 1 --2- 172.21.15.5:0/2360508886 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f85380a3c40 0x7f85380a6100 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f855407a5d0 tx=0x7f8540009260 comp rx=0 tx=0).stop 2024-04-24T06:40:41.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:41.159+0000 7f855b000700 1 -- 172.21.15.5:0/2360508886 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8554102b40 msgr2=0x7f855419b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:41.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:41.159+0000 7f855b000700 1 --2- 172.21.15.5:0/2360508886 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8554102b40 0x7f855419b510 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f854400d950 tx=0x7f854400dc60 comp rx=0 tx=0).stop 2024-04-24T06:40:41.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:41.160+0000 7f855b000700 1 -- 172.21.15.5:0/2360508886 shutdown_connections 2024-04-24T06:40:41.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:41.160+0000 7f855b000700 1 --2- 172.21.15.5:0/2360508886 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f85380a3c40 0x7f85380a6100 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:41.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:41.160+0000 7f855b000700 1 --2- 172.21.15.5:0/2360508886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8554108b90 0x7f855419fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:41.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:41.160+0000 7f855b000700 1 --2- 172.21.15.5:0/2360508886 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8554102b40 0x7f855419b510 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:41.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:41.160+0000 7f855b000700 1 --2- 172.21.15.5:0/2360508886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8554103500 0x7f855419ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:41.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:41.160+0000 7f855b000700 1 -- 172.21.15.5:0/2360508886 >> 172.21.15.5:0/2360508886 conn(0x7f8554079c50 msgr2=0x7f855410df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:41.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:41.160+0000 7f855b000700 1 -- 172.21.15.5:0/2360508886 shutdown_connections 2024-04-24T06:40:41.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:41.160+0000 7f855b000700 1 -- 172.21.15.5:0/2360508886 wait complete. 2024-04-24T06:40:41.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:40 smithi037 ceph-mon[32864]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:40:41.990 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:36:27.320342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:40:41.990 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-24T06:40:42.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:41 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/2360508886' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:40:42.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:41 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/2360508886' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:40:42.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:41 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/2360508886' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:40:42.458 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:40:42.948 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:42 smithi005 ceph-mon[27798]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:40:43.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:42 smithi137 ceph-mon[33017]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:40:43.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:42 smithi037 ceph-mon[32864]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:40:44.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.546+0000 7fbfe4a00700 1 -- 172.21.15.5:0/3953189734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfe0113cc0 msgr2=0x7fbfe01160b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:44.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.546+0000 7fbfe4a00700 1 --2- 172.21.15.5:0/3953189734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfe0113cc0 0x7fbfe01160b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbfd800b0a0 tx=0x7fbfd800b3b0 comp rx=0 tx=0).stop 2024-04-24T06:40:44.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.547+0000 7fbfe4a00700 1 -- 172.21.15.5:0/3953189734 shutdown_connections 2024-04-24T06:40:44.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.547+0000 7fbfe4a00700 1 --2- 172.21.15.5:0/3953189734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfe0113cc0 0x7fbfe01160b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:44.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.547+0000 7fbfe4a00700 1 --2- 172.21.15.5:0/3953189734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfe0074d20 0x7fbfe0075100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:44.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.547+0000 7fbfe4a00700 1 --2- 172.21.15.5:0/3953189734 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbfe0075640 0x7fbfe0113780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:44.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.547+0000 7fbfe4a00700 1 -- 172.21.15.5:0/3953189734 >> 172.21.15.5:0/3953189734 conn(0x7fbfe0079c50 msgr2=0x7fbfe007a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:44.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.547+0000 7fbfe4a00700 1 -- 172.21.15.5:0/3953189734 shutdown_connections 2024-04-24T06:40:44.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.547+0000 7fbfe4a00700 1 -- 172.21.15.5:0/3953189734 wait complete. 2024-04-24T06:40:44.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.548+0000 7fbfe4a00700 1 Processor -- start 2024-04-24T06:40:44.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.548+0000 7fbfe4a00700 1 -- start start 2024-04-24T06:40:44.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.549+0000 7fbfe4a00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfe0074d20 0x7fbfe01a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:44.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.549+0000 7fbfe4a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfe0075640 0x7fbfe01a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:44.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.549+0000 7fbfe4a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbfe0113cc0 0x7fbfe019f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:44.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.549+0000 7fbfe4a00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe0119420 con 0x7fbfe0074d20 2024-04-24T06:40:44.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.549+0000 7fbfe4a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe01192a0 con 0x7fbfe0075640 2024-04-24T06:40:44.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.549+0000 7fbfe4a00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe01195a0 con 0x7fbfe0113cc0 2024-04-24T06:40:44.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.549+0000 7fbfdd400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfe0075640 0x7fbfe01a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:44.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.549+0000 7fbfdde00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfe0074d20 0x7fbfe01a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:44.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.549+0000 7fbfdd400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfe0075640 0x7fbfe01a5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:51812/0 (socket says 172.21.15.5:51812) 2024-04-24T06:40:44.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.549+0000 7fbfdd400700 1 -- 172.21.15.5:0/3510591200 learned_addr learned my addr 172.21.15.5:0/3510591200 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:40:44.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.549+0000 7fbfde800700 1 --2- 172.21.15.5:0/3510591200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbfe0113cc0 0x7fbfe019f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:44.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.550+0000 7fbfdd400700 1 -- 172.21.15.5:0/3510591200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbfe0113cc0 msgr2=0x7fbfe019f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:44.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.550+0000 7fbfdd400700 1 --2- 172.21.15.5:0/3510591200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbfe0113cc0 0x7fbfe019f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:44.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.550+0000 7fbfdd400700 1 -- 172.21.15.5:0/3510591200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfe0074d20 msgr2=0x7fbfe01a55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:44.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.550+0000 7fbfdd400700 1 --2- 172.21.15.5:0/3510591200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfe0074d20 0x7fbfe01a55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:44.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.550+0000 7fbfdd400700 1 -- 172.21.15.5:0/3510591200 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfe019ff60 con 0x7fbfe0075640 2024-04-24T06:40:44.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.550+0000 7fbfdd400700 1 --2- 172.21.15.5:0/3510591200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfe0075640 0x7fbfe01a5af0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc00e9a0 tx=0x7fbfcc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:44.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.550+0000 7fbfd6a00700 1 -- 172.21.15.5:0/3510591200 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc00ca10 con 0x7fbfe0075640 2024-04-24T06:40:44.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.550+0000 7fbfd6a00700 1 -- 172.21.15.5:0/3510591200 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfcc00cbb0 con 0x7fbfe0075640 2024-04-24T06:40:44.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.550+0000 7fbfd6a00700 1 -- 172.21.15.5:0/3510591200 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc018680 con 0x7fbfe0075640 2024-04-24T06:40:44.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.550+0000 7fbfe4a00700 1 -- 172.21.15.5:0/3510591200 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfe01a0250 con 0x7fbfe0075640 2024-04-24T06:40:44.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.550+0000 7fbfe4a00700 1 -- 172.21.15.5:0/3510591200 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfe01ac530 con 0x7fbfe0075640 2024-04-24T06:40:44.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.552+0000 7fbfe4a00700 1 -- 172.21.15.5:0/3510591200 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfe0076410 con 0x7fbfe0075640 2024-04-24T06:40:44.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.554+0000 7fbfd6a00700 1 -- 172.21.15.5:0/3510591200 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fbfcc010430 con 0x7fbfe0075640 2024-04-24T06:40:44.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.555+0000 7fbfd6a00700 1 --2- 172.21.15.5:0/3510591200 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fbfc40a3cc0 0x7fbfc40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:44.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.555+0000 7fbfd6a00700 1 -- 172.21.15.5:0/3510591200 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fbfcc0d0130 con 0x7fbfe0075640 2024-04-24T06:40:44.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.555+0000 7fbfdde00700 1 --2- 172.21.15.5:0/3510591200 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fbfc40a3cc0 0x7fbfc40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:44.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.556+0000 7fbfdde00700 1 --2- 172.21.15.5:0/3510591200 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fbfc40a3cc0 0x7fbfc40a6180 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0009c60 tx=0x7fbfd0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:44.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.559+0000 7fbfd6a00700 1 -- 172.21.15.5:0/3510591200 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbfcc09cf10 con 0x7fbfe0075640 2024-04-24T06:40:44.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.789+0000 7fbfe4a00700 1 -- 172.21.15.5:0/3510591200 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbfe01ac700 con 0x7fbfe0075640 2024-04-24T06:40:44.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.790+0000 7fbfd6a00700 1 -- 172.21.15.5:0/3510591200 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fbfcc09c810 con 0x7fbfe0075640 2024-04-24T06:40:44.792 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-24T06:40:44.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.793+0000 7fbfe4a00700 1 -- 172.21.15.5:0/3510591200 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fbfc40a3cc0 msgr2=0x7fbfc40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:44.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.793+0000 7fbfe4a00700 1 --2- 172.21.15.5:0/3510591200 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fbfc40a3cc0 0x7fbfc40a6180 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0009c60 tx=0x7fbfd0009260 comp rx=0 tx=0).stop 2024-04-24T06:40:44.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.793+0000 7fbfe4a00700 1 -- 172.21.15.5:0/3510591200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfe0075640 msgr2=0x7fbfe01a5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:44.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.793+0000 7fbfe4a00700 1 --2- 172.21.15.5:0/3510591200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfe0075640 0x7fbfe01a5af0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc00e9a0 tx=0x7fbfcc00ecb0 comp rx=0 tx=0).stop 2024-04-24T06:40:44.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.794+0000 7fbfe4a00700 1 -- 172.21.15.5:0/3510591200 shutdown_connections 2024-04-24T06:40:44.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.794+0000 7fbfe4a00700 1 --2- 172.21.15.5:0/3510591200 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fbfc40a3cc0 0x7fbfc40a6180 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:44.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.794+0000 7fbfe4a00700 1 --2- 172.21.15.5:0/3510591200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbfe0113cc0 0x7fbfe019f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:44.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.794+0000 7fbfe4a00700 1 --2- 172.21.15.5:0/3510591200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfe0074d20 0x7fbfe01a55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:44.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.794+0000 7fbfe4a00700 1 --2- 172.21.15.5:0/3510591200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfe0075640 0x7fbfe01a5af0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:44.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.794+0000 7fbfe4a00700 1 -- 172.21.15.5:0/3510591200 >> 172.21.15.5:0/3510591200 conn(0x7fbfe0079c50 msgr2=0x7fbfe0112a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:44.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.794+0000 7fbfe4a00700 1 -- 172.21.15.5:0/3510591200 shutdown_connections 2024-04-24T06:40:44.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:44.794+0000 7fbfe4a00700 1 -- 172.21.15.5:0/3510591200 wait complete. 2024-04-24T06:40:45.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:44 smithi005 ceph-mon[27798]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:40:45.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:44 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/3510591200' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T06:40:45.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:44 smithi137 ceph-mon[33017]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:40:45.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:44 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/3510591200' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T06:40:45.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:44 smithi037 ceph-mon[32864]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:40:45.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:44 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/3510591200' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T06:40:45.497 INFO:tasks.cephadm:Adding mds.a on smithi005 2024-04-24T06:40:45.497 INFO:tasks.cephadm:Adding mds.d on smithi005 2024-04-24T06:40:45.497 INFO:tasks.cephadm:Adding mds.g on smithi005 2024-04-24T06:40:45.497 INFO:tasks.cephadm:Adding mds.j on smithi005 2024-04-24T06:40:45.497 INFO:tasks.cephadm:Adding mds.b on smithi037 2024-04-24T06:40:45.497 INFO:tasks.cephadm:Adding mds.e on smithi037 2024-04-24T06:40:45.497 INFO:tasks.cephadm:Adding mds.h on smithi037 2024-04-24T06:40:45.497 INFO:tasks.cephadm:Adding mds.k on smithi037 2024-04-24T06:40:45.497 INFO:tasks.cephadm:Adding mds.c on smithi137 2024-04-24T06:40:45.497 INFO:tasks.cephadm:Adding mds.f on smithi137 2024-04-24T06:40:45.498 INFO:tasks.cephadm:Adding mds.i on smithi137 2024-04-24T06:40:45.498 INFO:tasks.cephadm:Adding mds.l on smithi137 2024-04-24T06:40:45.498 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi037=b;smithi037=e;smithi037=h;smithi037=k;smithi137=c;smithi137=f;smithi137=i;smithi137=l' 2024-04-24T06:40:45.746 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T06:40:47.086 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:46 smithi137 ceph-mon[33017]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:40:47.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:46 smithi005 ceph-mon[27798]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:40:47.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:46 smithi037 ceph-mon[32864]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:40:47.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.299+0000 7f8dd8584700 1 -- 172.21.15.137:0/497937388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8dd0101a50 msgr2=0x7f8dd0101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:47.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.299+0000 7f8dd8584700 1 --2- 172.21.15.137:0/497937388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8dd0101a50 0x7f8dd0101e70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc009990 tx=0x7f8dcc009ca0 comp rx=0 tx=0).stop 2024-04-24T06:40:47.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.300+0000 7f8dd8584700 1 -- 172.21.15.137:0/497937388 shutdown_connections 2024-04-24T06:40:47.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.300+0000 7f8dd8584700 1 --2- 172.21.15.137:0/497937388 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8dd01035f0 0x7f8dd0109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:47.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.300+0000 7f8dd8584700 1 --2- 172.21.15.137:0/497937388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8dd0101a50 0x7f8dd0101e70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:47.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.300+0000 7f8dd8584700 1 --2- 172.21.15.137:0/497937388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8dd0102c50 0x7f8dd01030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:47.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.300+0000 7f8dd8584700 1 -- 172.21.15.137:0/497937388 >> 172.21.15.137:0/497937388 conn(0x7f8dd00fd010 msgr2=0x7f8dd00ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:47.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.300+0000 7f8dd8584700 1 -- 172.21.15.137:0/497937388 shutdown_connections 2024-04-24T06:40:47.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.300+0000 7f8dd8584700 1 -- 172.21.15.137:0/497937388 wait complete. 2024-04-24T06:40:47.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.300+0000 7f8dd8584700 1 Processor -- start 2024-04-24T06:40:47.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.300+0000 7f8dd8584700 1 -- start start 2024-04-24T06:40:47.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.300+0000 7f8dd8584700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8dd0101a50 0x7f8dd0197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:47.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.300+0000 7f8dd8584700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8dd0102c50 0x7f8dd0198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:47.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.300+0000 7f8dd8584700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8dd01035f0 0x7f8dd019f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:47.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.300+0000 7f8dd8584700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd010d710 con 0x7f8dd0101a50 2024-04-24T06:40:47.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.300+0000 7f8dd8584700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd010d590 con 0x7f8dd01035f0 2024-04-24T06:40:47.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.300+0000 7f8dd8584700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd010d890 con 0x7f8dd0102c50 2024-04-24T06:40:47.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.301+0000 7f8dd5b1f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8dd0102c50 0x7f8dd0198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:47.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.301+0000 7f8dd5b1f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8dd0102c50 0x7f8dd0198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42314/0 (socket says 172.21.15.137:42314) 2024-04-24T06:40:47.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.301+0000 7f8dd5b1f700 1 -- 172.21.15.137:0/2488103415 learned_addr learned my addr 172.21.15.137:0/2488103415 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-04-24T06:40:47.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.301+0000 7f8dd6b21700 1 --2- 172.21.15.137:0/2488103415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8dd01035f0 0x7f8dd019f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:47.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.301+0000 7f8dd5b1f700 1 -- 172.21.15.137:0/2488103415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8dd01035f0 msgr2=0x7f8dd019f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:47.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.301+0000 7f8dd5b1f700 1 --2- 172.21.15.137:0/2488103415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8dd01035f0 0x7f8dd019f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:47.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.301+0000 7f8dd5b1f700 1 -- 172.21.15.137:0/2488103415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8dd0101a50 msgr2=0x7f8dd0197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:47.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.301+0000 7f8dd5b1f700 1 --2- 172.21.15.137:0/2488103415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8dd0101a50 0x7f8dd0197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:47.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.301+0000 7f8dd5b1f700 1 -- 172.21.15.137:0/2488103415 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dd019fa30 con 0x7f8dd0102c50 2024-04-24T06:40:47.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.301+0000 7f8dd6b21700 1 --2- 172.21.15.137:0/2488103415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8dd01035f0 0x7f8dd019f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:40:47.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.301+0000 7f8dd5b1f700 1 --2- 172.21.15.137:0/2488103415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8dd0102c50 0x7f8dd0198310 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8dc000da10 tx=0x7f8dc000ddd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:47.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.301+0000 7f8dc77fe700 1 -- 172.21.15.137:0/2488103415 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc001b070 con 0x7f8dd0102c50 2024-04-24T06:40:47.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.301+0000 7f8dc77fe700 1 -- 172.21.15.137:0/2488103415 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8dc000ecd0 con 0x7f8dd0102c50 2024-04-24T06:40:47.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.301+0000 7f8dd8584700 1 -- 172.21.15.137:0/2488103415 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dd006b2c0 con 0x7f8dd0102c50 2024-04-24T06:40:47.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.301+0000 7f8dc77fe700 1 -- 172.21.15.137:0/2488103415 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc0014760 con 0x7f8dd0102c50 2024-04-24T06:40:47.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.301+0000 7f8dd8584700 1 -- 172.21.15.137:0/2488103415 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dd006b8d0 con 0x7f8dd0102c50 2024-04-24T06:40:47.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.302+0000 7f8dd8584700 1 -- 172.21.15.137:0/2488103415 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dd00697e0 con 0x7f8dd0102c50 2024-04-24T06:40:47.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.303+0000 7f8dc77fe700 1 -- 172.21.15.137:0/2488103415 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f8dc001e070 con 0x7f8dd0102c50 2024-04-24T06:40:47.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.304+0000 7f8dc77fe700 1 --2- 172.21.15.137:0/2488103415 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f8dbc0a3cc0 0x7f8dbc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:47.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.304+0000 7f8dc77fe700 1 -- 172.21.15.137:0/2488103415 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f8dc00d14e0 con 0x7f8dd0102c50 2024-04-24T06:40:47.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.304+0000 7f8dd6320700 1 --2- 172.21.15.137:0/2488103415 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f8dbc0a3cc0 0x7f8dbc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:47.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.305+0000 7f8dd6320700 1 --2- 172.21.15.137:0/2488103415 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f8dbc0a3cc0 0x7f8dbc0a6180 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc005f50 tx=0x7f8dcc005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:47.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.309+0000 7f8dc77fe700 1 -- 172.21.15.137:0/2488103415 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8dc009e360 con 0x7f8dd0102c50 2024-04-24T06:40:47.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.517+0000 7f8dd8584700 1 -- 172.21.15.137:0/2488103415 --> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi037=b;smithi037=e;smithi037=h;smithi037=k;smithi137=c;smithi137=f;smithi137=i;smithi137=l", "target": ["mon-mgr", ""]}) -- 0x7f8dd006bea0 con 0x7f8dbc0a3cc0 2024-04-24T06:40:47.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.533+0000 7f8dc77fe700 1 -- 172.21.15.137:0/2488103415 <== mgr.14152 v2:172.21.15.5:6800/4042936965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f8dd006bea0 con 0x7f8dbc0a3cc0 2024-04-24T06:40:47.535 INFO:teuthology.orchestra.run.smithi137.stdout:Scheduled mds.all update... 2024-04-24T06:40:47.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.536+0000 7f8dd8584700 1 -- 172.21.15.137:0/2488103415 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f8dbc0a3cc0 msgr2=0x7f8dbc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:47.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.536+0000 7f8dd8584700 1 --2- 172.21.15.137:0/2488103415 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f8dbc0a3cc0 0x7f8dbc0a6180 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc005f50 tx=0x7f8dcc005e80 comp rx=0 tx=0).stop 2024-04-24T06:40:47.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.536+0000 7f8dd8584700 1 -- 172.21.15.137:0/2488103415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8dd0102c50 msgr2=0x7f8dd0198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:47.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.536+0000 7f8dd8584700 1 --2- 172.21.15.137:0/2488103415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8dd0102c50 0x7f8dd0198310 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8dc000da10 tx=0x7f8dc000ddd0 comp rx=0 tx=0).stop 2024-04-24T06:40:47.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.536+0000 7f8dd8584700 1 -- 172.21.15.137:0/2488103415 shutdown_connections 2024-04-24T06:40:47.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.536+0000 7f8dd8584700 1 --2- 172.21.15.137:0/2488103415 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f8dbc0a3cc0 0x7f8dbc0a6180 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:47.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.536+0000 7f8dd8584700 1 --2- 172.21.15.137:0/2488103415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8dd01035f0 0x7f8dd019f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:47.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.536+0000 7f8dd8584700 1 --2- 172.21.15.137:0/2488103415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8dd0101a50 0x7f8dd0197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:47.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.536+0000 7f8dd8584700 1 --2- 172.21.15.137:0/2488103415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8dd0102c50 0x7f8dd0198310 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:47.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.536+0000 7f8dd8584700 1 -- 172.21.15.137:0/2488103415 >> 172.21.15.137:0/2488103415 conn(0x7f8dd00fd010 msgr2=0x7f8dd0106020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:47.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.536+0000 7f8dd8584700 1 -- 172.21.15.137:0/2488103415 shutdown_connections 2024-04-24T06:40:47.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-24T06:40:47.536+0000 7f8dd8584700 1 -- 172.21.15.137:0/2488103415 wait complete. 2024-04-24T06:40:48.168 DEBUG:teuthology.orchestra.run.smithi005:mds.a> sudo journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mds.a.service 2024-04-24T06:40:48.175 DEBUG:teuthology.orchestra.run.smithi005:mds.d> sudo journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mds.d.service 2024-04-24T06:40:48.221 DEBUG:teuthology.orchestra.run.smithi005:mds.g> sudo journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mds.g.service 2024-04-24T06:40:48.269 DEBUG:teuthology.orchestra.run.smithi005:mds.j> sudo journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mds.j.service 2024-04-24T06:40:48.314 DEBUG:teuthology.orchestra.run.smithi037:mds.b> sudo journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mds.b.service 2024-04-24T06:40:48.317 DEBUG:teuthology.orchestra.run.smithi037:mds.e> sudo journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mds.e.service 2024-04-24T06:40:48.360 DEBUG:teuthology.orchestra.run.smithi037:mds.h> sudo journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mds.h.service 2024-04-24T06:40:48.404 DEBUG:teuthology.orchestra.run.smithi037:mds.k> sudo journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mds.k.service 2024-04-24T06:40:48.447 DEBUG:teuthology.orchestra.run.smithi137:mds.c> sudo journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mds.c.service 2024-04-24T06:40:48.449 DEBUG:teuthology.orchestra.run.smithi137:mds.f> sudo journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mds.f.service 2024-04-24T06:40:48.492 DEBUG:teuthology.orchestra.run.smithi137:mds.i> sudo journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mds.i.service 2024-04-24T06:40:48.535 DEBUG:teuthology.orchestra.run.smithi137:mds.l> sudo journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mds.l.service 2024-04-24T06:40:48.578 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-24T06:40:48.578 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:40:48.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:48 smithi137 ceph-mon[33017]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:40:48.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:48 smithi137 ceph-mon[33017]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi037=b;smithi037=e;smithi037=h;smithi037=k;smithi137=c;smithi137=f;smithi137=i;smithi137=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:40:48.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:48 smithi137 ceph-mon[33017]: Saving service mds.all spec with placement smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi037=b;smithi037=e;smithi037=h;smithi037=k;smithi137=c;smithi137=f;smithi137=i;smithi137=l;count:12 2024-04-24T06:40:48.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:48 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:48.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:48 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:40:48.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:48 smithi037 ceph-mon[32864]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:40:48.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:48 smithi037 ceph-mon[32864]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi037=b;smithi037=e;smithi037=h;smithi037=k;smithi137=c;smithi137=f;smithi137=i;smithi137=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:40:48.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:48 smithi037 ceph-mon[32864]: Saving service mds.all spec with placement smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi037=b;smithi037=e;smithi037=h;smithi037=k;smithi137=c;smithi137=f;smithi137=i;smithi137=l;count:12 2024-04-24T06:40:48.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:48 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:48.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:48 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:40:48.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:48.784+0000 7fb271200640 1 -- 172.21.15.5:0/1361030710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb26c105350 msgr2=0x7fb26c1057d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:48.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:48.784+0000 7fb271200640 1 --2- 172.21.15.5:0/1361030710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb26c105350 0x7fb26c1057d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb25c009ca0 tx=0x7fb25c02a8c0 comp rx=0 tx=0).stop 2024-04-24T06:40:48.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:48.784+0000 7fb271200640 1 -- 172.21.15.5:0/1361030710 shutdown_connections 2024-04-24T06:40:48.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:48.784+0000 7fb271200640 1 --2- 172.21.15.5:0/1361030710 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb26c105d10 0x7fb26c10c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:48.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:48.784+0000 7fb271200640 1 --2- 172.21.15.5:0/1361030710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb26c105350 0x7fb26c1057d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:48.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:48.784+0000 7fb271200640 1 --2- 172.21.15.5:0/1361030710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb26c104160 0x7fb26c104580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:48.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:48.784+0000 7fb271200640 1 -- 172.21.15.5:0/1361030710 >> 172.21.15.5:0/1361030710 conn(0x7fb26c0ff6d0 msgr2=0x7fb26c101b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:48.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:48.784+0000 7fb271200640 1 -- 172.21.15.5:0/1361030710 shutdown_connections 2024-04-24T06:40:48.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:48.784+0000 7fb271200640 1 -- 172.21.15.5:0/1361030710 wait complete. 2024-04-24T06:40:48.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:48.785+0000 7fb271200640 1 Processor -- start 2024-04-24T06:40:48.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:48.785+0000 7fb271200640 1 -- start start 2024-04-24T06:40:48.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:48.785+0000 7fb271200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb26c104160 0x7fb26c19a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:48.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:48.785+0000 7fb271200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb26c105350 0x7fb26c19a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:48.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:48.785+0000 7fb271200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb26c105d10 0x7fb26c1a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:48.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:48.785+0000 7fb271200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb26c10fdf0 con 0x7fb26c105350 2024-04-24T06:40:48.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:48.785+0000 7fb271200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb26c10fc70 con 0x7fb26c105d10 2024-04-24T06:40:48.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:48.785+0000 7fb271200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb26c10ff70 con 0x7fb26c104160 2024-04-24T06:40:48.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:48.786+0000 7fb26b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb26c104160 0x7fb26c19a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:48.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:48.786+0000 7fb26b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb26c104160 0x7fb26c19a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:43050/0 (socket says 172.21.15.5:43050) 2024-04-24T06:40:48.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:48.786+0000 7fb26b400640 1 -- 172.21.15.5:0/1316950573 learned_addr learned my addr 172.21.15.5:0/1316950573 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:40:48.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:48.786+0000 7fb26aa00640 1 --2- 172.21.15.5:0/1316950573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb26c105350 0x7fb26c19a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:48.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:48.786+0000 7fb26be00640 1 --2- 172.21.15.5:0/1316950573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb26c105d10 0x7fb26c1a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:48.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:48.786+0000 7fb26b400640 1 -- 172.21.15.5:0/1316950573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb26c105d10 msgr2=0x7fb26c1a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:48.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:48.786+0000 7fb26b400640 1 --2- 172.21.15.5:0/1316950573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb26c105d10 0x7fb26c1a19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:48.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:48.786+0000 7fb26b400640 1 -- 172.21.15.5:0/1316950573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb26c105350 msgr2=0x7fb26c19a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:48.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:48.786+0000 7fb26b400640 1 --2- 172.21.15.5:0/1316950573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb26c105350 0x7fb26c19a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:48.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:48.786+0000 7fb26b400640 1 -- 172.21.15.5:0/1316950573 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb26c1a2010 con 0x7fb26c104160 2024-04-24T06:40:48.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:48.786+0000 7fb26b400640 1 --2- 172.21.15.5:0/1316950573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb26c104160 0x7fb26c19a3f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb25400e9c0 tx=0x7fb25400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:48.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:48.787+0000 7fb263e00640 1 -- 172.21.15.5:0/1316950573 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25400cde0 con 0x7fb26c104160 2024-04-24T06:40:48.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:48.787+0000 7fb271200640 1 -- 172.21.15.5:0/1316950573 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb26c1a2300 con 0x7fb26c104160 2024-04-24T06:40:48.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:48.787+0000 7fb263e00640 1 -- 172.21.15.5:0/1316950573 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb254010910 con 0x7fb26c104160 2024-04-24T06:40:48.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:48.787+0000 7fb263e00640 1 -- 172.21.15.5:0/1316950573 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb254018a10 con 0x7fb26c104160 2024-04-24T06:40:48.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:48.787+0000 7fb271200640 1 -- 172.21.15.5:0/1316950573 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb26c1a28a0 con 0x7fb26c104160 2024-04-24T06:40:48.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:48.788+0000 7fb271200640 1 -- 172.21.15.5:0/1316950573 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb26c10b500 con 0x7fb26c104160 2024-04-24T06:40:48.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:48.790+0000 7fb263e00640 1 -- 172.21.15.5:0/1316950573 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fb25401f020 con 0x7fb26c104160 2024-04-24T06:40:48.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:48.791+0000 7fb263e00640 1 --2- 172.21.15.5:0/1316950573 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fb2500a3ba0 0x7fb2500a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:48.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:48.791+0000 7fb263e00640 1 -- 172.21.15.5:0/1316950573 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fb2540d01f0 con 0x7fb26c104160 2024-04-24T06:40:48.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:48.791+0000 7fb26aa00640 1 --2- 172.21.15.5:0/1316950573 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fb2500a3ba0 0x7fb2500a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:48.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:48.792+0000 7fb26aa00640 1 --2- 172.21.15.5:0/1316950573 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fb2500a3ba0 0x7fb2500a6060 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb26c19b900 tx=0x7fb25c000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:48.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:48.795+0000 7fb263e00640 1 -- 172.21.15.5:0/1316950573 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb25409cfd0 con 0x7fb26c104160 2024-04-24T06:40:49.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:48 smithi005 ceph-mon[27798]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:40:49.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:48 smithi005 ceph-mon[27798]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi037=b;smithi037=e;smithi037=h;smithi037=k;smithi137=c;smithi137=f;smithi137=i;smithi137=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:40:49.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:48 smithi005 ceph-mon[27798]: Saving service mds.all spec with placement smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi037=b;smithi037=e;smithi037=h;smithi037=k;smithi137=c;smithi137=f;smithi137=i;smithi137=l;count:12 2024-04-24T06:40:49.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:48 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:49.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:48 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:40:49.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.010+0000 7fb271200640 1 -- 172.21.15.5:0/1316950573 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb26c062c00 con 0x7fb26c104160 2024-04-24T06:40:49.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.011+0000 7fb263e00640 1 -- 172.21.15.5:0/1316950573 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18325 (secure 0 0 0) 0x7fb2540bb830 con 0x7fb26c104160 2024-04-24T06:40:49.013 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:40:49.013 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":82,"fsid":"43e9bad0-0204-11ef-bc93-c7b262605968","created":"2024-04-24T06:32:26.340555+0000","modified":"2024-04-24T06:40:37.808668+0000","last_up_change":"2024-04-24T06:40:36.790934+0000","last_in_change":"2024-04-24T06:40:17.803257+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:36:27.320342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c2e974c-745c-4dd5-8c3d-8e8da2eb1ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6803","nonce":2155858515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6805","nonce":2155858515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6809","nonce":2155858515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6807","nonce":2155858515}]},"public_addr":"172.21.15.5:6803/2155858515","cluster_addr":"172.21.15.5:6805/2155858515","heartbeat_back_addr":"172.21.15.5:6809/2155858515","heartbeat_front_addr":"172.21.15.5:6807/2155858515","state":["exists","up"]},{"osd":1,"uuid":"50467744-db20-4721-bf2d-1c5db938297b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6801","nonce":1302362650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6803","nonce":1302362650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6807","nonce":1302362650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6805","nonce":1302362650}]},"public_addr":"172.21.15.37:6801/1302362650","cluster_addr":"172.21.15.37:6803/1302362650","heartbeat_back_addr":"172.21.15.37:6807/1302362650","heartbeat_front_addr":"172.21.15.37:6805/1302362650","state":["exists","up"]},{"osd":2,"uuid":"0e190b15-d04c-4288-8d48-ecf6664392fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6801","nonce":2885027256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6803","nonce":2885027256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6807","nonce":2885027256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6805","nonce":2885027256}]},"public_addr":"172.21.15.137:6801/2885027256","cluster_addr":"172.21.15.137:6803/2885027256","heartbeat_back_addr":"172.21.15.137:6807/2885027256","heartbeat_front_addr":"172.21.15.137:6805/2885027256","state":["exists","up"]},{"osd":3,"uuid":"e05592ba-d96a-472a-9f63-09a535d04dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6811","nonce":3701747260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6813","nonce":3701747260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6817","nonce":3701747260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6815","nonce":3701747260}]},"public_addr":"172.21.15.5:6811/3701747260","cluster_addr":"172.21.15.5:6813/3701747260","heartbeat_back_addr":"172.21.15.5:6817/3701747260","heartbeat_front_addr":"172.21.15.5:6815/3701747260","state":["exists","up"]},{"osd":4,"uuid":"c3f36e51-7a3b-4dd9-af4b-a10558830c4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6809","nonce":759253844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6811","nonce":759253844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6815","nonce":759253844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6813","nonce":759253844}]},"public_addr":"172.21.15.37:6809/759253844","cluster_addr":"172.21.15.37:6811/759253844","heartbeat_back_addr":"172.21.15.37:6815/759253844","heartbeat_front_addr":"172.21.15.37:6813/759253844","state":["exists","up"]},{"osd":5,"uuid":"97a0500c-cbb8-4ff5-b476-8634b26b3821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6809","nonce":3771921560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6811","nonce":3771921560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6815","nonce":3771921560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6813","nonce":3771921560}]},"public_addr":"172.21.15.137:6809/3771921560","cluster_addr":"172.21.15.137:6811/3771921560","heartbeat_back_addr":"172.21.15.137:6815/3771921560","heartbeat_front_addr":"172.21.15.137:6813/3771921560","state":["exists","up"]},{"osd":6,"uuid":"33a33d62-bbb3-4ed9-834e-19ae93a97670","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6819","nonce":2064636084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6821","nonce":2064636084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6825","nonce":2064636084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6823","nonce":2064636084}]},"public_addr":"172.21.15.5:6819/2064636084","cluster_addr":"172.21.15.5:6821/2064636084","heartbeat_back_addr":"172.21.15.5:6825/2064636084","heartbeat_front_addr":"172.21.15.5:6823/2064636084","state":["exists","up"]},{"osd":7,"uuid":"c0f4e036-4717-4514-bb59-ab11fbaab063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6817","nonce":2952750659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6819","nonce":2952750659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6823","nonce":2952750659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6821","nonce":2952750659}]},"public_addr":"172.21.15.37:6817/2952750659","cluster_addr":"172.21.15.37:6819/2952750659","heartbeat_back_addr":"172.21.15.37:6823/2952750659","heartbeat_front_addr":"172.21.15.37:6821/2952750659","state":["exists","up"]},{"osd":8,"uuid":"5391fd87-44ea-440f-80da-c54d0327c04c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6817","nonce":29724945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6819","nonce":29724945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6823","nonce":29724945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6821","nonce":29724945}]},"public_addr":"172.21.15.137:6817/29724945","cluster_addr":"172.21.15.137:6819/29724945","heartbeat_back_addr":"172.21.15.137:6823/29724945","heartbeat_front_addr":"172.21.15.137:6821/29724945","state":["exists","up"]},{"osd":9,"uuid":"d94c9dbc-f43d-4237-ac7f-47a6a5ede1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6827","nonce":2257102107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6829","nonce":2257102107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6833","nonce":2257102107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6831","nonce":2257102107}]},"public_addr":"172.21.15.5:6827/2257102107","cluster_addr":"172.21.15.5:6829/2257102107","heartbeat_back_addr":"172.21.15.5:6833/2257102107","heartbeat_front_addr":"172.21.15.5:6831/2257102107","state":["exists","up"]},{"osd":10,"uuid":"e2455332-2a5e-44b1-85d3-8e47da399f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6825","nonce":2451570637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6827","nonce":2451570637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6831","nonce":2451570637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6829","nonce":2451570637}]},"public_addr":"172.21.15.37:6825/2451570637","cluster_addr":"172.21.15.37:6827/2451570637","heartbeat_back_addr":"172.21.15.37:6831/2451570637","heartbeat_front_addr":"172.21.15.37:6829/2451570637","state":["exists","up"]},{"osd":11,"uuid":"49d9396b-704a-4cef-8210-75d30c28e924","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6825","nonce":90600596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6827","nonce":90600596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6831","nonce":90600596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6829","nonce":90600596}]},"public_addr":"172.21.15.137:6825/90600596","cluster_addr":"172.21.15.137:6827/90600596","heartbeat_back_addr":"172.21.15.137:6831/90600596","heartbeat_front_addr":"172.21.15.137:6829/90600596","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:35:35.591619+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:00.568817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:24.195643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:56.451724+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:22.109293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:46.161749+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:19.278305+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:45.620256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:10.041165+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:43.813697+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:10.736026+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:35.384377+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/23307164":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6801/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:0/230785941":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/2940023597":"2024-04-25T06:33:07.040369+0000","172.21.15.5:6801/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3886613904":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/4199165437":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3459593499":"2024-04-25T06:33:41.161852+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:40:49.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.013+0000 7fb271200640 1 -- 172.21.15.5:0/1316950573 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fb2500a3ba0 msgr2=0x7fb2500a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:49.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.013+0000 7fb271200640 1 --2- 172.21.15.5:0/1316950573 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fb2500a3ba0 0x7fb2500a6060 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb26c19b900 tx=0x7fb25c000f40 comp rx=0 tx=0).stop 2024-04-24T06:40:49.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.013+0000 7fb271200640 1 -- 172.21.15.5:0/1316950573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb26c104160 msgr2=0x7fb26c19a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:49.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.013+0000 7fb271200640 1 --2- 172.21.15.5:0/1316950573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb26c104160 0x7fb26c19a3f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb25400e9c0 tx=0x7fb25400ee80 comp rx=0 tx=0).stop 2024-04-24T06:40:49.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.013+0000 7fb271200640 1 -- 172.21.15.5:0/1316950573 shutdown_connections 2024-04-24T06:40:49.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.013+0000 7fb271200640 1 --2- 172.21.15.5:0/1316950573 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fb2500a3ba0 0x7fb2500a6060 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:49.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.013+0000 7fb271200640 1 --2- 172.21.15.5:0/1316950573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb26c105d10 0x7fb26c1a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:49.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.013+0000 7fb271200640 1 --2- 172.21.15.5:0/1316950573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb26c105350 0x7fb26c19a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:49.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.013+0000 7fb271200640 1 --2- 172.21.15.5:0/1316950573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb26c104160 0x7fb26c19a3f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:49.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.013+0000 7fb271200640 1 -- 172.21.15.5:0/1316950573 >> 172.21.15.5:0/1316950573 conn(0x7fb26c0ff6d0 msgr2=0x7fb26c1086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:49.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.014+0000 7fb271200640 1 -- 172.21.15.5:0/1316950573 shutdown_connections 2024-04-24T06:40:49.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.014+0000 7fb271200640 1 -- 172.21.15.5:0/1316950573 wait complete. 2024-04-24T06:40:49.034 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:36:27.320342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:40:49.034 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:40:49.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.294+0000 7f24b7e00640 1 -- 172.21.15.5:0/3279103289 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24b8105270 msgr2=0x7f24b81056f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:49.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.294+0000 7f24b7e00640 1 --2- 172.21.15.5:0/3279103289 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24b8105270 0x7f24b81056f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f24a8009ca0 tx=0x7f24a802a8c0 comp rx=0 tx=0).stop 2024-04-24T06:40:49.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.294+0000 7f24b7e00640 1 -- 172.21.15.5:0/3279103289 shutdown_connections 2024-04-24T06:40:49.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.294+0000 7f24b7e00640 1 --2- 172.21.15.5:0/3279103289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24b8105c30 0x7f24b810c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:49.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.294+0000 7f24b7e00640 1 --2- 172.21.15.5:0/3279103289 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24b8105270 0x7f24b81056f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:49.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.294+0000 7f24b7e00640 1 --2- 172.21.15.5:0/3279103289 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24b8104080 0x7f24b81044a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:49.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.294+0000 7f24b7e00640 1 -- 172.21.15.5:0/3279103289 >> 172.21.15.5:0/3279103289 conn(0x7f24b80ff610 msgr2=0x7f24b8101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:49.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.294+0000 7f24b7e00640 1 -- 172.21.15.5:0/3279103289 shutdown_connections 2024-04-24T06:40:49.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.294+0000 7f24b7e00640 1 -- 172.21.15.5:0/3279103289 wait complete. 2024-04-24T06:40:49.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.295+0000 7f24b7e00640 1 Processor -- start 2024-04-24T06:40:49.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.295+0000 7f24b7e00640 1 -- start start 2024-04-24T06:40:49.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.296+0000 7f24b7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24b8104080 0x7f24b819a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:49.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.296+0000 7f24b7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24b8105270 0x7f24b819a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:49.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.296+0000 7f24b7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24b8105c30 0x7f24b81a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:49.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.296+0000 7f24b7e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f24b810fd60 con 0x7f24b8104080 2024-04-24T06:40:49.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.296+0000 7f24b7e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f24b810fbe0 con 0x7f24b8105270 2024-04-24T06:40:49.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.296+0000 7f24b7e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f24b810fee0 con 0x7f24b8105c30 2024-04-24T06:40:49.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.296+0000 7f24b6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24b8104080 0x7f24b819a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:49.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.296+0000 7f24b7400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24b8105c30 0x7f24b81a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:49.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.296+0000 7f24b6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24b8104080 0x7f24b819a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48642/0 (socket says 172.21.15.5:48642) 2024-04-24T06:40:49.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.296+0000 7f24b6a00640 1 -- 172.21.15.5:0/2423165346 learned_addr learned my addr 172.21.15.5:0/2423165346 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:40:49.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.296+0000 7f24b6000640 1 --2- 172.21.15.5:0/2423165346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24b8105270 0x7f24b819a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:49.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.296+0000 7f24b6a00640 1 -- 172.21.15.5:0/2423165346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24b8105c30 msgr2=0x7f24b81a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:49.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.296+0000 7f24b6a00640 1 --2- 172.21.15.5:0/2423165346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24b8105c30 0x7f24b81a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:49.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.296+0000 7f24b6a00640 1 -- 172.21.15.5:0/2423165346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24b8105270 msgr2=0x7f24b819a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:49.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.296+0000 7f24b6a00640 1 --2- 172.21.15.5:0/2423165346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24b8105270 0x7f24b819a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:49.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.296+0000 7f24b6a00640 1 -- 172.21.15.5:0/2423165346 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24b81a1f80 con 0x7f24b8104080 2024-04-24T06:40:49.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.296+0000 7f24b7400640 1 --2- 172.21.15.5:0/2423165346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24b8105c30 0x7f24b81a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:40:49.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.297+0000 7f24b6a00640 1 --2- 172.21.15.5:0/2423165346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24b8104080 0x7f24b819a360 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f24a000e9c0 tx=0x7f24a000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:49.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.297+0000 7f24bca00640 1 -- 172.21.15.5:0/2423165346 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a000cd80 con 0x7f24b8104080 2024-04-24T06:40:49.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.297+0000 7f24bca00640 1 -- 172.21.15.5:0/2423165346 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24a00035f0 con 0x7f24b8104080 2024-04-24T06:40:49.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.297+0000 7f24b7e00640 1 -- 172.21.15.5:0/2423165346 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24b81a2270 con 0x7f24b8104080 2024-04-24T06:40:49.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.297+0000 7f24bca00640 1 -- 172.21.15.5:0/2423165346 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a0010690 con 0x7f24b8104080 2024-04-24T06:40:49.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.297+0000 7f24b7e00640 1 -- 172.21.15.5:0/2423165346 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24b81a2740 con 0x7f24b8104080 2024-04-24T06:40:49.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.298+0000 7f24b7e00640 1 -- 172.21.15.5:0/2423165346 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24b810b3a0 con 0x7f24b8104080 2024-04-24T06:40:49.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.301+0000 7f24bca00640 1 -- 172.21.15.5:0/2423165346 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f24a00037a0 con 0x7f24b8104080 2024-04-24T06:40:49.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.303+0000 7f24bca00640 1 --2- 172.21.15.5:0/2423165346 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f249c0a3bb0 0x7f249c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:49.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.303+0000 7f24bca00640 1 -- 172.21.15.5:0/2423165346 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f24a00d0360 con 0x7f24b8104080 2024-04-24T06:40:49.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.303+0000 7f24b6000640 1 --2- 172.21.15.5:0/2423165346 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f249c0a3bb0 0x7f249c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:49.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.304+0000 7f24b6000640 1 --2- 172.21.15.5:0/2423165346 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f249c0a3bb0 0x7f249c0a6070 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f24a8005e30 tx=0x7f24a8005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:49.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.307+0000 7f24bca00640 1 -- 172.21.15.5:0/2423165346 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f24a009d140 con 0x7f24b8104080 2024-04-24T06:40:49.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.514+0000 7f24b7e00640 1 -- 172.21.15.5:0/2423165346 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f24b8062c00 con 0x7f24b8104080 2024-04-24T06:40:49.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.515+0000 7f24bca00640 1 -- 172.21.15.5:0/2423165346 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f24a009ca40 con 0x7f24b8104080 2024-04-24T06:40:49.516 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-24T06:40:49.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.517+0000 7f24b7e00640 1 -- 172.21.15.5:0/2423165346 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f249c0a3bb0 msgr2=0x7f249c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:49.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.517+0000 7f24b7e00640 1 --2- 172.21.15.5:0/2423165346 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f249c0a3bb0 0x7f249c0a6070 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f24a8005e30 tx=0x7f24a8005f50 comp rx=0 tx=0).stop 2024-04-24T06:40:49.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.517+0000 7f24b7e00640 1 -- 172.21.15.5:0/2423165346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24b8104080 msgr2=0x7f24b819a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:49.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.517+0000 7f24b7e00640 1 --2- 172.21.15.5:0/2423165346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24b8104080 0x7f24b819a360 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f24a000e9c0 tx=0x7f24a000ee80 comp rx=0 tx=0).stop 2024-04-24T06:40:49.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.518+0000 7f24b7e00640 1 -- 172.21.15.5:0/2423165346 shutdown_connections 2024-04-24T06:40:49.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.518+0000 7f24b7e00640 1 --2- 172.21.15.5:0/2423165346 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f249c0a3bb0 0x7f249c0a6070 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:49.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.518+0000 7f24b7e00640 1 --2- 172.21.15.5:0/2423165346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24b8105c30 0x7f24b81a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:49.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.518+0000 7f24b7e00640 1 --2- 172.21.15.5:0/2423165346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24b8105270 0x7f24b819a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:49.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.518+0000 7f24b7e00640 1 --2- 172.21.15.5:0/2423165346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24b8104080 0x7f24b819a360 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:49.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.518+0000 7f24b7e00640 1 -- 172.21.15.5:0/2423165346 >> 172.21.15.5:0/2423165346 conn(0x7f24b80ff610 msgr2=0x7f24b8108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:49.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.518+0000 7f24b7e00640 1 -- 172.21.15.5:0/2423165346 shutdown_connections 2024-04-24T06:40:49.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.518+0000 7f24b7e00640 1 -- 172.21.15.5:0/2423165346 wait complete. 2024-04-24T06:40:49.540 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:40:49.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:49 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:40:49.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:49 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:40:49.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:49 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:49.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:49 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T06:40:49.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:49 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T06:40:49.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:49 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:40:49.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:49 smithi137 ceph-mon[33017]: Deploying daemon mds.l on smithi137 2024-04-24T06:40:49.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:49 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/1316950573' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:40:49.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:49 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/2423165346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T06:40:49.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.736+0000 7f5e6f200640 1 -- 172.21.15.5:0/3999576276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e7010a100 msgr2=0x7f5e70110990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:49.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.736+0000 7f5e6f200640 1 --2- 172.21.15.5:0/3999576276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e7010a100 0x7f5e70110990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5e68009be0 tx=0x7f5e6802a6c0 comp rx=0 tx=0).stop 2024-04-24T06:40:49.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.736+0000 7f5e6f200640 1 -- 172.21.15.5:0/3999576276 shutdown_connections 2024-04-24T06:40:49.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.736+0000 7f5e6f200640 1 --2- 172.21.15.5:0/3999576276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e7010a100 0x7f5e70110990 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:49.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.736+0000 7f5e6f200640 1 --2- 172.21.15.5:0/3999576276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e70109740 0x7f5e70109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:49.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.736+0000 7f5e6f200640 1 --2- 172.21.15.5:0/3999576276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e701085a0 0x7f5e701089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:49.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.736+0000 7f5e6f200640 1 -- 172.21.15.5:0/3999576276 >> 172.21.15.5:0/3999576276 conn(0x7f5e700774f0 msgr2=0x7f5e70079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:49.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.736+0000 7f5e6f200640 1 -- 172.21.15.5:0/3999576276 shutdown_connections 2024-04-24T06:40:49.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.736+0000 7f5e6f200640 1 -- 172.21.15.5:0/3999576276 wait complete. 2024-04-24T06:40:49.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.737+0000 7f5e6f200640 1 Processor -- start 2024-04-24T06:40:49.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.737+0000 7f5e6f200640 1 -- start start 2024-04-24T06:40:49.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.738+0000 7f5e6f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e701085a0 0x7f5e7019e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:49.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.738+0000 7f5e6f200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e70109740 0x7f5e7019ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:49.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.738+0000 7f5e6f200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e7010a100 0x7f5e701a5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:49.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.738+0000 7f5e6f200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5e701141e0 con 0x7f5e701085a0 2024-04-24T06:40:49.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.738+0000 7f5e6f200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5e70114060 con 0x7f5e70109740 2024-04-24T06:40:49.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.738+0000 7f5e6f200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5e70114360 con 0x7f5e7010a100 2024-04-24T06:40:49.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.738+0000 7f5e65400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e70109740 0x7f5e7019ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:49.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.738+0000 7f5e65400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e70109740 0x7f5e7019ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:51860/0 (socket says 172.21.15.5:51860) 2024-04-24T06:40:49.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.738+0000 7f5e65400640 1 -- 172.21.15.5:0/1343784812 learned_addr learned my addr 172.21.15.5:0/1343784812 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:40:49.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.738+0000 7f5e6de00640 1 --2- 172.21.15.5:0/1343784812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e701085a0 0x7f5e7019e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:49.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.738+0000 7f5e6e800640 1 --2- 172.21.15.5:0/1343784812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e7010a100 0x7f5e701a5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:49.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.739+0000 7f5e65400640 1 -- 172.21.15.5:0/1343784812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e7010a100 msgr2=0x7f5e701a5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:49.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.739+0000 7f5e65400640 1 --2- 172.21.15.5:0/1343784812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e7010a100 0x7f5e701a5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:49.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.739+0000 7f5e65400640 1 -- 172.21.15.5:0/1343784812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e701085a0 msgr2=0x7f5e7019e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:49.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.739+0000 7f5e65400640 1 --2- 172.21.15.5:0/1343784812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e701085a0 0x7f5e7019e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:49.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.739+0000 7f5e65400640 1 -- 172.21.15.5:0/1343784812 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e701a63f0 con 0x7f5e70109740 2024-04-24T06:40:49.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.739+0000 7f5e6de00640 1 --2- 172.21.15.5:0/1343784812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e701085a0 0x7f5e7019e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:40:49.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.739+0000 7f5e65400640 1 --2- 172.21.15.5:0/1343784812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e70109740 0x7f5e7019ed10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5e5800e9f0 tx=0x7f5e5800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:49.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.739+0000 7f5e67400640 1 -- 172.21.15.5:0/1343784812 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5800ccc0 con 0x7f5e70109740 2024-04-24T06:40:49.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.739+0000 7f5e6f200640 1 -- 172.21.15.5:0/1343784812 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e701a66e0 con 0x7f5e70109740 2024-04-24T06:40:49.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.739+0000 7f5e67400640 1 -- 172.21.15.5:0/1343784812 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e58010910 con 0x7f5e70109740 2024-04-24T06:40:49.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.739+0000 7f5e67400640 1 -- 172.21.15.5:0/1343784812 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e580188a0 con 0x7f5e70109740 2024-04-24T06:40:49.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.739+0000 7f5e6f200640 1 -- 172.21.15.5:0/1343784812 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e701a6c30 con 0x7f5e70109740 2024-04-24T06:40:49.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.740+0000 7f5e6f200640 1 -- 172.21.15.5:0/1343784812 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e7010f7a0 con 0x7f5e70109740 2024-04-24T06:40:49.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.743+0000 7f5e67400640 1 -- 172.21.15.5:0/1343784812 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f5e58010ac0 con 0x7f5e70109740 2024-04-24T06:40:49.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.744+0000 7f5e67400640 1 --2- 172.21.15.5:0/1343784812 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f5e540a3ba0 0x7f5e540a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:49.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.744+0000 7f5e67400640 1 -- 172.21.15.5:0/1343784812 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f5e580d0440 con 0x7f5e70109740 2024-04-24T06:40:49.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.744+0000 7f5e6de00640 1 --2- 172.21.15.5:0/1343784812 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f5e540a3ba0 0x7f5e540a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:49.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.745+0000 7f5e6de00640 1 --2- 172.21.15.5:0/1343784812 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f5e540a3ba0 0x7f5e540a6060 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5e701095a0 tx=0x7f5e60009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:49.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.747+0000 7f5e67400640 1 -- 172.21.15.5:0/1343784812 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5e5809d220 con 0x7f5e70109740 2024-04-24T06:40:50.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:49.999+0000 7f5e6f200640 1 -- 172.21.15.5:0/1343784812 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e7002d620 con 0x7f5e70109740 2024-04-24T06:40:50.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:50.000+0000 7f5e67400640 1 -- 172.21.15.5:0/1343784812 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f5e5809cb20 con 0x7f5e70109740 2024-04-24T06:40:50.001 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:40:50.002 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-24T06:40:50.002 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 1 2024-04-24T06:40:50.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:50.003+0000 7f5e6f200640 1 -- 172.21.15.5:0/1343784812 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f5e540a3ba0 msgr2=0x7f5e540a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:50.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:50.003+0000 7f5e6f200640 1 --2- 172.21.15.5:0/1343784812 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f5e540a3ba0 0x7f5e540a6060 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5e701095a0 tx=0x7f5e60009260 comp rx=0 tx=0).stop 2024-04-24T06:40:50.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:50.003+0000 7f5e6f200640 1 -- 172.21.15.5:0/1343784812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e70109740 msgr2=0x7f5e7019ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:50.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:50.003+0000 7f5e6f200640 1 --2- 172.21.15.5:0/1343784812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e70109740 0x7f5e7019ed10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5e5800e9f0 tx=0x7f5e5800edf0 comp rx=0 tx=0).stop 2024-04-24T06:40:50.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:50.003+0000 7f5e6f200640 1 -- 172.21.15.5:0/1343784812 shutdown_connections 2024-04-24T06:40:50.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:50.003+0000 7f5e6f200640 1 --2- 172.21.15.5:0/1343784812 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f5e540a3ba0 0x7f5e540a6060 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:50.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:50.003+0000 7f5e6f200640 1 --2- 172.21.15.5:0/1343784812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e7010a100 0x7f5e701a5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:50.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:50.003+0000 7f5e6f200640 1 --2- 172.21.15.5:0/1343784812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e70109740 0x7f5e7019ed10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:50.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:50.003+0000 7f5e6f200640 1 --2- 172.21.15.5:0/1343784812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e701085a0 0x7f5e7019e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:50.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:50.003+0000 7f5e6f200640 1 -- 172.21.15.5:0/1343784812 >> 172.21.15.5:0/1343784812 conn(0x7f5e700774f0 msgr2=0x7f5e7010c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:50.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:50.004+0000 7f5e6f200640 1 -- 172.21.15.5:0/1343784812 shutdown_connections 2024-04-24T06:40:50.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:50.004+0000 7f5e6f200640 1 -- 172.21.15.5:0/1343784812 wait complete. 2024-04-24T06:40:50.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:49 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:40:50.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:49 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:40:50.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:49 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:50.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:49 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T06:40:50.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:49 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T06:40:50.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:49 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:40:50.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:49 smithi005 ceph-mon[27798]: Deploying daemon mds.l on smithi137 2024-04-24T06:40:50.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:49 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1316950573' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:40:50.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:49 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/2423165346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T06:40:50.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:49 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:40:50.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:49 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:40:50.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:49 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:50.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:49 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T06:40:50.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:49 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T06:40:50.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:49 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:40:50.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:49 smithi037 ceph-mon[32864]: Deploying daemon mds.l on smithi137 2024-04-24T06:40:50.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:49 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/1316950573' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:40:50.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:49 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/2423165346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T06:40:51.174 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:50 smithi137 ceph-mon[33017]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:40:51.174 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:50 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/1343784812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:40:51.175 INFO:journalctl@ceph.mds.l.smithi137.stdout:Apr 24 06:40:50 smithi137 systemd[1]: Starting Ceph mds.l for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:40:51.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:50 smithi005 ceph-mon[27798]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:40:51.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:50 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1343784812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:40:51.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:50 smithi037 ceph-mon[32864]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:40:51.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:50 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/1343784812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:40:51.427 INFO:journalctl@ceph.mds.l.smithi137.stdout:Apr 24 06:40:51 smithi137 podman[63127]: 2024-04-24 06:40:51.173297327 +0000 UTC m=+0.113743960 container create 1efb8334f844a2db7cce32fb5355cac7655458b103a29a53c92cd07aa1314a4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-l, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-04-24T06:40:51.427 INFO:journalctl@ceph.mds.l.smithi137.stdout:Apr 24 06:40:51 smithi137 podman[63127]: 2024-04-24 06:40:51.077541068 +0000 UTC m=+0.017987709 image pull b5c3183062a6ce19f361681cc58de6e3ac1de1a54c80e6094ec5695237cd98d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b 2024-04-24T06:40:51.427 INFO:journalctl@ceph.mds.l.smithi137.stdout:Apr 24 06:40:51 smithi137 podman[63127]: 2024-04-24 06:40:51.400587139 +0000 UTC m=+0.341033772 container init 1efb8334f844a2db7cce32fb5355cac7655458b103a29a53c92cd07aa1314a4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-l, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22) 2024-04-24T06:40:51.427 INFO:journalctl@ceph.mds.l.smithi137.stdout:Apr 24 06:40:51 smithi137 podman[63127]: 2024-04-24 06:40:51.403059057 +0000 UTC m=+0.343505691 container start 1efb8334f844a2db7cce32fb5355cac7655458b103a29a53c92cd07aa1314a4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True) 2024-04-24T06:40:51.427 INFO:journalctl@ceph.mds.l.smithi137.stdout:Apr 24 06:40:51 smithi137 ceph-mds[63146]: -- 172.21.15.137:0/2881392199 <== mon.1 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d343815a40 con 0x55d3437e9000 2024-04-24T06:40:51.696 INFO:journalctl@ceph.mds.l.smithi137.stdout:Apr 24 06:40:51 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-l[63142]: starting mds.l at 2024-04-24T06:40:51.696 INFO:journalctl@ceph.mds.l.smithi137.stdout:Apr 24 06:40:51 smithi137 bash[63127]: 1efb8334f844a2db7cce32fb5355cac7655458b103a29a53c92cd07aa1314a4e 2024-04-24T06:40:51.696 INFO:journalctl@ceph.mds.l.smithi137.stdout:Apr 24 06:40:51 smithi137 systemd[1]: Started Ceph mds.l for 43e9bad0-0204-11ef-bc93-c7b262605968. 2024-04-24T06:40:52.026 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:40:52.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.249+0000 7f371e600640 1 -- 172.21.15.5:0/1836678293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3718103080 msgr2=0x7f3718103500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:52.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.249+0000 7f371e600640 1 --2- 172.21.15.5:0/1836678293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3718103080 0x7f3718103500 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3700009900 tx=0x7f370002a3f0 comp rx=0 tx=0).stop 2024-04-24T06:40:52.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.250+0000 7f371e600640 1 -- 172.21.15.5:0/1836678293 shutdown_connections 2024-04-24T06:40:52.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.250+0000 7f371e600640 1 --2- 172.21.15.5:0/1836678293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3718103a40 0x7f371810a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:52.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.250+0000 7f371e600640 1 --2- 172.21.15.5:0/1836678293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3718103080 0x7f3718103500 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:52.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.250+0000 7f371e600640 1 --2- 172.21.15.5:0/1836678293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3718101f30 0x7f3718102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:52.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.250+0000 7f371e600640 1 -- 172.21.15.5:0/1836678293 >> 172.21.15.5:0/1836678293 conn(0x7f37180fd4c0 msgr2=0x7f37180ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:52.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.250+0000 7f371e600640 1 -- 172.21.15.5:0/1836678293 shutdown_connections 2024-04-24T06:40:52.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.250+0000 7f371e600640 1 -- 172.21.15.5:0/1836678293 wait complete. 2024-04-24T06:40:52.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.251+0000 7f371e600640 1 Processor -- start 2024-04-24T06:40:52.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.251+0000 7f371e600640 1 -- start start 2024-04-24T06:40:52.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.251+0000 7f371e600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3718101f30 0x7f37181982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:52.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.252+0000 7f371e600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3718103080 0x7f37181987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:52.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.252+0000 7f371e600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3718103a40 0x7f371819f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:52.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.252+0000 7f371e600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f371810dc10 con 0x7f3718101f30 2024-04-24T06:40:52.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.252+0000 7f371e600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f371810da90 con 0x7f3718103080 2024-04-24T06:40:52.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.252+0000 7f371e600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f371810dd90 con 0x7f3718103a40 2024-04-24T06:40:52.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.252+0000 7f371d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3718101f30 0x7f37181982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:52.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.252+0000 7f371d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3718101f30 0x7f37181982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48676/0 (socket says 172.21.15.5:48676) 2024-04-24T06:40:52.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.252+0000 7f371d200640 1 -- 172.21.15.5:0/132395356 learned_addr learned my addr 172.21.15.5:0/132395356 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:40:52.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.252+0000 7f371dc00640 1 --2- 172.21.15.5:0/132395356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3718103a40 0x7f371819f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:52.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.252+0000 7f3717e00640 1 --2- 172.21.15.5:0/132395356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3718103080 0x7f37181987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:52.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.252+0000 7f371d200640 1 -- 172.21.15.5:0/132395356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3718103a40 msgr2=0x7f371819f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:52.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.252+0000 7f371d200640 1 --2- 172.21.15.5:0/132395356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3718103a40 0x7f371819f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:52.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.252+0000 7f371d200640 1 -- 172.21.15.5:0/132395356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3718103080 msgr2=0x7f37181987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:52.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.252+0000 7f371d200640 1 --2- 172.21.15.5:0/132395356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3718103080 0x7f37181987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:52.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.252+0000 7f371d200640 1 -- 172.21.15.5:0/132395356 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f371819fed0 con 0x7f3718101f30 2024-04-24T06:40:52.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.253+0000 7f371d200640 1 --2- 172.21.15.5:0/132395356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3718101f30 0x7f37181982b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f370800e9f0 tx=0x7f370800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:52.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.253+0000 7f3715600640 1 -- 172.21.15.5:0/132395356 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370800cd80 con 0x7f3718101f30 2024-04-24T06:40:52.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.253+0000 7f3715600640 1 -- 172.21.15.5:0/132395356 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3708010910 con 0x7f3718101f30 2024-04-24T06:40:52.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.253+0000 7f371e600640 1 -- 172.21.15.5:0/132395356 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37181a01c0 con 0x7f3718101f30 2024-04-24T06:40:52.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.253+0000 7f3715600640 1 -- 172.21.15.5:0/132395356 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37080188f0 con 0x7f3718101f30 2024-04-24T06:40:52.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.253+0000 7f371e600640 1 -- 172.21.15.5:0/132395356 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37181a06e0 con 0x7f3718101f30 2024-04-24T06:40:52.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.254+0000 7f371e600640 1 -- 172.21.15.5:0/132395356 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3718109260 con 0x7f3718101f30 2024-04-24T06:40:52.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.256+0000 7f3715600640 1 -- 172.21.15.5:0/132395356 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f3708010ac0 con 0x7f3718101f30 2024-04-24T06:40:52.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.258+0000 7f3715600640 1 --2- 172.21.15.5:0/132395356 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f36fc0a3ba0 0x7f36fc0a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:52.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.258+0000 7f3715600640 1 -- 172.21.15.5:0/132395356 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f37080d0390 con 0x7f3718101f30 2024-04-24T06:40:52.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.258+0000 7f3717e00640 1 --2- 172.21.15.5:0/132395356 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f36fc0a3ba0 0x7f36fc0a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:52.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.259+0000 7f3717e00640 1 --2- 172.21.15.5:0/132395356 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f36fc0a3ba0 0x7f36fc0a6060 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f370002a9e0 tx=0x7f37000315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:52.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.263+0000 7f3715600640 1 -- 172.21.15.5:0/132395356 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f370809cb20 con 0x7f3718101f30 2024-04-24T06:40:52.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.513+0000 7f371e600640 1 -- 172.21.15.5:0/132395356 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f371802d620 con 0x7f3718101f30 2024-04-24T06:40:52.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.514+0000 7f3715600640 1 -- 172.21.15.5:0/132395356 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1393 (secure 0 0 0) 0x7f37080a0e20 con 0x7f3718101f30 2024-04-24T06:40:52.515 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:40:52.515 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-24T06:40:52.516 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 2 2024-04-24T06:40:52.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.516+0000 7f371e600640 1 -- 172.21.15.5:0/132395356 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f36fc0a3ba0 msgr2=0x7f36fc0a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:52.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.516+0000 7f371e600640 1 --2- 172.21.15.5:0/132395356 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f36fc0a3ba0 0x7f36fc0a6060 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f370002a9e0 tx=0x7f37000315b0 comp rx=0 tx=0).stop 2024-04-24T06:40:52.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.516+0000 7f371e600640 1 -- 172.21.15.5:0/132395356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3718101f30 msgr2=0x7f37181982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:52.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.516+0000 7f371e600640 1 --2- 172.21.15.5:0/132395356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3718101f30 0x7f37181982b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f370800e9f0 tx=0x7f370800eeb0 comp rx=0 tx=0).stop 2024-04-24T06:40:52.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.516+0000 7f371e600640 1 -- 172.21.15.5:0/132395356 shutdown_connections 2024-04-24T06:40:52.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.516+0000 7f371e600640 1 --2- 172.21.15.5:0/132395356 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f36fc0a3ba0 0x7f36fc0a6060 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:52.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.516+0000 7f371e600640 1 --2- 172.21.15.5:0/132395356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3718103a40 0x7f371819f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:52.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.516+0000 7f371e600640 1 --2- 172.21.15.5:0/132395356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3718103080 0x7f37181987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:52.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.516+0000 7f371e600640 1 --2- 172.21.15.5:0/132395356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3718101f30 0x7f37181982b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:52.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.517+0000 7f371e600640 1 -- 172.21.15.5:0/132395356 >> 172.21.15.5:0/132395356 conn(0x7f37180fd4c0 msgr2=0x7f3718106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:52.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.517+0000 7f371e600640 1 -- 172.21.15.5:0/132395356 shutdown_connections 2024-04-24T06:40:52.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:52.517+0000 7f371e600640 1 -- 172.21.15.5:0/132395356 wait complete. 2024-04-24T06:40:52.663 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:52 smithi037 ceph-mon[32864]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:40:52.663 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:52 smithi037 ceph-mon[32864]: mds.? [v2:172.21.15.137:6832/681129825,v1:172.21.15.137:6833/681129825] up:boot 2024-04-24T06:40:52.664 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:52 smithi037 ceph-mon[32864]: fsmap 1 up:standby 2024-04-24T06:40:52.664 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:52 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T06:40:52.664 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:52 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:52.664 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:52 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:52.664 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:52 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:52.664 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:52 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T06:40:52.664 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:52 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T06:40:52.664 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:52 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:40:52.664 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:52 smithi037 ceph-mon[32864]: Deploying daemon mds.k on smithi037 2024-04-24T06:40:52.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:52 smithi137 ceph-mon[33017]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:40:52.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:52 smithi137 ceph-mon[33017]: mds.? [v2:172.21.15.137:6832/681129825,v1:172.21.15.137:6833/681129825] up:boot 2024-04-24T06:40:52.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:52 smithi137 ceph-mon[33017]: fsmap 1 up:standby 2024-04-24T06:40:52.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:52 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T06:40:52.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:52 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:52.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:52 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:52.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:52 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:52.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:52 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T06:40:52.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:52 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T06:40:52.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:52 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:40:52.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:52 smithi137 ceph-mon[33017]: Deploying daemon mds.k on smithi037 2024-04-24T06:40:52.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:52 smithi005 ceph-mon[27798]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:40:52.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:52 smithi005 ceph-mon[27798]: mds.? [v2:172.21.15.137:6832/681129825,v1:172.21.15.137:6833/681129825] up:boot 2024-04-24T06:40:52.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:52 smithi005 ceph-mon[27798]: fsmap 1 up:standby 2024-04-24T06:40:52.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:52 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T06:40:52.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:52 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:52.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:52 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:52.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:52 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:52.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:52 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T06:40:52.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:52 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T06:40:52.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:52 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:40:52.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:52 smithi005 ceph-mon[27798]: Deploying daemon mds.k on smithi037 2024-04-24T06:40:53.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:53 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/132395356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:40:53.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:53 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/132395356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:40:53.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:53 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/132395356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:40:54.205 INFO:journalctl@ceph.mds.k.smithi037.stdout:Apr 24 06:40:53 smithi037 systemd[1]: Starting Ceph mds.k for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:40:54.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:54 smithi037 ceph-mon[32864]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:40:54.519 INFO:journalctl@ceph.mds.k.smithi037.stdout:Apr 24 06:40:54 smithi037 podman[62381]: 2024-04-24 06:40:54.204398723 +0000 UTC m=+0.114741162 container create 2d48a66b627449c18f74d743cb990ee408062e5b8fdc9497d95fccd9a1150bc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-k, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-24T06:40:54.519 INFO:journalctl@ceph.mds.k.smithi037.stdout:Apr 24 06:40:54 smithi037 podman[62381]: 2024-04-24 06:40:54.107823579 +0000 UTC m=+0.018165963 image pull b5c3183062a6ce19f361681cc58de6e3ac1de1a54c80e6094ec5695237cd98d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b 2024-04-24T06:40:54.538 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:40:54.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:54 smithi005 ceph-mon[27798]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:40:54.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:54.765+0000 7f85c0e00640 1 -- 172.21.15.5:0/1060075818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85bc1020e0 msgr2=0x7f85bc102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:54.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:54.765+0000 7f85c0e00640 1 --2- 172.21.15.5:0/1060075818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85bc1020e0 0x7f85bc102560 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f85a8009900 tx=0x7f85a802a3f0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:54.766+0000 7f85c0e00640 1 -- 172.21.15.5:0/1060075818 shutdown_connections 2024-04-24T06:40:54.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:54.766+0000 7f85c0e00640 1 --2- 172.21.15.5:0/1060075818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85bc102aa0 0x7f85bc1093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:54.766+0000 7f85c0e00640 1 --2- 172.21.15.5:0/1060075818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85bc1020e0 0x7f85bc102560 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:54.766+0000 7f85c0e00640 1 --2- 172.21.15.5:0/1060075818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f85bc100f10 0x7f85bc101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:54.766+0000 7f85c0e00640 1 -- 172.21.15.5:0/1060075818 >> 172.21.15.5:0/1060075818 conn(0x7f85bc0fc4d0 msgr2=0x7f85bc0fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:54.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:54.766+0000 7f85c0e00640 1 -- 172.21.15.5:0/1060075818 shutdown_connections 2024-04-24T06:40:54.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:54.766+0000 7f85c0e00640 1 -- 172.21.15.5:0/1060075818 wait complete. 2024-04-24T06:40:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:54.767+0000 7f85c0e00640 1 Processor -- start 2024-04-24T06:40:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:54.767+0000 7f85c0e00640 1 -- start start 2024-04-24T06:40:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:54.767+0000 7f85c0e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85bc100f10 0x7f85bc19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:54.767+0000 7f85c0e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85bc1020e0 0x7f85bc19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:54.767+0000 7f85c0e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f85bc102aa0 0x7f85bc1a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:54.767+0000 7f85c0e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f85bc10dd60 con 0x7f85bc100f10 2024-04-24T06:40:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:54.767+0000 7f85c0e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f85bc10dbe0 con 0x7f85bc1020e0 2024-04-24T06:40:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:54.767+0000 7f85c0e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f85bc10dee0 con 0x7f85bc102aa0 2024-04-24T06:40:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:54.767+0000 7f85b9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85bc100f10 0x7f85bc19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:54.767+0000 7f85b9400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85bc1020e0 0x7f85bc19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:54.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:54.768+0000 7f85b9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85bc100f10 0x7f85bc19a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57300/0 (socket says 172.21.15.5:57300) 2024-04-24T06:40:54.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:54.768+0000 7f85b9400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85bc1020e0 0x7f85bc19aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:42118/0 (socket says 172.21.15.5:42118) 2024-04-24T06:40:54.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:54.768+0000 7f85ba800640 1 --2- 172.21.15.5:0/977105464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f85bc102aa0 0x7f85bc1a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:54.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:54.768+0000 7f85b9e00640 1 -- 172.21.15.5:0/977105464 learned_addr learned my addr 172.21.15.5:0/977105464 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:40:54.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:54.768+0000 7f85b9400640 1 -- 172.21.15.5:0/977105464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f85bc102aa0 msgr2=0x7f85bc1a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:54.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:54.768+0000 7f85b9400640 1 --2- 172.21.15.5:0/977105464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f85bc102aa0 0x7f85bc1a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:54.768+0000 7f85b9400640 1 -- 172.21.15.5:0/977105464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85bc100f10 msgr2=0x7f85bc19a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:54.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:54.768+0000 7f85b9400640 1 --2- 172.21.15.5:0/977105464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85bc100f10 0x7f85bc19a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:54.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:54.768+0000 7f85b9400640 1 -- 172.21.15.5:0/977105464 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85bc1a2160 con 0x7f85bc1020e0 2024-04-24T06:40:54.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:54.768+0000 7f85b9400640 1 --2- 172.21.15.5:0/977105464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85bc1020e0 0x7f85bc19aa80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f85a802a910 tx=0x7f85a8032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:54.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:54.769+0000 7f85b2a00640 1 -- 172.21.15.5:0/977105464 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a803f030 con 0x7f85bc1020e0 2024-04-24T06:40:54.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:54.769+0000 7f85b2a00640 1 -- 172.21.15.5:0/977105464 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85a8003ce0 con 0x7f85bc1020e0 2024-04-24T06:40:54.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:54.769+0000 7f85b2a00640 1 -- 172.21.15.5:0/977105464 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a8035070 con 0x7f85bc1020e0 2024-04-24T06:40:54.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:54.769+0000 7f85c0e00640 1 -- 172.21.15.5:0/977105464 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85bc1a23f0 con 0x7f85bc1020e0 2024-04-24T06:40:54.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:54.769+0000 7f85c0e00640 1 -- 172.21.15.5:0/977105464 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85bc1a2900 con 0x7f85bc1020e0 2024-04-24T06:40:54.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:54.770+0000 7f85c0e00640 1 -- 172.21.15.5:0/977105464 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85bc1a2b60 con 0x7f85bc1020e0 2024-04-24T06:40:54.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:54.772+0000 7f85b2a00640 1 -- 172.21.15.5:0/977105464 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f85a8042030 con 0x7f85bc1020e0 2024-04-24T06:40:54.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:54.773+0000 7f85b2a00640 1 --2- 172.21.15.5:0/977105464 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f85a00a3c30 0x7f85a00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:54.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:54.773+0000 7f85b2a00640 1 -- 172.21.15.5:0/977105464 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f85a80f0340 con 0x7f85bc1020e0 2024-04-24T06:40:54.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:54.774+0000 7f85b9e00640 1 --2- 172.21.15.5:0/977105464 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f85a00a3c30 0x7f85a00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:54.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:54.774+0000 7f85b9e00640 1 --2- 172.21.15.5:0/977105464 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f85a00a3c30 0x7f85a00a60f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f85ac0058c0 tx=0x7f85ac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:54.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:54.778+0000 7f85b2a00640 1 -- 172.21.15.5:0/977105464 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f85a80bcad0 con 0x7f85bc1020e0 2024-04-24T06:40:54.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:54 smithi137 ceph-mon[33017]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:40:55.019 INFO:journalctl@ceph.mds.k.smithi037.stdout:Apr 24 06:40:54 smithi037 podman[62381]: 2024-04-24 06:40:54.531772433 +0000 UTC m=+0.442114813 container init 2d48a66b627449c18f74d743cb990ee408062e5b8fdc9497d95fccd9a1150bc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T06:40:55.019 INFO:journalctl@ceph.mds.k.smithi037.stdout:Apr 24 06:40:54 smithi037 podman[62381]: 2024-04-24 06:40:54.534857744 +0000 UTC m=+0.445200125 container start 2d48a66b627449c18f74d743cb990ee408062e5b8fdc9497d95fccd9a1150bc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-k, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-24T06:40:55.019 INFO:journalctl@ceph.mds.k.smithi037.stdout:Apr 24 06:40:54 smithi037 ceph-mds[62401]: -- 172.21.15.37:0/1620629761 <== mon.2 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562f4b667a40 con 0x562f4b63b000 2024-04-24T06:40:55.019 INFO:journalctl@ceph.mds.k.smithi037.stdout:Apr 24 06:40:54 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-k[62397]: starting mds.k at 2024-04-24T06:40:55.019 INFO:journalctl@ceph.mds.k.smithi037.stdout:Apr 24 06:40:54 smithi037 bash[62381]: 2d48a66b627449c18f74d743cb990ee408062e5b8fdc9497d95fccd9a1150bc6 2024-04-24T06:40:55.019 INFO:journalctl@ceph.mds.k.smithi037.stdout:Apr 24 06:40:54 smithi037 systemd[1]: Started Ceph mds.k for 43e9bad0-0204-11ef-bc93-c7b262605968. 2024-04-24T06:40:55.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:55.028+0000 7f85c0e00640 1 -- 172.21.15.5:0/977105464 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f85bc1a2ef0 con 0x7f85bc1020e0 2024-04-24T06:40:55.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:55.029+0000 7f85b2a00640 1 -- 172.21.15.5:0/977105464 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1393 (secure 0 0 0) 0x7f85a80c0dd0 con 0x7f85bc1020e0 2024-04-24T06:40:55.030 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:40:55.031 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-24T06:40:55.031 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 2 2024-04-24T06:40:55.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:55.032+0000 7f85c0e00640 1 -- 172.21.15.5:0/977105464 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f85a00a3c30 msgr2=0x7f85a00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:55.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:55.032+0000 7f85c0e00640 1 --2- 172.21.15.5:0/977105464 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f85a00a3c30 0x7f85a00a60f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f85ac0058c0 tx=0x7f85ac005f90 comp rx=0 tx=0).stop 2024-04-24T06:40:55.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:55.032+0000 7f85c0e00640 1 -- 172.21.15.5:0/977105464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85bc1020e0 msgr2=0x7f85bc19aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:55.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:55.032+0000 7f85c0e00640 1 --2- 172.21.15.5:0/977105464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85bc1020e0 0x7f85bc19aa80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f85a802a910 tx=0x7f85a8032ce0 comp rx=0 tx=0).stop 2024-04-24T06:40:55.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:55.032+0000 7f85c0e00640 1 -- 172.21.15.5:0/977105464 shutdown_connections 2024-04-24T06:40:55.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:55.032+0000 7f85c0e00640 1 --2- 172.21.15.5:0/977105464 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f85a00a3c30 0x7f85a00a60f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:55.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:55.032+0000 7f85c0e00640 1 --2- 172.21.15.5:0/977105464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f85bc102aa0 0x7f85bc1a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:55.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:55.032+0000 7f85c0e00640 1 --2- 172.21.15.5:0/977105464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85bc1020e0 0x7f85bc19aa80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:55.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:55.032+0000 7f85c0e00640 1 --2- 172.21.15.5:0/977105464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85bc100f10 0x7f85bc19a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:55.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:55.032+0000 7f85c0e00640 1 -- 172.21.15.5:0/977105464 >> 172.21.15.5:0/977105464 conn(0x7f85bc0fc4d0 msgr2=0x7f85bc109d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:55.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:55.033+0000 7f85c0e00640 1 -- 172.21.15.5:0/977105464 shutdown_connections 2024-04-24T06:40:55.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:55.033+0000 7f85c0e00640 1 -- 172.21.15.5:0/977105464 wait complete. 2024-04-24T06:40:55.937 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:55 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:55.937 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:55 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:55.937 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:55 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:55.937 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:55 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T06:40:55.937 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:55 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T06:40:55.937 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:55 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:40:55.937 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:55 smithi005 ceph-mon[27798]: Deploying daemon mds.j on smithi005 2024-04-24T06:40:55.937 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:55 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/977105464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:40:55.937 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:55 smithi005 ceph-mon[27798]: mds.? [v2:172.21.15.37:6832/2228938014,v1:172.21.15.37:6833/2228938014] up:boot 2024-04-24T06:40:55.937 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:55 smithi005 ceph-mon[27798]: fsmap 2 up:standby 2024-04-24T06:40:55.938 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:55 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T06:40:55.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:55 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:55.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:55 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:55.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:55 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:55.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:55 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T06:40:55.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:55 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T06:40:55.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:55 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:40:55.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:55 smithi137 ceph-mon[33017]: Deploying daemon mds.j on smithi005 2024-04-24T06:40:55.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:55 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/977105464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:40:55.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:55 smithi137 ceph-mon[33017]: mds.? [v2:172.21.15.37:6832/2228938014,v1:172.21.15.37:6833/2228938014] up:boot 2024-04-24T06:40:55.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:55 smithi137 ceph-mon[33017]: fsmap 2 up:standby 2024-04-24T06:40:55.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:55 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T06:40:55.976 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:55 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:55.977 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:55 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:55.977 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:55 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:55.977 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:55 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T06:40:55.977 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:55 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T06:40:55.977 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:55 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:40:55.977 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:55 smithi037 ceph-mon[32864]: Deploying daemon mds.j on smithi005 2024-04-24T06:40:55.977 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:55 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/977105464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:40:55.977 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:55 smithi037 ceph-mon[32864]: mds.? [v2:172.21.15.37:6832/2228938014,v1:172.21.15.37:6833/2228938014] up:boot 2024-04-24T06:40:55.977 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:55 smithi037 ceph-mon[32864]: fsmap 2 up:standby 2024-04-24T06:40:55.977 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:55 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T06:40:56.748 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:56 smithi005 ceph-mon[27798]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:40:57.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:56 smithi037 ceph-mon[32864]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:40:57.056 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:40:57.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:56 smithi137 ceph-mon[33017]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:40:57.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.400+0000 7f9833200640 1 -- 172.21.15.5:0/633475350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f982c10c060 msgr2=0x7f982c10e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:57.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.400+0000 7f9833200640 1 --2- 172.21.15.5:0/633475350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f982c10c060 0x7f982c10e450 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9824009ca0 tx=0x7f982402e9a0 comp rx=0 tx=0).stop 2024-04-24T06:40:57.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.400+0000 7f9833200640 1 -- 172.21.15.5:0/633475350 shutdown_connections 2024-04-24T06:40:57.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.400+0000 7f9833200640 1 --2- 172.21.15.5:0/633475350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f982c0731c0 0x7f982c111230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:57.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.400+0000 7f9833200640 1 --2- 172.21.15.5:0/633475350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f982c10c060 0x7f982c10e450 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:57.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.400+0000 7f9833200640 1 --2- 172.21.15.5:0/633475350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f982c109270 0x7f982c109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:57.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.400+0000 7f9833200640 1 -- 172.21.15.5:0/633475350 >> 172.21.15.5:0/633475350 conn(0x7f982c06f010 msgr2=0x7f982c071470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:57.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.400+0000 7f9833200640 1 -- 172.21.15.5:0/633475350 shutdown_connections 2024-04-24T06:40:57.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.401+0000 7f9833200640 1 -- 172.21.15.5:0/633475350 wait complete. 2024-04-24T06:40:57.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.402+0000 7f9833200640 1 Processor -- start 2024-04-24T06:40:57.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.402+0000 7f9833200640 1 -- start start 2024-04-24T06:40:57.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.403+0000 7f9833200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f982c0731c0 0x7f982c11a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:57.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.403+0000 7f9833200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f982c109270 0x7f982c11a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:57.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.403+0000 7f9833200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f982c10c060 0x7f982c118760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.404+0000 7f982be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f982c0731c0 0x7f982c11a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.404+0000 7f9831200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f982c10c060 0x7f982c118760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.404+0000 7f982be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f982c0731c0 0x7f982c11a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:58762/0 (socket says 172.21.15.5:58762) 2024-04-24T06:40:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.404+0000 7f982be00640 1 -- 172.21.15.5:0/2410920872 learned_addr learned my addr 172.21.15.5:0/2410920872 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:40:57.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.404+0000 7f982b400640 1 --2- 172.21.15.5:0/2410920872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f982c109270 0x7f982c11a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:57.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.404+0000 7f9833200640 1 -- 172.21.15.5:0/2410920872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f982c112fb0 con 0x7f982c109270 2024-04-24T06:40:57.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.404+0000 7f9833200640 1 -- 172.21.15.5:0/2410920872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f982c112e30 con 0x7f982c10c060 2024-04-24T06:40:57.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.404+0000 7f9833200640 1 -- 172.21.15.5:0/2410920872 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f982c113130 con 0x7f982c0731c0 2024-04-24T06:40:57.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.405+0000 7f9831200640 1 -- 172.21.15.5:0/2410920872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f982c0731c0 msgr2=0x7f982c11a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:57.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.405+0000 7f9831200640 1 --2- 172.21.15.5:0/2410920872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f982c0731c0 0x7f982c11a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:57.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.405+0000 7f9831200640 1 -- 172.21.15.5:0/2410920872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f982c109270 msgr2=0x7f982c11a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:57.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.405+0000 7f9831200640 1 --2- 172.21.15.5:0/2410920872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f982c109270 0x7f982c11a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:57.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.405+0000 7f9831200640 1 -- 172.21.15.5:0/2410920872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f982c118ee0 con 0x7f982c10c060 2024-04-24T06:40:57.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.405+0000 7f982be00640 1 --2- 172.21.15.5:0/2410920872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f982c0731c0 0x7f982c11a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:40:57.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.405+0000 7f982b400640 1 --2- 172.21.15.5:0/2410920872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f982c109270 0x7f982c11a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:40:57.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.406+0000 7f9831200640 1 --2- 172.21.15.5:0/2410920872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f982c10c060 0x7f982c118760 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f981c00e940 tx=0x7f981c00ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:57.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.406+0000 7f9828c00640 1 -- 172.21.15.5:0/2410920872 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c00ccc0 con 0x7f982c10c060 2024-04-24T06:40:57.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.406+0000 7f9828c00640 1 -- 172.21.15.5:0/2410920872 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f981c005020 con 0x7f982c10c060 2024-04-24T06:40:57.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.406+0000 7f9828c00640 1 -- 172.21.15.5:0/2410920872 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c004a00 con 0x7f982c10c060 2024-04-24T06:40:57.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.407+0000 7f9833200640 1 -- 172.21.15.5:0/2410920872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f982c1191d0 con 0x7f982c10c060 2024-04-24T06:40:57.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.407+0000 7f9833200640 1 -- 172.21.15.5:0/2410920872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f982c1b7e40 con 0x7f982c10c060 2024-04-24T06:40:57.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.408+0000 7f9833200640 1 -- 172.21.15.5:0/2410920872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f982c071b10 con 0x7f982c10c060 2024-04-24T06:40:57.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.410+0000 7f9828c00640 1 -- 172.21.15.5:0/2410920872 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f981c0051d0 con 0x7f982c10c060 2024-04-24T06:40:57.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.411+0000 7f9828c00640 1 --2- 172.21.15.5:0/2410920872 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f98100a3c80 0x7f98100a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:57.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.412+0000 7f9828c00640 1 -- 172.21.15.5:0/2410920872 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f981c0cfed0 con 0x7f982c10c060 2024-04-24T06:40:57.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.412+0000 7f982be00640 1 --2- 172.21.15.5:0/2410920872 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f98100a3c80 0x7f98100a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:57.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.412+0000 7f982be00640 1 --2- 172.21.15.5:0/2410920872 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f98100a3c80 0x7f98100a6140 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9818011550 tx=0x7f9818005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:57.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.416+0000 7f9828c00640 1 -- 172.21.15.5:0/2410920872 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f981c09ccb0 con 0x7f982c10c060 2024-04-24T06:40:57.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.674+0000 7f9833200640 1 -- 172.21.15.5:0/2410920872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f982c02d620 con 0x7f982c10c060 2024-04-24T06:40:57.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.675+0000 7f9828c00640 1 -- 172.21.15.5:0/2410920872 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2218 (secure 0 0 0) 0x7f981c09c5b0 con 0x7f982c10c060 2024-04-24T06:40:57.676 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:40:57.676 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-24T06:40:57.676 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 3 2024-04-24T06:40:57.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.678+0000 7f9833200640 1 -- 172.21.15.5:0/2410920872 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f98100a3c80 msgr2=0x7f98100a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:57.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.678+0000 7f9833200640 1 --2- 172.21.15.5:0/2410920872 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f98100a3c80 0x7f98100a6140 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9818011550 tx=0x7f9818005f90 comp rx=0 tx=0).stop 2024-04-24T06:40:57.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.678+0000 7f9833200640 1 -- 172.21.15.5:0/2410920872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f982c10c060 msgr2=0x7f982c118760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:57.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.678+0000 7f9833200640 1 --2- 172.21.15.5:0/2410920872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f982c10c060 0x7f982c118760 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f981c00e940 tx=0x7f981c00ee00 comp rx=0 tx=0).stop 2024-04-24T06:40:57.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.678+0000 7f9833200640 1 -- 172.21.15.5:0/2410920872 shutdown_connections 2024-04-24T06:40:57.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.678+0000 7f9833200640 1 --2- 172.21.15.5:0/2410920872 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f98100a3c80 0x7f98100a6140 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:57.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.678+0000 7f9833200640 1 --2- 172.21.15.5:0/2410920872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f982c10c060 0x7f982c118760 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:57.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.678+0000 7f9833200640 1 --2- 172.21.15.5:0/2410920872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f982c109270 0x7f982c11a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:57.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.678+0000 7f9833200640 1 --2- 172.21.15.5:0/2410920872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f982c0731c0 0x7f982c11a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:57.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.679+0000 7f9833200640 1 -- 172.21.15.5:0/2410920872 >> 172.21.15.5:0/2410920872 conn(0x7f982c06f010 msgr2=0x7f982c10d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:57.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.679+0000 7f9833200640 1 -- 172.21.15.5:0/2410920872 shutdown_connections 2024-04-24T06:40:57.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:57.679+0000 7f9833200640 1 -- 172.21.15.5:0/2410920872 wait complete. 2024-04-24T06:40:58.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:57 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/2410920872' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:40:58.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:57 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/2410920872' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:40:58.097 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:57 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/2410920872' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:40:58.364 INFO:journalctl@ceph.mds.j.smithi005.stdout:Apr 24 06:40:58 smithi005 podman[64608]: 2024-04-24 06:40:58.049009339 +0000 UTC m=+0.154037232 container create 708d1ca029d015e739334056bcaf017f109dc3a5a96c37bf451378fad15c6011 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-24T06:40:58.364 INFO:journalctl@ceph.mds.j.smithi005.stdout:Apr 24 06:40:58 smithi005 podman[64608]: 2024-04-24 06:40:58.361731777 +0000 UTC m=+0.466759706 container init 708d1ca029d015e739334056bcaf017f109dc3a5a96c37bf451378fad15c6011 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-j, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-24T06:40:58.745 INFO:journalctl@ceph.mds.j.smithi005.stdout:Apr 24 06:40:58 smithi005 podman[64608]: 2024-04-24 06:40:58.372828196 +0000 UTC m=+0.477856119 container start 708d1ca029d015e739334056bcaf017f109dc3a5a96c37bf451378fad15c6011 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-j, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-24T06:40:58.745 INFO:journalctl@ceph.mds.j.smithi005.stdout:Apr 24 06:40:58 smithi005 ceph-mds[64627]: -- 172.21.15.5:0/71904255 <== mon.1 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f4af7a7a40 con 0x55f4af77ac00 2024-04-24T06:40:58.745 INFO:journalctl@ceph.mds.j.smithi005.stdout:Apr 24 06:40:58 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-j[64623]: starting mds.j at 2024-04-24T06:40:58.745 INFO:journalctl@ceph.mds.j.smithi005.stdout:Apr 24 06:40:58 smithi005 bash[64608]: 708d1ca029d015e739334056bcaf017f109dc3a5a96c37bf451378fad15c6011 2024-04-24T06:40:58.746 INFO:journalctl@ceph.mds.j.smithi005.stdout:Apr 24 06:40:58 smithi005 systemd[1]: Started Ceph mds.j for 43e9bad0-0204-11ef-bc93-c7b262605968. 2024-04-24T06:40:59.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:58 smithi005 ceph-mon[27798]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:40:59.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:58 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:59.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:58 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:59.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:58 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:59.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:58 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T06:40:59.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:58 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T06:40:59.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:58 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:40:59.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:58 smithi037 ceph-mon[32864]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:40:59.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:58 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:59.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:58 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:59.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:58 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:59.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:58 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T06:40:59.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:58 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T06:40:59.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:58 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:40:59.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:58 smithi137 ceph-mon[33017]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:40:59.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:58 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:59.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:58 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:59.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:58 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:40:59.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:58 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T06:40:59.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:58 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T06:40:59.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:58 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:40:59.703 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:40:59.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:59.929+0000 7fcb9d800640 1 -- 172.21.15.5:0/785148493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb981052f0 msgr2=0x7fcb981076e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:59.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:59.929+0000 7fcb9d800640 1 --2- 172.21.15.5:0/785148493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb981052f0 0x7fcb981076e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcb84009900 tx=0x7fcb8402a3f0 comp rx=0 tx=0).stop 2024-04-24T06:40:59.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:59.930+0000 7fcb9d800640 1 -- 172.21.15.5:0/785148493 shutdown_connections 2024-04-24T06:40:59.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:59.930+0000 7fcb9d800640 1 --2- 172.21.15.5:0/785148493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb98107c20 0x7fcb9810a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:59.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:59.930+0000 7fcb9d800640 1 --2- 172.21.15.5:0/785148493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb981052f0 0x7fcb981076e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:59.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:59.930+0000 7fcb9d800640 1 --2- 172.21.15.5:0/785148493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb981029c0 0x7fcb98104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:59.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:59.930+0000 7fcb9d800640 1 -- 172.21.15.5:0/785148493 >> 172.21.15.5:0/785148493 conn(0x7fcb980fc3d0 msgr2=0x7fcb980fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:40:59.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:59.930+0000 7fcb9d800640 1 -- 172.21.15.5:0/785148493 shutdown_connections 2024-04-24T06:40:59.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:59.930+0000 7fcb9d800640 1 -- 172.21.15.5:0/785148493 wait complete. 2024-04-24T06:40:59.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:59.931+0000 7fcb9d800640 1 Processor -- start 2024-04-24T06:40:59.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:59.931+0000 7fcb9d800640 1 -- start start 2024-04-24T06:40:59.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:59.931+0000 7fcb9d800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb981029c0 0x7fcb9819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:59.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:59.931+0000 7fcb9d800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb981052f0 0x7fcb9819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:59.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:59.931+0000 7fcb9d800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb98107c20 0x7fcb981a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:59.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:59.931+0000 7fcb9d800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9810dc00 con 0x7fcb98107c20 2024-04-24T06:40:59.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:59.931+0000 7fcb9d800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9810da80 con 0x7fcb981052f0 2024-04-24T06:40:59.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:59.931+0000 7fcb9d800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9810dd80 con 0x7fcb981029c0 2024-04-24T06:40:59.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:59.931+0000 7fcb97400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb981052f0 0x7fcb9819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:59.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:59.931+0000 7fcb9ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb98107c20 0x7fcb981a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:59.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:59.931+0000 7fcb97400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb981052f0 0x7fcb9819a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:42174/0 (socket says 172.21.15.5:42174) 2024-04-24T06:40:59.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:59.931+0000 7fcb97400640 1 -- 172.21.15.5:0/957209231 learned_addr learned my addr 172.21.15.5:0/957209231 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:40:59.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:59.932+0000 7fcb97e00640 1 --2- 172.21.15.5:0/957209231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb981029c0 0x7fcb9819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:59.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:59.932+0000 7fcb97400640 1 -- 172.21.15.5:0/957209231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb981029c0 msgr2=0x7fcb9819a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:59.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:59.932+0000 7fcb97400640 1 --2- 172.21.15.5:0/957209231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb981029c0 0x7fcb9819a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:59.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:59.932+0000 7fcb97400640 1 -- 172.21.15.5:0/957209231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb98107c20 msgr2=0x7fcb981a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:40:59.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:59.932+0000 7fcb97400640 1 --2- 172.21.15.5:0/957209231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb98107c20 0x7fcb981a1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:40:59.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:59.932+0000 7fcb97400640 1 -- 172.21.15.5:0/957209231 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb981a2090 con 0x7fcb981052f0 2024-04-24T06:40:59.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:59.932+0000 7fcb9ce00640 1 --2- 172.21.15.5:0/957209231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb98107c20 0x7fcb981a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:40:59.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:59.932+0000 7fcb97400640 1 --2- 172.21.15.5:0/957209231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb981052f0 0x7fcb9819a9b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcb84031440 tx=0x7fcb84032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:59.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:59.933+0000 7fcb94c00640 1 -- 172.21.15.5:0/957209231 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8403f030 con 0x7fcb981052f0 2024-04-24T06:40:59.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:59.933+0000 7fcb94c00640 1 -- 172.21.15.5:0/957209231 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb84003ce0 con 0x7fcb981052f0 2024-04-24T06:40:59.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:59.933+0000 7fcb9d800640 1 -- 172.21.15.5:0/957209231 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb981a2320 con 0x7fcb981052f0 2024-04-24T06:40:59.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:59.933+0000 7fcb94c00640 1 -- 172.21.15.5:0/957209231 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb84035070 con 0x7fcb981052f0 2024-04-24T06:40:59.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:59.933+0000 7fcb9d800640 1 -- 172.21.15.5:0/957209231 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb981a2810 con 0x7fcb981052f0 2024-04-24T06:40:59.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:59.934+0000 7fcb9d800640 1 -- 172.21.15.5:0/957209231 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb98109770 con 0x7fcb981052f0 2024-04-24T06:40:59.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:59.936+0000 7fcb94c00640 1 -- 172.21.15.5:0/957209231 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fcb84042030 con 0x7fcb981052f0 2024-04-24T06:40:59.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:59.937+0000 7fcb94c00640 1 --2- 172.21.15.5:0/957209231 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fcb7c0a3ba0 0x7fcb7c0a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:40:59.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:59.937+0000 7fcb94c00640 1 -- 172.21.15.5:0/957209231 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fcb840f0440 con 0x7fcb981052f0 2024-04-24T06:40:59.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:59.937+0000 7fcb97e00640 1 --2- 172.21.15.5:0/957209231 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fcb7c0a3ba0 0x7fcb7c0a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:40:59.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:59.938+0000 7fcb97e00640 1 --2- 172.21.15.5:0/957209231 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fcb7c0a3ba0 0x7fcb7c0a6060 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcb88009bd0 tx=0x7fcb88005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:40:59.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:40:59.941+0000 7fcb94c00640 1 -- 172.21.15.5:0/957209231 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcb840bcbd0 con 0x7fcb981052f0 2024-04-24T06:40:59.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:59 smithi137 ceph-mon[33017]: Deploying daemon mds.d on smithi005 2024-04-24T06:40:59.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:59 smithi137 ceph-mon[33017]: mds.? [v2:172.21.15.5:6834/2026364804,v1:172.21.15.5:6835/2026364804] up:boot 2024-04-24T06:40:59.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:59 smithi137 ceph-mon[33017]: fsmap 3 up:standby 2024-04-24T06:40:59.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:40:59 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T06:41:00.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:59 smithi005 ceph-mon[27798]: Deploying daemon mds.d on smithi005 2024-04-24T06:41:00.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:59 smithi005 ceph-mon[27798]: mds.? [v2:172.21.15.5:6834/2026364804,v1:172.21.15.5:6835/2026364804] up:boot 2024-04-24T06:41:00.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:59 smithi005 ceph-mon[27798]: fsmap 3 up:standby 2024-04-24T06:41:00.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:40:59 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T06:41:00.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:59 smithi037 ceph-mon[32864]: Deploying daemon mds.d on smithi005 2024-04-24T06:41:00.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:59 smithi037 ceph-mon[32864]: mds.? [v2:172.21.15.5:6834/2026364804,v1:172.21.15.5:6835/2026364804] up:boot 2024-04-24T06:41:00.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:59 smithi037 ceph-mon[32864]: fsmap 3 up:standby 2024-04-24T06:41:00.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:40:59 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T06:41:00.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:00.205+0000 7fcb9d800640 1 -- 172.21.15.5:0/957209231 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb9802d620 con 0x7fcb981052f0 2024-04-24T06:41:00.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:00.206+0000 7fcb94c00640 1 -- 172.21.15.5:0/957209231 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3040 (secure 0 0 0) 0x7fcb840c0ed0 con 0x7fcb981052f0 2024-04-24T06:41:00.207 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:41:00.207 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[]} 2024-04-24T06:41:00.207 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 4 2024-04-24T06:41:00.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:00.208+0000 7fcb9d800640 1 -- 172.21.15.5:0/957209231 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fcb7c0a3ba0 msgr2=0x7fcb7c0a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:00.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:00.208+0000 7fcb9d800640 1 --2- 172.21.15.5:0/957209231 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fcb7c0a3ba0 0x7fcb7c0a6060 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcb88009bd0 tx=0x7fcb88005f90 comp rx=0 tx=0).stop 2024-04-24T06:41:00.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:00.208+0000 7fcb9d800640 1 -- 172.21.15.5:0/957209231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb981052f0 msgr2=0x7fcb9819a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:00.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:00.209+0000 7fcb9d800640 1 --2- 172.21.15.5:0/957209231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb981052f0 0x7fcb9819a9b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcb84031440 tx=0x7fcb84032ce0 comp rx=0 tx=0).stop 2024-04-24T06:41:00.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:00.209+0000 7fcb9d800640 1 -- 172.21.15.5:0/957209231 shutdown_connections 2024-04-24T06:41:00.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:00.209+0000 7fcb9d800640 1 --2- 172.21.15.5:0/957209231 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fcb7c0a3ba0 0x7fcb7c0a6060 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:00.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:00.209+0000 7fcb9d800640 1 --2- 172.21.15.5:0/957209231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb98107c20 0x7fcb981a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:00.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:00.209+0000 7fcb9d800640 1 --2- 172.21.15.5:0/957209231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb981052f0 0x7fcb9819a9b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:00.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:00.209+0000 7fcb9d800640 1 --2- 172.21.15.5:0/957209231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb981029c0 0x7fcb9819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:00.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:00.209+0000 7fcb9d800640 1 -- 172.21.15.5:0/957209231 >> 172.21.15.5:0/957209231 conn(0x7fcb980fc3d0 msgr2=0x7fcb981020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:00.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:00.209+0000 7fcb9d800640 1 -- 172.21.15.5:0/957209231 shutdown_connections 2024-04-24T06:41:00.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:00.209+0000 7fcb9d800640 1 -- 172.21.15.5:0/957209231 wait complete. 2024-04-24T06:41:01.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:00 smithi005 ceph-mon[27798]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:01.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:00 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/957209231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:01.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:00 smithi037 ceph-mon[32864]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:01.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:00 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/957209231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:01.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:00 smithi137 ceph-mon[33017]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:01.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:00 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/957209231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:02.230 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:41:02.256 INFO:journalctl@ceph.mds.d.smithi005.stdout:Apr 24 06:41:01 smithi005 systemd[1]: Starting Ceph mds.d for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:41:02.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.441+0000 7fb53fc00640 1 -- 172.21.15.5:0/3034433252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5381020e0 msgr2=0x7fb538102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:02.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.441+0000 7fb53fc00640 1 --2- 172.21.15.5:0/3034433252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5381020e0 0x7fb538102560 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb520009900 tx=0x7fb52002a410 comp rx=0 tx=0).stop 2024-04-24T06:41:02.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.441+0000 7fb53fc00640 1 -- 172.21.15.5:0/3034433252 shutdown_connections 2024-04-24T06:41:02.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.441+0000 7fb53fc00640 1 --2- 172.21.15.5:0/3034433252 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb538102aa0 0x7fb5381093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:02.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.441+0000 7fb53fc00640 1 --2- 172.21.15.5:0/3034433252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5381020e0 0x7fb538102560 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:02.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.441+0000 7fb53fc00640 1 --2- 172.21.15.5:0/3034433252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb538100f10 0x7fb538101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:02.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.441+0000 7fb53fc00640 1 -- 172.21.15.5:0/3034433252 >> 172.21.15.5:0/3034433252 conn(0x7fb5380fc4d0 msgr2=0x7fb5380fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:02.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.441+0000 7fb53fc00640 1 -- 172.21.15.5:0/3034433252 shutdown_connections 2024-04-24T06:41:02.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.441+0000 7fb53fc00640 1 -- 172.21.15.5:0/3034433252 wait complete. 2024-04-24T06:41:02.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.442+0000 7fb53fc00640 1 Processor -- start 2024-04-24T06:41:02.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.442+0000 7fb53fc00640 1 -- start start 2024-04-24T06:41:02.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.443+0000 7fb53fc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb538100f10 0x7fb53819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:02.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.443+0000 7fb53fc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5381020e0 0x7fb53819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:02.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.443+0000 7fb53fc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb538102aa0 0x7fb5381a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:02.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.443+0000 7fb53fc00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb53810cc30 con 0x7fb538100f10 2024-04-24T06:41:02.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.443+0000 7fb53fc00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb53810cab0 con 0x7fb538102aa0 2024-04-24T06:41:02.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.443+0000 7fb53fc00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb53810cdb0 con 0x7fb5381020e0 2024-04-24T06:41:02.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.443+0000 7fb53d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb538100f10 0x7fb53819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:02.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.443+0000 7fb52fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5381020e0 0x7fb53819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:02.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.443+0000 7fb53d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb538100f10 0x7fb53819a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57360/0 (socket says 172.21.15.5:57360) 2024-04-24T06:41:02.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.443+0000 7fb53d200640 1 -- 172.21.15.5:0/1171606548 learned_addr learned my addr 172.21.15.5:0/1171606548 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:41:02.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.443+0000 7fb53dc00640 1 --2- 172.21.15.5:0/1171606548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb538102aa0 0x7fb5381a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:02.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.443+0000 7fb53d200640 1 -- 172.21.15.5:0/1171606548 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5381020e0 msgr2=0x7fb53819aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:02.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.443+0000 7fb53d200640 1 --2- 172.21.15.5:0/1171606548 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5381020e0 0x7fb53819aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:02.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.443+0000 7fb53d200640 1 -- 172.21.15.5:0/1171606548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb538102aa0 msgr2=0x7fb5381a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:02.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.443+0000 7fb53d200640 1 --2- 172.21.15.5:0/1171606548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb538102aa0 0x7fb5381a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:02.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.443+0000 7fb53d200640 1 -- 172.21.15.5:0/1171606548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5381a2160 con 0x7fb538100f10 2024-04-24T06:41:02.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.444+0000 7fb53d200640 1 --2- 172.21.15.5:0/1171606548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb538100f10 0x7fb53819a540 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb52800d950 tx=0x7fb52800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:02.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.444+0000 7fb536000640 1 -- 172.21.15.5:0/1171606548 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb528017020 con 0x7fb538100f10 2024-04-24T06:41:02.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.444+0000 7fb536000640 1 -- 172.21.15.5:0/1171606548 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb528009c30 con 0x7fb538100f10 2024-04-24T06:41:02.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.444+0000 7fb536000640 1 -- 172.21.15.5:0/1171606548 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52800f720 con 0x7fb538100f10 2024-04-24T06:41:02.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.444+0000 7fb53fc00640 1 -- 172.21.15.5:0/1171606548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5381a2450 con 0x7fb538100f10 2024-04-24T06:41:02.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.444+0000 7fb53fc00640 1 -- 172.21.15.5:0/1171606548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5381a2920 con 0x7fb538100f10 2024-04-24T06:41:02.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.446+0000 7fb53fc00640 1 -- 172.21.15.5:0/1171606548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5381a2be0 con 0x7fb538100f10 2024-04-24T06:41:02.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.447+0000 7fb536000640 1 -- 172.21.15.5:0/1171606548 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fb52801a020 con 0x7fb538100f10 2024-04-24T06:41:02.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.450+0000 7fb536000640 1 --2- 172.21.15.5:0/1171606548 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fb51c0a3c30 0x7fb51c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:02.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.450+0000 7fb536000640 1 -- 172.21.15.5:0/1171606548 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fb5280cfe50 con 0x7fb538100f10 2024-04-24T06:41:02.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.450+0000 7fb52fe00640 1 --2- 172.21.15.5:0/1171606548 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fb51c0a3c30 0x7fb51c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:02.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.451+0000 7fb52fe00640 1 --2- 172.21.15.5:0/1171606548 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fb51c0a3c30 0x7fb51c0a60f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb520031440 tx=0x7fb5200315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:02.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.455+0000 7fb536000640 1 -- 172.21.15.5:0/1171606548 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb528021020 con 0x7fb538100f10 2024-04-24T06:41:02.548 INFO:journalctl@ceph.mds.d.smithi005.stdout:Apr 24 06:41:02 smithi005 podman[64951]: 2024-04-24 06:41:02.175125962 +0000 UTC m=+0.035998234 image pull b5c3183062a6ce19f361681cc58de6e3ac1de1a54c80e6094ec5695237cd98d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b 2024-04-24T06:41:02.548 INFO:journalctl@ceph.mds.d.smithi005.stdout:Apr 24 06:41:02 smithi005 podman[64951]: 2024-04-24 06:41:02.293283939 +0000 UTC m=+0.154156217 container create 0b6768e60a1130572350dca189046fa20a3709a408071061e4041165a95897fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-24T06:41:02.548 INFO:journalctl@ceph.mds.d.smithi005.stdout:Apr 24 06:41:02 smithi005 podman[64951]: 2024-04-24 06:41:02.546694184 +0000 UTC m=+0.407566466 container init 0b6768e60a1130572350dca189046fa20a3709a408071061e4041165a95897fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-24T06:41:02.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.718+0000 7fb53fc00640 1 -- 172.21.15.5:0/1171606548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5381a2db0 con 0x7fb538100f10 2024-04-24T06:41:02.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.719+0000 7fb536000640 1 -- 172.21.15.5:0/1171606548 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3040 (secure 0 0 0) 0x7fb528009de0 con 0x7fb538100f10 2024-04-24T06:41:02.720 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:41:02.720 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[]} 2024-04-24T06:41:02.720 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 4 2024-04-24T06:41:02.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.721+0000 7fb53fc00640 1 -- 172.21.15.5:0/1171606548 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fb51c0a3c30 msgr2=0x7fb51c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:02.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.721+0000 7fb53fc00640 1 --2- 172.21.15.5:0/1171606548 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fb51c0a3c30 0x7fb51c0a60f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb520031440 tx=0x7fb5200315b0 comp rx=0 tx=0).stop 2024-04-24T06:41:02.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.721+0000 7fb53fc00640 1 -- 172.21.15.5:0/1171606548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb538100f10 msgr2=0x7fb53819a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:02.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.721+0000 7fb53fc00640 1 --2- 172.21.15.5:0/1171606548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb538100f10 0x7fb53819a540 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb52800d950 tx=0x7fb52800de10 comp rx=0 tx=0).stop 2024-04-24T06:41:02.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.721+0000 7fb53fc00640 1 -- 172.21.15.5:0/1171606548 shutdown_connections 2024-04-24T06:41:02.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.721+0000 7fb53fc00640 1 --2- 172.21.15.5:0/1171606548 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fb51c0a3c30 0x7fb51c0a60f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:02.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.721+0000 7fb53fc00640 1 --2- 172.21.15.5:0/1171606548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb538102aa0 0x7fb5381a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:02.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.721+0000 7fb53fc00640 1 --2- 172.21.15.5:0/1171606548 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5381020e0 0x7fb53819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:02.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.721+0000 7fb53fc00640 1 --2- 172.21.15.5:0/1171606548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb538100f10 0x7fb53819a540 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:02.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.722+0000 7fb53fc00640 1 -- 172.21.15.5:0/1171606548 >> 172.21.15.5:0/1171606548 conn(0x7fb5380fc4d0 msgr2=0x7fb5380fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:02.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.722+0000 7fb53fc00640 1 -- 172.21.15.5:0/1171606548 shutdown_connections 2024-04-24T06:41:02.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:02.722+0000 7fb53fc00640 1 -- 172.21.15.5:0/1171606548 wait complete. 2024-04-24T06:41:03.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:02 smithi005 ceph-mon[27798]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:03.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:02 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1171606548' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:03.007 INFO:journalctl@ceph.mds.d.smithi005.stdout:Apr 24 06:41:02 smithi005 podman[64951]: 2024-04-24 06:41:02.557151022 +0000 UTC m=+0.418023298 container start 0b6768e60a1130572350dca189046fa20a3709a408071061e4041165a95897fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-d, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-24T06:41:03.007 INFO:journalctl@ceph.mds.d.smithi005.stdout:Apr 24 06:41:02 smithi005 ceph-mds[65009]: -- 172.21.15.5:0/4150013598 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a3bca99a40 con 0x55a3bca6d000 2024-04-24T06:41:03.007 INFO:journalctl@ceph.mds.d.smithi005.stdout:Apr 24 06:41:02 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-d[64986]: starting mds.d at 2024-04-24T06:41:03.007 INFO:journalctl@ceph.mds.d.smithi005.stdout:Apr 24 06:41:02 smithi005 bash[64951]: 0b6768e60a1130572350dca189046fa20a3709a408071061e4041165a95897fc 2024-04-24T06:41:03.007 INFO:journalctl@ceph.mds.d.smithi005.stdout:Apr 24 06:41:02 smithi005 systemd[1]: Started Ceph mds.d for 43e9bad0-0204-11ef-bc93-c7b262605968. 2024-04-24T06:41:03.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:02 smithi037 ceph-mon[32864]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:03.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:02 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/1171606548' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:03.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:02 smithi137 ceph-mon[33017]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:03.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:02 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/1171606548' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:04.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:03 smithi037 ceph-mon[32864]: mds.? [v2:172.21.15.5:6836/1735462750,v1:172.21.15.5:6837/1735462750] up:boot 2024-04-24T06:41:04.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:03 smithi037 ceph-mon[32864]: fsmap 4 up:standby 2024-04-24T06:41:04.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:03 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T06:41:04.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:03 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:04.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:03 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:04.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:03 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:04.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:03 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T06:41:04.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:03 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T06:41:04.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:03 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:41:04.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:03 smithi037 ceph-mon[32864]: Deploying daemon mds.e on smithi037 2024-04-24T06:41:04.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:03 smithi137 ceph-mon[33017]: mds.? [v2:172.21.15.5:6836/1735462750,v1:172.21.15.5:6837/1735462750] up:boot 2024-04-24T06:41:04.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:03 smithi137 ceph-mon[33017]: fsmap 4 up:standby 2024-04-24T06:41:04.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:03 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T06:41:04.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:03 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:04.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:03 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:04.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:03 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:04.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:03 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T06:41:04.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:03 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T06:41:04.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:03 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:41:04.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:03 smithi137 ceph-mon[33017]: Deploying daemon mds.e on smithi037 2024-04-24T06:41:04.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:03 smithi005 ceph-mon[27798]: mds.? [v2:172.21.15.5:6836/1735462750,v1:172.21.15.5:6837/1735462750] up:boot 2024-04-24T06:41:04.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:03 smithi005 ceph-mon[27798]: fsmap 4 up:standby 2024-04-24T06:41:04.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:03 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T06:41:04.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:03 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:04.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:03 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:04.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:03 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:04.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:03 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T06:41:04.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:03 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T06:41:04.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:03 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:41:04.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:03 smithi005 ceph-mon[27798]: Deploying daemon mds.e on smithi037 2024-04-24T06:41:04.742 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:41:04.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:04.966+0000 7efd61000640 1 -- 172.21.15.5:0/674915878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd5c074a60 msgr2=0x7efd5c074ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:04.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:04.966+0000 7efd61000640 1 --2- 172.21.15.5:0/674915878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd5c074a60 0x7efd5c074ee0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efd44009930 tx=0x7efd4402a420 comp rx=0 tx=0).stop 2024-04-24T06:41:04.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:04.966+0000 7efd61000640 1 -- 172.21.15.5:0/674915878 shutdown_connections 2024-04-24T06:41:04.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:04.966+0000 7efd61000640 1 --2- 172.21.15.5:0/674915878 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd5c10e500 0x7efd5c1108f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:04.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:04.966+0000 7efd61000640 1 --2- 172.21.15.5:0/674915878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd5c074a60 0x7efd5c074ee0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:04.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:04.966+0000 7efd61000640 1 --2- 172.21.15.5:0/674915878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd5c076040 0x7efd5c074490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:04.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:04.966+0000 7efd61000640 1 -- 172.21.15.5:0/674915878 >> 172.21.15.5:0/674915878 conn(0x7efd5c0fd4c0 msgr2=0x7efd5c0ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:04.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:04.966+0000 7efd61000640 1 -- 172.21.15.5:0/674915878 shutdown_connections 2024-04-24T06:41:04.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:04.966+0000 7efd61000640 1 -- 172.21.15.5:0/674915878 wait complete. 2024-04-24T06:41:04.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:04.967+0000 7efd61000640 1 Processor -- start 2024-04-24T06:41:04.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:04.967+0000 7efd61000640 1 -- start start 2024-04-24T06:41:04.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:04.968+0000 7efd61000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd5c074a60 0x7efd5c19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:04.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:04.968+0000 7efd61000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd5c076040 0x7efd5c19edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:04.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:04.968+0000 7efd61000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd5c10e500 0x7efd5c1a5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:04.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:04.968+0000 7efd61000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c114210 con 0x7efd5c076040 2024-04-24T06:41:04.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:04.968+0000 7efd61000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c114090 con 0x7efd5c074a60 2024-04-24T06:41:04.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:04.968+0000 7efd61000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c114390 con 0x7efd5c10e500 2024-04-24T06:41:04.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:04.968+0000 7efd5b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd5c074a60 0x7efd5c19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:04.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:04.968+0000 7efd52a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd5c076040 0x7efd5c19edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:04.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:04.968+0000 7efd5b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd5c074a60 0x7efd5c19e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:50978/0 (socket says 172.21.15.5:50978) 2024-04-24T06:41:04.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:04.968+0000 7efd5b400640 1 -- 172.21.15.5:0/1492776920 learned_addr learned my addr 172.21.15.5:0/1492776920 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:41:04.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:04.968+0000 7efd52a00640 1 --2- 172.21.15.5:0/1492776920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd5c076040 0x7efd5c19edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55354/0 (socket says 172.21.15.5:55354) 2024-04-24T06:41:04.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:04.968+0000 7efd5be00640 1 --2- 172.21.15.5:0/1492776920 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd5c10e500 0x7efd5c1a5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:04.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:04.968+0000 7efd5b400640 1 -- 172.21.15.5:0/1492776920 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd5c10e500 msgr2=0x7efd5c1a5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:04.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:04.968+0000 7efd5b400640 1 --2- 172.21.15.5:0/1492776920 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd5c10e500 0x7efd5c1a5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:04.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:04.968+0000 7efd5b400640 1 -- 172.21.15.5:0/1492776920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd5c076040 msgr2=0x7efd5c19edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:04.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:04.968+0000 7efd5b400640 1 --2- 172.21.15.5:0/1492776920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd5c076040 0x7efd5c19edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:04.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:04.968+0000 7efd5b400640 1 -- 172.21.15.5:0/1492776920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd5c1a64b0 con 0x7efd5c074a60 2024-04-24T06:41:04.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:04.969+0000 7efd5b400640 1 --2- 172.21.15.5:0/1492776920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd5c074a60 0x7efd5c19e890 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efd4c009d50 tx=0x7efd4c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:04.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:04.969+0000 7efd58c00640 1 -- 172.21.15.5:0/1492776920 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c003b70 con 0x7efd5c074a60 2024-04-24T06:41:04.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:04.969+0000 7efd61000640 1 -- 172.21.15.5:0/1492776920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd5c1a67a0 con 0x7efd5c074a60 2024-04-24T06:41:04.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:04.969+0000 7efd58c00640 1 -- 172.21.15.5:0/1492776920 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd4c003d10 con 0x7efd5c074a60 2024-04-24T06:41:04.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:04.969+0000 7efd58c00640 1 -- 172.21.15.5:0/1492776920 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c010a20 con 0x7efd5c074a60 2024-04-24T06:41:04.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:04.969+0000 7efd61000640 1 -- 172.21.15.5:0/1492776920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd5c1a6d10 con 0x7efd5c074a60 2024-04-24T06:41:04.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:04.970+0000 7efd61000640 1 -- 172.21.15.5:0/1492776920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd5c1046d0 con 0x7efd5c074a60 2024-04-24T06:41:04.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:04.972+0000 7efd58c00640 1 -- 172.21.15.5:0/1492776920 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7efd4c010bc0 con 0x7efd5c074a60 2024-04-24T06:41:04.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:04.973+0000 7efd58c00640 1 --2- 172.21.15.5:0/1492776920 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7efd400a3c20 0x7efd400a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:04.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:04.974+0000 7efd58c00640 1 -- 172.21.15.5:0/1492776920 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7efd4c0d07a0 con 0x7efd5c074a60 2024-04-24T06:41:04.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:04.974+0000 7efd52a00640 1 --2- 172.21.15.5:0/1492776920 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7efd400a3c20 0x7efd400a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:04.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:04.974+0000 7efd52a00640 1 --2- 172.21.15.5:0/1492776920 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7efd400a3c20 0x7efd400a60e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7efd44009900 tx=0x7efd440315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:04.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:04.977+0000 7efd58c00640 1 -- 172.21.15.5:0/1492776920 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efd4c09d580 con 0x7efd5c074a60 2024-04-24T06:41:05.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:04 smithi037 ceph-mon[32864]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:05.019 INFO:journalctl@ceph.mds.e.smithi037.stdout:Apr 24 06:41:04 smithi037 systemd[1]: Starting Ceph mds.e for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:41:05.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:04 smithi137 ceph-mon[33017]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:05.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:05.223+0000 7efd61000640 1 -- 172.21.15.5:0/1492776920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd5c02d620 con 0x7efd5c074a60 2024-04-24T06:41:05.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:05.225+0000 7efd58c00640 1 -- 172.21.15.5:0/1492776920 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3862 (secure 0 0 0) 0x7efd4c09ff30 con 0x7efd5c074a60 2024-04-24T06:41:05.227 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:41:05.227 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1735462750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1735462750},{"type":"v1","addr":"172.21.15.5:6837","nonce":1735462750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[]} 2024-04-24T06:41:05.227 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 5 2024-04-24T06:41:05.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:05.227+0000 7efd61000640 1 -- 172.21.15.5:0/1492776920 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7efd400a3c20 msgr2=0x7efd400a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:05.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:05.227+0000 7efd61000640 1 --2- 172.21.15.5:0/1492776920 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7efd400a3c20 0x7efd400a60e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7efd44009900 tx=0x7efd440315b0 comp rx=0 tx=0).stop 2024-04-24T06:41:05.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:05.227+0000 7efd61000640 1 -- 172.21.15.5:0/1492776920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd5c074a60 msgr2=0x7efd5c19e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:05.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:05.227+0000 7efd61000640 1 --2- 172.21.15.5:0/1492776920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd5c074a60 0x7efd5c19e890 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efd4c009d50 tx=0x7efd4c00c6a0 comp rx=0 tx=0).stop 2024-04-24T06:41:05.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:05.228+0000 7efd61000640 1 -- 172.21.15.5:0/1492776920 shutdown_connections 2024-04-24T06:41:05.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:05.228+0000 7efd61000640 1 --2- 172.21.15.5:0/1492776920 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7efd400a3c20 0x7efd400a60e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:05.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:05.228+0000 7efd61000640 1 --2- 172.21.15.5:0/1492776920 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd5c10e500 0x7efd5c1a5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:05.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:05.228+0000 7efd61000640 1 --2- 172.21.15.5:0/1492776920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd5c076040 0x7efd5c19edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:05.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:05.228+0000 7efd61000640 1 --2- 172.21.15.5:0/1492776920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd5c074a60 0x7efd5c19e890 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:05.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:05.228+0000 7efd61000640 1 -- 172.21.15.5:0/1492776920 >> 172.21.15.5:0/1492776920 conn(0x7efd5c0fd4c0 msgr2=0x7efd5c1018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:05.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:05.228+0000 7efd61000640 1 -- 172.21.15.5:0/1492776920 shutdown_connections 2024-04-24T06:41:05.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:05.228+0000 7efd61000640 1 -- 172.21.15.5:0/1492776920 wait complete. 2024-04-24T06:41:05.245 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:04 smithi005 ceph-mon[27798]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:05.322 INFO:journalctl@ceph.mds.e.smithi037.stdout:Apr 24 06:41:05 smithi037 podman[62703]: 2024-04-24 06:41:05.073210294 +0000 UTC m=+0.018330443 image pull b5c3183062a6ce19f361681cc58de6e3ac1de1a54c80e6094ec5695237cd98d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b 2024-04-24T06:41:05.322 INFO:journalctl@ceph.mds.e.smithi037.stdout:Apr 24 06:41:05 smithi037 podman[62703]: 2024-04-24 06:41:05.204338204 +0000 UTC m=+0.149458353 container create c97d3e315a8d120e55ec98250377cd69c9c49437bee9cb24ebbf33712baaa82b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-e, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T06:41:05.769 INFO:journalctl@ceph.mds.e.smithi037.stdout:Apr 24 06:41:05 smithi037 podman[62703]: 2024-04-24 06:41:05.415247207 +0000 UTC m=+0.360367358 container init c97d3e315a8d120e55ec98250377cd69c9c49437bee9cb24ebbf33712baaa82b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-e, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-24T06:41:05.769 INFO:journalctl@ceph.mds.e.smithi037.stdout:Apr 24 06:41:05 smithi037 podman[62703]: 2024-04-24 06:41:05.418626693 +0000 UTC m=+0.363746850 container start c97d3e315a8d120e55ec98250377cd69c9c49437bee9cb24ebbf33712baaa82b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-24T06:41:05.769 INFO:journalctl@ceph.mds.e.smithi037.stdout:Apr 24 06:41:05 smithi037 ceph-mds[62722]: -- 172.21.15.37:0/2182782472 <== mon.2 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b8f04a7a40 con 0x55b8f047ac00 2024-04-24T06:41:05.769 INFO:journalctl@ceph.mds.e.smithi037.stdout:Apr 24 06:41:05 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-e[62718]: starting mds.e at 2024-04-24T06:41:05.769 INFO:journalctl@ceph.mds.e.smithi037.stdout:Apr 24 06:41:05 smithi037 bash[62703]: c97d3e315a8d120e55ec98250377cd69c9c49437bee9cb24ebbf33712baaa82b 2024-04-24T06:41:05.769 INFO:journalctl@ceph.mds.e.smithi037.stdout:Apr 24 06:41:05 smithi037 systemd[1]: Started Ceph mds.e for 43e9bad0-0204-11ef-bc93-c7b262605968. 2024-04-24T06:41:06.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:05 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/1492776920' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:06.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:05 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:06.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:05 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:06.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:05 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:06.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:05 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T06:41:06.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:05 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T06:41:06.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:05 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:41:06.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:05 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1492776920' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:06.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:05 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:06.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:05 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:06.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:05 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:06.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:05 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T06:41:06.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:05 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T06:41:06.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:05 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:41:06.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:05 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/1492776920' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:06.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:05 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:06.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:05 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:06.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:05 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:06.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:05 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T06:41:06.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:05 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T06:41:06.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:05 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:41:07.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:06 smithi137 ceph-mon[33017]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:07.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:06 smithi137 ceph-mon[33017]: Deploying daemon mds.f on smithi137 2024-04-24T06:41:07.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:06 smithi137 ceph-mon[33017]: mds.? [v2:172.21.15.37:6834/3705638847,v1:172.21.15.37:6835/3705638847] up:boot 2024-04-24T06:41:07.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:06 smithi137 ceph-mon[33017]: fsmap 5 up:standby 2024-04-24T06:41:07.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:06 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T06:41:07.248 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:41:07.249 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:06 smithi037 ceph-mon[32864]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:07.249 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:06 smithi037 ceph-mon[32864]: Deploying daemon mds.f on smithi137 2024-04-24T06:41:07.249 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:06 smithi037 ceph-mon[32864]: mds.? [v2:172.21.15.37:6834/3705638847,v1:172.21.15.37:6835/3705638847] up:boot 2024-04-24T06:41:07.249 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:06 smithi037 ceph-mon[32864]: fsmap 5 up:standby 2024-04-24T06:41:07.249 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:06 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T06:41:07.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:06 smithi005 ceph-mon[27798]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:07.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:06 smithi005 ceph-mon[27798]: Deploying daemon mds.f on smithi137 2024-04-24T06:41:07.258 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:06 smithi005 ceph-mon[27798]: mds.? [v2:172.21.15.37:6834/3705638847,v1:172.21.15.37:6835/3705638847] up:boot 2024-04-24T06:41:07.258 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:06 smithi005 ceph-mon[27798]: fsmap 5 up:standby 2024-04-24T06:41:07.258 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:06 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T06:41:07.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.472+0000 7f42f9e00640 1 -- 172.21.15.5:0/4234340355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42f41029b0 msgr2=0x7f42f4104da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:07.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.472+0000 7f42f9e00640 1 --2- 172.21.15.5:0/4234340355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42f41029b0 0x7f42f4104da0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f42e4009860 tx=0x7f42e402a340 comp rx=0 tx=0).stop 2024-04-24T06:41:07.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.472+0000 7f42f9e00640 1 -- 172.21.15.5:0/4234340355 shutdown_connections 2024-04-24T06:41:07.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.472+0000 7f42f9e00640 1 --2- 172.21.15.5:0/4234340355 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42f4107c10 0x7f42f410a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:07.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.472+0000 7f42f9e00640 1 --2- 172.21.15.5:0/4234340355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42f41052e0 0x7f42f41076d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:07.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.472+0000 7f42f9e00640 1 --2- 172.21.15.5:0/4234340355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42f41029b0 0x7f42f4104da0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:07.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.472+0000 7f42f9e00640 1 -- 172.21.15.5:0/4234340355 >> 172.21.15.5:0/4234340355 conn(0x7f42f40fc3e0 msgr2=0x7f42f40fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:07.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.472+0000 7f42f9e00640 1 -- 172.21.15.5:0/4234340355 shutdown_connections 2024-04-24T06:41:07.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.472+0000 7f42f9e00640 1 -- 172.21.15.5:0/4234340355 wait complete. 2024-04-24T06:41:07.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.473+0000 7f42f9e00640 1 Processor -- start 2024-04-24T06:41:07.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.474+0000 7f42f9e00640 1 -- start start 2024-04-24T06:41:07.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.474+0000 7f42f9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42f41029b0 0x7f42f419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:07.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.474+0000 7f42f9e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42f41052e0 0x7f42f419a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:07.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.474+0000 7f42f9e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42f4107c10 0x7f42f41a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:07.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.474+0000 7f42f9e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f42f410dbb0 con 0x7f42f41029b0 2024-04-24T06:41:07.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.474+0000 7f42f9e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f42f410da30 con 0x7f42f41052e0 2024-04-24T06:41:07.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.474+0000 7f42f9e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f42f410dd30 con 0x7f42f4107c10 2024-04-24T06:41:07.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.474+0000 7f42ebe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42f41052e0 0x7f42f419a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:07.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.474+0000 7f42f8a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42f41029b0 0x7f42f419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:07.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.474+0000 7f42ebe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42f41052e0 0x7f42f419a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:51004/0 (socket says 172.21.15.5:51004) 2024-04-24T06:41:07.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.474+0000 7f42ebe00640 1 -- 172.21.15.5:0/4056868362 learned_addr learned my addr 172.21.15.5:0/4056868362 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:41:07.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.475+0000 7f42f9400640 1 --2- 172.21.15.5:0/4056868362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42f4107c10 0x7f42f41a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:07.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.475+0000 7f42ebe00640 1 -- 172.21.15.5:0/4056868362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42f4107c10 msgr2=0x7f42f41a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:07.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.475+0000 7f42ebe00640 1 --2- 172.21.15.5:0/4056868362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42f4107c10 0x7f42f41a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:07.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.475+0000 7f42ebe00640 1 -- 172.21.15.5:0/4056868362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42f41029b0 msgr2=0x7f42f419a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:07.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.475+0000 7f42ebe00640 1 --2- 172.21.15.5:0/4056868362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42f41029b0 0x7f42f419a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:07.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.475+0000 7f42ebe00640 1 -- 172.21.15.5:0/4056868362 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f41a2040 con 0x7f42f41052e0 2024-04-24T06:41:07.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.475+0000 7f42f9400640 1 --2- 172.21.15.5:0/4056868362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42f4107c10 0x7f42f41a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:41:07.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.475+0000 7f42ebe00640 1 --2- 172.21.15.5:0/4056868362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42f41052e0 0x7f42f419a960 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f42dc00e9f0 tx=0x7f42dc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:07.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.475+0000 7f42f2000640 1 -- 172.21.15.5:0/4056868362 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc00ccc0 con 0x7f42f41052e0 2024-04-24T06:41:07.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.475+0000 7f42f2000640 1 -- 172.21.15.5:0/4056868362 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42dc010910 con 0x7f42f41052e0 2024-04-24T06:41:07.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.475+0000 7f42f2000640 1 -- 172.21.15.5:0/4056868362 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc018960 con 0x7f42f41052e0 2024-04-24T06:41:07.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.475+0000 7f42f9e00640 1 -- 172.21.15.5:0/4056868362 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42f41a2330 con 0x7f42f41052e0 2024-04-24T06:41:07.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.476+0000 7f42f9e00640 1 -- 172.21.15.5:0/4056868362 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42f41a28d0 con 0x7f42f41052e0 2024-04-24T06:41:07.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.477+0000 7f42f9e00640 1 -- 172.21.15.5:0/4056868362 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42f4109760 con 0x7f42f41052e0 2024-04-24T06:41:07.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.479+0000 7f42f2000640 1 -- 172.21.15.5:0/4056868362 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f42dc010ac0 con 0x7f42f41052e0 2024-04-24T06:41:07.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.480+0000 7f42f2000640 1 --2- 172.21.15.5:0/4056868362 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f42d80a3c20 0x7f42d80a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:07.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.480+0000 7f42f2000640 1 -- 172.21.15.5:0/4056868362 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f42dc0d02e0 con 0x7f42f41052e0 2024-04-24T06:41:07.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.480+0000 7f42f8a00640 1 --2- 172.21.15.5:0/4056868362 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f42d80a3c20 0x7f42d80a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:07.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.481+0000 7f42f8a00640 1 --2- 172.21.15.5:0/4056868362 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f42d80a3c20 0x7f42d80a60e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f42e4009830 tx=0x7f42e4030550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:07.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.484+0000 7f42f2000640 1 -- 172.21.15.5:0/4056868362 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f42dc09d0c0 con 0x7f42f41052e0 2024-04-24T06:41:07.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.730+0000 7f42f9e00640 1 -- 172.21.15.5:0/4056868362 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f42f402d620 con 0x7f42f41052e0 2024-04-24T06:41:07.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.731+0000 7f42f2000640 1 -- 172.21.15.5:0/4056868362 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4687 (secure 0 0 0) 0x7f42dc09c9c0 con 0x7f42f41052e0 2024-04-24T06:41:07.734 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:41:07.734 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1735462750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1735462750},{"type":"v1","addr":"172.21.15.5:6837","nonce":1735462750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3705638847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3705638847},{"type":"v1","addr":"172.21.15.37:6835","nonce":3705638847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-24T06:41:07.734 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 6 2024-04-24T06:41:07.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.734+0000 7f42f9e00640 1 -- 172.21.15.5:0/4056868362 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f42d80a3c20 msgr2=0x7f42d80a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:07.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.734+0000 7f42f9e00640 1 --2- 172.21.15.5:0/4056868362 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f42d80a3c20 0x7f42d80a60e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f42e4009830 tx=0x7f42e4030550 comp rx=0 tx=0).stop 2024-04-24T06:41:07.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.734+0000 7f42f9e00640 1 -- 172.21.15.5:0/4056868362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42f41052e0 msgr2=0x7f42f419a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:07.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.734+0000 7f42f9e00640 1 --2- 172.21.15.5:0/4056868362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42f41052e0 0x7f42f419a960 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f42dc00e9f0 tx=0x7f42dc00eeb0 comp rx=0 tx=0).stop 2024-04-24T06:41:07.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.734+0000 7f42f9e00640 1 -- 172.21.15.5:0/4056868362 shutdown_connections 2024-04-24T06:41:07.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.734+0000 7f42f9e00640 1 --2- 172.21.15.5:0/4056868362 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f42d80a3c20 0x7f42d80a60e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:07.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.734+0000 7f42f9e00640 1 --2- 172.21.15.5:0/4056868362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42f4107c10 0x7f42f41a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:07.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.734+0000 7f42f9e00640 1 --2- 172.21.15.5:0/4056868362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42f41052e0 0x7f42f419a960 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:07.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.734+0000 7f42f9e00640 1 --2- 172.21.15.5:0/4056868362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42f41029b0 0x7f42f419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:07.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.734+0000 7f42f9e00640 1 -- 172.21.15.5:0/4056868362 >> 172.21.15.5:0/4056868362 conn(0x7f42f40fc3e0 msgr2=0x7f42f4102140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:07.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.734+0000 7f42f9e00640 1 -- 172.21.15.5:0/4056868362 shutdown_connections 2024-04-24T06:41:07.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:07.734+0000 7f42f9e00640 1 -- 172.21.15.5:0/4056868362 wait complete. 2024-04-24T06:41:08.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:07 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/4056868362' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:08.122 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:07 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/4056868362' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:08.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:07 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/4056868362' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:08.446 INFO:journalctl@ceph.mds.f.smithi137.stdout:Apr 24 06:41:08 smithi137 podman[63440]: 2024-04-24 06:41:08.368006085 +0000 UTC m=+0.143950626 container create 8a32b4df1c7fbdbbde4a59fa66064e7bad0177a0e1fe0a7f38dbbbc858413493 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-f, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-24T06:41:08.861 INFO:journalctl@ceph.mds.f.smithi137.stdout:Apr 24 06:41:08 smithi137 podman[63440]: 2024-04-24 06:41:08.595597617 +0000 UTC m=+0.371542158 container init 8a32b4df1c7fbdbbde4a59fa66064e7bad0177a0e1fe0a7f38dbbbc858413493 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-f, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-24T06:41:08.861 INFO:journalctl@ceph.mds.f.smithi137.stdout:Apr 24 06:41:08 smithi137 podman[63440]: 2024-04-24 06:41:08.598142902 +0000 UTC m=+0.374087453 container start 8a32b4df1c7fbdbbde4a59fa66064e7bad0177a0e1fe0a7f38dbbbc858413493 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-f, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T06:41:08.861 INFO:journalctl@ceph.mds.f.smithi137.stdout:Apr 24 06:41:08 smithi137 ceph-mds[63459]: -- 172.21.15.137:0/1884517413 <== mon.1 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e712ccba40 con 0x55e712c9e800 2024-04-24T06:41:08.861 INFO:journalctl@ceph.mds.f.smithi137.stdout:Apr 24 06:41:08 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-f[63455]: starting mds.f at 2024-04-24T06:41:08.862 INFO:journalctl@ceph.mds.f.smithi137.stdout:Apr 24 06:41:08 smithi137 bash[63440]: 8a32b4df1c7fbdbbde4a59fa66064e7bad0177a0e1fe0a7f38dbbbc858413493 2024-04-24T06:41:08.862 INFO:journalctl@ceph.mds.f.smithi137.stdout:Apr 24 06:41:08 smithi137 systemd[1]: Started Ceph mds.f for 43e9bad0-0204-11ef-bc93-c7b262605968. 2024-04-24T06:41:09.166 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:08 smithi137 ceph-mon[33017]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:09.166 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:08 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:09.166 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:08 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:09.166 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:08 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:09.166 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:08 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T06:41:09.166 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:08 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T06:41:09.166 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:08 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:41:09.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:08 smithi005 ceph-mon[27798]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:09.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:08 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:09.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:08 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:09.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:08 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:09.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:08 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T06:41:09.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:08 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T06:41:09.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:08 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:41:09.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:08 smithi037 ceph-mon[32864]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:09.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:08 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:09.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:08 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:09.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:08 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:09.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:08 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T06:41:09.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:08 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T06:41:09.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:08 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:41:09.758 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:41:09.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:09.973+0000 7f92e8c00640 1 -- 172.21.15.5:0/2800121507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92e4109740 msgr2=0x7f92e4109bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:09.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:09.973+0000 7f92e8c00640 1 --2- 172.21.15.5:0/2800121507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92e4109740 0x7f92e4109bc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f92cc009ca0 tx=0x7f92cc02a8c0 comp rx=0 tx=0).stop 2024-04-24T06:41:09.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:09.974+0000 7f92e8c00640 1 -- 172.21.15.5:0/2800121507 shutdown_connections 2024-04-24T06:41:09.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:09.974+0000 7f92e8c00640 1 --2- 172.21.15.5:0/2800121507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92e410a100 0x7f92e4110990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:09.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:09.974+0000 7f92e8c00640 1 --2- 172.21.15.5:0/2800121507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92e4109740 0x7f92e4109bc0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:09.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:09.974+0000 7f92e8c00640 1 --2- 172.21.15.5:0/2800121507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92e41085a0 0x7f92e41089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:09.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:09.974+0000 7f92e8c00640 1 -- 172.21.15.5:0/2800121507 >> 172.21.15.5:0/2800121507 conn(0x7f92e40774f0 msgr2=0x7f92e4079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:09.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:09.974+0000 7f92e8c00640 1 -- 172.21.15.5:0/2800121507 shutdown_connections 2024-04-24T06:41:09.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:09.974+0000 7f92e8c00640 1 -- 172.21.15.5:0/2800121507 wait complete. 2024-04-24T06:41:09.977 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:09 smithi137 ceph-mon[33017]: Deploying daemon mds.c on smithi137 2024-04-24T06:41:09.978 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:09 smithi137 ceph-mon[33017]: mds.? [v2:172.21.15.137:6834/3217754215,v1:172.21.15.137:6835/3217754215] up:boot 2024-04-24T06:41:09.978 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:09 smithi137 ceph-mon[33017]: fsmap 6 up:standby 2024-04-24T06:41:09.978 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:09 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T06:41:09.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:09.975+0000 7f92e8c00640 1 Processor -- start 2024-04-24T06:41:09.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:09.975+0000 7f92e8c00640 1 -- start start 2024-04-24T06:41:09.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:09.975+0000 7f92e8c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92e41085a0 0x7f92e419e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:09.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:09.975+0000 7f92e8c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92e4109740 0x7f92e419ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:09.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:09.975+0000 7f92e8c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92e410a100 0x7f92e41a5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:09.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:09.975+0000 7f92e8c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f92e4114230 con 0x7f92e41085a0 2024-04-24T06:41:09.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:09.975+0000 7f92e8c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f92e41140b0 con 0x7f92e410a100 2024-04-24T06:41:09.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:09.975+0000 7f92e8c00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f92e41143b0 con 0x7f92e4109740 2024-04-24T06:41:09.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:09.975+0000 7f92daa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92e4109740 0x7f92e419ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:09.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:09.975+0000 7f92e3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92e41085a0 0x7f92e419e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:09.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:09.976+0000 7f92daa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92e4109740 0x7f92e419ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:51712/0 (socket says 172.21.15.5:51712) 2024-04-24T06:41:09.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:09.976+0000 7f92daa00640 1 -- 172.21.15.5:0/2953042754 learned_addr learned my addr 172.21.15.5:0/2953042754 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:41:09.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:09.976+0000 7f92e3e00640 1 --2- 172.21.15.5:0/2953042754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92e410a100 0x7f92e41a5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:09.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:09.976+0000 7f92e3400640 1 -- 172.21.15.5:0/2953042754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92e4109740 msgr2=0x7f92e419ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:09.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:09.976+0000 7f92e3400640 1 --2- 172.21.15.5:0/2953042754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92e4109740 0x7f92e419ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:09.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:09.976+0000 7f92e3400640 1 -- 172.21.15.5:0/2953042754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92e410a100 msgr2=0x7f92e41a5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:09.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:09.976+0000 7f92e3400640 1 --2- 172.21.15.5:0/2953042754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92e410a100 0x7f92e41a5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:09.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:09.976+0000 7f92e3400640 1 -- 172.21.15.5:0/2953042754 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92e41a63f0 con 0x7f92e41085a0 2024-04-24T06:41:09.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:09.976+0000 7f92e3400640 1 --2- 172.21.15.5:0/2953042754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92e41085a0 0x7f92e419e7d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f92d400e9c0 tx=0x7f92d400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:09.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:09.977+0000 7f92e0c00640 1 -- 172.21.15.5:0/2953042754 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d400ccc0 con 0x7f92e41085a0 2024-04-24T06:41:09.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:09.977+0000 7f92e0c00640 1 -- 172.21.15.5:0/2953042754 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92d40035f0 con 0x7f92e41085a0 2024-04-24T06:41:09.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:09.977+0000 7f92e0c00640 1 -- 172.21.15.5:0/2953042754 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d400ce60 con 0x7f92e41085a0 2024-04-24T06:41:09.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:09.977+0000 7f92e8c00640 1 -- 172.21.15.5:0/2953042754 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92e41a66e0 con 0x7f92e41085a0 2024-04-24T06:41:09.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:09.977+0000 7f92e8c00640 1 -- 172.21.15.5:0/2953042754 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92e41a6c30 con 0x7f92e41085a0 2024-04-24T06:41:09.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:09.978+0000 7f92e8c00640 1 -- 172.21.15.5:0/2953042754 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92e410f810 con 0x7f92e41085a0 2024-04-24T06:41:09.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:09.980+0000 7f92e0c00640 1 -- 172.21.15.5:0/2953042754 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f92d40037a0 con 0x7f92e41085a0 2024-04-24T06:41:09.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:09.983+0000 7f92e0c00640 1 --2- 172.21.15.5:0/2953042754 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f92c80a3ba0 0x7f92c80a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:09.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:09.983+0000 7f92e0c00640 1 -- 172.21.15.5:0/2953042754 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f92d40d0440 con 0x7f92e41085a0 2024-04-24T06:41:09.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:09.983+0000 7f92daa00640 1 --2- 172.21.15.5:0/2953042754 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f92c80a3ba0 0x7f92c80a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:09.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:09.984+0000 7f92daa00640 1 --2- 172.21.15.5:0/2953042754 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f92c80a3ba0 0x7f92c80a6060 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f92cc005fe0 tx=0x7f92cc005f70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:09.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:09.986+0000 7f92e0c00640 1 -- 172.21.15.5:0/2953042754 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f92d409d220 con 0x7f92e41085a0 2024-04-24T06:41:10.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:10.236+0000 7f92e8c00640 1 -- 172.21.15.5:0/2953042754 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92e402d620 con 0x7f92e41085a0 2024-04-24T06:41:10.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:10.238+0000 7f92e0c00640 1 -- 172.21.15.5:0/2953042754 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5515 (secure 0 0 0) 0x7f92d409cb20 con 0x7f92e41085a0 2024-04-24T06:41:10.240 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:41:10.240 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1735462750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1735462750},{"type":"v1","addr":"172.21.15.5:6837","nonce":1735462750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3705638847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3705638847},{"type":"v1","addr":"172.21.15.37:6835","nonce":3705638847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24520,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3217754215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3217754215},{"type":"v1","addr":"172.21.15.137:6835","nonce":3217754215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-24T06:41:10.240 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 7 2024-04-24T06:41:10.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:10.240+0000 7f92e8c00640 1 -- 172.21.15.5:0/2953042754 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f92c80a3ba0 msgr2=0x7f92c80a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:10.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:10.240+0000 7f92e8c00640 1 --2- 172.21.15.5:0/2953042754 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f92c80a3ba0 0x7f92c80a6060 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f92cc005fe0 tx=0x7f92cc005f70 comp rx=0 tx=0).stop 2024-04-24T06:41:10.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:10.240+0000 7f92e8c00640 1 -- 172.21.15.5:0/2953042754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92e41085a0 msgr2=0x7f92e419e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:10.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:10.240+0000 7f92e8c00640 1 --2- 172.21.15.5:0/2953042754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92e41085a0 0x7f92e419e7d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f92d400e9c0 tx=0x7f92d400ee80 comp rx=0 tx=0).stop 2024-04-24T06:41:10.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:10.240+0000 7f92e8c00640 1 -- 172.21.15.5:0/2953042754 shutdown_connections 2024-04-24T06:41:10.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:10.240+0000 7f92e8c00640 1 --2- 172.21.15.5:0/2953042754 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f92c80a3ba0 0x7f92c80a6060 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:10.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:10.240+0000 7f92e8c00640 1 --2- 172.21.15.5:0/2953042754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92e410a100 0x7f92e41a5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:10.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:10.240+0000 7f92e8c00640 1 --2- 172.21.15.5:0/2953042754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92e4109740 0x7f92e419ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:10.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:10.240+0000 7f92e8c00640 1 --2- 172.21.15.5:0/2953042754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92e41085a0 0x7f92e419e7d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:10.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:10.240+0000 7f92e8c00640 1 -- 172.21.15.5:0/2953042754 >> 172.21.15.5:0/2953042754 conn(0x7f92e40774f0 msgr2=0x7f92e410c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:10.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:10.241+0000 7f92e8c00640 1 -- 172.21.15.5:0/2953042754 shutdown_connections 2024-04-24T06:41:10.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:10.241+0000 7f92e8c00640 1 -- 172.21.15.5:0/2953042754 wait complete. 2024-04-24T06:41:10.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:09 smithi005 ceph-mon[27798]: Deploying daemon mds.c on smithi137 2024-04-24T06:41:10.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:09 smithi005 ceph-mon[27798]: mds.? [v2:172.21.15.137:6834/3217754215,v1:172.21.15.137:6835/3217754215] up:boot 2024-04-24T06:41:10.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:09 smithi005 ceph-mon[27798]: fsmap 6 up:standby 2024-04-24T06:41:10.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:09 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T06:41:10.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:09 smithi037 ceph-mon[32864]: Deploying daemon mds.c on smithi137 2024-04-24T06:41:10.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:09 smithi037 ceph-mon[32864]: mds.? [v2:172.21.15.137:6834/3217754215,v1:172.21.15.137:6835/3217754215] up:boot 2024-04-24T06:41:10.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:09 smithi037 ceph-mon[32864]: fsmap 6 up:standby 2024-04-24T06:41:10.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:09 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T06:41:10.871 INFO:journalctl@ceph.mds.c.smithi137.stdout:Apr 24 06:41:10 smithi137 systemd[1]: Starting Ceph mds.c for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:41:11.141 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:10 smithi137 ceph-mon[33017]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:11.141 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:10 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/2953042754' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:11.141 INFO:journalctl@ceph.mds.c.smithi137.stdout:Apr 24 06:41:10 smithi137 podman[63746]: 2024-04-24 06:41:10.832941671 +0000 UTC m=+0.017385784 image pull b5c3183062a6ce19f361681cc58de6e3ac1de1a54c80e6094ec5695237cd98d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b 2024-04-24T06:41:11.141 INFO:journalctl@ceph.mds.c.smithi137.stdout:Apr 24 06:41:10 smithi137 podman[63746]: 2024-04-24 06:41:10.946321144 +0000 UTC m=+0.130765245 container create 2a46530d84e88b5e61af5ee6d1dadae4a3809f4476b403f81b1c4422658bec37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T06:41:11.141 INFO:journalctl@ceph.mds.c.smithi137.stdout:Apr 24 06:41:11 smithi137 podman[63746]: 2024-04-24 06:41:11.140567726 +0000 UTC m=+0.325011827 container init 2a46530d84e88b5e61af5ee6d1dadae4a3809f4476b403f81b1c4422658bec37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-c, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-24T06:41:11.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:10 smithi005 ceph-mon[27798]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:11.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:10 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/2953042754' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:11.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:10 smithi037 ceph-mon[32864]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:11.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:10 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/2953042754' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:11.446 INFO:journalctl@ceph.mds.c.smithi137.stdout:Apr 24 06:41:11 smithi137 podman[63746]: 2024-04-24 06:41:11.143294097 +0000 UTC m=+0.327738199 container start 2a46530d84e88b5e61af5ee6d1dadae4a3809f4476b403f81b1c4422658bec37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T06:41:11.446 INFO:journalctl@ceph.mds.c.smithi137.stdout:Apr 24 06:41:11 smithi137 ceph-mds[63766]: -- 172.21.15.137:0/1254733351 <== mon.1 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55db5106da40 con 0x55db51040c00 2024-04-24T06:41:11.446 INFO:journalctl@ceph.mds.c.smithi137.stdout:Apr 24 06:41:11 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-c[63762]: starting mds.c at 2024-04-24T06:41:11.446 INFO:journalctl@ceph.mds.c.smithi137.stdout:Apr 24 06:41:11 smithi137 bash[63746]: 2a46530d84e88b5e61af5ee6d1dadae4a3809f4476b403f81b1c4422658bec37 2024-04-24T06:41:11.446 INFO:journalctl@ceph.mds.c.smithi137.stdout:Apr 24 06:41:11 smithi137 systemd[1]: Started Ceph mds.c for 43e9bad0-0204-11ef-bc93-c7b262605968. 2024-04-24T06:41:12.263 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:41:12.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.486+0000 7f54f4200640 1 -- 172.21.15.5:0/245586398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54ec1032b0 msgr2=0x7f54ec103730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:12.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.486+0000 7f54f4200640 1 --2- 172.21.15.5:0/245586398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54ec1032b0 0x7f54ec103730 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f54e0009ca0 tx=0x7f54e002a8c0 comp rx=0 tx=0).stop 2024-04-24T06:41:12.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.486+0000 7f54f4200640 1 -- 172.21.15.5:0/245586398 shutdown_connections 2024-04-24T06:41:12.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.486+0000 7f54f4200640 1 --2- 172.21.15.5:0/245586398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54ec103c70 0x7f54ec10a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:12.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.486+0000 7f54f4200640 1 --2- 172.21.15.5:0/245586398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54ec1032b0 0x7f54ec103730 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:12.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.486+0000 7f54f4200640 1 --2- 172.21.15.5:0/245586398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54ec1020c0 0x7f54ec1024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:12.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.486+0000 7f54f4200640 1 -- 172.21.15.5:0/245586398 >> 172.21.15.5:0/245586398 conn(0x7f54ec0fd630 msgr2=0x7f54ec0ffa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:12.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.486+0000 7f54f4200640 1 -- 172.21.15.5:0/245586398 shutdown_connections 2024-04-24T06:41:12.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.486+0000 7f54f4200640 1 -- 172.21.15.5:0/245586398 wait complete. 2024-04-24T06:41:12.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.487+0000 7f54f4200640 1 Processor -- start 2024-04-24T06:41:12.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.487+0000 7f54f4200640 1 -- start start 2024-04-24T06:41:12.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.488+0000 7f54f4200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54ec1020c0 0x7f54ec198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:12.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.488+0000 7f54f4200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54ec1032b0 0x7f54ec198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:12.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.488+0000 7f54f4200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54ec103c70 0x7f54ec19f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:12.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.488+0000 7f54f4200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f54ec10dd30 con 0x7f54ec103c70 2024-04-24T06:41:12.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.488+0000 7f54f4200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f54ec10dbb0 con 0x7f54ec1020c0 2024-04-24T06:41:12.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.488+0000 7f54f4200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f54ec10deb0 con 0x7f54ec1032b0 2024-04-24T06:41:12.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.488+0000 7f54f1800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54ec1020c0 0x7f54ec198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:12.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.488+0000 7f54f2200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54ec103c70 0x7f54ec19f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:12.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.488+0000 7f54f1800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54ec1020c0 0x7f54ec198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:51058/0 (socket says 172.21.15.5:51058) 2024-04-24T06:41:12.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.488+0000 7f54f1800640 1 -- 172.21.15.5:0/3981167822 learned_addr learned my addr 172.21.15.5:0/3981167822 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:41:12.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.488+0000 7f54f0e00640 1 --2- 172.21.15.5:0/3981167822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54ec1032b0 0x7f54ec198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:12.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.488+0000 7f54f1800640 1 -- 172.21.15.5:0/3981167822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54ec1032b0 msgr2=0x7f54ec198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:12.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.488+0000 7f54f1800640 1 --2- 172.21.15.5:0/3981167822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54ec1032b0 0x7f54ec198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:12.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.488+0000 7f54f1800640 1 -- 172.21.15.5:0/3981167822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54ec103c70 msgr2=0x7f54ec19f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:12.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.488+0000 7f54f1800640 1 --2- 172.21.15.5:0/3981167822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54ec103c70 0x7f54ec19f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:12.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.488+0000 7f54f1800640 1 -- 172.21.15.5:0/3981167822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54ec19ff30 con 0x7f54ec1020c0 2024-04-24T06:41:12.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.488+0000 7f54f2200640 1 --2- 172.21.15.5:0/3981167822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54ec103c70 0x7f54ec19f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:41:12.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.489+0000 7f54f1800640 1 --2- 172.21.15.5:0/3981167822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54ec1020c0 0x7f54ec198310 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f54dc00e9c0 tx=0x7f54dc00edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:12.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.489+0000 7f54ea000640 1 -- 172.21.15.5:0/3981167822 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54dc00ccc0 con 0x7f54ec1020c0 2024-04-24T06:41:12.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.489+0000 7f54ea000640 1 -- 172.21.15.5:0/3981167822 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54dc0035f0 con 0x7f54ec1020c0 2024-04-24T06:41:12.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.489+0000 7f54ea000640 1 -- 172.21.15.5:0/3981167822 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54dc00ce60 con 0x7f54ec1020c0 2024-04-24T06:41:12.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.489+0000 7f54f4200640 1 -- 172.21.15.5:0/3981167822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54ec1a0220 con 0x7f54ec1020c0 2024-04-24T06:41:12.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.489+0000 7f54f4200640 1 -- 172.21.15.5:0/3981167822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54ec1a0790 con 0x7f54ec1020c0 2024-04-24T06:41:12.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.490+0000 7f54f4200640 1 -- 172.21.15.5:0/3981167822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54ec1093e0 con 0x7f54ec1020c0 2024-04-24T06:41:12.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.492+0000 7f54ea000640 1 -- 172.21.15.5:0/3981167822 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f54dc0037a0 con 0x7f54ec1020c0 2024-04-24T06:41:12.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.493+0000 7f54ea000640 1 --2- 172.21.15.5:0/3981167822 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f54d40a3c30 0x7f54d40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:12.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.494+0000 7f54ea000640 1 -- 172.21.15.5:0/3981167822 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f54dc0d02b0 con 0x7f54ec1020c0 2024-04-24T06:41:12.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.494+0000 7f54f0e00640 1 --2- 172.21.15.5:0/3981167822 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f54d40a3c30 0x7f54d40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:12.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.494+0000 7f54f0e00640 1 --2- 172.21.15.5:0/3981167822 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f54d40a3c30 0x7f54d40a60f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f54e0005e30 tx=0x7f54e0005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:12.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.498+0000 7f54ea000640 1 -- 172.21.15.5:0/3981167822 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f54dc09d090 con 0x7f54ec1020c0 2024-04-24T06:41:12.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:12 smithi137 ceph-mon[33017]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:12.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:12 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:12.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:12 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:12.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:12 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:12.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:12 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T06:41:12.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:12 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T06:41:12.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:12 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:41:12.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:12 smithi137 ceph-mon[33017]: Deploying daemon mds.h on smithi037 2024-04-24T06:41:12.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:12 smithi137 ceph-mon[33017]: mds.? [v2:172.21.15.137:6836/3044222976,v1:172.21.15.137:6837/3044222976] up:boot 2024-04-24T06:41:12.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:12 smithi137 ceph-mon[33017]: fsmap 7 up:standby 2024-04-24T06:41:12.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:12 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T06:41:12.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.743+0000 7f54f4200640 1 -- 172.21.15.5:0/3981167822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f54ec02d620 con 0x7f54ec1020c0 2024-04-24T06:41:12.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.745+0000 7f54ea000640 1 -- 172.21.15.5:0/3981167822 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6343 (secure 0 0 0) 0x7f54dc09c990 con 0x7f54ec1020c0 2024-04-24T06:41:12.747 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:41:12.747 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1735462750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1735462750},{"type":"v1","addr":"172.21.15.5:6837","nonce":1735462750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3705638847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3705638847},{"type":"v1","addr":"172.21.15.37:6835","nonce":3705638847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24520,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3217754215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3217754215},{"type":"v1","addr":"172.21.15.137:6835","nonce":3217754215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24532,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/3044222976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3044222976},{"type":"v1","addr":"172.21.15.137:6837","nonce":3044222976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-24T06:41:12.747 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 8 2024-04-24T06:41:12.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.747+0000 7f54f4200640 1 -- 172.21.15.5:0/3981167822 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f54d40a3c30 msgr2=0x7f54d40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:12.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.747+0000 7f54f4200640 1 --2- 172.21.15.5:0/3981167822 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f54d40a3c30 0x7f54d40a60f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f54e0005e30 tx=0x7f54e0005f50 comp rx=0 tx=0).stop 2024-04-24T06:41:12.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.747+0000 7f54f4200640 1 -- 172.21.15.5:0/3981167822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54ec1020c0 msgr2=0x7f54ec198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:12.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.747+0000 7f54f4200640 1 --2- 172.21.15.5:0/3981167822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54ec1020c0 0x7f54ec198310 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f54dc00e9c0 tx=0x7f54dc00edc0 comp rx=0 tx=0).stop 2024-04-24T06:41:12.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.748+0000 7f54f4200640 1 -- 172.21.15.5:0/3981167822 shutdown_connections 2024-04-24T06:41:12.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.748+0000 7f54f4200640 1 --2- 172.21.15.5:0/3981167822 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f54d40a3c30 0x7f54d40a60f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:12.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.748+0000 7f54f4200640 1 --2- 172.21.15.5:0/3981167822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54ec103c70 0x7f54ec19f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:12.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.748+0000 7f54f4200640 1 --2- 172.21.15.5:0/3981167822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54ec1032b0 0x7f54ec198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:12.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.748+0000 7f54f4200640 1 --2- 172.21.15.5:0/3981167822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54ec1020c0 0x7f54ec198310 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:12.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.748+0000 7f54f4200640 1 -- 172.21.15.5:0/3981167822 >> 172.21.15.5:0/3981167822 conn(0x7f54ec0fd630 msgr2=0x7f54ec1065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:12.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.748+0000 7f54f4200640 1 -- 172.21.15.5:0/3981167822 shutdown_connections 2024-04-24T06:41:12.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:12.748+0000 7f54f4200640 1 -- 172.21.15.5:0/3981167822 wait complete. 2024-04-24T06:41:12.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:12 smithi005 ceph-mon[27798]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:12.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:12 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:12.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:12 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:12.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:12 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:12.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:12 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T06:41:12.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:12 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T06:41:12.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:12 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:41:12.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:12 smithi005 ceph-mon[27798]: Deploying daemon mds.h on smithi037 2024-04-24T06:41:12.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:12 smithi005 ceph-mon[27798]: mds.? [v2:172.21.15.137:6836/3044222976,v1:172.21.15.137:6837/3044222976] up:boot 2024-04-24T06:41:12.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:12 smithi005 ceph-mon[27798]: fsmap 7 up:standby 2024-04-24T06:41:12.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:12 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T06:41:12.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:12 smithi037 ceph-mon[32864]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:12.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:12 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:12.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:12 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:12.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:12 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:12.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:12 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T06:41:12.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:12 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T06:41:12.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:12 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:41:12.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:12 smithi037 ceph-mon[32864]: Deploying daemon mds.h on smithi037 2024-04-24T06:41:12.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:12 smithi037 ceph-mon[32864]: mds.? [v2:172.21.15.137:6836/3044222976,v1:172.21.15.137:6837/3044222976] up:boot 2024-04-24T06:41:12.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:12 smithi037 ceph-mon[32864]: fsmap 7 up:standby 2024-04-24T06:41:12.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:12 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T06:41:13.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:13 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/3981167822' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:13.519 INFO:journalctl@ceph.mds.h.smithi037.stdout:Apr 24 06:41:13 smithi037 systemd[1]: Starting Ceph mds.h for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:41:13.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:13 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/3981167822' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:13.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:13 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/3981167822' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:13.911 INFO:journalctl@ceph.mds.h.smithi037.stdout:Apr 24 06:41:13 smithi037 podman[63011]: 2024-04-24 06:41:13.56159778 +0000 UTC m=+0.018322664 image pull b5c3183062a6ce19f361681cc58de6e3ac1de1a54c80e6094ec5695237cd98d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b 2024-04-24T06:41:13.911 INFO:journalctl@ceph.mds.h.smithi037.stdout:Apr 24 06:41:13 smithi037 podman[63011]: 2024-04-24 06:41:13.674611554 +0000 UTC m=+0.131336425 container create 3aaa91ad501856d64c05aa1dff7ed8447b6bb4a0d11259c63183bc608bb1e7a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-24T06:41:13.911 INFO:journalctl@ceph.mds.h.smithi037.stdout:Apr 24 06:41:13 smithi037 podman[63011]: 2024-04-24 06:41:13.910375055 +0000 UTC m=+0.367099930 container init 3aaa91ad501856d64c05aa1dff7ed8447b6bb4a0d11259c63183bc608bb1e7a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-h, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_CLEAN=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T06:41:14.269 INFO:journalctl@ceph.mds.h.smithi037.stdout:Apr 24 06:41:13 smithi037 podman[63011]: 2024-04-24 06:41:13.913050847 +0000 UTC m=+0.369775719 container start 3aaa91ad501856d64c05aa1dff7ed8447b6bb4a0d11259c63183bc608bb1e7a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-h, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T06:41:14.269 INFO:journalctl@ceph.mds.h.smithi037.stdout:Apr 24 06:41:13 smithi037 ceph-mds[63030]: -- 172.21.15.37:0/3770474101 <== mon.2 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55eafedaba40 con 0x55eafed7ec00 2024-04-24T06:41:14.269 INFO:journalctl@ceph.mds.h.smithi037.stdout:Apr 24 06:41:13 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-h[63026]: starting mds.h at 2024-04-24T06:41:14.269 INFO:journalctl@ceph.mds.h.smithi037.stdout:Apr 24 06:41:14 smithi037 bash[63011]: 3aaa91ad501856d64c05aa1dff7ed8447b6bb4a0d11259c63183bc608bb1e7a1 2024-04-24T06:41:14.269 INFO:journalctl@ceph.mds.h.smithi037.stdout:Apr 24 06:41:14 smithi037 systemd[1]: Started Ceph mds.h for 43e9bad0-0204-11ef-bc93-c7b262605968. 2024-04-24T06:41:14.772 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:41:14.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:14.994+0000 7f0761400640 1 -- 172.21.15.5:0/4066300127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f075c1029c0 msgr2=0x7f075c104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:14.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:14.994+0000 7f0761400640 1 --2- 172.21.15.5:0/4066300127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f075c1029c0 0x7f075c104db0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f074c009860 tx=0x7f074c02a340 comp rx=0 tx=0).stop 2024-04-24T06:41:14.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:14.994+0000 7f0761400640 1 -- 172.21.15.5:0/4066300127 shutdown_connections 2024-04-24T06:41:14.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:14.994+0000 7f0761400640 1 --2- 172.21.15.5:0/4066300127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f075c107c20 0x7f075c10a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:14.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:14.994+0000 7f0761400640 1 --2- 172.21.15.5:0/4066300127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f075c1052f0 0x7f075c1076e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:14.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:14.994+0000 7f0761400640 1 --2- 172.21.15.5:0/4066300127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f075c1029c0 0x7f075c104db0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:14.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:14.994+0000 7f0761400640 1 -- 172.21.15.5:0/4066300127 >> 172.21.15.5:0/4066300127 conn(0x7f075c0fc3d0 msgr2=0x7f075c0fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:14.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:14.994+0000 7f0761400640 1 -- 172.21.15.5:0/4066300127 shutdown_connections 2024-04-24T06:41:14.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:14.994+0000 7f0761400640 1 -- 172.21.15.5:0/4066300127 wait complete. 2024-04-24T06:41:14.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:14.995+0000 7f0761400640 1 Processor -- start 2024-04-24T06:41:14.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:14.995+0000 7f0761400640 1 -- start start 2024-04-24T06:41:14.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:14.996+0000 7f0761400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f075c1029c0 0x7f075c19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:14.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:14.996+0000 7f0761400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f075c1052f0 0x7f075c19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:14.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:14.996+0000 7f0761400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f075c107c20 0x7f075c1a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:14.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:14.996+0000 7f0761400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f075c10dbe0 con 0x7f075c107c20 2024-04-24T06:41:14.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:14.996+0000 7f0761400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f075c10da60 con 0x7f075c1029c0 2024-04-24T06:41:14.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:14.996+0000 7f0761400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f075c10dd60 con 0x7f075c1052f0 2024-04-24T06:41:14.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:14.996+0000 7f075be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f075c1029c0 0x7f075c19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:14.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:14.996+0000 7f075be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f075c1029c0 0x7f075c19a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:33156/0 (socket says 172.21.15.5:33156) 2024-04-24T06:41:14.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:14.996+0000 7f0753400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f075c1052f0 0x7f075c19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:14.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:14.996+0000 7f075be00640 1 -- 172.21.15.5:0/341523546 learned_addr learned my addr 172.21.15.5:0/341523546 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:41:14.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:14.996+0000 7f0760a00640 1 --2- 172.21.15.5:0/341523546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f075c107c20 0x7f075c1a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:14.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:14.996+0000 7f075be00640 1 -- 172.21.15.5:0/341523546 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f075c1052f0 msgr2=0x7f075c19a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:14.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:14.996+0000 7f075be00640 1 --2- 172.21.15.5:0/341523546 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f075c1052f0 0x7f075c19a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:14.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:14.996+0000 7f075be00640 1 -- 172.21.15.5:0/341523546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f075c107c20 msgr2=0x7f075c1a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:14.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:14.996+0000 7f075be00640 1 --2- 172.21.15.5:0/341523546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f075c107c20 0x7f075c1a1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:14.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:14.996+0000 7f075be00640 1 -- 172.21.15.5:0/341523546 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f075c1a2090 con 0x7f075c1029c0 2024-04-24T06:41:14.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:14.996+0000 7f0753400640 1 --2- 172.21.15.5:0/341523546 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f075c1052f0 0x7f075c19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:41:14.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:14.997+0000 7f075be00640 1 --2- 172.21.15.5:0/341523546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f075c1029c0 0x7f075c19a470 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f074c035040 tx=0x7f074c033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:15.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:14.997+0000 7f0759600640 1 -- 172.21.15.5:0/341523546 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c037040 con 0x7f075c1029c0 2024-04-24T06:41:15.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:14.997+0000 7f0759600640 1 -- 172.21.15.5:0/341523546 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f074c032770 con 0x7f075c1029c0 2024-04-24T06:41:15.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:14.997+0000 7f0761400640 1 -- 172.21.15.5:0/341523546 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f075c1a2320 con 0x7f075c1029c0 2024-04-24T06:41:15.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:14.997+0000 7f0759600640 1 -- 172.21.15.5:0/341523546 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c0046c0 con 0x7f075c1029c0 2024-04-24T06:41:15.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:14.997+0000 7f0761400640 1 -- 172.21.15.5:0/341523546 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f075c1a2810 con 0x7f075c1029c0 2024-04-24T06:41:15.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:14.998+0000 7f0761400640 1 -- 172.21.15.5:0/341523546 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f075c109770 con 0x7f075c1029c0 2024-04-24T06:41:15.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:15.000+0000 7f0759600640 1 -- 172.21.15.5:0/341523546 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f074c004050 con 0x7f075c1029c0 2024-04-24T06:41:15.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:15.002+0000 7f0759600640 1 --2- 172.21.15.5:0/341523546 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f07400a3ba0 0x7f07400a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:15.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:15.002+0000 7f0759600640 1 -- 172.21.15.5:0/341523546 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f074c0efea0 con 0x7f075c1029c0 2024-04-24T06:41:15.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:15.002+0000 7f0753400640 1 --2- 172.21.15.5:0/341523546 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f07400a3ba0 0x7f07400a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:15.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:15.003+0000 7f0753400640 1 --2- 172.21.15.5:0/341523546 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f07400a3ba0 0x7f07400a6060 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f075c19b980 tx=0x7f0744009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:15.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:15.006+0000 7f0759600640 1 -- 172.21.15.5:0/341523546 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f074c0bcc80 con 0x7f075c1029c0 2024-04-24T06:41:15.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:15 smithi005 ceph-mon[27798]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:15.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:15 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:15.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:15 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:15.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:15 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:15.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:15 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T06:41:15.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:15 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T06:41:15.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:15 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:41:15.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:15 smithi005 ceph-mon[27798]: Deploying daemon mds.i on smithi137 2024-04-24T06:41:15.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:15 smithi005 ceph-mon[27798]: mds.? [v2:172.21.15.37:6836/1390103296,v1:172.21.15.37:6837/1390103296] up:boot 2024-04-24T06:41:15.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:15 smithi005 ceph-mon[27798]: fsmap 8 up:standby 2024-04-24T06:41:15.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:15 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T06:41:15.258 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:15 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T06:41:15.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:15.259+0000 7f0761400640 1 -- 172.21.15.5:0/341523546 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f075c02d620 con 0x7f075c1029c0 2024-04-24T06:41:15.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:15.260+0000 7f0759600640 1 -- 172.21.15.5:0/341523546 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7168 (secure 0 0 0) 0x7f074c0bc580 con 0x7f075c1029c0 2024-04-24T06:41:15.261 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:41:15.261 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1735462750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1735462750},{"type":"v1","addr":"172.21.15.5:6837","nonce":1735462750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3705638847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3705638847},{"type":"v1","addr":"172.21.15.37:6835","nonce":3705638847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24515,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1390103296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1390103296},{"type":"v1","addr":"172.21.15.37:6837","nonce":1390103296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24520,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3217754215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3217754215},{"type":"v1","addr":"172.21.15.137:6835","nonce":3217754215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24532,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/3044222976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3044222976},{"type":"v1","addr":"172.21.15.137:6837","nonce":3044222976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-24T06:41:15.262 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 9 2024-04-24T06:41:15.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:15.263+0000 7f0761400640 1 -- 172.21.15.5:0/341523546 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f07400a3ba0 msgr2=0x7f07400a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:15.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:15.263+0000 7f0761400640 1 --2- 172.21.15.5:0/341523546 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f07400a3ba0 0x7f07400a6060 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f075c19b980 tx=0x7f0744009260 comp rx=0 tx=0).stop 2024-04-24T06:41:15.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:15.263+0000 7f0761400640 1 -- 172.21.15.5:0/341523546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f075c1029c0 msgr2=0x7f075c19a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:15.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:15.263+0000 7f0761400640 1 --2- 172.21.15.5:0/341523546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f075c1029c0 0x7f075c19a470 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f074c035040 tx=0x7f074c033d90 comp rx=0 tx=0).stop 2024-04-24T06:41:15.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:15.263+0000 7f0761400640 1 -- 172.21.15.5:0/341523546 shutdown_connections 2024-04-24T06:41:15.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:15.263+0000 7f0761400640 1 --2- 172.21.15.5:0/341523546 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f07400a3ba0 0x7f07400a6060 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:15.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:15.263+0000 7f0761400640 1 --2- 172.21.15.5:0/341523546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f075c107c20 0x7f075c1a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:15.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:15.263+0000 7f0761400640 1 --2- 172.21.15.5:0/341523546 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f075c1052f0 0x7f075c19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:15.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:15.263+0000 7f0761400640 1 --2- 172.21.15.5:0/341523546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f075c1029c0 0x7f075c19a470 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:15.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:15.263+0000 7f0761400640 1 -- 172.21.15.5:0/341523546 >> 172.21.15.5:0/341523546 conn(0x7f075c0fc3d0 msgr2=0x7f075c1020f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:15.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:15.263+0000 7f0761400640 1 -- 172.21.15.5:0/341523546 shutdown_connections 2024-04-24T06:41:15.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:15.263+0000 7f0761400640 1 -- 172.21.15.5:0/341523546 wait complete. 2024-04-24T06:41:15.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:15 smithi137 ceph-mon[33017]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:15.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:15 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:15.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:15 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:15.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:15 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:15.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:15 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T06:41:15.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:15 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T06:41:15.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:15 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:41:15.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:15 smithi137 ceph-mon[33017]: Deploying daemon mds.i on smithi137 2024-04-24T06:41:15.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:15 smithi137 ceph-mon[33017]: mds.? [v2:172.21.15.37:6836/1390103296,v1:172.21.15.37:6837/1390103296] up:boot 2024-04-24T06:41:15.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:15 smithi137 ceph-mon[33017]: fsmap 8 up:standby 2024-04-24T06:41:15.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:15 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T06:41:15.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:15 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T06:41:15.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:15 smithi037 ceph-mon[32864]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:15.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:15 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:15.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:15 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:15.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:15 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:15.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:15 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T06:41:15.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:15 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T06:41:15.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:15 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:41:15.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:15 smithi037 ceph-mon[32864]: Deploying daemon mds.i on smithi137 2024-04-24T06:41:15.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:15 smithi037 ceph-mon[32864]: mds.? [v2:172.21.15.37:6836/1390103296,v1:172.21.15.37:6837/1390103296] up:boot 2024-04-24T06:41:15.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:15 smithi037 ceph-mon[32864]: fsmap 8 up:standby 2024-04-24T06:41:15.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:15 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T06:41:15.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:15 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T06:41:16.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:16 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/341523546' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:16.447 INFO:journalctl@ceph.mds.i.smithi137.stdout:Apr 24 06:41:16 smithi137 systemd[1]: Starting Ceph mds.i for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:41:16.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:16 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/341523546' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:16.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:16 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/341523546' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:16.753 INFO:journalctl@ceph.mds.i.smithi137.stdout:Apr 24 06:41:16 smithi137 podman[64057]: 2024-04-24 06:41:16.348962103 +0000 UTC m=+0.017717760 image pull b5c3183062a6ce19f361681cc58de6e3ac1de1a54c80e6094ec5695237cd98d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b 2024-04-24T06:41:16.753 INFO:journalctl@ceph.mds.i.smithi137.stdout:Apr 24 06:41:16 smithi137 podman[64057]: 2024-04-24 06:41:16.491254994 +0000 UTC m=+0.160010631 container create 65c868131cb6f58b1c9a8151791d0cfc06299b5bdb93ac6312e7824e3f6d6e29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-i, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-24T06:41:16.753 INFO:journalctl@ceph.mds.i.smithi137.stdout:Apr 24 06:41:16 smithi137 podman[64057]: 2024-04-24 06:41:16.752313706 +0000 UTC m=+0.421069346 container init 65c868131cb6f58b1c9a8151791d0cfc06299b5bdb93ac6312e7824e3f6d6e29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416) 2024-04-24T06:41:17.078 INFO:journalctl@ceph.mds.i.smithi137.stdout:Apr 24 06:41:16 smithi137 podman[64057]: 2024-04-24 06:41:16.755535976 +0000 UTC m=+0.424291614 container start 65c868131cb6f58b1c9a8151791d0cfc06299b5bdb93ac6312e7824e3f6d6e29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-i, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T06:41:17.079 INFO:journalctl@ceph.mds.i.smithi137.stdout:Apr 24 06:41:16 smithi137 ceph-mds[64077]: -- 172.21.15.137:0/3270349903 <== mon.1 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555c705e9a40 con 0x555c705bd000 2024-04-24T06:41:17.079 INFO:journalctl@ceph.mds.i.smithi137.stdout:Apr 24 06:41:16 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-i[64073]: starting mds.i at 2024-04-24T06:41:17.079 INFO:journalctl@ceph.mds.i.smithi137.stdout:Apr 24 06:41:16 smithi137 bash[64057]: 65c868131cb6f58b1c9a8151791d0cfc06299b5bdb93ac6312e7824e3f6d6e29 2024-04-24T06:41:17.079 INFO:journalctl@ceph.mds.i.smithi137.stdout:Apr 24 06:41:16 smithi137 systemd[1]: Started Ceph mds.i for 43e9bad0-0204-11ef-bc93-c7b262605968. 2024-04-24T06:41:17.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:17 smithi005 ceph-mon[27798]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:17.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:17 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:17.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:17 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:17.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:17 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:17.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:17 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T06:41:17.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:17 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T06:41:17.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:17 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:41:17.285 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:41:17.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:17 smithi137 ceph-mon[33017]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:17.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:17 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:17.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:17 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:17.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:17 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:17.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:17 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T06:41:17.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:17 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T06:41:17.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:17 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:41:17.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.486+0000 7f2b5fe00640 1 -- 172.21.15.5:0/3537127312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b601030d0 msgr2=0x7f2b60103550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:17.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.486+0000 7f2b5fe00640 1 --2- 172.21.15.5:0/3537127312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b601030d0 0x7f2b60103550 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2b48009860 tx=0x7f2b4802a340 comp rx=0 tx=0).stop 2024-04-24T06:41:17.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.486+0000 7f2b5fe00640 1 -- 172.21.15.5:0/3537127312 shutdown_connections 2024-04-24T06:41:17.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.486+0000 7f2b5fe00640 1 --2- 172.21.15.5:0/3537127312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b60103a90 0x7f2b6010a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:17.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.486+0000 7f2b5fe00640 1 --2- 172.21.15.5:0/3537127312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b601030d0 0x7f2b60103550 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:17.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.486+0000 7f2b5fe00640 1 --2- 172.21.15.5:0/3537127312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b60101f00 0x7f2b60102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:17.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.486+0000 7f2b5fe00640 1 -- 172.21.15.5:0/3537127312 >> 172.21.15.5:0/3537127312 conn(0x7f2b600fd4c0 msgr2=0x7f2b600ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:17.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.486+0000 7f2b5fe00640 1 -- 172.21.15.5:0/3537127312 shutdown_connections 2024-04-24T06:41:17.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.486+0000 7f2b5fe00640 1 -- 172.21.15.5:0/3537127312 wait complete. 2024-04-24T06:41:17.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.487+0000 7f2b5fe00640 1 Processor -- start 2024-04-24T06:41:17.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.487+0000 7f2b5fe00640 1 -- start start 2024-04-24T06:41:17.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.488+0000 7f2b5fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b60101f00 0x7f2b6019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:17.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.488+0000 7f2b5fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b601030d0 0x7f2b6019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:17.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.488+0000 7f2b5fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b60103a90 0x7f2b601a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:17.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.488+0000 7f2b5fe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6010dbb0 con 0x7f2b60101f00 2024-04-24T06:41:17.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.488+0000 7f2b5fe00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6010da30 con 0x7f2b601030d0 2024-04-24T06:41:17.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.488+0000 7f2b5fe00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6010dd30 con 0x7f2b60103a90 2024-04-24T06:41:17.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.488+0000 7f2b5ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b60101f00 0x7f2b6019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:17.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.488+0000 7f2b57e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b601030d0 0x7f2b6019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:17.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.488+0000 7f2b57e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b601030d0 0x7f2b6019a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:33184/0 (socket says 172.21.15.5:33184) 2024-04-24T06:41:17.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.488+0000 7f2b57e00640 1 -- 172.21.15.5:0/442790793 learned_addr learned my addr 172.21.15.5:0/442790793 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:41:17.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.488+0000 7f2b5f400640 1 --2- 172.21.15.5:0/442790793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b60103a90 0x7f2b601a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:17.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.488+0000 7f2b57e00640 1 -- 172.21.15.5:0/442790793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b60103a90 msgr2=0x7f2b601a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:17.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.488+0000 7f2b57e00640 1 --2- 172.21.15.5:0/442790793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b60103a90 0x7f2b601a19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:17.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.488+0000 7f2b57e00640 1 -- 172.21.15.5:0/442790793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b60101f00 msgr2=0x7f2b6019a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:17.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.488+0000 7f2b57e00640 1 --2- 172.21.15.5:0/442790793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b60101f00 0x7f2b6019a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:17.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.488+0000 7f2b57e00640 1 -- 172.21.15.5:0/442790793 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b601a2050 con 0x7f2b601030d0 2024-04-24T06:41:17.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.489+0000 7f2b57e00640 1 --2- 172.21.15.5:0/442790793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b601030d0 0x7f2b6019a9c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2b48035040 tx=0x7f2b48033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:17.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.489+0000 7f2b57400640 1 -- 172.21.15.5:0/442790793 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b48037040 con 0x7f2b601030d0 2024-04-24T06:41:17.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.489+0000 7f2b57400640 1 -- 172.21.15.5:0/442790793 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b48030850 con 0x7f2b601030d0 2024-04-24T06:41:17.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.489+0000 7f2b5fe00640 1 -- 172.21.15.5:0/442790793 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b601a22e0 con 0x7f2b601030d0 2024-04-24T06:41:17.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.489+0000 7f2b57400640 1 -- 172.21.15.5:0/442790793 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b48004920 con 0x7f2b601030d0 2024-04-24T06:41:17.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.489+0000 7f2b5fe00640 1 -- 172.21.15.5:0/442790793 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b601a27d0 con 0x7f2b601030d0 2024-04-24T06:41:17.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.490+0000 7f2b5fe00640 1 -- 172.21.15.5:0/442790793 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b601a2a30 con 0x7f2b601030d0 2024-04-24T06:41:17.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.492+0000 7f2b57400640 1 -- 172.21.15.5:0/442790793 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f2b48004050 con 0x7f2b601030d0 2024-04-24T06:41:17.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.494+0000 7f2b57400640 1 --2- 172.21.15.5:0/442790793 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f2b440a3ba0 0x7f2b440a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:17.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.494+0000 7f2b57400640 1 -- 172.21.15.5:0/442790793 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f2b480efed0 con 0x7f2b601030d0 2024-04-24T06:41:17.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.494+0000 7f2b5ea00640 1 --2- 172.21.15.5:0/442790793 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f2b440a3ba0 0x7f2b440a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:17.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.495+0000 7f2b5ea00640 1 --2- 172.21.15.5:0/442790793 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f2b440a3ba0 0x7f2b440a6060 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2b50009530 tx=0x7f2b50009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:17.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.496+0000 7f2b57400640 1 -- 172.21.15.5:0/442790793 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2b480bccb0 con 0x7f2b601030d0 2024-04-24T06:41:17.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:17 smithi037 ceph-mon[32864]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:17.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:17 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:17.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:17 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:17.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:17 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:17.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:17 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T06:41:17.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:17 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T06:41:17.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:17 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:41:17.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.742+0000 7f2b5fe00640 1 -- 172.21.15.5:0/442790793 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b601a2c00 con 0x7f2b601030d0 2024-04-24T06:41:17.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.743+0000 7f2b57400640 1 -- 172.21.15.5:0/442790793 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+7998 (secure 0 0 0) 0x7f2b480bc5b0 con 0x7f2b601030d0 2024-04-24T06:41:17.744 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:41:17.744 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1735462750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1735462750},{"type":"v1","addr":"172.21.15.5:6837","nonce":1735462750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3705638847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3705638847},{"type":"v1","addr":"172.21.15.37:6835","nonce":3705638847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24515,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1390103296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1390103296},{"type":"v1","addr":"172.21.15.37:6837","nonce":1390103296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24520,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3217754215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3217754215},{"type":"v1","addr":"172.21.15.137:6835","nonce":3217754215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24532,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/3044222976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3044222976},{"type":"v1","addr":"172.21.15.137:6837","nonce":3044222976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/1889774232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1889774232},{"type":"v1","addr":"172.21.15.137:6839","nonce":1889774232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-24T06:41:17.745 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 10 2024-04-24T06:41:17.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.745+0000 7f2b5fe00640 1 -- 172.21.15.5:0/442790793 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f2b440a3ba0 msgr2=0x7f2b440a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:17.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.745+0000 7f2b5fe00640 1 --2- 172.21.15.5:0/442790793 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f2b440a3ba0 0x7f2b440a6060 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2b50009530 tx=0x7f2b50009260 comp rx=0 tx=0).stop 2024-04-24T06:41:17.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.745+0000 7f2b5fe00640 1 -- 172.21.15.5:0/442790793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b601030d0 msgr2=0x7f2b6019a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:17.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.745+0000 7f2b5fe00640 1 --2- 172.21.15.5:0/442790793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b601030d0 0x7f2b6019a9c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2b48035040 tx=0x7f2b48033d90 comp rx=0 tx=0).stop 2024-04-24T06:41:17.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.746+0000 7f2b5fe00640 1 -- 172.21.15.5:0/442790793 shutdown_connections 2024-04-24T06:41:17.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.746+0000 7f2b5fe00640 1 --2- 172.21.15.5:0/442790793 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f2b440a3ba0 0x7f2b440a6060 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:17.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.746+0000 7f2b5fe00640 1 --2- 172.21.15.5:0/442790793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b60103a90 0x7f2b601a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:17.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.746+0000 7f2b5fe00640 1 --2- 172.21.15.5:0/442790793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b601030d0 0x7f2b6019a9c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:17.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.746+0000 7f2b5fe00640 1 --2- 172.21.15.5:0/442790793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b60101f00 0x7f2b6019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:17.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.746+0000 7f2b5fe00640 1 -- 172.21.15.5:0/442790793 >> 172.21.15.5:0/442790793 conn(0x7f2b600fd4c0 msgr2=0x7f2b60074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:17.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.746+0000 7f2b5fe00640 1 -- 172.21.15.5:0/442790793 shutdown_connections 2024-04-24T06:41:17.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:17.746+0000 7f2b5fe00640 1 -- 172.21.15.5:0/442790793 wait complete. 2024-04-24T06:41:18.410 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:18 smithi005 ceph-mon[27798]: Deploying daemon mds.g on smithi005 2024-04-24T06:41:18.410 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:18 smithi005 ceph-mon[27798]: mds.? [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] up:boot 2024-04-24T06:41:18.410 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:18 smithi005 ceph-mon[27798]: fsmap 9 up:standby 2024-04-24T06:41:18.410 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:18 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T06:41:18.410 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:18 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/442790793' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:18.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:18 smithi137 ceph-mon[33017]: Deploying daemon mds.g on smithi005 2024-04-24T06:41:18.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:18 smithi137 ceph-mon[33017]: mds.? [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] up:boot 2024-04-24T06:41:18.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:18 smithi137 ceph-mon[33017]: fsmap 9 up:standby 2024-04-24T06:41:18.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:18 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T06:41:18.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:18 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/442790793' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:18.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:18 smithi037 ceph-mon[32864]: Deploying daemon mds.g on smithi005 2024-04-24T06:41:18.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:18 smithi037 ceph-mon[32864]: mds.? [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] up:boot 2024-04-24T06:41:18.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:18 smithi037 ceph-mon[32864]: fsmap 9 up:standby 2024-04-24T06:41:18.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:18 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T06:41:18.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:18 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/442790793' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:19.162 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:19 smithi005 ceph-mon[27798]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:19.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:19 smithi137 ceph-mon[33017]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:19.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:19 smithi037 ceph-mon[32864]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:19.770 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:41:20.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.088+0000 7ffa48000640 1 -- 172.21.15.5:0/1576491870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa40074a60 msgr2=0x7ffa40074ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:20.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.088+0000 7ffa48000640 1 --2- 172.21.15.5:0/1576491870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa40074a60 0x7ffa40074ee0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c009900 tx=0x7ffa2c02a3f0 comp rx=0 tx=0).stop 2024-04-24T06:41:20.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.089+0000 7ffa48000640 1 -- 172.21.15.5:0/1576491870 shutdown_connections 2024-04-24T06:41:20.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.089+0000 7ffa48000640 1 --2- 172.21.15.5:0/1576491870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa40109f20 0x7ffa4010c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:20.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.089+0000 7ffa48000640 1 --2- 172.21.15.5:0/1576491870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa40074a60 0x7ffa40074ee0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:20.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.089+0000 7ffa48000640 1 --2- 172.21.15.5:0/1576491870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa40076040 0x7ffa40074490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:20.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.089+0000 7ffa48000640 1 -- 172.21.15.5:0/1576491870 >> 172.21.15.5:0/1576491870 conn(0x7ffa400fd5f0 msgr2=0x7ffa400ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:20.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.089+0000 7ffa48000640 1 -- 172.21.15.5:0/1576491870 shutdown_connections 2024-04-24T06:41:20.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.089+0000 7ffa48000640 1 -- 172.21.15.5:0/1576491870 wait complete. 2024-04-24T06:41:20.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.090+0000 7ffa48000640 1 Processor -- start 2024-04-24T06:41:20.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.090+0000 7ffa48000640 1 -- start start 2024-04-24T06:41:20.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.090+0000 7ffa48000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa40074a60 0x7ffa4019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:20.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.090+0000 7ffa48000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa40076040 0x7ffa4019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:20.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.090+0000 7ffa48000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa40109f20 0x7ffa401a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:20.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.090+0000 7ffa48000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4010ff30 con 0x7ffa40074a60 2024-04-24T06:41:20.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.090+0000 7ffa48000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4010fdb0 con 0x7ffa40076040 2024-04-24T06:41:20.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.090+0000 7ffa48000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ffa401100b0 con 0x7ffa40109f20 2024-04-24T06:41:20.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.090+0000 7ffa45600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa40074a60 0x7ffa4019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:20.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.090+0000 7ffa45600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa40074a60 0x7ffa4019a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52012/0 (socket says 172.21.15.5:52012) 2024-04-24T06:41:20.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.090+0000 7ffa45600640 1 -- 172.21.15.5:0/1842519052 learned_addr learned my addr 172.21.15.5:0/1842519052 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:41:20.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.090+0000 7ffa44c00640 1 --2- 172.21.15.5:0/1842519052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa40076040 0x7ffa4019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:20.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.091+0000 7ffa46000640 1 --2- 172.21.15.5:0/1842519052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa40109f20 0x7ffa401a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:20.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.091+0000 7ffa45600640 1 -- 172.21.15.5:0/1842519052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa40109f20 msgr2=0x7ffa401a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:20.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.091+0000 7ffa45600640 1 --2- 172.21.15.5:0/1842519052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa40109f20 0x7ffa401a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:20.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.091+0000 7ffa45600640 1 -- 172.21.15.5:0/1842519052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa40076040 msgr2=0x7ffa4019aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:20.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.091+0000 7ffa45600640 1 --2- 172.21.15.5:0/1842519052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa40076040 0x7ffa4019aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:20.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.091+0000 7ffa45600640 1 -- 172.21.15.5:0/1842519052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa401a2160 con 0x7ffa40074a60 2024-04-24T06:41:20.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.091+0000 7ffa44c00640 1 --2- 172.21.15.5:0/1842519052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa40076040 0x7ffa4019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:41:20.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.091+0000 7ffa46000640 1 --2- 172.21.15.5:0/1842519052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa40109f20 0x7ffa401a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:41:20.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.091+0000 7ffa45600640 1 --2- 172.21.15.5:0/1842519052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa40074a60 0x7ffa4019a540 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ffa3400ebc0 tx=0x7ffa3400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:20.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.091+0000 7ffa3e000640 1 -- 172.21.15.5:0/1842519052 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa34010630 con 0x7ffa40074a60 2024-04-24T06:41:20.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.091+0000 7ffa3e000640 1 -- 172.21.15.5:0/1842519052 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa34010cb0 con 0x7ffa40074a60 2024-04-24T06:41:20.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.091+0000 7ffa48000640 1 -- 172.21.15.5:0/1842519052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa401a2450 con 0x7ffa40074a60 2024-04-24T06:41:20.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.091+0000 7ffa3e000640 1 -- 172.21.15.5:0/1842519052 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa34018bd0 con 0x7ffa40074a60 2024-04-24T06:41:20.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.092+0000 7ffa48000640 1 -- 172.21.15.5:0/1842519052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa401a29c0 con 0x7ffa40074a60 2024-04-24T06:41:20.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.094+0000 7ffa48000640 1 -- 172.21.15.5:0/1842519052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa40108c50 con 0x7ffa40074a60 2024-04-24T06:41:20.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.096+0000 7ffa3e000640 1 -- 172.21.15.5:0/1842519052 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7ffa340107d0 con 0x7ffa40074a60 2024-04-24T06:41:20.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.098+0000 7ffa3e000640 1 --2- 172.21.15.5:0/1842519052 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ffa280a3c30 0x7ffa280a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:20.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.098+0000 7ffa3e000640 1 -- 172.21.15.5:0/1842519052 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ffa340d0700 con 0x7ffa40074a60 2024-04-24T06:41:20.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.098+0000 7ffa44c00640 1 --2- 172.21.15.5:0/1842519052 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ffa280a3c30 0x7ffa280a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:20.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.099+0000 7ffa44c00640 1 --2- 172.21.15.5:0/1842519052 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ffa280a3c30 0x7ffa280a60f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c006010 tx=0x7ffa2c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:20.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.102+0000 7ffa3e000640 1 -- 172.21.15.5:0/1842519052 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffa3409d4e0 con 0x7ffa40074a60 2024-04-24T06:41:20.256 INFO:journalctl@ceph.mds.g.smithi005.stdout:Apr 24 06:41:20 smithi005 systemd[1]: Starting Ceph mds.g for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:41:20.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.371+0000 7ffa48000640 1 -- 172.21.15.5:0/1842519052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa4002d620 con 0x7ffa40074a60 2024-04-24T06:41:20.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.372+0000 7ffa3e000640 1 -- 172.21.15.5:0/1842519052 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+7998 (secure 0 0 0) 0x7ffa3409cde0 con 0x7ffa40074a60 2024-04-24T06:41:20.373 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:41:20.374 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1735462750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1735462750},{"type":"v1","addr":"172.21.15.5:6837","nonce":1735462750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3705638847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3705638847},{"type":"v1","addr":"172.21.15.37:6835","nonce":3705638847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24515,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1390103296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1390103296},{"type":"v1","addr":"172.21.15.37:6837","nonce":1390103296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24520,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3217754215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3217754215},{"type":"v1","addr":"172.21.15.137:6835","nonce":3217754215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24532,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/3044222976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3044222976},{"type":"v1","addr":"172.21.15.137:6837","nonce":3044222976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/1889774232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1889774232},{"type":"v1","addr":"172.21.15.137:6839","nonce":1889774232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-24T06:41:20.374 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 10 2024-04-24T06:41:20.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.374+0000 7ffa48000640 1 -- 172.21.15.5:0/1842519052 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ffa280a3c30 msgr2=0x7ffa280a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:20.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.374+0000 7ffa48000640 1 --2- 172.21.15.5:0/1842519052 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ffa280a3c30 0x7ffa280a60f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c006010 tx=0x7ffa2c000bc0 comp rx=0 tx=0).stop 2024-04-24T06:41:20.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.375+0000 7ffa48000640 1 -- 172.21.15.5:0/1842519052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa40074a60 msgr2=0x7ffa4019a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:20.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.375+0000 7ffa48000640 1 --2- 172.21.15.5:0/1842519052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa40074a60 0x7ffa4019a540 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ffa3400ebc0 tx=0x7ffa3400c6a0 comp rx=0 tx=0).stop 2024-04-24T06:41:20.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.375+0000 7ffa48000640 1 -- 172.21.15.5:0/1842519052 shutdown_connections 2024-04-24T06:41:20.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.375+0000 7ffa48000640 1 --2- 172.21.15.5:0/1842519052 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ffa280a3c30 0x7ffa280a60f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:20.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.375+0000 7ffa48000640 1 --2- 172.21.15.5:0/1842519052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa40109f20 0x7ffa401a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:20.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.375+0000 7ffa48000640 1 --2- 172.21.15.5:0/1842519052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa40076040 0x7ffa4019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:20.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.375+0000 7ffa48000640 1 --2- 172.21.15.5:0/1842519052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa40074a60 0x7ffa4019a540 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:20.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.375+0000 7ffa48000640 1 -- 172.21.15.5:0/1842519052 >> 172.21.15.5:0/1842519052 conn(0x7ffa400fd5f0 msgr2=0x7ffa40105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:20.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.375+0000 7ffa48000640 1 -- 172.21.15.5:0/1842519052 shutdown_connections 2024-04-24T06:41:20.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:20.375+0000 7ffa48000640 1 -- 172.21.15.5:0/1842519052 wait complete. 2024-04-24T06:41:20.719 INFO:journalctl@ceph.mds.g.smithi005.stdout:Apr 24 06:41:20 smithi005 podman[65587]: 2024-04-24 06:41:20.282647014 +0000 UTC m=+0.036487703 image pull b5c3183062a6ce19f361681cc58de6e3ac1de1a54c80e6094ec5695237cd98d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b 2024-04-24T06:41:20.719 INFO:journalctl@ceph.mds.g.smithi005.stdout:Apr 24 06:41:20 smithi005 podman[65587]: 2024-04-24 06:41:20.404750336 +0000 UTC m=+0.158591016 container create fa9159430b1334b7e5c85b6b942065a4c505748ec48fcb97fe125d3de3edef0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-g, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T06:41:20.719 INFO:journalctl@ceph.mds.g.smithi005.stdout:Apr 24 06:41:20 smithi005 podman[65587]: 2024-04-24 06:41:20.717024091 +0000 UTC m=+0.470864817 container init fa9159430b1334b7e5c85b6b942065a4c505748ec48fcb97fe125d3de3edef0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-g, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-24T06:41:21.007 INFO:journalctl@ceph.mds.g.smithi005.stdout:Apr 24 06:41:20 smithi005 podman[65587]: 2024-04-24 06:41:20.728413102 +0000 UTC m=+0.482253794 container start fa9159430b1334b7e5c85b6b942065a4c505748ec48fcb97fe125d3de3edef0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-g, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-24T06:41:21.007 INFO:journalctl@ceph.mds.g.smithi005.stdout:Apr 24 06:41:20 smithi005 ceph-mds[65607]: -- 172.21.15.5:0/1744342328 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557b169d7a40 con 0x557b169aa800 2024-04-24T06:41:21.007 INFO:journalctl@ceph.mds.g.smithi005.stdout:Apr 24 06:41:20 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-g[65603]: starting mds.g at 2024-04-24T06:41:21.007 INFO:journalctl@ceph.mds.g.smithi005.stdout:Apr 24 06:41:20 smithi005 bash[65587]: fa9159430b1334b7e5c85b6b942065a4c505748ec48fcb97fe125d3de3edef0b 2024-04-24T06:41:21.007 INFO:journalctl@ceph.mds.g.smithi005.stdout:Apr 24 06:41:20 smithi005 systemd[1]: Started Ceph mds.g for 43e9bad0-0204-11ef-bc93-c7b262605968. 2024-04-24T06:41:21.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:21 smithi137 ceph-mon[33017]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:21.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:21 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/1842519052' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:21.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:21 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:21.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:21 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:21.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:21 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:21.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:21 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T06:41:21.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:21 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T06:41:21.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:21 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:41:21.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:21 smithi005 ceph-mon[27798]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:21.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:21 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1842519052' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:21.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:21 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:21.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:21 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:21.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:21 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:21.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:21 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T06:41:21.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:21 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T06:41:21.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:21 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:41:21.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:21 smithi037 ceph-mon[32864]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:21.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:21 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/1842519052' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:21.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:21 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:21.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:21 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:21.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:21 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:21.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:21 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T06:41:21.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:21 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T06:41:21.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:21 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:41:22.363 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:22 smithi005 ceph-mon[27798]: Deploying daemon mds.a on smithi005 2024-04-24T06:41:22.363 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:22 smithi005 ceph-mon[27798]: mds.? [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] up:boot 2024-04-24T06:41:22.363 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:22 smithi005 ceph-mon[27798]: fsmap 10 up:standby 2024-04-24T06:41:22.363 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:22 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T06:41:22.397 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:41:22.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:22 smithi137 ceph-mon[33017]: Deploying daemon mds.a on smithi005 2024-04-24T06:41:22.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:22 smithi137 ceph-mon[33017]: mds.? [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] up:boot 2024-04-24T06:41:22.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:22 smithi137 ceph-mon[33017]: fsmap 10 up:standby 2024-04-24T06:41:22.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:22 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T06:41:22.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:22 smithi037 ceph-mon[32864]: Deploying daemon mds.a on smithi005 2024-04-24T06:41:22.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:22 smithi037 ceph-mon[32864]: mds.? [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] up:boot 2024-04-24T06:41:22.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:22 smithi037 ceph-mon[32864]: fsmap 10 up:standby 2024-04-24T06:41:22.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:22 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T06:41:22.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.590+0000 7f66f9e00640 1 -- 172.21.15.5:0/3485172053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66f4109740 msgr2=0x7f66f4109bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:22.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.590+0000 7f66f9e00640 1 --2- 172.21.15.5:0/3485172053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66f4109740 0x7f66f4109bc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f66e0009900 tx=0x7f66e002a3f0 comp rx=0 tx=0).stop 2024-04-24T06:41:22.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.590+0000 7f66f9e00640 1 -- 172.21.15.5:0/3485172053 shutdown_connections 2024-04-24T06:41:22.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.590+0000 7f66f9e00640 1 --2- 172.21.15.5:0/3485172053 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66f410a100 0x7f66f4110990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:22.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.590+0000 7f66f9e00640 1 --2- 172.21.15.5:0/3485172053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66f4109740 0x7f66f4109bc0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:22.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.590+0000 7f66f9e00640 1 --2- 172.21.15.5:0/3485172053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66f41085a0 0x7f66f41089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:22.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.590+0000 7f66f9e00640 1 -- 172.21.15.5:0/3485172053 >> 172.21.15.5:0/3485172053 conn(0x7f66f40774f0 msgr2=0x7f66f4079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:22.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.590+0000 7f66f9e00640 1 -- 172.21.15.5:0/3485172053 shutdown_connections 2024-04-24T06:41:22.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.590+0000 7f66f9e00640 1 -- 172.21.15.5:0/3485172053 wait complete. 2024-04-24T06:41:22.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.591+0000 7f66f9e00640 1 Processor -- start 2024-04-24T06:41:22.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.591+0000 7f66f9e00640 1 -- start start 2024-04-24T06:41:22.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.591+0000 7f66f9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66f41085a0 0x7f66f419e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:22.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.591+0000 7f66f9e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66f4109740 0x7f66f419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:22.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.591+0000 7f66f9e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66f410a100 0x7f66f41a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:22.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.591+0000 7f66f9e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f66f4114230 con 0x7f66f41085a0 2024-04-24T06:41:22.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.591+0000 7f66f9e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f66f41140b0 con 0x7f66f4109740 2024-04-24T06:41:22.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.591+0000 7f66f9e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f66f41143b0 con 0x7f66f410a100 2024-04-24T06:41:22.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.592+0000 7f66f8a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66f41085a0 0x7f66f419e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:22.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.592+0000 7f66f8a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66f41085a0 0x7f66f419e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52046/0 (socket says 172.21.15.5:52046) 2024-04-24T06:41:22.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.592+0000 7f66f8a00640 1 -- 172.21.15.5:0/1027696532 learned_addr learned my addr 172.21.15.5:0/1027696532 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:41:22.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.592+0000 7f66f9400640 1 --2- 172.21.15.5:0/1027696532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66f410a100 0x7f66f41a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:22.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.592+0000 7f66f3e00640 1 --2- 172.21.15.5:0/1027696532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66f4109740 0x7f66f419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:22.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.592+0000 7f66f8a00640 1 -- 172.21.15.5:0/1027696532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66f410a100 msgr2=0x7f66f41a5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:22.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.592+0000 7f66f8a00640 1 --2- 172.21.15.5:0/1027696532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66f410a100 0x7f66f41a5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:22.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.592+0000 7f66f8a00640 1 -- 172.21.15.5:0/1027696532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66f4109740 msgr2=0x7f66f419ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:22.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.592+0000 7f66f8a00640 1 --2- 172.21.15.5:0/1027696532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66f4109740 0x7f66f419ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:22.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.592+0000 7f66f8a00640 1 -- 172.21.15.5:0/1027696532 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66f41a6420 con 0x7f66f41085a0 2024-04-24T06:41:22.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.592+0000 7f66f9400640 1 --2- 172.21.15.5:0/1027696532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66f410a100 0x7f66f41a5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:41:22.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.593+0000 7f66f8a00640 1 --2- 172.21.15.5:0/1027696532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66f41085a0 0x7f66f419e800 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f66e800e9c0 tx=0x7f66e800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:22.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.593+0000 7f66f1600640 1 -- 172.21.15.5:0/1027696532 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e800cd80 con 0x7f66f41085a0 2024-04-24T06:41:22.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.593+0000 7f66f9e00640 1 -- 172.21.15.5:0/1027696532 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66f41a6710 con 0x7f66f41085a0 2024-04-24T06:41:22.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.593+0000 7f66f1600640 1 -- 172.21.15.5:0/1027696532 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66e80035f0 con 0x7f66f41085a0 2024-04-24T06:41:22.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.593+0000 7f66f1600640 1 -- 172.21.15.5:0/1027696532 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e8010690 con 0x7f66f41085a0 2024-04-24T06:41:22.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.593+0000 7f66f9e00640 1 -- 172.21.15.5:0/1027696532 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66f41a6c80 con 0x7f66f41085a0 2024-04-24T06:41:22.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.594+0000 7f66f9e00640 1 -- 172.21.15.5:0/1027696532 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66f410f810 con 0x7f66f41085a0 2024-04-24T06:41:22.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.596+0000 7f66f1600640 1 -- 172.21.15.5:0/1027696532 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f66e80037a0 con 0x7f66f41085a0 2024-04-24T06:41:22.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.598+0000 7f66f1600640 1 --2- 172.21.15.5:0/1027696532 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f66dc0a3ba0 0x7f66dc0a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:22.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.598+0000 7f66f1600640 1 -- 172.21.15.5:0/1027696532 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f66e80d03c0 con 0x7f66f41085a0 2024-04-24T06:41:22.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.598+0000 7f66f3e00640 1 --2- 172.21.15.5:0/1027696532 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f66dc0a3ba0 0x7f66dc0a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:22.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.599+0000 7f66f3e00640 1 --2- 172.21.15.5:0/1027696532 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f66dc0a3ba0 0x7f66dc0a6060 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f66e0031440 tx=0x7f66e00315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:22.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.603+0000 7f66f1600640 1 -- 172.21.15.5:0/1027696532 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f66e809d1a0 con 0x7f66f41085a0 2024-04-24T06:41:22.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.868+0000 7f66f9e00640 1 -- 172.21.15.5:0/1027696532 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66f402d620 con 0x7f66f41085a0 2024-04-24T06:41:22.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.869+0000 7f66f1600640 1 -- 172.21.15.5:0/1027696532 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8821 (secure 0 0 0) 0x7f66e809caa0 con 0x7f66f41085a0 2024-04-24T06:41:22.871 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:41:22.871 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1735462750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1735462750},{"type":"v1","addr":"172.21.15.5:6837","nonce":1735462750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3705638847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3705638847},{"type":"v1","addr":"172.21.15.37:6835","nonce":3705638847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24515,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1390103296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1390103296},{"type":"v1","addr":"172.21.15.37:6837","nonce":1390103296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24520,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3217754215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3217754215},{"type":"v1","addr":"172.21.15.137:6835","nonce":3217754215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24532,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/3044222976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3044222976},{"type":"v1","addr":"172.21.15.137:6837","nonce":3044222976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24539,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3173201459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3173201459},{"type":"v1","addr":"172.21.15.5:6839","nonce":3173201459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/1889774232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1889774232},{"type":"v1","addr":"172.21.15.137:6839","nonce":1889774232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-24T06:41:22.871 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 11 2024-04-24T06:41:22.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.872+0000 7f66f9e00640 1 -- 172.21.15.5:0/1027696532 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f66dc0a3ba0 msgr2=0x7f66dc0a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:22.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.872+0000 7f66f9e00640 1 --2- 172.21.15.5:0/1027696532 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f66dc0a3ba0 0x7f66dc0a6060 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f66e0031440 tx=0x7f66e00315b0 comp rx=0 tx=0).stop 2024-04-24T06:41:22.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.872+0000 7f66f9e00640 1 -- 172.21.15.5:0/1027696532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66f41085a0 msgr2=0x7f66f419e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:22.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.872+0000 7f66f9e00640 1 --2- 172.21.15.5:0/1027696532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66f41085a0 0x7f66f419e800 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f66e800e9c0 tx=0x7f66e800ee80 comp rx=0 tx=0).stop 2024-04-24T06:41:22.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.872+0000 7f66f9e00640 1 -- 172.21.15.5:0/1027696532 shutdown_connections 2024-04-24T06:41:22.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.872+0000 7f66f9e00640 1 --2- 172.21.15.5:0/1027696532 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f66dc0a3ba0 0x7f66dc0a6060 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:22.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.872+0000 7f66f9e00640 1 --2- 172.21.15.5:0/1027696532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66f410a100 0x7f66f41a5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:22.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.872+0000 7f66f9e00640 1 --2- 172.21.15.5:0/1027696532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66f4109740 0x7f66f419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:22.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.872+0000 7f66f9e00640 1 --2- 172.21.15.5:0/1027696532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66f41085a0 0x7f66f419e800 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:22.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.872+0000 7f66f9e00640 1 -- 172.21.15.5:0/1027696532 >> 172.21.15.5:0/1027696532 conn(0x7f66f40774f0 msgr2=0x7f66f410c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:22.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.873+0000 7f66f9e00640 1 -- 172.21.15.5:0/1027696532 shutdown_connections 2024-04-24T06:41:22.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:22.873+0000 7f66f9e00640 1 -- 172.21.15.5:0/1027696532 wait complete. 2024-04-24T06:41:23.231 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:23 smithi005 ceph-mon[27798]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:23.232 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:23 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1027696532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:23.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:23 smithi137 ceph-mon[33017]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:23.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:23 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/1027696532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:23.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:23 smithi037 ceph-mon[32864]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:23.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:23 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/1027696532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:24.507 INFO:journalctl@ceph.mds.a.smithi005.stdout:Apr 24 06:41:24 smithi005 systemd[1]: Starting Ceph mds.a for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:41:24.797 INFO:journalctl@ceph.mds.a.smithi005.stdout:Apr 24 06:41:24 smithi005 podman[65936]: 2024-04-24 06:41:24.524550046 +0000 UTC m=+0.129518950 container create 0acdc9dffbd4732a386dd7d5199d155cd4c529cc7b30f359b684d33f617b93c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-a, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T06:41:24.797 INFO:journalctl@ceph.mds.a.smithi005.stdout:Apr 24 06:41:24 smithi005 podman[65936]: 2024-04-24 06:41:24.434224796 +0000 UTC m=+0.039193684 image pull b5c3183062a6ce19f361681cc58de6e3ac1de1a54c80e6094ec5695237cd98d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b 2024-04-24T06:41:24.797 INFO:journalctl@ceph.mds.a.smithi005.stdout:Apr 24 06:41:24 smithi005 podman[65936]: 2024-04-24 06:41:24.79508648 +0000 UTC m=+0.400055427 container init 0acdc9dffbd4732a386dd7d5199d155cd4c529cc7b30f359b684d33f617b93c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-a, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-24T06:41:24.894 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:41:25.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.107+0000 7f598b200640 1 -- 172.21.15.5:0/4184877739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f598c105350 msgr2=0x7f598c1057d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.107+0000 7f598b200640 1 --2- 172.21.15.5:0/4184877739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f598c105350 0x7f598c1057d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5974009860 tx=0x7f597402a340 comp rx=0 tx=0).stop 2024-04-24T06:41:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.107+0000 7f598b200640 1 -- 172.21.15.5:0/4184877739 shutdown_connections 2024-04-24T06:41:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.107+0000 7f598b200640 1 --2- 172.21.15.5:0/4184877739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f598c105d10 0x7f598c10c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.107+0000 7f598b200640 1 --2- 172.21.15.5:0/4184877739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f598c105350 0x7f598c1057d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.107+0000 7f598b200640 1 --2- 172.21.15.5:0/4184877739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f598c104160 0x7f598c104580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.107+0000 7f598b200640 1 -- 172.21.15.5:0/4184877739 >> 172.21.15.5:0/4184877739 conn(0x7f598c0ff6d0 msgr2=0x7f598c101b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.107+0000 7f598b200640 1 -- 172.21.15.5:0/4184877739 shutdown_connections 2024-04-24T06:41:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.107+0000 7f598b200640 1 -- 172.21.15.5:0/4184877739 wait complete. 2024-04-24T06:41:25.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.108+0000 7f598b200640 1 Processor -- start 2024-04-24T06:41:25.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.108+0000 7f598b200640 1 -- start start 2024-04-24T06:41:25.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.109+0000 7f598b200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f598c104160 0x7f598c19a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:25.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.109+0000 7f598b200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f598c105350 0x7f598c19a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:25.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.109+0000 7f598b200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f598c105d10 0x7f598c1a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:25.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.109+0000 7f598b200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f598c10fdf0 con 0x7f598c105d10 2024-04-24T06:41:25.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.109+0000 7f598b200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f598c10fc70 con 0x7f598c104160 2024-04-24T06:41:25.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.109+0000 7f598b200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f598c10ff70 con 0x7f598c105350 2024-04-24T06:41:25.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.109+0000 7f5989e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f598c104160 0x7f598c19a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:25.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.109+0000 7f5989e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f598c104160 0x7f598c19a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:38778/0 (socket says 172.21.15.5:38778) 2024-04-24T06:41:25.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.109+0000 7f5989e00640 1 -- 172.21.15.5:0/647457478 learned_addr learned my addr 172.21.15.5:0/647457478 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:41:25.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.109+0000 7f598a800640 1 --2- 172.21.15.5:0/647457478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f598c105d10 0x7f598c1a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.109+0000 7f5989400640 1 --2- 172.21.15.5:0/647457478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f598c105350 0x7f598c19a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.109+0000 7f5989e00640 1 -- 172.21.15.5:0/647457478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f598c105350 msgr2=0x7f598c19a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.110+0000 7f5989e00640 1 --2- 172.21.15.5:0/647457478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f598c105350 0x7f598c19a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.110+0000 7f5989e00640 1 -- 172.21.15.5:0/647457478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f598c105d10 msgr2=0x7f598c1a19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.110+0000 7f5989e00640 1 --2- 172.21.15.5:0/647457478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f598c105d10 0x7f598c1a19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.110+0000 7f5989e00640 1 -- 172.21.15.5:0/647457478 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f598c1a2030 con 0x7f598c104160 2024-04-24T06:41:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.110+0000 7f5989400640 1 --2- 172.21.15.5:0/647457478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f598c105350 0x7f598c19a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:41:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.110+0000 7f5989e00640 1 --2- 172.21.15.5:0/647457478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f598c104160 0x7f598c19a410 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f597c00c3f0 tx=0x7f597c009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.110+0000 7f5982a00640 1 -- 172.21.15.5:0/647457478 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597c01b070 con 0x7f598c104160 2024-04-24T06:41:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.110+0000 7f598b200640 1 -- 172.21.15.5:0/647457478 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f598c1a2320 con 0x7f598c104160 2024-04-24T06:41:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.110+0000 7f5982a00640 1 -- 172.21.15.5:0/647457478 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f597c00bb10 con 0x7f598c104160 2024-04-24T06:41:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.110+0000 7f5982a00640 1 -- 172.21.15.5:0/647457478 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597c016d80 con 0x7f598c104160 2024-04-24T06:41:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.110+0000 7f598b200640 1 -- 172.21.15.5:0/647457478 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f598c1a2840 con 0x7f598c104160 2024-04-24T06:41:25.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.111+0000 7f598b200640 1 -- 172.21.15.5:0/647457478 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f598c10b420 con 0x7f598c104160 2024-04-24T06:41:25.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.113+0000 7f5982a00640 1 -- 172.21.15.5:0/647457478 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f597c01e070 con 0x7f598c104160 2024-04-24T06:41:25.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.115+0000 7f5982a00640 1 --2- 172.21.15.5:0/647457478 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f59700a3ba0 0x7f59700a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:25.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.115+0000 7f5982a00640 1 -- 172.21.15.5:0/647457478 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f597c0cfeb0 con 0x7f598c104160 2024-04-24T06:41:25.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.115+0000 7f5989400640 1 --2- 172.21.15.5:0/647457478 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f59700a3ba0 0x7f59700a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:25.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.116+0000 7f5989400640 1 --2- 172.21.15.5:0/647457478 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f59700a3ba0 0x7f59700a6060 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5974009830 tx=0x7f5974030550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:25.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.120+0000 7f5982a00640 1 -- 172.21.15.5:0/647457478 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f597c09c640 con 0x7f598c104160 2024-04-24T06:41:25.138 INFO:journalctl@ceph.mds.a.smithi005.stdout:Apr 24 06:41:24 smithi005 podman[65936]: 2024-04-24 06:41:24.808468836 +0000 UTC m=+0.413437743 container start 0acdc9dffbd4732a386dd7d5199d155cd4c529cc7b30f359b684d33f617b93c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-a, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-24T06:41:25.138 INFO:journalctl@ceph.mds.a.smithi005.stdout:Apr 24 06:41:24 smithi005 ceph-mds[65955]: -- 172.21.15.5:0/3984393088 <== mon.2 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e41e839a40 con 0x55e41e80c800 2024-04-24T06:41:25.138 INFO:journalctl@ceph.mds.a.smithi005.stdout:Apr 24 06:41:24 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-a[65951]: starting mds.a at 2024-04-24T06:41:25.138 INFO:journalctl@ceph.mds.a.smithi005.stdout:Apr 24 06:41:24 smithi005 bash[65936]: 0acdc9dffbd4732a386dd7d5199d155cd4c529cc7b30f359b684d33f617b93c0 2024-04-24T06:41:25.139 INFO:journalctl@ceph.mds.a.smithi005.stdout:Apr 24 06:41:24 smithi005 systemd[1]: Started Ceph mds.a for 43e9bad0-0204-11ef-bc93-c7b262605968. 2024-04-24T06:41:25.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.374+0000 7f598b200640 1 -- 172.21.15.5:0/647457478 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f598c02d620 con 0x7f598c104160 2024-04-24T06:41:25.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.375+0000 7f5982a00640 1 -- 172.21.15.5:0/647457478 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9644 (secure 0 0 0) 0x7f597c098020 con 0x7f598c104160 2024-04-24T06:41:25.376 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:41:25.376 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1735462750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1735462750},{"type":"v1","addr":"172.21.15.5:6837","nonce":1735462750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1499469370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1499469370},{"type":"v1","addr":"172.21.15.5:6841","nonce":1499469370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3705638847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3705638847},{"type":"v1","addr":"172.21.15.37:6835","nonce":3705638847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24515,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1390103296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1390103296},{"type":"v1","addr":"172.21.15.37:6837","nonce":1390103296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24520,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3217754215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3217754215},{"type":"v1","addr":"172.21.15.137:6835","nonce":3217754215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24532,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/3044222976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3044222976},{"type":"v1","addr":"172.21.15.137:6837","nonce":3044222976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24539,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3173201459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3173201459},{"type":"v1","addr":"172.21.15.5:6839","nonce":3173201459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/1889774232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1889774232},{"type":"v1","addr":"172.21.15.137:6839","nonce":1889774232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-24T06:41:25.376 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 12 2024-04-24T06:41:25.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.377+0000 7f598b200640 1 -- 172.21.15.5:0/647457478 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f59700a3ba0 msgr2=0x7f59700a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:25.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.377+0000 7f598b200640 1 --2- 172.21.15.5:0/647457478 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f59700a3ba0 0x7f59700a6060 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5974009830 tx=0x7f5974030550 comp rx=0 tx=0).stop 2024-04-24T06:41:25.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.377+0000 7f598b200640 1 -- 172.21.15.5:0/647457478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f598c104160 msgr2=0x7f598c19a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:25.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.377+0000 7f598b200640 1 --2- 172.21.15.5:0/647457478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f598c104160 0x7f598c19a410 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f597c00c3f0 tx=0x7f597c009f90 comp rx=0 tx=0).stop 2024-04-24T06:41:25.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.378+0000 7f598b200640 1 -- 172.21.15.5:0/647457478 shutdown_connections 2024-04-24T06:41:25.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.378+0000 7f598b200640 1 --2- 172.21.15.5:0/647457478 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f59700a3ba0 0x7f59700a6060 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:25.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.378+0000 7f598b200640 1 --2- 172.21.15.5:0/647457478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f598c105d10 0x7f598c1a19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:25.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.378+0000 7f598b200640 1 --2- 172.21.15.5:0/647457478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f598c105350 0x7f598c19a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:25.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.378+0000 7f598b200640 1 --2- 172.21.15.5:0/647457478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f598c104160 0x7f598c19a410 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:25.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.378+0000 7f598b200640 1 -- 172.21.15.5:0/647457478 >> 172.21.15.5:0/647457478 conn(0x7f598c0ff6d0 msgr2=0x7f598c1085f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:25.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.378+0000 7f598b200640 1 -- 172.21.15.5:0/647457478 shutdown_connections 2024-04-24T06:41:25.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:25.378+0000 7f598b200640 1 -- 172.21.15.5:0/647457478 wait complete. 2024-04-24T06:41:25.395 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:25 smithi005 ceph-mon[27798]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:25.396 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:25 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:25.396 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:25 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:25.396 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:25 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:25.396 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:25 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T06:41:25.396 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:25 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T06:41:25.396 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:25 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:41:25.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:25 smithi137 ceph-mon[33017]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:25.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:25 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:25.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:25 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:25.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:25 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:25.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:25 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T06:41:25.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:25 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T06:41:25.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:25 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:41:25.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:25 smithi037 ceph-mon[32864]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:25.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:25 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:25.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:25 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:25.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:25 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:25.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:25 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T06:41:25.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:25 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T06:41:25.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:25 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:41:26.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:26 smithi037 ceph-mon[32864]: Deploying daemon mds.b on smithi037 2024-04-24T06:41:26.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:26 smithi037 ceph-mon[32864]: mds.? [v2:172.21.15.5:6840/1499469370,v1:172.21.15.5:6841/1499469370] up:boot 2024-04-24T06:41:26.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:26 smithi037 ceph-mon[32864]: fsmap 11 up:standby 2024-04-24T06:41:26.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:26 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T06:41:26.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:26 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/647457478' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:26.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:26 smithi137 ceph-mon[33017]: Deploying daemon mds.b on smithi037 2024-04-24T06:41:26.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:26 smithi137 ceph-mon[33017]: mds.? [v2:172.21.15.5:6840/1499469370,v1:172.21.15.5:6841/1499469370] up:boot 2024-04-24T06:41:26.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:26 smithi137 ceph-mon[33017]: fsmap 11 up:standby 2024-04-24T06:41:26.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:26 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T06:41:26.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:26 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/647457478' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:26.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:26 smithi005 ceph-mon[27798]: Deploying daemon mds.b on smithi037 2024-04-24T06:41:26.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:26 smithi005 ceph-mon[27798]: mds.? [v2:172.21.15.5:6840/1499469370,v1:172.21.15.5:6841/1499469370] up:boot 2024-04-24T06:41:26.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:26 smithi005 ceph-mon[27798]: fsmap 11 up:standby 2024-04-24T06:41:26.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:26 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T06:41:26.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:26 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/647457478' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:27.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:27 smithi037 ceph-mon[32864]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:27.269 INFO:journalctl@ceph.mds.b.smithi037.stdout:Apr 24 06:41:27 smithi037 systemd[1]: Starting Ceph mds.b for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:41:27.400 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:41:27.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:27 smithi137 ceph-mon[33017]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:27.483 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:27 smithi005 ceph-mon[27798]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:27.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.623+0000 7f0bdb200640 1 -- 172.21.15.5:0/2387066683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bdc0ffcb0 msgr2=0x7f0bdc1000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:27.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.623+0000 7f0bdb200640 1 --2- 172.21.15.5:0/2387066683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bdc0ffcb0 0x7f0bdc1000d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009900 tx=0x7f0bcc02a3f0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.623+0000 7f0bdb200640 1 -- 172.21.15.5:0/2387066683 shutdown_connections 2024-04-24T06:41:27.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.623+0000 7f0bdb200640 1 --2- 172.21.15.5:0/2387066683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bdc107ba0 0x7f0bdc10a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.623+0000 7f0bdb200640 1 --2- 172.21.15.5:0/2387066683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bdc100610 0x7f0bdc107610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.623+0000 7f0bdb200640 1 --2- 172.21.15.5:0/2387066683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bdc0ffcb0 0x7f0bdc1000d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.623+0000 7f0bdb200640 1 -- 172.21.15.5:0/2387066683 >> 172.21.15.5:0/2387066683 conn(0x7f0bdc0fb700 msgr2=0x7f0bdc0fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:27.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.623+0000 7f0bdb200640 1 -- 172.21.15.5:0/2387066683 shutdown_connections 2024-04-24T06:41:27.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.623+0000 7f0bdb200640 1 -- 172.21.15.5:0/2387066683 wait complete. 2024-04-24T06:41:27.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.624+0000 7f0bdb200640 1 Processor -- start 2024-04-24T06:41:27.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.624+0000 7f0bdb200640 1 -- start start 2024-04-24T06:41:27.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.624+0000 7f0bdb200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bdc0ffcb0 0x7f0bdc19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:27.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.624+0000 7f0bdb200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bdc100610 0x7f0bdc19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:27.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.624+0000 7f0bdb200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bdc107ba0 0x7f0bdc1a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:27.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.624+0000 7f0bdb200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc10dbb0 con 0x7f0bdc107ba0 2024-04-24T06:41:27.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.624+0000 7f0bdb200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc10da30 con 0x7f0bdc100610 2024-04-24T06:41:27.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.624+0000 7f0bdb200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc10dd30 con 0x7f0bdc0ffcb0 2024-04-24T06:41:27.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.625+0000 7f0bd9400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bdc100610 0x7f0bdc19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:27.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.625+0000 7f0bd9400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bdc100610 0x7f0bdc19a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:38794/0 (socket says 172.21.15.5:38794) 2024-04-24T06:41:27.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.625+0000 7f0bd9400640 1 -- 172.21.15.5:0/1139819335 learned_addr learned my addr 172.21.15.5:0/1139819335 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:41:27.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.625+0000 7f0bda800640 1 --2- 172.21.15.5:0/1139819335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bdc107ba0 0x7f0bdc1a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:27.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.625+0000 7f0bd9e00640 1 --2- 172.21.15.5:0/1139819335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bdc0ffcb0 0x7f0bdc19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:27.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.625+0000 7f0bd9400640 1 -- 172.21.15.5:0/1139819335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bdc0ffcb0 msgr2=0x7f0bdc19a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:27.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.625+0000 7f0bd9400640 1 --2- 172.21.15.5:0/1139819335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bdc0ffcb0 0x7f0bdc19a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.625+0000 7f0bd9400640 1 -- 172.21.15.5:0/1139819335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bdc107ba0 msgr2=0x7f0bdc1a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:27.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.625+0000 7f0bd9400640 1 --2- 172.21.15.5:0/1139819335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bdc107ba0 0x7f0bdc1a1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.625+0000 7f0bd9400640 1 -- 172.21.15.5:0/1139819335 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bdc1a2080 con 0x7f0bdc100610 2024-04-24T06:41:27.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.625+0000 7f0bd9400640 1 --2- 172.21.15.5:0/1139819335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bdc100610 0x7f0bdc19a9f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0bc800e9c0 tx=0x7f0bc800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:27.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.626+0000 7f0bd2a00640 1 -- 172.21.15.5:0/1139819335 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc800cd80 con 0x7f0bdc100610 2024-04-24T06:41:27.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.626+0000 7f0bd2a00640 1 -- 172.21.15.5:0/1139819335 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bc80035f0 con 0x7f0bdc100610 2024-04-24T06:41:27.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.626+0000 7f0bd2a00640 1 -- 172.21.15.5:0/1139819335 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc8010690 con 0x7f0bdc100610 2024-04-24T06:41:27.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.626+0000 7f0bdb200640 1 -- 172.21.15.5:0/1139819335 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bdc1a2370 con 0x7f0bdc100610 2024-04-24T06:41:27.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.626+0000 7f0bdb200640 1 -- 172.21.15.5:0/1139819335 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bdc1a2870 con 0x7f0bdc100610 2024-04-24T06:41:27.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.627+0000 7f0bdb200640 1 -- 172.21.15.5:0/1139819335 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bdc1a2b30 con 0x7f0bdc100610 2024-04-24T06:41:27.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.630+0000 7f0bd2a00640 1 -- 172.21.15.5:0/1139819335 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f0bc80037a0 con 0x7f0bdc100610 2024-04-24T06:41:27.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.631+0000 7f0bd2a00640 1 --2- 172.21.15.5:0/1139819335 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f0bc00a3ba0 0x7f0bc00a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:27.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.631+0000 7f0bd2a00640 1 -- 172.21.15.5:0/1139819335 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f0bc80d04b0 con 0x7f0bdc100610 2024-04-24T06:41:27.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.631+0000 7f0bd9e00640 1 --2- 172.21.15.5:0/1139819335 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f0bc00a3ba0 0x7f0bc00a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:27.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.632+0000 7f0bd9e00640 1 --2- 172.21.15.5:0/1139819335 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f0bc00a3ba0 0x7f0bc00a6060 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc000c00 tx=0x7f0bcc0315d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:27.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.635+0000 7f0bd2a00640 1 -- 172.21.15.5:0/1139819335 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0bc809d290 con 0x7f0bdc100610 2024-04-24T06:41:27.722 INFO:journalctl@ceph.mds.b.smithi037.stdout:Apr 24 06:41:27 smithi037 podman[63315]: 2024-04-24 06:41:27.280269068 +0000 UTC m=+0.018588417 image pull b5c3183062a6ce19f361681cc58de6e3ac1de1a54c80e6094ec5695237cd98d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b 2024-04-24T06:41:27.722 INFO:journalctl@ceph.mds.b.smithi037.stdout:Apr 24 06:41:27 smithi037 podman[63315]: 2024-04-24 06:41:27.377793022 +0000 UTC m=+0.116112365 container create fd4020104cab49cbb86a2c65378dc7b1f489fdddd822a068f201358fc1138541 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-24T06:41:27.722 INFO:journalctl@ceph.mds.b.smithi037.stdout:Apr 24 06:41:27 smithi037 podman[63315]: 2024-04-24 06:41:27.721985732 +0000 UTC m=+0.460305073 container init fd4020104cab49cbb86a2c65378dc7b1f489fdddd822a068f201358fc1138541 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-24T06:41:27.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.890+0000 7f0bdb200640 1 -- 172.21.15.5:0/1139819335 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0bdc1a2d00 con 0x7f0bdc100610 2024-04-24T06:41:27.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.891+0000 7f0bd2a00640 1 -- 172.21.15.5:0/1139819335 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9644 (secure 0 0 0) 0x7f0bc809cb90 con 0x7f0bdc100610 2024-04-24T06:41:27.892 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:41:27.892 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1735462750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1735462750},{"type":"v1","addr":"172.21.15.5:6837","nonce":1735462750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1499469370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1499469370},{"type":"v1","addr":"172.21.15.5:6841","nonce":1499469370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3705638847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3705638847},{"type":"v1","addr":"172.21.15.37:6835","nonce":3705638847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24515,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1390103296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1390103296},{"type":"v1","addr":"172.21.15.37:6837","nonce":1390103296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24520,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3217754215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3217754215},{"type":"v1","addr":"172.21.15.137:6835","nonce":3217754215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24532,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/3044222976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3044222976},{"type":"v1","addr":"172.21.15.137:6837","nonce":3044222976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24539,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3173201459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3173201459},{"type":"v1","addr":"172.21.15.5:6839","nonce":3173201459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/1889774232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1889774232},{"type":"v1","addr":"172.21.15.137:6839","nonce":1889774232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-24T06:41:27.893 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 12 2024-04-24T06:41:27.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.894+0000 7f0bdb200640 1 -- 172.21.15.5:0/1139819335 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f0bc00a3ba0 msgr2=0x7f0bc00a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:27.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.894+0000 7f0bdb200640 1 --2- 172.21.15.5:0/1139819335 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f0bc00a3ba0 0x7f0bc00a6060 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc000c00 tx=0x7f0bcc0315d0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.894+0000 7f0bdb200640 1 -- 172.21.15.5:0/1139819335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bdc100610 msgr2=0x7f0bdc19a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:27.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.894+0000 7f0bdb200640 1 --2- 172.21.15.5:0/1139819335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bdc100610 0x7f0bdc19a9f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0bc800e9c0 tx=0x7f0bc800ee80 comp rx=0 tx=0).stop 2024-04-24T06:41:27.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.894+0000 7f0bdb200640 1 -- 172.21.15.5:0/1139819335 shutdown_connections 2024-04-24T06:41:27.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.894+0000 7f0bdb200640 1 --2- 172.21.15.5:0/1139819335 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f0bc00a3ba0 0x7f0bc00a6060 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.894+0000 7f0bdb200640 1 --2- 172.21.15.5:0/1139819335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bdc107ba0 0x7f0bdc1a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.894+0000 7f0bdb200640 1 --2- 172.21.15.5:0/1139819335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bdc100610 0x7f0bdc19a9f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.894+0000 7f0bdb200640 1 --2- 172.21.15.5:0/1139819335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bdc0ffcb0 0x7f0bdc19a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:27.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.894+0000 7f0bdb200640 1 -- 172.21.15.5:0/1139819335 >> 172.21.15.5:0/1139819335 conn(0x7f0bdc0fb700 msgr2=0x7f0bdc0fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:27.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.894+0000 7f0bdb200640 1 -- 172.21.15.5:0/1139819335 shutdown_connections 2024-04-24T06:41:27.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:27.894+0000 7f0bdb200640 1 -- 172.21.15.5:0/1139819335 wait complete. 2024-04-24T06:41:28.018 INFO:journalctl@ceph.mds.b.smithi037.stdout:Apr 24 06:41:27 smithi037 podman[63315]: 2024-04-24 06:41:27.725080787 +0000 UTC m=+0.463400127 container start fd4020104cab49cbb86a2c65378dc7b1f489fdddd822a068f201358fc1138541 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-b, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T06:41:28.019 INFO:journalctl@ceph.mds.b.smithi037.stdout:Apr 24 06:41:27 smithi037 ceph-mds[63334]: -- 172.21.15.37:0/1390676025 <== mon.2 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561a8d687a40 con 0x561a8d65ac00 2024-04-24T06:41:28.019 INFO:journalctl@ceph.mds.b.smithi037.stdout:Apr 24 06:41:27 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-b[63330]: starting mds.b at 2024-04-24T06:41:28.019 INFO:journalctl@ceph.mds.b.smithi037.stdout:Apr 24 06:41:27 smithi037 bash[63315]: fd4020104cab49cbb86a2c65378dc7b1f489fdddd822a068f201358fc1138541 2024-04-24T06:41:28.019 INFO:journalctl@ceph.mds.b.smithi037.stdout:Apr 24 06:41:27 smithi037 systemd[1]: Started Ceph mds.b for 43e9bad0-0204-11ef-bc93-c7b262605968. 2024-04-24T06:41:29.131 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:28 smithi137 ceph-mon[33017]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:29.131 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:28 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:29.131 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:28 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:29.131 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:28 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/1139819335' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:29.131 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:28 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:29.131 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:28 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:29.131 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:28 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:29.131 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:28 smithi137 ceph-mon[33017]: mds.? [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] up:boot 2024-04-24T06:41:29.131 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:28 smithi137 ceph-mon[33017]: fsmap 12 up:standby 2024-04-24T06:41:29.131 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:28 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T06:41:29.132 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:28 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:41:29.252 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:28 smithi037 ceph-mon[32864]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:29.252 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:28 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:29.252 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:28 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:29.252 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:28 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/1139819335' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:29.252 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:28 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:29.252 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:28 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:29.252 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:28 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:29.252 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:28 smithi037 ceph-mon[32864]: mds.? [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] up:boot 2024-04-24T06:41:29.253 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:28 smithi037 ceph-mon[32864]: fsmap 12 up:standby 2024-04-24T06:41:29.253 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:28 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T06:41:29.253 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:28 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:41:29.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:28 smithi005 ceph-mon[27798]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:29.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:28 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:29.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:28 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:29.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:28 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1139819335' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:29.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:28 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:29.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:28 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:29.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:28 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:29.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:28 smithi005 ceph-mon[27798]: mds.? [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] up:boot 2024-04-24T06:41:29.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:28 smithi005 ceph-mon[27798]: fsmap 12 up:standby 2024-04-24T06:41:29.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:28 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T06:41:29.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:28 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:41:29.916 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:41:30.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.131+0000 7f523a400640 1 -- 172.21.15.5:0/816615959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5234100200 msgr2=0x7f5234100620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:30.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.131+0000 7f523a400640 1 --2- 172.21.15.5:0/816615959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5234100200 0x7f5234100620 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5228009900 tx=0x7f522802a410 comp rx=0 tx=0).stop 2024-04-24T06:41:30.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.131+0000 7f523a400640 1 -- 172.21.15.5:0/816615959 shutdown_connections 2024-04-24T06:41:30.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.131+0000 7f523a400640 1 --2- 172.21.15.5:0/816615959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5234103b90 0x7f523410a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:30.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.131+0000 7f523a400640 1 --2- 172.21.15.5:0/816615959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52341031d0 0x7f5234103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:30.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.131+0000 7f523a400640 1 --2- 172.21.15.5:0/816615959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5234100200 0x7f5234100620 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:30.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.131+0000 7f523a400640 1 -- 172.21.15.5:0/816615959 >> 172.21.15.5:0/816615959 conn(0x7f52340fbb70 msgr2=0x7f52340fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:30.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.131+0000 7f523a400640 1 -- 172.21.15.5:0/816615959 shutdown_connections 2024-04-24T06:41:30.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.131+0000 7f523a400640 1 -- 172.21.15.5:0/816615959 wait complete. 2024-04-24T06:41:30.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.132+0000 7f523a400640 1 Processor -- start 2024-04-24T06:41:30.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.132+0000 7f523a400640 1 -- start start 2024-04-24T06:41:30.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.132+0000 7f523a400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5234100200 0x7f523419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:30.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.132+0000 7f523a400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52341031d0 0x7f523419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:30.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.132+0000 7f523a400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5234103b90 0x7f52341a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:30.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.132+0000 7f523a400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f523410dc30 con 0x7f52341031d0 2024-04-24T06:41:30.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.132+0000 7f523a400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f523410dab0 con 0x7f5234103b90 2024-04-24T06:41:30.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.132+0000 7f523a400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f523410ddb0 con 0x7f5234100200 2024-04-24T06:41:30.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.133+0000 7f5239000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5234100200 0x7f523419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:30.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.133+0000 7f5233e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52341031d0 0x7f523419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:30.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.133+0000 7f5239000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5234100200 0x7f523419a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:59624/0 (socket says 172.21.15.5:59624) 2024-04-24T06:41:30.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.133+0000 7f5239000640 1 -- 172.21.15.5:0/658237717 learned_addr learned my addr 172.21.15.5:0/658237717 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:41:30.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.133+0000 7f5239a00640 1 --2- 172.21.15.5:0/658237717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5234103b90 0x7f52341a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:30.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.133+0000 7f5233e00640 1 -- 172.21.15.5:0/658237717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5234100200 msgr2=0x7f523419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:30.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.133+0000 7f5233e00640 1 --2- 172.21.15.5:0/658237717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5234100200 0x7f523419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:30.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.133+0000 7f5233e00640 1 -- 172.21.15.5:0/658237717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5234103b90 msgr2=0x7f52341a1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:30.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.133+0000 7f5233e00640 1 --2- 172.21.15.5:0/658237717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5234103b90 0x7f52341a1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:30.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.133+0000 7f5233e00640 1 -- 172.21.15.5:0/658237717 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52341a20a0 con 0x7f52341031d0 2024-04-24T06:41:30.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.133+0000 7f5239000640 1 --2- 172.21.15.5:0/658237717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5234100200 0x7f523419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:41:30.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.133+0000 7f5239a00640 1 --2- 172.21.15.5:0/658237717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5234103b90 0x7f52341a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:41:30.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.133+0000 7f5233e00640 1 --2- 172.21.15.5:0/658237717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52341031d0 0x7f523419a9c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f522000e9f0 tx=0x7f522000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:30.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.134+0000 7f5231600640 1 -- 172.21.15.5:0/658237717 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522000ccc0 con 0x7f52341031d0 2024-04-24T06:41:30.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.134+0000 7f5231600640 1 -- 172.21.15.5:0/658237717 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5220010910 con 0x7f52341031d0 2024-04-24T06:41:30.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.134+0000 7f523a400640 1 -- 172.21.15.5:0/658237717 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52341a2390 con 0x7f52341031d0 2024-04-24T06:41:30.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.134+0000 7f5231600640 1 -- 172.21.15.5:0/658237717 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5220018960 con 0x7f52341031d0 2024-04-24T06:41:30.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.134+0000 7f523a400640 1 -- 172.21.15.5:0/658237717 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52341a28e0 con 0x7f52341031d0 2024-04-24T06:41:30.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.135+0000 7f523a400640 1 -- 172.21.15.5:0/658237717 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52341a2b40 con 0x7f52341031d0 2024-04-24T06:41:30.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.137+0000 7f5231600640 1 -- 172.21.15.5:0/658237717 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f5220010ac0 con 0x7f52341031d0 2024-04-24T06:41:30.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.139+0000 7f5231600640 1 --2- 172.21.15.5:0/658237717 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f521c0a3ba0 0x7f521c0a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:30.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.139+0000 7f5231600640 1 -- 172.21.15.5:0/658237717 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f52200d0340 con 0x7f52341031d0 2024-04-24T06:41:30.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.139+0000 7f5239000640 1 --2- 172.21.15.5:0/658237717 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f521c0a3ba0 0x7f521c0a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:30.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.140+0000 7f5239000640 1 --2- 172.21.15.5:0/658237717 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f521c0a3ba0 0x7f521c0a6060 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5228006010 tx=0x7f5228005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:30.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.143+0000 7f5231600640 1 -- 172.21.15.5:0/658237717 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f522009cad0 con 0x7f52341031d0 2024-04-24T06:41:30.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.424+0000 7f523a400640 1 -- 172.21.15.5:0/658237717 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52341a2d10 con 0x7f52341031d0 2024-04-24T06:41:30.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.425+0000 7f5231600640 1 -- 172.21.15.5:0/658237717 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10470 (secure 0 0 0) 0x7f52200a0dd0 con 0x7f52341031d0 2024-04-24T06:41:30.427 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:41:30.427 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1735462750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1735462750},{"type":"v1","addr":"172.21.15.5:6837","nonce":1735462750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1499469370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1499469370},{"type":"v1","addr":"172.21.15.5:6841","nonce":1499469370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3705638847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3705638847},{"type":"v1","addr":"172.21.15.37:6835","nonce":3705638847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24515,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1390103296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1390103296},{"type":"v1","addr":"172.21.15.37:6837","nonce":1390103296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24520,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3217754215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3217754215},{"type":"v1","addr":"172.21.15.137:6835","nonce":3217754215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24532,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/3044222976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3044222976},{"type":"v1","addr":"172.21.15.137:6837","nonce":3044222976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24539,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3173201459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3173201459},{"type":"v1","addr":"172.21.15.5:6839","nonce":3173201459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/1889774232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1889774232},{"type":"v1","addr":"172.21.15.137:6839","nonce":1889774232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24569,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/3536721753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":3536721753},{"type":"v1","addr":"172.21.15.37:6839","nonce":3536721753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-24T06:41:30.427 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 13 2024-04-24T06:41:30.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.428+0000 7f523a400640 1 -- 172.21.15.5:0/658237717 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f521c0a3ba0 msgr2=0x7f521c0a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:30.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.428+0000 7f523a400640 1 --2- 172.21.15.5:0/658237717 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f521c0a3ba0 0x7f521c0a6060 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5228006010 tx=0x7f5228005a20 comp rx=0 tx=0).stop 2024-04-24T06:41:30.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.428+0000 7f523a400640 1 -- 172.21.15.5:0/658237717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52341031d0 msgr2=0x7f523419a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:30.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.428+0000 7f523a400640 1 --2- 172.21.15.5:0/658237717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52341031d0 0x7f523419a9c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f522000e9f0 tx=0x7f522000eeb0 comp rx=0 tx=0).stop 2024-04-24T06:41:30.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.428+0000 7f523a400640 1 -- 172.21.15.5:0/658237717 shutdown_connections 2024-04-24T06:41:30.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.428+0000 7f523a400640 1 --2- 172.21.15.5:0/658237717 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f521c0a3ba0 0x7f521c0a6060 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:30.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.428+0000 7f523a400640 1 --2- 172.21.15.5:0/658237717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5234103b90 0x7f52341a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:30.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.428+0000 7f523a400640 1 --2- 172.21.15.5:0/658237717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52341031d0 0x7f523419a9c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:30.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.428+0000 7f523a400640 1 --2- 172.21.15.5:0/658237717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5234100200 0x7f523419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:30.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.428+0000 7f523a400640 1 -- 172.21.15.5:0/658237717 >> 172.21.15.5:0/658237717 conn(0x7f52340fbb70 msgr2=0x7f52341089a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:30.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.429+0000 7f523a400640 1 -- 172.21.15.5:0/658237717 shutdown_connections 2024-04-24T06:41:30.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.429+0000 7f523a400640 1 -- 172.21.15.5:0/658237717 wait complete. 2024-04-24T06:41:30.460 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:41:30.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.670+0000 7f47cbc00640 1 -- 172.21.15.5:0/1444493293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47c4103c70 msgr2=0x7f47c410a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:30.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.670+0000 7f47cbc00640 1 --2- 172.21.15.5:0/1444493293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47c4103c70 0x7f47c410a500 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f47bc009900 tx=0x7f47bc02a3f0 comp rx=0 tx=0).stop 2024-04-24T06:41:30.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.670+0000 7f47cbc00640 1 -- 172.21.15.5:0/1444493293 shutdown_connections 2024-04-24T06:41:30.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.670+0000 7f47cbc00640 1 --2- 172.21.15.5:0/1444493293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47c4103c70 0x7f47c410a500 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:30.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.670+0000 7f47cbc00640 1 --2- 172.21.15.5:0/1444493293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47c41032b0 0x7f47c4103730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:30.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.670+0000 7f47cbc00640 1 --2- 172.21.15.5:0/1444493293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47c41020c0 0x7f47c41024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:30.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.670+0000 7f47cbc00640 1 -- 172.21.15.5:0/1444493293 >> 172.21.15.5:0/1444493293 conn(0x7f47c40fd630 msgr2=0x7f47c40ffa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:30.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.670+0000 7f47cbc00640 1 -- 172.21.15.5:0/1444493293 shutdown_connections 2024-04-24T06:41:30.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.670+0000 7f47cbc00640 1 -- 172.21.15.5:0/1444493293 wait complete. 2024-04-24T06:41:30.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.671+0000 7f47cbc00640 1 Processor -- start 2024-04-24T06:41:30.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.672+0000 7f47cbc00640 1 -- start start 2024-04-24T06:41:30.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.672+0000 7f47cbc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47c41020c0 0x7f47c41982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:30.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.672+0000 7f47cbc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47c41032b0 0x7f47c41987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:30.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.672+0000 7f47cbc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47c4103c70 0x7f47c419f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:30.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.672+0000 7f47cbc00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f47c410dd30 con 0x7f47c4103c70 2024-04-24T06:41:30.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.672+0000 7f47cbc00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f47c410dbb0 con 0x7f47c41020c0 2024-04-24T06:41:30.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.672+0000 7f47cbc00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f47c410deb0 con 0x7f47c41032b0 2024-04-24T06:41:30.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.672+0000 7f47c9c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47c4103c70 0x7f47c419f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:30.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.672+0000 7f47c3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47c41032b0 0x7f47c41987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:30.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.672+0000 7f47c3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47c41032b0 0x7f47c41987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:59638/0 (socket says 172.21.15.5:59638) 2024-04-24T06:41:30.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.672+0000 7f47c9c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47c4103c70 0x7f47c419f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50486/0 (socket says 172.21.15.5:50486) 2024-04-24T06:41:30.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.672+0000 7f47c9200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47c41020c0 0x7f47c41982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:30.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.673+0000 7f47c3e00640 1 -- 172.21.15.5:0/3242219957 learned_addr learned my addr 172.21.15.5:0/3242219957 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:41:30.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.673+0000 7f47c3e00640 1 -- 172.21.15.5:0/3242219957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47c41020c0 msgr2=0x7f47c41982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:30.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.673+0000 7f47c3e00640 1 --2- 172.21.15.5:0/3242219957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47c41020c0 0x7f47c41982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:30.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.673+0000 7f47c3e00640 1 -- 172.21.15.5:0/3242219957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47c4103c70 msgr2=0x7f47c419f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:30.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.673+0000 7f47c3e00640 1 --2- 172.21.15.5:0/3242219957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47c4103c70 0x7f47c419f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:30.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.673+0000 7f47c3e00640 1 -- 172.21.15.5:0/3242219957 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c419fed0 con 0x7f47c41032b0 2024-04-24T06:41:30.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.673+0000 7f47c3e00640 1 --2- 172.21.15.5:0/3242219957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47c41032b0 0x7f47c41987f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f47b000e9f0 tx=0x7f47b000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:30.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.673+0000 7f47c1600640 1 -- 172.21.15.5:0/3242219957 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b000cd80 con 0x7f47c41032b0 2024-04-24T06:41:30.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.673+0000 7f47c1600640 1 -- 172.21.15.5:0/3242219957 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47b0010910 con 0x7f47c41032b0 2024-04-24T06:41:30.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.673+0000 7f47c1600640 1 -- 172.21.15.5:0/3242219957 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b0018990 con 0x7f47c41032b0 2024-04-24T06:41:30.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.674+0000 7f47cbc00640 1 -- 172.21.15.5:0/3242219957 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47c4074290 con 0x7f47c41032b0 2024-04-24T06:41:30.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.674+0000 7f47cbc00640 1 -- 172.21.15.5:0/3242219957 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47c4074830 con 0x7f47c41032b0 2024-04-24T06:41:30.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.675+0000 7f47cbc00640 1 -- 172.21.15.5:0/3242219957 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47c4109380 con 0x7f47c41032b0 2024-04-24T06:41:30.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.678+0000 7f47c1600640 1 -- 172.21.15.5:0/3242219957 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f47b0010ac0 con 0x7f47c41032b0 2024-04-24T06:41:30.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.679+0000 7f47c1600640 1 --2- 172.21.15.5:0/3242219957 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f47a80a3c30 0x7f47a80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:30.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.679+0000 7f47c1600640 1 -- 172.21.15.5:0/3242219957 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f47b00d0480 con 0x7f47c41032b0 2024-04-24T06:41:30.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.679+0000 7f47c9200640 1 --2- 172.21.15.5:0/3242219957 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f47a80a3c30 0x7f47a80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:30.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.680+0000 7f47c1600640 1 -- 172.21.15.5:0/3242219957 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f47b009cc10 con 0x7f47c41032b0 2024-04-24T06:41:30.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.680+0000 7f47c9200640 1 --2- 172.21.15.5:0/3242219957 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f47a80a3c30 0x7f47a80a60f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f47c4103110 tx=0x7f47b4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:30.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.907+0000 7f47cbc00640 1 -- 172.21.15.5:0/3242219957 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f47c402c4a0 con 0x7f47c41032b0 2024-04-24T06:41:30.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.909+0000 7f47c1600640 1 -- 172.21.15.5:0/3242219957 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18325 (secure 0 0 0) 0x7f47b00bbac0 con 0x7f47c41032b0 2024-04-24T06:41:30.910 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:41:30.910 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":82,"fsid":"43e9bad0-0204-11ef-bc93-c7b262605968","created":"2024-04-24T06:32:26.340555+0000","modified":"2024-04-24T06:40:37.808668+0000","last_up_change":"2024-04-24T06:40:36.790934+0000","last_in_change":"2024-04-24T06:40:17.803257+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:36:27.320342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c2e974c-745c-4dd5-8c3d-8e8da2eb1ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6803","nonce":2155858515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6805","nonce":2155858515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6809","nonce":2155858515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6807","nonce":2155858515}]},"public_addr":"172.21.15.5:6803/2155858515","cluster_addr":"172.21.15.5:6805/2155858515","heartbeat_back_addr":"172.21.15.5:6809/2155858515","heartbeat_front_addr":"172.21.15.5:6807/2155858515","state":["exists","up"]},{"osd":1,"uuid":"50467744-db20-4721-bf2d-1c5db938297b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6801","nonce":1302362650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6803","nonce":1302362650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6807","nonce":1302362650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6805","nonce":1302362650}]},"public_addr":"172.21.15.37:6801/1302362650","cluster_addr":"172.21.15.37:6803/1302362650","heartbeat_back_addr":"172.21.15.37:6807/1302362650","heartbeat_front_addr":"172.21.15.37:6805/1302362650","state":["exists","up"]},{"osd":2,"uuid":"0e190b15-d04c-4288-8d48-ecf6664392fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6801","nonce":2885027256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6803","nonce":2885027256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6807","nonce":2885027256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6805","nonce":2885027256}]},"public_addr":"172.21.15.137:6801/2885027256","cluster_addr":"172.21.15.137:6803/2885027256","heartbeat_back_addr":"172.21.15.137:6807/2885027256","heartbeat_front_addr":"172.21.15.137:6805/2885027256","state":["exists","up"]},{"osd":3,"uuid":"e05592ba-d96a-472a-9f63-09a535d04dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6811","nonce":3701747260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6813","nonce":3701747260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6817","nonce":3701747260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6815","nonce":3701747260}]},"public_addr":"172.21.15.5:6811/3701747260","cluster_addr":"172.21.15.5:6813/3701747260","heartbeat_back_addr":"172.21.15.5:6817/3701747260","heartbeat_front_addr":"172.21.15.5:6815/3701747260","state":["exists","up"]},{"osd":4,"uuid":"c3f36e51-7a3b-4dd9-af4b-a10558830c4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6809","nonce":759253844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6811","nonce":759253844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6815","nonce":759253844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6813","nonce":759253844}]},"public_addr":"172.21.15.37:6809/759253844","cluster_addr":"172.21.15.37:6811/759253844","heartbeat_back_addr":"172.21.15.37:6815/759253844","heartbeat_front_addr":"172.21.15.37:6813/759253844","state":["exists","up"]},{"osd":5,"uuid":"97a0500c-cbb8-4ff5-b476-8634b26b3821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6809","nonce":3771921560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6811","nonce":3771921560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6815","nonce":3771921560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6813","nonce":3771921560}]},"public_addr":"172.21.15.137:6809/3771921560","cluster_addr":"172.21.15.137:6811/3771921560","heartbeat_back_addr":"172.21.15.137:6815/3771921560","heartbeat_front_addr":"172.21.15.137:6813/3771921560","state":["exists","up"]},{"osd":6,"uuid":"33a33d62-bbb3-4ed9-834e-19ae93a97670","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6819","nonce":2064636084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6821","nonce":2064636084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6825","nonce":2064636084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6823","nonce":2064636084}]},"public_addr":"172.21.15.5:6819/2064636084","cluster_addr":"172.21.15.5:6821/2064636084","heartbeat_back_addr":"172.21.15.5:6825/2064636084","heartbeat_front_addr":"172.21.15.5:6823/2064636084","state":["exists","up"]},{"osd":7,"uuid":"c0f4e036-4717-4514-bb59-ab11fbaab063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6817","nonce":2952750659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6819","nonce":2952750659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6823","nonce":2952750659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6821","nonce":2952750659}]},"public_addr":"172.21.15.37:6817/2952750659","cluster_addr":"172.21.15.37:6819/2952750659","heartbeat_back_addr":"172.21.15.37:6823/2952750659","heartbeat_front_addr":"172.21.15.37:6821/2952750659","state":["exists","up"]},{"osd":8,"uuid":"5391fd87-44ea-440f-80da-c54d0327c04c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6817","nonce":29724945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6819","nonce":29724945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6823","nonce":29724945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6821","nonce":29724945}]},"public_addr":"172.21.15.137:6817/29724945","cluster_addr":"172.21.15.137:6819/29724945","heartbeat_back_addr":"172.21.15.137:6823/29724945","heartbeat_front_addr":"172.21.15.137:6821/29724945","state":["exists","up"]},{"osd":9,"uuid":"d94c9dbc-f43d-4237-ac7f-47a6a5ede1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6827","nonce":2257102107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6829","nonce":2257102107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6833","nonce":2257102107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6831","nonce":2257102107}]},"public_addr":"172.21.15.5:6827/2257102107","cluster_addr":"172.21.15.5:6829/2257102107","heartbeat_back_addr":"172.21.15.5:6833/2257102107","heartbeat_front_addr":"172.21.15.5:6831/2257102107","state":["exists","up"]},{"osd":10,"uuid":"e2455332-2a5e-44b1-85d3-8e47da399f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6825","nonce":2451570637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6827","nonce":2451570637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6831","nonce":2451570637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6829","nonce":2451570637}]},"public_addr":"172.21.15.37:6825/2451570637","cluster_addr":"172.21.15.37:6827/2451570637","heartbeat_back_addr":"172.21.15.37:6831/2451570637","heartbeat_front_addr":"172.21.15.37:6829/2451570637","state":["exists","up"]},{"osd":11,"uuid":"49d9396b-704a-4cef-8210-75d30c28e924","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6825","nonce":90600596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6827","nonce":90600596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6831","nonce":90600596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6829","nonce":90600596}]},"public_addr":"172.21.15.137:6825/90600596","cluster_addr":"172.21.15.137:6827/90600596","heartbeat_back_addr":"172.21.15.137:6831/90600596","heartbeat_front_addr":"172.21.15.137:6829/90600596","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:35:35.591619+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:00.568817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:24.195643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:56.451724+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:22.109293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:46.161749+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:19.278305+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:45.620256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:10.041165+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:43.813697+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:10.736026+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:35.384377+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/23307164":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6801/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:0/230785941":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/2940023597":"2024-04-25T06:33:07.040369+0000","172.21.15.5:6801/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3886613904":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/4199165437":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3459593499":"2024-04-25T06:33:41.161852+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:41:30.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.912+0000 7f47cbc00640 1 -- 172.21.15.5:0/3242219957 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f47a80a3c30 msgr2=0x7f47a80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:30.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.912+0000 7f47cbc00640 1 --2- 172.21.15.5:0/3242219957 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f47a80a3c30 0x7f47a80a60f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f47c4103110 tx=0x7f47b4009500 comp rx=0 tx=0).stop 2024-04-24T06:41:30.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.912+0000 7f47cbc00640 1 -- 172.21.15.5:0/3242219957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47c41032b0 msgr2=0x7f47c41987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:30.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.912+0000 7f47cbc00640 1 --2- 172.21.15.5:0/3242219957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47c41032b0 0x7f47c41987f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f47b000e9f0 tx=0x7f47b000eeb0 comp rx=0 tx=0).stop 2024-04-24T06:41:30.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.912+0000 7f47cbc00640 1 -- 172.21.15.5:0/3242219957 shutdown_connections 2024-04-24T06:41:30.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.912+0000 7f47cbc00640 1 --2- 172.21.15.5:0/3242219957 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f47a80a3c30 0x7f47a80a60f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:30.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.912+0000 7f47cbc00640 1 --2- 172.21.15.5:0/3242219957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47c4103c70 0x7f47c419f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:30.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.912+0000 7f47cbc00640 1 --2- 172.21.15.5:0/3242219957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47c41032b0 0x7f47c41987f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:30.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.912+0000 7f47cbc00640 1 --2- 172.21.15.5:0/3242219957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47c41020c0 0x7f47c41982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:30.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.912+0000 7f47cbc00640 1 -- 172.21.15.5:0/3242219957 >> 172.21.15.5:0/3242219957 conn(0x7f47c40fd630 msgr2=0x7f47c4106550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:30.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.912+0000 7f47cbc00640 1 -- 172.21.15.5:0/3242219957 shutdown_connections 2024-04-24T06:41:30.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:30.912+0000 7f47cbc00640 1 -- 172.21.15.5:0/3242219957 wait complete. 2024-04-24T06:41:30.935 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:36:27.320342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:41:30.936 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:41:31.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.120+0000 7fa3ed400640 1 -- 172.21.15.5:0/151728992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3e8104260 msgr2=0x7fa3e8104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:31.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.120+0000 7fa3ed400640 1 --2- 172.21.15.5:0/151728992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3e8104260 0x7fa3e8104680 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8009900 tx=0x7fa3d802a410 comp rx=0 tx=0).stop 2024-04-24T06:41:31.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.120+0000 7fa3ed400640 1 -- 172.21.15.5:0/151728992 shutdown_connections 2024-04-24T06:41:31.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.120+0000 7fa3ed400640 1 --2- 172.21.15.5:0/151728992 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3e8105e10 0x7fa3e810c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.120+0000 7fa3ed400640 1 --2- 172.21.15.5:0/151728992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3e8105450 0x7fa3e81058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.120+0000 7fa3ed400640 1 --2- 172.21.15.5:0/151728992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3e8104260 0x7fa3e8104680 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.120+0000 7fa3ed400640 1 -- 172.21.15.5:0/151728992 >> 172.21.15.5:0/151728992 conn(0x7fa3e80ff7f0 msgr2=0x7fa3e8101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:31.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.120+0000 7fa3ed400640 1 -- 172.21.15.5:0/151728992 shutdown_connections 2024-04-24T06:41:31.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.120+0000 7fa3ed400640 1 -- 172.21.15.5:0/151728992 wait complete. 2024-04-24T06:41:31.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.121+0000 7fa3ed400640 1 Processor -- start 2024-04-24T06:41:31.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.121+0000 7fa3ed400640 1 -- start start 2024-04-24T06:41:31.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.122+0000 7fa3ed400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3e8104260 0x7fa3e819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:31.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.122+0000 7fa3ed400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3e8105450 0x7fa3e819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:31.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.122+0000 7fa3ed400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3e8105e10 0x7fa3e81a1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:31.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.122+0000 7fa3ed400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e810ff40 con 0x7fa3e8105e10 2024-04-24T06:41:31.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.122+0000 7fa3ed400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e810fdc0 con 0x7fa3e8105450 2024-04-24T06:41:31.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.122+0000 7fa3ed400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e81100c0 con 0x7fa3e8104260 2024-04-24T06:41:31.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.122+0000 7fa3e6800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3e8104260 0x7fa3e819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:31.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.122+0000 7fa3e6800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3e8104260 0x7fa3e819a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:59650/0 (socket says 172.21.15.5:59650) 2024-04-24T06:41:31.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.122+0000 7fa3e6800640 1 -- 172.21.15.5:0/2857086122 learned_addr learned my addr 172.21.15.5:0/2857086122 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:41:31.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.122+0000 7fa3e5e00640 1 --2- 172.21.15.5:0/2857086122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3e8105450 0x7fa3e819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:31.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.122+0000 7fa3e7200640 1 --2- 172.21.15.5:0/2857086122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3e8105e10 0x7fa3e81a1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:31.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.123+0000 7fa3e6800640 1 -- 172.21.15.5:0/2857086122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3e8105450 msgr2=0x7fa3e819ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:31.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.123+0000 7fa3e6800640 1 --2- 172.21.15.5:0/2857086122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3e8105450 0x7fa3e819ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.123+0000 7fa3e6800640 1 -- 172.21.15.5:0/2857086122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3e8105e10 msgr2=0x7fa3e81a1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:31.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.123+0000 7fa3e6800640 1 --2- 172.21.15.5:0/2857086122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3e8105e10 0x7fa3e81a1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.123+0000 7fa3e6800640 1 -- 172.21.15.5:0/2857086122 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e81a21f0 con 0x7fa3e8104260 2024-04-24T06:41:31.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.123+0000 7fa3e7200640 1 --2- 172.21.15.5:0/2857086122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3e8105e10 0x7fa3e81a1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:41:31.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.123+0000 7fa3e6800640 1 --2- 172.21.15.5:0/2857086122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3e8104260 0x7fa3e819a5d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8031440 tx=0x7fa3d8032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:31.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.123+0000 7fa3df400640 1 -- 172.21.15.5:0/2857086122 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d803f030 con 0x7fa3e8104260 2024-04-24T06:41:31.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.123+0000 7fa3df400640 1 -- 172.21.15.5:0/2857086122 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3d8003ce0 con 0x7fa3e8104260 2024-04-24T06:41:31.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.123+0000 7fa3df400640 1 -- 172.21.15.5:0/2857086122 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d8035070 con 0x7fa3e8104260 2024-04-24T06:41:31.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.123+0000 7fa3ed400640 1 -- 172.21.15.5:0/2857086122 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3e81a2480 con 0x7fa3e8104260 2024-04-24T06:41:31.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.124+0000 7fa3ed400640 1 -- 172.21.15.5:0/2857086122 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3e81a2920 con 0x7fa3e8104260 2024-04-24T06:41:31.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.125+0000 7fa3ed400640 1 -- 172.21.15.5:0/2857086122 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3e810b5d0 con 0x7fa3e8104260 2024-04-24T06:41:31.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.127+0000 7fa3df400640 1 -- 172.21.15.5:0/2857086122 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fa3d8042030 con 0x7fa3e8104260 2024-04-24T06:41:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.128+0000 7fa3df400640 1 --2- 172.21.15.5:0/2857086122 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fa3cc0a3c30 0x7fa3cc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.128+0000 7fa3df400640 1 -- 172.21.15.5:0/2857086122 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa3d80f0460 con 0x7fa3e8104260 2024-04-24T06:41:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.129+0000 7fa3e5e00640 1 --2- 172.21.15.5:0/2857086122 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fa3cc0a3c30 0x7fa3cc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.129+0000 7fa3e5e00640 1 --2- 172.21.15.5:0/2857086122 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fa3cc0a3c30 0x7fa3cc0a60f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa3e819bae0 tx=0x7fa3d4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:31.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.132+0000 7fa3df400640 1 -- 172.21.15.5:0/2857086122 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa3d80bd240 con 0x7fa3e8104260 2024-04-24T06:41:31.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:30 smithi137 ceph-mon[33017]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:31.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:30 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:31.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:30 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:31.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:30 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:31.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:30 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:31.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:30 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/658237717' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:31.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:30 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/3242219957' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:41:31.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:30 smithi005 ceph-mon[27798]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:31.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:30 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:31.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:30 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:31.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:30 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:31.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:30 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:31.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:30 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/658237717' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:31.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:30 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/3242219957' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:41:31.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:30 smithi037 ceph-mon[32864]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:31.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:30 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:31.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:30 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:31.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:30 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:31.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:30 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:31.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:30 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/658237717' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:31.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:30 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/3242219957' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:41:31.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.347+0000 7fa3ed400640 1 -- 172.21.15.5:0/2857086122 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa3e8062c00 con 0x7fa3e8104260 2024-04-24T06:41:31.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.348+0000 7fa3df400640 1 -- 172.21.15.5:0/2857086122 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fa3d80bcb40 con 0x7fa3e8104260 2024-04-24T06:41:31.349 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-24T06:41:31.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.350+0000 7fa3ed400640 1 -- 172.21.15.5:0/2857086122 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fa3cc0a3c30 msgr2=0x7fa3cc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:31.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.350+0000 7fa3ed400640 1 --2- 172.21.15.5:0/2857086122 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fa3cc0a3c30 0x7fa3cc0a60f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa3e819bae0 tx=0x7fa3d4005f90 comp rx=0 tx=0).stop 2024-04-24T06:41:31.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.350+0000 7fa3ed400640 1 -- 172.21.15.5:0/2857086122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3e8104260 msgr2=0x7fa3e819a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:31.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.350+0000 7fa3ed400640 1 --2- 172.21.15.5:0/2857086122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3e8104260 0x7fa3e819a5d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8031440 tx=0x7fa3d8032ce0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.350+0000 7fa3ed400640 1 -- 172.21.15.5:0/2857086122 shutdown_connections 2024-04-24T06:41:31.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.350+0000 7fa3ed400640 1 --2- 172.21.15.5:0/2857086122 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fa3cc0a3c30 0x7fa3cc0a60f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.350+0000 7fa3ed400640 1 --2- 172.21.15.5:0/2857086122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3e8105e10 0x7fa3e81a1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.350+0000 7fa3ed400640 1 --2- 172.21.15.5:0/2857086122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3e8105450 0x7fa3e819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.350+0000 7fa3ed400640 1 --2- 172.21.15.5:0/2857086122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3e8104260 0x7fa3e819a5d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.350+0000 7fa3ed400640 1 -- 172.21.15.5:0/2857086122 >> 172.21.15.5:0/2857086122 conn(0x7fa3e80ff7f0 msgr2=0x7fa3e81087a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:31.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.350+0000 7fa3ed400640 1 -- 172.21.15.5:0/2857086122 shutdown_connections 2024-04-24T06:41:31.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.350+0000 7fa3ed400640 1 -- 172.21.15.5:0/2857086122 wait complete. 2024-04-24T06:41:31.369 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-24T06:41:31.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.557+0000 7f4c67000640 1 -- 172.21.15.5:0/754971359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c60077c00 msgr2=0x7f4c60078060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:31.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.557+0000 7f4c67000640 1 --2- 172.21.15.5:0/754971359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c60077c00 0x7f4c60078060 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009900 tx=0x7f4c4c02a410 comp rx=0 tx=0).stop 2024-04-24T06:41:31.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.558+0000 7f4c67000640 1 -- 172.21.15.5:0/754971359 shutdown_connections 2024-04-24T06:41:31.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.558+0000 7f4c67000640 1 --2- 172.21.15.5:0/754971359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c600785a0 0x7f4c60078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.558+0000 7f4c67000640 1 --2- 172.21.15.5:0/754971359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c60077c00 0x7f4c60078060 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.558+0000 7f4c67000640 1 --2- 172.21.15.5:0/754971359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c600769c0 0x7f4c60076de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.558+0000 7f4c67000640 1 -- 172.21.15.5:0/754971359 >> 172.21.15.5:0/754971359 conn(0x7f4c600ff850 msgr2=0x7f4c60101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:31.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.558+0000 7f4c67000640 1 -- 172.21.15.5:0/754971359 shutdown_connections 2024-04-24T06:41:31.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.558+0000 7f4c67000640 1 -- 172.21.15.5:0/754971359 wait complete. 2024-04-24T06:41:31.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.559+0000 7f4c67000640 1 Processor -- start 2024-04-24T06:41:31.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.559+0000 7f4c67000640 1 -- start start 2024-04-24T06:41:31.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.559+0000 7f4c67000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c600769c0 0x7f4c6019e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:31.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.559+0000 7f4c67000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c60077c00 0x7f4c6019ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:31.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.559+0000 7f4c67000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c600785a0 0x7f4c601a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:31.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.559+0000 7f4c67000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4c601142c0 con 0x7f4c600769c0 2024-04-24T06:41:31.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.559+0000 7f4c67000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4c60114140 con 0x7f4c600785a0 2024-04-24T06:41:31.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.559+0000 7f4c67000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4c60114440 con 0x7f4c60077c00 2024-04-24T06:41:31.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.560+0000 7f4c5fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c600769c0 0x7f4c6019e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:31.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.560+0000 7f4c5fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c600769c0 0x7f4c6019e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50528/0 (socket says 172.21.15.5:50528) 2024-04-24T06:41:31.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.560+0000 7f4c65000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c600785a0 0x7f4c601a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:31.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.560+0000 7f4c5fe00640 1 -- 172.21.15.5:0/2978007032 learned_addr learned my addr 172.21.15.5:0/2978007032 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:41:31.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.560+0000 7f4c5f400640 1 --2- 172.21.15.5:0/2978007032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c60077c00 0x7f4c6019ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:31.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.560+0000 7f4c5fe00640 1 -- 172.21.15.5:0/2978007032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c60077c00 msgr2=0x7f4c6019ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:31.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.560+0000 7f4c5fe00640 1 --2- 172.21.15.5:0/2978007032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c60077c00 0x7f4c6019ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.560+0000 7f4c5fe00640 1 -- 172.21.15.5:0/2978007032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c600785a0 msgr2=0x7f4c601a5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:31.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.560+0000 7f4c5fe00640 1 --2- 172.21.15.5:0/2978007032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c600785a0 0x7f4c601a5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.560+0000 7f4c5fe00640 1 -- 172.21.15.5:0/2978007032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c601a6560 con 0x7f4c600769c0 2024-04-24T06:41:31.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.560+0000 7f4c65000640 1 --2- 172.21.15.5:0/2978007032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c600785a0 0x7f4c601a5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:41:31.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.560+0000 7f4c5fe00640 1 --2- 172.21.15.5:0/2978007032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c600769c0 0x7f4c6019e940 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4c5000e9f0 tx=0x7f4c5000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:31.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.561+0000 7f4c5cc00640 1 -- 172.21.15.5:0/2978007032 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5000cd80 con 0x7f4c600769c0 2024-04-24T06:41:31.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.561+0000 7f4c5cc00640 1 -- 172.21.15.5:0/2978007032 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c50010910 con 0x7f4c600769c0 2024-04-24T06:41:31.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.561+0000 7f4c67000640 1 -- 172.21.15.5:0/2978007032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c601a6850 con 0x7f4c600769c0 2024-04-24T06:41:31.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.561+0000 7f4c5cc00640 1 -- 172.21.15.5:0/2978007032 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c50018990 con 0x7f4c600769c0 2024-04-24T06:41:31.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.561+0000 7f4c67000640 1 -- 172.21.15.5:0/2978007032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c601a6d70 con 0x7f4c600769c0 2024-04-24T06:41:31.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.562+0000 7f4c67000640 1 -- 172.21.15.5:0/2978007032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c60110e00 con 0x7f4c600769c0 2024-04-24T06:41:31.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.565+0000 7f4c5cc00640 1 -- 172.21.15.5:0/2978007032 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f4c50010ac0 con 0x7f4c600769c0 2024-04-24T06:41:31.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.566+0000 7f4c5cc00640 1 --2- 172.21.15.5:0/2978007032 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f4c440a3bb0 0x7f4c440a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:31.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.566+0000 7f4c5cc00640 1 -- 172.21.15.5:0/2978007032 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f4c500d0320 con 0x7f4c600769c0 2024-04-24T06:41:31.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.566+0000 7f4c5f400640 1 --2- 172.21.15.5:0/2978007032 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f4c440a3bb0 0x7f4c440a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:31.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.567+0000 7f4c5f400640 1 --2- 172.21.15.5:0/2978007032 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f4c440a3bb0 0x7f4c440a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c0314e0 tx=0x7f4c4c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:31.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.572+0000 7f4c5cc00640 1 -- 172.21.15.5:0/2978007032 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4c5009cab0 con 0x7f4c600769c0 2024-04-24T06:41:31.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.781+0000 7f4c67000640 1 -- 172.21.15.5:0/2978007032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f4c60062c00 con 0x7f4c600769c0 2024-04-24T06:41:31.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.782+0000 7f4c5cc00640 1 -- 172.21.15.5:0/2978007032 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7f4c500a0db0 con 0x7f4c600769c0 2024-04-24T06:41:31.784 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:41:31.784 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-04-24T06:41:31.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:41:31.784 INFO:teuthology.orchestra.run.smithi005.stdout: "poolnum": 1, 2024-04-24T06:41:31.784 INFO:teuthology.orchestra.run.smithi005.stdout: "poolname": ".mgr" 2024-04-24T06:41:31.784 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:41:31.784 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-04-24T06:41:31.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.785+0000 7f4c67000640 1 -- 172.21.15.5:0/2978007032 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f4c440a3bb0 msgr2=0x7f4c440a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:31.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.785+0000 7f4c67000640 1 --2- 172.21.15.5:0/2978007032 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f4c440a3bb0 0x7f4c440a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c0314e0 tx=0x7f4c4c000bc0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.785+0000 7f4c67000640 1 -- 172.21.15.5:0/2978007032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c600769c0 msgr2=0x7f4c6019e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:31.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.785+0000 7f4c67000640 1 --2- 172.21.15.5:0/2978007032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c600769c0 0x7f4c6019e940 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4c5000e9f0 tx=0x7f4c5000eeb0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.785+0000 7f4c67000640 1 -- 172.21.15.5:0/2978007032 shutdown_connections 2024-04-24T06:41:31.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.785+0000 7f4c67000640 1 --2- 172.21.15.5:0/2978007032 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f4c440a3bb0 0x7f4c440a6070 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.785+0000 7f4c67000640 1 --2- 172.21.15.5:0/2978007032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c600785a0 0x7f4c601a5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.785+0000 7f4c67000640 1 --2- 172.21.15.5:0/2978007032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c60077c00 0x7f4c6019ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.785+0000 7f4c67000640 1 --2- 172.21.15.5:0/2978007032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c600769c0 0x7f4c6019e940 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:31.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.785+0000 7f4c67000640 1 -- 172.21.15.5:0/2978007032 >> 172.21.15.5:0/2978007032 conn(0x7f4c600ff850 msgr2=0x7f4c60100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:31.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.785+0000 7f4c67000640 1 -- 172.21.15.5:0/2978007032 shutdown_connections 2024-04-24T06:41:31.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:31.785+0000 7f4c67000640 1 -- 172.21.15.5:0/2978007032 wait complete. 2024-04-24T06:41:31.807 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-24T06:41:31.807 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-24T06:41:32.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.016+0000 7f2f43000640 1 -- 172.21.15.5:0/2820479975 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f3c105450 msgr2=0x7f2f3c1058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:32.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.016+0000 7f2f43000640 1 --2- 172.21.15.5:0/2820479975 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f3c105450 0x7f2f3c1058d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2f28009860 tx=0x7f2f2802a340 comp rx=0 tx=0).stop 2024-04-24T06:41:32.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.016+0000 7f2f43000640 1 -- 172.21.15.5:0/2820479975 shutdown_connections 2024-04-24T06:41:32.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.016+0000 7f2f43000640 1 --2- 172.21.15.5:0/2820479975 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f3c105e10 0x7f2f3c10c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:32.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.016+0000 7f2f43000640 1 --2- 172.21.15.5:0/2820479975 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f3c105450 0x7f2f3c1058d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:32.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.016+0000 7f2f43000640 1 --2- 172.21.15.5:0/2820479975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f3c104260 0x7f2f3c104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:32.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.016+0000 7f2f43000640 1 -- 172.21.15.5:0/2820479975 >> 172.21.15.5:0/2820479975 conn(0x7f2f3c0ff7f0 msgr2=0x7f2f3c101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:32.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.016+0000 7f2f43000640 1 -- 172.21.15.5:0/2820479975 shutdown_connections 2024-04-24T06:41:32.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.016+0000 7f2f43000640 1 -- 172.21.15.5:0/2820479975 wait complete. 2024-04-24T06:41:32.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.017+0000 7f2f43000640 1 Processor -- start 2024-04-24T06:41:32.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.017+0000 7f2f43000640 1 -- start start 2024-04-24T06:41:32.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.018+0000 7f2f43000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f3c104260 0x7f2f3c19a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:32.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.018+0000 7f2f43000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f3c105450 0x7f2f3c19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.018+0000 7f2f43000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f3c105e10 0x7f2f3c1a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.018+0000 7f2f43000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3c10fef0 con 0x7f2f3c105450 2024-04-24T06:41:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.018+0000 7f2f43000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3c10fd70 con 0x7f2f3c105e10 2024-04-24T06:41:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.018+0000 7f2f43000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3c110070 con 0x7f2f3c104260 2024-04-24T06:41:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.018+0000 7f2f3be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f3c104260 0x7f2f3c19a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.018+0000 7f2f3b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f3c105450 0x7f2f3c19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.018+0000 7f2f3be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f3c104260 0x7f2f3c19a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:59686/0 (socket says 172.21.15.5:59686) 2024-04-24T06:41:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.018+0000 7f2f3be00640 1 -- 172.21.15.5:0/2597750163 learned_addr learned my addr 172.21.15.5:0/2597750163 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:41:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.018+0000 7f2f41000640 1 --2- 172.21.15.5:0/2597750163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f3c105e10 0x7f2f3c1a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.018+0000 7f2f3be00640 1 -- 172.21.15.5:0/2597750163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f3c105e10 msgr2=0x7f2f3c1a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.018+0000 7f2f3be00640 1 --2- 172.21.15.5:0/2597750163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f3c105e10 0x7f2f3c1a1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.018+0000 7f2f3be00640 1 -- 172.21.15.5:0/2597750163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f3c105450 msgr2=0x7f2f3c19aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.018+0000 7f2f3be00640 1 --2- 172.21.15.5:0/2597750163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f3c105450 0x7f2f3c19aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.018+0000 7f2f3be00640 1 -- 172.21.15.5:0/2597750163 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f3c1a2150 con 0x7f2f3c104260 2024-04-24T06:41:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.018+0000 7f2f3b400640 1 --2- 172.21.15.5:0/2597750163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f3c105450 0x7f2f3c19aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:41:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.019+0000 7f2f3be00640 1 --2- 172.21.15.5:0/2597750163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f3c104260 0x7f2f3c19a530 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00e9f0 tx=0x7f2f2c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.019+0000 7f2f38c00640 1 -- 172.21.15.5:0/2597750163 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c00ccc0 con 0x7f2f3c104260 2024-04-24T06:41:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.019+0000 7f2f43000640 1 -- 172.21.15.5:0/2597750163 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f3c1a2440 con 0x7f2f3c104260 2024-04-24T06:41:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.019+0000 7f2f38c00640 1 -- 172.21.15.5:0/2597750163 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f2c0035f0 con 0x7f2f3c104260 2024-04-24T06:41:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.019+0000 7f2f38c00640 1 -- 172.21.15.5:0/2597750163 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c00ce60 con 0x7f2f3c104260 2024-04-24T06:41:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.019+0000 7f2f43000640 1 -- 172.21.15.5:0/2597750163 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f3c1a29b0 con 0x7f2f3c104260 2024-04-24T06:41:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.021+0000 7f2f43000640 1 -- 172.21.15.5:0/2597750163 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f3c10b580 con 0x7f2f3c104260 2024-04-24T06:41:32.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.022+0000 7f2f38c00640 1 -- 172.21.15.5:0/2597750163 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f2f2c0037a0 con 0x7f2f3c104260 2024-04-24T06:41:32.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.024+0000 7f2f38c00640 1 --2- 172.21.15.5:0/2597750163 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f2f200a3c30 0x7f2f200a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:32.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.024+0000 7f2f38c00640 1 -- 172.21.15.5:0/2597750163 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f2f2c0d01b0 con 0x7f2f3c104260 2024-04-24T06:41:32.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.024+0000 7f2f3b400640 1 --2- 172.21.15.5:0/2597750163 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f2f200a3c30 0x7f2f200a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:32.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.025+0000 7f2f3b400640 1 --2- 172.21.15.5:0/2597750163 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f2f200a3c30 0x7f2f200a60f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2f28009830 tx=0x7f2f28030550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:32.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.029+0000 7f2f38c00640 1 -- 172.21.15.5:0/2597750163 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2f2c09cf90 con 0x7f2f3c104260 2024-04-24T06:41:32.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:32.242+0000 7f2f43000640 1 -- 172.21.15.5:0/2597750163 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f2f3c02d620 con 0x7f2f3c104260 2024-04-24T06:41:32.485 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:32 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:32.485 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:32 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:32.485 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:32 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:41:32.486 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:32 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:41:32.486 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:32 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:32.486 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:32 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:32.486 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:32 smithi137 ceph-mon[33017]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:32.486 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:32 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/2857086122' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T06:41:32.486 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:32 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:41:32.486 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:32 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/2978007032' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-24T06:41:32.488 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:32 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:32.488 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:32 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:32.488 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:32 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:41:32.488 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:32 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:41:32.488 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:32 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:32.489 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:32 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:32.489 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:32 smithi037 ceph-mon[32864]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:32.489 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:32 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/2857086122' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T06:41:32.489 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:32 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:41:32.489 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:32 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/2978007032' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-24T06:41:32.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:32 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:32.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:32 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:32.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:32 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:41:32.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:32 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:41:32.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:32 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:32.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:32 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:32.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:32 smithi005 ceph-mon[27798]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:32.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:32 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/2857086122' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T06:41:32.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:32 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:41:32.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:32 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/2978007032' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-24T06:41:33.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.248+0000 7f2f38c00640 1 -- 172.21.15.5:0/2597750163 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7f2f2c085e20 con 0x7f2f3c104260 2024-04-24T06:41:33.250 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' created 2024-04-24T06:41:33.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.257+0000 7f2f43000640 1 -- 172.21.15.5:0/2597750163 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f2f200a3c30 msgr2=0x7f2f200a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:33.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.257+0000 7f2f43000640 1 --2- 172.21.15.5:0/2597750163 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f2f200a3c30 0x7f2f200a60f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2f28009830 tx=0x7f2f28030550 comp rx=0 tx=0).stop 2024-04-24T06:41:33.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.257+0000 7f2f43000640 1 -- 172.21.15.5:0/2597750163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f3c104260 msgr2=0x7f2f3c19a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:33.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.257+0000 7f2f43000640 1 --2- 172.21.15.5:0/2597750163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f3c104260 0x7f2f3c19a530 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00e9f0 tx=0x7f2f2c00eeb0 comp rx=0 tx=0).stop 2024-04-24T06:41:33.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.258+0000 7f2f43000640 1 -- 172.21.15.5:0/2597750163 shutdown_connections 2024-04-24T06:41:33.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.258+0000 7f2f43000640 1 --2- 172.21.15.5:0/2597750163 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f2f200a3c30 0x7f2f200a60f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:33.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.258+0000 7f2f43000640 1 --2- 172.21.15.5:0/2597750163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f3c105e10 0x7f2f3c1a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:33.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.258+0000 7f2f43000640 1 --2- 172.21.15.5:0/2597750163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f3c105450 0x7f2f3c19aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:33.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.258+0000 7f2f43000640 1 --2- 172.21.15.5:0/2597750163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f3c104260 0x7f2f3c19a530 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:33.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.258+0000 7f2f43000640 1 -- 172.21.15.5:0/2597750163 >> 172.21.15.5:0/2597750163 conn(0x7f2f3c0ff7f0 msgr2=0x7f2f3c108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:33.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.258+0000 7f2f43000640 1 -- 172.21.15.5:0/2597750163 shutdown_connections 2024-04-24T06:41:33.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.259+0000 7f2f43000640 1 -- 172.21.15.5:0/2597750163 wait complete. 2024-04-24T06:41:33.283 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-24T06:41:33.344 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:33 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/2597750163' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-24T06:41:33.344 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:33 smithi005 ceph-mon[27798]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-24T06:41:33.344 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:33 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:33.344 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:33 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:33.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.489+0000 7f56e8200640 1 -- 172.21.15.5:0/2608735570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56e0103240 msgr2=0x7f56e01056b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:33.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.489+0000 7f56e8200640 1 --2- 172.21.15.5:0/2608735570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56e0103240 0x7f56e01056b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f56cc009900 tx=0x7f56cc02a410 comp rx=0 tx=0).stop 2024-04-24T06:41:33.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.489+0000 7f56e8200640 1 -- 172.21.15.5:0/2608735570 shutdown_connections 2024-04-24T06:41:33.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.489+0000 7f56e8200640 1 --2- 172.21.15.5:0/2608735570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56e0105bf0 0x7f56e0106070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:33.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.489+0000 7f56e8200640 1 --2- 172.21.15.5:0/2608735570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56e0103240 0x7f56e01056b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:33.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.489+0000 7f56e8200640 1 --2- 172.21.15.5:0/2608735570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56e01008d0 0x7f56e0102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:33.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.489+0000 7f56e8200640 1 -- 172.21.15.5:0/2608735570 >> 172.21.15.5:0/2608735570 conn(0x7f56e00fc4d0 msgr2=0x7f56e00fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:33.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.489+0000 7f56e8200640 1 -- 172.21.15.5:0/2608735570 shutdown_connections 2024-04-24T06:41:33.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.489+0000 7f56e8200640 1 -- 172.21.15.5:0/2608735570 wait complete. 2024-04-24T06:41:33.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.490+0000 7f56e8200640 1 Processor -- start 2024-04-24T06:41:33.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.490+0000 7f56e8200640 1 -- start start 2024-04-24T06:41:33.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.490+0000 7f56e8200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56e01008d0 0x7f56e019ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:33.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.490+0000 7f56e8200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56e0103240 0x7f56e019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:33.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.490+0000 7f56e8200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56e0105bf0 0x7f56e01a5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:33.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.491+0000 7f56e8200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f56e0114320 con 0x7f56e0105bf0 2024-04-24T06:41:33.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.491+0000 7f56e8200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f56e01141a0 con 0x7f56e01008d0 2024-04-24T06:41:33.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.491+0000 7f56e8200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f56e01144a0 con 0x7f56e0103240 2024-04-24T06:41:33.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.491+0000 7f56e4e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56e0103240 0x7f56e019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:33.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.491+0000 7f56e4e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56e0103240 0x7f56e019ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:59700/0 (socket says 172.21.15.5:59700) 2024-04-24T06:41:33.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.491+0000 7f56e4e00640 1 -- 172.21.15.5:0/2964135213 learned_addr learned my addr 172.21.15.5:0/2964135213 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:41:33.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.491+0000 7f56e5800640 1 --2- 172.21.15.5:0/2964135213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56e01008d0 0x7f56e019ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:33.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.491+0000 7f56e6200640 1 --2- 172.21.15.5:0/2964135213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56e0105bf0 0x7f56e01a5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:33.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.491+0000 7f56e5800640 1 -- 172.21.15.5:0/2964135213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56e0103240 msgr2=0x7f56e019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:33.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.491+0000 7f56e5800640 1 --2- 172.21.15.5:0/2964135213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56e0103240 0x7f56e019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:33.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.491+0000 7f56e5800640 1 -- 172.21.15.5:0/2964135213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56e0105bf0 msgr2=0x7f56e01a5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:33.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.491+0000 7f56e5800640 1 --2- 172.21.15.5:0/2964135213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56e0105bf0 0x7f56e01a5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:33.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.491+0000 7f56e5800640 1 -- 172.21.15.5:0/2964135213 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56e01a65b0 con 0x7f56e01008d0 2024-04-24T06:41:33.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.491+0000 7f56e4e00640 1 --2- 172.21.15.5:0/2964135213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56e0103240 0x7f56e019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:41:33.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.492+0000 7f56e5800640 1 --2- 172.21.15.5:0/2964135213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56e01008d0 0x7f56e019ea30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f56d000e9f0 tx=0x7f56d000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:33.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.492+0000 7f56de000640 1 -- 172.21.15.5:0/2964135213 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d000ccc0 con 0x7f56e01008d0 2024-04-24T06:41:33.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.492+0000 7f56e8200640 1 -- 172.21.15.5:0/2964135213 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56e01a68a0 con 0x7f56e01008d0 2024-04-24T06:41:33.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.492+0000 7f56de000640 1 -- 172.21.15.5:0/2964135213 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56d0010910 con 0x7f56e01008d0 2024-04-24T06:41:33.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.492+0000 7f56de000640 1 -- 172.21.15.5:0/2964135213 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d0018960 con 0x7f56e01008d0 2024-04-24T06:41:33.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.492+0000 7f56e8200640 1 -- 172.21.15.5:0/2964135213 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56e01a6e40 con 0x7f56e01008d0 2024-04-24T06:41:33.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.494+0000 7f56e8200640 1 -- 172.21.15.5:0/2964135213 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56e0068620 con 0x7f56e01008d0 2024-04-24T06:41:33.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.497+0000 7f56de000640 1 -- 172.21.15.5:0/2964135213 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f56d0010ac0 con 0x7f56e01008d0 2024-04-24T06:41:33.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.498+0000 7f56de000640 1 --2- 172.21.15.5:0/2964135213 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f56c40a3bb0 0x7f56c40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:33.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.498+0000 7f56de000640 1 -- 172.21.15.5:0/2964135213 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7497+0+0 (secure 0 0 0) 0x7f56d00d0490 con 0x7f56e01008d0 2024-04-24T06:41:33.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.498+0000 7f56e4e00640 1 --2- 172.21.15.5:0/2964135213 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f56c40a3bb0 0x7f56c40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:33.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.499+0000 7f56e4e00640 1 --2- 172.21.15.5:0/2964135213 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f56c40a3bb0 0x7f56c40a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f56e019fea0 tx=0x7f56cc0315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:33.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.502+0000 7f56de000640 1 -- 172.21.15.5:0/2964135213 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f56d009cab0 con 0x7f56e01008d0 2024-04-24T06:41:33.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:33 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/2597750163' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-24T06:41:33.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:33 smithi037 ceph-mon[32864]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-24T06:41:33.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:33 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:33.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:33 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:33.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:33 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/2597750163' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-24T06:41:33.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:33 smithi137 ceph-mon[33017]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-24T06:41:33.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:33 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:33.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:33 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:33.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:33.727+0000 7f56e8200640 1 -- 172.21.15.5:0/2964135213 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f56e00fdfc0 con 0x7f56e01008d0 2024-04-24T06:41:34.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.260+0000 7f56de000640 1 -- 172.21.15.5:0/2964135213 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) ==== 170+0+0 (secure 0 0 0) 0x7f56e00fdfc0 con 0x7f56e01008d0 2024-04-24T06:41:34.263 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' created 2024-04-24T06:41:34.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.269+0000 7f56e8200640 1 -- 172.21.15.5:0/2964135213 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f56c40a3bb0 msgr2=0x7f56c40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:34.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.269+0000 7f56e8200640 1 --2- 172.21.15.5:0/2964135213 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f56c40a3bb0 0x7f56c40a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f56e019fea0 tx=0x7f56cc0315b0 comp rx=0 tx=0).stop 2024-04-24T06:41:34.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.269+0000 7f56e8200640 1 -- 172.21.15.5:0/2964135213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56e01008d0 msgr2=0x7f56e019ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:34.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.269+0000 7f56e8200640 1 --2- 172.21.15.5:0/2964135213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56e01008d0 0x7f56e019ea30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f56d000e9f0 tx=0x7f56d000eeb0 comp rx=0 tx=0).stop 2024-04-24T06:41:34.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.272+0000 7f56e8200640 1 -- 172.21.15.5:0/2964135213 shutdown_connections 2024-04-24T06:41:34.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.272+0000 7f56e8200640 1 --2- 172.21.15.5:0/2964135213 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f56c40a3bb0 0x7f56c40a6070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:34.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.272+0000 7f56e8200640 1 --2- 172.21.15.5:0/2964135213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56e0105bf0 0x7f56e01a5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:34.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.272+0000 7f56e8200640 1 --2- 172.21.15.5:0/2964135213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56e0103240 0x7f56e019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:34.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.272+0000 7f56e8200640 1 --2- 172.21.15.5:0/2964135213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56e01008d0 0x7f56e019ea30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:34.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.272+0000 7f56e8200640 1 -- 172.21.15.5:0/2964135213 >> 172.21.15.5:0/2964135213 conn(0x7f56e00fc4d0 msgr2=0x7f56e0101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:34.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.272+0000 7f56e8200640 1 -- 172.21.15.5:0/2964135213 shutdown_connections 2024-04-24T06:41:34.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.272+0000 7f56e8200640 1 -- 172.21.15.5:0/2964135213 wait complete. 2024-04-24T06:41:34.305 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-24T06:41:34.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.492+0000 7fa71a600640 1 -- 172.21.15.5:0/2010757169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa714105440 msgr2=0x7fa7141058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:34.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.492+0000 7fa71a600640 1 --2- 172.21.15.5:0/2010757169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa714105440 0x7fa7141058c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa700009900 tx=0x7fa70002a410 comp rx=0 tx=0).stop 2024-04-24T06:41:34.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.492+0000 7fa71a600640 1 -- 172.21.15.5:0/2010757169 shutdown_connections 2024-04-24T06:41:34.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.492+0000 7fa71a600640 1 --2- 172.21.15.5:0/2010757169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa714105e00 0x7fa71410c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:34.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.492+0000 7fa71a600640 1 --2- 172.21.15.5:0/2010757169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa714105440 0x7fa7141058c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:34.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.492+0000 7fa71a600640 1 --2- 172.21.15.5:0/2010757169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa714104250 0x7fa714104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:34.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.492+0000 7fa71a600640 1 -- 172.21.15.5:0/2010757169 >> 172.21.15.5:0/2010757169 conn(0x7fa7140ff7c0 msgr2=0x7fa714101c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:34.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.492+0000 7fa71a600640 1 -- 172.21.15.5:0/2010757169 shutdown_connections 2024-04-24T06:41:34.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.492+0000 7fa71a600640 1 -- 172.21.15.5:0/2010757169 wait complete. 2024-04-24T06:41:34.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.493+0000 7fa71a600640 1 Processor -- start 2024-04-24T06:41:34.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.493+0000 7fa71a600640 1 -- start start 2024-04-24T06:41:34.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.494+0000 7fa71a600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa714104250 0x7fa71419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:34.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.494+0000 7fa71a600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa714105440 0x7fa71419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:34.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.494+0000 7fa71a600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa714105e00 0x7fa7141a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:34.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.494+0000 7fa71a600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa71410ff30 con 0x7fa714105440 2024-04-24T06:41:34.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.494+0000 7fa71a600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa71410fdb0 con 0x7fa714104250 2024-04-24T06:41:34.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.494+0000 7fa71a600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa7141100b0 con 0x7fa714105e00 2024-04-24T06:41:34.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.494+0000 7fa713400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa714104250 0x7fa71419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:34.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.494+0000 7fa713e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa714105e00 0x7fa7141a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:34.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.494+0000 7fa713e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa714105e00 0x7fa7141a1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:42968/0 (socket says 172.21.15.5:42968) 2024-04-24T06:41:34.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.494+0000 7fa713e00640 1 -- 172.21.15.5:0/2585866774 learned_addr learned my addr 172.21.15.5:0/2585866774 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:41:34.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.494+0000 7fa713400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa714104250 0x7fa71419a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:35630/0 (socket says 172.21.15.5:35630) 2024-04-24T06:41:34.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.494+0000 7fa712a00640 1 --2- 172.21.15.5:0/2585866774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa714105440 0x7fa71419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:34.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.494+0000 7fa713e00640 1 -- 172.21.15.5:0/2585866774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa714104250 msgr2=0x7fa71419a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:34.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.494+0000 7fa713e00640 1 --2- 172.21.15.5:0/2585866774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa714104250 0x7fa71419a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:34.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.494+0000 7fa713e00640 1 -- 172.21.15.5:0/2585866774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa714105440 msgr2=0x7fa71419aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:34.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.494+0000 7fa713e00640 1 --2- 172.21.15.5:0/2585866774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa714105440 0x7fa71419aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:34.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.494+0000 7fa713e00640 1 -- 172.21.15.5:0/2585866774 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7141a2170 con 0x7fa714105e00 2024-04-24T06:41:34.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.494+0000 7fa713400640 1 --2- 172.21.15.5:0/2585866774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa714104250 0x7fa71419a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:41:34.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.495+0000 7fa713e00640 1 --2- 172.21.15.5:0/2585866774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa714105e00 0x7fa7141a1b00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa70c00eb10 tx=0x7fa70c00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:34.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.495+0000 7fa70be00640 1 -- 172.21.15.5:0/2585866774 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c004b20 con 0x7fa714105e00 2024-04-24T06:41:34.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.495+0000 7fa70be00640 1 -- 172.21.15.5:0/2585866774 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa70c004cc0 con 0x7fa714105e00 2024-04-24T06:41:34.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.495+0000 7fa70be00640 1 -- 172.21.15.5:0/2585866774 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c010760 con 0x7fa714105e00 2024-04-24T06:41:34.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.495+0000 7fa71a600640 1 -- 172.21.15.5:0/2585866774 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7141a2460 con 0x7fa714105e00 2024-04-24T06:41:34.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.495+0000 7fa71a600640 1 -- 172.21.15.5:0/2585866774 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7141a2930 con 0x7fa714105e00 2024-04-24T06:41:34.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.496+0000 7fa71a600640 1 -- 172.21.15.5:0/2585866774 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa71410b510 con 0x7fa714105e00 2024-04-24T06:41:34.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.498+0000 7fa70be00640 1 -- 172.21.15.5:0/2585866774 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fa70c005020 con 0x7fa714105e00 2024-04-24T06:41:34.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.499+0000 7fa70be00640 1 --2- 172.21.15.5:0/2585866774 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fa6f80a3c30 0x7fa6f80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:34.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.500+0000 7fa70be00640 1 -- 172.21.15.5:0/2585866774 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7fa70c0d07e0 con 0x7fa714105e00 2024-04-24T06:41:34.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.500+0000 7fa713400640 1 --2- 172.21.15.5:0/2585866774 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fa6f80a3c30 0x7fa6f80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:34.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.500+0000 7fa713400640 1 --2- 172.21.15.5:0/2585866774 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fa6f80a3c30 0x7fa6f80a60f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa7141052a0 tx=0x7fa704005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:34.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.503+0000 7fa70be00640 1 -- 172.21.15.5:0/2585866774 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa70c09d2d0 con 0x7fa714105e00 2024-04-24T06:41:34.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:34 smithi037 ceph-mon[32864]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-24T06:41:34.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:34 smithi037 ceph-mon[32864]: osdmap e83: 12 total, 12 up, 12 in 2024-04-24T06:41:34.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:34 smithi037 ceph-mon[32864]: pgmap v309: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:34.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:34 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:34.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:34 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:34.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:34 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/2964135213' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-24T06:41:34.521 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:34 smithi037 ceph-mon[32864]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-24T06:41:34.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:34 smithi137 ceph-mon[33017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-24T06:41:34.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:34 smithi137 ceph-mon[33017]: osdmap e83: 12 total, 12 up, 12 in 2024-04-24T06:41:34.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:34 smithi137 ceph-mon[33017]: pgmap v309: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:34.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:34 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:34.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:34 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:34.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:34 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/2964135213' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-24T06:41:34.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:34 smithi137 ceph-mon[33017]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-24T06:41:34.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:34 smithi005 ceph-mon[27798]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-24T06:41:34.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:34 smithi005 ceph-mon[27798]: osdmap e83: 12 total, 12 up, 12 in 2024-04-24T06:41:34.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:34 smithi005 ceph-mon[27798]: pgmap v309: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:34.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:34 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:34.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:34 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:34.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:34 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/2964135213' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-24T06:41:34.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:34 smithi005 ceph-mon[27798]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-24T06:41:34.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:34.762+0000 7fa71a600640 1 -- 172.21.15.5:0/2585866774 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa71402d620 con 0x7fa714105e00 2024-04-24T06:41:35.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.439+0000 7fa70be00640 1 -- 172.21.15.5:0/2585866774 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fa70c09cbd0 con 0x7fa714105e00 2024-04-24T06:41:35.440 INFO:teuthology.orchestra.run.smithi005.stderr:new fs with metadata pool 2 and data pool 3 2024-04-24T06:41:35.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.444+0000 7fa71a600640 1 -- 172.21.15.5:0/2585866774 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fa6f80a3c30 msgr2=0x7fa6f80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:35.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.444+0000 7fa71a600640 1 --2- 172.21.15.5:0/2585866774 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fa6f80a3c30 0x7fa6f80a60f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa7141052a0 tx=0x7fa704005f90 comp rx=0 tx=0).stop 2024-04-24T06:41:35.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.444+0000 7fa71a600640 1 -- 172.21.15.5:0/2585866774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa714105e00 msgr2=0x7fa7141a1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:35.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.444+0000 7fa71a600640 1 --2- 172.21.15.5:0/2585866774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa714105e00 0x7fa7141a1b00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa70c00eb10 tx=0x7fa70c00c2d0 comp rx=0 tx=0).stop 2024-04-24T06:41:35.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.444+0000 7fa71a600640 1 -- 172.21.15.5:0/2585866774 shutdown_connections 2024-04-24T06:41:35.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.444+0000 7fa71a600640 1 --2- 172.21.15.5:0/2585866774 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fa6f80a3c30 0x7fa6f80a60f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:35.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.444+0000 7fa71a600640 1 --2- 172.21.15.5:0/2585866774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa714105e00 0x7fa7141a1b00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:35.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.445+0000 7fa71a600640 1 --2- 172.21.15.5:0/2585866774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa714105440 0x7fa71419aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:35.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.445+0000 7fa71a600640 1 --2- 172.21.15.5:0/2585866774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa714104250 0x7fa71419a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:35.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.445+0000 7fa71a600640 1 -- 172.21.15.5:0/2585866774 >> 172.21.15.5:0/2585866774 conn(0x7fa7140ff7c0 msgr2=0x7fa7141086e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:35.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.445+0000 7fa71a600640 1 -- 172.21.15.5:0/2585866774 shutdown_connections 2024-04-24T06:41:35.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.445+0000 7fa71a600640 1 -- 172.21.15.5:0/2585866774 wait complete. 2024-04-24T06:41:35.471 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:41:35.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:35 smithi037 ceph-mon[32864]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T06:41:35.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:35 smithi037 ceph-mon[32864]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-24T06:41:35.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:35 smithi037 ceph-mon[32864]: osdmap e84: 12 total, 12 up, 12 in 2024-04-24T06:41:35.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:35 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:35.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:35 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:35.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:35 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:41:35.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:35 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:41:35.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:35 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:35.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:35 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/2585866774' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-24T06:41:35.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:35 smithi037 ceph-mon[32864]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-24T06:41:35.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:35 smithi037 ceph-mon[32864]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-24T06:41:35.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:35 smithi005 ceph-mon[27798]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T06:41:35.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:35 smithi005 ceph-mon[27798]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-24T06:41:35.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:35 smithi005 ceph-mon[27798]: osdmap e84: 12 total, 12 up, 12 in 2024-04-24T06:41:35.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:35 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:35.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:35 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:35.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:35 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:41:35.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:35 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:41:35.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:35 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:35.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:35 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/2585866774' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-24T06:41:35.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:35 smithi005 ceph-mon[27798]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-24T06:41:35.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:35 smithi005 ceph-mon[27798]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-24T06:41:35.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.675+0000 7f4752000640 1 -- 172.21.15.5:0/1996164527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f474c1085a0 msgr2=0x7f474c1089c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:35.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.675+0000 7f4752000640 1 --2- 172.21.15.5:0/1996164527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f474c1085a0 0x7f474c1089c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4740009900 tx=0x7f474002a410 comp rx=0 tx=0).stop 2024-04-24T06:41:35.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.675+0000 7f4752000640 1 -- 172.21.15.5:0/1996164527 shutdown_connections 2024-04-24T06:41:35.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.675+0000 7f4752000640 1 --2- 172.21.15.5:0/1996164527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f474c10a100 0x7f474c110990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:35.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.675+0000 7f4752000640 1 --2- 172.21.15.5:0/1996164527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f474c109740 0x7f474c109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:35.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.675+0000 7f4752000640 1 --2- 172.21.15.5:0/1996164527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f474c1085a0 0x7f474c1089c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:35.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.675+0000 7f4752000640 1 -- 172.21.15.5:0/1996164527 >> 172.21.15.5:0/1996164527 conn(0x7f474c0774f0 msgr2=0x7f474c079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:35.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.675+0000 7f4752000640 1 -- 172.21.15.5:0/1996164527 shutdown_connections 2024-04-24T06:41:35.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.675+0000 7f4752000640 1 -- 172.21.15.5:0/1996164527 wait complete. 2024-04-24T06:41:35.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.677+0000 7f4752000640 1 Processor -- start 2024-04-24T06:41:35.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.677+0000 7f4752000640 1 -- start start 2024-04-24T06:41:35.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.677+0000 7f4752000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f474c1085a0 0x7f474c19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:35.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.677+0000 7f4752000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f474c109740 0x7f474c19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:35.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.677+0000 7f4752000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f474c10a100 0x7f474c1a5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:35.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.677+0000 7f4752000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f474c114230 con 0x7f474c109740 2024-04-24T06:41:35.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.677+0000 7f4752000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f474c1140b0 con 0x7f474c1085a0 2024-04-24T06:41:35.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.677+0000 7f4752000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f474c1143b0 con 0x7f474c10a100 2024-04-24T06:41:35.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.678+0000 7f474be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f474c109740 0x7f474c19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:35.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.678+0000 7f4750c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f474c1085a0 0x7f474c19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:35.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.678+0000 7f474be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f474c109740 0x7f474c19ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60572/0 (socket says 172.21.15.5:60572) 2024-04-24T06:41:35.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.678+0000 7f474be00640 1 -- 172.21.15.5:0/2291194249 learned_addr learned my addr 172.21.15.5:0/2291194249 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:41:35.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.678+0000 7f4751600640 1 --2- 172.21.15.5:0/2291194249 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f474c10a100 0x7f474c1a5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:35.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.678+0000 7f4750c00640 1 -- 172.21.15.5:0/2291194249 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f474c10a100 msgr2=0x7f474c1a5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:35.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.678+0000 7f4750c00640 1 --2- 172.21.15.5:0/2291194249 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f474c10a100 0x7f474c1a5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:35.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.678+0000 7f4750c00640 1 -- 172.21.15.5:0/2291194249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f474c109740 msgr2=0x7f474c19ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:35.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.678+0000 7f4750c00640 1 --2- 172.21.15.5:0/2291194249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f474c109740 0x7f474c19ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:35.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.678+0000 7f4750c00640 1 -- 172.21.15.5:0/2291194249 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f474c1a6440 con 0x7f474c1085a0 2024-04-24T06:41:35.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.678+0000 7f474be00640 1 --2- 172.21.15.5:0/2291194249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f474c109740 0x7f474c19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:41:35.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.678+0000 7f4750c00640 1 --2- 172.21.15.5:0/2291194249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f474c1085a0 0x7f474c19e820 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4740009230 tx=0x7f4740032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:35.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.679+0000 7f4749600640 1 -- 172.21.15.5:0/2291194249 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474003f030 con 0x7f474c1085a0 2024-04-24T06:41:35.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.679+0000 7f4749600640 1 -- 172.21.15.5:0/2291194249 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4740003ce0 con 0x7f474c1085a0 2024-04-24T06:41:35.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.679+0000 7f4749600640 1 -- 172.21.15.5:0/2291194249 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4740035070 con 0x7f474c1085a0 2024-04-24T06:41:35.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.679+0000 7f4752000640 1 -- 172.21.15.5:0/2291194249 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f474c1a66d0 con 0x7f474c1085a0 2024-04-24T06:41:35.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.679+0000 7f4752000640 1 -- 172.21.15.5:0/2291194249 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f474c1a6be0 con 0x7f474c1085a0 2024-04-24T06:41:35.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.680+0000 7f4752000640 1 -- 172.21.15.5:0/2291194249 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f474c10f870 con 0x7f474c1085a0 2024-04-24T06:41:35.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.683+0000 7f4749600640 1 -- 172.21.15.5:0/2291194249 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f4740042030 con 0x7f474c1085a0 2024-04-24T06:41:35.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.684+0000 7f4749600640 1 --2- 172.21.15.5:0/2291194249 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f47340a3c20 0x7f47340a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:35.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.684+0000 7f4749600640 1 -- 172.21.15.5:0/2291194249 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f47400f0b10 con 0x7f474c1085a0 2024-04-24T06:41:35.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.685+0000 7f474be00640 1 --2- 172.21.15.5:0/2291194249 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f47340a3c20 0x7f47340a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:35.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.686+0000 7f474be00640 1 --2- 172.21.15.5:0/2291194249 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f47340a3c20 0x7f47340a60e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f474c19fd30 tx=0x7f4738005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:35.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.686+0000 7f4749600640 1 -- 172.21.15.5:0/2291194249 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f47400bcf50 con 0x7f474c1085a0 2024-04-24T06:41:35.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:35 smithi137 ceph-mon[33017]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T06:41:35.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:35 smithi137 ceph-mon[33017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-24T06:41:35.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:35 smithi137 ceph-mon[33017]: osdmap e84: 12 total, 12 up, 12 in 2024-04-24T06:41:35.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:35 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:35.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:35 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:35.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:35 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:41:35.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:35 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:41:35.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:35 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:41:35.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:35 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/2585866774' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-24T06:41:35.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:35 smithi137 ceph-mon[33017]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-24T06:41:35.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:35 smithi137 ceph-mon[33017]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-24T06:41:35.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.894+0000 7f4752000640 1 -- 172.21.15.5:0/2291194249 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f474c062c00 con 0x7f474c1085a0 2024-04-24T06:41:35.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.896+0000 7f4749600640 1 -- 172.21.15.5:0/2291194249 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22101 (secure 0 0 0) 0x7f47400c1250 con 0x7f474c1085a0 2024-04-24T06:41:35.898 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:41:35.899 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":85,"fsid":"43e9bad0-0204-11ef-bc93-c7b262605968","created":"2024-04-24T06:32:26.340555+0000","modified":"2024-04-24T06:41:34.764544+0000","last_up_change":"2024-04-24T06:40:36.790934+0000","last_in_change":"2024-04-24T06:40:17.803257+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:36:27.320342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:41:32.254502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:41:33.739067+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c2e974c-745c-4dd5-8c3d-8e8da2eb1ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6803","nonce":2155858515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6805","nonce":2155858515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6809","nonce":2155858515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6807","nonce":2155858515}]},"public_addr":"172.21.15.5:6803/2155858515","cluster_addr":"172.21.15.5:6805/2155858515","heartbeat_back_addr":"172.21.15.5:6809/2155858515","heartbeat_front_addr":"172.21.15.5:6807/2155858515","state":["exists","up"]},{"osd":1,"uuid":"50467744-db20-4721-bf2d-1c5db938297b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6801","nonce":1302362650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6803","nonce":1302362650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6807","nonce":1302362650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6805","nonce":1302362650}]},"public_addr":"172.21.15.37:6801/1302362650","cluster_addr":"172.21.15.37:6803/1302362650","heartbeat_back_addr":"172.21.15.37:6807/1302362650","heartbeat_front_addr":"172.21.15.37:6805/1302362650","state":["exists","up"]},{"osd":2,"uuid":"0e190b15-d04c-4288-8d48-ecf6664392fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6801","nonce":2885027256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6803","nonce":2885027256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6807","nonce":2885027256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6805","nonce":2885027256}]},"public_addr":"172.21.15.137:6801/2885027256","cluster_addr":"172.21.15.137:6803/2885027256","heartbeat_back_addr":"172.21.15.137:6807/2885027256","heartbeat_front_addr":"172.21.15.137:6805/2885027256","state":["exists","up"]},{"osd":3,"uuid":"e05592ba-d96a-472a-9f63-09a535d04dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6811","nonce":3701747260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6813","nonce":3701747260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6817","nonce":3701747260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6815","nonce":3701747260}]},"public_addr":"172.21.15.5:6811/3701747260","cluster_addr":"172.21.15.5:6813/3701747260","heartbeat_back_addr":"172.21.15.5:6817/3701747260","heartbeat_front_addr":"172.21.15.5:6815/3701747260","state":["exists","up"]},{"osd":4,"uuid":"c3f36e51-7a3b-4dd9-af4b-a10558830c4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6809","nonce":759253844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6811","nonce":759253844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6815","nonce":759253844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6813","nonce":759253844}]},"public_addr":"172.21.15.37:6809/759253844","cluster_addr":"172.21.15.37:6811/759253844","heartbeat_back_addr":"172.21.15.37:6815/759253844","heartbeat_front_addr":"172.21.15.37:6813/759253844","state":["exists","up"]},{"osd":5,"uuid":"97a0500c-cbb8-4ff5-b476-8634b26b3821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6809","nonce":3771921560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6811","nonce":3771921560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6815","nonce":3771921560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6813","nonce":3771921560}]},"public_addr":"172.21.15.137:6809/3771921560","cluster_addr":"172.21.15.137:6811/3771921560","heartbeat_back_addr":"172.21.15.137:6815/3771921560","heartbeat_front_addr":"172.21.15.137:6813/3771921560","state":["exists","up"]},{"osd":6,"uuid":"33a33d62-bbb3-4ed9-834e-19ae93a97670","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6819","nonce":2064636084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6821","nonce":2064636084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6825","nonce":2064636084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6823","nonce":2064636084}]},"public_addr":"172.21.15.5:6819/2064636084","cluster_addr":"172.21.15.5:6821/2064636084","heartbeat_back_addr":"172.21.15.5:6825/2064636084","heartbeat_front_addr":"172.21.15.5:6823/2064636084","state":["exists","up"]},{"osd":7,"uuid":"c0f4e036-4717-4514-bb59-ab11fbaab063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6817","nonce":2952750659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6819","nonce":2952750659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6823","nonce":2952750659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6821","nonce":2952750659}]},"public_addr":"172.21.15.37:6817/2952750659","cluster_addr":"172.21.15.37:6819/2952750659","heartbeat_back_addr":"172.21.15.37:6823/2952750659","heartbeat_front_addr":"172.21.15.37:6821/2952750659","state":["exists","up"]},{"osd":8,"uuid":"5391fd87-44ea-440f-80da-c54d0327c04c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6817","nonce":29724945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6819","nonce":29724945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6823","nonce":29724945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6821","nonce":29724945}]},"public_addr":"172.21.15.137:6817/29724945","cluster_addr":"172.21.15.137:6819/29724945","heartbeat_back_addr":"172.21.15.137:6823/29724945","heartbeat_front_addr":"172.21.15.137:6821/29724945","state":["exists","up"]},{"osd":9,"uuid":"d94c9dbc-f43d-4237-ac7f-47a6a5ede1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6827","nonce":2257102107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6829","nonce":2257102107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6833","nonce":2257102107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6831","nonce":2257102107}]},"public_addr":"172.21.15.5:6827/2257102107","cluster_addr":"172.21.15.5:6829/2257102107","heartbeat_back_addr":"172.21.15.5:6833/2257102107","heartbeat_front_addr":"172.21.15.5:6831/2257102107","state":["exists","up"]},{"osd":10,"uuid":"e2455332-2a5e-44b1-85d3-8e47da399f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6825","nonce":2451570637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6827","nonce":2451570637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6831","nonce":2451570637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6829","nonce":2451570637}]},"public_addr":"172.21.15.37:6825/2451570637","cluster_addr":"172.21.15.37:6827/2451570637","heartbeat_back_addr":"172.21.15.37:6831/2451570637","heartbeat_front_addr":"172.21.15.37:6829/2451570637","state":["exists","up"]},{"osd":11,"uuid":"49d9396b-704a-4cef-8210-75d30c28e924","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6825","nonce":90600596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6827","nonce":90600596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6831","nonce":90600596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6829","nonce":90600596}]},"public_addr":"172.21.15.137:6825/90600596","cluster_addr":"172.21.15.137:6827/90600596","heartbeat_back_addr":"172.21.15.137:6831/90600596","heartbeat_front_addr":"172.21.15.137:6829/90600596","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:35:35.591619+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:00.568817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:24.195643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:56.451724+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:22.109293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:46.161749+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:19.278305+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:45.620256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:10.041165+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:43.813697+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:10.736026+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:35.384377+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/23307164":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6801/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:0/230785941":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/2940023597":"2024-04-25T06:33:07.040369+0000","172.21.15.5:6801/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3886613904":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/4199165437":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3459593499":"2024-04-25T06:33:41.161852+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:41:35.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.899+0000 7f4752000640 1 -- 172.21.15.5:0/2291194249 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f47340a3c20 msgr2=0x7f47340a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:35.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.899+0000 7f4752000640 1 --2- 172.21.15.5:0/2291194249 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f47340a3c20 0x7f47340a60e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f474c19fd30 tx=0x7f4738005f90 comp rx=0 tx=0).stop 2024-04-24T06:41:35.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.899+0000 7f4752000640 1 -- 172.21.15.5:0/2291194249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f474c1085a0 msgr2=0x7f474c19e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:35.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.899+0000 7f4752000640 1 --2- 172.21.15.5:0/2291194249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f474c1085a0 0x7f474c19e820 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4740009230 tx=0x7f4740032ce0 comp rx=0 tx=0).stop 2024-04-24T06:41:35.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.899+0000 7f4752000640 1 -- 172.21.15.5:0/2291194249 shutdown_connections 2024-04-24T06:41:35.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.899+0000 7f4752000640 1 --2- 172.21.15.5:0/2291194249 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f47340a3c20 0x7f47340a60e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:35.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.899+0000 7f4752000640 1 --2- 172.21.15.5:0/2291194249 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f474c10a100 0x7f474c1a5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:35.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.899+0000 7f4752000640 1 --2- 172.21.15.5:0/2291194249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f474c109740 0x7f474c19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:35.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.899+0000 7f4752000640 1 --2- 172.21.15.5:0/2291194249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f474c1085a0 0x7f474c19e820 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:35.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.899+0000 7f4752000640 1 -- 172.21.15.5:0/2291194249 >> 172.21.15.5:0/2291194249 conn(0x7f474c0774f0 msgr2=0x7f474c10ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:35.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.900+0000 7f4752000640 1 -- 172.21.15.5:0/2291194249 shutdown_connections 2024-04-24T06:41:35.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:35.900+0000 7f4752000640 1 -- 172.21.15.5:0/2291194249 wait complete. 2024-04-24T06:41:35.921 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:41:36.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.128+0000 7fbbcae00640 1 -- 172.21.15.5:0/446006668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbbc4109740 msgr2=0x7fbbc4109bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:36.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.128+0000 7fbbcae00640 1 --2- 172.21.15.5:0/446006668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbbc4109740 0x7fbbc4109bc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0009900 tx=0x7fbbb002a410 comp rx=0 tx=0).stop 2024-04-24T06:41:36.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.129+0000 7fbbcae00640 1 -- 172.21.15.5:0/446006668 shutdown_connections 2024-04-24T06:41:36.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.129+0000 7fbbcae00640 1 --2- 172.21.15.5:0/446006668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbbc410a100 0x7fbbc4110990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:36.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.129+0000 7fbbcae00640 1 --2- 172.21.15.5:0/446006668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbbc4109740 0x7fbbc4109bc0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:36.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.129+0000 7fbbcae00640 1 --2- 172.21.15.5:0/446006668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbc41085a0 0x7fbbc41089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:36.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.129+0000 7fbbcae00640 1 -- 172.21.15.5:0/446006668 >> 172.21.15.5:0/446006668 conn(0x7fbbc40774f0 msgr2=0x7fbbc4079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:36.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.129+0000 7fbbcae00640 1 -- 172.21.15.5:0/446006668 shutdown_connections 2024-04-24T06:41:36.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.129+0000 7fbbcae00640 1 -- 172.21.15.5:0/446006668 wait complete. 2024-04-24T06:41:36.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.130+0000 7fbbcae00640 1 Processor -- start 2024-04-24T06:41:36.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.130+0000 7fbbcae00640 1 -- start start 2024-04-24T06:41:36.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.130+0000 7fbbcae00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbbc41085a0 0x7fbbc419e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:36.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.130+0000 7fbbcae00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbbc4109740 0x7fbbc419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:36.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.130+0000 7fbbcae00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbc410a100 0x7fbbc41a5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:36.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.130+0000 7fbbcae00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc41141e0 con 0x7fbbc410a100 2024-04-24T06:41:36.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.130+0000 7fbbcae00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc4114060 con 0x7fbbc41085a0 2024-04-24T06:41:36.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.130+0000 7fbbcae00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc4114360 con 0x7fbbc4109740 2024-04-24T06:41:36.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.130+0000 7fbbc9a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbbc41085a0 0x7fbbc419e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:36.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.130+0000 7fbbca400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbc410a100 0x7fbbc41a5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:36.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.130+0000 7fbbc9a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbbc41085a0 0x7fbbc419e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:35674/0 (socket says 172.21.15.5:35674) 2024-04-24T06:41:36.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.130+0000 7fbbc9a00640 1 -- 172.21.15.5:0/3358798806 learned_addr learned my addr 172.21.15.5:0/3358798806 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:41:36.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.130+0000 7fbbc9000640 1 --2- 172.21.15.5:0/3358798806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbbc4109740 0x7fbbc419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:36.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.131+0000 7fbbca400640 1 -- 172.21.15.5:0/3358798806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbbc4109740 msgr2=0x7fbbc419ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:36.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.131+0000 7fbbca400640 1 --2- 172.21.15.5:0/3358798806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbbc4109740 0x7fbbc419ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:36.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.131+0000 7fbbca400640 1 -- 172.21.15.5:0/3358798806 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbbc41085a0 msgr2=0x7fbbc419e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:36.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.131+0000 7fbbca400640 1 --2- 172.21.15.5:0/3358798806 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbbc41085a0 0x7fbbc419e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:36.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.131+0000 7fbbca400640 1 -- 172.21.15.5:0/3358798806 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbc41a6460 con 0x7fbbc410a100 2024-04-24T06:41:36.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.131+0000 7fbbc9a00640 1 --2- 172.21.15.5:0/3358798806 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbbc41085a0 0x7fbbc419e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:41:36.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.131+0000 7fbbca400640 1 --2- 172.21.15.5:0/3358798806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbc410a100 0x7fbbc41a5df0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc009fd0 tx=0x7fbbbc00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:36.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.132+0000 7fbbc2000640 1 -- 172.21.15.5:0/3358798806 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc00ce20 con 0x7fbbc410a100 2024-04-24T06:41:36.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.132+0000 7fbbcae00640 1 -- 172.21.15.5:0/3358798806 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbc41a6750 con 0x7fbbc410a100 2024-04-24T06:41:36.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.132+0000 7fbbc2000640 1 -- 172.21.15.5:0/3358798806 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbbc005020 con 0x7fbbc410a100 2024-04-24T06:41:36.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.132+0000 7fbbc2000640 1 -- 172.21.15.5:0/3358798806 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc010720 con 0x7fbbc410a100 2024-04-24T06:41:36.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.132+0000 7fbbcae00640 1 -- 172.21.15.5:0/3358798806 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbc41a6c70 con 0x7fbbc410a100 2024-04-24T06:41:36.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.133+0000 7fbbcae00640 1 -- 172.21.15.5:0/3358798806 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbc410f870 con 0x7fbbc410a100 2024-04-24T06:41:36.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.135+0000 7fbbc2000640 1 -- 172.21.15.5:0/3358798806 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fbbbc005540 con 0x7fbbc410a100 2024-04-24T06:41:36.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.137+0000 7fbbc2000640 1 --2- 172.21.15.5:0/3358798806 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fbbac0a3c20 0x7fbbac0a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:36.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.137+0000 7fbbc2000640 1 -- 172.21.15.5:0/3358798806 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fbbbc0d0970 con 0x7fbbc410a100 2024-04-24T06:41:36.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.137+0000 7fbbc9a00640 1 --2- 172.21.15.5:0/3358798806 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fbbac0a3c20 0x7fbbac0a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:36.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.138+0000 7fbbc9a00640 1 --2- 172.21.15.5:0/3358798806 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fbbac0a3c20 0x7fbbac0a60e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbbb80058c0 tx=0x7fbbb8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:36.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.140+0000 7fbbc2000640 1 -- 172.21.15.5:0/3358798806 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbbbc09d400 con 0x7fbbc410a100 2024-04-24T06:41:36.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.353+0000 7fbbcae00640 1 -- 172.21.15.5:0/3358798806 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbbc4062c00 con 0x7fbbc410a100 2024-04-24T06:41:36.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.356+0000 7fbbc2000640 1 -- 172.21.15.5:0/3358798806 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22088 (secure 0 0 0) 0x7fbbbc09cd00 con 0x7fbbc410a100 2024-04-24T06:41:36.357 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:41:36.357 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":86,"fsid":"43e9bad0-0204-11ef-bc93-c7b262605968","created":"2024-04-24T06:32:26.340555+0000","modified":"2024-04-24T06:41:36.259462+0000","last_up_change":"2024-04-24T06:40:36.790934+0000","last_in_change":"2024-04-24T06:40:17.803257+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:36:27.320342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:41:32.254502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:41:33.739067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c2e974c-745c-4dd5-8c3d-8e8da2eb1ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6803","nonce":2155858515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6805","nonce":2155858515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6809","nonce":2155858515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6807","nonce":2155858515}]},"public_addr":"172.21.15.5:6803/2155858515","cluster_addr":"172.21.15.5:6805/2155858515","heartbeat_back_addr":"172.21.15.5:6809/2155858515","heartbeat_front_addr":"172.21.15.5:6807/2155858515","state":["exists","up"]},{"osd":1,"uuid":"50467744-db20-4721-bf2d-1c5db938297b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6801","nonce":1302362650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6803","nonce":1302362650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6807","nonce":1302362650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6805","nonce":1302362650}]},"public_addr":"172.21.15.37:6801/1302362650","cluster_addr":"172.21.15.37:6803/1302362650","heartbeat_back_addr":"172.21.15.37:6807/1302362650","heartbeat_front_addr":"172.21.15.37:6805/1302362650","state":["exists","up"]},{"osd":2,"uuid":"0e190b15-d04c-4288-8d48-ecf6664392fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6801","nonce":2885027256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6803","nonce":2885027256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6807","nonce":2885027256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6805","nonce":2885027256}]},"public_addr":"172.21.15.137:6801/2885027256","cluster_addr":"172.21.15.137:6803/2885027256","heartbeat_back_addr":"172.21.15.137:6807/2885027256","heartbeat_front_addr":"172.21.15.137:6805/2885027256","state":["exists","up"]},{"osd":3,"uuid":"e05592ba-d96a-472a-9f63-09a535d04dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6811","nonce":3701747260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6813","nonce":3701747260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6817","nonce":3701747260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6815","nonce":3701747260}]},"public_addr":"172.21.15.5:6811/3701747260","cluster_addr":"172.21.15.5:6813/3701747260","heartbeat_back_addr":"172.21.15.5:6817/3701747260","heartbeat_front_addr":"172.21.15.5:6815/3701747260","state":["exists","up"]},{"osd":4,"uuid":"c3f36e51-7a3b-4dd9-af4b-a10558830c4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6809","nonce":759253844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6811","nonce":759253844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6815","nonce":759253844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6813","nonce":759253844}]},"public_addr":"172.21.15.37:6809/759253844","cluster_addr":"172.21.15.37:6811/759253844","heartbeat_back_addr":"172.21.15.37:6815/759253844","heartbeat_front_addr":"172.21.15.37:6813/759253844","state":["exists","up"]},{"osd":5,"uuid":"97a0500c-cbb8-4ff5-b476-8634b26b3821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6809","nonce":3771921560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6811","nonce":3771921560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6815","nonce":3771921560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6813","nonce":3771921560}]},"public_addr":"172.21.15.137:6809/3771921560","cluster_addr":"172.21.15.137:6811/3771921560","heartbeat_back_addr":"172.21.15.137:6815/3771921560","heartbeat_front_addr":"172.21.15.137:6813/3771921560","state":["exists","up"]},{"osd":6,"uuid":"33a33d62-bbb3-4ed9-834e-19ae93a97670","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6819","nonce":2064636084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6821","nonce":2064636084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6825","nonce":2064636084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6823","nonce":2064636084}]},"public_addr":"172.21.15.5:6819/2064636084","cluster_addr":"172.21.15.5:6821/2064636084","heartbeat_back_addr":"172.21.15.5:6825/2064636084","heartbeat_front_addr":"172.21.15.5:6823/2064636084","state":["exists","up"]},{"osd":7,"uuid":"c0f4e036-4717-4514-bb59-ab11fbaab063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6817","nonce":2952750659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6819","nonce":2952750659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6823","nonce":2952750659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6821","nonce":2952750659}]},"public_addr":"172.21.15.37:6817/2952750659","cluster_addr":"172.21.15.37:6819/2952750659","heartbeat_back_addr":"172.21.15.37:6823/2952750659","heartbeat_front_addr":"172.21.15.37:6821/2952750659","state":["exists","up"]},{"osd":8,"uuid":"5391fd87-44ea-440f-80da-c54d0327c04c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6817","nonce":29724945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6819","nonce":29724945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6823","nonce":29724945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6821","nonce":29724945}]},"public_addr":"172.21.15.137:6817/29724945","cluster_addr":"172.21.15.137:6819/29724945","heartbeat_back_addr":"172.21.15.137:6823/29724945","heartbeat_front_addr":"172.21.15.137:6821/29724945","state":["exists","up"]},{"osd":9,"uuid":"d94c9dbc-f43d-4237-ac7f-47a6a5ede1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6827","nonce":2257102107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6829","nonce":2257102107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6833","nonce":2257102107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6831","nonce":2257102107}]},"public_addr":"172.21.15.5:6827/2257102107","cluster_addr":"172.21.15.5:6829/2257102107","heartbeat_back_addr":"172.21.15.5:6833/2257102107","heartbeat_front_addr":"172.21.15.5:6831/2257102107","state":["exists","up"]},{"osd":10,"uuid":"e2455332-2a5e-44b1-85d3-8e47da399f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6825","nonce":2451570637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6827","nonce":2451570637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6831","nonce":2451570637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6829","nonce":2451570637}]},"public_addr":"172.21.15.37:6825/2451570637","cluster_addr":"172.21.15.37:6827/2451570637","heartbeat_back_addr":"172.21.15.37:6831/2451570637","heartbeat_front_addr":"172.21.15.37:6829/2451570637","state":["exists","up"]},{"osd":11,"uuid":"49d9396b-704a-4cef-8210-75d30c28e924","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6825","nonce":90600596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6827","nonce":90600596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6831","nonce":90600596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6829","nonce":90600596}]},"public_addr":"172.21.15.137:6825/90600596","cluster_addr":"172.21.15.137:6827/90600596","heartbeat_back_addr":"172.21.15.137:6831/90600596","heartbeat_front_addr":"172.21.15.137:6829/90600596","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:35:35.591619+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:00.568817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:24.195643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:56.451724+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:22.109293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:46.161749+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:19.278305+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:45.620256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:10.041165+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:43.813697+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:10.736026+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:35.384377+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/23307164":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6801/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:0/230785941":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/2940023597":"2024-04-25T06:33:07.040369+0000","172.21.15.5:6801/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3886613904":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/4199165437":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3459593499":"2024-04-25T06:33:41.161852+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:41:36.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.358+0000 7fbbcae00640 1 -- 172.21.15.5:0/3358798806 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fbbac0a3c20 msgr2=0x7fbbac0a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:36.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.358+0000 7fbbcae00640 1 --2- 172.21.15.5:0/3358798806 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fbbac0a3c20 0x7fbbac0a60e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbbb80058c0 tx=0x7fbbb8005f90 comp rx=0 tx=0).stop 2024-04-24T06:41:36.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.358+0000 7fbbcae00640 1 -- 172.21.15.5:0/3358798806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbc410a100 msgr2=0x7fbbc41a5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:36.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.358+0000 7fbbcae00640 1 --2- 172.21.15.5:0/3358798806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbc410a100 0x7fbbc41a5df0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc009fd0 tx=0x7fbbbc00ef10 comp rx=0 tx=0).stop 2024-04-24T06:41:36.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.359+0000 7fbbcae00640 1 -- 172.21.15.5:0/3358798806 shutdown_connections 2024-04-24T06:41:36.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.359+0000 7fbbcae00640 1 --2- 172.21.15.5:0/3358798806 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fbbac0a3c20 0x7fbbac0a60e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:36.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.359+0000 7fbbcae00640 1 --2- 172.21.15.5:0/3358798806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbc410a100 0x7fbbc41a5df0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:36.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.359+0000 7fbbcae00640 1 --2- 172.21.15.5:0/3358798806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbbc4109740 0x7fbbc419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:36.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.359+0000 7fbbcae00640 1 --2- 172.21.15.5:0/3358798806 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbbc41085a0 0x7fbbc419e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:36.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.359+0000 7fbbcae00640 1 -- 172.21.15.5:0/3358798806 >> 172.21.15.5:0/3358798806 conn(0x7fbbc40774f0 msgr2=0x7fbbc410ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:36.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.359+0000 7fbbcae00640 1 -- 172.21.15.5:0/3358798806 shutdown_connections 2024-04-24T06:41:36.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.359+0000 7fbbcae00640 1 -- 172.21.15.5:0/3358798806 wait complete. 2024-04-24T06:41:36.381 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-24T06:41:36.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.595+0000 7f7653600640 1 -- 172.21.15.5:0/2883318372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f764c108660 msgr2=0x7f764c108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:36.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.595+0000 7f7653600640 1 --2- 172.21.15.5:0/2883318372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f764c108660 0x7f764c108a80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7640009900 tx=0x7f764002a410 comp rx=0 tx=0).stop 2024-04-24T06:41:36.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.596+0000 7f7653600640 1 -- 172.21.15.5:0/2883318372 shutdown_connections 2024-04-24T06:41:36.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.596+0000 7f7653600640 1 --2- 172.21.15.5:0/2883318372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f764c10a210 0x7f764c110aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:36.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.596+0000 7f7653600640 1 --2- 172.21.15.5:0/2883318372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f764c109850 0x7f764c109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:36.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.596+0000 7f7653600640 1 --2- 172.21.15.5:0/2883318372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f764c108660 0x7f764c108a80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:36.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.596+0000 7f7653600640 1 -- 172.21.15.5:0/2883318372 >> 172.21.15.5:0/2883318372 conn(0x7f764c077610 msgr2=0x7f764c079a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:36.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.596+0000 7f7653600640 1 -- 172.21.15.5:0/2883318372 shutdown_connections 2024-04-24T06:41:36.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.596+0000 7f7653600640 1 -- 172.21.15.5:0/2883318372 wait complete. 2024-04-24T06:41:36.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.597+0000 7f7653600640 1 Processor -- start 2024-04-24T06:41:36.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.597+0000 7f7653600640 1 -- start start 2024-04-24T06:41:36.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.598+0000 7f7653600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f764c108660 0x7f764c19ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:36.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.598+0000 7f7653600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f764c109850 0x7f764c19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:36.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.598+0000 7f7653600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f764c10a210 0x7f764c1a5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:36.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.598+0000 7f7653600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f764c114330 con 0x7f764c108660 2024-04-24T06:41:36.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.598+0000 7f7653600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f764c1141b0 con 0x7f764c10a210 2024-04-24T06:41:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.598+0000 7f7653600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f764c1144b0 con 0x7f764c109850 2024-04-24T06:41:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.598+0000 7f7650c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f764c108660 0x7f764c19ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.598+0000 7f764be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f764c109850 0x7f764c19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.598+0000 7f7650c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f764c108660 0x7f764c19ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60618/0 (socket says 172.21.15.5:60618) 2024-04-24T06:41:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.598+0000 7f7650c00640 1 -- 172.21.15.5:0/4101404828 learned_addr learned my addr 172.21.15.5:0/4101404828 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:41:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.598+0000 7f7651600640 1 --2- 172.21.15.5:0/4101404828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f764c10a210 0x7f764c1a5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.598+0000 7f764be00640 1 -- 172.21.15.5:0/4101404828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f764c10a210 msgr2=0x7f764c1a5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.598+0000 7f764be00640 1 --2- 172.21.15.5:0/4101404828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f764c10a210 0x7f764c1a5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.598+0000 7f764be00640 1 -- 172.21.15.5:0/4101404828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f764c108660 msgr2=0x7f764c19ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.599+0000 7f764be00640 1 --2- 172.21.15.5:0/4101404828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f764c108660 0x7f764c19ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.599+0000 7f764be00640 1 -- 172.21.15.5:0/4101404828 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f764c1a6630 con 0x7f764c109850 2024-04-24T06:41:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.599+0000 7f7651600640 1 --2- 172.21.15.5:0/4101404828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f764c10a210 0x7f764c1a5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:41:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.599+0000 7f7650c00640 1 --2- 172.21.15.5:0/4101404828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f764c108660 0x7f764c19ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:41:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.599+0000 7f764be00640 1 --2- 172.21.15.5:0/4101404828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f764c109850 0x7f764c19ef50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f763800d720 tx=0x7f763800db20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.599+0000 7f7649600640 1 -- 172.21.15.5:0/4101404828 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7638017070 con 0x7f764c109850 2024-04-24T06:41:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.599+0000 7f7649600640 1 -- 172.21.15.5:0/4101404828 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7638005500 con 0x7f764c109850 2024-04-24T06:41:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.599+0000 7f7649600640 1 -- 172.21.15.5:0/4101404828 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76380047a0 con 0x7f764c109850 2024-04-24T06:41:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.599+0000 7f7653600640 1 -- 172.21.15.5:0/4101404828 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f764c1a6920 con 0x7f764c109850 2024-04-24T06:41:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.600+0000 7f7653600640 1 -- 172.21.15.5:0/4101404828 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f764c1a6e20 con 0x7f764c109850 2024-04-24T06:41:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.601+0000 7f7653600640 1 -- 172.21.15.5:0/4101404828 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f764c10f9f0 con 0x7f764c109850 2024-04-24T06:41:36.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.604+0000 7f7649600640 1 -- 172.21.15.5:0/4101404828 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f7638005020 con 0x7f764c109850 2024-04-24T06:41:36.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.605+0000 7f7649600640 1 --2- 172.21.15.5:0/4101404828 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f76340a3bb0 0x7f76340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:36.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.606+0000 7f7649600640 1 -- 172.21.15.5:0/4101404828 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f76380cff30 con 0x7f764c109850 2024-04-24T06:41:36.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.606+0000 7f7650c00640 1 --2- 172.21.15.5:0/4101404828 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f76340a3bb0 0x7f76340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:36.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.607+0000 7f7650c00640 1 --2- 172.21.15.5:0/4101404828 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f76340a3bb0 0x7f76340a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7640000c00 tx=0x7f76400315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:36.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.610+0000 7f7649600640 1 -- 172.21.15.5:0/4101404828 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7638020020 con 0x7f764c109850 2024-04-24T06:41:36.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:36 smithi137 ceph-mon[33017]: osdmap e85: 12 total, 12 up, 12 in 2024-04-24T06:41:36.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:36 smithi137 ceph-mon[33017]: pgmap v312: 97 pgs: 10 creating+peering, 4 active+clean, 83 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:36.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:36 smithi137 ceph-mon[33017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-24T06:41:36.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:36 smithi137 ceph-mon[33017]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-24T06:41:36.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:36 smithi137 ceph-mon[33017]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-24T06:41:36.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:36 smithi137 ceph-mon[33017]: mds.? [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] up:active 2024-04-24T06:41:36.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:36 smithi137 ceph-mon[33017]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T06:41:36.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:36 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/2291194249' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:41:36.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:36 smithi005 ceph-mon[27798]: osdmap e85: 12 total, 12 up, 12 in 2024-04-24T06:41:36.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:36 smithi005 ceph-mon[27798]: pgmap v312: 97 pgs: 10 creating+peering, 4 active+clean, 83 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:36.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:36 smithi005 ceph-mon[27798]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-24T06:41:36.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:36 smithi005 ceph-mon[27798]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-24T06:41:36.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:36 smithi005 ceph-mon[27798]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-24T06:41:36.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:36 smithi005 ceph-mon[27798]: mds.? [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] up:active 2024-04-24T06:41:36.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:36 smithi005 ceph-mon[27798]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T06:41:36.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:36 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/2291194249' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:41:36.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:36 smithi037 ceph-mon[32864]: osdmap e85: 12 total, 12 up, 12 in 2024-04-24T06:41:36.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:36 smithi037 ceph-mon[32864]: pgmap v312: 97 pgs: 10 creating+peering, 4 active+clean, 83 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:41:36.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:36 smithi037 ceph-mon[32864]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-24T06:41:36.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:36 smithi037 ceph-mon[32864]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-24T06:41:36.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:36 smithi037 ceph-mon[32864]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-24T06:41:36.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:36 smithi037 ceph-mon[32864]: mds.? [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] up:active 2024-04-24T06:41:36.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:36 smithi037 ceph-mon[32864]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T06:41:36.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:36 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/2291194249' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:41:36.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:36.862+0000 7f7653600640 1 -- 172.21.15.5:0/4101404828 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f764c02d620 con 0x7f764c109850 2024-04-24T06:41:37.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.286+0000 7f7649600640 1 -- 172.21.15.5:0/4101404828 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f763809ca00 con 0x7f764c109850 2024-04-24T06:41:37.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.289+0000 7f7653600640 1 -- 172.21.15.5:0/4101404828 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f76340a3bb0 msgr2=0x7f76340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:37.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.289+0000 7f7653600640 1 --2- 172.21.15.5:0/4101404828 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f76340a3bb0 0x7f76340a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7640000c00 tx=0x7f76400315b0 comp rx=0 tx=0).stop 2024-04-24T06:41:37.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.289+0000 7f7653600640 1 -- 172.21.15.5:0/4101404828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f764c109850 msgr2=0x7f764c19ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:37.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.289+0000 7f7653600640 1 --2- 172.21.15.5:0/4101404828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f764c109850 0x7f764c19ef50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f763800d720 tx=0x7f763800db20 comp rx=0 tx=0).stop 2024-04-24T06:41:37.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.289+0000 7f7653600640 1 -- 172.21.15.5:0/4101404828 shutdown_connections 2024-04-24T06:41:37.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.289+0000 7f7653600640 1 --2- 172.21.15.5:0/4101404828 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f76340a3bb0 0x7f76340a6070 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:37.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.289+0000 7f7653600640 1 --2- 172.21.15.5:0/4101404828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f764c10a210 0x7f764c1a5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:37.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.289+0000 7f7653600640 1 --2- 172.21.15.5:0/4101404828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f764c109850 0x7f764c19ef50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:37.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.289+0000 7f7653600640 1 --2- 172.21.15.5:0/4101404828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f764c108660 0x7f764c19ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:37.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.290+0000 7f7653600640 1 -- 172.21.15.5:0/4101404828 >> 172.21.15.5:0/4101404828 conn(0x7f764c077610 msgr2=0x7f764c10cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:37.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.290+0000 7f7653600640 1 -- 172.21.15.5:0/4101404828 shutdown_connections 2024-04-24T06:41:37.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.290+0000 7f7653600640 1 -- 172.21.15.5:0/4101404828 wait complete. 2024-04-24T06:41:37.315 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-04-24T06:41:37.315 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-04-24T06:41:37.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.536+0000 7ff387e00640 1 -- 172.21.15.5:0/2388708478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff388101c50 msgr2=0x7ff3881084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:37.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.536+0000 7ff387e00640 1 --2- 172.21.15.5:0/2388708478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff388101c50 0x7ff3881084e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff380009ca0 tx=0x7ff38002a8c0 comp rx=0 tx=0).stop 2024-04-24T06:41:37.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.536+0000 7ff387e00640 1 -- 172.21.15.5:0/2388708478 shutdown_connections 2024-04-24T06:41:37.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.536+0000 7ff387e00640 1 --2- 172.21.15.5:0/2388708478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff388101c50 0x7ff3881084e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:37.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.536+0000 7ff387e00640 1 --2- 172.21.15.5:0/2388708478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff388101290 0x7ff388101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:37.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.536+0000 7ff387e00640 1 --2- 172.21.15.5:0/2388708478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3881000a0 0x7ff3881004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:37.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.536+0000 7ff387e00640 1 -- 172.21.15.5:0/2388708478 >> 172.21.15.5:0/2388708478 conn(0x7ff3880fb610 msgr2=0x7ff3880fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:37.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.536+0000 7ff387e00640 1 -- 172.21.15.5:0/2388708478 shutdown_connections 2024-04-24T06:41:37.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.536+0000 7ff387e00640 1 -- 172.21.15.5:0/2388708478 wait complete. 2024-04-24T06:41:37.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.537+0000 7ff387e00640 1 Processor -- start 2024-04-24T06:41:37.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.537+0000 7ff387e00640 1 -- start start 2024-04-24T06:41:37.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.537+0000 7ff387e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3881000a0 0x7ff388104030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:37.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.537+0000 7ff387e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff388101290 0x7ff388102670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:37.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.537+0000 7ff387e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff388101c50 0x7ff388102bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:37.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.537+0000 7ff387e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff38810b940 con 0x7ff388101c50 2024-04-24T06:41:37.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.537+0000 7ff387e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff38810b7c0 con 0x7ff3881000a0 2024-04-24T06:41:37.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.537+0000 7ff387e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff38810bac0 con 0x7ff388101290 2024-04-24T06:41:37.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.538+0000 7ff386a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3881000a0 0x7ff388104030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:37.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.538+0000 7ff386a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3881000a0 0x7ff388104030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:35714/0 (socket says 172.21.15.5:35714) 2024-04-24T06:41:37.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.538+0000 7ff386a00640 1 -- 172.21.15.5:0/3946084744 learned_addr learned my addr 172.21.15.5:0/3946084744 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:41:37.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.538+0000 7ff386000640 1 --2- 172.21.15.5:0/3946084744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff388101290 0x7ff388102670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:37.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.538+0000 7ff387400640 1 --2- 172.21.15.5:0/3946084744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff388101c50 0x7ff388102bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:37.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.538+0000 7ff386a00640 1 -- 172.21.15.5:0/3946084744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff388101290 msgr2=0x7ff388102670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:37.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.538+0000 7ff386a00640 1 --2- 172.21.15.5:0/3946084744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff388101290 0x7ff388102670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:37.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.538+0000 7ff386a00640 1 -- 172.21.15.5:0/3946084744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff388101c50 msgr2=0x7ff388102bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:37.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.538+0000 7ff386a00640 1 --2- 172.21.15.5:0/3946084744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff388101c50 0x7ff388102bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:37.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.538+0000 7ff386a00640 1 -- 172.21.15.5:0/3946084744 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff388103440 con 0x7ff3881000a0 2024-04-24T06:41:37.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.538+0000 7ff387400640 1 --2- 172.21.15.5:0/3946084744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff388101c50 0x7ff388102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:41:37.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.539+0000 7ff386a00640 1 --2- 172.21.15.5:0/3946084744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3881000a0 0x7ff388104030 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff3780097d0 tx=0x7ff378009c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:37.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.539+0000 7ff37f400640 1 -- 172.21.15.5:0/3946084744 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff378018030 con 0x7ff3881000a0 2024-04-24T06:41:37.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.539+0000 7ff37f400640 1 -- 172.21.15.5:0/3946084744 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff37800e8c0 con 0x7ff3881000a0 2024-04-24T06:41:37.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.539+0000 7ff387e00640 1 -- 172.21.15.5:0/3946084744 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3881a2290 con 0x7ff3881000a0 2024-04-24T06:41:37.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.539+0000 7ff37f400640 1 -- 172.21.15.5:0/3946084744 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff378003890 con 0x7ff3881000a0 2024-04-24T06:41:37.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.539+0000 7ff387e00640 1 -- 172.21.15.5:0/3946084744 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3881a2830 con 0x7ff3881000a0 2024-04-24T06:41:37.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.540+0000 7ff387e00640 1 -- 172.21.15.5:0/3946084744 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3881a2b50 con 0x7ff3881000a0 2024-04-24T06:41:37.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.542+0000 7ff37f400640 1 -- 172.21.15.5:0/3946084744 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7ff37801b070 con 0x7ff3881000a0 2024-04-24T06:41:37.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.543+0000 7ff37f400640 1 --2- 172.21.15.5:0/3946084744 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ff36c0a3c30 0x7ff36c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:37.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.544+0000 7ff37f400640 1 -- 172.21.15.5:0/3946084744 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7ff3780cfb10 con 0x7ff3881000a0 2024-04-24T06:41:37.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.544+0000 7ff386000640 1 --2- 172.21.15.5:0/3946084744 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ff36c0a3c30 0x7ff36c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:37.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.544+0000 7ff386000640 1 --2- 172.21.15.5:0/3946084744 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ff36c0a3c30 0x7ff36c0a60f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff388103da0 tx=0x7ff374005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:37.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.548+0000 7ff37f400640 1 -- 172.21.15.5:0/3946084744 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff378022070 con 0x7ff3881000a0 2024-04-24T06:41:37.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:37 smithi137 ceph-mon[33017]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T06:41:37.770 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:37 smithi137 ceph-mon[33017]: Cluster is now healthy 2024-04-24T06:41:37.770 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:37 smithi137 ceph-mon[33017]: osdmap e86: 12 total, 12 up, 12 in 2024-04-24T06:41:37.770 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:37 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/3358798806' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:41:37.771 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:37 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/4101404828' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-24T06:41:37.771 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:37 smithi137 ceph-mon[33017]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-24T06:41:37.772 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:37 smithi037 ceph-mon[32864]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T06:41:37.772 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:37 smithi037 ceph-mon[32864]: Cluster is now healthy 2024-04-24T06:41:37.772 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:37 smithi037 ceph-mon[32864]: osdmap e86: 12 total, 12 up, 12 in 2024-04-24T06:41:37.772 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:37 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/3358798806' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:41:37.772 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:37 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/4101404828' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-24T06:41:37.772 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:37 smithi037 ceph-mon[32864]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-24T06:41:37.772 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:37 smithi005 ceph-mon[27798]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T06:41:37.772 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:37 smithi005 ceph-mon[27798]: Cluster is now healthy 2024-04-24T06:41:37.772 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:37 smithi005 ceph-mon[27798]: osdmap e86: 12 total, 12 up, 12 in 2024-04-24T06:41:37.772 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:37 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/3358798806' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:41:37.773 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:37 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/4101404828' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-24T06:41:37.773 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:37 smithi005 ceph-mon[27798]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-24T06:41:37.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:37.800+0000 7ff387e00640 1 -- 172.21.15.5:0/3946084744 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7ff3881a2d20 con 0x7ff3881000a0 2024-04-24T06:41:38.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.297+0000 7ff37f400640 1 -- 172.21.15.5:0/3946084744 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v17) ==== 105+0+0 (secure 0 0 0) 0x7ff378011070 con 0x7ff3881000a0 2024-04-24T06:41:38.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.301+0000 7ff387e00640 1 -- 172.21.15.5:0/3946084744 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ff36c0a3c30 msgr2=0x7ff36c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:38.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.301+0000 7ff387e00640 1 --2- 172.21.15.5:0/3946084744 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ff36c0a3c30 0x7ff36c0a60f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff388103da0 tx=0x7ff374005f90 comp rx=0 tx=0).stop 2024-04-24T06:41:38.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.301+0000 7ff387e00640 1 -- 172.21.15.5:0/3946084744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3881000a0 msgr2=0x7ff388104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:38.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.301+0000 7ff387e00640 1 --2- 172.21.15.5:0/3946084744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3881000a0 0x7ff388104030 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff3780097d0 tx=0x7ff378009c90 comp rx=0 tx=0).stop 2024-04-24T06:41:38.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.301+0000 7ff387e00640 1 -- 172.21.15.5:0/3946084744 shutdown_connections 2024-04-24T06:41:38.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.301+0000 7ff387e00640 1 --2- 172.21.15.5:0/3946084744 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ff36c0a3c30 0x7ff36c0a60f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:38.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.301+0000 7ff387e00640 1 --2- 172.21.15.5:0/3946084744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff388101c50 0x7ff388102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:38.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.301+0000 7ff387e00640 1 --2- 172.21.15.5:0/3946084744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff388101290 0x7ff388102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:38.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.301+0000 7ff387e00640 1 --2- 172.21.15.5:0/3946084744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3881000a0 0x7ff388104030 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:38.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.301+0000 7ff387e00640 1 -- 172.21.15.5:0/3946084744 >> 172.21.15.5:0/3946084744 conn(0x7ff3880fb610 msgr2=0x7ff388106740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:38.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.301+0000 7ff387e00640 1 -- 172.21.15.5:0/3946084744 shutdown_connections 2024-04-24T06:41:38.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.302+0000 7ff387e00640 1 -- 172.21.15.5:0/3946084744 wait complete. 2024-04-24T06:41:38.332 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-24T06:41:38.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.537+0000 7faeff000640 1 -- 172.21.15.5:0/3118198959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faef8105230 msgr2=0x7faef8107620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:38.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.537+0000 7faeff000640 1 --2- 172.21.15.5:0/3118198959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faef8105230 0x7faef8107620 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faee4009900 tx=0x7faee402a3f0 comp rx=0 tx=0).stop 2024-04-24T06:41:38.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.538+0000 7faeff000640 1 -- 172.21.15.5:0/3118198959 shutdown_connections 2024-04-24T06:41:38.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.538+0000 7faeff000640 1 --2- 172.21.15.5:0/3118198959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faef8107b60 0x7faef8109fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:38.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.538+0000 7faeff000640 1 --2- 172.21.15.5:0/3118198959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faef8105230 0x7faef8107620 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:38.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.538+0000 7faeff000640 1 --2- 172.21.15.5:0/3118198959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faef8102900 0x7faef8104cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:38.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.538+0000 7faeff000640 1 -- 172.21.15.5:0/3118198959 >> 172.21.15.5:0/3118198959 conn(0x7faef80fc2f0 msgr2=0x7faef80fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:38.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.538+0000 7faeff000640 1 -- 172.21.15.5:0/3118198959 shutdown_connections 2024-04-24T06:41:38.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.538+0000 7faeff000640 1 -- 172.21.15.5:0/3118198959 wait complete. 2024-04-24T06:41:38.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.539+0000 7faeff000640 1 Processor -- start 2024-04-24T06:41:38.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.539+0000 7faeff000640 1 -- start start 2024-04-24T06:41:38.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.539+0000 7faeff000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faef8102900 0x7faef8102270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:38.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.539+0000 7faeff000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faef8105230 0x7faef81008b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:38.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.539+0000 7faeff000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faef8107b60 0x7faef8100df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:38.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.539+0000 7faeff000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faef810daf0 con 0x7faef8102900 2024-04-24T06:41:38.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.539+0000 7faeff000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faef810d970 con 0x7faef8105230 2024-04-24T06:41:38.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.540+0000 7faeff000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faef810dc70 con 0x7faef8107b60 2024-04-24T06:41:38.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.540+0000 7faefdc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faef8102900 0x7faef8102270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:38.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.540+0000 7faefdc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faef8102900 0x7faef8102270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60666/0 (socket says 172.21.15.5:60666) 2024-04-24T06:41:38.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.540+0000 7faefdc00640 1 -- 172.21.15.5:0/3198956517 learned_addr learned my addr 172.21.15.5:0/3198956517 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:41:38.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.540+0000 7faefd200640 1 --2- 172.21.15.5:0/3198956517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faef8105230 0x7faef81008b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:38.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.540+0000 7faefe600640 1 --2- 172.21.15.5:0/3198956517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faef8107b60 0x7faef8100df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:38.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.540+0000 7faefd200640 1 -- 172.21.15.5:0/3198956517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faef8107b60 msgr2=0x7faef8100df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:38.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.540+0000 7faefd200640 1 --2- 172.21.15.5:0/3198956517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faef8107b60 0x7faef8100df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:38.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.540+0000 7faefd200640 1 -- 172.21.15.5:0/3198956517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faef8102900 msgr2=0x7faef8102270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:38.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.540+0000 7faefd200640 1 --2- 172.21.15.5:0/3198956517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faef8102900 0x7faef8102270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:38.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.540+0000 7faefd200640 1 -- 172.21.15.5:0/3198956517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef8101680 con 0x7faef8105230 2024-04-24T06:41:38.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.540+0000 7faefdc00640 1 --2- 172.21.15.5:0/3198956517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faef8102900 0x7faef8102270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:41:38.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.541+0000 7faefd200640 1 --2- 172.21.15.5:0/3198956517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faef8105230 0x7faef81008b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7faee4009230 tx=0x7faee4004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:38.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.541+0000 7faef6000640 1 -- 172.21.15.5:0/3198956517 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee403f070 con 0x7faef8105230 2024-04-24T06:41:38.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.541+0000 7faeff000640 1 -- 172.21.15.5:0/3198956517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faef81a22b0 con 0x7faef8105230 2024-04-24T06:41:38.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.541+0000 7faef6000640 1 -- 172.21.15.5:0/3198956517 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faee4003cd0 con 0x7faef8105230 2024-04-24T06:41:38.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.541+0000 7faef6000640 1 -- 172.21.15.5:0/3198956517 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee4035070 con 0x7faef8105230 2024-04-24T06:41:38.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.541+0000 7faeff000640 1 -- 172.21.15.5:0/3198956517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faef81a27d0 con 0x7faef8105230 2024-04-24T06:41:38.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.542+0000 7faeff000640 1 -- 172.21.15.5:0/3198956517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faef80fdde0 con 0x7faef8105230 2024-04-24T06:41:38.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.544+0000 7faef6000640 1 -- 172.21.15.5:0/3198956517 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7faee403b460 con 0x7faef8105230 2024-04-24T06:41:38.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.546+0000 7faef6000640 1 --2- 172.21.15.5:0/3198956517 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7faee00a3bb0 0x7faee00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:38.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.546+0000 7faef6000640 1 -- 172.21.15.5:0/3198956517 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7faee40f0820 con 0x7faef8105230 2024-04-24T06:41:38.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.546+0000 7faefdc00640 1 --2- 172.21.15.5:0/3198956517 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7faee00a3bb0 0x7faee00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:38.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.547+0000 7faefdc00640 1 --2- 172.21.15.5:0/3198956517 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7faee00a3bb0 0x7faee00a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7faeec011550 tx=0x7faeec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:38.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.551+0000 7faef6000640 1 -- 172.21.15.5:0/3198956517 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faee40bd2b0 con 0x7faef8105230 2024-04-24T06:41:38.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:38 smithi137 ceph-mon[33017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-24T06:41:38.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:38 smithi137 ceph-mon[33017]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T06:41:38.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:38 smithi137 ceph-mon[33017]: pgmap v314: 97 pgs: 10 creating+peering, 28 active+clean, 59 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 504 B/s wr, 6 op/s 2024-04-24T06:41:38.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:38 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/3946084744' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-24T06:41:38.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:38 smithi137 ceph-mon[33017]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-24T06:41:38.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:38 smithi005 ceph-mon[27798]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-24T06:41:38.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:38 smithi005 ceph-mon[27798]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T06:41:38.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:38 smithi005 ceph-mon[27798]: pgmap v314: 97 pgs: 10 creating+peering, 28 active+clean, 59 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 504 B/s wr, 6 op/s 2024-04-24T06:41:38.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:38 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/3946084744' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-24T06:41:38.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:38 smithi005 ceph-mon[27798]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-24T06:41:38.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:38 smithi037 ceph-mon[32864]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-24T06:41:38.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:38 smithi037 ceph-mon[32864]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T06:41:38.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:38 smithi037 ceph-mon[32864]: pgmap v314: 97 pgs: 10 creating+peering, 28 active+clean, 59 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 504 B/s wr, 6 op/s 2024-04-24T06:41:38.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:38 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/3946084744' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-24T06:41:38.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:38 smithi037 ceph-mon[32864]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-24T06:41:38.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:38.800+0000 7faeff000640 1 -- 172.21.15.5:0/3198956517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7faef802d620 con 0x7faef8105230 2024-04-24T06:41:39.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.313+0000 7faef6000640 1 -- 172.21.15.5:0/3198956517 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v19) ==== 121+0+0 (secure 0 0 0) 0x7faee40bcbb0 con 0x7faef8105230 2024-04-24T06:41:39.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.316+0000 7faeff000640 1 -- 172.21.15.5:0/3198956517 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7faee00a3bb0 msgr2=0x7faee00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:39.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.316+0000 7faeff000640 1 --2- 172.21.15.5:0/3198956517 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7faee00a3bb0 0x7faee00a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7faeec011550 tx=0x7faeec005f90 comp rx=0 tx=0).stop 2024-04-24T06:41:39.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.316+0000 7faeff000640 1 -- 172.21.15.5:0/3198956517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faef8105230 msgr2=0x7faef81008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:39.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.316+0000 7faeff000640 1 --2- 172.21.15.5:0/3198956517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faef8105230 0x7faef81008b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7faee4009230 tx=0x7faee4004270 comp rx=0 tx=0).stop 2024-04-24T06:41:39.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.317+0000 7faeff000640 1 -- 172.21.15.5:0/3198956517 shutdown_connections 2024-04-24T06:41:39.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.317+0000 7faeff000640 1 --2- 172.21.15.5:0/3198956517 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7faee00a3bb0 0x7faee00a6070 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:39.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.317+0000 7faeff000640 1 --2- 172.21.15.5:0/3198956517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faef8107b60 0x7faef8100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:39.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.317+0000 7faeff000640 1 --2- 172.21.15.5:0/3198956517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faef8105230 0x7faef81008b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:39.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.317+0000 7faeff000640 1 --2- 172.21.15.5:0/3198956517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faef8102900 0x7faef8102270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:39.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.317+0000 7faeff000640 1 -- 172.21.15.5:0/3198956517 >> 172.21.15.5:0/3198956517 conn(0x7faef80fc2f0 msgr2=0x7faef8103240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:39.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.317+0000 7faeff000640 1 -- 172.21.15.5:0/3198956517 shutdown_connections 2024-04-24T06:41:39.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.317+0000 7faeff000640 1 -- 172.21.15.5:0/3198956517 wait complete. 2024-04-24T06:41:39.346 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-24T06:41:39.518 INFO:journalctl@ceph.mds.b.smithi037.stdout:Apr 24 06:41:39 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-b[63330]: 2024-04-24T06:41:39.168+0000 7f4e6146c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T06:41:39.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:39 smithi037 ceph-mon[32864]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-24T06:41:39.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:39 smithi037 ceph-mon[32864]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-24T06:41:39.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:39 smithi037 ceph-mon[32864]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-24T06:41:39.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:39 smithi037 ceph-mon[32864]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T06:41:39.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:39 smithi037 ceph-mon[32864]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-24T06:41:39.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:39 smithi037 ceph-mon[32864]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-24T06:41:39.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:39 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/3198956517' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-24T06:41:39.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:39 smithi037 ceph-mon[32864]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-24T06:41:39.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.531+0000 7fd191600640 1 -- 172.21.15.5:0/3027555863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd18c109670 msgr2=0x7fd18c109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:39.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.531+0000 7fd191600640 1 --2- 172.21.15.5:0/3027555863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd18c109670 0x7fd18c109af0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd174009900 tx=0x7fd17402a3f0 comp rx=0 tx=0).stop 2024-04-24T06:41:39.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.532+0000 7fd191600640 1 -- 172.21.15.5:0/3027555863 shutdown_connections 2024-04-24T06:41:39.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.532+0000 7fd191600640 1 --2- 172.21.15.5:0/3027555863 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd18c10a030 0x7fd18c1108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:39.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.532+0000 7fd191600640 1 --2- 172.21.15.5:0/3027555863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd18c109670 0x7fd18c109af0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:39.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.532+0000 7fd191600640 1 --2- 172.21.15.5:0/3027555863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd18c108480 0x7fd18c1088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:39.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.532+0000 7fd191600640 1 -- 172.21.15.5:0/3027555863 >> 172.21.15.5:0/3027555863 conn(0x7fd18c077440 msgr2=0x7fd18c079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:39.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.532+0000 7fd191600640 1 -- 172.21.15.5:0/3027555863 shutdown_connections 2024-04-24T06:41:39.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.532+0000 7fd191600640 1 -- 172.21.15.5:0/3027555863 wait complete. 2024-04-24T06:41:39.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.533+0000 7fd191600640 1 Processor -- start 2024-04-24T06:41:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.533+0000 7fd191600640 1 -- start start 2024-04-24T06:41:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.534+0000 7fd191600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd18c108480 0x7fd18c19e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.534+0000 7fd191600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd18c109670 0x7fd18c19eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:39.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.534+0000 7fd191600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd18c10a030 0x7fd18c1a5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:39.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.534+0000 7fd18be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd18c108480 0x7fd18c19e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:39.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.534+0000 7fd18be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd18c108480 0x7fd18c19e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:35758/0 (socket says 172.21.15.5:35758) 2024-04-24T06:41:39.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.534+0000 7fd18be00640 1 -- 172.21.15.5:0/1099844948 learned_addr learned my addr 172.21.15.5:0/1099844948 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:41:39.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.534+0000 7fd191600640 1 -- 172.21.15.5:0/1099844948 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd18c114160 con 0x7fd18c10a030 2024-04-24T06:41:39.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.534+0000 7fd191600640 1 -- 172.21.15.5:0/1099844948 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd18c113fe0 con 0x7fd18c108480 2024-04-24T06:41:39.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.534+0000 7fd191600640 1 -- 172.21.15.5:0/1099844948 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd18c1142e0 con 0x7fd18c109670 2024-04-24T06:41:39.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.534+0000 7fd183e00640 1 --2- 172.21.15.5:0/1099844948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd18c109670 0x7fd18c19eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:39.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.534+0000 7fd190c00640 1 --2- 172.21.15.5:0/1099844948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd18c10a030 0x7fd18c1a5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:39.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.534+0000 7fd18be00640 1 -- 172.21.15.5:0/1099844948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd18c109670 msgr2=0x7fd18c19eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:39.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.534+0000 7fd18be00640 1 --2- 172.21.15.5:0/1099844948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd18c109670 0x7fd18c19eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:39.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.534+0000 7fd18be00640 1 -- 172.21.15.5:0/1099844948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd18c10a030 msgr2=0x7fd18c1a5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:39.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.534+0000 7fd18be00640 1 --2- 172.21.15.5:0/1099844948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd18c10a030 0x7fd18c1a5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:39.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.534+0000 7fd18be00640 1 -- 172.21.15.5:0/1099844948 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd18c1a6380 con 0x7fd18c108480 2024-04-24T06:41:39.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.534+0000 7fd190c00640 1 --2- 172.21.15.5:0/1099844948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd18c10a030 0x7fd18c1a5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:41:39.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.535+0000 7fd18be00640 1 --2- 172.21.15.5:0/1099844948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd18c108480 0x7fd18c19e760 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00e9c0 tx=0x7fd17c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:39.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.535+0000 7fd189600640 1 -- 172.21.15.5:0/1099844948 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c00cde0 con 0x7fd18c108480 2024-04-24T06:41:39.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.535+0000 7fd189600640 1 -- 172.21.15.5:0/1099844948 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd17c010910 con 0x7fd18c108480 2024-04-24T06:41:39.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.535+0000 7fd191600640 1 -- 172.21.15.5:0/1099844948 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd18c1a6670 con 0x7fd18c108480 2024-04-24T06:41:39.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.535+0000 7fd189600640 1 -- 172.21.15.5:0/1099844948 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c018a00 con 0x7fd18c108480 2024-04-24T06:41:39.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.535+0000 7fd191600640 1 -- 172.21.15.5:0/1099844948 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd18c1a6be0 con 0x7fd18c108480 2024-04-24T06:41:39.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.536+0000 7fd191600640 1 -- 172.21.15.5:0/1099844948 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd18c10f7a0 con 0x7fd18c108480 2024-04-24T06:41:39.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.538+0000 7fd189600640 1 -- 172.21.15.5:0/1099844948 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fd17c01f020 con 0x7fd18c108480 2024-04-24T06:41:39.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.540+0000 7fd189600640 1 --2- 172.21.15.5:0/1099844948 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fd1700a3c30 0x7fd1700a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:39.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.540+0000 7fd189600640 1 -- 172.21.15.5:0/1099844948 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fd17c0d0550 con 0x7fd18c108480 2024-04-24T06:41:39.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.540+0000 7fd183e00640 1 --2- 172.21.15.5:0/1099844948 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fd1700a3c30 0x7fd1700a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:39.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.541+0000 7fd183e00640 1 --2- 172.21.15.5:0/1099844948 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fd1700a3c30 0x7fd1700a60f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd174031440 tx=0x7fd1740315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:39.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.544+0000 7fd189600640 1 -- 172.21.15.5:0/1099844948 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd17c09cfe0 con 0x7fd18c108480 2024-04-24T06:41:39.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:39 smithi137 ceph-mon[33017]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-24T06:41:39.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:39 smithi137 ceph-mon[33017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-24T06:41:39.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:39 smithi137 ceph-mon[33017]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-24T06:41:39.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:39 smithi137 ceph-mon[33017]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T06:41:39.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:39 smithi137 ceph-mon[33017]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-24T06:41:39.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:39 smithi137 ceph-mon[33017]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-24T06:41:39.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:39 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/3198956517' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-24T06:41:39.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:39 smithi137 ceph-mon[33017]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-24T06:41:39.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:39 smithi005 ceph-mon[27798]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-24T06:41:39.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:39 smithi005 ceph-mon[27798]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-24T06:41:39.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:39 smithi005 ceph-mon[27798]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-24T06:41:39.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:39 smithi005 ceph-mon[27798]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T06:41:39.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:39 smithi005 ceph-mon[27798]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-24T06:41:39.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:39 smithi005 ceph-mon[27798]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-24T06:41:39.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:39 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/3198956517' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-24T06:41:39.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:39 smithi005 ceph-mon[27798]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-24T06:41:39.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:39.799+0000 7fd191600640 1 -- 172.21.15.5:0/1099844948 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fd18c02d620 con 0x7fd18c108480 2024-04-24T06:41:40.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.324+0000 7fd189600640 1 -- 172.21.15.5:0/1099844948 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v21) ==== 115+0+0 (secure 0 0 0) 0x7fd17c09c8e0 con 0x7fd18c108480 2024-04-24T06:41:40.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.328+0000 7fd191600640 1 -- 172.21.15.5:0/1099844948 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fd1700a3c30 msgr2=0x7fd1700a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:40.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.328+0000 7fd191600640 1 --2- 172.21.15.5:0/1099844948 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fd1700a3c30 0x7fd1700a60f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd174031440 tx=0x7fd1740315b0 comp rx=0 tx=0).stop 2024-04-24T06:41:40.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.328+0000 7fd191600640 1 -- 172.21.15.5:0/1099844948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd18c108480 msgr2=0x7fd18c19e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:40.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.328+0000 7fd191600640 1 --2- 172.21.15.5:0/1099844948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd18c108480 0x7fd18c19e760 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00e9c0 tx=0x7fd17c00ee80 comp rx=0 tx=0).stop 2024-04-24T06:41:40.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.329+0000 7fd191600640 1 -- 172.21.15.5:0/1099844948 shutdown_connections 2024-04-24T06:41:40.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.329+0000 7fd191600640 1 --2- 172.21.15.5:0/1099844948 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fd1700a3c30 0x7fd1700a60f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:40.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.329+0000 7fd191600640 1 --2- 172.21.15.5:0/1099844948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd18c10a030 0x7fd18c1a5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:40.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.329+0000 7fd191600640 1 --2- 172.21.15.5:0/1099844948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd18c109670 0x7fd18c19eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:40.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.329+0000 7fd191600640 1 --2- 172.21.15.5:0/1099844948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd18c108480 0x7fd18c19e760 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:40.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.329+0000 7fd191600640 1 -- 172.21.15.5:0/1099844948 >> 172.21.15.5:0/1099844948 conn(0x7fd18c077440 msgr2=0x7fd18c10c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:40.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.329+0000 7fd191600640 1 -- 172.21.15.5:0/1099844948 shutdown_connections 2024-04-24T06:41:40.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.329+0000 7fd191600640 1 -- 172.21.15.5:0/1099844948 wait complete. 2024-04-24T06:41:40.356 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-04-24T06:41:40.356 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-24T06:41:40.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.570+0000 7f574b200640 1 -- 172.21.15.5:0/2688447314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f574c10a030 msgr2=0x7f574c1108c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:40.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.570+0000 7f574b200640 1 --2- 172.21.15.5:0/2688447314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f574c10a030 0x7f574c1108c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5744009ca0 tx=0x7f574402a8c0 comp rx=0 tx=0).stop 2024-04-24T06:41:40.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.570+0000 7f574b200640 1 -- 172.21.15.5:0/2688447314 shutdown_connections 2024-04-24T06:41:40.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.570+0000 7f574b200640 1 --2- 172.21.15.5:0/2688447314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f574c10a030 0x7f574c1108c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:40.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.570+0000 7f574b200640 1 --2- 172.21.15.5:0/2688447314 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f574c109670 0x7f574c109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:40.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.570+0000 7f574b200640 1 --2- 172.21.15.5:0/2688447314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f574c108480 0x7f574c1088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:40.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.570+0000 7f574b200640 1 -- 172.21.15.5:0/2688447314 >> 172.21.15.5:0/2688447314 conn(0x7f574c077440 msgr2=0x7f574c079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:40.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.570+0000 7f574b200640 1 -- 172.21.15.5:0/2688447314 shutdown_connections 2024-04-24T06:41:40.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.570+0000 7f574b200640 1 -- 172.21.15.5:0/2688447314 wait complete. 2024-04-24T06:41:40.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.571+0000 7f574b200640 1 Processor -- start 2024-04-24T06:41:40.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.572+0000 7f574b200640 1 -- start start 2024-04-24T06:41:40.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.572+0000 7f574b200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f574c108480 0x7f574c19e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:40.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.572+0000 7f574b200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f574c109670 0x7f574c19ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:40.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.572+0000 7f574b200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f574c10a030 0x7f574c1a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:40.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.572+0000 7f574b200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f574c1140e0 con 0x7f574c108480 2024-04-24T06:41:40.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.572+0000 7f574b200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f574c113f60 con 0x7f574c109670 2024-04-24T06:41:40.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.572+0000 7f574b200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f574c114260 con 0x7f574c10a030 2024-04-24T06:41:40.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.572+0000 7f5749400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f574c109670 0x7f574c19ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:40.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.572+0000 7f5749400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f574c109670 0x7f574c19ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:35780/0 (socket says 172.21.15.5:35780) 2024-04-24T06:41:40.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.572+0000 7f5749400640 1 -- 172.21.15.5:0/737206321 learned_addr learned my addr 172.21.15.5:0/737206321 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:41:40.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.572+0000 7f5749e00640 1 --2- 172.21.15.5:0/737206321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f574c108480 0x7f574c19e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:40.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.572+0000 7f574a800640 1 --2- 172.21.15.5:0/737206321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f574c10a030 0x7f574c1a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:40.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.573+0000 7f5749400640 1 -- 172.21.15.5:0/737206321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f574c10a030 msgr2=0x7f574c1a5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:40.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.573+0000 7f5749400640 1 --2- 172.21.15.5:0/737206321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f574c10a030 0x7f574c1a5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:40.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.573+0000 7f5749400640 1 -- 172.21.15.5:0/737206321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f574c108480 msgr2=0x7f574c19e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:40.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.573+0000 7f5749400640 1 --2- 172.21.15.5:0/737206321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f574c108480 0x7f574c19e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:40.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.573+0000 7f5749400640 1 -- 172.21.15.5:0/737206321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f574c1a6350 con 0x7f574c109670 2024-04-24T06:41:40.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.573+0000 7f574a800640 1 --2- 172.21.15.5:0/737206321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f574c10a030 0x7f574c1a5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:41:40.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.573+0000 7f5749e00640 1 --2- 172.21.15.5:0/737206321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f574c108480 0x7f574c19e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:41:40.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.573+0000 7f5749400640 1 --2- 172.21.15.5:0/737206321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f574c109670 0x7f574c19ec70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f573800e9f0 tx=0x7f573800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:40.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.573+0000 7f5742a00640 1 -- 172.21.15.5:0/737206321 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573800cd80 con 0x7f574c109670 2024-04-24T06:41:40.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.573+0000 7f5742a00640 1 -- 172.21.15.5:0/737206321 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5738010910 con 0x7f574c109670 2024-04-24T06:41:40.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.574+0000 7f5742a00640 1 -- 172.21.15.5:0/737206321 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5738018990 con 0x7f574c109670 2024-04-24T06:41:40.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.574+0000 7f574b200640 1 -- 172.21.15.5:0/737206321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f574c1a6640 con 0x7f574c109670 2024-04-24T06:41:40.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.574+0000 7f574b200640 1 -- 172.21.15.5:0/737206321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f574c1a6b40 con 0x7f574c109670 2024-04-24T06:41:40.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.575+0000 7f574b200640 1 -- 172.21.15.5:0/737206321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f574c10f7a0 con 0x7f574c109670 2024-04-24T06:41:40.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.577+0000 7f5742a00640 1 -- 172.21.15.5:0/737206321 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f5738010ac0 con 0x7f574c109670 2024-04-24T06:41:40.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.578+0000 7f5742a00640 1 --2- 172.21.15.5:0/737206321 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f57300a3bb0 0x7f57300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:40.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.578+0000 7f5742a00640 1 -- 172.21.15.5:0/737206321 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f57380d0670 con 0x7f574c109670 2024-04-24T06:41:40.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.578+0000 7f5749e00640 1 --2- 172.21.15.5:0/737206321 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f57300a3bb0 0x7f57300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:40.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.579+0000 7f5749e00640 1 --2- 172.21.15.5:0/737206321 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f57300a3bb0 0x7f57300a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f574c1094d0 tx=0x7f573c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:40.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.582+0000 7f5742a00640 1 -- 172.21.15.5:0/737206321 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f573809cab0 con 0x7f574c109670 2024-04-24T06:41:40.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:40 smithi137 ceph-mon[33017]: pgmap v315: 97 pgs: 2 creating+peering, 95 active+clean; 580 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s 2024-04-24T06:41:40.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:40 smithi137 ceph-mon[33017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-24T06:41:40.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:40 smithi137 ceph-mon[33017]: mds.? [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] up:active 2024-04-24T06:41:40.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:40 smithi137 ceph-mon[33017]: daemon mds.g assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-24T06:41:40.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:40 smithi137 ceph-mon[33017]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-24T06:41:40.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:40 smithi137 ceph-mon[33017]: Cluster is now healthy 2024-04-24T06:41:40.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:40 smithi137 ceph-mon[33017]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-24T06:41:40.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:40 smithi137 ceph-mon[33017]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=g=up:creating} 9 up:standby 2024-04-24T06:41:40.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:40 smithi137 ceph-mon[33017]: daemon mds.g is now active in filesystem cephfs as rank 2 2024-04-24T06:41:40.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:40 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/1099844948' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-24T06:41:40.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:40 smithi137 ceph-mon[33017]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-24T06:41:40.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:40 smithi005 ceph-mon[27798]: pgmap v315: 97 pgs: 2 creating+peering, 95 active+clean; 580 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s 2024-04-24T06:41:40.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:40 smithi005 ceph-mon[27798]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-24T06:41:40.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:40 smithi005 ceph-mon[27798]: mds.? [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] up:active 2024-04-24T06:41:40.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:40 smithi005 ceph-mon[27798]: daemon mds.g assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-24T06:41:40.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:40 smithi005 ceph-mon[27798]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-24T06:41:40.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:40 smithi005 ceph-mon[27798]: Cluster is now healthy 2024-04-24T06:41:40.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:40 smithi005 ceph-mon[27798]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-24T06:41:40.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:40 smithi005 ceph-mon[27798]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=g=up:creating} 9 up:standby 2024-04-24T06:41:40.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:40 smithi005 ceph-mon[27798]: daemon mds.g is now active in filesystem cephfs as rank 2 2024-04-24T06:41:40.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:40 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1099844948' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-24T06:41:40.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:40 smithi005 ceph-mon[27798]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-24T06:41:40.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:40 smithi037 ceph-mon[32864]: pgmap v315: 97 pgs: 2 creating+peering, 95 active+clean; 580 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s 2024-04-24T06:41:40.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:40 smithi037 ceph-mon[32864]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-24T06:41:40.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:40 smithi037 ceph-mon[32864]: mds.? [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] up:active 2024-04-24T06:41:40.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:40 smithi037 ceph-mon[32864]: daemon mds.g assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-24T06:41:40.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:40 smithi037 ceph-mon[32864]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-24T06:41:40.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:40 smithi037 ceph-mon[32864]: Cluster is now healthy 2024-04-24T06:41:40.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:40 smithi037 ceph-mon[32864]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-24T06:41:40.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:40 smithi037 ceph-mon[32864]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=g=up:creating} 9 up:standby 2024-04-24T06:41:40.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:40 smithi037 ceph-mon[32864]: daemon mds.g is now active in filesystem cephfs as rank 2 2024-04-24T06:41:40.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:40 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/1099844948' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-24T06:41:40.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:40 smithi037 ceph-mon[32864]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-24T06:41:40.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.816+0000 7f574b200640 1 -- 172.21.15.5:0/737206321 --> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f574c062440 con 0x7f57300a3bb0 2024-04-24T06:41:40.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.847+0000 7f5742a00640 1 -- 172.21.15.5:0/737206321 <== mgr.14152 v2:172.21.15.5:6800/4042936965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f574c062440 con 0x7f57300a3bb0 2024-04-24T06:41:40.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.849+0000 7f574b200640 1 -- 172.21.15.5:0/737206321 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f57300a3bb0 msgr2=0x7f57300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:40.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.849+0000 7f574b200640 1 --2- 172.21.15.5:0/737206321 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f57300a3bb0 0x7f57300a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f574c1094d0 tx=0x7f573c005f90 comp rx=0 tx=0).stop 2024-04-24T06:41:40.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.849+0000 7f574b200640 1 -- 172.21.15.5:0/737206321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f574c109670 msgr2=0x7f574c19ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:40.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.849+0000 7f574b200640 1 --2- 172.21.15.5:0/737206321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f574c109670 0x7f574c19ec70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f573800e9f0 tx=0x7f573800eeb0 comp rx=0 tx=0).stop 2024-04-24T06:41:40.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.850+0000 7f574b200640 1 -- 172.21.15.5:0/737206321 shutdown_connections 2024-04-24T06:41:40.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.850+0000 7f574b200640 1 --2- 172.21.15.5:0/737206321 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f57300a3bb0 0x7f57300a6070 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:40.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.850+0000 7f574b200640 1 --2- 172.21.15.5:0/737206321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f574c10a030 0x7f574c1a5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:40.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.850+0000 7f574b200640 1 --2- 172.21.15.5:0/737206321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f574c109670 0x7f574c19ec70 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:40.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.850+0000 7f574b200640 1 --2- 172.21.15.5:0/737206321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f574c108480 0x7f574c19e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:40.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.850+0000 7f574b200640 1 -- 172.21.15.5:0/737206321 >> 172.21.15.5:0/737206321 conn(0x7f574c077440 msgr2=0x7f574c10c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:40.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.850+0000 7f574b200640 1 -- 172.21.15.5:0/737206321 shutdown_connections 2024-04-24T06:41:40.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:40.850+0000 7f574b200640 1 -- 172.21.15.5:0/737206321 wait complete. 2024-04-24T06:41:40.873 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-24T06:41:41.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.078+0000 7f0e31a00640 1 -- 172.21.15.5:0/320533047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e2c105c30 msgr2=0x7f0e2c10c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:41.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.078+0000 7f0e31a00640 1 --2- 172.21.15.5:0/320533047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e2c105c30 0x7f0e2c10c4c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0e24009ca0 tx=0x7f0e2402a8c0 comp rx=0 tx=0).stop 2024-04-24T06:41:41.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.078+0000 7f0e31a00640 1 -- 172.21.15.5:0/320533047 shutdown_connections 2024-04-24T06:41:41.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.078+0000 7f0e31a00640 1 --2- 172.21.15.5:0/320533047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e2c105c30 0x7f0e2c10c4c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:41.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.078+0000 7f0e31a00640 1 --2- 172.21.15.5:0/320533047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e2c105270 0x7f0e2c1056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:41.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.078+0000 7f0e31a00640 1 --2- 172.21.15.5:0/320533047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e2c104080 0x7f0e2c1044a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:41.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.078+0000 7f0e31a00640 1 -- 172.21.15.5:0/320533047 >> 172.21.15.5:0/320533047 conn(0x7f0e2c0ff610 msgr2=0x7f0e2c101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:41.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.078+0000 7f0e31a00640 1 -- 172.21.15.5:0/320533047 shutdown_connections 2024-04-24T06:41:41.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.078+0000 7f0e31a00640 1 -- 172.21.15.5:0/320533047 wait complete. 2024-04-24T06:41:41.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.079+0000 7f0e31a00640 1 Processor -- start 2024-04-24T06:41:41.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.079+0000 7f0e31a00640 1 -- start start 2024-04-24T06:41:41.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.080+0000 7f0e31a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e2c104080 0x7f0e2c19a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:41.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.080+0000 7f0e31a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e2c105270 0x7f0e2c19a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:41.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.080+0000 7f0e31a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e2c105c30 0x7f0e2c1a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:41.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.080+0000 7f0e31a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2c10fd20 con 0x7f0e2c105270 2024-04-24T06:41:41.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.080+0000 7f0e31a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2c10fba0 con 0x7f0e2c104080 2024-04-24T06:41:41.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.080+0000 7f0e31a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2c10fea0 con 0x7f0e2c105c30 2024-04-24T06:41:41.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.080+0000 7f0e2be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e2c104080 0x7f0e2c19a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:41.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.080+0000 7f0e23e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e2c105270 0x7f0e2c19a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:41.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.080+0000 7f0e2be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e2c104080 0x7f0e2c19a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:35808/0 (socket says 172.21.15.5:35808) 2024-04-24T06:41:41.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.080+0000 7f0e2be00640 1 -- 172.21.15.5:0/1592101622 learned_addr learned my addr 172.21.15.5:0/1592101622 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:41:41.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.080+0000 7f0e23e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e2c105270 0x7f0e2c19a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60738/0 (socket says 172.21.15.5:60738) 2024-04-24T06:41:41.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.080+0000 7f0e31000640 1 --2- 172.21.15.5:0/1592101622 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e2c105c30 0x7f0e2c1a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:41.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.081+0000 7f0e2be00640 1 -- 172.21.15.5:0/1592101622 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e2c105c30 msgr2=0x7f0e2c1a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:41.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.081+0000 7f0e2be00640 1 --2- 172.21.15.5:0/1592101622 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e2c105c30 0x7f0e2c1a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:41.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.081+0000 7f0e2be00640 1 -- 172.21.15.5:0/1592101622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e2c105270 msgr2=0x7f0e2c19a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:41.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.081+0000 7f0e2be00640 1 --2- 172.21.15.5:0/1592101622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e2c105270 0x7f0e2c19a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:41.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.081+0000 7f0e2be00640 1 -- 172.21.15.5:0/1592101622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e2c1a1f20 con 0x7f0e2c104080 2024-04-24T06:41:41.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.081+0000 7f0e23e00640 1 --2- 172.21.15.5:0/1592101622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e2c105270 0x7f0e2c19a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:41:41.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.081+0000 7f0e2be00640 1 --2- 172.21.15.5:0/1592101622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e2c104080 0x7f0e2c19a300 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c00e9c0 tx=0x7f0e1c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:41.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.081+0000 7f0e29600640 1 -- 172.21.15.5:0/1592101622 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c00cd80 con 0x7f0e2c104080 2024-04-24T06:41:41.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.081+0000 7f0e31a00640 1 -- 172.21.15.5:0/1592101622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e2c1a2210 con 0x7f0e2c104080 2024-04-24T06:41:41.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.081+0000 7f0e29600640 1 -- 172.21.15.5:0/1592101622 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e1c0035f0 con 0x7f0e2c104080 2024-04-24T06:41:41.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.081+0000 7f0e29600640 1 -- 172.21.15.5:0/1592101622 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c010690 con 0x7f0e2c104080 2024-04-24T06:41:41.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.081+0000 7f0e31a00640 1 -- 172.21.15.5:0/1592101622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e2c1a2760 con 0x7f0e2c104080 2024-04-24T06:41:41.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.083+0000 7f0e31a00640 1 -- 172.21.15.5:0/1592101622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e2c10b340 con 0x7f0e2c104080 2024-04-24T06:41:41.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.085+0000 7f0e29600640 1 -- 172.21.15.5:0/1592101622 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f0e1c0037a0 con 0x7f0e2c104080 2024-04-24T06:41:41.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.087+0000 7f0e29600640 1 --2- 172.21.15.5:0/1592101622 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f0e100a3bb0 0x7f0e100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:41.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.087+0000 7f0e29600640 1 -- 172.21.15.5:0/1592101622 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f0e1c0d0730 con 0x7f0e2c104080 2024-04-24T06:41:41.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.087+0000 7f0e23e00640 1 --2- 172.21.15.5:0/1592101622 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f0e100a3bb0 0x7f0e100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:41.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.088+0000 7f0e23e00640 1 --2- 172.21.15.5:0/1592101622 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f0e100a3bb0 0x7f0e100a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c19b810 tx=0x7f0e14005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:41.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.091+0000 7f0e29600640 1 -- 172.21.15.5:0/1592101622 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0e1c09d1c0 con 0x7f0e2c104080 2024-04-24T06:41:41.257 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:41:40 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:41:40.829+0000 7f59cc600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T06:41:41.257 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:41:40 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:41:40.829+0000 7f59cc600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T06:41:41.258 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:41:40 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:41:40.829+0000 7f59cc600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T06:41:41.258 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:41:40 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:41:40.829+0000 7f59cc600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T06:41:41.258 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:41:40 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:41:40.829+0000 7f59cc600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T06:41:41.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.334+0000 7f0e31a00640 1 -- 172.21.15.5:0/1592101622 --> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f0e2c062440 con 0x7f0e100a3bb0 2024-04-24T06:41:41.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.371+0000 7f0e29600640 1 -- 172.21.15.5:0/1592101622 <== mgr.14152 v2:172.21.15.5:6800/4042936965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0e2c062440 con 0x7f0e100a3bb0 2024-04-24T06:41:41.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.373+0000 7f0e31a00640 1 -- 172.21.15.5:0/1592101622 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f0e100a3bb0 msgr2=0x7f0e100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:41.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.373+0000 7f0e31a00640 1 --2- 172.21.15.5:0/1592101622 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f0e100a3bb0 0x7f0e100a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c19b810 tx=0x7f0e14005f90 comp rx=0 tx=0).stop 2024-04-24T06:41:41.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.373+0000 7f0e31a00640 1 -- 172.21.15.5:0/1592101622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e2c104080 msgr2=0x7f0e2c19a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:41.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.373+0000 7f0e31a00640 1 --2- 172.21.15.5:0/1592101622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e2c104080 0x7f0e2c19a300 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c00e9c0 tx=0x7f0e1c00ee80 comp rx=0 tx=0).stop 2024-04-24T06:41:41.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.374+0000 7f0e31a00640 1 -- 172.21.15.5:0/1592101622 shutdown_connections 2024-04-24T06:41:41.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.374+0000 7f0e31a00640 1 --2- 172.21.15.5:0/1592101622 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f0e100a3bb0 0x7f0e100a6070 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:41.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.374+0000 7f0e31a00640 1 --2- 172.21.15.5:0/1592101622 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e2c105c30 0x7f0e2c1a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:41.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.374+0000 7f0e31a00640 1 --2- 172.21.15.5:0/1592101622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e2c105270 0x7f0e2c19a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:41.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.374+0000 7f0e31a00640 1 --2- 172.21.15.5:0/1592101622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e2c104080 0x7f0e2c19a300 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:41.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.374+0000 7f0e31a00640 1 -- 172.21.15.5:0/1592101622 >> 172.21.15.5:0/1592101622 conn(0x7f0e2c0ff610 msgr2=0x7f0e2c108510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:41.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.374+0000 7f0e31a00640 1 -- 172.21.15.5:0/1592101622 shutdown_connections 2024-04-24T06:41:41.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.374+0000 7f0e31a00640 1 -- 172.21.15.5:0/1592101622 wait complete. 2024-04-24T06:41:41.394 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-24T06:41:41.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.602+0000 7f33bb200640 1 -- 172.21.15.5:0/2307069295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33bc1028f0 msgr2=0x7f33bc104ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:41.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.602+0000 7f33bb200640 1 --2- 172.21.15.5:0/2307069295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33bc1028f0 0x7f33bc104ce0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f33ac009900 tx=0x7f33ac02a3f0 comp rx=0 tx=0).stop 2024-04-24T06:41:41.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.602+0000 7f33bb200640 1 -- 172.21.15.5:0/2307069295 shutdown_connections 2024-04-24T06:41:41.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.602+0000 7f33bb200640 1 --2- 172.21.15.5:0/2307069295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33bc107b50 0x7f33bc109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:41.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.602+0000 7f33bb200640 1 --2- 172.21.15.5:0/2307069295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33bc105220 0x7f33bc107610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:41.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.602+0000 7f33bb200640 1 --2- 172.21.15.5:0/2307069295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33bc1028f0 0x7f33bc104ce0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:41.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.602+0000 7f33bb200640 1 -- 172.21.15.5:0/2307069295 >> 172.21.15.5:0/2307069295 conn(0x7f33bc0fc2e0 msgr2=0x7f33bc0fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:41.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.602+0000 7f33bb200640 1 -- 172.21.15.5:0/2307069295 shutdown_connections 2024-04-24T06:41:41.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.602+0000 7f33bb200640 1 -- 172.21.15.5:0/2307069295 wait complete. 2024-04-24T06:41:41.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.603+0000 7f33bb200640 1 Processor -- start 2024-04-24T06:41:41.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.603+0000 7f33bb200640 1 -- start start 2024-04-24T06:41:41.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.604+0000 7f33bb200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33bc1028f0 0x7f33bc19a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:41.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.604+0000 7f33bb200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33bc105220 0x7f33bc19a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:41.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.604+0000 7f33bb200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33bc107b50 0x7f33bc1a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:41.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.604+0000 7f33bb200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc10db20 con 0x7f33bc105220 2024-04-24T06:41:41.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.604+0000 7f33bb200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc10d9a0 con 0x7f33bc1028f0 2024-04-24T06:41:41.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.604+0000 7f33bb200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc10dca0 con 0x7f33bc107b50 2024-04-24T06:41:41.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.604+0000 7f33b1400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33bc105220 0x7f33bc19a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:41.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.604+0000 7f33b1400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33bc105220 0x7f33bc19a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60758/0 (socket says 172.21.15.5:60758) 2024-04-24T06:41:41.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.604+0000 7f33b1400640 1 -- 172.21.15.5:0/1520262081 learned_addr learned my addr 172.21.15.5:0/1520262081 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:41:41.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.604+0000 7f33b9e00640 1 --2- 172.21.15.5:0/1520262081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33bc1028f0 0x7f33bc19a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:41.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.604+0000 7f33ba800640 1 --2- 172.21.15.5:0/1520262081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33bc107b50 0x7f33bc1a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:41.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.604+0000 7f33b1400640 1 -- 172.21.15.5:0/1520262081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33bc107b50 msgr2=0x7f33bc1a1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:41.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.604+0000 7f33b1400640 1 --2- 172.21.15.5:0/1520262081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33bc107b50 0x7f33bc1a1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:41.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.604+0000 7f33b1400640 1 -- 172.21.15.5:0/1520262081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33bc1028f0 msgr2=0x7f33bc19a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:41.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.604+0000 7f33b1400640 1 --2- 172.21.15.5:0/1520262081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33bc1028f0 0x7f33bc19a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.604+0000 7f33b1400640 1 -- 172.21.15.5:0/1520262081 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33bc1a1ff0 con 0x7f33bc105220 2024-04-24T06:41:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.605+0000 7f33b1400640 1 --2- 172.21.15.5:0/1520262081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33bc105220 0x7f33bc19a910 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f33a400e9f0 tx=0x7f33a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.605+0000 7f33b3400640 1 -- 172.21.15.5:0/1520262081 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a400ccc0 con 0x7f33bc105220 2024-04-24T06:41:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.605+0000 7f33b3400640 1 -- 172.21.15.5:0/1520262081 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33a4005500 con 0x7f33bc105220 2024-04-24T06:41:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.605+0000 7f33bb200640 1 -- 172.21.15.5:0/1520262081 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33bc1a22e0 con 0x7f33bc105220 2024-04-24T06:41:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.605+0000 7f33b3400640 1 -- 172.21.15.5:0/1520262081 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a4010690 con 0x7f33bc105220 2024-04-24T06:41:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.605+0000 7f33bb200640 1 -- 172.21.15.5:0/1520262081 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33bc1a27e0 con 0x7f33bc105220 2024-04-24T06:41:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.606+0000 7f33bb200640 1 -- 172.21.15.5:0/1520262081 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33bc1096a0 con 0x7f33bc105220 2024-04-24T06:41:41.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.608+0000 7f33b3400640 1 -- 172.21.15.5:0/1520262081 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f33a40046c0 con 0x7f33bc105220 2024-04-24T06:41:41.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.610+0000 7f33b3400640 1 --2- 172.21.15.5:0/1520262081 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f33a00a3bb0 0x7f33a00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:41.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.610+0000 7f33b3400640 1 -- 172.21.15.5:0/1520262081 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f33a40d0740 con 0x7f33bc105220 2024-04-24T06:41:41.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.610+0000 7f33b9e00640 1 --2- 172.21.15.5:0/1520262081 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f33a00a3bb0 0x7f33a00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:41.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.611+0000 7f33b9e00640 1 --2- 172.21.15.5:0/1520262081 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f33a00a3bb0 0x7f33a00a6070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f33ac0314e0 tx=0x7f33ac000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:41.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.615+0000 7f33b3400640 1 -- 172.21.15.5:0/1520262081 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f33a409d1d0 con 0x7f33bc105220 2024-04-24T06:41:41.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:41 smithi137 ceph-mon[33017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-24T06:41:41.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:41 smithi137 ceph-mon[33017]: mds.? [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] up:active 2024-04-24T06:41:41.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:41 smithi137 ceph-mon[33017]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=g=up:active} 9 up:standby 2024-04-24T06:41:41.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:41 smithi137 ceph-mon[33017]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T06:41:41.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:41 smithi137 ceph-mon[33017]: from='client.24653 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:41:41.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:41 smithi005 ceph-mon[27798]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-24T06:41:41.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:41 smithi005 ceph-mon[27798]: mds.? [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] up:active 2024-04-24T06:41:41.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:41 smithi005 ceph-mon[27798]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=g=up:active} 9 up:standby 2024-04-24T06:41:41.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:41 smithi005 ceph-mon[27798]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T06:41:41.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:41 smithi005 ceph-mon[27798]: from='client.24653 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:41:41.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:41 smithi037 ceph-mon[32864]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-24T06:41:41.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:41 smithi037 ceph-mon[32864]: mds.? [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] up:active 2024-04-24T06:41:41.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:41 smithi037 ceph-mon[32864]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=g=up:active} 9 up:standby 2024-04-24T06:41:41.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:41 smithi037 ceph-mon[32864]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T06:41:41.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:41 smithi037 ceph-mon[32864]: from='client.24653 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:41:41.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.850+0000 7f33bb200640 1 -- 172.21.15.5:0/1520262081 --> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f33bc062440 con 0x7f33a00a3bb0 2024-04-24T06:41:41.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.863+0000 7f33b3400640 1 -- 172.21.15.5:0/1520262081 <== mgr.14152 v2:172.21.15.5:6800/4042936965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f33bc062440 con 0x7f33a00a3bb0 2024-04-24T06:41:41.865 INFO:teuthology.orchestra.run.smithi005.stdout:/volumes/qa/sv_0/e4ecbbf6-3e7e-4157-8345-c524cc801569 2024-04-24T06:41:41.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.866+0000 7f33bb200640 1 -- 172.21.15.5:0/1520262081 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f33a00a3bb0 msgr2=0x7f33a00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:41.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.866+0000 7f33bb200640 1 --2- 172.21.15.5:0/1520262081 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f33a00a3bb0 0x7f33a00a6070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f33ac0314e0 tx=0x7f33ac000bc0 comp rx=0 tx=0).stop 2024-04-24T06:41:41.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.866+0000 7f33bb200640 1 -- 172.21.15.5:0/1520262081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33bc105220 msgr2=0x7f33bc19a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:41.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.866+0000 7f33bb200640 1 --2- 172.21.15.5:0/1520262081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33bc105220 0x7f33bc19a910 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f33a400e9f0 tx=0x7f33a400eeb0 comp rx=0 tx=0).stop 2024-04-24T06:41:41.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.866+0000 7f33bb200640 1 -- 172.21.15.5:0/1520262081 shutdown_connections 2024-04-24T06:41:41.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.866+0000 7f33bb200640 1 --2- 172.21.15.5:0/1520262081 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f33a00a3bb0 0x7f33a00a6070 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:41.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.866+0000 7f33bb200640 1 --2- 172.21.15.5:0/1520262081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33bc107b50 0x7f33bc1a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:41.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.866+0000 7f33bb200640 1 --2- 172.21.15.5:0/1520262081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33bc105220 0x7f33bc19a910 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:41.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.866+0000 7f33bb200640 1 --2- 172.21.15.5:0/1520262081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33bc1028f0 0x7f33bc19a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:41.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.866+0000 7f33bb200640 1 -- 172.21.15.5:0/1520262081 >> 172.21.15.5:0/1520262081 conn(0x7f33bc0fc2e0 msgr2=0x7f33bc1020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:41.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.867+0000 7f33bb200640 1 -- 172.21.15.5:0/1520262081 shutdown_connections 2024-04-24T06:41:41.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:41.867+0000 7f33bb200640 1 -- 172.21.15.5:0/1520262081 wait complete. 2024-04-24T06:41:41.888 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-24T06:41:42.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.109+0000 7f5ecfe00640 1 -- 172.21.15.5:0/3588825318 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ed00785a0 msgr2=0x7f5ed0078e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:42.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.109+0000 7f5ecfe00640 1 --2- 172.21.15.5:0/3588825318 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ed00785a0 0x7f5ed0078e40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5ec8009ca0 tx=0x7f5ec802a8c0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.109+0000 7f5ecfe00640 1 -- 172.21.15.5:0/3588825318 shutdown_connections 2024-04-24T06:41:42.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.109+0000 7f5ecfe00640 1 --2- 172.21.15.5:0/3588825318 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ed00785a0 0x7f5ed0078e40 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.109+0000 7f5ecfe00640 1 --2- 172.21.15.5:0/3588825318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ed0077c00 0x7f5ed0078060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.109+0000 7f5ecfe00640 1 --2- 172.21.15.5:0/3588825318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ed00769c0 0x7f5ed0076de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.109+0000 7f5ecfe00640 1 -- 172.21.15.5:0/3588825318 >> 172.21.15.5:0/3588825318 conn(0x7f5ed00ff610 msgr2=0x7f5ed0101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:42.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.110+0000 7f5ecfe00640 1 -- 172.21.15.5:0/3588825318 shutdown_connections 2024-04-24T06:41:42.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.110+0000 7f5ecfe00640 1 -- 172.21.15.5:0/3588825318 wait complete. 2024-04-24T06:41:42.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.111+0000 7f5ecfe00640 1 Processor -- start 2024-04-24T06:41:42.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.111+0000 7f5ecfe00640 1 -- start start 2024-04-24T06:41:42.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.112+0000 7f5ecfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ed00769c0 0x7f5ed019e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:42.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.112+0000 7f5ecfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ed0077c00 0x7f5ed019ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:42.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.112+0000 7f5ecfe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ed00785a0 0x7f5ed01a5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:42.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.112+0000 7f5ecfe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed0114150 con 0x7f5ed00769c0 2024-04-24T06:41:42.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.112+0000 7f5ecfe00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed0113fd0 con 0x7f5ed0077c00 2024-04-24T06:41:42.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.112+0000 7f5ecfe00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed01142d0 con 0x7f5ed00785a0 2024-04-24T06:41:42.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.112+0000 7f5ecea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ed00769c0 0x7f5ed019e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:42.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.112+0000 7f5ecea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ed00769c0 0x7f5ed019e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60774/0 (socket says 172.21.15.5:60774) 2024-04-24T06:41:42.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.112+0000 7f5ecea00640 1 -- 172.21.15.5:0/2524261044 learned_addr learned my addr 172.21.15.5:0/2524261044 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:41:42.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.112+0000 7f5ecf400640 1 --2- 172.21.15.5:0/2524261044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ed00785a0 0x7f5ed01a5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:42.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.112+0000 7f5ece000640 1 --2- 172.21.15.5:0/2524261044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ed0077c00 0x7f5ed019ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:42.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.112+0000 7f5ecea00640 1 -- 172.21.15.5:0/2524261044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ed00785a0 msgr2=0x7f5ed01a5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:42.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.112+0000 7f5ecea00640 1 --2- 172.21.15.5:0/2524261044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ed00785a0 0x7f5ed01a5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.112+0000 7f5ecea00640 1 -- 172.21.15.5:0/2524261044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ed0077c00 msgr2=0x7f5ed019ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:42.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.112+0000 7f5ecea00640 1 --2- 172.21.15.5:0/2524261044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ed0077c00 0x7f5ed019ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.112+0000 7f5ecea00640 1 -- 172.21.15.5:0/2524261044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed01a6390 con 0x7f5ed00769c0 2024-04-24T06:41:42.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.113+0000 7f5ecea00640 1 --2- 172.21.15.5:0/2524261044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ed00769c0 0x7f5ed019e770 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5ec000ea60 tx=0x7f5ec000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:42.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.113+0000 7f5ec7400640 1 -- 172.21.15.5:0/2524261044 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec000cde0 con 0x7f5ed00769c0 2024-04-24T06:41:42.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.113+0000 7f5ec7400640 1 -- 172.21.15.5:0/2524261044 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ec00035f0 con 0x7f5ed00769c0 2024-04-24T06:41:42.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.113+0000 7f5ecfe00640 1 -- 172.21.15.5:0/2524261044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ed01a6680 con 0x7f5ed00769c0 2024-04-24T06:41:42.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.113+0000 7f5ec7400640 1 -- 172.21.15.5:0/2524261044 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0010670 con 0x7f5ed00769c0 2024-04-24T06:41:42.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.113+0000 7f5ecfe00640 1 -- 172.21.15.5:0/2524261044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ed01a6bd0 con 0x7f5ed00769c0 2024-04-24T06:41:42.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.114+0000 7f5ecfe00640 1 -- 172.21.15.5:0/2524261044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ed01109f0 con 0x7f5ed00769c0 2024-04-24T06:41:42.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.116+0000 7f5ec7400640 1 -- 172.21.15.5:0/2524261044 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f5ec00037a0 con 0x7f5ed00769c0 2024-04-24T06:41:42.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.118+0000 7f5ec7400640 1 --2- 172.21.15.5:0/2524261044 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f5eb40a3bb0 0x7f5eb40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:42.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.118+0000 7f5ec7400640 1 -- 172.21.15.5:0/2524261044 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f5ec00d0640 con 0x7f5ed00769c0 2024-04-24T06:41:42.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.118+0000 7f5ece000640 1 --2- 172.21.15.5:0/2524261044 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f5eb40a3bb0 0x7f5eb40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:42.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.119+0000 7f5ece000640 1 --2- 172.21.15.5:0/2524261044 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f5eb40a3bb0 0x7f5eb40a6070 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5ed019fc80 tx=0x7f5ebc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:42.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.122+0000 7f5ec7400640 1 -- 172.21.15.5:0/2524261044 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5ec009d0d0 con 0x7f5ed00769c0 2024-04-24T06:41:42.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.366+0000 7f5ec7400640 1 -- 172.21.15.5:0/2524261044 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f5ec0099040 con 0x7f5ed00769c0 2024-04-24T06:41:42.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.373+0000 7f5ecfe00640 1 -- 172.21.15.5:0/2524261044 --> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f5ed0062440 con 0x7f5eb40a3bb0 2024-04-24T06:41:42.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.417+0000 7f5ec7400640 1 -- 172.21.15.5:0/2524261044 <== mgr.14152 v2:172.21.15.5:6800/4042936965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5ed0062440 con 0x7f5eb40a3bb0 2024-04-24T06:41:42.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.419+0000 7f5ecfe00640 1 -- 172.21.15.5:0/2524261044 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f5eb40a3bb0 msgr2=0x7f5eb40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:42.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.419+0000 7f5ecfe00640 1 --2- 172.21.15.5:0/2524261044 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f5eb40a3bb0 0x7f5eb40a6070 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5ed019fc80 tx=0x7f5ebc005f90 comp rx=0 tx=0).stop 2024-04-24T06:41:42.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.419+0000 7f5ecfe00640 1 -- 172.21.15.5:0/2524261044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ed00769c0 msgr2=0x7f5ed019e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:42.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.419+0000 7f5ecfe00640 1 --2- 172.21.15.5:0/2524261044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ed00769c0 0x7f5ed019e770 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5ec000ea60 tx=0x7f5ec000ef20 comp rx=0 tx=0).stop 2024-04-24T06:41:42.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.420+0000 7f5ecfe00640 1 -- 172.21.15.5:0/2524261044 shutdown_connections 2024-04-24T06:41:42.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.420+0000 7f5ecfe00640 1 --2- 172.21.15.5:0/2524261044 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f5eb40a3bb0 0x7f5eb40a6070 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.420+0000 7f5ecfe00640 1 --2- 172.21.15.5:0/2524261044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ed00785a0 0x7f5ed01a5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.420+0000 7f5ecfe00640 1 --2- 172.21.15.5:0/2524261044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ed0077c00 0x7f5ed019ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.420+0000 7f5ecfe00640 1 --2- 172.21.15.5:0/2524261044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ed00769c0 0x7f5ed019e770 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.420+0000 7f5ecfe00640 1 -- 172.21.15.5:0/2524261044 >> 172.21.15.5:0/2524261044 conn(0x7f5ed00ff610 msgr2=0x7f5ed0101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:42.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.420+0000 7f5ecfe00640 1 -- 172.21.15.5:0/2524261044 shutdown_connections 2024-04-24T06:41:42.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.420+0000 7f5ecfe00640 1 -- 172.21.15.5:0/2524261044 wait complete. 2024-04-24T06:41:42.442 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-24T06:41:42.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.654+0000 7fc216000640 1 -- 172.21.15.5:0/956138175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2100785a0 msgr2=0x7fc210078e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:42.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.654+0000 7fc216000640 1 --2- 172.21.15.5:0/956138175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2100785a0 0x7fc210078e40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc208009900 tx=0x7fc20802a3f0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.654+0000 7fc216000640 1 -- 172.21.15.5:0/956138175 shutdown_connections 2024-04-24T06:41:42.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.654+0000 7fc216000640 1 --2- 172.21.15.5:0/956138175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2100785a0 0x7fc210078e40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.654+0000 7fc216000640 1 --2- 172.21.15.5:0/956138175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc210077c00 0x7fc210078060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.654+0000 7fc216000640 1 --2- 172.21.15.5:0/956138175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2100769c0 0x7fc210076de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.654+0000 7fc216000640 1 -- 172.21.15.5:0/956138175 >> 172.21.15.5:0/956138175 conn(0x7fc2100ff610 msgr2=0x7fc210101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:42.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.655+0000 7fc216000640 1 -- 172.21.15.5:0/956138175 shutdown_connections 2024-04-24T06:41:42.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.655+0000 7fc216000640 1 -- 172.21.15.5:0/956138175 wait complete. 2024-04-24T06:41:42.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.656+0000 7fc216000640 1 Processor -- start 2024-04-24T06:41:42.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.656+0000 7fc216000640 1 -- start start 2024-04-24T06:41:42.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.657+0000 7fc216000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2100769c0 0x7fc21019e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:42.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.657+0000 7fc216000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc210077c00 0x7fc21019ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:42.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.657+0000 7fc216000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2100785a0 0x7fc2101a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:42.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.657+0000 7fc216000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc210114170 con 0x7fc210077c00 2024-04-24T06:41:42.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.657+0000 7fc216000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc210113ff0 con 0x7fc2100769c0 2024-04-24T06:41:42.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.657+0000 7fc216000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc2101142f0 con 0x7fc2100785a0 2024-04-24T06:41:42.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.657+0000 7fc20fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc210077c00 0x7fc21019ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:42.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.657+0000 7fc214c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2100769c0 0x7fc21019e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:42.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.657+0000 7fc215600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2100785a0 0x7fc2101a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:42.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.657+0000 7fc20fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc210077c00 0x7fc21019ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60802/0 (socket says 172.21.15.5:60802) 2024-04-24T06:41:42.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.657+0000 7fc214c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2100769c0 0x7fc21019e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:35862/0 (socket says 172.21.15.5:35862) 2024-04-24T06:41:42.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.657+0000 7fc20fe00640 1 -- 172.21.15.5:0/2111831479 learned_addr learned my addr 172.21.15.5:0/2111831479 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:41:42.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.657+0000 7fc20fe00640 1 -- 172.21.15.5:0/2111831479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2100785a0 msgr2=0x7fc2101a5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:42.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.657+0000 7fc20fe00640 1 --2- 172.21.15.5:0/2111831479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2100785a0 0x7fc2101a5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.657+0000 7fc20fe00640 1 -- 172.21.15.5:0/2111831479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2100769c0 msgr2=0x7fc21019e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:42.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.657+0000 7fc20fe00640 1 --2- 172.21.15.5:0/2111831479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2100769c0 0x7fc21019e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.657+0000 7fc20fe00640 1 -- 172.21.15.5:0/2111831479 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2101a6330 con 0x7fc210077c00 2024-04-24T06:41:42.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.657+0000 7fc214c00640 1 --2- 172.21.15.5:0/2111831479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2100769c0 0x7fc21019e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:41:42.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.658+0000 7fc20fe00640 1 --2- 172.21.15.5:0/2111831479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc210077c00 0x7fc21019ec50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc20000ebc0 tx=0x7fc20000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:42.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.658+0000 7fc20d600640 1 -- 172.21.15.5:0/2111831479 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc200010630 con 0x7fc210077c00 2024-04-24T06:41:42.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.658+0000 7fc20d600640 1 -- 172.21.15.5:0/2111831479 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc200010cb0 con 0x7fc210077c00 2024-04-24T06:41:42.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.658+0000 7fc20d600640 1 -- 172.21.15.5:0/2111831479 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc200018bd0 con 0x7fc210077c00 2024-04-24T06:41:42.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.658+0000 7fc216000640 1 -- 172.21.15.5:0/2111831479 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2101a6620 con 0x7fc210077c00 2024-04-24T06:41:42.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.658+0000 7fc216000640 1 -- 172.21.15.5:0/2111831479 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2101a6b20 con 0x7fc210077c00 2024-04-24T06:41:42.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.659+0000 7fc216000640 1 -- 172.21.15.5:0/2111831479 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc210110a10 con 0x7fc210077c00 2024-04-24T06:41:42.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.661+0000 7fc20d600640 1 -- 172.21.15.5:0/2111831479 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7fc2000107d0 con 0x7fc210077c00 2024-04-24T06:41:42.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.663+0000 7fc20d600640 1 --2- 172.21.15.5:0/2111831479 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fc1f40a3cc0 0x7fc1f40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:42.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.663+0000 7fc20d600640 1 -- 172.21.15.5:0/2111831479 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fc2000d0a80 con 0x7fc210077c00 2024-04-24T06:41:42.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.663+0000 7fc214c00640 1 --2- 172.21.15.5:0/2111831479 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fc1f40a3cc0 0x7fc1f40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:42.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.664+0000 7fc214c00640 1 --2- 172.21.15.5:0/2111831479 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fc1f40a3cc0 0x7fc1f40a6180 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc1f80095f0 tx=0x7fc1f8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:42.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.668+0000 7fc20d600640 1 -- 172.21.15.5:0/2111831479 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc2000d2e20 con 0x7fc210077c00 2024-04-24T06:41:42.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:42 smithi137 ceph-mon[33017]: pgmap v316: 97 pgs: 97 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 145 B/s rd, 3.8 KiB/s wr, 10 op/s 2024-04-24T06:41:42.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:42 smithi137 ceph-mon[33017]: from='client.24665 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:41:42.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:42 smithi137 ceph-mon[33017]: from='client.14577 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:41:42.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:42 smithi005 ceph-mon[27798]: pgmap v316: 97 pgs: 97 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 145 B/s rd, 3.8 KiB/s wr, 10 op/s 2024-04-24T06:41:42.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:42 smithi005 ceph-mon[27798]: from='client.24665 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:41:42.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:42 smithi005 ceph-mon[27798]: from='client.14577 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:41:42.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:42 smithi037 ceph-mon[32864]: pgmap v316: 97 pgs: 97 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 145 B/s rd, 3.8 KiB/s wr, 10 op/s 2024-04-24T06:41:42.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:42 smithi037 ceph-mon[32864]: from='client.24665 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:41:42.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:42 smithi037 ceph-mon[32864]: from='client.14577 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:41:42.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.909+0000 7fc216000640 1 -- 172.21.15.5:0/2111831479 --> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fc2100624d0 con 0x7fc1f40a3cc0 2024-04-24T06:41:42.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.917+0000 7fc20d600640 1 -- 172.21.15.5:0/2111831479 <== mgr.14152 v2:172.21.15.5:6800/4042936965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fc2100624d0 con 0x7fc1f40a3cc0 2024-04-24T06:41:42.919 INFO:teuthology.orchestra.run.smithi005.stdout:/volumes/qa/sv_1/45532007-e9dd-4bc8-88b6-1eb4c5182393 2024-04-24T06:41:42.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.920+0000 7fc216000640 1 -- 172.21.15.5:0/2111831479 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fc1f40a3cc0 msgr2=0x7fc1f40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:42.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.920+0000 7fc216000640 1 --2- 172.21.15.5:0/2111831479 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fc1f40a3cc0 0x7fc1f40a6180 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc1f80095f0 tx=0x7fc1f8005f90 comp rx=0 tx=0).stop 2024-04-24T06:41:42.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.920+0000 7fc216000640 1 -- 172.21.15.5:0/2111831479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc210077c00 msgr2=0x7fc21019ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:42.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.920+0000 7fc216000640 1 --2- 172.21.15.5:0/2111831479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc210077c00 0x7fc21019ec50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc20000ebc0 tx=0x7fc20000c6a0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.920+0000 7fc216000640 1 -- 172.21.15.5:0/2111831479 shutdown_connections 2024-04-24T06:41:42.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.920+0000 7fc216000640 1 --2- 172.21.15.5:0/2111831479 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fc1f40a3cc0 0x7fc1f40a6180 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.920+0000 7fc216000640 1 --2- 172.21.15.5:0/2111831479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2100785a0 0x7fc2101a5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.920+0000 7fc216000640 1 --2- 172.21.15.5:0/2111831479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc210077c00 0x7fc21019ec50 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.920+0000 7fc216000640 1 --2- 172.21.15.5:0/2111831479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2100769c0 0x7fc21019e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:42.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.920+0000 7fc216000640 1 -- 172.21.15.5:0/2111831479 >> 172.21.15.5:0/2111831479 conn(0x7fc2100ff610 msgr2=0x7fc210101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:42.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.920+0000 7fc216000640 1 -- 172.21.15.5:0/2111831479 shutdown_connections 2024-04-24T06:41:42.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:42.920+0000 7fc216000640 1 -- 172.21.15.5:0/2111831479 wait complete. 2024-04-24T06:41:42.941 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:41:43.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.154+0000 7f648a800640 1 -- 172.21.15.5:0/514829324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6484077c00 msgr2=0x7f6484078060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:43.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.154+0000 7f648a800640 1 --2- 172.21.15.5:0/514829324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6484077c00 0x7f6484078060 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f646c009900 tx=0x7f646c02a3f0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.154+0000 7f648a800640 1 -- 172.21.15.5:0/514829324 shutdown_connections 2024-04-24T06:41:43.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.155+0000 7f648a800640 1 --2- 172.21.15.5:0/514829324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f64840785a0 0x7f6484078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.155+0000 7f648a800640 1 --2- 172.21.15.5:0/514829324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6484077c00 0x7f6484078060 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.155+0000 7f648a800640 1 --2- 172.21.15.5:0/514829324 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64840769c0 0x7f6484076de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.155+0000 7f648a800640 1 -- 172.21.15.5:0/514829324 >> 172.21.15.5:0/514829324 conn(0x7f64840ff880 msgr2=0x7f6484101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:43.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.155+0000 7f648a800640 1 -- 172.21.15.5:0/514829324 shutdown_connections 2024-04-24T06:41:43.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.155+0000 7f648a800640 1 -- 172.21.15.5:0/514829324 wait complete. 2024-04-24T06:41:43.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.156+0000 7f648a800640 1 Processor -- start 2024-04-24T06:41:43.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.156+0000 7f648a800640 1 -- start start 2024-04-24T06:41:43.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.156+0000 7f648a800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64840769c0 0x7f648419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.156+0000 7f648a800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6484077c00 0x7f648419ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.156+0000 7f648a800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64840785a0 0x7f64841a5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.156+0000 7f648a800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6484114330 con 0x7f64840785a0 2024-04-24T06:41:43.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.156+0000 7f648a800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f64841141b0 con 0x7f64840769c0 2024-04-24T06:41:43.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.156+0000 7f648a800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f64841144b0 con 0x7f6484077c00 2024-04-24T06:41:43.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.157+0000 7f6483e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64840785a0 0x7f64841a5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:43.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.157+0000 7f6483e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64840785a0 0x7f64841a5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60812/0 (socket says 172.21.15.5:60812) 2024-04-24T06:41:43.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.157+0000 7f6483e00640 1 -- 172.21.15.5:0/3035687116 learned_addr learned my addr 172.21.15.5:0/3035687116 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:41:43.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.157+0000 7f6482a00640 1 --2- 172.21.15.5:0/3035687116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6484077c00 0x7f648419ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:43.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.157+0000 7f6483400640 1 --2- 172.21.15.5:0/3035687116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64840769c0 0x7f648419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:43.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.157+0000 7f6483e00640 1 -- 172.21.15.5:0/3035687116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6484077c00 msgr2=0x7f648419ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:43.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.157+0000 7f6483e00640 1 --2- 172.21.15.5:0/3035687116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6484077c00 0x7f648419ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.157+0000 7f6483e00640 1 -- 172.21.15.5:0/3035687116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64840769c0 msgr2=0x7f648419e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:43.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.157+0000 7f6483e00640 1 --2- 172.21.15.5:0/3035687116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64840769c0 0x7f648419e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.157+0000 7f6483e00640 1 -- 172.21.15.5:0/3035687116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64841a6550 con 0x7f64840785a0 2024-04-24T06:41:43.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.157+0000 7f6483e00640 1 --2- 172.21.15.5:0/3035687116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64840785a0 0x7f64841a5ee0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f647c00e970 tx=0x7f647c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:43.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.158+0000 7f647be00640 1 -- 172.21.15.5:0/3035687116 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647c00ccc0 con 0x7f64840785a0 2024-04-24T06:41:43.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.158+0000 7f647be00640 1 -- 172.21.15.5:0/3035687116 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f647c00ce60 con 0x7f64840785a0 2024-04-24T06:41:43.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.158+0000 7f648a800640 1 -- 172.21.15.5:0/3035687116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64841a6840 con 0x7f64840785a0 2024-04-24T06:41:43.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.158+0000 7f647be00640 1 -- 172.21.15.5:0/3035687116 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647c010490 con 0x7f64840785a0 2024-04-24T06:41:43.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.158+0000 7f648a800640 1 -- 172.21.15.5:0/3035687116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64841a6db0 con 0x7f64840785a0 2024-04-24T06:41:43.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.159+0000 7f648a800640 1 -- 172.21.15.5:0/3035687116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6484110d90 con 0x7f64840785a0 2024-04-24T06:41:43.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.161+0000 7f647be00640 1 -- 172.21.15.5:0/3035687116 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f647c005020 con 0x7f64840785a0 2024-04-24T06:41:43.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.163+0000 7f647be00640 1 --2- 172.21.15.5:0/3035687116 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f64680a3cc0 0x7f64680a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.164+0000 7f647be00640 1 -- 172.21.15.5:0/3035687116 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f647c0d0850 con 0x7f64840785a0 2024-04-24T06:41:43.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.164+0000 7f6483400640 1 --2- 172.21.15.5:0/3035687116 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f64680a3cc0 0x7f64680a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:43.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.165+0000 7f6483400640 1 --2- 172.21.15.5:0/3035687116 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f64680a3cc0 0x7f64680a6180 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6474005fd0 tx=0x7f6474009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:43.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.168+0000 7f647be00640 1 -- 172.21.15.5:0/3035687116 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f647c09d180 con 0x7f64840785a0 2024-04-24T06:41:43.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.428+0000 7f648a800640 1 -- 172.21.15.5:0/3035687116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f648402d620 con 0x7f64840785a0 2024-04-24T06:41:43.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.429+0000 7f647be00640 1 -- 172.21.15.5:0/3035687116 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11856 (secure 0 0 0) 0x7f647c09ca80 con 0x7f64840785a0 2024-04-24T06:41:43.431 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:41:43.431 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1735462750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1735462750},{"type":"v1","addr":"172.21.15.5:6837","nonce":1735462750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1499469370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1499469370},{"type":"v1","addr":"172.21.15.5:6841","nonce":1499469370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3705638847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3705638847},{"type":"v1","addr":"172.21.15.37:6835","nonce":3705638847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T06:41:34.764490+0000","modified":"2024-04-24T06:41:40.330991+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24550,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6837/1390103296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1390103296},{"type":"v1","addr":"172.21.15.37:6837","nonce":1390103296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3217754215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3217754215},{"type":"v1","addr":"172.21.15.137:6835","nonce":3217754215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6837/3044222976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3044222976},{"type":"v1","addr":"172.21.15.137:6837","nonce":3044222976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.5:6839/3173201459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3173201459},{"type":"v1","addr":"172.21.15.5:6839","nonce":3173201459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6839/1889774232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1889774232},{"type":"v1","addr":"172.21.15.137:6839","nonce":1889774232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6839/3536721753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":3536721753},{"type":"v1","addr":"172.21.15.37:6839","nonce":3536721753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24539,24550,24569]},"id":1}]} 2024-04-24T06:41:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 22 2024-04-24T06:41:43.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.433+0000 7f648a800640 1 -- 172.21.15.5:0/3035687116 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f64680a3cc0 msgr2=0x7f64680a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:43.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.433+0000 7f648a800640 1 --2- 172.21.15.5:0/3035687116 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f64680a3cc0 0x7f64680a6180 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6474005fd0 tx=0x7f6474009500 comp rx=0 tx=0).stop 2024-04-24T06:41:43.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.433+0000 7f648a800640 1 -- 172.21.15.5:0/3035687116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64840785a0 msgr2=0x7f64841a5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:43.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.433+0000 7f648a800640 1 --2- 172.21.15.5:0/3035687116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64840785a0 0x7f64841a5ee0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f647c00e970 tx=0x7f647c00ed70 comp rx=0 tx=0).stop 2024-04-24T06:41:43.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.433+0000 7f648a800640 1 -- 172.21.15.5:0/3035687116 shutdown_connections 2024-04-24T06:41:43.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.433+0000 7f648a800640 1 --2- 172.21.15.5:0/3035687116 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f64680a3cc0 0x7f64680a6180 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.433+0000 7f648a800640 1 --2- 172.21.15.5:0/3035687116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64840785a0 0x7f64841a5ee0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.433+0000 7f648a800640 1 --2- 172.21.15.5:0/3035687116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6484077c00 0x7f648419ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.433+0000 7f648a800640 1 --2- 172.21.15.5:0/3035687116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64840769c0 0x7f648419e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.433+0000 7f648a800640 1 -- 172.21.15.5:0/3035687116 >> 172.21.15.5:0/3035687116 conn(0x7f64840ff880 msgr2=0x7f6484100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:43.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.433+0000 7f648a800640 1 -- 172.21.15.5:0/3035687116 shutdown_connections 2024-04-24T06:41:43.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.433+0000 7f648a800640 1 -- 172.21.15.5:0/3035687116 wait complete. 2024-04-24T06:41:43.456 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:41:43.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.666+0000 7fd863200640 1 -- 172.21.15.5:0/2073250159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8641018a0 msgr2=0x7fd864101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:43.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.666+0000 7fd863200640 1 --2- 172.21.15.5:0/2073250159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8641018a0 0x7fd864101d20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd84c00aa40 tx=0x7fd84c02b750 comp rx=0 tx=0).stop 2024-04-24T06:41:43.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.666+0000 7fd863200640 1 -- 172.21.15.5:0/2073250159 shutdown_connections 2024-04-24T06:41:43.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.666+0000 7fd863200640 1 --2- 172.21.15.5:0/2073250159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8641065a0 0x7fd864108990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.666+0000 7fd863200640 1 --2- 172.21.15.5:0/2073250159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8641018a0 0x7fd864101d20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.666+0000 7fd863200640 1 --2- 172.21.15.5:0/2073250159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd864068620 0x7fd864101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.666+0000 7fd863200640 1 -- 172.21.15.5:0/2073250159 >> 172.21.15.5:0/2073250159 conn(0x7fd8640fb700 msgr2=0x7fd8640fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:43.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.666+0000 7fd863200640 1 -- 172.21.15.5:0/2073250159 shutdown_connections 2024-04-24T06:41:43.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.666+0000 7fd863200640 1 -- 172.21.15.5:0/2073250159 wait complete. 2024-04-24T06:41:43.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.667+0000 7fd863200640 1 Processor -- start 2024-04-24T06:41:43.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.667+0000 7fd863200640 1 -- start start 2024-04-24T06:41:43.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.668+0000 7fd863200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd864068620 0x7fd86419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.668+0000 7fd863200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8641065a0 0x7fd86419ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.668+0000 7fd863200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd86419fca0 0x7fd8641a2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.668+0000 7fd863200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd86410b990 con 0x7fd86419fca0 2024-04-24T06:41:43.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.668+0000 7fd863200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd86410b810 con 0x7fd864068620 2024-04-24T06:41:43.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.668+0000 7fd863200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd86410bb10 con 0x7fd8641065a0 2024-04-24T06:41:43.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.668+0000 7fd861e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd864068620 0x7fd86419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:43.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.668+0000 7fd862800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd86419fca0 0x7fd8641a2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:43.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.668+0000 7fd861e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd864068620 0x7fd86419a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:35902/0 (socket says 172.21.15.5:35902) 2024-04-24T06:41:43.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.668+0000 7fd861e00640 1 -- 172.21.15.5:0/3969552520 learned_addr learned my addr 172.21.15.5:0/3969552520 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:41:43.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.668+0000 7fd85be00640 1 --2- 172.21.15.5:0/3969552520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8641065a0 0x7fd86419ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:43.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.668+0000 7fd861e00640 1 -- 172.21.15.5:0/3969552520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8641065a0 msgr2=0x7fd86419ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:43.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.668+0000 7fd861e00640 1 --2- 172.21.15.5:0/3969552520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8641065a0 0x7fd86419ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.668+0000 7fd861e00640 1 -- 172.21.15.5:0/3969552520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd86419fca0 msgr2=0x7fd8641a2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:43.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.668+0000 7fd861e00640 1 --2- 172.21.15.5:0/3969552520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd86419fca0 0x7fd8641a2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.668+0000 7fd861e00640 1 -- 172.21.15.5:0/3969552520 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8641a2700 con 0x7fd864068620 2024-04-24T06:41:43.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.669+0000 7fd861e00640 1 --2- 172.21.15.5:0/3969552520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd864068620 0x7fd86419a700 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd85400e9c0 tx=0x7fd85400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:43.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.669+0000 7fd85aa00640 1 -- 172.21.15.5:0/3969552520 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85400cde0 con 0x7fd864068620 2024-04-24T06:41:43.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.669+0000 7fd85aa00640 1 -- 172.21.15.5:0/3969552520 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd854010910 con 0x7fd864068620 2024-04-24T06:41:43.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.669+0000 7fd863200640 1 -- 172.21.15.5:0/3969552520 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8641a29f0 con 0x7fd864068620 2024-04-24T06:41:43.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.669+0000 7fd85aa00640 1 -- 172.21.15.5:0/3969552520 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd854018a00 con 0x7fd864068620 2024-04-24T06:41:43.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.669+0000 7fd863200640 1 -- 172.21.15.5:0/3969552520 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8641a2f10 con 0x7fd864068620 2024-04-24T06:41:43.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.670+0000 7fd863200640 1 -- 172.21.15.5:0/3969552520 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8641053b0 con 0x7fd864068620 2024-04-24T06:41:43.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.672+0000 7fd85aa00640 1 -- 172.21.15.5:0/3969552520 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7fd85401f020 con 0x7fd864068620 2024-04-24T06:41:43.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.673+0000 7fd85aa00640 1 --2- 172.21.15.5:0/3969552520 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fd8480a3c30 0x7fd8480a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:43.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.674+0000 7fd85aa00640 1 -- 172.21.15.5:0/3969552520 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fd8540d05b0 con 0x7fd864068620 2024-04-24T06:41:43.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.674+0000 7fd85be00640 1 --2- 172.21.15.5:0/3969552520 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fd8480a3c30 0x7fd8480a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:43.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.674+0000 7fd85be00640 1 --2- 172.21.15.5:0/3969552520 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fd8480a3c30 0x7fd8480a60f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd84c00a340 tx=0x7fd84c030510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:43.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.678+0000 7fd85aa00640 1 -- 172.21.15.5:0/3969552520 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd85409cfe0 con 0x7fd864068620 2024-04-24T06:41:43.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:43 smithi137 ceph-mon[33017]: mgrmap e17: x(active, since 8m), standbys: y, z 2024-04-24T06:41:43.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:43 smithi137 ceph-mon[33017]: from='client.14583 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:41:43.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:43 smithi137 ceph-mon[33017]: from='client.14589 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:41:43.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:43 smithi005 ceph-mon[27798]: mgrmap e17: x(active, since 8m), standbys: y, z 2024-04-24T06:41:43.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:43 smithi005 ceph-mon[27798]: from='client.14583 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:41:43.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:43 smithi005 ceph-mon[27798]: from='client.14589 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:41:43.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:43 smithi037 ceph-mon[32864]: mgrmap e17: x(active, since 8m), standbys: y, z 2024-04-24T06:41:43.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:43 smithi037 ceph-mon[32864]: from='client.14583 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:41:43.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:43 smithi037 ceph-mon[32864]: from='client.14589 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:41:43.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.887+0000 7fd863200640 1 -- 172.21.15.5:0/3969552520 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd864062c00 con 0x7fd864068620 2024-04-24T06:41:43.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.889+0000 7fd85aa00640 1 -- 172.21.15.5:0/3969552520 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22088 (secure 0 0 0) 0x7fd85409c8e0 con 0x7fd864068620 2024-04-24T06:41:43.891 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:41:43.892 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":86,"fsid":"43e9bad0-0204-11ef-bc93-c7b262605968","created":"2024-04-24T06:32:26.340555+0000","modified":"2024-04-24T06:41:36.259462+0000","last_up_change":"2024-04-24T06:40:36.790934+0000","last_in_change":"2024-04-24T06:40:17.803257+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:36:27.320342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:41:32.254502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:41:33.739067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c2e974c-745c-4dd5-8c3d-8e8da2eb1ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6803","nonce":2155858515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6805","nonce":2155858515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6809","nonce":2155858515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6807","nonce":2155858515}]},"public_addr":"172.21.15.5:6803/2155858515","cluster_addr":"172.21.15.5:6805/2155858515","heartbeat_back_addr":"172.21.15.5:6809/2155858515","heartbeat_front_addr":"172.21.15.5:6807/2155858515","state":["exists","up"]},{"osd":1,"uuid":"50467744-db20-4721-bf2d-1c5db938297b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6801","nonce":1302362650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6803","nonce":1302362650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6807","nonce":1302362650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6805","nonce":1302362650}]},"public_addr":"172.21.15.37:6801/1302362650","cluster_addr":"172.21.15.37:6803/1302362650","heartbeat_back_addr":"172.21.15.37:6807/1302362650","heartbeat_front_addr":"172.21.15.37:6805/1302362650","state":["exists","up"]},{"osd":2,"uuid":"0e190b15-d04c-4288-8d48-ecf6664392fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6801","nonce":2885027256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6803","nonce":2885027256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6807","nonce":2885027256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6805","nonce":2885027256}]},"public_addr":"172.21.15.137:6801/2885027256","cluster_addr":"172.21.15.137:6803/2885027256","heartbeat_back_addr":"172.21.15.137:6807/2885027256","heartbeat_front_addr":"172.21.15.137:6805/2885027256","state":["exists","up"]},{"osd":3,"uuid":"e05592ba-d96a-472a-9f63-09a535d04dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6811","nonce":3701747260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6813","nonce":3701747260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6817","nonce":3701747260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6815","nonce":3701747260}]},"public_addr":"172.21.15.5:6811/3701747260","cluster_addr":"172.21.15.5:6813/3701747260","heartbeat_back_addr":"172.21.15.5:6817/3701747260","heartbeat_front_addr":"172.21.15.5:6815/3701747260","state":["exists","up"]},{"osd":4,"uuid":"c3f36e51-7a3b-4dd9-af4b-a10558830c4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6809","nonce":759253844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6811","nonce":759253844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6815","nonce":759253844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6813","nonce":759253844}]},"public_addr":"172.21.15.37:6809/759253844","cluster_addr":"172.21.15.37:6811/759253844","heartbeat_back_addr":"172.21.15.37:6815/759253844","heartbeat_front_addr":"172.21.15.37:6813/759253844","state":["exists","up"]},{"osd":5,"uuid":"97a0500c-cbb8-4ff5-b476-8634b26b3821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6809","nonce":3771921560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6811","nonce":3771921560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6815","nonce":3771921560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6813","nonce":3771921560}]},"public_addr":"172.21.15.137:6809/3771921560","cluster_addr":"172.21.15.137:6811/3771921560","heartbeat_back_addr":"172.21.15.137:6815/3771921560","heartbeat_front_addr":"172.21.15.137:6813/3771921560","state":["exists","up"]},{"osd":6,"uuid":"33a33d62-bbb3-4ed9-834e-19ae93a97670","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6819","nonce":2064636084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6821","nonce":2064636084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6825","nonce":2064636084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6823","nonce":2064636084}]},"public_addr":"172.21.15.5:6819/2064636084","cluster_addr":"172.21.15.5:6821/2064636084","heartbeat_back_addr":"172.21.15.5:6825/2064636084","heartbeat_front_addr":"172.21.15.5:6823/2064636084","state":["exists","up"]},{"osd":7,"uuid":"c0f4e036-4717-4514-bb59-ab11fbaab063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6817","nonce":2952750659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6819","nonce":2952750659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6823","nonce":2952750659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6821","nonce":2952750659}]},"public_addr":"172.21.15.37:6817/2952750659","cluster_addr":"172.21.15.37:6819/2952750659","heartbeat_back_addr":"172.21.15.37:6823/2952750659","heartbeat_front_addr":"172.21.15.37:6821/2952750659","state":["exists","up"]},{"osd":8,"uuid":"5391fd87-44ea-440f-80da-c54d0327c04c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6817","nonce":29724945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6819","nonce":29724945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6823","nonce":29724945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6821","nonce":29724945}]},"public_addr":"172.21.15.137:6817/29724945","cluster_addr":"172.21.15.137:6819/29724945","heartbeat_back_addr":"172.21.15.137:6823/29724945","heartbeat_front_addr":"172.21.15.137:6821/29724945","state":["exists","up"]},{"osd":9,"uuid":"d94c9dbc-f43d-4237-ac7f-47a6a5ede1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6827","nonce":2257102107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6829","nonce":2257102107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6833","nonce":2257102107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6831","nonce":2257102107}]},"public_addr":"172.21.15.5:6827/2257102107","cluster_addr":"172.21.15.5:6829/2257102107","heartbeat_back_addr":"172.21.15.5:6833/2257102107","heartbeat_front_addr":"172.21.15.5:6831/2257102107","state":["exists","up"]},{"osd":10,"uuid":"e2455332-2a5e-44b1-85d3-8e47da399f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6825","nonce":2451570637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6827","nonce":2451570637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6831","nonce":2451570637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6829","nonce":2451570637}]},"public_addr":"172.21.15.37:6825/2451570637","cluster_addr":"172.21.15.37:6827/2451570637","heartbeat_back_addr":"172.21.15.37:6831/2451570637","heartbeat_front_addr":"172.21.15.37:6829/2451570637","state":["exists","up"]},{"osd":11,"uuid":"49d9396b-704a-4cef-8210-75d30c28e924","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6825","nonce":90600596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6827","nonce":90600596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6831","nonce":90600596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6829","nonce":90600596}]},"public_addr":"172.21.15.137:6825/90600596","cluster_addr":"172.21.15.137:6827/90600596","heartbeat_back_addr":"172.21.15.137:6831/90600596","heartbeat_front_addr":"172.21.15.137:6829/90600596","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:35:35.591619+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:00.568817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:24.195643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:56.451724+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:22.109293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:46.161749+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:19.278305+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:45.620256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:10.041165+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:43.813697+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:10.736026+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:35.384377+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/23307164":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6801/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:0/230785941":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/2940023597":"2024-04-25T06:33:07.040369+0000","172.21.15.5:6801/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3886613904":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/4199165437":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3459593499":"2024-04-25T06:33:41.161852+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:41:43.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.892+0000 7fd863200640 1 -- 172.21.15.5:0/3969552520 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fd8480a3c30 msgr2=0x7fd8480a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:43.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.892+0000 7fd863200640 1 --2- 172.21.15.5:0/3969552520 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fd8480a3c30 0x7fd8480a60f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd84c00a340 tx=0x7fd84c030510 comp rx=0 tx=0).stop 2024-04-24T06:41:43.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.892+0000 7fd863200640 1 -- 172.21.15.5:0/3969552520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd864068620 msgr2=0x7fd86419a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:43.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.892+0000 7fd863200640 1 --2- 172.21.15.5:0/3969552520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd864068620 0x7fd86419a700 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd85400e9c0 tx=0x7fd85400ee80 comp rx=0 tx=0).stop 2024-04-24T06:41:43.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.892+0000 7fd863200640 1 -- 172.21.15.5:0/3969552520 shutdown_connections 2024-04-24T06:41:43.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.892+0000 7fd863200640 1 --2- 172.21.15.5:0/3969552520 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fd8480a3c30 0x7fd8480a60f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.892+0000 7fd863200640 1 --2- 172.21.15.5:0/3969552520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd86419fca0 0x7fd8641a2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.892+0000 7fd863200640 1 --2- 172.21.15.5:0/3969552520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8641065a0 0x7fd86419ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.892+0000 7fd863200640 1 --2- 172.21.15.5:0/3969552520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd864068620 0x7fd86419a700 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:43.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.892+0000 7fd863200640 1 -- 172.21.15.5:0/3969552520 >> 172.21.15.5:0/3969552520 conn(0x7fd8640fb700 msgr2=0x7fd864102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:43.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.893+0000 7fd863200640 1 -- 172.21.15.5:0/3969552520 shutdown_connections 2024-04-24T06:41:43.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:43.893+0000 7fd863200640 1 -- 172.21.15.5:0/3969552520 wait complete. 2024-04-24T06:41:43.915 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-24T06:41:43.916 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T06:41:44.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.134+0000 7f2c8ee00640 1 -- 172.21.15.5:0/1059751983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c88104160 msgr2=0x7f2c88104580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.134+0000 7f2c8ee00640 1 --2- 172.21.15.5:0/1059751983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c88104160 0x7f2c88104580 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c009900 tx=0x7f2c7c02a3f0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.134+0000 7f2c8ee00640 1 -- 172.21.15.5:0/1059751983 shutdown_connections 2024-04-24T06:41:44.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.134+0000 7f2c8ee00640 1 --2- 172.21.15.5:0/1059751983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c88105d10 0x7f2c8810c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.134+0000 7f2c8ee00640 1 --2- 172.21.15.5:0/1059751983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c88105350 0x7f2c881057d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.134+0000 7f2c8ee00640 1 --2- 172.21.15.5:0/1059751983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c88104160 0x7f2c88104580 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.134+0000 7f2c8ee00640 1 -- 172.21.15.5:0/1059751983 >> 172.21.15.5:0/1059751983 conn(0x7f2c880ff6d0 msgr2=0x7f2c88101b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:44.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.134+0000 7f2c8ee00640 1 -- 172.21.15.5:0/1059751983 shutdown_connections 2024-04-24T06:41:44.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.134+0000 7f2c8ee00640 1 -- 172.21.15.5:0/1059751983 wait complete. 2024-04-24T06:41:44.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.136+0000 7f2c8ee00640 1 Processor -- start 2024-04-24T06:41:44.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.136+0000 7f2c8ee00640 1 -- start start 2024-04-24T06:41:44.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.136+0000 7f2c8ee00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c88104160 0x7f2c8819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.136+0000 7f2c8ee00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c88105350 0x7f2c8819a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.136+0000 7f2c8ee00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c88105d10 0x7f2c881a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.136+0000 7f2c8ee00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8810fe40 con 0x7f2c88104160 2024-04-24T06:41:44.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.137+0000 7f2c8ee00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8810fcc0 con 0x7f2c88105d10 2024-04-24T06:41:44.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.137+0000 7f2c8ee00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8810ffc0 con 0x7f2c88105350 2024-04-24T06:41:44.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.137+0000 7f2c8da00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c88104160 0x7f2c8819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.137+0000 7f2c8da00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c88104160 0x7f2c8819a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43928/0 (socket says 172.21.15.5:43928) 2024-04-24T06:41:44.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.137+0000 7f2c8da00640 1 -- 172.21.15.5:0/997588856 learned_addr learned my addr 172.21.15.5:0/997588856 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:41:44.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.137+0000 7f2c8d000640 1 --2- 172.21.15.5:0/997588856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c88105350 0x7f2c8819a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.137+0000 7f2c8e400640 1 --2- 172.21.15.5:0/997588856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c88105d10 0x7f2c881a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.137+0000 7f2c8da00640 1 -- 172.21.15.5:0/997588856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c88105350 msgr2=0x7f2c8819a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.137+0000 7f2c8da00640 1 --2- 172.21.15.5:0/997588856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c88105350 0x7f2c8819a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.137+0000 7f2c8da00640 1 -- 172.21.15.5:0/997588856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c88105d10 msgr2=0x7f2c881a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.137+0000 7f2c8da00640 1 --2- 172.21.15.5:0/997588856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c88105d10 0x7f2c881a1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.137+0000 7f2c8da00640 1 -- 172.21.15.5:0/997588856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c881a2080 con 0x7f2c88104160 2024-04-24T06:41:44.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.138+0000 7f2c8da00640 1 --2- 172.21.15.5:0/997588856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c88104160 0x7f2c8819a460 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c031440 tx=0x7f2c7c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.138+0000 7f2c86000640 1 -- 172.21.15.5:0/997588856 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c03f030 con 0x7f2c88104160 2024-04-24T06:41:44.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.138+0000 7f2c86000640 1 -- 172.21.15.5:0/997588856 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c7c003ce0 con 0x7f2c88104160 2024-04-24T06:41:44.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.138+0000 7f2c8ee00640 1 -- 172.21.15.5:0/997588856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c881a2310 con 0x7f2c88104160 2024-04-24T06:41:44.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.138+0000 7f2c86000640 1 -- 172.21.15.5:0/997588856 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c035070 con 0x7f2c88104160 2024-04-24T06:41:44.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.138+0000 7f2c8ee00640 1 -- 172.21.15.5:0/997588856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c881a2800 con 0x7f2c88104160 2024-04-24T06:41:44.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.139+0000 7f2c8ee00640 1 -- 172.21.15.5:0/997588856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c8810b480 con 0x7f2c88104160 2024-04-24T06:41:44.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.141+0000 7f2c86000640 1 -- 172.21.15.5:0/997588856 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f2c7c042030 con 0x7f2c88104160 2024-04-24T06:41:44.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.143+0000 7f2c86000640 1 --2- 172.21.15.5:0/997588856 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f2c700a3c30 0x7f2c700a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.143+0000 7f2c86000640 1 -- 172.21.15.5:0/997588856 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f2c7c0f0d30 con 0x7f2c88104160 2024-04-24T06:41:44.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.143+0000 7f2c8d000640 1 --2- 172.21.15.5:0/997588856 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f2c700a3c30 0x7f2c700a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.144+0000 7f2c8d000640 1 --2- 172.21.15.5:0/997588856 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f2c700a3c30 0x7f2c700a60f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2c8819b970 tx=0x7f2c78005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.147+0000 7f2c86000640 1 -- 172.21.15.5:0/997588856 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2c7c0bd660 con 0x7f2c88104160 2024-04-24T06:41:44.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.349+0000 7f2c8ee00640 1 -- 172.21.15.5:0/997588856 --> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2c88062440 con 0x7f2c700a3c30 2024-04-24T06:41:44.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.369+0000 7f2c86000640 1 -- 172.21.15.5:0/997588856 <== mgr.14152 v2:172.21.15.5:6800/4042936965 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258754 (secure 0 0 0) 0x7f2c88062440 con 0x7f2c700a3c30 2024-04-24T06:41:44.375 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:41:44.377 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-24T06:41:44.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.375+0000 7f2c8ee00640 1 -- 172.21.15.5:0/997588856 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f2c700a3c30 msgr2=0x7f2c700a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.375+0000 7f2c8ee00640 1 --2- 172.21.15.5:0/997588856 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f2c700a3c30 0x7f2c700a60f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2c8819b970 tx=0x7f2c78005f90 comp rx=0 tx=0).stop 2024-04-24T06:41:44.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.375+0000 7f2c8ee00640 1 -- 172.21.15.5:0/997588856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c88104160 msgr2=0x7f2c8819a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.375+0000 7f2c8ee00640 1 --2- 172.21.15.5:0/997588856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c88104160 0x7f2c8819a460 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c031440 tx=0x7f2c7c032ce0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.375+0000 7f2c8ee00640 1 -- 172.21.15.5:0/997588856 shutdown_connections 2024-04-24T06:41:44.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.375+0000 7f2c8ee00640 1 --2- 172.21.15.5:0/997588856 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f2c700a3c30 0x7f2c700a60f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.375+0000 7f2c8ee00640 1 --2- 172.21.15.5:0/997588856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c88105d10 0x7f2c881a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.375+0000 7f2c8ee00640 1 --2- 172.21.15.5:0/997588856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c88105350 0x7f2c8819a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.375+0000 7f2c8ee00640 1 --2- 172.21.15.5:0/997588856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c88104160 0x7f2c8819a460 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.375+0000 7f2c8ee00640 1 -- 172.21.15.5:0/997588856 >> 172.21.15.5:0/997588856 conn(0x7f2c880ff6d0 msgr2=0x7f2c88108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:44.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.376+0000 7f2c8ee00640 1 -- 172.21.15.5:0/997588856 shutdown_connections 2024-04-24T06:41:44.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.376+0000 7f2c8ee00640 1 -- 172.21.15.5:0/997588856 wait complete. 2024-04-24T06:41:44.398 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-04-24T06:41:43.304495+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":633840,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":158,"num_read_kb":155,"num_write":321,"num_write_kb":3019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":259,"ondisk_log_size":259,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374488,"kb_used_data":12840,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567336,"statfs":{"total":1151940427776,"available":1151556952064,"internally_reserved":0,"allocated":13148160,"data_stored":7140272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":43472,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":22,"num_read_kb":36,"num_write":140,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.044327"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.470195+0000","last_change":"2024-04-24T06:41:35.296451+0000","last_active":"2024-04-24T06:41:36.470195+0000","last_peered":"2024-04-24T06:41:36.470195+0000","last_clean":"2024-04-24T06:41:36.470195+0000","last_became_active":"2024-04-24T06:41:35.295733+0000","last_became_peered":"2024-04-24T06:41:35.295733+0000","last_unstale":"2024-04-24T06:41:36.470195+0000","last_undegraded":"2024-04-24T06:41:36.470195+0000","last_fullsized":"2024-04-24T06:41:36.470195+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:04:15.072644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.470648+0000","last_change":"2024-04-24T06:41:35.293359+0000","last_active":"2024-04-24T06:41:36.470648+0000","last_peered":"2024-04-24T06:41:36.470648+0000","last_clean":"2024-04-24T06:41:36.470648+0000","last_became_active":"2024-04-24T06:41:35.287371+0000","last_became_peered":"2024-04-24T06:41:35.287371+0000","last_unstale":"2024-04-24T06:41:36.470648+0000","last_undegraded":"2024-04-24T06:41:36.470648+0000","last_fullsized":"2024-04-24T06:41:36.470648+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:03:39.376495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.470076+0000","last_change":"2024-04-24T06:41:35.272734+0000","last_active":"2024-04-24T06:41:36.470076+0000","last_peered":"2024-04-24T06:41:36.470076+0000","last_clean":"2024-04-24T06:41:36.470076+0000","last_became_active":"2024-04-24T06:41:35.271685+0000","last_became_peered":"2024-04-24T06:41:35.271685+0000","last_unstale":"2024-04-24T06:41:36.470076+0000","last_undegraded":"2024-04-24T06:41:36.470076+0000","last_fullsized":"2024-04-24T06:41:36.470076+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:08:15.057918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.470211+0000","last_change":"2024-04-24T06:41:35.288574+0000","last_active":"2024-04-24T06:41:36.470211+0000","last_peered":"2024-04-24T06:41:36.470211+0000","last_clean":"2024-04-24T06:41:36.470211+0000","last_became_active":"2024-04-24T06:41:35.288229+0000","last_became_peered":"2024-04-24T06:41:35.288229+0000","last_unstale":"2024-04-24T06:41:36.470211+0000","last_undegraded":"2024-04-24T06:41:36.470211+0000","last_fullsized":"2024-04-24T06:41:36.470211+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:58:18.028067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.446788+0000","last_change":"2024-04-24T06:41:35.310226+0000","last_active":"2024-04-24T06:41:36.446788+0000","last_peered":"2024-04-24T06:41:36.446788+0000","last_clean":"2024-04-24T06:41:36.446788+0000","last_became_active":"2024-04-24T06:41:35.309853+0000","last_became_peered":"2024-04-24T06:41:35.309853+0000","last_unstale":"2024-04-24T06:41:36.446788+0000","last_undegraded":"2024-04-24T06:41:36.446788+0000","last_fullsized":"2024-04-24T06:41:36.446788+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:34:57.767158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.470638+0000","last_change":"2024-04-24T06:41:35.294259+0000","last_active":"2024-04-24T06:41:36.470638+0000","last_peered":"2024-04-24T06:41:36.470638+0000","last_clean":"2024-04-24T06:41:36.470638+0000","last_became_active":"2024-04-24T06:41:35.293827+0000","last_became_peered":"2024-04-24T06:41:35.293827+0000","last_unstale":"2024-04-24T06:41:36.470638+0000","last_undegraded":"2024-04-24T06:41:36.470638+0000","last_fullsized":"2024-04-24T06:41:36.470638+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:09:23.302827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.470783+0000","last_change":"2024-04-24T06:41:35.295095+0000","last_active":"2024-04-24T06:41:36.470783+0000","last_peered":"2024-04-24T06:41:36.470783+0000","last_clean":"2024-04-24T06:41:36.470783+0000","last_became_active":"2024-04-24T06:41:35.294743+0000","last_became_peered":"2024-04-24T06:41:35.294743+0000","last_unstale":"2024-04-24T06:41:36.470783+0000","last_undegraded":"2024-04-24T06:41:36.470783+0000","last_fullsized":"2024-04-24T06:41:36.470783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:06:09.473123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.276149+0000","last_change":"2024-04-24T06:41:35.302644+0000","last_active":"2024-04-24T06:41:36.276149+0000","last_peered":"2024-04-24T06:41:36.276149+0000","last_clean":"2024-04-24T06:41:36.276149+0000","last_became_active":"2024-04-24T06:41:35.302272+0000","last_became_peered":"2024-04-24T06:41:35.302272+0000","last_unstale":"2024-04-24T06:41:36.276149+0000","last_undegraded":"2024-04-24T06:41:36.276149+0000","last_fullsized":"2024-04-24T06:41:36.276149+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:14:26.176607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.469543+0000","last_change":"2024-04-24T06:41:35.283275+0000","last_active":"2024-04-24T06:41:36.469543+0000","last_peered":"2024-04-24T06:41:36.469543+0000","last_clean":"2024-04-24T06:41:36.469543+0000","last_became_active":"2024-04-24T06:41:35.282586+0000","last_became_peered":"2024-04-24T06:41:35.282586+0000","last_unstale":"2024-04-24T06:41:36.469543+0000","last_undegraded":"2024-04-24T06:41:36.469543+0000","last_fullsized":"2024-04-24T06:41:36.469543+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:42:30.716149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.471497+0000","last_change":"2024-04-24T06:41:35.284986+0000","last_active":"2024-04-24T06:41:36.471497+0000","last_peered":"2024-04-24T06:41:36.471497+0000","last_clean":"2024-04-24T06:41:36.471497+0000","last_became_active":"2024-04-24T06:41:35.284628+0000","last_became_peered":"2024-04-24T06:41:35.284628+0000","last_unstale":"2024-04-24T06:41:36.471497+0000","last_undegraded":"2024-04-24T06:41:36.471497+0000","last_fullsized":"2024-04-24T06:41:36.471497+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:29:38.479593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.471727+0000","last_change":"2024-04-24T06:41:35.290353+0000","last_active":"2024-04-24T06:41:36.471727+0000","last_peered":"2024-04-24T06:41:36.471727+0000","last_clean":"2024-04-24T06:41:36.471727+0000","last_became_active":"2024-04-24T06:41:35.289987+0000","last_became_peered":"2024-04-24T06:41:35.289987+0000","last_unstale":"2024-04-24T06:41:36.471727+0000","last_undegraded":"2024-04-24T06:41:36.471727+0000","last_fullsized":"2024-04-24T06:41:36.471727+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:56:54.914426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.470085+0000","last_change":"2024-04-24T06:41:35.289462+0000","last_active":"2024-04-24T06:41:36.470085+0000","last_peered":"2024-04-24T06:41:36.470085+0000","last_clean":"2024-04-24T06:41:36.470085+0000","last_became_active":"2024-04-24T06:41:35.289123+0000","last_became_peered":"2024-04-24T06:41:35.289123+0000","last_unstale":"2024-04-24T06:41:36.470085+0000","last_undegraded":"2024-04-24T06:41:36.470085+0000","last_fullsized":"2024-04-24T06:41:36.470085+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:51:06.792508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.276261+0000","last_change":"2024-04-24T06:41:35.298320+0000","last_active":"2024-04-24T06:41:36.276261+0000","last_peered":"2024-04-24T06:41:36.276261+0000","last_clean":"2024-04-24T06:41:36.276261+0000","last_became_active":"2024-04-24T06:41:35.297839+0000","last_became_peered":"2024-04-24T06:41:35.297839+0000","last_unstale":"2024-04-24T06:41:36.276261+0000","last_undegraded":"2024-04-24T06:41:36.276261+0000","last_fullsized":"2024-04-24T06:41:36.276261+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:10:36.423514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.470082+0000","last_change":"2024-04-24T06:41:35.301924+0000","last_active":"2024-04-24T06:41:36.470082+0000","last_peered":"2024-04-24T06:41:36.470082+0000","last_clean":"2024-04-24T06:41:36.470082+0000","last_became_active":"2024-04-24T06:41:35.299114+0000","last_became_peered":"2024-04-24T06:41:35.299114+0000","last_unstale":"2024-04-24T06:41:36.470082+0000","last_undegraded":"2024-04-24T06:41:36.470082+0000","last_fullsized":"2024-04-24T06:41:36.470082+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:34:46.194266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.469752+0000","last_change":"2024-04-24T06:41:35.284885+0000","last_active":"2024-04-24T06:41:36.469752+0000","last_peered":"2024-04-24T06:41:36.469752+0000","last_clean":"2024-04-24T06:41:36.469752+0000","last_became_active":"2024-04-24T06:41:35.284244+0000","last_became_peered":"2024-04-24T06:41:35.284244+0000","last_unstale":"2024-04-24T06:41:36.469752+0000","last_undegraded":"2024-04-24T06:41:36.469752+0000","last_fullsized":"2024-04-24T06:41:36.469752+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:30:40.279905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.468945+0000","last_change":"2024-04-24T06:41:35.286363+0000","last_active":"2024-04-24T06:41:36.468945+0000","last_peered":"2024-04-24T06:41:36.468945+0000","last_clean":"2024-04-24T06:41:36.468945+0000","last_became_active":"2024-04-24T06:41:35.286024+0000","last_became_peered":"2024-04-24T06:41:35.286024+0000","last_unstale":"2024-04-24T06:41:36.468945+0000","last_undegraded":"2024-04-24T06:41:36.468945+0000","last_fullsized":"2024-04-24T06:41:36.468945+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:43:35.728990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.470296+0000","last_change":"2024-04-24T06:41:35.304005+0000","last_active":"2024-04-24T06:41:36.470296+0000","last_peered":"2024-04-24T06:41:36.470296+0000","last_clean":"2024-04-24T06:41:36.470296+0000","last_became_active":"2024-04-24T06:41:35.303659+0000","last_became_peered":"2024-04-24T06:41:35.303659+0000","last_unstale":"2024-04-24T06:41:36.470296+0000","last_undegraded":"2024-04-24T06:41:36.470296+0000","last_fullsized":"2024-04-24T06:41:36.470296+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:05:07.215927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.444526+0000","last_change":"2024-04-24T06:41:35.296218+0000","last_active":"2024-04-24T06:41:36.444526+0000","last_peered":"2024-04-24T06:41:36.444526+0000","last_clean":"2024-04-24T06:41:36.444526+0000","last_became_active":"2024-04-24T06:41:35.295486+0000","last_became_peered":"2024-04-24T06:41:35.295486+0000","last_unstale":"2024-04-24T06:41:36.444526+0000","last_undegraded":"2024-04-24T06:41:36.444526+0000","last_fullsized":"2024-04-24T06:41:36.444526+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:17:25.844899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.469995+0000","last_change":"2024-04-24T06:41:35.305064+0000","last_active":"2024-04-24T06:41:36.469995+0000","last_peered":"2024-04-24T06:41:36.469995+0000","last_clean":"2024-04-24T06:41:36.469995+0000","last_became_active":"2024-04-24T06:41:35.304695+0000","last_became_peered":"2024-04-24T06:41:35.304695+0000","last_unstale":"2024-04-24T06:41:36.469995+0000","last_undegraded":"2024-04-24T06:41:36.469995+0000","last_fullsized":"2024-04-24T06:41:36.469995+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:35:13.065601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:39.337530+0000","last_change":"2024-04-24T06:41:34.282140+0000","last_active":"2024-04-24T06:41:39.337530+0000","last_peered":"2024-04-24T06:41:39.337530+0000","last_clean":"2024-04-24T06:41:39.337530+0000","last_became_active":"2024-04-24T06:41:34.281608+0000","last_became_peered":"2024-04-24T06:41:34.281608+0000","last_unstale":"2024-04-24T06:41:39.337530+0000","last_undegraded":"2024-04-24T06:41:39.337530+0000","last_fullsized":"2024-04-24T06:41:39.337530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:32:06.727639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.278254+0000","last_change":"2024-04-24T06:41:35.308847+0000","last_active":"2024-04-24T06:41:36.278254+0000","last_peered":"2024-04-24T06:41:36.278254+0000","last_clean":"2024-04-24T06:41:36.278254+0000","last_became_active":"2024-04-24T06:41:35.308256+0000","last_became_peered":"2024-04-24T06:41:35.308256+0000","last_unstale":"2024-04-24T06:41:36.278254+0000","last_undegraded":"2024-04-24T06:41:36.278254+0000","last_fullsized":"2024-04-24T06:41:36.278254+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:21:49.338737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.278214+0000","last_change":"2024-04-24T06:41:34.285819+0000","last_active":"2024-04-24T06:41:36.278214+0000","last_peered":"2024-04-24T06:41:36.278214+0000","last_clean":"2024-04-24T06:41:36.278214+0000","last_became_active":"2024-04-24T06:41:34.282393+0000","last_became_peered":"2024-04-24T06:41:34.282393+0000","last_unstale":"2024-04-24T06:41:36.278214+0000","last_undegraded":"2024-04-24T06:41:36.278214+0000","last_fullsized":"2024-04-24T06:41:36.278214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:11:18.969178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.470688+0000","last_change":"2024-04-24T06:41:35.297437+0000","last_active":"2024-04-24T06:41:36.470688+0000","last_peered":"2024-04-24T06:41:36.470688+0000","last_clean":"2024-04-24T06:41:36.470688+0000","last_became_active":"2024-04-24T06:41:35.295195+0000","last_became_peered":"2024-04-24T06:41:35.295195+0000","last_unstale":"2024-04-24T06:41:36.470688+0000","last_undegraded":"2024-04-24T06:41:36.470688+0000","last_fullsized":"2024-04-24T06:41:36.470688+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:21:51.219219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:39.330880+0000","last_change":"2024-04-24T06:41:34.275768+0000","last_active":"2024-04-24T06:41:39.330880+0000","last_peered":"2024-04-24T06:41:39.330880+0000","last_clean":"2024-04-24T06:41:39.330880+0000","last_became_active":"2024-04-24T06:41:34.275426+0000","last_became_peered":"2024-04-24T06:41:34.275426+0000","last_unstale":"2024-04-24T06:41:39.330880+0000","last_undegraded":"2024-04-24T06:41:39.330880+0000","last_fullsized":"2024-04-24T06:41:39.330880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:01:08.735480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.469042+0000","last_change":"2024-04-24T06:41:35.305734+0000","last_active":"2024-04-24T06:41:36.469042+0000","last_peered":"2024-04-24T06:41:36.469042+0000","last_clean":"2024-04-24T06:41:36.469042+0000","last_became_active":"2024-04-24T06:41:35.305365+0000","last_became_peered":"2024-04-24T06:41:35.305365+0000","last_unstale":"2024-04-24T06:41:36.469042+0000","last_undegraded":"2024-04-24T06:41:36.469042+0000","last_fullsized":"2024-04-24T06:41:36.469042+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:38:52.740107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.447003+0000","last_change":"2024-04-24T06:41:34.278317+0000","last_active":"2024-04-24T06:41:36.447003+0000","last_peered":"2024-04-24T06:41:36.447003+0000","last_clean":"2024-04-24T06:41:36.447003+0000","last_became_active":"2024-04-24T06:41:34.277862+0000","last_became_peered":"2024-04-24T06:41:34.277862+0000","last_unstale":"2024-04-24T06:41:36.447003+0000","last_undegraded":"2024-04-24T06:41:36.447003+0000","last_fullsized":"2024-04-24T06:41:36.447003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:19:50.444136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.276716+0000","last_change":"2024-04-24T06:41:35.282648+0000","last_active":"2024-04-24T06:41:36.276716+0000","last_peered":"2024-04-24T06:41:36.276716+0000","last_clean":"2024-04-24T06:41:36.276716+0000","last_became_active":"2024-04-24T06:41:35.282335+0000","last_became_peered":"2024-04-24T06:41:35.282335+0000","last_unstale":"2024-04-24T06:41:36.276716+0000","last_undegraded":"2024-04-24T06:41:36.276716+0000","last_fullsized":"2024-04-24T06:41:36.276716+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:30:11.909067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:38.318497+0000","last_change":"2024-04-24T06:41:34.274405+0000","last_active":"2024-04-24T06:41:38.318497+0000","last_peered":"2024-04-24T06:41:38.318497+0000","last_clean":"2024-04-24T06:41:38.318497+0000","last_became_active":"2024-04-24T06:41:34.273820+0000","last_became_peered":"2024-04-24T06:41:34.273820+0000","last_unstale":"2024-04-24T06:41:38.318497+0000","last_undegraded":"2024-04-24T06:41:38.318497+0000","last_fullsized":"2024-04-24T06:41:38.318497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:11:26.119236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.446635+0000","last_change":"2024-04-24T06:41:35.294133+0000","last_active":"2024-04-24T06:41:36.446635+0000","last_peered":"2024-04-24T06:41:36.446635+0000","last_clean":"2024-04-24T06:41:36.446635+0000","last_became_active":"2024-04-24T06:41:35.293741+0000","last_became_peered":"2024-04-24T06:41:35.293741+0000","last_unstale":"2024-04-24T06:41:36.446635+0000","last_undegraded":"2024-04-24T06:41:36.446635+0000","last_fullsized":"2024-04-24T06:41:36.446635+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:09:12.495187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:39.334384+0000","last_change":"2024-04-24T06:41:34.276666+0000","last_active":"2024-04-24T06:41:39.334384+0000","last_peered":"2024-04-24T06:41:39.334384+0000","last_clean":"2024-04-24T06:41:39.334384+0000","last_became_active":"2024-04-24T06:41:34.276285+0000","last_became_peered":"2024-04-24T06:41:34.276285+0000","last_unstale":"2024-04-24T06:41:39.334384+0000","last_undegraded":"2024-04-24T06:41:39.334384+0000","last_fullsized":"2024-04-24T06:41:39.334384+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:02:31.230583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.276590+0000","last_change":"2024-04-24T06:41:35.296555+0000","last_active":"2024-04-24T06:41:36.276590+0000","last_peered":"2024-04-24T06:41:36.276590+0000","last_clean":"2024-04-24T06:41:36.276590+0000","last_became_active":"2024-04-24T06:41:35.296011+0000","last_became_peered":"2024-04-24T06:41:35.296011+0000","last_unstale":"2024-04-24T06:41:36.276590+0000","last_undegraded":"2024-04-24T06:41:36.276590+0000","last_fullsized":"2024-04-24T06:41:36.276590+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:34:03.081573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:39.335640+0000","last_change":"2024-04-24T06:41:34.284958+0000","last_active":"2024-04-24T06:41:39.335640+0000","last_peered":"2024-04-24T06:41:39.335640+0000","last_clean":"2024-04-24T06:41:39.335640+0000","last_became_active":"2024-04-24T06:41:34.284279+0000","last_became_peered":"2024-04-24T06:41:34.284279+0000","last_unstale":"2024-04-24T06:41:39.335640+0000","last_undegraded":"2024-04-24T06:41:39.335640+0000","last_fullsized":"2024-04-24T06:41:39.335640+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:58:19.316091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.275895+0000","last_change":"2024-04-24T06:41:35.307280+0000","last_active":"2024-04-24T06:41:36.275895+0000","last_peered":"2024-04-24T06:41:36.275895+0000","last_clean":"2024-04-24T06:41:36.275895+0000","last_became_active":"2024-04-24T06:41:35.306908+0000","last_became_peered":"2024-04-24T06:41:35.306908+0000","last_unstale":"2024-04-24T06:41:36.275895+0000","last_undegraded":"2024-04-24T06:41:36.275895+0000","last_fullsized":"2024-04-24T06:41:36.275895+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:54:31.306302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.276211+0000","last_change":"2024-04-24T06:41:34.272221+0000","last_active":"2024-04-24T06:41:36.276211+0000","last_peered":"2024-04-24T06:41:36.276211+0000","last_clean":"2024-04-24T06:41:36.276211+0000","last_became_active":"2024-04-24T06:41:34.271748+0000","last_became_peered":"2024-04-24T06:41:34.271748+0000","last_unstale":"2024-04-24T06:41:36.276211+0000","last_undegraded":"2024-04-24T06:41:36.276211+0000","last_fullsized":"2024-04-24T06:41:36.276211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:29:02.667285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.445003+0000","last_change":"2024-04-24T06:41:35.309608+0000","last_active":"2024-04-24T06:41:36.445003+0000","last_peered":"2024-04-24T06:41:36.445003+0000","last_clean":"2024-04-24T06:41:36.445003+0000","last_became_active":"2024-04-24T06:41:35.309227+0000","last_became_peered":"2024-04-24T06:41:35.309227+0000","last_unstale":"2024-04-24T06:41:36.445003+0000","last_undegraded":"2024-04-24T06:41:36.445003+0000","last_fullsized":"2024-04-24T06:41:36.445003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:51:23.151426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:41.343110+0000","last_change":"2024-04-24T06:41:34.281885+0000","last_active":"2024-04-24T06:41:41.343110+0000","last_peered":"2024-04-24T06:41:41.343110+0000","last_clean":"2024-04-24T06:41:41.343110+0000","last_became_active":"2024-04-24T06:41:34.281466+0000","last_became_peered":"2024-04-24T06:41:34.281466+0000","last_unstale":"2024-04-24T06:41:41.343110+0000","last_undegraded":"2024-04-24T06:41:41.343110+0000","last_fullsized":"2024-04-24T06:41:41.343110+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:20:08.500343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.469405+0000","last_change":"2024-04-24T06:41:35.292714+0000","last_active":"2024-04-24T06:41:36.469405+0000","last_peered":"2024-04-24T06:41:36.469405+0000","last_clean":"2024-04-24T06:41:36.469405+0000","last_became_active":"2024-04-24T06:41:35.291950+0000","last_became_peered":"2024-04-24T06:41:35.291950+0000","last_unstale":"2024-04-24T06:41:36.469405+0000","last_undegraded":"2024-04-24T06:41:36.469405+0000","last_fullsized":"2024-04-24T06:41:36.469405+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:42:18.049746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:38.316320+0000","last_change":"2024-04-24T06:41:34.275218+0000","last_active":"2024-04-24T06:41:38.316320+0000","last_peered":"2024-04-24T06:41:38.316320+0000","last_clean":"2024-04-24T06:41:38.316320+0000","last_became_active":"2024-04-24T06:41:34.274824+0000","last_became_peered":"2024-04-24T06:41:34.274824+0000","last_unstale":"2024-04-24T06:41:38.316320+0000","last_undegraded":"2024-04-24T06:41:38.316320+0000","last_fullsized":"2024-04-24T06:41:38.316320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:09:57.271304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.470888+0000","last_change":"2024-04-24T06:41:35.286311+0000","last_active":"2024-04-24T06:41:36.470888+0000","last_peered":"2024-04-24T06:41:36.470888+0000","last_clean":"2024-04-24T06:41:36.470888+0000","last_became_active":"2024-04-24T06:41:35.284643+0000","last_became_peered":"2024-04-24T06:41:35.284643+0000","last_unstale":"2024-04-24T06:41:36.470888+0000","last_undegraded":"2024-04-24T06:41:36.470888+0000","last_fullsized":"2024-04-24T06:41:36.470888+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:08:26.760816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.276775+0000","last_change":"2024-04-24T06:41:34.277372+0000","last_active":"2024-04-24T06:41:36.276775+0000","last_peered":"2024-04-24T06:41:36.276775+0000","last_clean":"2024-04-24T06:41:36.276775+0000","last_became_active":"2024-04-24T06:41:34.276997+0000","last_became_peered":"2024-04-24T06:41:34.276997+0000","last_unstale":"2024-04-24T06:41:36.276775+0000","last_undegraded":"2024-04-24T06:41:36.276775+0000","last_fullsized":"2024-04-24T06:41:36.276775+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:52:35.695109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.469671+0000","last_change":"2024-04-24T06:41:35.290233+0000","last_active":"2024-04-24T06:41:36.469671+0000","last_peered":"2024-04-24T06:41:36.469671+0000","last_clean":"2024-04-24T06:41:36.469671+0000","last_became_active":"2024-04-24T06:41:35.289885+0000","last_became_peered":"2024-04-24T06:41:35.289885+0000","last_unstale":"2024-04-24T06:41:36.469671+0000","last_undegraded":"2024-04-24T06:41:36.469671+0000","last_fullsized":"2024-04-24T06:41:36.469671+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:35:54.293308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'19","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:42.401418+0000","last_change":"2024-04-24T06:41:34.267693+0000","last_active":"2024-04-24T06:41:42.401418+0000","last_peered":"2024-04-24T06:41:42.401418+0000","last_clean":"2024-04-24T06:41:42.401418+0000","last_became_active":"2024-04-24T06:41:34.267298+0000","last_became_peered":"2024-04-24T06:41:34.267298+0000","last_unstale":"2024-04-24T06:41:42.401418+0000","last_undegraded":"2024-04-24T06:41:42.401418+0000","last_fullsized":"2024-04-24T06:41:42.401418+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:21:16.015695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39628,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":10,"num_read_kb":24,"num_write":20,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.446944+0000","last_change":"2024-04-24T06:41:35.286587+0000","last_active":"2024-04-24T06:41:36.446944+0000","last_peered":"2024-04-24T06:41:36.446944+0000","last_clean":"2024-04-24T06:41:36.446944+0000","last_became_active":"2024-04-24T06:41:35.286239+0000","last_became_peered":"2024-04-24T06:41:35.286239+0000","last_unstale":"2024-04-24T06:41:36.446944+0000","last_undegraded":"2024-04-24T06:41:36.446944+0000","last_fullsized":"2024-04-24T06:41:36.446944+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:18:56.013425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:39.333120+0000","last_change":"2024-04-24T06:41:34.278044+0000","last_active":"2024-04-24T06:41:39.333120+0000","last_peered":"2024-04-24T06:41:39.333120+0000","last_clean":"2024-04-24T06:41:39.333120+0000","last_became_active":"2024-04-24T06:41:34.276133+0000","last_became_peered":"2024-04-24T06:41:34.276133+0000","last_unstale":"2024-04-24T06:41:39.333120+0000","last_undegraded":"2024-04-24T06:41:39.333120+0000","last_fullsized":"2024-04-24T06:41:39.333120+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:54:32.394751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.276063+0000","last_change":"2024-04-24T06:41:35.306175+0000","last_active":"2024-04-24T06:41:36.276063+0000","last_peered":"2024-04-24T06:41:36.276063+0000","last_clean":"2024-04-24T06:41:36.276063+0000","last_became_active":"2024-04-24T06:41:35.305840+0000","last_became_peered":"2024-04-24T06:41:35.305840+0000","last_unstale":"2024-04-24T06:41:36.276063+0000","last_undegraded":"2024-04-24T06:41:36.276063+0000","last_fullsized":"2024-04-24T06:41:36.276063+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:42:46.238490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:38.322819+0000","last_change":"2024-04-24T06:41:34.277212+0000","last_active":"2024-04-24T06:41:38.322819+0000","last_peered":"2024-04-24T06:41:38.322819+0000","last_clean":"2024-04-24T06:41:38.322819+0000","last_became_active":"2024-04-24T06:41:34.276695+0000","last_became_peered":"2024-04-24T06:41:34.276695+0000","last_unstale":"2024-04-24T06:41:38.322819+0000","last_undegraded":"2024-04-24T06:41:38.322819+0000","last_fullsized":"2024-04-24T06:41:38.322819+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:00:47.169384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.471612+0000","last_change":"2024-04-24T06:41:35.305348+0000","last_active":"2024-04-24T06:41:36.471612+0000","last_peered":"2024-04-24T06:41:36.471612+0000","last_clean":"2024-04-24T06:41:36.471612+0000","last_became_active":"2024-04-24T06:41:35.304899+0000","last_became_peered":"2024-04-24T06:41:35.304899+0000","last_unstale":"2024-04-24T06:41:36.471612+0000","last_undegraded":"2024-04-24T06:41:36.471612+0000","last_fullsized":"2024-04-24T06:41:36.471612+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:49:35.737711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:39.335266+0000","last_change":"2024-04-24T06:41:34.272902+0000","last_active":"2024-04-24T06:41:39.335266+0000","last_peered":"2024-04-24T06:41:39.335266+0000","last_clean":"2024-04-24T06:41:39.335266+0000","last_became_active":"2024-04-24T06:41:34.271726+0000","last_became_peered":"2024-04-24T06:41:34.271726+0000","last_unstale":"2024-04-24T06:41:39.335266+0000","last_undegraded":"2024-04-24T06:41:39.335266+0000","last_fullsized":"2024-04-24T06:41:39.335266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:07:38.318110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.470594+0000","last_change":"2024-04-24T06:41:35.283764+0000","last_active":"2024-04-24T06:41:36.470594+0000","last_peered":"2024-04-24T06:41:36.470594+0000","last_clean":"2024-04-24T06:41:36.470594+0000","last_became_active":"2024-04-24T06:41:35.283400+0000","last_became_peered":"2024-04-24T06:41:35.283400+0000","last_unstale":"2024-04-24T06:41:36.470594+0000","last_undegraded":"2024-04-24T06:41:36.470594+0000","last_fullsized":"2024-04-24T06:41:36.470594+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:36:47.034068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:39.338140+0000","last_change":"2024-04-24T06:41:34.280020+0000","last_active":"2024-04-24T06:41:39.338140+0000","last_peered":"2024-04-24T06:41:39.338140+0000","last_clean":"2024-04-24T06:41:39.338140+0000","last_became_active":"2024-04-24T06:41:34.279659+0000","last_became_peered":"2024-04-24T06:41:34.279659+0000","last_unstale":"2024-04-24T06:41:39.338140+0000","last_undegraded":"2024-04-24T06:41:39.338140+0000","last_fullsized":"2024-04-24T06:41:39.338140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:13:29.529123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.276340+0000","last_change":"2024-04-24T06:41:35.284500+0000","last_active":"2024-04-24T06:41:36.276340+0000","last_peered":"2024-04-24T06:41:36.276340+0000","last_clean":"2024-04-24T06:41:36.276340+0000","last_became_active":"2024-04-24T06:41:35.283171+0000","last_became_peered":"2024-04-24T06:41:35.283171+0000","last_unstale":"2024-04-24T06:41:36.276340+0000","last_undegraded":"2024-04-24T06:41:36.276340+0000","last_fullsized":"2024-04-24T06:41:36.276340+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:33:19.527226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:40.330284+0000","last_change":"2024-04-24T06:41:34.279306+0000","last_active":"2024-04-24T06:41:40.330284+0000","last_peered":"2024-04-24T06:41:40.330284+0000","last_clean":"2024-04-24T06:41:40.330284+0000","last_became_active":"2024-04-24T06:41:34.278946+0000","last_became_peered":"2024-04-24T06:41:34.278946+0000","last_unstale":"2024-04-24T06:41:40.330284+0000","last_undegraded":"2024-04-24T06:41:40.330284+0000","last_fullsized":"2024-04-24T06:41:40.330284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:13:40.845871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.470113+0000","last_change":"2024-04-24T06:41:35.283716+0000","last_active":"2024-04-24T06:41:36.470113+0000","last_peered":"2024-04-24T06:41:36.470113+0000","last_clean":"2024-04-24T06:41:36.470113+0000","last_became_active":"2024-04-24T06:41:35.283330+0000","last_became_peered":"2024-04-24T06:41:35.283330+0000","last_unstale":"2024-04-24T06:41:36.470113+0000","last_undegraded":"2024-04-24T06:41:36.470113+0000","last_fullsized":"2024-04-24T06:41:36.470113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:18:13.403264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.444356+0000","last_change":"2024-04-24T06:41:34.288831+0000","last_active":"2024-04-24T06:41:36.444356+0000","last_peered":"2024-04-24T06:41:36.444356+0000","last_clean":"2024-04-24T06:41:36.444356+0000","last_became_active":"2024-04-24T06:41:34.288405+0000","last_became_peered":"2024-04-24T06:41:34.288405+0000","last_unstale":"2024-04-24T06:41:36.444356+0000","last_undegraded":"2024-04-24T06:41:36.444356+0000","last_fullsized":"2024-04-24T06:41:36.444356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:50:22.312561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:37.668529+0000","last_change":"2024-04-24T06:41:35.306607+0000","last_active":"2024-04-24T06:41:37.668529+0000","last_peered":"2024-04-24T06:41:37.668529+0000","last_clean":"2024-04-24T06:41:37.668529+0000","last_became_active":"2024-04-24T06:41:35.306217+0000","last_became_peered":"2024-04-24T06:41:35.306217+0000","last_unstale":"2024-04-24T06:41:37.668529+0000","last_undegraded":"2024-04-24T06:41:37.668529+0000","last_fullsized":"2024-04-24T06:41:37.668529+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:07:46.068876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:38.135100+0000","last_change":"2024-04-24T06:41:34.285319+0000","last_active":"2024-04-24T06:41:38.135100+0000","last_peered":"2024-04-24T06:41:38.135100+0000","last_clean":"2024-04-24T06:41:38.135100+0000","last_became_active":"2024-04-24T06:41:34.284912+0000","last_became_peered":"2024-04-24T06:41:34.284912+0000","last_unstale":"2024-04-24T06:41:38.135100+0000","last_undegraded":"2024-04-24T06:41:38.135100+0000","last_fullsized":"2024-04-24T06:41:38.135100+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:34:56.454587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.470931+0000","last_change":"2024-04-24T06:41:35.296294+0000","last_active":"2024-04-24T06:41:36.470931+0000","last_peered":"2024-04-24T06:41:36.470931+0000","last_clean":"2024-04-24T06:41:36.470931+0000","last_became_active":"2024-04-24T06:41:35.295792+0000","last_became_peered":"2024-04-24T06:41:35.295792+0000","last_unstale":"2024-04-24T06:41:36.470931+0000","last_undegraded":"2024-04-24T06:41:36.470931+0000","last_fullsized":"2024-04-24T06:41:36.470931+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:31:35.573858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.276425+0000","last_change":"2024-04-24T06:41:35.282558+0000","last_active":"2024-04-24T06:41:36.276425+0000","last_peered":"2024-04-24T06:41:36.276425+0000","last_clean":"2024-04-24T06:41:36.276425+0000","last_became_active":"2024-04-24T06:41:35.282163+0000","last_became_peered":"2024-04-24T06:41:35.282163+0000","last_unstale":"2024-04-24T06:41:36.276425+0000","last_undegraded":"2024-04-24T06:41:36.276425+0000","last_fullsized":"2024-04-24T06:41:36.276425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:12:23.569849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:39.338130+0000","last_change":"2024-04-24T06:41:34.284294+0000","last_active":"2024-04-24T06:41:39.338130+0000","last_peered":"2024-04-24T06:41:39.338130+0000","last_clean":"2024-04-24T06:41:39.338130+0000","last_became_active":"2024-04-24T06:41:34.283933+0000","last_became_peered":"2024-04-24T06:41:34.283933+0000","last_unstale":"2024-04-24T06:41:39.338130+0000","last_undegraded":"2024-04-24T06:41:39.338130+0000","last_fullsized":"2024-04-24T06:41:39.338130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:35:26.042614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.277119+0000","last_change":"2024-04-24T06:41:35.302361+0000","last_active":"2024-04-24T06:41:36.277119+0000","last_peered":"2024-04-24T06:41:36.277119+0000","last_clean":"2024-04-24T06:41:36.277119+0000","last_became_active":"2024-04-24T06:41:35.301652+0000","last_became_peered":"2024-04-24T06:41:35.301652+0000","last_unstale":"2024-04-24T06:41:36.277119+0000","last_undegraded":"2024-04-24T06:41:36.277119+0000","last_fullsized":"2024-04-24T06:41:36.277119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:35:58.010853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:38.321768+0000","last_change":"2024-04-24T06:41:34.282156+0000","last_active":"2024-04-24T06:41:38.321768+0000","last_peered":"2024-04-24T06:41:38.321768+0000","last_clean":"2024-04-24T06:41:38.321768+0000","last_became_active":"2024-04-24T06:41:34.281771+0000","last_became_peered":"2024-04-24T06:41:34.281771+0000","last_unstale":"2024-04-24T06:41:38.321768+0000","last_undegraded":"2024-04-24T06:41:38.321768+0000","last_fullsized":"2024-04-24T06:41:38.321768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:41:49.584620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":350,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.470757+0000","last_change":"2024-04-24T06:40:16.681695+0000","last_active":"2024-04-24T06:41:36.470757+0000","last_peered":"2024-04-24T06:41:36.470757+0000","last_clean":"2024-04-24T06:41:36.470757+0000","last_became_active":"2024-04-24T06:40:16.681208+0000","last_became_peered":"2024-04-24T06:40:16.681208+0000","last_unstale":"2024-04-24T06:41:36.470757+0000","last_undegraded":"2024-04-24T06:41:36.470757+0000","last_fullsized":"2024-04-24T06:41:36.470757+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:36:27.950574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:36:27.950574+0000","last_clean_scrub_stamp":"2024-04-24T06:36:27.950574+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:39:01.985893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.274890+0000","last_change":"2024-04-24T06:41:35.303109+0000","last_active":"2024-04-24T06:41:36.274890+0000","last_peered":"2024-04-24T06:41:36.274890+0000","last_clean":"2024-04-24T06:41:36.274890+0000","last_became_active":"2024-04-24T06:41:35.302755+0000","last_became_peered":"2024-04-24T06:41:35.302755+0000","last_unstale":"2024-04-24T06:41:36.274890+0000","last_undegraded":"2024-04-24T06:41:36.274890+0000","last_fullsized":"2024-04-24T06:41:36.274890+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:22:56.075740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:39.331760+0000","last_change":"2024-04-24T06:41:34.273872+0000","last_active":"2024-04-24T06:41:39.331760+0000","last_peered":"2024-04-24T06:41:39.331760+0000","last_clean":"2024-04-24T06:41:39.331760+0000","last_became_active":"2024-04-24T06:41:34.273473+0000","last_became_peered":"2024-04-24T06:41:34.273473+0000","last_unstale":"2024-04-24T06:41:39.331760+0000","last_undegraded":"2024-04-24T06:41:39.331760+0000","last_fullsized":"2024-04-24T06:41:39.331760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:47:11.914985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:38.320281+0000","last_change":"2024-04-24T06:41:34.289293+0000","last_active":"2024-04-24T06:41:38.320281+0000","last_peered":"2024-04-24T06:41:38.320281+0000","last_clean":"2024-04-24T06:41:38.320281+0000","last_became_active":"2024-04-24T06:41:34.288939+0000","last_became_peered":"2024-04-24T06:41:34.288939+0000","last_unstale":"2024-04-24T06:41:38.320281+0000","last_undegraded":"2024-04-24T06:41:38.320281+0000","last_fullsized":"2024-04-24T06:41:38.320281+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:03:01.026792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.277972+0000","last_change":"2024-04-24T06:41:35.302153+0000","last_active":"2024-04-24T06:41:36.277972+0000","last_peered":"2024-04-24T06:41:36.277972+0000","last_clean":"2024-04-24T06:41:36.277972+0000","last_became_active":"2024-04-24T06:41:35.301666+0000","last_became_peered":"2024-04-24T06:41:35.301666+0000","last_unstale":"2024-04-24T06:41:36.277972+0000","last_undegraded":"2024-04-24T06:41:36.277972+0000","last_fullsized":"2024-04-24T06:41:36.277972+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:45:09.642482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:39.338104+0000","last_change":"2024-04-24T06:41:34.291570+0000","last_active":"2024-04-24T06:41:39.338104+0000","last_peered":"2024-04-24T06:41:39.338104+0000","last_clean":"2024-04-24T06:41:39.338104+0000","last_became_active":"2024-04-24T06:41:34.289700+0000","last_became_peered":"2024-04-24T06:41:34.289700+0000","last_unstale":"2024-04-24T06:41:39.338104+0000","last_undegraded":"2024-04-24T06:41:39.338104+0000","last_fullsized":"2024-04-24T06:41:39.338104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:57:20.537299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.278921+0000","last_change":"2024-04-24T06:41:35.304703+0000","last_active":"2024-04-24T06:41:36.278921+0000","last_peered":"2024-04-24T06:41:36.278921+0000","last_clean":"2024-04-24T06:41:36.278921+0000","last_became_active":"2024-04-24T06:41:35.304319+0000","last_became_peered":"2024-04-24T06:41:35.304319+0000","last_unstale":"2024-04-24T06:41:36.278921+0000","last_undegraded":"2024-04-24T06:41:36.278921+0000","last_fullsized":"2024-04-24T06:41:36.278921+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:00:56.069509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:38.318761+0000","last_change":"2024-04-24T06:41:34.276866+0000","last_active":"2024-04-24T06:41:38.318761+0000","last_peered":"2024-04-24T06:41:38.318761+0000","last_clean":"2024-04-24T06:41:38.318761+0000","last_became_active":"2024-04-24T06:41:34.276478+0000","last_became_peered":"2024-04-24T06:41:34.276478+0000","last_unstale":"2024-04-24T06:41:38.318761+0000","last_undegraded":"2024-04-24T06:41:38.318761+0000","last_fullsized":"2024-04-24T06:41:38.318761+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:10:35.047960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.470382+0000","last_change":"2024-04-24T06:41:35.300951+0000","last_active":"2024-04-24T06:41:36.470382+0000","last_peered":"2024-04-24T06:41:36.470382+0000","last_clean":"2024-04-24T06:41:36.470382+0000","last_became_active":"2024-04-24T06:41:35.296176+0000","last_became_peered":"2024-04-24T06:41:35.296176+0000","last_unstale":"2024-04-24T06:41:36.470382+0000","last_undegraded":"2024-04-24T06:41:36.470382+0000","last_fullsized":"2024-04-24T06:41:36.470382+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:53:58.497437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:38.320407+0000","last_change":"2024-04-24T06:41:34.276154+0000","last_active":"2024-04-24T06:41:38.320407+0000","last_peered":"2024-04-24T06:41:38.320407+0000","last_clean":"2024-04-24T06:41:38.320407+0000","last_became_active":"2024-04-24T06:41:34.275734+0000","last_became_peered":"2024-04-24T06:41:34.275734+0000","last_unstale":"2024-04-24T06:41:38.320407+0000","last_undegraded":"2024-04-24T06:41:38.320407+0000","last_fullsized":"2024-04-24T06:41:38.320407+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:28:54.618061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.446629+0000","last_change":"2024-04-24T06:41:35.285837+0000","last_active":"2024-04-24T06:41:36.446629+0000","last_peered":"2024-04-24T06:41:36.446629+0000","last_clean":"2024-04-24T06:41:36.446629+0000","last_became_active":"2024-04-24T06:41:35.285505+0000","last_became_peered":"2024-04-24T06:41:35.285505+0000","last_unstale":"2024-04-24T06:41:36.446629+0000","last_undegraded":"2024-04-24T06:41:36.446629+0000","last_fullsized":"2024-04-24T06:41:36.446629+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:36:53.460780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:38.323741+0000","last_change":"2024-04-24T06:41:34.290039+0000","last_active":"2024-04-24T06:41:38.323741+0000","last_peered":"2024-04-24T06:41:38.323741+0000","last_clean":"2024-04-24T06:41:38.323741+0000","last_became_active":"2024-04-24T06:41:34.289228+0000","last_became_peered":"2024-04-24T06:41:34.289228+0000","last_unstale":"2024-04-24T06:41:38.323741+0000","last_undegraded":"2024-04-24T06:41:38.323741+0000","last_fullsized":"2024-04-24T06:41:38.323741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:07:20.342359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.469847+0000","last_change":"2024-04-24T06:41:35.296214+0000","last_active":"2024-04-24T06:41:36.469847+0000","last_peered":"2024-04-24T06:41:36.469847+0000","last_clean":"2024-04-24T06:41:36.469847+0000","last_became_active":"2024-04-24T06:41:35.295888+0000","last_became_peered":"2024-04-24T06:41:35.295888+0000","last_unstale":"2024-04-24T06:41:36.469847+0000","last_undegraded":"2024-04-24T06:41:36.469847+0000","last_fullsized":"2024-04-24T06:41:36.469847+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:55:19.520479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:37.379729+0000","last_change":"2024-04-24T06:41:34.275885+0000","last_active":"2024-04-24T06:41:37.379729+0000","last_peered":"2024-04-24T06:41:37.379729+0000","last_clean":"2024-04-24T06:41:37.379729+0000","last_became_active":"2024-04-24T06:41:34.275503+0000","last_became_peered":"2024-04-24T06:41:34.275503+0000","last_unstale":"2024-04-24T06:41:37.379729+0000","last_undegraded":"2024-04-24T06:41:37.379729+0000","last_fullsized":"2024-04-24T06:41:37.379729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:50:58.216699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.277124+0000","last_change":"2024-04-24T06:41:35.305330+0000","last_active":"2024-04-24T06:41:36.277124+0000","last_peered":"2024-04-24T06:41:36.277124+0000","last_clean":"2024-04-24T06:41:36.277124+0000","last_became_active":"2024-04-24T06:41:35.301262+0000","last_became_peered":"2024-04-24T06:41:35.301262+0000","last_unstale":"2024-04-24T06:41:36.277124+0000","last_undegraded":"2024-04-24T06:41:36.277124+0000","last_fullsized":"2024-04-24T06:41:36.277124+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:58:41.029511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:38.317955+0000","last_change":"2024-04-24T06:41:34.276926+0000","last_active":"2024-04-24T06:41:38.317955+0000","last_peered":"2024-04-24T06:41:38.317955+0000","last_clean":"2024-04-24T06:41:38.317955+0000","last_became_active":"2024-04-24T06:41:34.276539+0000","last_became_peered":"2024-04-24T06:41:34.276539+0000","last_unstale":"2024-04-24T06:41:38.317955+0000","last_undegraded":"2024-04-24T06:41:38.317955+0000","last_fullsized":"2024-04-24T06:41:38.317955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:28:56.652584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.471444+0000","last_change":"2024-04-24T06:41:35.301052+0000","last_active":"2024-04-24T06:41:36.471444+0000","last_peered":"2024-04-24T06:41:36.471444+0000","last_clean":"2024-04-24T06:41:36.471444+0000","last_became_active":"2024-04-24T06:41:35.300650+0000","last_became_peered":"2024-04-24T06:41:35.300650+0000","last_unstale":"2024-04-24T06:41:36.471444+0000","last_undegraded":"2024-04-24T06:41:36.471444+0000","last_fullsized":"2024-04-24T06:41:36.471444+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:06:54.290353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:39.333903+0000","last_change":"2024-04-24T06:41:34.273773+0000","last_active":"2024-04-24T06:41:39.333903+0000","last_peered":"2024-04-24T06:41:39.333903+0000","last_clean":"2024-04-24T06:41:39.333903+0000","last_became_active":"2024-04-24T06:41:34.273324+0000","last_became_peered":"2024-04-24T06:41:34.273324+0000","last_unstale":"2024-04-24T06:41:39.333903+0000","last_undegraded":"2024-04-24T06:41:39.333903+0000","last_fullsized":"2024-04-24T06:41:39.333903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:30:30.760740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.471784+0000","last_change":"2024-04-24T06:41:35.289565+0000","last_active":"2024-04-24T06:41:36.471784+0000","last_peered":"2024-04-24T06:41:36.471784+0000","last_clean":"2024-04-24T06:41:36.471784+0000","last_became_active":"2024-04-24T06:41:35.289185+0000","last_became_peered":"2024-04-24T06:41:35.289185+0000","last_unstale":"2024-04-24T06:41:36.471784+0000","last_undegraded":"2024-04-24T06:41:36.471784+0000","last_fullsized":"2024-04-24T06:41:36.471784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:26:28.345162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.276540+0000","last_change":"2024-04-24T06:41:34.285804+0000","last_active":"2024-04-24T06:41:36.276540+0000","last_peered":"2024-04-24T06:41:36.276540+0000","last_clean":"2024-04-24T06:41:36.276540+0000","last_became_active":"2024-04-24T06:41:34.285501+0000","last_became_peered":"2024-04-24T06:41:34.285501+0000","last_unstale":"2024-04-24T06:41:36.276540+0000","last_undegraded":"2024-04-24T06:41:36.276540+0000","last_fullsized":"2024-04-24T06:41:36.276540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:25:15.804221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:38.135125+0000","last_change":"2024-04-24T06:41:35.300856+0000","last_active":"2024-04-24T06:41:38.135125+0000","last_peered":"2024-04-24T06:41:38.135125+0000","last_clean":"2024-04-24T06:41:38.135125+0000","last_became_active":"2024-04-24T06:41:35.300344+0000","last_became_peered":"2024-04-24T06:41:35.300344+0000","last_unstale":"2024-04-24T06:41:38.135125+0000","last_undegraded":"2024-04-24T06:41:38.135125+0000","last_fullsized":"2024-04-24T06:41:38.135125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:10:01.866954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.278567+0000","last_change":"2024-04-24T06:41:34.284112+0000","last_active":"2024-04-24T06:41:36.278567+0000","last_peered":"2024-04-24T06:41:36.278567+0000","last_clean":"2024-04-24T06:41:36.278567+0000","last_became_active":"2024-04-24T06:41:34.279317+0000","last_became_peered":"2024-04-24T06:41:34.279317+0000","last_unstale":"2024-04-24T06:41:36.278567+0000","last_undegraded":"2024-04-24T06:41:36.278567+0000","last_fullsized":"2024-04-24T06:41:36.278567+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:49:45.673353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.471288+0000","last_change":"2024-04-24T06:41:35.288360+0000","last_active":"2024-04-24T06:41:36.471288+0000","last_peered":"2024-04-24T06:41:36.471288+0000","last_clean":"2024-04-24T06:41:36.471288+0000","last_became_active":"2024-04-24T06:41:35.288019+0000","last_became_peered":"2024-04-24T06:41:35.288019+0000","last_unstale":"2024-04-24T06:41:36.471288+0000","last_undegraded":"2024-04-24T06:41:36.471288+0000","last_fullsized":"2024-04-24T06:41:36.471288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:33:49.927591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.469862+0000","last_change":"2024-04-24T06:41:35.286137+0000","last_active":"2024-04-24T06:41:36.469862+0000","last_peered":"2024-04-24T06:41:36.469862+0000","last_clean":"2024-04-24T06:41:36.469862+0000","last_became_active":"2024-04-24T06:41:35.285773+0000","last_became_peered":"2024-04-24T06:41:35.285773+0000","last_unstale":"2024-04-24T06:41:36.469862+0000","last_undegraded":"2024-04-24T06:41:36.469862+0000","last_fullsized":"2024-04-24T06:41:36.469862+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:29:32.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.469496+0000","last_change":"2024-04-24T06:41:35.283977+0000","last_active":"2024-04-24T06:41:36.469496+0000","last_peered":"2024-04-24T06:41:36.469496+0000","last_clean":"2024-04-24T06:41:36.469496+0000","last_became_active":"2024-04-24T06:41:35.282312+0000","last_became_peered":"2024-04-24T06:41:35.282312+0000","last_unstale":"2024-04-24T06:41:36.469496+0000","last_undegraded":"2024-04-24T06:41:36.469496+0000","last_fullsized":"2024-04-24T06:41:36.469496+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:41:36.463803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.469605+0000","last_change":"2024-04-24T06:41:35.300372+0000","last_active":"2024-04-24T06:41:36.469605+0000","last_peered":"2024-04-24T06:41:36.469605+0000","last_clean":"2024-04-24T06:41:36.469605+0000","last_became_active":"2024-04-24T06:41:35.300034+0000","last_became_peered":"2024-04-24T06:41:35.300034+0000","last_unstale":"2024-04-24T06:41:36.469605+0000","last_undegraded":"2024-04-24T06:41:36.469605+0000","last_fullsized":"2024-04-24T06:41:36.469605+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:56:23.732798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.470495+0000","last_change":"2024-04-24T06:41:35.301619+0000","last_active":"2024-04-24T06:41:36.470495+0000","last_peered":"2024-04-24T06:41:36.470495+0000","last_clean":"2024-04-24T06:41:36.470495+0000","last_became_active":"2024-04-24T06:41:35.298897+0000","last_became_peered":"2024-04-24T06:41:35.298897+0000","last_unstale":"2024-04-24T06:41:36.470495+0000","last_undegraded":"2024-04-24T06:41:36.470495+0000","last_fullsized":"2024-04-24T06:41:36.470495+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:26:11.583079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.278245+0000","last_change":"2024-04-24T06:41:35.304905+0000","last_active":"2024-04-24T06:41:36.278245+0000","last_peered":"2024-04-24T06:41:36.278245+0000","last_clean":"2024-04-24T06:41:36.278245+0000","last_became_active":"2024-04-24T06:41:35.304407+0000","last_became_peered":"2024-04-24T06:41:35.304407+0000","last_unstale":"2024-04-24T06:41:36.278245+0000","last_undegraded":"2024-04-24T06:41:36.278245+0000","last_fullsized":"2024-04-24T06:41:36.278245+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:38:07.417898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.469810+0000","last_change":"2024-04-24T06:41:35.288646+0000","last_active":"2024-04-24T06:41:36.469810+0000","last_peered":"2024-04-24T06:41:36.469810+0000","last_clean":"2024-04-24T06:41:36.469810+0000","last_became_active":"2024-04-24T06:41:35.288266+0000","last_became_peered":"2024-04-24T06:41:35.288266+0000","last_unstale":"2024-04-24T06:41:36.469810+0000","last_undegraded":"2024-04-24T06:41:36.469810+0000","last_fullsized":"2024-04-24T06:41:36.469810+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:59:16.526712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.470133+0000","last_change":"2024-04-24T06:41:35.285001+0000","last_active":"2024-04-24T06:41:36.470133+0000","last_peered":"2024-04-24T06:41:36.470133+0000","last_clean":"2024-04-24T06:41:36.470133+0000","last_became_active":"2024-04-24T06:41:35.284671+0000","last_became_peered":"2024-04-24T06:41:35.284671+0000","last_unstale":"2024-04-24T06:41:36.470133+0000","last_undegraded":"2024-04-24T06:41:36.470133+0000","last_fullsized":"2024-04-24T06:41:36.470133+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:37:43.534018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.278660+0000","last_change":"2024-04-24T06:41:35.299124+0000","last_active":"2024-04-24T06:41:36.278660+0000","last_peered":"2024-04-24T06:41:36.278660+0000","last_clean":"2024-04-24T06:41:36.278660+0000","last_became_active":"2024-04-24T06:41:35.298246+0000","last_became_peered":"2024-04-24T06:41:35.298246+0000","last_unstale":"2024-04-24T06:41:36.278660+0000","last_undegraded":"2024-04-24T06:41:36.278660+0000","last_fullsized":"2024-04-24T06:41:36.278660+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:07:22.649874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.276423+0000","last_change":"2024-04-24T06:41:35.296588+0000","last_active":"2024-04-24T06:41:36.276423+0000","last_peered":"2024-04-24T06:41:36.276423+0000","last_clean":"2024-04-24T06:41:36.276423+0000","last_became_active":"2024-04-24T06:41:35.296026+0000","last_became_peered":"2024-04-24T06:41:35.296026+0000","last_unstale":"2024-04-24T06:41:36.276423+0000","last_undegraded":"2024-04-24T06:41:36.276423+0000","last_fullsized":"2024-04-24T06:41:36.276423+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:17:50.627727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.277958+0000","last_change":"2024-04-24T06:41:35.301619+0000","last_active":"2024-04-24T06:41:36.277958+0000","last_peered":"2024-04-24T06:41:36.277958+0000","last_clean":"2024-04-24T06:41:36.277958+0000","last_became_active":"2024-04-24T06:41:35.297987+0000","last_became_peered":"2024-04-24T06:41:35.297987+0000","last_unstale":"2024-04-24T06:41:36.277958+0000","last_undegraded":"2024-04-24T06:41:36.277958+0000","last_fullsized":"2024-04-24T06:41:36.277958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:06:01.235593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.469686+0000","last_change":"2024-04-24T06:41:35.304347+0000","last_active":"2024-04-24T06:41:36.469686+0000","last_peered":"2024-04-24T06:41:36.469686+0000","last_clean":"2024-04-24T06:41:36.469686+0000","last_became_active":"2024-04-24T06:41:35.303971+0000","last_became_peered":"2024-04-24T06:41:35.303971+0000","last_unstale":"2024-04-24T06:41:36.469686+0000","last_undegraded":"2024-04-24T06:41:36.469686+0000","last_fullsized":"2024-04-24T06:41:36.469686+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:31:54.813243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.276274+0000","last_change":"2024-04-24T06:41:35.308314+0000","last_active":"2024-04-24T06:41:36.276274+0000","last_peered":"2024-04-24T06:41:36.276274+0000","last_clean":"2024-04-24T06:41:36.276274+0000","last_became_active":"2024-04-24T06:41:35.306581+0000","last_became_peered":"2024-04-24T06:41:35.306581+0000","last_unstale":"2024-04-24T06:41:36.276274+0000","last_undegraded":"2024-04-24T06:41:36.276274+0000","last_fullsized":"2024-04-24T06:41:36.276274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:34:43.095087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.469706+0000","last_change":"2024-04-24T06:41:35.289166+0000","last_active":"2024-04-24T06:41:36.469706+0000","last_peered":"2024-04-24T06:41:36.469706+0000","last_clean":"2024-04-24T06:41:36.469706+0000","last_became_active":"2024-04-24T06:41:35.287672+0000","last_became_peered":"2024-04-24T06:41:35.287672+0000","last_unstale":"2024-04-24T06:41:36.469706+0000","last_undegraded":"2024-04-24T06:41:36.469706+0000","last_fullsized":"2024-04-24T06:41:36.469706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:31:45.403387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":43238,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":22,"num_read_kb":36,"num_write":138,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":72091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350990,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":443597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612627,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1585152,"data_stored":1055973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776152,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":946176,"data_stored":443833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037791,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":443753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299428,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28520,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716632,"statfs":{"total":95995035648,"available":95965831168,"internally_reserved":0,"allocated":1540096,"data_stored":1035292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462954,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":443105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":937984,"data_stored":444627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":929792,"data_stored":443105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411395,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":978944,"data_stored":465260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182457,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":443209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920821,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":443659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724592,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32692,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712460,"statfs":{"total":95995035648,"available":95961559040,"internally_reserved":0,"allocated":1548288,"data_stored":1034859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":22245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":41206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T06:41:44.403 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T06:41:44.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.654+0000 7fa104e00640 1 -- 172.21.15.5:0/3611737944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa100103a40 msgr2=0x7fa10010a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.654+0000 7fa104e00640 1 --2- 172.21.15.5:0/3611737944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa100103a40 0x7fa10010a360 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8009ca0 tx=0x7fa0f802a8c0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.654+0000 7fa104e00640 1 -- 172.21.15.5:0/3611737944 shutdown_connections 2024-04-24T06:41:44.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.654+0000 7fa104e00640 1 --2- 172.21.15.5:0/3611737944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa100103a40 0x7fa10010a360 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.654+0000 7fa104e00640 1 --2- 172.21.15.5:0/3611737944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa100103080 0x7fa100103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.654+0000 7fa104e00640 1 --2- 172.21.15.5:0/3611737944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa100101f30 0x7fa100102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.654+0000 7fa104e00640 1 -- 172.21.15.5:0/3611737944 >> 172.21.15.5:0/3611737944 conn(0x7fa1000fd4c0 msgr2=0x7fa1000ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:44.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.654+0000 7fa104e00640 1 -- 172.21.15.5:0/3611737944 shutdown_connections 2024-04-24T06:41:44.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.654+0000 7fa104e00640 1 -- 172.21.15.5:0/3611737944 wait complete. 2024-04-24T06:41:44.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.655+0000 7fa104e00640 1 Processor -- start 2024-04-24T06:41:44.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.655+0000 7fa104e00640 1 -- start start 2024-04-24T06:41:44.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.656+0000 7fa104e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa100101f30 0x7fa1001981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.656+0000 7fa104e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa100103080 0x7fa100198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.656+0000 7fa104e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa100103a40 0x7fa10019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.656+0000 7fa104e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa10010dc30 con 0x7fa100103080 2024-04-24T06:41:44.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.656+0000 7fa104e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa10010dab0 con 0x7fa100103a40 2024-04-24T06:41:44.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.656+0000 7fa104e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa10010ddb0 con 0x7fa100101f30 2024-04-24T06:41:44.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.656+0000 7fa0fea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa100103080 0x7fa100198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.656+0000 7fa0fea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa100103080 0x7fa100198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43948/0 (socket says 172.21.15.5:43948) 2024-04-24T06:41:44.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.656+0000 7fa0ffe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa100103a40 0x7fa10019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.656+0000 7fa0fea00640 1 -- 172.21.15.5:0/3325353376 learned_addr learned my addr 172.21.15.5:0/3325353376 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:41:44.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.656+0000 7fa0ff400640 1 --2- 172.21.15.5:0/3325353376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa100101f30 0x7fa1001981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.656+0000 7fa0fea00640 1 -- 172.21.15.5:0/3325353376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa100101f30 msgr2=0x7fa1001981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.656+0000 7fa0fea00640 1 --2- 172.21.15.5:0/3325353376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa100101f30 0x7fa1001981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.656+0000 7fa0fea00640 1 -- 172.21.15.5:0/3325353376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa100103a40 msgr2=0x7fa10019f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.656+0000 7fa0fea00640 1 --2- 172.21.15.5:0/3325353376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa100103a40 0x7fa10019f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.656+0000 7fa0fea00640 1 -- 172.21.15.5:0/3325353376 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa10019fe00 con 0x7fa100103080 2024-04-24T06:41:44.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.657+0000 7fa0fea00640 1 --2- 172.21.15.5:0/3325353376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa100103080 0x7fa100198720 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00e9c0 tx=0x7fa0ec00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.657+0000 7fa0f7e00640 1 -- 172.21.15.5:0/3325353376 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec00cd80 con 0x7fa100103080 2024-04-24T06:41:44.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.657+0000 7fa104e00640 1 -- 172.21.15.5:0/3325353376 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1001a00f0 con 0x7fa100103080 2024-04-24T06:41:44.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.657+0000 7fa0f7e00640 1 -- 172.21.15.5:0/3325353376 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0ec0035f0 con 0x7fa100103080 2024-04-24T06:41:44.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.657+0000 7fa0f7e00640 1 -- 172.21.15.5:0/3325353376 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec010690 con 0x7fa100103080 2024-04-24T06:41:44.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.657+0000 7fa104e00640 1 -- 172.21.15.5:0/3325353376 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1001a0640 con 0x7fa100103080 2024-04-24T06:41:44.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.659+0000 7fa104e00640 1 -- 172.21.15.5:0/3325353376 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1001091e0 con 0x7fa100103080 2024-04-24T06:41:44.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.661+0000 7fa0f7e00640 1 -- 172.21.15.5:0/3325353376 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7fa0ec0037a0 con 0x7fa100103080 2024-04-24T06:41:44.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.663+0000 7fa0f7e00640 1 --2- 172.21.15.5:0/3325353376 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fa0e40a3c30 0x7fa0e40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:44.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.663+0000 7fa0f7e00640 1 -- 172.21.15.5:0/3325353376 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fa0ec0d0840 con 0x7fa100103080 2024-04-24T06:41:44.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.663+0000 7fa0ff400640 1 --2- 172.21.15.5:0/3325353376 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fa0e40a3c30 0x7fa0e40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:44.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.664+0000 7fa0ff400640 1 --2- 172.21.15.5:0/3325353376 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fa0e40a3c30 0x7fa0e40a60f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0009bd0 tx=0x7fa0f0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:44.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.667+0000 7fa0f7e00640 1 -- 172.21.15.5:0/3325353376 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa0ec09d170 con 0x7fa100103080 2024-04-24T06:41:44.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:44 smithi137 ceph-mon[33017]: pgmap v317: 97 pgs: 97 active+clean; 619 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-04-24T06:41:44.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:44 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/3035687116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:44.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:44 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/3969552520' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:41:44.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:44 smithi005 ceph-mon[27798]: pgmap v317: 97 pgs: 97 active+clean; 619 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-04-24T06:41:44.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:44 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/3035687116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:44.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:44 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/3969552520' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:41:44.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:44 smithi037 ceph-mon[32864]: pgmap v317: 97 pgs: 97 active+clean; 619 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 KiB/s wr, 20 op/s 2024-04-24T06:41:44.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:44 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/3035687116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:41:44.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:44 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/3969552520' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:41:44.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.871+0000 7fa104e00640 1 -- 172.21.15.5:0/3325353376 --> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa100062440 con 0x7fa0e40a3c30 2024-04-24T06:41:44.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.895+0000 7fa0f7e00640 1 -- 172.21.15.5:0/3325353376 <== mgr.14152 v2:172.21.15.5:6800/4042936965 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258754 (secure 0 0 0) 0x7fa100062440 con 0x7fa0e40a3c30 2024-04-24T06:41:44.900 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:41:44.901 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-24T06:41:44.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.901+0000 7fa104e00640 1 -- 172.21.15.5:0/3325353376 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fa0e40a3c30 msgr2=0x7fa0e40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.901+0000 7fa104e00640 1 --2- 172.21.15.5:0/3325353376 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fa0e40a3c30 0x7fa0e40a60f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0009bd0 tx=0x7fa0f0005f90 comp rx=0 tx=0).stop 2024-04-24T06:41:44.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.901+0000 7fa104e00640 1 -- 172.21.15.5:0/3325353376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa100103080 msgr2=0x7fa100198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:44.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.901+0000 7fa104e00640 1 --2- 172.21.15.5:0/3325353376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa100103080 0x7fa100198720 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00e9c0 tx=0x7fa0ec00ee80 comp rx=0 tx=0).stop 2024-04-24T06:41:44.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.901+0000 7fa104e00640 1 -- 172.21.15.5:0/3325353376 shutdown_connections 2024-04-24T06:41:44.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.901+0000 7fa104e00640 1 --2- 172.21.15.5:0/3325353376 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fa0e40a3c30 0x7fa0e40a60f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.901+0000 7fa104e00640 1 --2- 172.21.15.5:0/3325353376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa100103a40 0x7fa10019f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.901+0000 7fa104e00640 1 --2- 172.21.15.5:0/3325353376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa100103080 0x7fa100198720 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.901+0000 7fa104e00640 1 --2- 172.21.15.5:0/3325353376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa100101f30 0x7fa1001981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:44.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.901+0000 7fa104e00640 1 -- 172.21.15.5:0/3325353376 >> 172.21.15.5:0/3325353376 conn(0x7fa1000fd4c0 msgr2=0x7fa1001063b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:44.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.901+0000 7fa104e00640 1 -- 172.21.15.5:0/3325353376 shutdown_connections 2024-04-24T06:41:44.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:44.901+0000 7fa104e00640 1 -- 172.21.15.5:0/3325353376 wait complete. 2024-04-24T06:41:44.923 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-04-24T06:41:43.304495+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":633840,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":158,"num_read_kb":155,"num_write":321,"num_write_kb":3019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":259,"ondisk_log_size":259,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374488,"kb_used_data":12840,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567336,"statfs":{"total":1151940427776,"available":1151556952064,"internally_reserved":0,"allocated":13148160,"data_stored":7140272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":43472,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":22,"num_read_kb":36,"num_write":140,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.044327"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.470195+0000","last_change":"2024-04-24T06:41:35.296451+0000","last_active":"2024-04-24T06:41:36.470195+0000","last_peered":"2024-04-24T06:41:36.470195+0000","last_clean":"2024-04-24T06:41:36.470195+0000","last_became_active":"2024-04-24T06:41:35.295733+0000","last_became_peered":"2024-04-24T06:41:35.295733+0000","last_unstale":"2024-04-24T06:41:36.470195+0000","last_undegraded":"2024-04-24T06:41:36.470195+0000","last_fullsized":"2024-04-24T06:41:36.470195+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:04:15.072644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.470648+0000","last_change":"2024-04-24T06:41:35.293359+0000","last_active":"2024-04-24T06:41:36.470648+0000","last_peered":"2024-04-24T06:41:36.470648+0000","last_clean":"2024-04-24T06:41:36.470648+0000","last_became_active":"2024-04-24T06:41:35.287371+0000","last_became_peered":"2024-04-24T06:41:35.287371+0000","last_unstale":"2024-04-24T06:41:36.470648+0000","last_undegraded":"2024-04-24T06:41:36.470648+0000","last_fullsized":"2024-04-24T06:41:36.470648+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:03:39.376495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.470076+0000","last_change":"2024-04-24T06:41:35.272734+0000","last_active":"2024-04-24T06:41:36.470076+0000","last_peered":"2024-04-24T06:41:36.470076+0000","last_clean":"2024-04-24T06:41:36.470076+0000","last_became_active":"2024-04-24T06:41:35.271685+0000","last_became_peered":"2024-04-24T06:41:35.271685+0000","last_unstale":"2024-04-24T06:41:36.470076+0000","last_undegraded":"2024-04-24T06:41:36.470076+0000","last_fullsized":"2024-04-24T06:41:36.470076+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:08:15.057918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.470211+0000","last_change":"2024-04-24T06:41:35.288574+0000","last_active":"2024-04-24T06:41:36.470211+0000","last_peered":"2024-04-24T06:41:36.470211+0000","last_clean":"2024-04-24T06:41:36.470211+0000","last_became_active":"2024-04-24T06:41:35.288229+0000","last_became_peered":"2024-04-24T06:41:35.288229+0000","last_unstale":"2024-04-24T06:41:36.470211+0000","last_undegraded":"2024-04-24T06:41:36.470211+0000","last_fullsized":"2024-04-24T06:41:36.470211+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:58:18.028067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.446788+0000","last_change":"2024-04-24T06:41:35.310226+0000","last_active":"2024-04-24T06:41:36.446788+0000","last_peered":"2024-04-24T06:41:36.446788+0000","last_clean":"2024-04-24T06:41:36.446788+0000","last_became_active":"2024-04-24T06:41:35.309853+0000","last_became_peered":"2024-04-24T06:41:35.309853+0000","last_unstale":"2024-04-24T06:41:36.446788+0000","last_undegraded":"2024-04-24T06:41:36.446788+0000","last_fullsized":"2024-04-24T06:41:36.446788+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:34:57.767158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.470638+0000","last_change":"2024-04-24T06:41:35.294259+0000","last_active":"2024-04-24T06:41:36.470638+0000","last_peered":"2024-04-24T06:41:36.470638+0000","last_clean":"2024-04-24T06:41:36.470638+0000","last_became_active":"2024-04-24T06:41:35.293827+0000","last_became_peered":"2024-04-24T06:41:35.293827+0000","last_unstale":"2024-04-24T06:41:36.470638+0000","last_undegraded":"2024-04-24T06:41:36.470638+0000","last_fullsized":"2024-04-24T06:41:36.470638+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:09:23.302827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.470783+0000","last_change":"2024-04-24T06:41:35.295095+0000","last_active":"2024-04-24T06:41:36.470783+0000","last_peered":"2024-04-24T06:41:36.470783+0000","last_clean":"2024-04-24T06:41:36.470783+0000","last_became_active":"2024-04-24T06:41:35.294743+0000","last_became_peered":"2024-04-24T06:41:35.294743+0000","last_unstale":"2024-04-24T06:41:36.470783+0000","last_undegraded":"2024-04-24T06:41:36.470783+0000","last_fullsized":"2024-04-24T06:41:36.470783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:06:09.473123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.276149+0000","last_change":"2024-04-24T06:41:35.302644+0000","last_active":"2024-04-24T06:41:36.276149+0000","last_peered":"2024-04-24T06:41:36.276149+0000","last_clean":"2024-04-24T06:41:36.276149+0000","last_became_active":"2024-04-24T06:41:35.302272+0000","last_became_peered":"2024-04-24T06:41:35.302272+0000","last_unstale":"2024-04-24T06:41:36.276149+0000","last_undegraded":"2024-04-24T06:41:36.276149+0000","last_fullsized":"2024-04-24T06:41:36.276149+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:14:26.176607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.469543+0000","last_change":"2024-04-24T06:41:35.283275+0000","last_active":"2024-04-24T06:41:36.469543+0000","last_peered":"2024-04-24T06:41:36.469543+0000","last_clean":"2024-04-24T06:41:36.469543+0000","last_became_active":"2024-04-24T06:41:35.282586+0000","last_became_peered":"2024-04-24T06:41:35.282586+0000","last_unstale":"2024-04-24T06:41:36.469543+0000","last_undegraded":"2024-04-24T06:41:36.469543+0000","last_fullsized":"2024-04-24T06:41:36.469543+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:42:30.716149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.471497+0000","last_change":"2024-04-24T06:41:35.284986+0000","last_active":"2024-04-24T06:41:36.471497+0000","last_peered":"2024-04-24T06:41:36.471497+0000","last_clean":"2024-04-24T06:41:36.471497+0000","last_became_active":"2024-04-24T06:41:35.284628+0000","last_became_peered":"2024-04-24T06:41:35.284628+0000","last_unstale":"2024-04-24T06:41:36.471497+0000","last_undegraded":"2024-04-24T06:41:36.471497+0000","last_fullsized":"2024-04-24T06:41:36.471497+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:29:38.479593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.471727+0000","last_change":"2024-04-24T06:41:35.290353+0000","last_active":"2024-04-24T06:41:36.471727+0000","last_peered":"2024-04-24T06:41:36.471727+0000","last_clean":"2024-04-24T06:41:36.471727+0000","last_became_active":"2024-04-24T06:41:35.289987+0000","last_became_peered":"2024-04-24T06:41:35.289987+0000","last_unstale":"2024-04-24T06:41:36.471727+0000","last_undegraded":"2024-04-24T06:41:36.471727+0000","last_fullsized":"2024-04-24T06:41:36.471727+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:56:54.914426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.470085+0000","last_change":"2024-04-24T06:41:35.289462+0000","last_active":"2024-04-24T06:41:36.470085+0000","last_peered":"2024-04-24T06:41:36.470085+0000","last_clean":"2024-04-24T06:41:36.470085+0000","last_became_active":"2024-04-24T06:41:35.289123+0000","last_became_peered":"2024-04-24T06:41:35.289123+0000","last_unstale":"2024-04-24T06:41:36.470085+0000","last_undegraded":"2024-04-24T06:41:36.470085+0000","last_fullsized":"2024-04-24T06:41:36.470085+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:51:06.792508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.276261+0000","last_change":"2024-04-24T06:41:35.298320+0000","last_active":"2024-04-24T06:41:36.276261+0000","last_peered":"2024-04-24T06:41:36.276261+0000","last_clean":"2024-04-24T06:41:36.276261+0000","last_became_active":"2024-04-24T06:41:35.297839+0000","last_became_peered":"2024-04-24T06:41:35.297839+0000","last_unstale":"2024-04-24T06:41:36.276261+0000","last_undegraded":"2024-04-24T06:41:36.276261+0000","last_fullsized":"2024-04-24T06:41:36.276261+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:10:36.423514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.470082+0000","last_change":"2024-04-24T06:41:35.301924+0000","last_active":"2024-04-24T06:41:36.470082+0000","last_peered":"2024-04-24T06:41:36.470082+0000","last_clean":"2024-04-24T06:41:36.470082+0000","last_became_active":"2024-04-24T06:41:35.299114+0000","last_became_peered":"2024-04-24T06:41:35.299114+0000","last_unstale":"2024-04-24T06:41:36.470082+0000","last_undegraded":"2024-04-24T06:41:36.470082+0000","last_fullsized":"2024-04-24T06:41:36.470082+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:34:46.194266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.469752+0000","last_change":"2024-04-24T06:41:35.284885+0000","last_active":"2024-04-24T06:41:36.469752+0000","last_peered":"2024-04-24T06:41:36.469752+0000","last_clean":"2024-04-24T06:41:36.469752+0000","last_became_active":"2024-04-24T06:41:35.284244+0000","last_became_peered":"2024-04-24T06:41:35.284244+0000","last_unstale":"2024-04-24T06:41:36.469752+0000","last_undegraded":"2024-04-24T06:41:36.469752+0000","last_fullsized":"2024-04-24T06:41:36.469752+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:30:40.279905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.468945+0000","last_change":"2024-04-24T06:41:35.286363+0000","last_active":"2024-04-24T06:41:36.468945+0000","last_peered":"2024-04-24T06:41:36.468945+0000","last_clean":"2024-04-24T06:41:36.468945+0000","last_became_active":"2024-04-24T06:41:35.286024+0000","last_became_peered":"2024-04-24T06:41:35.286024+0000","last_unstale":"2024-04-24T06:41:36.468945+0000","last_undegraded":"2024-04-24T06:41:36.468945+0000","last_fullsized":"2024-04-24T06:41:36.468945+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:43:35.728990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.470296+0000","last_change":"2024-04-24T06:41:35.304005+0000","last_active":"2024-04-24T06:41:36.470296+0000","last_peered":"2024-04-24T06:41:36.470296+0000","last_clean":"2024-04-24T06:41:36.470296+0000","last_became_active":"2024-04-24T06:41:35.303659+0000","last_became_peered":"2024-04-24T06:41:35.303659+0000","last_unstale":"2024-04-24T06:41:36.470296+0000","last_undegraded":"2024-04-24T06:41:36.470296+0000","last_fullsized":"2024-04-24T06:41:36.470296+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:05:07.215927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.444526+0000","last_change":"2024-04-24T06:41:35.296218+0000","last_active":"2024-04-24T06:41:36.444526+0000","last_peered":"2024-04-24T06:41:36.444526+0000","last_clean":"2024-04-24T06:41:36.444526+0000","last_became_active":"2024-04-24T06:41:35.295486+0000","last_became_peered":"2024-04-24T06:41:35.295486+0000","last_unstale":"2024-04-24T06:41:36.444526+0000","last_undegraded":"2024-04-24T06:41:36.444526+0000","last_fullsized":"2024-04-24T06:41:36.444526+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:17:25.844899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.469995+0000","last_change":"2024-04-24T06:41:35.305064+0000","last_active":"2024-04-24T06:41:36.469995+0000","last_peered":"2024-04-24T06:41:36.469995+0000","last_clean":"2024-04-24T06:41:36.469995+0000","last_became_active":"2024-04-24T06:41:35.304695+0000","last_became_peered":"2024-04-24T06:41:35.304695+0000","last_unstale":"2024-04-24T06:41:36.469995+0000","last_undegraded":"2024-04-24T06:41:36.469995+0000","last_fullsized":"2024-04-24T06:41:36.469995+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:35:13.065601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:39.337530+0000","last_change":"2024-04-24T06:41:34.282140+0000","last_active":"2024-04-24T06:41:39.337530+0000","last_peered":"2024-04-24T06:41:39.337530+0000","last_clean":"2024-04-24T06:41:39.337530+0000","last_became_active":"2024-04-24T06:41:34.281608+0000","last_became_peered":"2024-04-24T06:41:34.281608+0000","last_unstale":"2024-04-24T06:41:39.337530+0000","last_undegraded":"2024-04-24T06:41:39.337530+0000","last_fullsized":"2024-04-24T06:41:39.337530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:32:06.727639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.278254+0000","last_change":"2024-04-24T06:41:35.308847+0000","last_active":"2024-04-24T06:41:36.278254+0000","last_peered":"2024-04-24T06:41:36.278254+0000","last_clean":"2024-04-24T06:41:36.278254+0000","last_became_active":"2024-04-24T06:41:35.308256+0000","last_became_peered":"2024-04-24T06:41:35.308256+0000","last_unstale":"2024-04-24T06:41:36.278254+0000","last_undegraded":"2024-04-24T06:41:36.278254+0000","last_fullsized":"2024-04-24T06:41:36.278254+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:21:49.338737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.278214+0000","last_change":"2024-04-24T06:41:34.285819+0000","last_active":"2024-04-24T06:41:36.278214+0000","last_peered":"2024-04-24T06:41:36.278214+0000","last_clean":"2024-04-24T06:41:36.278214+0000","last_became_active":"2024-04-24T06:41:34.282393+0000","last_became_peered":"2024-04-24T06:41:34.282393+0000","last_unstale":"2024-04-24T06:41:36.278214+0000","last_undegraded":"2024-04-24T06:41:36.278214+0000","last_fullsized":"2024-04-24T06:41:36.278214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:11:18.969178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.470688+0000","last_change":"2024-04-24T06:41:35.297437+0000","last_active":"2024-04-24T06:41:36.470688+0000","last_peered":"2024-04-24T06:41:36.470688+0000","last_clean":"2024-04-24T06:41:36.470688+0000","last_became_active":"2024-04-24T06:41:35.295195+0000","last_became_peered":"2024-04-24T06:41:35.295195+0000","last_unstale":"2024-04-24T06:41:36.470688+0000","last_undegraded":"2024-04-24T06:41:36.470688+0000","last_fullsized":"2024-04-24T06:41:36.470688+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:21:51.219219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:39.330880+0000","last_change":"2024-04-24T06:41:34.275768+0000","last_active":"2024-04-24T06:41:39.330880+0000","last_peered":"2024-04-24T06:41:39.330880+0000","last_clean":"2024-04-24T06:41:39.330880+0000","last_became_active":"2024-04-24T06:41:34.275426+0000","last_became_peered":"2024-04-24T06:41:34.275426+0000","last_unstale":"2024-04-24T06:41:39.330880+0000","last_undegraded":"2024-04-24T06:41:39.330880+0000","last_fullsized":"2024-04-24T06:41:39.330880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:01:08.735480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.469042+0000","last_change":"2024-04-24T06:41:35.305734+0000","last_active":"2024-04-24T06:41:36.469042+0000","last_peered":"2024-04-24T06:41:36.469042+0000","last_clean":"2024-04-24T06:41:36.469042+0000","last_became_active":"2024-04-24T06:41:35.305365+0000","last_became_peered":"2024-04-24T06:41:35.305365+0000","last_unstale":"2024-04-24T06:41:36.469042+0000","last_undegraded":"2024-04-24T06:41:36.469042+0000","last_fullsized":"2024-04-24T06:41:36.469042+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:38:52.740107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.447003+0000","last_change":"2024-04-24T06:41:34.278317+0000","last_active":"2024-04-24T06:41:36.447003+0000","last_peered":"2024-04-24T06:41:36.447003+0000","last_clean":"2024-04-24T06:41:36.447003+0000","last_became_active":"2024-04-24T06:41:34.277862+0000","last_became_peered":"2024-04-24T06:41:34.277862+0000","last_unstale":"2024-04-24T06:41:36.447003+0000","last_undegraded":"2024-04-24T06:41:36.447003+0000","last_fullsized":"2024-04-24T06:41:36.447003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:19:50.444136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.276716+0000","last_change":"2024-04-24T06:41:35.282648+0000","last_active":"2024-04-24T06:41:36.276716+0000","last_peered":"2024-04-24T06:41:36.276716+0000","last_clean":"2024-04-24T06:41:36.276716+0000","last_became_active":"2024-04-24T06:41:35.282335+0000","last_became_peered":"2024-04-24T06:41:35.282335+0000","last_unstale":"2024-04-24T06:41:36.276716+0000","last_undegraded":"2024-04-24T06:41:36.276716+0000","last_fullsized":"2024-04-24T06:41:36.276716+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:30:11.909067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:38.318497+0000","last_change":"2024-04-24T06:41:34.274405+0000","last_active":"2024-04-24T06:41:38.318497+0000","last_peered":"2024-04-24T06:41:38.318497+0000","last_clean":"2024-04-24T06:41:38.318497+0000","last_became_active":"2024-04-24T06:41:34.273820+0000","last_became_peered":"2024-04-24T06:41:34.273820+0000","last_unstale":"2024-04-24T06:41:38.318497+0000","last_undegraded":"2024-04-24T06:41:38.318497+0000","last_fullsized":"2024-04-24T06:41:38.318497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:11:26.119236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.446635+0000","last_change":"2024-04-24T06:41:35.294133+0000","last_active":"2024-04-24T06:41:36.446635+0000","last_peered":"2024-04-24T06:41:36.446635+0000","last_clean":"2024-04-24T06:41:36.446635+0000","last_became_active":"2024-04-24T06:41:35.293741+0000","last_became_peered":"2024-04-24T06:41:35.293741+0000","last_unstale":"2024-04-24T06:41:36.446635+0000","last_undegraded":"2024-04-24T06:41:36.446635+0000","last_fullsized":"2024-04-24T06:41:36.446635+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:09:12.495187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:39.334384+0000","last_change":"2024-04-24T06:41:34.276666+0000","last_active":"2024-04-24T06:41:39.334384+0000","last_peered":"2024-04-24T06:41:39.334384+0000","last_clean":"2024-04-24T06:41:39.334384+0000","last_became_active":"2024-04-24T06:41:34.276285+0000","last_became_peered":"2024-04-24T06:41:34.276285+0000","last_unstale":"2024-04-24T06:41:39.334384+0000","last_undegraded":"2024-04-24T06:41:39.334384+0000","last_fullsized":"2024-04-24T06:41:39.334384+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:02:31.230583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.276590+0000","last_change":"2024-04-24T06:41:35.296555+0000","last_active":"2024-04-24T06:41:36.276590+0000","last_peered":"2024-04-24T06:41:36.276590+0000","last_clean":"2024-04-24T06:41:36.276590+0000","last_became_active":"2024-04-24T06:41:35.296011+0000","last_became_peered":"2024-04-24T06:41:35.296011+0000","last_unstale":"2024-04-24T06:41:36.276590+0000","last_undegraded":"2024-04-24T06:41:36.276590+0000","last_fullsized":"2024-04-24T06:41:36.276590+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:34:03.081573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:39.335640+0000","last_change":"2024-04-24T06:41:34.284958+0000","last_active":"2024-04-24T06:41:39.335640+0000","last_peered":"2024-04-24T06:41:39.335640+0000","last_clean":"2024-04-24T06:41:39.335640+0000","last_became_active":"2024-04-24T06:41:34.284279+0000","last_became_peered":"2024-04-24T06:41:34.284279+0000","last_unstale":"2024-04-24T06:41:39.335640+0000","last_undegraded":"2024-04-24T06:41:39.335640+0000","last_fullsized":"2024-04-24T06:41:39.335640+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:58:19.316091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.275895+0000","last_change":"2024-04-24T06:41:35.307280+0000","last_active":"2024-04-24T06:41:36.275895+0000","last_peered":"2024-04-24T06:41:36.275895+0000","last_clean":"2024-04-24T06:41:36.275895+0000","last_became_active":"2024-04-24T06:41:35.306908+0000","last_became_peered":"2024-04-24T06:41:35.306908+0000","last_unstale":"2024-04-24T06:41:36.275895+0000","last_undegraded":"2024-04-24T06:41:36.275895+0000","last_fullsized":"2024-04-24T06:41:36.275895+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:54:31.306302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.276211+0000","last_change":"2024-04-24T06:41:34.272221+0000","last_active":"2024-04-24T06:41:36.276211+0000","last_peered":"2024-04-24T06:41:36.276211+0000","last_clean":"2024-04-24T06:41:36.276211+0000","last_became_active":"2024-04-24T06:41:34.271748+0000","last_became_peered":"2024-04-24T06:41:34.271748+0000","last_unstale":"2024-04-24T06:41:36.276211+0000","last_undegraded":"2024-04-24T06:41:36.276211+0000","last_fullsized":"2024-04-24T06:41:36.276211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:29:02.667285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.445003+0000","last_change":"2024-04-24T06:41:35.309608+0000","last_active":"2024-04-24T06:41:36.445003+0000","last_peered":"2024-04-24T06:41:36.445003+0000","last_clean":"2024-04-24T06:41:36.445003+0000","last_became_active":"2024-04-24T06:41:35.309227+0000","last_became_peered":"2024-04-24T06:41:35.309227+0000","last_unstale":"2024-04-24T06:41:36.445003+0000","last_undegraded":"2024-04-24T06:41:36.445003+0000","last_fullsized":"2024-04-24T06:41:36.445003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:51:23.151426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:41.343110+0000","last_change":"2024-04-24T06:41:34.281885+0000","last_active":"2024-04-24T06:41:41.343110+0000","last_peered":"2024-04-24T06:41:41.343110+0000","last_clean":"2024-04-24T06:41:41.343110+0000","last_became_active":"2024-04-24T06:41:34.281466+0000","last_became_peered":"2024-04-24T06:41:34.281466+0000","last_unstale":"2024-04-24T06:41:41.343110+0000","last_undegraded":"2024-04-24T06:41:41.343110+0000","last_fullsized":"2024-04-24T06:41:41.343110+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:20:08.500343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.469405+0000","last_change":"2024-04-24T06:41:35.292714+0000","last_active":"2024-04-24T06:41:36.469405+0000","last_peered":"2024-04-24T06:41:36.469405+0000","last_clean":"2024-04-24T06:41:36.469405+0000","last_became_active":"2024-04-24T06:41:35.291950+0000","last_became_peered":"2024-04-24T06:41:35.291950+0000","last_unstale":"2024-04-24T06:41:36.469405+0000","last_undegraded":"2024-04-24T06:41:36.469405+0000","last_fullsized":"2024-04-24T06:41:36.469405+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:42:18.049746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:38.316320+0000","last_change":"2024-04-24T06:41:34.275218+0000","last_active":"2024-04-24T06:41:38.316320+0000","last_peered":"2024-04-24T06:41:38.316320+0000","last_clean":"2024-04-24T06:41:38.316320+0000","last_became_active":"2024-04-24T06:41:34.274824+0000","last_became_peered":"2024-04-24T06:41:34.274824+0000","last_unstale":"2024-04-24T06:41:38.316320+0000","last_undegraded":"2024-04-24T06:41:38.316320+0000","last_fullsized":"2024-04-24T06:41:38.316320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:09:57.271304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.470888+0000","last_change":"2024-04-24T06:41:35.286311+0000","last_active":"2024-04-24T06:41:36.470888+0000","last_peered":"2024-04-24T06:41:36.470888+0000","last_clean":"2024-04-24T06:41:36.470888+0000","last_became_active":"2024-04-24T06:41:35.284643+0000","last_became_peered":"2024-04-24T06:41:35.284643+0000","last_unstale":"2024-04-24T06:41:36.470888+0000","last_undegraded":"2024-04-24T06:41:36.470888+0000","last_fullsized":"2024-04-24T06:41:36.470888+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:08:26.760816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.276775+0000","last_change":"2024-04-24T06:41:34.277372+0000","last_active":"2024-04-24T06:41:36.276775+0000","last_peered":"2024-04-24T06:41:36.276775+0000","last_clean":"2024-04-24T06:41:36.276775+0000","last_became_active":"2024-04-24T06:41:34.276997+0000","last_became_peered":"2024-04-24T06:41:34.276997+0000","last_unstale":"2024-04-24T06:41:36.276775+0000","last_undegraded":"2024-04-24T06:41:36.276775+0000","last_fullsized":"2024-04-24T06:41:36.276775+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:52:35.695109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.469671+0000","last_change":"2024-04-24T06:41:35.290233+0000","last_active":"2024-04-24T06:41:36.469671+0000","last_peered":"2024-04-24T06:41:36.469671+0000","last_clean":"2024-04-24T06:41:36.469671+0000","last_became_active":"2024-04-24T06:41:35.289885+0000","last_became_peered":"2024-04-24T06:41:35.289885+0000","last_unstale":"2024-04-24T06:41:36.469671+0000","last_undegraded":"2024-04-24T06:41:36.469671+0000","last_fullsized":"2024-04-24T06:41:36.469671+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:35:54.293308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'19","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:42.401418+0000","last_change":"2024-04-24T06:41:34.267693+0000","last_active":"2024-04-24T06:41:42.401418+0000","last_peered":"2024-04-24T06:41:42.401418+0000","last_clean":"2024-04-24T06:41:42.401418+0000","last_became_active":"2024-04-24T06:41:34.267298+0000","last_became_peered":"2024-04-24T06:41:34.267298+0000","last_unstale":"2024-04-24T06:41:42.401418+0000","last_undegraded":"2024-04-24T06:41:42.401418+0000","last_fullsized":"2024-04-24T06:41:42.401418+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:21:16.015695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39628,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":10,"num_read_kb":24,"num_write":20,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.446944+0000","last_change":"2024-04-24T06:41:35.286587+0000","last_active":"2024-04-24T06:41:36.446944+0000","last_peered":"2024-04-24T06:41:36.446944+0000","last_clean":"2024-04-24T06:41:36.446944+0000","last_became_active":"2024-04-24T06:41:35.286239+0000","last_became_peered":"2024-04-24T06:41:35.286239+0000","last_unstale":"2024-04-24T06:41:36.446944+0000","last_undegraded":"2024-04-24T06:41:36.446944+0000","last_fullsized":"2024-04-24T06:41:36.446944+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:18:56.013425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:39.333120+0000","last_change":"2024-04-24T06:41:34.278044+0000","last_active":"2024-04-24T06:41:39.333120+0000","last_peered":"2024-04-24T06:41:39.333120+0000","last_clean":"2024-04-24T06:41:39.333120+0000","last_became_active":"2024-04-24T06:41:34.276133+0000","last_became_peered":"2024-04-24T06:41:34.276133+0000","last_unstale":"2024-04-24T06:41:39.333120+0000","last_undegraded":"2024-04-24T06:41:39.333120+0000","last_fullsized":"2024-04-24T06:41:39.333120+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:54:32.394751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.276063+0000","last_change":"2024-04-24T06:41:35.306175+0000","last_active":"2024-04-24T06:41:36.276063+0000","last_peered":"2024-04-24T06:41:36.276063+0000","last_clean":"2024-04-24T06:41:36.276063+0000","last_became_active":"2024-04-24T06:41:35.305840+0000","last_became_peered":"2024-04-24T06:41:35.305840+0000","last_unstale":"2024-04-24T06:41:36.276063+0000","last_undegraded":"2024-04-24T06:41:36.276063+0000","last_fullsized":"2024-04-24T06:41:36.276063+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:42:46.238490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:38.322819+0000","last_change":"2024-04-24T06:41:34.277212+0000","last_active":"2024-04-24T06:41:38.322819+0000","last_peered":"2024-04-24T06:41:38.322819+0000","last_clean":"2024-04-24T06:41:38.322819+0000","last_became_active":"2024-04-24T06:41:34.276695+0000","last_became_peered":"2024-04-24T06:41:34.276695+0000","last_unstale":"2024-04-24T06:41:38.322819+0000","last_undegraded":"2024-04-24T06:41:38.322819+0000","last_fullsized":"2024-04-24T06:41:38.322819+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:00:47.169384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.471612+0000","last_change":"2024-04-24T06:41:35.305348+0000","last_active":"2024-04-24T06:41:36.471612+0000","last_peered":"2024-04-24T06:41:36.471612+0000","last_clean":"2024-04-24T06:41:36.471612+0000","last_became_active":"2024-04-24T06:41:35.304899+0000","last_became_peered":"2024-04-24T06:41:35.304899+0000","last_unstale":"2024-04-24T06:41:36.471612+0000","last_undegraded":"2024-04-24T06:41:36.471612+0000","last_fullsized":"2024-04-24T06:41:36.471612+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:49:35.737711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:39.335266+0000","last_change":"2024-04-24T06:41:34.272902+0000","last_active":"2024-04-24T06:41:39.335266+0000","last_peered":"2024-04-24T06:41:39.335266+0000","last_clean":"2024-04-24T06:41:39.335266+0000","last_became_active":"2024-04-24T06:41:34.271726+0000","last_became_peered":"2024-04-24T06:41:34.271726+0000","last_unstale":"2024-04-24T06:41:39.335266+0000","last_undegraded":"2024-04-24T06:41:39.335266+0000","last_fullsized":"2024-04-24T06:41:39.335266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:07:38.318110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.470594+0000","last_change":"2024-04-24T06:41:35.283764+0000","last_active":"2024-04-24T06:41:36.470594+0000","last_peered":"2024-04-24T06:41:36.470594+0000","last_clean":"2024-04-24T06:41:36.470594+0000","last_became_active":"2024-04-24T06:41:35.283400+0000","last_became_peered":"2024-04-24T06:41:35.283400+0000","last_unstale":"2024-04-24T06:41:36.470594+0000","last_undegraded":"2024-04-24T06:41:36.470594+0000","last_fullsized":"2024-04-24T06:41:36.470594+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:36:47.034068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:39.338140+0000","last_change":"2024-04-24T06:41:34.280020+0000","last_active":"2024-04-24T06:41:39.338140+0000","last_peered":"2024-04-24T06:41:39.338140+0000","last_clean":"2024-04-24T06:41:39.338140+0000","last_became_active":"2024-04-24T06:41:34.279659+0000","last_became_peered":"2024-04-24T06:41:34.279659+0000","last_unstale":"2024-04-24T06:41:39.338140+0000","last_undegraded":"2024-04-24T06:41:39.338140+0000","last_fullsized":"2024-04-24T06:41:39.338140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:13:29.529123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.276340+0000","last_change":"2024-04-24T06:41:35.284500+0000","last_active":"2024-04-24T06:41:36.276340+0000","last_peered":"2024-04-24T06:41:36.276340+0000","last_clean":"2024-04-24T06:41:36.276340+0000","last_became_active":"2024-04-24T06:41:35.283171+0000","last_became_peered":"2024-04-24T06:41:35.283171+0000","last_unstale":"2024-04-24T06:41:36.276340+0000","last_undegraded":"2024-04-24T06:41:36.276340+0000","last_fullsized":"2024-04-24T06:41:36.276340+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:33:19.527226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:40.330284+0000","last_change":"2024-04-24T06:41:34.279306+0000","last_active":"2024-04-24T06:41:40.330284+0000","last_peered":"2024-04-24T06:41:40.330284+0000","last_clean":"2024-04-24T06:41:40.330284+0000","last_became_active":"2024-04-24T06:41:34.278946+0000","last_became_peered":"2024-04-24T06:41:34.278946+0000","last_unstale":"2024-04-24T06:41:40.330284+0000","last_undegraded":"2024-04-24T06:41:40.330284+0000","last_fullsized":"2024-04-24T06:41:40.330284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:13:40.845871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.470113+0000","last_change":"2024-04-24T06:41:35.283716+0000","last_active":"2024-04-24T06:41:36.470113+0000","last_peered":"2024-04-24T06:41:36.470113+0000","last_clean":"2024-04-24T06:41:36.470113+0000","last_became_active":"2024-04-24T06:41:35.283330+0000","last_became_peered":"2024-04-24T06:41:35.283330+0000","last_unstale":"2024-04-24T06:41:36.470113+0000","last_undegraded":"2024-04-24T06:41:36.470113+0000","last_fullsized":"2024-04-24T06:41:36.470113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:18:13.403264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.444356+0000","last_change":"2024-04-24T06:41:34.288831+0000","last_active":"2024-04-24T06:41:36.444356+0000","last_peered":"2024-04-24T06:41:36.444356+0000","last_clean":"2024-04-24T06:41:36.444356+0000","last_became_active":"2024-04-24T06:41:34.288405+0000","last_became_peered":"2024-04-24T06:41:34.288405+0000","last_unstale":"2024-04-24T06:41:36.444356+0000","last_undegraded":"2024-04-24T06:41:36.444356+0000","last_fullsized":"2024-04-24T06:41:36.444356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:50:22.312561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:37.668529+0000","last_change":"2024-04-24T06:41:35.306607+0000","last_active":"2024-04-24T06:41:37.668529+0000","last_peered":"2024-04-24T06:41:37.668529+0000","last_clean":"2024-04-24T06:41:37.668529+0000","last_became_active":"2024-04-24T06:41:35.306217+0000","last_became_peered":"2024-04-24T06:41:35.306217+0000","last_unstale":"2024-04-24T06:41:37.668529+0000","last_undegraded":"2024-04-24T06:41:37.668529+0000","last_fullsized":"2024-04-24T06:41:37.668529+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:07:46.068876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:38.135100+0000","last_change":"2024-04-24T06:41:34.285319+0000","last_active":"2024-04-24T06:41:38.135100+0000","last_peered":"2024-04-24T06:41:38.135100+0000","last_clean":"2024-04-24T06:41:38.135100+0000","last_became_active":"2024-04-24T06:41:34.284912+0000","last_became_peered":"2024-04-24T06:41:34.284912+0000","last_unstale":"2024-04-24T06:41:38.135100+0000","last_undegraded":"2024-04-24T06:41:38.135100+0000","last_fullsized":"2024-04-24T06:41:38.135100+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:34:56.454587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.470931+0000","last_change":"2024-04-24T06:41:35.296294+0000","last_active":"2024-04-24T06:41:36.470931+0000","last_peered":"2024-04-24T06:41:36.470931+0000","last_clean":"2024-04-24T06:41:36.470931+0000","last_became_active":"2024-04-24T06:41:35.295792+0000","last_became_peered":"2024-04-24T06:41:35.295792+0000","last_unstale":"2024-04-24T06:41:36.470931+0000","last_undegraded":"2024-04-24T06:41:36.470931+0000","last_fullsized":"2024-04-24T06:41:36.470931+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:31:35.573858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.276425+0000","last_change":"2024-04-24T06:41:35.282558+0000","last_active":"2024-04-24T06:41:36.276425+0000","last_peered":"2024-04-24T06:41:36.276425+0000","last_clean":"2024-04-24T06:41:36.276425+0000","last_became_active":"2024-04-24T06:41:35.282163+0000","last_became_peered":"2024-04-24T06:41:35.282163+0000","last_unstale":"2024-04-24T06:41:36.276425+0000","last_undegraded":"2024-04-24T06:41:36.276425+0000","last_fullsized":"2024-04-24T06:41:36.276425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:12:23.569849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:39.338130+0000","last_change":"2024-04-24T06:41:34.284294+0000","last_active":"2024-04-24T06:41:39.338130+0000","last_peered":"2024-04-24T06:41:39.338130+0000","last_clean":"2024-04-24T06:41:39.338130+0000","last_became_active":"2024-04-24T06:41:34.283933+0000","last_became_peered":"2024-04-24T06:41:34.283933+0000","last_unstale":"2024-04-24T06:41:39.338130+0000","last_undegraded":"2024-04-24T06:41:39.338130+0000","last_fullsized":"2024-04-24T06:41:39.338130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:35:26.042614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.277119+0000","last_change":"2024-04-24T06:41:35.302361+0000","last_active":"2024-04-24T06:41:36.277119+0000","last_peered":"2024-04-24T06:41:36.277119+0000","last_clean":"2024-04-24T06:41:36.277119+0000","last_became_active":"2024-04-24T06:41:35.301652+0000","last_became_peered":"2024-04-24T06:41:35.301652+0000","last_unstale":"2024-04-24T06:41:36.277119+0000","last_undegraded":"2024-04-24T06:41:36.277119+0000","last_fullsized":"2024-04-24T06:41:36.277119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:35:58.010853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:38.321768+0000","last_change":"2024-04-24T06:41:34.282156+0000","last_active":"2024-04-24T06:41:38.321768+0000","last_peered":"2024-04-24T06:41:38.321768+0000","last_clean":"2024-04-24T06:41:38.321768+0000","last_became_active":"2024-04-24T06:41:34.281771+0000","last_became_peered":"2024-04-24T06:41:34.281771+0000","last_unstale":"2024-04-24T06:41:38.321768+0000","last_undegraded":"2024-04-24T06:41:38.321768+0000","last_fullsized":"2024-04-24T06:41:38.321768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:41:49.584620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":350,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.470757+0000","last_change":"2024-04-24T06:40:16.681695+0000","last_active":"2024-04-24T06:41:36.470757+0000","last_peered":"2024-04-24T06:41:36.470757+0000","last_clean":"2024-04-24T06:41:36.470757+0000","last_became_active":"2024-04-24T06:40:16.681208+0000","last_became_peered":"2024-04-24T06:40:16.681208+0000","last_unstale":"2024-04-24T06:41:36.470757+0000","last_undegraded":"2024-04-24T06:41:36.470757+0000","last_fullsized":"2024-04-24T06:41:36.470757+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:36:27.950574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:36:27.950574+0000","last_clean_scrub_stamp":"2024-04-24T06:36:27.950574+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:39:01.985893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.274890+0000","last_change":"2024-04-24T06:41:35.303109+0000","last_active":"2024-04-24T06:41:36.274890+0000","last_peered":"2024-04-24T06:41:36.274890+0000","last_clean":"2024-04-24T06:41:36.274890+0000","last_became_active":"2024-04-24T06:41:35.302755+0000","last_became_peered":"2024-04-24T06:41:35.302755+0000","last_unstale":"2024-04-24T06:41:36.274890+0000","last_undegraded":"2024-04-24T06:41:36.274890+0000","last_fullsized":"2024-04-24T06:41:36.274890+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:22:56.075740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:39.331760+0000","last_change":"2024-04-24T06:41:34.273872+0000","last_active":"2024-04-24T06:41:39.331760+0000","last_peered":"2024-04-24T06:41:39.331760+0000","last_clean":"2024-04-24T06:41:39.331760+0000","last_became_active":"2024-04-24T06:41:34.273473+0000","last_became_peered":"2024-04-24T06:41:34.273473+0000","last_unstale":"2024-04-24T06:41:39.331760+0000","last_undegraded":"2024-04-24T06:41:39.331760+0000","last_fullsized":"2024-04-24T06:41:39.331760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:47:11.914985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:38.320281+0000","last_change":"2024-04-24T06:41:34.289293+0000","last_active":"2024-04-24T06:41:38.320281+0000","last_peered":"2024-04-24T06:41:38.320281+0000","last_clean":"2024-04-24T06:41:38.320281+0000","last_became_active":"2024-04-24T06:41:34.288939+0000","last_became_peered":"2024-04-24T06:41:34.288939+0000","last_unstale":"2024-04-24T06:41:38.320281+0000","last_undegraded":"2024-04-24T06:41:38.320281+0000","last_fullsized":"2024-04-24T06:41:38.320281+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:03:01.026792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.277972+0000","last_change":"2024-04-24T06:41:35.302153+0000","last_active":"2024-04-24T06:41:36.277972+0000","last_peered":"2024-04-24T06:41:36.277972+0000","last_clean":"2024-04-24T06:41:36.277972+0000","last_became_active":"2024-04-24T06:41:35.301666+0000","last_became_peered":"2024-04-24T06:41:35.301666+0000","last_unstale":"2024-04-24T06:41:36.277972+0000","last_undegraded":"2024-04-24T06:41:36.277972+0000","last_fullsized":"2024-04-24T06:41:36.277972+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:45:09.642482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:39.338104+0000","last_change":"2024-04-24T06:41:34.291570+0000","last_active":"2024-04-24T06:41:39.338104+0000","last_peered":"2024-04-24T06:41:39.338104+0000","last_clean":"2024-04-24T06:41:39.338104+0000","last_became_active":"2024-04-24T06:41:34.289700+0000","last_became_peered":"2024-04-24T06:41:34.289700+0000","last_unstale":"2024-04-24T06:41:39.338104+0000","last_undegraded":"2024-04-24T06:41:39.338104+0000","last_fullsized":"2024-04-24T06:41:39.338104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:57:20.537299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.278921+0000","last_change":"2024-04-24T06:41:35.304703+0000","last_active":"2024-04-24T06:41:36.278921+0000","last_peered":"2024-04-24T06:41:36.278921+0000","last_clean":"2024-04-24T06:41:36.278921+0000","last_became_active":"2024-04-24T06:41:35.304319+0000","last_became_peered":"2024-04-24T06:41:35.304319+0000","last_unstale":"2024-04-24T06:41:36.278921+0000","last_undegraded":"2024-04-24T06:41:36.278921+0000","last_fullsized":"2024-04-24T06:41:36.278921+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:00:56.069509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:38.318761+0000","last_change":"2024-04-24T06:41:34.276866+0000","last_active":"2024-04-24T06:41:38.318761+0000","last_peered":"2024-04-24T06:41:38.318761+0000","last_clean":"2024-04-24T06:41:38.318761+0000","last_became_active":"2024-04-24T06:41:34.276478+0000","last_became_peered":"2024-04-24T06:41:34.276478+0000","last_unstale":"2024-04-24T06:41:38.318761+0000","last_undegraded":"2024-04-24T06:41:38.318761+0000","last_fullsized":"2024-04-24T06:41:38.318761+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:10:35.047960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.470382+0000","last_change":"2024-04-24T06:41:35.300951+0000","last_active":"2024-04-24T06:41:36.470382+0000","last_peered":"2024-04-24T06:41:36.470382+0000","last_clean":"2024-04-24T06:41:36.470382+0000","last_became_active":"2024-04-24T06:41:35.296176+0000","last_became_peered":"2024-04-24T06:41:35.296176+0000","last_unstale":"2024-04-24T06:41:36.470382+0000","last_undegraded":"2024-04-24T06:41:36.470382+0000","last_fullsized":"2024-04-24T06:41:36.470382+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:53:58.497437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:38.320407+0000","last_change":"2024-04-24T06:41:34.276154+0000","last_active":"2024-04-24T06:41:38.320407+0000","last_peered":"2024-04-24T06:41:38.320407+0000","last_clean":"2024-04-24T06:41:38.320407+0000","last_became_active":"2024-04-24T06:41:34.275734+0000","last_became_peered":"2024-04-24T06:41:34.275734+0000","last_unstale":"2024-04-24T06:41:38.320407+0000","last_undegraded":"2024-04-24T06:41:38.320407+0000","last_fullsized":"2024-04-24T06:41:38.320407+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:28:54.618061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.446629+0000","last_change":"2024-04-24T06:41:35.285837+0000","last_active":"2024-04-24T06:41:36.446629+0000","last_peered":"2024-04-24T06:41:36.446629+0000","last_clean":"2024-04-24T06:41:36.446629+0000","last_became_active":"2024-04-24T06:41:35.285505+0000","last_became_peered":"2024-04-24T06:41:35.285505+0000","last_unstale":"2024-04-24T06:41:36.446629+0000","last_undegraded":"2024-04-24T06:41:36.446629+0000","last_fullsized":"2024-04-24T06:41:36.446629+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:36:53.460780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:38.323741+0000","last_change":"2024-04-24T06:41:34.290039+0000","last_active":"2024-04-24T06:41:38.323741+0000","last_peered":"2024-04-24T06:41:38.323741+0000","last_clean":"2024-04-24T06:41:38.323741+0000","last_became_active":"2024-04-24T06:41:34.289228+0000","last_became_peered":"2024-04-24T06:41:34.289228+0000","last_unstale":"2024-04-24T06:41:38.323741+0000","last_undegraded":"2024-04-24T06:41:38.323741+0000","last_fullsized":"2024-04-24T06:41:38.323741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:07:20.342359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.469847+0000","last_change":"2024-04-24T06:41:35.296214+0000","last_active":"2024-04-24T06:41:36.469847+0000","last_peered":"2024-04-24T06:41:36.469847+0000","last_clean":"2024-04-24T06:41:36.469847+0000","last_became_active":"2024-04-24T06:41:35.295888+0000","last_became_peered":"2024-04-24T06:41:35.295888+0000","last_unstale":"2024-04-24T06:41:36.469847+0000","last_undegraded":"2024-04-24T06:41:36.469847+0000","last_fullsized":"2024-04-24T06:41:36.469847+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:55:19.520479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:37.379729+0000","last_change":"2024-04-24T06:41:34.275885+0000","last_active":"2024-04-24T06:41:37.379729+0000","last_peered":"2024-04-24T06:41:37.379729+0000","last_clean":"2024-04-24T06:41:37.379729+0000","last_became_active":"2024-04-24T06:41:34.275503+0000","last_became_peered":"2024-04-24T06:41:34.275503+0000","last_unstale":"2024-04-24T06:41:37.379729+0000","last_undegraded":"2024-04-24T06:41:37.379729+0000","last_fullsized":"2024-04-24T06:41:37.379729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:50:58.216699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.277124+0000","last_change":"2024-04-24T06:41:35.305330+0000","last_active":"2024-04-24T06:41:36.277124+0000","last_peered":"2024-04-24T06:41:36.277124+0000","last_clean":"2024-04-24T06:41:36.277124+0000","last_became_active":"2024-04-24T06:41:35.301262+0000","last_became_peered":"2024-04-24T06:41:35.301262+0000","last_unstale":"2024-04-24T06:41:36.277124+0000","last_undegraded":"2024-04-24T06:41:36.277124+0000","last_fullsized":"2024-04-24T06:41:36.277124+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:58:41.029511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:38.317955+0000","last_change":"2024-04-24T06:41:34.276926+0000","last_active":"2024-04-24T06:41:38.317955+0000","last_peered":"2024-04-24T06:41:38.317955+0000","last_clean":"2024-04-24T06:41:38.317955+0000","last_became_active":"2024-04-24T06:41:34.276539+0000","last_became_peered":"2024-04-24T06:41:34.276539+0000","last_unstale":"2024-04-24T06:41:38.317955+0000","last_undegraded":"2024-04-24T06:41:38.317955+0000","last_fullsized":"2024-04-24T06:41:38.317955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:28:56.652584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.471444+0000","last_change":"2024-04-24T06:41:35.301052+0000","last_active":"2024-04-24T06:41:36.471444+0000","last_peered":"2024-04-24T06:41:36.471444+0000","last_clean":"2024-04-24T06:41:36.471444+0000","last_became_active":"2024-04-24T06:41:35.300650+0000","last_became_peered":"2024-04-24T06:41:35.300650+0000","last_unstale":"2024-04-24T06:41:36.471444+0000","last_undegraded":"2024-04-24T06:41:36.471444+0000","last_fullsized":"2024-04-24T06:41:36.471444+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:06:54.290353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:39.333903+0000","last_change":"2024-04-24T06:41:34.273773+0000","last_active":"2024-04-24T06:41:39.333903+0000","last_peered":"2024-04-24T06:41:39.333903+0000","last_clean":"2024-04-24T06:41:39.333903+0000","last_became_active":"2024-04-24T06:41:34.273324+0000","last_became_peered":"2024-04-24T06:41:34.273324+0000","last_unstale":"2024-04-24T06:41:39.333903+0000","last_undegraded":"2024-04-24T06:41:39.333903+0000","last_fullsized":"2024-04-24T06:41:39.333903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:30:30.760740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.471784+0000","last_change":"2024-04-24T06:41:35.289565+0000","last_active":"2024-04-24T06:41:36.471784+0000","last_peered":"2024-04-24T06:41:36.471784+0000","last_clean":"2024-04-24T06:41:36.471784+0000","last_became_active":"2024-04-24T06:41:35.289185+0000","last_became_peered":"2024-04-24T06:41:35.289185+0000","last_unstale":"2024-04-24T06:41:36.471784+0000","last_undegraded":"2024-04-24T06:41:36.471784+0000","last_fullsized":"2024-04-24T06:41:36.471784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:26:28.345162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.276540+0000","last_change":"2024-04-24T06:41:34.285804+0000","last_active":"2024-04-24T06:41:36.276540+0000","last_peered":"2024-04-24T06:41:36.276540+0000","last_clean":"2024-04-24T06:41:36.276540+0000","last_became_active":"2024-04-24T06:41:34.285501+0000","last_became_peered":"2024-04-24T06:41:34.285501+0000","last_unstale":"2024-04-24T06:41:36.276540+0000","last_undegraded":"2024-04-24T06:41:36.276540+0000","last_fullsized":"2024-04-24T06:41:36.276540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:25:15.804221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:38.135125+0000","last_change":"2024-04-24T06:41:35.300856+0000","last_active":"2024-04-24T06:41:38.135125+0000","last_peered":"2024-04-24T06:41:38.135125+0000","last_clean":"2024-04-24T06:41:38.135125+0000","last_became_active":"2024-04-24T06:41:35.300344+0000","last_became_peered":"2024-04-24T06:41:35.300344+0000","last_unstale":"2024-04-24T06:41:38.135125+0000","last_undegraded":"2024-04-24T06:41:38.135125+0000","last_fullsized":"2024-04-24T06:41:38.135125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:10:01.866954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.278567+0000","last_change":"2024-04-24T06:41:34.284112+0000","last_active":"2024-04-24T06:41:36.278567+0000","last_peered":"2024-04-24T06:41:36.278567+0000","last_clean":"2024-04-24T06:41:36.278567+0000","last_became_active":"2024-04-24T06:41:34.279317+0000","last_became_peered":"2024-04-24T06:41:34.279317+0000","last_unstale":"2024-04-24T06:41:36.278567+0000","last_undegraded":"2024-04-24T06:41:36.278567+0000","last_fullsized":"2024-04-24T06:41:36.278567+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:49:45.673353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.471288+0000","last_change":"2024-04-24T06:41:35.288360+0000","last_active":"2024-04-24T06:41:36.471288+0000","last_peered":"2024-04-24T06:41:36.471288+0000","last_clean":"2024-04-24T06:41:36.471288+0000","last_became_active":"2024-04-24T06:41:35.288019+0000","last_became_peered":"2024-04-24T06:41:35.288019+0000","last_unstale":"2024-04-24T06:41:36.471288+0000","last_undegraded":"2024-04-24T06:41:36.471288+0000","last_fullsized":"2024-04-24T06:41:36.471288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:33:49.927591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.469862+0000","last_change":"2024-04-24T06:41:35.286137+0000","last_active":"2024-04-24T06:41:36.469862+0000","last_peered":"2024-04-24T06:41:36.469862+0000","last_clean":"2024-04-24T06:41:36.469862+0000","last_became_active":"2024-04-24T06:41:35.285773+0000","last_became_peered":"2024-04-24T06:41:35.285773+0000","last_unstale":"2024-04-24T06:41:36.469862+0000","last_undegraded":"2024-04-24T06:41:36.469862+0000","last_fullsized":"2024-04-24T06:41:36.469862+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:29:32.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.469496+0000","last_change":"2024-04-24T06:41:35.283977+0000","last_active":"2024-04-24T06:41:36.469496+0000","last_peered":"2024-04-24T06:41:36.469496+0000","last_clean":"2024-04-24T06:41:36.469496+0000","last_became_active":"2024-04-24T06:41:35.282312+0000","last_became_peered":"2024-04-24T06:41:35.282312+0000","last_unstale":"2024-04-24T06:41:36.469496+0000","last_undegraded":"2024-04-24T06:41:36.469496+0000","last_fullsized":"2024-04-24T06:41:36.469496+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:41:36.463803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.469605+0000","last_change":"2024-04-24T06:41:35.300372+0000","last_active":"2024-04-24T06:41:36.469605+0000","last_peered":"2024-04-24T06:41:36.469605+0000","last_clean":"2024-04-24T06:41:36.469605+0000","last_became_active":"2024-04-24T06:41:35.300034+0000","last_became_peered":"2024-04-24T06:41:35.300034+0000","last_unstale":"2024-04-24T06:41:36.469605+0000","last_undegraded":"2024-04-24T06:41:36.469605+0000","last_fullsized":"2024-04-24T06:41:36.469605+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:56:23.732798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.470495+0000","last_change":"2024-04-24T06:41:35.301619+0000","last_active":"2024-04-24T06:41:36.470495+0000","last_peered":"2024-04-24T06:41:36.470495+0000","last_clean":"2024-04-24T06:41:36.470495+0000","last_became_active":"2024-04-24T06:41:35.298897+0000","last_became_peered":"2024-04-24T06:41:35.298897+0000","last_unstale":"2024-04-24T06:41:36.470495+0000","last_undegraded":"2024-04-24T06:41:36.470495+0000","last_fullsized":"2024-04-24T06:41:36.470495+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:26:11.583079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.278245+0000","last_change":"2024-04-24T06:41:35.304905+0000","last_active":"2024-04-24T06:41:36.278245+0000","last_peered":"2024-04-24T06:41:36.278245+0000","last_clean":"2024-04-24T06:41:36.278245+0000","last_became_active":"2024-04-24T06:41:35.304407+0000","last_became_peered":"2024-04-24T06:41:35.304407+0000","last_unstale":"2024-04-24T06:41:36.278245+0000","last_undegraded":"2024-04-24T06:41:36.278245+0000","last_fullsized":"2024-04-24T06:41:36.278245+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:38:07.417898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.469810+0000","last_change":"2024-04-24T06:41:35.288646+0000","last_active":"2024-04-24T06:41:36.469810+0000","last_peered":"2024-04-24T06:41:36.469810+0000","last_clean":"2024-04-24T06:41:36.469810+0000","last_became_active":"2024-04-24T06:41:35.288266+0000","last_became_peered":"2024-04-24T06:41:35.288266+0000","last_unstale":"2024-04-24T06:41:36.469810+0000","last_undegraded":"2024-04-24T06:41:36.469810+0000","last_fullsized":"2024-04-24T06:41:36.469810+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:59:16.526712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.470133+0000","last_change":"2024-04-24T06:41:35.285001+0000","last_active":"2024-04-24T06:41:36.470133+0000","last_peered":"2024-04-24T06:41:36.470133+0000","last_clean":"2024-04-24T06:41:36.470133+0000","last_became_active":"2024-04-24T06:41:35.284671+0000","last_became_peered":"2024-04-24T06:41:35.284671+0000","last_unstale":"2024-04-24T06:41:36.470133+0000","last_undegraded":"2024-04-24T06:41:36.470133+0000","last_fullsized":"2024-04-24T06:41:36.470133+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:37:43.534018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.278660+0000","last_change":"2024-04-24T06:41:35.299124+0000","last_active":"2024-04-24T06:41:36.278660+0000","last_peered":"2024-04-24T06:41:36.278660+0000","last_clean":"2024-04-24T06:41:36.278660+0000","last_became_active":"2024-04-24T06:41:35.298246+0000","last_became_peered":"2024-04-24T06:41:35.298246+0000","last_unstale":"2024-04-24T06:41:36.278660+0000","last_undegraded":"2024-04-24T06:41:36.278660+0000","last_fullsized":"2024-04-24T06:41:36.278660+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:07:22.649874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.276423+0000","last_change":"2024-04-24T06:41:35.296588+0000","last_active":"2024-04-24T06:41:36.276423+0000","last_peered":"2024-04-24T06:41:36.276423+0000","last_clean":"2024-04-24T06:41:36.276423+0000","last_became_active":"2024-04-24T06:41:35.296026+0000","last_became_peered":"2024-04-24T06:41:35.296026+0000","last_unstale":"2024-04-24T06:41:36.276423+0000","last_undegraded":"2024-04-24T06:41:36.276423+0000","last_fullsized":"2024-04-24T06:41:36.276423+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:17:50.627727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.277958+0000","last_change":"2024-04-24T06:41:35.301619+0000","last_active":"2024-04-24T06:41:36.277958+0000","last_peered":"2024-04-24T06:41:36.277958+0000","last_clean":"2024-04-24T06:41:36.277958+0000","last_became_active":"2024-04-24T06:41:35.297987+0000","last_became_peered":"2024-04-24T06:41:35.297987+0000","last_unstale":"2024-04-24T06:41:36.277958+0000","last_undegraded":"2024-04-24T06:41:36.277958+0000","last_fullsized":"2024-04-24T06:41:36.277958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:06:01.235593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.469686+0000","last_change":"2024-04-24T06:41:35.304347+0000","last_active":"2024-04-24T06:41:36.469686+0000","last_peered":"2024-04-24T06:41:36.469686+0000","last_clean":"2024-04-24T06:41:36.469686+0000","last_became_active":"2024-04-24T06:41:35.303971+0000","last_became_peered":"2024-04-24T06:41:35.303971+0000","last_unstale":"2024-04-24T06:41:36.469686+0000","last_undegraded":"2024-04-24T06:41:36.469686+0000","last_fullsized":"2024-04-24T06:41:36.469686+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:31:54.813243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.276274+0000","last_change":"2024-04-24T06:41:35.308314+0000","last_active":"2024-04-24T06:41:36.276274+0000","last_peered":"2024-04-24T06:41:36.276274+0000","last_clean":"2024-04-24T06:41:36.276274+0000","last_became_active":"2024-04-24T06:41:35.306581+0000","last_became_peered":"2024-04-24T06:41:35.306581+0000","last_unstale":"2024-04-24T06:41:36.276274+0000","last_undegraded":"2024-04-24T06:41:36.276274+0000","last_fullsized":"2024-04-24T06:41:36.276274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:34:43.095087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T06:41:36.469706+0000","last_change":"2024-04-24T06:41:35.289166+0000","last_active":"2024-04-24T06:41:36.469706+0000","last_peered":"2024-04-24T06:41:36.469706+0000","last_clean":"2024-04-24T06:41:36.469706+0000","last_became_active":"2024-04-24T06:41:35.287672+0000","last_became_peered":"2024-04-24T06:41:35.287672+0000","last_unstale":"2024-04-24T06:41:36.469706+0000","last_undegraded":"2024-04-24T06:41:36.469706+0000","last_fullsized":"2024-04-24T06:41:36.469706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:31:45.403387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":43238,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":22,"num_read_kb":36,"num_write":138,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":72091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350990,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":443597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612627,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1585152,"data_stored":1055973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776152,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":946176,"data_stored":443833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037791,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":443753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299428,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28520,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716632,"statfs":{"total":95995035648,"available":95965831168,"internally_reserved":0,"allocated":1540096,"data_stored":1035292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462954,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":443105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":937984,"data_stored":444627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":929792,"data_stored":443105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411395,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":978944,"data_stored":465260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182457,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":443209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920821,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":443659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724592,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32692,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712460,"statfs":{"total":95995035648,"available":95961559040,"internally_reserved":0,"allocated":1548288,"data_stored":1034859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":22245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":41206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T06:41:44.929 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-24T06:41:44.929 INFO:tasks.cephadm:Setting up client nodes... 2024-04-24T06:41:44.929 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-24T06:41:45.449 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:41:45.610 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:45 smithi005 ceph-mon[27798]: from='client.14607 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T06:41:45.610 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:45 smithi005 ceph-mon[27798]: from='client.14613 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T06:41:45.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:45 smithi137 ceph-mon[33017]: from='client.14607 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T06:41:45.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:45 smithi137 ceph-mon[33017]: from='client.14613 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T06:41:45.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:45 smithi037 ceph-mon[32864]: from='client.14607 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T06:41:45.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:45 smithi037 ceph-mon[32864]: from='client.14613 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T06:41:46.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:46 smithi005 ceph-mon[27798]: pgmap v318: 97 pgs: 97 active+clean; 619 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8.1 KiB/s wr, 18 op/s 2024-04-24T06:41:46.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:46 smithi137 ceph-mon[33017]: pgmap v318: 97 pgs: 97 active+clean; 619 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8.1 KiB/s wr, 18 op/s 2024-04-24T06:41:46.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:46 smithi037 ceph-mon[32864]: pgmap v318: 97 pgs: 97 active+clean; 619 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8.1 KiB/s wr, 18 op/s 2024-04-24T06:41:47.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.574+0000 7fd4da200700 1 -- 172.21.15.5:0/1026584667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4d4105db0 msgr2=0x7fd4d4106210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:47.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.574+0000 7fd4da200700 1 --2- 172.21.15.5:0/1026584667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4d4105db0 0x7fd4d4106210 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc009990 tx=0x7fd4bc009ca0 comp rx=0 tx=0).stop 2024-04-24T06:41:47.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.574+0000 7fd4da200700 1 -- 172.21.15.5:0/1026584667 shutdown_connections 2024-04-24T06:41:47.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.574+0000 7fd4da200700 1 --2- 172.21.15.5:0/1026584667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4d4106750 0x7fd4d410d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:47.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.574+0000 7fd4da200700 1 --2- 172.21.15.5:0/1026584667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4d4104bb0 0x7fd4d4104fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:47.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.574+0000 7fd4da200700 1 --2- 172.21.15.5:0/1026584667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4d4105db0 0x7fd4d4106210 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:47.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.574+0000 7fd4da200700 1 -- 172.21.15.5:0/1026584667 >> 172.21.15.5:0/1026584667 conn(0x7fd4d4100130 msgr2=0x7fd4d4102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:47.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.574+0000 7fd4da200700 1 -- 172.21.15.5:0/1026584667 shutdown_connections 2024-04-24T06:41:47.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.574+0000 7fd4da200700 1 -- 172.21.15.5:0/1026584667 wait complete. 2024-04-24T06:41:47.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.575+0000 7fd4da200700 1 Processor -- start 2024-04-24T06:41:47.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.575+0000 7fd4da200700 1 -- start start 2024-04-24T06:41:47.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.576+0000 7fd4da200700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4d4104bb0 0x7fd4d419ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:47.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.576+0000 7fd4da200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4d4105db0 0x7fd4d419b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:47.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.576+0000 7fd4da200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4d4106750 0x7fd4d41a2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:47.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.576+0000 7fd4da200700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d41107c0 con 0x7fd4d4104bb0 2024-04-24T06:41:47.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.576+0000 7fd4da200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d4110640 con 0x7fd4d4106750 2024-04-24T06:41:47.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.576+0000 7fd4da200700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d4110940 con 0x7fd4d4105db0 2024-04-24T06:41:47.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.576+0000 7fd4d8e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4d4104bb0 0x7fd4d419ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:47.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.576+0000 7fd4cbe00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4d4105db0 0x7fd4d419b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:47.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.576+0000 7fd4cbe00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4d4105db0 0x7fd4d419b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:48674/0 (socket says 172.21.15.5:48674) 2024-04-24T06:41:47.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.576+0000 7fd4d8e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4d4104bb0 0x7fd4d419ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43970/0 (socket says 172.21.15.5:43970) 2024-04-24T06:41:47.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.576+0000 7fd4cbe00700 1 -- 172.21.15.5:0/737543583 learned_addr learned my addr 172.21.15.5:0/737543583 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:41:47.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.576+0000 7fd4d9800700 1 --2- 172.21.15.5:0/737543583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4d4106750 0x7fd4d41a2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:47.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.576+0000 7fd4cbe00700 1 -- 172.21.15.5:0/737543583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4d4106750 msgr2=0x7fd4d41a2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:47.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.576+0000 7fd4cbe00700 1 --2- 172.21.15.5:0/737543583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4d4106750 0x7fd4d41a2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:47.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.576+0000 7fd4cbe00700 1 -- 172.21.15.5:0/737543583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4d4104bb0 msgr2=0x7fd4d419ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:47.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.576+0000 7fd4cbe00700 1 --2- 172.21.15.5:0/737543583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4d4104bb0 0x7fd4d419ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:47.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.576+0000 7fd4cbe00700 1 -- 172.21.15.5:0/737543583 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d41a2960 con 0x7fd4d4105db0 2024-04-24T06:41:47.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.577+0000 7fd4cbe00700 1 --2- 172.21.15.5:0/737543583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4d4105db0 0x7fd4d419b250 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc006010 tx=0x7fd4bc00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:47.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.577+0000 7fd4d2000700 1 -- 172.21.15.5:0/737543583 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc027070 con 0x7fd4d4105db0 2024-04-24T06:41:47.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.577+0000 7fd4d2000700 1 -- 172.21.15.5:0/737543583 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4bc022950 con 0x7fd4d4105db0 2024-04-24T06:41:47.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.577+0000 7fd4d2000700 1 -- 172.21.15.5:0/737543583 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc017650 con 0x7fd4d4105db0 2024-04-24T06:41:47.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.577+0000 7fd4da200700 1 -- 172.21.15.5:0/737543583 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4d41a2bf0 con 0x7fd4d4105db0 2024-04-24T06:41:47.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.577+0000 7fd4da200700 1 -- 172.21.15.5:0/737543583 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4d41a30a0 con 0x7fd4d4105db0 2024-04-24T06:41:47.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.579+0000 7fd4da200700 1 -- 172.21.15.5:0/737543583 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4d4064150 con 0x7fd4d4105db0 2024-04-24T06:41:47.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.580+0000 7fd4d2000700 1 -- 172.21.15.5:0/737543583 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7fd4bc02a070 con 0x7fd4d4105db0 2024-04-24T06:41:47.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.582+0000 7fd4d2000700 1 --2- 172.21.15.5:0/737543583 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fd4b80a3e00 0x7fd4b80a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:47.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.582+0000 7fd4d2000700 1 -- 172.21.15.5:0/737543583 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fd4bc0d1350 con 0x7fd4d4105db0 2024-04-24T06:41:47.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.582+0000 7fd4d8e00700 1 --2- 172.21.15.5:0/737543583 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fd4b80a3e00 0x7fd4b80a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:47.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.583+0000 7fd4d8e00700 1 --2- 172.21.15.5:0/737543583 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fd4b80a3e00 0x7fd4b80a62c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd4d419c1f0 tx=0x7fd4c4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:47.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.586+0000 7fd4d2000700 1 -- 172.21.15.5:0/737543583 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd4bc013070 con 0x7fd4d4105db0 2024-04-24T06:41:47.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.896+0000 7fd4da200700 1 -- 172.21.15.5:0/737543583 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fd4d402e100 con 0x7fd4d4105db0 2024-04-24T06:41:47.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.905+0000 7fd4d2000700 1 -- 172.21.15.5:0/737543583 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fd4d402e100 con 0x7fd4d4105db0 2024-04-24T06:41:47.907 INFO:teuthology.orchestra.run.smithi005.stdout:[client.0] 2024-04-24T06:41:47.907 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCrqShmymibNRAAUdSqCceA4/9fR0FWuwJfsw== 2024-04-24T06:41:47.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.908+0000 7fd4da200700 1 -- 172.21.15.5:0/737543583 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fd4b80a3e00 msgr2=0x7fd4b80a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:47.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.908+0000 7fd4da200700 1 --2- 172.21.15.5:0/737543583 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fd4b80a3e00 0x7fd4b80a62c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd4d419c1f0 tx=0x7fd4c4009260 comp rx=0 tx=0).stop 2024-04-24T06:41:47.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.908+0000 7fd4da200700 1 -- 172.21.15.5:0/737543583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4d4105db0 msgr2=0x7fd4d419b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:47.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.908+0000 7fd4da200700 1 --2- 172.21.15.5:0/737543583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4d4105db0 0x7fd4d419b250 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc006010 tx=0x7fd4bc00f690 comp rx=0 tx=0).stop 2024-04-24T06:41:47.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.908+0000 7fd4da200700 1 -- 172.21.15.5:0/737543583 shutdown_connections 2024-04-24T06:41:47.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.908+0000 7fd4da200700 1 --2- 172.21.15.5:0/737543583 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fd4b80a3e00 0x7fd4b80a62c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:47.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.908+0000 7fd4da200700 1 --2- 172.21.15.5:0/737543583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4d4106750 0x7fd4d41a2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:47.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.908+0000 7fd4da200700 1 --2- 172.21.15.5:0/737543583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4d4104bb0 0x7fd4d419ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:47.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.908+0000 7fd4da200700 1 --2- 172.21.15.5:0/737543583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4d4105db0 0x7fd4d419b250 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:47.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.909+0000 7fd4da200700 1 -- 172.21.15.5:0/737543583 >> 172.21.15.5:0/737543583 conn(0x7fd4d4100130 msgr2=0x7fd4d41090b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:47.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.909+0000 7fd4da200700 1 -- 172.21.15.5:0/737543583 shutdown_connections 2024-04-24T06:41:47.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:47.909+0000 7fd4da200700 1 -- 172.21.15.5:0/737543583 wait complete. 2024-04-24T06:41:48.596 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:48 smithi005 ceph-mon[27798]: pgmap v319: 97 pgs: 97 active+clean; 619 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 7.5 KiB/s wr, 16 op/s 2024-04-24T06:41:48.596 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:48 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/737543583' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T06:41:48.596 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:48 smithi005 ceph-mon[27798]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T06:41:48.596 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:48 smithi005 ceph-mon[27798]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-24T06:41:48.680 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-24T06:41:48.681 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-24T06:41:48.681 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-24T06:41:48.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:48 smithi137 ceph-mon[33017]: pgmap v319: 97 pgs: 97 active+clean; 619 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 7.5 KiB/s wr, 16 op/s 2024-04-24T06:41:48.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:48 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/737543583' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T06:41:48.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:48 smithi137 ceph-mon[33017]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T06:41:48.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:48 smithi137 ceph-mon[33017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-24T06:41:48.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:48 smithi037 ceph-mon[32864]: pgmap v319: 97 pgs: 97 active+clean; 619 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 7.5 KiB/s wr, 16 op/s 2024-04-24T06:41:48.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:48 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/737543583' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T06:41:48.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:48 smithi037 ceph-mon[32864]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T06:41:48.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:48 smithi037 ceph-mon[32864]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-24T06:41:48.813 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-24T06:41:48.813 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-24T06:41:48.813 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-24T06:41:49.317 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:41:50.665 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:50 smithi005 ceph-mon[27798]: pgmap v320: 97 pgs: 97 active+clean; 623 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-04-24T06:41:50.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:50 smithi137 ceph-mon[33017]: pgmap v320: 97 pgs: 97 active+clean; 623 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-04-24T06:41:50.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:50 smithi037 ceph-mon[32864]: pgmap v320: 97 pgs: 97 active+clean; 623 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-04-24T06:41:51.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:51.867+0000 7f0bff000700 1 -- 172.21.15.5:0/966197591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bf8109380 msgr2=0x7f0bf8109760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:51.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:51.867+0000 7f0bff000700 1 --2- 172.21.15.5:0/966197591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bf8109380 0x7f0bf8109760 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0be80099c0 tx=0x7f0be8009cd0 comp rx=0 tx=0).stop 2024-04-24T06:41:51.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:51.867+0000 7f0bff000700 1 -- 172.21.15.5:0/966197591 shutdown_connections 2024-04-24T06:41:51.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:51.867+0000 7f0bff000700 1 --2- 172.21.15.5:0/966197591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bf810e460 0x7f0bf8116030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:51.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:51.867+0000 7f0bff000700 1 --2- 172.21.15.5:0/966197591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bf8109380 0x7f0bf8109760 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:51.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:51.867+0000 7f0bff000700 1 --2- 172.21.15.5:0/966197591 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bf8109d30 0x7f0bf810dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:51.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:51.867+0000 7f0bff000700 1 -- 172.21.15.5:0/966197591 >> 172.21.15.5:0/966197591 conn(0x7f0bf80762c0 msgr2=0x7f0bf8107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:51.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:51.867+0000 7f0bff000700 1 -- 172.21.15.5:0/966197591 shutdown_connections 2024-04-24T06:41:51.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:51.867+0000 7f0bff000700 1 -- 172.21.15.5:0/966197591 wait complete. 2024-04-24T06:41:51.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:51.869+0000 7f0bff000700 1 Processor -- start 2024-04-24T06:41:51.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:51.869+0000 7f0bff000700 1 -- start start 2024-04-24T06:41:51.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:51.869+0000 7f0bff000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bf8109380 0x7f0bf81a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:51.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:51.869+0000 7f0bff000700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bf8109d30 0x7f0bf81a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:51.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:51.869+0000 7f0bff000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bf810e460 0x7f0bf819f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:51.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:51.869+0000 7f0bff000700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf8119490 con 0x7f0bf8109d30 2024-04-24T06:41:51.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:51.869+0000 7f0bff000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf8119310 con 0x7f0bf810e460 2024-04-24T06:41:51.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:51.869+0000 7f0bff000700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf8119610 con 0x7f0bf8109380 2024-04-24T06:41:51.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:51.870+0000 7f0bf7e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bf8109380 0x7f0bf81a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:51.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:51.870+0000 7f0bef400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bf8109d30 0x7f0bf81a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:51.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:51.870+0000 7f0bf7e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bf8109380 0x7f0bf81a56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:48688/0 (socket says 172.21.15.5:48688) 2024-04-24T06:41:51.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:51.870+0000 7f0bf7e00700 1 -- 172.21.15.5:0/2686871502 learned_addr learned my addr 172.21.15.5:0/2686871502 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:41:51.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:51.870+0000 7f0bfd000700 1 --2- 172.21.15.5:0/2686871502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bf810e460 0x7f0bf819f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:51.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:51.870+0000 7f0bf7e00700 1 -- 172.21.15.5:0/2686871502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bf810e460 msgr2=0x7f0bf819f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:51.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:51.870+0000 7f0bf7e00700 1 --2- 172.21.15.5:0/2686871502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bf810e460 0x7f0bf819f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:51.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:51.870+0000 7f0bf7e00700 1 -- 172.21.15.5:0/2686871502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bf8109d30 msgr2=0x7f0bf81a5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:51.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:51.870+0000 7f0bf7e00700 1 --2- 172.21.15.5:0/2686871502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bf8109d30 0x7f0bf81a5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:51.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:51.870+0000 7f0bf7e00700 1 -- 172.21.15.5:0/2686871502 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf81a0050 con 0x7f0bf8109380 2024-04-24T06:41:51.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:51.870+0000 7f0bf7e00700 1 --2- 172.21.15.5:0/2686871502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bf8109380 0x7f0bf81a56a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0be8006010 tx=0x7f0be8003e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:51.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:51.871+0000 7f0bf5600700 1 -- 172.21.15.5:0/2686871502 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be8009e60 con 0x7f0bf8109380 2024-04-24T06:41:51.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:51.871+0000 7f0bf5600700 1 -- 172.21.15.5:0/2686871502 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0be8003680 con 0x7f0bf8109380 2024-04-24T06:41:51.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:51.871+0000 7f0bf5600700 1 -- 172.21.15.5:0/2686871502 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be800fe50 con 0x7f0bf8109380 2024-04-24T06:41:51.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:51.871+0000 7f0bff000700 1 -- 172.21.15.5:0/2686871502 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bf81a02e0 con 0x7f0bf8109380 2024-04-24T06:41:51.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:51.871+0000 7f0bff000700 1 -- 172.21.15.5:0/2686871502 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bf81ac620 con 0x7f0bf8109380 2024-04-24T06:41:51.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:51.872+0000 7f0bff000700 1 -- 172.21.15.5:0/2686871502 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bf81a0640 con 0x7f0bf8109380 2024-04-24T06:41:51.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:51.874+0000 7f0bf5600700 1 -- 172.21.15.5:0/2686871502 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f0be800f580 con 0x7f0bf8109380 2024-04-24T06:41:51.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:51.876+0000 7f0bf5600700 1 --2- 172.21.15.5:0/2686871502 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f0bdc0a3ef0 0x7f0bdc0a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:51.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:51.876+0000 7f0bf5600700 1 -- 172.21.15.5:0/2686871502 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f0be80d12e0 con 0x7f0bf8109380 2024-04-24T06:41:51.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:51.876+0000 7f0bef400700 1 --2- 172.21.15.5:0/2686871502 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f0bdc0a3ef0 0x7f0bdc0a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:51.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:51.877+0000 7f0bef400700 1 --2- 172.21.15.5:0/2686871502 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f0bdc0a3ef0 0x7f0bdc0a63b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0bf81a0f30 tx=0x7f0be000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:51.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:51.878+0000 7f0bf5600700 1 -- 172.21.15.5:0/2686871502 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0be8013070 con 0x7f0bf8109380 2024-04-24T06:41:52.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:52.172+0000 7f0bff000700 1 -- 172.21.15.5:0/2686871502 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f0bf80697e0 con 0x7f0bf8109380 2024-04-24T06:41:52.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:52.178+0000 7f0bf5600700 1 -- 172.21.15.5:0/2686871502 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264987 (secure 0 0 0) 0x7f0be809dca0 con 0x7f0bf8109380 2024-04-24T06:41:52.186 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:41:52.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:52.184+0000 7f0bff000700 1 -- 172.21.15.5:0/2686871502 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f0bdc0a3ef0 msgr2=0x7f0bdc0a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:52.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:52.184+0000 7f0bff000700 1 --2- 172.21.15.5:0/2686871502 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f0bdc0a3ef0 0x7f0bdc0a63b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0bf81a0f30 tx=0x7f0be000b040 comp rx=0 tx=0).stop 2024-04-24T06:41:52.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:52.184+0000 7f0bff000700 1 -- 172.21.15.5:0/2686871502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bf8109380 msgr2=0x7f0bf81a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:52.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:52.184+0000 7f0bff000700 1 --2- 172.21.15.5:0/2686871502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bf8109380 0x7f0bf81a56a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0be8006010 tx=0x7f0be8003e00 comp rx=0 tx=0).stop 2024-04-24T06:41:52.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:52.184+0000 7f0bff000700 1 -- 172.21.15.5:0/2686871502 shutdown_connections 2024-04-24T06:41:52.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:52.184+0000 7f0bff000700 1 --2- 172.21.15.5:0/2686871502 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f0bdc0a3ef0 0x7f0bdc0a63b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:52.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:52.184+0000 7f0bff000700 1 --2- 172.21.15.5:0/2686871502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bf810e460 0x7f0bf819f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:52.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:52.184+0000 7f0bff000700 1 --2- 172.21.15.5:0/2686871502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bf8109380 0x7f0bf81a56a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:52.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:52.184+0000 7f0bff000700 1 --2- 172.21.15.5:0/2686871502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bf8109d30 0x7f0bf81a5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:52.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:52.184+0000 7f0bff000700 1 -- 172.21.15.5:0/2686871502 >> 172.21.15.5:0/2686871502 conn(0x7f0bf80762c0 msgr2=0x7f0bf8077e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:52.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:52.185+0000 7f0bff000700 1 -- 172.21.15.5:0/2686871502 shutdown_connections 2024-04-24T06:41:52.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:52.185+0000 7f0bff000700 1 -- 172.21.15.5:0/2686871502 wait complete. 2024-04-24T06:41:52.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:52 smithi005 ceph-mon[27798]: pgmap v321: 97 pgs: 97 active+clean; 623 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.8 KiB/s wr, 13 op/s 2024-04-24T06:41:52.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:52 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/2686871502' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-24T06:41:52.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:52 smithi137 ceph-mon[33017]: pgmap v321: 97 pgs: 97 active+clean; 623 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.8 KiB/s wr, 13 op/s 2024-04-24T06:41:52.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:52 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/2686871502' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-24T06:41:52.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:52 smithi037 ceph-mon[32864]: pgmap v321: 97 pgs: 97 active+clean; 623 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.8 KiB/s wr, 13 op/s 2024-04-24T06:41:52.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:52 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/2686871502' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-24T06:41:53.143 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":4042936965},{"type":"v1","addr":"172.21.15.5:6801","nonce":4042936965}]},"active_addr":"172.21.15.5:6801/4042936965","active_change":"2024-04-24T06:33:41.162084+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24112,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.5:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":805658012}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":2912680052}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":1012004002}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":3478347847}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":584944646}]}]} 2024-04-24T06:41:53.149 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-24T06:41:53.149 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-24T06:41:53.149 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-24T06:41:53.618 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:41:54.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:54 smithi137 ceph-mon[33017]: pgmap v322: 97 pgs: 97 active+clean; 630 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.7 KiB/s wr, 14 op/s 2024-04-24T06:41:54.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:54 smithi005 ceph-mon[27798]: pgmap v322: 97 pgs: 97 active+clean; 630 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.7 KiB/s wr, 14 op/s 2024-04-24T06:41:54.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:54 smithi037 ceph-mon[32864]: pgmap v322: 97 pgs: 97 active+clean; 630 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.7 KiB/s wr, 14 op/s 2024-04-24T06:41:55.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.761+0000 7f55bdc00700 1 -- 172.21.15.5:0/2648125208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55b81019f0 msgr2=0x7f55b8101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:55.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.761+0000 7f55bdc00700 1 --2- 172.21.15.5:0/2648125208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55b81019f0 0x7f55b8101e70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f55a0009990 tx=0x7f55a0009ca0 comp rx=0 tx=0).stop 2024-04-24T06:41:55.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.761+0000 7f55bdc00700 1 -- 172.21.15.5:0/2648125208 shutdown_connections 2024-04-24T06:41:55.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.761+0000 7f55bdc00700 1 --2- 172.21.15.5:0/2648125208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55b81023b0 0x7f55b810e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:55.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.761+0000 7f55bdc00700 1 --2- 172.21.15.5:0/2648125208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55b8107a40 0x7f55b8107e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:55.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.761+0000 7f55bdc00700 1 --2- 172.21.15.5:0/2648125208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55b81019f0 0x7f55b8101e70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:55.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.761+0000 7f55bdc00700 1 -- 172.21.15.5:0/2648125208 >> 172.21.15.5:0/2648125208 conn(0x7f55b80789a0 msgr2=0x7f55b8078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:55.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.761+0000 7f55bdc00700 1 -- 172.21.15.5:0/2648125208 shutdown_connections 2024-04-24T06:41:55.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.761+0000 7f55bdc00700 1 -- 172.21.15.5:0/2648125208 wait complete. 2024-04-24T06:41:55.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.763+0000 7f55bdc00700 1 Processor -- start 2024-04-24T06:41:55.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.763+0000 7f55bdc00700 1 -- start start 2024-04-24T06:41:55.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.763+0000 7f55bdc00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55b81019f0 0x7f55b819b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:55.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.763+0000 7f55bdc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55b81023b0 0x7f55b819ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:55.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.763+0000 7f55bdc00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55b8107a40 0x7f55b819fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:55.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.763+0000 7f55bdc00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f55b8111d40 con 0x7f55b8107a40 2024-04-24T06:41:55.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.763+0000 7f55bdc00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f55b8111bc0 con 0x7f55b81023b0 2024-04-24T06:41:55.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.763+0000 7f55bdc00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f55b8111ec0 con 0x7f55b81019f0 2024-04-24T06:41:55.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.764+0000 7f55b7400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55b8107a40 0x7f55b819fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:55.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.764+0000 7f55b6a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55b81019f0 0x7f55b819b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:55.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.764+0000 7f55ae000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55b81023b0 0x7f55b819ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:55.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.764+0000 7f55b6a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55b81019f0 0x7f55b819b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:45476/0 (socket says 172.21.15.5:45476) 2024-04-24T06:41:55.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.764+0000 7f55b6a00700 1 -- 172.21.15.5:0/3060496786 learned_addr learned my addr 172.21.15.5:0/3060496786 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:41:55.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.764+0000 7f55ae000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55b81023b0 0x7f55b819ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:47110/0 (socket says 172.21.15.5:47110) 2024-04-24T06:41:55.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.764+0000 7f55b6a00700 1 -- 172.21.15.5:0/3060496786 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55b81023b0 msgr2=0x7f55b819ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:55.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.764+0000 7f55b6a00700 1 --2- 172.21.15.5:0/3060496786 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55b81023b0 0x7f55b819ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:55.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.764+0000 7f55b6a00700 1 -- 172.21.15.5:0/3060496786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55b8107a40 msgr2=0x7f55b819fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:55.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.764+0000 7f55b6a00700 1 --2- 172.21.15.5:0/3060496786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55b8107a40 0x7f55b819fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:55.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.764+0000 7f55b6a00700 1 -- 172.21.15.5:0/3060496786 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b81a04f0 con 0x7f55b81019f0 2024-04-24T06:41:55.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.764+0000 7f55b7400700 1 --2- 172.21.15.5:0/3060496786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55b8107a40 0x7f55b819fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:41:55.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.765+0000 7f55b6a00700 1 --2- 172.21.15.5:0/3060496786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55b81019f0 0x7f55b819b4e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f55a800eb30 tx=0x7f55a800ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:55.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.765+0000 7f55afe00700 1 -- 172.21.15.5:0/3060496786 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a800cbd0 con 0x7f55b81019f0 2024-04-24T06:41:55.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.765+0000 7f55bdc00700 1 -- 172.21.15.5:0/3060496786 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55b81a07e0 con 0x7f55b81019f0 2024-04-24T06:41:55.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.765+0000 7f55afe00700 1 -- 172.21.15.5:0/3060496786 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55a8004500 con 0x7f55b81019f0 2024-04-24T06:41:55.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.765+0000 7f55afe00700 1 -- 172.21.15.5:0/3060496786 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a8010430 con 0x7f55b81019f0 2024-04-24T06:41:55.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.765+0000 7f55bdc00700 1 -- 172.21.15.5:0/3060496786 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55b81a8060 con 0x7f55b81019f0 2024-04-24T06:41:55.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.766+0000 7f55bdc00700 1 -- 172.21.15.5:0/3060496786 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55b80697e0 con 0x7f55b81019f0 2024-04-24T06:41:55.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.768+0000 7f55afe00700 1 -- 172.21.15.5:0/3060496786 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f55a8003f00 con 0x7f55b81019f0 2024-04-24T06:41:55.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.769+0000 7f55afe00700 1 --2- 172.21.15.5:0/3060496786 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f559c0a3cd0 0x7f559c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:55.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.770+0000 7f55afe00700 1 -- 172.21.15.5:0/3060496786 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f55a80d07b0 con 0x7f55b81019f0 2024-04-24T06:41:55.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.770+0000 7f55ae000700 1 --2- 172.21.15.5:0/3060496786 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f559c0a3cd0 0x7f559c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:55.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.770+0000 7f55ae000700 1 --2- 172.21.15.5:0/3060496786 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f559c0a3cd0 0x7f559c0a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f55b8100b40 tx=0x7f55a0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:55.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:55.774+0000 7f55afe00700 1 -- 172.21.15.5:0/3060496786 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f55a809d0e0 con 0x7f55b81019f0 2024-04-24T06:41:56.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:56.007+0000 7f55bdc00700 1 -- 172.21.15.5:0/3060496786 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f55b819c840 con 0x7f55b81019f0 2024-04-24T06:41:56.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:56.010+0000 7f55afe00700 1 -- 172.21.15.5:0/3060496786 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22088 (secure 0 0 0) 0x7f55a809c9e0 con 0x7f55b81019f0 2024-04-24T06:41:56.012 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:41:56.013 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":86,"fsid":"43e9bad0-0204-11ef-bc93-c7b262605968","created":"2024-04-24T06:32:26.340555+0000","modified":"2024-04-24T06:41:36.259462+0000","last_up_change":"2024-04-24T06:40:36.790934+0000","last_in_change":"2024-04-24T06:40:17.803257+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:36:27.320342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:41:32.254502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:41:33.739067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c2e974c-745c-4dd5-8c3d-8e8da2eb1ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6803","nonce":2155858515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6805","nonce":2155858515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6809","nonce":2155858515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6807","nonce":2155858515}]},"public_addr":"172.21.15.5:6803/2155858515","cluster_addr":"172.21.15.5:6805/2155858515","heartbeat_back_addr":"172.21.15.5:6809/2155858515","heartbeat_front_addr":"172.21.15.5:6807/2155858515","state":["exists","up"]},{"osd":1,"uuid":"50467744-db20-4721-bf2d-1c5db938297b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6801","nonce":1302362650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6803","nonce":1302362650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6807","nonce":1302362650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6805","nonce":1302362650}]},"public_addr":"172.21.15.37:6801/1302362650","cluster_addr":"172.21.15.37:6803/1302362650","heartbeat_back_addr":"172.21.15.37:6807/1302362650","heartbeat_front_addr":"172.21.15.37:6805/1302362650","state":["exists","up"]},{"osd":2,"uuid":"0e190b15-d04c-4288-8d48-ecf6664392fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6801","nonce":2885027256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6803","nonce":2885027256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6807","nonce":2885027256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6805","nonce":2885027256}]},"public_addr":"172.21.15.137:6801/2885027256","cluster_addr":"172.21.15.137:6803/2885027256","heartbeat_back_addr":"172.21.15.137:6807/2885027256","heartbeat_front_addr":"172.21.15.137:6805/2885027256","state":["exists","up"]},{"osd":3,"uuid":"e05592ba-d96a-472a-9f63-09a535d04dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6811","nonce":3701747260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6813","nonce":3701747260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6817","nonce":3701747260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6815","nonce":3701747260}]},"public_addr":"172.21.15.5:6811/3701747260","cluster_addr":"172.21.15.5:6813/3701747260","heartbeat_back_addr":"172.21.15.5:6817/3701747260","heartbeat_front_addr":"172.21.15.5:6815/3701747260","state":["exists","up"]},{"osd":4,"uuid":"c3f36e51-7a3b-4dd9-af4b-a10558830c4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6809","nonce":759253844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6811","nonce":759253844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6815","nonce":759253844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6813","nonce":759253844}]},"public_addr":"172.21.15.37:6809/759253844","cluster_addr":"172.21.15.37:6811/759253844","heartbeat_back_addr":"172.21.15.37:6815/759253844","heartbeat_front_addr":"172.21.15.37:6813/759253844","state":["exists","up"]},{"osd":5,"uuid":"97a0500c-cbb8-4ff5-b476-8634b26b3821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6809","nonce":3771921560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6811","nonce":3771921560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6815","nonce":3771921560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6813","nonce":3771921560}]},"public_addr":"172.21.15.137:6809/3771921560","cluster_addr":"172.21.15.137:6811/3771921560","heartbeat_back_addr":"172.21.15.137:6815/3771921560","heartbeat_front_addr":"172.21.15.137:6813/3771921560","state":["exists","up"]},{"osd":6,"uuid":"33a33d62-bbb3-4ed9-834e-19ae93a97670","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6819","nonce":2064636084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6821","nonce":2064636084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6825","nonce":2064636084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6823","nonce":2064636084}]},"public_addr":"172.21.15.5:6819/2064636084","cluster_addr":"172.21.15.5:6821/2064636084","heartbeat_back_addr":"172.21.15.5:6825/2064636084","heartbeat_front_addr":"172.21.15.5:6823/2064636084","state":["exists","up"]},{"osd":7,"uuid":"c0f4e036-4717-4514-bb59-ab11fbaab063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6817","nonce":2952750659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6819","nonce":2952750659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6823","nonce":2952750659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6821","nonce":2952750659}]},"public_addr":"172.21.15.37:6817/2952750659","cluster_addr":"172.21.15.37:6819/2952750659","heartbeat_back_addr":"172.21.15.37:6823/2952750659","heartbeat_front_addr":"172.21.15.37:6821/2952750659","state":["exists","up"]},{"osd":8,"uuid":"5391fd87-44ea-440f-80da-c54d0327c04c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6817","nonce":29724945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6819","nonce":29724945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6823","nonce":29724945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6821","nonce":29724945}]},"public_addr":"172.21.15.137:6817/29724945","cluster_addr":"172.21.15.137:6819/29724945","heartbeat_back_addr":"172.21.15.137:6823/29724945","heartbeat_front_addr":"172.21.15.137:6821/29724945","state":["exists","up"]},{"osd":9,"uuid":"d94c9dbc-f43d-4237-ac7f-47a6a5ede1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6827","nonce":2257102107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6829","nonce":2257102107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6833","nonce":2257102107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6831","nonce":2257102107}]},"public_addr":"172.21.15.5:6827/2257102107","cluster_addr":"172.21.15.5:6829/2257102107","heartbeat_back_addr":"172.21.15.5:6833/2257102107","heartbeat_front_addr":"172.21.15.5:6831/2257102107","state":["exists","up"]},{"osd":10,"uuid":"e2455332-2a5e-44b1-85d3-8e47da399f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6825","nonce":2451570637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6827","nonce":2451570637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6831","nonce":2451570637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6829","nonce":2451570637}]},"public_addr":"172.21.15.37:6825/2451570637","cluster_addr":"172.21.15.37:6827/2451570637","heartbeat_back_addr":"172.21.15.37:6831/2451570637","heartbeat_front_addr":"172.21.15.37:6829/2451570637","state":["exists","up"]},{"osd":11,"uuid":"49d9396b-704a-4cef-8210-75d30c28e924","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6825","nonce":90600596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6827","nonce":90600596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6831","nonce":90600596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6829","nonce":90600596}]},"public_addr":"172.21.15.137:6825/90600596","cluster_addr":"172.21.15.137:6827/90600596","heartbeat_back_addr":"172.21.15.137:6831/90600596","heartbeat_front_addr":"172.21.15.137:6829/90600596","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:35:35.591619+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:00.568817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:24.195643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:56.451724+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:22.109293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:46.161749+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:19.278305+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:45.620256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:10.041165+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:43.813697+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:10.736026+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:35.384377+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/23307164":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6801/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:0/230785941":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/2940023597":"2024-04-25T06:33:07.040369+0000","172.21.15.5:6801/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3886613904":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/4199165437":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3459593499":"2024-04-25T06:33:41.161852+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:41:56.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:56.013+0000 7f55bdc00700 1 -- 172.21.15.5:0/3060496786 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f559c0a3cd0 msgr2=0x7f559c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:56.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:56.013+0000 7f55bdc00700 1 --2- 172.21.15.5:0/3060496786 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f559c0a3cd0 0x7f559c0a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f55b8100b40 tx=0x7f55a0005fd0 comp rx=0 tx=0).stop 2024-04-24T06:41:56.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:56.013+0000 7f55bdc00700 1 -- 172.21.15.5:0/3060496786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55b81019f0 msgr2=0x7f55b819b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:56.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:56.013+0000 7f55bdc00700 1 --2- 172.21.15.5:0/3060496786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55b81019f0 0x7f55b819b4e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f55a800eb30 tx=0x7f55a800ee40 comp rx=0 tx=0).stop 2024-04-24T06:41:56.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:56.013+0000 7f55bdc00700 1 -- 172.21.15.5:0/3060496786 shutdown_connections 2024-04-24T06:41:56.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:56.013+0000 7f55bdc00700 1 --2- 172.21.15.5:0/3060496786 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f559c0a3cd0 0x7f559c0a6190 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:56.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:56.013+0000 7f55bdc00700 1 --2- 172.21.15.5:0/3060496786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55b8107a40 0x7f55b819fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:56.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:56.013+0000 7f55bdc00700 1 --2- 172.21.15.5:0/3060496786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55b81019f0 0x7f55b819b4e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:56.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:56.013+0000 7f55bdc00700 1 --2- 172.21.15.5:0/3060496786 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55b81023b0 0x7f55b819ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:56.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:56.013+0000 7f55bdc00700 1 -- 172.21.15.5:0/3060496786 >> 172.21.15.5:0/3060496786 conn(0x7f55b80789a0 msgr2=0x7f55b80ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:56.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:56.014+0000 7f55bdc00700 1 -- 172.21.15.5:0/3060496786 shutdown_connections 2024-04-24T06:41:56.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:56.014+0000 7f55bdc00700 1 -- 172.21.15.5:0/3060496786 wait complete. 2024-04-24T06:41:56.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:56 smithi137 ceph-mon[33017]: pgmap v323: 97 pgs: 97 active+clean; 630 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-24T06:41:56.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:56 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/3060496786' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:41:56.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:56 smithi005 ceph-mon[27798]: pgmap v323: 97 pgs: 97 active+clean; 630 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-24T06:41:56.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:56 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/3060496786' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:41:56.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:56 smithi037 ceph-mon[32864]: pgmap v323: 97 pgs: 97 active+clean; 630 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-24T06:41:56.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:56 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/3060496786' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:41:56.979 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-24T06:41:56.980 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-24T06:41:57.451 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:41:58.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:41:58 smithi137 ceph-mon[33017]: pgmap v324: 97 pgs: 97 active+clean; 630 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-24T06:41:58.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:41:58 smithi005 ceph-mon[27798]: pgmap v324: 97 pgs: 97 active+clean; 630 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-24T06:41:58.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:41:58 smithi037 ceph-mon[32864]: pgmap v324: 97 pgs: 97 active+clean; 630 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-24T06:41:59.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.743+0000 7f733ce00700 1 -- 172.21.15.5:0/2184130095 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f733806b090 msgr2=0x7f733806b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:59.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.743+0000 7f733ce00700 1 --2- 172.21.15.5:0/2184130095 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f733806b090 0x7f733806b4f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7324009990 tx=0x7f7324009ca0 comp rx=0 tx=0).stop 2024-04-24T06:41:59.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.743+0000 7f733ce00700 1 -- 172.21.15.5:0/2184130095 shutdown_connections 2024-04-24T06:41:59.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.743+0000 7f733ce00700 1 --2- 172.21.15.5:0/2184130095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f733806ba30 0x7f733810f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:59.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.743+0000 7f733ce00700 1 --2- 172.21.15.5:0/2184130095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7338108b60 0x7f7338108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:59.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.743+0000 7f733ce00700 1 --2- 172.21.15.5:0/2184130095 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f733806b090 0x7f733806b4f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:59.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.743+0000 7f733ce00700 1 -- 172.21.15.5:0/2184130095 >> 172.21.15.5:0/2184130095 conn(0x7f7338078960 msgr2=0x7f7338078d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:59.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.743+0000 7f733ce00700 1 -- 172.21.15.5:0/2184130095 shutdown_connections 2024-04-24T06:41:59.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.743+0000 7f733ce00700 1 -- 172.21.15.5:0/2184130095 wait complete. 2024-04-24T06:41:59.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.745+0000 7f733ce00700 1 Processor -- start 2024-04-24T06:41:59.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.746+0000 7f733ce00700 1 -- start start 2024-04-24T06:41:59.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.746+0000 7f733ce00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f733806b090 0x7f733819b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:59.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.746+0000 7f733ce00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f733806ba30 0x7f733819b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:59.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.746+0000 7f733ce00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7338108b60 0x7f733819fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:59.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.746+0000 7f733ce00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7338112de0 con 0x7f7338108b60 2024-04-24T06:41:59.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.746+0000 7f733ce00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7338112c60 con 0x7f733806ba30 2024-04-24T06:41:59.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.746+0000 7f733ce00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7338112f60 con 0x7f733806b090 2024-04-24T06:41:59.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.746+0000 7f7335400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f733806ba30 0x7f733819b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:59.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.747+0000 7f7335e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f733806b090 0x7f733819b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:59.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.747+0000 7f7335400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f733806ba30 0x7f733819b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:47118/0 (socket says 172.21.15.5:47118) 2024-04-24T06:41:59.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.747+0000 7f7335400700 1 -- 172.21.15.5:0/1223784413 learned_addr learned my addr 172.21.15.5:0/1223784413 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:41:59.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.747+0000 7f7336800700 1 --2- 172.21.15.5:0/1223784413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7338108b60 0x7f733819fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:59.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.747+0000 7f7335400700 1 -- 172.21.15.5:0/1223784413 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f733806b090 msgr2=0x7f733819b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:59.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.747+0000 7f7335400700 1 --2- 172.21.15.5:0/1223784413 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f733806b090 0x7f733819b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:59.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.747+0000 7f7335400700 1 -- 172.21.15.5:0/1223784413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7338108b60 msgr2=0x7f733819fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:59.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.747+0000 7f7335400700 1 --2- 172.21.15.5:0/1223784413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7338108b60 0x7f733819fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:59.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.747+0000 7f7335400700 1 -- 172.21.15.5:0/1223784413 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73381a0430 con 0x7f733806ba30 2024-04-24T06:41:59.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.747+0000 7f7335400700 1 --2- 172.21.15.5:0/1223784413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f733806ba30 0x7f733819b9b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7324000c00 tx=0x7f732400bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:59.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.748+0000 7f732ea00700 1 -- 172.21.15.5:0/1223784413 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7324027070 con 0x7f733806ba30 2024-04-24T06:41:59.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.748+0000 7f732ea00700 1 -- 172.21.15.5:0/1223784413 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7324003d70 con 0x7f733806ba30 2024-04-24T06:41:59.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.748+0000 7f732ea00700 1 -- 172.21.15.5:0/1223784413 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732400f460 con 0x7f733806ba30 2024-04-24T06:41:59.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.748+0000 7f733ce00700 1 -- 172.21.15.5:0/1223784413 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73381a06c0 con 0x7f733806ba30 2024-04-24T06:41:59.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.748+0000 7f733ce00700 1 -- 172.21.15.5:0/1223784413 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73381a7fa0 con 0x7f733806ba30 2024-04-24T06:41:59.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.749+0000 7f733ce00700 1 -- 172.21.15.5:0/1223784413 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73381041d0 con 0x7f733806ba30 2024-04-24T06:41:59.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.751+0000 7f732ea00700 1 -- 172.21.15.5:0/1223784413 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f732402a070 con 0x7f733806ba30 2024-04-24T06:41:59.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.753+0000 7f732ea00700 1 --2- 172.21.15.5:0/1223784413 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f731c0a3cd0 0x7f731c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:41:59.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.753+0000 7f732ea00700 1 -- 172.21.15.5:0/1223784413 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f73240d1270 con 0x7f733806ba30 2024-04-24T06:41:59.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.753+0000 7f7335e00700 1 --2- 172.21.15.5:0/1223784413 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f731c0a3cd0 0x7f731c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:41:59.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.754+0000 7f7335e00700 1 --2- 172.21.15.5:0/1223784413 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f731c0a3cd0 0x7f731c0a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f732800f540 tx=0x7f732800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:41:59.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.757+0000 7f732ea00700 1 -- 172.21.15.5:0/1223784413 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7324013070 con 0x7f733806ba30 2024-04-24T06:41:59.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.990+0000 7f733ce00700 1 -- 172.21.15.5:0/1223784413 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f733802dfe0 con 0x7f733806ba30 2024-04-24T06:41:59.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.991+0000 7f732ea00700 1 -- 172.21.15.5:0/1223784413 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22088 (secure 0 0 0) 0x7f732409dc30 con 0x7f733806ba30 2024-04-24T06:41:59.994 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:41:59.994 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":86,"fsid":"43e9bad0-0204-11ef-bc93-c7b262605968","created":"2024-04-24T06:32:26.340555+0000","modified":"2024-04-24T06:41:36.259462+0000","last_up_change":"2024-04-24T06:40:36.790934+0000","last_in_change":"2024-04-24T06:40:17.803257+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:36:27.320342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:41:32.254502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:41:33.739067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c2e974c-745c-4dd5-8c3d-8e8da2eb1ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6803","nonce":2155858515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6805","nonce":2155858515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6809","nonce":2155858515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6807","nonce":2155858515}]},"public_addr":"172.21.15.5:6803/2155858515","cluster_addr":"172.21.15.5:6805/2155858515","heartbeat_back_addr":"172.21.15.5:6809/2155858515","heartbeat_front_addr":"172.21.15.5:6807/2155858515","state":["exists","up"]},{"osd":1,"uuid":"50467744-db20-4721-bf2d-1c5db938297b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6801","nonce":1302362650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6803","nonce":1302362650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6807","nonce":1302362650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6805","nonce":1302362650}]},"public_addr":"172.21.15.37:6801/1302362650","cluster_addr":"172.21.15.37:6803/1302362650","heartbeat_back_addr":"172.21.15.37:6807/1302362650","heartbeat_front_addr":"172.21.15.37:6805/1302362650","state":["exists","up"]},{"osd":2,"uuid":"0e190b15-d04c-4288-8d48-ecf6664392fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6801","nonce":2885027256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6803","nonce":2885027256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6807","nonce":2885027256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6805","nonce":2885027256}]},"public_addr":"172.21.15.137:6801/2885027256","cluster_addr":"172.21.15.137:6803/2885027256","heartbeat_back_addr":"172.21.15.137:6807/2885027256","heartbeat_front_addr":"172.21.15.137:6805/2885027256","state":["exists","up"]},{"osd":3,"uuid":"e05592ba-d96a-472a-9f63-09a535d04dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6811","nonce":3701747260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6813","nonce":3701747260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6817","nonce":3701747260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6815","nonce":3701747260}]},"public_addr":"172.21.15.5:6811/3701747260","cluster_addr":"172.21.15.5:6813/3701747260","heartbeat_back_addr":"172.21.15.5:6817/3701747260","heartbeat_front_addr":"172.21.15.5:6815/3701747260","state":["exists","up"]},{"osd":4,"uuid":"c3f36e51-7a3b-4dd9-af4b-a10558830c4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6809","nonce":759253844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6811","nonce":759253844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6815","nonce":759253844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6813","nonce":759253844}]},"public_addr":"172.21.15.37:6809/759253844","cluster_addr":"172.21.15.37:6811/759253844","heartbeat_back_addr":"172.21.15.37:6815/759253844","heartbeat_front_addr":"172.21.15.37:6813/759253844","state":["exists","up"]},{"osd":5,"uuid":"97a0500c-cbb8-4ff5-b476-8634b26b3821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6809","nonce":3771921560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6811","nonce":3771921560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6815","nonce":3771921560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6813","nonce":3771921560}]},"public_addr":"172.21.15.137:6809/3771921560","cluster_addr":"172.21.15.137:6811/3771921560","heartbeat_back_addr":"172.21.15.137:6815/3771921560","heartbeat_front_addr":"172.21.15.137:6813/3771921560","state":["exists","up"]},{"osd":6,"uuid":"33a33d62-bbb3-4ed9-834e-19ae93a97670","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6819","nonce":2064636084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6821","nonce":2064636084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6825","nonce":2064636084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6823","nonce":2064636084}]},"public_addr":"172.21.15.5:6819/2064636084","cluster_addr":"172.21.15.5:6821/2064636084","heartbeat_back_addr":"172.21.15.5:6825/2064636084","heartbeat_front_addr":"172.21.15.5:6823/2064636084","state":["exists","up"]},{"osd":7,"uuid":"c0f4e036-4717-4514-bb59-ab11fbaab063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6817","nonce":2952750659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6819","nonce":2952750659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6823","nonce":2952750659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6821","nonce":2952750659}]},"public_addr":"172.21.15.37:6817/2952750659","cluster_addr":"172.21.15.37:6819/2952750659","heartbeat_back_addr":"172.21.15.37:6823/2952750659","heartbeat_front_addr":"172.21.15.37:6821/2952750659","state":["exists","up"]},{"osd":8,"uuid":"5391fd87-44ea-440f-80da-c54d0327c04c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6817","nonce":29724945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6819","nonce":29724945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6823","nonce":29724945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6821","nonce":29724945}]},"public_addr":"172.21.15.137:6817/29724945","cluster_addr":"172.21.15.137:6819/29724945","heartbeat_back_addr":"172.21.15.137:6823/29724945","heartbeat_front_addr":"172.21.15.137:6821/29724945","state":["exists","up"]},{"osd":9,"uuid":"d94c9dbc-f43d-4237-ac7f-47a6a5ede1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6827","nonce":2257102107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6829","nonce":2257102107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6833","nonce":2257102107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6831","nonce":2257102107}]},"public_addr":"172.21.15.5:6827/2257102107","cluster_addr":"172.21.15.5:6829/2257102107","heartbeat_back_addr":"172.21.15.5:6833/2257102107","heartbeat_front_addr":"172.21.15.5:6831/2257102107","state":["exists","up"]},{"osd":10,"uuid":"e2455332-2a5e-44b1-85d3-8e47da399f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6825","nonce":2451570637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6827","nonce":2451570637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6831","nonce":2451570637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6829","nonce":2451570637}]},"public_addr":"172.21.15.37:6825/2451570637","cluster_addr":"172.21.15.37:6827/2451570637","heartbeat_back_addr":"172.21.15.37:6831/2451570637","heartbeat_front_addr":"172.21.15.37:6829/2451570637","state":["exists","up"]},{"osd":11,"uuid":"49d9396b-704a-4cef-8210-75d30c28e924","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6825","nonce":90600596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6827","nonce":90600596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6831","nonce":90600596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6829","nonce":90600596}]},"public_addr":"172.21.15.137:6825/90600596","cluster_addr":"172.21.15.137:6827/90600596","heartbeat_back_addr":"172.21.15.137:6831/90600596","heartbeat_front_addr":"172.21.15.137:6829/90600596","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:35:35.591619+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:00.568817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:24.195643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:56.451724+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:22.109293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:46.161749+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:19.278305+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:45.620256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:10.041165+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:43.813697+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:10.736026+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:35.384377+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/23307164":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6801/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:0/230785941":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/2940023597":"2024-04-25T06:33:07.040369+0000","172.21.15.5:6801/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3886613904":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/4199165437":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3459593499":"2024-04-25T06:33:41.161852+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:41:59.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.995+0000 7f733ce00700 1 -- 172.21.15.5:0/1223784413 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f731c0a3cd0 msgr2=0x7f731c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:59.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.995+0000 7f733ce00700 1 --2- 172.21.15.5:0/1223784413 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f731c0a3cd0 0x7f731c0a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f732800f540 tx=0x7f732800b040 comp rx=0 tx=0).stop 2024-04-24T06:41:59.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.995+0000 7f733ce00700 1 -- 172.21.15.5:0/1223784413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f733806ba30 msgr2=0x7f733819b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:41:59.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.995+0000 7f733ce00700 1 --2- 172.21.15.5:0/1223784413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f733806ba30 0x7f733819b9b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7324000c00 tx=0x7f732400bab0 comp rx=0 tx=0).stop 2024-04-24T06:41:59.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.996+0000 7f733ce00700 1 -- 172.21.15.5:0/1223784413 shutdown_connections 2024-04-24T06:41:59.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.996+0000 7f733ce00700 1 --2- 172.21.15.5:0/1223784413 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f731c0a3cd0 0x7f731c0a6190 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:59.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.996+0000 7f733ce00700 1 --2- 172.21.15.5:0/1223784413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7338108b60 0x7f733819fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:59.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.996+0000 7f733ce00700 1 --2- 172.21.15.5:0/1223784413 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f733806b090 0x7f733819b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:59.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.996+0000 7f733ce00700 1 --2- 172.21.15.5:0/1223784413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f733806ba30 0x7f733819b9b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:41:59.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.996+0000 7f733ce00700 1 -- 172.21.15.5:0/1223784413 >> 172.21.15.5:0/1223784413 conn(0x7f7338078960 msgr2=0x7f733810dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:41:59.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.996+0000 7f733ce00700 1 -- 172.21.15.5:0/1223784413 shutdown_connections 2024-04-24T06:41:59.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:41:59.996+0000 7f733ce00700 1 -- 172.21.15.5:0/1223784413 wait complete. 2024-04-24T06:42:00.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:00 smithi137 ceph-mon[33017]: pgmap v325: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-04-24T06:42:00.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:00 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/1223784413' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:42:00.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:00 smithi005 ceph-mon[27798]: pgmap v325: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-04-24T06:42:00.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:00 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1223784413' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:42:00.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:00 smithi037 ceph-mon[32864]: pgmap v325: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-04-24T06:42:00.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:00 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/1223784413' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:42:00.884 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-24T06:42:00.885 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-24T06:42:00.885 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-24T06:42:00.885 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-24T06:42:00.886 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-24T06:42:00.886 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-24T06:42:00.886 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-24T06:42:00.887 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-24T06:42:00.887 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-24T06:42:00.888 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-24T06:42:00.888 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-24T06:42:00.888 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-24T06:42:02.120 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:42:02.120 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:42:02.125 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:42:02.127 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:42:02.137 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:42:02.138 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:42:02.138 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:42:02.139 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:42:02.152 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:42:02.153 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:42:02.154 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:42:02.156 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:42:02.605 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:02 smithi005 ceph-mon[27798]: pgmap v326: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-24T06:42:02.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:02 smithi137 ceph-mon[33017]: pgmap v326: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-24T06:42:02.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:02 smithi037 ceph-mon[32864]: pgmap v326: 97 pgs: 97 active+clean; 638 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-24T06:42:04.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:04 smithi137 ceph-mon[33017]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-24T06:42:04.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:04 smithi005 ceph-mon[27798]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-24T06:42:04.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:04 smithi037 ceph-mon[32864]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-24T06:42:06.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:06 smithi005 ceph-mon[27798]: pgmap v328: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-24T06:42:06.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:06 smithi137 ceph-mon[33017]: pgmap v328: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-24T06:42:06.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:06 smithi037 ceph-mon[32864]: pgmap v328: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-24T06:42:08.701 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:08 smithi005 ceph-mon[27798]: pgmap v329: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1023 B/s wr, 5 op/s 2024-04-24T06:42:08.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:08 smithi037 ceph-mon[32864]: pgmap v329: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1023 B/s wr, 5 op/s 2024-04-24T06:42:08.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:08 smithi137 ceph-mon[33017]: pgmap v329: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1023 B/s wr, 5 op/s 2024-04-24T06:42:10.725 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:10 smithi005 ceph-mon[27798]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-24T06:42:10.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:10 smithi037 ceph-mon[32864]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-24T06:42:10.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:10 smithi137 ceph-mon[33017]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-24T06:42:12.594 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:12 smithi005 ceph-mon[27798]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-24T06:42:12.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.761+0000 7f45a6a00700 1 -- 172.21.15.5:0/375201415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45a0107df0 msgr2=0x7f45a010f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:12.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.761+0000 7f45a6a00700 1 --2- 172.21.15.5:0/375201415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45a0107df0 0x7f45a010f9a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f459800b0a0 tx=0x7f459800b3b0 comp rx=0 tx=0).stop 2024-04-24T06:42:12.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.761+0000 7f45a6a00700 1 -- 172.21.15.5:0/375201415 shutdown_connections 2024-04-24T06:42:12.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.761+0000 7f45a6a00700 1 --2- 172.21.15.5:0/375201415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45a0107df0 0x7f45a010f9a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:12.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.761+0000 7f45a6a00700 1 --2- 172.21.15.5:0/375201415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45a0101580 0x7f45a0101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:12.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.761+0000 7f45a6a00700 1 --2- 172.21.15.5:0/375201415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45a0101f30 0x7f45a0107710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:12.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.761+0000 7f45a6a00700 1 -- 172.21.15.5:0/375201415 >> 172.21.15.5:0/375201415 conn(0x7f45a0076ca0 msgr2=0x7f45a00770b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:12.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.761+0000 7f45a6a00700 1 -- 172.21.15.5:0/375201415 shutdown_connections 2024-04-24T06:42:12.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.761+0000 7f45a6a00700 1 -- 172.21.15.5:0/375201415 wait complete. 2024-04-24T06:42:12.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.762+0000 7f45a6a00700 1 Processor -- start 2024-04-24T06:42:12.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.762+0000 7f45a6a00700 1 -- start start 2024-04-24T06:42:12.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.763+0000 7f45a6a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45a0101580 0x7f45a019b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:12.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.763+0000 7f45a6a00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45a0101f30 0x7f45a019b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:12.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.763+0000 7f45a6a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45a0107df0 0x7f45a019fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:12.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.763+0000 7f45a6a00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f45a0112e00 con 0x7f45a0101f30 2024-04-24T06:42:12.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.763+0000 7f45a6a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f45a0112c80 con 0x7f45a0101580 2024-04-24T06:42:12.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.763+0000 7f45a6a00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f45a0112f80 con 0x7f45a0107df0 2024-04-24T06:42:12.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.763+0000 7f45a4c00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45a0101f30 0x7f45a019b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:12.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.763+0000 7f45a5600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45a0101580 0x7f45a019b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:12.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.763+0000 7f45a5600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45a0101580 0x7f45a019b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:60016/0 (socket says 172.21.15.5:60016) 2024-04-24T06:42:12.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.763+0000 7f45a4c00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45a0101f30 0x7f45a019b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40990/0 (socket says 172.21.15.5:40990) 2024-04-24T06:42:12.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.763+0000 7f45a5600700 1 -- 172.21.15.5:0/32243311 learned_addr learned my addr 172.21.15.5:0/32243311 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:42:12.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.763+0000 7f45a6000700 1 --2- 172.21.15.5:0/32243311 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45a0107df0 0x7f45a019fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:12.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.764+0000 7f45a5600700 1 -- 172.21.15.5:0/32243311 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45a0107df0 msgr2=0x7f45a019fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:12.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.764+0000 7f45a5600700 1 --2- 172.21.15.5:0/32243311 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45a0107df0 0x7f45a019fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:12.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.764+0000 7f45a5600700 1 -- 172.21.15.5:0/32243311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45a0101f30 msgr2=0x7f45a019b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:12.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.764+0000 7f45a5600700 1 --2- 172.21.15.5:0/32243311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45a0101f30 0x7f45a019b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:12.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.764+0000 7f45a5600700 1 -- 172.21.15.5:0/32243311 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45a01a0380 con 0x7f45a0101580 2024-04-24T06:42:12.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.764+0000 7f45a4c00700 1 --2- 172.21.15.5:0/32243311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45a0101f30 0x7f45a019b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:42:12.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.764+0000 7f45a5600700 1 --2- 172.21.15.5:0/32243311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45a0101580 0x7f45a019b370 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f459400e9a0 tx=0x7f459400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:12.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.764+0000 7f459e000700 1 -- 172.21.15.5:0/32243311 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459400ca10 con 0x7f45a0101580 2024-04-24T06:42:12.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.764+0000 7f45a6a00700 1 -- 172.21.15.5:0/32243311 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45a01a0670 con 0x7f45a0101580 2024-04-24T06:42:12.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.765+0000 7f45a6a00700 1 -- 172.21.15.5:0/32243311 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45a01a7ef0 con 0x7f45a0101580 2024-04-24T06:42:12.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.765+0000 7f459e000700 1 -- 172.21.15.5:0/32243311 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f459400cbb0 con 0x7f45a0101580 2024-04-24T06:42:12.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.765+0000 7f459e000700 1 -- 172.21.15.5:0/32243311 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4594018680 con 0x7f45a0101580 2024-04-24T06:42:12.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.765+0000 7f45a6a00700 1 -- 172.21.15.5:0/32243311 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4580001030 con 0x7f45a0101580 2024-04-24T06:42:12.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:12 smithi037 ceph-mon[32864]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-24T06:42:12.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.768+0000 7f459e000700 1 -- 172.21.15.5:0/32243311 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f4594018820 con 0x7f45a0101580 2024-04-24T06:42:12.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.769+0000 7f459e000700 1 --2- 172.21.15.5:0/32243311 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f45880a3d50 0x7f45880a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:12.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.769+0000 7f459e000700 1 -- 172.21.15.5:0/32243311 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f45940d0850 con 0x7f45a0101580 2024-04-24T06:42:12.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.770+0000 7f45a4c00700 1 --2- 172.21.15.5:0/32243311 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f45880a3d50 0x7f45880a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:12.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.770+0000 7f459e000700 1 --2- 172.21.15.5:0/32243311 >> [v2:172.21.15.37:6800/1302362650,v1:172.21.15.37:6801/1302362650] conn(0x7f45880ad530 0x7f45880af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:12.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.770+0000 7f459e000700 1 -- 172.21.15.5:0/32243311 --> [v2:172.21.15.37:6800/1302362650,v1:172.21.15.37:6801/1302362650] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4594001d30 con 0x7f45880ad530 2024-04-24T06:42:12.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.770+0000 7f459e000700 1 -- 172.21.15.5:0/32243311 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f45940d0c00 con 0x7f45a0101580 2024-04-24T06:42:12.772 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:12 smithi137 ceph-mon[33017]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-24T06:42:12.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.770+0000 7f45a6000700 1 --2- 172.21.15.5:0/32243311 >> [v2:172.21.15.37:6800/1302362650,v1:172.21.15.37:6801/1302362650] conn(0x7f45880ad530 0x7f45880af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:12.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.770+0000 7f45a4c00700 1 --2- 172.21.15.5:0/32243311 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f45880a3d50 0x7f45880a6210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f45a010fdc0 tx=0x7f458c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:12.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.771+0000 7f45a6000700 1 --2- 172.21.15.5:0/32243311 >> [v2:172.21.15.37:6800/1302362650,v1:172.21.15.37:6801/1302362650] conn(0x7f45880ad530 0x7f45880af950 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:12.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.772+0000 7f459e000700 1 -- 172.21.15.5:0/32243311 <== osd.1 v2:172.21.15.37:6800/1302362650 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4594001d30 con 0x7f45880ad530 2024-04-24T06:42:12.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.790+0000 7f45a6a00700 1 -- 172.21.15.5:0/32243311 --> [v2:172.21.15.37:6800/1302362650,v1:172.21.15.37:6801/1302362650] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4580002e60 con 0x7f45880ad530 2024-04-24T06:42:12.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.791+0000 7f459e000700 1 -- 172.21.15.5:0/32243311 <== osd.1 v2:172.21.15.37:6800/1302362650 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4580002e60 con 0x7f45880ad530 2024-04-24T06:42:12.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.792+0000 7f45a6a00700 1 -- 172.21.15.5:0/32243311 >> [v2:172.21.15.37:6800/1302362650,v1:172.21.15.37:6801/1302362650] conn(0x7f45880ad530 msgr2=0x7f45880af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:12.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.792+0000 7f45a6a00700 1 --2- 172.21.15.5:0/32243311 >> [v2:172.21.15.37:6800/1302362650,v1:172.21.15.37:6801/1302362650] conn(0x7f45880ad530 0x7f45880af950 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:12.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.792+0000 7f45a6a00700 1 -- 172.21.15.5:0/32243311 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f45880a3d50 msgr2=0x7f45880a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:12.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.792+0000 7f45a6a00700 1 --2- 172.21.15.5:0/32243311 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f45880a3d50 0x7f45880a6210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f45a010fdc0 tx=0x7f458c0073d0 comp rx=0 tx=0).stop 2024-04-24T06:42:12.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.792+0000 7f45a6a00700 1 -- 172.21.15.5:0/32243311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45a0101580 msgr2=0x7f45a019b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:12.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.792+0000 7f45a6a00700 1 --2- 172.21.15.5:0/32243311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45a0101580 0x7f45a019b370 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f459400e9a0 tx=0x7f459400ecb0 comp rx=0 tx=0).stop 2024-04-24T06:42:12.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.792+0000 7f45a6000700 1 -- 172.21.15.5:0/32243311 reap_dead start 2024-04-24T06:42:12.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.792+0000 7f45a6a00700 1 -- 172.21.15.5:0/32243311 shutdown_connections 2024-04-24T06:42:12.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.792+0000 7f45a6a00700 1 -- 172.21.15.5:0/32243311 >> 172.21.15.5:0/32243311 conn(0x7f45a0076ca0 msgr2=0x7f45a00ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:12.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.792+0000 7f45a6a00700 1 -- 172.21.15.5:0/32243311 shutdown_connections 2024-04-24T06:42:12.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:12.792+0000 7f45a6a00700 1 -- 172.21.15.5:0/32243311 wait complete. 2024-04-24T06:42:14.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:14 smithi005 ceph-mon[27798]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-24T06:42:14.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:14 smithi037 ceph-mon[32864]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-24T06:42:14.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:14 smithi137 ceph-mon[33017]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-24T06:42:16.721 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:16 smithi005 ceph-mon[27798]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:42:16.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:16 smithi037 ceph-mon[32864]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:42:16.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:16 smithi137 ceph-mon[33017]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:42:17.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.333+0000 7fa05de00700 1 -- 172.21.15.5:0/753799073 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa058102b40 msgr2=0x7fa058102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:17.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.333+0000 7fa05de00700 1 --2- 172.21.15.5:0/753799073 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa058102b40 0x7fa058102fc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa044009990 tx=0x7fa044009ca0 comp rx=0 tx=0).stop 2024-04-24T06:42:17.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.333+0000 7fa05de00700 1 -- 172.21.15.5:0/753799073 shutdown_connections 2024-04-24T06:42:17.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.333+0000 7fa05de00700 1 --2- 172.21.15.5:0/753799073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa058103500 0x7fa05810f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:17.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.333+0000 7fa05de00700 1 --2- 172.21.15.5:0/753799073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa058108b90 0x7fa058108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:17.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.333+0000 7fa05de00700 1 --2- 172.21.15.5:0/753799073 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa058102b40 0x7fa058102fc0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:17.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.333+0000 7fa05de00700 1 -- 172.21.15.5:0/753799073 >> 172.21.15.5:0/753799073 conn(0x7fa058079c50 msgr2=0x7fa05807a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:17.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.333+0000 7fa05de00700 1 -- 172.21.15.5:0/753799073 shutdown_connections 2024-04-24T06:42:17.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.333+0000 7fa05de00700 1 -- 172.21.15.5:0/753799073 wait complete. 2024-04-24T06:42:17.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.334+0000 7fa05de00700 1 Processor -- start 2024-04-24T06:42:17.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.335+0000 7fa05de00700 1 -- start start 2024-04-24T06:42:17.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.335+0000 7fa05de00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa058102b40 0x7fa05819b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:17.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.335+0000 7fa05de00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa058103500 0x7fa05819ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:17.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.335+0000 7fa05de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa058108b90 0x7fa05819fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:17.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.335+0000 7fa05de00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa058112e40 con 0x7fa058102b40 2024-04-24T06:42:17.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.335+0000 7fa05de00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa058112cc0 con 0x7fa058108b90 2024-04-24T06:42:17.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.335+0000 7fa05de00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa058112fc0 con 0x7fa058103500 2024-04-24T06:42:17.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.336+0000 7fa056a00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa058102b40 0x7fa05819b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:17.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.336+0000 7fa056a00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa058102b40 0x7fa05819b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52556/0 (socket says 172.21.15.5:52556) 2024-04-24T06:42:17.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.336+0000 7fa056a00700 1 -- 172.21.15.5:0/769640332 learned_addr learned my addr 172.21.15.5:0/769640332 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:42:17.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.336+0000 7fa056000700 1 --2- 172.21.15.5:0/769640332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa058103500 0x7fa05819ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:17.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.336+0000 7fa057400700 1 --2- 172.21.15.5:0/769640332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa058108b90 0x7fa05819fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:17.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.336+0000 7fa056a00700 1 -- 172.21.15.5:0/769640332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa058103500 msgr2=0x7fa05819ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:17.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.336+0000 7fa056a00700 1 --2- 172.21.15.5:0/769640332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa058103500 0x7fa05819ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:17.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.336+0000 7fa056a00700 1 -- 172.21.15.5:0/769640332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa058108b90 msgr2=0x7fa05819fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:17.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.336+0000 7fa056a00700 1 --2- 172.21.15.5:0/769640332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa058108b90 0x7fa05819fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:17.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.336+0000 7fa056a00700 1 -- 172.21.15.5:0/769640332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0581a0560 con 0x7fa058102b40 2024-04-24T06:42:17.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.336+0000 7fa056a00700 1 --2- 172.21.15.5:0/769640332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa058102b40 0x7fa05819b550 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa04800e9a0 tx=0x7fa04800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:17.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.337+0000 7fa05ca00700 1 -- 172.21.15.5:0/769640332 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04800ca10 con 0x7fa058102b40 2024-04-24T06:42:17.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.337+0000 7fa05ca00700 1 -- 172.21.15.5:0/769640332 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa04800cbb0 con 0x7fa058102b40 2024-04-24T06:42:17.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.337+0000 7fa05ca00700 1 -- 172.21.15.5:0/769640332 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa048018680 con 0x7fa058102b40 2024-04-24T06:42:17.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.337+0000 7fa05de00700 1 -- 172.21.15.5:0/769640332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0581a0850 con 0x7fa058102b40 2024-04-24T06:42:17.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.337+0000 7fa05de00700 1 -- 172.21.15.5:0/769640332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0581a80d0 con 0x7fa058102b40 2024-04-24T06:42:17.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.338+0000 7fa05de00700 1 -- 172.21.15.5:0/769640332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa058102050 con 0x7fa058102b40 2024-04-24T06:42:17.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.340+0000 7fa05ca00700 1 -- 172.21.15.5:0/769640332 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7fa048010430 con 0x7fa058102b40 2024-04-24T06:42:17.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.342+0000 7fa05ca00700 1 --2- 172.21.15.5:0/769640332 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fa03c0a3cd0 0x7fa03c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:17.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.342+0000 7fa05ca00700 1 -- 172.21.15.5:0/769640332 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fa0480d0e50 con 0x7fa058102b40 2024-04-24T06:42:17.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.342+0000 7fa056000700 1 --2- 172.21.15.5:0/769640332 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fa03c0a3cd0 0x7fa03c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:17.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.342+0000 7fa05ca00700 1 --2- 172.21.15.5:0/769640332 >> [v2:172.21.15.5:6810/3701747260,v1:172.21.15.5:6811/3701747260] conn(0x7fa03c0ad4b0 0x7fa03c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:17.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.342+0000 7fa05ca00700 1 -- 172.21.15.5:0/769640332 --> [v2:172.21.15.5:6810/3701747260,v1:172.21.15.5:6811/3701747260] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa048027e50 con 0x7fa03c0ad4b0 2024-04-24T06:42:17.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.342+0000 7fa05ca00700 1 -- 172.21.15.5:0/769640332 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fa0480d1200 con 0x7fa058102b40 2024-04-24T06:42:17.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.342+0000 7fa057400700 1 --2- 172.21.15.5:0/769640332 >> [v2:172.21.15.5:6810/3701747260,v1:172.21.15.5:6811/3701747260] conn(0x7fa03c0ad4b0 0x7fa03c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:17.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.343+0000 7fa056000700 1 --2- 172.21.15.5:0/769640332 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fa03c0a3cd0 0x7fa03c0a6190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa05810fda0 tx=0x7fa04400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:17.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.343+0000 7fa057400700 1 --2- 172.21.15.5:0/769640332 >> [v2:172.21.15.5:6810/3701747260,v1:172.21.15.5:6811/3701747260] conn(0x7fa03c0ad4b0 0x7fa03c0af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:17.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.344+0000 7fa05ca00700 1 -- 172.21.15.5:0/769640332 <== osd.3 v2:172.21.15.5:6810/3701747260 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa048027e50 con 0x7fa03c0ad4b0 2024-04-24T06:42:17.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.364+0000 7fa05de00700 1 -- 172.21.15.5:0/769640332 --> [v2:172.21.15.5:6810/3701747260,v1:172.21.15.5:6811/3701747260] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa0581041f0 con 0x7fa03c0ad4b0 2024-04-24T06:42:17.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.364+0000 7fa05ca00700 1 -- 172.21.15.5:0/769640332 <== osd.3 v2:172.21.15.5:6810/3701747260 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa0581041f0 con 0x7fa03c0ad4b0 2024-04-24T06:42:17.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.365+0000 7fa05de00700 1 -- 172.21.15.5:0/769640332 >> [v2:172.21.15.5:6810/3701747260,v1:172.21.15.5:6811/3701747260] conn(0x7fa03c0ad4b0 msgr2=0x7fa03c0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:17.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.365+0000 7fa05de00700 1 --2- 172.21.15.5:0/769640332 >> [v2:172.21.15.5:6810/3701747260,v1:172.21.15.5:6811/3701747260] conn(0x7fa03c0ad4b0 0x7fa03c0af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:17.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.365+0000 7fa05de00700 1 -- 172.21.15.5:0/769640332 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fa03c0a3cd0 msgr2=0x7fa03c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:17.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.365+0000 7fa05de00700 1 --2- 172.21.15.5:0/769640332 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fa03c0a3cd0 0x7fa03c0a6190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa05810fda0 tx=0x7fa04400b260 comp rx=0 tx=0).stop 2024-04-24T06:42:17.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.365+0000 7fa05de00700 1 -- 172.21.15.5:0/769640332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa058102b40 msgr2=0x7fa05819b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:17.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.365+0000 7fa05de00700 1 --2- 172.21.15.5:0/769640332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa058102b40 0x7fa05819b550 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa04800e9a0 tx=0x7fa04800ecb0 comp rx=0 tx=0).stop 2024-04-24T06:42:17.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.365+0000 7fa057400700 1 -- 172.21.15.5:0/769640332 reap_dead start 2024-04-24T06:42:17.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.365+0000 7fa05de00700 1 -- 172.21.15.5:0/769640332 shutdown_connections 2024-04-24T06:42:17.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.365+0000 7fa05de00700 1 -- 172.21.15.5:0/769640332 >> 172.21.15.5:0/769640332 conn(0x7fa058079c50 msgr2=0x7fa0581015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:17.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.365+0000 7fa05de00700 1 -- 172.21.15.5:0/769640332 shutdown_connections 2024-04-24T06:42:17.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:17.365+0000 7fa05de00700 1 -- 172.21.15.5:0/769640332 wait complete. 2024-04-24T06:42:18.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.868+0000 7f9febe00700 1 -- 172.21.15.5:0/190289791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fec1022e0 msgr2=0x7f9fec10e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:18.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.868+0000 7f9febe00700 1 --2- 172.21.15.5:0/190289791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fec1022e0 0x7f9fec10e7f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4009990 tx=0x7f9fe4009ca0 comp rx=0 tx=0).stop 2024-04-24T06:42:18.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.869+0000 7f9febe00700 1 -- 172.21.15.5:0/190289791 shutdown_connections 2024-04-24T06:42:18.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.869+0000 7f9febe00700 1 --2- 172.21.15.5:0/190289791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fec1022e0 0x7f9fec10e7f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:18.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.869+0000 7f9febe00700 1 --2- 172.21.15.5:0/190289791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fec107920 0x7f9fec107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:18.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.869+0000 7f9febe00700 1 --2- 172.21.15.5:0/190289791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9fec101920 0x7f9fec101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:18.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.869+0000 7f9febe00700 1 -- 172.21.15.5:0/190289791 >> 172.21.15.5:0/190289791 conn(0x7f9fec0fd410 msgr2=0x7f9fec0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:18.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.869+0000 7f9febe00700 1 -- 172.21.15.5:0/190289791 shutdown_connections 2024-04-24T06:42:18.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.869+0000 7f9febe00700 1 -- 172.21.15.5:0/190289791 wait complete. 2024-04-24T06:42:18.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.870+0000 7f9febe00700 1 Processor -- start 2024-04-24T06:42:18.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.870+0000 7f9febe00700 1 -- start start 2024-04-24T06:42:18.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.870+0000 7f9febe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fec101920 0x7f9fec1991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:18.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.870+0000 7f9febe00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fec1022e0 0x7f9fec199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:18.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.870+0000 7f9febe00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9fec107920 0x7f9fec19dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:18.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.870+0000 7f9febe00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9fec111ca0 con 0x7f9fec1022e0 2024-04-24T06:42:18.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.870+0000 7f9febe00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9fec111b20 con 0x7f9fec101920 2024-04-24T06:42:18.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.870+0000 7f9febe00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9fec111e20 con 0x7f9fec107920 2024-04-24T06:42:18.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.871+0000 7f9feaa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fec101920 0x7f9fec1991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:18.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.871+0000 7f9fe3e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fec1022e0 0x7f9fec199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:18.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.871+0000 7f9feaa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fec101920 0x7f9fec1991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:45546/0 (socket says 172.21.15.5:45546) 2024-04-24T06:42:18.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.871+0000 7f9feaa00700 1 -- 172.21.15.5:0/655731132 learned_addr learned my addr 172.21.15.5:0/655731132 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:42:18.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.871+0000 7f9feb400700 1 --2- 172.21.15.5:0/655731132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9fec107920 0x7f9fec19dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:18.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.871+0000 7f9feaa00700 1 -- 172.21.15.5:0/655731132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9fec107920 msgr2=0x7f9fec19dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:18.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.871+0000 7f9feaa00700 1 --2- 172.21.15.5:0/655731132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9fec107920 0x7f9fec19dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:18.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.871+0000 7f9feaa00700 1 -- 172.21.15.5:0/655731132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fec1022e0 msgr2=0x7f9fec199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:18.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.871+0000 7f9feaa00700 1 --2- 172.21.15.5:0/655731132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fec1022e0 0x7f9fec199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:18.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.871+0000 7f9feaa00700 1 -- 172.21.15.5:0/655731132 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fec19e1f0 con 0x7f9fec101920 2024-04-24T06:42:18.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.871+0000 7f9fe3e00700 1 --2- 172.21.15.5:0/655731132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fec1022e0 0x7f9fec199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:42:18.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.871+0000 7f9feaa00700 1 --2- 172.21.15.5:0/655731132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fec101920 0x7f9fec1991e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00ef10 tx=0x7f9fdc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:18.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.872+0000 7f9fe3400700 1 -- 172.21.15.5:0/655731132 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc00c9c0 con 0x7f9fec101920 2024-04-24T06:42:18.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.872+0000 7f9febe00700 1 -- 172.21.15.5:0/655731132 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fec19e480 con 0x7f9fec101920 2024-04-24T06:42:18.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.872+0000 7f9fe3400700 1 -- 172.21.15.5:0/655731132 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fdc019070 con 0x7f9fec101920 2024-04-24T06:42:18.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.872+0000 7f9fe3400700 1 -- 172.21.15.5:0/655731132 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc007e60 con 0x7f9fec101920 2024-04-24T06:42:18.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.872+0000 7f9febe00700 1 -- 172.21.15.5:0/655731132 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fec1a5d60 con 0x7f9fec101920 2024-04-24T06:42:18.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.873+0000 7f9febe00700 1 -- 172.21.15.5:0/655731132 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9fec063200 con 0x7f9fec101920 2024-04-24T06:42:18.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.875+0000 7f9fe3400700 1 -- 172.21.15.5:0/655731132 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f9fdc012050 con 0x7f9fec101920 2024-04-24T06:42:18.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.876+0000 7f9fe3400700 1 --2- 172.21.15.5:0/655731132 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f9fd00a3cd0 0x7f9fd00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:18.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.876+0000 7f9fe3400700 1 -- 172.21.15.5:0/655731132 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f9fdc0d0e90 con 0x7f9fec101920 2024-04-24T06:42:18.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.876+0000 7f9fe3e00700 1 --2- 172.21.15.5:0/655731132 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f9fd00a3cd0 0x7f9fd00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:18.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.876+0000 7f9fe3400700 1 --2- 172.21.15.5:0/655731132 >> [v2:172.21.15.137:6816/29724945,v1:172.21.15.137:6817/29724945] conn(0x7f9fd00ad4b0 0x7f9fd00af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:18.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.876+0000 7f9fe3400700 1 -- 172.21.15.5:0/655731132 --> [v2:172.21.15.137:6816/29724945,v1:172.21.15.137:6817/29724945] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9fd00aff80 con 0x7f9fd00ad4b0 2024-04-24T06:42:18.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.877+0000 7f9fe3400700 1 -- 172.21.15.5:0/655731132 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f9fdc0d1240 con 0x7f9fec101920 2024-04-24T06:42:18.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.877+0000 7f9feb400700 1 --2- 172.21.15.5:0/655731132 >> [v2:172.21.15.137:6816/29724945,v1:172.21.15.137:6817/29724945] conn(0x7f9fd00ad4b0 0x7f9fd00af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:18.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.877+0000 7f9fe3e00700 1 --2- 172.21.15.5:0/655731132 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f9fd00a3cd0 0x7f9fd00a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9fec106f40 tx=0x7f9fd4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:18.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.877+0000 7f9feb400700 1 --2- 172.21.15.5:0/655731132 >> [v2:172.21.15.137:6816/29724945,v1:172.21.15.137:6817/29724945] conn(0x7f9fd00ad4b0 0x7f9fd00af8d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:18.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.878+0000 7f9fe3400700 1 -- 172.21.15.5:0/655731132 <== osd.8 v2:172.21.15.137:6816/29724945 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9fd00aff80 con 0x7f9fd00ad4b0 2024-04-24T06:42:18.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.896+0000 7f9febe00700 1 -- 172.21.15.5:0/655731132 --> [v2:172.21.15.137:6816/29724945,v1:172.21.15.137:6817/29724945] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9fec063600 con 0x7f9fd00ad4b0 2024-04-24T06:42:18.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.897+0000 7f9fe3400700 1 -- 172.21.15.5:0/655731132 <== osd.8 v2:172.21.15.137:6816/29724945 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9fec063600 con 0x7f9fd00ad4b0 2024-04-24T06:42:18.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.898+0000 7f9febe00700 1 -- 172.21.15.5:0/655731132 >> [v2:172.21.15.137:6816/29724945,v1:172.21.15.137:6817/29724945] conn(0x7f9fd00ad4b0 msgr2=0x7f9fd00af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:18.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.898+0000 7f9febe00700 1 --2- 172.21.15.5:0/655731132 >> [v2:172.21.15.137:6816/29724945,v1:172.21.15.137:6817/29724945] conn(0x7f9fd00ad4b0 0x7f9fd00af8d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:18.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.898+0000 7f9febe00700 1 -- 172.21.15.5:0/655731132 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f9fd00a3cd0 msgr2=0x7f9fd00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:18.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.898+0000 7f9febe00700 1 --2- 172.21.15.5:0/655731132 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f9fd00a3cd0 0x7f9fd00a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9fec106f40 tx=0x7f9fd4009260 comp rx=0 tx=0).stop 2024-04-24T06:42:18.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.898+0000 7f9febe00700 1 -- 172.21.15.5:0/655731132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fec101920 msgr2=0x7f9fec1991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:18.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.898+0000 7f9febe00700 1 --2- 172.21.15.5:0/655731132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fec101920 0x7f9fec1991e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00ef10 tx=0x7f9fdc00c3a0 comp rx=0 tx=0).stop 2024-04-24T06:42:18.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.898+0000 7f9feb400700 1 -- 172.21.15.5:0/655731132 reap_dead start 2024-04-24T06:42:18.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.898+0000 7f9febe00700 1 -- 172.21.15.5:0/655731132 shutdown_connections 2024-04-24T06:42:18.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.898+0000 7f9febe00700 1 -- 172.21.15.5:0/655731132 >> 172.21.15.5:0/655731132 conn(0x7f9fec0fd410 msgr2=0x7f9fec06aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:18.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.898+0000 7f9febe00700 1 -- 172.21.15.5:0/655731132 shutdown_connections 2024-04-24T06:42:18.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.898+0000 7f9febe00700 1 -- 172.21.15.5:0/655731132 wait complete. 2024-04-24T06:42:18.745 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:18 smithi005 ceph-mon[27798]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:42:18.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:18 smithi037 ceph-mon[32864]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:42:18.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:18 smithi137 ceph-mon[33017]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:42:19.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.460+0000 7f9f8f600700 1 -- 172.21.15.5:0/2412236741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f88075640 msgr2=0x7f9f8810efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:19.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.460+0000 7f9f8f600700 1 --2- 172.21.15.5:0/2412236741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f88075640 0x7f9f8810efb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9f740098f0 tx=0x7f9f74009c00 comp rx=0 tx=0).stop 2024-04-24T06:42:19.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.460+0000 7f9f8f600700 1 -- 172.21.15.5:0/2412236741 shutdown_connections 2024-04-24T06:42:19.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.460+0000 7f9f8f600700 1 --2- 172.21.15.5:0/2412236741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f8810f4f0 0x7f9f881118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:19.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.460+0000 7f9f8f600700 1 --2- 172.21.15.5:0/2412236741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f88074d20 0x7f9f88075100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:19.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.460+0000 7f9f8f600700 1 --2- 172.21.15.5:0/2412236741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f88075640 0x7f9f8810efb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:19.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.460+0000 7f9f8f600700 1 -- 172.21.15.5:0/2412236741 >> 172.21.15.5:0/2412236741 conn(0x7f9f88079c50 msgr2=0x7f9f8807a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:19.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.460+0000 7f9f8f600700 1 -- 172.21.15.5:0/2412236741 shutdown_connections 2024-04-24T06:42:19.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.460+0000 7f9f8f600700 1 -- 172.21.15.5:0/2412236741 wait complete. 2024-04-24T06:42:19.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.462+0000 7f9f8f600700 1 Processor -- start 2024-04-24T06:42:19.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.462+0000 7f9f8f600700 1 -- start start 2024-04-24T06:42:19.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.462+0000 7f9f8f600700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f88074d20 0x7f9f8819b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:19.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.462+0000 7f9f8f600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f88075640 0x7f9f8819ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:19.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.462+0000 7f9f8f600700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f8810f4f0 0x7f9f8819fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:19.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.462+0000 7f9f8f600700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9f88115070 con 0x7f9f8810f4f0 2024-04-24T06:42:19.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.462+0000 7f9f8f600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9f88114ef0 con 0x7f9f88075640 2024-04-24T06:42:19.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.462+0000 7f9f8f600700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9f881151f0 con 0x7f9f88074d20 2024-04-24T06:42:19.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.463+0000 7f9f87e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f88075640 0x7f9f8819ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:19.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.463+0000 7f9f8d600700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f8810f4f0 0x7f9f8819fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:19.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.463+0000 7f9f87e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f88075640 0x7f9f8819ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:45536/0 (socket says 172.21.15.5:45536) 2024-04-24T06:42:19.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.463+0000 7f9f87e00700 1 -- 172.21.15.5:0/4280275348 learned_addr learned my addr 172.21.15.5:0/4280275348 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:42:19.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.463+0000 7f9f8cc00700 1 --2- 172.21.15.5:0/4280275348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f88074d20 0x7f9f8819b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:19.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.463+0000 7f9f87e00700 1 -- 172.21.15.5:0/4280275348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f88074d20 msgr2=0x7f9f8819b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:19.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.463+0000 7f9f87e00700 1 --2- 172.21.15.5:0/4280275348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f88074d20 0x7f9f8819b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:19.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.463+0000 7f9f87e00700 1 -- 172.21.15.5:0/4280275348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f8810f4f0 msgr2=0x7f9f8819fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:19.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.463+0000 7f9f87e00700 1 --2- 172.21.15.5:0/4280275348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f8810f4f0 0x7f9f8819fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:19.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.463+0000 7f9f87e00700 1 -- 172.21.15.5:0/4280275348 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f881a04c0 con 0x7f9f88075640 2024-04-24T06:42:19.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.463+0000 7f9f8d600700 1 --2- 172.21.15.5:0/4280275348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f8810f4f0 0x7f9f8819fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:42:19.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.464+0000 7f9f87e00700 1 --2- 172.21.15.5:0/4280275348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f88075640 0x7f9f8819ba40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9f740094f0 tx=0x7f9f7400fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:19.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.464+0000 7f9f85600700 1 -- 172.21.15.5:0/4280275348 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f74027070 con 0x7f9f88075640 2024-04-24T06:42:19.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.464+0000 7f9f85600700 1 -- 172.21.15.5:0/4280275348 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f740055a0 con 0x7f9f88075640 2024-04-24T06:42:19.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.464+0000 7f9f85600700 1 -- 172.21.15.5:0/4280275348 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f74004930 con 0x7f9f88075640 2024-04-24T06:42:19.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.464+0000 7f9f8f600700 1 -- 172.21.15.5:0/4280275348 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f881a0750 con 0x7f9f88075640 2024-04-24T06:42:19.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.464+0000 7f9f8f600700 1 -- 172.21.15.5:0/4280275348 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f881a8030 con 0x7f9f88075640 2024-04-24T06:42:19.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.465+0000 7f9f8f600700 1 -- 172.21.15.5:0/4280275348 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9f88100760 con 0x7f9f88075640 2024-04-24T06:42:19.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.471+0000 7f9f85600700 1 -- 172.21.15.5:0/4280275348 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f9f7402a070 con 0x7f9f88075640 2024-04-24T06:42:19.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.472+0000 7f9f85600700 1 --2- 172.21.15.5:0/4280275348 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f9f700a3d50 0x7f9f700a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:19.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.472+0000 7f9f85600700 1 -- 172.21.15.5:0/4280275348 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f9f740d14a0 con 0x7f9f88075640 2024-04-24T06:42:19.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.472+0000 7f9f8cc00700 1 --2- 172.21.15.5:0/4280275348 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f9f700a3d50 0x7f9f700a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:19.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.472+0000 7f9f85600700 1 --2- 172.21.15.5:0/4280275348 >> [v2:172.21.15.37:6808/759253844,v1:172.21.15.37:6809/759253844] conn(0x7f9f700ad530 0x7f9f700af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:19.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.472+0000 7f9f85600700 1 -- 172.21.15.5:0/4280275348 --> [v2:172.21.15.37:6808/759253844,v1:172.21.15.37:6809/759253844] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9f700b0000 con 0x7f9f700ad530 2024-04-24T06:42:19.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.472+0000 7f9f85600700 1 -- 172.21.15.5:0/4280275348 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f9f740d1850 con 0x7f9f88075640 2024-04-24T06:42:19.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.473+0000 7f9f8d600700 1 --2- 172.21.15.5:0/4280275348 >> [v2:172.21.15.37:6808/759253844,v1:172.21.15.37:6809/759253844] conn(0x7f9f700ad530 0x7f9f700af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:19.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.473+0000 7f9f8cc00700 1 --2- 172.21.15.5:0/4280275348 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f9f700a3d50 0x7f9f700a6210 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c009750 tx=0x7f9f7c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:19.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.474+0000 7f9f8d600700 1 --2- 172.21.15.5:0/4280275348 >> [v2:172.21.15.37:6808/759253844,v1:172.21.15.37:6809/759253844] conn(0x7f9f700ad530 0x7f9f700af950 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:19.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.475+0000 7f9f85600700 1 -- 172.21.15.5:0/4280275348 <== osd.4 v2:172.21.15.37:6808/759253844 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9f700b0000 con 0x7f9f700ad530 2024-04-24T06:42:19.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.495+0000 7f9f8f600700 1 -- 172.21.15.5:0/4280275348 --> [v2:172.21.15.37:6808/759253844,v1:172.21.15.37:6809/759253844] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9f88076410 con 0x7f9f700ad530 2024-04-24T06:42:19.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.496+0000 7f9f85600700 1 -- 172.21.15.5:0/4280275348 <== osd.4 v2:172.21.15.37:6808/759253844 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9f88076410 con 0x7f9f700ad530 2024-04-24T06:42:19.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.496+0000 7f9f8f600700 1 -- 172.21.15.5:0/4280275348 >> [v2:172.21.15.37:6808/759253844,v1:172.21.15.37:6809/759253844] conn(0x7f9f700ad530 msgr2=0x7f9f700af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:19.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.496+0000 7f9f8f600700 1 --2- 172.21.15.5:0/4280275348 >> [v2:172.21.15.37:6808/759253844,v1:172.21.15.37:6809/759253844] conn(0x7f9f700ad530 0x7f9f700af950 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:19.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.496+0000 7f9f8f600700 1 -- 172.21.15.5:0/4280275348 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f9f700a3d50 msgr2=0x7f9f700a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:19.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.496+0000 7f9f8f600700 1 --2- 172.21.15.5:0/4280275348 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f9f700a3d50 0x7f9f700a6210 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c009750 tx=0x7f9f7c009310 comp rx=0 tx=0).stop 2024-04-24T06:42:19.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.496+0000 7f9f8f600700 1 -- 172.21.15.5:0/4280275348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f88075640 msgr2=0x7f9f8819ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:19.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.496+0000 7f9f8f600700 1 --2- 172.21.15.5:0/4280275348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f88075640 0x7f9f8819ba40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9f740094f0 tx=0x7f9f7400fa00 comp rx=0 tx=0).stop 2024-04-24T06:42:19.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.497+0000 7f9f8d600700 1 -- 172.21.15.5:0/4280275348 reap_dead start 2024-04-24T06:42:19.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.497+0000 7f9f8f600700 1 -- 172.21.15.5:0/4280275348 shutdown_connections 2024-04-24T06:42:19.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.497+0000 7f9f8f600700 1 -- 172.21.15.5:0/4280275348 >> 172.21.15.5:0/4280275348 conn(0x7f9f88079c50 msgr2=0x7f9f880ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:19.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.497+0000 7f9f8f600700 1 -- 172.21.15.5:0/4280275348 shutdown_connections 2024-04-24T06:42:19.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:16.497+0000 7f9f8f600700 1 -- 172.21.15.5:0/4280275348 wait complete. 2024-04-24T06:42:20.399 INFO:teuthology.orchestra.run.smithi005.stdout:141733920829 2024-04-24T06:42:20.399 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-24T06:42:20.627 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:20 smithi005 ceph-mon[27798]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T06:42:20.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:20 smithi037 ceph-mon[32864]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T06:42:20.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:20 smithi137 ceph-mon[33017]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T06:42:21.784 INFO:teuthology.orchestra.run.smithi005.stdout:107374182465 2024-04-24T06:42:21.784 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-24T06:42:21.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.091+0000 7f08cd200700 1 -- 172.21.15.5:0/2749306164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08c8107ab0 msgr2=0x7f08c8107e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:21.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.091+0000 7f08cd200700 1 --2- 172.21.15.5:0/2749306164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08c8107ab0 0x7f08c8107e90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009990 tx=0x7f08b8009ca0 comp rx=0 tx=0).stop 2024-04-24T06:42:21.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.091+0000 7f08cd200700 1 -- 172.21.15.5:0/2749306164 shutdown_connections 2024-04-24T06:42:21.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.091+0000 7f08cd200700 1 --2- 172.21.15.5:0/2749306164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08c8103540 0x7f08c810fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:21.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.091+0000 7f08cd200700 1 --2- 172.21.15.5:0/2749306164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08c8107ab0 0x7f08c8107e90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:21.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.091+0000 7f08cd200700 1 --2- 172.21.15.5:0/2749306164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08c8102ba0 0x7f08c8103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:21.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.091+0000 7f08cd200700 1 -- 172.21.15.5:0/2749306164 >> 172.21.15.5:0/2749306164 conn(0x7f08c8079c50 msgr2=0x7f08c807a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:21.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.091+0000 7f08cd200700 1 -- 172.21.15.5:0/2749306164 shutdown_connections 2024-04-24T06:42:21.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.091+0000 7f08cd200700 1 -- 172.21.15.5:0/2749306164 wait complete. 2024-04-24T06:42:21.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.092+0000 7f08cd200700 1 Processor -- start 2024-04-24T06:42:21.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.092+0000 7f08cd200700 1 -- start start 2024-04-24T06:42:21.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.093+0000 7f08cd200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08c8102ba0 0x7f08c819b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:21.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.093+0000 7f08cd200700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08c8103540 0x7f08c819ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:21.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.093+0000 7f08cd200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08c8107ab0 0x7f08c819fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:21.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.093+0000 7f08cd200700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f08c8112e70 con 0x7f08c8103540 2024-04-24T06:42:21.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.093+0000 7f08cd200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f08c8112cf0 con 0x7f08c8107ab0 2024-04-24T06:42:21.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.093+0000 7f08cd200700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f08c8112ff0 con 0x7f08c8102ba0 2024-04-24T06:42:21.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.093+0000 7f08c5e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08c8102ba0 0x7f08c819b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:21.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.093+0000 7f08c5400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08c8103540 0x7f08c819ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:21.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.093+0000 7f08c5e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08c8102ba0 0x7f08c819b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:54840/0 (socket says 172.21.15.5:54840) 2024-04-24T06:42:21.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.093+0000 7f08c5e00700 1 -- 172.21.15.5:0/2005940075 learned_addr learned my addr 172.21.15.5:0/2005940075 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:42:21.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.093+0000 7f08c6800700 1 --2- 172.21.15.5:0/2005940075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08c8107ab0 0x7f08c819fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:21.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.094+0000 7f08c5e00700 1 -- 172.21.15.5:0/2005940075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08c8107ab0 msgr2=0x7f08c819fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:21.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.094+0000 7f08c5e00700 1 --2- 172.21.15.5:0/2005940075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08c8107ab0 0x7f08c819fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:21.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.094+0000 7f08c5e00700 1 -- 172.21.15.5:0/2005940075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08c8103540 msgr2=0x7f08c819ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:21.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.094+0000 7f08c5e00700 1 --2- 172.21.15.5:0/2005940075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08c8103540 0x7f08c819ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:21.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.094+0000 7f08c5e00700 1 -- 172.21.15.5:0/2005940075 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c81a0500 con 0x7f08c8102ba0 2024-04-24T06:42:21.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.094+0000 7f08c5e00700 1 --2- 172.21.15.5:0/2005940075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08c8102ba0 0x7f08c819b4f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f08b8000c00 tx=0x7f08b800ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:21.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.094+0000 7f08bea00700 1 -- 172.21.15.5:0/2005940075 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b8027070 con 0x7f08c8102ba0 2024-04-24T06:42:21.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.095+0000 7f08bea00700 1 -- 172.21.15.5:0/2005940075 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08b8003d70 con 0x7f08c8102ba0 2024-04-24T06:42:21.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.095+0000 7f08bea00700 1 -- 172.21.15.5:0/2005940075 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b800f460 con 0x7f08c8102ba0 2024-04-24T06:42:21.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.095+0000 7f08cd200700 1 -- 172.21.15.5:0/2005940075 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08c81a0790 con 0x7f08c8102ba0 2024-04-24T06:42:21.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.095+0000 7f08cd200700 1 -- 172.21.15.5:0/2005940075 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08c81a8070 con 0x7f08c8102ba0 2024-04-24T06:42:21.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.096+0000 7f08cd200700 1 -- 172.21.15.5:0/2005940075 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f08c81a0c20 con 0x7f08c8102ba0 2024-04-24T06:42:21.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.098+0000 7f08bea00700 1 -- 172.21.15.5:0/2005940075 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f08b802a070 con 0x7f08c8102ba0 2024-04-24T06:42:21.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.099+0000 7f08bea00700 1 --2- 172.21.15.5:0/2005940075 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f08ac0a3cd0 0x7f08ac0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:21.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.100+0000 7f08bea00700 1 -- 172.21.15.5:0/2005940075 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f08b80d1990 con 0x7f08c8102ba0 2024-04-24T06:42:21.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.100+0000 7f08c5400700 1 --2- 172.21.15.5:0/2005940075 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f08ac0a3cd0 0x7f08ac0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:21.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.100+0000 7f08bea00700 1 --2- 172.21.15.5:0/2005940075 >> [v2:172.21.15.37:6816/2952750659,v1:172.21.15.37:6817/2952750659] conn(0x7f08ac0ad4b0 0x7f08ac0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:21.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.100+0000 7f08bea00700 1 -- 172.21.15.5:0/2005940075 --> [v2:172.21.15.37:6816/2952750659,v1:172.21.15.37:6817/2952750659] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f08b8009030 con 0x7f08ac0ad4b0 2024-04-24T06:42:21.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.100+0000 7f08bea00700 1 -- 172.21.15.5:0/2005940075 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f08b80d1d40 con 0x7f08c8102ba0 2024-04-24T06:42:21.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.100+0000 7f08c6800700 1 --2- 172.21.15.5:0/2005940075 >> [v2:172.21.15.37:6816/2952750659,v1:172.21.15.37:6817/2952750659] conn(0x7f08ac0ad4b0 0x7f08ac0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:21.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.101+0000 7f08c6800700 1 --2- 172.21.15.5:0/2005940075 >> [v2:172.21.15.37:6816/2952750659,v1:172.21.15.37:6817/2952750659] conn(0x7f08ac0ad4b0 0x7f08ac0af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:21.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.101+0000 7f08c5400700 1 --2- 172.21.15.5:0/2005940075 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f08ac0a3cd0 0x7f08ac0a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f08c8109320 tx=0x7f08b400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:21.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.102+0000 7f08bea00700 1 -- 172.21.15.5:0/2005940075 <== osd.7 v2:172.21.15.37:6816/2952750659 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f08b8009030 con 0x7f08ac0ad4b0 2024-04-24T06:42:21.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.120+0000 7f08cd200700 1 -- 172.21.15.5:0/2005940075 --> [v2:172.21.15.37:6816/2952750659,v1:172.21.15.37:6817/2952750659] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f08c8101700 con 0x7f08ac0ad4b0 2024-04-24T06:42:21.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.121+0000 7f08bea00700 1 -- 172.21.15.5:0/2005940075 <== osd.7 v2:172.21.15.37:6816/2952750659 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f08c8101700 con 0x7f08ac0ad4b0 2024-04-24T06:42:21.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.121+0000 7f08cd200700 1 -- 172.21.15.5:0/2005940075 >> [v2:172.21.15.37:6816/2952750659,v1:172.21.15.37:6817/2952750659] conn(0x7f08ac0ad4b0 msgr2=0x7f08ac0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:21.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.121+0000 7f08cd200700 1 --2- 172.21.15.5:0/2005940075 >> [v2:172.21.15.37:6816/2952750659,v1:172.21.15.37:6817/2952750659] conn(0x7f08ac0ad4b0 0x7f08ac0af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:21.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.121+0000 7f08cd200700 1 -- 172.21.15.5:0/2005940075 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f08ac0a3cd0 msgr2=0x7f08ac0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:21.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.121+0000 7f08cd200700 1 --2- 172.21.15.5:0/2005940075 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f08ac0a3cd0 0x7f08ac0a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f08c8109320 tx=0x7f08b400b410 comp rx=0 tx=0).stop 2024-04-24T06:42:21.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.121+0000 7f08cd200700 1 -- 172.21.15.5:0/2005940075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08c8102ba0 msgr2=0x7f08c819b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:21.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.121+0000 7f08cd200700 1 --2- 172.21.15.5:0/2005940075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08c8102ba0 0x7f08c819b4f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f08b8000c00 tx=0x7f08b800ba90 comp rx=0 tx=0).stop 2024-04-24T06:42:21.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.121+0000 7f08c6800700 1 -- 172.21.15.5:0/2005940075 reap_dead start 2024-04-24T06:42:21.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.121+0000 7f08cd200700 1 -- 172.21.15.5:0/2005940075 shutdown_connections 2024-04-24T06:42:21.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.121+0000 7f08cd200700 1 -- 172.21.15.5:0/2005940075 >> 172.21.15.5:0/2005940075 conn(0x7f08c8079c50 msgr2=0x7f08c8100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:21.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.122+0000 7f08cd200700 1 -- 172.21.15.5:0/2005940075 shutdown_connections 2024-04-24T06:42:21.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.122+0000 7f08cd200700 1 -- 172.21.15.5:0/2005940075 wait complete. 2024-04-24T06:42:22.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.577+0000 7f39e3200700 1 -- 172.21.15.5:0/112088317 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39e40757e0 msgr2=0x7f39e410ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.577+0000 7f39e3200700 1 --2- 172.21.15.5:0/112088317 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39e40757e0 0x7f39e410ed70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f39cc0098f0 tx=0x7f39cc009c00 comp rx=0 tx=0).stop 2024-04-24T06:42:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.578+0000 7f39e3200700 1 -- 172.21.15.5:0/112088317 shutdown_connections 2024-04-24T06:42:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.578+0000 7f39e3200700 1 --2- 172.21.15.5:0/112088317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39e410f2b0 0x7f39e41116c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.578+0000 7f39e3200700 1 --2- 172.21.15.5:0/112088317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39e4074ec0 0x7f39e40752a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.578+0000 7f39e3200700 1 --2- 172.21.15.5:0/112088317 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39e40757e0 0x7f39e410ed70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.578+0000 7f39e3200700 1 -- 172.21.15.5:0/112088317 >> 172.21.15.5:0/112088317 conn(0x7f39e40fe500 msgr2=0x7f39e4100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.578+0000 7f39e3200700 1 -- 172.21.15.5:0/112088317 shutdown_connections 2024-04-24T06:42:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.578+0000 7f39e3200700 1 -- 172.21.15.5:0/112088317 wait complete. 2024-04-24T06:42:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.579+0000 7f39e3200700 1 Processor -- start 2024-04-24T06:42:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.579+0000 7f39e3200700 1 -- start start 2024-04-24T06:42:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.579+0000 7f39e3200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39e4074ec0 0x7f39e419b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.579+0000 7f39e3200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39e40757e0 0x7f39e419ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.579+0000 7f39e3200700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39e410f2b0 0x7f39e419fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.579+0000 7f39e3200700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39e4114f50 con 0x7f39e410f2b0 2024-04-24T06:42:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.579+0000 7f39e3200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f39e4114dd0 con 0x7f39e4074ec0 2024-04-24T06:42:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.579+0000 7f39e3200700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f39e41150d0 con 0x7f39e40757e0 2024-04-24T06:42:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.580+0000 7f39e2800700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39e410f2b0 0x7f39e419fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.580+0000 7f39dbe00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39e40757e0 0x7f39e419ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.580+0000 7f39e2800700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39e410f2b0 0x7f39e419fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52644/0 (socket says 172.21.15.5:52644) 2024-04-24T06:42:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.580+0000 7f39dbe00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39e40757e0 0x7f39e419ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:54866/0 (socket says 172.21.15.5:54866) 2024-04-24T06:42:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.580+0000 7f39e2800700 1 -- 172.21.15.5:0/2845481975 learned_addr learned my addr 172.21.15.5:0/2845481975 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:42:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.580+0000 7f39e1e00700 1 --2- 172.21.15.5:0/2845481975 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39e4074ec0 0x7f39e419b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.580+0000 7f39dbe00700 1 -- 172.21.15.5:0/2845481975 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39e4074ec0 msgr2=0x7f39e419b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.580+0000 7f39dbe00700 1 --2- 172.21.15.5:0/2845481975 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39e4074ec0 0x7f39e419b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.580+0000 7f39dbe00700 1 -- 172.21.15.5:0/2845481975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39e410f2b0 msgr2=0x7f39e419fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.580+0000 7f39dbe00700 1 --2- 172.21.15.5:0/2845481975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39e410f2b0 0x7f39e419fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.580+0000 7f39dbe00700 1 -- 172.21.15.5:0/2845481975 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e41a04f0 con 0x7f39e40757e0 2024-04-24T06:42:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.580+0000 7f39dbe00700 1 --2- 172.21.15.5:0/2845481975 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39e40757e0 0x7f39e419ba20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f39cc00c010 tx=0x7f39cc00fa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:22.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.581+0000 7f39daa00700 1 -- 172.21.15.5:0/2845481975 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc005180 con 0x7f39e40757e0 2024-04-24T06:42:22.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.581+0000 7f39daa00700 1 -- 172.21.15.5:0/2845481975 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39cc00b560 con 0x7f39e40757e0 2024-04-24T06:42:22.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.581+0000 7f39e3200700 1 -- 172.21.15.5:0/2845481975 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39e41a0780 con 0x7f39e40757e0 2024-04-24T06:42:22.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.581+0000 7f39daa00700 1 -- 172.21.15.5:0/2845481975 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc003a80 con 0x7f39e40757e0 2024-04-24T06:42:22.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.581+0000 7f39e3200700 1 -- 172.21.15.5:0/2845481975 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39e41a8060 con 0x7f39e40757e0 2024-04-24T06:42:22.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.582+0000 7f39e3200700 1 -- 172.21.15.5:0/2845481975 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f39e4063200 con 0x7f39e40757e0 2024-04-24T06:42:22.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.584+0000 7f39daa00700 1 -- 172.21.15.5:0/2845481975 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f39cc00b710 con 0x7f39e40757e0 2024-04-24T06:42:22.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.585+0000 7f39daa00700 1 --2- 172.21.15.5:0/2845481975 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f39c80a3cd0 0x7f39c80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:22.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.585+0000 7f39daa00700 1 -- 172.21.15.5:0/2845481975 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f39cc0d1820 con 0x7f39e40757e0 2024-04-24T06:42:22.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.586+0000 7f39e1e00700 1 --2- 172.21.15.5:0/2845481975 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f39c80a3cd0 0x7f39c80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:22.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.586+0000 7f39daa00700 1 --2- 172.21.15.5:0/2845481975 >> [v2:172.21.15.137:6824/90600596,v1:172.21.15.137:6825/90600596] conn(0x7f39c80ad4b0 0x7f39c80af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:22.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.586+0000 7f39daa00700 1 -- 172.21.15.5:0/2845481975 --> [v2:172.21.15.137:6824/90600596,v1:172.21.15.137:6825/90600596] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f39cc00bde0 con 0x7f39c80ad4b0 2024-04-24T06:42:22.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.586+0000 7f39daa00700 1 -- 172.21.15.5:0/2845481975 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f39cc0d1bd0 con 0x7f39e40757e0 2024-04-24T06:42:22.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.586+0000 7f39e2800700 1 --2- 172.21.15.5:0/2845481975 >> [v2:172.21.15.137:6824/90600596,v1:172.21.15.137:6825/90600596] conn(0x7f39c80ad4b0 0x7f39c80af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:22.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.586+0000 7f39e1e00700 1 --2- 172.21.15.5:0/2845481975 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f39c80a3cd0 0x7f39c80a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f39d4009750 tx=0x7f39d4009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:22.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.587+0000 7f39e2800700 1 --2- 172.21.15.5:0/2845481975 >> [v2:172.21.15.137:6824/90600596,v1:172.21.15.137:6825/90600596] conn(0x7f39c80ad4b0 0x7f39c80af8d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:22.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.588+0000 7f39daa00700 1 -- 172.21.15.5:0/2845481975 <== osd.11 v2:172.21.15.137:6824/90600596 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f39cc00bde0 con 0x7f39c80ad4b0 2024-04-24T06:42:22.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.607+0000 7f39e3200700 1 -- 172.21.15.5:0/2845481975 --> [v2:172.21.15.137:6824/90600596,v1:172.21.15.137:6825/90600596] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f39e4063600 con 0x7f39c80ad4b0 2024-04-24T06:42:22.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.608+0000 7f39daa00700 1 -- 172.21.15.5:0/2845481975 <== osd.11 v2:172.21.15.137:6824/90600596 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f39e4063600 con 0x7f39c80ad4b0 2024-04-24T06:42:22.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.609+0000 7f39e3200700 1 -- 172.21.15.5:0/2845481975 >> [v2:172.21.15.137:6824/90600596,v1:172.21.15.137:6825/90600596] conn(0x7f39c80ad4b0 msgr2=0x7f39c80af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:22.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.609+0000 7f39e3200700 1 --2- 172.21.15.5:0/2845481975 >> [v2:172.21.15.137:6824/90600596,v1:172.21.15.137:6825/90600596] conn(0x7f39c80ad4b0 0x7f39c80af8d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:22.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.610+0000 7f39e3200700 1 -- 172.21.15.5:0/2845481975 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f39c80a3cd0 msgr2=0x7f39c80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:22.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.610+0000 7f39e3200700 1 --2- 172.21.15.5:0/2845481975 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f39c80a3cd0 0x7f39c80a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f39d4009750 tx=0x7f39d4009310 comp rx=0 tx=0).stop 2024-04-24T06:42:22.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.610+0000 7f39e3200700 1 -- 172.21.15.5:0/2845481975 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39e40757e0 msgr2=0x7f39e419ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:22.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.610+0000 7f39e3200700 1 --2- 172.21.15.5:0/2845481975 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39e40757e0 0x7f39e419ba20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f39cc00c010 tx=0x7f39cc00fa70 comp rx=0 tx=0).stop 2024-04-24T06:42:22.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.610+0000 7f39e2800700 1 -- 172.21.15.5:0/2845481975 reap_dead start 2024-04-24T06:42:22.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.610+0000 7f39e3200700 1 -- 172.21.15.5:0/2845481975 shutdown_connections 2024-04-24T06:42:22.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.610+0000 7f39e3200700 1 -- 172.21.15.5:0/2845481975 >> 172.21.15.5:0/2845481975 conn(0x7f39e40fe500 msgr2=0x7f39e40ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:22.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.610+0000 7f39e3200700 1 -- 172.21.15.5:0/2845481975 shutdown_connections 2024-04-24T06:42:22.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:21.610+0000 7f39e3200700 1 -- 172.21.15.5:0/2845481975 wait complete. 2024-04-24T06:42:22.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:18.996+0000 7f9b9b800700 1 -- 172.21.15.5:0/2060733958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b94108b60 msgr2=0x7f9b94108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:22.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:18.996+0000 7f9b9b800700 1 --2- 172.21.15.5:0/2060733958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b94108b60 0x7f9b94108f40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9b880098f0 tx=0x7f9b88009c00 comp rx=0 tx=0).stop 2024-04-24T06:42:22.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:18.997+0000 7f9b9b800700 1 -- 172.21.15.5:0/2060733958 shutdown_connections 2024-04-24T06:42:22.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:18.997+0000 7f9b9b800700 1 --2- 172.21.15.5:0/2060733958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b9406ba30 0x7f9b9410f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:22.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:18.997+0000 7f9b9b800700 1 --2- 172.21.15.5:0/2060733958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b94108b60 0x7f9b94108f40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:22.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:18.997+0000 7f9b9b800700 1 --2- 172.21.15.5:0/2060733958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b9406b090 0x7f9b9406b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:22.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:18.997+0000 7f9b9b800700 1 -- 172.21.15.5:0/2060733958 >> 172.21.15.5:0/2060733958 conn(0x7f9b94078960 msgr2=0x7f9b94078d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:22.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:18.997+0000 7f9b9b800700 1 -- 172.21.15.5:0/2060733958 shutdown_connections 2024-04-24T06:42:22.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:18.997+0000 7f9b9b800700 1 -- 172.21.15.5:0/2060733958 wait complete. 2024-04-24T06:42:22.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:18.998+0000 7f9b9b800700 1 Processor -- start 2024-04-24T06:42:22.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:18.998+0000 7f9b9b800700 1 -- start start 2024-04-24T06:42:22.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:18.999+0000 7f9b9b800700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b9406b090 0x7f9b9419b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:22.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:18.999+0000 7f9b9b800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b9406ba30 0x7f9b9419bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:22.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:18.999+0000 7f9b9b800700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b94108b60 0x7f9b9419fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:22.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:18.999+0000 7f9b9b800700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9b94112dd0 con 0x7f9b94108b60 2024-04-24T06:42:22.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:18.999+0000 7f9b9b800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9b94112c50 con 0x7f9b9406ba30 2024-04-24T06:42:22.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:18.999+0000 7f9b9b800700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9b94112f50 con 0x7f9b9406b090 2024-04-24T06:42:22.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:18.999+0000 7f9b98e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b9406b090 0x7f9b9419b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:22.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:18.999+0000 7f9b99800700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b94108b60 0x7f9b9419fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:22.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:18.999+0000 7f9b98e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b9406b090 0x7f9b9419b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:54822/0 (socket says 172.21.15.5:54822) 2024-04-24T06:42:22.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:18.999+0000 7f9b98e00700 1 -- 172.21.15.5:0/3777862131 learned_addr learned my addr 172.21.15.5:0/3777862131 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:42:22.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:18.999+0000 7f9b93e00700 1 --2- 172.21.15.5:0/3777862131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b9406ba30 0x7f9b9419bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:22.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:18.999+0000 7f9b98e00700 1 -- 172.21.15.5:0/3777862131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b9406ba30 msgr2=0x7f9b9419bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:22.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:18.999+0000 7f9b98e00700 1 --2- 172.21.15.5:0/3777862131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b9406ba30 0x7f9b9419bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:22.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:18.999+0000 7f9b98e00700 1 -- 172.21.15.5:0/3777862131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b94108b60 msgr2=0x7f9b9419fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:22.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:18.999+0000 7f9b98e00700 1 --2- 172.21.15.5:0/3777862131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b94108b60 0x7f9b9419fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:22.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:18.999+0000 7f9b98e00700 1 -- 172.21.15.5:0/3777862131 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b941a0590 con 0x7f9b9406b090 2024-04-24T06:42:22.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.000+0000 7f9b98e00700 1 --2- 172.21.15.5:0/3777862131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b9406b090 0x7f9b9419b580 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9b88009230 tx=0x7f9b8800fa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:22.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.000+0000 7f9b91600700 1 -- 172.21.15.5:0/3777862131 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b88005180 con 0x7f9b9406b090 2024-04-24T06:42:22.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.000+0000 7f9b91600700 1 -- 172.21.15.5:0/3777862131 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b8800b560 con 0x7f9b9406b090 2024-04-24T06:42:22.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.000+0000 7f9b9b800700 1 -- 172.21.15.5:0/3777862131 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b941a0820 con 0x7f9b9406b090 2024-04-24T06:42:22.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.000+0000 7f9b91600700 1 -- 172.21.15.5:0/3777862131 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b88003a70 con 0x7f9b9406b090 2024-04-24T06:42:22.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.000+0000 7f9b9b800700 1 -- 172.21.15.5:0/3777862131 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b941a8100 con 0x7f9b9406b090 2024-04-24T06:42:22.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.001+0000 7f9b9b800700 1 -- 172.21.15.5:0/3777862131 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9b941001f0 con 0x7f9b9406b090 2024-04-24T06:42:22.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.004+0000 7f9b91600700 1 -- 172.21.15.5:0/3777862131 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f9b88029060 con 0x7f9b9406b090 2024-04-24T06:42:22.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.005+0000 7f9b91600700 1 --2- 172.21.15.5:0/3777862131 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f9b7c0a3d50 0x7f9b7c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:22.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.005+0000 7f9b91600700 1 -- 172.21.15.5:0/3777862131 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f9b880d1430 con 0x7f9b9406b090 2024-04-24T06:42:22.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.005+0000 7f9b93e00700 1 --2- 172.21.15.5:0/3777862131 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f9b7c0a3d50 0x7f9b7c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:22.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.006+0000 7f9b91600700 1 --2- 172.21.15.5:0/3777862131 >> [v2:172.21.15.137:6808/3771921560,v1:172.21.15.137:6809/3771921560] conn(0x7f9b7c0ad400 0x7f9b7c0af820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:22.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.006+0000 7f9b91600700 1 -- 172.21.15.5:0/3777862131 --> [v2:172.21.15.137:6808/3771921560,v1:172.21.15.137:6809/3771921560] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9b8800bde0 con 0x7f9b7c0ad400 2024-04-24T06:42:22.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.006+0000 7f9b91600700 1 -- 172.21.15.5:0/3777862131 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f9b880d17e0 con 0x7f9b9406b090 2024-04-24T06:42:22.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.006+0000 7f9b99800700 1 --2- 172.21.15.5:0/3777862131 >> [v2:172.21.15.137:6808/3771921560,v1:172.21.15.137:6809/3771921560] conn(0x7f9b7c0ad400 0x7f9b7c0af820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:22.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.006+0000 7f9b93e00700 1 --2- 172.21.15.5:0/3777862131 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f9b7c0a3d50 0x7f9b7c0a6210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9b9410fde0 tx=0x7f9b80009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:22.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.007+0000 7f9b99800700 1 --2- 172.21.15.5:0/3777862131 >> [v2:172.21.15.137:6808/3771921560,v1:172.21.15.137:6809/3771921560] conn(0x7f9b7c0ad400 0x7f9b7c0af820 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:22.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.008+0000 7f9b91600700 1 -- 172.21.15.5:0/3777862131 <== osd.5 v2:172.21.15.137:6808/3771921560 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9b8800bde0 con 0x7f9b7c0ad400 2024-04-24T06:42:22.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.034+0000 7f9b9b800700 1 -- 172.21.15.5:0/3777862131 --> [v2:172.21.15.137:6808/3771921560,v1:172.21.15.137:6809/3771921560] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9b941041d0 con 0x7f9b7c0ad400 2024-04-24T06:42:22.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.035+0000 7f9b91600700 1 -- 172.21.15.5:0/3777862131 <== osd.5 v2:172.21.15.137:6808/3771921560 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9b941041d0 con 0x7f9b7c0ad400 2024-04-24T06:42:22.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.036+0000 7f9b9b800700 1 -- 172.21.15.5:0/3777862131 >> [v2:172.21.15.137:6808/3771921560,v1:172.21.15.137:6809/3771921560] conn(0x7f9b7c0ad400 msgr2=0x7f9b7c0af820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:22.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.036+0000 7f9b9b800700 1 --2- 172.21.15.5:0/3777862131 >> [v2:172.21.15.137:6808/3771921560,v1:172.21.15.137:6809/3771921560] conn(0x7f9b7c0ad400 0x7f9b7c0af820 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:22.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.036+0000 7f9b9b800700 1 -- 172.21.15.5:0/3777862131 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f9b7c0a3d50 msgr2=0x7f9b7c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:22.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.036+0000 7f9b9b800700 1 --2- 172.21.15.5:0/3777862131 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f9b7c0a3d50 0x7f9b7c0a6210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9b9410fde0 tx=0x7f9b80009260 comp rx=0 tx=0).stop 2024-04-24T06:42:22.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.036+0000 7f9b9b800700 1 -- 172.21.15.5:0/3777862131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b9406b090 msgr2=0x7f9b9419b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:22.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.036+0000 7f9b9b800700 1 --2- 172.21.15.5:0/3777862131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b9406b090 0x7f9b9419b580 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9b88009230 tx=0x7f9b8800fa70 comp rx=0 tx=0).stop 2024-04-24T06:42:22.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.037+0000 7f9b99800700 1 -- 172.21.15.5:0/3777862131 reap_dead start 2024-04-24T06:42:22.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.037+0000 7f9b9b800700 1 -- 172.21.15.5:0/3777862131 shutdown_connections 2024-04-24T06:42:22.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.037+0000 7f9b9b800700 1 -- 172.21.15.5:0/3777862131 >> 172.21.15.5:0/3777862131 conn(0x7f9b94078960 msgr2=0x7f9b940ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:22.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.037+0000 7f9b9b800700 1 -- 172.21.15.5:0/3777862131 shutdown_connections 2024-04-24T06:42:22.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:19.037+0000 7f9b9b800700 1 -- 172.21.15.5:0/3777862131 wait complete. 2024-04-24T06:42:22.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:22 smithi005 ceph-mon[27798]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-04-24T06:42:22.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:22 smithi037 ceph-mon[32864]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-04-24T06:42:22.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:22 smithi137 ceph-mon[33017]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-04-24T06:42:23.824 INFO:teuthology.orchestra.run.smithi005.stdout:55834574924 2024-04-24T06:42:23.824 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-24T06:42:24.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.746+0000 7f8c6c200700 1 -- 172.21.15.5:0/1195795005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c64108b80 msgr2=0x7f8c64108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:24.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.746+0000 7f8c6c200700 1 --2- 172.21.15.5:0/1195795005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c64108b80 0x7f8c64108f60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8c58009990 tx=0x7f8c58009ca0 comp rx=0 tx=0).stop 2024-04-24T06:42:24.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.747+0000 7f8c6c200700 1 -- 172.21.15.5:0/1195795005 shutdown_connections 2024-04-24T06:42:24.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.747+0000 7f8c6c200700 1 --2- 172.21.15.5:0/1195795005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c64103530 0x7f8c6410f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:24.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.747+0000 7f8c6c200700 1 --2- 172.21.15.5:0/1195795005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c64108b80 0x7f8c64108f60 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:24.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.747+0000 7f8c6c200700 1 --2- 172.21.15.5:0/1195795005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c64102b70 0x7f8c64102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:24.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.747+0000 7f8c6c200700 1 -- 172.21.15.5:0/1195795005 >> 172.21.15.5:0/1195795005 conn(0x7f8c64079ca0 msgr2=0x7f8c6407a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:24.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.747+0000 7f8c6c200700 1 -- 172.21.15.5:0/1195795005 shutdown_connections 2024-04-24T06:42:24.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.747+0000 7f8c6c200700 1 -- 172.21.15.5:0/1195795005 wait complete. 2024-04-24T06:42:24.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.748+0000 7f8c6c200700 1 Processor -- start 2024-04-24T06:42:24.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.748+0000 7f8c6c200700 1 -- start start 2024-04-24T06:42:24.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.749+0000 7f8c6c200700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c64102b70 0x7f8c6419b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:24.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.749+0000 7f8c6c200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c64103530 0x7f8c6419b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:24.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.749+0000 7f8c6c200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c64108b80 0x7f8c6419fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:24.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.749+0000 7f8c6c200700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8c64112e60 con 0x7f8c64102b70 2024-04-24T06:42:24.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.749+0000 7f8c6c200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8c64112ce0 con 0x7f8c64103530 2024-04-24T06:42:24.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.749+0000 7f8c6c200700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8c64112fe0 con 0x7f8c64108b80 2024-04-24T06:42:24.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.749+0000 7f8c69800700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c64102b70 0x7f8c6419b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:24.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.749+0000 7f8c69800700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c64102b70 0x7f8c6419b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52670/0 (socket says 172.21.15.5:52670) 2024-04-24T06:42:24.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.749+0000 7f8c69800700 1 -- 172.21.15.5:0/3743062096 learned_addr learned my addr 172.21.15.5:0/3743062096 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:42:24.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.749+0000 7f8c68e00700 1 --2- 172.21.15.5:0/3743062096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c64103530 0x7f8c6419b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:24.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.749+0000 7f8c6a200700 1 --2- 172.21.15.5:0/3743062096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c64108b80 0x7f8c6419fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:24.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.750+0000 7f8c69800700 1 -- 172.21.15.5:0/3743062096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c64108b80 msgr2=0x7f8c6419fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:24.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.750+0000 7f8c69800700 1 --2- 172.21.15.5:0/3743062096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c64108b80 0x7f8c6419fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:24.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.750+0000 7f8c69800700 1 -- 172.21.15.5:0/3743062096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c64103530 msgr2=0x7f8c6419b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:24.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.750+0000 7f8c69800700 1 --2- 172.21.15.5:0/3743062096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c64103530 0x7f8c6419b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:24.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.750+0000 7f8c69800700 1 -- 172.21.15.5:0/3743062096 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c641a0400 con 0x7f8c64102b70 2024-04-24T06:42:24.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.750+0000 7f8c69800700 1 --2- 172.21.15.5:0/3743062096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c64102b70 0x7f8c6419b440 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8c58000c00 tx=0x7f8c5800ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:24.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.750+0000 7f8c62000700 1 -- 172.21.15.5:0/3743062096 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c58027070 con 0x7f8c64102b70 2024-04-24T06:42:24.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.750+0000 7f8c62000700 1 -- 172.21.15.5:0/3743062096 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c58003d70 con 0x7f8c64102b70 2024-04-24T06:42:24.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.750+0000 7f8c62000700 1 -- 172.21.15.5:0/3743062096 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5800f460 con 0x7f8c64102b70 2024-04-24T06:42:24.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.750+0000 7f8c6c200700 1 -- 172.21.15.5:0/3743062096 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c641a0690 con 0x7f8c64102b70 2024-04-24T06:42:24.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.751+0000 7f8c6c200700 1 -- 172.21.15.5:0/3743062096 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c641a7f70 con 0x7f8c64102b70 2024-04-24T06:42:24.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.752+0000 7f8c6c200700 1 -- 172.21.15.5:0/3743062096 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8c641a0b20 con 0x7f8c64102b70 2024-04-24T06:42:24.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.754+0000 7f8c62000700 1 -- 172.21.15.5:0/3743062096 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f8c5802a070 con 0x7f8c64102b70 2024-04-24T06:42:24.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.755+0000 7f8c62000700 1 --2- 172.21.15.5:0/3743062096 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f8c4c0a3d50 0x7f8c4c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:24.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.755+0000 7f8c62000700 1 -- 172.21.15.5:0/3743062096 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f8c580d1e50 con 0x7f8c64102b70 2024-04-24T06:42:24.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.755+0000 7f8c68e00700 1 --2- 172.21.15.5:0/3743062096 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f8c4c0a3d50 0x7f8c4c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:24.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.756+0000 7f8c62000700 1 --2- 172.21.15.5:0/3743062096 >> [v2:172.21.15.137:6800/2885027256,v1:172.21.15.137:6801/2885027256] conn(0x7f8c4c0ad530 0x7f8c4c0af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:24.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.756+0000 7f8c62000700 1 -- 172.21.15.5:0/3743062096 --> [v2:172.21.15.137:6800/2885027256,v1:172.21.15.137:6801/2885027256] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8c4c0b0000 con 0x7f8c4c0ad530 2024-04-24T06:42:24.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.756+0000 7f8c62000700 1 -- 172.21.15.5:0/3743062096 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f8c580d2200 con 0x7f8c64102b70 2024-04-24T06:42:24.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.756+0000 7f8c68e00700 1 --2- 172.21.15.5:0/3743062096 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f8c4c0a3d50 0x7f8c4c0a6210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8c6407a4c0 tx=0x7f8c50009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:24.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.756+0000 7f8c6a200700 1 --2- 172.21.15.5:0/3743062096 >> [v2:172.21.15.137:6800/2885027256,v1:172.21.15.137:6801/2885027256] conn(0x7f8c4c0ad530 0x7f8c4c0af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:24.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.757+0000 7f8c6a200700 1 --2- 172.21.15.5:0/3743062096 >> [v2:172.21.15.137:6800/2885027256,v1:172.21.15.137:6801/2885027256] conn(0x7f8c4c0ad530 0x7f8c4c0af950 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:24.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.758+0000 7f8c62000700 1 -- 172.21.15.5:0/3743062096 <== osd.2 v2:172.21.15.137:6800/2885027256 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8c4c0b0000 con 0x7f8c4c0ad530 2024-04-24T06:42:24.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.777+0000 7f8c6c200700 1 -- 172.21.15.5:0/3743062096 --> [v2:172.21.15.137:6800/2885027256,v1:172.21.15.137:6801/2885027256] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8c641041f0 con 0x7f8c4c0ad530 2024-04-24T06:42:24.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.778+0000 7f8c62000700 1 -- 172.21.15.5:0/3743062096 <== osd.2 v2:172.21.15.137:6800/2885027256 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8c641041f0 con 0x7f8c4c0ad530 2024-04-24T06:42:24.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.778+0000 7f8c6c200700 1 -- 172.21.15.5:0/3743062096 >> [v2:172.21.15.137:6800/2885027256,v1:172.21.15.137:6801/2885027256] conn(0x7f8c4c0ad530 msgr2=0x7f8c4c0af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:24.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.778+0000 7f8c6c200700 1 --2- 172.21.15.5:0/3743062096 >> [v2:172.21.15.137:6800/2885027256,v1:172.21.15.137:6801/2885027256] conn(0x7f8c4c0ad530 0x7f8c4c0af950 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:24.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.779+0000 7f8c6c200700 1 -- 172.21.15.5:0/3743062096 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f8c4c0a3d50 msgr2=0x7f8c4c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:24.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.779+0000 7f8c6c200700 1 --2- 172.21.15.5:0/3743062096 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f8c4c0a3d50 0x7f8c4c0a6210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8c6407a4c0 tx=0x7f8c50009260 comp rx=0 tx=0).stop 2024-04-24T06:42:24.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.779+0000 7f8c6c200700 1 -- 172.21.15.5:0/3743062096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c64102b70 msgr2=0x7f8c6419b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:24.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.779+0000 7f8c6c200700 1 --2- 172.21.15.5:0/3743062096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c64102b70 0x7f8c6419b440 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8c58000c00 tx=0x7f8c5800ba90 comp rx=0 tx=0).stop 2024-04-24T06:42:24.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.779+0000 7f8c6a200700 1 -- 172.21.15.5:0/3743062096 reap_dead start 2024-04-24T06:42:24.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.779+0000 7f8c6c200700 1 -- 172.21.15.5:0/3743062096 shutdown_connections 2024-04-24T06:42:24.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.779+0000 7f8c6c200700 1 -- 172.21.15.5:0/3743062096 >> 172.21.15.5:0/3743062096 conn(0x7f8c64079ca0 msgr2=0x7f8c64074fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:24.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.779+0000 7f8c6c200700 1 -- 172.21.15.5:0/3743062096 shutdown_connections 2024-04-24T06:42:24.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:23.779+0000 7f8c6c200700 1 -- 172.21.15.5:0/3743062096 wait complete. 2024-04-24T06:42:24.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.491+0000 7feff9e00700 1 -- 172.21.15.5:0/1399418344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feff4107ab0 msgr2=0x7feff4107e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:24.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.491+0000 7feff9e00700 1 --2- 172.21.15.5:0/1399418344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feff4107ab0 0x7feff4107e90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fefe40098f0 tx=0x7fefe4009c00 comp rx=0 tx=0).stop 2024-04-24T06:42:24.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.491+0000 7feff9e00700 1 -- 172.21.15.5:0/1399418344 shutdown_connections 2024-04-24T06:42:24.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.491+0000 7feff9e00700 1 --2- 172.21.15.5:0/1399418344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feff4103540 0x7feff410fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:24.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.491+0000 7feff9e00700 1 --2- 172.21.15.5:0/1399418344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feff4107ab0 0x7feff4107e90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:24.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.491+0000 7feff9e00700 1 --2- 172.21.15.5:0/1399418344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feff4102ba0 0x7feff4103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:24.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.491+0000 7feff9e00700 1 -- 172.21.15.5:0/1399418344 >> 172.21.15.5:0/1399418344 conn(0x7feff4079c50 msgr2=0x7feff407a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:24.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.491+0000 7feff9e00700 1 -- 172.21.15.5:0/1399418344 shutdown_connections 2024-04-24T06:42:24.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.491+0000 7feff9e00700 1 -- 172.21.15.5:0/1399418344 wait complete. 2024-04-24T06:42:24.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.493+0000 7feff9e00700 1 Processor -- start 2024-04-24T06:42:24.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.494+0000 7feff9e00700 1 -- start start 2024-04-24T06:42:24.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.494+0000 7feff9e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feff4102ba0 0x7feff419b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:24.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.494+0000 7feff9e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feff4103540 0x7feff419ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:24.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.494+0000 7feff9e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feff4107ab0 0x7feff419fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:24.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.494+0000 7feff9e00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feff4112e80 con 0x7feff4102ba0 2024-04-24T06:42:24.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.494+0000 7feff2a00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feff4102ba0 0x7feff419b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:24.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.495+0000 7feff2a00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feff4102ba0 0x7feff419b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53742/0 (socket says 172.21.15.5:53742) 2024-04-24T06:42:24.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.495+0000 7feff2a00700 1 -- 172.21.15.5:0/317197561 learned_addr learned my addr 172.21.15.5:0/317197561 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:42:24.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.495+0000 7feff2000700 1 --2- 172.21.15.5:0/317197561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feff4103540 0x7feff419ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:24.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.495+0000 7feff9e00700 1 -- 172.21.15.5:0/317197561 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7feff4112d00 con 0x7feff4107ab0 2024-04-24T06:42:24.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.495+0000 7feff9e00700 1 -- 172.21.15.5:0/317197561 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7feff4113000 con 0x7feff4103540 2024-04-24T06:42:24.743 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:24 smithi005 ceph-mon[27798]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T06:42:24.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.495+0000 7feff3400700 1 --2- 172.21.15.5:0/317197561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feff4107ab0 0x7feff419fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:24.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.495+0000 7feff2000700 1 -- 172.21.15.5:0/317197561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feff4107ab0 msgr2=0x7feff419fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:24.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.495+0000 7feff2000700 1 --2- 172.21.15.5:0/317197561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feff4107ab0 0x7feff419fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:24.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.495+0000 7feff2000700 1 -- 172.21.15.5:0/317197561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feff4102ba0 msgr2=0x7feff419b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:24.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.495+0000 7feff2000700 1 --2- 172.21.15.5:0/317197561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feff4102ba0 0x7feff419b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:24.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.495+0000 7feff2000700 1 -- 172.21.15.5:0/317197561 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff41a0550 con 0x7feff4103540 2024-04-24T06:42:24.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.495+0000 7feff2000700 1 --2- 172.21.15.5:0/317197561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feff4103540 0x7feff419ba80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fefe000e9c0 tx=0x7fefe000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:24.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.496+0000 7feff8a00700 1 -- 172.21.15.5:0/317197561 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe000c9c0 con 0x7feff4103540 2024-04-24T06:42:24.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.496+0000 7feff8a00700 1 -- 172.21.15.5:0/317197561 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefe000cb60 con 0x7feff4103540 2024-04-24T06:42:24.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.496+0000 7feff8a00700 1 -- 172.21.15.5:0/317197561 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe0018680 con 0x7feff4103540 2024-04-24T06:42:24.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.496+0000 7feff9e00700 1 -- 172.21.15.5:0/317197561 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feff41a0840 con 0x7feff4103540 2024-04-24T06:42:24.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.496+0000 7feff9e00700 1 -- 172.21.15.5:0/317197561 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feff41a80c0 con 0x7feff4103540 2024-04-24T06:42:24.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.497+0000 7feff9e00700 1 -- 172.21.15.5:0/317197561 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7feff4101470 con 0x7feff4103540 2024-04-24T06:42:24.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.499+0000 7feff8a00700 1 -- 172.21.15.5:0/317197561 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7fefe000cd10 con 0x7feff4103540 2024-04-24T06:42:24.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.500+0000 7feff8a00700 1 --2- 172.21.15.5:0/317197561 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fefd80a3cd0 0x7fefd80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:24.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.500+0000 7feff8a00700 1 -- 172.21.15.5:0/317197561 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fefe00d05b0 con 0x7feff4103540 2024-04-24T06:42:24.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.500+0000 7feff8a00700 1 --2- 172.21.15.5:0/317197561 >> [v2:172.21.15.5:6818/2064636084,v1:172.21.15.5:6819/2064636084] conn(0x7fefd80ad4b0 0x7fefd80af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:24.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.501+0000 7feff8a00700 1 -- 172.21.15.5:0/317197561 --> [v2:172.21.15.5:6818/2064636084,v1:172.21.15.5:6819/2064636084] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fefd80aff80 con 0x7fefd80ad4b0 2024-04-24T06:42:24.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.501+0000 7feff8a00700 1 -- 172.21.15.5:0/317197561 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fefe00d7050 con 0x7feff4103540 2024-04-24T06:42:24.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.501+0000 7feff2a00700 1 --2- 172.21.15.5:0/317197561 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fefd80a3cd0 0x7fefd80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:24.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.501+0000 7feff3400700 1 --2- 172.21.15.5:0/317197561 >> [v2:172.21.15.5:6818/2064636084,v1:172.21.15.5:6819/2064636084] conn(0x7fefd80ad4b0 0x7fefd80af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:24.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.501+0000 7feff2a00700 1 --2- 172.21.15.5:0/317197561 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fefd80a3cd0 0x7fefd80a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fefe4009fd0 tx=0x7fefe4019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:24.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.502+0000 7feff3400700 1 --2- 172.21.15.5:0/317197561 >> [v2:172.21.15.5:6818/2064636084,v1:172.21.15.5:6819/2064636084] conn(0x7fefd80ad4b0 0x7fefd80af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:24.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.503+0000 7feff8a00700 1 -- 172.21.15.5:0/317197561 <== osd.6 v2:172.21.15.5:6818/2064636084 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fefd80aff80 con 0x7fefd80ad4b0 2024-04-24T06:42:24.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.521+0000 7feff9e00700 1 -- 172.21.15.5:0/317197561 --> [v2:172.21.15.5:6818/2064636084,v1:172.21.15.5:6819/2064636084] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7feff41041f0 con 0x7fefd80ad4b0 2024-04-24T06:42:24.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.522+0000 7feff8a00700 1 -- 172.21.15.5:0/317197561 <== osd.6 v2:172.21.15.5:6818/2064636084 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7feff41041f0 con 0x7fefd80ad4b0 2024-04-24T06:42:24.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.522+0000 7feff9e00700 1 -- 172.21.15.5:0/317197561 >> [v2:172.21.15.5:6818/2064636084,v1:172.21.15.5:6819/2064636084] conn(0x7fefd80ad4b0 msgr2=0x7fefd80af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:24.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.522+0000 7feff9e00700 1 --2- 172.21.15.5:0/317197561 >> [v2:172.21.15.5:6818/2064636084,v1:172.21.15.5:6819/2064636084] conn(0x7fefd80ad4b0 0x7fefd80af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:24.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.523+0000 7feff9e00700 1 -- 172.21.15.5:0/317197561 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fefd80a3cd0 msgr2=0x7fefd80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:24.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.523+0000 7feff9e00700 1 --2- 172.21.15.5:0/317197561 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fefd80a3cd0 0x7fefd80a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fefe4009fd0 tx=0x7fefe4019040 comp rx=0 tx=0).stop 2024-04-24T06:42:24.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.523+0000 7feff9e00700 1 -- 172.21.15.5:0/317197561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feff4103540 msgr2=0x7feff419ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:24.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.523+0000 7feff9e00700 1 --2- 172.21.15.5:0/317197561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feff4103540 0x7feff419ba80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fefe000e9c0 tx=0x7fefe000ecd0 comp rx=0 tx=0).stop 2024-04-24T06:42:24.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.523+0000 7feff3400700 1 -- 172.21.15.5:0/317197561 reap_dead start 2024-04-24T06:42:24.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.523+0000 7feff9e00700 1 -- 172.21.15.5:0/317197561 shutdown_connections 2024-04-24T06:42:24.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.523+0000 7feff9e00700 1 -- 172.21.15.5:0/317197561 >> 172.21.15.5:0/317197561 conn(0x7feff4079c50 msgr2=0x7feff4100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:24.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.523+0000 7feff9e00700 1 -- 172.21.15.5:0/317197561 shutdown_connections 2024-04-24T06:42:24.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:24.523+0000 7feff9e00700 1 -- 172.21.15.5:0/317197561 wait complete. 2024-04-24T06:42:24.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:24 smithi037 ceph-mon[32864]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T06:42:24.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:24 smithi137 ceph-mon[33017]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T06:42:25.355 INFO:teuthology.orchestra.run.smithi005.stdout:167503724601 2024-04-24T06:42:25.356 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-24T06:42:25.943 INFO:teuthology.orchestra.run.smithi005.stdout:347892350998 2024-04-24T06:42:25.943 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-24T06:42:26.204 INFO:teuthology.orchestra.run.smithi005.stdout:77309411404 2024-04-24T06:42:26.204 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-24T06:42:26.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:26 smithi005 ceph-mon[27798]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-24T06:42:26.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:26 smithi037 ceph-mon[32864]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-24T06:42:26.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:26 smithi137 ceph-mon[33017]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-24T06:42:27.019 INFO:teuthology.orchestra.run.smithi005.stdout:223338299436 2024-04-24T06:42:27.019 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-24T06:42:27.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.811+0000 7f6e8c800700 1 -- 172.21.15.5:0/867395442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e84102b40 msgr2=0x7f6e84102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:27.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.811+0000 7f6e8c800700 1 --2- 172.21.15.5:0/867395442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e84102b40 0x7f6e84102fc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6e740098f0 tx=0x7f6e74009c00 comp rx=0 tx=0).stop 2024-04-24T06:42:27.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.811+0000 7f6e8c800700 1 -- 172.21.15.5:0/867395442 shutdown_connections 2024-04-24T06:42:27.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.811+0000 7f6e8c800700 1 --2- 172.21.15.5:0/867395442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e84103500 0x7f6e8410f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:27.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.811+0000 7f6e8c800700 1 --2- 172.21.15.5:0/867395442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e84108b90 0x7f6e84108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:27.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.811+0000 7f6e8c800700 1 --2- 172.21.15.5:0/867395442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e84102b40 0x7f6e84102fc0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:27.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.811+0000 7f6e8c800700 1 -- 172.21.15.5:0/867395442 >> 172.21.15.5:0/867395442 conn(0x7f6e84079c50 msgr2=0x7f6e8407a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:27.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.811+0000 7f6e8c800700 1 -- 172.21.15.5:0/867395442 shutdown_connections 2024-04-24T06:42:27.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.811+0000 7f6e8c800700 1 -- 172.21.15.5:0/867395442 wait complete. 2024-04-24T06:42:27.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.812+0000 7f6e8c800700 1 Processor -- start 2024-04-24T06:42:27.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.813+0000 7f6e8c800700 1 -- start start 2024-04-24T06:42:27.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.813+0000 7f6e8c800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e84102b40 0x7f6e8419b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:27.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.813+0000 7f6e8c800700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e84103500 0x7f6e8419bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:27.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.813+0000 7f6e8c800700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e84108b90 0x7f6e8419fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:27.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.813+0000 7f6e8c800700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6e84112e40 con 0x7f6e84108b90 2024-04-24T06:42:27.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.813+0000 7f6e8c800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6e84112cc0 con 0x7f6e84102b40 2024-04-24T06:42:27.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.813+0000 7f6e8c800700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6e84112fc0 con 0x7f6e84103500 2024-04-24T06:42:27.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.813+0000 7f6e8a800700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e84108b90 0x7f6e8419fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:27.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.813+0000 7f6e89e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e84102b40 0x7f6e8419b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:27.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.813+0000 7f6e8a800700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e84108b90 0x7f6e8419fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52616/0 (socket says 172.21.15.5:52616) 2024-04-24T06:42:27.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.813+0000 7f6e8a800700 1 -- 172.21.15.5:0/3963138609 learned_addr learned my addr 172.21.15.5:0/3963138609 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:42:27.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.813+0000 7f6e89400700 1 --2- 172.21.15.5:0/3963138609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e84103500 0x7f6e8419bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:27.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.814+0000 7f6e8a800700 1 -- 172.21.15.5:0/3963138609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e84103500 msgr2=0x7f6e8419bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:27.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.814+0000 7f6e8a800700 1 --2- 172.21.15.5:0/3963138609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e84103500 0x7f6e8419bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:27.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.814+0000 7f6e8a800700 1 -- 172.21.15.5:0/3963138609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e84102b40 msgr2=0x7f6e8419b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:27.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.814+0000 7f6e8a800700 1 --2- 172.21.15.5:0/3963138609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e84102b40 0x7f6e8419b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:27.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.814+0000 7f6e8a800700 1 -- 172.21.15.5:0/3963138609 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e841a0610 con 0x7f6e84108b90 2024-04-24T06:42:27.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.814+0000 7f6e89e00700 1 --2- 172.21.15.5:0/3963138609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e84102b40 0x7f6e8419b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:42:27.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.814+0000 7f6e89400700 1 --2- 172.21.15.5:0/3963138609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e84103500 0x7f6e8419bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:42:27.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.814+0000 7f6e8a800700 1 --2- 172.21.15.5:0/3963138609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e84108b90 0x7f6e8419fed0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6e8000c110 tx=0x7f6e8000c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:27.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.814+0000 7f6e7ea00700 1 -- 172.21.15.5:0/3963138609 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e800092e0 con 0x7f6e84108b90 2024-04-24T06:42:27.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.815+0000 7f6e7ea00700 1 -- 172.21.15.5:0/3963138609 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e80004bb0 con 0x7f6e84108b90 2024-04-24T06:42:27.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.815+0000 7f6e8c800700 1 -- 172.21.15.5:0/3963138609 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e841a08a0 con 0x7f6e84108b90 2024-04-24T06:42:27.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.815+0000 7f6e7ea00700 1 -- 172.21.15.5:0/3963138609 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e80007d10 con 0x7f6e84108b90 2024-04-24T06:42:27.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.815+0000 7f6e8c800700 1 -- 172.21.15.5:0/3963138609 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e841a8180 con 0x7f6e84108b90 2024-04-24T06:42:27.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.816+0000 7f6e8c800700 1 -- 172.21.15.5:0/3963138609 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6e84101fc0 con 0x7f6e84108b90 2024-04-24T06:42:27.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.818+0000 7f6e7ea00700 1 -- 172.21.15.5:0/3963138609 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f6e8001a020 con 0x7f6e84108b90 2024-04-24T06:42:27.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.819+0000 7f6e7ea00700 1 --2- 172.21.15.5:0/3963138609 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f6e6c0a3cd0 0x7f6e6c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:27.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.820+0000 7f6e7ea00700 1 -- 172.21.15.5:0/3963138609 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f6e800d09c0 con 0x7f6e84108b90 2024-04-24T06:42:27.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.820+0000 7f6e89e00700 1 --2- 172.21.15.5:0/3963138609 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f6e6c0a3cd0 0x7f6e6c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:27.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.820+0000 7f6e7ea00700 1 --2- 172.21.15.5:0/3963138609 >> [v2:172.21.15.37:6824/2451570637,v1:172.21.15.37:6825/2451570637] conn(0x7f6e6c0ad4b0 0x7f6e6c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:27.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.820+0000 7f6e7ea00700 1 -- 172.21.15.5:0/3963138609 --> [v2:172.21.15.37:6824/2451570637,v1:172.21.15.37:6825/2451570637] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6e6c0aff80 con 0x7f6e6c0ad4b0 2024-04-24T06:42:27.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.820+0000 7f6e7ea00700 1 -- 172.21.15.5:0/3963138609 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f6e800d0d70 con 0x7f6e84108b90 2024-04-24T06:42:27.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.820+0000 7f6e89400700 1 --2- 172.21.15.5:0/3963138609 >> [v2:172.21.15.37:6824/2451570637,v1:172.21.15.37:6825/2451570637] conn(0x7f6e6c0ad4b0 0x7f6e6c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:27.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.820+0000 7f6e89e00700 1 --2- 172.21.15.5:0/3963138609 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f6e6c0a3cd0 0x7f6e6c0a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6e78009fd0 tx=0x7f6e78009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:27.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.821+0000 7f6e89400700 1 --2- 172.21.15.5:0/3963138609 >> [v2:172.21.15.37:6824/2451570637,v1:172.21.15.37:6825/2451570637] conn(0x7f6e6c0ad4b0 0x7f6e6c0af8d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:27.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.822+0000 7f6e7ea00700 1 -- 172.21.15.5:0/3963138609 <== osd.10 v2:172.21.15.37:6824/2451570637 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6e6c0aff80 con 0x7f6e6c0ad4b0 2024-04-24T06:42:27.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.841+0000 7f6e8c800700 1 -- 172.21.15.5:0/3963138609 --> [v2:172.21.15.37:6824/2451570637,v1:172.21.15.37:6825/2451570637] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6e841041f0 con 0x7f6e6c0ad4b0 2024-04-24T06:42:27.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.842+0000 7f6e7ea00700 1 -- 172.21.15.5:0/3963138609 <== osd.10 v2:172.21.15.37:6824/2451570637 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6e841041f0 con 0x7f6e6c0ad4b0 2024-04-24T06:42:27.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.842+0000 7f6e8c800700 1 -- 172.21.15.5:0/3963138609 >> [v2:172.21.15.37:6824/2451570637,v1:172.21.15.37:6825/2451570637] conn(0x7f6e6c0ad4b0 msgr2=0x7f6e6c0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:27.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.842+0000 7f6e8c800700 1 --2- 172.21.15.5:0/3963138609 >> [v2:172.21.15.37:6824/2451570637,v1:172.21.15.37:6825/2451570637] conn(0x7f6e6c0ad4b0 0x7f6e6c0af8d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:27.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.842+0000 7f6e8c800700 1 -- 172.21.15.5:0/3963138609 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f6e6c0a3cd0 msgr2=0x7f6e6c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:27.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.842+0000 7f6e8c800700 1 --2- 172.21.15.5:0/3963138609 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f6e6c0a3cd0 0x7f6e6c0a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6e78009fd0 tx=0x7f6e78009260 comp rx=0 tx=0).stop 2024-04-24T06:42:27.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.842+0000 7f6e8c800700 1 -- 172.21.15.5:0/3963138609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e84108b90 msgr2=0x7f6e8419fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:27.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.842+0000 7f6e8c800700 1 --2- 172.21.15.5:0/3963138609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e84108b90 0x7f6e8419fed0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6e8000c110 tx=0x7f6e8000c420 comp rx=0 tx=0).stop 2024-04-24T06:42:27.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.842+0000 7f6e8a800700 1 -- 172.21.15.5:0/3963138609 reap_dead start 2024-04-24T06:42:27.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.842+0000 7f6e8c800700 1 -- 172.21.15.5:0/3963138609 shutdown_connections 2024-04-24T06:42:27.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.842+0000 7f6e8c800700 1 -- 172.21.15.5:0/3963138609 >> 172.21.15.5:0/3963138609 conn(0x7f6e84079c50 msgr2=0x7f6e84101620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:27.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.843+0000 7f6e8c800700 1 -- 172.21.15.5:0/3963138609 shutdown_connections 2024-04-24T06:42:27.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:20.843+0000 7f6e8c800700 1 -- 172.21.15.5:0/3963138609 wait complete. 2024-04-24T06:42:27.783 INFO:teuthology.orchestra.run.smithi005.stdout:313532612635 2024-04-24T06:42:27.784 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-24T06:42:28.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.022+0000 7f854c600700 1 -- 172.21.15.5:0/288340932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8544108b60 msgr2=0x7f8544108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:28.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.022+0000 7f854c600700 1 --2- 172.21.15.5:0/288340932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8544108b60 0x7f8544108f40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8534009990 tx=0x7f8534009ca0 comp rx=0 tx=0).stop 2024-04-24T06:42:28.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.022+0000 7f854c600700 1 -- 172.21.15.5:0/288340932 shutdown_connections 2024-04-24T06:42:28.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.022+0000 7f854c600700 1 --2- 172.21.15.5:0/288340932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f854406ba30 0x7f854410f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:28.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.022+0000 7f854c600700 1 --2- 172.21.15.5:0/288340932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8544108b60 0x7f8544108f40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:28.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.022+0000 7f854c600700 1 --2- 172.21.15.5:0/288340932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f854406b090 0x7f854406b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:28.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.022+0000 7f854c600700 1 -- 172.21.15.5:0/288340932 >> 172.21.15.5:0/288340932 conn(0x7f8544078960 msgr2=0x7f8544078d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:28.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.022+0000 7f854c600700 1 -- 172.21.15.5:0/288340932 shutdown_connections 2024-04-24T06:42:28.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.022+0000 7f854c600700 1 -- 172.21.15.5:0/288340932 wait complete. 2024-04-24T06:42:28.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.023+0000 7f854c600700 1 Processor -- start 2024-04-24T06:42:28.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.023+0000 7f854c600700 1 -- start start 2024-04-24T06:42:28.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.024+0000 7f854c600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f854406b090 0x7f854419b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:28.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.024+0000 7f854c600700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f854406ba30 0x7f854419b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:28.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.024+0000 7f854c600700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8544108b60 0x7f854419fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:28.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.024+0000 7f854c600700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8544112db0 con 0x7f854406ba30 2024-04-24T06:42:28.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.024+0000 7f854c600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8544112c30 con 0x7f854406b090 2024-04-24T06:42:28.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.024+0000 7f854c600700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8544112f30 con 0x7f8544108b60 2024-04-24T06:42:28.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.024+0000 7f8549200700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f854406ba30 0x7f854419b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:28.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.024+0000 7f8549c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f854406b090 0x7f854419b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:28.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.024+0000 7f8549200700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f854406ba30 0x7f854419b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53766/0 (socket says 172.21.15.5:53766) 2024-04-24T06:42:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.024+0000 7f8549200700 1 -- 172.21.15.5:0/627589722 learned_addr learned my addr 172.21.15.5:0/627589722 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:42:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.024+0000 7f854a600700 1 --2- 172.21.15.5:0/627589722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8544108b60 0x7f854419fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.024+0000 7f8549200700 1 -- 172.21.15.5:0/627589722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8544108b60 msgr2=0x7f854419fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.024+0000 7f8549200700 1 --2- 172.21.15.5:0/627589722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8544108b60 0x7f854419fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.024+0000 7f8549200700 1 -- 172.21.15.5:0/627589722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f854406b090 msgr2=0x7f854419b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.025+0000 7f8549200700 1 --2- 172.21.15.5:0/627589722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f854406b090 0x7f854419b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.025+0000 7f8549200700 1 -- 172.21.15.5:0/627589722 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85441a0430 con 0x7f854406ba30 2024-04-24T06:42:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.025+0000 7f8549c00700 1 --2- 172.21.15.5:0/627589722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f854406b090 0x7f854419b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:42:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.025+0000 7f8549200700 1 --2- 172.21.15.5:0/627589722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f854406ba30 0x7f854419b9b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f853800e9a0 tx=0x7f853800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.025+0000 7f8542000700 1 -- 172.21.15.5:0/627589722 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853800ca10 con 0x7f854406ba30 2024-04-24T06:42:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.025+0000 7f8542000700 1 -- 172.21.15.5:0/627589722 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f853800cbb0 con 0x7f854406ba30 2024-04-24T06:42:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.025+0000 7f854c600700 1 -- 172.21.15.5:0/627589722 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85441a0720 con 0x7f854406ba30 2024-04-24T06:42:28.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.025+0000 7f8542000700 1 -- 172.21.15.5:0/627589722 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8538018680 con 0x7f854406ba30 2024-04-24T06:42:28.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.026+0000 7f854c600700 1 -- 172.21.15.5:0/627589722 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85441a7fa0 con 0x7f854406ba30 2024-04-24T06:42:28.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.026+0000 7f854c600700 1 -- 172.21.15.5:0/627589722 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8544063240 con 0x7f854406ba30 2024-04-24T06:42:28.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.029+0000 7f8542000700 1 -- 172.21.15.5:0/627589722 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f8538010430 con 0x7f854406ba30 2024-04-24T06:42:28.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.030+0000 7f8542000700 1 --2- 172.21.15.5:0/627589722 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f852c0a3d50 0x7f852c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:28.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.031+0000 7f8542000700 1 -- 172.21.15.5:0/627589722 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f85380d0ee0 con 0x7f854406ba30 2024-04-24T06:42:28.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.031+0000 7f8549c00700 1 --2- 172.21.15.5:0/627589722 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f852c0a3d50 0x7f852c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:28.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.031+0000 7f8542000700 1 --2- 172.21.15.5:0/627589722 >> [v2:172.21.15.5:6826/2257102107,v1:172.21.15.5:6827/2257102107] conn(0x7f852c0ad530 0x7f852c0af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:28.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.031+0000 7f8542000700 1 -- 172.21.15.5:0/627589722 --> [v2:172.21.15.5:6826/2257102107,v1:172.21.15.5:6827/2257102107] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8538027e50 con 0x7f852c0ad530 2024-04-24T06:42:28.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.031+0000 7f8542000700 1 -- 172.21.15.5:0/627589722 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f85380d1290 con 0x7f854406ba30 2024-04-24T06:42:28.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.031+0000 7f854a600700 1 --2- 172.21.15.5:0/627589722 >> [v2:172.21.15.5:6826/2257102107,v1:172.21.15.5:6827/2257102107] conn(0x7f852c0ad530 0x7f852c0af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:28.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.032+0000 7f8549c00700 1 --2- 172.21.15.5:0/627589722 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f852c0a3d50 0x7f852c0a6210 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f853400b2e0 tx=0x7f8534005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:28.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.032+0000 7f854a600700 1 --2- 172.21.15.5:0/627589722 >> [v2:172.21.15.5:6826/2257102107,v1:172.21.15.5:6827/2257102107] conn(0x7f852c0ad530 0x7f852c0af950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:28.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.034+0000 7f8542000700 1 -- 172.21.15.5:0/627589722 <== osd.9 v2:172.21.15.5:6826/2257102107 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8538027e50 con 0x7f852c0ad530 2024-04-24T06:42:28.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.052+0000 7f854c600700 1 -- 172.21.15.5:0/627589722 --> [v2:172.21.15.5:6826/2257102107,v1:172.21.15.5:6827/2257102107] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f85440636b0 con 0x7f852c0ad530 2024-04-24T06:42:28.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.053+0000 7f8542000700 1 -- 172.21.15.5:0/627589722 <== osd.9 v2:172.21.15.5:6826/2257102107 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f85440636b0 con 0x7f852c0ad530 2024-04-24T06:42:28.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.053+0000 7f854c600700 1 -- 172.21.15.5:0/627589722 >> [v2:172.21.15.5:6826/2257102107,v1:172.21.15.5:6827/2257102107] conn(0x7f852c0ad530 msgr2=0x7f852c0af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:28.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.053+0000 7f854c600700 1 --2- 172.21.15.5:0/627589722 >> [v2:172.21.15.5:6826/2257102107,v1:172.21.15.5:6827/2257102107] conn(0x7f852c0ad530 0x7f852c0af950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:28.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.053+0000 7f854c600700 1 -- 172.21.15.5:0/627589722 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f852c0a3d50 msgr2=0x7f852c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:28.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.053+0000 7f854c600700 1 --2- 172.21.15.5:0/627589722 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f852c0a3d50 0x7f852c0a6210 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f853400b2e0 tx=0x7f8534005960 comp rx=0 tx=0).stop 2024-04-24T06:42:28.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.053+0000 7f854c600700 1 -- 172.21.15.5:0/627589722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f854406ba30 msgr2=0x7f854419b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:28.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.053+0000 7f854c600700 1 --2- 172.21.15.5:0/627589722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f854406ba30 0x7f854419b9b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f853800e9a0 tx=0x7f853800ecb0 comp rx=0 tx=0).stop 2024-04-24T06:42:28.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.053+0000 7f854a600700 1 -- 172.21.15.5:0/627589722 reap_dead start 2024-04-24T06:42:28.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.054+0000 7f854c600700 1 -- 172.21.15.5:0/627589722 shutdown_connections 2024-04-24T06:42:28.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.054+0000 7f854c600700 1 -- 172.21.15.5:0/627589722 >> 172.21.15.5:0/627589722 conn(0x7f8544078960 msgr2=0x7f854410ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:28.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.054+0000 7f854c600700 1 -- 172.21.15.5:0/627589722 shutdown_connections 2024-04-24T06:42:28.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.054+0000 7f854c600700 1 -- 172.21.15.5:0/627589722 wait complete. 2024-04-24T06:42:28.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.261+0000 7f9343000700 1 -- 172.21.15.5:0/865260101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f933c10e460 msgr2=0x7f933c116030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:28.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.261+0000 7f9343000700 1 --2- 172.21.15.5:0/865260101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f933c10e460 0x7f933c116030 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f933400b0d0 tx=0x7f933400b3e0 comp rx=0 tx=0).stop 2024-04-24T06:42:28.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.261+0000 7f9343000700 1 -- 172.21.15.5:0/865260101 shutdown_connections 2024-04-24T06:42:28.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.261+0000 7f9343000700 1 --2- 172.21.15.5:0/865260101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f933c10e460 0x7f933c116030 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:28.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.261+0000 7f9343000700 1 --2- 172.21.15.5:0/865260101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f933c109380 0x7f933c109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:28.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.261+0000 7f9343000700 1 --2- 172.21.15.5:0/865260101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f933c109d30 0x7f933c10dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:28.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.261+0000 7f9343000700 1 -- 172.21.15.5:0/865260101 >> 172.21.15.5:0/865260101 conn(0x7f933c0762c0 msgr2=0x7f933c107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:28.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.261+0000 7f9343000700 1 -- 172.21.15.5:0/865260101 shutdown_connections 2024-04-24T06:42:28.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.261+0000 7f9343000700 1 -- 172.21.15.5:0/865260101 wait complete. 2024-04-24T06:42:28.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.262+0000 7f9343000700 1 Processor -- start 2024-04-24T06:42:28.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.262+0000 7f9343000700 1 -- start start 2024-04-24T06:42:28.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.262+0000 7f9343000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f933c109380 0x7f933c1a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:28.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.263+0000 7f9343000700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f933c109d30 0x7f933c1a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:28.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.263+0000 7f9343000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f933c10e460 0x7f933c19f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:28.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.263+0000 7f9343000700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f933c1194a0 con 0x7f933c109d30 2024-04-24T06:42:28.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.263+0000 7f9343000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f933c119320 con 0x7f933c10e460 2024-04-24T06:42:28.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.263+0000 7f9343000700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f933c119620 con 0x7f933c109380 2024-04-24T06:42:28.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.263+0000 7f933b400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f933c109d30 0x7f933c1a5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:28.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.263+0000 7f933b400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f933c109d30 0x7f933c1a5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53780/0 (socket says 172.21.15.5:53780) 2024-04-24T06:42:28.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.263+0000 7f933b400700 1 -- 172.21.15.5:0/2286065405 learned_addr learned my addr 172.21.15.5:0/2286065405 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:42:28.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.263+0000 7f933be00700 1 --2- 172.21.15.5:0/2286065405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f933c109380 0x7f933c1a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:28.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.263+0000 7f9341000700 1 --2- 172.21.15.5:0/2286065405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f933c10e460 0x7f933c19f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:28.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.263+0000 7f933b400700 1 -- 172.21.15.5:0/2286065405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f933c109380 msgr2=0x7f933c1a55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:28.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.263+0000 7f933b400700 1 --2- 172.21.15.5:0/2286065405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f933c109380 0x7f933c1a55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:28.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.263+0000 7f933b400700 1 -- 172.21.15.5:0/2286065405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f933c10e460 msgr2=0x7f933c19f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:28.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.263+0000 7f933b400700 1 --2- 172.21.15.5:0/2286065405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f933c10e460 0x7f933c19f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:28.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.263+0000 7f933b400700 1 -- 172.21.15.5:0/2286065405 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f933c19ff80 con 0x7f933c109d30 2024-04-24T06:42:28.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.264+0000 7f933b400700 1 --2- 172.21.15.5:0/2286065405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f933c109d30 0x7f933c1a5b10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f932800ea30 tx=0x7f932800ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:28.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.264+0000 7f9338c00700 1 -- 172.21.15.5:0/2286065405 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932800cbd0 con 0x7f933c109d30 2024-04-24T06:42:28.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.264+0000 7f9338c00700 1 -- 172.21.15.5:0/2286065405 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9328004500 con 0x7f933c109d30 2024-04-24T06:42:28.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.264+0000 7f9338c00700 1 -- 172.21.15.5:0/2286065405 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9328003e30 con 0x7f933c109d30 2024-04-24T06:42:28.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.264+0000 7f9343000700 1 -- 172.21.15.5:0/2286065405 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f933c1a0270 con 0x7f933c109d30 2024-04-24T06:42:28.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.264+0000 7f9343000700 1 -- 172.21.15.5:0/2286065405 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f933c1ac550 con 0x7f933c109d30 2024-04-24T06:42:28.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.265+0000 7f9343000700 1 -- 172.21.15.5:0/2286065405 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f933c0771c0 con 0x7f933c109d30 2024-04-24T06:42:28.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.267+0000 7f9338c00700 1 -- 172.21.15.5:0/2286065405 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f9328004000 con 0x7f933c109d30 2024-04-24T06:42:28.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.269+0000 7f9338c00700 1 --2- 172.21.15.5:0/2286065405 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f93200a3cd0 0x7f93200a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:28.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.269+0000 7f9338c00700 1 -- 172.21.15.5:0/2286065405 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f93280d06f0 con 0x7f933c109d30 2024-04-24T06:42:28.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.269+0000 7f933be00700 1 --2- 172.21.15.5:0/2286065405 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f93200a3cd0 0x7f93200a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:28.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.269+0000 7f9338c00700 1 --2- 172.21.15.5:0/2286065405 >> [v2:172.21.15.5:6802/2155858515,v1:172.21.15.5:6803/2155858515] conn(0x7f93200ad4b0 0x7f93200af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:28.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.269+0000 7f9338c00700 1 -- 172.21.15.5:0/2286065405 --> [v2:172.21.15.5:6802/2155858515,v1:172.21.15.5:6803/2155858515] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f93200aff80 con 0x7f93200ad4b0 2024-04-24T06:42:28.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.269+0000 7f9338c00700 1 -- 172.21.15.5:0/2286065405 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f93280d0aa0 con 0x7f933c109d30 2024-04-24T06:42:28.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.269+0000 7f9341000700 1 --2- 172.21.15.5:0/2286065405 >> [v2:172.21.15.5:6802/2155858515,v1:172.21.15.5:6803/2155858515] conn(0x7f93200ad4b0 0x7f93200af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:28.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.270+0000 7f933be00700 1 --2- 172.21.15.5:0/2286065405 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f93200a3cd0 0x7f93200a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f932c009680 tx=0x7f932c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:28.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.270+0000 7f9341000700 1 --2- 172.21.15.5:0/2286065405 >> [v2:172.21.15.5:6802/2155858515,v1:172.21.15.5:6803/2155858515] conn(0x7f93200ad4b0 0x7f93200af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:28.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.271+0000 7f9338c00700 1 -- 172.21.15.5:0/2286065405 <== osd.0 v2:172.21.15.5:6802/2155858515 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f93200aff80 con 0x7f93200ad4b0 2024-04-24T06:42:28.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.290+0000 7f9343000700 1 -- 172.21.15.5:0/2286065405 --> [v2:172.21.15.5:6802/2155858515,v1:172.21.15.5:6803/2155858515] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f933c10a860 con 0x7f93200ad4b0 2024-04-24T06:42:28.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.290+0000 7f9338c00700 1 -- 172.21.15.5:0/2286065405 <== osd.0 v2:172.21.15.5:6802/2155858515 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f933c10a860 con 0x7f93200ad4b0 2024-04-24T06:42:28.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.291+0000 7f9343000700 1 -- 172.21.15.5:0/2286065405 >> [v2:172.21.15.5:6802/2155858515,v1:172.21.15.5:6803/2155858515] conn(0x7f93200ad4b0 msgr2=0x7f93200af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:28.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.291+0000 7f9343000700 1 --2- 172.21.15.5:0/2286065405 >> [v2:172.21.15.5:6802/2155858515,v1:172.21.15.5:6803/2155858515] conn(0x7f93200ad4b0 0x7f93200af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:28.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.291+0000 7f9343000700 1 -- 172.21.15.5:0/2286065405 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f93200a3cd0 msgr2=0x7f93200a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:28.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.291+0000 7f9343000700 1 --2- 172.21.15.5:0/2286065405 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f93200a3cd0 0x7f93200a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f932c009680 tx=0x7f932c009260 comp rx=0 tx=0).stop 2024-04-24T06:42:28.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.291+0000 7f9343000700 1 -- 172.21.15.5:0/2286065405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f933c109d30 msgr2=0x7f933c1a5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:28.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.291+0000 7f9343000700 1 --2- 172.21.15.5:0/2286065405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f933c109d30 0x7f933c1a5b10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f932800ea30 tx=0x7f932800ed40 comp rx=0 tx=0).stop 2024-04-24T06:42:28.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.291+0000 7f9341000700 1 -- 172.21.15.5:0/2286065405 reap_dead start 2024-04-24T06:42:28.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.291+0000 7f9343000700 1 -- 172.21.15.5:0/2286065405 shutdown_connections 2024-04-24T06:42:28.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.291+0000 7f9343000700 1 -- 172.21.15.5:0/2286065405 >> 172.21.15.5:0/2286065405 conn(0x7f933c0762c0 msgr2=0x7f933c0767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:28.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.291+0000 7f9343000700 1 -- 172.21.15.5:0/2286065405 shutdown_connections 2024-04-24T06:42:28.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:28.291+0000 7f9343000700 1 -- 172.21.15.5:0/2286065405 wait complete. 2024-04-24T06:42:28.627 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:28 smithi005 ceph-mon[27798]: pgmap v339: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:42:28.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:28 smithi037 ceph-mon[32864]: pgmap v339: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:42:28.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:28 smithi137 ceph-mon[33017]: pgmap v339: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:42:29.393 INFO:teuthology.orchestra.run.smithi005.stdout:201863462963 2024-04-24T06:42:29.394 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-24T06:42:29.728 INFO:teuthology.orchestra.run.smithi005.stdout:34359738453 2024-04-24T06:42:29.728 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-24T06:42:29.969 INFO:teuthology.orchestra.run.smithi005.stdout:257698037798 2024-04-24T06:42:29.969 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-24T06:42:30.408 INFO:teuthology.orchestra.run.smithi005.stdout:292057776162 2024-04-24T06:42:30.408 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-24T06:42:30.523 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:42:30.529 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:42:30.532 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:42:30.538 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:42:30.545 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:42:30.552 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:42:30.562 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:42:30.626 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:42:30.719 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:42:30.727 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:42:30.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:30 smithi005 ceph-mon[27798]: pgmap v340: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T06:42:30.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:30 smithi037 ceph-mon[32864]: pgmap v340: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T06:42:30.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:30 smithi137 ceph-mon[33017]: pgmap v340: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T06:42:33.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:32 smithi005 ceph-mon[27798]: pgmap v341: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:42:33.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:32 smithi137 ceph-mon[33017]: pgmap v341: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:42:33.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:32 smithi037 ceph-mon[32864]: pgmap v341: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:42:34.779 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:42:35.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:34 smithi005 ceph-mon[27798]: pgmap v342: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-24T06:42:35.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:34 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:42:35.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:34 smithi137 ceph-mon[33017]: pgmap v342: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-24T06:42:35.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:34 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:42:35.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:34 smithi037 ceph-mon[32864]: pgmap v342: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-24T06:42:35.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:34 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:42:37.087 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:36 smithi005 ceph-mon[27798]: pgmap v343: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:42:37.087 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:36 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:42:37.087 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:36 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:42:37.087 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:36 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:42:37.087 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:36 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:42:37.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:36 smithi137 ceph-mon[33017]: pgmap v343: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:42:37.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:42:37.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:42:37.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:42:37.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:36 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:42:37.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:36 smithi037 ceph-mon[32864]: pgmap v343: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:42:37.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:36 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:42:37.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:36 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:42:37.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:36 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:42:37.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:36 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:42:39.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:38 smithi005 ceph-mon[27798]: pgmap v344: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-24T06:42:39.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:38 smithi137 ceph-mon[33017]: pgmap v344: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-24T06:42:39.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:38 smithi037 ceph-mon[32864]: pgmap v344: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-24T06:42:41.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:40 smithi005 ceph-mon[27798]: pgmap v345: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T06:42:41.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:40 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/2637123935' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-24T06:42:41.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:40 smithi137 ceph-mon[33017]: pgmap v345: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T06:42:41.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:40 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/2637123935' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-24T06:42:41.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:40 smithi037 ceph-mon[32864]: pgmap v345: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T06:42:41.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:40 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/2637123935' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-24T06:42:41.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.557+0000 7fbd87e00700 1 -- 172.21.15.5:0/3505677615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd881138c0 msgr2=0x7fbd88115cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:41.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.557+0000 7fbd87e00700 1 --2- 172.21.15.5:0/3505677615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd881138c0 0x7fbd88115cb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbd80009f90 tx=0x7fbd8000b330 comp rx=0 tx=0).stop 2024-04-24T06:42:41.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.557+0000 7fbd87e00700 1 -- 172.21.15.5:0/3505677615 shutdown_connections 2024-04-24T06:42:41.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.557+0000 7fbd87e00700 1 --2- 172.21.15.5:0/3505677615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd881138c0 0x7fbd88115cb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:41.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.557+0000 7fbd87e00700 1 --2- 172.21.15.5:0/3505677615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd88077020 0x7fbd88077400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:41.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.557+0000 7fbd87e00700 1 --2- 172.21.15.5:0/3505677615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd88077940 0x7fbd88113380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:41.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.557+0000 7fbd87e00700 1 -- 172.21.15.5:0/3505677615 >> 172.21.15.5:0/3505677615 conn(0x7fbd881006d0 msgr2=0x7fbd88102af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:41.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.557+0000 7fbd87e00700 1 -- 172.21.15.5:0/3505677615 shutdown_connections 2024-04-24T06:42:41.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.557+0000 7fbd87e00700 1 -- 172.21.15.5:0/3505677615 wait complete. 2024-04-24T06:42:41.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.558+0000 7fbd87e00700 1 Processor -- start 2024-04-24T06:42:41.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.558+0000 7fbd87e00700 1 -- start start 2024-04-24T06:42:41.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.559+0000 7fbd87e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd88077020 0x7fbd881a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:41.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.559+0000 7fbd87e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd88077940 0x7fbd881a5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:41.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.559+0000 7fbd87e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd881138c0 0x7fbd8819f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:41.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.559+0000 7fbd87e00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbd88119390 con 0x7fbd881138c0 2024-04-24T06:42:41.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.559+0000 7fbd87e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbd88119210 con 0x7fbd88077940 2024-04-24T06:42:41.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.559+0000 7fbd87e00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbd88119510 con 0x7fbd88077020 2024-04-24T06:42:41.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.559+0000 7fbd86a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd88077020 0x7fbd881a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:41.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.559+0000 7fbd86a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd88077020 0x7fbd881a5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:60838/0 (socket says 172.21.15.5:60838) 2024-04-24T06:42:41.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.559+0000 7fbd86a00700 1 -- 172.21.15.5:0/3183283956 learned_addr learned my addr 172.21.15.5:0/3183283956 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:42:41.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.559+0000 7fbd87400700 1 --2- 172.21.15.5:0/3183283956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd881138c0 0x7fbd8819f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:41.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.559+0000 7fbd7fe00700 1 --2- 172.21.15.5:0/3183283956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd88077940 0x7fbd881a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:41.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.560+0000 7fbd86a00700 1 -- 172.21.15.5:0/3183283956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd88077940 msgr2=0x7fbd881a5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:41.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.560+0000 7fbd86a00700 1 --2- 172.21.15.5:0/3183283956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd88077940 0x7fbd881a5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:41.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.560+0000 7fbd86a00700 1 -- 172.21.15.5:0/3183283956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd881138c0 msgr2=0x7fbd8819f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:41.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.560+0000 7fbd86a00700 1 --2- 172.21.15.5:0/3183283956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd881138c0 0x7fbd8819f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:41.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.560+0000 7fbd86a00700 1 -- 172.21.15.5:0/3183283956 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd8819fd30 con 0x7fbd88077020 2024-04-24T06:42:41.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.560+0000 7fbd87400700 1 --2- 172.21.15.5:0/3183283956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd881138c0 0x7fbd8819f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:42:41.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.560+0000 7fbd86a00700 1 --2- 172.21.15.5:0/3183283956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd88077020 0x7fbd881a5410 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbd7800ec10 tx=0x7fbd7800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:41.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.560+0000 7fbd7f400700 1 -- 172.21.15.5:0/3183283956 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7800cc90 con 0x7fbd88077020 2024-04-24T06:42:41.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.560+0000 7fbd87e00700 1 -- 172.21.15.5:0/3183283956 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd881a0020 con 0x7fbd88077020 2024-04-24T06:42:41.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.560+0000 7fbd7f400700 1 -- 172.21.15.5:0/3183283956 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd7800ce30 con 0x7fbd88077020 2024-04-24T06:42:41.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.560+0000 7fbd7f400700 1 -- 172.21.15.5:0/3183283956 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd78010690 con 0x7fbd88077020 2024-04-24T06:42:41.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.561+0000 7fbd87e00700 1 -- 172.21.15.5:0/3183283956 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd881ac2e0 con 0x7fbd88077020 2024-04-24T06:42:41.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.562+0000 7fbd87e00700 1 -- 172.21.15.5:0/3183283956 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd880697e0 con 0x7fbd88077020 2024-04-24T06:42:41.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.564+0000 7fbd7f400700 1 -- 172.21.15.5:0/3183283956 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7fbd78003680 con 0x7fbd88077020 2024-04-24T06:42:41.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.565+0000 7fbd7f400700 1 --2- 172.21.15.5:0/3183283956 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fbd6c0a3d50 0x7fbd6c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:41.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.565+0000 7fbd7f400700 1 -- 172.21.15.5:0/3183283956 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fbd780d0960 con 0x7fbd88077020 2024-04-24T06:42:41.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.565+0000 7fbd7fe00700 1 --2- 172.21.15.5:0/3183283956 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fbd6c0a3d50 0x7fbd6c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:41.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.566+0000 7fbd7fe00700 1 --2- 172.21.15.5:0/3183283956 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fbd6c0a3d50 0x7fbd6c0a6210 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbd88104070 tx=0x7fbd70009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:41.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.568+0000 7fbd7f400700 1 -- 172.21.15.5:0/3183283956 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbd7809d290 con 0x7fbd88077020 2024-04-24T06:42:41.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.807+0000 7fbd87e00700 1 -- 172.21.15.5:0/3183283956 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fbd881a0b00 con 0x7fbd88077020 2024-04-24T06:42:41.810 INFO:teuthology.orchestra.run.smithi005.stdout:167503724605 2024-04-24T06:42:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.807+0000 7fbd7f400700 1 -- 172.21.15.5:0/3183283956 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbd7809cb90 con 0x7fbd88077020 2024-04-24T06:42:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.811+0000 7fbd87e00700 1 -- 172.21.15.5:0/3183283956 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fbd6c0a3d50 msgr2=0x7fbd6c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.811+0000 7fbd87e00700 1 --2- 172.21.15.5:0/3183283956 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fbd6c0a3d50 0x7fbd6c0a6210 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbd88104070 tx=0x7fbd70009260 comp rx=0 tx=0).stop 2024-04-24T06:42:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.811+0000 7fbd87e00700 1 -- 172.21.15.5:0/3183283956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd88077020 msgr2=0x7fbd881a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.811+0000 7fbd87e00700 1 --2- 172.21.15.5:0/3183283956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd88077020 0x7fbd881a5410 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbd7800ec10 tx=0x7fbd7800ef20 comp rx=0 tx=0).stop 2024-04-24T06:42:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.812+0000 7fbd87e00700 1 -- 172.21.15.5:0/3183283956 shutdown_connections 2024-04-24T06:42:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.812+0000 7fbd87e00700 1 --2- 172.21.15.5:0/3183283956 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fbd6c0a3d50 0x7fbd6c0a6210 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.812+0000 7fbd87e00700 1 --2- 172.21.15.5:0/3183283956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd881138c0 0x7fbd8819f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.812+0000 7fbd87e00700 1 --2- 172.21.15.5:0/3183283956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd88077020 0x7fbd881a5410 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.812+0000 7fbd87e00700 1 --2- 172.21.15.5:0/3183283956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd88077940 0x7fbd881a5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.812+0000 7fbd87e00700 1 -- 172.21.15.5:0/3183283956 >> 172.21.15.5:0/3183283956 conn(0x7fbd881006d0 msgr2=0x7fbd88102ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.812+0000 7fbd87e00700 1 -- 172.21.15.5:0/3183283956 shutdown_connections 2024-04-24T06:42:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:41.812+0000 7fbd87e00700 1 -- 172.21.15.5:0/3183283956 wait complete. 2024-04-24T06:42:41.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:41 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-04-24T06:42:41.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:41 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-24T06:42:41.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:41 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-04-24T06:42:41.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:41 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-24T06:42:41.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:41 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-04-24T06:42:41.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:41 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 7]}]: dispatch 2024-04-24T06:42:41.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:41 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]: dispatch 2024-04-24T06:42:41.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:41 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]: dispatch 2024-04-24T06:42:41.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:41 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-04-24T06:42:41.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:41 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/3183283956' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-24T06:42:42.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:41 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-04-24T06:42:42.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:41 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-24T06:42:42.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:41 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-04-24T06:42:42.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:41 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-24T06:42:42.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:41 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-04-24T06:42:42.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:41 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 7]}]: dispatch 2024-04-24T06:42:42.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:41 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]: dispatch 2024-04-24T06:42:42.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:41 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]: dispatch 2024-04-24T06:42:42.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:41 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-04-24T06:42:42.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:41 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/3183283956' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-24T06:42:42.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:41 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-04-24T06:42:42.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:41 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-24T06:42:42.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:41 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-04-24T06:42:42.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:41 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-24T06:42:42.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:41 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-04-24T06:42:42.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:41 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 7]}]: dispatch 2024-04-24T06:42:42.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:41 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]: dispatch 2024-04-24T06:42:42.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:41 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]: dispatch 2024-04-24T06:42:42.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:41 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-04-24T06:42:42.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:41 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/3183283956' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-24T06:42:43.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:42 smithi005 ceph-mon[27798]: pgmap v346: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:42:43.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:42 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-04-24T06:42:43.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:42 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-24T06:42:43.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:42 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-04-24T06:42:43.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:42 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-24T06:42:43.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:42 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-04-24T06:42:43.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:42 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 7]}]': finished 2024-04-24T06:42:43.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:42 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]': finished 2024-04-24T06:42:43.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:42 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]': finished 2024-04-24T06:42:43.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:42 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-04-24T06:42:43.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:42 smithi005 ceph-mon[27798]: osdmap e87: 12 total, 12 up, 12 in 2024-04-24T06:42:43.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:42 smithi137 ceph-mon[33017]: pgmap v346: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:42:43.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:42 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-04-24T06:42:43.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:42 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-24T06:42:43.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:42 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-04-24T06:42:43.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:42 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-24T06:42:43.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:42 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-04-24T06:42:43.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:42 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 7]}]': finished 2024-04-24T06:42:43.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:42 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]': finished 2024-04-24T06:42:43.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:42 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]': finished 2024-04-24T06:42:43.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:42 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-04-24T06:42:43.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:42 smithi137 ceph-mon[33017]: osdmap e87: 12 total, 12 up, 12 in 2024-04-24T06:42:43.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:42 smithi037 ceph-mon[32864]: pgmap v346: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:42:43.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:42 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-04-24T06:42:43.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:42 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-24T06:42:43.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:42 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-04-24T06:42:43.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:42 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-24T06:42:43.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:42 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-04-24T06:42:43.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:42 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 7]}]': finished 2024-04-24T06:42:43.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:42 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]': finished 2024-04-24T06:42:43.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:42 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]': finished 2024-04-24T06:42:43.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:42 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-04-24T06:42:43.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:42 smithi037 ceph-mon[32864]: osdmap e87: 12 total, 12 up, 12 in 2024-04-24T06:42:44.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:43 smithi137 ceph-mon[33017]: osdmap e88: 12 total, 12 up, 12 in 2024-04-24T06:42:44.213 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:43 smithi005 ceph-mon[27798]: osdmap e88: 12 total, 12 up, 12 in 2024-04-24T06:42:44.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:43 smithi037 ceph-mon[32864]: osdmap e88: 12 total, 12 up, 12 in 2024-04-24T06:42:44.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.387+0000 7f76fa400700 1 -- 172.21.15.5:0/205258519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76f410f380 msgr2=0x7f76f4111770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:44.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.387+0000 7f76fa400700 1 --2- 172.21.15.5:0/205258519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76f410f380 0x7f76f4111770 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f76ec009f90 tx=0x7f76ec00b330 comp rx=0 tx=0).stop 2024-04-24T06:42:44.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.387+0000 7f76fa400700 1 -- 172.21.15.5:0/205258519 shutdown_connections 2024-04-24T06:42:44.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.387+0000 7f76fa400700 1 --2- 172.21.15.5:0/205258519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76f410f380 0x7f76f4111770 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:44.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.387+0000 7f76fa400700 1 --2- 172.21.15.5:0/205258519 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76f41017d0 0x7f76f4101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:44.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.387+0000 7f76fa400700 1 --2- 172.21.15.5:0/205258519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f4102180 0x7f76f410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:44.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.387+0000 7f76fa400700 1 -- 172.21.15.5:0/205258519 >> 172.21.15.5:0/205258519 conn(0x7f76f40fd410 msgr2=0x7f76f40ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:44.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.387+0000 7f76fa400700 1 -- 172.21.15.5:0/205258519 shutdown_connections 2024-04-24T06:42:44.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.387+0000 7f76fa400700 1 -- 172.21.15.5:0/205258519 wait complete. 2024-04-24T06:42:44.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.388+0000 7f76fa400700 1 Processor -- start 2024-04-24T06:42:44.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.388+0000 7f76fa400700 1 -- start start 2024-04-24T06:42:44.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.389+0000 7f76fa400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f41017d0 0x7f76f419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:44.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.389+0000 7f76fa400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76f4102180 0x7f76f419b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:44.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.389+0000 7f76fa400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76f410f380 0x7f76f419fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:44.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.389+0000 7f76fa400700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f76f4114f80 con 0x7f76f41017d0 2024-04-24T06:42:44.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.389+0000 7f76fa400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f76f4114e00 con 0x7f76f410f380 2024-04-24T06:42:44.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.389+0000 7f76fa400700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f76f4115100 con 0x7f76f4102180 2024-04-24T06:42:44.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.389+0000 7f76f9000700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f41017d0 0x7f76f419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:44.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.389+0000 7f76f9000700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f41017d0 0x7f76f419b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43848/0 (socket says 172.21.15.5:43848) 2024-04-24T06:42:44.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.389+0000 7f76f9000700 1 -- 172.21.15.5:0/1267393344 learned_addr learned my addr 172.21.15.5:0/1267393344 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:42:44.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.389+0000 7f76f9a00700 1 --2- 172.21.15.5:0/1267393344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76f410f380 0x7f76f419fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:44.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.390+0000 7f76f9000700 1 -- 172.21.15.5:0/1267393344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76f4102180 msgr2=0x7f76f419b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:44.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.390+0000 7f76f3e00700 1 --2- 172.21.15.5:0/1267393344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76f4102180 0x7f76f419b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:44.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.390+0000 7f76f9000700 1 --2- 172.21.15.5:0/1267393344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76f4102180 0x7f76f419b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:44.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.390+0000 7f76f9000700 1 -- 172.21.15.5:0/1267393344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76f410f380 msgr2=0x7f76f419fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:44.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.390+0000 7f76f9000700 1 --2- 172.21.15.5:0/1267393344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76f410f380 0x7f76f419fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:44.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.390+0000 7f76f9000700 1 -- 172.21.15.5:0/1267393344 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f41a0340 con 0x7f76f41017d0 2024-04-24T06:42:44.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.390+0000 7f76f3e00700 1 --2- 172.21.15.5:0/1267393344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76f4102180 0x7f76f419b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:42:44.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.390+0000 7f76f9000700 1 --2- 172.21.15.5:0/1267393344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f41017d0 0x7f76f419b330 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f76e800d980 tx=0x7f76e800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:44.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.390+0000 7f76f1600700 1 -- 172.21.15.5:0/1267393344 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e8016070 con 0x7f76f41017d0 2024-04-24T06:42:44.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.390+0000 7f76f1600700 1 -- 172.21.15.5:0/1267393344 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76e8004d10 con 0x7f76f41017d0 2024-04-24T06:42:44.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.390+0000 7f76fa400700 1 -- 172.21.15.5:0/1267393344 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76f41a0630 con 0x7f76f41017d0 2024-04-24T06:42:44.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.390+0000 7f76f1600700 1 -- 172.21.15.5:0/1267393344 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e80055f0 con 0x7f76f41017d0 2024-04-24T06:42:44.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.391+0000 7f76fa400700 1 -- 172.21.15.5:0/1267393344 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76f41a7eb0 con 0x7f76f41017d0 2024-04-24T06:42:44.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.392+0000 7f76fa400700 1 -- 172.21.15.5:0/1267393344 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76f419c690 con 0x7f76f41017d0 2024-04-24T06:42:44.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.394+0000 7f76f1600700 1 -- 172.21.15.5:0/1267393344 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f76e8004750 con 0x7f76f41017d0 2024-04-24T06:42:44.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.396+0000 7f76f1600700 1 --2- 172.21.15.5:0/1267393344 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f76dc0a3d50 0x7f76dc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:44.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.396+0000 7f76f1600700 1 -- 172.21.15.5:0/1267393344 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f76e80d0360 con 0x7f76f41017d0 2024-04-24T06:42:44.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.397+0000 7f76f3e00700 1 --2- 172.21.15.5:0/1267393344 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f76dc0a3d50 0x7f76dc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:44.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.397+0000 7f76f3e00700 1 --2- 172.21.15.5:0/1267393344 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f76dc0a3d50 0x7f76dc0a6210 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f76f40ff850 tx=0x7f76e0009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:44.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.403+0000 7f76f1600700 1 -- 172.21.15.5:0/1267393344 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f76e801f020 con 0x7f76f41017d0 2024-04-24T06:42:44.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.645+0000 7f76fa400700 1 -- 172.21.15.5:0/1267393344 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f76f402dfe0 con 0x7f76f41017d0 2024-04-24T06:42:44.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.646+0000 7f76f1600700 1 -- 172.21.15.5:0/1267393344 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f76e809cba0 con 0x7f76f41017d0 2024-04-24T06:42:44.648 INFO:teuthology.orchestra.run.smithi005.stdout:34359738456 2024-04-24T06:42:44.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.648+0000 7f76fa400700 1 -- 172.21.15.5:0/1267393344 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f76dc0a3d50 msgr2=0x7f76dc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:44.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.648+0000 7f76fa400700 1 --2- 172.21.15.5:0/1267393344 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f76dc0a3d50 0x7f76dc0a6210 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f76f40ff850 tx=0x7f76e0009310 comp rx=0 tx=0).stop 2024-04-24T06:42:44.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.648+0000 7f76fa400700 1 -- 172.21.15.5:0/1267393344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f41017d0 msgr2=0x7f76f419b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:44.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.648+0000 7f76fa400700 1 --2- 172.21.15.5:0/1267393344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f41017d0 0x7f76f419b330 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f76e800d980 tx=0x7f76e800dc90 comp rx=0 tx=0).stop 2024-04-24T06:42:44.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.648+0000 7f76fa400700 1 -- 172.21.15.5:0/1267393344 shutdown_connections 2024-04-24T06:42:44.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.648+0000 7f76fa400700 1 --2- 172.21.15.5:0/1267393344 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f76dc0a3d50 0x7f76dc0a6210 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:44.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.648+0000 7f76fa400700 1 --2- 172.21.15.5:0/1267393344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76f410f380 0x7f76f419fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:44.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.648+0000 7f76fa400700 1 --2- 172.21.15.5:0/1267393344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f41017d0 0x7f76f419b330 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:44.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.648+0000 7f76fa400700 1 --2- 172.21.15.5:0/1267393344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76f4102180 0x7f76f419b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:44.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.648+0000 7f76fa400700 1 -- 172.21.15.5:0/1267393344 >> 172.21.15.5:0/1267393344 conn(0x7f76f40fd410 msgr2=0x7f76f4110580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:44.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.649+0000 7f76fa400700 1 -- 172.21.15.5:0/1267393344 shutdown_connections 2024-04-24T06:42:44.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.649+0000 7f76fa400700 1 -- 172.21.15.5:0/1267393344 wait complete. 2024-04-24T06:42:45.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:44 smithi005 ceph-mon[27798]: pgmap v349: 97 pgs: 3 peering, 94 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T06:42:45.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:44 smithi005 ceph-mon[27798]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-04-24T06:42:45.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:44 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1267393344' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-24T06:42:45.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:44 smithi137 ceph-mon[33017]: pgmap v349: 97 pgs: 3 peering, 94 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T06:42:45.453 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:44 smithi137 ceph-mon[33017]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-04-24T06:42:45.453 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:44 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/1267393344' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-24T06:42:45.454 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:44 smithi037 ceph-mon[32864]: pgmap v349: 97 pgs: 3 peering, 94 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T06:42:45.454 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:44 smithi037 ceph-mon[32864]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-04-24T06:42:45.454 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:44 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/1267393344' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-24T06:42:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.459+0000 7fd476000700 1 -- 172.21.15.5:0/816101934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd47010a040 msgr2=0x7fd470111c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:45.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.459+0000 7fd476000700 1 --2- 172.21.15.5:0/816101934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd47010a040 0x7fd470111c10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd46800b0d0 tx=0x7fd46800b3e0 comp rx=0 tx=0).stop 2024-04-24T06:42:45.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.459+0000 7fd476000700 1 -- 172.21.15.5:0/816101934 shutdown_connections 2024-04-24T06:42:45.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.459+0000 7fd476000700 1 --2- 172.21.15.5:0/816101934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd47010a040 0x7fd470111c10 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:45.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.459+0000 7fd476000700 1 --2- 172.21.15.5:0/816101934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd470104f60 0x7fd470105340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:45.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.459+0000 7fd476000700 1 --2- 172.21.15.5:0/816101934 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd470105910 0x7fd470109960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:45.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.459+0000 7fd476000700 1 -- 172.21.15.5:0/816101934 >> 172.21.15.5:0/816101934 conn(0x7fd470100830 msgr2=0x7fd470102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:45.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.460+0000 7fd476000700 1 -- 172.21.15.5:0/816101934 shutdown_connections 2024-04-24T06:42:45.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.460+0000 7fd476000700 1 -- 172.21.15.5:0/816101934 wait complete. 2024-04-24T06:42:45.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.461+0000 7fd476000700 1 Processor -- start 2024-04-24T06:42:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.461+0000 7fd476000700 1 -- start start 2024-04-24T06:42:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.461+0000 7fd476000700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd470104f60 0x7fd47019b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.461+0000 7fd476000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd470105910 0x7fd47019b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.461+0000 7fd476000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd47010a040 0x7fd47019fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.461+0000 7fd476000700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd470115080 con 0x7fd470104f60 2024-04-24T06:42:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.461+0000 7fd476000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd470114f00 con 0x7fd47010a040 2024-04-24T06:42:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.461+0000 7fd476000700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd470115200 con 0x7fd470105910 2024-04-24T06:42:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.461+0000 7fd46f400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd470104f60 0x7fd47019b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.461+0000 7fd46f400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd470104f60 0x7fd47019b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55972/0 (socket says 172.21.15.5:55972) 2024-04-24T06:42:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.462+0000 7fd46f400700 1 -- 172.21.15.5:0/2637123935 learned_addr learned my addr 172.21.15.5:0/2637123935 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:42:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.462+0000 7fd46ea00700 1 --2- 172.21.15.5:0/2637123935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd470105910 0x7fd47019b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:45.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.462+0000 7fd46fe00700 1 --2- 172.21.15.5:0/2637123935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd47010a040 0x7fd47019fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:45.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.462+0000 7fd46f400700 1 -- 172.21.15.5:0/2637123935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd470105910 msgr2=0x7fd47019b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:45.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.462+0000 7fd46f400700 1 --2- 172.21.15.5:0/2637123935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd470105910 0x7fd47019b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:45.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.462+0000 7fd46f400700 1 -- 172.21.15.5:0/2637123935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd47010a040 msgr2=0x7fd47019fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:45.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.462+0000 7fd46f400700 1 --2- 172.21.15.5:0/2637123935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd47010a040 0x7fd47019fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:45.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.462+0000 7fd46f400700 1 -- 172.21.15.5:0/2637123935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4701a0460 con 0x7fd470104f60 2024-04-24T06:42:45.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.462+0000 7fd46ea00700 1 --2- 172.21.15.5:0/2637123935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd470105910 0x7fd47019b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:42:45.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.462+0000 7fd46fe00700 1 --2- 172.21.15.5:0/2637123935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd47010a040 0x7fd47019fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:42:45.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.462+0000 7fd46f400700 1 --2- 172.21.15.5:0/2637123935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd470104f60 0x7fd47019b450 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd46000e9a0 tx=0x7fd46000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:45.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.463+0000 7fd467e00700 1 -- 172.21.15.5:0/2637123935 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46000ca10 con 0x7fd470104f60 2024-04-24T06:42:45.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.463+0000 7fd476000700 1 -- 172.21.15.5:0/2637123935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4701a0750 con 0x7fd470104f60 2024-04-24T06:42:45.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.463+0000 7fd467e00700 1 -- 172.21.15.5:0/2637123935 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd46000cbb0 con 0x7fd470104f60 2024-04-24T06:42:45.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.463+0000 7fd467e00700 1 -- 172.21.15.5:0/2637123935 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd460018680 con 0x7fd470104f60 2024-04-24T06:42:45.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.463+0000 7fd476000700 1 -- 172.21.15.5:0/2637123935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4701a7fd0 con 0x7fd470104f60 2024-04-24T06:42:45.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.464+0000 7fd476000700 1 -- 172.21.15.5:0/2637123935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4700697e0 con 0x7fd470104f60 2024-04-24T06:42:45.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.466+0000 7fd467e00700 1 -- 172.21.15.5:0/2637123935 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7fd460010430 con 0x7fd470104f60 2024-04-24T06:42:45.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.468+0000 7fd467e00700 1 --2- 172.21.15.5:0/2637123935 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fd4540a3cd0 0x7fd4540a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:45.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.468+0000 7fd467e00700 1 -- 172.21.15.5:0/2637123935 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fd4600d06d0 con 0x7fd470104f60 2024-04-24T06:42:45.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.468+0000 7fd46ea00700 1 --2- 172.21.15.5:0/2637123935 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fd4540a3cd0 0x7fd4540a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:45.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.469+0000 7fd46ea00700 1 --2- 172.21.15.5:0/2637123935 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fd4540a3cd0 0x7fd4540a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd47010b540 tx=0x7fd45c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:45.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.472+0000 7fd467e00700 1 -- 172.21.15.5:0/2637123935 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd46009d000 con 0x7fd470104f60 2024-04-24T06:42:45.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.714+0000 7fd476000700 1 -- 172.21.15.5:0/2637123935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fd4701a8330 con 0x7fd470104f60 2024-04-24T06:42:45.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.715+0000 7fd467e00700 1 -- 172.21.15.5:0/2637123935 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd46009c900 con 0x7fd470104f60 2024-04-24T06:42:45.527 INFO:teuthology.orchestra.run.smithi005.stdout:223338299440 2024-04-24T06:42:45.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.719+0000 7fd476000700 1 -- 172.21.15.5:0/2637123935 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fd4540a3cd0 msgr2=0x7fd4540a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:45.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.719+0000 7fd476000700 1 --2- 172.21.15.5:0/2637123935 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fd4540a3cd0 0x7fd4540a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd47010b540 tx=0x7fd45c009260 comp rx=0 tx=0).stop 2024-04-24T06:42:45.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.719+0000 7fd476000700 1 -- 172.21.15.5:0/2637123935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd470104f60 msgr2=0x7fd47019b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:45.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.719+0000 7fd476000700 1 --2- 172.21.15.5:0/2637123935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd470104f60 0x7fd47019b450 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd46000e9a0 tx=0x7fd46000ecb0 comp rx=0 tx=0).stop 2024-04-24T06:42:45.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.719+0000 7fd476000700 1 -- 172.21.15.5:0/2637123935 shutdown_connections 2024-04-24T06:42:45.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.719+0000 7fd476000700 1 --2- 172.21.15.5:0/2637123935 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fd4540a3cd0 0x7fd4540a6190 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:45.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.719+0000 7fd476000700 1 --2- 172.21.15.5:0/2637123935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd47010a040 0x7fd47019fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:45.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.719+0000 7fd476000700 1 --2- 172.21.15.5:0/2637123935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd470104f60 0x7fd47019b450 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:45.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.719+0000 7fd476000700 1 --2- 172.21.15.5:0/2637123935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd470105910 0x7fd47019b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:45.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.719+0000 7fd476000700 1 -- 172.21.15.5:0/2637123935 >> 172.21.15.5:0/2637123935 conn(0x7fd470100830 msgr2=0x7fd470101240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:45.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.719+0000 7fd476000700 1 -- 172.21.15.5:0/2637123935 shutdown_connections 2024-04-24T06:42:45.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:40.720+0000 7fd476000700 1 -- 172.21.15.5:0/2637123935 wait complete. 2024-04-24T06:42:46.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:45 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/2880439470' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-24T06:42:46.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:45 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/2368064069' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-24T06:42:46.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.496+0000 7f967c600700 1 -- 172.21.15.5:0/3169998890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9674102b40 msgr2=0x7f9674102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:46.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.496+0000 7f967c600700 1 --2- 172.21.15.5:0/3169998890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9674102b40 0x7f9674102fc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9660009990 tx=0x7f9660009ca0 comp rx=0 tx=0).stop 2024-04-24T06:42:46.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.496+0000 7f967c600700 1 -- 172.21.15.5:0/3169998890 shutdown_connections 2024-04-24T06:42:46.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.496+0000 7f967c600700 1 --2- 172.21.15.5:0/3169998890 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9674103500 0x7f967410f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:46.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.496+0000 7f967c600700 1 --2- 172.21.15.5:0/3169998890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9674108b90 0x7f9674108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:46.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.496+0000 7f967c600700 1 --2- 172.21.15.5:0/3169998890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9674102b40 0x7f9674102fc0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:46.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.496+0000 7f967c600700 1 -- 172.21.15.5:0/3169998890 >> 172.21.15.5:0/3169998890 conn(0x7f9674079c50 msgr2=0x7f967407a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:46.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.496+0000 7f967c600700 1 -- 172.21.15.5:0/3169998890 shutdown_connections 2024-04-24T06:42:46.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.496+0000 7f967c600700 1 -- 172.21.15.5:0/3169998890 wait complete. 2024-04-24T06:42:46.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.498+0000 7f967c600700 1 Processor -- start 2024-04-24T06:42:46.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.498+0000 7f967c600700 1 -- start start 2024-04-24T06:42:46.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.498+0000 7f967c600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9674102b40 0x7f96741992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:46.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.498+0000 7f967c600700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9674103500 0x7f9674199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:46.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.498+0000 7f967c600700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9674108b90 0x7f967419db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:46.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.498+0000 7f967c600700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9674112e00 con 0x7f9674103500 2024-04-24T06:42:46.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.498+0000 7f967c600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9674112c80 con 0x7f9674102b40 2024-04-24T06:42:46.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.498+0000 7f967c600700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9674112f80 con 0x7f9674108b90 2024-04-24T06:42:46.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.499+0000 7f9679200700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9674103500 0x7f9674199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:46.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.499+0000 7f9679200700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9674103500 0x7f9674199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43898/0 (socket says 172.21.15.5:43898) 2024-04-24T06:42:46.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.499+0000 7f9679200700 1 -- 172.21.15.5:0/2368064069 learned_addr learned my addr 172.21.15.5:0/2368064069 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:42:46.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.499+0000 7f967a600700 1 --2- 172.21.15.5:0/2368064069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9674108b90 0x7f967419db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:46.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.499+0000 7f9679c00700 1 --2- 172.21.15.5:0/2368064069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9674102b40 0x7f96741992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:46.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.499+0000 7f9679200700 1 -- 172.21.15.5:0/2368064069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9674108b90 msgr2=0x7f967419db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:46.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.499+0000 7f9679200700 1 --2- 172.21.15.5:0/2368064069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9674108b90 0x7f967419db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:46.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.499+0000 7f9679200700 1 -- 172.21.15.5:0/2368064069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9674102b40 msgr2=0x7f96741992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:46.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.499+0000 7f9679200700 1 --2- 172.21.15.5:0/2368064069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9674102b40 0x7f96741992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:46.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.499+0000 7f9679200700 1 -- 172.21.15.5:0/2368064069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f967419e2b0 con 0x7f9674103500 2024-04-24T06:42:46.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.499+0000 7f9679c00700 1 --2- 172.21.15.5:0/2368064069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9674102b40 0x7f96741992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:42:46.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.500+0000 7f9679200700 1 --2- 172.21.15.5:0/2368064069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9674103500 0x7f9674199830 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9660000c00 tx=0x7f966000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.500+0000 7f9672000700 1 -- 172.21.15.5:0/2368064069 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9660027070 con 0x7f9674103500 2024-04-24T06:42:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.500+0000 7f9672000700 1 -- 172.21.15.5:0/2368064069 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9660003d70 con 0x7f9674103500 2024-04-24T06:42:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.500+0000 7f967c600700 1 -- 172.21.15.5:0/2368064069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f967419e540 con 0x7f9674103500 2024-04-24T06:42:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.500+0000 7f9672000700 1 -- 172.21.15.5:0/2368064069 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f966000f460 con 0x7f9674103500 2024-04-24T06:42:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.500+0000 7f967c600700 1 -- 172.21.15.5:0/2368064069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96741a5e20 con 0x7f9674103500 2024-04-24T06:42:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.502+0000 7f967c600700 1 -- 172.21.15.5:0/2368064069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96740697e0 con 0x7f9674103500 2024-04-24T06:42:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.503+0000 7f9672000700 1 -- 172.21.15.5:0/2368064069 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f966002a070 con 0x7f9674103500 2024-04-24T06:42:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.504+0000 7f9672000700 1 --2- 172.21.15.5:0/2368064069 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f965c0a3d50 0x7f965c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.505+0000 7f9672000700 1 -- 172.21.15.5:0/2368064069 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f96600d1340 con 0x7f9674103500 2024-04-24T06:42:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.505+0000 7f9679c00700 1 --2- 172.21.15.5:0/2368064069 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f965c0a3d50 0x7f965c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.506+0000 7f9679c00700 1 --2- 172.21.15.5:0/2368064069 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f965c0a3d50 0x7f965c0a6210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9668007f50 tx=0x7f96680073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.509+0000 7f9672000700 1 -- 172.21.15.5:0/2368064069 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9660013070 con 0x7f9674103500 2024-04-24T06:42:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.737+0000 7f967c600700 1 -- 172.21.15.5:0/2368064069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f967402dfe0 con 0x7f9674103500 2024-04-24T06:42:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.737+0000 7f9672000700 1 -- 172.21.15.5:0/2368064069 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f966009dd30 con 0x7f9674103500 2024-04-24T06:42:46.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.741+0000 7f967c600700 1 -- 172.21.15.5:0/2368064069 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f965c0a3d50 msgr2=0x7f965c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:46.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.741+0000 7f967c600700 1 --2- 172.21.15.5:0/2368064069 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f965c0a3d50 0x7f965c0a6210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9668007f50 tx=0x7f96680073d0 comp rx=0 tx=0).stop 2024-04-24T06:42:46.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.741+0000 7f967c600700 1 -- 172.21.15.5:0/2368064069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9674103500 msgr2=0x7f9674199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:46.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.741+0000 7f967c600700 1 --2- 172.21.15.5:0/2368064069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9674103500 0x7f9674199830 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9660000c00 tx=0x7f966000bab0 comp rx=0 tx=0).stop 2024-04-24T06:42:46.164 INFO:teuthology.orchestra.run.smithi005.stdout:201863462967 2024-04-24T06:42:46.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.741+0000 7f967c600700 1 -- 172.21.15.5:0/2368064069 shutdown_connections 2024-04-24T06:42:46.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.741+0000 7f967c600700 1 --2- 172.21.15.5:0/2368064069 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f965c0a3d50 0x7f965c0a6210 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:46.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.741+0000 7f967c600700 1 --2- 172.21.15.5:0/2368064069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9674108b90 0x7f967419db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:46.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.741+0000 7f967c600700 1 --2- 172.21.15.5:0/2368064069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9674102b40 0x7f96741992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:46.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.741+0000 7f967c600700 1 --2- 172.21.15.5:0/2368064069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9674103500 0x7f9674199830 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:46.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.741+0000 7f967c600700 1 -- 172.21.15.5:0/2368064069 >> 172.21.15.5:0/2368064069 conn(0x7f9674079c50 msgr2=0x7f9674101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:46.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.742+0000 7f967c600700 1 -- 172.21.15.5:0/2368064069 shutdown_connections 2024-04-24T06:42:46.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:45.742+0000 7f967c600700 1 -- 172.21.15.5:0/2368064069 wait complete. 2024-04-24T06:42:46.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:45 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/2880439470' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-24T06:42:46.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:45 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/2368064069' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-24T06:42:46.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:45 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/2880439470' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-24T06:42:46.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:45 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/2368064069' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-24T06:42:46.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:46.760+0000 7f6bb1800700 1 -- 172.21.15.5:0/3855057964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bac10d010 msgr2=0x7f6bac10f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:46.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:46.760+0000 7f6bb1800700 1 --2- 172.21.15.5:0/3855057964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bac10d010 0x7f6bac10f4d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6ba400b0a0 tx=0x7f6ba400b3b0 comp rx=0 tx=0).stop 2024-04-24T06:42:46.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:46.760+0000 7f6bb1800700 1 -- 172.21.15.5:0/3855057964 shutdown_connections 2024-04-24T06:42:46.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:46.760+0000 7f6bb1800700 1 --2- 172.21.15.5:0/3855057964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bac10d010 0x7f6bac10f4d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:46.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:46.760+0000 7f6bb1800700 1 --2- 172.21.15.5:0/3855057964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bac103be0 0x7f6bac103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:46.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:46.760+0000 7f6bb1800700 1 --2- 172.21.15.5:0/3855057964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6bac104500 0x7f6bac10c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:46.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:46.760+0000 7f6bb1800700 1 -- 172.21.15.5:0/3855057964 >> 172.21.15.5:0/3855057964 conn(0x7f6bac0fd410 msgr2=0x7f6bac0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:46.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:46.760+0000 7f6bb1800700 1 -- 172.21.15.5:0/3855057964 shutdown_connections 2024-04-24T06:42:46.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:46.760+0000 7f6bb1800700 1 -- 172.21.15.5:0/3855057964 wait complete. 2024-04-24T06:42:46.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:46.761+0000 7f6bb1800700 1 Processor -- start 2024-04-24T06:42:46.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:46.761+0000 7f6bb1800700 1 -- start start 2024-04-24T06:42:46.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:46.762+0000 7f6bb1800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6bac103be0 0x7f6bac19b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:46.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:46.762+0000 7f6bb1800700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bac104500 0x7f6bac19b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:46.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:46.762+0000 7f6bb1800700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bac10d010 0x7f6bac19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:46.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:46.762+0000 7f6bb1800700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6bac112d50 con 0x7f6bac104500 2024-04-24T06:42:46.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:46.762+0000 7f6bb1800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6bac112bd0 con 0x7f6bac103be0 2024-04-24T06:42:46.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:46.762+0000 7f6bb1800700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6bac112ed0 con 0x7f6bac10d010 2024-04-24T06:42:46.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:46.762+0000 7f6bab400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bac104500 0x7f6bac19b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:46.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:46.762+0000 7f6babe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6bac103be0 0x7f6bac19b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:46.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:46.762+0000 7f6bb0e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bac10d010 0x7f6bac19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:46.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:46.762+0000 7f6bab400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bac104500 0x7f6bac19b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43912/0 (socket says 172.21.15.5:43912) 2024-04-24T06:42:46.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:46.762+0000 7f6bab400700 1 -- 172.21.15.5:0/4131557166 learned_addr learned my addr 172.21.15.5:0/4131557166 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:42:46.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:46.763+0000 7f6babe00700 1 -- 172.21.15.5:0/4131557166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bac10d010 msgr2=0x7f6bac19fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:46.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:46.763+0000 7f6babe00700 1 --2- 172.21.15.5:0/4131557166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bac10d010 0x7f6bac19fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:46.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:46.763+0000 7f6babe00700 1 -- 172.21.15.5:0/4131557166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bac104500 msgr2=0x7f6bac19b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:46.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:46.763+0000 7f6babe00700 1 --2- 172.21.15.5:0/4131557166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bac104500 0x7f6bac19b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:46.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:46.763+0000 7f6babe00700 1 -- 172.21.15.5:0/4131557166 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bac1a03d0 con 0x7f6bac103be0 2024-04-24T06:42:46.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:46.763+0000 7f6babe00700 1 --2- 172.21.15.5:0/4131557166 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6bac103be0 0x7f6bac19b3c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6ba000d750 tx=0x7f6ba000da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:46.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:46.763+0000 7f6ba8c00700 1 -- 172.21.15.5:0/4131557166 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0014030 con 0x7f6bac103be0 2024-04-24T06:42:46.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:46.763+0000 7f6ba8c00700 1 -- 172.21.15.5:0/4131557166 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ba0004500 con 0x7f6bac103be0 2024-04-24T06:42:46.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:46.763+0000 7f6bb1800700 1 -- 172.21.15.5:0/4131557166 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bac1a06c0 con 0x7f6bac103be0 2024-04-24T06:42:46.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:46.763+0000 7f6ba8c00700 1 -- 172.21.15.5:0/4131557166 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0005420 con 0x7f6bac103be0 2024-04-24T06:42:46.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:46.763+0000 7f6bb1800700 1 -- 172.21.15.5:0/4131557166 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bac1a7f40 con 0x7f6bac103be0 2024-04-24T06:42:46.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:46.765+0000 7f6bb1800700 1 -- 172.21.15.5:0/4131557166 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bac0697e0 con 0x7f6bac103be0 2024-04-24T06:42:46.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:46.767+0000 7f6ba8c00700 1 -- 172.21.15.5:0/4131557166 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f6ba0004750 con 0x7f6bac103be0 2024-04-24T06:42:46.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:46.768+0000 7f6ba8c00700 1 --2- 172.21.15.5:0/4131557166 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f6b940a3cd0 0x7f6b940a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:46.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:46.768+0000 7f6ba8c00700 1 -- 172.21.15.5:0/4131557166 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f6ba00d0220 con 0x7f6bac103be0 2024-04-24T06:42:46.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:46.768+0000 7f6bab400700 1 --2- 172.21.15.5:0/4131557166 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f6b940a3cd0 0x7f6b940a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:46.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:46.769+0000 7f6bab400700 1 --2- 172.21.15.5:0/4131557166 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f6b940a3cd0 0x7f6b940a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6bac10e2c0 tx=0x7f6b98009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:46.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:46.772+0000 7f6ba8c00700 1 -- 172.21.15.5:0/4131557166 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6ba009ca60 con 0x7f6bac103be0 2024-04-24T06:42:47.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:47.000+0000 7f6bb1800700 1 -- 172.21.15.5:0/4131557166 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f6bac1a8110 con 0x7f6bac103be0 2024-04-24T06:42:47.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:47.001+0000 7f6ba8c00700 1 -- 172.21.15.5:0/4131557166 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6ba009c360 con 0x7f6bac103be0 2024-04-24T06:42:47.003 INFO:teuthology.orchestra.run.smithi005.stdout:55834574931 2024-04-24T06:42:47.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:47.003+0000 7f6bb1800700 1 -- 172.21.15.5:0/4131557166 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f6b940a3cd0 msgr2=0x7f6b940a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:47.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:47.003+0000 7f6bb1800700 1 --2- 172.21.15.5:0/4131557166 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f6b940a3cd0 0x7f6b940a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6bac10e2c0 tx=0x7f6b98009260 comp rx=0 tx=0).stop 2024-04-24T06:42:47.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:47.003+0000 7f6bb1800700 1 -- 172.21.15.5:0/4131557166 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6bac103be0 msgr2=0x7f6bac19b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:47.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:47.003+0000 7f6bb1800700 1 --2- 172.21.15.5:0/4131557166 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6bac103be0 0x7f6bac19b3c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6ba000d750 tx=0x7f6ba000da60 comp rx=0 tx=0).stop 2024-04-24T06:42:47.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:47.003+0000 7f6bb1800700 1 -- 172.21.15.5:0/4131557166 shutdown_connections 2024-04-24T06:42:47.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:47.004+0000 7f6bb1800700 1 --2- 172.21.15.5:0/4131557166 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f6b940a3cd0 0x7f6b940a6190 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:47.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:47.004+0000 7f6bb1800700 1 --2- 172.21.15.5:0/4131557166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bac10d010 0x7f6bac19fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:47.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:47.004+0000 7f6bb1800700 1 --2- 172.21.15.5:0/4131557166 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6bac103be0 0x7f6bac19b3c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:47.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:47.004+0000 7f6bb1800700 1 --2- 172.21.15.5:0/4131557166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bac104500 0x7f6bac19b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:47.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:47.004+0000 7f6bb1800700 1 -- 172.21.15.5:0/4131557166 >> 172.21.15.5:0/4131557166 conn(0x7f6bac0fd410 msgr2=0x7f6bac10f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:47.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:47.004+0000 7f6bb1800700 1 -- 172.21.15.5:0/4131557166 shutdown_connections 2024-04-24T06:42:47.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:47.004+0000 7f6bb1800700 1 -- 172.21.15.5:0/4131557166 wait complete. 2024-04-24T06:42:47.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:46 smithi137 ceph-mon[33017]: pgmap v350: 97 pgs: 3 peering, 94 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-24T06:42:47.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:46 smithi005 ceph-mon[27798]: pgmap v350: 97 pgs: 3 peering, 94 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-24T06:42:47.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:46 smithi037 ceph-mon[32864]: pgmap v350: 97 pgs: 3 peering, 94 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-24T06:42:47.368 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299436 got 223338299440 for osd.7 2024-04-24T06:42:47.368 DEBUG:teuthology.parallel:result is None 2024-04-24T06:42:47.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.653+0000 7f7861600700 1 -- 172.21.15.5:0/2407008846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f785c074e10 msgr2=0x7f785c0751f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:47.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.653+0000 7f7861600700 1 --2- 172.21.15.5:0/2407008846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f785c074e10 0x7f785c0751f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f784c009990 tx=0x7f784c009ca0 comp rx=0 tx=0).stop 2024-04-24T06:42:47.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.654+0000 7f7861600700 1 -- 172.21.15.5:0/2407008846 shutdown_connections 2024-04-24T06:42:47.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.654+0000 7f7861600700 1 --2- 172.21.15.5:0/2407008846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f785c10f2c0 0x7f785c1116b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:47.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.654+0000 7f7861600700 1 --2- 172.21.15.5:0/2407008846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f785c074e10 0x7f785c0751f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:47.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.654+0000 7f7861600700 1 --2- 172.21.15.5:0/2407008846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f785c075730 0x7f785c10ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:47.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.654+0000 7f7861600700 1 -- 172.21.15.5:0/2407008846 >> 172.21.15.5:0/2407008846 conn(0x7f785c0fe510 msgr2=0x7f785c100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:47.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.654+0000 7f7861600700 1 -- 172.21.15.5:0/2407008846 shutdown_connections 2024-04-24T06:42:47.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.654+0000 7f7861600700 1 -- 172.21.15.5:0/2407008846 wait complete. 2024-04-24T06:42:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.655+0000 7f7861600700 1 Processor -- start 2024-04-24T06:42:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.655+0000 7f7861600700 1 -- start start 2024-04-24T06:42:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.655+0000 7f7861600700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f785c074e10 0x7f785c19b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.655+0000 7f7861600700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f785c075730 0x7f785c19b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.655+0000 7f7861600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f785c10f2c0 0x7f785c19fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.655+0000 7f7861600700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f785c114f70 con 0x7f785c074e10 2024-04-24T06:42:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.655+0000 7f7861600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f785c114df0 con 0x7f785c10f2c0 2024-04-24T06:42:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.655+0000 7f7861600700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f785c1150f0 con 0x7f785c075730 2024-04-24T06:42:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.656+0000 7f785be00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f785c074e10 0x7f785c19b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.656+0000 7f785b400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f785c075730 0x7f785c19b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.656+0000 7f785be00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f785c074e10 0x7f785c19b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43872/0 (socket says 172.21.15.5:43872) 2024-04-24T06:42:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.656+0000 7f785be00700 1 -- 172.21.15.5:0/2880439470 learned_addr learned my addr 172.21.15.5:0/2880439470 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:42:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.656+0000 7f7860c00700 1 --2- 172.21.15.5:0/2880439470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f785c10f2c0 0x7f785c19fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.656+0000 7f785b400700 1 -- 172.21.15.5:0/2880439470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f785c10f2c0 msgr2=0x7f785c19fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.656+0000 7f785b400700 1 --2- 172.21.15.5:0/2880439470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f785c10f2c0 0x7f785c19fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.656+0000 7f785b400700 1 -- 172.21.15.5:0/2880439470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f785c074e10 msgr2=0x7f785c19b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.656+0000 7f785b400700 1 --2- 172.21.15.5:0/2880439470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f785c074e10 0x7f785c19b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.656+0000 7f785b400700 1 -- 172.21.15.5:0/2880439470 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f785c1a03b0 con 0x7f785c075730 2024-04-24T06:42:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.657+0000 7f785b400700 1 --2- 172.21.15.5:0/2880439470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f785c075730 0x7f785c19b8e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f784800e9f0 tx=0x7f784800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.657+0000 7f7858c00700 1 -- 172.21.15.5:0/2880439470 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7848004220 con 0x7f785c075730 2024-04-24T06:42:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.657+0000 7f7858c00700 1 -- 172.21.15.5:0/2880439470 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7848004d10 con 0x7f785c075730 2024-04-24T06:42:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.657+0000 7f7861600700 1 -- 172.21.15.5:0/2880439470 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f785c1a06a0 con 0x7f785c075730 2024-04-24T06:42:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.657+0000 7f7858c00700 1 -- 172.21.15.5:0/2880439470 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7848005020 con 0x7f785c075730 2024-04-24T06:42:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.657+0000 7f7861600700 1 -- 172.21.15.5:0/2880439470 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f785c1a7f20 con 0x7f785c075730 2024-04-24T06:42:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.658+0000 7f7861600700 1 -- 172.21.15.5:0/2880439470 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f785c0697e0 con 0x7f785c075730 2024-04-24T06:42:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.660+0000 7f7858c00700 1 -- 172.21.15.5:0/2880439470 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f7848004750 con 0x7f785c075730 2024-04-24T06:42:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.662+0000 7f7858c00700 1 --2- 172.21.15.5:0/2880439470 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f78400a3cd0 0x7f78400a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.663+0000 7f7858c00700 1 -- 172.21.15.5:0/2880439470 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f78480d0490 con 0x7f785c075730 2024-04-24T06:42:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.663+0000 7f785be00700 1 --2- 172.21.15.5:0/2880439470 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f78400a3cd0 0x7f78400a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.663+0000 7f785be00700 1 --2- 172.21.15.5:0/2880439470 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f78400a3cd0 0x7f78400a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f784c009750 tx=0x7f784c00b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.666+0000 7f7858c00700 1 -- 172.21.15.5:0/2880439470 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f784809ccd0 con 0x7f785c075730 2024-04-24T06:42:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.922+0000 7f7861600700 1 -- 172.21.15.5:0/2880439470 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f785c19c700 con 0x7f785c075730 2024-04-24T06:42:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.923+0000 7f7858c00700 1 -- 172.21.15.5:0/2880439470 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f784809c5d0 con 0x7f785c075730 2024-04-24T06:42:47.496 INFO:teuthology.orchestra.run.smithi005.stdout:347892351003 2024-04-24T06:42:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.927+0000 7f7861600700 1 -- 172.21.15.5:0/2880439470 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f78400a3cd0 msgr2=0x7f78400a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:47.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.927+0000 7f7861600700 1 --2- 172.21.15.5:0/2880439470 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f78400a3cd0 0x7f78400a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f784c009750 tx=0x7f784c00b280 comp rx=0 tx=0).stop 2024-04-24T06:42:47.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.927+0000 7f7861600700 1 -- 172.21.15.5:0/2880439470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f785c075730 msgr2=0x7f785c19b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:47.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.927+0000 7f7861600700 1 --2- 172.21.15.5:0/2880439470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f785c075730 0x7f785c19b8e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f784800e9f0 tx=0x7f784800ed00 comp rx=0 tx=0).stop 2024-04-24T06:42:47.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.927+0000 7f7861600700 1 -- 172.21.15.5:0/2880439470 shutdown_connections 2024-04-24T06:42:47.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.927+0000 7f7861600700 1 --2- 172.21.15.5:0/2880439470 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f78400a3cd0 0x7f78400a6190 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:47.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.927+0000 7f7861600700 1 --2- 172.21.15.5:0/2880439470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f785c10f2c0 0x7f785c19fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:47.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.927+0000 7f7861600700 1 --2- 172.21.15.5:0/2880439470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f785c074e10 0x7f785c19b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:47.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.927+0000 7f7861600700 1 --2- 172.21.15.5:0/2880439470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f785c075730 0x7f785c19b8e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:47.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.927+0000 7f7861600700 1 -- 172.21.15.5:0/2880439470 >> 172.21.15.5:0/2880439470 conn(0x7f785c0fe510 msgr2=0x7f785c0ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:47.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.927+0000 7f7861600700 1 -- 172.21.15.5:0/2880439470 shutdown_connections 2024-04-24T06:42:47.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:44.927+0000 7f7861600700 1 -- 172.21.15.5:0/2880439470 wait complete. 2024-04-24T06:42:48.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:47 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/4131557166' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-24T06:42:48.197 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:47 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/4131557166' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-24T06:42:48.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:47 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/4131557166' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-24T06:42:48.537 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892350998 got 347892351003 for osd.11 2024-04-24T06:42:48.537 DEBUG:teuthology.parallel:result is None 2024-04-24T06:42:48.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.216+0000 7f9465e00700 1 -- 172.21.15.5:0/32590584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f946006a800 msgr2=0x7f946006ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:48.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.216+0000 7f9465e00700 1 --2- 172.21.15.5:0/32590584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f946006a800 0x7f946006ac60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f944c009990 tx=0x7f944c009ca0 comp rx=0 tx=0).stop 2024-04-24T06:42:48.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.216+0000 7f9465e00700 1 -- 172.21.15.5:0/32590584 shutdown_connections 2024-04-24T06:42:48.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.216+0000 7f9465e00700 1 --2- 172.21.15.5:0/32590584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f946006b1a0 0x7f946010f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:48.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.216+0000 7f9465e00700 1 --2- 172.21.15.5:0/32590584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9460108b70 0x7f9460108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:48.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.216+0000 7f9465e00700 1 --2- 172.21.15.5:0/32590584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f946006a800 0x7f946006ac60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:48.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.216+0000 7f9465e00700 1 -- 172.21.15.5:0/32590584 >> 172.21.15.5:0/32590584 conn(0x7f94600789a0 msgr2=0x7f9460078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:48.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.216+0000 7f9465e00700 1 -- 172.21.15.5:0/32590584 shutdown_connections 2024-04-24T06:42:48.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.216+0000 7f9465e00700 1 -- 172.21.15.5:0/32590584 wait complete. 2024-04-24T06:42:48.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.217+0000 7f9465e00700 1 Processor -- start 2024-04-24T06:42:48.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.218+0000 7f9465e00700 1 -- start start 2024-04-24T06:42:48.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.218+0000 7f9465e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f946006a800 0x7f946019b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:48.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.218+0000 7f9465e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f946006b1a0 0x7f946019bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:48.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.218+0000 7f9465e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9460108b70 0x7f946019fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:48.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.218+0000 7f9465e00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9460112e30 con 0x7f946006b1a0 2024-04-24T06:42:48.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.218+0000 7f9465e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9460112cb0 con 0x7f946006a800 2024-04-24T06:42:48.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.218+0000 7f9465e00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9460112fb0 con 0x7f9460108b70 2024-04-24T06:42:48.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.218+0000 7f945e000700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f946006b1a0 0x7f946019bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:48.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.218+0000 7f945e000700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f946006b1a0 0x7f946019bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43936/0 (socket says 172.21.15.5:43936) 2024-04-24T06:42:48.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.218+0000 7f945e000700 1 -- 172.21.15.5:0/3220232830 learned_addr learned my addr 172.21.15.5:0/3220232830 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:42:48.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.219+0000 7f945f400700 1 --2- 172.21.15.5:0/3220232830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9460108b70 0x7f946019fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:48.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.219+0000 7f945ea00700 1 --2- 172.21.15.5:0/3220232830 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f946006a800 0x7f946019b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:48.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.219+0000 7f945e000700 1 -- 172.21.15.5:0/3220232830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9460108b70 msgr2=0x7f946019fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:48.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.219+0000 7f945e000700 1 --2- 172.21.15.5:0/3220232830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9460108b70 0x7f946019fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:48.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.219+0000 7f945e000700 1 -- 172.21.15.5:0/3220232830 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f946006a800 msgr2=0x7f946019b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:48.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.219+0000 7f945e000700 1 --2- 172.21.15.5:0/3220232830 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f946006a800 0x7f946019b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:48.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.219+0000 7f945e000700 1 -- 172.21.15.5:0/3220232830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94601a0590 con 0x7f946006b1a0 2024-04-24T06:42:48.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.219+0000 7f945f400700 1 --2- 172.21.15.5:0/3220232830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9460108b70 0x7f946019fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:42:48.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.219+0000 7f945e000700 1 --2- 172.21.15.5:0/3220232830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f946006b1a0 0x7f946019bac0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f944c000c00 tx=0x7f944c00bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:48.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.220+0000 7f9464a00700 1 -- 172.21.15.5:0/3220232830 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944c027070 con 0x7f946006b1a0 2024-04-24T06:42:48.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.220+0000 7f9464a00700 1 -- 172.21.15.5:0/3220232830 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f944c0056d0 con 0x7f946006b1a0 2024-04-24T06:42:48.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.220+0000 7f9465e00700 1 -- 172.21.15.5:0/3220232830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94601a0820 con 0x7f946006b1a0 2024-04-24T06:42:48.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.220+0000 7f9464a00700 1 -- 172.21.15.5:0/3220232830 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944c022e10 con 0x7f946006b1a0 2024-04-24T06:42:48.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.220+0000 7f9465e00700 1 -- 172.21.15.5:0/3220232830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94601a8100 con 0x7f946006b1a0 2024-04-24T06:42:48.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.221+0000 7f9465e00700 1 -- 172.21.15.5:0/3220232830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94601041e0 con 0x7f946006b1a0 2024-04-24T06:42:48.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.223+0000 7f9464a00700 1 -- 172.21.15.5:0/3220232830 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f944c02a070 con 0x7f946006b1a0 2024-04-24T06:42:48.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.225+0000 7f9464a00700 1 --2- 172.21.15.5:0/3220232830 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f94440a3cd0 0x7f94440a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:48.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.225+0000 7f9464a00700 1 -- 172.21.15.5:0/3220232830 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f944c0d1210 con 0x7f946006b1a0 2024-04-24T06:42:48.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.225+0000 7f945ea00700 1 --2- 172.21.15.5:0/3220232830 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f94440a3cd0 0x7f94440a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:48.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.226+0000 7f945ea00700 1 --2- 172.21.15.5:0/3220232830 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f94440a3cd0 0x7f94440a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9450009be0 tx=0x7f9450009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:48.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.230+0000 7f9464a00700 1 -- 172.21.15.5:0/3220232830 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f944c013070 con 0x7f946006b1a0 2024-04-24T06:42:48.635 INFO:teuthology.orchestra.run.smithi005.stdout:313532612640 2024-04-24T06:42:48.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.471+0000 7f9465e00700 1 -- 172.21.15.5:0/3220232830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f946002dfe0 con 0x7f946006b1a0 2024-04-24T06:42:48.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.472+0000 7f9464a00700 1 -- 172.21.15.5:0/3220232830 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f944c09dc00 con 0x7f946006b1a0 2024-04-24T06:42:48.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.477+0000 7f9465e00700 1 -- 172.21.15.5:0/3220232830 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f94440a3cd0 msgr2=0x7f94440a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:48.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.477+0000 7f9465e00700 1 --2- 172.21.15.5:0/3220232830 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f94440a3cd0 0x7f94440a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9450009be0 tx=0x7f9450009260 comp rx=0 tx=0).stop 2024-04-24T06:42:48.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.477+0000 7f9465e00700 1 -- 172.21.15.5:0/3220232830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f946006b1a0 msgr2=0x7f946019bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:48.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.477+0000 7f9465e00700 1 --2- 172.21.15.5:0/3220232830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f946006b1a0 0x7f946019bac0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f944c000c00 tx=0x7f944c00bab0 comp rx=0 tx=0).stop 2024-04-24T06:42:48.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.480+0000 7f9465e00700 1 -- 172.21.15.5:0/3220232830 shutdown_connections 2024-04-24T06:42:48.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.480+0000 7f9465e00700 1 --2- 172.21.15.5:0/3220232830 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f94440a3cd0 0x7f94440a6190 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:48.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.480+0000 7f9465e00700 1 --2- 172.21.15.5:0/3220232830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9460108b70 0x7f946019fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:48.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.480+0000 7f9465e00700 1 --2- 172.21.15.5:0/3220232830 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f946006a800 0x7f946019b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:48.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.480+0000 7f9465e00700 1 --2- 172.21.15.5:0/3220232830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f946006b1a0 0x7f946019bac0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:48.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.480+0000 7f9465e00700 1 -- 172.21.15.5:0/3220232830 >> 172.21.15.5:0/3220232830 conn(0x7f94600789a0 msgr2=0x7f946010e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:48.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.480+0000 7f9465e00700 1 -- 172.21.15.5:0/3220232830 shutdown_connections 2024-04-24T06:42:48.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:48.480+0000 7f9465e00700 1 -- 172.21.15.5:0/3220232830 wait complete. 2024-04-24T06:42:49.165 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:48 smithi005 ceph-mon[27798]: pgmap v351: 97 pgs: 3 peering, 94 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:42:49.165 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:48 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/3220232830' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-24T06:42:49.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:48 smithi137 ceph-mon[33017]: pgmap v351: 97 pgs: 3 peering, 94 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:42:49.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:48 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/3220232830' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-24T06:42:49.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:48 smithi037 ceph-mon[32864]: pgmap v351: 97 pgs: 3 peering, 94 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:42:49.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:48 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/3220232830' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-24T06:42:50.095 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:49 smithi005 ceph-mon[27798]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-04-24T06:42:50.095 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:49 smithi005 ceph-mon[27798]: Cluster is now healthy 2024-04-24T06:42:50.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:49 smithi037 ceph-mon[32864]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-04-24T06:42:50.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:49 smithi037 ceph-mon[32864]: Cluster is now healthy 2024-04-24T06:42:50.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:49 smithi137 ceph-mon[33017]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-04-24T06:42:50.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:49 smithi137 ceph-mon[33017]: Cluster is now healthy 2024-04-24T06:42:50.508 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462963 got 201863462967 for osd.6 2024-04-24T06:42:50.509 DEBUG:teuthology.parallel:result is None 2024-04-24T06:42:50.961 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724601 got 167503724605 for osd.5 2024-04-24T06:42:50.961 DEBUG:teuthology.parallel:result is None 2024-04-24T06:42:51.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.270+0000 7f5a4f000700 1 -- 172.21.15.5:0/2446926535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a48109f60 msgr2=0x7f5a48111b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:51.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.270+0000 7f5a4f000700 1 --2- 172.21.15.5:0/2446926535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a48109f60 0x7f5a48111b30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5a40009f90 tx=0x7f5a4000b330 comp rx=0 tx=0).stop 2024-04-24T06:42:51.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.270+0000 7f5a4f000700 1 -- 172.21.15.5:0/2446926535 shutdown_connections 2024-04-24T06:42:51.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.270+0000 7f5a4f000700 1 --2- 172.21.15.5:0/2446926535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a48109f60 0x7f5a48111b30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:51.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.270+0000 7f5a4f000700 1 --2- 172.21.15.5:0/2446926535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a48104e80 0x7f5a48105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:51.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.270+0000 7f5a4f000700 1 --2- 172.21.15.5:0/2446926535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a48105830 0x7f5a48109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:51.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.270+0000 7f5a4f000700 1 -- 172.21.15.5:0/2446926535 >> 172.21.15.5:0/2446926535 conn(0x7f5a48100710 msgr2=0x7f5a48102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:51.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.270+0000 7f5a4f000700 1 -- 172.21.15.5:0/2446926535 shutdown_connections 2024-04-24T06:42:51.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.270+0000 7f5a4f000700 1 -- 172.21.15.5:0/2446926535 wait complete. 2024-04-24T06:42:51.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.271+0000 7f5a4f000700 1 Processor -- start 2024-04-24T06:42:51.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.271+0000 7f5a4f000700 1 -- start start 2024-04-24T06:42:51.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.272+0000 7f5a4f000700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a48104e80 0x7f5a4819b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:51.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.272+0000 7f5a4f000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a48105830 0x7f5a4819b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:51.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.272+0000 7f5a4f000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a48109f60 0x7f5a4819fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:51.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.272+0000 7f5a4f000700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5a48114f40 con 0x7f5a48104e80 2024-04-24T06:42:51.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.272+0000 7f5a4f000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5a48114dc0 con 0x7f5a48105830 2024-04-24T06:42:51.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.272+0000 7f5a4f000700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5a481150c0 con 0x7f5a48109f60 2024-04-24T06:42:51.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.272+0000 7f5a4dc00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a48104e80 0x7f5a4819b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:51.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.272+0000 7f5a4dc00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a48104e80 0x7f5a4819b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43952/0 (socket says 172.21.15.5:43952) 2024-04-24T06:42:51.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.272+0000 7f5a4dc00700 1 -- 172.21.15.5:0/1190978779 learned_addr learned my addr 172.21.15.5:0/1190978779 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:42:51.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.272+0000 7f5a4d200700 1 --2- 172.21.15.5:0/1190978779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a48105830 0x7f5a4819b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:51.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.272+0000 7f5a4e600700 1 --2- 172.21.15.5:0/1190978779 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a48109f60 0x7f5a4819fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:51.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.273+0000 7f5a4dc00700 1 -- 172.21.15.5:0/1190978779 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a48109f60 msgr2=0x7f5a4819fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:51.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.273+0000 7f5a4dc00700 1 --2- 172.21.15.5:0/1190978779 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a48109f60 0x7f5a4819fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:51.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.273+0000 7f5a4dc00700 1 -- 172.21.15.5:0/1190978779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a48105830 msgr2=0x7f5a4819b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:51.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.273+0000 7f5a4dc00700 1 --2- 172.21.15.5:0/1190978779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a48105830 0x7f5a4819b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:51.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.273+0000 7f5a4dc00700 1 -- 172.21.15.5:0/1190978779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a481a0390 con 0x7f5a48104e80 2024-04-24T06:42:51.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.273+0000 7f5a4e600700 1 --2- 172.21.15.5:0/1190978779 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a48109f60 0x7f5a4819fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:42:51.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.273+0000 7f5a4d200700 1 --2- 172.21.15.5:0/1190978779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a48105830 0x7f5a4819b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:42:51.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.273+0000 7f5a4dc00700 1 --2- 172.21.15.5:0/1190978779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a48104e80 0x7f5a4819b3d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c00eb30 tx=0x7f5a3c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:51.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.273+0000 7f5a46000700 1 -- 172.21.15.5:0/1190978779 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c00cbd0 con 0x7f5a48104e80 2024-04-24T06:42:51.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.273+0000 7f5a46000700 1 -- 172.21.15.5:0/1190978779 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a3c004500 con 0x7f5a48104e80 2024-04-24T06:42:51.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.273+0000 7f5a4f000700 1 -- 172.21.15.5:0/1190978779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a481a0680 con 0x7f5a48104e80 2024-04-24T06:42:51.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.273+0000 7f5a46000700 1 -- 172.21.15.5:0/1190978779 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c010430 con 0x7f5a48104e80 2024-04-24T06:42:51.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.273+0000 7f5a4f000700 1 -- 172.21.15.5:0/1190978779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a481a7f00 con 0x7f5a48104e80 2024-04-24T06:42:51.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.275+0000 7f5a4f000700 1 -- 172.21.15.5:0/1190978779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a480697e0 con 0x7f5a48104e80 2024-04-24T06:42:51.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.277+0000 7f5a46000700 1 -- 172.21.15.5:0/1190978779 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f5a3c003f00 con 0x7f5a48104e80 2024-04-24T06:42:51.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.279+0000 7f5a46000700 1 --2- 172.21.15.5:0/1190978779 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f5a300a3cd0 0x7f5a300a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:51.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.279+0000 7f5a46000700 1 -- 172.21.15.5:0/1190978779 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f5a3c0d09b0 con 0x7f5a48104e80 2024-04-24T06:42:51.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.279+0000 7f5a4d200700 1 --2- 172.21.15.5:0/1190978779 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f5a300a3cd0 0x7f5a300a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:51.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.280+0000 7f5a4d200700 1 --2- 172.21.15.5:0/1190978779 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f5a300a3cd0 0x7f5a300a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5a48103e20 tx=0x7f5a34009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:51.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.282+0000 7f5a46000700 1 -- 172.21.15.5:0/1190978779 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5a3c09d280 con 0x7f5a48104e80 2024-04-24T06:42:51.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.522+0000 7f5a4f000700 1 -- 172.21.15.5:0/1190978779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f5a4819c6e0 con 0x7f5a48104e80 2024-04-24T06:42:51.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.523+0000 7f5a46000700 1 -- 172.21.15.5:0/1190978779 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5a3c09cb80 con 0x7f5a48104e80 2024-04-24T06:42:51.127 INFO:teuthology.orchestra.run.smithi005.stdout:141733920835 2024-04-24T06:42:51.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.526+0000 7f5a4f000700 1 -- 172.21.15.5:0/1190978779 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f5a300a3cd0 msgr2=0x7f5a300a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:51.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.526+0000 7f5a4f000700 1 --2- 172.21.15.5:0/1190978779 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f5a300a3cd0 0x7f5a300a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5a48103e20 tx=0x7f5a34009310 comp rx=0 tx=0).stop 2024-04-24T06:42:51.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.526+0000 7f5a4f000700 1 -- 172.21.15.5:0/1190978779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a48104e80 msgr2=0x7f5a4819b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:51.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.526+0000 7f5a4f000700 1 --2- 172.21.15.5:0/1190978779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a48104e80 0x7f5a4819b3d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c00eb30 tx=0x7f5a3c00ee40 comp rx=0 tx=0).stop 2024-04-24T06:42:51.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.527+0000 7f5a4f000700 1 -- 172.21.15.5:0/1190978779 shutdown_connections 2024-04-24T06:42:51.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.527+0000 7f5a4f000700 1 --2- 172.21.15.5:0/1190978779 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f5a300a3cd0 0x7f5a300a6190 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:51.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.527+0000 7f5a4f000700 1 --2- 172.21.15.5:0/1190978779 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a48109f60 0x7f5a4819fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:51.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.527+0000 7f5a4f000700 1 --2- 172.21.15.5:0/1190978779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a48104e80 0x7f5a4819b3d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:51.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.527+0000 7f5a4f000700 1 --2- 172.21.15.5:0/1190978779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a48105830 0x7f5a4819b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:51.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.527+0000 7f5a4f000700 1 -- 172.21.15.5:0/1190978779 >> 172.21.15.5:0/1190978779 conn(0x7f5a48100710 msgr2=0x7f5a481011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:51.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.527+0000 7f5a4f000700 1 -- 172.21.15.5:0/1190978779 shutdown_connections 2024-04-24T06:42:51.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:50.527+0000 7f5a4f000700 1 -- 172.21.15.5:0/1190978779 wait complete. 2024-04-24T06:42:51.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:50 smithi005 ceph-mon[27798]: pgmap v352: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-04-24T06:42:51.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:50 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1190978779' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-24T06:42:51.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:50 smithi037 ceph-mon[32864]: pgmap v352: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-04-24T06:42:51.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:50 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/1190978779' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-24T06:42:51.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:50 smithi137 ceph-mon[33017]: pgmap v352: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-04-24T06:42:51.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:50 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/1190978779' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-24T06:42:51.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.648+0000 7feee5200700 1 -- 172.21.15.5:0/1824792472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feee010d010 msgr2=0x7feee010f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:51.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.648+0000 7feee5200700 1 --2- 172.21.15.5:0/1824792472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feee010d010 0x7feee010f4d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7feed800b0a0 tx=0x7feed800b3b0 comp rx=0 tx=0).stop 2024-04-24T06:42:51.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.648+0000 7feee5200700 1 -- 172.21.15.5:0/1824792472 shutdown_connections 2024-04-24T06:42:51.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.648+0000 7feee5200700 1 --2- 172.21.15.5:0/1824792472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feee010d010 0x7feee010f4d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:51.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.648+0000 7feee5200700 1 --2- 172.21.15.5:0/1824792472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feee0103be0 0x7feee0103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:51.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.648+0000 7feee5200700 1 --2- 172.21.15.5:0/1824792472 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feee0104500 0x7feee010c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:51.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.648+0000 7feee5200700 1 -- 172.21.15.5:0/1824792472 >> 172.21.15.5:0/1824792472 conn(0x7feee00fd410 msgr2=0x7feee00ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:51.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.649+0000 7feee5200700 1 -- 172.21.15.5:0/1824792472 shutdown_connections 2024-04-24T06:42:51.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.649+0000 7feee5200700 1 -- 172.21.15.5:0/1824792472 wait complete. 2024-04-24T06:42:51.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.650+0000 7feee5200700 1 Processor -- start 2024-04-24T06:42:51.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.650+0000 7feee5200700 1 -- start start 2024-04-24T06:42:51.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.650+0000 7feee5200700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feee0103be0 0x7feee0101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:51.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.650+0000 7feee5200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feee0104500 0x7feee0101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:51.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.650+0000 7feee5200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feee010d010 0x7feee01022c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:51.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.650+0000 7feee5200700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feee0112dc0 con 0x7feee0103be0 2024-04-24T06:42:51.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.650+0000 7feee5200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7feee0112c40 con 0x7feee010d010 2024-04-24T06:42:51.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.650+0000 7feee5200700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7feee0112f40 con 0x7feee0104500 2024-04-24T06:42:51.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.651+0000 7feed7400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feee0103be0 0x7feee0101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:51.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.651+0000 7feedf400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feee0104500 0x7feee0101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:51.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.651+0000 7feedfe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feee010d010 0x7feee01022c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:51.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.651+0000 7feed7400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feee0103be0 0x7feee0101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43976/0 (socket says 172.21.15.5:43976) 2024-04-24T06:42:51.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.651+0000 7feed7400700 1 -- 172.21.15.5:0/2504942172 learned_addr learned my addr 172.21.15.5:0/2504942172 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:42:51.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.651+0000 7feedf400700 1 -- 172.21.15.5:0/2504942172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feee010d010 msgr2=0x7feee01022c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:51.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.651+0000 7feedf400700 1 --2- 172.21.15.5:0/2504942172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feee010d010 0x7feee01022c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:51.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.651+0000 7feedf400700 1 -- 172.21.15.5:0/2504942172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feee0103be0 msgr2=0x7feee0101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:51.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.651+0000 7feedf400700 1 --2- 172.21.15.5:0/2504942172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feee0103be0 0x7feee0101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:51.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.651+0000 7feedf400700 1 -- 172.21.15.5:0/2504942172 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feee01a76e0 con 0x7feee0104500 2024-04-24T06:42:51.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.651+0000 7feed7400700 1 --2- 172.21.15.5:0/2504942172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feee0103be0 0x7feee0101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:42:51.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.652+0000 7feedf400700 1 --2- 172.21.15.5:0/2504942172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feee0104500 0x7feee0101d80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7feec800e9c0 tx=0x7feec800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:51.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.652+0000 7feedcc00700 1 -- 172.21.15.5:0/2504942172 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec80095c0 con 0x7feee0104500 2024-04-24T06:42:51.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.652+0000 7feee5200700 1 -- 172.21.15.5:0/2504942172 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feee01a79d0 con 0x7feee0104500 2024-04-24T06:42:51.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.652+0000 7feedcc00700 1 -- 172.21.15.5:0/2504942172 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feec8004d10 con 0x7feee0104500 2024-04-24T06:42:51.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.652+0000 7feedcc00700 1 -- 172.21.15.5:0/2504942172 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec8005020 con 0x7feee0104500 2024-04-24T06:42:51.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.652+0000 7feee5200700 1 -- 172.21.15.5:0/2504942172 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feee01a7f10 con 0x7feee0104500 2024-04-24T06:42:51.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.654+0000 7feee5200700 1 -- 172.21.15.5:0/2504942172 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feebc005330 con 0x7feee0104500 2024-04-24T06:42:51.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.657+0000 7feedcc00700 1 -- 172.21.15.5:0/2504942172 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7feec8004220 con 0x7feee0104500 2024-04-24T06:42:51.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.657+0000 7feedcc00700 1 --2- 172.21.15.5:0/2504942172 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7feec40a3cd0 0x7feec40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:51.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.657+0000 7feedcc00700 1 -- 172.21.15.5:0/2504942172 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7feec80d0200 con 0x7feee0104500 2024-04-24T06:42:51.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.657+0000 7feed7400700 1 --2- 172.21.15.5:0/2504942172 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7feec40a3cd0 0x7feec40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:51.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.658+0000 7feed7400700 1 --2- 172.21.15.5:0/2504942172 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7feec40a3cd0 0x7feec40a6190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7feed0009d70 tx=0x7feed0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:51.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.662+0000 7feedcc00700 1 -- 172.21.15.5:0/2504942172 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feec809ca40 con 0x7feee0104500 2024-04-24T06:42:51.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.900+0000 7feee5200700 1 -- 172.21.15.5:0/2504942172 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7feebc0058f0 con 0x7feee0104500 2024-04-24T06:42:51.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.901+0000 7feedcc00700 1 -- 172.21.15.5:0/2504942172 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7feec809c340 con 0x7feee0104500 2024-04-24T06:42:51.902 INFO:teuthology.orchestra.run.smithi005.stdout:77309411409 2024-04-24T06:42:51.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.905+0000 7feee5200700 1 -- 172.21.15.5:0/2504942172 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7feec40a3cd0 msgr2=0x7feec40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:51.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.905+0000 7feee5200700 1 --2- 172.21.15.5:0/2504942172 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7feec40a3cd0 0x7feec40a6190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7feed0009d70 tx=0x7feed0009260 comp rx=0 tx=0).stop 2024-04-24T06:42:51.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.905+0000 7feee5200700 1 -- 172.21.15.5:0/2504942172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feee0104500 msgr2=0x7feee0101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:51.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.905+0000 7feee5200700 1 --2- 172.21.15.5:0/2504942172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feee0104500 0x7feee0101d80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7feec800e9c0 tx=0x7feec800ecd0 comp rx=0 tx=0).stop 2024-04-24T06:42:51.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.905+0000 7feee5200700 1 -- 172.21.15.5:0/2504942172 shutdown_connections 2024-04-24T06:42:51.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.905+0000 7feee5200700 1 --2- 172.21.15.5:0/2504942172 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7feec40a3cd0 0x7feec40a6190 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:51.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.905+0000 7feee5200700 1 --2- 172.21.15.5:0/2504942172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feee010d010 0x7feee01022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:51.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.905+0000 7feee5200700 1 --2- 172.21.15.5:0/2504942172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feee0103be0 0x7feee0101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:51.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.905+0000 7feee5200700 1 --2- 172.21.15.5:0/2504942172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feee0104500 0x7feee0101d80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:51.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.905+0000 7feee5200700 1 -- 172.21.15.5:0/2504942172 >> 172.21.15.5:0/2504942172 conn(0x7feee00fd410 msgr2=0x7feee00fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:51.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.905+0000 7feee5200700 1 -- 172.21.15.5:0/2504942172 shutdown_connections 2024-04-24T06:42:51.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:51.905+0000 7feee5200700 1 -- 172.21.15.5:0/2504942172 wait complete. 2024-04-24T06:42:52.179 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:51 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/2504942172' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-24T06:42:52.241 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738453 got 34359738456 for osd.0 2024-04-24T06:42:52.241 DEBUG:teuthology.parallel:result is None 2024-04-24T06:42:52.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:51 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/2504942172' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-24T06:42:52.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:51 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/2504942172' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-24T06:42:52.869 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920829 got 141733920835 for osd.4 2024-04-24T06:42:52.869 DEBUG:teuthology.parallel:result is None 2024-04-24T06:42:53.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:52 smithi005 ceph-mon[27798]: pgmap v353: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-04-24T06:42:53.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.117+0000 7f0113e00700 1 -- 172.21.15.5:0/3002159963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f010c06ad40 msgr2=0x7f010c06b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:53.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.117+0000 7f0113e00700 1 --2- 172.21.15.5:0/3002159963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f010c06ad40 0x7f010c06b1a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f00f8009990 tx=0x7f00f8009ca0 comp rx=0 tx=0).stop 2024-04-24T06:42:53.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.117+0000 7f0113e00700 1 -- 172.21.15.5:0/3002159963 shutdown_connections 2024-04-24T06:42:53.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.117+0000 7f0113e00700 1 --2- 172.21.15.5:0/3002159963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f010c105530 0x7f010c1059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:53.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.117+0000 7f0113e00700 1 --2- 172.21.15.5:0/3002159963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f010c107a20 0x7f010c06a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:53.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.117+0000 7f0113e00700 1 --2- 172.21.15.5:0/3002159963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f010c06ad40 0x7f010c06b1a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:53.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.117+0000 7f0113e00700 1 -- 172.21.15.5:0/3002159963 >> 172.21.15.5:0/3002159963 conn(0x7f010c0789a0 msgr2=0x7f010c078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:53.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.117+0000 7f0113e00700 1 -- 172.21.15.5:0/3002159963 shutdown_connections 2024-04-24T06:42:53.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.117+0000 7f0113e00700 1 -- 172.21.15.5:0/3002159963 wait complete. 2024-04-24T06:42:53.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.118+0000 7f0113e00700 1 Processor -- start 2024-04-24T06:42:53.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.118+0000 7f0113e00700 1 -- start start 2024-04-24T06:42:53.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.118+0000 7f0113e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f010c06ad40 0x7f010c1a5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:53.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.118+0000 7f0113e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f010c105530 0x7f010c1a5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:53.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.118+0000 7f0113e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f010c107a20 0x7f010c19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:53.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.118+0000 7f0113e00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f010c119440 con 0x7f010c107a20 2024-04-24T06:42:53.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.118+0000 7f0113e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f010c1192c0 con 0x7f010c105530 2024-04-24T06:42:53.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.118+0000 7f0113e00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f010c1195c0 con 0x7f010c06ad40 2024-04-24T06:42:53.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.119+0000 7f0111400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f010c06ad40 0x7f010c1a5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:53.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.119+0000 7f0111e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f010c107a20 0x7f010c19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:53.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.119+0000 7f0111400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f010c06ad40 0x7f010c1a5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:44768/0 (socket says 172.21.15.5:44768) 2024-04-24T06:42:53.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.119+0000 7f0111400700 1 -- 172.21.15.5:0/3843059869 learned_addr learned my addr 172.21.15.5:0/3843059869 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:42:53.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.119+0000 7f0110a00700 1 --2- 172.21.15.5:0/3843059869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f010c105530 0x7f010c1a5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:53.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.119+0000 7f0111e00700 1 -- 172.21.15.5:0/3843059869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f010c06ad40 msgr2=0x7f010c1a5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:53.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.119+0000 7f0111e00700 1 --2- 172.21.15.5:0/3843059869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f010c06ad40 0x7f010c1a5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:53.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.119+0000 7f0111e00700 1 -- 172.21.15.5:0/3843059869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f010c105530 msgr2=0x7f010c1a5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:53.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.119+0000 7f0111e00700 1 --2- 172.21.15.5:0/3843059869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f010c105530 0x7f010c1a5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:53.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.119+0000 7f0111e00700 1 -- 172.21.15.5:0/3843059869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f010c19ffe0 con 0x7f010c107a20 2024-04-24T06:42:53.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.119+0000 7f0111400700 1 --2- 172.21.15.5:0/3843059869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f010c06ad40 0x7f010c1a5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:42:53.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.120+0000 7f0111e00700 1 --2- 172.21.15.5:0/3843059869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f010c107a20 0x7f010c19f700 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f010400e970 tx=0x7f010400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:53.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.120+0000 7f010a000700 1 -- 172.21.15.5:0/3843059869 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0104004220 con 0x7f010c107a20 2024-04-24T06:42:53.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.120+0000 7f010a000700 1 -- 172.21.15.5:0/3843059869 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0104004d10 con 0x7f010c107a20 2024-04-24T06:42:53.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.120+0000 7f010a000700 1 -- 172.21.15.5:0/3843059869 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0104010700 con 0x7f010c107a20 2024-04-24T06:42:53.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.120+0000 7f0113e00700 1 -- 172.21.15.5:0/3843059869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f010c1a02d0 con 0x7f010c107a20 2024-04-24T06:42:53.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.120+0000 7f0113e00700 1 -- 172.21.15.5:0/3843059869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f010c1ac5b0 con 0x7f010c107a20 2024-04-24T06:42:53.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.121+0000 7f0113e00700 1 -- 172.21.15.5:0/3843059869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f010c102010 con 0x7f010c107a20 2024-04-24T06:42:53.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.123+0000 7f010a000700 1 -- 172.21.15.5:0/3843059869 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f0104004750 con 0x7f010c107a20 2024-04-24T06:42:53.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.125+0000 7f010a000700 1 --2- 172.21.15.5:0/3843059869 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f00f40a3cd0 0x7f00f40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:53.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.125+0000 7f010a000700 1 -- 172.21.15.5:0/3843059869 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f01040d0380 con 0x7f010c107a20 2024-04-24T06:42:53.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.125+0000 7f0111400700 1 --2- 172.21.15.5:0/3843059869 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f00f40a3cd0 0x7f00f40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:53.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.126+0000 7f0111400700 1 --2- 172.21.15.5:0/3843059869 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f00f40a3cd0 0x7f00f40a6190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0100009c10 tx=0x7f010000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:53.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.130+0000 7f010a000700 1 -- 172.21.15.5:0/3843059869 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f010401e300 con 0x7f010c107a20 2024-04-24T06:42:53.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:52 smithi037 ceph-mon[32864]: pgmap v353: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-04-24T06:42:53.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.367+0000 7f0113e00700 1 -- 172.21.15.5:0/3843059869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f010c1ac780 con 0x7f010c107a20 2024-04-24T06:42:53.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.369+0000 7f010a000700 1 -- 172.21.15.5:0/3843059869 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f010409cbc0 con 0x7f010c107a20 2024-04-24T06:42:53.370 INFO:teuthology.orchestra.run.smithi005.stdout:107374182472 2024-04-24T06:42:53.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.373+0000 7f0113e00700 1 -- 172.21.15.5:0/3843059869 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f00f40a3cd0 msgr2=0x7f00f40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:53.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.373+0000 7f0113e00700 1 --2- 172.21.15.5:0/3843059869 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f00f40a3cd0 0x7f00f40a6190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0100009c10 tx=0x7f010000b040 comp rx=0 tx=0).stop 2024-04-24T06:42:53.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.373+0000 7f0113e00700 1 -- 172.21.15.5:0/3843059869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f010c107a20 msgr2=0x7f010c19f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:53.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.373+0000 7f0113e00700 1 --2- 172.21.15.5:0/3843059869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f010c107a20 0x7f010c19f700 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f010400e970 tx=0x7f010400ec80 comp rx=0 tx=0).stop 2024-04-24T06:42:53.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.373+0000 7f0113e00700 1 -- 172.21.15.5:0/3843059869 shutdown_connections 2024-04-24T06:42:53.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.373+0000 7f0113e00700 1 --2- 172.21.15.5:0/3843059869 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f00f40a3cd0 0x7f00f40a6190 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:53.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.373+0000 7f0113e00700 1 --2- 172.21.15.5:0/3843059869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f010c107a20 0x7f010c19f700 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:53.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.373+0000 7f0113e00700 1 --2- 172.21.15.5:0/3843059869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f010c06ad40 0x7f010c1a5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:53.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.373+0000 7f0113e00700 1 --2- 172.21.15.5:0/3843059869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f010c105530 0x7f010c1a5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:53.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.373+0000 7f0113e00700 1 -- 172.21.15.5:0/3843059869 >> 172.21.15.5:0/3843059869 conn(0x7f010c0789a0 msgr2=0x7f010c104510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:53.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.374+0000 7f0113e00700 1 -- 172.21.15.5:0/3843059869 shutdown_connections 2024-04-24T06:42:53.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:53.374+0000 7f0113e00700 1 -- 172.21.15.5:0/3843059869 wait complete. 2024-04-24T06:42:53.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:52 smithi137 ceph-mon[33017]: pgmap v353: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-04-24T06:42:54.159 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:42:54.195 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612635 got 313532612640 for osd.10 2024-04-24T06:42:54.195 DEBUG:teuthology.parallel:result is None 2024-04-24T06:42:54.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:53 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/3843059869' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-24T06:42:54.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:53 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/3843059869' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-24T06:42:54.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:53 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/3843059869' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-24T06:42:55.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.069+0000 7ff665400700 1 -- 172.21.15.5:0/2917963678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff660103500 msgr2=0x7ff66010f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:55.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.069+0000 7ff665400700 1 --2- 172.21.15.5:0/2917963678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff660103500 0x7ff66010f960 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff658009f90 tx=0x7ff65800b330 comp rx=0 tx=0).stop 2024-04-24T06:42:55.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.069+0000 7ff665400700 1 -- 172.21.15.5:0/2917963678 shutdown_connections 2024-04-24T06:42:55.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.069+0000 7ff665400700 1 --2- 172.21.15.5:0/2917963678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff660103500 0x7ff66010f960 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:55.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.069+0000 7ff665400700 1 --2- 172.21.15.5:0/2917963678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff660108b90 0x7ff660108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:55.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.069+0000 7ff665400700 1 --2- 172.21.15.5:0/2917963678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff660102b40 0x7ff660102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:55.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.069+0000 7ff665400700 1 -- 172.21.15.5:0/2917963678 >> 172.21.15.5:0/2917963678 conn(0x7ff660079c50 msgr2=0x7ff66007a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:55.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.070+0000 7ff665400700 1 -- 172.21.15.5:0/2917963678 shutdown_connections 2024-04-24T06:42:55.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.070+0000 7ff665400700 1 -- 172.21.15.5:0/2917963678 wait complete. 2024-04-24T06:42:55.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.071+0000 7ff665400700 1 Processor -- start 2024-04-24T06:42:55.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.071+0000 7ff665400700 1 -- start start 2024-04-24T06:42:55.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.072+0000 7ff665400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff660102b40 0x7ff66019b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:55.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.072+0000 7ff665400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff660103500 0x7ff66019b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:55.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.072+0000 7ff665400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff660108b90 0x7ff66019fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:55.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.072+0000 7ff665400700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff660112e40 con 0x7ff660108b90 2024-04-24T06:42:55.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.072+0000 7ff665400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff660112cc0 con 0x7ff660103500 2024-04-24T06:42:55.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.072+0000 7ff665400700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff660112fc0 con 0x7ff660102b40 2024-04-24T06:42:55.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.072+0000 7ff655e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff660103500 0x7ff66019b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:55.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.072+0000 7ff655e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff660103500 0x7ff66019b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:40140/0 (socket says 172.21.15.5:40140) 2024-04-24T06:42:55.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.072+0000 7ff655e00700 1 -- 172.21.15.5:0/99155558 learned_addr learned my addr 172.21.15.5:0/99155558 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:42:55.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.072+0000 7ff65e800700 1 --2- 172.21.15.5:0/99155558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff660102b40 0x7ff66019b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:55.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.072+0000 7ff65f200700 1 --2- 172.21.15.5:0/99155558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff660108b90 0x7ff66019fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:55.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.073+0000 7ff655e00700 1 -- 172.21.15.5:0/99155558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff660102b40 msgr2=0x7ff66019b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:55.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.073+0000 7ff655e00700 1 --2- 172.21.15.5:0/99155558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff660102b40 0x7ff66019b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:55.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.073+0000 7ff655e00700 1 -- 172.21.15.5:0/99155558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff660108b90 msgr2=0x7ff66019fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:55.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.073+0000 7ff655e00700 1 --2- 172.21.15.5:0/99155558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff660108b90 0x7ff66019fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:55.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.073+0000 7ff655e00700 1 -- 172.21.15.5:0/99155558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6601a0460 con 0x7ff660103500 2024-04-24T06:42:55.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.073+0000 7ff65f200700 1 --2- 172.21.15.5:0/99155558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff660108b90 0x7ff66019fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:42:55.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.073+0000 7ff655e00700 1 --2- 172.21.15.5:0/99155558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff660103500 0x7ff66019b990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff64800e9c0 tx=0x7ff64800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:55.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.073+0000 7ff657e00700 1 -- 172.21.15.5:0/99155558 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64800cbd0 con 0x7ff660103500 2024-04-24T06:42:55.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.073+0000 7ff657e00700 1 -- 172.21.15.5:0/99155558 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff64800cd70 con 0x7ff660103500 2024-04-24T06:42:55.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.073+0000 7ff657e00700 1 -- 172.21.15.5:0/99155558 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff648005660 con 0x7ff660103500 2024-04-24T06:42:55.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.073+0000 7ff665400700 1 -- 172.21.15.5:0/99155558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6601a0750 con 0x7ff660103500 2024-04-24T06:42:55.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.074+0000 7ff665400700 1 -- 172.21.15.5:0/99155558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6601a7fd0 con 0x7ff660103500 2024-04-24T06:42:55.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.075+0000 7ff665400700 1 -- 172.21.15.5:0/99155558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6600697e0 con 0x7ff660103500 2024-04-24T06:42:55.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.077+0000 7ff657e00700 1 -- 172.21.15.5:0/99155558 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7ff648004750 con 0x7ff660103500 2024-04-24T06:42:55.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.078+0000 7ff657e00700 1 --2- 172.21.15.5:0/99155558 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ff6440a3d50 0x7ff6440a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:55.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.078+0000 7ff657e00700 1 -- 172.21.15.5:0/99155558 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7ff6480d06a0 con 0x7ff660103500 2024-04-24T06:42:55.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.078+0000 7ff65e800700 1 --2- 172.21.15.5:0/99155558 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ff6440a3d50 0x7ff6440a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:55.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.079+0000 7ff65e800700 1 --2- 172.21.15.5:0/99155558 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ff6440a3d50 0x7ff6440a6210 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff650009be0 tx=0x7ff650009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:55.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.083+0000 7ff657e00700 1 -- 172.21.15.5:0/99155558 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff64809cee0 con 0x7ff660103500 2024-04-24T06:42:55.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:54 smithi005 ceph-mon[27798]: pgmap v354: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 3 B/s, 0 keys/s, 0 objects/s recovering 2024-04-24T06:42:55.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:54 smithi037 ceph-mon[32864]: pgmap v354: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 3 B/s, 0 keys/s, 0 objects/s recovering 2024-04-24T06:42:55.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.329+0000 7ff665400700 1 -- 172.21.15.5:0/99155558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7ff6601a82c0 con 0x7ff660103500 2024-04-24T06:42:55.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.330+0000 7ff657e00700 1 -- 172.21.15.5:0/99155558 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff64809c7e0 con 0x7ff660103500 2024-04-24T06:42:55.331 INFO:teuthology.orchestra.run.smithi005.stdout:257698037806 2024-04-24T06:42:55.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.333+0000 7ff665400700 1 -- 172.21.15.5:0/99155558 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ff6440a3d50 msgr2=0x7ff6440a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:55.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.333+0000 7ff665400700 1 --2- 172.21.15.5:0/99155558 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ff6440a3d50 0x7ff6440a6210 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff650009be0 tx=0x7ff650009260 comp rx=0 tx=0).stop 2024-04-24T06:42:55.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.333+0000 7ff665400700 1 -- 172.21.15.5:0/99155558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff660103500 msgr2=0x7ff66019b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:55.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.333+0000 7ff665400700 1 --2- 172.21.15.5:0/99155558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff660103500 0x7ff66019b990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff64800e9c0 tx=0x7ff64800ecd0 comp rx=0 tx=0).stop 2024-04-24T06:42:55.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.333+0000 7ff665400700 1 -- 172.21.15.5:0/99155558 shutdown_connections 2024-04-24T06:42:55.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.333+0000 7ff665400700 1 --2- 172.21.15.5:0/99155558 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ff6440a3d50 0x7ff6440a6210 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:55.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.333+0000 7ff665400700 1 --2- 172.21.15.5:0/99155558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff660108b90 0x7ff66019fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:55.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.333+0000 7ff665400700 1 --2- 172.21.15.5:0/99155558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff660102b40 0x7ff66019b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:55.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.333+0000 7ff665400700 1 --2- 172.21.15.5:0/99155558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff660103500 0x7ff66019b990 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:55.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.333+0000 7ff665400700 1 -- 172.21.15.5:0/99155558 >> 172.21.15.5:0/99155558 conn(0x7ff660079c50 msgr2=0x7ff66010dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:55.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.334+0000 7ff665400700 1 -- 172.21.15.5:0/99155558 shutdown_connections 2024-04-24T06:42:55.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:55.334+0000 7ff665400700 1 -- 172.21.15.5:0/99155558 wait complete. 2024-04-24T06:42:55.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:54 smithi137 ceph-mon[33017]: pgmap v354: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 3 B/s, 0 keys/s, 0 objects/s recovering 2024-04-24T06:42:55.964 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574924 got 55834574931 for osd.1 2024-04-24T06:42:55.964 DEBUG:teuthology.parallel:result is None 2024-04-24T06:42:56.107 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411404 got 77309411409 for osd.2 2024-04-24T06:42:56.107 DEBUG:teuthology.parallel:result is None 2024-04-24T06:42:56.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:55 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/99155558' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-24T06:42:56.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:55 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/99155558' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-24T06:42:56.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:55 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/99155558' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-24T06:42:56.534 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037798 got 257698037806 for osd.8 2024-04-24T06:42:56.534 DEBUG:teuthology.parallel:result is None 2024-04-24T06:42:56.698 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182465 got 107374182472 for osd.3 2024-04-24T06:42:56.698 DEBUG:teuthology.parallel:result is None 2024-04-24T06:42:57.125 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:56 smithi005 ceph-mon[27798]: pgmap v355: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-04-24T06:42:57.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:56 smithi037 ceph-mon[32864]: pgmap v355: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-04-24T06:42:57.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:56 smithi137 ceph-mon[33017]: pgmap v355: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-04-24T06:42:59.109 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:42:59 smithi005 ceph-mon[27798]: pgmap v356: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-04-24T06:42:59.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.175+0000 7f1e6da00700 1 -- 172.21.15.5:0/2822571350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e6810f4f0 msgr2=0x7f1e681118e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:59.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.175+0000 7f1e6da00700 1 --2- 172.21.15.5:0/2822571350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e6810f4f0 0x7f1e681118e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1e6000b0a0 tx=0x7f1e6000b3b0 comp rx=0 tx=0).stop 2024-04-24T06:42:59.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.175+0000 7f1e6da00700 1 -- 172.21.15.5:0/2822571350 shutdown_connections 2024-04-24T06:42:59.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.175+0000 7f1e6da00700 1 --2- 172.21.15.5:0/2822571350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e6810f4f0 0x7f1e681118e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:59.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.175+0000 7f1e6da00700 1 --2- 172.21.15.5:0/2822571350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e68074d20 0x7f1e68075100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:59.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.175+0000 7f1e6da00700 1 --2- 172.21.15.5:0/2822571350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e68075640 0x7f1e6810efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:59.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.175+0000 7f1e6da00700 1 -- 172.21.15.5:0/2822571350 >> 172.21.15.5:0/2822571350 conn(0x7f1e68079ca0 msgr2=0x7f1e6807a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:59.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.175+0000 7f1e6da00700 1 -- 172.21.15.5:0/2822571350 shutdown_connections 2024-04-24T06:42:59.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.175+0000 7f1e6da00700 1 -- 172.21.15.5:0/2822571350 wait complete. 2024-04-24T06:42:59.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.176+0000 7f1e6da00700 1 Processor -- start 2024-04-24T06:42:59.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.176+0000 7f1e6da00700 1 -- start start 2024-04-24T06:42:59.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.177+0000 7f1e6da00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e68074d20 0x7f1e6819b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:59.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.177+0000 7f1e6da00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e68075640 0x7f1e6819bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:59.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.177+0000 7f1e6da00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e6810f4f0 0x7f1e6819fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:59.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.177+0000 7f1e6da00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1e68115090 con 0x7f1e68075640 2024-04-24T06:42:59.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.177+0000 7f1e6da00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1e68114f10 con 0x7f1e68074d20 2024-04-24T06:42:59.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.177+0000 7f1e6da00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1e68115210 con 0x7f1e6810f4f0 2024-04-24T06:42:59.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.177+0000 7f1e5fe00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e68075640 0x7f1e6819bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:59.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.177+0000 7f1e5fe00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e68075640 0x7f1e6819bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36888/0 (socket says 172.21.15.5:36888) 2024-04-24T06:42:59.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.177+0000 7f1e5fe00700 1 -- 172.21.15.5:0/3917030055 learned_addr learned my addr 172.21.15.5:0/3917030055 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:42:59.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.177+0000 7f1e66a00700 1 --2- 172.21.15.5:0/3917030055 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e68074d20 0x7f1e6819b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:59.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.177+0000 7f1e67400700 1 --2- 172.21.15.5:0/3917030055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e6810f4f0 0x7f1e6819fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:59.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.178+0000 7f1e5fe00700 1 -- 172.21.15.5:0/3917030055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e6810f4f0 msgr2=0x7f1e6819fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:59.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.178+0000 7f1e5fe00700 1 --2- 172.21.15.5:0/3917030055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e6810f4f0 0x7f1e6819fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:59.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.178+0000 7f1e5fe00700 1 -- 172.21.15.5:0/3917030055 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e68074d20 msgr2=0x7f1e6819b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:59.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.178+0000 7f1e5fe00700 1 --2- 172.21.15.5:0/3917030055 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e68074d20 0x7f1e6819b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:59.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.178+0000 7f1e5fe00700 1 -- 172.21.15.5:0/3917030055 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e681a05d0 con 0x7f1e68075640 2024-04-24T06:42:59.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.178+0000 7f1e66a00700 1 --2- 172.21.15.5:0/3917030055 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e68074d20 0x7f1e6819b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:42:59.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.178+0000 7f1e5fe00700 1 --2- 172.21.15.5:0/3917030055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e68075640 0x7f1e6819bb00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1e5000e9c0 tx=0x7f1e5000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:59.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.178+0000 7f1e5f400700 1 -- 172.21.15.5:0/3917030055 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5000cbd0 con 0x7f1e68075640 2024-04-24T06:42:59.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.179+0000 7f1e5f400700 1 -- 172.21.15.5:0/3917030055 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e5000ee60 con 0x7f1e68075640 2024-04-24T06:42:59.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.179+0000 7f1e6da00700 1 -- 172.21.15.5:0/3917030055 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e681a08c0 con 0x7f1e68075640 2024-04-24T06:42:59.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.179+0000 7f1e6da00700 1 -- 172.21.15.5:0/3917030055 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e681a8140 con 0x7f1e68075640 2024-04-24T06:42:59.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.179+0000 7f1e5f400700 1 -- 172.21.15.5:0/3917030055 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e50005630 con 0x7f1e68075640 2024-04-24T06:42:59.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.180+0000 7f1e6da00700 1 -- 172.21.15.5:0/3917030055 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e680697e0 con 0x7f1e68075640 2024-04-24T06:42:59.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.183+0000 7f1e5f400700 1 -- 172.21.15.5:0/3917030055 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f1e50004750 con 0x7f1e68075640 2024-04-24T06:42:59.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.184+0000 7f1e5f400700 1 --2- 172.21.15.5:0/3917030055 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f1e4c0a3cd0 0x7f1e4c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:42:59.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.185+0000 7f1e5f400700 1 -- 172.21.15.5:0/3917030055 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f1e500d09a0 con 0x7f1e68075640 2024-04-24T06:42:59.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.185+0000 7f1e66a00700 1 --2- 172.21.15.5:0/3917030055 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f1e4c0a3cd0 0x7f1e4c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:42:59.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.186+0000 7f1e66a00700 1 --2- 172.21.15.5:0/3917030055 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f1e4c0a3cd0 0x7f1e4c0a6190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1e58009680 tx=0x7f1e58009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:42:59.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.187+0000 7f1e5f400700 1 -- 172.21.15.5:0/3917030055 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1e50053ff0 con 0x7f1e68075640 2024-04-24T06:42:59.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:42:59 smithi037 ceph-mon[32864]: pgmap v356: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-04-24T06:42:59.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:42:59 smithi137 ceph-mon[33017]: pgmap v356: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-04-24T06:42:59.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.453+0000 7f1e6da00700 1 -- 172.21.15.5:0/3917030055 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f1e6819c920 con 0x7f1e68075640 2024-04-24T06:42:59.455 INFO:teuthology.orchestra.run.smithi005.stdout:292057776168 2024-04-24T06:42:59.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.454+0000 7f1e5f400700 1 -- 172.21.15.5:0/3917030055 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1e6819c920 con 0x7f1e68075640 2024-04-24T06:42:59.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.458+0000 7f1e6da00700 1 -- 172.21.15.5:0/3917030055 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f1e4c0a3cd0 msgr2=0x7f1e4c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:59.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.458+0000 7f1e6da00700 1 --2- 172.21.15.5:0/3917030055 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f1e4c0a3cd0 0x7f1e4c0a6190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1e58009680 tx=0x7f1e58009260 comp rx=0 tx=0).stop 2024-04-24T06:42:59.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.458+0000 7f1e6da00700 1 -- 172.21.15.5:0/3917030055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e68075640 msgr2=0x7f1e6819bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:42:59.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.458+0000 7f1e6da00700 1 --2- 172.21.15.5:0/3917030055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e68075640 0x7f1e6819bb00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1e5000e9c0 tx=0x7f1e5000ecd0 comp rx=0 tx=0).stop 2024-04-24T06:42:59.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.458+0000 7f1e6da00700 1 -- 172.21.15.5:0/3917030055 shutdown_connections 2024-04-24T06:42:59.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.458+0000 7f1e6da00700 1 --2- 172.21.15.5:0/3917030055 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f1e4c0a3cd0 0x7f1e4c0a6190 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:59.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.458+0000 7f1e6da00700 1 --2- 172.21.15.5:0/3917030055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e6810f4f0 0x7f1e6819fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:59.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.458+0000 7f1e6da00700 1 --2- 172.21.15.5:0/3917030055 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e68074d20 0x7f1e6819b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:59.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.458+0000 7f1e6da00700 1 --2- 172.21.15.5:0/3917030055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e68075640 0x7f1e6819bb00 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:42:59.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.458+0000 7f1e6da00700 1 -- 172.21.15.5:0/3917030055 >> 172.21.15.5:0/3917030055 conn(0x7f1e68079ca0 msgr2=0x7f1e68100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:42:59.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.458+0000 7f1e6da00700 1 -- 172.21.15.5:0/3917030055 shutdown_connections 2024-04-24T06:42:59.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:42:59.458+0000 7f1e6da00700 1 -- 172.21.15.5:0/3917030055 wait complete. 2024-04-24T06:43:00.188 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:00 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/3917030055' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-24T06:43:00.259 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776162 got 292057776168 for osd.9 2024-04-24T06:43:00.259 DEBUG:teuthology.parallel:result is None 2024-04-24T06:43:00.260 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-24T06:43:00.260 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-24T06:43:00.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:00 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/3917030055' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-24T06:43:00.348 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:00 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/3917030055' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-24T06:43:00.695 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:43:01.125 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:01 smithi005 ceph-mon[27798]: pgmap v357: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-04-24T06:43:01.125 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:01 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:43:01.125 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:01 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:43:01.268 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:01 smithi037 ceph-mon[32864]: pgmap v357: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-04-24T06:43:01.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:01 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:43:01.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:01 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:43:01.360 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:01 smithi137 ceph-mon[33017]: pgmap v357: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-04-24T06:43:01.360 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:01 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:43:01.360 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:01 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:43:02.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:02.890+0000 7f2591e00700 1 -- 172.21.15.5:0/3798643674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f258c10a050 msgr2=0x7f258c111c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:02.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:02.890+0000 7f2591e00700 1 --2- 172.21.15.5:0/3798643674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f258c10a050 0x7f258c111c20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f25840099c0 tx=0x7f2584009cd0 comp rx=0 tx=0).stop 2024-04-24T06:43:02.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:02.890+0000 7f2591e00700 1 -- 172.21.15.5:0/3798643674 shutdown_connections 2024-04-24T06:43:02.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:02.890+0000 7f2591e00700 1 --2- 172.21.15.5:0/3798643674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f258c10a050 0x7f258c111c20 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:02.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:02.890+0000 7f2591e00700 1 --2- 172.21.15.5:0/3798643674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f258c104f70 0x7f258c105350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:02.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:02.890+0000 7f2591e00700 1 --2- 172.21.15.5:0/3798643674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f258c105920 0x7f258c109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:02.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:02.891+0000 7f2591e00700 1 -- 172.21.15.5:0/3798643674 >> 172.21.15.5:0/3798643674 conn(0x7f258c100800 msgr2=0x7f258c102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:02.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:02.891+0000 7f2591e00700 1 -- 172.21.15.5:0/3798643674 shutdown_connections 2024-04-24T06:43:02.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:02.891+0000 7f2591e00700 1 -- 172.21.15.5:0/3798643674 wait complete. 2024-04-24T06:43:02.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:02.892+0000 7f2591e00700 1 Processor -- start 2024-04-24T06:43:02.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:02.892+0000 7f2591e00700 1 -- start start 2024-04-24T06:43:02.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:02.893+0000 7f2591e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f258c104f70 0x7f258c19b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:02.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:02.893+0000 7f2591e00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f258c105920 0x7f258c19b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:02.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:02.893+0000 7f2591e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f258c10a050 0x7f258c19fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:02.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:02.893+0000 7f2591e00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f258c1150b0 con 0x7f258c105920 2024-04-24T06:43:02.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:02.893+0000 7f2591e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f258c114f30 con 0x7f258c104f70 2024-04-24T06:43:02.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:02.893+0000 7f2591e00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f258c115230 con 0x7f258c10a050 2024-04-24T06:43:02.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:02.893+0000 7f258a000700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f258c105920 0x7f258c19b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:02.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:02.893+0000 7f258a000700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f258c105920 0x7f258c19b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36894/0 (socket says 172.21.15.5:36894) 2024-04-24T06:43:02.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:02.893+0000 7f258aa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f258c104f70 0x7f258c19b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:02.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:02.893+0000 7f258a000700 1 -- 172.21.15.5:0/3968667554 learned_addr learned my addr 172.21.15.5:0/3968667554 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:02.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:02.893+0000 7f258b400700 1 --2- 172.21.15.5:0/3968667554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f258c10a050 0x7f258c19fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:02.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:02.894+0000 7f258a000700 1 -- 172.21.15.5:0/3968667554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f258c10a050 msgr2=0x7f258c19fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:02.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:02.894+0000 7f258a000700 1 --2- 172.21.15.5:0/3968667554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f258c10a050 0x7f258c19fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:02.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:02.894+0000 7f258a000700 1 -- 172.21.15.5:0/3968667554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f258c104f70 msgr2=0x7f258c19b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:02.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:02.894+0000 7f258a000700 1 --2- 172.21.15.5:0/3968667554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f258c104f70 0x7f258c19b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:02.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:02.894+0000 7f258a000700 1 -- 172.21.15.5:0/3968667554 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f258c1a0470 con 0x7f258c105920 2024-04-24T06:43:02.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:02.894+0000 7f258aa00700 1 --2- 172.21.15.5:0/3968667554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f258c104f70 0x7f258c19b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:43:02.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:02.894+0000 7f258a000700 1 --2- 172.21.15.5:0/3968667554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f258c105920 0x7f258c19b9a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f257800e970 tx=0x7f257800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:02.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:02.894+0000 7f2590a00700 1 -- 172.21.15.5:0/3968667554 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2578004a30 con 0x7f258c105920 2024-04-24T06:43:02.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:02.894+0000 7f2591e00700 1 -- 172.21.15.5:0/3968667554 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f258c1a0760 con 0x7f258c105920 2024-04-24T06:43:02.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:02.894+0000 7f2590a00700 1 -- 172.21.15.5:0/3968667554 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f257800ccc0 con 0x7f258c105920 2024-04-24T06:43:02.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:02.894+0000 7f2590a00700 1 -- 172.21.15.5:0/3968667554 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2578005650 con 0x7f258c105920 2024-04-24T06:43:02.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:02.895+0000 7f2591e00700 1 -- 172.21.15.5:0/3968667554 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f258c1a7fe0 con 0x7f258c105920 2024-04-24T06:43:02.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:02.896+0000 7f2591e00700 1 -- 172.21.15.5:0/3968667554 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f258c0697e0 con 0x7f258c105920 2024-04-24T06:43:02.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:02.898+0000 7f2590a00700 1 -- 172.21.15.5:0/3968667554 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f2578004bd0 con 0x7f258c105920 2024-04-24T06:43:02.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:02.900+0000 7f2590a00700 1 --2- 172.21.15.5:0/3968667554 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f25700a3d50 0x7f25700a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:02.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:02.900+0000 7f2590a00700 1 -- 172.21.15.5:0/3968667554 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f25780d06f0 con 0x7f258c105920 2024-04-24T06:43:02.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:02.901+0000 7f258aa00700 1 --2- 172.21.15.5:0/3968667554 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f25700a3d50 0x7f25700a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:02.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:02.901+0000 7f258aa00700 1 --2- 172.21.15.5:0/3968667554 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f25700a3d50 0x7f25700a6210 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f257c003e30 tx=0x7f257c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:02.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:02.905+0000 7f2590a00700 1 -- 172.21.15.5:0/3968667554 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f257809cf30 con 0x7f258c105920 2024-04-24T06:43:03.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:03.135+0000 7f2591e00700 1 -- 172.21.15.5:0/3968667554 --> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f258c101910 con 0x7f25700a3d50 2024-04-24T06:43:03.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:03.143+0000 7f2590a00700 1 -- 172.21.15.5:0/3968667554 <== mgr.14152 v2:172.21.15.5:6800/4042936965 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258708 (secure 0 0 0) 0x7f258c101910 con 0x7f25700a3d50 2024-04-24T06:43:03.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:03 smithi005 ceph-mon[27798]: pgmap v358: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:43:03.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:03 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:43:03.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:03 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:43:03.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:03 smithi005 ceph-mon[27798]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:43:03.146 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:43:03.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:03.151+0000 7f2591e00700 1 -- 172.21.15.5:0/3968667554 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f25700a3d50 msgr2=0x7f25700a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:03.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:03.151+0000 7f2591e00700 1 --2- 172.21.15.5:0/3968667554 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f25700a3d50 0x7f25700a6210 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f257c003e30 tx=0x7f257c00b040 comp rx=0 tx=0).stop 2024-04-24T06:43:03.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:03.151+0000 7f2591e00700 1 -- 172.21.15.5:0/3968667554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f258c105920 msgr2=0x7f258c19b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:03.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:03.151+0000 7f2591e00700 1 --2- 172.21.15.5:0/3968667554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f258c105920 0x7f258c19b9a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f257800e970 tx=0x7f257800ec80 comp rx=0 tx=0).stop 2024-04-24T06:43:03.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:03.151+0000 7f2591e00700 1 -- 172.21.15.5:0/3968667554 shutdown_connections 2024-04-24T06:43:03.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:03.151+0000 7f2591e00700 1 --2- 172.21.15.5:0/3968667554 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f25700a3d50 0x7f25700a6210 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:03.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:03.151+0000 7f2591e00700 1 --2- 172.21.15.5:0/3968667554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f258c10a050 0x7f258c19fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:03.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:03.151+0000 7f2591e00700 1 --2- 172.21.15.5:0/3968667554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f258c104f70 0x7f258c19b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:03.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:03.151+0000 7f2591e00700 1 --2- 172.21.15.5:0/3968667554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f258c105920 0x7f258c19b9a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:03.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:03.151+0000 7f2591e00700 1 -- 172.21.15.5:0/3968667554 >> 172.21.15.5:0/3968667554 conn(0x7f258c100800 msgr2=0x7f258c101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:03.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:03.152+0000 7f2591e00700 1 -- 172.21.15.5:0/3968667554 shutdown_connections 2024-04-24T06:43:03.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:03.152+0000 7f2591e00700 1 -- 172.21.15.5:0/3968667554 wait complete. 2024-04-24T06:43:03.154 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-24T06:43:03.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:03 smithi137 ceph-mon[33017]: pgmap v358: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:43:03.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:03 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:43:03.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:03 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:43:03.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:03 smithi137 ceph-mon[33017]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:43:03.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:03 smithi037 ceph-mon[32864]: pgmap v358: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:43:03.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:03 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:43:03.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:03 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:43:03.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:03 smithi037 ceph-mon[32864]: from='mgr.14152 172.21.15.5:0/475619974' entity='mgr.x' 2024-04-24T06:43:04.088 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":358,"stamp":"2024-04-24T06:43:01.338257+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654456,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":636,"num_read_kb":436,"num_write":339,"num_write_kb":3046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":273,"ondisk_log_size":273,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375108,"kb_used_data":13404,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566716,"statfs":{"total":1151940427776,"available":1151556317184,"internally_reserved":0,"allocated":13725696,"data_stored":7485912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005001"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.870668+0000","last_change":"2024-04-24T06:41:35.296451+0000","last_active":"2024-04-24T06:42:42.870668+0000","last_peered":"2024-04-24T06:42:42.870668+0000","last_clean":"2024-04-24T06:42:42.870668+0000","last_became_active":"2024-04-24T06:41:35.295733+0000","last_became_peered":"2024-04-24T06:41:35.295733+0000","last_unstale":"2024-04-24T06:42:42.870668+0000","last_undegraded":"2024-04-24T06:42:42.870668+0000","last_fullsized":"2024-04-24T06:42:42.870668+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:04:15.072644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.873777+0000","last_change":"2024-04-24T06:41:35.293359+0000","last_active":"2024-04-24T06:42:42.873777+0000","last_peered":"2024-04-24T06:42:42.873777+0000","last_clean":"2024-04-24T06:42:42.873777+0000","last_became_active":"2024-04-24T06:41:35.287371+0000","last_became_peered":"2024-04-24T06:41:35.287371+0000","last_unstale":"2024-04-24T06:42:42.873777+0000","last_undegraded":"2024-04-24T06:42:42.873777+0000","last_fullsized":"2024-04-24T06:42:42.873777+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:03:39.376495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.873042+0000","last_change":"2024-04-24T06:41:35.272734+0000","last_active":"2024-04-24T06:42:42.873042+0000","last_peered":"2024-04-24T06:42:42.873042+0000","last_clean":"2024-04-24T06:42:42.873042+0000","last_became_active":"2024-04-24T06:41:35.271685+0000","last_became_peered":"2024-04-24T06:41:35.271685+0000","last_unstale":"2024-04-24T06:42:42.873042+0000","last_undegraded":"2024-04-24T06:42:42.873042+0000","last_fullsized":"2024-04-24T06:42:42.873042+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:08:15.057918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.922346+0000","last_change":"2024-04-24T06:41:35.288574+0000","last_active":"2024-04-24T06:42:42.922346+0000","last_peered":"2024-04-24T06:42:42.922346+0000","last_clean":"2024-04-24T06:42:42.922346+0000","last_became_active":"2024-04-24T06:41:35.288229+0000","last_became_peered":"2024-04-24T06:41:35.288229+0000","last_unstale":"2024-04-24T06:42:42.922346+0000","last_undegraded":"2024-04-24T06:42:42.922346+0000","last_fullsized":"2024-04-24T06:42:42.922346+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:58:18.028067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.874961+0000","last_change":"2024-04-24T06:41:35.310226+0000","last_active":"2024-04-24T06:42:42.874961+0000","last_peered":"2024-04-24T06:42:42.874961+0000","last_clean":"2024-04-24T06:42:42.874961+0000","last_became_active":"2024-04-24T06:41:35.309853+0000","last_became_peered":"2024-04-24T06:41:35.309853+0000","last_unstale":"2024-04-24T06:42:42.874961+0000","last_undegraded":"2024-04-24T06:42:42.874961+0000","last_fullsized":"2024-04-24T06:42:42.874961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:34:57.767158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.873418+0000","last_change":"2024-04-24T06:41:35.294259+0000","last_active":"2024-04-24T06:42:42.873418+0000","last_peered":"2024-04-24T06:42:42.873418+0000","last_clean":"2024-04-24T06:42:42.873418+0000","last_became_active":"2024-04-24T06:41:35.293827+0000","last_became_peered":"2024-04-24T06:41:35.293827+0000","last_unstale":"2024-04-24T06:42:42.873418+0000","last_undegraded":"2024-04-24T06:42:42.873418+0000","last_fullsized":"2024-04-24T06:42:42.873418+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:09:23.302827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:43.165104+0000","last_change":"2024-04-24T06:41:35.295095+0000","last_active":"2024-04-24T06:42:43.165104+0000","last_peered":"2024-04-24T06:42:43.165104+0000","last_clean":"2024-04-24T06:42:43.165104+0000","last_became_active":"2024-04-24T06:41:35.294743+0000","last_became_peered":"2024-04-24T06:41:35.294743+0000","last_unstale":"2024-04-24T06:42:43.165104+0000","last_undegraded":"2024-04-24T06:42:43.165104+0000","last_fullsized":"2024-04-24T06:42:43.165104+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:06:09.473123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.889034+0000","last_change":"2024-04-24T06:42:42.886072+0000","last_active":"2024-04-24T06:42:42.889034+0000","last_peered":"2024-04-24T06:42:42.889034+0000","last_clean":"2024-04-24T06:42:42.889034+0000","last_became_active":"2024-04-24T06:42:42.882624+0000","last_became_peered":"2024-04-24T06:42:42.882624+0000","last_unstale":"2024-04-24T06:42:42.889034+0000","last_undegraded":"2024-04-24T06:42:42.889034+0000","last_fullsized":"2024-04-24T06:42:42.889034+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:53:02.523012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.922808+0000","last_change":"2024-04-24T06:41:35.283275+0000","last_active":"2024-04-24T06:42:42.922808+0000","last_peered":"2024-04-24T06:42:42.922808+0000","last_clean":"2024-04-24T06:42:42.922808+0000","last_became_active":"2024-04-24T06:41:35.282586+0000","last_became_peered":"2024-04-24T06:41:35.282586+0000","last_unstale":"2024-04-24T06:42:42.922808+0000","last_undegraded":"2024-04-24T06:42:42.922808+0000","last_fullsized":"2024-04-24T06:42:42.922808+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:42:30.716149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.873161+0000","last_change":"2024-04-24T06:41:35.284986+0000","last_active":"2024-04-24T06:42:42.873161+0000","last_peered":"2024-04-24T06:42:42.873161+0000","last_clean":"2024-04-24T06:42:42.873161+0000","last_became_active":"2024-04-24T06:41:35.284628+0000","last_became_peered":"2024-04-24T06:41:35.284628+0000","last_unstale":"2024-04-24T06:42:42.873161+0000","last_undegraded":"2024-04-24T06:42:42.873161+0000","last_fullsized":"2024-04-24T06:42:42.873161+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:29:38.479593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:43.165654+0000","last_change":"2024-04-24T06:41:35.290353+0000","last_active":"2024-04-24T06:42:43.165654+0000","last_peered":"2024-04-24T06:42:43.165654+0000","last_clean":"2024-04-24T06:42:43.165654+0000","last_became_active":"2024-04-24T06:41:35.289987+0000","last_became_peered":"2024-04-24T06:41:35.289987+0000","last_unstale":"2024-04-24T06:42:43.165654+0000","last_undegraded":"2024-04-24T06:42:43.165654+0000","last_fullsized":"2024-04-24T06:42:43.165654+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:56:54.914426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.922207+0000","last_change":"2024-04-24T06:41:35.289462+0000","last_active":"2024-04-24T06:42:42.922207+0000","last_peered":"2024-04-24T06:42:42.922207+0000","last_clean":"2024-04-24T06:42:42.922207+0000","last_became_active":"2024-04-24T06:41:35.289123+0000","last_became_peered":"2024-04-24T06:41:35.289123+0000","last_unstale":"2024-04-24T06:42:42.922207+0000","last_undegraded":"2024-04-24T06:42:42.922207+0000","last_fullsized":"2024-04-24T06:42:42.922207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:51:06.792508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.880246+0000","last_change":"2024-04-24T06:41:35.298320+0000","last_active":"2024-04-24T06:42:42.880246+0000","last_peered":"2024-04-24T06:42:42.880246+0000","last_clean":"2024-04-24T06:42:42.880246+0000","last_became_active":"2024-04-24T06:41:35.297839+0000","last_became_peered":"2024-04-24T06:41:35.297839+0000","last_unstale":"2024-04-24T06:42:42.880246+0000","last_undegraded":"2024-04-24T06:42:42.880246+0000","last_fullsized":"2024-04-24T06:42:42.880246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:10:36.423514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.872554+0000","last_change":"2024-04-24T06:41:35.301924+0000","last_active":"2024-04-24T06:42:42.872554+0000","last_peered":"2024-04-24T06:42:42.872554+0000","last_clean":"2024-04-24T06:42:42.872554+0000","last_became_active":"2024-04-24T06:41:35.299114+0000","last_became_peered":"2024-04-24T06:41:35.299114+0000","last_unstale":"2024-04-24T06:42:42.872554+0000","last_undegraded":"2024-04-24T06:42:42.872554+0000","last_fullsized":"2024-04-24T06:42:42.872554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:34:46.194266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.882096+0000","last_change":"2024-04-24T06:41:35.284885+0000","last_active":"2024-04-24T06:42:42.882096+0000","last_peered":"2024-04-24T06:42:42.882096+0000","last_clean":"2024-04-24T06:42:42.882096+0000","last_became_active":"2024-04-24T06:41:35.284244+0000","last_became_peered":"2024-04-24T06:41:35.284244+0000","last_unstale":"2024-04-24T06:42:42.882096+0000","last_undegraded":"2024-04-24T06:42:42.882096+0000","last_fullsized":"2024-04-24T06:42:42.882096+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:30:40.279905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.921571+0000","last_change":"2024-04-24T06:41:35.286363+0000","last_active":"2024-04-24T06:42:42.921571+0000","last_peered":"2024-04-24T06:42:42.921571+0000","last_clean":"2024-04-24T06:42:42.921571+0000","last_became_active":"2024-04-24T06:41:35.286024+0000","last_became_peered":"2024-04-24T06:41:35.286024+0000","last_unstale":"2024-04-24T06:42:42.921571+0000","last_undegraded":"2024-04-24T06:42:42.921571+0000","last_fullsized":"2024-04-24T06:42:42.921571+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:43:35.728990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.870846+0000","last_change":"2024-04-24T06:41:35.304005+0000","last_active":"2024-04-24T06:42:42.870846+0000","last_peered":"2024-04-24T06:42:42.870846+0000","last_clean":"2024-04-24T06:42:42.870846+0000","last_became_active":"2024-04-24T06:41:35.303659+0000","last_became_peered":"2024-04-24T06:41:35.303659+0000","last_unstale":"2024-04-24T06:42:42.870846+0000","last_undegraded":"2024-04-24T06:42:42.870846+0000","last_fullsized":"2024-04-24T06:42:42.870846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:05:07.215927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.875662+0000","last_change":"2024-04-24T06:41:35.296218+0000","last_active":"2024-04-24T06:42:42.875662+0000","last_peered":"2024-04-24T06:42:42.875662+0000","last_clean":"2024-04-24T06:42:42.875662+0000","last_became_active":"2024-04-24T06:41:35.295486+0000","last_became_peered":"2024-04-24T06:41:35.295486+0000","last_unstale":"2024-04-24T06:42:42.875662+0000","last_undegraded":"2024-04-24T06:42:42.875662+0000","last_fullsized":"2024-04-24T06:42:42.875662+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:17:25.844899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.886291+0000","last_change":"2024-04-24T06:42:42.886291+0000","last_active":"2024-04-24T06:42:42.886291+0000","last_peered":"2024-04-24T06:42:42.886291+0000","last_clean":"2024-04-24T06:42:42.886291+0000","last_became_active":"2024-04-24T06:42:42.885781+0000","last_became_peered":"2024-04-24T06:42:42.885781+0000","last_unstale":"2024-04-24T06:42:42.886291+0000","last_undegraded":"2024-04-24T06:42:42.886291+0000","last_fullsized":"2024-04-24T06:42:42.886291+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:01:44.377335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.877179+0000","last_change":"2024-04-24T06:41:34.282140+0000","last_active":"2024-04-24T06:42:42.877179+0000","last_peered":"2024-04-24T06:42:42.877179+0000","last_clean":"2024-04-24T06:42:42.877179+0000","last_became_active":"2024-04-24T06:41:34.281608+0000","last_became_peered":"2024-04-24T06:41:34.281608+0000","last_unstale":"2024-04-24T06:42:42.877179+0000","last_undegraded":"2024-04-24T06:42:42.877179+0000","last_fullsized":"2024-04-24T06:42:42.877179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:32:06.727639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.877322+0000","last_change":"2024-04-24T06:41:35.308847+0000","last_active":"2024-04-24T06:42:42.877322+0000","last_peered":"2024-04-24T06:42:42.877322+0000","last_clean":"2024-04-24T06:42:42.877322+0000","last_became_active":"2024-04-24T06:41:35.308256+0000","last_became_peered":"2024-04-24T06:41:35.308256+0000","last_unstale":"2024-04-24T06:42:42.877322+0000","last_undegraded":"2024-04-24T06:42:42.877322+0000","last_fullsized":"2024-04-24T06:42:42.877322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:21:49.338737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.877564+0000","last_change":"2024-04-24T06:41:34.285819+0000","last_active":"2024-04-24T06:42:42.877564+0000","last_peered":"2024-04-24T06:42:42.877564+0000","last_clean":"2024-04-24T06:42:42.877564+0000","last_became_active":"2024-04-24T06:41:34.282393+0000","last_became_peered":"2024-04-24T06:41:34.282393+0000","last_unstale":"2024-04-24T06:42:42.877564+0000","last_undegraded":"2024-04-24T06:42:42.877564+0000","last_fullsized":"2024-04-24T06:42:42.877564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:11:18.969178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.876983+0000","last_change":"2024-04-24T06:41:35.297437+0000","last_active":"2024-04-24T06:42:42.876983+0000","last_peered":"2024-04-24T06:42:42.876983+0000","last_clean":"2024-04-24T06:42:42.876983+0000","last_became_active":"2024-04-24T06:41:35.295195+0000","last_became_peered":"2024-04-24T06:41:35.295195+0000","last_unstale":"2024-04-24T06:42:42.876983+0000","last_undegraded":"2024-04-24T06:42:42.876983+0000","last_fullsized":"2024-04-24T06:42:42.876983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:21:51.219219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.875586+0000","last_change":"2024-04-24T06:41:34.275768+0000","last_active":"2024-04-24T06:42:42.875586+0000","last_peered":"2024-04-24T06:42:42.875586+0000","last_clean":"2024-04-24T06:42:42.875586+0000","last_became_active":"2024-04-24T06:41:34.275426+0000","last_became_peered":"2024-04-24T06:41:34.275426+0000","last_unstale":"2024-04-24T06:42:42.875586+0000","last_undegraded":"2024-04-24T06:42:42.875586+0000","last_fullsized":"2024-04-24T06:42:42.875586+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:01:08.735480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.921904+0000","last_change":"2024-04-24T06:41:35.305734+0000","last_active":"2024-04-24T06:42:42.921904+0000","last_peered":"2024-04-24T06:42:42.921904+0000","last_clean":"2024-04-24T06:42:42.921904+0000","last_became_active":"2024-04-24T06:41:35.305365+0000","last_became_peered":"2024-04-24T06:41:35.305365+0000","last_unstale":"2024-04-24T06:42:42.921904+0000","last_undegraded":"2024-04-24T06:42:42.921904+0000","last_fullsized":"2024-04-24T06:42:42.921904+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:38:52.740107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.875262+0000","last_change":"2024-04-24T06:41:34.278317+0000","last_active":"2024-04-24T06:42:42.875262+0000","last_peered":"2024-04-24T06:42:42.875262+0000","last_clean":"2024-04-24T06:42:42.875262+0000","last_became_active":"2024-04-24T06:41:34.277862+0000","last_became_peered":"2024-04-24T06:41:34.277862+0000","last_unstale":"2024-04-24T06:42:42.875262+0000","last_undegraded":"2024-04-24T06:42:42.875262+0000","last_fullsized":"2024-04-24T06:42:42.875262+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:19:50.444136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.876546+0000","last_change":"2024-04-24T06:41:35.282648+0000","last_active":"2024-04-24T06:42:42.876546+0000","last_peered":"2024-04-24T06:42:42.876546+0000","last_clean":"2024-04-24T06:42:42.876546+0000","last_became_active":"2024-04-24T06:41:35.282335+0000","last_became_peered":"2024-04-24T06:41:35.282335+0000","last_unstale":"2024-04-24T06:42:42.876546+0000","last_undegraded":"2024-04-24T06:42:42.876546+0000","last_fullsized":"2024-04-24T06:42:42.876546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:30:11.909067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":105,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:57.467595+0000","last_change":"2024-04-24T06:41:34.274405+0000","last_active":"2024-04-24T06:42:57.467595+0000","last_peered":"2024-04-24T06:42:57.467595+0000","last_clean":"2024-04-24T06:42:57.467595+0000","last_became_active":"2024-04-24T06:41:34.273820+0000","last_became_peered":"2024-04-24T06:41:34.273820+0000","last_unstale":"2024-04-24T06:42:57.467595+0000","last_undegraded":"2024-04-24T06:42:57.467595+0000","last_fullsized":"2024-04-24T06:42:57.467595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:11:26.119236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.875343+0000","last_change":"2024-04-24T06:41:35.294133+0000","last_active":"2024-04-24T06:42:42.875343+0000","last_peered":"2024-04-24T06:42:42.875343+0000","last_clean":"2024-04-24T06:42:42.875343+0000","last_became_active":"2024-04-24T06:41:35.293741+0000","last_became_peered":"2024-04-24T06:41:35.293741+0000","last_unstale":"2024-04-24T06:42:42.875343+0000","last_undegraded":"2024-04-24T06:42:42.875343+0000","last_fullsized":"2024-04-24T06:42:42.875343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:09:12.495187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.927397+0000","last_change":"2024-04-24T06:42:42.927397+0000","last_active":"2024-04-24T06:42:42.927397+0000","last_peered":"2024-04-24T06:42:42.927397+0000","last_clean":"2024-04-24T06:42:42.927397+0000","last_became_active":"2024-04-24T06:42:42.924070+0000","last_became_peered":"2024-04-24T06:42:42.924070+0000","last_unstale":"2024-04-24T06:42:42.927397+0000","last_undegraded":"2024-04-24T06:42:42.927397+0000","last_fullsized":"2024-04-24T06:42:42.927397+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:19:47.319762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:43.166909+0000","last_change":"2024-04-24T06:42:43.166909+0000","last_active":"2024-04-24T06:42:43.166909+0000","last_peered":"2024-04-24T06:42:43.166909+0000","last_clean":"2024-04-24T06:42:43.166909+0000","last_became_active":"2024-04-24T06:42:43.166559+0000","last_became_peered":"2024-04-24T06:42:43.166559+0000","last_unstale":"2024-04-24T06:42:43.166909+0000","last_undegraded":"2024-04-24T06:42:43.166909+0000","last_fullsized":"2024-04-24T06:42:43.166909+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:26:42.537506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.879678+0000","last_change":"2024-04-24T06:41:34.284958+0000","last_active":"2024-04-24T06:42:42.879678+0000","last_peered":"2024-04-24T06:42:42.879678+0000","last_clean":"2024-04-24T06:42:42.879678+0000","last_became_active":"2024-04-24T06:41:34.284279+0000","last_became_peered":"2024-04-24T06:41:34.284279+0000","last_unstale":"2024-04-24T06:42:42.879678+0000","last_undegraded":"2024-04-24T06:42:42.879678+0000","last_fullsized":"2024-04-24T06:42:42.879678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:58:19.316091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.878164+0000","last_change":"2024-04-24T06:41:35.307280+0000","last_active":"2024-04-24T06:42:42.878164+0000","last_peered":"2024-04-24T06:42:42.878164+0000","last_clean":"2024-04-24T06:42:42.878164+0000","last_became_active":"2024-04-24T06:41:35.306908+0000","last_became_peered":"2024-04-24T06:41:35.306908+0000","last_unstale":"2024-04-24T06:42:42.878164+0000","last_undegraded":"2024-04-24T06:42:42.878164+0000","last_fullsized":"2024-04-24T06:42:42.878164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:54:31.306302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.876562+0000","last_change":"2024-04-24T06:41:34.272221+0000","last_active":"2024-04-24T06:42:42.876562+0000","last_peered":"2024-04-24T06:42:42.876562+0000","last_clean":"2024-04-24T06:42:42.876562+0000","last_became_active":"2024-04-24T06:41:34.271748+0000","last_became_peered":"2024-04-24T06:41:34.271748+0000","last_unstale":"2024-04-24T06:42:42.876562+0000","last_undegraded":"2024-04-24T06:42:42.876562+0000","last_fullsized":"2024-04-24T06:42:42.876562+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:29:02.667285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.877845+0000","last_change":"2024-04-24T06:41:35.309608+0000","last_active":"2024-04-24T06:42:42.877845+0000","last_peered":"2024-04-24T06:42:42.877845+0000","last_clean":"2024-04-24T06:42:42.877845+0000","last_became_active":"2024-04-24T06:41:35.309227+0000","last_became_peered":"2024-04-24T06:41:35.309227+0000","last_unstale":"2024-04-24T06:42:42.877845+0000","last_undegraded":"2024-04-24T06:42:42.877845+0000","last_fullsized":"2024-04-24T06:42:42.877845+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:51:23.151426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":100,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:56.465254+0000","last_change":"2024-04-24T06:41:34.281885+0000","last_active":"2024-04-24T06:42:56.465254+0000","last_peered":"2024-04-24T06:42:56.465254+0000","last_clean":"2024-04-24T06:42:56.465254+0000","last_became_active":"2024-04-24T06:41:34.281466+0000","last_became_peered":"2024-04-24T06:41:34.281466+0000","last_unstale":"2024-04-24T06:42:56.465254+0000","last_undegraded":"2024-04-24T06:42:56.465254+0000","last_fullsized":"2024-04-24T06:42:56.465254+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:20:08.500343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":78,"num_read_kb":78,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.922456+0000","last_change":"2024-04-24T06:41:35.292714+0000","last_active":"2024-04-24T06:42:42.922456+0000","last_peered":"2024-04-24T06:42:42.922456+0000","last_clean":"2024-04-24T06:42:42.922456+0000","last_became_active":"2024-04-24T06:41:35.291950+0000","last_became_peered":"2024-04-24T06:41:35.291950+0000","last_unstale":"2024-04-24T06:42:42.922456+0000","last_undegraded":"2024-04-24T06:42:42.922456+0000","last_fullsized":"2024-04-24T06:42:42.922456+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:42:18.049746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:43.165729+0000","last_change":"2024-04-24T06:41:34.275218+0000","last_active":"2024-04-24T06:42:43.165729+0000","last_peered":"2024-04-24T06:42:43.165729+0000","last_clean":"2024-04-24T06:42:43.165729+0000","last_became_active":"2024-04-24T06:41:34.274824+0000","last_became_peered":"2024-04-24T06:41:34.274824+0000","last_unstale":"2024-04-24T06:42:43.165729+0000","last_undegraded":"2024-04-24T06:42:43.165729+0000","last_fullsized":"2024-04-24T06:42:43.165729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:09:57.271304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.883478+0000","last_change":"2024-04-24T06:42:42.883478+0000","last_active":"2024-04-24T06:42:42.883478+0000","last_peered":"2024-04-24T06:42:42.883478+0000","last_clean":"2024-04-24T06:42:42.883478+0000","last_became_active":"2024-04-24T06:42:42.882808+0000","last_became_peered":"2024-04-24T06:42:42.882808+0000","last_unstale":"2024-04-24T06:42:42.883478+0000","last_undegraded":"2024-04-24T06:42:42.883478+0000","last_fullsized":"2024-04-24T06:42:42.883478+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:17:11.514041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.876684+0000","last_change":"2024-04-24T06:41:34.277372+0000","last_active":"2024-04-24T06:42:42.876684+0000","last_peered":"2024-04-24T06:42:42.876684+0000","last_clean":"2024-04-24T06:42:42.876684+0000","last_became_active":"2024-04-24T06:41:34.276997+0000","last_became_peered":"2024-04-24T06:41:34.276997+0000","last_unstale":"2024-04-24T06:42:42.876684+0000","last_undegraded":"2024-04-24T06:42:42.876684+0000","last_fullsized":"2024-04-24T06:42:42.876684+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:52:35.695109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.922095+0000","last_change":"2024-04-24T06:41:35.290233+0000","last_active":"2024-04-24T06:42:42.922095+0000","last_peered":"2024-04-24T06:42:42.922095+0000","last_clean":"2024-04-24T06:42:42.922095+0000","last_became_active":"2024-04-24T06:41:35.289885+0000","last_became_peered":"2024-04-24T06:41:35.289885+0000","last_unstale":"2024-04-24T06:42:42.922095+0000","last_undegraded":"2024-04-24T06:42:42.922095+0000","last_fullsized":"2024-04-24T06:42:42.922095+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:35:54.293308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'27","reported_seq":210,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:57.467015+0000","last_change":"2024-04-24T06:41:34.267693+0000","last_active":"2024-04-24T06:42:57.467015+0000","last_peered":"2024-04-24T06:42:57.467015+0000","last_clean":"2024-04-24T06:42:57.467015+0000","last_became_active":"2024-04-24T06:41:34.267298+0000","last_became_peered":"2024-04-24T06:41:34.267298+0000","last_unstale":"2024-04-24T06:42:57.467015+0000","last_undegraded":"2024-04-24T06:42:57.467015+0000","last_fullsized":"2024-04-24T06:42:57.467015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:21:16.015695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60244,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":164,"num_read_kb":140,"num_write":28,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.875369+0000","last_change":"2024-04-24T06:41:35.286587+0000","last_active":"2024-04-24T06:42:42.875369+0000","last_peered":"2024-04-24T06:42:42.875369+0000","last_clean":"2024-04-24T06:42:42.875369+0000","last_became_active":"2024-04-24T06:41:35.286239+0000","last_became_peered":"2024-04-24T06:41:35.286239+0000","last_unstale":"2024-04-24T06:42:42.875369+0000","last_undegraded":"2024-04-24T06:42:42.875369+0000","last_fullsized":"2024-04-24T06:42:42.875369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:18:56.013425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.892090+0000","last_change":"2024-04-24T06:42:42.892090+0000","last_active":"2024-04-24T06:42:42.892090+0000","last_peered":"2024-04-24T06:42:42.892090+0000","last_clean":"2024-04-24T06:42:42.892090+0000","last_became_active":"2024-04-24T06:42:42.879904+0000","last_became_peered":"2024-04-24T06:42:42.879904+0000","last_unstale":"2024-04-24T06:42:42.892090+0000","last_undegraded":"2024-04-24T06:42:42.892090+0000","last_fullsized":"2024-04-24T06:42:42.892090+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:36:02.420438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.883128+0000","last_change":"2024-04-24T06:41:35.306175+0000","last_active":"2024-04-24T06:42:42.883128+0000","last_peered":"2024-04-24T06:42:42.883128+0000","last_clean":"2024-04-24T06:42:42.883128+0000","last_became_active":"2024-04-24T06:41:35.305840+0000","last_became_peered":"2024-04-24T06:41:35.305840+0000","last_unstale":"2024-04-24T06:42:42.883128+0000","last_undegraded":"2024-04-24T06:42:42.883128+0000","last_fullsized":"2024-04-24T06:42:42.883128+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:42:46.238490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.876274+0000","last_change":"2024-04-24T06:41:34.277212+0000","last_active":"2024-04-24T06:42:42.876274+0000","last_peered":"2024-04-24T06:42:42.876274+0000","last_clean":"2024-04-24T06:42:42.876274+0000","last_became_active":"2024-04-24T06:41:34.276695+0000","last_became_peered":"2024-04-24T06:41:34.276695+0000","last_unstale":"2024-04-24T06:42:42.876274+0000","last_undegraded":"2024-04-24T06:42:42.876274+0000","last_fullsized":"2024-04-24T06:42:42.876274+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:00:47.169384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:43.165743+0000","last_change":"2024-04-24T06:41:35.305348+0000","last_active":"2024-04-24T06:42:43.165743+0000","last_peered":"2024-04-24T06:42:43.165743+0000","last_clean":"2024-04-24T06:42:43.165743+0000","last_became_active":"2024-04-24T06:41:35.304899+0000","last_became_peered":"2024-04-24T06:41:35.304899+0000","last_unstale":"2024-04-24T06:42:43.165743+0000","last_undegraded":"2024-04-24T06:42:43.165743+0000","last_fullsized":"2024-04-24T06:42:43.165743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:49:35.737711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":104,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:58.485527+0000","last_change":"2024-04-24T06:41:34.272902+0000","last_active":"2024-04-24T06:42:58.485527+0000","last_peered":"2024-04-24T06:42:58.485527+0000","last_clean":"2024-04-24T06:42:58.485527+0000","last_became_active":"2024-04-24T06:41:34.271726+0000","last_became_peered":"2024-04-24T06:41:34.271726+0000","last_unstale":"2024-04-24T06:42:58.485527+0000","last_undegraded":"2024-04-24T06:42:58.485527+0000","last_fullsized":"2024-04-24T06:42:58.485527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:07:38.318110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":83,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:43.164962+0000","last_change":"2024-04-24T06:41:35.283764+0000","last_active":"2024-04-24T06:42:43.164962+0000","last_peered":"2024-04-24T06:42:43.164962+0000","last_clean":"2024-04-24T06:42:43.164962+0000","last_became_active":"2024-04-24T06:41:35.283400+0000","last_became_peered":"2024-04-24T06:41:35.283400+0000","last_unstale":"2024-04-24T06:42:43.164962+0000","last_undegraded":"2024-04-24T06:42:43.164962+0000","last_fullsized":"2024-04-24T06:42:43.164962+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:36:47.034068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.874899+0000","last_change":"2024-04-24T06:41:34.280020+0000","last_active":"2024-04-24T06:42:42.874899+0000","last_peered":"2024-04-24T06:42:42.874899+0000","last_clean":"2024-04-24T06:42:42.874899+0000","last_became_active":"2024-04-24T06:41:34.279659+0000","last_became_peered":"2024-04-24T06:41:34.279659+0000","last_unstale":"2024-04-24T06:42:42.874899+0000","last_undegraded":"2024-04-24T06:42:42.874899+0000","last_fullsized":"2024-04-24T06:42:42.874899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:13:29.529123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.876091+0000","last_change":"2024-04-24T06:41:35.284500+0000","last_active":"2024-04-24T06:42:42.876091+0000","last_peered":"2024-04-24T06:42:42.876091+0000","last_clean":"2024-04-24T06:42:42.876091+0000","last_became_active":"2024-04-24T06:41:35.283171+0000","last_became_peered":"2024-04-24T06:41:35.283171+0000","last_unstale":"2024-04-24T06:42:42.876091+0000","last_undegraded":"2024-04-24T06:42:42.876091+0000","last_fullsized":"2024-04-24T06:42:42.876091+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:33:19.527226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.875426+0000","last_change":"2024-04-24T06:41:34.279306+0000","last_active":"2024-04-24T06:42:42.875426+0000","last_peered":"2024-04-24T06:42:42.875426+0000","last_clean":"2024-04-24T06:42:42.875426+0000","last_became_active":"2024-04-24T06:41:34.278946+0000","last_became_peered":"2024-04-24T06:41:34.278946+0000","last_unstale":"2024-04-24T06:42:42.875426+0000","last_undegraded":"2024-04-24T06:42:42.875426+0000","last_fullsized":"2024-04-24T06:42:42.875426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:13:40.845871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.870469+0000","last_change":"2024-04-24T06:41:35.283716+0000","last_active":"2024-04-24T06:42:42.870469+0000","last_peered":"2024-04-24T06:42:42.870469+0000","last_clean":"2024-04-24T06:42:42.870469+0000","last_became_active":"2024-04-24T06:41:35.283330+0000","last_became_peered":"2024-04-24T06:41:35.283330+0000","last_unstale":"2024-04-24T06:42:42.870469+0000","last_undegraded":"2024-04-24T06:42:42.870469+0000","last_fullsized":"2024-04-24T06:42:42.870469+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:18:13.403264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.887349+0000","last_change":"2024-04-24T06:42:42.887349+0000","last_active":"2024-04-24T06:42:42.887349+0000","last_peered":"2024-04-24T06:42:42.887349+0000","last_clean":"2024-04-24T06:42:42.887349+0000","last_became_active":"2024-04-24T06:42:42.886813+0000","last_became_peered":"2024-04-24T06:42:42.886813+0000","last_unstale":"2024-04-24T06:42:42.887349+0000","last_undegraded":"2024-04-24T06:42:42.887349+0000","last_fullsized":"2024-04-24T06:42:42.887349+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:05:40.577189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.872971+0000","last_change":"2024-04-24T06:41:35.306607+0000","last_active":"2024-04-24T06:42:42.872971+0000","last_peered":"2024-04-24T06:42:42.872971+0000","last_clean":"2024-04-24T06:42:42.872971+0000","last_became_active":"2024-04-24T06:41:35.306217+0000","last_became_peered":"2024-04-24T06:41:35.306217+0000","last_unstale":"2024-04-24T06:42:42.872971+0000","last_undegraded":"2024-04-24T06:42:42.872971+0000","last_fullsized":"2024-04-24T06:42:42.872971+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:07:46.068876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:43.165230+0000","last_change":"2024-04-24T06:41:34.285319+0000","last_active":"2024-04-24T06:42:43.165230+0000","last_peered":"2024-04-24T06:42:43.165230+0000","last_clean":"2024-04-24T06:42:43.165230+0000","last_became_active":"2024-04-24T06:41:34.284912+0000","last_became_peered":"2024-04-24T06:41:34.284912+0000","last_unstale":"2024-04-24T06:42:43.165230+0000","last_undegraded":"2024-04-24T06:42:43.165230+0000","last_fullsized":"2024-04-24T06:42:43.165230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:34:56.454587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.873274+0000","last_change":"2024-04-24T06:41:35.296294+0000","last_active":"2024-04-24T06:42:42.873274+0000","last_peered":"2024-04-24T06:42:42.873274+0000","last_clean":"2024-04-24T06:42:42.873274+0000","last_became_active":"2024-04-24T06:41:35.295792+0000","last_became_peered":"2024-04-24T06:41:35.295792+0000","last_unstale":"2024-04-24T06:42:42.873274+0000","last_undegraded":"2024-04-24T06:42:42.873274+0000","last_fullsized":"2024-04-24T06:42:42.873274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:31:35.573858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.876723+0000","last_change":"2024-04-24T06:41:35.282558+0000","last_active":"2024-04-24T06:42:42.876723+0000","last_peered":"2024-04-24T06:42:42.876723+0000","last_clean":"2024-04-24T06:42:42.876723+0000","last_became_active":"2024-04-24T06:41:35.282163+0000","last_became_peered":"2024-04-24T06:41:35.282163+0000","last_unstale":"2024-04-24T06:42:42.876723+0000","last_undegraded":"2024-04-24T06:42:42.876723+0000","last_fullsized":"2024-04-24T06:42:42.876723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:12:23.569849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":104,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:59.486355+0000","last_change":"2024-04-24T06:41:34.284294+0000","last_active":"2024-04-24T06:42:59.486355+0000","last_peered":"2024-04-24T06:42:59.486355+0000","last_clean":"2024-04-24T06:42:59.486355+0000","last_became_active":"2024-04-24T06:41:34.283933+0000","last_became_peered":"2024-04-24T06:41:34.283933+0000","last_unstale":"2024-04-24T06:42:59.486355+0000","last_undegraded":"2024-04-24T06:42:59.486355+0000","last_fullsized":"2024-04-24T06:42:59.486355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:35:26.042614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":81,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.878680+0000","last_change":"2024-04-24T06:41:35.302361+0000","last_active":"2024-04-24T06:42:42.878680+0000","last_peered":"2024-04-24T06:42:42.878680+0000","last_clean":"2024-04-24T06:42:42.878680+0000","last_became_active":"2024-04-24T06:41:35.301652+0000","last_became_peered":"2024-04-24T06:41:35.301652+0000","last_unstale":"2024-04-24T06:42:42.878680+0000","last_undegraded":"2024-04-24T06:42:42.878680+0000","last_fullsized":"2024-04-24T06:42:42.878680+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:35:58.010853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.922470+0000","last_change":"2024-04-24T06:41:34.282156+0000","last_active":"2024-04-24T06:42:42.922470+0000","last_peered":"2024-04-24T06:42:42.922470+0000","last_clean":"2024-04-24T06:42:42.922470+0000","last_became_active":"2024-04-24T06:41:34.281771+0000","last_became_peered":"2024-04-24T06:41:34.281771+0000","last_unstale":"2024-04-24T06:42:42.922470+0000","last_undegraded":"2024-04-24T06:42:42.922470+0000","last_fullsized":"2024-04-24T06:42:42.922470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:41:49.584620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":355,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:43.165134+0000","last_change":"2024-04-24T06:40:16.681695+0000","last_active":"2024-04-24T06:42:43.165134+0000","last_peered":"2024-04-24T06:42:43.165134+0000","last_clean":"2024-04-24T06:42:43.165134+0000","last_became_active":"2024-04-24T06:40:16.681208+0000","last_became_peered":"2024-04-24T06:40:16.681208+0000","last_unstale":"2024-04-24T06:42:43.165134+0000","last_undegraded":"2024-04-24T06:42:43.165134+0000","last_fullsized":"2024-04-24T06:42:43.165134+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:36:27.950574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:36:27.950574+0000","last_clean_scrub_stamp":"2024-04-24T06:36:27.950574+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:39:01.985893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.885312+0000","last_change":"2024-04-24T06:42:42.885312+0000","last_active":"2024-04-24T06:42:42.885312+0000","last_peered":"2024-04-24T06:42:42.885312+0000","last_clean":"2024-04-24T06:42:42.885312+0000","last_became_active":"2024-04-24T06:42:42.884702+0000","last_became_peered":"2024-04-24T06:42:42.884702+0000","last_unstale":"2024-04-24T06:42:42.885312+0000","last_undegraded":"2024-04-24T06:42:42.885312+0000","last_fullsized":"2024-04-24T06:42:42.885312+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:33:44.590555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:43.165235+0000","last_change":"2024-04-24T06:41:34.273872+0000","last_active":"2024-04-24T06:42:43.165235+0000","last_peered":"2024-04-24T06:42:43.165235+0000","last_clean":"2024-04-24T06:42:43.165235+0000","last_became_active":"2024-04-24T06:41:34.273473+0000","last_became_peered":"2024-04-24T06:41:34.273473+0000","last_unstale":"2024-04-24T06:42:43.165235+0000","last_undegraded":"2024-04-24T06:42:43.165235+0000","last_fullsized":"2024-04-24T06:42:43.165235+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:47:11.914985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.875436+0000","last_change":"2024-04-24T06:41:34.289293+0000","last_active":"2024-04-24T06:42:42.875436+0000","last_peered":"2024-04-24T06:42:42.875436+0000","last_clean":"2024-04-24T06:42:42.875436+0000","last_became_active":"2024-04-24T06:41:34.288939+0000","last_became_peered":"2024-04-24T06:41:34.288939+0000","last_unstale":"2024-04-24T06:42:42.875436+0000","last_undegraded":"2024-04-24T06:42:42.875436+0000","last_fullsized":"2024-04-24T06:42:42.875436+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:03:01.026792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.879987+0000","last_change":"2024-04-24T06:41:35.302153+0000","last_active":"2024-04-24T06:42:42.879987+0000","last_peered":"2024-04-24T06:42:42.879987+0000","last_clean":"2024-04-24T06:42:42.879987+0000","last_became_active":"2024-04-24T06:41:35.301666+0000","last_became_peered":"2024-04-24T06:41:35.301666+0000","last_unstale":"2024-04-24T06:42:42.879987+0000","last_undegraded":"2024-04-24T06:42:42.879987+0000","last_fullsized":"2024-04-24T06:42:42.879987+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:45:09.642482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.875466+0000","last_change":"2024-04-24T06:41:34.291570+0000","last_active":"2024-04-24T06:42:42.875466+0000","last_peered":"2024-04-24T06:42:42.875466+0000","last_clean":"2024-04-24T06:42:42.875466+0000","last_became_active":"2024-04-24T06:41:34.289700+0000","last_became_peered":"2024-04-24T06:41:34.289700+0000","last_unstale":"2024-04-24T06:42:42.875466+0000","last_undegraded":"2024-04-24T06:42:42.875466+0000","last_fullsized":"2024-04-24T06:42:42.875466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:57:20.537299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.878064+0000","last_change":"2024-04-24T06:41:35.304703+0000","last_active":"2024-04-24T06:42:42.878064+0000","last_peered":"2024-04-24T06:42:42.878064+0000","last_clean":"2024-04-24T06:42:42.878064+0000","last_became_active":"2024-04-24T06:41:35.304319+0000","last_became_peered":"2024-04-24T06:41:35.304319+0000","last_unstale":"2024-04-24T06:42:42.878064+0000","last_undegraded":"2024-04-24T06:42:42.878064+0000","last_fullsized":"2024-04-24T06:42:42.878064+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:00:56.069509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.922285+0000","last_change":"2024-04-24T06:41:34.276866+0000","last_active":"2024-04-24T06:42:42.922285+0000","last_peered":"2024-04-24T06:42:42.922285+0000","last_clean":"2024-04-24T06:42:42.922285+0000","last_became_active":"2024-04-24T06:41:34.276478+0000","last_became_peered":"2024-04-24T06:41:34.276478+0000","last_unstale":"2024-04-24T06:42:42.922285+0000","last_undegraded":"2024-04-24T06:42:42.922285+0000","last_fullsized":"2024-04-24T06:42:42.922285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:10:35.047960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.873046+0000","last_change":"2024-04-24T06:41:35.300951+0000","last_active":"2024-04-24T06:42:42.873046+0000","last_peered":"2024-04-24T06:42:42.873046+0000","last_clean":"2024-04-24T06:42:42.873046+0000","last_became_active":"2024-04-24T06:41:35.296176+0000","last_became_peered":"2024-04-24T06:41:35.296176+0000","last_unstale":"2024-04-24T06:42:42.873046+0000","last_undegraded":"2024-04-24T06:42:42.873046+0000","last_fullsized":"2024-04-24T06:42:42.873046+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:53:58.497437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.921587+0000","last_change":"2024-04-24T06:41:34.276154+0000","last_active":"2024-04-24T06:42:42.921587+0000","last_peered":"2024-04-24T06:42:42.921587+0000","last_clean":"2024-04-24T06:42:42.921587+0000","last_became_active":"2024-04-24T06:41:34.275734+0000","last_became_peered":"2024-04-24T06:41:34.275734+0000","last_unstale":"2024-04-24T06:42:42.921587+0000","last_undegraded":"2024-04-24T06:42:42.921587+0000","last_fullsized":"2024-04-24T06:42:42.921587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:28:54.618061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.874889+0000","last_change":"2024-04-24T06:41:35.285837+0000","last_active":"2024-04-24T06:42:42.874889+0000","last_peered":"2024-04-24T06:42:42.874889+0000","last_clean":"2024-04-24T06:42:42.874889+0000","last_became_active":"2024-04-24T06:41:35.285505+0000","last_became_peered":"2024-04-24T06:41:35.285505+0000","last_unstale":"2024-04-24T06:42:42.874889+0000","last_undegraded":"2024-04-24T06:42:42.874889+0000","last_fullsized":"2024-04-24T06:42:42.874889+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:36:53.460780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.876765+0000","last_change":"2024-04-24T06:41:34.290039+0000","last_active":"2024-04-24T06:42:42.876765+0000","last_peered":"2024-04-24T06:42:42.876765+0000","last_clean":"2024-04-24T06:42:42.876765+0000","last_became_active":"2024-04-24T06:41:34.289228+0000","last_became_peered":"2024-04-24T06:41:34.289228+0000","last_unstale":"2024-04-24T06:42:42.876765+0000","last_undegraded":"2024-04-24T06:42:42.876765+0000","last_fullsized":"2024-04-24T06:42:42.876765+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:07:20.342359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.870454+0000","last_change":"2024-04-24T06:41:35.296214+0000","last_active":"2024-04-24T06:42:42.870454+0000","last_peered":"2024-04-24T06:42:42.870454+0000","last_clean":"2024-04-24T06:42:42.870454+0000","last_became_active":"2024-04-24T06:41:35.295888+0000","last_became_peered":"2024-04-24T06:41:35.295888+0000","last_unstale":"2024-04-24T06:42:42.870454+0000","last_undegraded":"2024-04-24T06:42:42.870454+0000","last_fullsized":"2024-04-24T06:42:42.870454+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:55:19.520479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.872600+0000","last_change":"2024-04-24T06:41:34.275885+0000","last_active":"2024-04-24T06:42:42.872600+0000","last_peered":"2024-04-24T06:42:42.872600+0000","last_clean":"2024-04-24T06:42:42.872600+0000","last_became_active":"2024-04-24T06:41:34.275503+0000","last_became_peered":"2024-04-24T06:41:34.275503+0000","last_unstale":"2024-04-24T06:42:42.872600+0000","last_undegraded":"2024-04-24T06:42:42.872600+0000","last_fullsized":"2024-04-24T06:42:42.872600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:50:58.216699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.877980+0000","last_change":"2024-04-24T06:41:35.305330+0000","last_active":"2024-04-24T06:42:42.877980+0000","last_peered":"2024-04-24T06:42:42.877980+0000","last_clean":"2024-04-24T06:42:42.877980+0000","last_became_active":"2024-04-24T06:41:35.301262+0000","last_became_peered":"2024-04-24T06:41:35.301262+0000","last_unstale":"2024-04-24T06:42:42.877980+0000","last_undegraded":"2024-04-24T06:42:42.877980+0000","last_fullsized":"2024-04-24T06:42:42.877980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:58:41.029511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.875278+0000","last_change":"2024-04-24T06:41:34.276926+0000","last_active":"2024-04-24T06:42:42.875278+0000","last_peered":"2024-04-24T06:42:42.875278+0000","last_clean":"2024-04-24T06:42:42.875278+0000","last_became_active":"2024-04-24T06:41:34.276539+0000","last_became_peered":"2024-04-24T06:41:34.276539+0000","last_unstale":"2024-04-24T06:42:42.875278+0000","last_undegraded":"2024-04-24T06:42:42.875278+0000","last_fullsized":"2024-04-24T06:42:42.875278+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:28:56.652584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:43.165259+0000","last_change":"2024-04-24T06:41:35.301052+0000","last_active":"2024-04-24T06:42:43.165259+0000","last_peered":"2024-04-24T06:42:43.165259+0000","last_clean":"2024-04-24T06:42:43.165259+0000","last_became_active":"2024-04-24T06:41:35.300650+0000","last_became_peered":"2024-04-24T06:41:35.300650+0000","last_unstale":"2024-04-24T06:42:43.165259+0000","last_undegraded":"2024-04-24T06:42:43.165259+0000","last_fullsized":"2024-04-24T06:42:43.165259+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:06:54.290353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.873168+0000","last_change":"2024-04-24T06:41:34.273773+0000","last_active":"2024-04-24T06:42:42.873168+0000","last_peered":"2024-04-24T06:42:42.873168+0000","last_clean":"2024-04-24T06:42:42.873168+0000","last_became_active":"2024-04-24T06:41:34.273324+0000","last_became_peered":"2024-04-24T06:41:34.273324+0000","last_unstale":"2024-04-24T06:42:42.873168+0000","last_undegraded":"2024-04-24T06:42:42.873168+0000","last_fullsized":"2024-04-24T06:42:42.873168+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:30:30.760740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:43.165693+0000","last_change":"2024-04-24T06:41:35.289565+0000","last_active":"2024-04-24T06:42:43.165693+0000","last_peered":"2024-04-24T06:42:43.165693+0000","last_clean":"2024-04-24T06:42:43.165693+0000","last_became_active":"2024-04-24T06:41:35.289185+0000","last_became_peered":"2024-04-24T06:41:35.289185+0000","last_unstale":"2024-04-24T06:42:43.165693+0000","last_undegraded":"2024-04-24T06:42:43.165693+0000","last_fullsized":"2024-04-24T06:42:43.165693+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:26:28.345162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.876467+0000","last_change":"2024-04-24T06:41:34.285804+0000","last_active":"2024-04-24T06:42:42.876467+0000","last_peered":"2024-04-24T06:42:42.876467+0000","last_clean":"2024-04-24T06:42:42.876467+0000","last_became_active":"2024-04-24T06:41:34.285501+0000","last_became_peered":"2024-04-24T06:41:34.285501+0000","last_unstale":"2024-04-24T06:42:42.876467+0000","last_undegraded":"2024-04-24T06:42:42.876467+0000","last_fullsized":"2024-04-24T06:42:42.876467+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:25:15.804221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:43.165519+0000","last_change":"2024-04-24T06:41:35.300856+0000","last_active":"2024-04-24T06:42:43.165519+0000","last_peered":"2024-04-24T06:42:43.165519+0000","last_clean":"2024-04-24T06:42:43.165519+0000","last_became_active":"2024-04-24T06:41:35.300344+0000","last_became_peered":"2024-04-24T06:41:35.300344+0000","last_unstale":"2024-04-24T06:42:43.165519+0000","last_undegraded":"2024-04-24T06:42:43.165519+0000","last_fullsized":"2024-04-24T06:42:43.165519+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:10:01.866954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.880835+0000","last_change":"2024-04-24T06:41:34.284112+0000","last_active":"2024-04-24T06:42:42.880835+0000","last_peered":"2024-04-24T06:42:42.880835+0000","last_clean":"2024-04-24T06:42:42.880835+0000","last_became_active":"2024-04-24T06:41:34.279317+0000","last_became_peered":"2024-04-24T06:41:34.279317+0000","last_unstale":"2024-04-24T06:42:42.880835+0000","last_undegraded":"2024-04-24T06:42:42.880835+0000","last_fullsized":"2024-04-24T06:42:42.880835+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:49:45.673353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:43.165795+0000","last_change":"2024-04-24T06:41:35.288360+0000","last_active":"2024-04-24T06:42:43.165795+0000","last_peered":"2024-04-24T06:42:43.165795+0000","last_clean":"2024-04-24T06:42:43.165795+0000","last_became_active":"2024-04-24T06:41:35.288019+0000","last_became_peered":"2024-04-24T06:41:35.288019+0000","last_unstale":"2024-04-24T06:42:43.165795+0000","last_undegraded":"2024-04-24T06:42:43.165795+0000","last_fullsized":"2024-04-24T06:42:43.165795+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:33:49.927591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.870599+0000","last_change":"2024-04-24T06:41:35.286137+0000","last_active":"2024-04-24T06:42:42.870599+0000","last_peered":"2024-04-24T06:42:42.870599+0000","last_clean":"2024-04-24T06:42:42.870599+0000","last_became_active":"2024-04-24T06:41:35.285773+0000","last_became_peered":"2024-04-24T06:41:35.285773+0000","last_unstale":"2024-04-24T06:42:42.870599+0000","last_undegraded":"2024-04-24T06:42:42.870599+0000","last_fullsized":"2024-04-24T06:42:42.870599+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:29:32.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.872717+0000","last_change":"2024-04-24T06:41:35.283977+0000","last_active":"2024-04-24T06:42:42.872717+0000","last_peered":"2024-04-24T06:42:42.872717+0000","last_clean":"2024-04-24T06:42:42.872717+0000","last_became_active":"2024-04-24T06:41:35.282312+0000","last_became_peered":"2024-04-24T06:41:35.282312+0000","last_unstale":"2024-04-24T06:42:42.872717+0000","last_undegraded":"2024-04-24T06:42:42.872717+0000","last_fullsized":"2024-04-24T06:42:42.872717+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:41:36.463803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.872593+0000","last_change":"2024-04-24T06:41:35.300372+0000","last_active":"2024-04-24T06:42:42.872593+0000","last_peered":"2024-04-24T06:42:42.872593+0000","last_clean":"2024-04-24T06:42:42.872593+0000","last_became_active":"2024-04-24T06:41:35.300034+0000","last_became_peered":"2024-04-24T06:41:35.300034+0000","last_unstale":"2024-04-24T06:42:42.872593+0000","last_undegraded":"2024-04-24T06:42:42.872593+0000","last_fullsized":"2024-04-24T06:42:42.872593+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:56:23.732798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.874606+0000","last_change":"2024-04-24T06:41:35.301619+0000","last_active":"2024-04-24T06:42:42.874606+0000","last_peered":"2024-04-24T06:42:42.874606+0000","last_clean":"2024-04-24T06:42:42.874606+0000","last_became_active":"2024-04-24T06:41:35.298897+0000","last_became_peered":"2024-04-24T06:41:35.298897+0000","last_unstale":"2024-04-24T06:42:42.874606+0000","last_undegraded":"2024-04-24T06:42:42.874606+0000","last_fullsized":"2024-04-24T06:42:42.874606+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:26:11.583079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.880110+0000","last_change":"2024-04-24T06:41:35.304905+0000","last_active":"2024-04-24T06:42:42.880110+0000","last_peered":"2024-04-24T06:42:42.880110+0000","last_clean":"2024-04-24T06:42:42.880110+0000","last_became_active":"2024-04-24T06:41:35.304407+0000","last_became_peered":"2024-04-24T06:41:35.304407+0000","last_unstale":"2024-04-24T06:42:42.880110+0000","last_undegraded":"2024-04-24T06:42:42.880110+0000","last_fullsized":"2024-04-24T06:42:42.880110+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:38:07.417898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.870617+0000","last_change":"2024-04-24T06:41:35.288646+0000","last_active":"2024-04-24T06:42:42.870617+0000","last_peered":"2024-04-24T06:42:42.870617+0000","last_clean":"2024-04-24T06:42:42.870617+0000","last_became_active":"2024-04-24T06:41:35.288266+0000","last_became_peered":"2024-04-24T06:41:35.288266+0000","last_unstale":"2024-04-24T06:42:42.870617+0000","last_undegraded":"2024-04-24T06:42:42.870617+0000","last_fullsized":"2024-04-24T06:42:42.870617+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:59:16.526712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.873283+0000","last_change":"2024-04-24T06:41:35.285001+0000","last_active":"2024-04-24T06:42:42.873283+0000","last_peered":"2024-04-24T06:42:42.873283+0000","last_clean":"2024-04-24T06:42:42.873283+0000","last_became_active":"2024-04-24T06:41:35.284671+0000","last_became_peered":"2024-04-24T06:41:35.284671+0000","last_unstale":"2024-04-24T06:42:42.873283+0000","last_undegraded":"2024-04-24T06:42:42.873283+0000","last_fullsized":"2024-04-24T06:42:42.873283+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:37:43.534018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.877784+0000","last_change":"2024-04-24T06:41:35.299124+0000","last_active":"2024-04-24T06:42:42.877784+0000","last_peered":"2024-04-24T06:42:42.877784+0000","last_clean":"2024-04-24T06:42:42.877784+0000","last_became_active":"2024-04-24T06:41:35.298246+0000","last_became_peered":"2024-04-24T06:41:35.298246+0000","last_unstale":"2024-04-24T06:42:42.877784+0000","last_undegraded":"2024-04-24T06:42:42.877784+0000","last_fullsized":"2024-04-24T06:42:42.877784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:07:22.649874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.889015+0000","last_change":"2024-04-24T06:42:42.888183+0000","last_active":"2024-04-24T06:42:42.889015+0000","last_peered":"2024-04-24T06:42:42.889015+0000","last_clean":"2024-04-24T06:42:42.889015+0000","last_became_active":"2024-04-24T06:42:42.887373+0000","last_became_peered":"2024-04-24T06:42:42.887373+0000","last_unstale":"2024-04-24T06:42:42.889015+0000","last_undegraded":"2024-04-24T06:42:42.889015+0000","last_fullsized":"2024-04-24T06:42:42.889015+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:44:51.456753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.876609+0000","last_change":"2024-04-24T06:41:35.301619+0000","last_active":"2024-04-24T06:42:42.876609+0000","last_peered":"2024-04-24T06:42:42.876609+0000","last_clean":"2024-04-24T06:42:42.876609+0000","last_became_active":"2024-04-24T06:41:35.297987+0000","last_became_peered":"2024-04-24T06:41:35.297987+0000","last_unstale":"2024-04-24T06:42:42.876609+0000","last_undegraded":"2024-04-24T06:42:42.876609+0000","last_fullsized":"2024-04-24T06:42:42.876609+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:06:01.235593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.921904+0000","last_change":"2024-04-24T06:41:35.304347+0000","last_active":"2024-04-24T06:42:42.921904+0000","last_peered":"2024-04-24T06:42:42.921904+0000","last_clean":"2024-04-24T06:42:42.921904+0000","last_became_active":"2024-04-24T06:41:35.303971+0000","last_became_peered":"2024-04-24T06:41:35.303971+0000","last_unstale":"2024-04-24T06:42:42.921904+0000","last_undegraded":"2024-04-24T06:42:42.921904+0000","last_fullsized":"2024-04-24T06:42:42.921904+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:31:54.813243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.880070+0000","last_change":"2024-04-24T06:41:35.308314+0000","last_active":"2024-04-24T06:42:42.880070+0000","last_peered":"2024-04-24T06:42:42.880070+0000","last_clean":"2024-04-24T06:42:42.880070+0000","last_became_active":"2024-04-24T06:41:35.306581+0000","last_became_peered":"2024-04-24T06:41:35.306581+0000","last_unstale":"2024-04-24T06:42:42.880070+0000","last_undegraded":"2024-04-24T06:42:42.880070+0000","last_fullsized":"2024-04-24T06:42:42.880070+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:34:43.095087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.870482+0000","last_change":"2024-04-24T06:41:35.289166+0000","last_active":"2024-04-24T06:42:42.870482+0000","last_peered":"2024-04-24T06:42:42.870482+0000","last_clean":"2024-04-24T06:42:42.870482+0000","last_became_active":"2024-04-24T06:41:35.287672+0000","last_became_peered":"2024-04-24T06:41:35.287672+0000","last_unstale":"2024-04-24T06:42:42.870482+0000","last_undegraded":"2024-04-24T06:42:42.870482+0000","last_fullsized":"2024-04-24T06:42:42.870482+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:31:45.403387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63854,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":500,"num_read_kb":317,"num_write":156,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":191562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351006,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":460828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612643,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1658880,"data_stored":1112519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776168,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":978944,"data_stored":461030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037807,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":460950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299444,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28556,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716596,"statfs":{"total":95995035648,"available":95965794304,"internally_reserved":0,"allocated":1572864,"data_stored":1052489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462971,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":460419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1028096,"data_stored":520456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":987136,"data_stored":461690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1048576,"data_stored":521772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":460730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920838,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":460856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724609,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32728,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712424,"statfs":{"total":95995035648,"available":95961522176,"internally_reserved":0,"allocated":1585152,"data_stored":1052173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":61560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T06:43:04.094 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-24T06:43:04.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:04 smithi137 ceph-mon[33017]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T06:43:04.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:04 smithi005 ceph-mon[27798]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T06:43:04.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:04 smithi037 ceph-mon[32864]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T06:43:04.610 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:43:05.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:05 smithi137 ceph-mon[33017]: pgmap v359: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T06:43:05.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:05 smithi005 ceph-mon[27798]: pgmap v359: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T06:43:05.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:05 smithi037 ceph-mon[32864]: pgmap v359: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T06:43:06.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.862+0000 7f4b61000700 1 -- 172.21.15.5:0/860168259 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b5c103500 msgr2=0x7f4b5c10f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:06.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.862+0000 7f4b61000700 1 --2- 172.21.15.5:0/860168259 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b5c103500 0x7f4b5c10f960 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4b54009f90 tx=0x7f4b5400b330 comp rx=0 tx=0).stop 2024-04-24T06:43:06.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.862+0000 7f4b61000700 1 -- 172.21.15.5:0/860168259 shutdown_connections 2024-04-24T06:43:06.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.862+0000 7f4b61000700 1 --2- 172.21.15.5:0/860168259 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b5c103500 0x7f4b5c10f960 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:06.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.862+0000 7f4b61000700 1 --2- 172.21.15.5:0/860168259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b5c108b90 0x7f4b5c108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:06.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.862+0000 7f4b61000700 1 --2- 172.21.15.5:0/860168259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b5c102b40 0x7f4b5c102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:06.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.862+0000 7f4b61000700 1 -- 172.21.15.5:0/860168259 >> 172.21.15.5:0/860168259 conn(0x7f4b5c079c50 msgr2=0x7f4b5c07a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:06.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.863+0000 7f4b61000700 1 -- 172.21.15.5:0/860168259 shutdown_connections 2024-04-24T06:43:06.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.863+0000 7f4b61000700 1 -- 172.21.15.5:0/860168259 wait complete. 2024-04-24T06:43:06.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.864+0000 7f4b61000700 1 Processor -- start 2024-04-24T06:43:06.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.864+0000 7f4b61000700 1 -- start start 2024-04-24T06:43:06.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.864+0000 7f4b61000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b5c102b40 0x7f4b5c074df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:06.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.864+0000 7f4b61000700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b5c103500 0x7f4b5c075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:06.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.864+0000 7f4b61000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b5c108b90 0x7f4b5c075870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:06.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.864+0000 7f4b61000700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5c112e40 con 0x7f4b5c103500 2024-04-24T06:43:06.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.864+0000 7f4b61000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5c112cc0 con 0x7f4b5c108b90 2024-04-24T06:43:06.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.864+0000 7f4b61000700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5c112fc0 con 0x7f4b5c102b40 2024-04-24T06:43:06.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.865+0000 7f4b59e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b5c102b40 0x7f4b5c074df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:06.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.865+0000 7f4b59400700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b5c103500 0x7f4b5c075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:06.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.865+0000 7f4b59e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b5c102b40 0x7f4b5c074df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:47032/0 (socket says 172.21.15.5:47032) 2024-04-24T06:43:06.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.865+0000 7f4b59e00700 1 -- 172.21.15.5:0/1943819051 learned_addr learned my addr 172.21.15.5:0/1943819051 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:06.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.865+0000 7f4b5a800700 1 --2- 172.21.15.5:0/1943819051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b5c108b90 0x7f4b5c075870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:06.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.865+0000 7f4b59e00700 1 -- 172.21.15.5:0/1943819051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b5c108b90 msgr2=0x7f4b5c075870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:06.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.865+0000 7f4b59e00700 1 --2- 172.21.15.5:0/1943819051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b5c108b90 0x7f4b5c075870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:06.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.865+0000 7f4b59e00700 1 -- 172.21.15.5:0/1943819051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b5c103500 msgr2=0x7f4b5c075330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:06.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.865+0000 7f4b59e00700 1 --2- 172.21.15.5:0/1943819051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b5c103500 0x7f4b5c075330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:06.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.865+0000 7f4b59e00700 1 -- 172.21.15.5:0/1943819051 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b5c1a77e0 con 0x7f4b5c102b40 2024-04-24T06:43:06.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.865+0000 7f4b5a800700 1 --2- 172.21.15.5:0/1943819051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b5c108b90 0x7f4b5c075870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:43:06.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.865+0000 7f4b59400700 1 --2- 172.21.15.5:0/1943819051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b5c103500 0x7f4b5c075330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:43:06.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.865+0000 7f4b59e00700 1 --2- 172.21.15.5:0/1943819051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b5c102b40 0x7f4b5c074df0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00e9c0 tx=0x7f4b4c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:06.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.866+0000 7f4b52a00700 1 -- 172.21.15.5:0/1943819051 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c00cbd0 con 0x7f4b5c102b40 2024-04-24T06:43:06.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.866+0000 7f4b52a00700 1 -- 172.21.15.5:0/1943819051 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b4c00cd70 con 0x7f4b5c102b40 2024-04-24T06:43:06.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.866+0000 7f4b61000700 1 -- 172.21.15.5:0/1943819051 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b5c1a7ad0 con 0x7f4b5c102b40 2024-04-24T06:43:06.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.866+0000 7f4b61000700 1 -- 172.21.15.5:0/1943819051 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b5c1a8060 con 0x7f4b5c102b40 2024-04-24T06:43:06.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.866+0000 7f4b52a00700 1 -- 172.21.15.5:0/1943819051 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c005660 con 0x7f4b5c102b40 2024-04-24T06:43:06.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.867+0000 7f4b61000700 1 -- 172.21.15.5:0/1943819051 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b3c005330 con 0x7f4b5c102b40 2024-04-24T06:43:06.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.869+0000 7f4b52a00700 1 -- 172.21.15.5:0/1943819051 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f4b4c004750 con 0x7f4b5c102b40 2024-04-24T06:43:06.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.871+0000 7f4b52a00700 1 --2- 172.21.15.5:0/1943819051 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f4b400a3d50 0x7f4b400a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:06.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.871+0000 7f4b52a00700 1 -- 172.21.15.5:0/1943819051 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f4b4c0d04e0 con 0x7f4b5c102b40 2024-04-24T06:43:06.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.871+0000 7f4b59400700 1 --2- 172.21.15.5:0/1943819051 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f4b400a3d50 0x7f4b400a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:06.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.872+0000 7f4b59400700 1 --2- 172.21.15.5:0/1943819051 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f4b400a3d50 0x7f4b400a6210 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c07a230 tx=0x7f4b48009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:06.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:06.875+0000 7f4b52a00700 1 -- 172.21.15.5:0/1943819051 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4b4c016330 con 0x7f4b5c102b40 2024-04-24T06:43:07.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:07.106+0000 7f4b61000700 1 -- 172.21.15.5:0/1943819051 --> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4b3c000bc0 con 0x7f4b400a3d50 2024-04-24T06:43:07.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:07.114+0000 7f4b52a00700 1 -- 172.21.15.5:0/1943819051 <== mgr.14152 v2:172.21.15.5:6800/4042936965 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258708 (secure 0 0 0) 0x7f4b3c000bc0 con 0x7f4b400a3d50 2024-04-24T06:43:07.117 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:43:07.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:07.123+0000 7f4b61000700 1 -- 172.21.15.5:0/1943819051 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f4b400a3d50 msgr2=0x7f4b400a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:07.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:07.123+0000 7f4b61000700 1 --2- 172.21.15.5:0/1943819051 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f4b400a3d50 0x7f4b400a6210 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c07a230 tx=0x7f4b48009260 comp rx=0 tx=0).stop 2024-04-24T06:43:07.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:07.123+0000 7f4b61000700 1 -- 172.21.15.5:0/1943819051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b5c102b40 msgr2=0x7f4b5c074df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:07.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:07.123+0000 7f4b61000700 1 --2- 172.21.15.5:0/1943819051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b5c102b40 0x7f4b5c074df0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00e9c0 tx=0x7f4b4c00ecd0 comp rx=0 tx=0).stop 2024-04-24T06:43:07.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:07.123+0000 7f4b61000700 1 -- 172.21.15.5:0/1943819051 shutdown_connections 2024-04-24T06:43:07.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:07.123+0000 7f4b61000700 1 --2- 172.21.15.5:0/1943819051 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f4b400a3d50 0x7f4b400a6210 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:07.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:07.123+0000 7f4b61000700 1 --2- 172.21.15.5:0/1943819051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b5c108b90 0x7f4b5c075870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:07.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:07.123+0000 7f4b61000700 1 --2- 172.21.15.5:0/1943819051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b5c102b40 0x7f4b5c074df0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:07.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:07.123+0000 7f4b61000700 1 --2- 172.21.15.5:0/1943819051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b5c103500 0x7f4b5c075330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:07.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:07.123+0000 7f4b61000700 1 -- 172.21.15.5:0/1943819051 >> 172.21.15.5:0/1943819051 conn(0x7f4b5c079c50 msgr2=0x7f4b5c1017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:07.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:07.123+0000 7f4b61000700 1 -- 172.21.15.5:0/1943819051 shutdown_connections 2024-04-24T06:43:07.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:07.123+0000 7f4b61000700 1 -- 172.21.15.5:0/1943819051 wait complete. 2024-04-24T06:43:07.127 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-24T06:43:07.321 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:07 smithi037 ceph-mon[32864]: pgmap v360: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:43:07.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:07 smithi137 ceph-mon[33017]: pgmap v360: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:43:07.487 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:07 smithi005 ceph-mon[27798]: pgmap v360: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:43:08.042 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-04-24T06:43:05.340176+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654456,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":661,"num_read_kb":446,"num_write":339,"num_write_kb":3046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":273,"ondisk_log_size":273,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375108,"kb_used_data":13404,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566716,"statfs":{"total":1151940427776,"available":1151556317184,"internally_reserved":0,"allocated":13725696,"data_stored":7485912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004986"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.870668+0000","last_change":"2024-04-24T06:41:35.296451+0000","last_active":"2024-04-24T06:42:42.870668+0000","last_peered":"2024-04-24T06:42:42.870668+0000","last_clean":"2024-04-24T06:42:42.870668+0000","last_became_active":"2024-04-24T06:41:35.295733+0000","last_became_peered":"2024-04-24T06:41:35.295733+0000","last_unstale":"2024-04-24T06:42:42.870668+0000","last_undegraded":"2024-04-24T06:42:42.870668+0000","last_fullsized":"2024-04-24T06:42:42.870668+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:04:15.072644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.873777+0000","last_change":"2024-04-24T06:41:35.293359+0000","last_active":"2024-04-24T06:42:42.873777+0000","last_peered":"2024-04-24T06:42:42.873777+0000","last_clean":"2024-04-24T06:42:42.873777+0000","last_became_active":"2024-04-24T06:41:35.287371+0000","last_became_peered":"2024-04-24T06:41:35.287371+0000","last_unstale":"2024-04-24T06:42:42.873777+0000","last_undegraded":"2024-04-24T06:42:42.873777+0000","last_fullsized":"2024-04-24T06:42:42.873777+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:03:39.376495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.873042+0000","last_change":"2024-04-24T06:41:35.272734+0000","last_active":"2024-04-24T06:42:42.873042+0000","last_peered":"2024-04-24T06:42:42.873042+0000","last_clean":"2024-04-24T06:42:42.873042+0000","last_became_active":"2024-04-24T06:41:35.271685+0000","last_became_peered":"2024-04-24T06:41:35.271685+0000","last_unstale":"2024-04-24T06:42:42.873042+0000","last_undegraded":"2024-04-24T06:42:42.873042+0000","last_fullsized":"2024-04-24T06:42:42.873042+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:08:15.057918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.922346+0000","last_change":"2024-04-24T06:41:35.288574+0000","last_active":"2024-04-24T06:42:42.922346+0000","last_peered":"2024-04-24T06:42:42.922346+0000","last_clean":"2024-04-24T06:42:42.922346+0000","last_became_active":"2024-04-24T06:41:35.288229+0000","last_became_peered":"2024-04-24T06:41:35.288229+0000","last_unstale":"2024-04-24T06:42:42.922346+0000","last_undegraded":"2024-04-24T06:42:42.922346+0000","last_fullsized":"2024-04-24T06:42:42.922346+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:58:18.028067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.874961+0000","last_change":"2024-04-24T06:41:35.310226+0000","last_active":"2024-04-24T06:42:42.874961+0000","last_peered":"2024-04-24T06:42:42.874961+0000","last_clean":"2024-04-24T06:42:42.874961+0000","last_became_active":"2024-04-24T06:41:35.309853+0000","last_became_peered":"2024-04-24T06:41:35.309853+0000","last_unstale":"2024-04-24T06:42:42.874961+0000","last_undegraded":"2024-04-24T06:42:42.874961+0000","last_fullsized":"2024-04-24T06:42:42.874961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:34:57.767158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.873418+0000","last_change":"2024-04-24T06:41:35.294259+0000","last_active":"2024-04-24T06:42:42.873418+0000","last_peered":"2024-04-24T06:42:42.873418+0000","last_clean":"2024-04-24T06:42:42.873418+0000","last_became_active":"2024-04-24T06:41:35.293827+0000","last_became_peered":"2024-04-24T06:41:35.293827+0000","last_unstale":"2024-04-24T06:42:42.873418+0000","last_undegraded":"2024-04-24T06:42:42.873418+0000","last_fullsized":"2024-04-24T06:42:42.873418+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:09:23.302827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:43.165104+0000","last_change":"2024-04-24T06:41:35.295095+0000","last_active":"2024-04-24T06:42:43.165104+0000","last_peered":"2024-04-24T06:42:43.165104+0000","last_clean":"2024-04-24T06:42:43.165104+0000","last_became_active":"2024-04-24T06:41:35.294743+0000","last_became_peered":"2024-04-24T06:41:35.294743+0000","last_unstale":"2024-04-24T06:42:43.165104+0000","last_undegraded":"2024-04-24T06:42:43.165104+0000","last_fullsized":"2024-04-24T06:42:43.165104+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:06:09.473123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.889034+0000","last_change":"2024-04-24T06:42:42.886072+0000","last_active":"2024-04-24T06:42:42.889034+0000","last_peered":"2024-04-24T06:42:42.889034+0000","last_clean":"2024-04-24T06:42:42.889034+0000","last_became_active":"2024-04-24T06:42:42.882624+0000","last_became_peered":"2024-04-24T06:42:42.882624+0000","last_unstale":"2024-04-24T06:42:42.889034+0000","last_undegraded":"2024-04-24T06:42:42.889034+0000","last_fullsized":"2024-04-24T06:42:42.889034+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:53:02.523012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.922808+0000","last_change":"2024-04-24T06:41:35.283275+0000","last_active":"2024-04-24T06:42:42.922808+0000","last_peered":"2024-04-24T06:42:42.922808+0000","last_clean":"2024-04-24T06:42:42.922808+0000","last_became_active":"2024-04-24T06:41:35.282586+0000","last_became_peered":"2024-04-24T06:41:35.282586+0000","last_unstale":"2024-04-24T06:42:42.922808+0000","last_undegraded":"2024-04-24T06:42:42.922808+0000","last_fullsized":"2024-04-24T06:42:42.922808+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:42:30.716149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.873161+0000","last_change":"2024-04-24T06:41:35.284986+0000","last_active":"2024-04-24T06:42:42.873161+0000","last_peered":"2024-04-24T06:42:42.873161+0000","last_clean":"2024-04-24T06:42:42.873161+0000","last_became_active":"2024-04-24T06:41:35.284628+0000","last_became_peered":"2024-04-24T06:41:35.284628+0000","last_unstale":"2024-04-24T06:42:42.873161+0000","last_undegraded":"2024-04-24T06:42:42.873161+0000","last_fullsized":"2024-04-24T06:42:42.873161+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:29:38.479593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:43.165654+0000","last_change":"2024-04-24T06:41:35.290353+0000","last_active":"2024-04-24T06:42:43.165654+0000","last_peered":"2024-04-24T06:42:43.165654+0000","last_clean":"2024-04-24T06:42:43.165654+0000","last_became_active":"2024-04-24T06:41:35.289987+0000","last_became_peered":"2024-04-24T06:41:35.289987+0000","last_unstale":"2024-04-24T06:42:43.165654+0000","last_undegraded":"2024-04-24T06:42:43.165654+0000","last_fullsized":"2024-04-24T06:42:43.165654+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:56:54.914426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.922207+0000","last_change":"2024-04-24T06:41:35.289462+0000","last_active":"2024-04-24T06:42:42.922207+0000","last_peered":"2024-04-24T06:42:42.922207+0000","last_clean":"2024-04-24T06:42:42.922207+0000","last_became_active":"2024-04-24T06:41:35.289123+0000","last_became_peered":"2024-04-24T06:41:35.289123+0000","last_unstale":"2024-04-24T06:42:42.922207+0000","last_undegraded":"2024-04-24T06:42:42.922207+0000","last_fullsized":"2024-04-24T06:42:42.922207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:51:06.792508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.880246+0000","last_change":"2024-04-24T06:41:35.298320+0000","last_active":"2024-04-24T06:42:42.880246+0000","last_peered":"2024-04-24T06:42:42.880246+0000","last_clean":"2024-04-24T06:42:42.880246+0000","last_became_active":"2024-04-24T06:41:35.297839+0000","last_became_peered":"2024-04-24T06:41:35.297839+0000","last_unstale":"2024-04-24T06:42:42.880246+0000","last_undegraded":"2024-04-24T06:42:42.880246+0000","last_fullsized":"2024-04-24T06:42:42.880246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:10:36.423514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.872554+0000","last_change":"2024-04-24T06:41:35.301924+0000","last_active":"2024-04-24T06:42:42.872554+0000","last_peered":"2024-04-24T06:42:42.872554+0000","last_clean":"2024-04-24T06:42:42.872554+0000","last_became_active":"2024-04-24T06:41:35.299114+0000","last_became_peered":"2024-04-24T06:41:35.299114+0000","last_unstale":"2024-04-24T06:42:42.872554+0000","last_undegraded":"2024-04-24T06:42:42.872554+0000","last_fullsized":"2024-04-24T06:42:42.872554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:34:46.194266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.882096+0000","last_change":"2024-04-24T06:41:35.284885+0000","last_active":"2024-04-24T06:42:42.882096+0000","last_peered":"2024-04-24T06:42:42.882096+0000","last_clean":"2024-04-24T06:42:42.882096+0000","last_became_active":"2024-04-24T06:41:35.284244+0000","last_became_peered":"2024-04-24T06:41:35.284244+0000","last_unstale":"2024-04-24T06:42:42.882096+0000","last_undegraded":"2024-04-24T06:42:42.882096+0000","last_fullsized":"2024-04-24T06:42:42.882096+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:30:40.279905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.921571+0000","last_change":"2024-04-24T06:41:35.286363+0000","last_active":"2024-04-24T06:42:42.921571+0000","last_peered":"2024-04-24T06:42:42.921571+0000","last_clean":"2024-04-24T06:42:42.921571+0000","last_became_active":"2024-04-24T06:41:35.286024+0000","last_became_peered":"2024-04-24T06:41:35.286024+0000","last_unstale":"2024-04-24T06:42:42.921571+0000","last_undegraded":"2024-04-24T06:42:42.921571+0000","last_fullsized":"2024-04-24T06:42:42.921571+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:43:35.728990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.870846+0000","last_change":"2024-04-24T06:41:35.304005+0000","last_active":"2024-04-24T06:42:42.870846+0000","last_peered":"2024-04-24T06:42:42.870846+0000","last_clean":"2024-04-24T06:42:42.870846+0000","last_became_active":"2024-04-24T06:41:35.303659+0000","last_became_peered":"2024-04-24T06:41:35.303659+0000","last_unstale":"2024-04-24T06:42:42.870846+0000","last_undegraded":"2024-04-24T06:42:42.870846+0000","last_fullsized":"2024-04-24T06:42:42.870846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:05:07.215927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.875662+0000","last_change":"2024-04-24T06:41:35.296218+0000","last_active":"2024-04-24T06:42:42.875662+0000","last_peered":"2024-04-24T06:42:42.875662+0000","last_clean":"2024-04-24T06:42:42.875662+0000","last_became_active":"2024-04-24T06:41:35.295486+0000","last_became_peered":"2024-04-24T06:41:35.295486+0000","last_unstale":"2024-04-24T06:42:42.875662+0000","last_undegraded":"2024-04-24T06:42:42.875662+0000","last_fullsized":"2024-04-24T06:42:42.875662+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:17:25.844899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.886291+0000","last_change":"2024-04-24T06:42:42.886291+0000","last_active":"2024-04-24T06:42:42.886291+0000","last_peered":"2024-04-24T06:42:42.886291+0000","last_clean":"2024-04-24T06:42:42.886291+0000","last_became_active":"2024-04-24T06:42:42.885781+0000","last_became_peered":"2024-04-24T06:42:42.885781+0000","last_unstale":"2024-04-24T06:42:42.886291+0000","last_undegraded":"2024-04-24T06:42:42.886291+0000","last_fullsized":"2024-04-24T06:42:42.886291+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:01:44.377335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.877179+0000","last_change":"2024-04-24T06:41:34.282140+0000","last_active":"2024-04-24T06:42:42.877179+0000","last_peered":"2024-04-24T06:42:42.877179+0000","last_clean":"2024-04-24T06:42:42.877179+0000","last_became_active":"2024-04-24T06:41:34.281608+0000","last_became_peered":"2024-04-24T06:41:34.281608+0000","last_unstale":"2024-04-24T06:42:42.877179+0000","last_undegraded":"2024-04-24T06:42:42.877179+0000","last_fullsized":"2024-04-24T06:42:42.877179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:32:06.727639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.877322+0000","last_change":"2024-04-24T06:41:35.308847+0000","last_active":"2024-04-24T06:42:42.877322+0000","last_peered":"2024-04-24T06:42:42.877322+0000","last_clean":"2024-04-24T06:42:42.877322+0000","last_became_active":"2024-04-24T06:41:35.308256+0000","last_became_peered":"2024-04-24T06:41:35.308256+0000","last_unstale":"2024-04-24T06:42:42.877322+0000","last_undegraded":"2024-04-24T06:42:42.877322+0000","last_fullsized":"2024-04-24T06:42:42.877322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:21:49.338737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.877564+0000","last_change":"2024-04-24T06:41:34.285819+0000","last_active":"2024-04-24T06:42:42.877564+0000","last_peered":"2024-04-24T06:42:42.877564+0000","last_clean":"2024-04-24T06:42:42.877564+0000","last_became_active":"2024-04-24T06:41:34.282393+0000","last_became_peered":"2024-04-24T06:41:34.282393+0000","last_unstale":"2024-04-24T06:42:42.877564+0000","last_undegraded":"2024-04-24T06:42:42.877564+0000","last_fullsized":"2024-04-24T06:42:42.877564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:11:18.969178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.876983+0000","last_change":"2024-04-24T06:41:35.297437+0000","last_active":"2024-04-24T06:42:42.876983+0000","last_peered":"2024-04-24T06:42:42.876983+0000","last_clean":"2024-04-24T06:42:42.876983+0000","last_became_active":"2024-04-24T06:41:35.295195+0000","last_became_peered":"2024-04-24T06:41:35.295195+0000","last_unstale":"2024-04-24T06:42:42.876983+0000","last_undegraded":"2024-04-24T06:42:42.876983+0000","last_fullsized":"2024-04-24T06:42:42.876983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:21:51.219219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.875586+0000","last_change":"2024-04-24T06:41:34.275768+0000","last_active":"2024-04-24T06:42:42.875586+0000","last_peered":"2024-04-24T06:42:42.875586+0000","last_clean":"2024-04-24T06:42:42.875586+0000","last_became_active":"2024-04-24T06:41:34.275426+0000","last_became_peered":"2024-04-24T06:41:34.275426+0000","last_unstale":"2024-04-24T06:42:42.875586+0000","last_undegraded":"2024-04-24T06:42:42.875586+0000","last_fullsized":"2024-04-24T06:42:42.875586+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:01:08.735480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.921904+0000","last_change":"2024-04-24T06:41:35.305734+0000","last_active":"2024-04-24T06:42:42.921904+0000","last_peered":"2024-04-24T06:42:42.921904+0000","last_clean":"2024-04-24T06:42:42.921904+0000","last_became_active":"2024-04-24T06:41:35.305365+0000","last_became_peered":"2024-04-24T06:41:35.305365+0000","last_unstale":"2024-04-24T06:42:42.921904+0000","last_undegraded":"2024-04-24T06:42:42.921904+0000","last_fullsized":"2024-04-24T06:42:42.921904+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:38:52.740107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.875262+0000","last_change":"2024-04-24T06:41:34.278317+0000","last_active":"2024-04-24T06:42:42.875262+0000","last_peered":"2024-04-24T06:42:42.875262+0000","last_clean":"2024-04-24T06:42:42.875262+0000","last_became_active":"2024-04-24T06:41:34.277862+0000","last_became_peered":"2024-04-24T06:41:34.277862+0000","last_unstale":"2024-04-24T06:42:42.875262+0000","last_undegraded":"2024-04-24T06:42:42.875262+0000","last_fullsized":"2024-04-24T06:42:42.875262+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:19:50.444136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.876546+0000","last_change":"2024-04-24T06:41:35.282648+0000","last_active":"2024-04-24T06:42:42.876546+0000","last_peered":"2024-04-24T06:42:42.876546+0000","last_clean":"2024-04-24T06:42:42.876546+0000","last_became_active":"2024-04-24T06:41:35.282335+0000","last_became_peered":"2024-04-24T06:41:35.282335+0000","last_unstale":"2024-04-24T06:42:42.876546+0000","last_undegraded":"2024-04-24T06:42:42.876546+0000","last_fullsized":"2024-04-24T06:42:42.876546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:30:11.909067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":110,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:43:02.475003+0000","last_change":"2024-04-24T06:41:34.274405+0000","last_active":"2024-04-24T06:43:02.475003+0000","last_peered":"2024-04-24T06:43:02.475003+0000","last_clean":"2024-04-24T06:43:02.475003+0000","last_became_active":"2024-04-24T06:41:34.273820+0000","last_became_peered":"2024-04-24T06:41:34.273820+0000","last_unstale":"2024-04-24T06:43:02.475003+0000","last_undegraded":"2024-04-24T06:43:02.475003+0000","last_fullsized":"2024-04-24T06:43:02.475003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:11:26.119236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.875343+0000","last_change":"2024-04-24T06:41:35.294133+0000","last_active":"2024-04-24T06:42:42.875343+0000","last_peered":"2024-04-24T06:42:42.875343+0000","last_clean":"2024-04-24T06:42:42.875343+0000","last_became_active":"2024-04-24T06:41:35.293741+0000","last_became_peered":"2024-04-24T06:41:35.293741+0000","last_unstale":"2024-04-24T06:42:42.875343+0000","last_undegraded":"2024-04-24T06:42:42.875343+0000","last_fullsized":"2024-04-24T06:42:42.875343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:09:12.495187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.927397+0000","last_change":"2024-04-24T06:42:42.927397+0000","last_active":"2024-04-24T06:42:42.927397+0000","last_peered":"2024-04-24T06:42:42.927397+0000","last_clean":"2024-04-24T06:42:42.927397+0000","last_became_active":"2024-04-24T06:42:42.924070+0000","last_became_peered":"2024-04-24T06:42:42.924070+0000","last_unstale":"2024-04-24T06:42:42.927397+0000","last_undegraded":"2024-04-24T06:42:42.927397+0000","last_fullsized":"2024-04-24T06:42:42.927397+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:19:47.319762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:43.166909+0000","last_change":"2024-04-24T06:42:43.166909+0000","last_active":"2024-04-24T06:42:43.166909+0000","last_peered":"2024-04-24T06:42:43.166909+0000","last_clean":"2024-04-24T06:42:43.166909+0000","last_became_active":"2024-04-24T06:42:43.166559+0000","last_became_peered":"2024-04-24T06:42:43.166559+0000","last_unstale":"2024-04-24T06:42:43.166909+0000","last_undegraded":"2024-04-24T06:42:43.166909+0000","last_fullsized":"2024-04-24T06:42:43.166909+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:26:42.537506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.879678+0000","last_change":"2024-04-24T06:41:34.284958+0000","last_active":"2024-04-24T06:42:42.879678+0000","last_peered":"2024-04-24T06:42:42.879678+0000","last_clean":"2024-04-24T06:42:42.879678+0000","last_became_active":"2024-04-24T06:41:34.284279+0000","last_became_peered":"2024-04-24T06:41:34.284279+0000","last_unstale":"2024-04-24T06:42:42.879678+0000","last_undegraded":"2024-04-24T06:42:42.879678+0000","last_fullsized":"2024-04-24T06:42:42.879678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:58:19.316091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.878164+0000","last_change":"2024-04-24T06:41:35.307280+0000","last_active":"2024-04-24T06:42:42.878164+0000","last_peered":"2024-04-24T06:42:42.878164+0000","last_clean":"2024-04-24T06:42:42.878164+0000","last_became_active":"2024-04-24T06:41:35.306908+0000","last_became_peered":"2024-04-24T06:41:35.306908+0000","last_unstale":"2024-04-24T06:42:42.878164+0000","last_undegraded":"2024-04-24T06:42:42.878164+0000","last_fullsized":"2024-04-24T06:42:42.878164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:54:31.306302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.876562+0000","last_change":"2024-04-24T06:41:34.272221+0000","last_active":"2024-04-24T06:42:42.876562+0000","last_peered":"2024-04-24T06:42:42.876562+0000","last_clean":"2024-04-24T06:42:42.876562+0000","last_became_active":"2024-04-24T06:41:34.271748+0000","last_became_peered":"2024-04-24T06:41:34.271748+0000","last_unstale":"2024-04-24T06:42:42.876562+0000","last_undegraded":"2024-04-24T06:42:42.876562+0000","last_fullsized":"2024-04-24T06:42:42.876562+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:29:02.667285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.877845+0000","last_change":"2024-04-24T06:41:35.309608+0000","last_active":"2024-04-24T06:42:42.877845+0000","last_peered":"2024-04-24T06:42:42.877845+0000","last_clean":"2024-04-24T06:42:42.877845+0000","last_became_active":"2024-04-24T06:41:35.309227+0000","last_became_peered":"2024-04-24T06:41:35.309227+0000","last_unstale":"2024-04-24T06:42:42.877845+0000","last_undegraded":"2024-04-24T06:42:42.877845+0000","last_fullsized":"2024-04-24T06:42:42.877845+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:51:23.151426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":105,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:43:01.473320+0000","last_change":"2024-04-24T06:41:34.281885+0000","last_active":"2024-04-24T06:43:01.473320+0000","last_peered":"2024-04-24T06:43:01.473320+0000","last_clean":"2024-04-24T06:43:01.473320+0000","last_became_active":"2024-04-24T06:41:34.281466+0000","last_became_peered":"2024-04-24T06:41:34.281466+0000","last_unstale":"2024-04-24T06:43:01.473320+0000","last_undegraded":"2024-04-24T06:43:01.473320+0000","last_fullsized":"2024-04-24T06:43:01.473320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:20:08.500343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":83,"num_read_kb":83,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.922456+0000","last_change":"2024-04-24T06:41:35.292714+0000","last_active":"2024-04-24T06:42:42.922456+0000","last_peered":"2024-04-24T06:42:42.922456+0000","last_clean":"2024-04-24T06:42:42.922456+0000","last_became_active":"2024-04-24T06:41:35.291950+0000","last_became_peered":"2024-04-24T06:41:35.291950+0000","last_unstale":"2024-04-24T06:42:42.922456+0000","last_undegraded":"2024-04-24T06:42:42.922456+0000","last_fullsized":"2024-04-24T06:42:42.922456+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:42:18.049746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:43.165729+0000","last_change":"2024-04-24T06:41:34.275218+0000","last_active":"2024-04-24T06:42:43.165729+0000","last_peered":"2024-04-24T06:42:43.165729+0000","last_clean":"2024-04-24T06:42:43.165729+0000","last_became_active":"2024-04-24T06:41:34.274824+0000","last_became_peered":"2024-04-24T06:41:34.274824+0000","last_unstale":"2024-04-24T06:42:43.165729+0000","last_undegraded":"2024-04-24T06:42:43.165729+0000","last_fullsized":"2024-04-24T06:42:43.165729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:09:57.271304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.883478+0000","last_change":"2024-04-24T06:42:42.883478+0000","last_active":"2024-04-24T06:42:42.883478+0000","last_peered":"2024-04-24T06:42:42.883478+0000","last_clean":"2024-04-24T06:42:42.883478+0000","last_became_active":"2024-04-24T06:42:42.882808+0000","last_became_peered":"2024-04-24T06:42:42.882808+0000","last_unstale":"2024-04-24T06:42:42.883478+0000","last_undegraded":"2024-04-24T06:42:42.883478+0000","last_fullsized":"2024-04-24T06:42:42.883478+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:17:11.514041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.876684+0000","last_change":"2024-04-24T06:41:34.277372+0000","last_active":"2024-04-24T06:42:42.876684+0000","last_peered":"2024-04-24T06:42:42.876684+0000","last_clean":"2024-04-24T06:42:42.876684+0000","last_became_active":"2024-04-24T06:41:34.276997+0000","last_became_peered":"2024-04-24T06:41:34.276997+0000","last_unstale":"2024-04-24T06:42:42.876684+0000","last_undegraded":"2024-04-24T06:42:42.876684+0000","last_fullsized":"2024-04-24T06:42:42.876684+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:52:35.695109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.922095+0000","last_change":"2024-04-24T06:41:35.290233+0000","last_active":"2024-04-24T06:42:42.922095+0000","last_peered":"2024-04-24T06:42:42.922095+0000","last_clean":"2024-04-24T06:42:42.922095+0000","last_became_active":"2024-04-24T06:41:35.289885+0000","last_became_peered":"2024-04-24T06:41:35.289885+0000","last_unstale":"2024-04-24T06:42:42.922095+0000","last_undegraded":"2024-04-24T06:42:42.922095+0000","last_fullsized":"2024-04-24T06:42:42.922095+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:35:54.293308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'27","reported_seq":220,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:43:02.474670+0000","last_change":"2024-04-24T06:41:34.267693+0000","last_active":"2024-04-24T06:43:02.474670+0000","last_peered":"2024-04-24T06:43:02.474670+0000","last_clean":"2024-04-24T06:43:02.474670+0000","last_became_active":"2024-04-24T06:41:34.267298+0000","last_became_peered":"2024-04-24T06:41:34.267298+0000","last_unstale":"2024-04-24T06:43:02.474670+0000","last_undegraded":"2024-04-24T06:43:02.474670+0000","last_fullsized":"2024-04-24T06:43:02.474670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:21:16.015695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60244,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":174,"num_read_kb":145,"num_write":28,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.875369+0000","last_change":"2024-04-24T06:41:35.286587+0000","last_active":"2024-04-24T06:42:42.875369+0000","last_peered":"2024-04-24T06:42:42.875369+0000","last_clean":"2024-04-24T06:42:42.875369+0000","last_became_active":"2024-04-24T06:41:35.286239+0000","last_became_peered":"2024-04-24T06:41:35.286239+0000","last_unstale":"2024-04-24T06:42:42.875369+0000","last_undegraded":"2024-04-24T06:42:42.875369+0000","last_fullsized":"2024-04-24T06:42:42.875369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:18:56.013425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.892090+0000","last_change":"2024-04-24T06:42:42.892090+0000","last_active":"2024-04-24T06:42:42.892090+0000","last_peered":"2024-04-24T06:42:42.892090+0000","last_clean":"2024-04-24T06:42:42.892090+0000","last_became_active":"2024-04-24T06:42:42.879904+0000","last_became_peered":"2024-04-24T06:42:42.879904+0000","last_unstale":"2024-04-24T06:42:42.892090+0000","last_undegraded":"2024-04-24T06:42:42.892090+0000","last_fullsized":"2024-04-24T06:42:42.892090+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:36:02.420438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.883128+0000","last_change":"2024-04-24T06:41:35.306175+0000","last_active":"2024-04-24T06:42:42.883128+0000","last_peered":"2024-04-24T06:42:42.883128+0000","last_clean":"2024-04-24T06:42:42.883128+0000","last_became_active":"2024-04-24T06:41:35.305840+0000","last_became_peered":"2024-04-24T06:41:35.305840+0000","last_unstale":"2024-04-24T06:42:42.883128+0000","last_undegraded":"2024-04-24T06:42:42.883128+0000","last_fullsized":"2024-04-24T06:42:42.883128+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:42:46.238490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.876274+0000","last_change":"2024-04-24T06:41:34.277212+0000","last_active":"2024-04-24T06:42:42.876274+0000","last_peered":"2024-04-24T06:42:42.876274+0000","last_clean":"2024-04-24T06:42:42.876274+0000","last_became_active":"2024-04-24T06:41:34.276695+0000","last_became_peered":"2024-04-24T06:41:34.276695+0000","last_unstale":"2024-04-24T06:42:42.876274+0000","last_undegraded":"2024-04-24T06:42:42.876274+0000","last_fullsized":"2024-04-24T06:42:42.876274+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:00:47.169384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:43.165743+0000","last_change":"2024-04-24T06:41:35.305348+0000","last_active":"2024-04-24T06:42:43.165743+0000","last_peered":"2024-04-24T06:42:43.165743+0000","last_clean":"2024-04-24T06:42:43.165743+0000","last_became_active":"2024-04-24T06:41:35.304899+0000","last_became_peered":"2024-04-24T06:41:35.304899+0000","last_unstale":"2024-04-24T06:42:43.165743+0000","last_undegraded":"2024-04-24T06:42:43.165743+0000","last_fullsized":"2024-04-24T06:42:43.165743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:49:35.737711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":109,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:43:03.494978+0000","last_change":"2024-04-24T06:41:34.272902+0000","last_active":"2024-04-24T06:43:03.494978+0000","last_peered":"2024-04-24T06:43:03.494978+0000","last_clean":"2024-04-24T06:43:03.494978+0000","last_became_active":"2024-04-24T06:41:34.271726+0000","last_became_peered":"2024-04-24T06:41:34.271726+0000","last_unstale":"2024-04-24T06:43:03.494978+0000","last_undegraded":"2024-04-24T06:43:03.494978+0000","last_fullsized":"2024-04-24T06:43:03.494978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:07:38.318110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":88,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:43.164962+0000","last_change":"2024-04-24T06:41:35.283764+0000","last_active":"2024-04-24T06:42:43.164962+0000","last_peered":"2024-04-24T06:42:43.164962+0000","last_clean":"2024-04-24T06:42:43.164962+0000","last_became_active":"2024-04-24T06:41:35.283400+0000","last_became_peered":"2024-04-24T06:41:35.283400+0000","last_unstale":"2024-04-24T06:42:43.164962+0000","last_undegraded":"2024-04-24T06:42:43.164962+0000","last_fullsized":"2024-04-24T06:42:43.164962+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:36:47.034068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.874899+0000","last_change":"2024-04-24T06:41:34.280020+0000","last_active":"2024-04-24T06:42:42.874899+0000","last_peered":"2024-04-24T06:42:42.874899+0000","last_clean":"2024-04-24T06:42:42.874899+0000","last_became_active":"2024-04-24T06:41:34.279659+0000","last_became_peered":"2024-04-24T06:41:34.279659+0000","last_unstale":"2024-04-24T06:42:42.874899+0000","last_undegraded":"2024-04-24T06:42:42.874899+0000","last_fullsized":"2024-04-24T06:42:42.874899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:13:29.529123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.876091+0000","last_change":"2024-04-24T06:41:35.284500+0000","last_active":"2024-04-24T06:42:42.876091+0000","last_peered":"2024-04-24T06:42:42.876091+0000","last_clean":"2024-04-24T06:42:42.876091+0000","last_became_active":"2024-04-24T06:41:35.283171+0000","last_became_peered":"2024-04-24T06:41:35.283171+0000","last_unstale":"2024-04-24T06:42:42.876091+0000","last_undegraded":"2024-04-24T06:42:42.876091+0000","last_fullsized":"2024-04-24T06:42:42.876091+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:33:19.527226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.875426+0000","last_change":"2024-04-24T06:41:34.279306+0000","last_active":"2024-04-24T06:42:42.875426+0000","last_peered":"2024-04-24T06:42:42.875426+0000","last_clean":"2024-04-24T06:42:42.875426+0000","last_became_active":"2024-04-24T06:41:34.278946+0000","last_became_peered":"2024-04-24T06:41:34.278946+0000","last_unstale":"2024-04-24T06:42:42.875426+0000","last_undegraded":"2024-04-24T06:42:42.875426+0000","last_fullsized":"2024-04-24T06:42:42.875426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:13:40.845871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.870469+0000","last_change":"2024-04-24T06:41:35.283716+0000","last_active":"2024-04-24T06:42:42.870469+0000","last_peered":"2024-04-24T06:42:42.870469+0000","last_clean":"2024-04-24T06:42:42.870469+0000","last_became_active":"2024-04-24T06:41:35.283330+0000","last_became_peered":"2024-04-24T06:41:35.283330+0000","last_unstale":"2024-04-24T06:42:42.870469+0000","last_undegraded":"2024-04-24T06:42:42.870469+0000","last_fullsized":"2024-04-24T06:42:42.870469+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:18:13.403264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.887349+0000","last_change":"2024-04-24T06:42:42.887349+0000","last_active":"2024-04-24T06:42:42.887349+0000","last_peered":"2024-04-24T06:42:42.887349+0000","last_clean":"2024-04-24T06:42:42.887349+0000","last_became_active":"2024-04-24T06:42:42.886813+0000","last_became_peered":"2024-04-24T06:42:42.886813+0000","last_unstale":"2024-04-24T06:42:42.887349+0000","last_undegraded":"2024-04-24T06:42:42.887349+0000","last_fullsized":"2024-04-24T06:42:42.887349+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:05:40.577189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.872971+0000","last_change":"2024-04-24T06:41:35.306607+0000","last_active":"2024-04-24T06:42:42.872971+0000","last_peered":"2024-04-24T06:42:42.872971+0000","last_clean":"2024-04-24T06:42:42.872971+0000","last_became_active":"2024-04-24T06:41:35.306217+0000","last_became_peered":"2024-04-24T06:41:35.306217+0000","last_unstale":"2024-04-24T06:42:42.872971+0000","last_undegraded":"2024-04-24T06:42:42.872971+0000","last_fullsized":"2024-04-24T06:42:42.872971+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:07:46.068876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:43.165230+0000","last_change":"2024-04-24T06:41:34.285319+0000","last_active":"2024-04-24T06:42:43.165230+0000","last_peered":"2024-04-24T06:42:43.165230+0000","last_clean":"2024-04-24T06:42:43.165230+0000","last_became_active":"2024-04-24T06:41:34.284912+0000","last_became_peered":"2024-04-24T06:41:34.284912+0000","last_unstale":"2024-04-24T06:42:43.165230+0000","last_undegraded":"2024-04-24T06:42:43.165230+0000","last_fullsized":"2024-04-24T06:42:43.165230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:34:56.454587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.873274+0000","last_change":"2024-04-24T06:41:35.296294+0000","last_active":"2024-04-24T06:42:42.873274+0000","last_peered":"2024-04-24T06:42:42.873274+0000","last_clean":"2024-04-24T06:42:42.873274+0000","last_became_active":"2024-04-24T06:41:35.295792+0000","last_became_peered":"2024-04-24T06:41:35.295792+0000","last_unstale":"2024-04-24T06:42:42.873274+0000","last_undegraded":"2024-04-24T06:42:42.873274+0000","last_fullsized":"2024-04-24T06:42:42.873274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:31:35.573858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.876723+0000","last_change":"2024-04-24T06:41:35.282558+0000","last_active":"2024-04-24T06:42:42.876723+0000","last_peered":"2024-04-24T06:42:42.876723+0000","last_clean":"2024-04-24T06:42:42.876723+0000","last_became_active":"2024-04-24T06:41:35.282163+0000","last_became_peered":"2024-04-24T06:41:35.282163+0000","last_unstale":"2024-04-24T06:42:42.876723+0000","last_undegraded":"2024-04-24T06:42:42.876723+0000","last_fullsized":"2024-04-24T06:42:42.876723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:12:23.569849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":104,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:59.486355+0000","last_change":"2024-04-24T06:41:34.284294+0000","last_active":"2024-04-24T06:42:59.486355+0000","last_peered":"2024-04-24T06:42:59.486355+0000","last_clean":"2024-04-24T06:42:59.486355+0000","last_became_active":"2024-04-24T06:41:34.283933+0000","last_became_peered":"2024-04-24T06:41:34.283933+0000","last_unstale":"2024-04-24T06:42:59.486355+0000","last_undegraded":"2024-04-24T06:42:59.486355+0000","last_fullsized":"2024-04-24T06:42:59.486355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:35:26.042614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":81,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.878680+0000","last_change":"2024-04-24T06:41:35.302361+0000","last_active":"2024-04-24T06:42:42.878680+0000","last_peered":"2024-04-24T06:42:42.878680+0000","last_clean":"2024-04-24T06:42:42.878680+0000","last_became_active":"2024-04-24T06:41:35.301652+0000","last_became_peered":"2024-04-24T06:41:35.301652+0000","last_unstale":"2024-04-24T06:42:42.878680+0000","last_undegraded":"2024-04-24T06:42:42.878680+0000","last_fullsized":"2024-04-24T06:42:42.878680+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:35:58.010853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.922470+0000","last_change":"2024-04-24T06:41:34.282156+0000","last_active":"2024-04-24T06:42:42.922470+0000","last_peered":"2024-04-24T06:42:42.922470+0000","last_clean":"2024-04-24T06:42:42.922470+0000","last_became_active":"2024-04-24T06:41:34.281771+0000","last_became_peered":"2024-04-24T06:41:34.281771+0000","last_unstale":"2024-04-24T06:42:42.922470+0000","last_undegraded":"2024-04-24T06:42:42.922470+0000","last_fullsized":"2024-04-24T06:42:42.922470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:41:49.584620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":355,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:43.165134+0000","last_change":"2024-04-24T06:40:16.681695+0000","last_active":"2024-04-24T06:42:43.165134+0000","last_peered":"2024-04-24T06:42:43.165134+0000","last_clean":"2024-04-24T06:42:43.165134+0000","last_became_active":"2024-04-24T06:40:16.681208+0000","last_became_peered":"2024-04-24T06:40:16.681208+0000","last_unstale":"2024-04-24T06:42:43.165134+0000","last_undegraded":"2024-04-24T06:42:43.165134+0000","last_fullsized":"2024-04-24T06:42:43.165134+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:36:27.950574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:36:27.950574+0000","last_clean_scrub_stamp":"2024-04-24T06:36:27.950574+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:39:01.985893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.885312+0000","last_change":"2024-04-24T06:42:42.885312+0000","last_active":"2024-04-24T06:42:42.885312+0000","last_peered":"2024-04-24T06:42:42.885312+0000","last_clean":"2024-04-24T06:42:42.885312+0000","last_became_active":"2024-04-24T06:42:42.884702+0000","last_became_peered":"2024-04-24T06:42:42.884702+0000","last_unstale":"2024-04-24T06:42:42.885312+0000","last_undegraded":"2024-04-24T06:42:42.885312+0000","last_fullsized":"2024-04-24T06:42:42.885312+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:33:44.590555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:43.165235+0000","last_change":"2024-04-24T06:41:34.273872+0000","last_active":"2024-04-24T06:42:43.165235+0000","last_peered":"2024-04-24T06:42:43.165235+0000","last_clean":"2024-04-24T06:42:43.165235+0000","last_became_active":"2024-04-24T06:41:34.273473+0000","last_became_peered":"2024-04-24T06:41:34.273473+0000","last_unstale":"2024-04-24T06:42:43.165235+0000","last_undegraded":"2024-04-24T06:42:43.165235+0000","last_fullsized":"2024-04-24T06:42:43.165235+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:47:11.914985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.875436+0000","last_change":"2024-04-24T06:41:34.289293+0000","last_active":"2024-04-24T06:42:42.875436+0000","last_peered":"2024-04-24T06:42:42.875436+0000","last_clean":"2024-04-24T06:42:42.875436+0000","last_became_active":"2024-04-24T06:41:34.288939+0000","last_became_peered":"2024-04-24T06:41:34.288939+0000","last_unstale":"2024-04-24T06:42:42.875436+0000","last_undegraded":"2024-04-24T06:42:42.875436+0000","last_fullsized":"2024-04-24T06:42:42.875436+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:03:01.026792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.879987+0000","last_change":"2024-04-24T06:41:35.302153+0000","last_active":"2024-04-24T06:42:42.879987+0000","last_peered":"2024-04-24T06:42:42.879987+0000","last_clean":"2024-04-24T06:42:42.879987+0000","last_became_active":"2024-04-24T06:41:35.301666+0000","last_became_peered":"2024-04-24T06:41:35.301666+0000","last_unstale":"2024-04-24T06:42:42.879987+0000","last_undegraded":"2024-04-24T06:42:42.879987+0000","last_fullsized":"2024-04-24T06:42:42.879987+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:45:09.642482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.875466+0000","last_change":"2024-04-24T06:41:34.291570+0000","last_active":"2024-04-24T06:42:42.875466+0000","last_peered":"2024-04-24T06:42:42.875466+0000","last_clean":"2024-04-24T06:42:42.875466+0000","last_became_active":"2024-04-24T06:41:34.289700+0000","last_became_peered":"2024-04-24T06:41:34.289700+0000","last_unstale":"2024-04-24T06:42:42.875466+0000","last_undegraded":"2024-04-24T06:42:42.875466+0000","last_fullsized":"2024-04-24T06:42:42.875466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:57:20.537299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.878064+0000","last_change":"2024-04-24T06:41:35.304703+0000","last_active":"2024-04-24T06:42:42.878064+0000","last_peered":"2024-04-24T06:42:42.878064+0000","last_clean":"2024-04-24T06:42:42.878064+0000","last_became_active":"2024-04-24T06:41:35.304319+0000","last_became_peered":"2024-04-24T06:41:35.304319+0000","last_unstale":"2024-04-24T06:42:42.878064+0000","last_undegraded":"2024-04-24T06:42:42.878064+0000","last_fullsized":"2024-04-24T06:42:42.878064+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:00:56.069509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.922285+0000","last_change":"2024-04-24T06:41:34.276866+0000","last_active":"2024-04-24T06:42:42.922285+0000","last_peered":"2024-04-24T06:42:42.922285+0000","last_clean":"2024-04-24T06:42:42.922285+0000","last_became_active":"2024-04-24T06:41:34.276478+0000","last_became_peered":"2024-04-24T06:41:34.276478+0000","last_unstale":"2024-04-24T06:42:42.922285+0000","last_undegraded":"2024-04-24T06:42:42.922285+0000","last_fullsized":"2024-04-24T06:42:42.922285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:10:35.047960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.873046+0000","last_change":"2024-04-24T06:41:35.300951+0000","last_active":"2024-04-24T06:42:42.873046+0000","last_peered":"2024-04-24T06:42:42.873046+0000","last_clean":"2024-04-24T06:42:42.873046+0000","last_became_active":"2024-04-24T06:41:35.296176+0000","last_became_peered":"2024-04-24T06:41:35.296176+0000","last_unstale":"2024-04-24T06:42:42.873046+0000","last_undegraded":"2024-04-24T06:42:42.873046+0000","last_fullsized":"2024-04-24T06:42:42.873046+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:53:58.497437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.921587+0000","last_change":"2024-04-24T06:41:34.276154+0000","last_active":"2024-04-24T06:42:42.921587+0000","last_peered":"2024-04-24T06:42:42.921587+0000","last_clean":"2024-04-24T06:42:42.921587+0000","last_became_active":"2024-04-24T06:41:34.275734+0000","last_became_peered":"2024-04-24T06:41:34.275734+0000","last_unstale":"2024-04-24T06:42:42.921587+0000","last_undegraded":"2024-04-24T06:42:42.921587+0000","last_fullsized":"2024-04-24T06:42:42.921587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:28:54.618061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.874889+0000","last_change":"2024-04-24T06:41:35.285837+0000","last_active":"2024-04-24T06:42:42.874889+0000","last_peered":"2024-04-24T06:42:42.874889+0000","last_clean":"2024-04-24T06:42:42.874889+0000","last_became_active":"2024-04-24T06:41:35.285505+0000","last_became_peered":"2024-04-24T06:41:35.285505+0000","last_unstale":"2024-04-24T06:42:42.874889+0000","last_undegraded":"2024-04-24T06:42:42.874889+0000","last_fullsized":"2024-04-24T06:42:42.874889+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:36:53.460780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.876765+0000","last_change":"2024-04-24T06:41:34.290039+0000","last_active":"2024-04-24T06:42:42.876765+0000","last_peered":"2024-04-24T06:42:42.876765+0000","last_clean":"2024-04-24T06:42:42.876765+0000","last_became_active":"2024-04-24T06:41:34.289228+0000","last_became_peered":"2024-04-24T06:41:34.289228+0000","last_unstale":"2024-04-24T06:42:42.876765+0000","last_undegraded":"2024-04-24T06:42:42.876765+0000","last_fullsized":"2024-04-24T06:42:42.876765+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:07:20.342359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.870454+0000","last_change":"2024-04-24T06:41:35.296214+0000","last_active":"2024-04-24T06:42:42.870454+0000","last_peered":"2024-04-24T06:42:42.870454+0000","last_clean":"2024-04-24T06:42:42.870454+0000","last_became_active":"2024-04-24T06:41:35.295888+0000","last_became_peered":"2024-04-24T06:41:35.295888+0000","last_unstale":"2024-04-24T06:42:42.870454+0000","last_undegraded":"2024-04-24T06:42:42.870454+0000","last_fullsized":"2024-04-24T06:42:42.870454+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:55:19.520479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.872600+0000","last_change":"2024-04-24T06:41:34.275885+0000","last_active":"2024-04-24T06:42:42.872600+0000","last_peered":"2024-04-24T06:42:42.872600+0000","last_clean":"2024-04-24T06:42:42.872600+0000","last_became_active":"2024-04-24T06:41:34.275503+0000","last_became_peered":"2024-04-24T06:41:34.275503+0000","last_unstale":"2024-04-24T06:42:42.872600+0000","last_undegraded":"2024-04-24T06:42:42.872600+0000","last_fullsized":"2024-04-24T06:42:42.872600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:50:58.216699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.877980+0000","last_change":"2024-04-24T06:41:35.305330+0000","last_active":"2024-04-24T06:42:42.877980+0000","last_peered":"2024-04-24T06:42:42.877980+0000","last_clean":"2024-04-24T06:42:42.877980+0000","last_became_active":"2024-04-24T06:41:35.301262+0000","last_became_peered":"2024-04-24T06:41:35.301262+0000","last_unstale":"2024-04-24T06:42:42.877980+0000","last_undegraded":"2024-04-24T06:42:42.877980+0000","last_fullsized":"2024-04-24T06:42:42.877980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:58:41.029511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.875278+0000","last_change":"2024-04-24T06:41:34.276926+0000","last_active":"2024-04-24T06:42:42.875278+0000","last_peered":"2024-04-24T06:42:42.875278+0000","last_clean":"2024-04-24T06:42:42.875278+0000","last_became_active":"2024-04-24T06:41:34.276539+0000","last_became_peered":"2024-04-24T06:41:34.276539+0000","last_unstale":"2024-04-24T06:42:42.875278+0000","last_undegraded":"2024-04-24T06:42:42.875278+0000","last_fullsized":"2024-04-24T06:42:42.875278+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:28:56.652584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:43.165259+0000","last_change":"2024-04-24T06:41:35.301052+0000","last_active":"2024-04-24T06:42:43.165259+0000","last_peered":"2024-04-24T06:42:43.165259+0000","last_clean":"2024-04-24T06:42:43.165259+0000","last_became_active":"2024-04-24T06:41:35.300650+0000","last_became_peered":"2024-04-24T06:41:35.300650+0000","last_unstale":"2024-04-24T06:42:43.165259+0000","last_undegraded":"2024-04-24T06:42:43.165259+0000","last_fullsized":"2024-04-24T06:42:43.165259+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:06:54.290353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.873168+0000","last_change":"2024-04-24T06:41:34.273773+0000","last_active":"2024-04-24T06:42:42.873168+0000","last_peered":"2024-04-24T06:42:42.873168+0000","last_clean":"2024-04-24T06:42:42.873168+0000","last_became_active":"2024-04-24T06:41:34.273324+0000","last_became_peered":"2024-04-24T06:41:34.273324+0000","last_unstale":"2024-04-24T06:42:42.873168+0000","last_undegraded":"2024-04-24T06:42:42.873168+0000","last_fullsized":"2024-04-24T06:42:42.873168+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:30:30.760740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:43.165693+0000","last_change":"2024-04-24T06:41:35.289565+0000","last_active":"2024-04-24T06:42:43.165693+0000","last_peered":"2024-04-24T06:42:43.165693+0000","last_clean":"2024-04-24T06:42:43.165693+0000","last_became_active":"2024-04-24T06:41:35.289185+0000","last_became_peered":"2024-04-24T06:41:35.289185+0000","last_unstale":"2024-04-24T06:42:43.165693+0000","last_undegraded":"2024-04-24T06:42:43.165693+0000","last_fullsized":"2024-04-24T06:42:43.165693+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:26:28.345162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.876467+0000","last_change":"2024-04-24T06:41:34.285804+0000","last_active":"2024-04-24T06:42:42.876467+0000","last_peered":"2024-04-24T06:42:42.876467+0000","last_clean":"2024-04-24T06:42:42.876467+0000","last_became_active":"2024-04-24T06:41:34.285501+0000","last_became_peered":"2024-04-24T06:41:34.285501+0000","last_unstale":"2024-04-24T06:42:42.876467+0000","last_undegraded":"2024-04-24T06:42:42.876467+0000","last_fullsized":"2024-04-24T06:42:42.876467+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:25:15.804221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:43.165519+0000","last_change":"2024-04-24T06:41:35.300856+0000","last_active":"2024-04-24T06:42:43.165519+0000","last_peered":"2024-04-24T06:42:43.165519+0000","last_clean":"2024-04-24T06:42:43.165519+0000","last_became_active":"2024-04-24T06:41:35.300344+0000","last_became_peered":"2024-04-24T06:41:35.300344+0000","last_unstale":"2024-04-24T06:42:43.165519+0000","last_undegraded":"2024-04-24T06:42:43.165519+0000","last_fullsized":"2024-04-24T06:42:43.165519+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:10:01.866954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.880835+0000","last_change":"2024-04-24T06:41:34.284112+0000","last_active":"2024-04-24T06:42:42.880835+0000","last_peered":"2024-04-24T06:42:42.880835+0000","last_clean":"2024-04-24T06:42:42.880835+0000","last_became_active":"2024-04-24T06:41:34.279317+0000","last_became_peered":"2024-04-24T06:41:34.279317+0000","last_unstale":"2024-04-24T06:42:42.880835+0000","last_undegraded":"2024-04-24T06:42:42.880835+0000","last_fullsized":"2024-04-24T06:42:42.880835+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:49:45.673353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:43.165795+0000","last_change":"2024-04-24T06:41:35.288360+0000","last_active":"2024-04-24T06:42:43.165795+0000","last_peered":"2024-04-24T06:42:43.165795+0000","last_clean":"2024-04-24T06:42:43.165795+0000","last_became_active":"2024-04-24T06:41:35.288019+0000","last_became_peered":"2024-04-24T06:41:35.288019+0000","last_unstale":"2024-04-24T06:42:43.165795+0000","last_undegraded":"2024-04-24T06:42:43.165795+0000","last_fullsized":"2024-04-24T06:42:43.165795+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:33:49.927591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.870599+0000","last_change":"2024-04-24T06:41:35.286137+0000","last_active":"2024-04-24T06:42:42.870599+0000","last_peered":"2024-04-24T06:42:42.870599+0000","last_clean":"2024-04-24T06:42:42.870599+0000","last_became_active":"2024-04-24T06:41:35.285773+0000","last_became_peered":"2024-04-24T06:41:35.285773+0000","last_unstale":"2024-04-24T06:42:42.870599+0000","last_undegraded":"2024-04-24T06:42:42.870599+0000","last_fullsized":"2024-04-24T06:42:42.870599+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:29:32.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.872717+0000","last_change":"2024-04-24T06:41:35.283977+0000","last_active":"2024-04-24T06:42:42.872717+0000","last_peered":"2024-04-24T06:42:42.872717+0000","last_clean":"2024-04-24T06:42:42.872717+0000","last_became_active":"2024-04-24T06:41:35.282312+0000","last_became_peered":"2024-04-24T06:41:35.282312+0000","last_unstale":"2024-04-24T06:42:42.872717+0000","last_undegraded":"2024-04-24T06:42:42.872717+0000","last_fullsized":"2024-04-24T06:42:42.872717+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:41:36.463803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.872593+0000","last_change":"2024-04-24T06:41:35.300372+0000","last_active":"2024-04-24T06:42:42.872593+0000","last_peered":"2024-04-24T06:42:42.872593+0000","last_clean":"2024-04-24T06:42:42.872593+0000","last_became_active":"2024-04-24T06:41:35.300034+0000","last_became_peered":"2024-04-24T06:41:35.300034+0000","last_unstale":"2024-04-24T06:42:42.872593+0000","last_undegraded":"2024-04-24T06:42:42.872593+0000","last_fullsized":"2024-04-24T06:42:42.872593+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:56:23.732798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.874606+0000","last_change":"2024-04-24T06:41:35.301619+0000","last_active":"2024-04-24T06:42:42.874606+0000","last_peered":"2024-04-24T06:42:42.874606+0000","last_clean":"2024-04-24T06:42:42.874606+0000","last_became_active":"2024-04-24T06:41:35.298897+0000","last_became_peered":"2024-04-24T06:41:35.298897+0000","last_unstale":"2024-04-24T06:42:42.874606+0000","last_undegraded":"2024-04-24T06:42:42.874606+0000","last_fullsized":"2024-04-24T06:42:42.874606+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:26:11.583079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.880110+0000","last_change":"2024-04-24T06:41:35.304905+0000","last_active":"2024-04-24T06:42:42.880110+0000","last_peered":"2024-04-24T06:42:42.880110+0000","last_clean":"2024-04-24T06:42:42.880110+0000","last_became_active":"2024-04-24T06:41:35.304407+0000","last_became_peered":"2024-04-24T06:41:35.304407+0000","last_unstale":"2024-04-24T06:42:42.880110+0000","last_undegraded":"2024-04-24T06:42:42.880110+0000","last_fullsized":"2024-04-24T06:42:42.880110+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:38:07.417898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.870617+0000","last_change":"2024-04-24T06:41:35.288646+0000","last_active":"2024-04-24T06:42:42.870617+0000","last_peered":"2024-04-24T06:42:42.870617+0000","last_clean":"2024-04-24T06:42:42.870617+0000","last_became_active":"2024-04-24T06:41:35.288266+0000","last_became_peered":"2024-04-24T06:41:35.288266+0000","last_unstale":"2024-04-24T06:42:42.870617+0000","last_undegraded":"2024-04-24T06:42:42.870617+0000","last_fullsized":"2024-04-24T06:42:42.870617+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:59:16.526712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.873283+0000","last_change":"2024-04-24T06:41:35.285001+0000","last_active":"2024-04-24T06:42:42.873283+0000","last_peered":"2024-04-24T06:42:42.873283+0000","last_clean":"2024-04-24T06:42:42.873283+0000","last_became_active":"2024-04-24T06:41:35.284671+0000","last_became_peered":"2024-04-24T06:41:35.284671+0000","last_unstale":"2024-04-24T06:42:42.873283+0000","last_undegraded":"2024-04-24T06:42:42.873283+0000","last_fullsized":"2024-04-24T06:42:42.873283+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:37:43.534018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.877784+0000","last_change":"2024-04-24T06:41:35.299124+0000","last_active":"2024-04-24T06:42:42.877784+0000","last_peered":"2024-04-24T06:42:42.877784+0000","last_clean":"2024-04-24T06:42:42.877784+0000","last_became_active":"2024-04-24T06:41:35.298246+0000","last_became_peered":"2024-04-24T06:41:35.298246+0000","last_unstale":"2024-04-24T06:42:42.877784+0000","last_undegraded":"2024-04-24T06:42:42.877784+0000","last_fullsized":"2024-04-24T06:42:42.877784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:07:22.649874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.889015+0000","last_change":"2024-04-24T06:42:42.888183+0000","last_active":"2024-04-24T06:42:42.889015+0000","last_peered":"2024-04-24T06:42:42.889015+0000","last_clean":"2024-04-24T06:42:42.889015+0000","last_became_active":"2024-04-24T06:42:42.887373+0000","last_became_peered":"2024-04-24T06:42:42.887373+0000","last_unstale":"2024-04-24T06:42:42.889015+0000","last_undegraded":"2024-04-24T06:42:42.889015+0000","last_fullsized":"2024-04-24T06:42:42.889015+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:44:51.456753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.876609+0000","last_change":"2024-04-24T06:41:35.301619+0000","last_active":"2024-04-24T06:42:42.876609+0000","last_peered":"2024-04-24T06:42:42.876609+0000","last_clean":"2024-04-24T06:42:42.876609+0000","last_became_active":"2024-04-24T06:41:35.297987+0000","last_became_peered":"2024-04-24T06:41:35.297987+0000","last_unstale":"2024-04-24T06:42:42.876609+0000","last_undegraded":"2024-04-24T06:42:42.876609+0000","last_fullsized":"2024-04-24T06:42:42.876609+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:06:01.235593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.921904+0000","last_change":"2024-04-24T06:41:35.304347+0000","last_active":"2024-04-24T06:42:42.921904+0000","last_peered":"2024-04-24T06:42:42.921904+0000","last_clean":"2024-04-24T06:42:42.921904+0000","last_became_active":"2024-04-24T06:41:35.303971+0000","last_became_peered":"2024-04-24T06:41:35.303971+0000","last_unstale":"2024-04-24T06:42:42.921904+0000","last_undegraded":"2024-04-24T06:42:42.921904+0000","last_fullsized":"2024-04-24T06:42:42.921904+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:31:54.813243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.880070+0000","last_change":"2024-04-24T06:41:35.308314+0000","last_active":"2024-04-24T06:42:42.880070+0000","last_peered":"2024-04-24T06:42:42.880070+0000","last_clean":"2024-04-24T06:42:42.880070+0000","last_became_active":"2024-04-24T06:41:35.306581+0000","last_became_peered":"2024-04-24T06:41:35.306581+0000","last_unstale":"2024-04-24T06:42:42.880070+0000","last_undegraded":"2024-04-24T06:42:42.880070+0000","last_fullsized":"2024-04-24T06:42:42.880070+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:34:43.095087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T06:42:42.870482+0000","last_change":"2024-04-24T06:41:35.289166+0000","last_active":"2024-04-24T06:42:42.870482+0000","last_peered":"2024-04-24T06:42:42.870482+0000","last_clean":"2024-04-24T06:42:42.870482+0000","last_became_active":"2024-04-24T06:41:35.287672+0000","last_became_peered":"2024-04-24T06:41:35.287672+0000","last_unstale":"2024-04-24T06:42:42.870482+0000","last_undegraded":"2024-04-24T06:42:42.870482+0000","last_fullsized":"2024-04-24T06:42:42.870482+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:31:45.403387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63854,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":525,"num_read_kb":327,"num_write":156,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":191562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351007,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":460828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612644,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1658880,"data_stored":1112519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776169,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":978944,"data_stored":461030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037808,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":460950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299445,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28556,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716596,"statfs":{"total":95995035648,"available":95965794304,"internally_reserved":0,"allocated":1572864,"data_stored":1052489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462971,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":460419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1028096,"data_stored":520456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":987136,"data_stored":461690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1048576,"data_stored":521772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182475,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":460730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":460856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724610,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32728,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712424,"statfs":{"total":95995035648,"available":95961522176,"internally_reserved":0,"allocated":1585152,"data_stored":1052173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":61560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T06:43:08.046 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-24T06:43:08.046 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-24T06:43:08.046 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-24T06:43:08.046 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-24T06:43:08.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:08 smithi005 ceph-mon[27798]: from='client.24910 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T06:43:08.333 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:08 smithi037 ceph-mon[32864]: from='client.24910 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T06:43:08.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:08 smithi137 ceph-mon[33017]: from='client.24910 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T06:43:08.556 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:43:09.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:09 smithi137 ceph-mon[33017]: pgmap v361: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:43:09.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:09 smithi005 ceph-mon[27798]: pgmap v361: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:43:09.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:09 smithi037 ceph-mon[32864]: pgmap v361: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:43:10.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:10.923+0000 7f7a4b600700 1 -- 172.21.15.5:0/633620922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a44105920 msgr2=0x7f7a44109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:10.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:10.923+0000 7f7a4b600700 1 --2- 172.21.15.5:0/633620922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a44105920 0x7f7a44109970 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7a340099c0 tx=0x7f7a34009cd0 comp rx=0 tx=0).stop 2024-04-24T06:43:10.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:10.923+0000 7f7a4b600700 1 -- 172.21.15.5:0/633620922 shutdown_connections 2024-04-24T06:43:10.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:10.923+0000 7f7a4b600700 1 --2- 172.21.15.5:0/633620922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a4410a050 0x7f7a44111c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:10.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:10.923+0000 7f7a4b600700 1 --2- 172.21.15.5:0/633620922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a44104f70 0x7f7a44105350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:10.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:10.923+0000 7f7a4b600700 1 --2- 172.21.15.5:0/633620922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a44105920 0x7f7a44109970 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:10.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:10.923+0000 7f7a4b600700 1 -- 172.21.15.5:0/633620922 >> 172.21.15.5:0/633620922 conn(0x7f7a44100800 msgr2=0x7f7a44102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:10.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:10.923+0000 7f7a4b600700 1 -- 172.21.15.5:0/633620922 shutdown_connections 2024-04-24T06:43:10.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:10.923+0000 7f7a4b600700 1 -- 172.21.15.5:0/633620922 wait complete. 2024-04-24T06:43:10.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:10.924+0000 7f7a4b600700 1 Processor -- start 2024-04-24T06:43:10.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:10.924+0000 7f7a4b600700 1 -- start start 2024-04-24T06:43:10.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:10.925+0000 7f7a4b600700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a44104f70 0x7f7a4419b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:10.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:10.925+0000 7f7a4b600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a44105920 0x7f7a4419ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:10.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:10.925+0000 7f7a4b600700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a4410a050 0x7f7a4419fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:10.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:10.925+0000 7f7a4b600700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7a44115030 con 0x7f7a44104f70 2024-04-24T06:43:10.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:10.925+0000 7f7a4b600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7a44114eb0 con 0x7f7a44105920 2024-04-24T06:43:10.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:10.925+0000 7f7a4b600700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7a441151b0 con 0x7f7a4410a050 2024-04-24T06:43:10.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:10.925+0000 7f7a48c00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a44104f70 0x7f7a4419b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:10.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:10.925+0000 7f7a48c00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a44104f70 0x7f7a4419b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33522/0 (socket says 172.21.15.5:33522) 2024-04-24T06:43:10.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:10.925+0000 7f7a48c00700 1 -- 172.21.15.5:0/3256992612 learned_addr learned my addr 172.21.15.5:0/3256992612 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:10.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:10.925+0000 7f7a43e00700 1 --2- 172.21.15.5:0/3256992612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a44105920 0x7f7a4419ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:10.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:10.925+0000 7f7a49600700 1 --2- 172.21.15.5:0/3256992612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a4410a050 0x7f7a4419fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:10.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:10.926+0000 7f7a48c00700 1 -- 172.21.15.5:0/3256992612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a4410a050 msgr2=0x7f7a4419fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:10.926+0000 7f7a48c00700 1 --2- 172.21.15.5:0/3256992612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a4410a050 0x7f7a4419fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:10.926+0000 7f7a48c00700 1 -- 172.21.15.5:0/3256992612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a44105920 msgr2=0x7f7a4419ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:10.926+0000 7f7a48c00700 1 --2- 172.21.15.5:0/3256992612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a44105920 0x7f7a4419ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:10.926+0000 7f7a48c00700 1 -- 172.21.15.5:0/3256992612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a441a04d0 con 0x7f7a44104f70 2024-04-24T06:43:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:10.926+0000 7f7a43e00700 1 --2- 172.21.15.5:0/3256992612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a44105920 0x7f7a4419ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:43:10.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:10.926+0000 7f7a48c00700 1 --2- 172.21.15.5:0/3256992612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a44104f70 0x7f7a4419b4c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800e9c0 tx=0x7f7a3800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:10.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:10.926+0000 7f7a41600700 1 -- 172.21.15.5:0/3256992612 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3800cbd0 con 0x7f7a44104f70 2024-04-24T06:43:10.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:10.926+0000 7f7a41600700 1 -- 172.21.15.5:0/3256992612 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a3800ee60 con 0x7f7a44104f70 2024-04-24T06:43:10.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:10.926+0000 7f7a4b600700 1 -- 172.21.15.5:0/3256992612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a441a07c0 con 0x7f7a44104f70 2024-04-24T06:43:10.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:10.926+0000 7f7a41600700 1 -- 172.21.15.5:0/3256992612 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a38005630 con 0x7f7a44104f70 2024-04-24T06:43:10.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:10.927+0000 7f7a4b600700 1 -- 172.21.15.5:0/3256992612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a441a8040 con 0x7f7a44104f70 2024-04-24T06:43:10.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:10.928+0000 7f7a4b600700 1 -- 172.21.15.5:0/3256992612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a44106450 con 0x7f7a44104f70 2024-04-24T06:43:10.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:10.930+0000 7f7a41600700 1 -- 172.21.15.5:0/3256992612 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f7a38004750 con 0x7f7a44104f70 2024-04-24T06:43:10.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:10.931+0000 7f7a41600700 1 --2- 172.21.15.5:0/3256992612 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f7a2c0a3cd0 0x7f7a2c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:10.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:10.931+0000 7f7a41600700 1 -- 172.21.15.5:0/3256992612 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f7a380d0850 con 0x7f7a44104f70 2024-04-24T06:43:10.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:10.931+0000 7f7a43e00700 1 --2- 172.21.15.5:0/3256992612 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f7a2c0a3cd0 0x7f7a2c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:10.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:10.932+0000 7f7a43e00700 1 --2- 172.21.15.5:0/3256992612 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f7a2c0a3cd0 0x7f7a2c0a6190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7a34000c00 tx=0x7f7a34005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:10.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:10.936+0000 7f7a41600700 1 -- 172.21.15.5:0/3256992612 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7a3809d090 con 0x7f7a44104f70 2024-04-24T06:43:11.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:11.219+0000 7f7a4b600700 1 -- 172.21.15.5:0/3256992612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f7a4419c820 con 0x7f7a44104f70 2024-04-24T06:43:11.224 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:11 smithi005 ceph-mon[27798]: pgmap v362: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T06:43:11.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:11.220+0000 7f7a41600700 1 -- 172.21.15.5:0/3256992612 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f7a3809c990 con 0x7f7a44104f70 2024-04-24T06:43:11.224 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:43:11.224 INFO:teuthology.orchestra.run.smithi005.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-24T06:43:11.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:11.223+0000 7f7a4b600700 1 -- 172.21.15.5:0/3256992612 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f7a2c0a3cd0 msgr2=0x7f7a2c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:11.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:11.223+0000 7f7a4b600700 1 --2- 172.21.15.5:0/3256992612 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f7a2c0a3cd0 0x7f7a2c0a6190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7a34000c00 tx=0x7f7a34005fb0 comp rx=0 tx=0).stop 2024-04-24T06:43:11.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:11.223+0000 7f7a4b600700 1 -- 172.21.15.5:0/3256992612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a44104f70 msgr2=0x7f7a4419b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:11.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:11.223+0000 7f7a4b600700 1 --2- 172.21.15.5:0/3256992612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a44104f70 0x7f7a4419b4c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800e9c0 tx=0x7f7a3800ecd0 comp rx=0 tx=0).stop 2024-04-24T06:43:11.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:11.224+0000 7f7a4b600700 1 -- 172.21.15.5:0/3256992612 shutdown_connections 2024-04-24T06:43:11.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:11.224+0000 7f7a4b600700 1 --2- 172.21.15.5:0/3256992612 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f7a2c0a3cd0 0x7f7a2c0a6190 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:11.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:11.224+0000 7f7a4b600700 1 --2- 172.21.15.5:0/3256992612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a4410a050 0x7f7a4419fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:11.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:11.224+0000 7f7a4b600700 1 --2- 172.21.15.5:0/3256992612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a44104f70 0x7f7a4419b4c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:11.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:11.224+0000 7f7a4b600700 1 --2- 172.21.15.5:0/3256992612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a44105920 0x7f7a4419ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:11.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:11.224+0000 7f7a4b600700 1 -- 172.21.15.5:0/3256992612 >> 172.21.15.5:0/3256992612 conn(0x7f7a44100800 msgr2=0x7f7a44102ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:11.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:11.224+0000 7f7a4b600700 1 -- 172.21.15.5:0/3256992612 shutdown_connections 2024-04-24T06:43:11.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:11.224+0000 7f7a4b600700 1 -- 172.21.15.5:0/3256992612 wait complete. 2024-04-24T06:43:11.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:11 smithi137 ceph-mon[33017]: pgmap v362: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T06:43:11.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:11 smithi037 ceph-mon[32864]: pgmap v362: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T06:43:12.022 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-24T06:43:12.023 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:43:12.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.237+0000 7fe393200640 1 -- 172.21.15.5:0/3006591088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3941052f0 msgr2=0x7fe3941076e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:12.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.237+0000 7fe393200640 1 --2- 172.21.15.5:0/3006591088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3941052f0 0x7fe3941076e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe384009be0 tx=0x7fe38402a6d0 comp rx=0 tx=0).stop 2024-04-24T06:43:12.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.237+0000 7fe393200640 1 -- 172.21.15.5:0/3006591088 shutdown_connections 2024-04-24T06:43:12.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.237+0000 7fe393200640 1 --2- 172.21.15.5:0/3006591088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe394107c20 0x7fe39410a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:12.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.237+0000 7fe393200640 1 --2- 172.21.15.5:0/3006591088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3941052f0 0x7fe3941076e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:12.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.237+0000 7fe393200640 1 --2- 172.21.15.5:0/3006591088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3941029c0 0x7fe394104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:12.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.237+0000 7fe393200640 1 -- 172.21.15.5:0/3006591088 >> 172.21.15.5:0/3006591088 conn(0x7fe3940fc3d0 msgr2=0x7fe3940fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:12.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.237+0000 7fe393200640 1 -- 172.21.15.5:0/3006591088 shutdown_connections 2024-04-24T06:43:12.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.237+0000 7fe393200640 1 -- 172.21.15.5:0/3006591088 wait complete. 2024-04-24T06:43:12.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.238+0000 7fe393200640 1 Processor -- start 2024-04-24T06:43:12.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.239+0000 7fe393200640 1 -- start start 2024-04-24T06:43:12.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.239+0000 7fe393200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3941029c0 0x7fe39419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:12.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.239+0000 7fe393200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3941052f0 0x7fe39419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:12.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.239+0000 7fe393200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe394107c20 0x7fe3941a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:12.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.239+0000 7fe393200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe39410dbb0 con 0x7fe394107c20 2024-04-24T06:43:12.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.239+0000 7fe393200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe39410da30 con 0x7fe3941029c0 2024-04-24T06:43:12.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.239+0000 7fe393200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe39410dd30 con 0x7fe3941052f0 2024-04-24T06:43:12.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.239+0000 7fe392800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe394107c20 0x7fe3941a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:12.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.239+0000 7fe392800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe394107c20 0x7fe3941a1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33530/0 (socket says 172.21.15.5:33530) 2024-04-24T06:43:12.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.239+0000 7fe392800640 1 -- 172.21.15.5:0/2627067356 learned_addr learned my addr 172.21.15.5:0/2627067356 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:12.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.239+0000 7fe391400640 1 --2- 172.21.15.5:0/2627067356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3941052f0 0x7fe39419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:12.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.239+0000 7fe391e00640 1 --2- 172.21.15.5:0/2627067356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3941029c0 0x7fe39419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:12.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.240+0000 7fe392800640 1 -- 172.21.15.5:0/2627067356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3941052f0 msgr2=0x7fe39419a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:12.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.240+0000 7fe392800640 1 --2- 172.21.15.5:0/2627067356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3941052f0 0x7fe39419a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:12.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.240+0000 7fe392800640 1 -- 172.21.15.5:0/2627067356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3941029c0 msgr2=0x7fe39419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:12.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.240+0000 7fe392800640 1 --2- 172.21.15.5:0/2627067356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3941029c0 0x7fe39419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:12.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.240+0000 7fe392800640 1 -- 172.21.15.5:0/2627067356 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3941a20a0 con 0x7fe394107c20 2024-04-24T06:43:12.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.240+0000 7fe392800640 1 --2- 172.21.15.5:0/2627067356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe394107c20 0x7fe3941a1a30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe38c00e970 tx=0x7fe38c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:12.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.240+0000 7fe38aa00640 1 -- 172.21.15.5:0/2627067356 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38c00ccc0 con 0x7fe394107c20 2024-04-24T06:43:12.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.240+0000 7fe38aa00640 1 -- 172.21.15.5:0/2627067356 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe38c00ce60 con 0x7fe394107c20 2024-04-24T06:43:12.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.240+0000 7fe38aa00640 1 -- 172.21.15.5:0/2627067356 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38c010490 con 0x7fe394107c20 2024-04-24T06:43:12.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.240+0000 7fe393200640 1 -- 172.21.15.5:0/2627067356 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3941a2390 con 0x7fe394107c20 2024-04-24T06:43:12.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.241+0000 7fe393200640 1 -- 172.21.15.5:0/2627067356 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3941a2860 con 0x7fe394107c20 2024-04-24T06:43:12.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.242+0000 7fe393200640 1 -- 172.21.15.5:0/2627067356 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe394109770 con 0x7fe394107c20 2024-04-24T06:43:12.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.244+0000 7fe38aa00640 1 -- 172.21.15.5:0/2627067356 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7fe38c005020 con 0x7fe394107c20 2024-04-24T06:43:12.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.245+0000 7fe38aa00640 1 --2- 172.21.15.5:0/2627067356 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fe3780a3ba0 0x7fe3780a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:12.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.245+0000 7fe38aa00640 1 -- 172.21.15.5:0/2627067356 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fe38c0d0860 con 0x7fe394107c20 2024-04-24T06:43:12.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.245+0000 7fe391e00640 1 --2- 172.21.15.5:0/2627067356 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fe3780a3ba0 0x7fe3780a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:12.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.246+0000 7fe391e00640 1 --2- 172.21.15.5:0/2627067356 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fe3780a3ba0 0x7fe3780a6060 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe37c0096c0 tx=0x7fe37c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:12.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.250+0000 7fe38aa00640 1 -- 172.21.15.5:0/2627067356 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe38c09d200 con 0x7fe394107c20 2024-04-24T06:43:12.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:12 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/3256992612' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-24T06:43:12.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:12 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/3256992612' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-24T06:43:12.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.459+0000 7fe393200640 1 -- 172.21.15.5:0/2627067356 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe394062c00 con 0x7fe394107c20 2024-04-24T06:43:12.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.463+0000 7fe38aa00640 1 -- 172.21.15.5:0/2627067356 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22513 (secure 0 0 0) 0x7fe38c09cb00 con 0x7fe394107c20 2024-04-24T06:43:12.465 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:43:12.465 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":88,"fsid":"43e9bad0-0204-11ef-bc93-c7b262605968","created":"2024-04-24T06:32:26.340555+0000","modified":"2024-04-24T06:42:42.855911+0000","last_up_change":"2024-04-24T06:40:36.790934+0000","last_in_change":"2024-04-24T06:40:17.803257+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:36:27.320342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:41:32.254502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:41:33.739067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c2e974c-745c-4dd5-8c3d-8e8da2eb1ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6803","nonce":2155858515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6805","nonce":2155858515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6809","nonce":2155858515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6807","nonce":2155858515}]},"public_addr":"172.21.15.5:6803/2155858515","cluster_addr":"172.21.15.5:6805/2155858515","heartbeat_back_addr":"172.21.15.5:6809/2155858515","heartbeat_front_addr":"172.21.15.5:6807/2155858515","state":["exists","up"]},{"osd":1,"uuid":"50467744-db20-4721-bf2d-1c5db938297b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6801","nonce":1302362650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6803","nonce":1302362650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6807","nonce":1302362650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6805","nonce":1302362650}]},"public_addr":"172.21.15.37:6801/1302362650","cluster_addr":"172.21.15.37:6803/1302362650","heartbeat_back_addr":"172.21.15.37:6807/1302362650","heartbeat_front_addr":"172.21.15.37:6805/1302362650","state":["exists","up"]},{"osd":2,"uuid":"0e190b15-d04c-4288-8d48-ecf6664392fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6801","nonce":2885027256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6803","nonce":2885027256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6807","nonce":2885027256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6805","nonce":2885027256}]},"public_addr":"172.21.15.137:6801/2885027256","cluster_addr":"172.21.15.137:6803/2885027256","heartbeat_back_addr":"172.21.15.137:6807/2885027256","heartbeat_front_addr":"172.21.15.137:6805/2885027256","state":["exists","up"]},{"osd":3,"uuid":"e05592ba-d96a-472a-9f63-09a535d04dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6811","nonce":3701747260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6813","nonce":3701747260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6817","nonce":3701747260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6815","nonce":3701747260}]},"public_addr":"172.21.15.5:6811/3701747260","cluster_addr":"172.21.15.5:6813/3701747260","heartbeat_back_addr":"172.21.15.5:6817/3701747260","heartbeat_front_addr":"172.21.15.5:6815/3701747260","state":["exists","up"]},{"osd":4,"uuid":"c3f36e51-7a3b-4dd9-af4b-a10558830c4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6809","nonce":759253844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6811","nonce":759253844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6815","nonce":759253844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6813","nonce":759253844}]},"public_addr":"172.21.15.37:6809/759253844","cluster_addr":"172.21.15.37:6811/759253844","heartbeat_back_addr":"172.21.15.37:6815/759253844","heartbeat_front_addr":"172.21.15.37:6813/759253844","state":["exists","up"]},{"osd":5,"uuid":"97a0500c-cbb8-4ff5-b476-8634b26b3821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6809","nonce":3771921560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6811","nonce":3771921560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6815","nonce":3771921560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6813","nonce":3771921560}]},"public_addr":"172.21.15.137:6809/3771921560","cluster_addr":"172.21.15.137:6811/3771921560","heartbeat_back_addr":"172.21.15.137:6815/3771921560","heartbeat_front_addr":"172.21.15.137:6813/3771921560","state":["exists","up"]},{"osd":6,"uuid":"33a33d62-bbb3-4ed9-834e-19ae93a97670","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6819","nonce":2064636084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6821","nonce":2064636084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6825","nonce":2064636084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6823","nonce":2064636084}]},"public_addr":"172.21.15.5:6819/2064636084","cluster_addr":"172.21.15.5:6821/2064636084","heartbeat_back_addr":"172.21.15.5:6825/2064636084","heartbeat_front_addr":"172.21.15.5:6823/2064636084","state":["exists","up"]},{"osd":7,"uuid":"c0f4e036-4717-4514-bb59-ab11fbaab063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6817","nonce":2952750659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6819","nonce":2952750659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6823","nonce":2952750659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6821","nonce":2952750659}]},"public_addr":"172.21.15.37:6817/2952750659","cluster_addr":"172.21.15.37:6819/2952750659","heartbeat_back_addr":"172.21.15.37:6823/2952750659","heartbeat_front_addr":"172.21.15.37:6821/2952750659","state":["exists","up"]},{"osd":8,"uuid":"5391fd87-44ea-440f-80da-c54d0327c04c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6817","nonce":29724945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6819","nonce":29724945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6823","nonce":29724945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6821","nonce":29724945}]},"public_addr":"172.21.15.137:6817/29724945","cluster_addr":"172.21.15.137:6819/29724945","heartbeat_back_addr":"172.21.15.137:6823/29724945","heartbeat_front_addr":"172.21.15.137:6821/29724945","state":["exists","up"]},{"osd":9,"uuid":"d94c9dbc-f43d-4237-ac7f-47a6a5ede1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6827","nonce":2257102107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6829","nonce":2257102107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6833","nonce":2257102107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6831","nonce":2257102107}]},"public_addr":"172.21.15.5:6827/2257102107","cluster_addr":"172.21.15.5:6829/2257102107","heartbeat_back_addr":"172.21.15.5:6833/2257102107","heartbeat_front_addr":"172.21.15.5:6831/2257102107","state":["exists","up"]},{"osd":10,"uuid":"e2455332-2a5e-44b1-85d3-8e47da399f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6825","nonce":2451570637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6827","nonce":2451570637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6831","nonce":2451570637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6829","nonce":2451570637}]},"public_addr":"172.21.15.37:6825/2451570637","cluster_addr":"172.21.15.37:6827/2451570637","heartbeat_back_addr":"172.21.15.37:6831/2451570637","heartbeat_front_addr":"172.21.15.37:6829/2451570637","state":["exists","up"]},{"osd":11,"uuid":"49d9396b-704a-4cef-8210-75d30c28e924","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6825","nonce":90600596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6827","nonce":90600596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6831","nonce":90600596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6829","nonce":90600596}]},"public_addr":"172.21.15.137:6825/90600596","cluster_addr":"172.21.15.137:6827/90600596","heartbeat_back_addr":"172.21.15.137:6831/90600596","heartbeat_front_addr":"172.21.15.137:6829/90600596","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:35:35.591619+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:00.568817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:24.195643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:56.451724+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:22.109293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:46.161749+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:19.278305+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:45.620256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:10.041165+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:43.813697+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:10.736026+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:35.384377+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/23307164":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6801/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:0/230785941":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/2940023597":"2024-04-25T06:33:07.040369+0000","172.21.15.5:6801/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3886613904":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/4199165437":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3459593499":"2024-04-25T06:33:41.161852+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:43:12.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.466+0000 7fe393200640 1 -- 172.21.15.5:0/2627067356 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fe3780a3ba0 msgr2=0x7fe3780a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:12.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.466+0000 7fe393200640 1 --2- 172.21.15.5:0/2627067356 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fe3780a3ba0 0x7fe3780a6060 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe37c0096c0 tx=0x7fe37c009260 comp rx=0 tx=0).stop 2024-04-24T06:43:12.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.466+0000 7fe393200640 1 -- 172.21.15.5:0/2627067356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe394107c20 msgr2=0x7fe3941a1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:12.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.466+0000 7fe393200640 1 --2- 172.21.15.5:0/2627067356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe394107c20 0x7fe3941a1a30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe38c00e970 tx=0x7fe38c00ed70 comp rx=0 tx=0).stop 2024-04-24T06:43:12.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.466+0000 7fe393200640 1 -- 172.21.15.5:0/2627067356 shutdown_connections 2024-04-24T06:43:12.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.466+0000 7fe393200640 1 --2- 172.21.15.5:0/2627067356 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fe3780a3ba0 0x7fe3780a6060 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:12.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.467+0000 7fe393200640 1 --2- 172.21.15.5:0/2627067356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe394107c20 0x7fe3941a1a30 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:12.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.467+0000 7fe393200640 1 --2- 172.21.15.5:0/2627067356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3941052f0 0x7fe39419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:12.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.467+0000 7fe393200640 1 --2- 172.21.15.5:0/2627067356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3941029c0 0x7fe39419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:12.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.467+0000 7fe393200640 1 -- 172.21.15.5:0/2627067356 >> 172.21.15.5:0/2627067356 conn(0x7fe3940fc3d0 msgr2=0x7fe394102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:12.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.467+0000 7fe393200640 1 -- 172.21.15.5:0/2627067356 shutdown_connections 2024-04-24T06:43:12.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.467+0000 7fe393200640 1 -- 172.21.15.5:0/2627067356 wait complete. 2024-04-24T06:43:12.489 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:36:27.320342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:41:32.254502+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:41:33.739067+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:43:12.489 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:43:12.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:12 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/3256992612' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-24T06:43:12.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.707+0000 7f3e47e00640 1 -- 172.21.15.5:0/85132262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e48074a00 msgr2=0x7f3e48074e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:12.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.707+0000 7f3e47e00640 1 --2- 172.21.15.5:0/85132262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e48074a00 0x7f3e48074e60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3e34009860 tx=0x7f3e3402a3d0 comp rx=0 tx=0).stop 2024-04-24T06:43:12.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.708+0000 7f3e47e00640 1 -- 172.21.15.5:0/85132262 shutdown_connections 2024-04-24T06:43:12.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.708+0000 7f3e47e00640 1 --2- 172.21.15.5:0/85132262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e48109d20 0x7f3e4810c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:12.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.708+0000 7f3e47e00640 1 --2- 172.21.15.5:0/85132262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e48074a00 0x7f3e48074e60 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:12.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.708+0000 7f3e47e00640 1 --2- 172.21.15.5:0/85132262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e48075fe0 0x7f3e48074430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:12.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.708+0000 7f3e47e00640 1 -- 172.21.15.5:0/85132262 >> 172.21.15.5:0/85132262 conn(0x7f3e480fd400 msgr2=0x7f3e480ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:12.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.708+0000 7f3e47e00640 1 -- 172.21.15.5:0/85132262 shutdown_connections 2024-04-24T06:43:12.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.708+0000 7f3e47e00640 1 -- 172.21.15.5:0/85132262 wait complete. 2024-04-24T06:43:12.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.709+0000 7f3e47e00640 1 Processor -- start 2024-04-24T06:43:12.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.709+0000 7f3e47e00640 1 -- start start 2024-04-24T06:43:12.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.709+0000 7f3e47e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e48074a00 0x7f3e4819a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:12.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.710+0000 7f3e47e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e48075fe0 0x7f3e4819a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:12.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.710+0000 7f3e47e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e48109d20 0x7f3e481a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:12.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.710+0000 7f3e47e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4810fd20 con 0x7f3e48109d20 2024-04-24T06:43:12.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.710+0000 7f3e47e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4810fba0 con 0x7f3e48074a00 2024-04-24T06:43:12.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.710+0000 7f3e47e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4810fea0 con 0x7f3e48075fe0 2024-04-24T06:43:12.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.710+0000 7f3e46000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e48075fe0 0x7f3e4819a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:12.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.710+0000 7f3e46a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e48074a00 0x7f3e4819a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:12.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.710+0000 7f3e47400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e48109d20 0x7f3e481a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:12.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.710+0000 7f3e46000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e48075fe0 0x7f3e4819a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:47098/0 (socket says 172.21.15.5:47098) 2024-04-24T06:43:12.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.710+0000 7f3e46000640 1 -- 172.21.15.5:0/3702412247 learned_addr learned my addr 172.21.15.5:0/3702412247 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:12.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.710+0000 7f3e46a00640 1 -- 172.21.15.5:0/3702412247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e48075fe0 msgr2=0x7f3e4819a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:12.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.710+0000 7f3e46a00640 1 --2- 172.21.15.5:0/3702412247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e48075fe0 0x7f3e4819a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:12.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.710+0000 7f3e46a00640 1 -- 172.21.15.5:0/3702412247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e48109d20 msgr2=0x7f3e481a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:12.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.710+0000 7f3e46a00640 1 --2- 172.21.15.5:0/3702412247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e48109d20 0x7f3e481a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:12.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.710+0000 7f3e46a00640 1 -- 172.21.15.5:0/3702412247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e481a1f80 con 0x7f3e48074a00 2024-04-24T06:43:12.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.710+0000 7f3e46000640 1 --2- 172.21.15.5:0/3702412247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e48075fe0 0x7f3e4819a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:43:12.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.711+0000 7f3e46a00640 1 --2- 172.21.15.5:0/3702412247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e48074a00 0x7f3e4819a360 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3e3800ef80 tx=0x7f3e3800c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:12.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.711+0000 7f3e3f400640 1 -- 172.21.15.5:0/3702412247 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e38003970 con 0x7f3e48074a00 2024-04-24T06:43:12.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.711+0000 7f3e3f400640 1 -- 172.21.15.5:0/3702412247 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e38004020 con 0x7f3e48074a00 2024-04-24T06:43:12.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.711+0000 7f3e47e00640 1 -- 172.21.15.5:0/3702412247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e481a2270 con 0x7f3e48074a00 2024-04-24T06:43:12.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.711+0000 7f3e3f400640 1 -- 172.21.15.5:0/3702412247 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3800ba10 con 0x7f3e48074a00 2024-04-24T06:43:12.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.711+0000 7f3e47e00640 1 -- 172.21.15.5:0/3702412247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e481a27c0 con 0x7f3e48074a00 2024-04-24T06:43:12.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.712+0000 7f3e47e00640 1 -- 172.21.15.5:0/3702412247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e48108ae0 con 0x7f3e48074a00 2024-04-24T06:43:12.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.714+0000 7f3e3f400640 1 -- 172.21.15.5:0/3702412247 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f3e3801d060 con 0x7f3e48074a00 2024-04-24T06:43:12.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.715+0000 7f3e3f400640 1 --2- 172.21.15.5:0/3702412247 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f3e2c0a3c30 0x7f3e2c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:12.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.716+0000 7f3e3f400640 1 -- 172.21.15.5:0/3702412247 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f3e3801b070 con 0x7f3e48074a00 2024-04-24T06:43:12.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.716+0000 7f3e46000640 1 --2- 172.21.15.5:0/3702412247 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f3e2c0a3c30 0x7f3e2c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:12.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.717+0000 7f3e46000640 1 --2- 172.21.15.5:0/3702412247 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f3e2c0a3c30 0x7f3e2c0a60f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3e340095f0 tx=0x7f3e34030ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:12.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.720+0000 7f3e3f400640 1 -- 172.21.15.5:0/3702412247 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3e3809cac0 con 0x7f3e48074a00 2024-04-24T06:43:12.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.933+0000 7f3e47e00640 1 -- 172.21.15.5:0/3702412247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3e48062c00 con 0x7f3e48074a00 2024-04-24T06:43:12.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.934+0000 7f3e3f400640 1 -- 172.21.15.5:0/3702412247 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f3e3809c3c0 con 0x7f3e48074a00 2024-04-24T06:43:12.936 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-24T06:43:12.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.937+0000 7f3e47e00640 1 -- 172.21.15.5:0/3702412247 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f3e2c0a3c30 msgr2=0x7f3e2c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:12.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.937+0000 7f3e47e00640 1 --2- 172.21.15.5:0/3702412247 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f3e2c0a3c30 0x7f3e2c0a60f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3e340095f0 tx=0x7f3e34030ab0 comp rx=0 tx=0).stop 2024-04-24T06:43:12.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.937+0000 7f3e47e00640 1 -- 172.21.15.5:0/3702412247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e48074a00 msgr2=0x7f3e4819a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:12.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.937+0000 7f3e47e00640 1 --2- 172.21.15.5:0/3702412247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e48074a00 0x7f3e4819a360 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3e3800ef80 tx=0x7f3e3800c800 comp rx=0 tx=0).stop 2024-04-24T06:43:12.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.938+0000 7f3e47e00640 1 -- 172.21.15.5:0/3702412247 shutdown_connections 2024-04-24T06:43:12.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.938+0000 7f3e47e00640 1 --2- 172.21.15.5:0/3702412247 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f3e2c0a3c30 0x7f3e2c0a60f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:12.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.938+0000 7f3e47e00640 1 --2- 172.21.15.5:0/3702412247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e48109d20 0x7f3e481a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:12.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.938+0000 7f3e47e00640 1 --2- 172.21.15.5:0/3702412247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e48075fe0 0x7f3e4819a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:12.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.938+0000 7f3e47e00640 1 --2- 172.21.15.5:0/3702412247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e48074a00 0x7f3e4819a360 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:12.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.938+0000 7f3e47e00640 1 -- 172.21.15.5:0/3702412247 >> 172.21.15.5:0/3702412247 conn(0x7f3e480fd400 msgr2=0x7f3e48105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:12.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.938+0000 7f3e47e00640 1 -- 172.21.15.5:0/3702412247 shutdown_connections 2024-04-24T06:43:12.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:12.938+0000 7f3e47e00640 1 -- 172.21.15.5:0/3702412247 wait complete. 2024-04-24T06:43:12.959 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:43:13.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.178+0000 7fa20ca00640 1 -- 172.21.15.5:0/3514028704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa208104250 msgr2=0x7fa208104670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:13.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.178+0000 7fa20ca00640 1 --2- 172.21.15.5:0/3514028704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa208104250 0x7fa208104670 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa200009900 tx=0x7fa20002a3f0 comp rx=0 tx=0).stop 2024-04-24T06:43:13.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.178+0000 7fa20ca00640 1 -- 172.21.15.5:0/3514028704 shutdown_connections 2024-04-24T06:43:13.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.178+0000 7fa20ca00640 1 --2- 172.21.15.5:0/3514028704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa208105e00 0x7fa20810c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:13.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.178+0000 7fa20ca00640 1 --2- 172.21.15.5:0/3514028704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa208105440 0x7fa2081058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:13.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.178+0000 7fa20ca00640 1 --2- 172.21.15.5:0/3514028704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa208104250 0x7fa208104670 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:13.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.178+0000 7fa20ca00640 1 -- 172.21.15.5:0/3514028704 >> 172.21.15.5:0/3514028704 conn(0x7fa2080ff7c0 msgr2=0x7fa208101c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:13.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.178+0000 7fa20ca00640 1 -- 172.21.15.5:0/3514028704 shutdown_connections 2024-04-24T06:43:13.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.178+0000 7fa20ca00640 1 -- 172.21.15.5:0/3514028704 wait complete. 2024-04-24T06:43:13.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.179+0000 7fa20ca00640 1 Processor -- start 2024-04-24T06:43:13.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.179+0000 7fa20ca00640 1 -- start start 2024-04-24T06:43:13.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.179+0000 7fa20ca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa208104250 0x7fa20819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:13.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.179+0000 7fa20ca00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa208105440 0x7fa20819aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:13.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.179+0000 7fa20ca00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa208105e00 0x7fa2081a1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:13.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.180+0000 7fa20ca00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa20810feb0 con 0x7fa208104250 2024-04-24T06:43:13.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.180+0000 7fa20ca00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa20810fd30 con 0x7fa208105440 2024-04-24T06:43:13.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.180+0000 7fa20ca00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa208110030 con 0x7fa208105e00 2024-04-24T06:43:13.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.180+0000 7fa205e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa208104250 0x7fa20819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:13.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.180+0000 7fa205e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa208104250 0x7fa20819a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33566/0 (socket says 172.21.15.5:33566) 2024-04-24T06:43:13.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.180+0000 7fa205e00640 1 -- 172.21.15.5:0/3767620946 learned_addr learned my addr 172.21.15.5:0/3767620946 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:13.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.180+0000 7fa206800640 1 --2- 172.21.15.5:0/3767620946 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa208105e00 0x7fa2081a1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:13.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.180+0000 7fa205400640 1 --2- 172.21.15.5:0/3767620946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa208105440 0x7fa20819aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:13.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.180+0000 7fa205e00640 1 -- 172.21.15.5:0/3767620946 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa208105e00 msgr2=0x7fa2081a1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:13.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.180+0000 7fa205e00640 1 --2- 172.21.15.5:0/3767620946 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa208105e00 0x7fa2081a1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:13.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.180+0000 7fa205e00640 1 -- 172.21.15.5:0/3767620946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa208105440 msgr2=0x7fa20819aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:13.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.180+0000 7fa205e00640 1 --2- 172.21.15.5:0/3767620946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa208105440 0x7fa20819aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:13.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.180+0000 7fa205e00640 1 -- 172.21.15.5:0/3767620946 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2081a21d0 con 0x7fa208104250 2024-04-24T06:43:13.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.180+0000 7fa205400640 1 --2- 172.21.15.5:0/3767620946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa208105440 0x7fa20819aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:43:13.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.181+0000 7fa205e00640 1 --2- 172.21.15.5:0/3767620946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa208104250 0x7fa20819a5b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa200031610 tx=0x7fa200004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:13.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.181+0000 7fa1fea00640 1 -- 172.21.15.5:0/3767620946 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa20003f030 con 0x7fa208104250 2024-04-24T06:43:13.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.181+0000 7fa1fea00640 1 -- 172.21.15.5:0/3767620946 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa200003ad0 con 0x7fa208104250 2024-04-24T06:43:13.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.181+0000 7fa20ca00640 1 -- 172.21.15.5:0/3767620946 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2081a2460 con 0x7fa208104250 2024-04-24T06:43:13.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.181+0000 7fa1fea00640 1 -- 172.21.15.5:0/3767620946 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa200049060 con 0x7fa208104250 2024-04-24T06:43:13.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.181+0000 7fa20ca00640 1 -- 172.21.15.5:0/3767620946 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2081a2950 con 0x7fa208104250 2024-04-24T06:43:13.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.182+0000 7fa20ca00640 1 -- 172.21.15.5:0/3767620946 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa20810b5c0 con 0x7fa208104250 2024-04-24T06:43:13.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.184+0000 7fa1fea00640 1 -- 172.21.15.5:0/3767620946 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7fa200042030 con 0x7fa208104250 2024-04-24T06:43:13.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.186+0000 7fa1fea00640 1 --2- 172.21.15.5:0/3767620946 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fa1ec0a3bb0 0x7fa1ec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:13.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.186+0000 7fa1fea00640 1 -- 172.21.15.5:0/3767620946 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fa2000f0d50 con 0x7fa208104250 2024-04-24T06:43:13.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.186+0000 7fa205400640 1 --2- 172.21.15.5:0/3767620946 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fa1ec0a3bb0 0x7fa1ec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:13.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.187+0000 7fa205400640 1 --2- 172.21.15.5:0/3767620946 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fa1ec0a3bb0 0x7fa1ec0a6070 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00099a0 tx=0x7fa1f0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:13.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.190+0000 7fa1fea00640 1 -- 172.21.15.5:0/3767620946 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa2000bd6f0 con 0x7fa208104250 2024-04-24T06:43:13.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:13 smithi005 ceph-mon[27798]: pgmap v363: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:43:13.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:13 smithi005 ceph-mon[27798]: mgrmap e18: x(active, since 9m), standbys: y, z 2024-04-24T06:43:13.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:13 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/2627067356' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:43:13.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:13 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/3702412247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T06:43:13.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.395+0000 7fa20ca00640 1 -- 172.21.15.5:0/3767620946 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa208062c00 con 0x7fa208104250 2024-04-24T06:43:13.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.396+0000 7fa1fea00640 1 -- 172.21.15.5:0/3767620946 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7fa2000dd020 con 0x7fa208104250 2024-04-24T06:43:13.397 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-24T06:43:13.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.399+0000 7fa20ca00640 1 -- 172.21.15.5:0/3767620946 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fa1ec0a3bb0 msgr2=0x7fa1ec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:13.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.399+0000 7fa20ca00640 1 --2- 172.21.15.5:0/3767620946 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fa1ec0a3bb0 0x7fa1ec0a6070 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00099a0 tx=0x7fa1f0009500 comp rx=0 tx=0).stop 2024-04-24T06:43:13.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.399+0000 7fa20ca00640 1 -- 172.21.15.5:0/3767620946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa208104250 msgr2=0x7fa20819a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:13.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.399+0000 7fa20ca00640 1 --2- 172.21.15.5:0/3767620946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa208104250 0x7fa20819a5b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa200031610 tx=0x7fa200004270 comp rx=0 tx=0).stop 2024-04-24T06:43:13.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.399+0000 7fa20ca00640 1 -- 172.21.15.5:0/3767620946 shutdown_connections 2024-04-24T06:43:13.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.399+0000 7fa20ca00640 1 --2- 172.21.15.5:0/3767620946 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fa1ec0a3bb0 0x7fa1ec0a6070 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:13.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.399+0000 7fa20ca00640 1 --2- 172.21.15.5:0/3767620946 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa208105e00 0x7fa2081a1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:13.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.399+0000 7fa20ca00640 1 --2- 172.21.15.5:0/3767620946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa208105440 0x7fa20819aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:13.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.399+0000 7fa20ca00640 1 --2- 172.21.15.5:0/3767620946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa208104250 0x7fa20819a5b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:13.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.399+0000 7fa20ca00640 1 -- 172.21.15.5:0/3767620946 >> 172.21.15.5:0/3767620946 conn(0x7fa2080ff7c0 msgr2=0x7fa208108790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:13.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.399+0000 7fa20ca00640 1 -- 172.21.15.5:0/3767620946 shutdown_connections 2024-04-24T06:43:13.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.399+0000 7fa20ca00640 1 -- 172.21.15.5:0/3767620946 wait complete. 2024-04-24T06:43:13.421 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:43:13.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:13 smithi137 ceph-mon[33017]: pgmap v363: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:43:13.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:13 smithi137 ceph-mon[33017]: mgrmap e18: x(active, since 9m), standbys: y, z 2024-04-24T06:43:13.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:13 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/2627067356' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:43:13.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:13 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/3702412247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T06:43:13.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:13 smithi037 ceph-mon[32864]: pgmap v363: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:43:13.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:13 smithi037 ceph-mon[32864]: mgrmap e18: x(active, since 9m), standbys: y, z 2024-04-24T06:43:13.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:13 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/2627067356' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:43:13.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:13 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/3702412247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T06:43:13.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.642+0000 7f4ed3e00640 1 -- 172.21.15.5:0/4257552570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ed40785a0 msgr2=0x7f4ed4078e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:13.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.642+0000 7f4ed3e00640 1 --2- 172.21.15.5:0/4257552570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ed40785a0 0x7f4ed4078e40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc009ca0 tx=0x7f4ecc02a8c0 comp rx=0 tx=0).stop 2024-04-24T06:43:13.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.642+0000 7f4ed3e00640 1 -- 172.21.15.5:0/4257552570 shutdown_connections 2024-04-24T06:43:13.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.642+0000 7f4ed3e00640 1 --2- 172.21.15.5:0/4257552570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ed40785a0 0x7f4ed4078e40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:13.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.642+0000 7f4ed3e00640 1 --2- 172.21.15.5:0/4257552570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ed4077c00 0x7f4ed4078060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:13.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.642+0000 7f4ed3e00640 1 --2- 172.21.15.5:0/4257552570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ed40769c0 0x7f4ed4076de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:13.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.642+0000 7f4ed3e00640 1 -- 172.21.15.5:0/4257552570 >> 172.21.15.5:0/4257552570 conn(0x7f4ed40ff610 msgr2=0x7f4ed4101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:13.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.642+0000 7f4ed3e00640 1 -- 172.21.15.5:0/4257552570 shutdown_connections 2024-04-24T06:43:13.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.642+0000 7f4ed3e00640 1 -- 172.21.15.5:0/4257552570 wait complete. 2024-04-24T06:43:13.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.643+0000 7f4ed3e00640 1 Processor -- start 2024-04-24T06:43:13.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.643+0000 7f4ed3e00640 1 -- start start 2024-04-24T06:43:13.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.644+0000 7f4ed3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ed40769c0 0x7f4ed419e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:13.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.644+0000 7f4ed3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ed4077c00 0x7f4ed419ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:13.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.644+0000 7f4ed3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ed40785a0 0x7f4ed41a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:13.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.644+0000 7f4ed3e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed4114150 con 0x7f4ed40785a0 2024-04-24T06:43:13.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.644+0000 7f4ed3e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed4113fd0 con 0x7f4ed40769c0 2024-04-24T06:43:13.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.644+0000 7f4ed3e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed41142d0 con 0x7f4ed4077c00 2024-04-24T06:43:13.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.644+0000 7f4ed2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ed40769c0 0x7f4ed419e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:13.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.644+0000 7f4ed2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ed40769c0 0x7f4ed419e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:59660/0 (socket says 172.21.15.5:59660) 2024-04-24T06:43:13.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.644+0000 7f4ed3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ed40785a0 0x7f4ed41a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:13.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.644+0000 7f4ed2a00640 1 -- 172.21.15.5:0/2995139409 learned_addr learned my addr 172.21.15.5:0/2995139409 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:13.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.644+0000 7f4ed2000640 1 --2- 172.21.15.5:0/2995139409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ed4077c00 0x7f4ed419ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:13.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.645+0000 7f4ed2a00640 1 -- 172.21.15.5:0/2995139409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ed4077c00 msgr2=0x7f4ed419ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:13.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.645+0000 7f4ed2a00640 1 --2- 172.21.15.5:0/2995139409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ed4077c00 0x7f4ed419ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:13.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.645+0000 7f4ed2a00640 1 -- 172.21.15.5:0/2995139409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ed40785a0 msgr2=0x7f4ed41a5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:13.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.645+0000 7f4ed2a00640 1 --2- 172.21.15.5:0/2995139409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ed40785a0 0x7f4ed41a5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:13.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.645+0000 7f4ed2a00640 1 -- 172.21.15.5:0/2995139409 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed41a6330 con 0x7f4ed40769c0 2024-04-24T06:43:13.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.645+0000 7f4ed2000640 1 --2- 172.21.15.5:0/2995139409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ed4077c00 0x7f4ed419ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:43:13.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.645+0000 7f4ed2a00640 1 --2- 172.21.15.5:0/2995139409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ed40769c0 0x7f4ed419e710 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4ec400e9f0 tx=0x7f4ec400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:13.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.645+0000 7f4ecb400640 1 -- 172.21.15.5:0/2995139409 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec400cd80 con 0x7f4ed40769c0 2024-04-24T06:43:13.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.645+0000 7f4ed3e00640 1 -- 172.21.15.5:0/2995139409 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ed41a6620 con 0x7f4ed40769c0 2024-04-24T06:43:13.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.645+0000 7f4ecb400640 1 -- 172.21.15.5:0/2995139409 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ec4005500 con 0x7f4ed40769c0 2024-04-24T06:43:13.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.645+0000 7f4ecb400640 1 -- 172.21.15.5:0/2995139409 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4010690 con 0x7f4ed40769c0 2024-04-24T06:43:13.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.645+0000 7f4ed3e00640 1 -- 172.21.15.5:0/2995139409 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ed41a6b20 con 0x7f4ed40769c0 2024-04-24T06:43:13.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.646+0000 7f4ed3e00640 1 -- 172.21.15.5:0/2995139409 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ed41109f0 con 0x7f4ed40769c0 2024-04-24T06:43:13.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.649+0000 7f4ecb400640 1 -- 172.21.15.5:0/2995139409 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f4ec40046c0 con 0x7f4ed40769c0 2024-04-24T06:43:13.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.650+0000 7f4ecb400640 1 --2- 172.21.15.5:0/2995139409 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f4eb80a3bb0 0x7f4eb80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:13.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.650+0000 7f4ecb400640 1 -- 172.21.15.5:0/2995139409 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f4ec40d07f0 con 0x7f4ed40769c0 2024-04-24T06:43:13.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.650+0000 7f4ed2000640 1 --2- 172.21.15.5:0/2995139409 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f4eb80a3bb0 0x7f4eb80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:13.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.651+0000 7f4ed2000640 1 --2- 172.21.15.5:0/2995139409 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f4eb80a3bb0 0x7f4eb80a6070 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00095f0 tx=0x7f4ec0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:13.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.653+0000 7f4ecb400640 1 -- 172.21.15.5:0/2995139409 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4ec409d190 con 0x7f4ed40769c0 2024-04-24T06:43:13.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.859+0000 7f4ed3e00640 1 -- 172.21.15.5:0/2995139409 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4ed4062c30 con 0x7f4ed40769c0 2024-04-24T06:43:13.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.860+0000 7f4ecb400640 1 -- 172.21.15.5:0/2995139409 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f4ec40bd0d0 con 0x7f4ed40769c0 2024-04-24T06:43:13.861 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-24T06:43:13.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.863+0000 7f4ed3e00640 1 -- 172.21.15.5:0/2995139409 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f4eb80a3bb0 msgr2=0x7f4eb80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:13.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.863+0000 7f4ed3e00640 1 --2- 172.21.15.5:0/2995139409 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f4eb80a3bb0 0x7f4eb80a6070 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00095f0 tx=0x7f4ec0005f90 comp rx=0 tx=0).stop 2024-04-24T06:43:13.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.863+0000 7f4ed3e00640 1 -- 172.21.15.5:0/2995139409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ed40769c0 msgr2=0x7f4ed419e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:13.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.863+0000 7f4ed3e00640 1 --2- 172.21.15.5:0/2995139409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ed40769c0 0x7f4ed419e710 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4ec400e9f0 tx=0x7f4ec400edf0 comp rx=0 tx=0).stop 2024-04-24T06:43:13.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.863+0000 7f4ed3e00640 1 -- 172.21.15.5:0/2995139409 shutdown_connections 2024-04-24T06:43:13.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.863+0000 7f4ed3e00640 1 --2- 172.21.15.5:0/2995139409 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f4eb80a3bb0 0x7f4eb80a6070 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:13.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.863+0000 7f4ed3e00640 1 --2- 172.21.15.5:0/2995139409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ed40785a0 0x7f4ed41a5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:13.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.863+0000 7f4ed3e00640 1 --2- 172.21.15.5:0/2995139409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ed4077c00 0x7f4ed419ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:13.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.863+0000 7f4ed3e00640 1 --2- 172.21.15.5:0/2995139409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ed40769c0 0x7f4ed419e710 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:13.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.863+0000 7f4ed3e00640 1 -- 172.21.15.5:0/2995139409 >> 172.21.15.5:0/2995139409 conn(0x7f4ed40ff610 msgr2=0x7f4ed4101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:13.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.863+0000 7f4ed3e00640 1 -- 172.21.15.5:0/2995139409 shutdown_connections 2024-04-24T06:43:13.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:13.863+0000 7f4ed3e00640 1 -- 172.21.15.5:0/2995139409 wait complete. 2024-04-24T06:43:13.884 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:43:14.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.101+0000 7fa55a800640 1 -- 172.21.15.5:0/3776433417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa554108680 msgr2=0x7fa554108aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:14.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.101+0000 7fa55a800640 1 --2- 172.21.15.5:0/3776433417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa554108680 0x7fa554108aa0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa544009860 tx=0x7fa54402a340 comp rx=0 tx=0).stop 2024-04-24T06:43:14.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.101+0000 7fa55a800640 1 -- 172.21.15.5:0/3776433417 shutdown_connections 2024-04-24T06:43:14.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.101+0000 7fa55a800640 1 --2- 172.21.15.5:0/3776433417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa55410a230 0x7fa554110ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:14.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.101+0000 7fa55a800640 1 --2- 172.21.15.5:0/3776433417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa554109870 0x7fa554109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:14.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.101+0000 7fa55a800640 1 --2- 172.21.15.5:0/3776433417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa554108680 0x7fa554108aa0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:14.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.101+0000 7fa55a800640 1 -- 172.21.15.5:0/3776433417 >> 172.21.15.5:0/3776433417 conn(0x7fa554077650 msgr2=0x7fa554079a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:14.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.101+0000 7fa55a800640 1 -- 172.21.15.5:0/3776433417 shutdown_connections 2024-04-24T06:43:14.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.101+0000 7fa55a800640 1 -- 172.21.15.5:0/3776433417 wait complete. 2024-04-24T06:43:14.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.103+0000 7fa55a800640 1 Processor -- start 2024-04-24T06:43:14.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.103+0000 7fa55a800640 1 -- start start 2024-04-24T06:43:14.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.103+0000 7fa55a800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa554108680 0x7fa55419e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:14.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.103+0000 7fa55a800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa554109870 0x7fa55419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:14.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.103+0000 7fa55a800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa55410a230 0x7fa5541a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:14.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.103+0000 7fa55a800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa554114300 con 0x7fa554109870 2024-04-24T06:43:14.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.103+0000 7fa55a800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa554114180 con 0x7fa55410a230 2024-04-24T06:43:14.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.103+0000 7fa55a800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa554114480 con 0x7fa554108680 2024-04-24T06:43:14.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.103+0000 7fa553400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa554108680 0x7fa55419e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:14.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.103+0000 7fa553400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa554108680 0x7fa55419e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:44500/0 (socket says 172.21.15.5:44500) 2024-04-24T06:43:14.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.103+0000 7fa553400640 1 -- 172.21.15.5:0/1333998809 learned_addr learned my addr 172.21.15.5:0/1333998809 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:14.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.104+0000 7fa54be00640 1 --2- 172.21.15.5:0/1333998809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa554109870 0x7fa55419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:14.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.104+0000 7fa553e00640 1 --2- 172.21.15.5:0/1333998809 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa55410a230 0x7fa5541a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:14.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.104+0000 7fa553400640 1 -- 172.21.15.5:0/1333998809 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa55410a230 msgr2=0x7fa5541a5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:14.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.104+0000 7fa553400640 1 --2- 172.21.15.5:0/1333998809 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa55410a230 0x7fa5541a5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:14.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.104+0000 7fa553400640 1 -- 172.21.15.5:0/1333998809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa554109870 msgr2=0x7fa55419eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:14.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.104+0000 7fa553400640 1 --2- 172.21.15.5:0/1333998809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa554109870 0x7fa55419eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:14.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.104+0000 7fa553400640 1 -- 172.21.15.5:0/1333998809 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5541a6590 con 0x7fa554108680 2024-04-24T06:43:14.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.104+0000 7fa553400640 1 --2- 172.21.15.5:0/1333998809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa554108680 0x7fa55419e970 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa544035040 tx=0x7fa544033d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:14.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.104+0000 7fa550c00640 1 -- 172.21.15.5:0/1333998809 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa544037040 con 0x7fa554108680 2024-04-24T06:43:14.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.104+0000 7fa550c00640 1 -- 172.21.15.5:0/1333998809 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa544030850 con 0x7fa554108680 2024-04-24T06:43:14.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.104+0000 7fa550c00640 1 -- 172.21.15.5:0/1333998809 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa544004920 con 0x7fa554108680 2024-04-24T06:43:14.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.105+0000 7fa55a800640 1 -- 172.21.15.5:0/1333998809 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5541a6820 con 0x7fa554108680 2024-04-24T06:43:14.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.105+0000 7fa55a800640 1 -- 172.21.15.5:0/1333998809 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5541a6cc0 con 0x7fa554108680 2024-04-24T06:43:14.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.106+0000 7fa55a800640 1 -- 172.21.15.5:0/1333998809 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa55410f9a0 con 0x7fa554108680 2024-04-24T06:43:14.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.108+0000 7fa550c00640 1 -- 172.21.15.5:0/1333998809 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7fa544004050 con 0x7fa554108680 2024-04-24T06:43:14.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.109+0000 7fa550c00640 1 --2- 172.21.15.5:0/1333998809 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fa5340a3c30 0x7fa5340a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:14.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.109+0000 7fa550c00640 1 -- 172.21.15.5:0/1333998809 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fa5440f0360 con 0x7fa554108680 2024-04-24T06:43:14.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.109+0000 7fa54be00640 1 --2- 172.21.15.5:0/1333998809 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fa5340a3c30 0x7fa5340a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:14.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.110+0000 7fa54be00640 1 --2- 172.21.15.5:0/1333998809 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fa5340a3c30 0x7fa5340a60f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa55419fe80 tx=0x7fa53c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:14.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.114+0000 7fa550c00640 1 -- 172.21.15.5:0/1333998809 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa5440bcd00 con 0x7fa554108680 2024-04-24T06:43:14.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:14 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/3767620946' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T06:43:14.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:14 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/2995139409' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T06:43:14.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.362+0000 7fa55a800640 1 -- 172.21.15.5:0/1333998809 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa55402d620 con 0x7fa554108680 2024-04-24T06:43:14.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.364+0000 7fa550c00640 1 -- 172.21.15.5:0/1333998809 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11856 (secure 0 0 0) 0x7fa5440bc600 con 0x7fa554108680 2024-04-24T06:43:14.365 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:43:14.365 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1735462750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1735462750},{"type":"v1","addr":"172.21.15.5:6837","nonce":1735462750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1499469370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1499469370},{"type":"v1","addr":"172.21.15.5:6841","nonce":1499469370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3705638847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3705638847},{"type":"v1","addr":"172.21.15.37:6835","nonce":3705638847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T06:41:34.764490+0000","modified":"2024-04-24T06:41:40.330991+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24550,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6837/1390103296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1390103296},{"type":"v1","addr":"172.21.15.37:6837","nonce":1390103296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3217754215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3217754215},{"type":"v1","addr":"172.21.15.137:6835","nonce":3217754215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6837/3044222976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3044222976},{"type":"v1","addr":"172.21.15.137:6837","nonce":3044222976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.5:6839/3173201459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3173201459},{"type":"v1","addr":"172.21.15.5:6839","nonce":3173201459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6839/1889774232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1889774232},{"type":"v1","addr":"172.21.15.137:6839","nonce":1889774232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6839/3536721753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":3536721753},{"type":"v1","addr":"172.21.15.37:6839","nonce":3536721753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24539,24550,24569]},"id":1}]} 2024-04-24T06:43:14.365 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 22 2024-04-24T06:43:14.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.366+0000 7fa55a800640 1 -- 172.21.15.5:0/1333998809 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fa5340a3c30 msgr2=0x7fa5340a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:14.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.366+0000 7fa55a800640 1 --2- 172.21.15.5:0/1333998809 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fa5340a3c30 0x7fa5340a60f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa55419fe80 tx=0x7fa53c009260 comp rx=0 tx=0).stop 2024-04-24T06:43:14.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.366+0000 7fa55a800640 1 -- 172.21.15.5:0/1333998809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa554108680 msgr2=0x7fa55419e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:14.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.366+0000 7fa55a800640 1 --2- 172.21.15.5:0/1333998809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa554108680 0x7fa55419e970 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa544035040 tx=0x7fa544033d90 comp rx=0 tx=0).stop 2024-04-24T06:43:14.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.366+0000 7fa55a800640 1 -- 172.21.15.5:0/1333998809 shutdown_connections 2024-04-24T06:43:14.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.366+0000 7fa55a800640 1 --2- 172.21.15.5:0/1333998809 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fa5340a3c30 0x7fa5340a60f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:14.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.366+0000 7fa55a800640 1 --2- 172.21.15.5:0/1333998809 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa55410a230 0x7fa5541a5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:14.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.366+0000 7fa55a800640 1 --2- 172.21.15.5:0/1333998809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa554109870 0x7fa55419eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:14.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.366+0000 7fa55a800640 1 --2- 172.21.15.5:0/1333998809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa554108680 0x7fa55419e970 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:14.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.366+0000 7fa55a800640 1 -- 172.21.15.5:0/1333998809 >> 172.21.15.5:0/1333998809 conn(0x7fa554077650 msgr2=0x7fa55410cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:14.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.367+0000 7fa55a800640 1 -- 172.21.15.5:0/1333998809 shutdown_connections 2024-04-24T06:43:14.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.367+0000 7fa55a800640 1 -- 172.21.15.5:0/1333998809 wait complete. 2024-04-24T06:43:14.388 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:43:14.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:14 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/3767620946' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T06:43:14.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:14 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/2995139409' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T06:43:14.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:14 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/3767620946' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T06:43:14.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:14 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/2995139409' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T06:43:14.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.601+0000 7f44c3e00640 1 -- 172.21.15.5:0/2375042439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44c4074a60 msgr2=0x7f44c4074ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:14.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.602+0000 7f44c3e00640 1 --2- 172.21.15.5:0/2375042439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44c4074a60 0x7f44c4074ee0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f44ac009900 tx=0x7f44ac02a410 comp rx=0 tx=0).stop 2024-04-24T06:43:14.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.602+0000 7f44c3e00640 1 -- 172.21.15.5:0/2375042439 shutdown_connections 2024-04-24T06:43:14.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.602+0000 7f44c3e00640 1 --2- 172.21.15.5:0/2375042439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44c4109e10 0x7f44c410c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:14.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.602+0000 7f44c3e00640 1 --2- 172.21.15.5:0/2375042439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44c4074a60 0x7f44c4074ee0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:14.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.602+0000 7f44c3e00640 1 --2- 172.21.15.5:0/2375042439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44c4076040 0x7f44c4074490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:14.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.602+0000 7f44c3e00640 1 -- 172.21.15.5:0/2375042439 >> 172.21.15.5:0/2375042439 conn(0x7f44c40fd4c0 msgr2=0x7f44c40ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:14.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.602+0000 7f44c3e00640 1 -- 172.21.15.5:0/2375042439 shutdown_connections 2024-04-24T06:43:14.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.602+0000 7f44c3e00640 1 -- 172.21.15.5:0/2375042439 wait complete. 2024-04-24T06:43:14.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.603+0000 7f44c3e00640 1 Processor -- start 2024-04-24T06:43:14.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.603+0000 7f44c3e00640 1 -- start start 2024-04-24T06:43:14.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.604+0000 7f44c3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44c4074a60 0x7f44c419a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:14.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.604+0000 7f44c3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44c4076040 0x7f44c419a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:14.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.604+0000 7f44c3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44c4109e10 0x7f44c41a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:14.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.604+0000 7f44c3e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f44c410fe20 con 0x7f44c4109e10 2024-04-24T06:43:14.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.604+0000 7f44c3e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f44c410fca0 con 0x7f44c4076040 2024-04-24T06:43:14.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.604+0000 7f44c3e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f44c410ffa0 con 0x7f44c4074a60 2024-04-24T06:43:14.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.604+0000 7f44c2a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44c4074a60 0x7f44c419a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:14.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.604+0000 7f44ba000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44c4076040 0x7f44c419a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:14.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.604+0000 7f44c3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44c4109e10 0x7f44c41a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:14.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.604+0000 7f44c2a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44c4074a60 0x7f44c419a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:44518/0 (socket says 172.21.15.5:44518) 2024-04-24T06:43:14.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.604+0000 7f44c2a00640 1 -- 172.21.15.5:0/3798894371 learned_addr learned my addr 172.21.15.5:0/3798894371 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:14.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.605+0000 7f44c2a00640 1 -- 172.21.15.5:0/3798894371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44c4076040 msgr2=0x7f44c419a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:14.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.605+0000 7f44c2a00640 1 --2- 172.21.15.5:0/3798894371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44c4076040 0x7f44c419a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:14.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.605+0000 7f44c2a00640 1 -- 172.21.15.5:0/3798894371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44c4109e10 msgr2=0x7f44c41a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:14.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.605+0000 7f44c2a00640 1 --2- 172.21.15.5:0/3798894371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44c4109e10 0x7f44c41a19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:14.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.605+0000 7f44c2a00640 1 -- 172.21.15.5:0/3798894371 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44c41a2010 con 0x7f44c4074a60 2024-04-24T06:43:14.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.605+0000 7f44c2a00640 1 --2- 172.21.15.5:0/3798894371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44c4074a60 0x7f44c419a3f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f44b400e9f0 tx=0x7f44b400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:14.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.605+0000 7f44bbe00640 1 -- 172.21.15.5:0/3798894371 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b400cd80 con 0x7f44c4074a60 2024-04-24T06:43:14.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.605+0000 7f44c3e00640 1 -- 172.21.15.5:0/3798894371 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44c41a2300 con 0x7f44c4074a60 2024-04-24T06:43:14.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.605+0000 7f44bbe00640 1 -- 172.21.15.5:0/3798894371 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44b4010910 con 0x7f44c4074a60 2024-04-24T06:43:14.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.605+0000 7f44bbe00640 1 -- 172.21.15.5:0/3798894371 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b40188f0 con 0x7f44c4074a60 2024-04-24T06:43:14.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.605+0000 7f44c3e00640 1 -- 172.21.15.5:0/3798894371 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44c41a2870 con 0x7f44c4074a60 2024-04-24T06:43:14.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.606+0000 7f44c3e00640 1 -- 172.21.15.5:0/3798894371 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44c4108b40 con 0x7f44c4074a60 2024-04-24T06:43:14.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.608+0000 7f44bbe00640 1 -- 172.21.15.5:0/3798894371 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f44b4010ac0 con 0x7f44c4074a60 2024-04-24T06:43:14.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.610+0000 7f44bbe00640 1 --2- 172.21.15.5:0/3798894371 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f44a40a3c20 0x7f44a40a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:14.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.610+0000 7f44bbe00640 1 -- 172.21.15.5:0/3798894371 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f44b40d0740 con 0x7f44c4074a60 2024-04-24T06:43:14.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.610+0000 7f44ba000640 1 --2- 172.21.15.5:0/3798894371 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f44a40a3c20 0x7f44a40a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:14.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.611+0000 7f44ba000640 1 --2- 172.21.15.5:0/3798894371 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f44a40a3c20 0x7f44a40a60e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f44ac031440 tx=0x7f44ac0315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:14.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.614+0000 7f44bbe00640 1 -- 172.21.15.5:0/3798894371 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f44b409ca90 con 0x7f44c4074a60 2024-04-24T06:43:14.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.824+0000 7f44c3e00640 1 -- 172.21.15.5:0/3798894371 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f44c402d620 con 0x7f44c4074a60 2024-04-24T06:43:14.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.826+0000 7f44bbe00640 1 -- 172.21.15.5:0/3798894371 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22513 (secure 0 0 0) 0x7f44b40a0d90 con 0x7f44c4074a60 2024-04-24T06:43:14.827 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:43:14.828 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":88,"fsid":"43e9bad0-0204-11ef-bc93-c7b262605968","created":"2024-04-24T06:32:26.340555+0000","modified":"2024-04-24T06:42:42.855911+0000","last_up_change":"2024-04-24T06:40:36.790934+0000","last_in_change":"2024-04-24T06:40:17.803257+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:36:27.320342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:41:32.254502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:41:33.739067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c2e974c-745c-4dd5-8c3d-8e8da2eb1ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6803","nonce":2155858515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6805","nonce":2155858515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6809","nonce":2155858515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6807","nonce":2155858515}]},"public_addr":"172.21.15.5:6803/2155858515","cluster_addr":"172.21.15.5:6805/2155858515","heartbeat_back_addr":"172.21.15.5:6809/2155858515","heartbeat_front_addr":"172.21.15.5:6807/2155858515","state":["exists","up"]},{"osd":1,"uuid":"50467744-db20-4721-bf2d-1c5db938297b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6801","nonce":1302362650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6803","nonce":1302362650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6807","nonce":1302362650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6805","nonce":1302362650}]},"public_addr":"172.21.15.37:6801/1302362650","cluster_addr":"172.21.15.37:6803/1302362650","heartbeat_back_addr":"172.21.15.37:6807/1302362650","heartbeat_front_addr":"172.21.15.37:6805/1302362650","state":["exists","up"]},{"osd":2,"uuid":"0e190b15-d04c-4288-8d48-ecf6664392fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6801","nonce":2885027256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6803","nonce":2885027256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6807","nonce":2885027256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6805","nonce":2885027256}]},"public_addr":"172.21.15.137:6801/2885027256","cluster_addr":"172.21.15.137:6803/2885027256","heartbeat_back_addr":"172.21.15.137:6807/2885027256","heartbeat_front_addr":"172.21.15.137:6805/2885027256","state":["exists","up"]},{"osd":3,"uuid":"e05592ba-d96a-472a-9f63-09a535d04dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6811","nonce":3701747260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6813","nonce":3701747260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6817","nonce":3701747260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6815","nonce":3701747260}]},"public_addr":"172.21.15.5:6811/3701747260","cluster_addr":"172.21.15.5:6813/3701747260","heartbeat_back_addr":"172.21.15.5:6817/3701747260","heartbeat_front_addr":"172.21.15.5:6815/3701747260","state":["exists","up"]},{"osd":4,"uuid":"c3f36e51-7a3b-4dd9-af4b-a10558830c4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6809","nonce":759253844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6811","nonce":759253844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6815","nonce":759253844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6813","nonce":759253844}]},"public_addr":"172.21.15.37:6809/759253844","cluster_addr":"172.21.15.37:6811/759253844","heartbeat_back_addr":"172.21.15.37:6815/759253844","heartbeat_front_addr":"172.21.15.37:6813/759253844","state":["exists","up"]},{"osd":5,"uuid":"97a0500c-cbb8-4ff5-b476-8634b26b3821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6809","nonce":3771921560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6811","nonce":3771921560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6815","nonce":3771921560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6813","nonce":3771921560}]},"public_addr":"172.21.15.137:6809/3771921560","cluster_addr":"172.21.15.137:6811/3771921560","heartbeat_back_addr":"172.21.15.137:6815/3771921560","heartbeat_front_addr":"172.21.15.137:6813/3771921560","state":["exists","up"]},{"osd":6,"uuid":"33a33d62-bbb3-4ed9-834e-19ae93a97670","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6819","nonce":2064636084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6821","nonce":2064636084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6825","nonce":2064636084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6823","nonce":2064636084}]},"public_addr":"172.21.15.5:6819/2064636084","cluster_addr":"172.21.15.5:6821/2064636084","heartbeat_back_addr":"172.21.15.5:6825/2064636084","heartbeat_front_addr":"172.21.15.5:6823/2064636084","state":["exists","up"]},{"osd":7,"uuid":"c0f4e036-4717-4514-bb59-ab11fbaab063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6817","nonce":2952750659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6819","nonce":2952750659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6823","nonce":2952750659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6821","nonce":2952750659}]},"public_addr":"172.21.15.37:6817/2952750659","cluster_addr":"172.21.15.37:6819/2952750659","heartbeat_back_addr":"172.21.15.37:6823/2952750659","heartbeat_front_addr":"172.21.15.37:6821/2952750659","state":["exists","up"]},{"osd":8,"uuid":"5391fd87-44ea-440f-80da-c54d0327c04c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6817","nonce":29724945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6819","nonce":29724945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6823","nonce":29724945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6821","nonce":29724945}]},"public_addr":"172.21.15.137:6817/29724945","cluster_addr":"172.21.15.137:6819/29724945","heartbeat_back_addr":"172.21.15.137:6823/29724945","heartbeat_front_addr":"172.21.15.137:6821/29724945","state":["exists","up"]},{"osd":9,"uuid":"d94c9dbc-f43d-4237-ac7f-47a6a5ede1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6827","nonce":2257102107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6829","nonce":2257102107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6833","nonce":2257102107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6831","nonce":2257102107}]},"public_addr":"172.21.15.5:6827/2257102107","cluster_addr":"172.21.15.5:6829/2257102107","heartbeat_back_addr":"172.21.15.5:6833/2257102107","heartbeat_front_addr":"172.21.15.5:6831/2257102107","state":["exists","up"]},{"osd":10,"uuid":"e2455332-2a5e-44b1-85d3-8e47da399f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6825","nonce":2451570637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6827","nonce":2451570637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6831","nonce":2451570637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6829","nonce":2451570637}]},"public_addr":"172.21.15.37:6825/2451570637","cluster_addr":"172.21.15.37:6827/2451570637","heartbeat_back_addr":"172.21.15.37:6831/2451570637","heartbeat_front_addr":"172.21.15.37:6829/2451570637","state":["exists","up"]},{"osd":11,"uuid":"49d9396b-704a-4cef-8210-75d30c28e924","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6825","nonce":90600596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6827","nonce":90600596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6831","nonce":90600596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6829","nonce":90600596}]},"public_addr":"172.21.15.137:6825/90600596","cluster_addr":"172.21.15.137:6827/90600596","heartbeat_back_addr":"172.21.15.137:6831/90600596","heartbeat_front_addr":"172.21.15.137:6829/90600596","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:35:35.591619+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:00.568817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:24.195643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:56.451724+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:22.109293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:46.161749+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:19.278305+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:45.620256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:10.041165+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:43.813697+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:10.736026+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:35.384377+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/23307164":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6801/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:0/230785941":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/2940023597":"2024-04-25T06:33:07.040369+0000","172.21.15.5:6801/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3886613904":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/4199165437":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3459593499":"2024-04-25T06:33:41.161852+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:43:14.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.829+0000 7f44c3e00640 1 -- 172.21.15.5:0/3798894371 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f44a40a3c20 msgr2=0x7f44a40a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:14.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.829+0000 7f44c3e00640 1 --2- 172.21.15.5:0/3798894371 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f44a40a3c20 0x7f44a40a60e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f44ac031440 tx=0x7f44ac0315b0 comp rx=0 tx=0).stop 2024-04-24T06:43:14.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.829+0000 7f44c3e00640 1 -- 172.21.15.5:0/3798894371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44c4074a60 msgr2=0x7f44c419a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:14.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.829+0000 7f44c3e00640 1 --2- 172.21.15.5:0/3798894371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44c4074a60 0x7f44c419a3f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f44b400e9f0 tx=0x7f44b400eeb0 comp rx=0 tx=0).stop 2024-04-24T06:43:14.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.829+0000 7f44c3e00640 1 -- 172.21.15.5:0/3798894371 shutdown_connections 2024-04-24T06:43:14.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.829+0000 7f44c3e00640 1 --2- 172.21.15.5:0/3798894371 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f44a40a3c20 0x7f44a40a60e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:14.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.829+0000 7f44c3e00640 1 --2- 172.21.15.5:0/3798894371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44c4109e10 0x7f44c41a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:14.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.829+0000 7f44c3e00640 1 --2- 172.21.15.5:0/3798894371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44c4076040 0x7f44c419a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:14.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.829+0000 7f44c3e00640 1 --2- 172.21.15.5:0/3798894371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44c4074a60 0x7f44c419a3f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:14.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.829+0000 7f44c3e00640 1 -- 172.21.15.5:0/3798894371 >> 172.21.15.5:0/3798894371 conn(0x7f44c40fd4c0 msgr2=0x7f44c4105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:14.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.829+0000 7f44c3e00640 1 -- 172.21.15.5:0/3798894371 shutdown_connections 2024-04-24T06:43:14.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:14.829+0000 7f44c3e00640 1 -- 172.21.15.5:0/3798894371 wait complete. 2024-04-24T06:43:14.851 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:36:27.320342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:41:32.254502+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:41:33.739067+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:43:14.851 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:43:15.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.073+0000 7fabaee00640 1 -- 172.21.15.5:0/1294148358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faba8105290 msgr2=0x7faba8105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:15.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.073+0000 7fabaee00640 1 --2- 172.21.15.5:0/1294148358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faba8105290 0x7faba8105710 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fab94009900 tx=0x7fab9402a400 comp rx=0 tx=0).stop 2024-04-24T06:43:15.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.073+0000 7fabaee00640 1 -- 172.21.15.5:0/1294148358 shutdown_connections 2024-04-24T06:43:15.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.073+0000 7fabaee00640 1 --2- 172.21.15.5:0/1294148358 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faba8105c50 0x7faba810c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:15.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.073+0000 7fabaee00640 1 --2- 172.21.15.5:0/1294148358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faba8105290 0x7faba8105710 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:15.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.073+0000 7fabaee00640 1 --2- 172.21.15.5:0/1294148358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faba81040a0 0x7faba81044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:15.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.073+0000 7fabaee00640 1 -- 172.21.15.5:0/1294148358 >> 172.21.15.5:0/1294148358 conn(0x7faba80ff610 msgr2=0x7faba8101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:15.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.073+0000 7fabaee00640 1 -- 172.21.15.5:0/1294148358 shutdown_connections 2024-04-24T06:43:15.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.073+0000 7fabaee00640 1 -- 172.21.15.5:0/1294148358 wait complete. 2024-04-24T06:43:15.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.074+0000 7fabaee00640 1 Processor -- start 2024-04-24T06:43:15.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.075+0000 7fabaee00640 1 -- start start 2024-04-24T06:43:15.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.075+0000 7fabaee00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faba81040a0 0x7faba819a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:15.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.075+0000 7fabaee00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faba8105290 0x7faba819a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:15.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.075+0000 7fabaee00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faba8105c50 0x7faba81a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:15.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.075+0000 7fabaee00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faba810fd80 con 0x7faba8105290 2024-04-24T06:43:15.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.075+0000 7fabaee00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faba810fc00 con 0x7faba81040a0 2024-04-24T06:43:15.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.075+0000 7fabaee00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faba810ff00 con 0x7faba8105c50 2024-04-24T06:43:15.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.075+0000 7fabad000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faba8105290 0x7faba819a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:15.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.076+0000 7fabada00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faba81040a0 0x7faba819a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:15.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.076+0000 7fabad000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faba8105290 0x7faba819a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58572/0 (socket says 172.21.15.5:58572) 2024-04-24T06:43:15.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.076+0000 7fabad000640 1 -- 172.21.15.5:0/1358745883 learned_addr learned my addr 172.21.15.5:0/1358745883 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:15.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.076+0000 7fabae400640 1 --2- 172.21.15.5:0/1358745883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faba8105c50 0x7faba81a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:15.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.076+0000 7fabad000640 1 -- 172.21.15.5:0/1358745883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faba8105c50 msgr2=0x7faba81a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:15.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.076+0000 7fabad000640 1 --2- 172.21.15.5:0/1358745883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faba8105c50 0x7faba81a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:15.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.076+0000 7fabad000640 1 -- 172.21.15.5:0/1358745883 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faba81040a0 msgr2=0x7faba819a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:15.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.076+0000 7fabad000640 1 --2- 172.21.15.5:0/1358745883 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faba81040a0 0x7faba819a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:15.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.076+0000 7fabad000640 1 -- 172.21.15.5:0/1358745883 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba81a1fa0 con 0x7faba8105290 2024-04-24T06:43:15.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.076+0000 7fabada00640 1 --2- 172.21.15.5:0/1358745883 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faba81040a0 0x7faba819a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:43:15.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.076+0000 7fabad000640 1 --2- 172.21.15.5:0/1358745883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faba8105290 0x7faba819a8c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fab94031440 tx=0x7fab94032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:15.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.077+0000 7faba6000640 1 -- 172.21.15.5:0/1358745883 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9403f030 con 0x7faba8105290 2024-04-24T06:43:15.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.077+0000 7faba6000640 1 -- 172.21.15.5:0/1358745883 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab94003ce0 con 0x7faba8105290 2024-04-24T06:43:15.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.077+0000 7fabaee00640 1 -- 172.21.15.5:0/1358745883 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faba81a2230 con 0x7faba8105290 2024-04-24T06:43:15.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.077+0000 7faba6000640 1 -- 172.21.15.5:0/1358745883 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab94035070 con 0x7faba8105290 2024-04-24T06:43:15.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.077+0000 7fabaee00640 1 -- 172.21.15.5:0/1358745883 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faba81a2770 con 0x7faba8105290 2024-04-24T06:43:15.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.078+0000 7fabaee00640 1 -- 172.21.15.5:0/1358745883 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faba810b3c0 con 0x7faba8105290 2024-04-24T06:43:15.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.080+0000 7faba6000640 1 -- 172.21.15.5:0/1358745883 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7fab94042030 con 0x7faba8105290 2024-04-24T06:43:15.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.083+0000 7faba6000640 1 --2- 172.21.15.5:0/1358745883 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fab900a3c30 0x7fab900a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:15.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.083+0000 7faba6000640 1 -- 172.21.15.5:0/1358745883 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fab940f04f0 con 0x7faba8105290 2024-04-24T06:43:15.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.083+0000 7fabada00640 1 --2- 172.21.15.5:0/1358745883 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fab900a3c30 0x7fab900a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:15.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.084+0000 7fabada00640 1 --2- 172.21.15.5:0/1358745883 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fab900a3c30 0x7fab900a60f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7faba81050f0 tx=0x7fab9c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:15.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.086+0000 7faba6000640 1 -- 172.21.15.5:0/1358745883 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fab940bc840 con 0x7faba8105290 2024-04-24T06:43:15.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:15 smithi005 ceph-mon[27798]: pgmap v364: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T06:43:15.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:15 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1333998809' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:43:15.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:15 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/3798894371' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:43:15.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.287+0000 7fabaee00640 1 -- 172.21.15.5:0/1358745883 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7faba8062c00 con 0x7faba8105290 2024-04-24T06:43:15.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.289+0000 7faba6000640 1 -- 172.21.15.5:0/1358745883 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fab940c0b40 con 0x7faba8105290 2024-04-24T06:43:15.290 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-24T06:43:15.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.291+0000 7fabaee00640 1 -- 172.21.15.5:0/1358745883 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fab900a3c30 msgr2=0x7fab900a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:15.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.291+0000 7fabaee00640 1 --2- 172.21.15.5:0/1358745883 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fab900a3c30 0x7fab900a60f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7faba81050f0 tx=0x7fab9c005f90 comp rx=0 tx=0).stop 2024-04-24T06:43:15.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.291+0000 7fabaee00640 1 -- 172.21.15.5:0/1358745883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faba8105290 msgr2=0x7faba819a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:15.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.291+0000 7fabaee00640 1 --2- 172.21.15.5:0/1358745883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faba8105290 0x7faba819a8c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fab94031440 tx=0x7fab94032ce0 comp rx=0 tx=0).stop 2024-04-24T06:43:15.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.291+0000 7fabaee00640 1 -- 172.21.15.5:0/1358745883 shutdown_connections 2024-04-24T06:43:15.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.291+0000 7fabaee00640 1 --2- 172.21.15.5:0/1358745883 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fab900a3c30 0x7fab900a60f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:15.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.291+0000 7fabaee00640 1 --2- 172.21.15.5:0/1358745883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faba8105c50 0x7faba81a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:15.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.291+0000 7fabaee00640 1 --2- 172.21.15.5:0/1358745883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faba8105290 0x7faba819a8c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:15.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.291+0000 7fabaee00640 1 --2- 172.21.15.5:0/1358745883 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faba81040a0 0x7faba819a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:15.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.291+0000 7fabaee00640 1 -- 172.21.15.5:0/1358745883 >> 172.21.15.5:0/1358745883 conn(0x7faba80ff610 msgr2=0x7faba8108590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:15.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.292+0000 7fabaee00640 1 -- 172.21.15.5:0/1358745883 shutdown_connections 2024-04-24T06:43:15.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.292+0000 7fabaee00640 1 -- 172.21.15.5:0/1358745883 wait complete. 2024-04-24T06:43:15.312 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:43:15.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:15 smithi137 ceph-mon[33017]: pgmap v364: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T06:43:15.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:15 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/1333998809' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:43:15.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:15 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/3798894371' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:43:15.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.511+0000 7f6648c00640 1 -- 172.21.15.5:0/824126948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66441040a0 msgr2=0x7f66441044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:15.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.511+0000 7f6648c00640 1 --2- 172.21.15.5:0/824126948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66441040a0 0x7f66441044c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6634009900 tx=0x7f663402a410 comp rx=0 tx=0).stop 2024-04-24T06:43:15.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.512+0000 7f6648c00640 1 -- 172.21.15.5:0/824126948 shutdown_connections 2024-04-24T06:43:15.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.512+0000 7f6648c00640 1 --2- 172.21.15.5:0/824126948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6644105c50 0x7f664410c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:15.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.512+0000 7f6648c00640 1 --2- 172.21.15.5:0/824126948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6644105290 0x7f6644105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:15.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.512+0000 7f6648c00640 1 --2- 172.21.15.5:0/824126948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66441040a0 0x7f66441044c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:15.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.512+0000 7f6648c00640 1 -- 172.21.15.5:0/824126948 >> 172.21.15.5:0/824126948 conn(0x7f66440ff610 msgr2=0x7f6644101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:15.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.512+0000 7f6648c00640 1 -- 172.21.15.5:0/824126948 shutdown_connections 2024-04-24T06:43:15.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.512+0000 7f6648c00640 1 -- 172.21.15.5:0/824126948 wait complete. 2024-04-24T06:43:15.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.513+0000 7f6648c00640 1 Processor -- start 2024-04-24T06:43:15.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.513+0000 7f6648c00640 1 -- start start 2024-04-24T06:43:15.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.513+0000 7f6648c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66441040a0 0x7f664419a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:15.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.513+0000 7f6648c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6644105290 0x7f664419a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:15.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.513+0000 7f6648c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6644105c50 0x7f66441a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:15.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.513+0000 7f6648c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f664410fd30 con 0x7f6644105290 2024-04-24T06:43:15.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.513+0000 7f6648c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f664410fbb0 con 0x7f66441040a0 2024-04-24T06:43:15.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.513+0000 7f6648c00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f664410feb0 con 0x7f6644105c50 2024-04-24T06:43:15.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.513+0000 7f6642a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6644105290 0x7f664419a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:15.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.514+0000 7f6642a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6644105290 0x7f664419a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58596/0 (socket says 172.21.15.5:58596) 2024-04-24T06:43:15.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.514+0000 7f6642a00640 1 -- 172.21.15.5:0/3734623447 learned_addr learned my addr 172.21.15.5:0/3734623447 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:15.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.514+0000 7f6643e00640 1 --2- 172.21.15.5:0/3734623447 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6644105c50 0x7f66441a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:15.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.514+0000 7f6643400640 1 --2- 172.21.15.5:0/3734623447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66441040a0 0x7f664419a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:15.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.514+0000 7f6642a00640 1 -- 172.21.15.5:0/3734623447 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6644105c50 msgr2=0x7f66441a1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:15.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.514+0000 7f6642a00640 1 --2- 172.21.15.5:0/3734623447 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6644105c50 0x7f66441a1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:15.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.514+0000 7f6642a00640 1 -- 172.21.15.5:0/3734623447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66441040a0 msgr2=0x7f664419a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:15.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.514+0000 7f6642a00640 1 --2- 172.21.15.5:0/3734623447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66441040a0 0x7f664419a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:15.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.514+0000 7f6642a00640 1 -- 172.21.15.5:0/3734623447 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66441a1fe0 con 0x7f6644105290 2024-04-24T06:43:15.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.514+0000 7f6642a00640 1 --2- 172.21.15.5:0/3734623447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6644105290 0x7f664419a900 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f663000e9f0 tx=0x7f663000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:15.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.515+0000 7f663be00640 1 -- 172.21.15.5:0/3734623447 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663000cd80 con 0x7f6644105290 2024-04-24T06:43:15.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.515+0000 7f6648c00640 1 -- 172.21.15.5:0/3734623447 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66441a22d0 con 0x7f6644105290 2024-04-24T06:43:15.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.515+0000 7f663be00640 1 -- 172.21.15.5:0/3734623447 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6630010910 con 0x7f6644105290 2024-04-24T06:43:15.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.515+0000 7f6648c00640 1 -- 172.21.15.5:0/3734623447 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66441a2870 con 0x7f6644105290 2024-04-24T06:43:15.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.515+0000 7f663be00640 1 -- 172.21.15.5:0/3734623447 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66300188f0 con 0x7f6644105290 2024-04-24T06:43:15.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.516+0000 7f6648c00640 1 -- 172.21.15.5:0/3734623447 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f664410b3c0 con 0x7f6644105290 2024-04-24T06:43:15.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:15 smithi037 ceph-mon[32864]: pgmap v364: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T06:43:15.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:15 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/1333998809' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:43:15.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:15 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/3798894371' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:43:15.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.519+0000 7f663be00640 1 -- 172.21.15.5:0/3734623447 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f6630010ac0 con 0x7f6644105290 2024-04-24T06:43:15.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.520+0000 7f663be00640 1 --2- 172.21.15.5:0/3734623447 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f66280a3c30 0x7f66280a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:15.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.520+0000 7f663be00640 1 -- 172.21.15.5:0/3734623447 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f66300d0760 con 0x7f6644105290 2024-04-24T06:43:15.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.521+0000 7f6643400640 1 --2- 172.21.15.5:0/3734623447 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f66280a3c30 0x7f66280a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:15.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.521+0000 7f6643400640 1 --2- 172.21.15.5:0/3734623447 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f66280a3c30 0x7f66280a60f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6634005ab0 tx=0x7f6634000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:15.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.525+0000 7f663be00640 1 -- 172.21.15.5:0/3734623447 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f663009cab0 con 0x7f6644105290 2024-04-24T06:43:15.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.729+0000 7f6648c00640 1 -- 172.21.15.5:0/3734623447 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6644062c00 con 0x7f6644105290 2024-04-24T06:43:15.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.730+0000 7f663be00640 1 -- 172.21.15.5:0/3734623447 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f66300a0db0 con 0x7f6644105290 2024-04-24T06:43:15.731 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-24T06:43:15.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.732+0000 7f6648c00640 1 -- 172.21.15.5:0/3734623447 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f66280a3c30 msgr2=0x7f66280a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:15.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.732+0000 7f6648c00640 1 --2- 172.21.15.5:0/3734623447 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f66280a3c30 0x7f66280a60f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6634005ab0 tx=0x7f6634000bc0 comp rx=0 tx=0).stop 2024-04-24T06:43:15.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.732+0000 7f6648c00640 1 -- 172.21.15.5:0/3734623447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6644105290 msgr2=0x7f664419a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:15.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.732+0000 7f6648c00640 1 --2- 172.21.15.5:0/3734623447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6644105290 0x7f664419a900 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f663000e9f0 tx=0x7f663000eeb0 comp rx=0 tx=0).stop 2024-04-24T06:43:15.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.732+0000 7f6648c00640 1 -- 172.21.15.5:0/3734623447 shutdown_connections 2024-04-24T06:43:15.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.732+0000 7f6648c00640 1 --2- 172.21.15.5:0/3734623447 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f66280a3c30 0x7f66280a60f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:15.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.732+0000 7f6648c00640 1 --2- 172.21.15.5:0/3734623447 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6644105c50 0x7f66441a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:15.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.732+0000 7f6648c00640 1 --2- 172.21.15.5:0/3734623447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6644105290 0x7f664419a900 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:15.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.732+0000 7f6648c00640 1 --2- 172.21.15.5:0/3734623447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66441040a0 0x7f664419a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:15.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.732+0000 7f6648c00640 1 -- 172.21.15.5:0/3734623447 >> 172.21.15.5:0/3734623447 conn(0x7f66440ff610 msgr2=0x7f6644108590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:15.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.733+0000 7f6648c00640 1 -- 172.21.15.5:0/3734623447 shutdown_connections 2024-04-24T06:43:15.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.733+0000 7f6648c00640 1 -- 172.21.15.5:0/3734623447 wait complete. 2024-04-24T06:43:15.750 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:43:15.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.956+0000 7f7f38800640 1 -- 172.21.15.5:0/3092135932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f30103c10 msgr2=0x7f7f3010a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:15.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.956+0000 7f7f38800640 1 --2- 172.21.15.5:0/3092135932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f30103c10 0x7f7f3010a4a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c009be0 tx=0x7f7f2c02a6c0 comp rx=0 tx=0).stop 2024-04-24T06:43:15.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.956+0000 7f7f38800640 1 -- 172.21.15.5:0/3092135932 shutdown_connections 2024-04-24T06:43:15.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.956+0000 7f7f38800640 1 --2- 172.21.15.5:0/3092135932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f30103c10 0x7f7f3010a4a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:15.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.956+0000 7f7f38800640 1 --2- 172.21.15.5:0/3092135932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f30103250 0x7f7f301036d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:15.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.956+0000 7f7f38800640 1 --2- 172.21.15.5:0/3092135932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f30102060 0x7f7f30102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:15.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.956+0000 7f7f38800640 1 -- 172.21.15.5:0/3092135932 >> 172.21.15.5:0/3092135932 conn(0x7f7f300fd5d0 msgr2=0x7f7f300ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:15.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.956+0000 7f7f38800640 1 -- 172.21.15.5:0/3092135932 shutdown_connections 2024-04-24T06:43:15.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.956+0000 7f7f38800640 1 -- 172.21.15.5:0/3092135932 wait complete. 2024-04-24T06:43:15.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.957+0000 7f7f38800640 1 Processor -- start 2024-04-24T06:43:15.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.957+0000 7f7f38800640 1 -- start start 2024-04-24T06:43:15.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.957+0000 7f7f38800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f30102060 0x7f7f3019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:15.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.957+0000 7f7f38800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f30103250 0x7f7f3019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:15.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.957+0000 7f7f38800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f30103c10 0x7f7f301a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:15.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.957+0000 7f7f38800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3010dca0 con 0x7f7f30102060 2024-04-24T06:43:15.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.957+0000 7f7f38800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3010db20 con 0x7f7f30103250 2024-04-24T06:43:15.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.957+0000 7f7f38800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3010de20 con 0x7f7f30103c10 2024-04-24T06:43:15.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.958+0000 7f7f35e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f30102060 0x7f7f3019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:15.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.958+0000 7f7f35e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f30102060 0x7f7f3019a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58626/0 (socket says 172.21.15.5:58626) 2024-04-24T06:43:15.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.958+0000 7f7f35e00640 1 -- 172.21.15.5:0/1414093749 learned_addr learned my addr 172.21.15.5:0/1414093749 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:15.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.958+0000 7f7f35400640 1 --2- 172.21.15.5:0/1414093749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f30103250 0x7f7f3019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:15.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.958+0000 7f7f36800640 1 --2- 172.21.15.5:0/1414093749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f30103c10 0x7f7f301a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:15.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.958+0000 7f7f35e00640 1 -- 172.21.15.5:0/1414093749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f30103c10 msgr2=0x7f7f301a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:15.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.958+0000 7f7f35e00640 1 --2- 172.21.15.5:0/1414093749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f30103c10 0x7f7f301a1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:15.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.958+0000 7f7f35e00640 1 -- 172.21.15.5:0/1414093749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f30103250 msgr2=0x7f7f3019aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:15.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.958+0000 7f7f35e00640 1 --2- 172.21.15.5:0/1414093749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f30103250 0x7f7f3019aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:15.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.958+0000 7f7f35e00640 1 -- 172.21.15.5:0/1414093749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f301a2190 con 0x7f7f30102060 2024-04-24T06:43:15.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.958+0000 7f7f36800640 1 --2- 172.21.15.5:0/1414093749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f30103c10 0x7f7f301a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:43:15.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.958+0000 7f7f35400640 1 --2- 172.21.15.5:0/1414093749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f30103250 0x7f7f3019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:43:15.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.958+0000 7f7f35e00640 1 --2- 172.21.15.5:0/1414093749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f30102060 0x7f7f3019a570 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7f2400e9c0 tx=0x7f7f2400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:15.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.959+0000 7f7f2aa00640 1 -- 172.21.15.5:0/1414093749 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2400ccc0 con 0x7f7f30102060 2024-04-24T06:43:15.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.959+0000 7f7f2aa00640 1 -- 172.21.15.5:0/1414093749 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f240035f0 con 0x7f7f30102060 2024-04-24T06:43:15.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.959+0000 7f7f38800640 1 -- 172.21.15.5:0/1414093749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f301a2480 con 0x7f7f30102060 2024-04-24T06:43:15.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.959+0000 7f7f2aa00640 1 -- 172.21.15.5:0/1414093749 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2400ce60 con 0x7f7f30102060 2024-04-24T06:43:15.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.959+0000 7f7f38800640 1 -- 172.21.15.5:0/1414093749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f301a29d0 con 0x7f7f30102060 2024-04-24T06:43:15.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.960+0000 7f7f38800640 1 -- 172.21.15.5:0/1414093749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f301a2c90 con 0x7f7f30102060 2024-04-24T06:43:15.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.962+0000 7f7f2aa00640 1 -- 172.21.15.5:0/1414093749 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f7f240037a0 con 0x7f7f30102060 2024-04-24T06:43:15.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.963+0000 7f7f2aa00640 1 --2- 172.21.15.5:0/1414093749 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f7f180a3bb0 0x7f7f180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:15.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.964+0000 7f7f2aa00640 1 -- 172.21.15.5:0/1414093749 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f7f240d08a0 con 0x7f7f30102060 2024-04-24T06:43:15.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.964+0000 7f7f35400640 1 --2- 172.21.15.5:0/1414093749 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f7f180a3bb0 0x7f7f180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:15.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.965+0000 7f7f35400640 1 --2- 172.21.15.5:0/1414093749 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f7f180a3bb0 0x7f7f180a6070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7f3019ba80 tx=0x7f7f20009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:15.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:15.968+0000 7f7f2aa00640 1 -- 172.21.15.5:0/1414093749 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7f2409d240 con 0x7f7f30102060 2024-04-24T06:43:16.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.176+0000 7f7f38800640 1 -- 172.21.15.5:0/1414093749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7f301a2e60 con 0x7f7f30102060 2024-04-24T06:43:16.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.177+0000 7f7f2aa00640 1 -- 172.21.15.5:0/1414093749 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f7f2409cb40 con 0x7f7f30102060 2024-04-24T06:43:16.178 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-24T06:43:16.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.179+0000 7f7f38800640 1 -- 172.21.15.5:0/1414093749 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f7f180a3bb0 msgr2=0x7f7f180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:16.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.179+0000 7f7f38800640 1 --2- 172.21.15.5:0/1414093749 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f7f180a3bb0 0x7f7f180a6070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7f3019ba80 tx=0x7f7f20009260 comp rx=0 tx=0).stop 2024-04-24T06:43:16.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.179+0000 7f7f38800640 1 -- 172.21.15.5:0/1414093749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f30102060 msgr2=0x7f7f3019a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:16.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.179+0000 7f7f38800640 1 --2- 172.21.15.5:0/1414093749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f30102060 0x7f7f3019a570 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7f2400e9c0 tx=0x7f7f2400edc0 comp rx=0 tx=0).stop 2024-04-24T06:43:16.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.179+0000 7f7f38800640 1 -- 172.21.15.5:0/1414093749 shutdown_connections 2024-04-24T06:43:16.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.179+0000 7f7f38800640 1 --2- 172.21.15.5:0/1414093749 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f7f180a3bb0 0x7f7f180a6070 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:16.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.179+0000 7f7f38800640 1 --2- 172.21.15.5:0/1414093749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f30103c10 0x7f7f301a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:16.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.179+0000 7f7f38800640 1 --2- 172.21.15.5:0/1414093749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f30103250 0x7f7f3019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:16.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.179+0000 7f7f38800640 1 --2- 172.21.15.5:0/1414093749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f30102060 0x7f7f3019a570 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:16.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.179+0000 7f7f38800640 1 -- 172.21.15.5:0/1414093749 >> 172.21.15.5:0/1414093749 conn(0x7f7f300fd5d0 msgr2=0x7f7f30074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:16.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.180+0000 7f7f38800640 1 -- 172.21.15.5:0/1414093749 shutdown_connections 2024-04-24T06:43:16.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.180+0000 7f7f38800640 1 -- 172.21.15.5:0/1414093749 wait complete. 2024-04-24T06:43:16.196 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:16 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1358745883' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T06:43:16.196 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:16 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/3734623447' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T06:43:16.200 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:43:16.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.392+0000 7f1d77e00640 1 -- 172.21.15.5:0/524629351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d78077c00 msgr2=0x7f1d78078060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:16.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.392+0000 7f1d77e00640 1 --2- 172.21.15.5:0/524629351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d78077c00 0x7f1d78078060 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1d64009900 tx=0x7f1d6402a3f0 comp rx=0 tx=0).stop 2024-04-24T06:43:16.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.392+0000 7f1d77e00640 1 -- 172.21.15.5:0/524629351 shutdown_connections 2024-04-24T06:43:16.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.392+0000 7f1d77e00640 1 --2- 172.21.15.5:0/524629351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d780785a0 0x7f1d78078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:16.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.392+0000 7f1d77e00640 1 --2- 172.21.15.5:0/524629351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d78077c00 0x7f1d78078060 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:16.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.392+0000 7f1d77e00640 1 --2- 172.21.15.5:0/524629351 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d780769c0 0x7f1d78076de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:16.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.392+0000 7f1d77e00640 1 -- 172.21.15.5:0/524629351 >> 172.21.15.5:0/524629351 conn(0x7f1d780ff6d0 msgr2=0x7f1d78101b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:16.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.392+0000 7f1d77e00640 1 -- 172.21.15.5:0/524629351 shutdown_connections 2024-04-24T06:43:16.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.392+0000 7f1d77e00640 1 -- 172.21.15.5:0/524629351 wait complete. 2024-04-24T06:43:16.585 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:16 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/1358745883' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T06:43:16.585 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:16 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/3734623447' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T06:43:16.585 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:16 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/1358745883' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T06:43:16.585 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:16 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/3734623447' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T06:43:16.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.393+0000 7f1d77e00640 1 Processor -- start 2024-04-24T06:43:16.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.393+0000 7f1d77e00640 1 -- start start 2024-04-24T06:43:16.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.393+0000 7f1d77e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d780769c0 0x7f1d7819e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:16.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.393+0000 7f1d77e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d78077c00 0x7f1d7819eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:16.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.393+0000 7f1d77e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d780785a0 0x7f1d781a5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:16.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.393+0000 7f1d77e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1d78114230 con 0x7f1d780769c0 2024-04-24T06:43:16.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.393+0000 7f1d77e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1d781140b0 con 0x7f1d780785a0 2024-04-24T06:43:16.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.393+0000 7f1d77e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1d781143b0 con 0x7f1d78077c00 2024-04-24T06:43:16.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.393+0000 7f1d76a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d780769c0 0x7f1d7819e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:16.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.393+0000 7f1d76a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d780769c0 0x7f1d7819e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58636/0 (socket says 172.21.15.5:58636) 2024-04-24T06:43:16.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.394+0000 7f1d76a00640 1 -- 172.21.15.5:0/2088335495 learned_addr learned my addr 172.21.15.5:0/2088335495 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:16.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.394+0000 7f1d76000640 1 --2- 172.21.15.5:0/2088335495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d78077c00 0x7f1d7819eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:16.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.394+0000 7f1d77400640 1 --2- 172.21.15.5:0/2088335495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d780785a0 0x7f1d781a5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:16.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.394+0000 7f1d76a00640 1 -- 172.21.15.5:0/2088335495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d78077c00 msgr2=0x7f1d7819eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:16.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.394+0000 7f1d76a00640 1 --2- 172.21.15.5:0/2088335495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d78077c00 0x7f1d7819eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:16.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.394+0000 7f1d76a00640 1 -- 172.21.15.5:0/2088335495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d780785a0 msgr2=0x7f1d781a5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:16.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.394+0000 7f1d76a00640 1 --2- 172.21.15.5:0/2088335495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d780785a0 0x7f1d781a5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:16.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.394+0000 7f1d76a00640 1 -- 172.21.15.5:0/2088335495 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d781a6480 con 0x7f1d780769c0 2024-04-24T06:43:16.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.394+0000 7f1d77400640 1 --2- 172.21.15.5:0/2088335495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d780785a0 0x7f1d781a5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:43:16.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.394+0000 7f1d76a00640 1 --2- 172.21.15.5:0/2088335495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d780769c0 0x7f1d7819e860 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1d6800e9f0 tx=0x7f1d6800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:16.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.394+0000 7f1d6f400640 1 -- 172.21.15.5:0/2088335495 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6800cd80 con 0x7f1d780769c0 2024-04-24T06:43:16.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.395+0000 7f1d77e00640 1 -- 172.21.15.5:0/2088335495 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d781a6770 con 0x7f1d780769c0 2024-04-24T06:43:16.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.395+0000 7f1d6f400640 1 -- 172.21.15.5:0/2088335495 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d68010910 con 0x7f1d780769c0 2024-04-24T06:43:16.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.395+0000 7f1d6f400640 1 -- 172.21.15.5:0/2088335495 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d680188f0 con 0x7f1d780769c0 2024-04-24T06:43:16.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.395+0000 7f1d77e00640 1 -- 172.21.15.5:0/2088335495 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d781a6ce0 con 0x7f1d780769c0 2024-04-24T06:43:16.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.396+0000 7f1d77e00640 1 -- 172.21.15.5:0/2088335495 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d78110ad0 con 0x7f1d780769c0 2024-04-24T06:43:16.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.398+0000 7f1d6f400640 1 -- 172.21.15.5:0/2088335495 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f1d68010ac0 con 0x7f1d780769c0 2024-04-24T06:43:16.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.399+0000 7f1d6f400640 1 --2- 172.21.15.5:0/2088335495 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f1d5c0a3c20 0x7f1d5c0a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:16.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.399+0000 7f1d6f400640 1 -- 172.21.15.5:0/2088335495 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f1d680d07f0 con 0x7f1d780769c0 2024-04-24T06:43:16.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.400+0000 7f1d76000640 1 --2- 172.21.15.5:0/2088335495 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f1d5c0a3c20 0x7f1d5c0a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:16.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.400+0000 7f1d76000640 1 --2- 172.21.15.5:0/2088335495 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f1d5c0a3c20 0x7f1d5c0a60e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1d64031440 tx=0x7f1d640315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:16.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.403+0000 7f1d6f400640 1 -- 172.21.15.5:0/2088335495 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1d6809d190 con 0x7f1d780769c0 2024-04-24T06:43:16.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.648+0000 7f1d77e00640 1 -- 172.21.15.5:0/2088335495 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d7802d620 con 0x7f1d780769c0 2024-04-24T06:43:16.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.649+0000 7f1d6f400640 1 -- 172.21.15.5:0/2088335495 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11856 (secure 0 0 0) 0x7f1d6809ca90 con 0x7f1d780769c0 2024-04-24T06:43:16.655 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:43:16.655 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1735462750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1735462750},{"type":"v1","addr":"172.21.15.5:6837","nonce":1735462750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1499469370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1499469370},{"type":"v1","addr":"172.21.15.5:6841","nonce":1499469370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3705638847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3705638847},{"type":"v1","addr":"172.21.15.37:6835","nonce":3705638847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T06:41:34.764490+0000","modified":"2024-04-24T06:41:40.330991+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24550,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6837/1390103296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1390103296},{"type":"v1","addr":"172.21.15.37:6837","nonce":1390103296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3217754215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3217754215},{"type":"v1","addr":"172.21.15.137:6835","nonce":3217754215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6837/3044222976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3044222976},{"type":"v1","addr":"172.21.15.137:6837","nonce":3044222976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.5:6839/3173201459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3173201459},{"type":"v1","addr":"172.21.15.5:6839","nonce":3173201459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6839/1889774232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1889774232},{"type":"v1","addr":"172.21.15.137:6839","nonce":1889774232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6839/3536721753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":3536721753},{"type":"v1","addr":"172.21.15.37:6839","nonce":3536721753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24539,24550,24569]},"id":1}]} 2024-04-24T06:43:16.655 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 22 2024-04-24T06:43:16.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.651+0000 7f1d77e00640 1 -- 172.21.15.5:0/2088335495 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f1d5c0a3c20 msgr2=0x7f1d5c0a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:16.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.652+0000 7f1d77e00640 1 --2- 172.21.15.5:0/2088335495 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f1d5c0a3c20 0x7f1d5c0a60e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1d64031440 tx=0x7f1d640315b0 comp rx=0 tx=0).stop 2024-04-24T06:43:16.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.652+0000 7f1d77e00640 1 -- 172.21.15.5:0/2088335495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d780769c0 msgr2=0x7f1d7819e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:16.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.652+0000 7f1d77e00640 1 --2- 172.21.15.5:0/2088335495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d780769c0 0x7f1d7819e860 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1d6800e9f0 tx=0x7f1d6800eeb0 comp rx=0 tx=0).stop 2024-04-24T06:43:16.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.652+0000 7f1d77e00640 1 -- 172.21.15.5:0/2088335495 shutdown_connections 2024-04-24T06:43:16.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.652+0000 7f1d77e00640 1 --2- 172.21.15.5:0/2088335495 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f1d5c0a3c20 0x7f1d5c0a60e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:16.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.652+0000 7f1d77e00640 1 --2- 172.21.15.5:0/2088335495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d780785a0 0x7f1d781a5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:16.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.652+0000 7f1d77e00640 1 --2- 172.21.15.5:0/2088335495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d78077c00 0x7f1d7819eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:16.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.652+0000 7f1d77e00640 1 --2- 172.21.15.5:0/2088335495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d780769c0 0x7f1d7819e860 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:16.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.652+0000 7f1d77e00640 1 -- 172.21.15.5:0/2088335495 >> 172.21.15.5:0/2088335495 conn(0x7f1d780ff6d0 msgr2=0x7f1d781011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:16.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.652+0000 7f1d77e00640 1 -- 172.21.15.5:0/2088335495 shutdown_connections 2024-04-24T06:43:16.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.652+0000 7f1d77e00640 1 -- 172.21.15.5:0/2088335495 wait complete. 2024-04-24T06:43:16.672 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:43:16.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.868+0000 7f7734800640 1 -- 172.21.15.5:0/3049768276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f772c1042c0 msgr2=0x7f772c1046e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:16.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.868+0000 7f7734800640 1 --2- 172.21.15.5:0/3049768276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f772c1042c0 0x7f772c1046e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7720009900 tx=0x7f772002a410 comp rx=0 tx=0).stop 2024-04-24T06:43:16.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.868+0000 7f7734800640 1 -- 172.21.15.5:0/3049768276 shutdown_connections 2024-04-24T06:43:16.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.868+0000 7f7734800640 1 --2- 172.21.15.5:0/3049768276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f772c105e70 0x7f772c10c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:16.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.868+0000 7f7734800640 1 --2- 172.21.15.5:0/3049768276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f772c1054b0 0x7f772c105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:16.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.868+0000 7f7734800640 1 --2- 172.21.15.5:0/3049768276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f772c1042c0 0x7f772c1046e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:16.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.868+0000 7f7734800640 1 -- 172.21.15.5:0/3049768276 >> 172.21.15.5:0/3049768276 conn(0x7f772c0ff850 msgr2=0x7f772c101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:16.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.868+0000 7f7734800640 1 -- 172.21.15.5:0/3049768276 shutdown_connections 2024-04-24T06:43:16.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.868+0000 7f7734800640 1 -- 172.21.15.5:0/3049768276 wait complete. 2024-04-24T06:43:16.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.869+0000 7f7734800640 1 Processor -- start 2024-04-24T06:43:16.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.869+0000 7f7734800640 1 -- start start 2024-04-24T06:43:16.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.870+0000 7f7734800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f772c1042c0 0x7f772c19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:16.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.870+0000 7f7734800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f772c1054b0 0x7f772c19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:16.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.870+0000 7f7734800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f772c105e70 0x7f772c1a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:16.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.870+0000 7f7734800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f772c10ff30 con 0x7f772c105e70 2024-04-24T06:43:16.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.870+0000 7f7734800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f772c10fdb0 con 0x7f772c1042c0 2024-04-24T06:43:16.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.870+0000 7f7734800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f772c1100b0 con 0x7f772c1054b0 2024-04-24T06:43:16.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.870+0000 7f7731e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f772c1042c0 0x7f772c19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:16.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.870+0000 7f7731400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f772c1054b0 0x7f772c19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:16.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.870+0000 7f7731e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f772c1042c0 0x7f772c19a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:53042/0 (socket says 172.21.15.5:53042) 2024-04-24T06:43:16.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.870+0000 7f7731e00640 1 -- 172.21.15.5:0/2773162493 learned_addr learned my addr 172.21.15.5:0/2773162493 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:16.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.870+0000 7f7732800640 1 --2- 172.21.15.5:0/2773162493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f772c105e70 0x7f772c1a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:16.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.870+0000 7f7731400640 1 -- 172.21.15.5:0/2773162493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f772c1042c0 msgr2=0x7f772c19a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:16.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.870+0000 7f7731400640 1 --2- 172.21.15.5:0/2773162493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f772c1042c0 0x7f772c19a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:16.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.870+0000 7f7731400640 1 -- 172.21.15.5:0/2773162493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f772c105e70 msgr2=0x7f772c1a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:16.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.870+0000 7f7731400640 1 --2- 172.21.15.5:0/2773162493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f772c105e70 0x7f772c1a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:16.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.870+0000 7f7731400640 1 -- 172.21.15.5:0/2773162493 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f772c1a2160 con 0x7f772c1054b0 2024-04-24T06:43:16.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.871+0000 7f7731400640 1 --2- 172.21.15.5:0/2773162493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f772c1054b0 0x7f772c19aa80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f771c00e9f0 tx=0x7f771c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:16.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.871+0000 7f7726a00640 1 -- 172.21.15.5:0/2773162493 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771c00cd80 con 0x7f772c1054b0 2024-04-24T06:43:16.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.871+0000 7f7734800640 1 -- 172.21.15.5:0/2773162493 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f772c1a2450 con 0x7f772c1054b0 2024-04-24T06:43:16.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.871+0000 7f7726a00640 1 -- 172.21.15.5:0/2773162493 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f771c0035f0 con 0x7f772c1054b0 2024-04-24T06:43:16.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.871+0000 7f7726a00640 1 -- 172.21.15.5:0/2773162493 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771c018690 con 0x7f772c1054b0 2024-04-24T06:43:16.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.871+0000 7f7734800640 1 -- 172.21.15.5:0/2773162493 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f772c1a29f0 con 0x7f772c1054b0 2024-04-24T06:43:16.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.872+0000 7f7734800640 1 -- 172.21.15.5:0/2773162493 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f772c10b660 con 0x7f772c1054b0 2024-04-24T06:43:16.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.874+0000 7f7726a00640 1 -- 172.21.15.5:0/2773162493 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f771c004020 con 0x7f772c1054b0 2024-04-24T06:43:16.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.876+0000 7f7726a00640 1 --2- 172.21.15.5:0/2773162493 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f77140a3c30 0x7f77140a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:16.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.876+0000 7f7726a00640 1 -- 172.21.15.5:0/2773162493 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f771c0d0640 con 0x7f772c1054b0 2024-04-24T06:43:16.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.876+0000 7f7731e00640 1 --2- 172.21.15.5:0/2773162493 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f77140a3c30 0x7f77140a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:16.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.877+0000 7f7731e00640 1 --2- 172.21.15.5:0/2773162493 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f77140a3c30 0x7f77140a60f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7720000c00 tx=0x7f77200315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:16.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:16.880+0000 7f7726a00640 1 -- 172.21.15.5:0/2773162493 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f771c09cfe0 con 0x7f772c1054b0 2024-04-24T06:43:17.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.085+0000 7f7734800640 1 -- 172.21.15.5:0/2773162493 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f772c062c00 con 0x7f772c1054b0 2024-04-24T06:43:17.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.088+0000 7f7726a00640 1 -- 172.21.15.5:0/2773162493 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22513 (secure 0 0 0) 0x7f771c09c8e0 con 0x7f772c1054b0 2024-04-24T06:43:17.089 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:43:17.089 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":88,"fsid":"43e9bad0-0204-11ef-bc93-c7b262605968","created":"2024-04-24T06:32:26.340555+0000","modified":"2024-04-24T06:42:42.855911+0000","last_up_change":"2024-04-24T06:40:36.790934+0000","last_in_change":"2024-04-24T06:40:17.803257+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:36:27.320342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:41:32.254502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:41:33.739067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c2e974c-745c-4dd5-8c3d-8e8da2eb1ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6803","nonce":2155858515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6805","nonce":2155858515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6809","nonce":2155858515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6807","nonce":2155858515}]},"public_addr":"172.21.15.5:6803/2155858515","cluster_addr":"172.21.15.5:6805/2155858515","heartbeat_back_addr":"172.21.15.5:6809/2155858515","heartbeat_front_addr":"172.21.15.5:6807/2155858515","state":["exists","up"]},{"osd":1,"uuid":"50467744-db20-4721-bf2d-1c5db938297b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6801","nonce":1302362650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6803","nonce":1302362650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6807","nonce":1302362650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6805","nonce":1302362650}]},"public_addr":"172.21.15.37:6801/1302362650","cluster_addr":"172.21.15.37:6803/1302362650","heartbeat_back_addr":"172.21.15.37:6807/1302362650","heartbeat_front_addr":"172.21.15.37:6805/1302362650","state":["exists","up"]},{"osd":2,"uuid":"0e190b15-d04c-4288-8d48-ecf6664392fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6801","nonce":2885027256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6803","nonce":2885027256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6807","nonce":2885027256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6805","nonce":2885027256}]},"public_addr":"172.21.15.137:6801/2885027256","cluster_addr":"172.21.15.137:6803/2885027256","heartbeat_back_addr":"172.21.15.137:6807/2885027256","heartbeat_front_addr":"172.21.15.137:6805/2885027256","state":["exists","up"]},{"osd":3,"uuid":"e05592ba-d96a-472a-9f63-09a535d04dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6811","nonce":3701747260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6813","nonce":3701747260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6817","nonce":3701747260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6815","nonce":3701747260}]},"public_addr":"172.21.15.5:6811/3701747260","cluster_addr":"172.21.15.5:6813/3701747260","heartbeat_back_addr":"172.21.15.5:6817/3701747260","heartbeat_front_addr":"172.21.15.5:6815/3701747260","state":["exists","up"]},{"osd":4,"uuid":"c3f36e51-7a3b-4dd9-af4b-a10558830c4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6809","nonce":759253844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6811","nonce":759253844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6815","nonce":759253844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6813","nonce":759253844}]},"public_addr":"172.21.15.37:6809/759253844","cluster_addr":"172.21.15.37:6811/759253844","heartbeat_back_addr":"172.21.15.37:6815/759253844","heartbeat_front_addr":"172.21.15.37:6813/759253844","state":["exists","up"]},{"osd":5,"uuid":"97a0500c-cbb8-4ff5-b476-8634b26b3821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6809","nonce":3771921560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6811","nonce":3771921560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6815","nonce":3771921560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6813","nonce":3771921560}]},"public_addr":"172.21.15.137:6809/3771921560","cluster_addr":"172.21.15.137:6811/3771921560","heartbeat_back_addr":"172.21.15.137:6815/3771921560","heartbeat_front_addr":"172.21.15.137:6813/3771921560","state":["exists","up"]},{"osd":6,"uuid":"33a33d62-bbb3-4ed9-834e-19ae93a97670","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6819","nonce":2064636084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6821","nonce":2064636084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6825","nonce":2064636084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6823","nonce":2064636084}]},"public_addr":"172.21.15.5:6819/2064636084","cluster_addr":"172.21.15.5:6821/2064636084","heartbeat_back_addr":"172.21.15.5:6825/2064636084","heartbeat_front_addr":"172.21.15.5:6823/2064636084","state":["exists","up"]},{"osd":7,"uuid":"c0f4e036-4717-4514-bb59-ab11fbaab063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6817","nonce":2952750659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6819","nonce":2952750659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6823","nonce":2952750659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6821","nonce":2952750659}]},"public_addr":"172.21.15.37:6817/2952750659","cluster_addr":"172.21.15.37:6819/2952750659","heartbeat_back_addr":"172.21.15.37:6823/2952750659","heartbeat_front_addr":"172.21.15.37:6821/2952750659","state":["exists","up"]},{"osd":8,"uuid":"5391fd87-44ea-440f-80da-c54d0327c04c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6817","nonce":29724945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6819","nonce":29724945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6823","nonce":29724945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6821","nonce":29724945}]},"public_addr":"172.21.15.137:6817/29724945","cluster_addr":"172.21.15.137:6819/29724945","heartbeat_back_addr":"172.21.15.137:6823/29724945","heartbeat_front_addr":"172.21.15.137:6821/29724945","state":["exists","up"]},{"osd":9,"uuid":"d94c9dbc-f43d-4237-ac7f-47a6a5ede1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6827","nonce":2257102107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6829","nonce":2257102107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6833","nonce":2257102107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6831","nonce":2257102107}]},"public_addr":"172.21.15.5:6827/2257102107","cluster_addr":"172.21.15.5:6829/2257102107","heartbeat_back_addr":"172.21.15.5:6833/2257102107","heartbeat_front_addr":"172.21.15.5:6831/2257102107","state":["exists","up"]},{"osd":10,"uuid":"e2455332-2a5e-44b1-85d3-8e47da399f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6825","nonce":2451570637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6827","nonce":2451570637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6831","nonce":2451570637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6829","nonce":2451570637}]},"public_addr":"172.21.15.37:6825/2451570637","cluster_addr":"172.21.15.37:6827/2451570637","heartbeat_back_addr":"172.21.15.37:6831/2451570637","heartbeat_front_addr":"172.21.15.37:6829/2451570637","state":["exists","up"]},{"osd":11,"uuid":"49d9396b-704a-4cef-8210-75d30c28e924","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6825","nonce":90600596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6827","nonce":90600596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6831","nonce":90600596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6829","nonce":90600596}]},"public_addr":"172.21.15.137:6825/90600596","cluster_addr":"172.21.15.137:6827/90600596","heartbeat_back_addr":"172.21.15.137:6831/90600596","heartbeat_front_addr":"172.21.15.137:6829/90600596","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:35:35.591619+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:00.568817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:24.195643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:56.451724+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:22.109293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:46.161749+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:19.278305+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:45.620256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:10.041165+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:43.813697+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:10.736026+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:35.384377+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/23307164":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6801/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:0/230785941":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/2940023597":"2024-04-25T06:33:07.040369+0000","172.21.15.5:6801/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3886613904":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/4199165437":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3459593499":"2024-04-25T06:33:41.161852+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:43:17.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.090+0000 7f7734800640 1 -- 172.21.15.5:0/2773162493 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f77140a3c30 msgr2=0x7f77140a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:17.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.090+0000 7f7734800640 1 --2- 172.21.15.5:0/2773162493 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f77140a3c30 0x7f77140a60f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7720000c00 tx=0x7f77200315b0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.090+0000 7f7734800640 1 -- 172.21.15.5:0/2773162493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f772c1054b0 msgr2=0x7f772c19aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:17.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.090+0000 7f7734800640 1 --2- 172.21.15.5:0/2773162493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f772c1054b0 0x7f772c19aa80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f771c00e9f0 tx=0x7f771c00eeb0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.091+0000 7f7734800640 1 -- 172.21.15.5:0/2773162493 shutdown_connections 2024-04-24T06:43:17.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.091+0000 7f7734800640 1 --2- 172.21.15.5:0/2773162493 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f77140a3c30 0x7f77140a60f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.091+0000 7f7734800640 1 --2- 172.21.15.5:0/2773162493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f772c105e70 0x7f772c1a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.091+0000 7f7734800640 1 --2- 172.21.15.5:0/2773162493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f772c1054b0 0x7f772c19aa80 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.091+0000 7f7734800640 1 --2- 172.21.15.5:0/2773162493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f772c1042c0 0x7f772c19a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.091+0000 7f7734800640 1 -- 172.21.15.5:0/2773162493 >> 172.21.15.5:0/2773162493 conn(0x7f772c0ff850 msgr2=0x7f772c108830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:17.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.091+0000 7f7734800640 1 -- 172.21.15.5:0/2773162493 shutdown_connections 2024-04-24T06:43:17.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.091+0000 7f7734800640 1 -- 172.21.15.5:0/2773162493 wait complete. 2024-04-24T06:43:17.113 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:43:17.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.323+0000 7f5240a00640 1 -- 172.21.15.5:0/4193940268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f523c109740 msgr2=0x7f523c109bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:17.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.323+0000 7f5240a00640 1 --2- 172.21.15.5:0/4193940268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f523c109740 0x7f523c109bc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5228009900 tx=0x7f522802a3f0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.324+0000 7f5240a00640 1 -- 172.21.15.5:0/4193940268 shutdown_connections 2024-04-24T06:43:17.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.324+0000 7f5240a00640 1 --2- 172.21.15.5:0/4193940268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f523c10a100 0x7f523c110990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.324+0000 7f5240a00640 1 --2- 172.21.15.5:0/4193940268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f523c109740 0x7f523c109bc0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.324+0000 7f5240a00640 1 --2- 172.21.15.5:0/4193940268 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f523c1085a0 0x7f523c1089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.324+0000 7f5240a00640 1 -- 172.21.15.5:0/4193940268 >> 172.21.15.5:0/4193940268 conn(0x7f523c0774f0 msgr2=0x7f523c079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:17.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.324+0000 7f5240a00640 1 -- 172.21.15.5:0/4193940268 shutdown_connections 2024-04-24T06:43:17.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.324+0000 7f5240a00640 1 -- 172.21.15.5:0/4193940268 wait complete. 2024-04-24T06:43:17.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.325+0000 7f5240a00640 1 Processor -- start 2024-04-24T06:43:17.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.325+0000 7f5240a00640 1 -- start start 2024-04-24T06:43:17.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.325+0000 7f5240a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f523c1085a0 0x7f523c19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:17.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.325+0000 7f5240a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f523c109740 0x7f523c19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:17.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.325+0000 7f5240a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f523c10a100 0x7f523c1a5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:17.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.325+0000 7f5240a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f523c114230 con 0x7f523c109740 2024-04-24T06:43:17.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.325+0000 7f5240a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f523c1140b0 con 0x7f523c1085a0 2024-04-24T06:43:17.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.325+0000 7f5240a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f523c1143b0 con 0x7f523c10a100 2024-04-24T06:43:17.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.326+0000 7f523aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f523c109740 0x7f523c19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:17.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.326+0000 7f523aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f523c109740 0x7f523c19ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58664/0 (socket says 172.21.15.5:58664) 2024-04-24T06:43:17.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.326+0000 7f523aa00640 1 -- 172.21.15.5:0/289446811 learned_addr learned my addr 172.21.15.5:0/289446811 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:17.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.326+0000 7f523b400640 1 --2- 172.21.15.5:0/289446811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f523c1085a0 0x7f523c19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:17.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.326+0000 7f523be00640 1 --2- 172.21.15.5:0/289446811 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f523c10a100 0x7f523c1a5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:17.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.326+0000 7f523aa00640 1 -- 172.21.15.5:0/289446811 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f523c10a100 msgr2=0x7f523c1a5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:17.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.326+0000 7f523aa00640 1 --2- 172.21.15.5:0/289446811 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f523c10a100 0x7f523c1a5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.326+0000 7f523aa00640 1 -- 172.21.15.5:0/289446811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f523c1085a0 msgr2=0x7f523c19e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:17.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.326+0000 7f523aa00640 1 --2- 172.21.15.5:0/289446811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f523c1085a0 0x7f523c19e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.326+0000 7f523aa00640 1 -- 172.21.15.5:0/289446811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f523c1a6440 con 0x7f523c109740 2024-04-24T06:43:17.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.326+0000 7f523aa00640 1 --2- 172.21.15.5:0/289446811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f523c109740 0x7f523c19ed60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5228031440 tx=0x7f5228032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:17.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.327+0000 7f5233e00640 1 -- 172.21.15.5:0/289446811 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522803f030 con 0x7f523c109740 2024-04-24T06:43:17.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.327+0000 7f5233e00640 1 -- 172.21.15.5:0/289446811 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5228003ce0 con 0x7f523c109740 2024-04-24T06:43:17.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.327+0000 7f5240a00640 1 -- 172.21.15.5:0/289446811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f523c1a66d0 con 0x7f523c109740 2024-04-24T06:43:17.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.327+0000 7f5233e00640 1 -- 172.21.15.5:0/289446811 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5228035070 con 0x7f523c109740 2024-04-24T06:43:17.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.327+0000 7f5240a00640 1 -- 172.21.15.5:0/289446811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f523c1a6c10 con 0x7f523c109740 2024-04-24T06:43:17.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.328+0000 7f5240a00640 1 -- 172.21.15.5:0/289446811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f523c10f810 con 0x7f523c109740 2024-04-24T06:43:17.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.330+0000 7f5233e00640 1 -- 172.21.15.5:0/289446811 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f5228042030 con 0x7f523c109740 2024-04-24T06:43:17.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.331+0000 7f5233e00640 1 --2- 172.21.15.5:0/289446811 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f52200a3c20 0x7f52200a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:17.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.332+0000 7f5233e00640 1 -- 172.21.15.5:0/289446811 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f52280f05c0 con 0x7f523c109740 2024-04-24T06:43:17.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.332+0000 7f523b400640 1 --2- 172.21.15.5:0/289446811 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f52200a3c20 0x7f52200a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:17.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.332+0000 7f523b400640 1 --2- 172.21.15.5:0/289446811 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f52200a3c20 0x7f52200a60e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f523c1095a0 tx=0x7f522c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:17.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.336+0000 7f5233e00640 1 -- 172.21.15.5:0/289446811 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f52280bc910 con 0x7f523c109740 2024-04-24T06:43:17.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:17 smithi137 ceph-mon[33017]: pgmap v365: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T06:43:17.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:17 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/1414093749' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T06:43:17.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:17 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/2088335495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:43:17.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:17 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/2773162493' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:43:17.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:17 smithi005 ceph-mon[27798]: pgmap v365: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T06:43:17.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:17 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1414093749' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T06:43:17.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:17 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/2088335495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:43:17.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:17 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/2773162493' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:43:17.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:17 smithi037 ceph-mon[32864]: pgmap v365: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T06:43:17.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:17 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/1414093749' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T06:43:17.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:17 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/2088335495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:43:17.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:17 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/2773162493' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:43:17.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.574+0000 7f5240a00640 1 -- 172.21.15.5:0/289446811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f523c02d620 con 0x7f523c109740 2024-04-24T06:43:17.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.576+0000 7f5233e00640 1 -- 172.21.15.5:0/289446811 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11856 (secure 0 0 0) 0x7f52280c0c10 con 0x7f523c109740 2024-04-24T06:43:17.577 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:43:17.577 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1735462750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1735462750},{"type":"v1","addr":"172.21.15.5:6837","nonce":1735462750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1499469370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1499469370},{"type":"v1","addr":"172.21.15.5:6841","nonce":1499469370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3705638847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3705638847},{"type":"v1","addr":"172.21.15.37:6835","nonce":3705638847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T06:41:34.764490+0000","modified":"2024-04-24T06:41:40.330991+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24550,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6837/1390103296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1390103296},{"type":"v1","addr":"172.21.15.37:6837","nonce":1390103296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3217754215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3217754215},{"type":"v1","addr":"172.21.15.137:6835","nonce":3217754215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6837/3044222976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3044222976},{"type":"v1","addr":"172.21.15.137:6837","nonce":3044222976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.5:6839/3173201459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3173201459},{"type":"v1","addr":"172.21.15.5:6839","nonce":3173201459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6839/1889774232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1889774232},{"type":"v1","addr":"172.21.15.137:6839","nonce":1889774232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6839/3536721753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":3536721753},{"type":"v1","addr":"172.21.15.37:6839","nonce":3536721753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24539,24550,24569]},"id":1}]} 2024-04-24T06:43:17.578 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 22 2024-04-24T06:43:17.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.579+0000 7f5240a00640 1 -- 172.21.15.5:0/289446811 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f52200a3c20 msgr2=0x7f52200a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:17.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.579+0000 7f5240a00640 1 --2- 172.21.15.5:0/289446811 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f52200a3c20 0x7f52200a60e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f523c1095a0 tx=0x7f522c005f90 comp rx=0 tx=0).stop 2024-04-24T06:43:17.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.579+0000 7f5240a00640 1 -- 172.21.15.5:0/289446811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f523c109740 msgr2=0x7f523c19ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:17.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.579+0000 7f5240a00640 1 --2- 172.21.15.5:0/289446811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f523c109740 0x7f523c19ed60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5228031440 tx=0x7f5228032ce0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.579+0000 7f5240a00640 1 -- 172.21.15.5:0/289446811 shutdown_connections 2024-04-24T06:43:17.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.579+0000 7f5240a00640 1 --2- 172.21.15.5:0/289446811 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f52200a3c20 0x7f52200a60e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.579+0000 7f5240a00640 1 --2- 172.21.15.5:0/289446811 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f523c10a100 0x7f523c1a5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.579+0000 7f5240a00640 1 --2- 172.21.15.5:0/289446811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f523c109740 0x7f523c19ed60 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.579+0000 7f5240a00640 1 --2- 172.21.15.5:0/289446811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f523c1085a0 0x7f523c19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.579+0000 7f5240a00640 1 -- 172.21.15.5:0/289446811 >> 172.21.15.5:0/289446811 conn(0x7f523c0774f0 msgr2=0x7f523c10c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:17.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.579+0000 7f5240a00640 1 -- 172.21.15.5:0/289446811 shutdown_connections 2024-04-24T06:43:17.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.579+0000 7f5240a00640 1 -- 172.21.15.5:0/289446811 wait complete. 2024-04-24T06:43:17.600 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-24T06:41:34.764490+0000', 'modified': '2024-04-24T06:41:40.330991+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24569, 'mds_1': 24550, 'mds_2': 24539}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24515': {'gid': 24515, 'name': 'h', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.37:6837/1390103296', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6836', 'nonce': 1390103296}, {'type': 'v1', 'addr': '172.21.15.37:6837', 'nonce': 1390103296}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24520': {'gid': 24520, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.137:6835/3217754215', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6834', 'nonce': 3217754215}, {'type': 'v1', 'addr': '172.21.15.137:6835', 'nonce': 3217754215}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24532': {'gid': 24532, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.137:6837/3044222976', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6836', 'nonce': 3044222976}, {'type': 'v1', 'addr': '172.21.15.137:6837', 'nonce': 3044222976}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24539': {'gid': 24539, 'name': 'g', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.5:6839/3173201459', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6838', 'nonce': 3173201459}, {'type': 'v1', 'addr': '172.21.15.5:6839', 'nonce': 3173201459}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24550': {'gid': 24550, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.137:6839/1889774232', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6838', 'nonce': 1889774232}, {'type': 'v1', 'addr': '172.21.15.137:6839', 'nonce': 1889774232}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24569': {'gid': 24569, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.37:6839/3536721753', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6838', 'nonce': 3536721753}, {'type': 'v1', 'addr': '172.21.15.37:6839', 'nonce': 3536721753}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24569, 'qdb_cluster': [24539, 24550, 24569]} 2024-04-24T06:43:17.600 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-24T06:43:17.600 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2024-04-24T06:43:17.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.809+0000 7f2c06600640 1 -- 172.21.15.5:0/4211970944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c0010a210 msgr2=0x7f2c00110aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:17.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.809+0000 7f2c06600640 1 --2- 172.21.15.5:0/4211970944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c0010a210 0x7f2c00110aa0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8009ca0 tx=0x7f2bf802a8c0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.809+0000 7f2c06600640 1 -- 172.21.15.5:0/4211970944 shutdown_connections 2024-04-24T06:43:17.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.809+0000 7f2c06600640 1 --2- 172.21.15.5:0/4211970944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c0010a210 0x7f2c00110aa0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.809+0000 7f2c06600640 1 --2- 172.21.15.5:0/4211970944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c00109850 0x7f2c00109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.809+0000 7f2c06600640 1 --2- 172.21.15.5:0/4211970944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c00108660 0x7f2c00108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.809+0000 7f2c06600640 1 -- 172.21.15.5:0/4211970944 >> 172.21.15.5:0/4211970944 conn(0x7f2c00077610 msgr2=0x7f2c00079a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:17.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.809+0000 7f2c06600640 1 -- 172.21.15.5:0/4211970944 shutdown_connections 2024-04-24T06:43:17.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.809+0000 7f2c06600640 1 -- 172.21.15.5:0/4211970944 wait complete. 2024-04-24T06:43:17.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.810+0000 7f2c06600640 1 Processor -- start 2024-04-24T06:43:17.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.810+0000 7f2c06600640 1 -- start start 2024-04-24T06:43:17.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.810+0000 7f2c06600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c00108660 0x7f2c0019e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:17.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.810+0000 7f2c06600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c00109850 0x7f2c0019ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:17.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.810+0000 7f2c06600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c0010a210 0x7f2c001a5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:17.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.810+0000 7f2c06600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2c00114310 con 0x7f2c0010a210 2024-04-24T06:43:17.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.810+0000 7f2c06600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2c00114190 con 0x7f2c00109850 2024-04-24T06:43:17.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.810+0000 7f2c06600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2c00114490 con 0x7f2c00108660 2024-04-24T06:43:17.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.811+0000 7f2bff400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c00108660 0x7f2c0019e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:17.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.811+0000 7f2bffe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c0010a210 0x7f2c001a5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:17.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.811+0000 7f2bff400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c00108660 0x7f2c0019e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:44632/0 (socket says 172.21.15.5:44632) 2024-04-24T06:43:17.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.811+0000 7f2bff400640 1 -- 172.21.15.5:0/3225519711 learned_addr learned my addr 172.21.15.5:0/3225519711 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:17.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.811+0000 7f2bfea00640 1 --2- 172.21.15.5:0/3225519711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c00109850 0x7f2c0019ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:17.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.811+0000 7f2bffe00640 1 -- 172.21.15.5:0/3225519711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c00108660 msgr2=0x7f2c0019e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:17.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.811+0000 7f2bffe00640 1 --2- 172.21.15.5:0/3225519711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c00108660 0x7f2c0019e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.811+0000 7f2bffe00640 1 -- 172.21.15.5:0/3225519711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c00109850 msgr2=0x7f2c0019ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:17.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.811+0000 7f2bffe00640 1 --2- 172.21.15.5:0/3225519711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c00109850 0x7f2c0019ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.811+0000 7f2bffe00640 1 -- 172.21.15.5:0/3225519711 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c001a6550 con 0x7f2c0010a210 2024-04-24T06:43:17.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.811+0000 7f2bff400640 1 --2- 172.21.15.5:0/3225519711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c00108660 0x7f2c0019e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:43:17.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.811+0000 7f2bffe00640 1 --2- 172.21.15.5:0/3225519711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c0010a210 0x7f2c001a5ee0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8031ae0 tx=0x7f2bf8031750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:17.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.812+0000 7f2bf7e00640 1 -- 172.21.15.5:0/3225519711 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8003b70 con 0x7f2c0010a210 2024-04-24T06:43:17.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.812+0000 7f2bf7e00640 1 -- 172.21.15.5:0/3225519711 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bf8003d10 con 0x7f2c0010a210 2024-04-24T06:43:17.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.812+0000 7f2bf7e00640 1 -- 172.21.15.5:0/3225519711 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf802fb10 con 0x7f2c0010a210 2024-04-24T06:43:17.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.812+0000 7f2c06600640 1 -- 172.21.15.5:0/3225519711 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c001a67e0 con 0x7f2c0010a210 2024-04-24T06:43:17.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.812+0000 7f2c06600640 1 -- 172.21.15.5:0/3225519711 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c001a6d80 con 0x7f2c0010a210 2024-04-24T06:43:17.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.815+0000 7f2bf7e00640 1 -- 172.21.15.5:0/3225519711 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f2bf8004640 con 0x7f2c0010a210 2024-04-24T06:43:17.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.815+0000 7f2c09414740 1 Processor -- start 2024-04-24T06:43:17.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.815+0000 7f2c09414740 1 -- start start 2024-04-24T06:43:17.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.815+0000 7f2c09414740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b63ab9a090 0x55b63ab9a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:17.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.815+0000 7f2c09414740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b63ab9aa80 0x55b63ab9f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:17.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.816+0000 7f2c09414740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b63ab9fbf0 0x55b63aba2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:17.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.816+0000 7f2c09414740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b63aaffb10 con 0x55b63ab9a090 2024-04-24T06:43:17.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.816+0000 7f2c09414740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b63aaa62b0 con 0x55b63ab9aa80 2024-04-24T06:43:17.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.816+0000 7f2c09414740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b63aafbad0 con 0x55b63ab9fbf0 2024-04-24T06:43:17.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.816+0000 7f2bfea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b63ab9a090 0x55b63ab9a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:17.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.816+0000 7f2bff400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b63ab9aa80 0x55b63ab9f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:17.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.816+0000 7f2bfea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b63ab9a090 0x55b63ab9a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58686/0 (socket says 172.21.15.5:58686) 2024-04-24T06:43:17.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.816+0000 7f2bfea00640 1 -- 172.21.15.5:0/515112261 learned_addr learned my addr 172.21.15.5:0/515112261 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:17.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.816+0000 7f2bfea00640 1 --2- 172.21.15.5:0/515112261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b63ab9fbf0 0x55b63aba2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:17.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.816+0000 7f2bff400640 1 -- 172.21.15.5:0/515112261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b63ab9fbf0 msgr2=0x55b63aba2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:17.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.816+0000 7f2bff400640 1 --2- 172.21.15.5:0/515112261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b63ab9fbf0 0x55b63aba2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.816+0000 7f2bff400640 1 -- 172.21.15.5:0/515112261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b63ab9a090 msgr2=0x55b63ab9a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:17.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.816+0000 7f2bff400640 1 --2- 172.21.15.5:0/515112261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b63ab9a090 0x55b63ab9a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.816+0000 7f2bff400640 1 -- 172.21.15.5:0/515112261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b63a8122d0 con 0x55b63ab9aa80 2024-04-24T06:43:17.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.816+0000 7f2bfea00640 1 --2- 172.21.15.5:0/515112261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b63ab9fbf0 0x55b63aba2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:43:17.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.817+0000 7f2bff400640 1 --2- 172.21.15.5:0/515112261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b63ab9aa80 0x55b63ab9f580 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2bf00114c0 tx=0x7f2bf0011980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcd9384e3e9d4118 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:17.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.817+0000 7f2bf7e00640 1 --2- 172.21.15.5:0/3225519711 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f2be40a3c30 0x7f2be40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:17.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.817+0000 7f2bf4c00640 1 -- 172.21.15.5:0/515112261 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0005020 con 0x55b63ab9aa80 2024-04-24T06:43:17.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.817+0000 7f2bf7e00640 1 -- 172.21.15.5:0/3225519711 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f2bf80f1710 con 0x7f2c0010a210 2024-04-24T06:43:17.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.817+0000 7f2bf4c00640 1 -- 172.21.15.5:0/515112261 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bf00047a0 con 0x55b63ab9aa80 2024-04-24T06:43:17.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.817+0000 7f2bf4c00640 1 -- 172.21.15.5:0/515112261 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf000f8d0 con 0x55b63ab9aa80 2024-04-24T06:43:17.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.817+0000 7f2bfea00640 1 --2- 172.21.15.5:0/3225519711 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f2be40a3c30 0x7f2be40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:17.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.818+0000 7f2c09414740 1 -- 172.21.15.5:0/515112261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b63ab9aa80 msgr2=0x55b63ab9f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:17.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.818+0000 7f2bfea00640 1 --2- 172.21.15.5:0/3225519711 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f2be40a3c30 0x7f2be40a60f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x55b63ab9b2e0 tx=0x7f2bec015040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:17.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.818+0000 7f2c09414740 1 --2- 172.21.15.5:0/515112261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b63ab9aa80 0x55b63ab9f580 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2bf00114c0 tx=0x7f2bf0011980 comp rx=0 tx=0).stop 2024-04-24T06:43:17.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.818+0000 7f2c09414740 1 -- 172.21.15.5:0/515112261 shutdown_connections 2024-04-24T06:43:17.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.818+0000 7f2c09414740 1 --2- 172.21.15.5:0/515112261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b63ab9fbf0 0x55b63aba2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.818+0000 7f2c09414740 1 --2- 172.21.15.5:0/515112261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b63ab9aa80 0x55b63ab9f580 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.818+0000 7f2c09414740 1 --2- 172.21.15.5:0/515112261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b63ab9a090 0x55b63ab9a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.818+0000 7f2c09414740 1 -- 172.21.15.5:0/515112261 >> 172.21.15.5:0/515112261 conn(0x55b63ab95700 msgr2=0x55b63ab95b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:17.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.818+0000 7f2c09414740 1 -- 172.21.15.5:0/515112261 shutdown_connections 2024-04-24T06:43:17.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.818+0000 7f2c09414740 1 -- 172.21.15.5:0/515112261 wait complete. 2024-04-24T06:43:17.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.818+0000 7f2c09414740 1 -- start start 2024-04-24T06:43:17.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.819+0000 7f2c09414740 1 Processor -- start 2024-04-24T06:43:17.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.819+0000 7f2c09414740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b63ab9a090 0x55b63aba4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:17.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.819+0000 7f2c09414740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b63ab9aa80 0x55b63aba49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:17.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.819+0000 7f2c09414740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b63ab9fbf0 0x55b63ababa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:17.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.819+0000 7f2c09414740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b63aba2ba0 con 0x55b63ab9aa80 2024-04-24T06:43:17.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.819+0000 7f2c09414740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b63aab9590 con 0x55b63ab9a090 2024-04-24T06:43:17.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.819+0000 7f2c09414740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b63aba2d20 con 0x55b63ab9fbf0 2024-04-24T06:43:17.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.819+0000 7f2c09414740 10 client.0 _ll_register_callbacks cb 0x55b63aa9c3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:43:17.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.819+0000 7f2bfea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b63ab9aa80 0x55b63aba49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:17.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.819+0000 7f2bfea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b63ab9aa80 0x55b63aba49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58690/0 (socket says 172.21.15.5:58690) 2024-04-24T06:43:17.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.819+0000 7f2bfea00640 1 -- 172.21.15.5:0/871631658 learned_addr learned my addr 172.21.15.5:0/871631658 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:17.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.820+0000 7f2bffe00640 1 --2- 172.21.15.5:0/871631658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b63ab9fbf0 0x55b63ababa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:17.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.820+0000 7f2bff400640 1 --2- 172.21.15.5:0/871631658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b63ab9a090 0x55b63aba4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:17.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.820+0000 7f2bfea00640 1 -- 172.21.15.5:0/871631658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b63ab9fbf0 msgr2=0x55b63ababa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:17.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.820+0000 7f2bfea00640 1 --2- 172.21.15.5:0/871631658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b63ab9fbf0 0x55b63ababa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.820+0000 7f2bfea00640 1 -- 172.21.15.5:0/871631658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b63ab9a090 msgr2=0x55b63aba4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:17.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.820+0000 7f2bfea00640 1 --2- 172.21.15.5:0/871631658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b63ab9a090 0x55b63aba4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.820+0000 7f2bfea00640 1 -- 172.21.15.5:0/871631658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b63a8dc6b0 con 0x55b63ab9aa80 2024-04-24T06:43:17.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.820+0000 7f2bff400640 1 --2- 172.21.15.5:0/871631658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b63ab9a090 0x55b63aba4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:43:17.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.820+0000 7f2bfea00640 1 --2- 172.21.15.5:0/871631658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b63ab9aa80 0x55b63aba49d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2bec005b90 tx=0x7f2bec005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f5b59c9af5c6f56 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:17.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.820+0000 7f2be9600640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:43:17.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.821+0000 7f2be9600640 1 -- 172.21.15.5:0/871631658 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec00f750 con 0x55b63ab9aa80 2024-04-24T06:43:17.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.821+0000 7f2be9600640 1 -- 172.21.15.5:0/871631658 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f2bec00f8f0 con 0x55b63ab9aa80 2024-04-24T06:43:17.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.821+0000 7f2c09414740 1 -- 172.21.15.5:0/871631658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bf00047a0 con 0x55b63ab9aa80 2024-04-24T06:43:17.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.821+0000 7f2be9600640 1 -- 172.21.15.5:0/871631658 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f2bec00faf0 con 0x55b63ab9aa80 2024-04-24T06:43:17.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.821+0000 7f2c09414740 10 client.14883 fetch_fsmap learned FSMap version 22 2024-04-24T06:43:17.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.821+0000 7f2c09414740 1 -- 172.21.15.5:0/871631658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55b63abaee40 con 0x55b63ab9aa80 2024-04-24T06:43:17.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.821+0000 7f2be9600640 1 -- 172.21.15.5:0/871631658 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bf00047a0 con 0x55b63ab9aa80 2024-04-24T06:43:17.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.821+0000 7f2be9600640 1 -- 172.21.15.5:0/871631658 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f2bec01f070 con 0x55b63ab9aa80 2024-04-24T06:43:17.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.821+0000 7f2c09414740 10 client.14883 fetch_fsmap finished waiting for FSMap version 22 2024-04-24T06:43:17.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.822+0000 7f2c09414740 10 client.14883 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-24T06:43:17.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.822+0000 7f2c09414740 20 client.14883 populate_metadata read hostname 'smithi005' 2024-04-24T06:43:17.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.822+0000 7f2c09414740 1 --2- 172.21.15.5:0/871631658 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55b63abb15a0 0x55b63abb2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:17.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.822+0000 7f2c09414740 4 client.14883 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:43:17.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.822+0000 7f2c09414740 1 -- 172.21.15.5:0/871631658 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b63ab0d530 con 0x55b63abb15a0 2024-04-24T06:43:17.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.822+0000 7f2bff400640 1 --2- 172.21.15.5:0/871631658 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55b63abb15a0 0x55b63abb2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:17.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.822+0000 7f2bff400640 1 --2- 172.21.15.5:0/871631658 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55b63abb15a0 0x55b63abb2a10 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=47a393bd69e3fd5c server_cookie=b64d009b199445fd in_seq=0 out_seq=0 2024-04-24T06:43:17.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.822+0000 7f2be9600640 10 client.14883 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:43:17.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.824+0000 7f2be9600640 1 -- 172.21.15.5:0/871631658 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b63ab0d530 con 0x55b63abb15a0 2024-04-24T06:43:17.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.824+0000 7f2be9600640 10 client.14883 handle_command_reply: tid=0 2024-04-24T06:43:17.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.824+0000 7f2c09414740 1 client.14883 shutdown 2024-04-24T06:43:17.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.824+0000 7f2c09414740 1 -- 172.21.15.5:0/871631658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b63ab9aa80 msgr2=0x55b63aba49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:17.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.824+0000 7f2c09414740 1 --2- 172.21.15.5:0/871631658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b63ab9aa80 0x55b63aba49d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2bec005b90 tx=0x7f2bec005f90 comp rx=0 tx=0).stop 2024-04-24T06:43:17.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.824+0000 7f2c09414740 1 -- 172.21.15.5:0/871631658 shutdown_connections 2024-04-24T06:43:17.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.824+0000 7f2c09414740 1 --2- 172.21.15.5:0/871631658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b63ab9fbf0 0x55b63ababa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.824+0000 7f2c09414740 1 --2- 172.21.15.5:0/871631658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b63ab9aa80 0x55b63aba49d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.824+0000 7f2c09414740 1 --2- 172.21.15.5:0/871631658 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55b63abb15a0 0x55b63abb2a10 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.824+0000 7f2c09414740 1 --2- 172.21.15.5:0/871631658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b63ab9a090 0x55b63aba4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.824+0000 7f2be9600640 0 client.14883 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:43:17.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.824+0000 7f2c09414740 1 -- 172.21.15.5:0/871631658 >> 172.21.15.5:0/871631658 conn(0x55b63ab95700 msgr2=0x55b63ab948c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:17.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.824+0000 7f2c09414740 1 -- 172.21.15.5:0/871631658 shutdown_connections 2024-04-24T06:43:17.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.824+0000 7f2c09414740 1 -- 172.21.15.5:0/871631658 wait complete. 2024-04-24T06:43:17.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.824+0000 7f2c09414740 20 client.14883 trim_cache size 0 max 16384 2024-04-24T06:43:17.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.844+0000 7f2c09414740 1 Processor -- start 2024-04-24T06:43:17.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.844+0000 7f2c09414740 1 -- start start 2024-04-24T06:43:17.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.845+0000 7f2c09414740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b63ab9a090 0x55b63ab90750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:17.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.845+0000 7f2c09414740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b63ab9aa80 0x55b63abab9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:17.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.845+0000 7f2c09414740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b63ab9fbf0 0x55b63abb0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:17.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.845+0000 7f2c09414740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b63abb0d50 con 0x55b63ab9aa80 2024-04-24T06:43:17.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.845+0000 7f2c09414740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b63abb1420 con 0x55b63ab9fbf0 2024-04-24T06:43:17.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.845+0000 7f2c09414740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b63abb1af0 con 0x55b63ab9a090 2024-04-24T06:43:17.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.845+0000 7f2bfea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b63ab9aa80 0x55b63abab9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:17.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.845+0000 7f2bff400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b63ab9a090 0x55b63ab90750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:17.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.845+0000 7f2bff400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b63ab9a090 0x55b63ab90750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:44656/0 (socket says 172.21.15.5:44656) 2024-04-24T06:43:17.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.845+0000 7f2bff400640 1 -- 172.21.15.5:0/2625143697 learned_addr learned my addr 172.21.15.5:0/2625143697 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:17.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.845+0000 7f2bffe00640 1 --2- 172.21.15.5:0/2625143697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b63ab9fbf0 0x55b63abb0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:17.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.845+0000 7f2bff400640 1 -- 172.21.15.5:0/2625143697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b63ab9fbf0 msgr2=0x55b63abb0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:17.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.845+0000 7f2bff400640 1 --2- 172.21.15.5:0/2625143697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b63ab9fbf0 0x55b63abb0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.845+0000 7f2bff400640 1 -- 172.21.15.5:0/2625143697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b63ab9aa80 msgr2=0x55b63abab9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:17.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.845+0000 7f2bff400640 1 --2- 172.21.15.5:0/2625143697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b63ab9aa80 0x55b63abab9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.845+0000 7f2bff400640 1 -- 172.21.15.5:0/2625143697 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b63ab86e20 con 0x55b63ab9a090 2024-04-24T06:43:17.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.845+0000 7f2bfea00640 1 --2- 172.21.15.5:0/2625143697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b63ab9aa80 0x55b63abab9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:43:17.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.846+0000 7f2bff400640 1 --2- 172.21.15.5:0/2625143697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b63ab9a090 0x55b63ab90750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0011e80 tx=0x7f2bf0009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbc91f05b305d78c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:17.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.846+0000 7f2bebe00640 1 -- 172.21.15.5:0/2625143697 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf001d040 con 0x55b63ab9a090 2024-04-24T06:43:17.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.846+0000 7f2bebe00640 1 -- 172.21.15.5:0/2625143697 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bf0030070 con 0x55b63ab9a090 2024-04-24T06:43:17.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.846+0000 7f2bebe00640 1 -- 172.21.15.5:0/2625143697 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf00054f0 con 0x55b63ab9a090 2024-04-24T06:43:17.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.847+0000 7f2c09414740 1 -- 172.21.15.5:0/2625143697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b63ab9a090 msgr2=0x55b63ab90750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:17.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.847+0000 7f2c09414740 1 --2- 172.21.15.5:0/2625143697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b63ab9a090 0x55b63ab90750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0011e80 tx=0x7f2bf0009310 comp rx=0 tx=0).stop 2024-04-24T06:43:17.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.847+0000 7f2c09414740 1 -- 172.21.15.5:0/2625143697 shutdown_connections 2024-04-24T06:43:17.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.847+0000 7f2c09414740 1 --2- 172.21.15.5:0/2625143697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b63ab9fbf0 0x55b63abb0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.847+0000 7f2c09414740 1 --2- 172.21.15.5:0/2625143697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b63ab9aa80 0x55b63abab9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.847+0000 7f2c09414740 1 --2- 172.21.15.5:0/2625143697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b63ab9a090 0x55b63ab90750 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.847+0000 7f2c09414740 1 -- 172.21.15.5:0/2625143697 >> 172.21.15.5:0/2625143697 conn(0x55b63ab95700 msgr2=0x55b63ab95ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:17.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.847+0000 7f2c09414740 1 -- 172.21.15.5:0/2625143697 shutdown_connections 2024-04-24T06:43:17.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.847+0000 7f2c09414740 1 -- 172.21.15.5:0/2625143697 wait complete. 2024-04-24T06:43:17.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.847+0000 7f2c09414740 1 -- start start 2024-04-24T06:43:17.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.847+0000 7f2c09414740 1 Processor -- start 2024-04-24T06:43:17.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.848+0000 7f2c09414740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b63ab9a090 0x55b63ab9e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:17.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.848+0000 7f2c09414740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b63ab9aa80 0x55b63ab9e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:17.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.848+0000 7f2c09414740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b63ab9fbf0 0x55b63abaa100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:17.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.848+0000 7f2c09414740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b63abb2890 con 0x55b63ab9aa80 2024-04-24T06:43:17.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.848+0000 7f2c09414740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b63abb21c0 con 0x55b63ab9fbf0 2024-04-24T06:43:17.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.848+0000 7f2c09414740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b63abb2f60 con 0x55b63ab9a090 2024-04-24T06:43:17.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.848+0000 7f2c09414740 10 client.0 _ll_register_callbacks cb 0x7f2bec004410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:43:17.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.848+0000 7f2bfea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b63ab9aa80 0x55b63ab9e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:17.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.848+0000 7f2bfea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b63ab9aa80 0x55b63ab9e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58716/0 (socket says 172.21.15.5:58716) 2024-04-24T06:43:17.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.848+0000 7f2bfea00640 1 -- 172.21.15.5:0/3057785778 learned_addr learned my addr 172.21.15.5:0/3057785778 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:17.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.848+0000 7f2bff400640 1 --2- 172.21.15.5:0/3057785778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b63ab9a090 0x55b63ab9e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:17.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.848+0000 7f2bffe00640 1 --2- 172.21.15.5:0/3057785778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b63ab9fbf0 0x55b63abaa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:17.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.849+0000 7f2bfea00640 1 -- 172.21.15.5:0/3057785778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b63ab9a090 msgr2=0x55b63ab9e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:17.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.849+0000 7f2bfea00640 1 --2- 172.21.15.5:0/3057785778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b63ab9a090 0x55b63ab9e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.849+0000 7f2bfea00640 1 -- 172.21.15.5:0/3057785778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b63ab9fbf0 msgr2=0x55b63abaa100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:17.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.849+0000 7f2bfea00640 1 --2- 172.21.15.5:0/3057785778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b63ab9fbf0 0x55b63abaa100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.849+0000 7f2bfea00640 1 -- 172.21.15.5:0/3057785778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b63abaa740 con 0x55b63ab9aa80 2024-04-24T06:43:17.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.849+0000 7f2bfea00640 1 --2- 172.21.15.5:0/3057785778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b63ab9aa80 0x55b63ab9e540 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2bec005bc0 tx=0x7f2bec003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75de0582af02aa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.849+0000 7f2be9600640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:43:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.849+0000 7f2be9600640 1 -- 172.21.15.5:0/3057785778 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec03a020 con 0x55b63ab9aa80 2024-04-24T06:43:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.849+0000 7f2be9600640 1 -- 172.21.15.5:0/3057785778 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f2bec0219e0 con 0x55b63ab9aa80 2024-04-24T06:43:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.849+0000 7f2c09414740 1 -- 172.21.15.5:0/3057785778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bf0030070 con 0x55b63ab9aa80 2024-04-24T06:43:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.850+0000 7f2be9600640 1 -- 172.21.15.5:0/3057785778 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f2bec021be0 con 0x55b63ab9aa80 2024-04-24T06:43:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.850+0000 7f2c09414740 10 client.14889 fetch_fsmap learned FSMap version 22 2024-04-24T06:43:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.850+0000 7f2c09414740 1 -- 172.21.15.5:0/3057785778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55b63ababa70 con 0x55b63ab9aa80 2024-04-24T06:43:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.850+0000 7f2be9600640 1 -- 172.21.15.5:0/3057785778 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bf0030070 con 0x55b63ab9aa80 2024-04-24T06:43:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.850+0000 7f2be9600640 1 -- 172.21.15.5:0/3057785778 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f2bec01f050 con 0x55b63ab9aa80 2024-04-24T06:43:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.850+0000 7f2c09414740 10 client.14889 fetch_fsmap finished waiting for FSMap version 22 2024-04-24T06:43:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.850+0000 7f2c09414740 10 client.14889 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-24T06:43:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.850+0000 7f2c09414740 20 client.14889 populate_metadata read hostname 'smithi005' 2024-04-24T06:43:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.850+0000 7f2c09414740 1 --2- 172.21.15.5:0/3057785778 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55b63ac1bc60 0x55b63ac180a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.850+0000 7f2c09414740 4 client.14889 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T06:43:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.850+0000 7f2c09414740 1 -- 172.21.15.5:0/3057785778 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "status"}) -- 0x55b63ab073a0 con 0x55b63ac1bc60 2024-04-24T06:43:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.851+0000 7f2bff400640 1 --2- 172.21.15.5:0/3057785778 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55b63ac1bc60 0x55b63ac180a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.851+0000 7f2bff400640 1 --2- 172.21.15.5:0/3057785778 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55b63ac1bc60 0x55b63ac180a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b4e8d0fa6e3f5eab server_cookie=9f18b37da0ffcb29 in_seq=0 out_seq=0 2024-04-24T06:43:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.851+0000 7f2be9600640 10 client.14889 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:43:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.851+0000 7f2be9600640 1 -- 172.21.15.5:0/3057785778 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55b63ab073a0 con 0x55b63ac1bc60 2024-04-24T06:43:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.851+0000 7f2be9600640 10 client.14889 handle_command_reply: tid=0 2024-04-24T06:43:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.851+0000 7f2c09414740 1 client.14889 shutdown 2024-04-24T06:43:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.852+0000 7f2c09414740 1 -- 172.21.15.5:0/3057785778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b63ab9aa80 msgr2=0x55b63ab9e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.852+0000 7f2c09414740 1 --2- 172.21.15.5:0/3057785778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b63ab9aa80 0x55b63ab9e540 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2bec005bc0 tx=0x7f2bec003f90 comp rx=0 tx=0).stop 2024-04-24T06:43:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.852+0000 7f2c09414740 1 -- 172.21.15.5:0/3057785778 shutdown_connections 2024-04-24T06:43:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.852+0000 7f2c09414740 1 --2- 172.21.15.5:0/3057785778 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55b63ac1bc60 0x55b63ac180a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.852+0000 7f2c09414740 1 --2- 172.21.15.5:0/3057785778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b63ab9fbf0 0x55b63abaa100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.852+0000 7f2c09414740 1 --2- 172.21.15.5:0/3057785778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b63ab9aa80 0x55b63ab9e540 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.852+0000 7f2c09414740 1 --2- 172.21.15.5:0/3057785778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b63ab9a090 0x55b63ab9e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.852+0000 7f2be9600640 0 client.14889 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:43:17.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.852+0000 7f2c09414740 1 -- 172.21.15.5:0/3057785778 >> 172.21.15.5:0/3057785778 conn(0x55b63ab95700 msgr2=0x55b63ab90750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:17.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.852+0000 7f2c09414740 1 -- 172.21.15.5:0/3057785778 shutdown_connections 2024-04-24T06:43:17.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.852+0000 7f2c09414740 1 -- 172.21.15.5:0/3057785778 wait complete. 2024-04-24T06:43:17.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.852+0000 7f2c09414740 20 client.14889 trim_cache size 0 max 16384 2024-04-24T06:43:17.857 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:43:17.857 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "43e9bad0-0204-11ef-bc93-c7b262605968", 2024-04-24T06:43:17.857 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 2, 2024-04-24T06:43:17.857 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24539, 2024-04-24T06:43:17.857 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-04-24T06:43:17.857 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-04-24T06:43:17.857 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-04-24T06:43:17.857 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 98.531514537000007, 2024-04-24T06:43:17.858 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 22, 2024-04-24T06:43:17.858 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 88, 2024-04-24T06:43:17.858 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 86, 2024-04-24T06:43:17.858 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 117.068728773 2024-04-24T06:43:17.858 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:43:17.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.853+0000 7f2c06600640 1 -- 172.21.15.5:0/3225519711 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f2be40a3c30 msgr2=0x7f2be40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:17.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.853+0000 7f2c06600640 1 --2- 172.21.15.5:0/3225519711 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f2be40a3c30 0x7f2be40a60f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x55b63ab9b2e0 tx=0x7f2bec015040 comp rx=0 tx=0).stop 2024-04-24T06:43:17.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.853+0000 7f2c06600640 1 -- 172.21.15.5:0/3225519711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c0010a210 msgr2=0x7f2c001a5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:17.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.853+0000 7f2c06600640 1 --2- 172.21.15.5:0/3225519711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c0010a210 0x7f2c001a5ee0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8031ae0 tx=0x7f2bf8031750 comp rx=0 tx=0).stop 2024-04-24T06:43:17.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.853+0000 7f2c06600640 1 -- 172.21.15.5:0/3225519711 shutdown_connections 2024-04-24T06:43:17.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.853+0000 7f2c06600640 1 --2- 172.21.15.5:0/3225519711 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f2be40a3c30 0x7f2be40a60f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.853+0000 7f2c06600640 1 --2- 172.21.15.5:0/3225519711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c0010a210 0x7f2c001a5ee0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.853+0000 7f2c06600640 1 --2- 172.21.15.5:0/3225519711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c00109850 0x7f2c0019ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.853+0000 7f2c06600640 1 --2- 172.21.15.5:0/3225519711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c00108660 0x7f2c0019e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:17.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.853+0000 7f2c06600640 1 -- 172.21.15.5:0/3225519711 >> 172.21.15.5:0/3225519711 conn(0x7f2c00077610 msgr2=0x7f2c0010cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:17.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.853+0000 7f2c06600640 1 -- 172.21.15.5:0/3225519711 shutdown_connections 2024-04-24T06:43:17.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:17.853+0000 7f2c06600640 1 -- 172.21.15.5:0/3225519711 wait complete. 2024-04-24T06:43:17.873 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-24T06:43:18.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.086+0000 7f0467200640 1 -- 172.21.15.5:0/2012952678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f046806a020 msgr2=0x7f0468100a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:18.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.086+0000 7f0467200640 1 --2- 172.21.15.5:0/2012952678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f046806a020 0x7f0468100a60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0450009900 tx=0x7f045002a3f0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.086+0000 7f0467200640 1 -- 172.21.15.5:0/2012952678 shutdown_connections 2024-04-24T06:43:18.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.086+0000 7f0467200640 1 --2- 172.21.15.5:0/2012952678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04681016f0 0x7f0468101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.086+0000 7f0467200640 1 --2- 172.21.15.5:0/2012952678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f046806a020 0x7f0468100a60 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.086+0000 7f0467200640 1 --2- 172.21.15.5:0/2012952678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0468069b70 0x7f0468100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.086+0000 7f0467200640 1 -- 172.21.15.5:0/2012952678 >> 172.21.15.5:0/2012952678 conn(0x7f04680fba70 msgr2=0x7f04680fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:18.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.086+0000 7f0467200640 1 -- 172.21.15.5:0/2012952678 shutdown_connections 2024-04-24T06:43:18.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.086+0000 7f0467200640 1 -- 172.21.15.5:0/2012952678 wait complete. 2024-04-24T06:43:18.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.087+0000 7f0467200640 1 Processor -- start 2024-04-24T06:43:18.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.087+0000 7f0467200640 1 -- start start 2024-04-24T06:43:18.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.088+0000 7f0467200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0468069b70 0x7f04681055a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:18.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.088+0000 7f0467200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f046806a020 0x7f0468103be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:18.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.088+0000 7f0467200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04681016f0 0x7f0468104120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:18.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.088+0000 7f0467200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f046810c9d0 con 0x7f04681016f0 2024-04-24T06:43:18.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.088+0000 7f0467200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f046810c850 con 0x7f0468069b70 2024-04-24T06:43:18.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.088+0000 7f0467200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f046810cb50 con 0x7f046806a020 2024-04-24T06:43:18.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.088+0000 7f0465e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0468069b70 0x7f04681055a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:18.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.088+0000 7f0465e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0468069b70 0x7f04681055a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:53124/0 (socket says 172.21.15.5:53124) 2024-04-24T06:43:18.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.088+0000 7f045d400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f046806a020 0x7f0468103be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:18.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.088+0000 7f0465e00640 1 -- 172.21.15.5:0/2721917814 learned_addr learned my addr 172.21.15.5:0/2721917814 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:18.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.088+0000 7f0466800640 1 --2- 172.21.15.5:0/2721917814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04681016f0 0x7f0468104120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:18.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.088+0000 7f0465e00640 1 -- 172.21.15.5:0/2721917814 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f046806a020 msgr2=0x7f0468103be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:18.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.088+0000 7f0465e00640 1 --2- 172.21.15.5:0/2721917814 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f046806a020 0x7f0468103be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.088+0000 7f0465e00640 1 -- 172.21.15.5:0/2721917814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04681016f0 msgr2=0x7f0468104120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:18.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.088+0000 7f0465e00640 1 --2- 172.21.15.5:0/2721917814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04681016f0 0x7f0468104120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.088+0000 7f0465e00640 1 -- 172.21.15.5:0/2721917814 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0468104660 con 0x7f0468069b70 2024-04-24T06:43:18.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.089+0000 7f0465e00640 1 --2- 172.21.15.5:0/2721917814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0468069b70 0x7f04681055a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f045800e9c0 tx=0x7f045800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:18.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.089+0000 7f045f400640 1 -- 172.21.15.5:0/2721917814 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045800cde0 con 0x7f0468069b70 2024-04-24T06:43:18.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.089+0000 7f0467200640 1 -- 172.21.15.5:0/2721917814 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0468104950 con 0x7f0468069b70 2024-04-24T06:43:18.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.089+0000 7f045f400640 1 -- 172.21.15.5:0/2721917814 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0458010910 con 0x7f0468069b70 2024-04-24T06:43:18.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.089+0000 7f045f400640 1 -- 172.21.15.5:0/2721917814 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0458018a00 con 0x7f0468069b70 2024-04-24T06:43:18.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.089+0000 7f0467200640 1 -- 172.21.15.5:0/2721917814 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04681a6c40 con 0x7f0468069b70 2024-04-24T06:43:18.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.092+0000 7f046f82d740 1 Processor -- start 2024-04-24T06:43:18.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.092+0000 7f046f82d740 1 -- start start 2024-04-24T06:43:18.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.092+0000 7f045f400640 1 -- 172.21.15.5:0/2721917814 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f045801f020 con 0x7f0468069b70 2024-04-24T06:43:18.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.093+0000 7f046f82d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab72555260 0x55ab72555680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:18.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.093+0000 7f046f82d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab72555c50 0x55ab7255a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:18.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.093+0000 7f046f82d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab7255adc0 0x55ab7255d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:18.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.093+0000 7f046f82d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ab724bab10 con 0x55ab72555260 2024-04-24T06:43:18.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.093+0000 7f046f82d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ab724612b0 con 0x55ab7255adc0 2024-04-24T06:43:18.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.093+0000 7f046f82d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ab724b6ad0 con 0x55ab72555c50 2024-04-24T06:43:18.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.093+0000 7f0466800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab72555260 0x55ab72555680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:18.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.093+0000 7f0466800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab72555260 0x55ab72555680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58752/0 (socket says 172.21.15.5:58752) 2024-04-24T06:43:18.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.093+0000 7f0465e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab72555c50 0x55ab7255a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:18.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.093+0000 7f0466800640 1 -- 172.21.15.5:0/2244218688 learned_addr learned my addr 172.21.15.5:0/2244218688 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:18.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.093+0000 7f045d400640 1 --2- 172.21.15.5:0/2244218688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab7255adc0 0x55ab7255d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:18.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.093+0000 7f0466800640 1 -- 172.21.15.5:0/2244218688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab72555c50 msgr2=0x55ab7255a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:18.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.093+0000 7f0466800640 1 --2- 172.21.15.5:0/2244218688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab72555c50 0x55ab7255a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.093+0000 7f0466800640 1 -- 172.21.15.5:0/2244218688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab7255adc0 msgr2=0x55ab7255d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:18.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.093+0000 7f0466800640 1 --2- 172.21.15.5:0/2244218688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab7255adc0 0x55ab7255d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.093+0000 7f0466800640 1 -- 172.21.15.5:0/2244218688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab721cd2d0 con 0x55ab72555260 2024-04-24T06:43:18.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.094+0000 7f0466800640 1 --2- 172.21.15.5:0/2244218688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab72555260 0x55ab72555680 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0460009fd0 tx=0x7f04600118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a80bdcf0a57fa895 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:18.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.094+0000 7f0457000640 1 -- 172.21.15.5:0/2244218688 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0460004d20 con 0x55ab72555260 2024-04-24T06:43:18.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.094+0000 7f0457000640 1 -- 172.21.15.5:0/2244218688 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0460004050 con 0x55ab72555260 2024-04-24T06:43:18.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.094+0000 7f0457000640 1 -- 172.21.15.5:0/2244218688 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0460005260 con 0x55ab72555260 2024-04-24T06:43:18.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.094+0000 7f045f400640 1 --2- 172.21.15.5:0/2721917814 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f044c0a3c30 0x7f044c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:18.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.095+0000 7f045f400640 1 -- 172.21.15.5:0/2721917814 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f04580d0530 con 0x7f0468069b70 2024-04-24T06:43:18.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.095+0000 7f046f82d740 1 -- 172.21.15.5:0/2244218688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab72555260 msgr2=0x55ab72555680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:18.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.095+0000 7f046f82d740 1 --2- 172.21.15.5:0/2244218688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab72555260 0x55ab72555680 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0460009fd0 tx=0x7f04600118d0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.095+0000 7f0465e00640 1 --2- 172.21.15.5:0/2721917814 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f044c0a3c30 0x7f044c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:18.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.095+0000 7f046f82d740 1 -- 172.21.15.5:0/2244218688 shutdown_connections 2024-04-24T06:43:18.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.095+0000 7f046f82d740 1 --2- 172.21.15.5:0/2244218688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab7255adc0 0x55ab7255d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.095+0000 7f046f82d740 1 --2- 172.21.15.5:0/2244218688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab72555c50 0x55ab7255a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.095+0000 7f046f82d740 1 --2- 172.21.15.5:0/2244218688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab72555260 0x55ab72555680 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.095+0000 7f046f82d740 1 -- 172.21.15.5:0/2244218688 >> 172.21.15.5:0/2244218688 conn(0x55ab725508d0 msgr2=0x55ab72550cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:18.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.095+0000 7f046f82d740 1 -- 172.21.15.5:0/2244218688 shutdown_connections 2024-04-24T06:43:18.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.095+0000 7f046f82d740 1 -- 172.21.15.5:0/2244218688 wait complete. 2024-04-24T06:43:18.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.096+0000 7f0465e00640 1 --2- 172.21.15.5:0/2721917814 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f044c0a3c30 0x7f044c0a60f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x55ab72556050 tx=0x7f04580d6060 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:18.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.096+0000 7f046f82d740 1 -- start start 2024-04-24T06:43:18.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.096+0000 7f046f82d740 1 Processor -- start 2024-04-24T06:43:18.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.096+0000 7f046f82d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab72555260 0x55ab7255f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:18.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.096+0000 7f046f82d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab72555c50 0x55ab7255fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:18.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.096+0000 7f046f82d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab7255adc0 0x55ab72566bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:18.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.096+0000 7f046f82d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ab7255dd70 con 0x55ab72555260 2024-04-24T06:43:18.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.096+0000 7f046f82d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ab72474590 con 0x55ab7255adc0 2024-04-24T06:43:18.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.096+0000 7f046f82d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ab7255def0 con 0x55ab72555c50 2024-04-24T06:43:18.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.097+0000 7f046f82d740 10 client.0 _ll_register_callbacks cb 0x55ab724573a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:43:18.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.097+0000 7f045d400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab72555c50 0x55ab7255fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:18.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.097+0000 7f0466800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab72555260 0x55ab7255f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:18.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.097+0000 7f045d400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab72555c50 0x55ab7255fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:44704/0 (socket says 172.21.15.5:44704) 2024-04-24T06:43:18.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.097+0000 7f045d400640 1 -- 172.21.15.5:0/668176883 learned_addr learned my addr 172.21.15.5:0/668176883 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:18.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.097+0000 7f0466800640 1 --2- 172.21.15.5:0/668176883 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab7255adc0 0x55ab72566bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:18.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.097+0000 7f045d400640 1 -- 172.21.15.5:0/668176883 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab7255adc0 msgr2=0x55ab72566bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:18.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.097+0000 7f045d400640 1 --2- 172.21.15.5:0/668176883 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab7255adc0 0x55ab72566bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.097+0000 7f045d400640 1 -- 172.21.15.5:0/668176883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab72555260 msgr2=0x55ab7255f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:18.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.097+0000 7f045d400640 1 --2- 172.21.15.5:0/668176883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab72555260 0x55ab7255f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.097+0000 7f045d400640 1 -- 172.21.15.5:0/668176883 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab722976b0 con 0x55ab72555c50 2024-04-24T06:43:18.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.097+0000 7f0466800640 1 --2- 172.21.15.5:0/668176883 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab7255adc0 0x55ab72566bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:43:18.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.098+0000 7f045d400640 1 --2- 172.21.15.5:0/668176883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab72555c50 0x55ab7255fb70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0450000c00 tx=0x7f0450004300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c70dfd6fb2bead66 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:18.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.098+0000 7f044b400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:43:18.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.100+0000 7f044b400640 1 -- 172.21.15.5:0/668176883 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0450035040 con 0x55ab72555c50 2024-04-24T06:43:18.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.100+0000 7f044b400640 1 -- 172.21.15.5:0/668176883 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f0450005450 con 0x55ab72555c50 2024-04-24T06:43:18.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.100+0000 7f046f82d740 1 -- 172.21.15.5:0/668176883 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0460004050 con 0x55ab72555c50 2024-04-24T06:43:18.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.103+0000 7f044b400640 1 -- 172.21.15.5:0/668176883 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f0450005650 con 0x55ab72555c50 2024-04-24T06:43:18.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.103+0000 7f046f82d740 10 client.25018 fetch_fsmap learned FSMap version 22 2024-04-24T06:43:18.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.103+0000 7f046f82d740 1 -- 172.21.15.5:0/668176883 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55ab72569f90 con 0x55ab72555c50 2024-04-24T06:43:18.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.103+0000 7f044b400640 1 -- 172.21.15.5:0/668176883 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0460004050 con 0x55ab72555c50 2024-04-24T06:43:18.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.104+0000 7f044b400640 1 -- 172.21.15.5:0/668176883 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f0450031ba0 con 0x55ab72555c50 2024-04-24T06:43:18.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.104+0000 7f046f82d740 10 client.25018 fetch_fsmap finished waiting for FSMap version 22 2024-04-24T06:43:18.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.104+0000 7f046f82d740 10 client.25018 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-24T06:43:18.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.104+0000 7f046f82d740 20 client.25018 populate_metadata read hostname 'smithi005' 2024-04-24T06:43:18.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.104+0000 7f046f82d740 1 --2- 172.21.15.5:0/668176883 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55ab7256c5e0 0x55ab7256da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:18.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.104+0000 7f046f82d740 4 client.25018 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:43:18.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.104+0000 7f046f82d740 1 -- 172.21.15.5:0/668176883 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab724c8530 con 0x55ab7256c5e0 2024-04-24T06:43:18.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.104+0000 7f0466800640 1 --2- 172.21.15.5:0/668176883 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55ab7256c5e0 0x55ab7256da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:18.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.105+0000 7f0466800640 1 --2- 172.21.15.5:0/668176883 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55ab7256c5e0 0x55ab7256da50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e254fddcea7cf188 server_cookie=9c6c2f6c938ce881 in_seq=0 out_seq=0 2024-04-24T06:43:18.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.105+0000 7f044b400640 10 client.25018 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:43:18.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.106+0000 7f044b400640 1 -- 172.21.15.5:0/668176883 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ab724c8530 con 0x55ab7256c5e0 2024-04-24T06:43:18.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.106+0000 7f044b400640 10 client.25018 handle_command_reply: tid=0 2024-04-24T06:43:18.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.106+0000 7f046f82d740 1 client.25018 shutdown 2024-04-24T06:43:18.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.106+0000 7f046f82d740 1 -- 172.21.15.5:0/668176883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab72555c50 msgr2=0x55ab7255fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:18.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.106+0000 7f046f82d740 1 --2- 172.21.15.5:0/668176883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab72555c50 0x55ab7255fb70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0450000c00 tx=0x7f0450004300 comp rx=0 tx=0).stop 2024-04-24T06:43:18.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.106+0000 7f046f82d740 1 -- 172.21.15.5:0/668176883 shutdown_connections 2024-04-24T06:43:18.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.106+0000 7f046f82d740 1 --2- 172.21.15.5:0/668176883 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55ab7256c5e0 0x55ab7256da50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.106+0000 7f046f82d740 1 --2- 172.21.15.5:0/668176883 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab7255adc0 0x55ab72566bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.106+0000 7f046f82d740 1 --2- 172.21.15.5:0/668176883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab72555c50 0x55ab7255fb70 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.106+0000 7f044b400640 0 client.25018 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:43:18.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.106+0000 7f046f82d740 1 --2- 172.21.15.5:0/668176883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab72555260 0x55ab7255f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.106+0000 7f046f82d740 1 -- 172.21.15.5:0/668176883 >> 172.21.15.5:0/668176883 conn(0x55ab725508d0 msgr2=0x55ab7254f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:18.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.107+0000 7f046f82d740 1 -- 172.21.15.5:0/668176883 shutdown_connections 2024-04-24T06:43:18.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.107+0000 7f046f82d740 1 -- 172.21.15.5:0/668176883 wait complete. 2024-04-24T06:43:18.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.107+0000 7f046f82d740 20 client.25018 trim_cache size 0 max 16384 2024-04-24T06:43:18.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.126+0000 7f046f82d740 1 Processor -- start 2024-04-24T06:43:18.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.126+0000 7f046f82d740 1 -- start start 2024-04-24T06:43:18.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.127+0000 7f046f82d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab72555260 0x55ab72557010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:18.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.127+0000 7f046f82d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab72555c50 0x55ab72557550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:18.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.127+0000 7f046f82d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab7255adc0 0x55ab72566b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:18.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.127+0000 7f046f82d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ab7256bd90 con 0x55ab72555c50 2024-04-24T06:43:18.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.127+0000 7f046f82d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ab7256c460 con 0x55ab72555260 2024-04-24T06:43:18.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.127+0000 7f046f82d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ab7256cb30 con 0x55ab7255adc0 2024-04-24T06:43:18.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.127+0000 7f045d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab72555c50 0x55ab72557550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:18.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.127+0000 7f045d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab72555c50 0x55ab72557550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58764/0 (socket says 172.21.15.5:58764) 2024-04-24T06:43:18.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.127+0000 7f045d400640 1 -- 172.21.15.5:0/3068445086 learned_addr learned my addr 172.21.15.5:0/3068445086 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:18.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.127+0000 7f0466800640 1 --2- 172.21.15.5:0/3068445086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab72555260 0x55ab72557010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:18.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.127+0000 7f0466800640 1 --2- 172.21.15.5:0/3068445086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab7255adc0 0x55ab72566b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:18.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.127+0000 7f045d400640 1 -- 172.21.15.5:0/3068445086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab7255adc0 msgr2=0x55ab72566b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:18.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.128+0000 7f045d400640 1 --2- 172.21.15.5:0/3068445086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab7255adc0 0x55ab72566b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.128+0000 7f045d400640 1 -- 172.21.15.5:0/3068445086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab72555260 msgr2=0x55ab72557010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:18.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.128+0000 7f045d400640 1 --2- 172.21.15.5:0/3068445086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab72555260 0x55ab72557010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.128+0000 7f045d400640 1 -- 172.21.15.5:0/3068445086 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab72560620 con 0x55ab72555c50 2024-04-24T06:43:18.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.128+0000 7f045d400640 1 --2- 172.21.15.5:0/3068445086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab72555c50 0x55ab72557550 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0450009900 tx=0x7f0450031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e122a6eccf72feee server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:18.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.128+0000 7f0456600640 1 -- 172.21.15.5:0/3068445086 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0450035030 con 0x55ab72555c50 2024-04-24T06:43:18.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.128+0000 7f0456600640 1 -- 172.21.15.5:0/3068445086 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0450041070 con 0x55ab72555c50 2024-04-24T06:43:18.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.128+0000 7f0456600640 1 -- 172.21.15.5:0/3068445086 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0450003e60 con 0x55ab72555c50 2024-04-24T06:43:18.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.129+0000 7f046f82d740 1 -- 172.21.15.5:0/3068445086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab72555c50 msgr2=0x55ab72557550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:18.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.129+0000 7f046f82d740 1 --2- 172.21.15.5:0/3068445086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab72555c50 0x55ab72557550 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0450009900 tx=0x7f0450031400 comp rx=0 tx=0).stop 2024-04-24T06:43:18.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.129+0000 7f046f82d740 1 -- 172.21.15.5:0/3068445086 shutdown_connections 2024-04-24T06:43:18.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.129+0000 7f046f82d740 1 --2- 172.21.15.5:0/3068445086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab7255adc0 0x55ab72566b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.129+0000 7f046f82d740 1 --2- 172.21.15.5:0/3068445086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab72555c50 0x55ab72557550 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.129+0000 7f046f82d740 1 --2- 172.21.15.5:0/3068445086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab72555260 0x55ab72557010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.129+0000 7f046f82d740 1 -- 172.21.15.5:0/3068445086 >> 172.21.15.5:0/3068445086 conn(0x55ab725508d0 msgr2=0x55ab7254b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:18.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.129+0000 7f046f82d740 1 -- 172.21.15.5:0/3068445086 shutdown_connections 2024-04-24T06:43:18.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.129+0000 7f046f82d740 1 -- 172.21.15.5:0/3068445086 wait complete. 2024-04-24T06:43:18.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.129+0000 7f046f82d740 1 -- start start 2024-04-24T06:43:18.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.130+0000 7f046f82d740 1 Processor -- start 2024-04-24T06:43:18.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.130+0000 7f046f82d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab72555260 0x55ab72562360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:18.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.130+0000 7f046f82d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab72555c50 0x55ab725628a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:18.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.130+0000 7f046f82d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab7255adc0 0x55ab725658c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:18.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.130+0000 7f046f82d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ab7256d8d0 con 0x55ab72555260 2024-04-24T06:43:18.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.130+0000 7f046f82d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ab7256d200 con 0x55ab7255adc0 2024-04-24T06:43:18.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.130+0000 7f046f82d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ab7256dfa0 con 0x55ab72555c50 2024-04-24T06:43:18.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.130+0000 7f046f82d740 10 client.0 _ll_register_callbacks cb 0x7f0450009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:43:18.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.130+0000 7f0466800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab72555260 0x55ab72562360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:18.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.130+0000 7f0466800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab72555260 0x55ab72562360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58778/0 (socket says 172.21.15.5:58778) 2024-04-24T06:43:18.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.130+0000 7f0466800640 1 -- 172.21.15.5:0/2815704347 learned_addr learned my addr 172.21.15.5:0/2815704347 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:18.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.130+0000 7f045d400640 1 --2- 172.21.15.5:0/2815704347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab72555c50 0x55ab725628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:18.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.131+0000 7f0466800640 1 --2- 172.21.15.5:0/2815704347 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab7255adc0 0x55ab725658c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.131+0000 7f045d400640 1 -- 172.21.15.5:0/2815704347 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab7255adc0 msgr2=0x55ab725658c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.131+0000 7f045d400640 1 --2- 172.21.15.5:0/2815704347 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab7255adc0 0x55ab725658c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.131+0000 7f045d400640 1 -- 172.21.15.5:0/2815704347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab72555260 msgr2=0x55ab72562360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.131+0000 7f045d400640 1 --2- 172.21.15.5:0/2815704347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab72555260 0x55ab72562360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.131+0000 7f045d400640 1 -- 172.21.15.5:0/2815704347 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab72565eb0 con 0x55ab72555c50 2024-04-24T06:43:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.131+0000 7f0466800640 1 --2- 172.21.15.5:0/2815704347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab72555260 0x55ab72562360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:43:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.131+0000 7f045d400640 1 --2- 172.21.15.5:0/2815704347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab72555c50 0x55ab725628a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0450006010 tx=0x7f04500315b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=783424006d806d30 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.131+0000 7f044b400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:43:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.131+0000 7f044b400640 1 -- 172.21.15.5:0/2815704347 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0450035040 con 0x55ab72555c50 2024-04-24T06:43:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.131+0000 7f046f82d740 1 -- 172.21.15.5:0/2815704347 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0450041070 con 0x55ab72555c50 2024-04-24T06:43:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.131+0000 7f044b400640 1 -- 172.21.15.5:0/2815704347 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f0450031ac0 con 0x55ab72555c50 2024-04-24T06:43:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.132+0000 7f044b400640 1 -- 172.21.15.5:0/2815704347 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f0450031cc0 con 0x55ab72555c50 2024-04-24T06:43:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.132+0000 7f046f82d740 10 client.25024 fetch_fsmap learned FSMap version 22 2024-04-24T06:43:18.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.132+0000 7f046f82d740 1 -- 172.21.15.5:0/2815704347 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55ab725d25f0 con 0x55ab72555c50 2024-04-24T06:43:18.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.132+0000 7f044b400640 1 -- 172.21.15.5:0/2815704347 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0450041070 con 0x55ab72555c50 2024-04-24T06:43:18.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.132+0000 7f044b400640 1 -- 172.21.15.5:0/2815704347 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f0450060070 con 0x55ab72555c50 2024-04-24T06:43:18.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.132+0000 7f046f82d740 10 client.25024 fetch_fsmap finished waiting for FSMap version 22 2024-04-24T06:43:18.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.132+0000 7f046f82d740 10 client.25024 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-24T06:43:18.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.132+0000 7f046f82d740 20 client.25024 populate_metadata read hostname 'smithi005' 2024-04-24T06:43:18.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.132+0000 7f046f82d740 1 --2- 172.21.15.5:0/2815704347 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55ab725d6f70 0x55ab725d5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:18.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.132+0000 7f046f82d740 4 client.25024 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T06:43:18.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.132+0000 7f046f82d740 1 -- 172.21.15.5:0/2815704347 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "status"}) -- 0x55ab724c23a0 con 0x55ab725d6f70 2024-04-24T06:43:18.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.133+0000 7f0466800640 1 --2- 172.21.15.5:0/2815704347 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55ab725d6f70 0x55ab725d5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:18.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.133+0000 7f0466800640 1 --2- 172.21.15.5:0/2815704347 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55ab725d6f70 0x55ab725d5c80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4c0c3f7040050ed server_cookie=1730e5d739c0103c in_seq=0 out_seq=0 2024-04-24T06:43:18.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.133+0000 7f044b400640 10 client.25024 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:43:18.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.133+0000 7f044b400640 1 -- 172.21.15.5:0/2815704347 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ab724c23a0 con 0x55ab725d6f70 2024-04-24T06:43:18.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.133+0000 7f044b400640 10 client.25024 handle_command_reply: tid=0 2024-04-24T06:43:18.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.133+0000 7f046f82d740 1 client.25024 shutdown 2024-04-24T06:43:18.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.134+0000 7f046f82d740 1 -- 172.21.15.5:0/2815704347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab72555c50 msgr2=0x55ab725628a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:18.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.134+0000 7f046f82d740 1 --2- 172.21.15.5:0/2815704347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab72555c50 0x55ab725628a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0450006010 tx=0x7f04500315b0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.134+0000 7f046f82d740 1 -- 172.21.15.5:0/2815704347 shutdown_connections 2024-04-24T06:43:18.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.134+0000 7f046f82d740 1 --2- 172.21.15.5:0/2815704347 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55ab725d6f70 0x55ab725d5c80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.134+0000 7f046f82d740 1 --2- 172.21.15.5:0/2815704347 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab7255adc0 0x55ab725658c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.134+0000 7f046f82d740 1 --2- 172.21.15.5:0/2815704347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab72555c50 0x55ab725628a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.134+0000 7f046f82d740 1 --2- 172.21.15.5:0/2815704347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab72555260 0x55ab72562360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.134+0000 7f044b400640 0 client.25024 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:43:18.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.134+0000 7f046f82d740 1 -- 172.21.15.5:0/2815704347 >> 172.21.15.5:0/2815704347 conn(0x55ab725508d0 msgr2=0x55ab7254b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:18.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.134+0000 7f046f82d740 1 -- 172.21.15.5:0/2815704347 shutdown_connections 2024-04-24T06:43:18.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.134+0000 7f046f82d740 1 -- 172.21.15.5:0/2815704347 wait complete. 2024-04-24T06:43:18.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.134+0000 7f046f82d740 20 client.25024 trim_cache size 0 max 16384 2024-04-24T06:43:18.140 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:43:18.140 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "43e9bad0-0204-11ef-bc93-c7b262605968", 2024-04-24T06:43:18.140 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 1, 2024-04-24T06:43:18.140 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24550, 2024-04-24T06:43:18.140 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-04-24T06:43:18.140 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-04-24T06:43:18.140 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-04-24T06:43:18.140 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 99.825730704999998, 2024-04-24T06:43:18.141 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 22, 2024-04-24T06:43:18.141 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 88, 2024-04-24T06:43:18.141 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 86, 2024-04-24T06:43:18.141 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 121.353452202 2024-04-24T06:43:18.141 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:43:18.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.135+0000 7f0467200640 1 -- 172.21.15.5:0/2721917814 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f044c0a3c30 msgr2=0x7f044c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:18.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.135+0000 7f0467200640 1 --2- 172.21.15.5:0/2721917814 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f044c0a3c30 0x7f044c0a60f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x55ab72556050 tx=0x7f04580d6060 comp rx=0 tx=0).stop 2024-04-24T06:43:18.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.135+0000 7f0467200640 1 -- 172.21.15.5:0/2721917814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0468069b70 msgr2=0x7f04681055a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:18.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.135+0000 7f0467200640 1 --2- 172.21.15.5:0/2721917814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0468069b70 0x7f04681055a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f045800e9c0 tx=0x7f045800ee80 comp rx=0 tx=0).stop 2024-04-24T06:43:18.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.135+0000 7f0467200640 1 -- 172.21.15.5:0/2721917814 shutdown_connections 2024-04-24T06:43:18.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.135+0000 7f0467200640 1 --2- 172.21.15.5:0/2721917814 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f044c0a3c30 0x7f044c0a60f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.135+0000 7f0467200640 1 --2- 172.21.15.5:0/2721917814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04681016f0 0x7f0468104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.135+0000 7f0467200640 1 --2- 172.21.15.5:0/2721917814 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f046806a020 0x7f0468103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.135+0000 7f0467200640 1 --2- 172.21.15.5:0/2721917814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0468069b70 0x7f04681055a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.135+0000 7f0467200640 1 -- 172.21.15.5:0/2721917814 >> 172.21.15.5:0/2721917814 conn(0x7f04680fba70 msgr2=0x7f04681031a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:18.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.135+0000 7f0467200640 1 -- 172.21.15.5:0/2721917814 shutdown_connections 2024-04-24T06:43:18.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.135+0000 7f0467200640 1 -- 172.21.15.5:0/2721917814 wait complete. 2024-04-24T06:43:18.150 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:18 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/289446811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:43:18.154 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-24T06:43:18.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.357+0000 7fe3ae800640 1 -- 172.21.15.5:0/2592167695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3a8108480 msgr2=0x7fe3a81088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:18.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.357+0000 7fe3ae800640 1 --2- 172.21.15.5:0/2592167695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3a8108480 0x7fe3a81088a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe39c009860 tx=0x7fe39c02a340 comp rx=0 tx=0).stop 2024-04-24T06:43:18.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.358+0000 7fe3ae800640 1 -- 172.21.15.5:0/2592167695 shutdown_connections 2024-04-24T06:43:18.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.358+0000 7fe3ae800640 1 --2- 172.21.15.5:0/2592167695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3a810a030 0x7fe3a81108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.358+0000 7fe3ae800640 1 --2- 172.21.15.5:0/2592167695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3a8109670 0x7fe3a8109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.358+0000 7fe3ae800640 1 --2- 172.21.15.5:0/2592167695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3a8108480 0x7fe3a81088a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.358+0000 7fe3ae800640 1 -- 172.21.15.5:0/2592167695 >> 172.21.15.5:0/2592167695 conn(0x7fe3a8077440 msgr2=0x7fe3a8079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:18.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.358+0000 7fe3ae800640 1 -- 172.21.15.5:0/2592167695 shutdown_connections 2024-04-24T06:43:18.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.358+0000 7fe3ae800640 1 -- 172.21.15.5:0/2592167695 wait complete. 2024-04-24T06:43:18.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.359+0000 7fe3ae800640 1 Processor -- start 2024-04-24T06:43:18.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.359+0000 7fe3ae800640 1 -- start start 2024-04-24T06:43:18.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.359+0000 7fe3ae800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3a8108480 0x7fe3a819e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:18.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.359+0000 7fe3ae800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3a8109670 0x7fe3a819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:18.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.359+0000 7fe3ae800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3a810a030 0x7fe3a81a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:18.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.359+0000 7fe3ae800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a8114110 con 0x7fe3a810a030 2024-04-24T06:43:18.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.359+0000 7fe3ae800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a8113f90 con 0x7fe3a8109670 2024-04-24T06:43:18.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.359+0000 7fe3ae800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a8114290 con 0x7fe3a8108480 2024-04-24T06:43:18.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.360+0000 7fe3ade00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3a810a030 0x7fe3a81a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:18.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.360+0000 7fe3aca00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3a8109670 0x7fe3a819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:18.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.360+0000 7fe3aca00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3a8109670 0x7fe3a819ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:53170/0 (socket says 172.21.15.5:53170) 2024-04-24T06:43:18.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.360+0000 7fe3aca00640 1 -- 172.21.15.5:0/2308103326 learned_addr learned my addr 172.21.15.5:0/2308103326 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:18.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.360+0000 7fe3ad400640 1 --2- 172.21.15.5:0/2308103326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3a8108480 0x7fe3a819e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:18.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.360+0000 7fe3ad400640 1 -- 172.21.15.5:0/2308103326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3a8109670 msgr2=0x7fe3a819ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:18.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.360+0000 7fe3ad400640 1 --2- 172.21.15.5:0/2308103326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3a8109670 0x7fe3a819ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.360+0000 7fe3ad400640 1 -- 172.21.15.5:0/2308103326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3a810a030 msgr2=0x7fe3a81a5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:18.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.360+0000 7fe3ad400640 1 --2- 172.21.15.5:0/2308103326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3a810a030 0x7fe3a81a5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.360+0000 7fe3ad400640 1 -- 172.21.15.5:0/2308103326 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a81a63e0 con 0x7fe3a8108480 2024-04-24T06:43:18.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.360+0000 7fe3ad400640 1 --2- 172.21.15.5:0/2308103326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3a8108480 0x7fe3a819e7c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe39c02a860 tx=0x7fe39c033d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:18.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.361+0000 7fe3a6000640 1 -- 172.21.15.5:0/2308103326 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c037040 con 0x7fe3a8108480 2024-04-24T06:43:18.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.361+0000 7fe3a6000640 1 -- 172.21.15.5:0/2308103326 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe39c030850 con 0x7fe3a8108480 2024-04-24T06:43:18.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.361+0000 7fe3ae800640 1 -- 172.21.15.5:0/2308103326 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3a81a6670 con 0x7fe3a8108480 2024-04-24T06:43:18.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.361+0000 7fe3a6000640 1 -- 172.21.15.5:0/2308103326 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c004920 con 0x7fe3a8108480 2024-04-24T06:43:18.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.361+0000 7fe3ae800640 1 -- 172.21.15.5:0/2308103326 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3a81a6b80 con 0x7fe3a8108480 2024-04-24T06:43:18.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.364+0000 7fe3b2dd9740 1 Processor -- start 2024-04-24T06:43:18.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.364+0000 7fe3b2dd9740 1 -- start start 2024-04-24T06:43:18.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.364+0000 7fe3a6000640 1 -- 172.21.15.5:0/2308103326 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7fe39c004050 con 0x7fe3a8108480 2024-04-24T06:43:18.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.364+0000 7fe3b2dd9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ce391260 0x55e3ce391680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:18.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.364+0000 7fe3b2dd9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ce391c50 0x55e3ce396750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:18.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.364+0000 7fe3b2dd9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3ce396dc0 0x55e3ce399250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:18.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.364+0000 7fe3b2dd9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e3ce2f6b10 con 0x55e3ce391c50 2024-04-24T06:43:18.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.364+0000 7fe3b2dd9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e3ce29d2b0 con 0x55e3ce391260 2024-04-24T06:43:18.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.364+0000 7fe3b2dd9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e3ce2f2ad0 con 0x55e3ce396dc0 2024-04-24T06:43:18.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.365+0000 7fe3ad400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ce391c50 0x55e3ce396750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:18.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.365+0000 7fe3ad400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ce391c50 0x55e3ce396750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58792/0 (socket says 172.21.15.5:58792) 2024-04-24T06:43:18.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.365+0000 7fe3ad400640 1 -- 172.21.15.5:0/447729024 learned_addr learned my addr 172.21.15.5:0/447729024 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:18.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.365+0000 7fe3ade00640 1 --2- 172.21.15.5:0/447729024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ce391260 0x55e3ce391680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:18.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.365+0000 7fe3ad400640 1 -- 172.21.15.5:0/447729024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3ce396dc0 msgr2=0x55e3ce399250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:18.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.365+0000 7fe3ad400640 1 --2- 172.21.15.5:0/447729024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3ce396dc0 0x55e3ce399250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.365+0000 7fe3ad400640 1 -- 172.21.15.5:0/447729024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ce391260 msgr2=0x55e3ce391680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:18.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.365+0000 7fe3ad400640 1 --2- 172.21.15.5:0/447729024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ce391260 0x55e3ce391680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.365+0000 7fe3ad400640 1 -- 172.21.15.5:0/447729024 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3ce0092d0 con 0x55e3ce391c50 2024-04-24T06:43:18.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.365+0000 7fe3ade00640 1 --2- 172.21.15.5:0/447729024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ce391260 0x55e3ce391680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:43:18.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.365+0000 7fe3ad400640 1 --2- 172.21.15.5:0/447729024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ce391c50 0x55e3ce396750 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe39c0f8190 tx=0x7fe39c0f8280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90a2e7cbabb6833a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:18.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.366+0000 7fe39a600640 1 -- 172.21.15.5:0/447729024 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c0f8ab0 con 0x55e3ce391c50 2024-04-24T06:43:18.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.366+0000 7fe39a600640 1 -- 172.21.15.5:0/447729024 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe39c0fd030 con 0x55e3ce391c50 2024-04-24T06:43:18.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.366+0000 7fe39a600640 1 -- 172.21.15.5:0/447729024 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c101630 con 0x55e3ce391c50 2024-04-24T06:43:18.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.366+0000 7fe3a6000640 1 --2- 172.21.15.5:0/2308103326 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fe3900a3c30 0x7fe3900a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:18.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.366+0000 7fe3a6000640 1 -- 172.21.15.5:0/2308103326 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fe39c0f0160 con 0x7fe3a8108480 2024-04-24T06:43:18.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.366+0000 7fe3ade00640 1 --2- 172.21.15.5:0/2308103326 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fe3900a3c30 0x7fe3900a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:18.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.366+0000 7fe3b2dd9740 1 -- 172.21.15.5:0/447729024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ce391c50 msgr2=0x55e3ce396750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:18.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.366+0000 7fe3b2dd9740 1 --2- 172.21.15.5:0/447729024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ce391c50 0x55e3ce396750 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe39c0f8190 tx=0x7fe39c0f8280 comp rx=0 tx=0).stop 2024-04-24T06:43:18.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.367+0000 7fe3b2dd9740 1 -- 172.21.15.5:0/447729024 shutdown_connections 2024-04-24T06:43:18.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.367+0000 7fe3b2dd9740 1 --2- 172.21.15.5:0/447729024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3ce396dc0 0x55e3ce399250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.367+0000 7fe3b2dd9740 1 --2- 172.21.15.5:0/447729024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ce391c50 0x55e3ce396750 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.367+0000 7fe3b2dd9740 1 --2- 172.21.15.5:0/447729024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ce391260 0x55e3ce391680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.367+0000 7fe3b2dd9740 1 -- 172.21.15.5:0/447729024 >> 172.21.15.5:0/447729024 conn(0x55e3ce38c8d0 msgr2=0x55e3ce38ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:18.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.367+0000 7fe3b2dd9740 1 -- 172.21.15.5:0/447729024 shutdown_connections 2024-04-24T06:43:18.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.367+0000 7fe3b2dd9740 1 -- 172.21.15.5:0/447729024 wait complete. 2024-04-24T06:43:18.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.367+0000 7fe3ade00640 1 --2- 172.21.15.5:0/2308103326 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fe3900a3c30 0x7fe3900a60f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x55e3ce3924b0 tx=0x7fe3a0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:18.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.367+0000 7fe3b2dd9740 1 -- start start 2024-04-24T06:43:18.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.367+0000 7fe3b2dd9740 1 Processor -- start 2024-04-24T06:43:18.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.368+0000 7fe3b2dd9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ce391260 0x55e3ce39b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:18.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.368+0000 7fe3b2dd9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ce391c50 0x55e3ce39b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:18.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.368+0000 7fe3b2dd9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3ce396dc0 0x55e3ce3a2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:18.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.368+0000 7fe3b2dd9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e3ce399b30 con 0x55e3ce391260 2024-04-24T06:43:18.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.368+0000 7fe3b2dd9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e3ce2b0590 con 0x55e3ce391c50 2024-04-24T06:43:18.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.368+0000 7fe3b2dd9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e3ce399cb0 con 0x55e3ce396dc0 2024-04-24T06:43:18.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.368+0000 7fe3b2dd9740 10 client.0 _ll_register_callbacks cb 0x55e3ce2933a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:43:18.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.368+0000 7fe3ad400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ce391260 0x55e3ce39b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:18.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.368+0000 7fe3ad400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ce391260 0x55e3ce39b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58800/0 (socket says 172.21.15.5:58800) 2024-04-24T06:43:18.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.368+0000 7fe3ad400640 1 -- 172.21.15.5:0/4132568412 learned_addr learned my addr 172.21.15.5:0/4132568412 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:18.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.368+0000 7fe3aca00640 1 --2- 172.21.15.5:0/4132568412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ce391c50 0x55e3ce39b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:18.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.369+0000 7fe3ad400640 1 -- 172.21.15.5:0/4132568412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3ce396dc0 msgr2=0x55e3ce3a2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:18.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.369+0000 7fe3ad400640 1 --2- 172.21.15.5:0/4132568412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3ce396dc0 0x55e3ce3a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.369+0000 7fe3ad400640 1 -- 172.21.15.5:0/4132568412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ce391c50 msgr2=0x55e3ce39b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:18.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.369+0000 7fe3ad400640 1 --2- 172.21.15.5:0/4132568412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ce391c50 0x55e3ce39b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.369+0000 7fe3ad400640 1 -- 172.21.15.5:0/4132568412 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3ce0d36b0 con 0x55e3ce391260 2024-04-24T06:43:18.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.369+0000 7fe3aca00640 1 --2- 172.21.15.5:0/4132568412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ce391c50 0x55e3ce39b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:43:18.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.369+0000 7fe3ad400640 1 --2- 172.21.15.5:0/4132568412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ce391260 0x55e3ce39b3d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe39c0c4b40 tx=0x7fe39c02a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b30c1446effa17f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:18.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.369+0000 7fe38ea00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:43:18.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.369+0000 7fe38ea00640 1 -- 172.21.15.5:0/4132568412 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c0b86a0 con 0x55e3ce391260 2024-04-24T06:43:18.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.369+0000 7fe38ea00640 1 -- 172.21.15.5:0/4132568412 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fe39c0ba700 con 0x55e3ce391260 2024-04-24T06:43:18.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.369+0000 7fe3b2dd9740 1 -- 172.21.15.5:0/4132568412 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe39c0fd030 con 0x55e3ce391260 2024-04-24T06:43:18.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.370+0000 7fe38ea00640 1 -- 172.21.15.5:0/4132568412 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fe39c0bbd60 con 0x55e3ce391260 2024-04-24T06:43:18.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.370+0000 7fe3b2dd9740 10 client.14919 fetch_fsmap learned FSMap version 22 2024-04-24T06:43:18.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.370+0000 7fe3b2dd9740 1 -- 172.21.15.5:0/4132568412 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55e3ce399b30 con 0x55e3ce391260 2024-04-24T06:43:18.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.370+0000 7fe38ea00640 1 -- 172.21.15.5:0/4132568412 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe39c0fd030 con 0x55e3ce391260 2024-04-24T06:43:18.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.370+0000 7fe38ea00640 1 -- 172.21.15.5:0/4132568412 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fe39c03a070 con 0x55e3ce391260 2024-04-24T06:43:18.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.370+0000 7fe3b2dd9740 10 client.14919 fetch_fsmap finished waiting for FSMap version 22 2024-04-24T06:43:18.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.370+0000 7fe3b2dd9740 10 client.14919 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T06:43:18.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.370+0000 7fe3b2dd9740 20 client.14919 populate_metadata read hostname 'smithi005' 2024-04-24T06:43:18.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.370+0000 7fe3b2dd9740 1 --2- 172.21.15.5:0/4132568412 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55e3ce3a8160 0x55e3ce3a95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:18.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.370+0000 7fe3b2dd9740 4 client.14919 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:43:18.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.370+0000 7fe3b2dd9740 1 -- 172.21.15.5:0/4132568412 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3ce304530 con 0x55e3ce3a8160 2024-04-24T06:43:18.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.371+0000 7fe3aca00640 1 --2- 172.21.15.5:0/4132568412 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55e3ce3a8160 0x55e3ce3a95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:18.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.371+0000 7fe3aca00640 1 --2- 172.21.15.5:0/4132568412 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55e3ce3a8160 0x55e3ce3a95d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f83af297eb31b8b2 server_cookie=39fc65476ac99bf2 in_seq=0 out_seq=0 2024-04-24T06:43:18.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.371+0000 7fe38ea00640 10 client.14919 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:43:18.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.372+0000 7fe38ea00640 1 -- 172.21.15.5:0/4132568412 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e3ce304530 con 0x55e3ce3a8160 2024-04-24T06:43:18.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.372+0000 7fe38ea00640 10 client.14919 handle_command_reply: tid=0 2024-04-24T06:43:18.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.372+0000 7fe3b2dd9740 1 client.14919 shutdown 2024-04-24T06:43:18.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.372+0000 7fe3b2dd9740 1 -- 172.21.15.5:0/4132568412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ce391260 msgr2=0x55e3ce39b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:18.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.372+0000 7fe3b2dd9740 1 --2- 172.21.15.5:0/4132568412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ce391260 0x55e3ce39b3d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe39c0c4b40 tx=0x7fe39c02a340 comp rx=0 tx=0).stop 2024-04-24T06:43:18.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.372+0000 7fe3b2dd9740 1 -- 172.21.15.5:0/4132568412 shutdown_connections 2024-04-24T06:43:18.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.372+0000 7fe3b2dd9740 1 --2- 172.21.15.5:0/4132568412 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55e3ce3a8160 0x55e3ce3a95d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.372+0000 7fe3b2dd9740 1 --2- 172.21.15.5:0/4132568412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3ce396dc0 0x55e3ce3a2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.372+0000 7fe3b2dd9740 1 --2- 172.21.15.5:0/4132568412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ce391c50 0x55e3ce39b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.372+0000 7fe3b2dd9740 1 --2- 172.21.15.5:0/4132568412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ce391260 0x55e3ce39b3d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.372+0000 7fe3b2dd9740 1 -- 172.21.15.5:0/4132568412 >> 172.21.15.5:0/4132568412 conn(0x55e3ce38c8d0 msgr2=0x55e3ce38b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:18.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.372+0000 7fe38ea00640 0 client.14919 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:43:18.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.373+0000 7fe3b2dd9740 1 -- 172.21.15.5:0/4132568412 shutdown_connections 2024-04-24T06:43:18.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.373+0000 7fe3b2dd9740 1 -- 172.21.15.5:0/4132568412 wait complete. 2024-04-24T06:43:18.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.373+0000 7fe3b2dd9740 20 client.14919 trim_cache size 0 max 16384 2024-04-24T06:43:18.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.393+0000 7fe3b2dd9740 1 Processor -- start 2024-04-24T06:43:18.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.393+0000 7fe3b2dd9740 1 -- start start 2024-04-24T06:43:18.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.393+0000 7fe3b2dd9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ce391260 0x55e3ce3991b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:18.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.393+0000 7fe3b2dd9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3ce391c50 0x55e3ce3a2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:18.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.393+0000 7fe3b2dd9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ce396dc0 0x55e3ce3a6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:18.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.393+0000 7fe3b2dd9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e3ce3a7910 con 0x55e3ce396dc0 2024-04-24T06:43:18.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.393+0000 7fe3b2dd9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e3ce3a7fe0 con 0x55e3ce391260 2024-04-24T06:43:18.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.393+0000 7fe3b2dd9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e3ce3a86b0 con 0x55e3ce391c50 2024-04-24T06:43:18.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.393+0000 7fe3ade00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ce396dc0 0x55e3ce3a6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:18.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.393+0000 7fe3ade00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ce396dc0 0x55e3ce3a6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58812/0 (socket says 172.21.15.5:58812) 2024-04-24T06:43:18.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.394+0000 7fe3ade00640 1 -- 172.21.15.5:0/1618685880 learned_addr learned my addr 172.21.15.5:0/1618685880 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:18.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.394+0000 7fe3aca00640 1 --2- 172.21.15.5:0/1618685880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3ce391c50 0x55e3ce3a2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:18.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.394+0000 7fe3ad400640 1 --2- 172.21.15.5:0/1618685880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ce391260 0x55e3ce3991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:18.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.394+0000 7fe3ade00640 1 -- 172.21.15.5:0/1618685880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3ce391c50 msgr2=0x55e3ce3a2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:18.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.394+0000 7fe3ade00640 1 --2- 172.21.15.5:0/1618685880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3ce391c50 0x55e3ce3a2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.394+0000 7fe3ade00640 1 -- 172.21.15.5:0/1618685880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ce391260 msgr2=0x55e3ce3991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:18.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.394+0000 7fe3ade00640 1 --2- 172.21.15.5:0/1618685880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ce391260 0x55e3ce3991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.394+0000 7fe3ade00640 1 -- 172.21.15.5:0/1618685880 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3ce396b10 con 0x55e3ce396dc0 2024-04-24T06:43:18.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.394+0000 7fe3aca00640 1 --2- 172.21.15.5:0/1618685880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3ce391c50 0x55e3ce3a2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:43:18.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.395+0000 7fe3ade00640 1 --2- 172.21.15.5:0/1618685880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ce396dc0 0x55e3ce3a6e40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0005f40 tx=0x7fe3a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b71b1eafce8fc52 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:18.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.395+0000 7fe399c00640 1 -- 172.21.15.5:0/1618685880 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a000fd90 con 0x55e3ce396dc0 2024-04-24T06:43:18.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.395+0000 7fe399c00640 1 -- 172.21.15.5:0/1618685880 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3a0004520 con 0x55e3ce396dc0 2024-04-24T06:43:18.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.395+0000 7fe399c00640 1 -- 172.21.15.5:0/1618685880 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a001ec90 con 0x55e3ce396dc0 2024-04-24T06:43:18.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.396+0000 7fe3b2dd9740 1 -- 172.21.15.5:0/1618685880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ce396dc0 msgr2=0x55e3ce3a6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:18.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.396+0000 7fe3b2dd9740 1 --2- 172.21.15.5:0/1618685880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ce396dc0 0x55e3ce3a6e40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0005f40 tx=0x7fe3a0004a40 comp rx=0 tx=0).stop 2024-04-24T06:43:18.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.396+0000 7fe3b2dd9740 1 -- 172.21.15.5:0/1618685880 shutdown_connections 2024-04-24T06:43:18.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.396+0000 7fe3b2dd9740 1 --2- 172.21.15.5:0/1618685880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ce396dc0 0x55e3ce3a6e40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.396+0000 7fe3b2dd9740 1 --2- 172.21.15.5:0/1618685880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3ce391c50 0x55e3ce3a2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.396+0000 7fe3b2dd9740 1 --2- 172.21.15.5:0/1618685880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ce391260 0x55e3ce3991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.396+0000 7fe3b2dd9740 1 -- 172.21.15.5:0/1618685880 >> 172.21.15.5:0/1618685880 conn(0x55e3ce38c8d0 msgr2=0x55e3ce38ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:18.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.396+0000 7fe3b2dd9740 1 -- 172.21.15.5:0/1618685880 shutdown_connections 2024-04-24T06:43:18.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.396+0000 7fe3b2dd9740 1 -- 172.21.15.5:0/1618685880 wait complete. 2024-04-24T06:43:18.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.396+0000 7fe3b2dd9740 1 -- start start 2024-04-24T06:43:18.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.396+0000 7fe3b2dd9740 1 Processor -- start 2024-04-24T06:43:18.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.397+0000 7fe3b2dd9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ce391260 0x55e3ce3a0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:18.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.397+0000 7fe3b2dd9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3ce391c50 0x55e3ce40a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:18.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.397+0000 7fe3b2dd9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ce396dc0 0x55e3ce40c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:18.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.397+0000 7fe3b2dd9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e3ce3a9450 con 0x55e3ce391260 2024-04-24T06:43:18.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.397+0000 7fe3b2dd9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e3ce3a8d80 con 0x55e3ce396dc0 2024-04-24T06:43:18.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.397+0000 7fe3b2dd9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e3ce3a9b20 con 0x55e3ce391c50 2024-04-24T06:43:18.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.397+0000 7fe3b2dd9740 10 client.0 _ll_register_callbacks cb 0x7fe39c005a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:43:18.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.397+0000 7fe3ad400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ce391260 0x55e3ce3a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:18.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.397+0000 7fe3ad400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ce391260 0x55e3ce3a0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58818/0 (socket says 172.21.15.5:58818) 2024-04-24T06:43:18.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.397+0000 7fe3ad400640 1 -- 172.21.15.5:0/1569340086 learned_addr learned my addr 172.21.15.5:0/1569340086 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:18.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.397+0000 7fe3ade00640 1 --2- 172.21.15.5:0/1569340086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ce396dc0 0x55e3ce40c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:18.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.397+0000 7fe3aca00640 1 --2- 172.21.15.5:0/1569340086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3ce391c50 0x55e3ce40a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:18.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.398+0000 7fe3ad400640 1 -- 172.21.15.5:0/1569340086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3ce391c50 msgr2=0x55e3ce40a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:18.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.398+0000 7fe3ad400640 1 --2- 172.21.15.5:0/1569340086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3ce391c50 0x55e3ce40a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.398+0000 7fe3ad400640 1 -- 172.21.15.5:0/1569340086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ce396dc0 msgr2=0x55e3ce40c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:18.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.398+0000 7fe3ad400640 1 --2- 172.21.15.5:0/1569340086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ce396dc0 0x55e3ce40c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.398+0000 7fe3ad400640 1 -- 172.21.15.5:0/1569340086 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3ce40cdb0 con 0x55e3ce391260 2024-04-24T06:43:18.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.398+0000 7fe3ade00640 1 --2- 172.21.15.5:0/1569340086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ce396dc0 0x55e3ce40c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:43:18.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.398+0000 7fe3ad400640 1 --2- 172.21.15.5:0/1569340086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ce391260 0x55e3ce3a0900 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe39c035040 tx=0x7fe39c005330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a74069184e7bdf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:18.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.398+0000 7fe38ea00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:43:18.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.398+0000 7fe38ea00640 1 -- 172.21.15.5:0/1569340086 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c0bbf40 con 0x55e3ce391260 2024-04-24T06:43:18.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.398+0000 7fe38ea00640 1 -- 172.21.15.5:0/1569340086 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fe39c004340 con 0x55e3ce391260 2024-04-24T06:43:18.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.398+0000 7fe3b2dd9740 1 -- 172.21.15.5:0/1569340086 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3a0004520 con 0x55e3ce391260 2024-04-24T06:43:18.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.398+0000 7fe38ea00640 1 -- 172.21.15.5:0/1569340086 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fe39c0bc1d0 con 0x55e3ce391260 2024-04-24T06:43:18.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.398+0000 7fe3b2dd9740 10 client.14925 fetch_fsmap learned FSMap version 22 2024-04-24T06:43:18.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.398+0000 7fe3b2dd9740 1 -- 172.21.15.5:0/1569340086 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55e3ce40cdb0 con 0x55e3ce391260 2024-04-24T06:43:18.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.399+0000 7fe38ea00640 1 -- 172.21.15.5:0/1569340086 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3a0004520 con 0x55e3ce391260 2024-04-24T06:43:18.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.399+0000 7fe38ea00640 1 -- 172.21.15.5:0/1569340086 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fe39c0f3970 con 0x55e3ce391260 2024-04-24T06:43:18.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.399+0000 7fe3b2dd9740 10 client.14925 fetch_fsmap finished waiting for FSMap version 22 2024-04-24T06:43:18.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.399+0000 7fe3b2dd9740 10 client.14925 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T06:43:18.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.399+0000 7fe3b2dd9740 20 client.14925 populate_metadata read hostname 'smithi005' 2024-04-24T06:43:18.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.399+0000 7fe3b2dd9740 1 --2- 172.21.15.5:0/1569340086 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55e3ce412930 0x55e3ce40ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:18.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.399+0000 7fe3b2dd9740 4 client.14925 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T06:43:18.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.399+0000 7fe3b2dd9740 1 -- 172.21.15.5:0/1569340086 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "status"}) -- 0x55e3ce2fe3a0 con 0x55e3ce412930 2024-04-24T06:43:18.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.399+0000 7fe3aca00640 1 --2- 172.21.15.5:0/1569340086 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55e3ce412930 0x55e3ce40ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:18.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.400+0000 7fe3aca00640 1 --2- 172.21.15.5:0/1569340086 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55e3ce412930 0x55e3ce40ed70 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b56b899978821a27 server_cookie=5d48857a1345ab32 in_seq=0 out_seq=0 2024-04-24T06:43:18.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.400+0000 7fe38ea00640 10 client.14925 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:43:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.400+0000 7fe38ea00640 1 -- 172.21.15.5:0/1569340086 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55e3ce2fe3a0 con 0x55e3ce412930 2024-04-24T06:43:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.400+0000 7fe38ea00640 10 client.14925 handle_command_reply: tid=0 2024-04-24T06:43:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.400+0000 7fe3b2dd9740 1 client.14925 shutdown 2024-04-24T06:43:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.400+0000 7fe3b2dd9740 1 -- 172.21.15.5:0/1569340086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ce391260 msgr2=0x55e3ce3a0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.400+0000 7fe3b2dd9740 1 --2- 172.21.15.5:0/1569340086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ce391260 0x55e3ce3a0900 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe39c035040 tx=0x7fe39c005330 comp rx=0 tx=0).stop 2024-04-24T06:43:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.400+0000 7fe3b2dd9740 1 -- 172.21.15.5:0/1569340086 shutdown_connections 2024-04-24T06:43:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.400+0000 7fe3b2dd9740 1 --2- 172.21.15.5:0/1569340086 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55e3ce412930 0x55e3ce40ed70 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.400+0000 7fe3b2dd9740 1 --2- 172.21.15.5:0/1569340086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e3ce396dc0 0x55e3ce40c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.400+0000 7fe3b2dd9740 1 --2- 172.21.15.5:0/1569340086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3ce391c50 0x55e3ce40a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.400+0000 7fe3b2dd9740 1 --2- 172.21.15.5:0/1569340086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3ce391260 0x55e3ce3a0900 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.400+0000 7fe38ea00640 0 client.14925 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:43:18.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.400+0000 7fe3b2dd9740 1 -- 172.21.15.5:0/1569340086 >> 172.21.15.5:0/1569340086 conn(0x55e3ce38c8d0 msgr2=0x55e3ce3a6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:18.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.401+0000 7fe3b2dd9740 1 -- 172.21.15.5:0/1569340086 shutdown_connections 2024-04-24T06:43:18.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.401+0000 7fe3b2dd9740 1 -- 172.21.15.5:0/1569340086 wait complete. 2024-04-24T06:43:18.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.401+0000 7fe3b2dd9740 20 client.14925 trim_cache size 0 max 16384 2024-04-24T06:43:18.406 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:43:18.406 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "43e9bad0-0204-11ef-bc93-c7b262605968", 2024-04-24T06:43:18.406 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-04-24T06:43:18.407 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24569, 2024-04-24T06:43:18.407 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-04-24T06:43:18.407 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-04-24T06:43:18.407 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-04-24T06:43:18.407 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 102.963440495, 2024-04-24T06:43:18.407 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 22, 2024-04-24T06:43:18.407 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 88, 2024-04-24T06:43:18.407 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 86, 2024-04-24T06:43:18.407 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 110.649534776 2024-04-24T06:43:18.407 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:43:18.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.401+0000 7fe3ae800640 1 -- 172.21.15.5:0/2308103326 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fe3900a3c30 msgr2=0x7fe3900a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:18.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.401+0000 7fe3ae800640 1 --2- 172.21.15.5:0/2308103326 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fe3900a3c30 0x7fe3900a60f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x55e3ce3924b0 tx=0x7fe3a0009500 comp rx=0 tx=0).stop 2024-04-24T06:43:18.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.401+0000 7fe3ae800640 1 -- 172.21.15.5:0/2308103326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3a8108480 msgr2=0x7fe3a819e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:18.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.401+0000 7fe3ae800640 1 --2- 172.21.15.5:0/2308103326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3a8108480 0x7fe3a819e7c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe39c02a860 tx=0x7fe39c033d90 comp rx=0 tx=0).stop 2024-04-24T06:43:18.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.402+0000 7fe3ae800640 1 -- 172.21.15.5:0/2308103326 shutdown_connections 2024-04-24T06:43:18.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.402+0000 7fe3ae800640 1 --2- 172.21.15.5:0/2308103326 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fe3900a3c30 0x7fe3900a60f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.402+0000 7fe3ae800640 1 --2- 172.21.15.5:0/2308103326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3a810a030 0x7fe3a81a5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.402+0000 7fe3ae800640 1 --2- 172.21.15.5:0/2308103326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3a8109670 0x7fe3a819ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.402+0000 7fe3ae800640 1 --2- 172.21.15.5:0/2308103326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3a8108480 0x7fe3a819e7c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:18.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.402+0000 7fe3ae800640 1 -- 172.21.15.5:0/2308103326 >> 172.21.15.5:0/2308103326 conn(0x7fe3a8077440 msgr2=0x7fe3a810c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:18.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.402+0000 7fe3ae800640 1 -- 172.21.15.5:0/2308103326 shutdown_connections 2024-04-24T06:43:18.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:18.402+0000 7fe3ae800640 1 -- 172.21.15.5:0/2308103326 wait complete. 2024-04-24T06:43:18.421 INFO:tasks.cephadm:Setup complete, yielding 2024-04-24T06:43:18.421 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T06:43:18.434 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-04-24T06:43:18.434 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-24T06:43:18.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:18 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/289446811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:43:18.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:18 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/289446811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:43:18.894 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:43:19.378 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:19 smithi005 ceph-mon[27798]: pgmap v366: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T06:43:19.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:19 smithi137 ceph-mon[33017]: pgmap v366: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T06:43:19.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:19 smithi037 ceph-mon[32864]: pgmap v366: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-24T06:43:21.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.135+0000 7f3006000700 1 -- 172.21.15.5:0/2034096745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30000790b0 msgr2=0x7f3000079930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:21.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.135+0000 7f3006000700 1 --2- 172.21.15.5:0/2034096745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30000790b0 0x7f3000079930 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2ff800b0a0 tx=0x7f2ff800b3b0 comp rx=0 tx=0).stop 2024-04-24T06:43:21.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.135+0000 7f3006000700 1 -- 172.21.15.5:0/2034096745 shutdown_connections 2024-04-24T06:43:21.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.135+0000 7f3006000700 1 --2- 172.21.15.5:0/2034096745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30000790b0 0x7f3000079930 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:21.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.135+0000 7f3006000700 1 --2- 172.21.15.5:0/2034096745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30000774a0 0x7f30000778c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:21.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.135+0000 7f3006000700 1 --2- 172.21.15.5:0/2034096745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30000786f0 0x7f3000078b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:21.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.135+0000 7f3006000700 1 -- 172.21.15.5:0/2034096745 >> 172.21.15.5:0/2034096745 conn(0x7f3000100210 msgr2=0x7f3000102670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:21.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.135+0000 7f3006000700 1 -- 172.21.15.5:0/2034096745 shutdown_connections 2024-04-24T06:43:21.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.135+0000 7f3006000700 1 -- 172.21.15.5:0/2034096745 wait complete. 2024-04-24T06:43:21.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.136+0000 7f3006000700 1 Processor -- start 2024-04-24T06:43:21.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.136+0000 7f3006000700 1 -- start start 2024-04-24T06:43:21.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.136+0000 7f3006000700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30000774a0 0x7f300019f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:21.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.136+0000 7f3006000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30000786f0 0x7f300019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.136+0000 7f3006000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30000790b0 0x7f30001a6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.136+0000 7f3006000700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3000114c40 con 0x7f30000774a0 2024-04-24T06:43:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.136+0000 7f3006000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3000114ac0 con 0x7f30000790b0 2024-04-24T06:43:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.136+0000 7f3006000700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3000114dc0 con 0x7f30000786f0 2024-04-24T06:43:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.137+0000 7f3004c00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30000774a0 0x7f300019f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.137+0000 7f3004c00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30000774a0 0x7f300019f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58830/0 (socket says 172.21.15.5:58830) 2024-04-24T06:43:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.137+0000 7f3004c00700 1 -- 172.21.15.5:0/3920317564 learned_addr learned my addr 172.21.15.5:0/3920317564 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.137+0000 7f2fffe00700 1 --2- 172.21.15.5:0/3920317564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30000786f0 0x7f300019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.137+0000 7f3005600700 1 --2- 172.21.15.5:0/3920317564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30000790b0 0x7f30001a6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.137+0000 7f3004c00700 1 -- 172.21.15.5:0/3920317564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30000786f0 msgr2=0x7f300019f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.137+0000 7f3004c00700 1 --2- 172.21.15.5:0/3920317564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30000786f0 0x7f300019f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.137+0000 7f3004c00700 1 -- 172.21.15.5:0/3920317564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30000790b0 msgr2=0x7f30001a6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.137+0000 7f3004c00700 1 --2- 172.21.15.5:0/3920317564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30000790b0 0x7f30001a6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.137+0000 7f3004c00700 1 -- 172.21.15.5:0/3920317564 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30001a6da0 con 0x7f30000774a0 2024-04-24T06:43:21.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.138+0000 7f3004c00700 1 --2- 172.21.15.5:0/3920317564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30000774a0 0x7f300019f190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2ff400d9b0 tx=0x7f2ff400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:21.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.138+0000 7f2ffd600700 1 -- 172.21.15.5:0/3920317564 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4016070 con 0x7f30000774a0 2024-04-24T06:43:21.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.138+0000 7f2ffd600700 1 -- 172.21.15.5:0/3920317564 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ff4004d10 con 0x7f30000774a0 2024-04-24T06:43:21.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.138+0000 7f3006000700 1 -- 172.21.15.5:0/3920317564 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30001a7090 con 0x7f30000774a0 2024-04-24T06:43:21.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.138+0000 7f2ffd600700 1 -- 172.21.15.5:0/3920317564 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4005530 con 0x7f30000774a0 2024-04-24T06:43:21.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.138+0000 7f3006000700 1 -- 172.21.15.5:0/3920317564 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30001a7620 con 0x7f30000774a0 2024-04-24T06:43:21.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.139+0000 7f3006000700 1 -- 172.21.15.5:0/3920317564 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30000697e0 con 0x7f30000774a0 2024-04-24T06:43:21.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.141+0000 7f2ffd600700 1 -- 172.21.15.5:0/3920317564 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f2ff4004750 con 0x7f30000774a0 2024-04-24T06:43:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.142+0000 7f2ffd600700 1 --2- 172.21.15.5:0/3920317564 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f2fe80a3cc0 0x7f2fe80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.143+0000 7f2ffd600700 1 -- 172.21.15.5:0/3920317564 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f2ff40d01b0 con 0x7f30000774a0 2024-04-24T06:43:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.143+0000 7f2fffe00700 1 --2- 172.21.15.5:0/3920317564 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f2fe80a3cc0 0x7f2fe80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.144+0000 7f2fffe00700 1 --2- 172.21.15.5:0/3920317564 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f2fe80a3cc0 0x7f2fe80a6180 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2fec009be0 tx=0x7f2fec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:21.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.147+0000 7f2ffd600700 1 -- 172.21.15.5:0/3920317564 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2ff401f020 con 0x7f30000774a0 2024-04-24T06:43:21.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:21 smithi005 ceph-mon[27798]: pgmap v367: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-24T06:43:21.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.398+0000 7f3006000700 1 -- 172.21.15.5:0/3920317564 --> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f3000062f30 con 0x7f2fe80a3cc0 2024-04-24T06:43:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.401+0000 7f2ffd600700 1 -- 172.21.15.5:0/3920317564 <== mgr.14152 v2:172.21.15.5:6800/4042936965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f3000062f30 con 0x7f2fe80a3cc0 2024-04-24T06:43:21.402 INFO:teuthology.orchestra.run.smithi005.stdout:Backend: cephadm 2024-04-24T06:43:21.402 INFO:teuthology.orchestra.run.smithi005.stdout:Available: Yes 2024-04-24T06:43:21.402 INFO:teuthology.orchestra.run.smithi005.stdout:Paused: No 2024-04-24T06:43:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.404+0000 7f3006000700 1 -- 172.21.15.5:0/3920317564 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f2fe80a3cc0 msgr2=0x7f2fe80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.404+0000 7f3006000700 1 --2- 172.21.15.5:0/3920317564 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f2fe80a3cc0 0x7f2fe80a6180 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2fec009be0 tx=0x7f2fec009260 comp rx=0 tx=0).stop 2024-04-24T06:43:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.404+0000 7f3006000700 1 -- 172.21.15.5:0/3920317564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30000774a0 msgr2=0x7f300019f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.404+0000 7f3006000700 1 --2- 172.21.15.5:0/3920317564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30000774a0 0x7f300019f190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2ff400d9b0 tx=0x7f2ff400dcc0 comp rx=0 tx=0).stop 2024-04-24T06:43:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.404+0000 7f3006000700 1 -- 172.21.15.5:0/3920317564 shutdown_connections 2024-04-24T06:43:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.404+0000 7f3006000700 1 --2- 172.21.15.5:0/3920317564 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f2fe80a3cc0 0x7f2fe80a6180 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.404+0000 7f3006000700 1 --2- 172.21.15.5:0/3920317564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30000790b0 0x7f30001a6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.404+0000 7f3006000700 1 --2- 172.21.15.5:0/3920317564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30000774a0 0x7f300019f190 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.404+0000 7f3006000700 1 --2- 172.21.15.5:0/3920317564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30000786f0 0x7f300019f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.404+0000 7f3006000700 1 -- 172.21.15.5:0/3920317564 >> 172.21.15.5:0/3920317564 conn(0x7f3000100210 msgr2=0x7f3000101d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.404+0000 7f3006000700 1 -- 172.21.15.5:0/3920317564 shutdown_connections 2024-04-24T06:43:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:21.404+0000 7f3006000700 1 -- 172.21.15.5:0/3920317564 wait complete. 2024-04-24T06:43:21.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:21 smithi137 ceph-mon[33017]: pgmap v367: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-24T06:43:21.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:21 smithi037 ceph-mon[32864]: pgmap v367: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-24T06:43:22.568 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-24T06:43:23.025 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:43:23.367 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:23 smithi005 ceph-mon[27798]: pgmap v368: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T06:43:23.367 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:23 smithi005 ceph-mon[27798]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:43:23.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:23 smithi137 ceph-mon[33017]: pgmap v368: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T06:43:23.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:23 smithi137 ceph-mon[33017]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:43:23.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:23 smithi037 ceph-mon[32864]: pgmap v368: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T06:43:23.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:23 smithi037 ceph-mon[32864]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:43:25.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.334+0000 7f3810200700 1 -- 172.21.15.5:0/3815390549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3808077ad0 msgr2=0x7f3808079930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:25.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.334+0000 7f3810200700 1 --2- 172.21.15.5:0/3815390549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3808077ad0 0x7f3808079930 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f380000b0a0 tx=0x7f380000b3b0 comp rx=0 tx=0).stop 2024-04-24T06:43:25.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.335+0000 7f3810200700 1 -- 172.21.15.5:0/3815390549 shutdown_connections 2024-04-24T06:43:25.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.335+0000 7f3810200700 1 --2- 172.21.15.5:0/3815390549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3808077ad0 0x7f3808079930 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:25.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.335+0000 7f3810200700 1 --2- 172.21.15.5:0/3815390549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3808078b80 0x7f3808078fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:25.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.335+0000 7f3810200700 1 --2- 172.21.15.5:0/3815390549 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3808077110 0x7f3808077590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:25.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.335+0000 7f3810200700 1 -- 172.21.15.5:0/3815390549 >> 172.21.15.5:0/3815390549 conn(0x7f3808100340 msgr2=0x7f3808102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:25.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.335+0000 7f3810200700 1 -- 172.21.15.5:0/3815390549 shutdown_connections 2024-04-24T06:43:25.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.335+0000 7f3810200700 1 -- 172.21.15.5:0/3815390549 wait complete. 2024-04-24T06:43:25.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.336+0000 7f3810200700 1 Processor -- start 2024-04-24T06:43:25.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.336+0000 7f3810200700 1 -- start start 2024-04-24T06:43:25.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.336+0000 7f3810200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3808077110 0x7f380819f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:25.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.336+0000 7f3810200700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3808077ad0 0x7f380819f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:25.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.336+0000 7f3810200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3808078b80 0x7f38081a69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:25.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.336+0000 7f3810200700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3808114ec0 con 0x7f3808077ad0 2024-04-24T06:43:25.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.336+0000 7f3810200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3808114d40 con 0x7f3808078b80 2024-04-24T06:43:25.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.336+0000 7f3810200700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3808115040 con 0x7f3808077110 2024-04-24T06:43:25.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.337+0000 7f380e200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3808078b80 0x7f38081a69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:25.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.337+0000 7f380e200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3808078b80 0x7f38081a69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:58882/0 (socket says 172.21.15.5:58882) 2024-04-24T06:43:25.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.337+0000 7f380e200700 1 -- 172.21.15.5:0/3839255064 learned_addr learned my addr 172.21.15.5:0/3839255064 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:25.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.337+0000 7f380d800700 1 --2- 172.21.15.5:0/3839255064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3808077110 0x7f380819f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:25.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.337+0000 7f380ce00700 1 --2- 172.21.15.5:0/3839255064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3808077ad0 0x7f380819f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:25.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.337+0000 7f380e200700 1 -- 172.21.15.5:0/3839255064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3808077110 msgr2=0x7f380819f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:25.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.337+0000 7f380e200700 1 --2- 172.21.15.5:0/3839255064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3808077110 0x7f380819f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:25.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.337+0000 7f380e200700 1 -- 172.21.15.5:0/3839255064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3808077ad0 msgr2=0x7f380819f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:25.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.337+0000 7f380e200700 1 --2- 172.21.15.5:0/3839255064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3808077ad0 0x7f380819f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:25.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.337+0000 7f380e200700 1 -- 172.21.15.5:0/3839255064 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38081a7080 con 0x7f3808078b80 2024-04-24T06:43:25.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.337+0000 7f380e200700 1 --2- 172.21.15.5:0/3839255064 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3808078b80 0x7f38081a69e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3800009bc0 tx=0x7f3800004c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:25.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.338+0000 7f3806000700 1 -- 172.21.15.5:0/3839255064 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3800021070 con 0x7f3808078b80 2024-04-24T06:43:25.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.338+0000 7f3806000700 1 -- 172.21.15.5:0/3839255064 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3800007b80 con 0x7f3808078b80 2024-04-24T06:43:25.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.338+0000 7f3810200700 1 -- 172.21.15.5:0/3839255064 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38081a7310 con 0x7f3808078b80 2024-04-24T06:43:25.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.338+0000 7f3806000700 1 -- 172.21.15.5:0/3839255064 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38000048a0 con 0x7f3808078b80 2024-04-24T06:43:25.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.338+0000 7f3810200700 1 -- 172.21.15.5:0/3839255064 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38081a7810 con 0x7f3808078b80 2024-04-24T06:43:25.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.339+0000 7f3810200700 1 -- 172.21.15.5:0/3839255064 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38080697e0 con 0x7f3808078b80 2024-04-24T06:43:25.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.341+0000 7f3806000700 1 -- 172.21.15.5:0/3839255064 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f3800003a00 con 0x7f3808078b80 2024-04-24T06:43:25.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.343+0000 7f3806000700 1 --2- 172.21.15.5:0/3839255064 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f37f00a3e60 0x7f37f00a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:25.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.343+0000 7f3806000700 1 -- 172.21.15.5:0/3839255064 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f380000f020 con 0x7f3808078b80 2024-04-24T06:43:25.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.343+0000 7f380d800700 1 --2- 172.21.15.5:0/3839255064 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f37f00a3e60 0x7f37f00a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:25.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.344+0000 7f380d800700 1 --2- 172.21.15.5:0/3839255064 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f37f00a3e60 0x7f37f00a6320 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f37fc009680 tx=0x7f37fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:25.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.347+0000 7f3806000700 1 -- 172.21.15.5:0/3839255064 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f380002a020 con 0x7f3808078b80 2024-04-24T06:43:25.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:25 smithi137 ceph-mon[33017]: pgmap v369: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T06:43:25.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:25 smithi005 ceph-mon[27798]: pgmap v369: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T06:43:25.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:25 smithi037 ceph-mon[32864]: pgmap v369: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-24T06:43:25.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.588+0000 7f3810200700 1 -- 172.21.15.5:0/3839255064 --> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f3808062f30 con 0x7f37f00a3e60 2024-04-24T06:43:25.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.611+0000 7f3806000700 1 -- 172.21.15.5:0/3839255064 <== mgr.14152 v2:172.21.15.5:6800/4042936965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f3808062f30 con 0x7f37f00a3e60 2024-04-24T06:43:25.613 INFO:teuthology.orchestra.run.smithi005.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T06:43:25.613 INFO:teuthology.orchestra.run.smithi005.stdout:mds.a smithi005 running (2m) 25s ago 2m 12.3M - 19.0.0-3124-ga3c8a22b b5c3183062a6 0acdc9dffbd4 2024-04-24T06:43:25.613 INFO:teuthology.orchestra.run.smithi005.stdout:mds.b smithi037 running (118s) 49s ago 117s 25.6M - 19.0.0-3124-ga3c8a22b b5c3183062a6 fd4020104cab 2024-04-24T06:43:25.613 INFO:teuthology.orchestra.run.smithi005.stdout:mds.c smithi137 running (2m) 49s ago 2m 17.6M - 19.0.0-3124-ga3c8a22b b5c3183062a6 2a46530d84e8 2024-04-24T06:43:25.614 INFO:teuthology.orchestra.run.smithi005.stdout:mds.d smithi005 running (2m) 25s ago 2m 12.5M - 19.0.0-3124-ga3c8a22b b5c3183062a6 0b6768e60a11 2024-04-24T06:43:25.614 INFO:teuthology.orchestra.run.smithi005.stdout:mds.e smithi037 running (2m) 49s ago 2m 15.5M - 19.0.0-3124-ga3c8a22b b5c3183062a6 c97d3e315a8d 2024-04-24T06:43:25.614 INFO:teuthology.orchestra.run.smithi005.stdout:mds.f smithi137 running (2m) 49s ago 2m 18.4M - 19.0.0-3124-ga3c8a22b b5c3183062a6 8a32b4df1c7f 2024-04-24T06:43:25.614 INFO:teuthology.orchestra.run.smithi005.stdout:mds.g smithi005 running (2m) 25s ago 2m 16.7M - 19.0.0-3124-ga3c8a22b b5c3183062a6 fa9159430b13 2024-04-24T06:43:25.614 INFO:teuthology.orchestra.run.smithi005.stdout:mds.h smithi037 running (2m) 49s ago 2m 16.9M - 19.0.0-3124-ga3c8a22b b5c3183062a6 3aaa91ad5018 2024-04-24T06:43:25.614 INFO:teuthology.orchestra.run.smithi005.stdout:mds.i smithi137 running (2m) 49s ago 2m 19.8M - 19.0.0-3124-ga3c8a22b b5c3183062a6 65c868131cb6 2024-04-24T06:43:25.614 INFO:teuthology.orchestra.run.smithi005.stdout:mds.j smithi005 running (2m) 25s ago 2m 12.5M - 19.0.0-3124-ga3c8a22b b5c3183062a6 708d1ca029d0 2024-04-24T06:43:25.614 INFO:teuthology.orchestra.run.smithi005.stdout:mds.k smithi037 running (2m) 49s ago 2m 13.9M - 19.0.0-3124-ga3c8a22b b5c3183062a6 2d48a66b6274 2024-04-24T06:43:25.614 INFO:teuthology.orchestra.run.smithi005.stdout:mds.l smithi137 running (2m) 49s ago 2m 16.6M - 19.0.0-3124-ga3c8a22b b5c3183062a6 1efb8334f844 2024-04-24T06:43:25.614 INFO:teuthology.orchestra.run.smithi005.stdout:mgr.x smithi005 *:9283,8765 running (10m) 25s ago 10m 515M - 19.0.0-3124-ga3c8a22b b5c3183062a6 96ec027a3df5 2024-04-24T06:43:25.614 INFO:teuthology.orchestra.run.smithi005.stdout:mgr.y smithi037 *:8443,8765 running (8m) 49s ago 8m 454M - 19.0.0-3124-ga3c8a22b b5c3183062a6 9a3b3e8361ba 2024-04-24T06:43:25.615 INFO:teuthology.orchestra.run.smithi005.stdout:mgr.z smithi137 *:8443,8765 running (8m) 49s ago 8m 452M - 19.0.0-3124-ga3c8a22b b5c3183062a6 1b2a03d1f584 2024-04-24T06:43:25.615 INFO:teuthology.orchestra.run.smithi005.stdout:mon.a smithi005 running (10m) 25s ago 10m 49.4M 2048M 19.0.0-3124-ga3c8a22b b5c3183062a6 a5810a88074d 2024-04-24T06:43:25.615 INFO:teuthology.orchestra.run.smithi005.stdout:mon.b smithi037 running (8m) 49s ago 8m 45.9M 2048M 19.0.0-3124-ga3c8a22b b5c3183062a6 68e4b241a9f4 2024-04-24T06:43:25.615 INFO:teuthology.orchestra.run.smithi005.stdout:mon.c smithi137 running (8m) 49s ago 8m 39.9M 2048M 19.0.0-3124-ga3c8a22b b5c3183062a6 15c232db705d 2024-04-24T06:43:25.615 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0 smithi005 running (7m) 25s ago 7m 54.3M 4273M 19.0.0-3124-ga3c8a22b b5c3183062a6 7e96d6f9ef11 2024-04-24T06:43:25.615 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1 smithi037 running (7m) 49s ago 7m 56.0M 4273M 19.0.0-3124-ga3c8a22b b5c3183062a6 a365fa714acf 2024-04-24T06:43:25.615 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2 smithi137 running (7m) 49s ago 7m 56.1M 4272M 19.0.0-3124-ga3c8a22b b5c3183062a6 959703a0b51f 2024-04-24T06:43:25.615 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3 smithi005 running (6m) 25s ago 6m 45.7M 4273M 19.0.0-3124-ga3c8a22b b5c3183062a6 fdcb99b527f2 2024-04-24T06:43:25.615 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4 smithi037 running (6m) 49s ago 6m 43.9M 4273M 19.0.0-3124-ga3c8a22b b5c3183062a6 e3c22d9f3672 2024-04-24T06:43:25.615 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5 smithi137 running (5m) 49s ago 5m 55.1M 4272M 19.0.0-3124-ga3c8a22b b5c3183062a6 1b0e660e6aca 2024-04-24T06:43:25.615 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6 smithi005 running (5m) 25s ago 5m 44.0M 4273M 19.0.0-3124-ga3c8a22b b5c3183062a6 56254981f479 2024-04-24T06:43:25.615 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7 smithi037 running (4m) 49s ago 4m 47.1M 4273M 19.0.0-3124-ga3c8a22b b5c3183062a6 689aa2f5855e 2024-04-24T06:43:25.615 INFO:teuthology.orchestra.run.smithi005.stdout:osd.8 smithi137 running (4m) 49s ago 4m 47.7M 4272M 19.0.0-3124-ga3c8a22b b5c3183062a6 875574a3714d 2024-04-24T06:43:25.615 INFO:teuthology.orchestra.run.smithi005.stdout:osd.9 smithi005 running (3m) 25s ago 3m 49.0M 4273M 19.0.0-3124-ga3c8a22b b5c3183062a6 fb4abfa84440 2024-04-24T06:43:25.616 INFO:teuthology.orchestra.run.smithi005.stdout:osd.10 smithi037 running (3m) 49s ago 3m 48.9M 4273M 19.0.0-3124-ga3c8a22b b5c3183062a6 4321db2367ce 2024-04-24T06:43:25.616 INFO:teuthology.orchestra.run.smithi005.stdout:osd.11 smithi137 running (2m) 49s ago 2m 45.4M 4272M 19.0.0-3124-ga3c8a22b b5c3183062a6 74c111b171bc 2024-04-24T06:43:25.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.614+0000 7f3810200700 1 -- 172.21.15.5:0/3839255064 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f37f00a3e60 msgr2=0x7f37f00a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:25.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.614+0000 7f3810200700 1 --2- 172.21.15.5:0/3839255064 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f37f00a3e60 0x7f37f00a6320 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f37fc009680 tx=0x7f37fc009260 comp rx=0 tx=0).stop 2024-04-24T06:43:25.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.614+0000 7f3810200700 1 -- 172.21.15.5:0/3839255064 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3808078b80 msgr2=0x7f38081a69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:25.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.614+0000 7f3810200700 1 --2- 172.21.15.5:0/3839255064 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3808078b80 0x7f38081a69e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3800009bc0 tx=0x7f3800004c60 comp rx=0 tx=0).stop 2024-04-24T06:43:25.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.614+0000 7f3810200700 1 -- 172.21.15.5:0/3839255064 shutdown_connections 2024-04-24T06:43:25.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.614+0000 7f3810200700 1 --2- 172.21.15.5:0/3839255064 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f37f00a3e60 0x7f37f00a6320 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:25.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.614+0000 7f3810200700 1 --2- 172.21.15.5:0/3839255064 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3808078b80 0x7f38081a69e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:25.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.614+0000 7f3810200700 1 --2- 172.21.15.5:0/3839255064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3808077110 0x7f380819f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:25.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.614+0000 7f3810200700 1 --2- 172.21.15.5:0/3839255064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3808077ad0 0x7f380819f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:25.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.614+0000 7f3810200700 1 -- 172.21.15.5:0/3839255064 >> 172.21.15.5:0/3839255064 conn(0x7f3808100340 msgr2=0x7f3808101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:25.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.615+0000 7f3810200700 1 -- 172.21.15.5:0/3839255064 shutdown_connections 2024-04-24T06:43:25.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:25.615+0000 7f3810200700 1 -- 172.21.15.5:0/3839255064 wait complete. 2024-04-24T06:43:26.430 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-24T06:43:26.900 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:43:27.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:27 smithi005 ceph-mon[27798]: pgmap v370: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:43:27.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:27 smithi005 ceph-mon[27798]: from='client.25031 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:43:27.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:27 smithi137 ceph-mon[33017]: pgmap v370: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:43:27.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:27 smithi137 ceph-mon[33017]: from='client.25031 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:43:27.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:27 smithi037 ceph-mon[32864]: pgmap v370: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:43:27.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:27 smithi037 ceph-mon[32864]: from='client.25031 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:43:29.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.166+0000 7fce0c000700 1 -- 172.21.15.5:0/3014931448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce0410a390 msgr2=0x7fce0410a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:29.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.166+0000 7fce0c000700 1 --2- 172.21.15.5:0/3014931448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce0410a390 0x7fce0410a7f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcdf00099c0 tx=0x7fcdf0009cd0 comp rx=0 tx=0).stop 2024-04-24T06:43:29.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.166+0000 7fce0c000700 1 -- 172.21.15.5:0/3014931448 shutdown_connections 2024-04-24T06:43:29.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.166+0000 7fce0c000700 1 --2- 172.21.15.5:0/3014931448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce0410ad30 0x7fce041115e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:29.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.166+0000 7fce0c000700 1 --2- 172.21.15.5:0/3014931448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce04109190 0x7fce041095b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:29.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.166+0000 7fce0c000700 1 --2- 172.21.15.5:0/3014931448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce0410a390 0x7fce0410a7f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:29.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.166+0000 7fce0c000700 1 -- 172.21.15.5:0/3014931448 >> 172.21.15.5:0/3014931448 conn(0x7fce04078160 msgr2=0x7fce0407a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:29.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.166+0000 7fce0c000700 1 -- 172.21.15.5:0/3014931448 shutdown_connections 2024-04-24T06:43:29.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.166+0000 7fce0c000700 1 -- 172.21.15.5:0/3014931448 wait complete. 2024-04-24T06:43:29.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.167+0000 7fce0c000700 1 Processor -- start 2024-04-24T06:43:29.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.167+0000 7fce0c000700 1 -- start start 2024-04-24T06:43:29.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.168+0000 7fce0c000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce04109190 0x7fce0419f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:29.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.168+0000 7fce0c000700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce0410a390 0x7fce0419f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:29.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.168+0000 7fce0c000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce0410ad30 0x7fce041a6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:29.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.168+0000 7fce0c000700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fce04114da0 con 0x7fce0410a390 2024-04-24T06:43:29.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.168+0000 7fce0c000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fce04114c20 con 0x7fce0410ad30 2024-04-24T06:43:29.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.168+0000 7fce0c000700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fce04114f20 con 0x7fce04109190 2024-04-24T06:43:29.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.168+0000 7fce08c00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce0410a390 0x7fce0419f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:29.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.168+0000 7fce09600700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce04109190 0x7fce0419f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:29.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.168+0000 7fce09600700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce04109190 0x7fce0419f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:40226/0 (socket says 172.21.15.5:40226) 2024-04-24T06:43:29.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.168+0000 7fce09600700 1 -- 172.21.15.5:0/3544996045 learned_addr learned my addr 172.21.15.5:0/3544996045 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:29.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.168+0000 7fce0a000700 1 --2- 172.21.15.5:0/3544996045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce0410ad30 0x7fce041a6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:29.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.168+0000 7fce08c00700 1 -- 172.21.15.5:0/3544996045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce04109190 msgr2=0x7fce0419f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:29.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.168+0000 7fce08c00700 1 --2- 172.21.15.5:0/3544996045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce04109190 0x7fce0419f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:29.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.168+0000 7fce08c00700 1 -- 172.21.15.5:0/3544996045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce0410ad30 msgr2=0x7fce041a6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:29.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.168+0000 7fce08c00700 1 --2- 172.21.15.5:0/3544996045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce0410ad30 0x7fce041a6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:29.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.168+0000 7fce08c00700 1 -- 172.21.15.5:0/3544996045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce041a6fb0 con 0x7fce0410a390 2024-04-24T06:43:29.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.169+0000 7fce0a000700 1 --2- 172.21.15.5:0/3544996045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce0410ad30 0x7fce041a6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:43:29.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.169+0000 7fce08c00700 1 --2- 172.21.15.5:0/3544996045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce0410a390 0x7fce0419f890 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcdf00099c0 tx=0x7fcdf000bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:29.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.169+0000 7fce02000700 1 -- 172.21.15.5:0/3544996045 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0009e60 con 0x7fce0410a390 2024-04-24T06:43:29.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.169+0000 7fce0c000700 1 -- 172.21.15.5:0/3544996045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce041a7240 con 0x7fce0410a390 2024-04-24T06:43:29.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.169+0000 7fce02000700 1 -- 172.21.15.5:0/3544996045 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdf0003e30 con 0x7fce0410a390 2024-04-24T06:43:29.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.169+0000 7fce02000700 1 -- 172.21.15.5:0/3544996045 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf00173f0 con 0x7fce0410a390 2024-04-24T06:43:29.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.169+0000 7fce0c000700 1 -- 172.21.15.5:0/3544996045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce041a76f0 con 0x7fce0410a390 2024-04-24T06:43:29.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.170+0000 7fce0c000700 1 -- 172.21.15.5:0/3544996045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce040697e0 con 0x7fce0410a390 2024-04-24T06:43:29.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.173+0000 7fce02000700 1 -- 172.21.15.5:0/3544996045 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7fcdf000f580 con 0x7fce0410a390 2024-04-24T06:43:29.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.174+0000 7fce02000700 1 --2- 172.21.15.5:0/3544996045 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fcdec0a3c40 0x7fcdec0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:29.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.174+0000 7fce02000700 1 -- 172.21.15.5:0/3544996045 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fcdf00d11c0 con 0x7fce0410a390 2024-04-24T06:43:29.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.174+0000 7fce09600700 1 --2- 172.21.15.5:0/3544996045 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fcdec0a3c40 0x7fcdec0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:29.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.175+0000 7fce09600700 1 --2- 172.21.15.5:0/3544996045 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fcdec0a3c40 0x7fcdec0a6100 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8005950 tx=0x7fcdf8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:29.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.179+0000 7fce02000700 1 -- 172.21.15.5:0/3544996045 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcdf0013070 con 0x7fce0410a390 2024-04-24T06:43:29.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.416+0000 7fce0c000700 1 -- 172.21.15.5:0/3544996045 --> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fce04062f30 con 0x7fcdec0a3c40 2024-04-24T06:43:29.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:29 smithi005 ceph-mon[27798]: pgmap v371: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:43:29.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.432+0000 7fce02000700 1 -- 172.21.15.5:0/3544996045 <== mgr.14152 v2:172.21.15.5:6800/4042936965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fce04062f30 con 0x7fcdec0a3c40 2024-04-24T06:43:29.433 INFO:teuthology.orchestra.run.smithi005.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-24T06:43:29.434 INFO:teuthology.orchestra.run.smithi005.stdout:mds.all 12/12 53s ago 2m smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi037=b;smithi037=e;smithi037=h;smithi037=k;smithi137=c;smithi137=f;smithi137=i;smithi137=l;count:12 2024-04-24T06:43:29.434 INFO:teuthology.orchestra.run.smithi005.stdout:mgr 3/3 53s ago 8m smithi005=x;smithi037=y;smithi137=z;count:3 2024-04-24T06:43:29.434 INFO:teuthology.orchestra.run.smithi005.stdout:mon 3/3 53s ago 8m smithi005:172.21.15.5=a;smithi037:172.21.15.37=b;smithi137:172.21.15.137=c;count:3 2024-04-24T06:43:29.434 INFO:teuthology.orchestra.run.smithi005.stdout:osd 12 53s ago - 2024-04-24T06:43:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.435+0000 7fce0c000700 1 -- 172.21.15.5:0/3544996045 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fcdec0a3c40 msgr2=0x7fcdec0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.435+0000 7fce0c000700 1 --2- 172.21.15.5:0/3544996045 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fcdec0a3c40 0x7fcdec0a6100 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8005950 tx=0x7fcdf8009260 comp rx=0 tx=0).stop 2024-04-24T06:43:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.435+0000 7fce0c000700 1 -- 172.21.15.5:0/3544996045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce0410a390 msgr2=0x7fce0419f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.435+0000 7fce0c000700 1 --2- 172.21.15.5:0/3544996045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce0410a390 0x7fce0419f890 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcdf00099c0 tx=0x7fcdf000bee0 comp rx=0 tx=0).stop 2024-04-24T06:43:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.435+0000 7fce0c000700 1 -- 172.21.15.5:0/3544996045 shutdown_connections 2024-04-24T06:43:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.435+0000 7fce0c000700 1 --2- 172.21.15.5:0/3544996045 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fcdec0a3c40 0x7fcdec0a6100 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.435+0000 7fce0c000700 1 --2- 172.21.15.5:0/3544996045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce0410ad30 0x7fce041a6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.435+0000 7fce0c000700 1 --2- 172.21.15.5:0/3544996045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce04109190 0x7fce0419f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.435+0000 7fce0c000700 1 --2- 172.21.15.5:0/3544996045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce0410a390 0x7fce0419f890 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.435+0000 7fce0c000700 1 -- 172.21.15.5:0/3544996045 >> 172.21.15.5:0/3544996045 conn(0x7fce04078160 msgr2=0x7fce0410d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.436+0000 7fce0c000700 1 -- 172.21.15.5:0/3544996045 shutdown_connections 2024-04-24T06:43:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:29.436+0000 7fce0c000700 1 -- 172.21.15.5:0/3544996045 wait complete. 2024-04-24T06:43:29.445 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:29 smithi137 ceph-mon[33017]: pgmap v371: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:43:29.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:29 smithi037 ceph-mon[32864]: pgmap v371: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:43:30.259 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-24T06:43:30.724 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:43:31.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:31 smithi137 ceph-mon[33017]: pgmap v372: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T06:43:31.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:31 smithi137 ceph-mon[33017]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:43:31.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:31 smithi005 ceph-mon[27798]: pgmap v372: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T06:43:31.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:31 smithi005 ceph-mon[27798]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:43:31.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:31 smithi037 ceph-mon[32864]: pgmap v372: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T06:43:31.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:31 smithi037 ceph-mon[32864]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:43:32.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:32.842+0000 7f10c8600700 1 -- 172.21.15.5:0/3195974290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10c0105f10 msgr2=0x7f10c0108300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:32.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:32.842+0000 7f10c8600700 1 --2- 172.21.15.5:0/3195974290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10c0105f10 0x7f10c0108300 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f10ac009990 tx=0x7f10ac009ca0 comp rx=0 tx=0).stop 2024-04-24T06:43:32.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:32.842+0000 7f10c8600700 1 -- 172.21.15.5:0/3195974290 shutdown_connections 2024-04-24T06:43:32.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:32.842+0000 7f10c8600700 1 --2- 172.21.15.5:0/3195974290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10c0108840 0x7f10c010acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:32.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:32.842+0000 7f10c8600700 1 --2- 172.21.15.5:0/3195974290 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10c01035e0 0x7f10c01059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:32.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:32.842+0000 7f10c8600700 1 --2- 172.21.15.5:0/3195974290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10c0105f10 0x7f10c0108300 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:32.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:32.842+0000 7f10c8600700 1 -- 172.21.15.5:0/3195974290 >> 172.21.15.5:0/3195974290 conn(0x7f10c00fcff0 msgr2=0x7f10c00ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:32.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:32.843+0000 7f10c8600700 1 -- 172.21.15.5:0/3195974290 shutdown_connections 2024-04-24T06:43:32.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:32.843+0000 7f10c8600700 1 -- 172.21.15.5:0/3195974290 wait complete. 2024-04-24T06:43:32.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:32.844+0000 7f10c8600700 1 Processor -- start 2024-04-24T06:43:32.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:32.844+0000 7f10c8600700 1 -- start start 2024-04-24T06:43:32.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:32.844+0000 7f10c8600700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10c01035e0 0x7f10c0102f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:32.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:32.844+0000 7f10c8600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10c0105f10 0x7f10c0101590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:32.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:32.844+0000 7f10c8600700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10c0108840 0x7f10c0101ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:32.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:32.844+0000 7f10c8600700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f10c010e780 con 0x7f10c0108840 2024-04-24T06:43:32.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:32.844+0000 7f10c8600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f10c010e600 con 0x7f10c0105f10 2024-04-24T06:43:32.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:32.844+0000 7f10c8600700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f10c010e900 con 0x7f10c01035e0 2024-04-24T06:43:32.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:32.845+0000 7f10c6600700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10c0108840 0x7f10c0101ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:32.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:32.845+0000 7f10c5200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10c0105f10 0x7f10c0101590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:32.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:32.845+0000 7f10c5c00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10c01035e0 0x7f10c0102f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:32.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:32.845+0000 7f10c6600700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10c0108840 0x7f10c0101ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39970/0 (socket says 172.21.15.5:39970) 2024-04-24T06:43:32.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:32.845+0000 7f10c6600700 1 -- 172.21.15.5:0/1216419228 learned_addr learned my addr 172.21.15.5:0/1216419228 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:32.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:32.845+0000 7f10c5200700 1 -- 172.21.15.5:0/1216419228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10c01035e0 msgr2=0x7f10c0102f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:32.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:32.845+0000 7f10c5200700 1 --2- 172.21.15.5:0/1216419228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10c01035e0 0x7f10c0102f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:32.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:32.845+0000 7f10c5200700 1 -- 172.21.15.5:0/1216419228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10c0108840 msgr2=0x7f10c0101ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:32.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:32.845+0000 7f10c5200700 1 --2- 172.21.15.5:0/1216419228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10c0108840 0x7f10c0101ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:32.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:32.845+0000 7f10c5200700 1 -- 172.21.15.5:0/1216419228 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c0102380 con 0x7f10c0105f10 2024-04-24T06:43:32.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:32.846+0000 7f10c5200700 1 --2- 172.21.15.5:0/1216419228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10c0105f10 0x7f10c0101590 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f10ac0098e0 tx=0x7f10ac00bbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:32.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:32.846+0000 7f10be000700 1 -- 172.21.15.5:0/1216419228 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac027070 con 0x7f10c0105f10 2024-04-24T06:43:32.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:32.846+0000 7f10be000700 1 -- 172.21.15.5:0/1216419228 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10ac003d70 con 0x7f10c0105f10 2024-04-24T06:43:32.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:32.846+0000 7f10be000700 1 -- 172.21.15.5:0/1216419228 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac00f460 con 0x7f10c0105f10 2024-04-24T06:43:32.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:32.846+0000 7f10c8600700 1 -- 172.21.15.5:0/1216419228 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10c01a2f60 con 0x7f10c0105f10 2024-04-24T06:43:32.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:32.846+0000 7f10c8600700 1 -- 172.21.15.5:0/1216419228 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10c01a3300 con 0x7f10c0105f10 2024-04-24T06:43:32.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:32.847+0000 7f10c8600700 1 -- 172.21.15.5:0/1216419228 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10c00697e0 con 0x7f10c0105f10 2024-04-24T06:43:32.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:32.849+0000 7f10be000700 1 -- 172.21.15.5:0/1216419228 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f10ac02a070 con 0x7f10c0105f10 2024-04-24T06:43:32.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:32.851+0000 7f10be000700 1 --2- 172.21.15.5:0/1216419228 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f10a80a3c40 0x7f10a80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:32.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:32.851+0000 7f10be000700 1 -- 172.21.15.5:0/1216419228 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f10ac0d12a0 con 0x7f10c0105f10 2024-04-24T06:43:32.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:32.851+0000 7f10c5c00700 1 --2- 172.21.15.5:0/1216419228 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f10a80a3c40 0x7f10a80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:32.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:32.852+0000 7f10c5c00700 1 --2- 172.21.15.5:0/1216419228 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f10a80a3c40 0x7f10a80a6100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f10b4009680 tx=0x7f10b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:32.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:32.855+0000 7f10be000700 1 -- 172.21.15.5:0/1216419228 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f10ac05eff0 con 0x7f10c0105f10 2024-04-24T06:43:33.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:33.096+0000 7f10c8600700 1 -- 172.21.15.5:0/1216419228 --> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f10c0062f30 con 0x7f10a80a3c40 2024-04-24T06:43:33.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:33.100+0000 7f10be000700 1 -- 172.21.15.5:0/1216419228 <== mgr.14152 v2:172.21.15.5:6800/4042936965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f10c0062f30 con 0x7f10a80a3c40 2024-04-24T06:43:33.103 INFO:teuthology.orchestra.run.smithi005.stdout:HOST ADDR LABELS STATUS 2024-04-24T06:43:33.103 INFO:teuthology.orchestra.run.smithi005.stdout:smithi005 172.21.15.5 2024-04-24T06:43:33.103 INFO:teuthology.orchestra.run.smithi005.stdout:smithi037 172.21.15.37 2024-04-24T06:43:33.103 INFO:teuthology.orchestra.run.smithi005.stdout:smithi137 172.21.15.137 2024-04-24T06:43:33.103 INFO:teuthology.orchestra.run.smithi005.stdout:3 hosts in cluster 2024-04-24T06:43:33.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:33.103+0000 7f10c8600700 1 -- 172.21.15.5:0/1216419228 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f10a80a3c40 msgr2=0x7f10a80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:33.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:33.103+0000 7f10c8600700 1 --2- 172.21.15.5:0/1216419228 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f10a80a3c40 0x7f10a80a6100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f10b4009680 tx=0x7f10b4009260 comp rx=0 tx=0).stop 2024-04-24T06:43:33.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:33.103+0000 7f10c8600700 1 -- 172.21.15.5:0/1216419228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10c0105f10 msgr2=0x7f10c0101590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:33.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:33.103+0000 7f10c8600700 1 --2- 172.21.15.5:0/1216419228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10c0105f10 0x7f10c0101590 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f10ac0098e0 tx=0x7f10ac00bbb0 comp rx=0 tx=0).stop 2024-04-24T06:43:33.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:33.104+0000 7f10c8600700 1 -- 172.21.15.5:0/1216419228 shutdown_connections 2024-04-24T06:43:33.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:33.104+0000 7f10c8600700 1 --2- 172.21.15.5:0/1216419228 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f10a80a3c40 0x7f10a80a6100 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:33.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:33.104+0000 7f10c8600700 1 --2- 172.21.15.5:0/1216419228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10c0108840 0x7f10c0101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:33.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:33.104+0000 7f10c8600700 1 --2- 172.21.15.5:0/1216419228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10c01035e0 0x7f10c0102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:33.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:33.104+0000 7f10c8600700 1 --2- 172.21.15.5:0/1216419228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10c0105f10 0x7f10c0101590 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:33.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:33.104+0000 7f10c8600700 1 -- 172.21.15.5:0/1216419228 >> 172.21.15.5:0/1216419228 conn(0x7f10c00fcff0 msgr2=0x7f10c0104000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:33.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:33.104+0000 7f10c8600700 1 -- 172.21.15.5:0/1216419228 shutdown_connections 2024-04-24T06:43:33.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:33.104+0000 7f10c8600700 1 -- 172.21.15.5:0/1216419228 wait complete. 2024-04-24T06:43:33.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:33 smithi137 ceph-mon[33017]: pgmap v373: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:43:33.486 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:33 smithi005 ceph-mon[27798]: pgmap v373: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:43:33.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:33 smithi037 ceph-mon[32864]: pgmap v373: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:43:33.922 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-24T06:43:34.414 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:43:34.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:34 smithi137 ceph-mon[33017]: from='client.25043 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:43:34.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:34 smithi005 ceph-mon[27798]: from='client.25043 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:43:34.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:34 smithi037 ceph-mon[32864]: from='client.25043 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:43:35.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:35 smithi137 ceph-mon[33017]: pgmap v374: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T06:43:35.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:35 smithi005 ceph-mon[27798]: pgmap v374: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T06:43:35.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:35 smithi037 ceph-mon[32864]: pgmap v374: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T06:43:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.587+0000 7fd8bfe00700 1 -- 172.21.15.5:0/1780070365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8c0102680 msgr2=0x7fd8c0102b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.587+0000 7fd8bfe00700 1 --2- 172.21.15.5:0/1780070365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8c0102680 0x7fd8c0102b00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8009990 tx=0x7fd8b8009ca0 comp rx=0 tx=0).stop 2024-04-24T06:43:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.587+0000 7fd8bfe00700 1 -- 172.21.15.5:0/1780070365 shutdown_connections 2024-04-24T06:43:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.587+0000 7fd8bfe00700 1 --2- 172.21.15.5:0/1780070365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8c0102680 0x7fd8c0102b00 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.587+0000 7fd8bfe00700 1 --2- 172.21.15.5:0/1780070365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8c0101360 0x7fd8c0101780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.587+0000 7fd8bfe00700 1 --2- 172.21.15.5:0/1780070365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8c0101cc0 0x7fd8c0102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.587+0000 7fd8bfe00700 1 -- 172.21.15.5:0/1780070365 >> 172.21.15.5:0/1780070365 conn(0x7fd8c00fcf00 msgr2=0x7fd8c00ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.588+0000 7fd8bfe00700 1 -- 172.21.15.5:0/1780070365 shutdown_connections 2024-04-24T06:43:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.588+0000 7fd8bfe00700 1 -- 172.21.15.5:0/1780070365 wait complete. 2024-04-24T06:43:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.589+0000 7fd8bfe00700 1 Processor -- start 2024-04-24T06:43:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.589+0000 7fd8bfe00700 1 -- start start 2024-04-24T06:43:36.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.589+0000 7fd8bfe00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8c0101360 0x7fd8c019adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:36.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.589+0000 7fd8bfe00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8c0101cc0 0x7fd8c019b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:36.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.589+0000 7fd8bfe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8c0102680 0x7fd8c01a2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:36.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.589+0000 7fd8bfe00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c0110830 con 0x7fd8c0101360 2024-04-24T06:43:36.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.589+0000 7fd8bfe00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c01106b0 con 0x7fd8c0102680 2024-04-24T06:43:36.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.589+0000 7fd8bfe00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c01109b0 con 0x7fd8c0101cc0 2024-04-24T06:43:36.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.590+0000 7fd8bea00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8c0101360 0x7fd8c019adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:36.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.590+0000 7fd8bea00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8c0101360 0x7fd8c019adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43746/0 (socket says 172.21.15.5:43746) 2024-04-24T06:43:36.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.590+0000 7fd8b6000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8c0101cc0 0x7fd8c019b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:36.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.590+0000 7fd8bea00700 1 -- 172.21.15.5:0/256843719 learned_addr learned my addr 172.21.15.5:0/256843719 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:36.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.590+0000 7fd8bf400700 1 --2- 172.21.15.5:0/256843719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8c0102680 0x7fd8c01a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:36.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.590+0000 7fd8b6000700 1 -- 172.21.15.5:0/256843719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8c0102680 msgr2=0x7fd8c01a2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:36.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.590+0000 7fd8b6000700 1 --2- 172.21.15.5:0/256843719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8c0102680 0x7fd8c01a2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:36.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.590+0000 7fd8b6000700 1 -- 172.21.15.5:0/256843719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8c0101360 msgr2=0x7fd8c019adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:36.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.590+0000 7fd8b6000700 1 --2- 172.21.15.5:0/256843719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8c0101360 0x7fd8c019adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:36.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.590+0000 7fd8b6000700 1 -- 172.21.15.5:0/256843719 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c01a2a00 con 0x7fd8c0101cc0 2024-04-24T06:43:36.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.591+0000 7fd8b6000700 1 --2- 172.21.15.5:0/256843719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8c0101cc0 0x7fd8c019b2f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd8a800ef10 tx=0x7fd8a800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:36.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.591+0000 7fd8b7e00700 1 -- 172.21.15.5:0/256843719 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a800c9c0 con 0x7fd8c0101cc0 2024-04-24T06:43:36.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.591+0000 7fd8b7e00700 1 -- 172.21.15.5:0/256843719 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8a8019070 con 0x7fd8c0101cc0 2024-04-24T06:43:36.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.591+0000 7fd8bfe00700 1 -- 172.21.15.5:0/256843719 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8c01a2c90 con 0x7fd8c0101cc0 2024-04-24T06:43:36.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.591+0000 7fd8b7e00700 1 -- 172.21.15.5:0/256843719 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a80146e0 con 0x7fd8c0101cc0 2024-04-24T06:43:36.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.591+0000 7fd8bfe00700 1 -- 172.21.15.5:0/256843719 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8c01a3180 con 0x7fd8c0101cc0 2024-04-24T06:43:36.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.592+0000 7fd8bfe00700 1 -- 172.21.15.5:0/256843719 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8c00697e0 con 0x7fd8c0101cc0 2024-04-24T06:43:36.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.594+0000 7fd8b7e00700 1 -- 172.21.15.5:0/256843719 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7fd8a8012050 con 0x7fd8c0101cc0 2024-04-24T06:43:36.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.595+0000 7fd8b7e00700 1 --2- 172.21.15.5:0/256843719 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fd8a40a3c40 0x7fd8a40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:36.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.596+0000 7fd8b7e00700 1 -- 172.21.15.5:0/256843719 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fd8a80d0470 con 0x7fd8c0101cc0 2024-04-24T06:43:36.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.596+0000 7fd8bea00700 1 --2- 172.21.15.5:0/256843719 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fd8a40a3c40 0x7fd8a40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:36.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.596+0000 7fd8bea00700 1 --2- 172.21.15.5:0/256843719 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fd8a40a3c40 0x7fd8a40a6100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd8c019c290 tx=0x7fd8b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.600+0000 7fd8b7e00700 1 -- 172.21.15.5:0/256843719 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd8a8010040 con 0x7fd8c0101cc0 2024-04-24T06:43:36.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.840+0000 7fd8bfe00700 1 -- 172.21.15.5:0/256843719 --> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fd8c0062f30 con 0x7fd8a40a3c40 2024-04-24T06:43:36.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.844+0000 7fd8b7e00700 1 -- 172.21.15.5:0/256843719 <== mgr.14152 v2:172.21.15.5:6800/4042936965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fd8c0062f30 con 0x7fd8a40a3c40 2024-04-24T06:43:36.847 INFO:teuthology.orchestra.run.smithi005.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-24T06:43:36.847 INFO:teuthology.orchestra.run.smithi005.stdout:smithi005 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300EW480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-04-24T06:43:36.847 INFO:teuthology.orchestra.run.smithi005.stdout:smithi037 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5331005T400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-24T06:43:36.847 INFO:teuthology.orchestra.run.smithi005.stdout:smithi137 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233004A400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-24T06:43:36.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.847+0000 7fd8bfe00700 1 -- 172.21.15.5:0/256843719 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fd8a40a3c40 msgr2=0x7fd8a40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:36.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.847+0000 7fd8bfe00700 1 --2- 172.21.15.5:0/256843719 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fd8a40a3c40 0x7fd8a40a6100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd8c019c290 tx=0x7fd8b0009260 comp rx=0 tx=0).stop 2024-04-24T06:43:36.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.847+0000 7fd8bfe00700 1 -- 172.21.15.5:0/256843719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8c0101cc0 msgr2=0x7fd8c019b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:36.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.847+0000 7fd8bfe00700 1 --2- 172.21.15.5:0/256843719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8c0101cc0 0x7fd8c019b2f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd8a800ef10 tx=0x7fd8a800c3a0 comp rx=0 tx=0).stop 2024-04-24T06:43:36.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.847+0000 7fd8bfe00700 1 -- 172.21.15.5:0/256843719 shutdown_connections 2024-04-24T06:43:36.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.847+0000 7fd8bfe00700 1 --2- 172.21.15.5:0/256843719 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fd8a40a3c40 0x7fd8a40a6100 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:36.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.847+0000 7fd8bfe00700 1 --2- 172.21.15.5:0/256843719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8c0102680 0x7fd8c01a2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:36.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.847+0000 7fd8bfe00700 1 --2- 172.21.15.5:0/256843719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8c0101360 0x7fd8c019adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:36.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.847+0000 7fd8bfe00700 1 --2- 172.21.15.5:0/256843719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8c0101cc0 0x7fd8c019b2f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:36.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.847+0000 7fd8bfe00700 1 -- 172.21.15.5:0/256843719 >> 172.21.15.5:0/256843719 conn(0x7fd8c00fcf00 msgr2=0x7fd8c00fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:36.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.847+0000 7fd8bfe00700 1 -- 172.21.15.5:0/256843719 shutdown_connections 2024-04-24T06:43:36.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:36.848+0000 7fd8bfe00700 1 -- 172.21.15.5:0/256843719 wait complete. 2024-04-24T06:43:37.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:37 smithi137 ceph-mon[33017]: pgmap v375: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:43:37.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:37 smithi005 ceph-mon[27798]: pgmap v375: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:43:37.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:37 smithi037 ceph-mon[32864]: pgmap v375: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:43:37.661 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T06:43:37.671 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-04-24T06:43:37.671 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-24T06:43:38.138 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:43:38.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:38 smithi137 ceph-mon[33017]: from='client.25066 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:43:38.494 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:38 smithi005 ceph-mon[27798]: from='client.25066 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:43:38.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:38 smithi037 ceph-mon[32864]: from='client.25066 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:43:39.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:39 smithi137 ceph-mon[33017]: pgmap v376: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:43:39.482 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:39 smithi005 ceph-mon[27798]: pgmap v376: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:43:39.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:39 smithi037 ceph-mon[32864]: pgmap v376: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:43:40.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.303+0000 7f75c1a00700 1 -- 172.21.15.5:0/1482815535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75bc102b50 msgr2=0x7f75bc102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:40.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.303+0000 7f75c1a00700 1 --2- 172.21.15.5:0/1482815535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75bc102b50 0x7f75bc102f70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f75ac00b0d0 tx=0x7f75ac00b3e0 comp rx=0 tx=0).stop 2024-04-24T06:43:40.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.303+0000 7f75c1a00700 1 -- 172.21.15.5:0/1482815535 shutdown_connections 2024-04-24T06:43:40.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.303+0000 7f75c1a00700 1 --2- 172.21.15.5:0/1482815535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75bc1046f0 0x7f75bc10afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:40.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.303+0000 7f75c1a00700 1 --2- 172.21.15.5:0/1482815535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75bc102b50 0x7f75bc102f70 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:40.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.303+0000 7f75c1a00700 1 --2- 172.21.15.5:0/1482815535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75bc103d50 0x7f75bc1041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:40.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.303+0000 7f75c1a00700 1 -- 172.21.15.5:0/1482815535 >> 172.21.15.5:0/1482815535 conn(0x7f75bc0fe0f0 msgr2=0x7f75bc100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:40.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.303+0000 7f75c1a00700 1 -- 172.21.15.5:0/1482815535 shutdown_connections 2024-04-24T06:43:40.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.303+0000 7f75c1a00700 1 -- 172.21.15.5:0/1482815535 wait complete. 2024-04-24T06:43:40.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.304+0000 7f75c1a00700 1 Processor -- start 2024-04-24T06:43:40.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.304+0000 7f75c1a00700 1 -- start start 2024-04-24T06:43:40.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.305+0000 7f75c1a00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75bc102b50 0x7f75bc19af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:40.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.305+0000 7f75c1a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75bc103d50 0x7f75bc19b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:40.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.305+0000 7f75c1a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75bc1046f0 0x7f75bc1a2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:40.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.305+0000 7f75c1a00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f75bc10e710 con 0x7f75bc102b50 2024-04-24T06:43:40.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.305+0000 7f75c1a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f75bc10e590 con 0x7f75bc1046f0 2024-04-24T06:43:40.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.305+0000 7f75c1a00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f75bc10e890 con 0x7f75bc103d50 2024-04-24T06:43:40.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.305+0000 7f75b2000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75bc103d50 0x7f75bc19b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:40.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.305+0000 7f75baa00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75bc102b50 0x7f75bc19af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:40.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.305+0000 7f75bb400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75bc1046f0 0x7f75bc1a2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:40.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.305+0000 7f75b2000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75bc103d50 0x7f75bc19b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:39744/0 (socket says 172.21.15.5:39744) 2024-04-24T06:43:40.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.305+0000 7f75b2000700 1 -- 172.21.15.5:0/3271524581 learned_addr learned my addr 172.21.15.5:0/3271524581 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:40.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.306+0000 7f75baa00700 1 -- 172.21.15.5:0/3271524581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75bc103d50 msgr2=0x7f75bc19b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:40.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.306+0000 7f75baa00700 1 --2- 172.21.15.5:0/3271524581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75bc103d50 0x7f75bc19b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:40.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.306+0000 7f75baa00700 1 -- 172.21.15.5:0/3271524581 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75bc1046f0 msgr2=0x7f75bc1a2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:40.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.306+0000 7f75baa00700 1 --2- 172.21.15.5:0/3271524581 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75bc1046f0 0x7f75bc1a2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:40.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.306+0000 7f75baa00700 1 -- 172.21.15.5:0/3271524581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75bc1a2bf0 con 0x7f75bc102b50 2024-04-24T06:43:40.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.306+0000 7f75baa00700 1 --2- 172.21.15.5:0/3271524581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75bc102b50 0x7f75bc19af90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f75ac00b0d0 tx=0x7f75ac007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:40.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.306+0000 7f75b3e00700 1 -- 172.21.15.5:0/3271524581 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ac021020 con 0x7f75bc102b50 2024-04-24T06:43:40.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.306+0000 7f75b3e00700 1 -- 172.21.15.5:0/3271524581 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75ac007cc0 con 0x7f75bc102b50 2024-04-24T06:43:40.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.306+0000 7f75c1a00700 1 -- 172.21.15.5:0/3271524581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75bc1a2e80 con 0x7f75bc102b50 2024-04-24T06:43:40.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.306+0000 7f75b3e00700 1 -- 172.21.15.5:0/3271524581 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ac005290 con 0x7f75bc102b50 2024-04-24T06:43:40.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.307+0000 7f75c1a00700 1 -- 172.21.15.5:0/3271524581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75bc1a3280 con 0x7f75bc102b50 2024-04-24T06:43:40.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.308+0000 7f75c1a00700 1 -- 172.21.15.5:0/3271524581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75bc0697e0 con 0x7f75bc102b50 2024-04-24T06:43:40.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.310+0000 7f75b3e00700 1 -- 172.21.15.5:0/3271524581 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f75ac024020 con 0x7f75bc102b50 2024-04-24T06:43:40.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.311+0000 7f75b3e00700 1 --2- 172.21.15.5:0/3271524581 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f75a00a3cc0 0x7f75a00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:40.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.311+0000 7f75b3e00700 1 -- 172.21.15.5:0/3271524581 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f75ac00f020 con 0x7f75bc102b50 2024-04-24T06:43:40.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.312+0000 7f75b2000700 1 --2- 172.21.15.5:0/3271524581 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f75a00a3cc0 0x7f75a00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:40.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.312+0000 7f75b2000700 1 --2- 172.21.15.5:0/3271524581 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f75a00a3cc0 0x7f75a00a6180 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f75a4009d20 tx=0x7f75a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:40.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.314+0000 7f75b3e00700 1 -- 172.21.15.5:0/3271524581 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f75ac09e500 con 0x7f75bc102b50 2024-04-24T06:43:40.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.599+0000 7f75c1a00700 1 -- 172.21.15.5:0/3271524581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f75bc1a37d0 con 0x7f75bc102b50 2024-04-24T06:43:40.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.600+0000 7f75b3e00700 1 -- 172.21.15.5:0/3271524581 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) ==== 76+0+3040 (secure 0 0 0) 0x7f75ac09de00 con 0x7f75bc102b50 2024-04-24T06:43:40.602 INFO:teuthology.orchestra.run.smithi005.stdout:e22 2024-04-24T06:43:40.602 INFO:teuthology.orchestra.run.smithi005.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T06:43:40.602 INFO:teuthology.orchestra.run.smithi005.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T06:43:40.602 INFO:teuthology.orchestra.run.smithi005.stdout:legacy client fscid: 1 2024-04-24T06:43:40.602 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:43:40.602 INFO:teuthology.orchestra.run.smithi005.stdout:Filesystem 'cephfs' (1) 2024-04-24T06:43:40.602 INFO:teuthology.orchestra.run.smithi005.stdout:fs_name cephfs 2024-04-24T06:43:40.602 INFO:teuthology.orchestra.run.smithi005.stdout:epoch 22 2024-04-24T06:43:40.602 INFO:teuthology.orchestra.run.smithi005.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-24T06:43:40.603 INFO:teuthology.orchestra.run.smithi005.stdout:created 2024-04-24T06:41:34.764490+0000 2024-04-24T06:43:40.603 INFO:teuthology.orchestra.run.smithi005.stdout:modified 2024-04-24T06:41:40.330991+0000 2024-04-24T06:43:40.603 INFO:teuthology.orchestra.run.smithi005.stdout:tableserver 0 2024-04-24T06:43:40.603 INFO:teuthology.orchestra.run.smithi005.stdout:root 0 2024-04-24T06:43:40.603 INFO:teuthology.orchestra.run.smithi005.stdout:session_timeout 300 2024-04-24T06:43:40.603 INFO:teuthology.orchestra.run.smithi005.stdout:session_autoclose 300 2024-04-24T06:43:40.603 INFO:teuthology.orchestra.run.smithi005.stdout:max_file_size 1099511627776 2024-04-24T06:43:40.603 INFO:teuthology.orchestra.run.smithi005.stdout:max_xattr_size 65536 2024-04-24T06:43:40.603 INFO:teuthology.orchestra.run.smithi005.stdout:required_client_features {} 2024-04-24T06:43:40.603 INFO:teuthology.orchestra.run.smithi005.stdout:last_failure 0 2024-04-24T06:43:40.603 INFO:teuthology.orchestra.run.smithi005.stdout:last_failure_osd_epoch 0 2024-04-24T06:43:40.603 INFO:teuthology.orchestra.run.smithi005.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T06:43:40.603 INFO:teuthology.orchestra.run.smithi005.stdout:max_mds 3 2024-04-24T06:43:40.603 INFO:teuthology.orchestra.run.smithi005.stdout:in 0,1,2 2024-04-24T06:43:40.603 INFO:teuthology.orchestra.run.smithi005.stdout:up {0=24569,1=24550,2=24539} 2024-04-24T06:43:40.604 INFO:teuthology.orchestra.run.smithi005.stdout:failed 2024-04-24T06:43:40.604 INFO:teuthology.orchestra.run.smithi005.stdout:damaged 2024-04-24T06:43:40.604 INFO:teuthology.orchestra.run.smithi005.stdout:stopped 2024-04-24T06:43:40.604 INFO:teuthology.orchestra.run.smithi005.stdout:data_pools [3] 2024-04-24T06:43:40.604 INFO:teuthology.orchestra.run.smithi005.stdout:metadata_pool 2 2024-04-24T06:43:40.604 INFO:teuthology.orchestra.run.smithi005.stdout:inline_data disabled 2024-04-24T06:43:40.604 INFO:teuthology.orchestra.run.smithi005.stdout:balancer 2024-04-24T06:43:40.604 INFO:teuthology.orchestra.run.smithi005.stdout:bal_rank_mask -1 2024-04-24T06:43:40.604 INFO:teuthology.orchestra.run.smithi005.stdout:standby_count_wanted 0 2024-04-24T06:43:40.604 INFO:teuthology.orchestra.run.smithi005.stdout:qdb_cluster leader: 24569 members: 24539,24550,24569 2024-04-24T06:43:40.604 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.b{0:24569} state up:active seq 3 addr [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T06:43:40.604 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.c{0:24532} state up:standby-replay seq 1 addr [v2:172.21.15.137:6836/3044222976,v1:172.21.15.137:6837/3044222976] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T06:43:40.604 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.i{1:24550} state up:active seq 7 addr [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T06:43:40.604 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.f{1:24520} state up:standby-replay seq 1 addr [v2:172.21.15.137:6834/3217754215,v1:172.21.15.137:6835/3217754215] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T06:43:40.604 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.g{2:24539} state up:active seq 6 addr [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T06:43:40.605 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.h{2:24515} state up:standby-replay seq 1 addr [v2:172.21.15.37:6836/1390103296,v1:172.21.15.37:6837/1390103296] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T06:43:40.605 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:43:40.605 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:43:40.605 INFO:teuthology.orchestra.run.smithi005.stdout:Standby daemons: 2024-04-24T06:43:40.605 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:43:40.605 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.d{-1:14403} state up:standby seq 1 addr [v2:172.21.15.5:6836/1735462750,v1:172.21.15.5:6837/1735462750] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T06:43:40.605 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.a{-1:14463} state up:standby seq 1 addr [v2:172.21.15.5:6840/1499469370,v1:172.21.15.5:6841/1499469370] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T06:43:40.605 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.k{-1:24437} state up:standby seq 1 addr [v2:172.21.15.37:6832/2228938014,v1:172.21.15.37:6833/2228938014] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T06:43:40.605 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.l{-1:24451} state up:standby seq 1 addr [v2:172.21.15.137:6832/681129825,v1:172.21.15.137:6833/681129825] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T06:43:40.605 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.j{-1:24481} state up:standby seq 1 addr [v2:172.21.15.5:6834/2026364804,v1:172.21.15.5:6835/2026364804] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T06:43:40.605 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.e{-1:24485} state up:standby seq 1 addr [v2:172.21.15.37:6834/3705638847,v1:172.21.15.37:6835/3705638847] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T06:43:40.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.602+0000 7f75c1a00700 1 -- 172.21.15.5:0/3271524581 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f75a00a3cc0 msgr2=0x7f75a00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:40.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.602+0000 7f75c1a00700 1 --2- 172.21.15.5:0/3271524581 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f75a00a3cc0 0x7f75a00a6180 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f75a4009d20 tx=0x7f75a4009260 comp rx=0 tx=0).stop 2024-04-24T06:43:40.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.602+0000 7f75c1a00700 1 -- 172.21.15.5:0/3271524581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75bc102b50 msgr2=0x7f75bc19af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:40.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.602+0000 7f75c1a00700 1 --2- 172.21.15.5:0/3271524581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75bc102b50 0x7f75bc19af90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f75ac00b0d0 tx=0x7f75ac007a20 comp rx=0 tx=0).stop 2024-04-24T06:43:40.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.603+0000 7f75c1a00700 1 -- 172.21.15.5:0/3271524581 shutdown_connections 2024-04-24T06:43:40.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.603+0000 7f75c1a00700 1 --2- 172.21.15.5:0/3271524581 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f75a00a3cc0 0x7f75a00a6180 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:40.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.603+0000 7f75c1a00700 1 --2- 172.21.15.5:0/3271524581 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75bc1046f0 0x7f75bc1a2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:40.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.603+0000 7f75c1a00700 1 --2- 172.21.15.5:0/3271524581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75bc102b50 0x7f75bc19af90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:40.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.603+0000 7f75c1a00700 1 --2- 172.21.15.5:0/3271524581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75bc103d50 0x7f75bc19b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:40.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.603+0000 7f75c1a00700 1 -- 172.21.15.5:0/3271524581 >> 172.21.15.5:0/3271524581 conn(0x7f75bc0fe0f0 msgr2=0x7f75bc074d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:40.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.603+0000 7f75c1a00700 1 -- 172.21.15.5:0/3271524581 shutdown_connections 2024-04-24T06:43:40.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:40.603+0000 7f75c1a00700 1 -- 172.21.15.5:0/3271524581 wait complete. 2024-04-24T06:43:40.607 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 22 2024-04-24T06:43:41.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:41 smithi005 ceph-mon[27798]: pgmap v377: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T06:43:41.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:41 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/3271524581' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T06:43:41.408 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-24T06:43:41.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:41 smithi137 ceph-mon[33017]: pgmap v377: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T06:43:41.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:41 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/3271524581' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T06:43:41.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:41 smithi037 ceph-mon[32864]: pgmap v377: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T06:43:41.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:41 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/3271524581' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T06:43:41.893 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:43:43.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:43 smithi137 ceph-mon[33017]: pgmap v378: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:43:43.502 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:43 smithi005 ceph-mon[27798]: pgmap v378: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:43:43.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:43 smithi037 ceph-mon[32864]: pgmap v378: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:43:44.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.038+0000 7f6bd1c00700 1 -- 172.21.15.5:0/2367374813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bcc10a390 msgr2=0x7f6bcc10a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:44.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.038+0000 7f6bd1c00700 1 --2- 172.21.15.5:0/2367374813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bcc10a390 0x7f6bcc10a7f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6bb80098f0 tx=0x7f6bb8009c00 comp rx=0 tx=0).stop 2024-04-24T06:43:44.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.038+0000 7f6bd1c00700 1 -- 172.21.15.5:0/2367374813 shutdown_connections 2024-04-24T06:43:44.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.038+0000 7f6bd1c00700 1 --2- 172.21.15.5:0/2367374813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bcc10ad30 0x7f6bcc1115e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:44.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.038+0000 7f6bd1c00700 1 --2- 172.21.15.5:0/2367374813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6bcc109190 0x7f6bcc1095b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:44.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.038+0000 7f6bd1c00700 1 --2- 172.21.15.5:0/2367374813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bcc10a390 0x7f6bcc10a7f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:44.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.038+0000 7f6bd1c00700 1 -- 172.21.15.5:0/2367374813 >> 172.21.15.5:0/2367374813 conn(0x7f6bcc078160 msgr2=0x7f6bcc07a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:44.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.038+0000 7f6bd1c00700 1 -- 172.21.15.5:0/2367374813 shutdown_connections 2024-04-24T06:43:44.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.038+0000 7f6bd1c00700 1 -- 172.21.15.5:0/2367374813 wait complete. 2024-04-24T06:43:44.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.039+0000 7f6bd1c00700 1 Processor -- start 2024-04-24T06:43:44.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.040+0000 7f6bd1c00700 1 -- start start 2024-04-24T06:43:44.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.040+0000 7f6bd1c00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bcc109190 0x7f6bcc19f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:44.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.040+0000 7f6bd1c00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bcc10a390 0x7f6bcc19f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:44.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.040+0000 7f6bd1c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6bcc10ad30 0x7f6bcc1a6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:44.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.040+0000 7f6bd1c00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6bcc114da0 con 0x7f6bcc109190 2024-04-24T06:43:44.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.040+0000 7f6bd1c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6bcc114c20 con 0x7f6bcc10ad30 2024-04-24T06:43:44.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.040+0000 7f6bd1c00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6bcc114f20 con 0x7f6bcc10a390 2024-04-24T06:43:44.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.040+0000 7f6bcaa00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bcc109190 0x7f6bcc19f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:44.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.040+0000 7f6bcb400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6bcc10ad30 0x7f6bcc1a6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:44.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.040+0000 7f6bca000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bcc10a390 0x7f6bcc19f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:44.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.040+0000 7f6bcaa00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bcc109190 0x7f6bcc19f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39442/0 (socket says 172.21.15.5:39442) 2024-04-24T06:43:44.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.040+0000 7f6bcb400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6bcc10ad30 0x7f6bcc1a6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:60938/0 (socket says 172.21.15.5:60938) 2024-04-24T06:43:44.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.040+0000 7f6bcaa00700 1 -- 172.21.15.5:0/94562907 learned_addr learned my addr 172.21.15.5:0/94562907 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:44.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.041+0000 7f6bcb400700 1 -- 172.21.15.5:0/94562907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bcc10a390 msgr2=0x7f6bcc19f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:44.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.041+0000 7f6bcb400700 1 --2- 172.21.15.5:0/94562907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bcc10a390 0x7f6bcc19f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:44.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.041+0000 7f6bcb400700 1 -- 172.21.15.5:0/94562907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bcc109190 msgr2=0x7f6bcc19f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:44.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.041+0000 7f6bcb400700 1 --2- 172.21.15.5:0/94562907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bcc109190 0x7f6bcc19f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:44.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.041+0000 7f6bcb400700 1 -- 172.21.15.5:0/94562907 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bcc1a7010 con 0x7f6bcc10ad30 2024-04-24T06:43:44.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.041+0000 7f6bca000700 1 --2- 172.21.15.5:0/94562907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bcc10a390 0x7f6bcc19f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:43:44.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.041+0000 7f6bcb400700 1 --2- 172.21.15.5:0/94562907 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6bcc10ad30 0x7f6bcc1a6970 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6bc400efc0 tx=0x7f6bc400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:44.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.041+0000 7f6bc3400700 1 -- 172.21.15.5:0/94562907 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc40092e0 con 0x7f6bcc10ad30 2024-04-24T06:43:44.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.041+0000 7f6bc3400700 1 -- 172.21.15.5:0/94562907 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bc4004bb0 con 0x7f6bcc10ad30 2024-04-24T06:43:44.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.041+0000 7f6bc3400700 1 -- 172.21.15.5:0/94562907 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc4004280 con 0x7f6bcc10ad30 2024-04-24T06:43:44.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.041+0000 7f6bd1c00700 1 -- 172.21.15.5:0/94562907 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bcc1a72a0 con 0x7f6bcc10ad30 2024-04-24T06:43:44.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.042+0000 7f6bd1c00700 1 -- 172.21.15.5:0/94562907 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bcc1a7830 con 0x7f6bcc10ad30 2024-04-24T06:43:44.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.043+0000 7f6bd1c00700 1 -- 172.21.15.5:0/94562907 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bcc0697e0 con 0x7f6bcc10ad30 2024-04-24T06:43:44.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.045+0000 7f6bc3400700 1 -- 172.21.15.5:0/94562907 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f6bc401b030 con 0x7f6bcc10ad30 2024-04-24T06:43:44.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.046+0000 7f6bc3400700 1 --2- 172.21.15.5:0/94562907 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f6bb00a3cc0 0x7f6bb00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:44.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.046+0000 7f6bc3400700 1 -- 172.21.15.5:0/94562907 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f6bc40d08f0 con 0x7f6bcc10ad30 2024-04-24T06:43:44.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.047+0000 7f6bcaa00700 1 --2- 172.21.15.5:0/94562907 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f6bb00a3cc0 0x7f6bb00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:44.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.047+0000 7f6bcaa00700 1 --2- 172.21.15.5:0/94562907 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f6bb00a3cc0 0x7f6bb00a6180 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc009750 tx=0x7f6bbc009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:44.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.049+0000 7f6bc3400700 1 -- 172.21.15.5:0/94562907 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6bc4022020 con 0x7f6bcc10ad30 2024-04-24T06:43:44.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.282+0000 7f6bd1c00700 1 -- 172.21.15.5:0/94562907 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f6bcc063700 con 0x7f6bcc10ad30 2024-04-24T06:43:44.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.283+0000 7f6bc3400700 1 -- 172.21.15.5:0/94562907 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v88) ==== 56+0+5259 (secure 0 0 0) 0x7f6bc4010040 con 0x7f6bcc10ad30 2024-04-24T06:43:44.285 INFO:teuthology.orchestra.run.smithi005.stdout:epoch 88 2024-04-24T06:43:44.285 INFO:teuthology.orchestra.run.smithi005.stdout:fsid 43e9bad0-0204-11ef-bc93-c7b262605968 2024-04-24T06:43:44.285 INFO:teuthology.orchestra.run.smithi005.stdout:created 2024-04-24T06:32:26.340555+0000 2024-04-24T06:43:44.285 INFO:teuthology.orchestra.run.smithi005.stdout:modified 2024-04-24T06:42:42.855911+0000 2024-04-24T06:43:44.285 INFO:teuthology.orchestra.run.smithi005.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-24T06:43:44.285 INFO:teuthology.orchestra.run.smithi005.stdout:crush_version 26 2024-04-24T06:43:44.286 INFO:teuthology.orchestra.run.smithi005.stdout:full_ratio 0.95 2024-04-24T06:43:44.286 INFO:teuthology.orchestra.run.smithi005.stdout:backfillfull_ratio 0.9 2024-04-24T06:43:44.286 INFO:teuthology.orchestra.run.smithi005.stdout:nearfull_ratio 0.85 2024-04-24T06:43:44.286 INFO:teuthology.orchestra.run.smithi005.stdout:require_min_compat_client luminous 2024-04-24T06:43:44.286 INFO:teuthology.orchestra.run.smithi005.stdout:min_compat_client luminous 2024-04-24T06:43:44.286 INFO:teuthology.orchestra.run.smithi005.stdout:require_osd_release squid 2024-04-24T06:43:44.286 INFO:teuthology.orchestra.run.smithi005.stdout:stretch_mode_enabled false 2024-04-24T06:43:44.286 INFO:teuthology.orchestra.run.smithi005.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-24T06:43:44.286 INFO:teuthology.orchestra.run.smithi005.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-04-24T06:43:44.286 INFO:teuthology.orchestra.run.smithi005.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-24T06:43:44.286 INFO:teuthology.orchestra.run.smithi005.stdout:max_osd 12 2024-04-24T06:43:44.286 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6802/2155858515,v1:172.21.15.5:6803/2155858515] [v2:172.21.15.5:6804/2155858515,v1:172.21.15.5:6805/2155858515] exists,up 3c2e974c-745c-4dd5-8c3d-8e8da2eb1ce0 2024-04-24T06:43:44.286 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1 up in weight 1 up_from 13 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6800/1302362650,v1:172.21.15.37:6801/1302362650] [v2:172.21.15.37:6802/1302362650,v1:172.21.15.37:6803/1302362650] exists,up 50467744-db20-4721-bf2d-1c5db938297b 2024-04-24T06:43:44.287 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2 up in weight 1 up_from 18 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6800/2885027256,v1:172.21.15.137:6801/2885027256] [v2:172.21.15.137:6802/2885027256,v1:172.21.15.137:6803/2885027256] exists,up 0e190b15-d04c-4288-8d48-ecf6664392fe 2024-04-24T06:43:44.287 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3 up in weight 1 up_from 25 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6810/3701747260,v1:172.21.15.5:6811/3701747260] [v2:172.21.15.5:6812/3701747260,v1:172.21.15.5:6813/3701747260] exists,up e05592ba-d96a-472a-9f63-09a535d04dbb 2024-04-24T06:43:44.287 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4 up in weight 1 up_from 33 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6808/759253844,v1:172.21.15.37:6809/759253844] [v2:172.21.15.37:6810/759253844,v1:172.21.15.37:6811/759253844] exists,up c3f36e51-7a3b-4dd9-af4b-a10558830c4c 2024-04-24T06:43:44.287 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5 up in weight 1 up_from 39 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6808/3771921560,v1:172.21.15.137:6809/3771921560] [v2:172.21.15.137:6810/3771921560,v1:172.21.15.137:6811/3771921560] exists,up 97a0500c-cbb8-4ff5-b476-8634b26b3821 2024-04-24T06:43:44.287 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6 up in weight 1 up_from 47 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6818/2064636084,v1:172.21.15.5:6819/2064636084] [v2:172.21.15.5:6820/2064636084,v1:172.21.15.5:6821/2064636084] exists,up 33a33d62-bbb3-4ed9-834e-19ae93a97670 2024-04-24T06:43:44.287 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7 up in weight 1 up_from 52 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6816/2952750659,v1:172.21.15.37:6817/2952750659] [v2:172.21.15.37:6818/2952750659,v1:172.21.15.37:6819/2952750659] exists,up c0f4e036-4717-4514-bb59-ab11fbaab063 2024-04-24T06:43:44.287 INFO:teuthology.orchestra.run.smithi005.stdout:osd.8 up in weight 1 up_from 60 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6816/29724945,v1:172.21.15.137:6817/29724945] [v2:172.21.15.137:6818/29724945,v1:172.21.15.137:6819/29724945] exists,up 5391fd87-44ea-440f-80da-c54d0327c04c 2024-04-24T06:43:44.287 INFO:teuthology.orchestra.run.smithi005.stdout:osd.9 up in weight 1 up_from 68 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6826/2257102107,v1:172.21.15.5:6827/2257102107] [v2:172.21.15.5:6828/2257102107,v1:172.21.15.5:6829/2257102107] exists,up d94c9dbc-f43d-4237-ac7f-47a6a5ede1c4 2024-04-24T06:43:44.287 INFO:teuthology.orchestra.run.smithi005.stdout:osd.10 up in weight 1 up_from 73 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6824/2451570637,v1:172.21.15.37:6825/2451570637] [v2:172.21.15.37:6826/2451570637,v1:172.21.15.37:6827/2451570637] exists,up e2455332-2a5e-44b1-85d3-8e47da399f9e 2024-04-24T06:43:44.287 INFO:teuthology.orchestra.run.smithi005.stdout:osd.11 up in weight 1 up_from 81 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6824/90600596,v1:172.21.15.137:6825/90600596] [v2:172.21.15.137:6826/90600596,v1:172.21.15.137:6827/90600596] exists,up 49d9396b-704a-4cef-8210-75d30c28e924 2024-04-24T06:43:44.287 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.4 [2,11] 2024-04-24T06:43:44.287 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.13 [2,11] 2024-04-24T06:43:44.287 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.1a [8,4] 2024-04-24T06:43:44.287 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.1 [4,9] 2024-04-24T06:43:44.288 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.17 [4,8] 2024-04-24T06:43:44.288 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.1b [4,7] 2024-04-24T06:43:44.288 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.28 [5,6] 2024-04-24T06:43:44.288 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.2d [3,9] 2024-04-24T06:43:44.288 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.38 [3,9] 2024-04-24T06:43:44.288 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/23307164 expires 2024-04-25T06:33:41.161852+0000 2024-04-24T06:43:44.288 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6801/2147029578 expires 2024-04-25T06:33:41.161852+0000 2024-04-24T06:43:44.288 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/230785941 expires 2024-04-25T06:33:41.161852+0000 2024-04-24T06:43:44.288 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6800/2147029578 expires 2024-04-25T06:33:41.161852+0000 2024-04-24T06:43:44.288 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6800/2413496885 expires 2024-04-25T06:33:07.040369+0000 2024-04-24T06:43:44.288 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/2940023597 expires 2024-04-25T06:33:07.040369+0000 2024-04-24T06:43:44.288 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6801/2413496885 expires 2024-04-25T06:33:07.040369+0000 2024-04-24T06:43:44.288 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/3886613904 expires 2024-04-25T06:33:07.040369+0000 2024-04-24T06:43:44.288 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/4199165437 expires 2024-04-25T06:33:07.040369+0000 2024-04-24T06:43:44.288 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/3459593499 expires 2024-04-25T06:33:41.161852+0000 2024-04-24T06:43:44.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.286+0000 7f6bd1c00700 1 -- 172.21.15.5:0/94562907 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f6bb00a3cc0 msgr2=0x7f6bb00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:44.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.286+0000 7f6bd1c00700 1 --2- 172.21.15.5:0/94562907 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f6bb00a3cc0 0x7f6bb00a6180 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc009750 tx=0x7f6bbc009310 comp rx=0 tx=0).stop 2024-04-24T06:43:44.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.286+0000 7f6bd1c00700 1 -- 172.21.15.5:0/94562907 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6bcc10ad30 msgr2=0x7f6bcc1a6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:44.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.286+0000 7f6bd1c00700 1 --2- 172.21.15.5:0/94562907 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6bcc10ad30 0x7f6bcc1a6970 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6bc400efc0 tx=0x7f6bc400c3a0 comp rx=0 tx=0).stop 2024-04-24T06:43:44.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.286+0000 7f6bd1c00700 1 -- 172.21.15.5:0/94562907 shutdown_connections 2024-04-24T06:43:44.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.286+0000 7f6bd1c00700 1 --2- 172.21.15.5:0/94562907 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f6bb00a3cc0 0x7f6bb00a6180 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:44.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.286+0000 7f6bd1c00700 1 --2- 172.21.15.5:0/94562907 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6bcc10ad30 0x7f6bcc1a6970 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:44.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.286+0000 7f6bd1c00700 1 --2- 172.21.15.5:0/94562907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bcc109190 0x7f6bcc19f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:44.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.286+0000 7f6bd1c00700 1 --2- 172.21.15.5:0/94562907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bcc10a390 0x7f6bcc19f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:44.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.286+0000 7f6bd1c00700 1 -- 172.21.15.5:0/94562907 >> 172.21.15.5:0/94562907 conn(0x7f6bcc078160 msgr2=0x7f6bcc10d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:44.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.287+0000 7f6bd1c00700 1 -- 172.21.15.5:0/94562907 shutdown_connections 2024-04-24T06:43:44.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:44.287+0000 7f6bd1c00700 1 -- 172.21.15.5:0/94562907 wait complete. 2024-04-24T06:43:45.091 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-24T06:43:45.103 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:43:45.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.326+0000 7fc69ae00640 1 -- 172.21.15.5:0/723829038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc694101190 msgr2=0x7fc6941015f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:45.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.326+0000 7fc69ae00640 1 --2- 172.21.15.5:0/723829038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc694101190 0x7fc6941015f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc680009900 tx=0x7fc68002a480 comp rx=0 tx=0).stop 2024-04-24T06:43:45.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.326+0000 7fc69ae00640 1 -- 172.21.15.5:0/723829038 shutdown_connections 2024-04-24T06:43:45.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.326+0000 7fc69ae00640 1 --2- 172.21.15.5:0/723829038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc694101b30 0x7fc694101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:45.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.326+0000 7fc69ae00640 1 --2- 172.21.15.5:0/723829038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc694101190 0x7fc6941015f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:45.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.326+0000 7fc69ae00640 1 --2- 172.21.15.5:0/723829038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc694100830 0x7fc694100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:45.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.326+0000 7fc69ae00640 1 -- 172.21.15.5:0/723829038 >> 172.21.15.5:0/723829038 conn(0x7fc6940fc3d0 msgr2=0x7fc6940fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:45.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.326+0000 7fc69ae00640 1 -- 172.21.15.5:0/723829038 shutdown_connections 2024-04-24T06:43:45.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.326+0000 7fc69ae00640 1 -- 172.21.15.5:0/723829038 wait complete. 2024-04-24T06:43:45.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.327+0000 7fc69ae00640 1 Processor -- start 2024-04-24T06:43:45.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.328+0000 7fc69ae00640 1 -- start start 2024-04-24T06:43:45.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.328+0000 7fc69ae00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc694100830 0x7fc69419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:45.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.328+0000 7fc69ae00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc694101190 0x7fc69419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:45.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.328+0000 7fc69ae00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc694101b30 0x7fc6941a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:45.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.328+0000 7fc69ae00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc69410fe20 con 0x7fc694101190 2024-04-24T06:43:45.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.328+0000 7fc69ae00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc69410fca0 con 0x7fc694100830 2024-04-24T06:43:45.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.328+0000 7fc69ae00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc69410ffa0 con 0x7fc694101b30 2024-04-24T06:43:45.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.328+0000 7fc699000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc694101190 0x7fc69419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:45.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.328+0000 7fc699000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc694101190 0x7fc69419aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39466/0 (socket says 172.21.15.5:39466) 2024-04-24T06:43:45.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.328+0000 7fc699000640 1 -- 172.21.15.5:0/3009055848 learned_addr learned my addr 172.21.15.5:0/3009055848 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:45.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.328+0000 7fc699a00640 1 --2- 172.21.15.5:0/3009055848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc694100830 0x7fc69419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:45.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.328+0000 7fc69a400640 1 --2- 172.21.15.5:0/3009055848 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc694101b30 0x7fc6941a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:45.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.329+0000 7fc699a00640 1 -- 172.21.15.5:0/3009055848 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc694101b30 msgr2=0x7fc6941a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:45.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.329+0000 7fc699a00640 1 --2- 172.21.15.5:0/3009055848 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc694101b30 0x7fc6941a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:45.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.329+0000 7fc699a00640 1 -- 172.21.15.5:0/3009055848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc694101190 msgr2=0x7fc69419aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:45.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.329+0000 7fc699a00640 1 --2- 172.21.15.5:0/3009055848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc694101190 0x7fc69419aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:45.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.329+0000 7fc699a00640 1 -- 172.21.15.5:0/3009055848 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6941a20c0 con 0x7fc694100830 2024-04-24T06:43:45.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.329+0000 7fc699000640 1 --2- 172.21.15.5:0/3009055848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc694101190 0x7fc69419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:43:45.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.329+0000 7fc699a00640 1 --2- 172.21.15.5:0/3009055848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc694100830 0x7fc69419a4f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc68800e9f0 tx=0x7fc68800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:45.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.329+0000 7fc692000640 1 -- 172.21.15.5:0/3009055848 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68800ccc0 con 0x7fc694100830 2024-04-24T06:43:45.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.329+0000 7fc69ae00640 1 -- 172.21.15.5:0/3009055848 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6941a23b0 con 0x7fc694100830 2024-04-24T06:43:45.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.329+0000 7fc692000640 1 -- 172.21.15.5:0/3009055848 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc688010910 con 0x7fc694100830 2024-04-24T06:43:45.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.329+0000 7fc692000640 1 -- 172.21.15.5:0/3009055848 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc688018960 con 0x7fc694100830 2024-04-24T06:43:45.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.330+0000 7fc69ae00640 1 -- 172.21.15.5:0/3009055848 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6941a2950 con 0x7fc694100830 2024-04-24T06:43:45.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.331+0000 7fc69ae00640 1 -- 172.21.15.5:0/3009055848 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc69410b580 con 0x7fc694100830 2024-04-24T06:43:45.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.333+0000 7fc692000640 1 -- 172.21.15.5:0/3009055848 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7fc688010ac0 con 0x7fc694100830 2024-04-24T06:43:45.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.334+0000 7fc692000640 1 --2- 172.21.15.5:0/3009055848 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fc67c0a3ba0 0x7fc67c0a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:45.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.334+0000 7fc692000640 1 -- 172.21.15.5:0/3009055848 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fc6880d07e0 con 0x7fc694100830 2024-04-24T06:43:45.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.335+0000 7fc699000640 1 --2- 172.21.15.5:0/3009055848 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fc67c0a3ba0 0x7fc67c0a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:45.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.335+0000 7fc699000640 1 --2- 172.21.15.5:0/3009055848 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fc67c0a3ba0 0x7fc67c0a6060 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc69419b9b0 tx=0x7fc6800315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:45.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.337+0000 7fc692000640 1 -- 172.21.15.5:0/3009055848 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc68809cb30 con 0x7fc694100830 2024-04-24T06:43:45.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:45 smithi137 ceph-mon[33017]: pgmap v379: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T06:43:45.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:45 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/94562907' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-24T06:43:45.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:45 smithi005 ceph-mon[27798]: pgmap v379: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T06:43:45.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:45 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/94562907' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-24T06:43:45.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:45 smithi037 ceph-mon[32864]: pgmap v379: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T06:43:45.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:45 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/94562907' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-24T06:43:45.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.546+0000 7fc69ae00640 1 -- 172.21.15.5:0/3009055848 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc69402d620 con 0x7fc694100830 2024-04-24T06:43:45.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.548+0000 7fc692000640 1 -- 172.21.15.5:0/3009055848 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22513 (secure 0 0 0) 0x7fc6880a0e30 con 0x7fc694100830 2024-04-24T06:43:45.550 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:43:45.551 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":88,"fsid":"43e9bad0-0204-11ef-bc93-c7b262605968","created":"2024-04-24T06:32:26.340555+0000","modified":"2024-04-24T06:42:42.855911+0000","last_up_change":"2024-04-24T06:40:36.790934+0000","last_in_change":"2024-04-24T06:40:17.803257+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:36:27.320342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:41:32.254502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:41:33.739067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c2e974c-745c-4dd5-8c3d-8e8da2eb1ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6803","nonce":2155858515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6805","nonce":2155858515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6809","nonce":2155858515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6807","nonce":2155858515}]},"public_addr":"172.21.15.5:6803/2155858515","cluster_addr":"172.21.15.5:6805/2155858515","heartbeat_back_addr":"172.21.15.5:6809/2155858515","heartbeat_front_addr":"172.21.15.5:6807/2155858515","state":["exists","up"]},{"osd":1,"uuid":"50467744-db20-4721-bf2d-1c5db938297b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6801","nonce":1302362650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6803","nonce":1302362650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6807","nonce":1302362650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6805","nonce":1302362650}]},"public_addr":"172.21.15.37:6801/1302362650","cluster_addr":"172.21.15.37:6803/1302362650","heartbeat_back_addr":"172.21.15.37:6807/1302362650","heartbeat_front_addr":"172.21.15.37:6805/1302362650","state":["exists","up"]},{"osd":2,"uuid":"0e190b15-d04c-4288-8d48-ecf6664392fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6801","nonce":2885027256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6803","nonce":2885027256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6807","nonce":2885027256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6805","nonce":2885027256}]},"public_addr":"172.21.15.137:6801/2885027256","cluster_addr":"172.21.15.137:6803/2885027256","heartbeat_back_addr":"172.21.15.137:6807/2885027256","heartbeat_front_addr":"172.21.15.137:6805/2885027256","state":["exists","up"]},{"osd":3,"uuid":"e05592ba-d96a-472a-9f63-09a535d04dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6811","nonce":3701747260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6813","nonce":3701747260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6817","nonce":3701747260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6815","nonce":3701747260}]},"public_addr":"172.21.15.5:6811/3701747260","cluster_addr":"172.21.15.5:6813/3701747260","heartbeat_back_addr":"172.21.15.5:6817/3701747260","heartbeat_front_addr":"172.21.15.5:6815/3701747260","state":["exists","up"]},{"osd":4,"uuid":"c3f36e51-7a3b-4dd9-af4b-a10558830c4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6809","nonce":759253844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6811","nonce":759253844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6815","nonce":759253844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6813","nonce":759253844}]},"public_addr":"172.21.15.37:6809/759253844","cluster_addr":"172.21.15.37:6811/759253844","heartbeat_back_addr":"172.21.15.37:6815/759253844","heartbeat_front_addr":"172.21.15.37:6813/759253844","state":["exists","up"]},{"osd":5,"uuid":"97a0500c-cbb8-4ff5-b476-8634b26b3821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6809","nonce":3771921560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6811","nonce":3771921560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6815","nonce":3771921560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6813","nonce":3771921560}]},"public_addr":"172.21.15.137:6809/3771921560","cluster_addr":"172.21.15.137:6811/3771921560","heartbeat_back_addr":"172.21.15.137:6815/3771921560","heartbeat_front_addr":"172.21.15.137:6813/3771921560","state":["exists","up"]},{"osd":6,"uuid":"33a33d62-bbb3-4ed9-834e-19ae93a97670","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6819","nonce":2064636084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6821","nonce":2064636084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6825","nonce":2064636084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6823","nonce":2064636084}]},"public_addr":"172.21.15.5:6819/2064636084","cluster_addr":"172.21.15.5:6821/2064636084","heartbeat_back_addr":"172.21.15.5:6825/2064636084","heartbeat_front_addr":"172.21.15.5:6823/2064636084","state":["exists","up"]},{"osd":7,"uuid":"c0f4e036-4717-4514-bb59-ab11fbaab063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6817","nonce":2952750659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6819","nonce":2952750659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6823","nonce":2952750659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6821","nonce":2952750659}]},"public_addr":"172.21.15.37:6817/2952750659","cluster_addr":"172.21.15.37:6819/2952750659","heartbeat_back_addr":"172.21.15.37:6823/2952750659","heartbeat_front_addr":"172.21.15.37:6821/2952750659","state":["exists","up"]},{"osd":8,"uuid":"5391fd87-44ea-440f-80da-c54d0327c04c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6817","nonce":29724945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6819","nonce":29724945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6823","nonce":29724945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6821","nonce":29724945}]},"public_addr":"172.21.15.137:6817/29724945","cluster_addr":"172.21.15.137:6819/29724945","heartbeat_back_addr":"172.21.15.137:6823/29724945","heartbeat_front_addr":"172.21.15.137:6821/29724945","state":["exists","up"]},{"osd":9,"uuid":"d94c9dbc-f43d-4237-ac7f-47a6a5ede1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6827","nonce":2257102107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6829","nonce":2257102107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6833","nonce":2257102107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6831","nonce":2257102107}]},"public_addr":"172.21.15.5:6827/2257102107","cluster_addr":"172.21.15.5:6829/2257102107","heartbeat_back_addr":"172.21.15.5:6833/2257102107","heartbeat_front_addr":"172.21.15.5:6831/2257102107","state":["exists","up"]},{"osd":10,"uuid":"e2455332-2a5e-44b1-85d3-8e47da399f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6825","nonce":2451570637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6827","nonce":2451570637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6831","nonce":2451570637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6829","nonce":2451570637}]},"public_addr":"172.21.15.37:6825/2451570637","cluster_addr":"172.21.15.37:6827/2451570637","heartbeat_back_addr":"172.21.15.37:6831/2451570637","heartbeat_front_addr":"172.21.15.37:6829/2451570637","state":["exists","up"]},{"osd":11,"uuid":"49d9396b-704a-4cef-8210-75d30c28e924","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6825","nonce":90600596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6827","nonce":90600596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6831","nonce":90600596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6829","nonce":90600596}]},"public_addr":"172.21.15.137:6825/90600596","cluster_addr":"172.21.15.137:6827/90600596","heartbeat_back_addr":"172.21.15.137:6831/90600596","heartbeat_front_addr":"172.21.15.137:6829/90600596","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:35:35.591619+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:00.568817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:24.195643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:56.451724+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:22.109293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:46.161749+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:19.278305+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:45.620256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:10.041165+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:43.813697+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:10.736026+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:35.384377+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/23307164":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6801/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:0/230785941":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/2940023597":"2024-04-25T06:33:07.040369+0000","172.21.15.5:6801/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3886613904":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/4199165437":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3459593499":"2024-04-25T06:33:41.161852+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:43:45.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.551+0000 7fc69ae00640 1 -- 172.21.15.5:0/3009055848 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fc67c0a3ba0 msgr2=0x7fc67c0a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:45.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.551+0000 7fc69ae00640 1 --2- 172.21.15.5:0/3009055848 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fc67c0a3ba0 0x7fc67c0a6060 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc69419b9b0 tx=0x7fc6800315b0 comp rx=0 tx=0).stop 2024-04-24T06:43:45.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.551+0000 7fc69ae00640 1 -- 172.21.15.5:0/3009055848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc694100830 msgr2=0x7fc69419a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:45.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.551+0000 7fc69ae00640 1 --2- 172.21.15.5:0/3009055848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc694100830 0x7fc69419a4f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc68800e9f0 tx=0x7fc68800eeb0 comp rx=0 tx=0).stop 2024-04-24T06:43:45.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.552+0000 7fc69ae00640 1 -- 172.21.15.5:0/3009055848 shutdown_connections 2024-04-24T06:43:45.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.552+0000 7fc69ae00640 1 --2- 172.21.15.5:0/3009055848 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fc67c0a3ba0 0x7fc67c0a6060 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:45.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.552+0000 7fc69ae00640 1 --2- 172.21.15.5:0/3009055848 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc694101b30 0x7fc6941a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:45.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.552+0000 7fc69ae00640 1 --2- 172.21.15.5:0/3009055848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc694101190 0x7fc69419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:45.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.552+0000 7fc69ae00640 1 --2- 172.21.15.5:0/3009055848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc694100830 0x7fc69419a4f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:45.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.552+0000 7fc69ae00640 1 -- 172.21.15.5:0/3009055848 >> 172.21.15.5:0/3009055848 conn(0x7fc6940fc3d0 msgr2=0x7fc6940fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:45.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.552+0000 7fc69ae00640 1 -- 172.21.15.5:0/3009055848 shutdown_connections 2024-04-24T06:43:45.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.552+0000 7fc69ae00640 1 -- 172.21.15.5:0/3009055848 wait complete. 2024-04-24T06:43:45.574 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:36:27.320342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:41:32.254502+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:41:33.739067+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:43:45.574 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:43:45.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.794+0000 7f45bc000640 1 -- 172.21.15.5:0/3668902211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45b4109850 msgr2=0x7f45b4109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.794+0000 7f45bc000640 1 --2- 172.21.15.5:0/3668902211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45b4109850 0x7f45b4109cd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f45a0009900 tx=0x7f45a002a410 comp rx=0 tx=0).stop 2024-04-24T06:43:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.794+0000 7f45bc000640 1 -- 172.21.15.5:0/3668902211 shutdown_connections 2024-04-24T06:43:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.794+0000 7f45bc000640 1 --2- 172.21.15.5:0/3668902211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45b410a210 0x7f45b4110aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.794+0000 7f45bc000640 1 --2- 172.21.15.5:0/3668902211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45b4109850 0x7f45b4109cd0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.794+0000 7f45bc000640 1 --2- 172.21.15.5:0/3668902211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45b4108660 0x7f45b4108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.794+0000 7f45bc000640 1 -- 172.21.15.5:0/3668902211 >> 172.21.15.5:0/3668902211 conn(0x7f45b4077610 msgr2=0x7f45b4079a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.794+0000 7f45bc000640 1 -- 172.21.15.5:0/3668902211 shutdown_connections 2024-04-24T06:43:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.794+0000 7f45bc000640 1 -- 172.21.15.5:0/3668902211 wait complete. 2024-04-24T06:43:45.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.796+0000 7f45bc000640 1 Processor -- start 2024-04-24T06:43:45.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.796+0000 7f45bc000640 1 -- start start 2024-04-24T06:43:45.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.796+0000 7f45bc000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45b4108660 0x7f45b419e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:45.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.796+0000 7f45bc000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45b4109850 0x7f45b419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:45.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.796+0000 7f45bc000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45b410a210 0x7f45b41a5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:45.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.796+0000 7f45bc000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f45b4114310 con 0x7f45b410a210 2024-04-24T06:43:45.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.796+0000 7f45bc000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f45b4114190 con 0x7f45b4108660 2024-04-24T06:43:45.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.796+0000 7f45bc000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f45b4114490 con 0x7f45b4109850 2024-04-24T06:43:45.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.797+0000 7f45ba000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45b410a210 0x7f45b41a5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:45.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.797+0000 7f45b9600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45b4108660 0x7f45b419e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.797+0000 7f45ba000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45b410a210 0x7f45b41a5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39478/0 (socket says 172.21.15.5:39478) 2024-04-24T06:43:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.797+0000 7f45ba000640 1 -- 172.21.15.5:0/381094425 learned_addr learned my addr 172.21.15.5:0/381094425 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.797+0000 7f45b9600640 1 --2- 172.21.15.5:0/381094425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45b4108660 0x7f45b419e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:60974/0 (socket says 172.21.15.5:60974) 2024-04-24T06:43:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.797+0000 7f45b8c00640 1 --2- 172.21.15.5:0/381094425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45b4109850 0x7f45b419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.797+0000 7f45ba000640 1 -- 172.21.15.5:0/381094425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45b4109850 msgr2=0x7f45b419eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.797+0000 7f45ba000640 1 --2- 172.21.15.5:0/381094425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45b4109850 0x7f45b419eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.797+0000 7f45ba000640 1 -- 172.21.15.5:0/381094425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45b4108660 msgr2=0x7f45b419e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.797+0000 7f45ba000640 1 --2- 172.21.15.5:0/381094425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45b4108660 0x7f45b419e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.797+0000 7f45ba000640 1 -- 172.21.15.5:0/381094425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b41a65a0 con 0x7f45b410a210 2024-04-24T06:43:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.797+0000 7f45ba000640 1 --2- 172.21.15.5:0/381094425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45b410a210 0x7f45b41a5f30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00eb10 tx=0x7f45ac00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.798+0000 7f45b2000640 1 -- 172.21.15.5:0/381094425 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac004b20 con 0x7f45b410a210 2024-04-24T06:43:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.798+0000 7f45bc000640 1 -- 172.21.15.5:0/381094425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45b41a6890 con 0x7f45b410a210 2024-04-24T06:43:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.798+0000 7f45b2000640 1 -- 172.21.15.5:0/381094425 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45ac004cc0 con 0x7f45b410a210 2024-04-24T06:43:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.798+0000 7f45b2000640 1 -- 172.21.15.5:0/381094425 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac010760 con 0x7f45b410a210 2024-04-24T06:43:45.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.798+0000 7f45bc000640 1 -- 172.21.15.5:0/381094425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45b41a6db0 con 0x7f45b410a210 2024-04-24T06:43:45.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.799+0000 7f45bc000640 1 -- 172.21.15.5:0/381094425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45b410fa00 con 0x7f45b410a210 2024-04-24T06:43:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.801+0000 7f45b2000640 1 -- 172.21.15.5:0/381094425 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f45ac005020 con 0x7f45b410a210 2024-04-24T06:43:45.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.802+0000 7f45b2000640 1 --2- 172.21.15.5:0/381094425 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f459c0a3bb0 0x7f459c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:45.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.803+0000 7f45b2000640 1 -- 172.21.15.5:0/381094425 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f45ac0d09b0 con 0x7f45b410a210 2024-04-24T06:43:45.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.803+0000 7f45b9600640 1 --2- 172.21.15.5:0/381094425 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f459c0a3bb0 0x7f459c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:45.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.804+0000 7f45b9600640 1 --2- 172.21.15.5:0/381094425 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f459c0a3bb0 0x7f459c0a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f45a80095f0 tx=0x7f45a8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:45.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:45.807+0000 7f45b2000640 1 -- 172.21.15.5:0/381094425 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f45ac09d350 con 0x7f45b410a210 2024-04-24T06:43:46.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.014+0000 7f45bc000640 1 -- 172.21.15.5:0/381094425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f45b4062c00 con 0x7f45b410a210 2024-04-24T06:43:46.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.015+0000 7f45b2000640 1 -- 172.21.15.5:0/381094425 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f45ac09cc50 con 0x7f45b410a210 2024-04-24T06:43:46.016 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-24T06:43:46.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.018+0000 7f45bc000640 1 -- 172.21.15.5:0/381094425 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f459c0a3bb0 msgr2=0x7f459c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:46.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.018+0000 7f45bc000640 1 --2- 172.21.15.5:0/381094425 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f459c0a3bb0 0x7f459c0a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f45a80095f0 tx=0x7f45a8005f90 comp rx=0 tx=0).stop 2024-04-24T06:43:46.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.018+0000 7f45bc000640 1 -- 172.21.15.5:0/381094425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45b410a210 msgr2=0x7f45b41a5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:46.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.018+0000 7f45bc000640 1 --2- 172.21.15.5:0/381094425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45b410a210 0x7f45b41a5f30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00eb10 tx=0x7f45ac00c2d0 comp rx=0 tx=0).stop 2024-04-24T06:43:46.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.018+0000 7f45bc000640 1 -- 172.21.15.5:0/381094425 shutdown_connections 2024-04-24T06:43:46.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.018+0000 7f45bc000640 1 --2- 172.21.15.5:0/381094425 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f459c0a3bb0 0x7f459c0a6070 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:46.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.018+0000 7f45bc000640 1 --2- 172.21.15.5:0/381094425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45b410a210 0x7f45b41a5f30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:46.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.018+0000 7f45bc000640 1 --2- 172.21.15.5:0/381094425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45b4109850 0x7f45b419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:46.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.018+0000 7f45bc000640 1 --2- 172.21.15.5:0/381094425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45b4108660 0x7f45b419e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:46.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.018+0000 7f45bc000640 1 -- 172.21.15.5:0/381094425 >> 172.21.15.5:0/381094425 conn(0x7f45b4077610 msgr2=0x7f45b410cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:46.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.018+0000 7f45bc000640 1 -- 172.21.15.5:0/381094425 shutdown_connections 2024-04-24T06:43:46.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.018+0000 7f45bc000640 1 -- 172.21.15.5:0/381094425 wait complete. 2024-04-24T06:43:46.039 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:43:46.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.255+0000 7f710ba00640 1 -- 172.21.15.5:0/4204361895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7104105450 msgr2=0x7f71041058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:46.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.255+0000 7f710ba00640 1 --2- 172.21.15.5:0/4204361895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7104105450 0x7f71041058d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f70f0009860 tx=0x7f70f002a340 comp rx=0 tx=0).stop 2024-04-24T06:43:46.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.255+0000 7f710ba00640 1 -- 172.21.15.5:0/4204361895 shutdown_connections 2024-04-24T06:43:46.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.255+0000 7f710ba00640 1 --2- 172.21.15.5:0/4204361895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7104105e10 0x7f710410c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:46.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.255+0000 7f710ba00640 1 --2- 172.21.15.5:0/4204361895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7104105450 0x7f71041058d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:46.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.255+0000 7f710ba00640 1 --2- 172.21.15.5:0/4204361895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7104104260 0x7f7104104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:46.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.255+0000 7f710ba00640 1 -- 172.21.15.5:0/4204361895 >> 172.21.15.5:0/4204361895 conn(0x7f71040ff7f0 msgr2=0x7f7104101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:46.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.255+0000 7f710ba00640 1 -- 172.21.15.5:0/4204361895 shutdown_connections 2024-04-24T06:43:46.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.255+0000 7f710ba00640 1 -- 172.21.15.5:0/4204361895 wait complete. 2024-04-24T06:43:46.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.257+0000 7f710ba00640 1 Processor -- start 2024-04-24T06:43:46.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.257+0000 7f710ba00640 1 -- start start 2024-04-24T06:43:46.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.257+0000 7f710ba00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7104104260 0x7f710419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:46.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.257+0000 7f710ba00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7104105450 0x7f710419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:46.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.257+0000 7f710ba00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7104105e10 0x7f71041a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:46.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.257+0000 7f710ba00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f710410fef0 con 0x7f7104105e10 2024-04-24T06:43:46.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.257+0000 7f710ba00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f710410fd70 con 0x7f7104105450 2024-04-24T06:43:46.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.257+0000 7f710ba00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7104110070 con 0x7f7104104260 2024-04-24T06:43:46.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.258+0000 7f7109000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7104104260 0x7f710419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:46.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.258+0000 7f7109a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7104105e10 0x7f71041a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:46.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.258+0000 7f7109000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7104104260 0x7f710419a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:54196/0 (socket says 172.21.15.5:54196) 2024-04-24T06:43:46.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.258+0000 7f7109000640 1 -- 172.21.15.5:0/2210146135 learned_addr learned my addr 172.21.15.5:0/2210146135 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:46.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.258+0000 7f7103e00640 1 --2- 172.21.15.5:0/2210146135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7104105450 0x7f710419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:46.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.258+0000 7f7109a00640 1 -- 172.21.15.5:0/2210146135 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7104104260 msgr2=0x7f710419a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:46.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.258+0000 7f7109a00640 1 --2- 172.21.15.5:0/2210146135 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7104104260 0x7f710419a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:46.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.258+0000 7f7109a00640 1 -- 172.21.15.5:0/2210146135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7104105450 msgr2=0x7f710419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:46.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.258+0000 7f7109a00640 1 --2- 172.21.15.5:0/2210146135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7104105450 0x7f710419aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:46.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.258+0000 7f7109a00640 1 -- 172.21.15.5:0/2210146135 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71041a2160 con 0x7f7104105e10 2024-04-24T06:43:46.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.258+0000 7f7103e00640 1 --2- 172.21.15.5:0/2210146135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7104105450 0x7f710419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:43:46.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.258+0000 7f7109a00640 1 --2- 172.21.15.5:0/2210146135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7104105e10 0x7f71041a1af0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f70fc009fd0 tx=0x7f70fc00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:46.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.259+0000 7f7101600640 1 -- 172.21.15.5:0/2210146135 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc00ce20 con 0x7f7104105e10 2024-04-24T06:43:46.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.259+0000 7f7101600640 1 -- 172.21.15.5:0/2210146135 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70fc005020 con 0x7f7104105e10 2024-04-24T06:43:46.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.259+0000 7f710ba00640 1 -- 172.21.15.5:0/2210146135 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71041a2450 con 0x7f7104105e10 2024-04-24T06:43:46.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.259+0000 7f7101600640 1 -- 172.21.15.5:0/2210146135 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc010720 con 0x7f7104105e10 2024-04-24T06:43:46.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.259+0000 7f710ba00640 1 -- 172.21.15.5:0/2210146135 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71041a29c0 con 0x7f7104105e10 2024-04-24T06:43:46.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.260+0000 7f710ba00640 1 -- 172.21.15.5:0/2210146135 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f710410b580 con 0x7f7104105e10 2024-04-24T06:43:46.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.262+0000 7f7101600640 1 -- 172.21.15.5:0/2210146135 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f70fc005540 con 0x7f7104105e10 2024-04-24T06:43:46.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.264+0000 7f7101600640 1 --2- 172.21.15.5:0/2210146135 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f70ec0a3c30 0x7f70ec0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:46.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.264+0000 7f7101600640 1 -- 172.21.15.5:0/2210146135 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f70fc0d0a60 con 0x7f7104105e10 2024-04-24T06:43:46.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.264+0000 7f7109000640 1 --2- 172.21.15.5:0/2210146135 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f70ec0a3c30 0x7f70ec0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:46.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.265+0000 7f7109000640 1 --2- 172.21.15.5:0/2210146135 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f70ec0a3c30 0x7f70ec0a60f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f70f80096c0 tx=0x7f70f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:46.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.271+0000 7f7101600640 1 -- 172.21.15.5:0/2210146135 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f70fc09d490 con 0x7f7104105e10 2024-04-24T06:43:46.427 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:46 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/3009055848' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:43:46.427 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:46 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/381094425' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T06:43:46.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:46 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/3009055848' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:43:46.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:46 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/381094425' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T06:43:46.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.480+0000 7f710ba00640 1 -- 172.21.15.5:0/2210146135 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7104062c00 con 0x7f7104105e10 2024-04-24T06:43:46.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.481+0000 7f7101600640 1 -- 172.21.15.5:0/2210146135 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f70fc09cd90 con 0x7f7104105e10 2024-04-24T06:43:46.482 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-24T06:43:46.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.483+0000 7f710ba00640 1 -- 172.21.15.5:0/2210146135 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f70ec0a3c30 msgr2=0x7f70ec0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:46.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.483+0000 7f710ba00640 1 --2- 172.21.15.5:0/2210146135 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f70ec0a3c30 0x7f70ec0a60f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f70f80096c0 tx=0x7f70f8009260 comp rx=0 tx=0).stop 2024-04-24T06:43:46.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.484+0000 7f710ba00640 1 -- 172.21.15.5:0/2210146135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7104105e10 msgr2=0x7f71041a1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:46.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.484+0000 7f710ba00640 1 --2- 172.21.15.5:0/2210146135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7104105e10 0x7f71041a1af0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f70fc009fd0 tx=0x7f70fc00ef10 comp rx=0 tx=0).stop 2024-04-24T06:43:46.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.484+0000 7f710ba00640 1 -- 172.21.15.5:0/2210146135 shutdown_connections 2024-04-24T06:43:46.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.484+0000 7f710ba00640 1 --2- 172.21.15.5:0/2210146135 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f70ec0a3c30 0x7f70ec0a60f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:46.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.484+0000 7f710ba00640 1 --2- 172.21.15.5:0/2210146135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7104105e10 0x7f71041a1af0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:46.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.484+0000 7f710ba00640 1 --2- 172.21.15.5:0/2210146135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7104105450 0x7f710419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:46.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.484+0000 7f710ba00640 1 --2- 172.21.15.5:0/2210146135 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7104104260 0x7f710419a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:46.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.484+0000 7f710ba00640 1 -- 172.21.15.5:0/2210146135 >> 172.21.15.5:0/2210146135 conn(0x7f71040ff7f0 msgr2=0x7f7104108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:46.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.484+0000 7f710ba00640 1 -- 172.21.15.5:0/2210146135 shutdown_connections 2024-04-24T06:43:46.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.484+0000 7f710ba00640 1 -- 172.21.15.5:0/2210146135 wait complete. 2024-04-24T06:43:46.505 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:43:46.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:46 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/3009055848' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:43:46.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:46 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/381094425' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T06:43:46.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.728+0000 7fb83ae00640 1 -- 172.21.15.5:0/2653563271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb834103250 msgr2=0x7fb8341036d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:46.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.728+0000 7fb83ae00640 1 --2- 172.21.15.5:0/2653563271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb834103250 0x7fb8341036d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb820009900 tx=0x7fb82002a3f0 comp rx=0 tx=0).stop 2024-04-24T06:43:46.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.729+0000 7fb83ae00640 1 -- 172.21.15.5:0/2653563271 shutdown_connections 2024-04-24T06:43:46.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.729+0000 7fb83ae00640 1 --2- 172.21.15.5:0/2653563271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb834103c10 0x7fb83410a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:46.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.729+0000 7fb83ae00640 1 --2- 172.21.15.5:0/2653563271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb834103250 0x7fb8341036d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:46.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.729+0000 7fb83ae00640 1 --2- 172.21.15.5:0/2653563271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb834102060 0x7fb834102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:46.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.729+0000 7fb83ae00640 1 -- 172.21.15.5:0/2653563271 >> 172.21.15.5:0/2653563271 conn(0x7fb8340fd5d0 msgr2=0x7fb8340ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:46.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.729+0000 7fb83ae00640 1 -- 172.21.15.5:0/2653563271 shutdown_connections 2024-04-24T06:43:46.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.729+0000 7fb83ae00640 1 -- 172.21.15.5:0/2653563271 wait complete. 2024-04-24T06:43:46.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.730+0000 7fb83ae00640 1 Processor -- start 2024-04-24T06:43:46.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.730+0000 7fb83ae00640 1 -- start start 2024-04-24T06:43:46.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.730+0000 7fb83ae00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb834102060 0x7fb83419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:46.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.730+0000 7fb83ae00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb834103250 0x7fb83419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:46.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.731+0000 7fb83ae00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb834103c10 0x7fb8341a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:46.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.731+0000 7fb83ae00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb83410dcf0 con 0x7fb834103c10 2024-04-24T06:43:46.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.731+0000 7fb83ae00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb83410db70 con 0x7fb834103250 2024-04-24T06:43:46.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.731+0000 7fb83ae00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb83410de70 con 0x7fb834102060 2024-04-24T06:43:46.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.731+0000 7fb838e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb834103c10 0x7fb8341a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:46.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.731+0000 7fb838e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb834103c10 0x7fb8341a1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39512/0 (socket says 172.21.15.5:39512) 2024-04-24T06:43:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.731+0000 7fb838e00640 1 -- 172.21.15.5:0/987195387 learned_addr learned my addr 172.21.15.5:0/987195387 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.731+0000 7fb833400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb834103250 0x7fb83419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.731+0000 7fb833e00640 1 --2- 172.21.15.5:0/987195387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb834102060 0x7fb83419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.731+0000 7fb838e00640 1 -- 172.21.15.5:0/987195387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb834102060 msgr2=0x7fb83419a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.731+0000 7fb838e00640 1 --2- 172.21.15.5:0/987195387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb834102060 0x7fb83419a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.731+0000 7fb838e00640 1 -- 172.21.15.5:0/987195387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb834103250 msgr2=0x7fb83419aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.731+0000 7fb838e00640 1 --2- 172.21.15.5:0/987195387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb834103250 0x7fb83419aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.731+0000 7fb838e00640 1 -- 172.21.15.5:0/987195387 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8341a2170 con 0x7fb834103c10 2024-04-24T06:43:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.731+0000 7fb833400640 1 --2- 172.21.15.5:0/987195387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb834103250 0x7fb83419aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:43:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.732+0000 7fb838e00640 1 --2- 172.21.15.5:0/987195387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb834103c10 0x7fb8341a1b00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb82c00eb10 tx=0x7fb82c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.732+0000 7fb830c00640 1 -- 172.21.15.5:0/987195387 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c004b20 con 0x7fb834103c10 2024-04-24T06:43:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.732+0000 7fb830c00640 1 -- 172.21.15.5:0/987195387 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb82c004cc0 con 0x7fb834103c10 2024-04-24T06:43:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.732+0000 7fb83ae00640 1 -- 172.21.15.5:0/987195387 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8341a2460 con 0x7fb834103c10 2024-04-24T06:43:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.732+0000 7fb830c00640 1 -- 172.21.15.5:0/987195387 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c010760 con 0x7fb834103c10 2024-04-24T06:43:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.732+0000 7fb83ae00640 1 -- 172.21.15.5:0/987195387 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8341a2980 con 0x7fb834103c10 2024-04-24T06:43:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.734+0000 7fb83ae00640 1 -- 172.21.15.5:0/987195387 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8341a2c40 con 0x7fb834103c10 2024-04-24T06:43:46.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.736+0000 7fb830c00640 1 -- 172.21.15.5:0/987195387 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7fb82c005020 con 0x7fb834103c10 2024-04-24T06:43:46.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.737+0000 7fb830c00640 1 --2- 172.21.15.5:0/987195387 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fb8180a3bb0 0x7fb8180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:46.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.738+0000 7fb830c00640 1 -- 172.21.15.5:0/987195387 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fb82c0d0ad0 con 0x7fb834103c10 2024-04-24T06:43:46.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.738+0000 7fb833e00640 1 --2- 172.21.15.5:0/987195387 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fb8180a3bb0 0x7fb8180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.738+0000 7fb833e00640 1 --2- 172.21.15.5:0/987195387 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fb8180a3bb0 0x7fb8180a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb8341030b0 tx=0x7fb824005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:46.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.741+0000 7fb830c00640 1 -- 172.21.15.5:0/987195387 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb82c09d470 con 0x7fb834103c10 2024-04-24T06:43:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.948+0000 7fb83ae00640 1 -- 172.21.15.5:0/987195387 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb8341a2e10 con 0x7fb834103c10 2024-04-24T06:43:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.948+0000 7fb830c00640 1 -- 172.21.15.5:0/987195387 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7fb82c09cd70 con 0x7fb834103c10 2024-04-24T06:43:46.950 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-24T06:43:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.951+0000 7fb83ae00640 1 -- 172.21.15.5:0/987195387 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fb8180a3bb0 msgr2=0x7fb8180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.951+0000 7fb83ae00640 1 --2- 172.21.15.5:0/987195387 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fb8180a3bb0 0x7fb8180a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb8341030b0 tx=0x7fb824005f90 comp rx=0 tx=0).stop 2024-04-24T06:43:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.951+0000 7fb83ae00640 1 -- 172.21.15.5:0/987195387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb834103c10 msgr2=0x7fb8341a1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.951+0000 7fb83ae00640 1 --2- 172.21.15.5:0/987195387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb834103c10 0x7fb8341a1b00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb82c00eb10 tx=0x7fb82c00c2d0 comp rx=0 tx=0).stop 2024-04-24T06:43:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.951+0000 7fb83ae00640 1 -- 172.21.15.5:0/987195387 shutdown_connections 2024-04-24T06:43:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.951+0000 7fb83ae00640 1 --2- 172.21.15.5:0/987195387 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fb8180a3bb0 0x7fb8180a6070 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.951+0000 7fb83ae00640 1 --2- 172.21.15.5:0/987195387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb834103c10 0x7fb8341a1b00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.951+0000 7fb83ae00640 1 --2- 172.21.15.5:0/987195387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb834103250 0x7fb83419aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.951+0000 7fb83ae00640 1 --2- 172.21.15.5:0/987195387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb834102060 0x7fb83419a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.951+0000 7fb83ae00640 1 -- 172.21.15.5:0/987195387 >> 172.21.15.5:0/987195387 conn(0x7fb8340fd5d0 msgr2=0x7fb834074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.951+0000 7fb83ae00640 1 -- 172.21.15.5:0/987195387 shutdown_connections 2024-04-24T06:43:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:46.951+0000 7fb83ae00640 1 -- 172.21.15.5:0/987195387 wait complete. 2024-04-24T06:43:46.980 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:43:47.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.176+0000 7f5de7e00640 1 -- 172.21.15.5:0/177628205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5de00769c0 msgr2=0x7f5de0076de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:47.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.176+0000 7f5de7e00640 1 --2- 172.21.15.5:0/177628205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5de00769c0 0x7f5de0076de0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4009860 tx=0x7f5dd402a340 comp rx=0 tx=0).stop 2024-04-24T06:43:47.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.176+0000 7f5de7e00640 1 -- 172.21.15.5:0/177628205 shutdown_connections 2024-04-24T06:43:47.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.176+0000 7f5de7e00640 1 --2- 172.21.15.5:0/177628205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5de00785a0 0x7f5de0078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:47.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.176+0000 7f5de7e00640 1 --2- 172.21.15.5:0/177628205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5de0077c00 0x7f5de0078060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:47.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.176+0000 7f5de7e00640 1 --2- 172.21.15.5:0/177628205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5de00769c0 0x7f5de0076de0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:47.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.176+0000 7f5de7e00640 1 -- 172.21.15.5:0/177628205 >> 172.21.15.5:0/177628205 conn(0x7f5de00ff850 msgr2=0x7f5de0101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:47.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.176+0000 7f5de7e00640 1 -- 172.21.15.5:0/177628205 shutdown_connections 2024-04-24T06:43:47.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.176+0000 7f5de7e00640 1 -- 172.21.15.5:0/177628205 wait complete. 2024-04-24T06:43:47.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.177+0000 7f5de7e00640 1 Processor -- start 2024-04-24T06:43:47.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.177+0000 7f5de7e00640 1 -- start start 2024-04-24T06:43:47.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.178+0000 7f5de7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5de00769c0 0x7f5de019e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:47.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.178+0000 7f5de7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5de0077c00 0x7f5de019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:47.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.178+0000 7f5de7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5de00785a0 0x7f5de01a5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:47.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.178+0000 7f5de7e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5de0114300 con 0x7f5de0077c00 2024-04-24T06:43:47.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.178+0000 7f5de7e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5de0114180 con 0x7f5de00769c0 2024-04-24T06:43:47.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.178+0000 7f5de7e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5de0114480 con 0x7f5de00785a0 2024-04-24T06:43:47.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.178+0000 7f5de4a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5de0077c00 0x7f5de019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:47.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.178+0000 7f5de4a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5de0077c00 0x7f5de019eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39528/0 (socket says 172.21.15.5:39528) 2024-04-24T06:43:47.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.178+0000 7f5de4a00640 1 -- 172.21.15.5:0/578590792 learned_addr learned my addr 172.21.15.5:0/578590792 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:47.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.178+0000 7f5de5400640 1 --2- 172.21.15.5:0/578590792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5de00769c0 0x7f5de019e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:47.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.178+0000 7f5de5e00640 1 --2- 172.21.15.5:0/578590792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5de00785a0 0x7f5de01a5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:47.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.178+0000 7f5de5400640 1 -- 172.21.15.5:0/578590792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5de00785a0 msgr2=0x7f5de01a5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:47.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.178+0000 7f5de5400640 1 --2- 172.21.15.5:0/578590792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5de00785a0 0x7f5de01a5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:47.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.178+0000 7f5de5400640 1 -- 172.21.15.5:0/578590792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5de0077c00 msgr2=0x7f5de019eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:47.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.178+0000 7f5de5400640 1 --2- 172.21.15.5:0/578590792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5de0077c00 0x7f5de019eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:47.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.178+0000 7f5de5400640 1 -- 172.21.15.5:0/578590792 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de01a65a0 con 0x7f5de00769c0 2024-04-24T06:43:47.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.178+0000 7f5de4a00640 1 --2- 172.21.15.5:0/578590792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5de0077c00 0x7f5de019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:43:47.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.179+0000 7f5de5400640 1 --2- 172.21.15.5:0/578590792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5de00769c0 0x7f5de019e980 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4035040 tx=0x7f5dd4033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:47.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.179+0000 7f5dde000640 1 -- 172.21.15.5:0/578590792 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4037040 con 0x7f5de00769c0 2024-04-24T06:43:47.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.179+0000 7f5dde000640 1 -- 172.21.15.5:0/578590792 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dd4030850 con 0x7f5de00769c0 2024-04-24T06:43:47.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.179+0000 7f5dde000640 1 -- 172.21.15.5:0/578590792 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4004920 con 0x7f5de00769c0 2024-04-24T06:43:47.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.179+0000 7f5de7e00640 1 -- 172.21.15.5:0/578590792 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5de01a6830 con 0x7f5de00769c0 2024-04-24T06:43:47.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.179+0000 7f5de7e00640 1 -- 172.21.15.5:0/578590792 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5de01a6ca0 con 0x7f5de00769c0 2024-04-24T06:43:47.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.181+0000 7f5de7e00640 1 -- 172.21.15.5:0/578590792 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5de0110e40 con 0x7f5de00769c0 2024-04-24T06:43:47.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.182+0000 7f5dde000640 1 -- 172.21.15.5:0/578590792 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f5dd4004050 con 0x7f5de00769c0 2024-04-24T06:43:47.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.184+0000 7f5dde000640 1 --2- 172.21.15.5:0/578590792 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f5dc80a3c30 0x7f5dc80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:47.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.184+0000 7f5dde000640 1 -- 172.21.15.5:0/578590792 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f5dd40eff70 con 0x7f5de00769c0 2024-04-24T06:43:47.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.184+0000 7f5de4a00640 1 --2- 172.21.15.5:0/578590792 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f5dc80a3c30 0x7f5dc80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:47.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.185+0000 7f5de4a00640 1 --2- 172.21.15.5:0/578590792 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f5dc80a3c30 0x7f5dc80a60f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5de019fe90 tx=0x7f5dcc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:47.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.189+0000 7f5dde000640 1 -- 172.21.15.5:0/578590792 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5dd4049020 con 0x7f5de00769c0 2024-04-24T06:43:47.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.433+0000 7f5de7e00640 1 -- 172.21.15.5:0/578590792 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5de0062c00 con 0x7f5de00769c0 2024-04-24T06:43:47.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.434+0000 7f5dde000640 1 -- 172.21.15.5:0/578590792 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11856 (secure 0 0 0) 0x7f5dd40bc910 con 0x7f5de00769c0 2024-04-24T06:43:47.435 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:43:47.435 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1735462750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1735462750},{"type":"v1","addr":"172.21.15.5:6837","nonce":1735462750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1499469370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1499469370},{"type":"v1","addr":"172.21.15.5:6841","nonce":1499469370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3705638847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3705638847},{"type":"v1","addr":"172.21.15.37:6835","nonce":3705638847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T06:41:34.764490+0000","modified":"2024-04-24T06:41:40.330991+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24550,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6837/1390103296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1390103296},{"type":"v1","addr":"172.21.15.37:6837","nonce":1390103296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3217754215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3217754215},{"type":"v1","addr":"172.21.15.137:6835","nonce":3217754215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6837/3044222976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3044222976},{"type":"v1","addr":"172.21.15.137:6837","nonce":3044222976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.5:6839/3173201459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3173201459},{"type":"v1","addr":"172.21.15.5:6839","nonce":3173201459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6839/1889774232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1889774232},{"type":"v1","addr":"172.21.15.137:6839","nonce":1889774232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6839/3536721753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":3536721753},{"type":"v1","addr":"172.21.15.37:6839","nonce":3536721753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24539,24550,24569]},"id":1}]} 2024-04-24T06:43:47.435 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 22 2024-04-24T06:43:47.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.437+0000 7f5de7e00640 1 -- 172.21.15.5:0/578590792 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f5dc80a3c30 msgr2=0x7f5dc80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:47.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.437+0000 7f5de7e00640 1 --2- 172.21.15.5:0/578590792 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f5dc80a3c30 0x7f5dc80a60f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5de019fe90 tx=0x7f5dcc005f90 comp rx=0 tx=0).stop 2024-04-24T06:43:47.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.437+0000 7f5de7e00640 1 -- 172.21.15.5:0/578590792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5de00769c0 msgr2=0x7f5de019e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:47.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.437+0000 7f5de7e00640 1 --2- 172.21.15.5:0/578590792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5de00769c0 0x7f5de019e980 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4035040 tx=0x7f5dd4033d90 comp rx=0 tx=0).stop 2024-04-24T06:43:47.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.437+0000 7f5de7e00640 1 -- 172.21.15.5:0/578590792 shutdown_connections 2024-04-24T06:43:47.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.437+0000 7f5de7e00640 1 --2- 172.21.15.5:0/578590792 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f5dc80a3c30 0x7f5dc80a60f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:47.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.437+0000 7f5de7e00640 1 --2- 172.21.15.5:0/578590792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5de00785a0 0x7f5de01a5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:47.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.437+0000 7f5de7e00640 1 --2- 172.21.15.5:0/578590792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5de0077c00 0x7f5de019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:47.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.437+0000 7f5de7e00640 1 --2- 172.21.15.5:0/578590792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5de00769c0 0x7f5de019e980 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:47.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.437+0000 7f5de7e00640 1 -- 172.21.15.5:0/578590792 >> 172.21.15.5:0/578590792 conn(0x7f5de00ff850 msgr2=0x7f5de0100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:47.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.437+0000 7f5de7e00640 1 -- 172.21.15.5:0/578590792 shutdown_connections 2024-04-24T06:43:47.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.437+0000 7f5de7e00640 1 -- 172.21.15.5:0/578590792 wait complete. 2024-04-24T06:43:47.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:47 smithi137 ceph-mon[33017]: pgmap v380: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:43:47.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:47 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/2210146135' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T06:43:47.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:47 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/987195387' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T06:43:47.453 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:47 smithi005 ceph-mon[27798]: pgmap v380: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:43:47.453 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:47 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/2210146135' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T06:43:47.453 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:47 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/987195387' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T06:43:47.457 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:43:47.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:47 smithi037 ceph-mon[32864]: pgmap v380: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:43:47.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:47 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/2210146135' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T06:43:47.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:47 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/987195387' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T06:43:47.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.643+0000 7fa0b3e00640 1 -- 172.21.15.5:0/3542114541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0b4076040 msgr2=0x7fa0b4074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:47.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.643+0000 7fa0b3e00640 1 --2- 172.21.15.5:0/3542114541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0b4076040 0x7fa0b4074490 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa09c009900 tx=0x7fa09c02a410 comp rx=0 tx=0).stop 2024-04-24T06:43:47.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.643+0000 7fa0b3e00640 1 -- 172.21.15.5:0/3542114541 shutdown_connections 2024-04-24T06:43:47.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.643+0000 7fa0b3e00640 1 --2- 172.21.15.5:0/3542114541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0b4109e10 0x7fa0b410c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:47.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.643+0000 7fa0b3e00640 1 --2- 172.21.15.5:0/3542114541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0b4074a60 0x7fa0b4074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:47.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.643+0000 7fa0b3e00640 1 --2- 172.21.15.5:0/3542114541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0b4076040 0x7fa0b4074490 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:47.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.643+0000 7fa0b3e00640 1 -- 172.21.15.5:0/3542114541 >> 172.21.15.5:0/3542114541 conn(0x7fa0b40fd4c0 msgr2=0x7fa0b40ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:47.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.643+0000 7fa0b3e00640 1 -- 172.21.15.5:0/3542114541 shutdown_connections 2024-04-24T06:43:47.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.643+0000 7fa0b3e00640 1 -- 172.21.15.5:0/3542114541 wait complete. 2024-04-24T06:43:47.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.644+0000 7fa0b3e00640 1 Processor -- start 2024-04-24T06:43:47.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.645+0000 7fa0b3e00640 1 -- start start 2024-04-24T06:43:47.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.645+0000 7fa0b3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0b4074a60 0x7fa0b419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:47.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.645+0000 7fa0b3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0b4076040 0x7fa0b419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:47.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.645+0000 7fa0b3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0b4109e10 0x7fa0b41a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:47.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.645+0000 7fa0b3e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b410fe20 con 0x7fa0b4109e10 2024-04-24T06:43:47.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.645+0000 7fa0b3e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b410fca0 con 0x7fa0b4076040 2024-04-24T06:43:47.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.645+0000 7fa0b3e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b410ffa0 con 0x7fa0b4074a60 2024-04-24T06:43:47.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.645+0000 7fa0b2a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0b4074a60 0x7fa0b419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:47.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.645+0000 7fa0b2a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0b4074a60 0x7fa0b419a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:54256/0 (socket says 172.21.15.5:54256) 2024-04-24T06:43:47.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.645+0000 7fa0b2a00640 1 -- 172.21.15.5:0/1583127665 learned_addr learned my addr 172.21.15.5:0/1583127665 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:47.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.645+0000 7fa0b3400640 1 --2- 172.21.15.5:0/1583127665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0b4109e10 0x7fa0b41a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:47.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.646+0000 7fa0b2000640 1 --2- 172.21.15.5:0/1583127665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0b4076040 0x7fa0b419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:47.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.646+0000 7fa0b2a00640 1 -- 172.21.15.5:0/1583127665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0b4076040 msgr2=0x7fa0b419a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:47.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.646+0000 7fa0b2a00640 1 --2- 172.21.15.5:0/1583127665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0b4076040 0x7fa0b419a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:47.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.646+0000 7fa0b2a00640 1 -- 172.21.15.5:0/1583127665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0b4109e10 msgr2=0x7fa0b41a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:47.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.646+0000 7fa0b2a00640 1 --2- 172.21.15.5:0/1583127665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0b4109e10 0x7fa0b41a19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:47.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.646+0000 7fa0b2a00640 1 -- 172.21.15.5:0/1583127665 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b41a2050 con 0x7fa0b4074a60 2024-04-24T06:43:47.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.646+0000 7fa0b2a00640 1 --2- 172.21.15.5:0/1583127665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0b4074a60 0x7fa0b419a480 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa09c031440 tx=0x7fa09c032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:47.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.647+0000 7fa0ab400640 1 -- 172.21.15.5:0/1583127665 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09c03f030 con 0x7fa0b4074a60 2024-04-24T06:43:47.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.647+0000 7fa0ab400640 1 -- 172.21.15.5:0/1583127665 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa09c003ce0 con 0x7fa0b4074a60 2024-04-24T06:43:47.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.647+0000 7fa0b3e00640 1 -- 172.21.15.5:0/1583127665 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0b41a22e0 con 0x7fa0b4074a60 2024-04-24T06:43:47.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.647+0000 7fa0ab400640 1 -- 172.21.15.5:0/1583127665 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09c035070 con 0x7fa0b4074a60 2024-04-24T06:43:47.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.647+0000 7fa0b3e00640 1 -- 172.21.15.5:0/1583127665 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0b41a27d0 con 0x7fa0b4074a60 2024-04-24T06:43:47.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.648+0000 7fa0b3e00640 1 -- 172.21.15.5:0/1583127665 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0b4108b40 con 0x7fa0b4074a60 2024-04-24T06:43:47.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.650+0000 7fa0ab400640 1 -- 172.21.15.5:0/1583127665 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7fa09c042030 con 0x7fa0b4074a60 2024-04-24T06:43:47.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.652+0000 7fa0ab400640 1 --2- 172.21.15.5:0/1583127665 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fa0980a3ba0 0x7fa0980a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:47.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.652+0000 7fa0ab400640 1 -- 172.21.15.5:0/1583127665 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fa09c0f0d50 con 0x7fa0b4074a60 2024-04-24T06:43:47.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.652+0000 7fa0b2000640 1 --2- 172.21.15.5:0/1583127665 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fa0980a3ba0 0x7fa0980a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:47.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.653+0000 7fa0b2000640 1 --2- 172.21.15.5:0/1583127665 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fa0980a3ba0 0x7fa0980a6060 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa0b419b940 tx=0x7fa0a4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:47.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.654+0000 7fa0ab400640 1 -- 172.21.15.5:0/1583127665 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa09c0bd6f0 con 0x7fa0b4074a60 2024-04-24T06:43:47.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.851+0000 7fa0b3e00640 1 -- 172.21.15.5:0/1583127665 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa0b4062c00 con 0x7fa0b4074a60 2024-04-24T06:43:47.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.854+0000 7fa0ab400640 1 -- 172.21.15.5:0/1583127665 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22513 (secure 0 0 0) 0x7fa09c0dd070 con 0x7fa0b4074a60 2024-04-24T06:43:47.855 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:43:47.855 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":88,"fsid":"43e9bad0-0204-11ef-bc93-c7b262605968","created":"2024-04-24T06:32:26.340555+0000","modified":"2024-04-24T06:42:42.855911+0000","last_up_change":"2024-04-24T06:40:36.790934+0000","last_in_change":"2024-04-24T06:40:17.803257+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:36:27.320342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:41:32.254502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:41:33.739067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c2e974c-745c-4dd5-8c3d-8e8da2eb1ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6803","nonce":2155858515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6805","nonce":2155858515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6809","nonce":2155858515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6807","nonce":2155858515}]},"public_addr":"172.21.15.5:6803/2155858515","cluster_addr":"172.21.15.5:6805/2155858515","heartbeat_back_addr":"172.21.15.5:6809/2155858515","heartbeat_front_addr":"172.21.15.5:6807/2155858515","state":["exists","up"]},{"osd":1,"uuid":"50467744-db20-4721-bf2d-1c5db938297b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6801","nonce":1302362650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6803","nonce":1302362650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6807","nonce":1302362650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6805","nonce":1302362650}]},"public_addr":"172.21.15.37:6801/1302362650","cluster_addr":"172.21.15.37:6803/1302362650","heartbeat_back_addr":"172.21.15.37:6807/1302362650","heartbeat_front_addr":"172.21.15.37:6805/1302362650","state":["exists","up"]},{"osd":2,"uuid":"0e190b15-d04c-4288-8d48-ecf6664392fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6801","nonce":2885027256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6803","nonce":2885027256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6807","nonce":2885027256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6805","nonce":2885027256}]},"public_addr":"172.21.15.137:6801/2885027256","cluster_addr":"172.21.15.137:6803/2885027256","heartbeat_back_addr":"172.21.15.137:6807/2885027256","heartbeat_front_addr":"172.21.15.137:6805/2885027256","state":["exists","up"]},{"osd":3,"uuid":"e05592ba-d96a-472a-9f63-09a535d04dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6811","nonce":3701747260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6813","nonce":3701747260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6817","nonce":3701747260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6815","nonce":3701747260}]},"public_addr":"172.21.15.5:6811/3701747260","cluster_addr":"172.21.15.5:6813/3701747260","heartbeat_back_addr":"172.21.15.5:6817/3701747260","heartbeat_front_addr":"172.21.15.5:6815/3701747260","state":["exists","up"]},{"osd":4,"uuid":"c3f36e51-7a3b-4dd9-af4b-a10558830c4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6809","nonce":759253844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6811","nonce":759253844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6815","nonce":759253844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6813","nonce":759253844}]},"public_addr":"172.21.15.37:6809/759253844","cluster_addr":"172.21.15.37:6811/759253844","heartbeat_back_addr":"172.21.15.37:6815/759253844","heartbeat_front_addr":"172.21.15.37:6813/759253844","state":["exists","up"]},{"osd":5,"uuid":"97a0500c-cbb8-4ff5-b476-8634b26b3821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6809","nonce":3771921560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6811","nonce":3771921560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6815","nonce":3771921560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6813","nonce":3771921560}]},"public_addr":"172.21.15.137:6809/3771921560","cluster_addr":"172.21.15.137:6811/3771921560","heartbeat_back_addr":"172.21.15.137:6815/3771921560","heartbeat_front_addr":"172.21.15.137:6813/3771921560","state":["exists","up"]},{"osd":6,"uuid":"33a33d62-bbb3-4ed9-834e-19ae93a97670","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6819","nonce":2064636084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6821","nonce":2064636084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6825","nonce":2064636084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6823","nonce":2064636084}]},"public_addr":"172.21.15.5:6819/2064636084","cluster_addr":"172.21.15.5:6821/2064636084","heartbeat_back_addr":"172.21.15.5:6825/2064636084","heartbeat_front_addr":"172.21.15.5:6823/2064636084","state":["exists","up"]},{"osd":7,"uuid":"c0f4e036-4717-4514-bb59-ab11fbaab063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6817","nonce":2952750659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6819","nonce":2952750659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6823","nonce":2952750659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6821","nonce":2952750659}]},"public_addr":"172.21.15.37:6817/2952750659","cluster_addr":"172.21.15.37:6819/2952750659","heartbeat_back_addr":"172.21.15.37:6823/2952750659","heartbeat_front_addr":"172.21.15.37:6821/2952750659","state":["exists","up"]},{"osd":8,"uuid":"5391fd87-44ea-440f-80da-c54d0327c04c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6817","nonce":29724945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6819","nonce":29724945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6823","nonce":29724945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6821","nonce":29724945}]},"public_addr":"172.21.15.137:6817/29724945","cluster_addr":"172.21.15.137:6819/29724945","heartbeat_back_addr":"172.21.15.137:6823/29724945","heartbeat_front_addr":"172.21.15.137:6821/29724945","state":["exists","up"]},{"osd":9,"uuid":"d94c9dbc-f43d-4237-ac7f-47a6a5ede1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6827","nonce":2257102107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6829","nonce":2257102107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6833","nonce":2257102107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6831","nonce":2257102107}]},"public_addr":"172.21.15.5:6827/2257102107","cluster_addr":"172.21.15.5:6829/2257102107","heartbeat_back_addr":"172.21.15.5:6833/2257102107","heartbeat_front_addr":"172.21.15.5:6831/2257102107","state":["exists","up"]},{"osd":10,"uuid":"e2455332-2a5e-44b1-85d3-8e47da399f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6825","nonce":2451570637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6827","nonce":2451570637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6831","nonce":2451570637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6829","nonce":2451570637}]},"public_addr":"172.21.15.37:6825/2451570637","cluster_addr":"172.21.15.37:6827/2451570637","heartbeat_back_addr":"172.21.15.37:6831/2451570637","heartbeat_front_addr":"172.21.15.37:6829/2451570637","state":["exists","up"]},{"osd":11,"uuid":"49d9396b-704a-4cef-8210-75d30c28e924","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6825","nonce":90600596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6827","nonce":90600596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6831","nonce":90600596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6829","nonce":90600596}]},"public_addr":"172.21.15.137:6825/90600596","cluster_addr":"172.21.15.137:6827/90600596","heartbeat_back_addr":"172.21.15.137:6831/90600596","heartbeat_front_addr":"172.21.15.137:6829/90600596","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:35:35.591619+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:00.568817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:24.195643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:56.451724+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:22.109293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:46.161749+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:19.278305+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:45.620256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:10.041165+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:43.813697+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:10.736026+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:35.384377+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/23307164":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6801/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:0/230785941":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/2940023597":"2024-04-25T06:33:07.040369+0000","172.21.15.5:6801/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3886613904":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/4199165437":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3459593499":"2024-04-25T06:33:41.161852+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:43:47.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.856+0000 7fa0b3e00640 1 -- 172.21.15.5:0/1583127665 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fa0980a3ba0 msgr2=0x7fa0980a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:47.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.856+0000 7fa0b3e00640 1 --2- 172.21.15.5:0/1583127665 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fa0980a3ba0 0x7fa0980a6060 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa0b419b940 tx=0x7fa0a4005f90 comp rx=0 tx=0).stop 2024-04-24T06:43:47.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.856+0000 7fa0b3e00640 1 -- 172.21.15.5:0/1583127665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0b4074a60 msgr2=0x7fa0b419a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:47.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.856+0000 7fa0b3e00640 1 --2- 172.21.15.5:0/1583127665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0b4074a60 0x7fa0b419a480 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa09c031440 tx=0x7fa09c032ce0 comp rx=0 tx=0).stop 2024-04-24T06:43:47.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.856+0000 7fa0b3e00640 1 -- 172.21.15.5:0/1583127665 shutdown_connections 2024-04-24T06:43:47.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.856+0000 7fa0b3e00640 1 --2- 172.21.15.5:0/1583127665 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fa0980a3ba0 0x7fa0980a6060 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:47.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.856+0000 7fa0b3e00640 1 --2- 172.21.15.5:0/1583127665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0b4109e10 0x7fa0b41a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:47.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.856+0000 7fa0b3e00640 1 --2- 172.21.15.5:0/1583127665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0b4076040 0x7fa0b419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:47.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.856+0000 7fa0b3e00640 1 --2- 172.21.15.5:0/1583127665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0b4074a60 0x7fa0b419a480 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:47.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.856+0000 7fa0b3e00640 1 -- 172.21.15.5:0/1583127665 >> 172.21.15.5:0/1583127665 conn(0x7fa0b40fd4c0 msgr2=0x7fa0b4105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:47.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.856+0000 7fa0b3e00640 1 -- 172.21.15.5:0/1583127665 shutdown_connections 2024-04-24T06:43:47.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:47.856+0000 7fa0b3e00640 1 -- 172.21.15.5:0/1583127665 wait complete. 2024-04-24T06:43:47.875 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:36:27.320342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:41:32.254502+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:41:33.739067+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:43:47.876 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:43:48.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.084+0000 7f3089600640 1 -- 172.21.15.5:0/1974892299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3084100140 msgr2=0x7f3084100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:48.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.084+0000 7f3089600640 1 --2- 172.21.15.5:0/1974892299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3084100140 0x7f3084100560 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3074009930 tx=0x7f307402a420 comp rx=0 tx=0).stop 2024-04-24T06:43:48.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.084+0000 7f3089600640 1 -- 172.21.15.5:0/1974892299 shutdown_connections 2024-04-24T06:43:48.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.084+0000 7f3089600640 1 --2- 172.21.15.5:0/1974892299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3084103ad0 0x7f308410a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:48.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.084+0000 7f3089600640 1 --2- 172.21.15.5:0/1974892299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3084103110 0x7f3084103590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:48.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.084+0000 7f3089600640 1 --2- 172.21.15.5:0/1974892299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3084100140 0x7f3084100560 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:48.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.084+0000 7f3089600640 1 -- 172.21.15.5:0/1974892299 >> 172.21.15.5:0/1974892299 conn(0x7f30840fba90 msgr2=0x7f30840fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:48.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.084+0000 7f3089600640 1 -- 172.21.15.5:0/1974892299 shutdown_connections 2024-04-24T06:43:48.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.084+0000 7f3089600640 1 -- 172.21.15.5:0/1974892299 wait complete. 2024-04-24T06:43:48.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.085+0000 7f3089600640 1 Processor -- start 2024-04-24T06:43:48.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.086+0000 7f3089600640 1 -- start start 2024-04-24T06:43:48.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.086+0000 7f3089600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3084100140 0x7f308419a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:48.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.086+0000 7f3089600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3084103110 0x7f308419a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:48.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.086+0000 7f3089600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3084103ad0 0x7f30841a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:48.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.086+0000 7f3089600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f308410db40 con 0x7f3084103110 2024-04-24T06:43:48.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.086+0000 7f3089600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f308410d9c0 con 0x7f3084100140 2024-04-24T06:43:48.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.086+0000 7f3089600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f308410dcc0 con 0x7f3084103ad0 2024-04-24T06:43:48.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.086+0000 7f307be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3084103110 0x7f308419a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:48.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.086+0000 7f3083e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3084100140 0x7f308419a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:48.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.086+0000 7f3083e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3084100140 0x7f308419a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:32838/0 (socket says 172.21.15.5:32838) 2024-04-24T06:43:48.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.086+0000 7f3083e00640 1 -- 172.21.15.5:0/3477997803 learned_addr learned my addr 172.21.15.5:0/3477997803 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:48.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.086+0000 7f3088c00640 1 --2- 172.21.15.5:0/3477997803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3084103ad0 0x7f30841a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:48.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.087+0000 7f307be00640 1 -- 172.21.15.5:0/3477997803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3084103ad0 msgr2=0x7f30841a19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:48.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.087+0000 7f307be00640 1 --2- 172.21.15.5:0/3477997803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3084103ad0 0x7f30841a19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:48.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.087+0000 7f307be00640 1 -- 172.21.15.5:0/3477997803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3084100140 msgr2=0x7f308419a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:48.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.087+0000 7f307be00640 1 --2- 172.21.15.5:0/3477997803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3084100140 0x7f308419a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:48.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.087+0000 7f307be00640 1 -- 172.21.15.5:0/3477997803 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30841a2030 con 0x7f3084103110 2024-04-24T06:43:48.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.087+0000 7f307be00640 1 --2- 172.21.15.5:0/3477997803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3084103110 0x7f308419a950 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f306c005fd0 tx=0x7f306c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:48.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.087+0000 7f3081600640 1 -- 172.21.15.5:0/3477997803 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c009c30 con 0x7f3084103110 2024-04-24T06:43:48.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.087+0000 7f3081600640 1 -- 172.21.15.5:0/3477997803 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f306c009dd0 con 0x7f3084103110 2024-04-24T06:43:48.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.087+0000 7f3081600640 1 -- 172.21.15.5:0/3477997803 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c010460 con 0x7f3084103110 2024-04-24T06:43:48.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.087+0000 7f3089600640 1 -- 172.21.15.5:0/3477997803 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30841a2320 con 0x7f3084103110 2024-04-24T06:43:48.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.087+0000 7f3089600640 1 -- 172.21.15.5:0/3477997803 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30841a28c0 con 0x7f3084103110 2024-04-24T06:43:48.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.089+0000 7f3089600640 1 -- 172.21.15.5:0/3477997803 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30841a2b20 con 0x7f3084103110 2024-04-24T06:43:48.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.091+0000 7f3081600640 1 -- 172.21.15.5:0/3477997803 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f306c003f50 con 0x7f3084103110 2024-04-24T06:43:48.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.092+0000 7f3081600640 1 --2- 172.21.15.5:0/3477997803 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f30680a3c30 0x7f30680a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:48.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.092+0000 7f3081600640 1 -- 172.21.15.5:0/3477997803 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f306c0d02f0 con 0x7f3084103110 2024-04-24T06:43:48.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.093+0000 7f3083e00640 1 --2- 172.21.15.5:0/3477997803 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f30680a3c30 0x7f30680a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:48.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.093+0000 7f3083e00640 1 --2- 172.21.15.5:0/3477997803 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f30680a3c30 0x7f30680a60f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3074006010 tx=0x7f30740315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:48.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.097+0000 7f3081600640 1 -- 172.21.15.5:0/3477997803 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f306c017020 con 0x7f3084103110 2024-04-24T06:43:48.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.303+0000 7f3089600640 1 -- 172.21.15.5:0/3477997803 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f30841a2cf0 con 0x7f3084103110 2024-04-24T06:43:48.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.304+0000 7f3081600640 1 -- 172.21.15.5:0/3477997803 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f306c098020 con 0x7f3084103110 2024-04-24T06:43:48.305 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-24T06:43:48.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.307+0000 7f3089600640 1 -- 172.21.15.5:0/3477997803 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f30680a3c30 msgr2=0x7f30680a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:48.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.307+0000 7f3089600640 1 --2- 172.21.15.5:0/3477997803 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f30680a3c30 0x7f30680a60f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3074006010 tx=0x7f30740315b0 comp rx=0 tx=0).stop 2024-04-24T06:43:48.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.307+0000 7f3089600640 1 -- 172.21.15.5:0/3477997803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3084103110 msgr2=0x7f308419a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:48.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.307+0000 7f3089600640 1 --2- 172.21.15.5:0/3477997803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3084103110 0x7f308419a950 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f306c005fd0 tx=0x7f306c00dde0 comp rx=0 tx=0).stop 2024-04-24T06:43:48.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.307+0000 7f3089600640 1 -- 172.21.15.5:0/3477997803 shutdown_connections 2024-04-24T06:43:48.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.307+0000 7f3089600640 1 --2- 172.21.15.5:0/3477997803 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f30680a3c30 0x7f30680a60f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:48.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.307+0000 7f3089600640 1 --2- 172.21.15.5:0/3477997803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3084103ad0 0x7f30841a19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:48.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.307+0000 7f3089600640 1 --2- 172.21.15.5:0/3477997803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3084103110 0x7f308419a950 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:48.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.307+0000 7f3089600640 1 --2- 172.21.15.5:0/3477997803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3084100140 0x7f308419a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:48.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.307+0000 7f3089600640 1 -- 172.21.15.5:0/3477997803 >> 172.21.15.5:0/3477997803 conn(0x7f30840fba90 msgr2=0x7f30841088e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:48.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.307+0000 7f3089600640 1 -- 172.21.15.5:0/3477997803 shutdown_connections 2024-04-24T06:43:48.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.307+0000 7f3089600640 1 -- 172.21.15.5:0/3477997803 wait complete. 2024-04-24T06:43:48.327 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:43:48.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:48 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/578590792' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:43:48.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:48 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1583127665' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:43:48.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:48 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/578590792' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:43:48.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:48 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/1583127665' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:43:48.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.527+0000 7fd01d600640 1 -- 172.21.15.5:0/1606892025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd018105220 msgr2=0x7fd018107610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:48.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.527+0000 7fd01d600640 1 --2- 172.21.15.5:0/1606892025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd018105220 0x7fd018107610 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd004009860 tx=0x7fd00402a340 comp rx=0 tx=0).stop 2024-04-24T06:43:48.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.527+0000 7fd01d600640 1 -- 172.21.15.5:0/1606892025 shutdown_connections 2024-04-24T06:43:48.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.527+0000 7fd01d600640 1 --2- 172.21.15.5:0/1606892025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd018107b50 0x7fd018109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:48.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.527+0000 7fd01d600640 1 --2- 172.21.15.5:0/1606892025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd018105220 0x7fd018107610 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:48.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.527+0000 7fd01d600640 1 --2- 172.21.15.5:0/1606892025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0181028f0 0x7fd018104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:48.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.527+0000 7fd01d600640 1 -- 172.21.15.5:0/1606892025 >> 172.21.15.5:0/1606892025 conn(0x7fd0180fc2e0 msgr2=0x7fd0180fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:48.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.527+0000 7fd01d600640 1 -- 172.21.15.5:0/1606892025 shutdown_connections 2024-04-24T06:43:48.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.527+0000 7fd01d600640 1 -- 172.21.15.5:0/1606892025 wait complete. 2024-04-24T06:43:48.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.528+0000 7fd01d600640 1 Processor -- start 2024-04-24T06:43:48.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.528+0000 7fd01d600640 1 -- start start 2024-04-24T06:43:48.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.529+0000 7fd01d600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0181028f0 0x7fd0181981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:48.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.529+0000 7fd01d600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd018105220 0x7fd018198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:48.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.529+0000 7fd01d600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd018107b50 0x7fd01819f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:48.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.529+0000 7fd01d600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd01810db00 con 0x7fd018107b50 2024-04-24T06:43:48.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.529+0000 7fd01d600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd01810d980 con 0x7fd018105220 2024-04-24T06:43:48.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.529+0000 7fd01d600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd01810dc80 con 0x7fd0181028f0 2024-04-24T06:43:48.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.529+0000 7fd017e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0181028f0 0x7fd0181981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:48.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.529+0000 7fd017e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0181028f0 0x7fd0181981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:54292/0 (socket says 172.21.15.5:54292) 2024-04-24T06:43:48.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.529+0000 7fd017e00640 1 -- 172.21.15.5:0/254383933 learned_addr learned my addr 172.21.15.5:0/254383933 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:48.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.529+0000 7fd01cc00640 1 --2- 172.21.15.5:0/254383933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd018107b50 0x7fd01819f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:48.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.529+0000 7fd017400640 1 --2- 172.21.15.5:0/254383933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd018105220 0x7fd018198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:48.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.530+0000 7fd017e00640 1 -- 172.21.15.5:0/254383933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd018105220 msgr2=0x7fd018198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:48.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.530+0000 7fd017e00640 1 --2- 172.21.15.5:0/254383933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd018105220 0x7fd018198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:48.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.530+0000 7fd017e00640 1 -- 172.21.15.5:0/254383933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd018107b50 msgr2=0x7fd01819f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:48.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.530+0000 7fd017e00640 1 --2- 172.21.15.5:0/254383933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd018107b50 0x7fd01819f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:48.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.530+0000 7fd017e00640 1 -- 172.21.15.5:0/254383933 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd01819fdf0 con 0x7fd0181028f0 2024-04-24T06:43:48.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.530+0000 7fd01cc00640 1 --2- 172.21.15.5:0/254383933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd018107b50 0x7fd01819f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:43:48.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.530+0000 7fd017e00640 1 --2- 172.21.15.5:0/254383933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0181028f0 0x7fd0181981d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd00800c420 tx=0x7fd00800c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:48.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.530+0000 7fd014c00640 1 -- 172.21.15.5:0/254383933 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd008004b20 con 0x7fd0181028f0 2024-04-24T06:43:48.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.530+0000 7fd014c00640 1 -- 172.21.15.5:0/254383933 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd008003e50 con 0x7fd0181028f0 2024-04-24T06:43:48.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.530+0000 7fd01d600640 1 -- 172.21.15.5:0/254383933 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0181a00e0 con 0x7fd0181028f0 2024-04-24T06:43:48.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.530+0000 7fd014c00640 1 -- 172.21.15.5:0/254383933 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00800b630 con 0x7fd0181028f0 2024-04-24T06:43:48.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.531+0000 7fd01d600640 1 -- 172.21.15.5:0/254383933 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0181a05b0 con 0x7fd0181028f0 2024-04-24T06:43:48.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.532+0000 7fd01d600640 1 -- 172.21.15.5:0/254383933 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0180fddd0 con 0x7fd0181028f0 2024-04-24T06:43:48.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.534+0000 7fd014c00640 1 -- 172.21.15.5:0/254383933 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7fd00801d020 con 0x7fd0181028f0 2024-04-24T06:43:48.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.535+0000 7fd014c00640 1 --2- 172.21.15.5:0/254383933 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fcffc0a3bb0 0x7fcffc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:48.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.535+0000 7fd014c00640 1 -- 172.21.15.5:0/254383933 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fd0080d0420 con 0x7fd0181028f0 2024-04-24T06:43:48.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.535+0000 7fd017400640 1 --2- 172.21.15.5:0/254383933 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fcffc0a3bb0 0x7fcffc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:48.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.536+0000 7fd017400640 1 --2- 172.21.15.5:0/254383933 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fcffc0a3bb0 0x7fcffc0a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd004030410 tx=0x7fd004030550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:48.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.539+0000 7fd014c00640 1 -- 172.21.15.5:0/254383933 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd00809cdc0 con 0x7fd0181028f0 2024-04-24T06:43:48.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:48 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/578590792' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:43:48.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:48 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/1583127665' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:43:48.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.747+0000 7fd01d600640 1 -- 172.21.15.5:0/254383933 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd018062c00 con 0x7fd0181028f0 2024-04-24T06:43:48.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.748+0000 7fd014c00640 1 -- 172.21.15.5:0/254383933 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7fd00809c6c0 con 0x7fd0181028f0 2024-04-24T06:43:48.750 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-24T06:43:48.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.751+0000 7fd01d600640 1 -- 172.21.15.5:0/254383933 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fcffc0a3bb0 msgr2=0x7fcffc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:48.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.751+0000 7fd01d600640 1 --2- 172.21.15.5:0/254383933 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fcffc0a3bb0 0x7fcffc0a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd004030410 tx=0x7fd004030550 comp rx=0 tx=0).stop 2024-04-24T06:43:48.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.751+0000 7fd01d600640 1 -- 172.21.15.5:0/254383933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0181028f0 msgr2=0x7fd0181981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:48.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.751+0000 7fd01d600640 1 --2- 172.21.15.5:0/254383933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0181028f0 0x7fd0181981d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd00800c420 tx=0x7fd00800c8e0 comp rx=0 tx=0).stop 2024-04-24T06:43:48.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.751+0000 7fd01d600640 1 -- 172.21.15.5:0/254383933 shutdown_connections 2024-04-24T06:43:48.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.751+0000 7fd01d600640 1 --2- 172.21.15.5:0/254383933 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fcffc0a3bb0 0x7fcffc0a6070 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:48.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.751+0000 7fd01d600640 1 --2- 172.21.15.5:0/254383933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd018107b50 0x7fd01819f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:48.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.751+0000 7fd01d600640 1 --2- 172.21.15.5:0/254383933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd018105220 0x7fd018198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:48.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.751+0000 7fd01d600640 1 --2- 172.21.15.5:0/254383933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0181028f0 0x7fd0181981d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:48.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.751+0000 7fd01d600640 1 -- 172.21.15.5:0/254383933 >> 172.21.15.5:0/254383933 conn(0x7fd0180fc2e0 msgr2=0x7fd018103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:48.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.752+0000 7fd01d600640 1 -- 172.21.15.5:0/254383933 shutdown_connections 2024-04-24T06:43:48.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.752+0000 7fd01d600640 1 -- 172.21.15.5:0/254383933 wait complete. 2024-04-24T06:43:48.770 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:43:48.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.985+0000 7efde9400640 1 -- 172.21.15.5:0/2960931548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efde4105c30 msgr2=0x7efde410c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:48.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.985+0000 7efde9400640 1 --2- 172.21.15.5:0/2960931548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efde4105c30 0x7efde410c4c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efddc009ca0 tx=0x7efddc02a8c0 comp rx=0 tx=0).stop 2024-04-24T06:43:48.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.986+0000 7efde9400640 1 -- 172.21.15.5:0/2960931548 shutdown_connections 2024-04-24T06:43:48.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.986+0000 7efde9400640 1 --2- 172.21.15.5:0/2960931548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efde4105c30 0x7efde410c4c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:48.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.986+0000 7efde9400640 1 --2- 172.21.15.5:0/2960931548 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efde4105270 0x7efde41056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:48.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.986+0000 7efde9400640 1 --2- 172.21.15.5:0/2960931548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efde4104080 0x7efde41044a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:48.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.986+0000 7efde9400640 1 -- 172.21.15.5:0/2960931548 >> 172.21.15.5:0/2960931548 conn(0x7efde40ff610 msgr2=0x7efde4101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:48.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.986+0000 7efde9400640 1 -- 172.21.15.5:0/2960931548 shutdown_connections 2024-04-24T06:43:48.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.986+0000 7efde9400640 1 -- 172.21.15.5:0/2960931548 wait complete. 2024-04-24T06:43:48.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.987+0000 7efde9400640 1 Processor -- start 2024-04-24T06:43:48.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.987+0000 7efde9400640 1 -- start start 2024-04-24T06:43:48.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.987+0000 7efde9400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efde4104080 0x7efde419a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:48.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.987+0000 7efde9400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efde4105270 0x7efde419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:48.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.987+0000 7efde9400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efde4105c30 0x7efde41a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:48.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.987+0000 7efde9400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efde410fd10 con 0x7efde4105c30 2024-04-24T06:43:48.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.987+0000 7efde9400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efde410fb90 con 0x7efde4105270 2024-04-24T06:43:48.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.987+0000 7efde9400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efde410fe90 con 0x7efde4104080 2024-04-24T06:43:48.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.988+0000 7efde8a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efde4105c30 0x7efde41a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:48.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.988+0000 7efde3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efde4104080 0x7efde419a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:48.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.988+0000 7efde3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efde4104080 0x7efde419a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:54320/0 (socket says 172.21.15.5:54320) 2024-04-24T06:43:48.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.988+0000 7efde8a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efde4105c30 0x7efde41a18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39584/0 (socket says 172.21.15.5:39584) 2024-04-24T06:43:48.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.988+0000 7efde3e00640 1 -- 172.21.15.5:0/2746923473 learned_addr learned my addr 172.21.15.5:0/2746923473 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:48.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.988+0000 7efddb400640 1 --2- 172.21.15.5:0/2746923473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efde4105270 0x7efde419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:48.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.988+0000 7efde3e00640 1 -- 172.21.15.5:0/2746923473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efde4105270 msgr2=0x7efde419a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:48.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.988+0000 7efde3e00640 1 --2- 172.21.15.5:0/2746923473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efde4105270 0x7efde419a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:48.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.988+0000 7efde3e00640 1 -- 172.21.15.5:0/2746923473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efde4105c30 msgr2=0x7efde41a18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:48.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.988+0000 7efde3e00640 1 --2- 172.21.15.5:0/2746923473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efde4105c30 0x7efde41a18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:48.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.988+0000 7efde3e00640 1 -- 172.21.15.5:0/2746923473 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde41a1f30 con 0x7efde4104080 2024-04-24T06:43:48.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.988+0000 7efde8a00640 1 --2- 172.21.15.5:0/2746923473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efde4105c30 0x7efde41a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:43:48.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.988+0000 7efde3e00640 1 --2- 172.21.15.5:0/2746923473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efde4104080 0x7efde419a310 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7efdd400e9c0 tx=0x7efdd400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:48.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.989+0000 7efde1600640 1 -- 172.21.15.5:0/2746923473 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd400cde0 con 0x7efde4104080 2024-04-24T06:43:48.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.989+0000 7efde1600640 1 -- 172.21.15.5:0/2746923473 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdd4010910 con 0x7efde4104080 2024-04-24T06:43:48.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.989+0000 7efde9400640 1 -- 172.21.15.5:0/2746923473 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efde41a2220 con 0x7efde4104080 2024-04-24T06:43:48.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.989+0000 7efde1600640 1 -- 172.21.15.5:0/2746923473 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd4018a00 con 0x7efde4104080 2024-04-24T06:43:48.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.989+0000 7efde9400640 1 -- 172.21.15.5:0/2746923473 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efde41a27c0 con 0x7efde4104080 2024-04-24T06:43:48.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.990+0000 7efde9400640 1 -- 172.21.15.5:0/2746923473 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efde410b3a0 con 0x7efde4104080 2024-04-24T06:43:48.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.992+0000 7efde1600640 1 -- 172.21.15.5:0/2746923473 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7efdd401f020 con 0x7efde4104080 2024-04-24T06:43:48.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.993+0000 7efde1600640 1 --2- 172.21.15.5:0/2746923473 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7efdc80a3c30 0x7efdc80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:48.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.994+0000 7efde1600640 1 -- 172.21.15.5:0/2746923473 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7efdd40d0b60 con 0x7efde4104080 2024-04-24T06:43:48.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.994+0000 7efddb400640 1 --2- 172.21.15.5:0/2746923473 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7efdc80a3c30 0x7efdc80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:48.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.994+0000 7efddb400640 1 --2- 172.21.15.5:0/2746923473 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7efdc80a3c30 0x7efdc80a60f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efde419b820 tx=0x7efdcc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:48.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:48.995+0000 7efde1600640 1 -- 172.21.15.5:0/2746923473 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efdd409d500 con 0x7efde4104080 2024-04-24T06:43:49.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.205+0000 7efde9400640 1 -- 172.21.15.5:0/2746923473 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efde4062c00 con 0x7efde4104080 2024-04-24T06:43:49.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.209+0000 7efde1600640 1 -- 172.21.15.5:0/2746923473 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7efdd409ce00 con 0x7efde4104080 2024-04-24T06:43:49.210 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-24T06:43:49.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.211+0000 7efde9400640 1 -- 172.21.15.5:0/2746923473 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7efdc80a3c30 msgr2=0x7efdc80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:49.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.211+0000 7efde9400640 1 --2- 172.21.15.5:0/2746923473 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7efdc80a3c30 0x7efdc80a60f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efde419b820 tx=0x7efdcc005f90 comp rx=0 tx=0).stop 2024-04-24T06:43:49.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.211+0000 7efde9400640 1 -- 172.21.15.5:0/2746923473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efde4104080 msgr2=0x7efde419a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:49.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.211+0000 7efde9400640 1 --2- 172.21.15.5:0/2746923473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efde4104080 0x7efde419a310 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7efdd400e9c0 tx=0x7efdd400ee80 comp rx=0 tx=0).stop 2024-04-24T06:43:49.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.212+0000 7efde9400640 1 -- 172.21.15.5:0/2746923473 shutdown_connections 2024-04-24T06:43:49.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.212+0000 7efde9400640 1 --2- 172.21.15.5:0/2746923473 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7efdc80a3c30 0x7efdc80a60f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:49.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.212+0000 7efde9400640 1 --2- 172.21.15.5:0/2746923473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efde4105c30 0x7efde41a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:49.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.212+0000 7efde9400640 1 --2- 172.21.15.5:0/2746923473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efde4105270 0x7efde419a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:49.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.212+0000 7efde9400640 1 --2- 172.21.15.5:0/2746923473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efde4104080 0x7efde419a310 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:49.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.212+0000 7efde9400640 1 -- 172.21.15.5:0/2746923473 >> 172.21.15.5:0/2746923473 conn(0x7efde40ff610 msgr2=0x7efde4108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:49.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.212+0000 7efde9400640 1 -- 172.21.15.5:0/2746923473 shutdown_connections 2024-04-24T06:43:49.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.212+0000 7efde9400640 1 -- 172.21.15.5:0/2746923473 wait complete. 2024-04-24T06:43:49.233 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:43:49.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.429+0000 7f1b86e00640 1 -- 172.21.15.5:0/3923207718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b80105350 msgr2=0x7f1b801057d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:49.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.429+0000 7f1b86e00640 1 --2- 172.21.15.5:0/3923207718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b80105350 0x7f1b801057d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c009900 tx=0x7f1b6c02a410 comp rx=0 tx=0).stop 2024-04-24T06:43:49.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.429+0000 7f1b86e00640 1 -- 172.21.15.5:0/3923207718 shutdown_connections 2024-04-24T06:43:49.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.429+0000 7f1b86e00640 1 --2- 172.21.15.5:0/3923207718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b80105d10 0x7f1b8010c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:49.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.429+0000 7f1b86e00640 1 --2- 172.21.15.5:0/3923207718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b80105350 0x7f1b801057d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:49.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.429+0000 7f1b86e00640 1 --2- 172.21.15.5:0/3923207718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b80104160 0x7f1b80104580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:49.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.429+0000 7f1b86e00640 1 -- 172.21.15.5:0/3923207718 >> 172.21.15.5:0/3923207718 conn(0x7f1b800ff6d0 msgr2=0x7f1b80101b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:49.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.429+0000 7f1b86e00640 1 -- 172.21.15.5:0/3923207718 shutdown_connections 2024-04-24T06:43:49.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.429+0000 7f1b86e00640 1 -- 172.21.15.5:0/3923207718 wait complete. 2024-04-24T06:43:49.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.430+0000 7f1b86e00640 1 Processor -- start 2024-04-24T06:43:49.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.431+0000 7f1b86e00640 1 -- start start 2024-04-24T06:43:49.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.431+0000 7f1b86e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b80104160 0x7f1b8019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:49.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.431+0000 7f1b86e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b80105350 0x7f1b8019a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:49.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.431+0000 7f1b86e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b80105d10 0x7f1b801a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:49.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.431+0000 7f1b86e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8010fe40 con 0x7f1b80105d10 2024-04-24T06:43:49.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.431+0000 7f1b86e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8010fcc0 con 0x7f1b80104160 2024-04-24T06:43:49.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.431+0000 7f1b86e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8010ffc0 con 0x7f1b80105350 2024-04-24T06:43:49.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.431+0000 7f1b86400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b80105d10 0x7f1b801a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:49.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.431+0000 7f1b85a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b80104160 0x7f1b8019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:49.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.431+0000 7f1b85a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b80104160 0x7f1b8019a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:32896/0 (socket says 172.21.15.5:32896) 2024-04-24T06:43:49.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.431+0000 7f1b86400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b80105d10 0x7f1b801a19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39588/0 (socket says 172.21.15.5:39588) 2024-04-24T06:43:49.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.432+0000 7f1b85a00640 1 -- 172.21.15.5:0/2665838611 learned_addr learned my addr 172.21.15.5:0/2665838611 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:49.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.432+0000 7f1b85000640 1 --2- 172.21.15.5:0/2665838611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b80105350 0x7f1b8019a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:49.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.432+0000 7f1b86400640 1 -- 172.21.15.5:0/2665838611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b80105350 msgr2=0x7f1b8019a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:49.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.432+0000 7f1b86400640 1 --2- 172.21.15.5:0/2665838611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b80105350 0x7f1b8019a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:49.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.432+0000 7f1b86400640 1 -- 172.21.15.5:0/2665838611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b80104160 msgr2=0x7f1b8019a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:49.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.432+0000 7f1b86400640 1 --2- 172.21.15.5:0/2665838611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b80104160 0x7f1b8019a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:49.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.432+0000 7f1b86400640 1 -- 172.21.15.5:0/2665838611 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b801a2030 con 0x7f1b80105d10 2024-04-24T06:43:49.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.432+0000 7f1b85a00640 1 --2- 172.21.15.5:0/2665838611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b80104160 0x7f1b8019a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:43:49.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.432+0000 7f1b86400640 1 --2- 172.21.15.5:0/2665838611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b80105d10 0x7f1b801a19c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1b78009fd0 tx=0x7f1b7800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:49.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.433+0000 7f1b7e000640 1 -- 172.21.15.5:0/2665838611 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7800ce20 con 0x7f1b80105d10 2024-04-24T06:43:49.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.433+0000 7f1b7e000640 1 -- 172.21.15.5:0/2665838611 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b78005020 con 0x7f1b80105d10 2024-04-24T06:43:49.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.433+0000 7f1b86e00640 1 -- 172.21.15.5:0/2665838611 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b801a2320 con 0x7f1b80105d10 2024-04-24T06:43:49.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.433+0000 7f1b7e000640 1 -- 172.21.15.5:0/2665838611 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b78010720 con 0x7f1b80105d10 2024-04-24T06:43:49.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.433+0000 7f1b86e00640 1 -- 172.21.15.5:0/2665838611 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b801a27f0 con 0x7f1b80105d10 2024-04-24T06:43:49.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.434+0000 7f1b86e00640 1 -- 172.21.15.5:0/2665838611 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b8010b480 con 0x7f1b80105d10 2024-04-24T06:43:49.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.437+0000 7f1b7e000640 1 -- 172.21.15.5:0/2665838611 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f1b78005540 con 0x7f1b80105d10 2024-04-24T06:43:49.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.439+0000 7f1b7e000640 1 --2- 172.21.15.5:0/2665838611 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f1b680a3ba0 0x7f1b680a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:49.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.439+0000 7f1b7e000640 1 -- 172.21.15.5:0/2665838611 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f1b780d0ad0 con 0x7f1b80105d10 2024-04-24T06:43:49.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.439+0000 7f1b85a00640 1 --2- 172.21.15.5:0/2665838611 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f1b680a3ba0 0x7f1b680a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:49.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.440+0000 7f1b85a00640 1 --2- 172.21.15.5:0/2665838611 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f1b680a3ba0 0x7f1b680a6060 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1b740058c0 tx=0x7f1b74005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:49.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.441+0000 7f1b7e000640 1 -- 172.21.15.5:0/2665838611 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1b7809d470 con 0x7f1b80105d10 2024-04-24T06:43:49.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:49 smithi005 ceph-mon[27798]: pgmap v381: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:43:49.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:49 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/3477997803' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T06:43:49.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:49 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/254383933' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T06:43:49.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:49 smithi037 ceph-mon[32864]: pgmap v381: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:43:49.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:49 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/3477997803' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T06:43:49.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:49 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/254383933' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T06:43:49.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.688+0000 7f1b86e00640 1 -- 172.21.15.5:0/2665838611 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b8002d620 con 0x7f1b80105d10 2024-04-24T06:43:49.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.690+0000 7f1b7e000640 1 -- 172.21.15.5:0/2665838611 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11856 (secure 0 0 0) 0x7f1b7809cd70 con 0x7f1b80105d10 2024-04-24T06:43:49.691 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:43:49.691 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1735462750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1735462750},{"type":"v1","addr":"172.21.15.5:6837","nonce":1735462750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1499469370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1499469370},{"type":"v1","addr":"172.21.15.5:6841","nonce":1499469370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3705638847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3705638847},{"type":"v1","addr":"172.21.15.37:6835","nonce":3705638847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T06:41:34.764490+0000","modified":"2024-04-24T06:41:40.330991+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24550,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6837/1390103296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1390103296},{"type":"v1","addr":"172.21.15.37:6837","nonce":1390103296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3217754215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3217754215},{"type":"v1","addr":"172.21.15.137:6835","nonce":3217754215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6837/3044222976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3044222976},{"type":"v1","addr":"172.21.15.137:6837","nonce":3044222976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.5:6839/3173201459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3173201459},{"type":"v1","addr":"172.21.15.5:6839","nonce":3173201459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6839/1889774232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1889774232},{"type":"v1","addr":"172.21.15.137:6839","nonce":1889774232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6839/3536721753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":3536721753},{"type":"v1","addr":"172.21.15.37:6839","nonce":3536721753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24539,24550,24569]},"id":1}]} 2024-04-24T06:43:49.691 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 22 2024-04-24T06:43:49.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.692+0000 7f1b86e00640 1 -- 172.21.15.5:0/2665838611 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f1b680a3ba0 msgr2=0x7f1b680a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:49.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.692+0000 7f1b86e00640 1 --2- 172.21.15.5:0/2665838611 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f1b680a3ba0 0x7f1b680a6060 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1b740058c0 tx=0x7f1b74005f90 comp rx=0 tx=0).stop 2024-04-24T06:43:49.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.692+0000 7f1b86e00640 1 -- 172.21.15.5:0/2665838611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b80105d10 msgr2=0x7f1b801a19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:49.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.692+0000 7f1b86e00640 1 --2- 172.21.15.5:0/2665838611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b80105d10 0x7f1b801a19c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1b78009fd0 tx=0x7f1b7800ef10 comp rx=0 tx=0).stop 2024-04-24T06:43:49.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.693+0000 7f1b86e00640 1 -- 172.21.15.5:0/2665838611 shutdown_connections 2024-04-24T06:43:49.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.693+0000 7f1b86e00640 1 --2- 172.21.15.5:0/2665838611 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f1b680a3ba0 0x7f1b680a6060 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:49.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.693+0000 7f1b86e00640 1 --2- 172.21.15.5:0/2665838611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b80105d10 0x7f1b801a19c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:49.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.693+0000 7f1b86e00640 1 --2- 172.21.15.5:0/2665838611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b80105350 0x7f1b8019a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:49.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.693+0000 7f1b86e00640 1 --2- 172.21.15.5:0/2665838611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b80104160 0x7f1b8019a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:49.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.693+0000 7f1b86e00640 1 -- 172.21.15.5:0/2665838611 >> 172.21.15.5:0/2665838611 conn(0x7f1b800ff6d0 msgr2=0x7f1b80108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:49.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.693+0000 7f1b86e00640 1 -- 172.21.15.5:0/2665838611 shutdown_connections 2024-04-24T06:43:49.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.693+0000 7f1b86e00640 1 -- 172.21.15.5:0/2665838611 wait complete. 2024-04-24T06:43:49.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:49 smithi137 ceph-mon[33017]: pgmap v381: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:43:49.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:49 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/3477997803' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T06:43:49.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:49 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/254383933' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T06:43:49.713 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:43:49.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.930+0000 7f0258400640 1 -- 172.21.15.5:0/1591214512 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02501020c0 msgr2=0x7f02501024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:49.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.930+0000 7f0258400640 1 --2- 172.21.15.5:0/1591214512 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02501020c0 0x7f02501024e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0244009ca0 tx=0x7f024402a8c0 comp rx=0 tx=0).stop 2024-04-24T06:43:49.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.930+0000 7f0258400640 1 -- 172.21.15.5:0/1591214512 shutdown_connections 2024-04-24T06:43:49.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.930+0000 7f0258400640 1 --2- 172.21.15.5:0/1591214512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0250103c70 0x7f025010a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:49.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.930+0000 7f0258400640 1 --2- 172.21.15.5:0/1591214512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02501032b0 0x7f0250103730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:49.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.930+0000 7f0258400640 1 --2- 172.21.15.5:0/1591214512 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02501020c0 0x7f02501024e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:49.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.930+0000 7f0258400640 1 -- 172.21.15.5:0/1591214512 >> 172.21.15.5:0/1591214512 conn(0x7f02500fd630 msgr2=0x7f02500ffa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:49.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.930+0000 7f0258400640 1 -- 172.21.15.5:0/1591214512 shutdown_connections 2024-04-24T06:43:49.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.930+0000 7f0258400640 1 -- 172.21.15.5:0/1591214512 wait complete. 2024-04-24T06:43:49.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.932+0000 7f0258400640 1 Processor -- start 2024-04-24T06:43:49.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.932+0000 7f0258400640 1 -- start start 2024-04-24T06:43:49.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.932+0000 7f0258400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02501020c0 0x7f025019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:49.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.932+0000 7f0258400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02501032b0 0x7f025019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:49.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.932+0000 7f0258400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0250103c70 0x7f02501a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:49.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.932+0000 7f0258400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f025010dce0 con 0x7f02501020c0 2024-04-24T06:43:49.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.932+0000 7f0258400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f025010db60 con 0x7f02501032b0 2024-04-24T06:43:49.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.932+0000 7f0258400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f025010de60 con 0x7f0250103c70 2024-04-24T06:43:49.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.933+0000 7f0255a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02501020c0 0x7f025019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:49.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.933+0000 7f0255a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02501020c0 0x7f025019a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39604/0 (socket says 172.21.15.5:39604) 2024-04-24T06:43:49.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.933+0000 7f0255a00640 1 -- 172.21.15.5:0/3203593702 learned_addr learned my addr 172.21.15.5:0/3203593702 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:49.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.933+0000 7f0256400640 1 --2- 172.21.15.5:0/3203593702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0250103c70 0x7f02501a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:49.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.933+0000 7f0255000640 1 --2- 172.21.15.5:0/3203593702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02501032b0 0x7f025019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:49.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.933+0000 7f0255a00640 1 -- 172.21.15.5:0/3203593702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0250103c70 msgr2=0x7f02501a1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:49.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.933+0000 7f0255a00640 1 --2- 172.21.15.5:0/3203593702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0250103c70 0x7f02501a1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:49.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.933+0000 7f0255a00640 1 -- 172.21.15.5:0/3203593702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02501032b0 msgr2=0x7f025019aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:49.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.933+0000 7f0255a00640 1 --2- 172.21.15.5:0/3203593702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02501032b0 0x7f025019aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:49.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.933+0000 7f0255a00640 1 -- 172.21.15.5:0/3203593702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02501a21a0 con 0x7f02501020c0 2024-04-24T06:43:49.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.934+0000 7f0255a00640 1 --2- 172.21.15.5:0/3203593702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02501020c0 0x7f025019a580 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0244009bf0 tx=0x7f0244031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:49.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.934+0000 7f024e000640 1 -- 172.21.15.5:0/3203593702 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024402fae0 con 0x7f02501020c0 2024-04-24T06:43:49.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.934+0000 7f0258400640 1 -- 172.21.15.5:0/3203593702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02501a2430 con 0x7f02501020c0 2024-04-24T06:43:49.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.934+0000 7f024e000640 1 -- 172.21.15.5:0/3203593702 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f024402fc80 con 0x7f02501020c0 2024-04-24T06:43:49.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.934+0000 7f024e000640 1 -- 172.21.15.5:0/3203593702 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02440467e0 con 0x7f02501020c0 2024-04-24T06:43:49.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.934+0000 7f0258400640 1 -- 172.21.15.5:0/3203593702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02501a2980 con 0x7f02501020c0 2024-04-24T06:43:49.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.935+0000 7f0258400640 1 -- 172.21.15.5:0/3203593702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02501a2be0 con 0x7f02501020c0 2024-04-24T06:43:49.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.938+0000 7f024e000640 1 -- 172.21.15.5:0/3203593702 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f0244030b30 con 0x7f02501020c0 2024-04-24T06:43:49.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.939+0000 7f024e000640 1 --2- 172.21.15.5:0/3203593702 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f02380a3bb0 0x7f02380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:49.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.940+0000 7f024e000640 1 -- 172.21.15.5:0/3203593702 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f02440f1890 con 0x7f02501020c0 2024-04-24T06:43:49.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.940+0000 7f0255000640 1 --2- 172.21.15.5:0/3203593702 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f02380a3bb0 0x7f02380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:49.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.940+0000 7f0255000640 1 --2- 172.21.15.5:0/3203593702 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f02380a3bb0 0x7f02380a6070 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f025019ba90 tx=0x7f023c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:49.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:49.945+0000 7f024e000640 1 -- 172.21.15.5:0/3203593702 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f02440be230 con 0x7f02501020c0 2024-04-24T06:43:50.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.156+0000 7f0258400640 1 -- 172.21.15.5:0/3203593702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f02501a2db0 con 0x7f02501020c0 2024-04-24T06:43:50.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.159+0000 7f024e000640 1 -- 172.21.15.5:0/3203593702 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22513 (secure 0 0 0) 0x7f02440bdb30 con 0x7f02501020c0 2024-04-24T06:43:50.160 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:43:50.161 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":88,"fsid":"43e9bad0-0204-11ef-bc93-c7b262605968","created":"2024-04-24T06:32:26.340555+0000","modified":"2024-04-24T06:42:42.855911+0000","last_up_change":"2024-04-24T06:40:36.790934+0000","last_in_change":"2024-04-24T06:40:17.803257+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:36:27.320342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:41:32.254502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:41:33.739067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c2e974c-745c-4dd5-8c3d-8e8da2eb1ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6803","nonce":2155858515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6805","nonce":2155858515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6809","nonce":2155858515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6807","nonce":2155858515}]},"public_addr":"172.21.15.5:6803/2155858515","cluster_addr":"172.21.15.5:6805/2155858515","heartbeat_back_addr":"172.21.15.5:6809/2155858515","heartbeat_front_addr":"172.21.15.5:6807/2155858515","state":["exists","up"]},{"osd":1,"uuid":"50467744-db20-4721-bf2d-1c5db938297b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6801","nonce":1302362650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6803","nonce":1302362650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6807","nonce":1302362650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6805","nonce":1302362650}]},"public_addr":"172.21.15.37:6801/1302362650","cluster_addr":"172.21.15.37:6803/1302362650","heartbeat_back_addr":"172.21.15.37:6807/1302362650","heartbeat_front_addr":"172.21.15.37:6805/1302362650","state":["exists","up"]},{"osd":2,"uuid":"0e190b15-d04c-4288-8d48-ecf6664392fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6801","nonce":2885027256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6803","nonce":2885027256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6807","nonce":2885027256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6805","nonce":2885027256}]},"public_addr":"172.21.15.137:6801/2885027256","cluster_addr":"172.21.15.137:6803/2885027256","heartbeat_back_addr":"172.21.15.137:6807/2885027256","heartbeat_front_addr":"172.21.15.137:6805/2885027256","state":["exists","up"]},{"osd":3,"uuid":"e05592ba-d96a-472a-9f63-09a535d04dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6811","nonce":3701747260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6813","nonce":3701747260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6817","nonce":3701747260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6815","nonce":3701747260}]},"public_addr":"172.21.15.5:6811/3701747260","cluster_addr":"172.21.15.5:6813/3701747260","heartbeat_back_addr":"172.21.15.5:6817/3701747260","heartbeat_front_addr":"172.21.15.5:6815/3701747260","state":["exists","up"]},{"osd":4,"uuid":"c3f36e51-7a3b-4dd9-af4b-a10558830c4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6809","nonce":759253844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6811","nonce":759253844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6815","nonce":759253844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6813","nonce":759253844}]},"public_addr":"172.21.15.37:6809/759253844","cluster_addr":"172.21.15.37:6811/759253844","heartbeat_back_addr":"172.21.15.37:6815/759253844","heartbeat_front_addr":"172.21.15.37:6813/759253844","state":["exists","up"]},{"osd":5,"uuid":"97a0500c-cbb8-4ff5-b476-8634b26b3821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6809","nonce":3771921560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6811","nonce":3771921560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6815","nonce":3771921560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6813","nonce":3771921560}]},"public_addr":"172.21.15.137:6809/3771921560","cluster_addr":"172.21.15.137:6811/3771921560","heartbeat_back_addr":"172.21.15.137:6815/3771921560","heartbeat_front_addr":"172.21.15.137:6813/3771921560","state":["exists","up"]},{"osd":6,"uuid":"33a33d62-bbb3-4ed9-834e-19ae93a97670","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6819","nonce":2064636084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6821","nonce":2064636084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6825","nonce":2064636084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6823","nonce":2064636084}]},"public_addr":"172.21.15.5:6819/2064636084","cluster_addr":"172.21.15.5:6821/2064636084","heartbeat_back_addr":"172.21.15.5:6825/2064636084","heartbeat_front_addr":"172.21.15.5:6823/2064636084","state":["exists","up"]},{"osd":7,"uuid":"c0f4e036-4717-4514-bb59-ab11fbaab063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6817","nonce":2952750659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6819","nonce":2952750659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6823","nonce":2952750659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6821","nonce":2952750659}]},"public_addr":"172.21.15.37:6817/2952750659","cluster_addr":"172.21.15.37:6819/2952750659","heartbeat_back_addr":"172.21.15.37:6823/2952750659","heartbeat_front_addr":"172.21.15.37:6821/2952750659","state":["exists","up"]},{"osd":8,"uuid":"5391fd87-44ea-440f-80da-c54d0327c04c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6817","nonce":29724945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6819","nonce":29724945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6823","nonce":29724945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6821","nonce":29724945}]},"public_addr":"172.21.15.137:6817/29724945","cluster_addr":"172.21.15.137:6819/29724945","heartbeat_back_addr":"172.21.15.137:6823/29724945","heartbeat_front_addr":"172.21.15.137:6821/29724945","state":["exists","up"]},{"osd":9,"uuid":"d94c9dbc-f43d-4237-ac7f-47a6a5ede1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6827","nonce":2257102107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6829","nonce":2257102107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6833","nonce":2257102107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6831","nonce":2257102107}]},"public_addr":"172.21.15.5:6827/2257102107","cluster_addr":"172.21.15.5:6829/2257102107","heartbeat_back_addr":"172.21.15.5:6833/2257102107","heartbeat_front_addr":"172.21.15.5:6831/2257102107","state":["exists","up"]},{"osd":10,"uuid":"e2455332-2a5e-44b1-85d3-8e47da399f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6825","nonce":2451570637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6827","nonce":2451570637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6831","nonce":2451570637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6829","nonce":2451570637}]},"public_addr":"172.21.15.37:6825/2451570637","cluster_addr":"172.21.15.37:6827/2451570637","heartbeat_back_addr":"172.21.15.37:6831/2451570637","heartbeat_front_addr":"172.21.15.37:6829/2451570637","state":["exists","up"]},{"osd":11,"uuid":"49d9396b-704a-4cef-8210-75d30c28e924","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6825","nonce":90600596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6827","nonce":90600596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6831","nonce":90600596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6829","nonce":90600596}]},"public_addr":"172.21.15.137:6825/90600596","cluster_addr":"172.21.15.137:6827/90600596","heartbeat_back_addr":"172.21.15.137:6831/90600596","heartbeat_front_addr":"172.21.15.137:6829/90600596","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:35:35.591619+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:00.568817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:24.195643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:56.451724+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:22.109293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:46.161749+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:19.278305+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:45.620256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:10.041165+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:43.813697+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:10.736026+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:35.384377+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/23307164":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6801/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:0/230785941":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/2940023597":"2024-04-25T06:33:07.040369+0000","172.21.15.5:6801/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3886613904":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/4199165437":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3459593499":"2024-04-25T06:33:41.161852+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:43:50.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.162+0000 7f0258400640 1 -- 172.21.15.5:0/3203593702 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f02380a3bb0 msgr2=0x7f02380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:50.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.162+0000 7f0258400640 1 --2- 172.21.15.5:0/3203593702 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f02380a3bb0 0x7f02380a6070 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f025019ba90 tx=0x7f023c005f90 comp rx=0 tx=0).stop 2024-04-24T06:43:50.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.162+0000 7f0258400640 1 -- 172.21.15.5:0/3203593702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02501020c0 msgr2=0x7f025019a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:50.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.162+0000 7f0258400640 1 --2- 172.21.15.5:0/3203593702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02501020c0 0x7f025019a580 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0244009bf0 tx=0x7f0244031ee0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.162+0000 7f0258400640 1 -- 172.21.15.5:0/3203593702 shutdown_connections 2024-04-24T06:43:50.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.162+0000 7f0258400640 1 --2- 172.21.15.5:0/3203593702 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f02380a3bb0 0x7f02380a6070 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.162+0000 7f0258400640 1 --2- 172.21.15.5:0/3203593702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0250103c70 0x7f02501a1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.162+0000 7f0258400640 1 --2- 172.21.15.5:0/3203593702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02501032b0 0x7f025019aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.162+0000 7f0258400640 1 --2- 172.21.15.5:0/3203593702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02501020c0 0x7f025019a580 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.162+0000 7f0258400640 1 -- 172.21.15.5:0/3203593702 >> 172.21.15.5:0/3203593702 conn(0x7f02500fd630 msgr2=0x7f0250074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:50.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.162+0000 7f0258400640 1 -- 172.21.15.5:0/3203593702 shutdown_connections 2024-04-24T06:43:50.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.162+0000 7f0258400640 1 -- 172.21.15.5:0/3203593702 wait complete. 2024-04-24T06:43:50.182 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-24T06:41:34.764490+0000', 'modified': '2024-04-24T06:41:40.330991+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24569, 'mds_1': 24550, 'mds_2': 24539}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24515': {'gid': 24515, 'name': 'h', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.37:6837/1390103296', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6836', 'nonce': 1390103296}, {'type': 'v1', 'addr': '172.21.15.37:6837', 'nonce': 1390103296}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24520': {'gid': 24520, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.137:6835/3217754215', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6834', 'nonce': 3217754215}, {'type': 'v1', 'addr': '172.21.15.137:6835', 'nonce': 3217754215}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24532': {'gid': 24532, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.137:6837/3044222976', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6836', 'nonce': 3044222976}, {'type': 'v1', 'addr': '172.21.15.137:6837', 'nonce': 3044222976}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24539': {'gid': 24539, 'name': 'g', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.5:6839/3173201459', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6838', 'nonce': 3173201459}, {'type': 'v1', 'addr': '172.21.15.5:6839', 'nonce': 3173201459}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24550': {'gid': 24550, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.137:6839/1889774232', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6838', 'nonce': 1889774232}, {'type': 'v1', 'addr': '172.21.15.137:6839', 'nonce': 1889774232}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24569': {'gid': 24569, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.37:6839/3536721753', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6838', 'nonce': 3536721753}, {'type': 'v1', 'addr': '172.21.15.37:6839', 'nonce': 3536721753}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24569, 'qdb_cluster': [24539, 24550, 24569]} 2024-04-24T06:43:50.183 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-24T06:43:50.183 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2024-04-24T06:43:50.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.397+0000 7f0577e00640 1 -- 172.21.15.5:0/3719229423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0578109670 msgr2=0x7f0578109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:50.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.397+0000 7f0577e00640 1 --2- 172.21.15.5:0/3719229423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0578109670 0x7f0578109af0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0560009900 tx=0x7f056002a410 comp rx=0 tx=0).stop 2024-04-24T06:43:50.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.397+0000 7f0577e00640 1 -- 172.21.15.5:0/3719229423 shutdown_connections 2024-04-24T06:43:50.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.397+0000 7f0577e00640 1 --2- 172.21.15.5:0/3719229423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f057810a030 0x7f05781108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.397+0000 7f0577e00640 1 --2- 172.21.15.5:0/3719229423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0578109670 0x7f0578109af0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.397+0000 7f0577e00640 1 --2- 172.21.15.5:0/3719229423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0578108480 0x7f05781088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.397+0000 7f0577e00640 1 -- 172.21.15.5:0/3719229423 >> 172.21.15.5:0/3719229423 conn(0x7f0578077440 msgr2=0x7f0578079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:50.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.397+0000 7f0577e00640 1 -- 172.21.15.5:0/3719229423 shutdown_connections 2024-04-24T06:43:50.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.397+0000 7f0577e00640 1 -- 172.21.15.5:0/3719229423 wait complete. 2024-04-24T06:43:50.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.398+0000 7f0577e00640 1 Processor -- start 2024-04-24T06:43:50.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.398+0000 7f0577e00640 1 -- start start 2024-04-24T06:43:50.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.399+0000 7f0577e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0578108480 0x7f057819e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.399+0000 7f0577e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0578109670 0x7f057819ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.399+0000 7f0577e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f057810a030 0x7f05781a5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.399+0000 7f0577e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0578114110 con 0x7f0578109670 2024-04-24T06:43:50.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.399+0000 7f0577e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0578113f90 con 0x7f057810a030 2024-04-24T06:43:50.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.399+0000 7f0577e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0578114290 con 0x7f0578108480 2024-04-24T06:43:50.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.399+0000 7f056fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0578109670 0x7f057819ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.399+0000 7f056fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0578109670 0x7f057819ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39620/0 (socket says 172.21.15.5:39620) 2024-04-24T06:43:50.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.399+0000 7f056fe00640 1 -- 172.21.15.5:0/368301995 learned_addr learned my addr 172.21.15.5:0/368301995 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:50.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.399+0000 7f0577400640 1 --2- 172.21.15.5:0/368301995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f057810a030 0x7f05781a5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.399+0000 7f0576a00640 1 --2- 172.21.15.5:0/368301995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0578108480 0x7f057819e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.399+0000 7f056fe00640 1 -- 172.21.15.5:0/368301995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0578108480 msgr2=0x7f057819e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:50.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.399+0000 7f056fe00640 1 --2- 172.21.15.5:0/368301995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0578108480 0x7f057819e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.399+0000 7f056fe00640 1 -- 172.21.15.5:0/368301995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f057810a030 msgr2=0x7f05781a5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:50.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.399+0000 7f056fe00640 1 --2- 172.21.15.5:0/368301995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f057810a030 0x7f05781a5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.399+0000 7f056fe00640 1 -- 172.21.15.5:0/368301995 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05781a63b0 con 0x7f0578109670 2024-04-24T06:43:50.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.400+0000 7f056fe00640 1 --2- 172.21.15.5:0/368301995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0578109670 0x7f057819ecd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0560009230 tx=0x7f05600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:50.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.400+0000 7f056f400640 1 -- 172.21.15.5:0/368301995 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056003f030 con 0x7f0578109670 2024-04-24T06:43:50.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.400+0000 7f056f400640 1 -- 172.21.15.5:0/368301995 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0560003ce0 con 0x7f0578109670 2024-04-24T06:43:50.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.400+0000 7f0577e00640 1 -- 172.21.15.5:0/368301995 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05781a6640 con 0x7f0578109670 2024-04-24T06:43:50.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.400+0000 7f056f400640 1 -- 172.21.15.5:0/368301995 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0560035070 con 0x7f0578109670 2024-04-24T06:43:50.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.400+0000 7f0577e00640 1 -- 172.21.15.5:0/368301995 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05781a6b30 con 0x7f0578109670 2024-04-24T06:43:50.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.403+0000 7f056f400640 1 -- 172.21.15.5:0/368301995 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f0560042030 con 0x7f0578109670 2024-04-24T06:43:50.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.403+0000 7f058098e740 1 Processor -- start 2024-04-24T06:43:50.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.404+0000 7f058098e740 1 -- start start 2024-04-24T06:43:50.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.404+0000 7f058098e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c54816260 0x561c54816680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.404+0000 7f058098e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c54816c50 0x561c5481b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.404+0000 7f058098e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c5481bdc0 0x561c5481e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.404+0000 7f058098e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561c5477bb10 con 0x561c54816c50 2024-04-24T06:43:50.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.404+0000 7f058098e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561c547222b0 con 0x561c5481bdc0 2024-04-24T06:43:50.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.404+0000 7f058098e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561c54777ad0 con 0x561c54816260 2024-04-24T06:43:50.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.404+0000 7f0577400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c54816260 0x561c54816680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.404+0000 7f0576a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c54816c50 0x561c5481b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.404+0000 7f0577400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c54816260 0x561c54816680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:54356/0 (socket says 172.21.15.5:54356) 2024-04-24T06:43:50.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.404+0000 7f0577400640 1 -- 172.21.15.5:0/2674042895 learned_addr learned my addr 172.21.15.5:0/2674042895 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:50.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.404+0000 7f056fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c5481bdc0 0x561c5481e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.405+0000 7f0577400640 1 -- 172.21.15.5:0/2674042895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c5481bdc0 msgr2=0x561c5481e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:50.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.405+0000 7f0577400640 1 --2- 172.21.15.5:0/2674042895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c5481bdc0 0x561c5481e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.405+0000 7f0577400640 1 -- 172.21.15.5:0/2674042895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c54816c50 msgr2=0x561c5481b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:50.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.405+0000 7f0577400640 1 --2- 172.21.15.5:0/2674042895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c54816c50 0x561c5481b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.405+0000 7f0577400640 1 -- 172.21.15.5:0/2674042895 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c5448e2d0 con 0x561c54816260 2024-04-24T06:43:50.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.405+0000 7f0576a00640 1 --2- 172.21.15.5:0/2674042895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c54816c50 0x561c5481b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:43:50.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.405+0000 7f0577400640 1 --2- 172.21.15.5:0/2674042895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c54816260 0x561c54816680 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0570009fd0 tx=0x7f05700118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4566273555d5ca7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:50.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.405+0000 7f056f400640 1 --2- 172.21.15.5:0/368301995 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f055c0a3bb0 0x7f055c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.405+0000 7f0567e00640 1 -- 172.21.15.5:0/2674042895 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0570004d20 con 0x561c54816260 2024-04-24T06:43:50.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.405+0000 7f056f400640 1 -- 172.21.15.5:0/368301995 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f05600f08e0 con 0x7f0578109670 2024-04-24T06:43:50.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.406+0000 7f0567e00640 1 -- 172.21.15.5:0/2674042895 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0570004050 con 0x561c54816260 2024-04-24T06:43:50.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.406+0000 7f0567e00640 1 -- 172.21.15.5:0/2674042895 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0570005260 con 0x561c54816260 2024-04-24T06:43:50.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.406+0000 7f0576a00640 1 --2- 172.21.15.5:0/368301995 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f055c0a3bb0 0x7f055c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.406+0000 7f058098e740 1 -- 172.21.15.5:0/2674042895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c54816260 msgr2=0x561c54816680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:50.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.406+0000 7f0576a00640 1 --2- 172.21.15.5:0/368301995 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f055c0a3bb0 0x7f055c0a6070 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x561c54817050 tx=0x7f0568005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:50.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.406+0000 7f058098e740 1 --2- 172.21.15.5:0/2674042895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c54816260 0x561c54816680 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0570009fd0 tx=0x7f05700118d0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.406+0000 7f058098e740 1 -- 172.21.15.5:0/2674042895 shutdown_connections 2024-04-24T06:43:50.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.406+0000 7f058098e740 1 --2- 172.21.15.5:0/2674042895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c5481bdc0 0x561c5481e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.406+0000 7f058098e740 1 --2- 172.21.15.5:0/2674042895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c54816c50 0x561c5481b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.406+0000 7f058098e740 1 --2- 172.21.15.5:0/2674042895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c54816260 0x561c54816680 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.406+0000 7f058098e740 1 -- 172.21.15.5:0/2674042895 >> 172.21.15.5:0/2674042895 conn(0x561c548118d0 msgr2=0x561c54811cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:50.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.406+0000 7f058098e740 1 -- 172.21.15.5:0/2674042895 shutdown_connections 2024-04-24T06:43:50.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.407+0000 7f058098e740 1 -- 172.21.15.5:0/2674042895 wait complete. 2024-04-24T06:43:50.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.407+0000 7f058098e740 1 -- start start 2024-04-24T06:43:50.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.407+0000 7f058098e740 1 Processor -- start 2024-04-24T06:43:50.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.408+0000 7f058098e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c54816260 0x561c54820750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.408+0000 7f058098e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c54816c50 0x561c54820c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.408+0000 7f058098e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c5481bdc0 0x561c54827c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.408+0000 7f058098e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561c5481ed70 con 0x561c5481bdc0 2024-04-24T06:43:50.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.408+0000 7f058098e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561c54735590 con 0x561c54816260 2024-04-24T06:43:50.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.408+0000 7f058098e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561c5481eef0 con 0x561c54816c50 2024-04-24T06:43:50.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.408+0000 7f058098e740 10 client.0 _ll_register_callbacks cb 0x561c547183a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:43:50.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.408+0000 7f056fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c54816c50 0x561c54820c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.408+0000 7f056fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c54816c50 0x561c54820c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:54366/0 (socket says 172.21.15.5:54366) 2024-04-24T06:43:50.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.408+0000 7f056fe00640 1 -- 172.21.15.5:0/2089498243 learned_addr learned my addr 172.21.15.5:0/2089498243 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:50.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.408+0000 7f0577400640 1 --2- 172.21.15.5:0/2089498243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c5481bdc0 0x561c54827c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.408+0000 7f0576a00640 1 --2- 172.21.15.5:0/2089498243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c54816260 0x561c54820750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.408+0000 7f056fe00640 1 -- 172.21.15.5:0/2089498243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c54816260 msgr2=0x561c54820750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:50.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.408+0000 7f056fe00640 1 --2- 172.21.15.5:0/2089498243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c54816260 0x561c54820750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.408+0000 7f056fe00640 1 -- 172.21.15.5:0/2089498243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c5481bdc0 msgr2=0x561c54827c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:50.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.408+0000 7f056fe00640 1 --2- 172.21.15.5:0/2089498243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c5481bdc0 0x561c54827c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.408+0000 7f056fe00640 1 -- 172.21.15.5:0/2089498243 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c545586b0 con 0x561c54816c50 2024-04-24T06:43:50.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.409+0000 7f056fe00640 1 --2- 172.21.15.5:0/2089498243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c54816c50 0x561c54820c90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f05600c52c0 tx=0x7f0560009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d80ed5490b717036 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:50.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.409+0000 7f0564c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:43:50.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.409+0000 7f0564c00640 1 -- 172.21.15.5:0/2089498243 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0560049070 con 0x561c54816c50 2024-04-24T06:43:50.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.409+0000 7f0564c00640 1 -- 172.21.15.5:0/2089498243 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f05600dd070 con 0x561c54816c50 2024-04-24T06:43:50.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.409+0000 7f058098e740 1 -- 172.21.15.5:0/2089498243 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0570004050 con 0x561c54816c50 2024-04-24T06:43:50.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.409+0000 7f0564c00640 1 -- 172.21.15.5:0/2089498243 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f05600bca00 con 0x561c54816c50 2024-04-24T06:43:50.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.409+0000 7f058098e740 10 client.25156 fetch_fsmap learned FSMap version 22 2024-04-24T06:43:50.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.409+0000 7f058098e740 1 -- 172.21.15.5:0/2089498243 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x561c5481ede0 con 0x561c54816c50 2024-04-24T06:43:50.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.409+0000 7f0564c00640 1 -- 172.21.15.5:0/2089498243 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0570004050 con 0x561c54816c50 2024-04-24T06:43:50.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.410+0000 7f0564c00640 1 -- 172.21.15.5:0/2089498243 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f05600b92a0 con 0x561c54816c50 2024-04-24T06:43:50.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.410+0000 7f058098e740 10 client.25156 fetch_fsmap finished waiting for FSMap version 22 2024-04-24T06:43:50.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.410+0000 7f058098e740 10 client.25156 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-24T06:43:50.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.410+0000 7f058098e740 20 client.25156 populate_metadata read hostname 'smithi005' 2024-04-24T06:43:50.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.410+0000 7f058098e740 1 --2- 172.21.15.5:0/2089498243 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x561c5482d630 0x561c5482eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.410+0000 7f058098e740 4 client.25156 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:43:50.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.410+0000 7f058098e740 1 -- 172.21.15.5:0/2089498243 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c54789530 con 0x561c5482d630 2024-04-24T06:43:50.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.410+0000 7f0576a00640 1 --2- 172.21.15.5:0/2089498243 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x561c5482d630 0x561c5482eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.411+0000 7f0576a00640 1 --2- 172.21.15.5:0/2089498243 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x561c5482d630 0x561c5482eaa0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=75e0190a3f995db1 server_cookie=8e547d8a60ac8713 in_seq=0 out_seq=0 2024-04-24T06:43:50.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.411+0000 7f0564c00640 10 client.25156 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:43:50.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.411+0000 7f0564c00640 1 -- 172.21.15.5:0/2089498243 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561c54789530 con 0x561c5482d630 2024-04-24T06:43:50.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.411+0000 7f0564c00640 10 client.25156 handle_command_reply: tid=0 2024-04-24T06:43:50.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.412+0000 7f058098e740 1 client.25156 shutdown 2024-04-24T06:43:50.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.412+0000 7f058098e740 1 -- 172.21.15.5:0/2089498243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c54816c50 msgr2=0x561c54820c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:50.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.412+0000 7f058098e740 1 --2- 172.21.15.5:0/2089498243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c54816c50 0x561c54820c90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f05600c52c0 tx=0x7f0560009560 comp rx=0 tx=0).stop 2024-04-24T06:43:50.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.412+0000 7f058098e740 1 -- 172.21.15.5:0/2089498243 shutdown_connections 2024-04-24T06:43:50.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.412+0000 7f058098e740 1 --2- 172.21.15.5:0/2089498243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c5481bdc0 0x561c54827c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.412+0000 7f058098e740 1 --2- 172.21.15.5:0/2089498243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c54816c50 0x561c54820c90 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.412+0000 7f058098e740 1 --2- 172.21.15.5:0/2089498243 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x561c5482d630 0x561c5482eaa0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.412+0000 7f058098e740 1 --2- 172.21.15.5:0/2089498243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c54816260 0x561c54820750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.412+0000 7f0564c00640 0 client.25156 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:43:50.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.412+0000 7f058098e740 1 -- 172.21.15.5:0/2089498243 >> 172.21.15.5:0/2089498243 conn(0x561c548118d0 msgr2=0x561c54810970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:50.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.412+0000 7f058098e740 1 -- 172.21.15.5:0/2089498243 shutdown_connections 2024-04-24T06:43:50.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.412+0000 7f058098e740 1 -- 172.21.15.5:0/2089498243 wait complete. 2024-04-24T06:43:50.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.412+0000 7f058098e740 20 client.25156 trim_cache size 0 max 16384 2024-04-24T06:43:50.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.432+0000 7f058098e740 1 Processor -- start 2024-04-24T06:43:50.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.432+0000 7f058098e740 1 -- start start 2024-04-24T06:43:50.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.433+0000 7f058098e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c54816260 0x561c5481e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.433+0000 7f058098e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c54816c50 0x561c54827c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.433+0000 7f058098e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c5481bdc0 0x561c548147e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.433+0000 7f058098e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561c5482cde0 con 0x561c54816c50 2024-04-24T06:43:50.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.433+0000 7f058098e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561c5482d4b0 con 0x561c54816260 2024-04-24T06:43:50.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.433+0000 7f058098e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561c5482db80 con 0x561c5481bdc0 2024-04-24T06:43:50.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.433+0000 7f056fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c54816c50 0x561c54827c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.433+0000 7f056fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c54816c50 0x561c54827c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39656/0 (socket says 172.21.15.5:39656) 2024-04-24T06:43:50.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.433+0000 7f056fe00640 1 -- 172.21.15.5:0/1801166641 learned_addr learned my addr 172.21.15.5:0/1801166641 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:50.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.433+0000 7f0576a00640 1 --2- 172.21.15.5:0/1801166641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c54816260 0x561c5481e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.433+0000 7f0577400640 1 --2- 172.21.15.5:0/1801166641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c5481bdc0 0x561c548147e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.433+0000 7f056fe00640 1 -- 172.21.15.5:0/1801166641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c5481bdc0 msgr2=0x561c548147e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:50.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.433+0000 7f056fe00640 1 --2- 172.21.15.5:0/1801166641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c5481bdc0 0x561c548147e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.433+0000 7f056fe00640 1 -- 172.21.15.5:0/1801166641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c54816260 msgr2=0x561c5481e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:50.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.433+0000 7f056fe00640 1 --2- 172.21.15.5:0/1801166641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c54816260 0x561c5481e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.433+0000 7f056fe00640 1 -- 172.21.15.5:0/1801166641 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c548174e0 con 0x561c54816c50 2024-04-24T06:43:50.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.434+0000 7f056fe00640 1 --2- 172.21.15.5:0/1801166641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c54816c50 0x561c54827c20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0560004650 tx=0x7f05600d61f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=645de1930d931cb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:50.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.434+0000 7f0567400640 1 -- 172.21.15.5:0/1801166641 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0560049030 con 0x561c54816c50 2024-04-24T06:43:50.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.434+0000 7f0567400640 1 -- 172.21.15.5:0/1801166641 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05600f3e50 con 0x561c54816c50 2024-04-24T06:43:50.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.434+0000 7f0567400640 1 -- 172.21.15.5:0/1801166641 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05600bc590 con 0x561c54816c50 2024-04-24T06:43:50.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.435+0000 7f058098e740 1 -- 172.21.15.5:0/1801166641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c54816c50 msgr2=0x561c54827c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:50.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.435+0000 7f058098e740 1 --2- 172.21.15.5:0/1801166641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c54816c50 0x561c54827c20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0560004650 tx=0x7f05600d61f0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.435+0000 7f058098e740 1 -- 172.21.15.5:0/1801166641 shutdown_connections 2024-04-24T06:43:50.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.435+0000 7f058098e740 1 --2- 172.21.15.5:0/1801166641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c5481bdc0 0x561c548147e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.435+0000 7f058098e740 1 --2- 172.21.15.5:0/1801166641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c54816c50 0x561c54827c20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.435+0000 7f058098e740 1 --2- 172.21.15.5:0/1801166641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c54816260 0x561c5481e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.435+0000 7f058098e740 1 -- 172.21.15.5:0/1801166641 >> 172.21.15.5:0/1801166641 conn(0x561c548118d0 msgr2=0x561c5482c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:50.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.435+0000 7f058098e740 1 -- 172.21.15.5:0/1801166641 shutdown_connections 2024-04-24T06:43:50.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.435+0000 7f058098e740 1 -- 172.21.15.5:0/1801166641 wait complete. 2024-04-24T06:43:50.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.435+0000 7f058098e740 1 -- start start 2024-04-24T06:43:50.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.435+0000 7f058098e740 1 Processor -- start 2024-04-24T06:43:50.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.436+0000 7f058098e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c54816260 0x561c54823db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.436+0000 7f058098e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c54816c50 0x561c54824330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.436+0000 7f058098e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c5481bdc0 0x561c54827050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.436+0000 7f058098e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561c5482e920 con 0x561c54816260 2024-04-24T06:43:50.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.436+0000 7f058098e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561c5482e250 con 0x561c5481bdc0 2024-04-24T06:43:50.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.436+0000 7f058098e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561c5482eff0 con 0x561c54816c50 2024-04-24T06:43:50.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.436+0000 7f058098e740 10 client.0 _ll_register_callbacks cb 0x7f0560003e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:43:50.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.436+0000 7f0576a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c54816260 0x561c54823db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.436+0000 7f056fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c54816c50 0x561c54824330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.436+0000 7f0576a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c54816260 0x561c54823db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39672/0 (socket says 172.21.15.5:39672) 2024-04-24T06:43:50.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.436+0000 7f0576a00640 1 -- 172.21.15.5:0/3411363902 learned_addr learned my addr 172.21.15.5:0/3411363902 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:50.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.436+0000 7f0577400640 1 --2- 172.21.15.5:0/3411363902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c5481bdc0 0x561c54827050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.436+0000 7f0576a00640 1 -- 172.21.15.5:0/3411363902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c54816c50 msgr2=0x561c54824330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:50.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.436+0000 7f0576a00640 1 --2- 172.21.15.5:0/3411363902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c54816c50 0x561c54824330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.436+0000 7f0576a00640 1 -- 172.21.15.5:0/3411363902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c5481bdc0 msgr2=0x561c54827050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:50.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.436+0000 7f0576a00640 1 --2- 172.21.15.5:0/3411363902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c5481bdc0 0x561c54827050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.436+0000 7f0576a00640 1 -- 172.21.15.5:0/3411363902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c54827640 con 0x561c54816260 2024-04-24T06:43:50.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.436+0000 7f0577400640 1 --2- 172.21.15.5:0/3411363902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c5481bdc0 0x561c54827050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:43:50.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.436+0000 7f056fe00640 1 --2- 172.21.15.5:0/3411363902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c54816c50 0x561c54824330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:43:50.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.437+0000 7f0576a00640 1 --2- 172.21.15.5:0/3411363902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c54816260 0x561c54823db0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x561c548127a0 tx=0x7f0568017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3a9752aa5f469db server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:50.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.437+0000 7f0564c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:43:50.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.437+0000 7f0564c00640 1 -- 172.21.15.5:0/3411363902 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056802b430 con 0x561c54816260 2024-04-24T06:43:50.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.437+0000 7f0564c00640 1 -- 172.21.15.5:0/3411363902 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f056802b5d0 con 0x561c54816260 2024-04-24T06:43:50.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.437+0000 7f058098e740 1 -- 172.21.15.5:0/3411363902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05600f3e50 con 0x561c54816260 2024-04-24T06:43:50.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.437+0000 7f0564c00640 1 -- 172.21.15.5:0/3411363902 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f056802b810 con 0x561c54816260 2024-04-24T06:43:50.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.437+0000 7f058098e740 10 client.15051 fetch_fsmap learned FSMap version 22 2024-04-24T06:43:50.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.437+0000 7f058098e740 1 -- 172.21.15.5:0/3411363902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x561c54893e10 con 0x561c54816260 2024-04-24T06:43:50.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.438+0000 7f0564c00640 1 -- 172.21.15.5:0/3411363902 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05600f3e50 con 0x561c54816260 2024-04-24T06:43:50.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.438+0000 7f0564c00640 1 -- 172.21.15.5:0/3411363902 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f0568004bb0 con 0x561c54816260 2024-04-24T06:43:50.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.438+0000 7f058098e740 10 client.15051 fetch_fsmap finished waiting for FSMap version 22 2024-04-24T06:43:50.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.438+0000 7f058098e740 10 client.15051 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-24T06:43:50.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.438+0000 7f058098e740 20 client.15051 populate_metadata read hostname 'smithi005' 2024-04-24T06:43:50.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.438+0000 7f058098e740 1 --2- 172.21.15.5:0/3411363902 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x561c54898a30 0x561c54894e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.438+0000 7f058098e740 4 client.15051 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T06:43:50.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.438+0000 7f058098e740 1 -- 172.21.15.5:0/3411363902 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "status"}) -- 0x561c547833a0 con 0x561c54898a30 2024-04-24T06:43:50.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.438+0000 7f056fe00640 1 --2- 172.21.15.5:0/3411363902 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x561c54898a30 0x561c54894e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.439+0000 7f056fe00640 1 --2- 172.21.15.5:0/3411363902 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x561c54898a30 0x561c54894e70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e9f23f2a165c823f server_cookie=16077086acbbf446 in_seq=0 out_seq=0 2024-04-24T06:43:50.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.439+0000 7f0564c00640 10 client.15051 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:43:50.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.439+0000 7f0564c00640 1 -- 172.21.15.5:0/3411363902 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x561c547833a0 con 0x561c54898a30 2024-04-24T06:43:50.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.439+0000 7f0564c00640 10 client.15051 handle_command_reply: tid=0 2024-04-24T06:43:50.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.439+0000 7f058098e740 1 client.15051 shutdown 2024-04-24T06:43:50.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.439+0000 7f058098e740 1 -- 172.21.15.5:0/3411363902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c54816260 msgr2=0x561c54823db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:50.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.439+0000 7f058098e740 1 --2- 172.21.15.5:0/3411363902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c54816260 0x561c54823db0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x561c548127a0 tx=0x7f0568017000 comp rx=0 tx=0).stop 2024-04-24T06:43:50.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.439+0000 7f058098e740 1 -- 172.21.15.5:0/3411363902 shutdown_connections 2024-04-24T06:43:50.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.439+0000 7f058098e740 1 --2- 172.21.15.5:0/3411363902 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x561c54898a30 0x561c54894e70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.439+0000 7f058098e740 1 --2- 172.21.15.5:0/3411363902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561c5481bdc0 0x561c54827050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.439+0000 7f058098e740 1 --2- 172.21.15.5:0/3411363902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561c54816c50 0x561c54824330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.439+0000 7f0564c00640 0 client.15051 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:43:50.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.439+0000 7f058098e740 1 --2- 172.21.15.5:0/3411363902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c54816260 0x561c54823db0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.439+0000 7f058098e740 1 -- 172.21.15.5:0/3411363902 >> 172.21.15.5:0/3411363902 conn(0x561c548118d0 msgr2=0x561c548147e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:50.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.439+0000 7f058098e740 1 -- 172.21.15.5:0/3411363902 shutdown_connections 2024-04-24T06:43:50.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.439+0000 7f058098e740 1 -- 172.21.15.5:0/3411363902 wait complete. 2024-04-24T06:43:50.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.440+0000 7f058098e740 20 client.15051 trim_cache size 0 max 16384 2024-04-24T06:43:50.445 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:43:50.445 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "43e9bad0-0204-11ef-bc93-c7b262605968", 2024-04-24T06:43:50.445 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 2, 2024-04-24T06:43:50.446 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24539, 2024-04-24T06:43:50.446 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-04-24T06:43:50.446 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-04-24T06:43:50.446 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-04-24T06:43:50.446 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 131.11899853099999, 2024-04-24T06:43:50.446 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 22, 2024-04-24T06:43:50.446 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 88, 2024-04-24T06:43:50.446 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 86, 2024-04-24T06:43:50.446 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 149.65621066400001 2024-04-24T06:43:50.446 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:43:50.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.440+0000 7f0577e00640 1 -- 172.21.15.5:0/368301995 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f055c0a3bb0 msgr2=0x7f055c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:50.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.440+0000 7f0577e00640 1 --2- 172.21.15.5:0/368301995 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f055c0a3bb0 0x7f055c0a6070 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x561c54817050 tx=0x7f0568005f90 comp rx=0 tx=0).stop 2024-04-24T06:43:50.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.440+0000 7f0577e00640 1 -- 172.21.15.5:0/368301995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0578109670 msgr2=0x7f057819ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:50.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.440+0000 7f0577e00640 1 --2- 172.21.15.5:0/368301995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0578109670 0x7f057819ecd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0560009230 tx=0x7f05600042f0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.440+0000 7f0577e00640 1 -- 172.21.15.5:0/368301995 shutdown_connections 2024-04-24T06:43:50.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.440+0000 7f0577e00640 1 --2- 172.21.15.5:0/368301995 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f055c0a3bb0 0x7f055c0a6070 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.440+0000 7f0577e00640 1 --2- 172.21.15.5:0/368301995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f057810a030 0x7f05781a5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.440+0000 7f0577e00640 1 --2- 172.21.15.5:0/368301995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0578109670 0x7f057819ecd0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.440+0000 7f0577e00640 1 --2- 172.21.15.5:0/368301995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0578108480 0x7f057819e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.440+0000 7f0577e00640 1 -- 172.21.15.5:0/368301995 >> 172.21.15.5:0/368301995 conn(0x7f0578077440 msgr2=0x7f057810c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:50.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.441+0000 7f0577e00640 1 -- 172.21.15.5:0/368301995 shutdown_connections 2024-04-24T06:43:50.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.441+0000 7f0577e00640 1 -- 172.21.15.5:0/368301995 wait complete. 2024-04-24T06:43:50.455 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:50 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/2746923473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T06:43:50.456 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:50 smithi005 ceph-mon[27798]: pgmap v382: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T06:43:50.456 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:50 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/2665838611' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:43:50.456 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:50 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/3203593702' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:43:50.458 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-24T06:43:50.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:50 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/2746923473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T06:43:50.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:50 smithi037 ceph-mon[32864]: pgmap v382: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T06:43:50.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:50 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/2665838611' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:43:50.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:50 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/3203593702' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:43:50.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.652+0000 7f72e2000640 1 -- 172.21.15.5:0/3793496637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72dc068620 msgr2=0x7f72dc101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:50.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.652+0000 7f72e2000640 1 --2- 172.21.15.5:0/3793496637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72dc068620 0x7f72dc101290 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f72d0009900 tx=0x7f72d002a410 comp rx=0 tx=0).stop 2024-04-24T06:43:50.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.652+0000 7f72e2000640 1 -- 172.21.15.5:0/3793496637 shutdown_connections 2024-04-24T06:43:50.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.652+0000 7f72e2000640 1 --2- 172.21.15.5:0/3793496637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72dc1064d0 0x7f72dc1088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.652+0000 7f72e2000640 1 --2- 172.21.15.5:0/3793496637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72dc1017d0 0x7f72dc101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.652+0000 7f72e2000640 1 --2- 172.21.15.5:0/3793496637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72dc068620 0x7f72dc101290 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.652+0000 7f72e2000640 1 -- 172.21.15.5:0/3793496637 >> 172.21.15.5:0/3793496637 conn(0x7f72dc0fb610 msgr2=0x7f72dc0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:50.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.652+0000 7f72e2000640 1 -- 172.21.15.5:0/3793496637 shutdown_connections 2024-04-24T06:43:50.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.652+0000 7f72e2000640 1 -- 172.21.15.5:0/3793496637 wait complete. 2024-04-24T06:43:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.654+0000 7f72e2000640 1 Processor -- start 2024-04-24T06:43:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.654+0000 7f72e2000640 1 -- start start 2024-04-24T06:43:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.654+0000 7f72e2000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72dc1017d0 0x7f72dc1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.654+0000 7f72e2000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72dc1064d0 0x7f72dc198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.654+0000 7f72e2000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72dc19d970 0x7f72dc19fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.654+0000 7f72e2000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f72dc10db80 con 0x7f72dc1064d0 2024-04-24T06:43:50.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.654+0000 7f72e2000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f72dc10da00 con 0x7f72dc19d970 2024-04-24T06:43:50.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.654+0000 7f72e2000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f72dc10dd00 con 0x7f72dc1017d0 2024-04-24T06:43:50.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.654+0000 7f72dbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72dc1064d0 0x7f72dc198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.654+0000 7f72dbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72dc1064d0 0x7f72dc198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39690/0 (socket says 172.21.15.5:39690) 2024-04-24T06:43:50.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.654+0000 7f72dbe00640 1 -- 172.21.15.5:0/3486952357 learned_addr learned my addr 172.21.15.5:0/3486952357 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:50.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.654+0000 7f72e0c00640 1 --2- 172.21.15.5:0/3486952357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72dc1017d0 0x7f72dc1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.655+0000 7f72e1600640 1 --2- 172.21.15.5:0/3486952357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72dc19d970 0x7f72dc19fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.655+0000 7f72dbe00640 1 -- 172.21.15.5:0/3486952357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72dc1017d0 msgr2=0x7f72dc1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:50.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.655+0000 7f72dbe00640 1 --2- 172.21.15.5:0/3486952357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72dc1017d0 0x7f72dc1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.655+0000 7f72dbe00640 1 -- 172.21.15.5:0/3486952357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72dc19d970 msgr2=0x7f72dc19fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:50.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.655+0000 7f72dbe00640 1 --2- 172.21.15.5:0/3486952357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72dc19d970 0x7f72dc19fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.655+0000 7f72dbe00640 1 -- 172.21.15.5:0/3486952357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72dc1a03d0 con 0x7f72dc1064d0 2024-04-24T06:43:50.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.655+0000 7f72dbe00640 1 --2- 172.21.15.5:0/3486952357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72dc1064d0 0x7f72dc198910 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f72c800d770 tx=0x7f72c800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:50.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.655+0000 7f72d9600640 1 -- 172.21.15.5:0/3486952357 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c8017070 con 0x7f72dc1064d0 2024-04-24T06:43:50.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.655+0000 7f72d9600640 1 -- 172.21.15.5:0/3486952357 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72c8009c30 con 0x7f72dc1064d0 2024-04-24T06:43:50.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.656+0000 7f72e2000640 1 -- 172.21.15.5:0/3486952357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72dc1a06c0 con 0x7f72dc1064d0 2024-04-24T06:43:50.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.656+0000 7f72d9600640 1 -- 172.21.15.5:0/3486952357 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c8003dd0 con 0x7f72dc1064d0 2024-04-24T06:43:50.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.656+0000 7f72e2000640 1 -- 172.21.15.5:0/3486952357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72dc1a0c10 con 0x7f72dc1064d0 2024-04-24T06:43:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.659+0000 7f72e6431740 1 Processor -- start 2024-04-24T06:43:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.659+0000 7f72e6431740 1 -- start start 2024-04-24T06:43:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.659+0000 7f72d9600640 1 -- 172.21.15.5:0/3486952357 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f72c8003fb0 con 0x7f72dc1064d0 2024-04-24T06:43:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.659+0000 7f72e6431740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bebcde0670 0x55bebcde0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.659+0000 7f72e6431740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bebcde1060 0x55bebcde5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.659+0000 7f72e6431740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bebcde61d0 0x55bebcde8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.659+0000 7f72e6431740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bebcd41b10 con 0x55bebcde0670 2024-04-24T06:43:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.659+0000 7f72e6431740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bebcce82b0 con 0x55bebcde1060 2024-04-24T06:43:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.659+0000 7f72e6431740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55bebcd3dad0 con 0x55bebcde61d0 2024-04-24T06:43:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.660+0000 7f72e1600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bebcde0670 0x55bebcde0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.660+0000 7f72e1600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bebcde0670 0x55bebcde0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39698/0 (socket says 172.21.15.5:39698) 2024-04-24T06:43:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.660+0000 7f72e1600640 1 -- 172.21.15.5:0/3985115265 learned_addr learned my addr 172.21.15.5:0/3985115265 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.660+0000 7f72dbe00640 1 --2- 172.21.15.5:0/3985115265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bebcde61d0 0x55bebcde8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.660+0000 7f72e0c00640 1 --2- 172.21.15.5:0/3985115265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bebcde1060 0x55bebcde5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.660+0000 7f72e1600640 1 -- 172.21.15.5:0/3985115265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bebcde61d0 msgr2=0x55bebcde8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.660+0000 7f72e1600640 1 --2- 172.21.15.5:0/3985115265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bebcde61d0 0x55bebcde8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.660+0000 7f72e1600640 1 -- 172.21.15.5:0/3985115265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bebcde1060 msgr2=0x55bebcde5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:50.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.660+0000 7f72e1600640 1 --2- 172.21.15.5:0/3985115265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bebcde1060 0x55bebcde5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.660+0000 7f72e1600640 1 -- 172.21.15.5:0/3985115265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bebca542d0 con 0x55bebcde0670 2024-04-24T06:43:50.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.661+0000 7f72e1600640 1 --2- 172.21.15.5:0/3985115265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bebcde0670 0x55bebcde0a90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f72d4009fd0 tx=0x7f72d40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76361d7e5cbe0391 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:50.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.661+0000 7f72d9600640 1 --2- 172.21.15.5:0/3486952357 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f72c40a3bb0 0x7f72c40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.661+0000 7f72cdc00640 1 -- 172.21.15.5:0/3985115265 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d4004d20 con 0x55bebcde0670 2024-04-24T06:43:50.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.661+0000 7f72cdc00640 1 -- 172.21.15.5:0/3985115265 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72d4004050 con 0x55bebcde0670 2024-04-24T06:43:50.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.661+0000 7f72cdc00640 1 -- 172.21.15.5:0/3985115265 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d4005240 con 0x55bebcde0670 2024-04-24T06:43:50.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.661+0000 7f72d9600640 1 -- 172.21.15.5:0/3486952357 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f72c80cfcc0 con 0x7f72dc1064d0 2024-04-24T06:43:50.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.661+0000 7f72e0c00640 1 --2- 172.21.15.5:0/3486952357 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f72c40a3bb0 0x7f72c40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.662+0000 7f72e6431740 1 -- 172.21.15.5:0/3985115265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bebcde0670 msgr2=0x55bebcde0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:50.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.662+0000 7f72e6431740 1 --2- 172.21.15.5:0/3985115265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bebcde0670 0x55bebcde0a90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f72d4009fd0 tx=0x7f72d40118d0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.662+0000 7f72e6431740 1 -- 172.21.15.5:0/3985115265 shutdown_connections 2024-04-24T06:43:50.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.662+0000 7f72e0c00640 1 --2- 172.21.15.5:0/3486952357 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f72c40a3bb0 0x7f72c40a6070 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f72d00314e0 tx=0x7f72d00315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:50.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.662+0000 7f72e6431740 1 --2- 172.21.15.5:0/3985115265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bebcde61d0 0x55bebcde8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.662+0000 7f72e6431740 1 --2- 172.21.15.5:0/3985115265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bebcde1060 0x55bebcde5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.662+0000 7f72e6431740 1 --2- 172.21.15.5:0/3985115265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bebcde0670 0x55bebcde0a90 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.662+0000 7f72e6431740 1 -- 172.21.15.5:0/3985115265 >> 172.21.15.5:0/3985115265 conn(0x55bebcddbce0 msgr2=0x55bebcddc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:50.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.662+0000 7f72e6431740 1 -- 172.21.15.5:0/3985115265 shutdown_connections 2024-04-24T06:43:50.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.662+0000 7f72e6431740 1 -- 172.21.15.5:0/3985115265 wait complete. 2024-04-24T06:43:50.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.662+0000 7f72e6431740 1 -- start start 2024-04-24T06:43:50.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.663+0000 7f72e6431740 1 Processor -- start 2024-04-24T06:43:50.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.663+0000 7f72e6431740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bebcde0670 0x55bebcdeaac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.663+0000 7f72e6431740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bebcde1060 0x55bebcdeb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.663+0000 7f72e6431740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bebcde61d0 0x55bebcdf2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.663+0000 7f72e6431740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bebcde9180 con 0x55bebcde0670 2024-04-24T06:43:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.663+0000 7f72e6431740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bebccfb590 con 0x55bebcde1060 2024-04-24T06:43:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.663+0000 7f72e6431740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55bebcde9300 con 0x55bebcde61d0 2024-04-24T06:43:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.663+0000 7f72e6431740 10 client.0 _ll_register_callbacks cb 0x55bebccde3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:43:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.663+0000 7f72e0c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bebcde0670 0x55bebcdeaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.663+0000 7f72e0c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bebcde0670 0x55bebcdeaac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39702/0 (socket says 172.21.15.5:39702) 2024-04-24T06:43:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.663+0000 7f72e0c00640 1 -- 172.21.15.5:0/1133558380 learned_addr learned my addr 172.21.15.5:0/1133558380 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.664+0000 7f72dbe00640 1 --2- 172.21.15.5:0/1133558380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bebcde1060 0x55bebcdeb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.664+0000 7f72e1600640 1 --2- 172.21.15.5:0/1133558380 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bebcde61d0 0x55bebcdf2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.664+0000 7f72e0c00640 1 -- 172.21.15.5:0/1133558380 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bebcde61d0 msgr2=0x55bebcdf2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.664+0000 7f72e0c00640 1 --2- 172.21.15.5:0/1133558380 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bebcde61d0 0x55bebcdf2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.664+0000 7f72e0c00640 1 -- 172.21.15.5:0/1133558380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bebcde1060 msgr2=0x55bebcdeb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.664+0000 7f72e0c00640 1 --2- 172.21.15.5:0/1133558380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bebcde1060 0x55bebcdeb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.664+0000 7f72e0c00640 1 -- 172.21.15.5:0/1133558380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bebcb1e6b0 con 0x55bebcde0670 2024-04-24T06:43:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.664+0000 7f72e0c00640 1 --2- 172.21.15.5:0/1133558380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bebcde0670 0x55bebcdeaac0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f72d002aa00 tx=0x7f72d0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb4547b695bc2d2f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.664+0000 7f72c2000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:43:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.665+0000 7f72c2000640 1 -- 172.21.15.5:0/1133558380 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d004e070 con 0x55bebcde0670 2024-04-24T06:43:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.665+0000 7f72c2000640 1 -- 172.21.15.5:0/1133558380 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f72d0031d40 con 0x55bebcde0670 2024-04-24T06:43:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.665+0000 7f72e6431740 1 -- 172.21.15.5:0/1133558380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72d4004050 con 0x55bebcde0670 2024-04-24T06:43:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.665+0000 7f72c2000640 1 -- 172.21.15.5:0/1133558380 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f72d005a050 con 0x55bebcde0670 2024-04-24T06:43:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.665+0000 7f72e6431740 10 client.15063 fetch_fsmap learned FSMap version 22 2024-04-24T06:43:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.665+0000 7f72e6431740 1 -- 172.21.15.5:0/1133558380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55bebcdf5420 con 0x55bebcde0670 2024-04-24T06:43:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.665+0000 7f72c2000640 1 -- 172.21.15.5:0/1133558380 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72d4004050 con 0x55bebcde0670 2024-04-24T06:43:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.665+0000 7f72c2000640 1 -- 172.21.15.5:0/1133558380 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f72d004a750 con 0x55bebcde0670 2024-04-24T06:43:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.665+0000 7f72e6431740 10 client.15063 fetch_fsmap finished waiting for FSMap version 22 2024-04-24T06:43:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.666+0000 7f72e6431740 10 client.15063 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-24T06:43:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.666+0000 7f72e6431740 20 client.15063 populate_metadata read hostname 'smithi005' 2024-04-24T06:43:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.666+0000 7f72e6431740 1 --2- 172.21.15.5:0/1133558380 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55bebcdf7a50 0x55bebcdf8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.666+0000 7f72e6431740 4 client.15063 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:43:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.666+0000 7f72e6431740 1 -- 172.21.15.5:0/1133558380 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bebcd4f530 con 0x55bebcdf7a50 2024-04-24T06:43:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.666+0000 7f72dbe00640 1 --2- 172.21.15.5:0/1133558380 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55bebcdf7a50 0x55bebcdf8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.666+0000 7f72dbe00640 1 --2- 172.21.15.5:0/1133558380 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55bebcdf7a50 0x55bebcdf8ec0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8b902c431e556f11 server_cookie=61cde7b0a315f716 in_seq=0 out_seq=0 2024-04-24T06:43:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.666+0000 7f72c2000640 10 client.15063 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:43:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.667+0000 7f72c2000640 1 -- 172.21.15.5:0/1133558380 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bebcd4f530 con 0x55bebcdf7a50 2024-04-24T06:43:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.667+0000 7f72c2000640 10 client.15063 handle_command_reply: tid=0 2024-04-24T06:43:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.667+0000 7f72e6431740 1 client.15063 shutdown 2024-04-24T06:43:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.668+0000 7f72e6431740 1 -- 172.21.15.5:0/1133558380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bebcde0670 msgr2=0x55bebcdeaac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.668+0000 7f72e6431740 1 --2- 172.21.15.5:0/1133558380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bebcde0670 0x55bebcdeaac0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f72d002aa00 tx=0x7f72d0009260 comp rx=0 tx=0).stop 2024-04-24T06:43:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.668+0000 7f72e6431740 1 -- 172.21.15.5:0/1133558380 shutdown_connections 2024-04-24T06:43:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.668+0000 7f72e6431740 1 --2- 172.21.15.5:0/1133558380 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55bebcdf7a50 0x55bebcdf8ec0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.668+0000 7f72e6431740 1 --2- 172.21.15.5:0/1133558380 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bebcde61d0 0x55bebcdf2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.668+0000 7f72e6431740 1 --2- 172.21.15.5:0/1133558380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bebcde1060 0x55bebcdeb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.668+0000 7f72c2000640 0 client.15063 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:43:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.668+0000 7f72e6431740 1 --2- 172.21.15.5:0/1133558380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bebcde0670 0x55bebcdeaac0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.668+0000 7f72e6431740 1 -- 172.21.15.5:0/1133558380 >> 172.21.15.5:0/1133558380 conn(0x55bebcddbce0 msgr2=0x55bebcdd6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.668+0000 7f72e6431740 1 -- 172.21.15.5:0/1133558380 shutdown_connections 2024-04-24T06:43:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.668+0000 7f72e6431740 1 -- 172.21.15.5:0/1133558380 wait complete. 2024-04-24T06:43:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.668+0000 7f72e6431740 20 client.15063 trim_cache size 0 max 16384 2024-04-24T06:43:50.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:50 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/2746923473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T06:43:50.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:50 smithi137 ceph-mon[33017]: pgmap v382: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T06:43:50.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:50 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/2665838611' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:43:50.698 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:50 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/3203593702' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:43:50.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.688+0000 7f72e6431740 1 Processor -- start 2024-04-24T06:43:50.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.688+0000 7f72e6431740 1 -- start start 2024-04-24T06:43:50.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.689+0000 7f72e6431740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bebcde0670 0x55bebcdf87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.689+0000 7f72e6431740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bebcde1060 0x55bebcdf1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.689+0000 7f72e6431740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bebcde61d0 0x55bebcde85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.689+0000 7f72e6431740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bebcdf7200 con 0x55bebcde1060 2024-04-24T06:43:50.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.689+0000 7f72e6431740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bebcdf78d0 con 0x55bebcde0670 2024-04-24T06:43:50.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.689+0000 7f72e6431740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55bebcdf7fa0 con 0x55bebcde61d0 2024-04-24T06:43:50.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.689+0000 7f72dbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bebcde1060 0x55bebcdf1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.689+0000 7f72e0c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bebcde0670 0x55bebcdf87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.689+0000 7f72dbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bebcde1060 0x55bebcdf1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39708/0 (socket says 172.21.15.5:39708) 2024-04-24T06:43:50.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.689+0000 7f72dbe00640 1 -- 172.21.15.5:0/2558027953 learned_addr learned my addr 172.21.15.5:0/2558027953 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:50.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.689+0000 7f72e0c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bebcde0670 0x55bebcdf87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:32994/0 (socket says 172.21.15.5:32994) 2024-04-24T06:43:50.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.689+0000 7f72e1600640 1 --2- 172.21.15.5:0/2558027953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bebcde61d0 0x55bebcde85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.690+0000 7f72dbe00640 1 -- 172.21.15.5:0/2558027953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bebcde61d0 msgr2=0x55bebcde85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:50.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.690+0000 7f72dbe00640 1 --2- 172.21.15.5:0/2558027953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bebcde61d0 0x55bebcde85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.690+0000 7f72dbe00640 1 -- 172.21.15.5:0/2558027953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bebcde0670 msgr2=0x55bebcdf87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:50.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.690+0000 7f72dbe00640 1 --2- 172.21.15.5:0/2558027953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bebcde0670 0x55bebcdf87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.690+0000 7f72dbe00640 1 -- 172.21.15.5:0/2558027953 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bebcde5760 con 0x55bebcde1060 2024-04-24T06:43:50.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.690+0000 7f72e0c00640 1 --2- 172.21.15.5:0/2558027953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bebcde0670 0x55bebcdf87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:43:50.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.690+0000 7f72dbe00640 1 --2- 172.21.15.5:0/2558027953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bebcde1060 0x55bebcdf1fd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f72c800d740 tx=0x7f72c80d3740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec8ca309d81903ab server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:50.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.690+0000 7f72cd200640 1 -- 172.21.15.5:0/2558027953 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c8020040 con 0x55bebcde1060 2024-04-24T06:43:50.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.690+0000 7f72cd200640 1 -- 172.21.15.5:0/2558027953 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72c8009de0 con 0x55bebcde1060 2024-04-24T06:43:50.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.690+0000 7f72cd200640 1 -- 172.21.15.5:0/2558027953 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c80cd8d0 con 0x55bebcde1060 2024-04-24T06:43:50.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.691+0000 7f72e6431740 1 -- 172.21.15.5:0/2558027953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bebcde1060 msgr2=0x55bebcdf1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:50.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.691+0000 7f72e6431740 1 --2- 172.21.15.5:0/2558027953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bebcde1060 0x55bebcdf1fd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f72c800d740 tx=0x7f72c80d3740 comp rx=0 tx=0).stop 2024-04-24T06:43:50.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.691+0000 7f72e6431740 1 -- 172.21.15.5:0/2558027953 shutdown_connections 2024-04-24T06:43:50.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.691+0000 7f72e6431740 1 --2- 172.21.15.5:0/2558027953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bebcde61d0 0x55bebcde85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.691+0000 7f72e6431740 1 --2- 172.21.15.5:0/2558027953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bebcde1060 0x55bebcdf1fd0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.691+0000 7f72e6431740 1 --2- 172.21.15.5:0/2558027953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bebcde0670 0x55bebcdf87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.691+0000 7f72e6431740 1 -- 172.21.15.5:0/2558027953 >> 172.21.15.5:0/2558027953 conn(0x55bebcddbce0 msgr2=0x55bebcddc0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:50.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.691+0000 7f72e6431740 1 -- 172.21.15.5:0/2558027953 shutdown_connections 2024-04-24T06:43:50.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.691+0000 7f72e6431740 1 -- 172.21.15.5:0/2558027953 wait complete. 2024-04-24T06:43:50.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.692+0000 7f72e6431740 1 -- start start 2024-04-24T06:43:50.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.692+0000 7f72e6431740 1 Processor -- start 2024-04-24T06:43:50.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.692+0000 7f72e6431740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bebcde0670 0x55bebcdedb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.692+0000 7f72e6431740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bebcde61d0 0x55bebcdee0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.692+0000 7f72e6431740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bebcdf10e0 0x55bebcdf1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.692+0000 7f72e6431740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bebcdf8d40 con 0x55bebcdf10e0 2024-04-24T06:43:50.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.692+0000 7f72e6431740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bebcdf8670 con 0x55bebcde61d0 2024-04-24T06:43:50.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.692+0000 7f72e6431740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55bebcdf9410 con 0x55bebcde0670 2024-04-24T06:43:50.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.692+0000 7f72e6431740 10 client.0 _ll_register_callbacks cb 0x7f72d00042a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:43:50.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.692+0000 7f72e1600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bebcdf10e0 0x55bebcdf1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.692+0000 7f72e0c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bebcde0670 0x55bebcdedb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.693+0000 7f72e1600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bebcdf10e0 0x55bebcdf1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39718/0 (socket says 172.21.15.5:39718) 2024-04-24T06:43:50.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.693+0000 7f72e1600640 1 -- 172.21.15.5:0/2837317797 learned_addr learned my addr 172.21.15.5:0/2837317797 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:50.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.693+0000 7f72e0c00640 1 --2- 172.21.15.5:0/2837317797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bebcde0670 0x55bebcdedb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:54430/0 (socket says 172.21.15.5:54430) 2024-04-24T06:43:50.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.693+0000 7f72dbe00640 1 --2- 172.21.15.5:0/2837317797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bebcde61d0 0x55bebcdee0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.693+0000 7f72e0c00640 1 -- 172.21.15.5:0/2837317797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bebcde61d0 msgr2=0x55bebcdee0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:50.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.693+0000 7f72e0c00640 1 --2- 172.21.15.5:0/2837317797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bebcde61d0 0x55bebcdee0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.693+0000 7f72e0c00640 1 -- 172.21.15.5:0/2837317797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bebcdf10e0 msgr2=0x55bebcdf1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:50.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.693+0000 7f72e0c00640 1 --2- 172.21.15.5:0/2837317797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bebcdf10e0 0x55bebcdf1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.693+0000 7f72e0c00640 1 -- 172.21.15.5:0/2837317797 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bebcdf1aa0 con 0x55bebcde0670 2024-04-24T06:43:50.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.693+0000 7f72e1600640 1 --2- 172.21.15.5:0/2837317797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bebcdf10e0 0x55bebcdf1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:43:50.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.693+0000 7f72e0c00640 1 --2- 172.21.15.5:0/2837317797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bebcde0670 0x55bebcdedb80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f72d004ad30 tx=0x7f72d002a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=217aeb2393dbc867 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:50.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.693+0000 7f72c2000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:43:50.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.693+0000 7f72c2000640 1 -- 172.21.15.5:0/2837317797 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d0062070 con 0x55bebcde0670 2024-04-24T06:43:50.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.694+0000 7f72c2000640 1 -- 172.21.15.5:0/2837317797 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f72d003d520 con 0x55bebcde0670 2024-04-24T06:43:50.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.694+0000 7f72e6431740 1 -- 172.21.15.5:0/2837317797 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72c8009de0 con 0x55bebcde0670 2024-04-24T06:43:50.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.694+0000 7f72c2000640 1 -- 172.21.15.5:0/2837317797 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f72d006e050 con 0x55bebcde0670 2024-04-24T06:43:50.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.694+0000 7f72e6431740 10 client.25180 fetch_fsmap learned FSMap version 22 2024-04-24T06:43:50.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.694+0000 7f72e6431740 1 -- 172.21.15.5:0/2837317797 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55bebce5e250 con 0x55bebcde0670 2024-04-24T06:43:50.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.694+0000 7f72c2000640 1 -- 172.21.15.5:0/2837317797 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72c8009de0 con 0x55bebcde0670 2024-04-24T06:43:50.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.694+0000 7f72c2000640 1 -- 172.21.15.5:0/2837317797 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f72d0053670 con 0x55bebcde0670 2024-04-24T06:43:50.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.694+0000 7f72e6431740 10 client.25180 fetch_fsmap finished waiting for FSMap version 22 2024-04-24T06:43:50.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.694+0000 7f72e6431740 10 client.25180 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-24T06:43:50.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.694+0000 7f72e6431740 20 client.25180 populate_metadata read hostname 'smithi005' 2024-04-24T06:43:50.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.694+0000 7f72e6431740 1 --2- 172.21.15.5:0/2837317797 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55bebce62e30 0x55bebce61b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.695+0000 7f72e6431740 4 client.25180 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T06:43:50.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.695+0000 7f72e6431740 1 -- 172.21.15.5:0/2837317797 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "status"}) -- 0x55bebcd493a0 con 0x55bebce62e30 2024-04-24T06:43:50.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.695+0000 7f72dbe00640 1 --2- 172.21.15.5:0/2837317797 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55bebce62e30 0x55bebce61b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.695+0000 7f72dbe00640 1 --2- 172.21.15.5:0/2837317797 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55bebce62e30 0x55bebce61b40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d7b8987cd16778e6 server_cookie=8a1170f155d851d4 in_seq=0 out_seq=0 2024-04-24T06:43:50.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.695+0000 7f72c2000640 10 client.25180 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:43:50.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.695+0000 7f72c2000640 1 -- 172.21.15.5:0/2837317797 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55bebcd493a0 con 0x55bebce62e30 2024-04-24T06:43:50.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.695+0000 7f72c2000640 10 client.25180 handle_command_reply: tid=0 2024-04-24T06:43:50.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.695+0000 7f72e6431740 1 client.25180 shutdown 2024-04-24T06:43:50.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.696+0000 7f72e6431740 1 -- 172.21.15.5:0/2837317797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bebcde0670 msgr2=0x55bebcdedb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:50.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.696+0000 7f72e6431740 1 --2- 172.21.15.5:0/2837317797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bebcde0670 0x55bebcdedb80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f72d004ad30 tx=0x7f72d002a410 comp rx=0 tx=0).stop 2024-04-24T06:43:50.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.696+0000 7f72e6431740 1 -- 172.21.15.5:0/2837317797 shutdown_connections 2024-04-24T06:43:50.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.696+0000 7f72e6431740 1 --2- 172.21.15.5:0/2837317797 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55bebce62e30 0x55bebce61b40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.696+0000 7f72e6431740 1 --2- 172.21.15.5:0/2837317797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bebcdf10e0 0x55bebcdf1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.696+0000 7f72e6431740 1 --2- 172.21.15.5:0/2837317797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bebcde61d0 0x55bebcdee0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.696+0000 7f72c2000640 0 client.25180 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:43:50.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.696+0000 7f72e6431740 1 --2- 172.21.15.5:0/2837317797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bebcde0670 0x55bebcdedb80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.696+0000 7f72e6431740 1 -- 172.21.15.5:0/2837317797 >> 172.21.15.5:0/2837317797 conn(0x55bebcddbce0 msgr2=0x55bebcddc0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:50.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.696+0000 7f72e6431740 1 -- 172.21.15.5:0/2837317797 shutdown_connections 2024-04-24T06:43:50.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.696+0000 7f72e6431740 1 -- 172.21.15.5:0/2837317797 wait complete. 2024-04-24T06:43:50.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.696+0000 7f72e6431740 20 client.25180 trim_cache size 0 max 16384 2024-04-24T06:43:50.707 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:43:50.707 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "43e9bad0-0204-11ef-bc93-c7b262605968", 2024-04-24T06:43:50.707 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 1, 2024-04-24T06:43:50.707 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24550, 2024-04-24T06:43:50.707 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-04-24T06:43:50.707 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-04-24T06:43:50.707 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-04-24T06:43:50.708 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 132.387718404, 2024-04-24T06:43:50.708 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 22, 2024-04-24T06:43:50.708 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 88, 2024-04-24T06:43:50.708 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 86, 2024-04-24T06:43:50.708 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 153.91543740399999 2024-04-24T06:43:50.708 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:43:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.697+0000 7f72e2000640 1 -- 172.21.15.5:0/3486952357 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f72c40a3bb0 msgr2=0x7f72c40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.697+0000 7f72e2000640 1 --2- 172.21.15.5:0/3486952357 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f72c40a3bb0 0x7f72c40a6070 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f72d00314e0 tx=0x7f72d00315b0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.697+0000 7f72e2000640 1 -- 172.21.15.5:0/3486952357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72dc1064d0 msgr2=0x7f72dc198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.697+0000 7f72e2000640 1 --2- 172.21.15.5:0/3486952357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72dc1064d0 0x7f72dc198910 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f72c800d770 tx=0x7f72c800dc30 comp rx=0 tx=0).stop 2024-04-24T06:43:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.697+0000 7f72e2000640 1 -- 172.21.15.5:0/3486952357 shutdown_connections 2024-04-24T06:43:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.697+0000 7f72e2000640 1 --2- 172.21.15.5:0/3486952357 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f72c40a3bb0 0x7f72c40a6070 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.697+0000 7f72e2000640 1 --2- 172.21.15.5:0/3486952357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72dc19d970 0x7f72dc19fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.697+0000 7f72e2000640 1 --2- 172.21.15.5:0/3486952357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72dc1064d0 0x7f72dc198910 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.697+0000 7f72e2000640 1 --2- 172.21.15.5:0/3486952357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72dc1017d0 0x7f72dc1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.697+0000 7f72e2000640 1 -- 172.21.15.5:0/3486952357 >> 172.21.15.5:0/3486952357 conn(0x7f72dc0fb610 msgr2=0x7f72dc1024b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.697+0000 7f72e2000640 1 -- 172.21.15.5:0/3486952357 shutdown_connections 2024-04-24T06:43:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.697+0000 7f72e2000640 1 -- 172.21.15.5:0/3486952357 wait complete. 2024-04-24T06:43:50.717 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-24T06:43:50.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.929+0000 7ffbad600640 1 -- 172.21.15.5:0/431417091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffba8105450 msgr2=0x7ffba81058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:50.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.929+0000 7ffbad600640 1 --2- 172.21.15.5:0/431417091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffba8105450 0x7ffba81058d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ffb94009900 tx=0x7ffb9402a3f0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.929+0000 7ffbad600640 1 -- 172.21.15.5:0/431417091 shutdown_connections 2024-04-24T06:43:50.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.929+0000 7ffbad600640 1 --2- 172.21.15.5:0/431417091 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffba8105e10 0x7ffba810c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.929+0000 7ffbad600640 1 --2- 172.21.15.5:0/431417091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffba8105450 0x7ffba81058d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.929+0000 7ffbad600640 1 --2- 172.21.15.5:0/431417091 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffba8104260 0x7ffba8104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.929+0000 7ffbad600640 1 -- 172.21.15.5:0/431417091 >> 172.21.15.5:0/431417091 conn(0x7ffba80ff7f0 msgr2=0x7ffba8101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:50.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.929+0000 7ffbad600640 1 -- 172.21.15.5:0/431417091 shutdown_connections 2024-04-24T06:43:50.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.929+0000 7ffbad600640 1 -- 172.21.15.5:0/431417091 wait complete. 2024-04-24T06:43:50.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.930+0000 7ffbad600640 1 Processor -- start 2024-04-24T06:43:50.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.930+0000 7ffbad600640 1 -- start start 2024-04-24T06:43:50.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.931+0000 7ffbad600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffba8104260 0x7ffba819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.931+0000 7ffbad600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffba8105450 0x7ffba819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.931+0000 7ffbad600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffba8105e10 0x7ffba81a1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.931+0000 7ffbad600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffba810ff40 con 0x7ffba8105450 2024-04-24T06:43:50.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.931+0000 7ffbad600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffba810fdc0 con 0x7ffba8104260 2024-04-24T06:43:50.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.931+0000 7ffbad600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ffba81100c0 con 0x7ffba8105e10 2024-04-24T06:43:50.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.931+0000 7ffba6000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffba8105450 0x7ffba819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.931+0000 7ffba6000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffba8105450 0x7ffba819ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39724/0 (socket says 172.21.15.5:39724) 2024-04-24T06:43:50.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.931+0000 7ffba6a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffba8104260 0x7ffba819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.931+0000 7ffba6000640 1 -- 172.21.15.5:0/2625577476 learned_addr learned my addr 172.21.15.5:0/2625577476 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:50.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.931+0000 7ffba7400640 1 --2- 172.21.15.5:0/2625577476 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffba8105e10 0x7ffba81a1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.932+0000 7ffba6a00640 1 -- 172.21.15.5:0/2625577476 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffba8105e10 msgr2=0x7ffba81a1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:50.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.932+0000 7ffba6a00640 1 --2- 172.21.15.5:0/2625577476 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffba8105e10 0x7ffba81a1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.932+0000 7ffba6a00640 1 -- 172.21.15.5:0/2625577476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffba8105450 msgr2=0x7ffba819ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:50.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.932+0000 7ffba6a00640 1 --2- 172.21.15.5:0/2625577476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffba8105450 0x7ffba819ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.932+0000 7ffba6a00640 1 -- 172.21.15.5:0/2625577476 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba81a21f0 con 0x7ffba8104260 2024-04-24T06:43:50.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.932+0000 7ffba6000640 1 --2- 172.21.15.5:0/2625577476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffba8105450 0x7ffba819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:43:50.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.932+0000 7ffba6a00640 1 --2- 172.21.15.5:0/2625577476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffba8104260 0x7ffba819a5d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ffb9800e9c0 tx=0x7ffb9800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:50.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.932+0000 7ffb9f400640 1 -- 172.21.15.5:0/2625577476 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9800cd80 con 0x7ffba8104260 2024-04-24T06:43:50.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.932+0000 7ffb9f400640 1 -- 172.21.15.5:0/2625577476 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb98010910 con 0x7ffba8104260 2024-04-24T06:43:50.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.932+0000 7ffbad600640 1 -- 172.21.15.5:0/2625577476 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffba81a24e0 con 0x7ffba8104260 2024-04-24T06:43:50.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.932+0000 7ffb9f400640 1 -- 172.21.15.5:0/2625577476 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb98018980 con 0x7ffba8104260 2024-04-24T06:43:50.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.932+0000 7ffbad600640 1 -- 172.21.15.5:0/2625577476 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffba81a2a30 con 0x7ffba8104260 2024-04-24T06:43:50.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.936+0000 7ffbb0373740 1 Processor -- start 2024-04-24T06:43:50.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.937+0000 7ffbb0373740 1 -- start start 2024-04-24T06:43:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.937+0000 7ffbb0373740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555944352090 0x5559443524b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.937+0000 7ffbb0373740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555944352a80 0x555944357580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.937+0000 7ffbb0373740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555944357bf0 0x55594435a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.937+0000 7ffbb0373740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5559442b7b10 con 0x555944352a80 2024-04-24T06:43:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.937+0000 7ffbb0373740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55594425e2b0 con 0x555944352090 2024-04-24T06:43:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.937+0000 7ffbb0373740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5559442b3ad0 con 0x555944357bf0 2024-04-24T06:43:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.937+0000 7ffb9f400640 1 -- 172.21.15.5:0/2625577476 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7ffb9801f020 con 0x7ffba8104260 2024-04-24T06:43:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.938+0000 7ffba6000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555944357bf0 0x55594435a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.938+0000 7ffba7400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555944352090 0x5559443524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.938+0000 7ffba6000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555944357bf0 0x55594435a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:54474/0 (socket says 172.21.15.5:54474) 2024-04-24T06:43:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.938+0000 7ffba6000640 1 -- 172.21.15.5:0/1607967038 learned_addr learned my addr 172.21.15.5:0/1607967038 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:50.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.938+0000 7ffba6a00640 1 --2- 172.21.15.5:0/1607967038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555944352a80 0x555944357580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.938+0000 7ffba6000640 1 -- 172.21.15.5:0/1607967038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555944352090 msgr2=0x5559443524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:50.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.938+0000 7ffba6000640 1 --2- 172.21.15.5:0/1607967038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555944352090 0x5559443524b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.938+0000 7ffba6000640 1 -- 172.21.15.5:0/1607967038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555944352a80 msgr2=0x555944357580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:50.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.938+0000 7ffba6000640 1 --2- 172.21.15.5:0/1607967038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555944352a80 0x555944357580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.938+0000 7ffba6000640 1 -- 172.21.15.5:0/1607967038 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555943fca2d0 con 0x555944357bf0 2024-04-24T06:43:50.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.938+0000 7ffba6000640 1 --2- 172.21.15.5:0/1607967038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555944357bf0 0x55594435a080 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ffb94006010 tx=0x7ffb94005e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=368c9ccd38e33728 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:50.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.939+0000 7ffb93400640 1 -- 172.21.15.5:0/1607967038 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb94035040 con 0x555944357bf0 2024-04-24T06:43:50.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.939+0000 7ffb93400640 1 -- 172.21.15.5:0/1607967038 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb94004760 con 0x555944357bf0 2024-04-24T06:43:50.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.939+0000 7ffb93400640 1 -- 172.21.15.5:0/1607967038 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb94031a10 con 0x555944357bf0 2024-04-24T06:43:50.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.939+0000 7ffb9f400640 1 --2- 172.21.15.5:0/2625577476 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ffb8c0a3bb0 0x7ffb8c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.939+0000 7ffb9f400640 1 -- 172.21.15.5:0/2625577476 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7ffb980d03a0 con 0x7ffba8104260 2024-04-24T06:43:50.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.939+0000 7ffba7400640 1 --2- 172.21.15.5:0/2625577476 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ffb8c0a3bb0 0x7ffb8c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.940+0000 7ffbb0373740 1 -- 172.21.15.5:0/1607967038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555944357bf0 msgr2=0x55594435a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:50.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.940+0000 7ffbb0373740 1 --2- 172.21.15.5:0/1607967038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555944357bf0 0x55594435a080 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ffb94006010 tx=0x7ffb94005e60 comp rx=0 tx=0).stop 2024-04-24T06:43:50.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.940+0000 7ffbb0373740 1 -- 172.21.15.5:0/1607967038 shutdown_connections 2024-04-24T06:43:50.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.940+0000 7ffbb0373740 1 --2- 172.21.15.5:0/1607967038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555944357bf0 0x55594435a080 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.940+0000 7ffbb0373740 1 --2- 172.21.15.5:0/1607967038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555944352a80 0x555944357580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.940+0000 7ffbb0373740 1 --2- 172.21.15.5:0/1607967038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555944352090 0x5559443524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.940+0000 7ffbb0373740 1 -- 172.21.15.5:0/1607967038 >> 172.21.15.5:0/1607967038 conn(0x55594434d700 msgr2=0x55594434db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:50.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.940+0000 7ffbb0373740 1 -- 172.21.15.5:0/1607967038 shutdown_connections 2024-04-24T06:43:50.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.940+0000 7ffba7400640 1 --2- 172.21.15.5:0/2625577476 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ffb8c0a3bb0 0x7ffb8c0a6070 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x5559443532e0 tx=0x7ffba0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:50.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.940+0000 7ffbb0373740 1 -- 172.21.15.5:0/1607967038 wait complete. 2024-04-24T06:43:50.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.940+0000 7ffbb0373740 1 -- start start 2024-04-24T06:43:50.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.941+0000 7ffbb0373740 1 Processor -- start 2024-04-24T06:43:50.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.941+0000 7ffbb0373740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555944352090 0x55594435c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.941+0000 7ffbb0373740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555944352a80 0x55594435ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.941+0000 7ffbb0373740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555944357bf0 0x555944363a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.941+0000 7ffbb0373740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55594435ab40 con 0x555944352a80 2024-04-24T06:43:50.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.941+0000 7ffbb0373740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555944271590 con 0x555944357bf0 2024-04-24T06:43:50.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.941+0000 7ffbb0373740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55594435acc0 con 0x555944352090 2024-04-24T06:43:50.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.941+0000 7ffbb0373740 10 client.0 _ll_register_callbacks cb 0x5559442543a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:43:50.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.941+0000 7ffba6000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555944352a80 0x55594435ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.941+0000 7ffba6000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555944352a80 0x55594435ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39744/0 (socket says 172.21.15.5:39744) 2024-04-24T06:43:50.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.941+0000 7ffba6000640 1 -- 172.21.15.5:0/1049013783 learned_addr learned my addr 172.21.15.5:0/1049013783 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:50.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.941+0000 7ffba7400640 1 --2- 172.21.15.5:0/1049013783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555944357bf0 0x555944363a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.942+0000 7ffba6a00640 1 --2- 172.21.15.5:0/1049013783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555944352090 0x55594435c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.942+0000 7ffba6000640 1 -- 172.21.15.5:0/1049013783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555944352090 msgr2=0x55594435c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.942+0000 7ffba6000640 1 --2- 172.21.15.5:0/1049013783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555944352090 0x55594435c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.942+0000 7ffba6000640 1 -- 172.21.15.5:0/1049013783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555944357bf0 msgr2=0x555944363a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.942+0000 7ffba6000640 1 --2- 172.21.15.5:0/1049013783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555944357bf0 0x555944363a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.942+0000 7ffba6000640 1 -- 172.21.15.5:0/1049013783 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559440946b0 con 0x555944352a80 2024-04-24T06:43:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.942+0000 7ffba6000640 1 --2- 172.21.15.5:0/1049013783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555944352a80 0x55594435ca20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ffb94000c00 tx=0x7ffb94004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9c76c8a00336dc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.942+0000 7ffb87e00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:43:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.942+0000 7ffb87e00640 1 -- 172.21.15.5:0/1049013783 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb94053060 con 0x555944352a80 2024-04-24T06:43:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.943+0000 7ffb87e00640 1 -- 172.21.15.5:0/1049013783 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7ffb94005160 con 0x555944352a80 2024-04-24T06:43:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.943+0000 7ffbb0373740 1 -- 172.21.15.5:0/1049013783 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb94004760 con 0x555944352a80 2024-04-24T06:43:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.943+0000 7ffb87e00640 1 -- 172.21.15.5:0/1049013783 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7ffb94035050 con 0x555944352a80 2024-04-24T06:43:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.943+0000 7ffbb0373740 10 client.15081 fetch_fsmap learned FSMap version 22 2024-04-24T06:43:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.943+0000 7ffbb0373740 1 -- 172.21.15.5:0/1049013783 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x555944366dd0 con 0x555944352a80 2024-04-24T06:43:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.943+0000 7ffb87e00640 1 -- 172.21.15.5:0/1049013783 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb94004760 con 0x555944352a80 2024-04-24T06:43:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.943+0000 7ffb87e00640 1 -- 172.21.15.5:0/1049013783 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7ffb94058070 con 0x555944352a80 2024-04-24T06:43:50.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.944+0000 7ffbb0373740 10 client.15081 fetch_fsmap finished waiting for FSMap version 22 2024-04-24T06:43:50.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.944+0000 7ffbb0373740 10 client.15081 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T06:43:50.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.944+0000 7ffbb0373740 20 client.15081 populate_metadata read hostname 'smithi005' 2024-04-24T06:43:50.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.944+0000 7ffbb0373740 1 --2- 172.21.15.5:0/1049013783 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x555944368fd0 0x55594436a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.944+0000 7ffbb0373740 4 client.15081 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:43:50.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.944+0000 7ffbb0373740 1 -- 172.21.15.5:0/1049013783 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559442c5530 con 0x555944368fd0 2024-04-24T06:43:50.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.944+0000 7ffba6a00640 1 --2- 172.21.15.5:0/1049013783 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x555944368fd0 0x55594436a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.944+0000 7ffba6a00640 1 --2- 172.21.15.5:0/1049013783 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x555944368fd0 0x55594436a440 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87656c5909c53327 server_cookie=eea48c45af5be383 in_seq=0 out_seq=0 2024-04-24T06:43:50.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.944+0000 7ffb87e00640 10 client.15081 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:43:50.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.945+0000 7ffb87e00640 1 -- 172.21.15.5:0/1049013783 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5559442c5530 con 0x555944368fd0 2024-04-24T06:43:50.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.945+0000 7ffb87e00640 10 client.15081 handle_command_reply: tid=0 2024-04-24T06:43:50.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.945+0000 7ffbb0373740 1 client.15081 shutdown 2024-04-24T06:43:50.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.945+0000 7ffbb0373740 1 -- 172.21.15.5:0/1049013783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555944352a80 msgr2=0x55594435ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:50.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.945+0000 7ffbb0373740 1 --2- 172.21.15.5:0/1049013783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555944352a80 0x55594435ca20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ffb94000c00 tx=0x7ffb94004b40 comp rx=0 tx=0).stop 2024-04-24T06:43:50.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.946+0000 7ffbb0373740 1 -- 172.21.15.5:0/1049013783 shutdown_connections 2024-04-24T06:43:50.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.946+0000 7ffbb0373740 1 --2- 172.21.15.5:0/1049013783 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x555944368fd0 0x55594436a440 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.946+0000 7ffbb0373740 1 --2- 172.21.15.5:0/1049013783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555944357bf0 0x555944363a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.946+0000 7ffbb0373740 1 --2- 172.21.15.5:0/1049013783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555944352a80 0x55594435ca20 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.946+0000 7ffbb0373740 1 --2- 172.21.15.5:0/1049013783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555944352090 0x55594435c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.946+0000 7ffb87e00640 0 client.15081 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:43:50.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.946+0000 7ffbb0373740 1 -- 172.21.15.5:0/1049013783 >> 172.21.15.5:0/1049013783 conn(0x55594434d700 msgr2=0x55594434c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:50.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.946+0000 7ffbb0373740 1 -- 172.21.15.5:0/1049013783 shutdown_connections 2024-04-24T06:43:50.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.946+0000 7ffbb0373740 1 -- 172.21.15.5:0/1049013783 wait complete. 2024-04-24T06:43:50.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.946+0000 7ffbb0373740 20 client.15081 trim_cache size 0 max 16384 2024-04-24T06:43:50.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.966+0000 7ffbb0373740 1 Processor -- start 2024-04-24T06:43:50.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.966+0000 7ffbb0373740 1 -- start start 2024-04-24T06:43:50.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.966+0000 7ffbb0373740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555944352090 0x555944368120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.966+0000 7ffbb0373740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555944352a80 0x555944363a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.966+0000 7ffbb0373740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555944357bf0 0x555944353520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.966+0000 7ffbb0373740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555944368780 con 0x555944352090 2024-04-24T06:43:50.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.966+0000 7ffbb0373740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555944368e50 con 0x555944357bf0 2024-04-24T06:43:50.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.966+0000 7ffbb0373740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555944369520 con 0x555944352a80 2024-04-24T06:43:50.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.966+0000 7ffba6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555944352090 0x555944368120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.966+0000 7ffba6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555944352090 0x555944368120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39758/0 (socket says 172.21.15.5:39758) 2024-04-24T06:43:50.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.966+0000 7ffba6a00640 1 -- 172.21.15.5:0/4106571057 learned_addr learned my addr 172.21.15.5:0/4106571057 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:50.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.966+0000 7ffba6000640 1 --2- 172.21.15.5:0/4106571057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555944352a80 0x555944363a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.967+0000 7ffba7400640 1 --2- 172.21.15.5:0/4106571057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555944357bf0 0x555944353520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.967+0000 7ffba6a00640 1 -- 172.21.15.5:0/4106571057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555944352a80 msgr2=0x555944363a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:50.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.967+0000 7ffba6a00640 1 --2- 172.21.15.5:0/4106571057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555944352a80 0x555944363a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.967+0000 7ffba6a00640 1 -- 172.21.15.5:0/4106571057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555944357bf0 msgr2=0x555944353520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:50.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.967+0000 7ffba6a00640 1 --2- 172.21.15.5:0/4106571057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555944357bf0 0x555944353520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.967+0000 7ffba6a00640 1 -- 172.21.15.5:0/4106571057 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559443699d0 con 0x555944352090 2024-04-24T06:43:50.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.967+0000 7ffba6a00640 1 --2- 172.21.15.5:0/4106571057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555944352090 0x555944368120 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ffb9800eff0 tx=0x7ffb98087040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab53dd20706849d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:50.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.967+0000 7ffb92a00640 1 -- 172.21.15.5:0/4106571057 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9809c450 con 0x555944352090 2024-04-24T06:43:50.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.967+0000 7ffb92a00640 1 -- 172.21.15.5:0/4106571057 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb980870b0 con 0x555944352090 2024-04-24T06:43:50.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.967+0000 7ffb92a00640 1 -- 172.21.15.5:0/4106571057 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9809bfe0 con 0x555944352090 2024-04-24T06:43:50.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.968+0000 7ffbb0373740 1 -- 172.21.15.5:0/4106571057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555944352090 msgr2=0x555944368120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:50.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.968+0000 7ffbb0373740 1 --2- 172.21.15.5:0/4106571057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555944352090 0x555944368120 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ffb9800eff0 tx=0x7ffb98087040 comp rx=0 tx=0).stop 2024-04-24T06:43:50.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.968+0000 7ffbb0373740 1 -- 172.21.15.5:0/4106571057 shutdown_connections 2024-04-24T06:43:50.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.968+0000 7ffbb0373740 1 --2- 172.21.15.5:0/4106571057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555944357bf0 0x555944353520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.968+0000 7ffbb0373740 1 --2- 172.21.15.5:0/4106571057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555944352a80 0x555944363a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.968+0000 7ffbb0373740 1 --2- 172.21.15.5:0/4106571057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555944352090 0x555944368120 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.968+0000 7ffbb0373740 1 -- 172.21.15.5:0/4106571057 >> 172.21.15.5:0/4106571057 conn(0x55594434d700 msgr2=0x55594434dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:50.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.968+0000 7ffbb0373740 1 -- 172.21.15.5:0/4106571057 shutdown_connections 2024-04-24T06:43:50.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.968+0000 7ffbb0373740 1 -- 172.21.15.5:0/4106571057 wait complete. 2024-04-24T06:43:50.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.968+0000 7ffbb0373740 1 -- start start 2024-04-24T06:43:50.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.969+0000 7ffbb0373740 1 Processor -- start 2024-04-24T06:43:50.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.969+0000 7ffbb0373740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555944352090 0x55594435f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.969+0000 7ffbb0373740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555944352a80 0x55594435f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.969+0000 7ffbb0373740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555944357bf0 0x555944362680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.969+0000 7ffbb0373740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55594436a2c0 con 0x555944352a80 2024-04-24T06:43:50.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.969+0000 7ffbb0373740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555944369bf0 con 0x555944357bf0 2024-04-24T06:43:50.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.969+0000 7ffbb0373740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55594436a990 con 0x555944352090 2024-04-24T06:43:50.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.969+0000 7ffbb0373740 10 client.0 _ll_register_callbacks cb 0x7ffb94009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:43:50.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.969+0000 7ffba6000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555944352a80 0x55594435f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.969+0000 7ffba6000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555944352a80 0x55594435f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39760/0 (socket says 172.21.15.5:39760) 2024-04-24T06:43:50.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.969+0000 7ffba6000640 1 -- 172.21.15.5:0/2755550386 learned_addr learned my addr 172.21.15.5:0/2755550386 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:50.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.969+0000 7ffba7400640 1 --2- 172.21.15.5:0/2755550386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555944357bf0 0x555944362680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.969+0000 7ffba6a00640 1 --2- 172.21.15.5:0/2755550386 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555944352090 0x55594435f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.970+0000 7ffba6000640 1 -- 172.21.15.5:0/2755550386 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555944352090 msgr2=0x55594435f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:50.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.970+0000 7ffba6000640 1 --2- 172.21.15.5:0/2755550386 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555944352090 0x55594435f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.970+0000 7ffba6000640 1 -- 172.21.15.5:0/2755550386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555944357bf0 msgr2=0x555944362680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:43:50.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.970+0000 7ffba6000640 1 --2- 172.21.15.5:0/2755550386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555944357bf0 0x555944362680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.970+0000 7ffba6000640 1 -- 172.21.15.5:0/2755550386 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555944362c70 con 0x555944352a80 2024-04-24T06:43:50.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.970+0000 7ffba6000640 1 --2- 172.21.15.5:0/2755550386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555944352a80 0x55594435f6b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x555944353310 tx=0x7ffb94005390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdac2655eb64ed33 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:50.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.970+0000 7ffb87e00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:43:50.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.970+0000 7ffb87e00640 1 -- 172.21.15.5:0/2755550386 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9405e070 con 0x555944352a80 2024-04-24T06:43:50.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.970+0000 7ffb87e00640 1 -- 172.21.15.5:0/2755550386 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7ffb940038b0 con 0x555944352a80 2024-04-24T06:43:50.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.970+0000 7ffbb0373740 1 -- 172.21.15.5:0/2755550386 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5559442bf3a0 con 0x555944352a80 2024-04-24T06:43:50.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.971+0000 7ffb87e00640 1 -- 172.21.15.5:0/2755550386 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7ffb94004760 con 0x555944352a80 2024-04-24T06:43:50.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.971+0000 7ffbb0373740 10 client.15087 fetch_fsmap learned FSMap version 22 2024-04-24T06:43:50.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.971+0000 7ffbb0373740 1 -- 172.21.15.5:0/2755550386 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5559443cef80 con 0x555944352a80 2024-04-24T06:43:50.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.971+0000 7ffb87e00640 1 -- 172.21.15.5:0/2755550386 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5559442bf3a0 con 0x555944352a80 2024-04-24T06:43:50.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.971+0000 7ffb87e00640 1 -- 172.21.15.5:0/2755550386 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7ffb94061020 con 0x555944352a80 2024-04-24T06:43:50.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.971+0000 7ffbb0373740 10 client.15087 fetch_fsmap finished waiting for FSMap version 22 2024-04-24T06:43:50.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.971+0000 7ffbb0373740 10 client.15087 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T06:43:50.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.971+0000 7ffbb0373740 20 client.15087 populate_metadata read hostname 'smithi005' 2024-04-24T06:43:50.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.971+0000 7ffbb0373740 1 --2- 172.21.15.5:0/2755550386 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5559443d3a40 0x5559443d2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:50.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.971+0000 7ffbb0373740 4 client.15087 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T06:43:50.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.971+0000 7ffbb0373740 1 -- 172.21.15.5:0/2755550386 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "status"}) -- 0x55594423b860 con 0x5559443d3a40 2024-04-24T06:43:50.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.971+0000 7ffba6a00640 1 --2- 172.21.15.5:0/2755550386 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5559443d3a40 0x5559443d2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:50.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.972+0000 7ffba6a00640 1 --2- 172.21.15.5:0/2755550386 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5559443d3a40 0x5559443d2750 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56345ea63793eea7 server_cookie=35ac2a4d9d6a9d41 in_seq=0 out_seq=0 2024-04-24T06:43:50.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.972+0000 7ffb87e00640 10 client.15087 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:43:50.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.972+0000 7ffb87e00640 1 -- 172.21.15.5:0/2755550386 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55594423b860 con 0x5559443d3a40 2024-04-24T06:43:50.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.972+0000 7ffb87e00640 10 client.15087 handle_command_reply: tid=0 2024-04-24T06:43:50.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.972+0000 7ffbb0373740 1 client.15087 shutdown 2024-04-24T06:43:50.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.973+0000 7ffbb0373740 1 -- 172.21.15.5:0/2755550386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555944352a80 msgr2=0x55594435f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:50.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.973+0000 7ffbb0373740 1 --2- 172.21.15.5:0/2755550386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555944352a80 0x55594435f6b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x555944353310 tx=0x7ffb94005390 comp rx=0 tx=0).stop 2024-04-24T06:43:50.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.973+0000 7ffbb0373740 1 -- 172.21.15.5:0/2755550386 shutdown_connections 2024-04-24T06:43:50.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.973+0000 7ffbb0373740 1 --2- 172.21.15.5:0/2755550386 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5559443d3a40 0x5559443d2750 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.973+0000 7ffbb0373740 1 --2- 172.21.15.5:0/2755550386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555944357bf0 0x555944362680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.973+0000 7ffbb0373740 1 --2- 172.21.15.5:0/2755550386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555944352a80 0x55594435f6b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.973+0000 7ffbb0373740 1 --2- 172.21.15.5:0/2755550386 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555944352090 0x55594435f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.973+0000 7ffb87e00640 0 client.15087 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:43:50.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.973+0000 7ffbb0373740 1 -- 172.21.15.5:0/2755550386 >> 172.21.15.5:0/2755550386 conn(0x55594434d700 msgr2=0x55594434dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:50.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.973+0000 7ffbb0373740 1 -- 172.21.15.5:0/2755550386 shutdown_connections 2024-04-24T06:43:50.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.973+0000 7ffbb0373740 1 -- 172.21.15.5:0/2755550386 wait complete. 2024-04-24T06:43:50.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.973+0000 7ffbb0373740 20 client.15087 trim_cache size 0 max 16384 2024-04-24T06:43:50.978 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:43:50.979 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "43e9bad0-0204-11ef-bc93-c7b262605968", 2024-04-24T06:43:50.979 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-04-24T06:43:50.979 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24569, 2024-04-24T06:43:50.979 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-04-24T06:43:50.979 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-04-24T06:43:50.979 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-04-24T06:43:50.979 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 135.53566650499999, 2024-04-24T06:43:50.979 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 22, 2024-04-24T06:43:50.979 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 88, 2024-04-24T06:43:50.979 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 86, 2024-04-24T06:43:50.979 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 143.22176243000001 2024-04-24T06:43:50.979 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:43:50.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.974+0000 7ffbad600640 1 -- 172.21.15.5:0/2625577476 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ffb8c0a3bb0 msgr2=0x7ffb8c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:50.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.974+0000 7ffbad600640 1 --2- 172.21.15.5:0/2625577476 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ffb8c0a3bb0 0x7ffb8c0a6070 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x5559443532e0 tx=0x7ffba0009500 comp rx=0 tx=0).stop 2024-04-24T06:43:50.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.974+0000 7ffbad600640 1 -- 172.21.15.5:0/2625577476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffba8104260 msgr2=0x7ffba819a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:50.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.974+0000 7ffbad600640 1 --2- 172.21.15.5:0/2625577476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffba8104260 0x7ffba819a5d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ffb9800e9c0 tx=0x7ffb9800ee80 comp rx=0 tx=0).stop 2024-04-24T06:43:50.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.974+0000 7ffbad600640 1 -- 172.21.15.5:0/2625577476 shutdown_connections 2024-04-24T06:43:50.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.974+0000 7ffbad600640 1 --2- 172.21.15.5:0/2625577476 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7ffb8c0a3bb0 0x7ffb8c0a6070 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.974+0000 7ffbad600640 1 --2- 172.21.15.5:0/2625577476 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffba8105e10 0x7ffba81a1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.974+0000 7ffbad600640 1 --2- 172.21.15.5:0/2625577476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffba8105450 0x7ffba819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.974+0000 7ffbad600640 1 --2- 172.21.15.5:0/2625577476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffba8104260 0x7ffba819a5d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:50.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.974+0000 7ffbad600640 1 -- 172.21.15.5:0/2625577476 >> 172.21.15.5:0/2625577476 conn(0x7ffba80ff7f0 msgr2=0x7ffba81086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:50.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.974+0000 7ffbad600640 1 -- 172.21.15.5:0/2625577476 shutdown_connections 2024-04-24T06:43:50.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:50.974+0000 7ffbad600640 1 -- 172.21.15.5:0/2625577476 wait complete. 2024-04-24T06:43:50.994 INFO:teuthology.run_tasks:Running task sequential... 2024-04-24T06:43:51.007 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-24T06:43:51.007 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-24T06:43:51.007 INFO:teuthology.task.print:Enabling mgr modules 2024-04-24T06:43:51.008 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-24T06:43:51.008 INFO:teuthology.task.exec:Executing custom commands... 2024-04-24T06:43:51.009 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-04-24T06:43:51.009 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-24T06:43:51.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:51.213+0000 7f43d3200640 1 -- 172.21.15.5:0/4169504086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43d4101ea0 msgr2=0x7f43d41022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:51.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:51.213+0000 7f43d3200640 1 --2- 172.21.15.5:0/4169504086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43d4101ea0 0x7f43d41022c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f43c4009900 tx=0x7f43c402a3f0 comp rx=0 tx=0).stop 2024-04-24T06:43:51.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:51.213+0000 7f43d3200640 1 -- 172.21.15.5:0/4169504086 shutdown_connections 2024-04-24T06:43:51.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:51.213+0000 7f43d3200640 1 --2- 172.21.15.5:0/4169504086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43d41039b0 0x7f43d410a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:51.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:51.213+0000 7f43d3200640 1 --2- 172.21.15.5:0/4169504086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43d4102ff0 0x7f43d4103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:51.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:51.213+0000 7f43d3200640 1 --2- 172.21.15.5:0/4169504086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43d4101ea0 0x7f43d41022c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:51.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:51.213+0000 7f43d3200640 1 -- 172.21.15.5:0/4169504086 >> 172.21.15.5:0/4169504086 conn(0x7f43d40fd410 msgr2=0x7f43d40ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:51.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:51.213+0000 7f43d3200640 1 -- 172.21.15.5:0/4169504086 shutdown_connections 2024-04-24T06:43:51.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:51.213+0000 7f43d3200640 1 -- 172.21.15.5:0/4169504086 wait complete. 2024-04-24T06:43:51.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:51.214+0000 7f43d3200640 1 Processor -- start 2024-04-24T06:43:51.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:51.215+0000 7f43d3200640 1 -- start start 2024-04-24T06:43:51.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:51.215+0000 7f43d3200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43d4101ea0 0x7f43d419a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:51.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:51.215+0000 7f43d3200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43d4102ff0 0x7f43d419a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:51.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:51.215+0000 7f43d3200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43d41039b0 0x7f43d41a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:51.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:51.215+0000 7f43d3200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f43d410db70 con 0x7f43d41039b0 2024-04-24T06:43:51.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:51.215+0000 7f43d3200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f43d410d9f0 con 0x7f43d4102ff0 2024-04-24T06:43:51.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:51.215+0000 7f43d3200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f43d410dcf0 con 0x7f43d4101ea0 2024-04-24T06:43:51.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:51.215+0000 7f43d1e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43d4101ea0 0x7f43d419a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:51.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:51.215+0000 7f43c9400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43d4102ff0 0x7f43d419a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:51.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:51.215+0000 7f43d2800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43d41039b0 0x7f43d41a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:51.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:51.215+0000 7f43d1e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43d4101ea0 0x7f43d419a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:54494/0 (socket says 172.21.15.5:54494) 2024-04-24T06:43:51.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:51.215+0000 7f43d1e00640 1 -- 172.21.15.5:0/61679100 learned_addr learned my addr 172.21.15.5:0/61679100 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:51.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:51.216+0000 7f43d1e00640 1 -- 172.21.15.5:0/61679100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43d4102ff0 msgr2=0x7f43d419a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:51.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:51.216+0000 7f43d1e00640 1 --2- 172.21.15.5:0/61679100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43d4102ff0 0x7f43d419a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:51.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:51.216+0000 7f43d1e00640 1 -- 172.21.15.5:0/61679100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43d41039b0 msgr2=0x7f43d41a1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:51.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:51.216+0000 7f43d1e00640 1 --2- 172.21.15.5:0/61679100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43d41039b0 0x7f43d41a1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:51.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:51.216+0000 7f43d1e00640 1 -- 172.21.15.5:0/61679100 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d41a1ff0 con 0x7f43d4101ea0 2024-04-24T06:43:51.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:51.216+0000 7f43d1e00640 1 --2- 172.21.15.5:0/61679100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43d4101ea0 0x7f43d419a3d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f43c4031440 tx=0x7f43c4032e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:51.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:51.216+0000 7f43cb400640 1 -- 172.21.15.5:0/61679100 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c403f030 con 0x7f43d4101ea0 2024-04-24T06:43:51.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:51.216+0000 7f43cb400640 1 -- 172.21.15.5:0/61679100 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43c4003ce0 con 0x7f43d4101ea0 2024-04-24T06:43:51.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:51.216+0000 7f43cb400640 1 -- 172.21.15.5:0/61679100 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c4035070 con 0x7f43d4101ea0 2024-04-24T06:43:51.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:51.216+0000 7f43d3200640 1 -- 172.21.15.5:0/61679100 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43d41a2280 con 0x7f43d4101ea0 2024-04-24T06:43:51.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:51.217+0000 7f43d3200640 1 -- 172.21.15.5:0/61679100 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43d41a27c0 con 0x7f43d4101ea0 2024-04-24T06:43:51.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:51.218+0000 7f43d3200640 1 -- 172.21.15.5:0/61679100 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43d41a2a20 con 0x7f43d4101ea0 2024-04-24T06:43:51.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:51.220+0000 7f43cb400640 1 -- 172.21.15.5:0/61679100 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f43c4042030 con 0x7f43d4101ea0 2024-04-24T06:43:51.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:51.221+0000 7f43cb400640 1 --2- 172.21.15.5:0/61679100 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f43b80a3c10 0x7f43b80a60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:51.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:51.222+0000 7f43cb400640 1 -- 172.21.15.5:0/61679100 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f43c40f0d70 con 0x7f43d4101ea0 2024-04-24T06:43:51.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:51.222+0000 7f43c9400640 1 --2- 172.21.15.5:0/61679100 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f43b80a3c10 0x7f43b80a60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:51.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:51.222+0000 7f43c9400640 1 --2- 172.21.15.5:0/61679100 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f43b80a3c10 0x7f43b80a60d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f43d419b8e0 tx=0x7f43bc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:51.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:51.224+0000 7f43cb400640 1 -- 172.21.15.5:0/61679100 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f43c40bd0c0 con 0x7f43d4101ea0 2024-04-24T06:43:51.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:51.494+0000 7f43d3200640 1 -- 172.21.15.5:0/61679100 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f43d41a2bf0 con 0x7f43d4101ea0 2024-04-24T06:43:52.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.439+0000 7f43cb400640 1 -- 172.21.15.5:0/61679100 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mgrmap(e 19) ==== 138662+0+0 (secure 0 0 0) 0x7f43c4003800 con 0x7f43d4101ea0 2024-04-24T06:43:52.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.442+0000 7f43cb400640 1 -- 172.21.15.5:0/61679100 <== mon.1 v2:172.21.15.137:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f43c40f2820 con 0x7f43d4101ea0 2024-04-24T06:43:52.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.446+0000 7f43d3200640 1 -- 172.21.15.5:0/61679100 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f43b80a3c10 msgr2=0x7f43b80a60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:52.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.446+0000 7f43d3200640 1 --2- 172.21.15.5:0/61679100 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f43b80a3c10 0x7f43b80a60d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f43d419b8e0 tx=0x7f43bc00b390 comp rx=0 tx=0).stop 2024-04-24T06:43:52.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.446+0000 7f43d3200640 1 -- 172.21.15.5:0/61679100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43d4101ea0 msgr2=0x7f43d419a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:52.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.446+0000 7f43d3200640 1 --2- 172.21.15.5:0/61679100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43d4101ea0 0x7f43d419a3d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f43c4031440 tx=0x7f43c4032e20 comp rx=0 tx=0).stop 2024-04-24T06:43:52.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.446+0000 7f43d3200640 1 -- 172.21.15.5:0/61679100 shutdown_connections 2024-04-24T06:43:52.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.446+0000 7f43d3200640 1 --2- 172.21.15.5:0/61679100 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f43b80a3c10 0x7f43b80a60d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:52.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.446+0000 7f43d3200640 1 --2- 172.21.15.5:0/61679100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43d41039b0 0x7f43d41a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:52.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.446+0000 7f43d3200640 1 --2- 172.21.15.5:0/61679100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43d4102ff0 0x7f43d419a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:52.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.446+0000 7f43d3200640 1 --2- 172.21.15.5:0/61679100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43d4101ea0 0x7f43d419a3d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:52.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.446+0000 7f43d3200640 1 -- 172.21.15.5:0/61679100 >> 172.21.15.5:0/61679100 conn(0x7f43d40fd410 msgr2=0x7f43d4074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:52.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.446+0000 7f43d3200640 1 -- 172.21.15.5:0/61679100 shutdown_connections 2024-04-24T06:43:52.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.446+0000 7f43d3200640 1 -- 172.21.15.5:0/61679100 wait complete. 2024-04-24T06:43:52.469 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-24T06:43:52.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.652+0000 7f8897e00640 1 -- 172.21.15.5:0/2082611052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8890102b20 msgr2=0x7f8890104f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:52.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.652+0000 7f8897e00640 1 --2- 172.21.15.5:0/2082611052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8890102b20 0x7f8890104f10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8880009ca0 tx=0x7f888002a8c0 comp rx=0 tx=0).stop 2024-04-24T06:43:52.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.652+0000 7f8897e00640 1 -- 172.21.15.5:0/2082611052 shutdown_connections 2024-04-24T06:43:52.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.652+0000 7f8897e00640 1 --2- 172.21.15.5:0/2082611052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8890107d80 0x7f889010a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:52.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.652+0000 7f8897e00640 1 --2- 172.21.15.5:0/2082611052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8890105450 0x7f8890107840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:52.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.652+0000 7f8897e00640 1 --2- 172.21.15.5:0/2082611052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8890102b20 0x7f8890104f10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:52.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.652+0000 7f8897e00640 1 -- 172.21.15.5:0/2082611052 >> 172.21.15.5:0/2082611052 conn(0x7f88900fc510 msgr2=0x7f88900fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:52.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.653+0000 7f8897e00640 1 -- 172.21.15.5:0/2082611052 shutdown_connections 2024-04-24T06:43:52.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.653+0000 7f8897e00640 1 -- 172.21.15.5:0/2082611052 wait complete. 2024-04-24T06:43:52.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.654+0000 7f8897e00640 1 Processor -- start 2024-04-24T06:43:52.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.654+0000 7f8897e00640 1 -- start start 2024-04-24T06:43:52.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.654+0000 7f8897e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8890102b20 0x7f889019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:52.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.654+0000 7f8897e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8890105450 0x7f889019ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:52.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.654+0000 7f8897e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8890107d80 0x7f88901a1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:52.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.654+0000 7f8897e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f889010dd30 con 0x7f8890105450 2024-04-24T06:43:52.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.654+0000 7f8897e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f889010dbb0 con 0x7f8890102b20 2024-04-24T06:43:52.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.654+0000 7f8897e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f889010deb0 con 0x7f8890107d80 2024-04-24T06:43:52.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.654+0000 7f8894a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8890105450 0x7f889019ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:52.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.654+0000 7f8895400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8890102b20 0x7f889019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:52.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.655+0000 7f8895400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8890102b20 0x7f889019a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:33100/0 (socket says 172.21.15.5:33100) 2024-04-24T06:43:52.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.655+0000 7f8895400640 1 -- 172.21.15.5:0/3525606705 learned_addr learned my addr 172.21.15.5:0/3525606705 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:52.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.655+0000 7f8894a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8890105450 0x7f889019ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39810/0 (socket says 172.21.15.5:39810) 2024-04-24T06:43:52.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.655+0000 7f8895e00640 1 --2- 172.21.15.5:0/3525606705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8890107d80 0x7f88901a1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:52.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.655+0000 7f8895400640 1 -- 172.21.15.5:0/3525606705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8890107d80 msgr2=0x7f88901a1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.655+0000 7f8895400640 1 --2- 172.21.15.5:0/3525606705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8890107d80 0x7f88901a1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.655+0000 7f8895400640 1 -- 172.21.15.5:0/3525606705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8890105450 msgr2=0x7f889019ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.655+0000 7f8895400640 1 --2- 172.21.15.5:0/3525606705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8890105450 0x7f889019ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.655+0000 7f8895400640 1 -- 172.21.15.5:0/3525606705 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88901a21e0 con 0x7f8890102b20 2024-04-24T06:43:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.655+0000 7f8895400640 1 --2- 172.21.15.5:0/3525606705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8890102b20 0x7f889019a5c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8880000f80 tx=0x7f8880031ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.655+0000 7f888e000640 1 -- 172.21.15.5:0/3525606705 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888002fa20 con 0x7f8890102b20 2024-04-24T06:43:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.656+0000 7f888e000640 1 -- 172.21.15.5:0/3525606705 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f888002fbc0 con 0x7f8890102b20 2024-04-24T06:43:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.656+0000 7f8897e00640 1 -- 172.21.15.5:0/3525606705 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88901a2470 con 0x7f8890102b20 2024-04-24T06:43:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.656+0000 7f888e000640 1 -- 172.21.15.5:0/3525606705 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888003a6f0 con 0x7f8890102b20 2024-04-24T06:43:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.656+0000 7f8897e00640 1 -- 172.21.15.5:0/3525606705 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88901a2a10 con 0x7f8890102b20 2024-04-24T06:43:52.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.657+0000 7f8897e00640 1 -- 172.21.15.5:0/3525606705 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88901098d0 con 0x7f8890102b20 2024-04-24T06:43:52.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.659+0000 7f888e000640 1 -- 172.21.15.5:0/3525606705 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 138662+0+0 (secure 0 0 0) 0x7f888003a890 con 0x7f8890102b20 2024-04-24T06:43:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.660+0000 7f888e000640 1 --2- 172.21.15.5:0/3525606705 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f88740a3c70 0x7f88740a6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.660+0000 7f8894a00640 1 -- 172.21.15.5:0/3525606705 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f88740a3c70 msgr2=0x7f88740a6130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/4042936965 2024-04-24T06:43:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.660+0000 7f888e000640 1 -- 172.21.15.5:0/3525606705 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f88800f18e0 con 0x7f8890102b20 2024-04-24T06:43:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.660+0000 7f8894a00640 1 --2- 172.21.15.5:0/3525606705 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f88740a3c70 0x7f88740a6130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T06:43:52.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.664+0000 7f888e000640 1 -- 172.21.15.5:0/3525606705 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f88800be310 con 0x7f8890102b20 2024-04-24T06:43:52.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:52 smithi005 ceph-mon[27798]: pgmap v383: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:43:52.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:52 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/61679100' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-24T06:43:52.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:52 smithi005 ceph-mon[27798]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-24T06:43:52.760 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:43:52 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: ignoring --setuser ceph since I am not root 2024-04-24T06:43:52.760 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:43:52 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: ignoring --setgroup ceph since I am not root 2024-04-24T06:43:52.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:52 smithi037 ceph-mon[32864]: pgmap v383: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:43:52.771 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:52 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/61679100' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-24T06:43:52.771 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:52 smithi037 ceph-mon[32864]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-24T06:43:52.773 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:43:52 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: ignoring --setuser ceph since I am not root 2024-04-24T06:43:52.773 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:43:52 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: ignoring --setgroup ceph since I am not root 2024-04-24T06:43:52.774 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:43:52 smithi037 ceph-mgr[34263]: -- 172.21.15.37:0/2912238616 <== mon.2 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5566693a65a0 con 0x55666937a800 2024-04-24T06:43:52.774 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:43:52 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:43:52.613+0000 7f9ab7685200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T06:43:52.793 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:52 smithi137 ceph-mon[33017]: pgmap v383: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:43:52.793 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:52 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/61679100' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-24T06:43:52.793 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:52 smithi137 ceph-mon[33017]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-24T06:43:52.794 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:43:52 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: ignoring --setuser ceph since I am not root 2024-04-24T06:43:52.794 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:43:52 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: ignoring --setgroup ceph since I am not root 2024-04-24T06:43:52.794 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:43:52 smithi137 ceph-mgr[34889]: -- 172.21.15.137:0/3347056655 <== mon.1 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c08b23a5a0 con 0x55c08b20ec00 2024-04-24T06:43:52.794 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:43:52 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:43:52.608+0000 7f233d292200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T06:43:52.794 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:43:52 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:43:52.792+0000 7f233d292200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T06:43:52.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.861+0000 7f8894a00640 1 -- 172.21.15.5:0/3525606705 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f88740a3c70 msgr2=0x7f88740a6130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/4042936965 2024-04-24T06:43:52.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.861+0000 7f8894a00640 1 --2- 172.21.15.5:0/3525606705 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f88740a3c70 0x7f88740a6130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T06:43:52.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.874+0000 7f8897e00640 1 -- 172.21.15.5:0/3525606705 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f8890062c00 con 0x7f8890102b20 2024-04-24T06:43:52.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.883+0000 7f888e000640 1 -- 172.21.15.5:0/3525606705 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v28)=0 v28) ==== 138+0+0 (secure 0 0 0) 0x7f88800bdc10 con 0x7f8890102b20 2024-04-24T06:43:52.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.885+0000 7f8897e00640 1 -- 172.21.15.5:0/3525606705 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f88740a3c70 msgr2=0x7f88740a6130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T06:43:52.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.885+0000 7f8897e00640 1 --2- 172.21.15.5:0/3525606705 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f88740a3c70 0x7f88740a6130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:52.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.885+0000 7f8897e00640 1 -- 172.21.15.5:0/3525606705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8890102b20 msgr2=0x7f889019a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:52.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.885+0000 7f8897e00640 1 --2- 172.21.15.5:0/3525606705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8890102b20 0x7f889019a5c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8880000f80 tx=0x7f8880031ee0 comp rx=0 tx=0).stop 2024-04-24T06:43:52.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.885+0000 7f8897e00640 1 -- 172.21.15.5:0/3525606705 shutdown_connections 2024-04-24T06:43:52.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.885+0000 7f8897e00640 1 --2- 172.21.15.5:0/3525606705 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f88740a3c70 0x7f88740a6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:52.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.885+0000 7f8897e00640 1 --2- 172.21.15.5:0/3525606705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8890107d80 0x7f88901a1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:52.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.885+0000 7f8897e00640 1 --2- 172.21.15.5:0/3525606705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8890105450 0x7f889019ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:52.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.885+0000 7f8897e00640 1 --2- 172.21.15.5:0/3525606705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8890102b20 0x7f889019a5c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:52.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.885+0000 7f8897e00640 1 -- 172.21.15.5:0/3525606705 >> 172.21.15.5:0/3525606705 conn(0x7f88900fc510 msgr2=0x7f8890102300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:52.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.886+0000 7f8897e00640 1 -- 172.21.15.5:0/3525606705 shutdown_connections 2024-04-24T06:43:52.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:52.886+0000 7f8897e00640 1 -- 172.21.15.5:0/3525606705 wait complete. 2024-04-24T06:43:52.905 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-24T06:43:53.023 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:43:52 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:43:52.760+0000 7fd702ae5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T06:43:53.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.100+0000 7fc277a00640 1 -- 172.21.15.5:0/2828217370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc270104290 msgr2=0x7fc2701046b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:53.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.100+0000 7fc277a00640 1 --2- 172.21.15.5:0/2828217370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc270104290 0x7fc2701046b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc264009900 tx=0x7fc26402a3f0 comp rx=0 tx=0).stop 2024-04-24T06:43:53.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.100+0000 7fc277a00640 1 -- 172.21.15.5:0/2828217370 shutdown_connections 2024-04-24T06:43:53.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.100+0000 7fc277a00640 1 --2- 172.21.15.5:0/2828217370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc270105e40 0x7fc27010c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:53.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.100+0000 7fc277a00640 1 --2- 172.21.15.5:0/2828217370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc270105480 0x7fc270105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:53.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.100+0000 7fc277a00640 1 --2- 172.21.15.5:0/2828217370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc270104290 0x7fc2701046b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:53.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.100+0000 7fc277a00640 1 -- 172.21.15.5:0/2828217370 >> 172.21.15.5:0/2828217370 conn(0x7fc2700ff820 msgr2=0x7fc270101c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:53.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.100+0000 7fc277a00640 1 -- 172.21.15.5:0/2828217370 shutdown_connections 2024-04-24T06:43:53.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.100+0000 7fc277a00640 1 -- 172.21.15.5:0/2828217370 wait complete. 2024-04-24T06:43:53.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.101+0000 7fc277a00640 1 Processor -- start 2024-04-24T06:43:53.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.101+0000 7fc277a00640 1 -- start start 2024-04-24T06:43:53.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.101+0000 7fc277a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc270104290 0x7fc27019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:53.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.101+0000 7fc277a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc270105480 0x7fc27019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:53.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.101+0000 7fc277a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc270105e40 0x7fc2701a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:53.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.101+0000 7fc277a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc27010ff70 con 0x7fc270104290 2024-04-24T06:43:53.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.101+0000 7fc277a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc27010fdf0 con 0x7fc270105e40 2024-04-24T06:43:53.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.101+0000 7fc277a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc2701100f0 con 0x7fc270105480 2024-04-24T06:43:53.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.102+0000 7fc26fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc270105480 0x7fc27019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:53.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.102+0000 7fc275a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc270105e40 0x7fc2701a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:53.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.102+0000 7fc275000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc270104290 0x7fc27019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:53.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.102+0000 7fc26fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc270105480 0x7fc27019aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:54562/0 (socket says 172.21.15.5:54562) 2024-04-24T06:43:53.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.102+0000 7fc275a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc270105e40 0x7fc2701a1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:33122/0 (socket says 172.21.15.5:33122) 2024-04-24T06:43:53.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.102+0000 7fc26fe00640 1 -- 172.21.15.5:0/1208620053 learned_addr learned my addr 172.21.15.5:0/1208620053 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:53.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.102+0000 7fc26fe00640 1 -- 172.21.15.5:0/1208620053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc270105e40 msgr2=0x7fc2701a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:53.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.102+0000 7fc26fe00640 1 --2- 172.21.15.5:0/1208620053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc270105e40 0x7fc2701a1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:53.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.102+0000 7fc26fe00640 1 -- 172.21.15.5:0/1208620053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc270104290 msgr2=0x7fc27019a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:53.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.102+0000 7fc26fe00640 1 --2- 172.21.15.5:0/1208620053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc270104290 0x7fc27019a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:53.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.102+0000 7fc26fe00640 1 -- 172.21.15.5:0/1208620053 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2701a2190 con 0x7fc270105480 2024-04-24T06:43:53.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.102+0000 7fc275a00640 1 --2- 172.21.15.5:0/1208620053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc270105e40 0x7fc2701a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:43:53.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.102+0000 7fc275000640 1 --2- 172.21.15.5:0/1208620053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc270104290 0x7fc27019a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:43:53.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.102+0000 7fc26fe00640 1 --2- 172.21.15.5:0/1208620053 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc270105480 0x7fc27019aab0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc25c00ea60 tx=0x7fc25c00ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:53.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.103+0000 7fc26d600640 1 -- 172.21.15.5:0/1208620053 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25c00cd20 con 0x7fc270105480 2024-04-24T06:43:53.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.103+0000 7fc277a00640 1 -- 172.21.15.5:0/1208620053 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2701a2480 con 0x7fc270105480 2024-04-24T06:43:53.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.103+0000 7fc26d600640 1 -- 172.21.15.5:0/1208620053 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc25c0035f0 con 0x7fc270105480 2024-04-24T06:43:53.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.103+0000 7fc26d600640 1 -- 172.21.15.5:0/1208620053 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25c010690 con 0x7fc270105480 2024-04-24T06:43:53.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.103+0000 7fc277a00640 1 -- 172.21.15.5:0/1208620053 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2701a2a20 con 0x7fc270105480 2024-04-24T06:43:53.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.104+0000 7fc277a00640 1 -- 172.21.15.5:0/1208620053 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc27010b5b0 con 0x7fc270105480 2024-04-24T06:43:53.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.106+0000 7fc26d600640 1 -- 172.21.15.5:0/1208620053 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) ==== 138662+0+0 (secure 0 0 0) 0x7fc25c0037a0 con 0x7fc270105480 2024-04-24T06:43:53.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.108+0000 7fc26d600640 1 --2- 172.21.15.5:0/1208620053 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fc2580a3bf0 0x7fc2580a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:53.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.108+0000 7fc275000640 1 -- 172.21.15.5:0/1208620053 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fc2580a3bf0 msgr2=0x7fc2580a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/4042936965 2024-04-24T06:43:53.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.108+0000 7fc26d600640 1 -- 172.21.15.5:0/1208620053 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fc25c0d0780 con 0x7fc270105480 2024-04-24T06:43:53.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.108+0000 7fc275000640 1 --2- 172.21.15.5:0/1208620053 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fc2580a3bf0 0x7fc2580a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T06:43:53.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.112+0000 7fc26d600640 1 -- 172.21.15.5:0/1208620053 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc25c09d120 con 0x7fc270105480 2024-04-24T06:43:53.196 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:43:52 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:43:52.895+0000 7f233d292200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T06:43:53.232 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:43:52 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:43:52.804+0000 7f9ab7685200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T06:43:53.232 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:43:52 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:43:52.910+0000 7f9ab7685200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T06:43:53.232 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:43:53 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:43:53.231+0000 7f9ab7685200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T06:43:53.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.309+0000 7fc275000640 1 -- 172.21.15.5:0/1208620053 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fc2580a3bf0 msgr2=0x7fc2580a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/4042936965 2024-04-24T06:43:53.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.309+0000 7fc275000640 1 --2- 172.21.15.5:0/1208620053 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fc2580a3bf0 0x7fc2580a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T06:43:53.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.316+0000 7fc277a00640 1 -- 172.21.15.5:0/1208620053 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fc27002d620 con 0x7fc270105480 2024-04-24T06:43:53.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.323+0000 7fc26d600640 1 -- 172.21.15.5:0/1208620053 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v29)=0 v29) ==== 133+0+0 (secure 0 0 0) 0x7fc25c09ca20 con 0x7fc270105480 2024-04-24T06:43:53.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.325+0000 7fc277a00640 1 -- 172.21.15.5:0/1208620053 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fc2580a3bf0 msgr2=0x7fc2580a60b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T06:43:53.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.325+0000 7fc277a00640 1 --2- 172.21.15.5:0/1208620053 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fc2580a3bf0 0x7fc2580a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:53.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.325+0000 7fc277a00640 1 -- 172.21.15.5:0/1208620053 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc270105480 msgr2=0x7fc27019aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:53.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.325+0000 7fc277a00640 1 --2- 172.21.15.5:0/1208620053 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc270105480 0x7fc27019aab0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc25c00ea60 tx=0x7fc25c00ee60 comp rx=0 tx=0).stop 2024-04-24T06:43:53.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.325+0000 7fc277a00640 1 -- 172.21.15.5:0/1208620053 shutdown_connections 2024-04-24T06:43:53.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.325+0000 7fc277a00640 1 --2- 172.21.15.5:0/1208620053 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7fc2580a3bf0 0x7fc2580a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:53.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.325+0000 7fc277a00640 1 --2- 172.21.15.5:0/1208620053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc270105e40 0x7fc2701a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:53.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.325+0000 7fc277a00640 1 --2- 172.21.15.5:0/1208620053 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc270105480 0x7fc27019aab0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:53.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.325+0000 7fc277a00640 1 --2- 172.21.15.5:0/1208620053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc270104290 0x7fc27019a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:53.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.325+0000 7fc277a00640 1 -- 172.21.15.5:0/1208620053 >> 172.21.15.5:0/1208620053 conn(0x7fc2700ff820 msgr2=0x7fc270108780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:53.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.325+0000 7fc277a00640 1 -- 172.21.15.5:0/1208620053 shutdown_connections 2024-04-24T06:43:53.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:53.325+0000 7fc277a00640 1 -- 172.21.15.5:0/1208620053 wait complete. 2024-04-24T06:43:53.342 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:43:53 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:43:53.021+0000 7fd702ae5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T06:43:53.342 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:43:53 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:43:53.160+0000 7fd702ae5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T06:43:53.345 INFO:teuthology.run_tasks:Running task kclient... 2024-04-24T06:43:53.427 INFO:tasks.kclient:Mounting kernel clients... 2024-04-24T06:43:53.427 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-24T06:43:53.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:43:53.428 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-04-24T06:43:53.450 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:43:53 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:43:53.198+0000 7f233d292200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T06:43:53.450 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:43:53 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:43:53.281+0000 7f233d292200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T06:43:53.450 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:53 smithi137 ceph-mon[33017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-24T06:43:53.450 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:53 smithi137 ceph-mon[33017]: mgrmap e19: x(active, since 10m), standbys: y, z 2024-04-24T06:43:53.450 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:53 smithi137 ceph-mon[33017]: from='client.? ' entity='client.admin' 2024-04-24T06:43:53.450 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:53 smithi137 ceph-mon[33017]: from='client.? ' entity='client.admin' 2024-04-24T06:43:53.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:43:53.484 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link delete ceph-brx 2024-04-24T06:43:53.518 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:43:53 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:43:53.319+0000 7f9ab7685200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T06:43:53.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:53 smithi037 ceph-mon[32864]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-24T06:43:53.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:53 smithi037 ceph-mon[32864]: mgrmap e19: x(active, since 10m), standbys: y, z 2024-04-24T06:43:53.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:53 smithi037 ceph-mon[32864]: from='client.? ' entity='client.admin' 2024-04-24T06:43:53.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:53 smithi037 ceph-mon[32864]: from='client.? ' entity='client.admin' 2024-04-24T06:43:53.571 INFO:teuthology.orchestra.run.smithi005.stderr:Cannot find device "ceph-brx" 2024-04-24T06:43:53.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:43:53.575 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-04-24T06:43:53.575 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T06:43:53.576 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T06:43:53.576 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-24T06:43:53.641 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:53 smithi005 ceph-mon[27798]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-24T06:43:53.641 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:53 smithi005 ceph-mon[27798]: mgrmap e19: x(active, since 10m), standbys: y, z 2024-04-24T06:43:53.641 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:53 smithi005 ceph-mon[27798]: from='client.? ' entity='client.admin' 2024-04-24T06:43:53.642 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:53 smithi005 ceph-mon[27798]: from='client.? ' entity='client.admin' 2024-04-24T06:43:53.642 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:43:53 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:43:53.582+0000 7fd702ae5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T06:43:54.005 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:43:54.006 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:43:53 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:43:53.701+0000 7fd702ae5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T06:43:54.784 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:43:54 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:43:54.447+0000 7f233d292200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T06:43:54.784 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:43:54 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:43:54.614+0000 7f233d292200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T06:43:54.785 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:43:54 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:43:54.783+0000 7f233d292200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T06:43:54.914 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:43:54 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:43:54.559+0000 7f9ab7685200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T06:43:54.914 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:43:54 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:43:54.736+0000 7f9ab7685200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T06:43:55.215 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:43:54 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:43:54.912+0000 7f9ab7685200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T06:43:55.368 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:43:55 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:43:55.070+0000 7f233d292200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T06:43:55.368 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:43:55 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:43:55.164+0000 7f233d292200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T06:43:55.368 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:43:55 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:43:55.366+0000 7f233d292200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T06:43:55.518 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:43:55 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:43:55.213+0000 7f9ab7685200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T06:43:55.518 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:43:55 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:43:55.309+0000 7f9ab7685200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T06:43:55.600 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:43:55 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:43:55.346+0000 7fd702ae5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T06:43:55.696 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:43:55 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:43:55.560+0000 7f233d292200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T06:43:55.895 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:43:55 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:43:55.598+0000 7fd702ae5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T06:43:55.896 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:43:55 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:43:55.819+0000 7fd702ae5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T06:43:56.018 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:43:55 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:43:55.519+0000 7f9ab7685200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T06:43:56.019 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:43:55 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:43:55.720+0000 7f9ab7685200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T06:43:56.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.192+0000 7f71dd800700 1 -- 172.21.15.5:0/763741118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71d806a800 msgr2=0x7f71d806ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:56.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.192+0000 7f71dd800700 1 --2- 172.21.15.5:0/763741118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71d806a800 0x7f71d806ac60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f71c40098f0 tx=0x7f71c4009c00 comp rx=0 tx=0).stop 2024-04-24T06:43:56.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.192+0000 7f71dd800700 1 -- 172.21.15.5:0/763741118 shutdown_connections 2024-04-24T06:43:56.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.192+0000 7f71dd800700 1 --2- 172.21.15.5:0/763741118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71d806b1a0 0x7f71d810f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:56.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.192+0000 7f71dd800700 1 --2- 172.21.15.5:0/763741118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71d8108b70 0x7f71d8108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:56.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.192+0000 7f71dd800700 1 --2- 172.21.15.5:0/763741118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71d806a800 0x7f71d806ac60 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:56.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.192+0000 7f71dd800700 1 -- 172.21.15.5:0/763741118 >> 172.21.15.5:0/763741118 conn(0x7f71d80789a0 msgr2=0x7f71d8078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:56.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.193+0000 7f71dd800700 1 -- 172.21.15.5:0/763741118 shutdown_connections 2024-04-24T06:43:56.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.193+0000 7f71dd800700 1 -- 172.21.15.5:0/763741118 wait complete. 2024-04-24T06:43:56.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.194+0000 7f71dd800700 1 Processor -- start 2024-04-24T06:43:56.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.195+0000 7f71dd800700 1 -- start start 2024-04-24T06:43:56.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.195+0000 7f71dd800700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71d806a800 0x7f71d8101ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:56.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.195+0000 7f71dd800700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71d806b1a0 0x7f71d8102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:56.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.195+0000 7f71dd800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71d8108b70 0x7f71d8102540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:56.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.195+0000 7f71dd800700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f71d8112dd0 con 0x7f71d806a800 2024-04-24T06:43:56.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.195+0000 7f71dd800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f71d8112c50 con 0x7f71d8108b70 2024-04-24T06:43:56.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.195+0000 7f71dd800700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f71d8112f50 con 0x7f71d806b1a0 2024-04-24T06:43:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.195+0000 7f71d6000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71d806b1a0 0x7f71d8102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.195+0000 7f71d6000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71d806b1a0 0x7f71d8102000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:56620/0 (socket says 172.21.15.5:56620) 2024-04-24T06:43:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.195+0000 7f71d6000700 1 -- 172.21.15.5:0/1008044015 learned_addr learned my addr 172.21.15.5:0/1008044015 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:43:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.195+0000 7f71d6a00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71d806a800 0x7f71d8101ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.195+0000 7f71d7400700 1 --2- 172.21.15.5:0/1008044015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71d8108b70 0x7f71d8102540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:43:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.196+0000 7f71d6000700 1 -- 172.21.15.5:0/1008044015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71d8108b70 msgr2=0x7f71d8102540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.196+0000 7f71d6000700 1 --2- 172.21.15.5:0/1008044015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71d8108b70 0x7f71d8102540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.196+0000 7f71d6000700 1 -- 172.21.15.5:0/1008044015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71d806a800 msgr2=0x7f71d8101ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.196+0000 7f71d6000700 1 --2- 172.21.15.5:0/1008044015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71d806a800 0x7f71d8101ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.196+0000 7f71d6000700 1 -- 172.21.15.5:0/1008044015 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d81abcd0 con 0x7f71d806b1a0 2024-04-24T06:43:56.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.196+0000 7f71d6000700 1 --2- 172.21.15.5:0/1008044015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71d806b1a0 0x7f71d8102000 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f71c4009230 tx=0x7f71c400fa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:43:56.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.197+0000 7f71cf400700 1 -- 172.21.15.5:0/1008044015 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c4005180 con 0x7f71d806b1a0 2024-04-24T06:43:56.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.197+0000 7f71cf400700 1 -- 172.21.15.5:0/1008044015 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71c400b560 con 0x7f71d806b1a0 2024-04-24T06:43:56.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.197+0000 7f71dd800700 1 -- 172.21.15.5:0/1008044015 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71d81abf00 con 0x7f71d806b1a0 2024-04-24T06:43:56.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.197+0000 7f71cf400700 1 -- 172.21.15.5:0/1008044015 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c4003a70 con 0x7f71d806b1a0 2024-04-24T06:43:56.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.197+0000 7f71dd800700 1 -- 172.21.15.5:0/1008044015 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71d81ac2f0 con 0x7f71d806b1a0 2024-04-24T06:43:56.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.198+0000 7f71dd800700 1 -- 172.21.15.5:0/1008044015 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71d81041e0 con 0x7f71d806b1a0 2024-04-24T06:43:56.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.200+0000 7f71cf400700 1 -- 172.21.15.5:0/1008044015 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) ==== 138662+0+0 (secure 0 0 0) 0x7f71c4029060 con 0x7f71d806b1a0 2024-04-24T06:43:56.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.201+0000 7f71cf400700 1 --2- 172.21.15.5:0/1008044015 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f71bc0a3d10 0x7f71bc0a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:43:56.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.201+0000 7f71d6a00700 1 -- 172.21.15.5:0/1008044015 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f71bc0a3d10 msgr2=0x7f71bc0a61d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/4042936965 2024-04-24T06:43:56.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.201+0000 7f71cf400700 1 -- 172.21.15.5:0/1008044015 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f71c40d0c90 con 0x7f71d806b1a0 2024-04-24T06:43:56.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.201+0000 7f71d6a00700 1 --2- 172.21.15.5:0/1008044015 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f71bc0a3d10 0x7f71bc0a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T06:43:56.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.205+0000 7f71cf400700 1 -- 172.21.15.5:0/1008044015 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f71c4013030 con 0x7f71d806b1a0 2024-04-24T06:43:56.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.402+0000 7f71d6a00700 1 -- 172.21.15.5:0/1008044015 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f71bc0a3d10 msgr2=0x7f71bc0a61d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/4042936965 2024-04-24T06:43:56.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.402+0000 7f71d6a00700 1 --2- 172.21.15.5:0/1008044015 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f71bc0a3d10 0x7f71bc0a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T06:43:56.446 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:43:56 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:43:56.155+0000 7f233d292200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T06:43:56.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.485+0000 7f71dd800700 1 -- 172.21.15.5:0/1008044015 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f71d802dfe0 con 0x7f71d806b1a0 2024-04-24T06:43:56.487 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:43:56 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:43:56.249+0000 7fd702ae5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T06:43:56.487 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:43:56 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:43:56.384+0000 7fd702ae5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T06:43:56.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.486+0000 7f71cf400700 1 -- 172.21.15.5:0/1008044015 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v22) ==== 53+0+73 (secure 0 0 0) 0x7f71c409d6c0 con 0x7f71d806b1a0 2024-04-24T06:43:56.488 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T06:43:56.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.489+0000 7f71dd800700 1 -- 172.21.15.5:0/1008044015 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f71bc0a3d10 msgr2=0x7f71bc0a61d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T06:43:56.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.489+0000 7f71dd800700 1 --2- 172.21.15.5:0/1008044015 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f71bc0a3d10 0x7f71bc0a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:56.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.489+0000 7f71dd800700 1 -- 172.21.15.5:0/1008044015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71d806b1a0 msgr2=0x7f71d8102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:43:56.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.489+0000 7f71dd800700 1 --2- 172.21.15.5:0/1008044015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71d806b1a0 0x7f71d8102000 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f71c4009230 tx=0x7f71c400fa70 comp rx=0 tx=0).stop 2024-04-24T06:43:56.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.489+0000 7f71dd800700 1 -- 172.21.15.5:0/1008044015 shutdown_connections 2024-04-24T06:43:56.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.489+0000 7f71dd800700 1 --2- 172.21.15.5:0/1008044015 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f71bc0a3d10 0x7f71bc0a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:56.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.489+0000 7f71dd800700 1 --2- 172.21.15.5:0/1008044015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71d8108b70 0x7f71d8102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:56.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.489+0000 7f71dd800700 1 --2- 172.21.15.5:0/1008044015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71d806a800 0x7f71d8101ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:56.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.489+0000 7f71dd800700 1 --2- 172.21.15.5:0/1008044015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71d806b1a0 0x7f71d8102000 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:43:56.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.489+0000 7f71dd800700 1 -- 172.21.15.5:0/1008044015 >> 172.21.15.5:0/1008044015 conn(0x7f71d80789a0 msgr2=0x7f71d80ffaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:43:56.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.489+0000 7f71dd800700 1 -- 172.21.15.5:0/1008044015 shutdown_connections 2024-04-24T06:43:56.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:56.489+0000 7f71dd800700 1 -- 172.21.15.5:0/1008044015 wait complete. 2024-04-24T06:43:56.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:56 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1008044015' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T06:43:56.756 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:43:56 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:43:56.658+0000 7fd702ae5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T06:43:56.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:56 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/1008044015' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T06:43:56.768 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:43:56 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:43:56.348+0000 7f9ab7685200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T06:43:56.877 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:56 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/1008044015' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T06:43:56.878 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:43:56 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:43:56.579+0000 7f233d292200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T06:43:56.878 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:43:56 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:43:56.676+0000 7f233d292200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T06:43:57.097 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:43:56 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:43:56.786+0000 7f9ab7685200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T06:43:57.097 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:43:56 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:43:56.887+0000 7f9ab7685200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T06:43:57.097 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:43:57 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:43:57.095+0000 7f9ab7685200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T06:43:57.196 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:43:56 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:43:56.876+0000 7f233d292200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T06:43:57.216 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:43:56 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:43:56.930+0000 7fd702ae5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T06:43:57.268 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T06:43:57.268 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T06:43:57.268 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-04-24T06:43:57.268 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T06:43:57.268 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T06:43:57.268 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T06:43:57.268 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T06:43:57.268 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T06:43:57.268 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:43:57.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:43:57.269 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-04-24T06:43:57.321 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T06:43:57.322 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T06:43:57.322 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T06:43:57.322 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:43:57.322 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-04-24T06:43:57.322 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:43:57.322 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:43:57.322 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-04-24T06:43:57.322 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-04-24T06:43:57.322 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:43:57.322 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-04-24T06:43:57.323 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-04-24T06:43:57.323 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T06:43:57.323 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-04-24T06:43:57.323 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-04-24T06:43:57.323 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-24T06:43:57.323 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 41973sec preferred_lft 41973sec 2024-04-24T06:43:57.323 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-04-24T06:43:57.323 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-24T06:43:57.323 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T06:43:57.323 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-04-24T06:43:57.323 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-04-24T06:43:57.324 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-24T06:43:57.325 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T06:43:57.325 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-04-24T06:43:57.325 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add name ceph-brx type bridge 2024-04-24T06:43:57.325 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr flush dev ceph-brx 2024-04-24T06:43:57.325 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set ceph-brx up 2024-04-24T06:43:57.325 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-24T06:43:57.325 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-04-24T06:43:57.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T06:43:57.696 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:43:57 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:43:57.397+0000 7f233d292200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T06:43:57.696 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:43:57 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:43:57.490+0000 7f233d292200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T06:43:57.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:43:57.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:43:57.825 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T06:43:57.886 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-04-24T06:43:57.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:43:57.890 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-04-24T06:43:57.947 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-04-24T06:43:57.947 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T06:43:57.947 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T06:43:57.947 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T06:43:57.947 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T06:43:57.948 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T06:43:57.948 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-04-24T06:43:57.948 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T06:43:57.948 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T06:43:57.948 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T06:43:57.948 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-04-24T06:43:57.991 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:43:57 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:43:57.640+0000 7f9ab7685200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T06:43:57.991 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:43:57 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:43:57.736+0000 7f9ab7685200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T06:43:58.006 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:43:57 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:43:57.761+0000 7fd702ae5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T06:43:58.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T06:43:58.075 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:43:57 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:43:57.732+0000 7f233d292200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T06:43:58.075 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:43:57 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:43:57.864+0000 7f233d292200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T06:43:58.075 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:43:57 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:43:57.959+0000 7f233d292200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T06:43:58.075 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:43:58 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:43:58.073+0000 7f233d292200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T06:43:58.268 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:43:57 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:43:57.990+0000 7f9ab7685200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T06:43:58.269 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:43:58 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:43:58.128+0000 7f9ab7685200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T06:43:58.269 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:43:58 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:43:58.227+0000 7f9ab7685200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T06:43:58.649 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:43:58 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:43:58.361+0000 7fd702ae5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T06:43:58.649 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:43:58 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:43:58.496+0000 7fd702ae5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T06:43:58.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:43:58.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:43:58.660 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-04-24T06:43:58.661 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:43:58 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:43:58.348+0000 7f9ab7685200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T06:43:58.696 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:43:58 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:43:58.376+0000 7f233d292200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T06:43:58.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:43:58.711 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list-id 2024-04-24T06:43:58.754 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T06:43:58.754 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-04-24T06:43:58.754 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:43:58.754 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-24T06:43:58.754 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-04-24T06:43:58.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T06:43:58.924 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:43:58 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:43:58.777+0000 7fd702ae5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T06:43:58.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:43:58.978 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-24T06:43:58.978 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T06:43:58.979 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-04-24T06:43:58.979 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-24T06:43:58.979 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-24T06:43:58.979 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-24T06:43:58.979 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-24T06:43:58.979 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-24T06:43:58.979 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-04-24T06:43:59.018 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:43:58 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:43:58.659+0000 7f9ab7685200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T06:43:59.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T06:43:59.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:43:59.254 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:43:58 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:43:58.972+0000 7f233d292200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T06:43:59.254 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:43:59 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z[34885]: 2024-04-24T06:43:59.184+0000 7f233d292200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T06:43:59.260 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T06:43:59.260 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-04-24T06:43:59.260 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-04-24T06:43:59.260 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set dev brx.0 master ceph-brx 2024-04-24T06:43:59.260 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-04-24T06:43:59.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T06:43:59.433 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:59 smithi005 ceph-mon[27798]: Standby manager daemon z restarted 2024-04-24T06:43:59.433 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:59 smithi005 ceph-mon[27798]: Standby manager daemon z started 2024-04-24T06:43:59.433 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:59 smithi005 ceph-mon[27798]: from='mgr.? 172.21.15.137:0/4116872010' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T06:43:59.433 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:59 smithi005 ceph-mon[27798]: from='mgr.? 172.21.15.137:0/4116872010' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T06:43:59.433 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:59 smithi005 ceph-mon[27798]: from='mgr.? 172.21.15.137:0/4116872010' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T06:43:59.434 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:43:59 smithi005 ceph-mon[27798]: from='mgr.? 172.21.15.137:0/4116872010' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T06:43:59.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:43:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:43:59.452 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T06:43:59.452 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T06:43:59.491 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T06:43:59.491 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T06:43:59.491 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T06:43:59.517 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:43:59 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:43:59.289+0000 7f9ab7685200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T06:43:59.517 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:59 smithi037 ceph-mon[32864]: Standby manager daemon z restarted 2024-04-24T06:43:59.517 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:59 smithi037 ceph-mon[32864]: Standby manager daemon z started 2024-04-24T06:43:59.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:59 smithi037 ceph-mon[32864]: from='mgr.? 172.21.15.137:0/4116872010' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T06:43:59.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:59 smithi037 ceph-mon[32864]: from='mgr.? 172.21.15.137:0/4116872010' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T06:43:59.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:59 smithi037 ceph-mon[32864]: from='mgr.? 172.21.15.137:0/4116872010' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T06:43:59.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:43:59 smithi037 ceph-mon[32864]: from='mgr.? 172.21.15.137:0/4116872010' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T06:43:59.580 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-24T06:43:59.580 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-24T06:43:59.580 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/45532007-e9dd-4bc8-88b6-1eb4c5182393 2024-04-24T06:43:59.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:43:59.580 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/45532007-e9dd-4bc8-88b6-1eb4c5182393 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-24T06:43:59.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:59 smithi137 ceph-mon[33017]: Standby manager daemon z restarted 2024-04-24T06:43:59.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:59 smithi137 ceph-mon[33017]: Standby manager daemon z started 2024-04-24T06:43:59.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:59 smithi137 ceph-mon[33017]: from='mgr.? 172.21.15.137:0/4116872010' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T06:43:59.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:59 smithi137 ceph-mon[33017]: from='mgr.? 172.21.15.137:0/4116872010' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T06:43:59.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:59 smithi137 ceph-mon[33017]: from='mgr.? 172.21.15.137:0/4116872010' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T06:43:59.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:43:59 smithi137 ceph-mon[33017]: from='mgr.? 172.21.15.137:0/4116872010' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T06:43:59.744 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-24T06:43:59.745 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-04-24T06:43:59.756 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:43:59 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:43:59.482+0000 7fd702ae5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T06:43:59.756 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:43:59 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:43:59.617+0000 7fd702ae5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T06:43:59.768 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:43:59 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y[34259]: 2024-04-24T06:43:59.516+0000 7f9ab7685200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T06:44:00.152 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:43:59 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:43:59.960+0000 7fd702ae5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T06:44:00.448 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:00 smithi005 ceph-mon[27798]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-04-24T06:44:00.448 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:00 smithi005 ceph-mon[27798]: Standby manager daemon y restarted 2024-04-24T06:44:00.448 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:00 smithi005 ceph-mon[27798]: Standby manager daemon y started 2024-04-24T06:44:00.448 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:00 smithi005 ceph-mon[27798]: from='mgr.? 172.21.15.37:0/467863069' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T06:44:00.449 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:00 smithi005 ceph-mon[27798]: from='mgr.? 172.21.15.37:0/467863069' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T06:44:00.449 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:00 smithi005 ceph-mon[27798]: from='mgr.? 172.21.15.37:0/467863069' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T06:44:00.449 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:00 smithi005 ceph-mon[27798]: from='mgr.? 172.21.15.37:0/467863069' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T06:44:00.449 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:44:00 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:44:00.150+0000 7fd702ae5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T06:44:00.449 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:44:00 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:44:00.287+0000 7fd702ae5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T06:44:00.449 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:44:00 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:44:00.446+0000 7fd702ae5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T06:44:00.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:00 smithi037 ceph-mon[32864]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-04-24T06:44:00.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:00 smithi037 ceph-mon[32864]: Standby manager daemon y restarted 2024-04-24T06:44:00.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:00 smithi037 ceph-mon[32864]: Standby manager daemon y started 2024-04-24T06:44:00.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:00 smithi037 ceph-mon[32864]: from='mgr.? 172.21.15.37:0/467863069' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T06:44:00.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:00 smithi037 ceph-mon[32864]: from='mgr.? 172.21.15.37:0/467863069' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T06:44:00.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:00 smithi037 ceph-mon[32864]: from='mgr.? 172.21.15.37:0/467863069' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T06:44:00.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:00 smithi037 ceph-mon[32864]: from='mgr.? 172.21.15.37:0/467863069' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T06:44:00.570 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-24T06:44:00.571 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-04-24T06:44:00.571 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.37:3300,172.21.15.137:3300" 2024-04-24T06:44:00.571 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with new device syntax: 0@43e9bad0-0204-11ef-bc93-c7b262605968.cephfs=/volumes/qa/sv_1/45532007-e9dd-4bc8-88b6-1eb4c5182393 2024-04-24T06:44:00.571 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync,name=0,key=0,mon_addr=172.21.15.5:3300/172.21.15.37:3300/172.21.15.137:3300" will pass to kernel 2024-04-24T06:44:00.575 INFO:teuthology.orchestra.run.smithi005.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-24T06:44:00.575 INFO:teuthology.orchestra.run.smithi005.stdout: You just mounted a file system that supports labels which does not 2024-04-24T06:44:00.575 INFO:teuthology.orchestra.run.smithi005.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T06:44:00.576 INFO:teuthology.orchestra.run.smithi005.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T06:44:00.576 INFO:teuthology.orchestra.run.smithi005.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T06:44:00.580 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T06:44:00.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:44:00.581 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T06:44:00.664 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T06:44:00.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:00 smithi137 ceph-mon[33017]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-04-24T06:44:00.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:00 smithi137 ceph-mon[33017]: Standby manager daemon y restarted 2024-04-24T06:44:00.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:00 smithi137 ceph-mon[33017]: Standby manager daemon y started 2024-04-24T06:44:00.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:00 smithi137 ceph-mon[33017]: from='mgr.? 172.21.15.37:0/467863069' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T06:44:00.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:00 smithi137 ceph-mon[33017]: from='mgr.? 172.21.15.37:0/467863069' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T06:44:00.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:00 smithi137 ceph-mon[33017]: from='mgr.? 172.21.15.37:0/467863069' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T06:44:00.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:00 smithi137 ceph-mon[33017]: from='mgr.? 172.21.15.37:0/467863069' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T06:44:00.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T06:44:00.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:44:00.791 INFO:teuthology.orchestra.run.smithi005.stdout:client25240 2024-04-24T06:44:00.791 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-24T06:44:00.791 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:44:00.837 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T06:44:00.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T06:44:00.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:44:00.996 INFO:teuthology.orchestra.run.smithi005.stdout:client25240 2024-04-24T06:44:00.996 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T06:44:01.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T06:44:01.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:44:01.106 INFO:teuthology.orchestra.run.smithi005.stdout:client25240 2024-04-24T06:44:01.106 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/43e9bad0-0204-11ef-bc93-c7b262605968.client25240/status') 2024-04-24T06:44:01.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T06:44:01.241 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:44:00 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:44:00.846+0000 7fd702ae5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T06:44:01.248 INFO:teuthology.orchestra.run.smithi005.stdout:instance: client.25240 (3)172.21.15.5:0/1304624346 2024-04-24T06:44:01.248 INFO:teuthology.orchestra.run.smithi005.stdout:blocklisted: false 2024-04-24T06:44:01.249 INFO:teuthology.orchestra.run.smithi005.stderr:0+1 records in 2024-04-24T06:44:01.249 INFO:teuthology.orchestra.run.smithi005.stderr:0+1 records out 2024-04-24T06:44:01.249 INFO:teuthology.orchestra.run.smithi005.stderr:70 bytes copied, 9.7699e-05 s, 716 kB/s 2024-04-24T06:44:01.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T06:44:01.262 INFO:teuthology.run_tasks:Running task exec... 2024-04-24T06:44:01.271 INFO:teuthology.task.exec:Executing custom commands... 2024-04-24T06:44:01.271 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-04-24T06:44:01.272 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-24T06:44:01.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.453+0000 7f0a40a00640 1 -- 172.21.15.5:0/1941684722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a3c1052c0 msgr2=0x7f0a3c1076b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:01.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.453+0000 7f0a40a00640 1 --2- 172.21.15.5:0/1941684722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a3c1052c0 0x7f0a3c1076b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0a28009900 tx=0x7f0a2802a3f0 comp rx=0 tx=0).stop 2024-04-24T06:44:01.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.453+0000 7f0a40a00640 1 -- 172.21.15.5:0/1941684722 shutdown_connections 2024-04-24T06:44:01.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.453+0000 7f0a40a00640 1 --2- 172.21.15.5:0/1941684722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a3c107bf0 0x7f0a3c109fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:01.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.453+0000 7f0a40a00640 1 --2- 172.21.15.5:0/1941684722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a3c1052c0 0x7f0a3c1076b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:01.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.453+0000 7f0a40a00640 1 --2- 172.21.15.5:0/1941684722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a3c102990 0x7f0a3c104d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:01.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.453+0000 7f0a40a00640 1 -- 172.21.15.5:0/1941684722 >> 172.21.15.5:0/1941684722 conn(0x7f0a3c0fc2e0 msgr2=0x7f0a3c0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:01.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.453+0000 7f0a40a00640 1 -- 172.21.15.5:0/1941684722 shutdown_connections 2024-04-24T06:44:01.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.453+0000 7f0a40a00640 1 -- 172.21.15.5:0/1941684722 wait complete. 2024-04-24T06:44:01.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.454+0000 7f0a40a00640 1 Processor -- start 2024-04-24T06:44:01.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.454+0000 7f0a40a00640 1 -- start start 2024-04-24T06:44:01.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.454+0000 7f0a40a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a3c102990 0x7f0a3c102300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:01.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.454+0000 7f0a40a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a3c1052c0 0x7f0a3c100940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:01.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.454+0000 7f0a40a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a3c107bf0 0x7f0a3c100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:01.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.454+0000 7f0a40a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c10db20 con 0x7f0a3c107bf0 2024-04-24T06:44:01.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.454+0000 7f0a40a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c10d9a0 con 0x7f0a3c1052c0 2024-04-24T06:44:01.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.454+0000 7f0a40a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c10dca0 con 0x7f0a3c102990 2024-04-24T06:44:01.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.454+0000 7f0a3b200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a3c107bf0 0x7f0a3c100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:01.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.455+0000 7f0a39e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a3c1052c0 0x7f0a3c100940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:01.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.455+0000 7f0a3b200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a3c107bf0 0x7f0a3c100e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38518/0 (socket says 172.21.15.5:38518) 2024-04-24T06:44:01.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.455+0000 7f0a3b200640 1 -- 172.21.15.5:0/836464234 learned_addr learned my addr 172.21.15.5:0/836464234 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:01.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.455+0000 7f0a39e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a3c1052c0 0x7f0a3c100940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:59046/0 (socket says 172.21.15.5:59046) 2024-04-24T06:44:01.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.455+0000 7f0a3a800640 1 --2- 172.21.15.5:0/836464234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a3c102990 0x7f0a3c102300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:01.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.455+0000 7f0a3b200640 1 -- 172.21.15.5:0/836464234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a3c102990 msgr2=0x7f0a3c102300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:01.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.455+0000 7f0a3b200640 1 --2- 172.21.15.5:0/836464234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a3c102990 0x7f0a3c102300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:01.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.455+0000 7f0a3b200640 1 -- 172.21.15.5:0/836464234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a3c1052c0 msgr2=0x7f0a3c100940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:01.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.455+0000 7f0a3b200640 1 --2- 172.21.15.5:0/836464234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a3c1052c0 0x7f0a3c100940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:01.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.455+0000 7f0a3b200640 1 -- 172.21.15.5:0/836464234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a3c1013c0 con 0x7f0a3c107bf0 2024-04-24T06:44:01.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.455+0000 7f0a3b200640 1 --2- 172.21.15.5:0/836464234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a3c107bf0 0x7f0a3c100e80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0a3400eb10 tx=0x7f0a3400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:01.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.456+0000 7f0a33400640 1 -- 172.21.15.5:0/836464234 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a34004b20 con 0x7f0a3c107bf0 2024-04-24T06:44:01.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.456+0000 7f0a40a00640 1 -- 172.21.15.5:0/836464234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a3c1016b0 con 0x7f0a3c107bf0 2024-04-24T06:44:01.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.456+0000 7f0a33400640 1 -- 172.21.15.5:0/836464234 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a34004cc0 con 0x7f0a3c107bf0 2024-04-24T06:44:01.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.456+0000 7f0a33400640 1 -- 172.21.15.5:0/836464234 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a34010760 con 0x7f0a3c107bf0 2024-04-24T06:44:01.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.456+0000 7f0a40a00640 1 -- 172.21.15.5:0/836464234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a3c1a6c40 con 0x7f0a3c107bf0 2024-04-24T06:44:01.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.457+0000 7f0a40a00640 1 -- 172.21.15.5:0/836464234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a3c0688a0 con 0x7f0a3c107bf0 2024-04-24T06:44:01.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.459+0000 7f0a33400640 1 -- 172.21.15.5:0/836464234 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 21) ==== 138662+0+0 (secure 0 0 0) 0x7f0a34010900 con 0x7f0a3c107bf0 2024-04-24T06:44:01.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.460+0000 7f0a33400640 1 --2- 172.21.15.5:0/836464234 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f0a200a3c60 0x7f0a200a6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:01.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.460+0000 7f0a3a800640 1 -- 172.21.15.5:0/836464234 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f0a200a3c60 msgr2=0x7f0a200a6120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/4042936965 2024-04-24T06:44:01.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.460+0000 7f0a33400640 1 -- 172.21.15.5:0/836464234 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f0a340d0ba0 con 0x7f0a3c107bf0 2024-04-24T06:44:01.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.461+0000 7f0a3a800640 1 --2- 172.21.15.5:0/836464234 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f0a200a3c60 0x7f0a200a6120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T06:44:01.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.464+0000 7f0a33400640 1 -- 172.21.15.5:0/836464234 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0a34053ff0 con 0x7f0a3c107bf0 2024-04-24T06:44:01.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:01 smithi005 ceph-mon[27798]: mgrmap e21: x(active, since 10m), standbys: y, z 2024-04-24T06:44:01.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:01 smithi037 ceph-mon[32864]: mgrmap e21: x(active, since 10m), standbys: y, z 2024-04-24T06:44:01.597 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:01 smithi137 ceph-mon[33017]: mgrmap e21: x(active, since 10m), standbys: y, z 2024-04-24T06:44:01.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.662+0000 7f0a3a800640 1 -- 172.21.15.5:0/836464234 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f0a200a3c60 msgr2=0x7f0a200a6120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/4042936965 2024-04-24T06:44:01.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.662+0000 7f0a3a800640 1 --2- 172.21.15.5:0/836464234 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f0a200a3c60 0x7f0a200a6120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T06:44:01.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.709+0000 7f0a40a00640 1 -- 172.21.15.5:0/836464234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f0a3c0fddd0 con 0x7f0a3c107bf0 2024-04-24T06:44:01.963 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:44:01 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:44:01.660+0000 7fd702ae5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T06:44:01.963 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:44:01 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:44:01.961+0000 7fd702ae5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T06:44:01.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.987+0000 7f0a33400640 1 -- 172.21.15.5:0/836464234 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 22) ==== 138331+0+0 (secure 0 0 0) 0x7f0a340994b0 con 0x7f0a3c107bf0 2024-04-24T06:44:01.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.987+0000 7f0a33400640 1 -- 172.21.15.5:0/836464234 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f0a200a3c60 msgr2=0x7f0a200a6120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T06:44:01.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:01.987+0000 7f0a33400640 1 --2- 172.21.15.5:0/836464234 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f0a200a3c60 0x7f0a200a6120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:02.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.272+0000 7f0a33400640 1 -- 172.21.15.5:0/836464234 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v23) ==== 118+0+0 (secure 0 0 0) 0x7f0a3c0fddd0 con 0x7f0a3c107bf0 2024-04-24T06:44:02.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.275+0000 7f0a40a00640 1 -- 172.21.15.5:0/836464234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a3c107bf0 msgr2=0x7f0a3c100e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:02.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.275+0000 7f0a40a00640 1 --2- 172.21.15.5:0/836464234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a3c107bf0 0x7f0a3c100e80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0a3400eb10 tx=0x7f0a3400c2d0 comp rx=0 tx=0).stop 2024-04-24T06:44:02.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.275+0000 7f0a40a00640 1 -- 172.21.15.5:0/836464234 shutdown_connections 2024-04-24T06:44:02.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.276+0000 7f0a40a00640 1 --2- 172.21.15.5:0/836464234 >> [v2:172.21.15.5:6800/4042936965,v1:172.21.15.5:6801/4042936965] conn(0x7f0a200a3c60 0x7f0a200a6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:02.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.276+0000 7f0a40a00640 1 --2- 172.21.15.5:0/836464234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a3c107bf0 0x7f0a3c100e80 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:02.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.276+0000 7f0a40a00640 1 --2- 172.21.15.5:0/836464234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a3c1052c0 0x7f0a3c100940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:02.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.276+0000 7f0a40a00640 1 --2- 172.21.15.5:0/836464234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a3c102990 0x7f0a3c102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:02.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.276+0000 7f0a40a00640 1 -- 172.21.15.5:0/836464234 >> 172.21.15.5:0/836464234 conn(0x7f0a3c0fc2e0 msgr2=0x7f0a3c1033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:02.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.276+0000 7f0a40a00640 1 -- 172.21.15.5:0/836464234 shutdown_connections 2024-04-24T06:44:02.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.276+0000 7f0a40a00640 1 -- 172.21.15.5:0/836464234 wait complete. 2024-04-24T06:44:02.297 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-24T06:44:02.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/836464234' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-24T06:44:02.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: Active manager daemon x restarted 2024-04-24T06:44:02.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: Activating manager daemon x 2024-04-24T06:44:02.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: osdmap e89: 12 total, 12 up, 12 in 2024-04-24T06:44:02.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: mgrmap e22: x(active, starting, since 0.0276083s), standbys: y, z 2024-04-24T06:44:02.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T06:44:02.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T06:44:02.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T06:44:02.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T06:44:02.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T06:44:02.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T06:44:02.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T06:44:02.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T06:44:02.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T06:44:02.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T06:44:02.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T06:44:02.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T06:44:02.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T06:44:02.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T06:44:02.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T06:44:02.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T06:44:02.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T06:44:02.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T06:44:02.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T06:44:02.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T06:44:02.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T06:44:02.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T06:44:02.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T06:44:02.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T06:44:02.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T06:44:02.449 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T06:44:02.449 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T06:44:02.449 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T06:44:02.449 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T06:44:02.449 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T06:44:02.449 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T06:44:02.449 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T06:44:02.449 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T06:44:02.449 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: Manager daemon x is now available 2024-04-24T06:44:02.449 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:44:02.449 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T06:44:02.449 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T06:44:02.449 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T06:44:02.449 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:02 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T06:44:02.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.510+0000 7f5db1200640 1 -- 172.21.15.5:0/2199976644 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5dac101e70 msgr2=0x7f5dac102290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:02.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.510+0000 7f5db1200640 1 --2- 172.21.15.5:0/2199976644 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5dac101e70 0x7f5dac102290 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c009900 tx=0x7f5d9c02a3f0 comp rx=0 tx=0).stop 2024-04-24T06:44:02.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.510+0000 7f5db1200640 1 -- 172.21.15.5:0/2199976644 shutdown_connections 2024-04-24T06:44:02.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.510+0000 7f5db1200640 1 --2- 172.21.15.5:0/2199976644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5dac103a10 0x7f5dac10a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:02.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.510+0000 7f5db1200640 1 --2- 172.21.15.5:0/2199976644 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dac103050 0x7f5dac1034d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:02.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.510+0000 7f5db1200640 1 --2- 172.21.15.5:0/2199976644 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5dac101e70 0x7f5dac102290 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:02.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.510+0000 7f5db1200640 1 -- 172.21.15.5:0/2199976644 >> 172.21.15.5:0/2199976644 conn(0x7f5dac0fd430 msgr2=0x7f5dac0ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:02.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.510+0000 7f5db1200640 1 -- 172.21.15.5:0/2199976644 shutdown_connections 2024-04-24T06:44:02.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.510+0000 7f5db1200640 1 -- 172.21.15.5:0/2199976644 wait complete. 2024-04-24T06:44:02.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.512+0000 7f5db1200640 1 Processor -- start 2024-04-24T06:44:02.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.512+0000 7f5db1200640 1 -- start start 2024-04-24T06:44:02.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.512+0000 7f5db1200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5dac101e70 0x7f5dac075db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:02.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.512+0000 7f5db1200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5dac103050 0x7f5dac0743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:02.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.512+0000 7f5db1200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dac103a10 0x7f5dac074930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:02.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.512+0000 7f5db1200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac10dba0 con 0x7f5dac103050 2024-04-24T06:44:02.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.512+0000 7f5db1200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac10da20 con 0x7f5dac103a10 2024-04-24T06:44:02.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.512+0000 7f5db1200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac10dd20 con 0x7f5dac101e70 2024-04-24T06:44:02.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.513+0000 7f5dab400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5dac101e70 0x7f5dac075db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:02.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.513+0000 7f5daaa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5dac103050 0x7f5dac0743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:02.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.513+0000 7f5dab400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5dac101e70 0x7f5dac075db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:56818/0 (socket says 172.21.15.5:56818) 2024-04-24T06:44:02.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.513+0000 7f5dab400640 1 -- 172.21.15.5:0/3006385350 learned_addr learned my addr 172.21.15.5:0/3006385350 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:02.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.513+0000 7f5dabe00640 1 --2- 172.21.15.5:0/3006385350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dac103a10 0x7f5dac074930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:02.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.513+0000 7f5dab400640 1 -- 172.21.15.5:0/3006385350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dac103a10 msgr2=0x7f5dac074930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:02.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.513+0000 7f5dab400640 1 --2- 172.21.15.5:0/3006385350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dac103a10 0x7f5dac074930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:02.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.513+0000 7f5dab400640 1 -- 172.21.15.5:0/3006385350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5dac103050 msgr2=0x7f5dac0743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:02.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.513+0000 7f5dab400640 1 --2- 172.21.15.5:0/3006385350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5dac103050 0x7f5dac0743f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:02.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.513+0000 7f5dab400640 1 -- 172.21.15.5:0/3006385350 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dac0751c0 con 0x7f5dac101e70 2024-04-24T06:44:02.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.513+0000 7f5dab400640 1 --2- 172.21.15.5:0/3006385350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5dac101e70 0x7f5dac075db0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c031440 tx=0x7f5d9c032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:02.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.514+0000 7f5da3e00640 1 -- 172.21.15.5:0/3006385350 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c03f030 con 0x7f5dac101e70 2024-04-24T06:44:02.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.514+0000 7f5da3e00640 1 -- 172.21.15.5:0/3006385350 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d9c003ce0 con 0x7f5dac101e70 2024-04-24T06:44:02.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.514+0000 7f5db1200640 1 -- 172.21.15.5:0/3006385350 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dac1a22d0 con 0x7f5dac101e70 2024-04-24T06:44:02.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.514+0000 7f5da3e00640 1 -- 172.21.15.5:0/3006385350 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c035070 con 0x7f5dac101e70 2024-04-24T06:44:02.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.514+0000 7f5db1200640 1 -- 172.21.15.5:0/3006385350 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dac1a2820 con 0x7f5dac101e70 2024-04-24T06:44:02.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.515+0000 7f5db1200640 1 -- 172.21.15.5:0/3006385350 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dac109210 con 0x7f5dac101e70 2024-04-24T06:44:02.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.517+0000 7f5da3e00640 1 -- 172.21.15.5:0/3006385350 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 22) ==== 138331+0+0 (secure 0 0 0) 0x7f5d9c042030 con 0x7f5dac101e70 2024-04-24T06:44:02.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/836464234' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-24T06:44:02.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: Active manager daemon x restarted 2024-04-24T06:44:02.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: Activating manager daemon x 2024-04-24T06:44:02.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: osdmap e89: 12 total, 12 up, 12 in 2024-04-24T06:44:02.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: mgrmap e22: x(active, starting, since 0.0276083s), standbys: y, z 2024-04-24T06:44:02.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T06:44:02.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T06:44:02.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T06:44:02.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T06:44:02.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T06:44:02.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T06:44:02.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T06:44:02.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T06:44:02.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T06:44:02.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T06:44:02.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T06:44:02.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T06:44:02.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T06:44:02.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T06:44:02.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T06:44:02.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T06:44:02.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T06:44:02.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T06:44:02.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T06:44:02.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T06:44:02.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T06:44:02.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T06:44:02.520 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T06:44:02.521 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T06:44:02.521 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T06:44:02.521 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T06:44:02.521 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T06:44:02.521 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T06:44:02.521 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T06:44:02.521 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T06:44:02.521 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T06:44:02.521 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T06:44:02.521 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T06:44:02.521 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: Manager daemon x is now available 2024-04-24T06:44:02.521 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:44:02.521 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T06:44:02.521 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T06:44:02.521 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T06:44:02.521 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:02 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T06:44:02.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.518+0000 7f5da3e00640 1 -- 172.21.15.5:0/3006385350 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f5d9c0f04b0 con 0x7f5dac101e70 2024-04-24T06:44:02.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:02.522+0000 7f5da3e00640 1 -- 172.21.15.5:0/3006385350 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5d9c0bcad0 con 0x7f5dac101e70 2024-04-24T06:44:02.757 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:44:02.312+0000 7fd658e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T06:44:02.757 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:44:02.312+0000 7fd658e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T06:44:02.757 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:44:02.312+0000 7fd658e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T06:44:02.757 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:44:02.312+0000 7fd658e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T06:44:02.757 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:44:02.312+0000 7fd658e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T06:44:02.757 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:44:02.313+0000 7fd654800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T06:44:02.757 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:44:02.313+0000 7fd654800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T06:44:02.757 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:44:02.313+0000 7fd654800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T06:44:02.757 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:44:02.313+0000 7fd654800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T06:44:02.757 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:44:02.313+0000 7fd654800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T06:44:02.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/836464234' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-24T06:44:02.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: Active manager daemon x restarted 2024-04-24T06:44:02.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: Activating manager daemon x 2024-04-24T06:44:02.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: osdmap e89: 12 total, 12 up, 12 in 2024-04-24T06:44:02.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: mgrmap e22: x(active, starting, since 0.0276083s), standbys: y, z 2024-04-24T06:44:02.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T06:44:02.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T06:44:02.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T06:44:02.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T06:44:02.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T06:44:02.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T06:44:02.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T06:44:02.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T06:44:02.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T06:44:02.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T06:44:02.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T06:44:02.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T06:44:02.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T06:44:02.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T06:44:02.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T06:44:02.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T06:44:02.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T06:44:02.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T06:44:02.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T06:44:02.760 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T06:44:02.760 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T06:44:02.760 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T06:44:02.760 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T06:44:02.760 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T06:44:02.760 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T06:44:02.760 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T06:44:02.760 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T06:44:02.760 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T06:44:02.760 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T06:44:02.760 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T06:44:02.760 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T06:44:02.760 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T06:44:02.760 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T06:44:02.761 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: Manager daemon x is now available 2024-04-24T06:44:02.761 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:44:02.761 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T06:44:02.761 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T06:44:02.761 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T06:44:02.761 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:02 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T06:44:03.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.001+0000 7f5da3e00640 1 -- 172.21.15.5:0/3006385350 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mgrmap(e 23) ==== 138458+0+0 (secure 0 0 0) 0x7f5d9c03bd10 con 0x7f5dac101e70 2024-04-24T06:44:03.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.002+0000 7f5da3e00640 1 --2- 172.21.15.5:0/3006385350 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f5d900aa910 0x7f5d900acd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:03.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.002+0000 7f5da3e00640 1 -- 172.21.15.5:0/3006385350 --> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f5d9001dcb0 con 0x7f5d900aa910 2024-04-24T06:44:03.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.013+0000 7f5daaa00640 1 --2- 172.21.15.5:0/3006385350 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f5d900aa910 0x7f5d900acd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:03.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.017+0000 7f5daaa00640 1 --2- 172.21.15.5:0/3006385350 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f5d900aa910 0x7f5d900acd00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5dac075b20 tx=0x7f5d98005f90 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:03.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.046+0000 7f5da3e00640 1 -- 172.21.15.5:0/3006385350 <== mgr.25202 v2:172.21.15.5:6800/3154624651 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f5d9001dcb0 con 0x7f5d900aa910 2024-04-24T06:44:03.048 INFO:teuthology.orchestra.run.smithi005.stdout:{} 2024-04-24T06:44:03.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.049+0000 7f5db1200640 1 -- 172.21.15.5:0/3006385350 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f5d900aa910 msgr2=0x7f5d900acd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:03.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.049+0000 7f5db1200640 1 --2- 172.21.15.5:0/3006385350 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f5d900aa910 0x7f5d900acd00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5dac075b20 tx=0x7f5d98005f90 comp rx=0 tx=0).stop 2024-04-24T06:44:03.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.049+0000 7f5db1200640 1 -- 172.21.15.5:0/3006385350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5dac101e70 msgr2=0x7f5dac075db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:03.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.049+0000 7f5db1200640 1 --2- 172.21.15.5:0/3006385350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5dac101e70 0x7f5dac075db0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c031440 tx=0x7f5d9c032ce0 comp rx=0 tx=0).stop 2024-04-24T06:44:03.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.049+0000 7f5db1200640 1 -- 172.21.15.5:0/3006385350 shutdown_connections 2024-04-24T06:44:03.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.049+0000 7f5db1200640 1 --2- 172.21.15.5:0/3006385350 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f5d900aa910 0x7f5d900acd00 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:03.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.049+0000 7f5db1200640 1 --2- 172.21.15.5:0/3006385350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dac103a10 0x7f5dac074930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:03.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.049+0000 7f5db1200640 1 --2- 172.21.15.5:0/3006385350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5dac103050 0x7f5dac0743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:03.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.049+0000 7f5db1200640 1 --2- 172.21.15.5:0/3006385350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5dac101e70 0x7f5dac075db0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:03.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.049+0000 7f5db1200640 1 -- 172.21.15.5:0/3006385350 >> 172.21.15.5:0/3006385350 conn(0x7f5dac0fd430 msgr2=0x7f5dac1063e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:03.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.049+0000 7f5db1200640 1 -- 172.21.15.5:0/3006385350 shutdown_connections 2024-04-24T06:44:03.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.049+0000 7f5db1200640 1 -- 172.21.15.5:0/3006385350 wait complete. 2024-04-24T06:44:03.074 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-24T06:44:03.085 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-04-24T06:44:03.096 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:44:03.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.278+0000 7fac07e00640 1 -- 172.21.15.5:0/1090105831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac08105350 msgr2=0x7fac081057d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:03.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.278+0000 7fac07e00640 1 --2- 172.21.15.5:0/1090105831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac08105350 0x7fac081057d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fabf4009860 tx=0x7fabf402a340 comp rx=0 tx=0).stop 2024-04-24T06:44:03.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.279+0000 7fac07e00640 1 -- 172.21.15.5:0/1090105831 shutdown_connections 2024-04-24T06:44:03.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.279+0000 7fac07e00640 1 --2- 172.21.15.5:0/1090105831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac08105d10 0x7fac0810c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:03.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.279+0000 7fac07e00640 1 --2- 172.21.15.5:0/1090105831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac08105350 0x7fac081057d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:03.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.279+0000 7fac07e00640 1 --2- 172.21.15.5:0/1090105831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac08104160 0x7fac08104580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:03.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.279+0000 7fac07e00640 1 -- 172.21.15.5:0/1090105831 >> 172.21.15.5:0/1090105831 conn(0x7fac080ff6d0 msgr2=0x7fac08101b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:03.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.279+0000 7fac07e00640 1 -- 172.21.15.5:0/1090105831 shutdown_connections 2024-04-24T06:44:03.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.279+0000 7fac07e00640 1 -- 172.21.15.5:0/1090105831 wait complete. 2024-04-24T06:44:03.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.280+0000 7fac07e00640 1 Processor -- start 2024-04-24T06:44:03.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.280+0000 7fac07e00640 1 -- start start 2024-04-24T06:44:03.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.280+0000 7fac07e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac08104160 0x7fac0819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:03.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.280+0000 7fac07e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac08105350 0x7fac0819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:03.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.280+0000 7fac07e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac08105d10 0x7fac081a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:03.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.280+0000 7fac07e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fac0810fdf0 con 0x7fac08105350 2024-04-24T06:44:03.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.280+0000 7fac07e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fac0810fc70 con 0x7fac08104160 2024-04-24T06:44:03.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.280+0000 7fac07e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fac0810ff70 con 0x7fac08105d10 2024-04-24T06:44:03.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.281+0000 7fac06000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac08105350 0x7fac0819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:03.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.281+0000 7fac06a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac08104160 0x7fac0819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:03.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.281+0000 7fac06000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac08105350 0x7fac0819a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38734/0 (socket says 172.21.15.5:38734) 2024-04-24T06:44:03.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.281+0000 7fac06a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac08104160 0x7fac0819a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:59204/0 (socket says 172.21.15.5:59204) 2024-04-24T06:44:03.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.281+0000 7fac06000640 1 -- 172.21.15.5:0/222020223 learned_addr learned my addr 172.21.15.5:0/222020223 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:03.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.281+0000 7fac07400640 1 --2- 172.21.15.5:0/222020223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac08105d10 0x7fac081a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:03.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.281+0000 7fac06a00640 1 -- 172.21.15.5:0/222020223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac08105d10 msgr2=0x7fac081a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:03.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.281+0000 7fac06a00640 1 --2- 172.21.15.5:0/222020223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac08105d10 0x7fac081a1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:03.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.281+0000 7fac06a00640 1 -- 172.21.15.5:0/222020223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac08105350 msgr2=0x7fac0819a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:03.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.281+0000 7fac06a00640 1 --2- 172.21.15.5:0/222020223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac08105350 0x7fac0819a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:03.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.281+0000 7fac06a00640 1 -- 172.21.15.5:0/222020223 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac081a2090 con 0x7fac08104160 2024-04-24T06:44:03.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.281+0000 7fac06000640 1 --2- 172.21.15.5:0/222020223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac08105350 0x7fac0819a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:44:03.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.282+0000 7fac06a00640 1 --2- 172.21.15.5:0/222020223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac08104160 0x7fac0819a470 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fabf800e9c0 tx=0x7fabf800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:03.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.282+0000 7fabff400640 1 -- 172.21.15.5:0/222020223 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf800ccc0 con 0x7fac08104160 2024-04-24T06:44:03.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.282+0000 7fabff400640 1 -- 172.21.15.5:0/222020223 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabf80035f0 con 0x7fac08104160 2024-04-24T06:44:03.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.282+0000 7fabff400640 1 -- 172.21.15.5:0/222020223 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf800ce60 con 0x7fac08104160 2024-04-24T06:44:03.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.282+0000 7fac07e00640 1 -- 172.21.15.5:0/222020223 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac081a2380 con 0x7fac08104160 2024-04-24T06:44:03.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.282+0000 7fac07e00640 1 -- 172.21.15.5:0/222020223 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac081a2880 con 0x7fac08104160 2024-04-24T06:44:03.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.283+0000 7fac07e00640 1 -- 172.21.15.5:0/222020223 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac0810b480 con 0x7fac08104160 2024-04-24T06:44:03.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.285+0000 7fabff400640 1 -- 172.21.15.5:0/222020223 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 23) ==== 138458+0+0 (secure 0 0 0) 0x7fabf8010690 con 0x7fac08104160 2024-04-24T06:44:03.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.286+0000 7fabff400640 1 --2- 172.21.15.5:0/222020223 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fabec0a3990 0x7fabec0a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:03.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.286+0000 7fabff400640 1 -- 172.21.15.5:0/222020223 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fabf80d0620 con 0x7fac08104160 2024-04-24T06:44:03.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.286+0000 7fac06000640 1 --2- 172.21.15.5:0/222020223 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fabec0a3990 0x7fabec0a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:03.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.287+0000 7fac06000640 1 --2- 172.21.15.5:0/222020223 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fabec0a3990 0x7fabec0a5e50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fac0819b980 tx=0x7fabf4030550 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:03.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.290+0000 7fabff400640 1 -- 172.21.15.5:0/222020223 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fabf809d1a0 con 0x7fac08104160 2024-04-24T06:44:03.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:03 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/836464234' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-24T06:44:03.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:03 smithi005 ceph-mon[27798]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T06:44:03.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:03 smithi005 ceph-mon[27798]: mgrmap e23: x(active, since 1.03088s), standbys: y, z 2024-04-24T06:44:03.506 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:44:03 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:44:03.032+0000 7fd651600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T06:44:03.506 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:44:03 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:44:03.032+0000 7fd651600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T06:44:03.507 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:44:03 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:44:03.032+0000 7fd651600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T06:44:03.507 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:44:03 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:44:03.032+0000 7fd651600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T06:44:03.507 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:44:03 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:44:03.032+0000 7fd651600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T06:44:03.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.508+0000 7fac07e00640 1 -- 172.21.15.5:0/222020223 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fac08062c00 con 0x7fac08104160 2024-04-24T06:44:03.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.509+0000 7fabff400640 1 -- 172.21.15.5:0/222020223 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22884 (secure 0 0 0) 0x7fabf809caa0 con 0x7fac08104160 2024-04-24T06:44:03.511 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:44:03.511 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":89,"fsid":"43e9bad0-0204-11ef-bc93-c7b262605968","created":"2024-04-24T06:32:26.340555+0000","modified":"2024-04-24T06:44:01.965991+0000","last_up_change":"2024-04-24T06:40:36.790934+0000","last_in_change":"2024-04-24T06:40:17.803257+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:36:27.320342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:41:32.254502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:41:33.739067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c2e974c-745c-4dd5-8c3d-8e8da2eb1ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6803","nonce":2155858515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6805","nonce":2155858515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6809","nonce":2155858515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6807","nonce":2155858515}]},"public_addr":"172.21.15.5:6803/2155858515","cluster_addr":"172.21.15.5:6805/2155858515","heartbeat_back_addr":"172.21.15.5:6809/2155858515","heartbeat_front_addr":"172.21.15.5:6807/2155858515","state":["exists","up"]},{"osd":1,"uuid":"50467744-db20-4721-bf2d-1c5db938297b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6801","nonce":1302362650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6803","nonce":1302362650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6807","nonce":1302362650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6805","nonce":1302362650}]},"public_addr":"172.21.15.37:6801/1302362650","cluster_addr":"172.21.15.37:6803/1302362650","heartbeat_back_addr":"172.21.15.37:6807/1302362650","heartbeat_front_addr":"172.21.15.37:6805/1302362650","state":["exists","up"]},{"osd":2,"uuid":"0e190b15-d04c-4288-8d48-ecf6664392fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6801","nonce":2885027256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6803","nonce":2885027256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6807","nonce":2885027256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6805","nonce":2885027256}]},"public_addr":"172.21.15.137:6801/2885027256","cluster_addr":"172.21.15.137:6803/2885027256","heartbeat_back_addr":"172.21.15.137:6807/2885027256","heartbeat_front_addr":"172.21.15.137:6805/2885027256","state":["exists","up"]},{"osd":3,"uuid":"e05592ba-d96a-472a-9f63-09a535d04dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6811","nonce":3701747260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6813","nonce":3701747260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6817","nonce":3701747260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6815","nonce":3701747260}]},"public_addr":"172.21.15.5:6811/3701747260","cluster_addr":"172.21.15.5:6813/3701747260","heartbeat_back_addr":"172.21.15.5:6817/3701747260","heartbeat_front_addr":"172.21.15.5:6815/3701747260","state":["exists","up"]},{"osd":4,"uuid":"c3f36e51-7a3b-4dd9-af4b-a10558830c4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6809","nonce":759253844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6811","nonce":759253844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6815","nonce":759253844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6813","nonce":759253844}]},"public_addr":"172.21.15.37:6809/759253844","cluster_addr":"172.21.15.37:6811/759253844","heartbeat_back_addr":"172.21.15.37:6815/759253844","heartbeat_front_addr":"172.21.15.37:6813/759253844","state":["exists","up"]},{"osd":5,"uuid":"97a0500c-cbb8-4ff5-b476-8634b26b3821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6809","nonce":3771921560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6811","nonce":3771921560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6815","nonce":3771921560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6813","nonce":3771921560}]},"public_addr":"172.21.15.137:6809/3771921560","cluster_addr":"172.21.15.137:6811/3771921560","heartbeat_back_addr":"172.21.15.137:6815/3771921560","heartbeat_front_addr":"172.21.15.137:6813/3771921560","state":["exists","up"]},{"osd":6,"uuid":"33a33d62-bbb3-4ed9-834e-19ae93a97670","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6819","nonce":2064636084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6821","nonce":2064636084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6825","nonce":2064636084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6823","nonce":2064636084}]},"public_addr":"172.21.15.5:6819/2064636084","cluster_addr":"172.21.15.5:6821/2064636084","heartbeat_back_addr":"172.21.15.5:6825/2064636084","heartbeat_front_addr":"172.21.15.5:6823/2064636084","state":["exists","up"]},{"osd":7,"uuid":"c0f4e036-4717-4514-bb59-ab11fbaab063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6817","nonce":2952750659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6819","nonce":2952750659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6823","nonce":2952750659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6821","nonce":2952750659}]},"public_addr":"172.21.15.37:6817/2952750659","cluster_addr":"172.21.15.37:6819/2952750659","heartbeat_back_addr":"172.21.15.37:6823/2952750659","heartbeat_front_addr":"172.21.15.37:6821/2952750659","state":["exists","up"]},{"osd":8,"uuid":"5391fd87-44ea-440f-80da-c54d0327c04c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6817","nonce":29724945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6819","nonce":29724945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6823","nonce":29724945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6821","nonce":29724945}]},"public_addr":"172.21.15.137:6817/29724945","cluster_addr":"172.21.15.137:6819/29724945","heartbeat_back_addr":"172.21.15.137:6823/29724945","heartbeat_front_addr":"172.21.15.137:6821/29724945","state":["exists","up"]},{"osd":9,"uuid":"d94c9dbc-f43d-4237-ac7f-47a6a5ede1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6827","nonce":2257102107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6829","nonce":2257102107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6833","nonce":2257102107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6831","nonce":2257102107}]},"public_addr":"172.21.15.5:6827/2257102107","cluster_addr":"172.21.15.5:6829/2257102107","heartbeat_back_addr":"172.21.15.5:6833/2257102107","heartbeat_front_addr":"172.21.15.5:6831/2257102107","state":["exists","up"]},{"osd":10,"uuid":"e2455332-2a5e-44b1-85d3-8e47da399f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6825","nonce":2451570637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6827","nonce":2451570637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6831","nonce":2451570637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6829","nonce":2451570637}]},"public_addr":"172.21.15.37:6825/2451570637","cluster_addr":"172.21.15.37:6827/2451570637","heartbeat_back_addr":"172.21.15.37:6831/2451570637","heartbeat_front_addr":"172.21.15.37:6829/2451570637","state":["exists","up"]},{"osd":11,"uuid":"49d9396b-704a-4cef-8210-75d30c28e924","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6825","nonce":90600596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6827","nonce":90600596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6831","nonce":90600596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6829","nonce":90600596}]},"public_addr":"172.21.15.137:6825/90600596","cluster_addr":"172.21.15.137:6827/90600596","heartbeat_back_addr":"172.21.15.137:6831/90600596","heartbeat_front_addr":"172.21.15.137:6829/90600596","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:35:35.591619+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:00.568817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:24.195643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:56.451724+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:22.109293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:46.161749+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:19.278305+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:45.620256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:10.041165+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:43.813697+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:10.736026+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:35.384377+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6800/4042936965":"2024-04-25T06:44:01.965881+0000","172.21.15.5:0/2912680052":"2024-04-25T06:44:01.965881+0000","172.21.15.5:6801/4042936965":"2024-04-25T06:44:01.965881+0000","172.21.15.5:0/3478347847":"2024-04-25T06:44:01.965881+0000","172.21.15.5:0/23307164":"2024-04-25T06:33:41.161852+0000","172.21.15.5:0/1012004002":"2024-04-25T06:44:01.965881+0000","172.21.15.5:6801/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:0/230785941":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/805658012":"2024-04-25T06:44:01.965881+0000","172.21.15.5:0/2940023597":"2024-04-25T06:33:07.040369+0000","172.21.15.5:6801/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3886613904":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/4199165437":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3459593499":"2024-04-25T06:33:41.161852+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:44:03.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.512+0000 7fac07e00640 1 -- 172.21.15.5:0/222020223 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fabec0a3990 msgr2=0x7fabec0a5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:03.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.512+0000 7fac07e00640 1 --2- 172.21.15.5:0/222020223 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fabec0a3990 0x7fabec0a5e50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fac0819b980 tx=0x7fabf4030550 comp rx=0 tx=0).stop 2024-04-24T06:44:03.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.512+0000 7fac07e00640 1 -- 172.21.15.5:0/222020223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac08104160 msgr2=0x7fac0819a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:03.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.512+0000 7fac07e00640 1 --2- 172.21.15.5:0/222020223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac08104160 0x7fac0819a470 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fabf800e9c0 tx=0x7fabf800ee80 comp rx=0 tx=0).stop 2024-04-24T06:44:03.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.512+0000 7fac07e00640 1 -- 172.21.15.5:0/222020223 shutdown_connections 2024-04-24T06:44:03.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.512+0000 7fac07e00640 1 --2- 172.21.15.5:0/222020223 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fabec0a3990 0x7fabec0a5e50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:03.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.512+0000 7fac07e00640 1 --2- 172.21.15.5:0/222020223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac08105d10 0x7fac081a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:03.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.512+0000 7fac07e00640 1 --2- 172.21.15.5:0/222020223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac08105350 0x7fac0819a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:03.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.512+0000 7fac07e00640 1 --2- 172.21.15.5:0/222020223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac08104160 0x7fac0819a470 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:03.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.512+0000 7fac07e00640 1 -- 172.21.15.5:0/222020223 >> 172.21.15.5:0/222020223 conn(0x7fac080ff6d0 msgr2=0x7fac08108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:03.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.512+0000 7fac07e00640 1 -- 172.21.15.5:0/222020223 shutdown_connections 2024-04-24T06:44:03.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.512+0000 7fac07e00640 1 -- 172.21.15.5:0/222020223 wait complete. 2024-04-24T06:44:03.531 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:36:27.320342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:41:32.254502+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:41:33.739067+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:44:03.531 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:44:03.546 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:03 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/836464234' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-24T06:44:03.547 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:03 smithi037 ceph-mon[32864]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T06:44:03.547 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:03 smithi037 ceph-mon[32864]: mgrmap e23: x(active, since 1.03088s), standbys: y, z 2024-04-24T06:44:03.597 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:03 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/836464234' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-24T06:44:03.598 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:03 smithi137 ceph-mon[33017]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T06:44:03.598 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:03 smithi137 ceph-mon[33017]: mgrmap e23: x(active, since 1.03088s), standbys: y, z 2024-04-24T06:44:03.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.719+0000 7f01e2400640 1 -- 172.21.15.5:0/1369502765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01dc076040 msgr2=0x7f01dc074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:03.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.719+0000 7f01e2400640 1 --2- 172.21.15.5:0/1369502765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01dc076040 0x7f01dc074490 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f01cc009900 tx=0x7f01cc02a410 comp rx=0 tx=0).stop 2024-04-24T06:44:03.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.719+0000 7f01e2400640 1 -- 172.21.15.5:0/1369502765 shutdown_connections 2024-04-24T06:44:03.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.719+0000 7f01e2400640 1 --2- 172.21.15.5:0/1369502765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01dc109f10 0x7f01dc10c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:03.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.719+0000 7f01e2400640 1 --2- 172.21.15.5:0/1369502765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01dc074a60 0x7f01dc074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:03.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.719+0000 7f01e2400640 1 --2- 172.21.15.5:0/1369502765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01dc076040 0x7f01dc074490 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:03.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.719+0000 7f01e2400640 1 -- 172.21.15.5:0/1369502765 >> 172.21.15.5:0/1369502765 conn(0x7f01dc0fd5c0 msgr2=0x7f01dc0ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:03.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.719+0000 7f01e2400640 1 -- 172.21.15.5:0/1369502765 shutdown_connections 2024-04-24T06:44:03.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.719+0000 7f01e2400640 1 -- 172.21.15.5:0/1369502765 wait complete. 2024-04-24T06:44:03.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.720+0000 7f01e2400640 1 Processor -- start 2024-04-24T06:44:03.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.720+0000 7f01e2400640 1 -- start start 2024-04-24T06:44:03.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.721+0000 7f01e2400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01dc074a60 0x7f01dc19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:03.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.721+0000 7f01e2400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01dc076040 0x7f01dc19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:03.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.721+0000 7f01e2400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01dc109f10 0x7f01dc1a1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:03.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.721+0000 7f01e2400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc10fea0 con 0x7f01dc109f10 2024-04-24T06:44:03.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.721+0000 7f01e2400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc10fd20 con 0x7f01dc074a60 2024-04-24T06:44:03.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.721+0000 7f01e2400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc110020 con 0x7f01dc076040 2024-04-24T06:44:03.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.721+0000 7f01db400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01dc074a60 0x7f01dc19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:03.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.721+0000 7f01db400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01dc074a60 0x7f01dc19a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:59216/0 (socket says 172.21.15.5:59216) 2024-04-24T06:44:03.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.721+0000 7f01db400640 1 -- 172.21.15.5:0/3186013690 learned_addr learned my addr 172.21.15.5:0/3186013690 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:03.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.721+0000 7f01dbe00640 1 --2- 172.21.15.5:0/3186013690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01dc109f10 0x7f01dc1a1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:03.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.721+0000 7f01d3e00640 1 --2- 172.21.15.5:0/3186013690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01dc076040 0x7f01dc19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:03.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.721+0000 7f01db400640 1 -- 172.21.15.5:0/3186013690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01dc076040 msgr2=0x7f01dc19aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:03.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.721+0000 7f01db400640 1 --2- 172.21.15.5:0/3186013690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01dc076040 0x7f01dc19aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:03.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.721+0000 7f01db400640 1 -- 172.21.15.5:0/3186013690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01dc109f10 msgr2=0x7f01dc1a1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:03.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.721+0000 7f01db400640 1 --2- 172.21.15.5:0/3186013690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01dc109f10 0x7f01dc1a1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:03.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.722+0000 7f01db400640 1 -- 172.21.15.5:0/3186013690 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01dc1a21c0 con 0x7f01dc074a60 2024-04-24T06:44:03.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.722+0000 7f01db400640 1 --2- 172.21.15.5:0/3186013690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01dc074a60 0x7f01dc19a5a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f01cc009230 tx=0x7f01cc031b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:03.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.722+0000 7f01d8c00640 1 -- 172.21.15.5:0/3186013690 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01cc03f030 con 0x7f01dc074a60 2024-04-24T06:44:03.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.722+0000 7f01d8c00640 1 -- 172.21.15.5:0/3186013690 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01cc003ad0 con 0x7f01dc074a60 2024-04-24T06:44:03.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.722+0000 7f01d8c00640 1 -- 172.21.15.5:0/3186013690 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01cc035070 con 0x7f01dc074a60 2024-04-24T06:44:03.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.722+0000 7f01e2400640 1 -- 172.21.15.5:0/3186013690 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01dc1a2450 con 0x7f01dc074a60 2024-04-24T06:44:03.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.723+0000 7f01e2400640 1 -- 172.21.15.5:0/3186013690 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01dc1a2940 con 0x7f01dc074a60 2024-04-24T06:44:03.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.724+0000 7f01e2400640 1 -- 172.21.15.5:0/3186013690 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01dc108c40 con 0x7f01dc074a60 2024-04-24T06:44:03.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.727+0000 7f01d8c00640 1 -- 172.21.15.5:0/3186013690 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 23) ==== 138458+0+0 (secure 0 0 0) 0x7f01cc042030 con 0x7f01dc074a60 2024-04-24T06:44:03.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.728+0000 7f01d8c00640 1 --2- 172.21.15.5:0/3186013690 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f01c00a39b0 0x7f01c00a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:03.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.728+0000 7f01d8c00640 1 -- 172.21.15.5:0/3186013690 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f01cc0f0620 con 0x7f01dc074a60 2024-04-24T06:44:03.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.729+0000 7f01d3e00640 1 --2- 172.21.15.5:0/3186013690 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f01c00a39b0 0x7f01c00a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:03.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.729+0000 7f01d3e00640 1 --2- 172.21.15.5:0/3186013690 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f01c00a39b0 0x7f01c00a5e70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f01dc19bab0 tx=0x7f01c400b410 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:03.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.733+0000 7f01d8c00640 1 -- 172.21.15.5:0/3186013690 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f01cc0bcb50 con 0x7f01dc074a60 2024-04-24T06:44:03.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.930+0000 7f01e2400640 1 -- 172.21.15.5:0/3186013690 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f01dc062c00 con 0x7f01dc074a60 2024-04-24T06:44:03.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.931+0000 7f01d8c00640 1 -- 172.21.15.5:0/3186013690 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f01cc0c0e50 con 0x7f01dc074a60 2024-04-24T06:44:03.932 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-24T06:44:03.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.933+0000 7f01e2400640 1 -- 172.21.15.5:0/3186013690 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f01c00a39b0 msgr2=0x7f01c00a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:03.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.933+0000 7f01e2400640 1 --2- 172.21.15.5:0/3186013690 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f01c00a39b0 0x7f01c00a5e70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f01dc19bab0 tx=0x7f01c400b410 comp rx=0 tx=0).stop 2024-04-24T06:44:03.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.933+0000 7f01e2400640 1 -- 172.21.15.5:0/3186013690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01dc074a60 msgr2=0x7f01dc19a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:03.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.933+0000 7f01e2400640 1 --2- 172.21.15.5:0/3186013690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01dc074a60 0x7f01dc19a5a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f01cc009230 tx=0x7f01cc031b60 comp rx=0 tx=0).stop 2024-04-24T06:44:03.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.933+0000 7f01e2400640 1 -- 172.21.15.5:0/3186013690 shutdown_connections 2024-04-24T06:44:03.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.933+0000 7f01e2400640 1 --2- 172.21.15.5:0/3186013690 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f01c00a39b0 0x7f01c00a5e70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:03.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.933+0000 7f01e2400640 1 --2- 172.21.15.5:0/3186013690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01dc109f10 0x7f01dc1a1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:03.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.934+0000 7f01e2400640 1 --2- 172.21.15.5:0/3186013690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01dc076040 0x7f01dc19aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:03.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.934+0000 7f01e2400640 1 --2- 172.21.15.5:0/3186013690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01dc074a60 0x7f01dc19a5a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:03.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.934+0000 7f01e2400640 1 -- 172.21.15.5:0/3186013690 >> 172.21.15.5:0/3186013690 conn(0x7f01dc0fd5c0 msgr2=0x7f01dc105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:03.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.934+0000 7f01e2400640 1 -- 172.21.15.5:0/3186013690 shutdown_connections 2024-04-24T06:44:03.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:03.934+0000 7f01e2400640 1 -- 172.21.15.5:0/3186013690 wait complete. 2024-04-24T06:44:03.952 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:44:04.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.144+0000 7f9eadc00640 1 -- 172.21.15.5:0/3695805181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ea8102090 msgr2=0x7f9ea8102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:04.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.144+0000 7f9eadc00640 1 --2- 172.21.15.5:0/3695805181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ea8102090 0x7f9ea8102510 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9e90009900 tx=0x7f9e9002a410 comp rx=0 tx=0).stop 2024-04-24T06:44:04.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.144+0000 7f9eadc00640 1 -- 172.21.15.5:0/3695805181 shutdown_connections 2024-04-24T06:44:04.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.144+0000 7f9eadc00640 1 --2- 172.21.15.5:0/3695805181 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ea8102a50 0x7f9ea8109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:04.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.144+0000 7f9eadc00640 1 --2- 172.21.15.5:0/3695805181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ea8102090 0x7f9ea8102510 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:04.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.144+0000 7f9eadc00640 1 --2- 172.21.15.5:0/3695805181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ea8100f40 0x7f9ea8101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:04.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.144+0000 7f9eadc00640 1 -- 172.21.15.5:0/3695805181 >> 172.21.15.5:0/3695805181 conn(0x7f9ea80fc4d0 msgr2=0x7f9ea80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:04.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.144+0000 7f9eadc00640 1 -- 172.21.15.5:0/3695805181 shutdown_connections 2024-04-24T06:44:04.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.144+0000 7f9eadc00640 1 -- 172.21.15.5:0/3695805181 wait complete. 2024-04-24T06:44:04.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.145+0000 7f9eadc00640 1 Processor -- start 2024-04-24T06:44:04.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.145+0000 7f9eadc00640 1 -- start start 2024-04-24T06:44:04.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.146+0000 7f9eadc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ea8100f40 0x7f9ea819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:04.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.146+0000 7f9eadc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ea8102090 0x7f9ea819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:04.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.146+0000 7f9eadc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ea8102a50 0x7f9ea81a1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:04.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.146+0000 7f9eadc00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea810cbf0 con 0x7f9ea8102a50 2024-04-24T06:44:04.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.146+0000 7f9eadc00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea810ca70 con 0x7f9ea8102090 2024-04-24T06:44:04.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.146+0000 7f9eadc00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea810cd70 con 0x7f9ea8100f40 2024-04-24T06:44:04.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.146+0000 7f9ea6a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ea8100f40 0x7f9ea819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:04.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.146+0000 7f9e9fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ea8102090 0x7f9ea819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:04.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.146+0000 7f9ea6a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ea8100f40 0x7f9ea819a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:54140/0 (socket says 172.21.15.5:54140) 2024-04-24T06:44:04.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.146+0000 7f9ea6a00640 1 -- 172.21.15.5:0/2251351959 learned_addr learned my addr 172.21.15.5:0/2251351959 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:04.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.146+0000 7f9ea7400640 1 --2- 172.21.15.5:0/2251351959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ea8102a50 0x7f9ea81a1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:04.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.146+0000 7f9ea6a00640 1 -- 172.21.15.5:0/2251351959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ea8102090 msgr2=0x7f9ea819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:04.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.146+0000 7f9ea6a00640 1 --2- 172.21.15.5:0/2251351959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ea8102090 0x7f9ea819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:04.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.146+0000 7f9ea6a00640 1 -- 172.21.15.5:0/2251351959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ea8102a50 msgr2=0x7f9ea81a1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:04.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.146+0000 7f9ea6a00640 1 --2- 172.21.15.5:0/2251351959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ea8102a50 0x7f9ea81a1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:04.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.146+0000 7f9ea6a00640 1 -- 172.21.15.5:0/2251351959 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea81a2210 con 0x7f9ea8100f40 2024-04-24T06:44:04.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.146+0000 7f9ea7400640 1 --2- 172.21.15.5:0/2251351959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ea8102a50 0x7f9ea81a1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:44:04.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.147+0000 7f9ea6a00640 1 --2- 172.21.15.5:0/2251351959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ea8100f40 0x7f9ea819a5f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9e9800ea60 tx=0x7f9e9800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:04.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.147+0000 7f9e9f400640 1 -- 172.21.15.5:0/2251351959 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9800cde0 con 0x7f9ea8100f40 2024-04-24T06:44:04.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.147+0000 7f9eadc00640 1 -- 172.21.15.5:0/2251351959 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ea81a2500 con 0x7f9ea8100f40 2024-04-24T06:44:04.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.147+0000 7f9e9f400640 1 -- 172.21.15.5:0/2251351959 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e980035f0 con 0x7f9ea8100f40 2024-04-24T06:44:04.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.147+0000 7f9e9f400640 1 -- 172.21.15.5:0/2251351959 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e98010670 con 0x7f9ea8100f40 2024-04-24T06:44:04.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.147+0000 7f9eadc00640 1 -- 172.21.15.5:0/2251351959 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ea81a2a20 con 0x7f9ea8100f40 2024-04-24T06:44:04.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.148+0000 7f9eadc00640 1 -- 172.21.15.5:0/2251351959 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ea81a2ce0 con 0x7f9ea8100f40 2024-04-24T06:44:04.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.150+0000 7f9e9f400640 1 -- 172.21.15.5:0/2251351959 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 23) ==== 138458+0+0 (secure 0 0 0) 0x7f9e980037a0 con 0x7f9ea8100f40 2024-04-24T06:44:04.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.152+0000 7f9e9f400640 1 --2- 172.21.15.5:0/2251351959 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f9e8c0a39b0 0x7f9e8c0a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:04.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.152+0000 7f9e9f400640 1 -- 172.21.15.5:0/2251351959 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f9e980d04b0 con 0x7f9ea8100f40 2024-04-24T06:44:04.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.152+0000 7f9e9fe00640 1 --2- 172.21.15.5:0/2251351959 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f9e8c0a39b0 0x7f9e8c0a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:04.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.153+0000 7f9e9fe00640 1 --2- 172.21.15.5:0/2251351959 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f9e8c0a39b0 0x7f9e8c0a5e70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9e90005ab0 tx=0x7f9e90000bc0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:04.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.157+0000 7f9e9f400640 1 -- 172.21.15.5:0/2251351959 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9e9809d030 con 0x7f9ea8100f40 2024-04-24T06:44:04.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.375+0000 7f9e9f400640 1 -- 172.21.15.5:0/2251351959 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mgrmap(e 24) ==== 138816+0+0 (secure 0 0 0) 0x7f9e98098fa0 con 0x7f9ea8100f40 2024-04-24T06:44:04.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.390+0000 7f9eadc00640 1 -- 172.21.15.5:0/2251351959 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9ea81a2eb0 con 0x7f9ea8100f40 2024-04-24T06:44:04.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.390+0000 7f9e9f400640 1 -- 172.21.15.5:0/2251351959 <== mon.1 v2:172.21.15.137:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f9e98096a00 con 0x7f9ea8100f40 2024-04-24T06:44:04.391 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-24T06:44:04.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.393+0000 7f9eadc00640 1 -- 172.21.15.5:0/2251351959 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f9e8c0a39b0 msgr2=0x7f9e8c0a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:04.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.393+0000 7f9eadc00640 1 --2- 172.21.15.5:0/2251351959 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f9e8c0a39b0 0x7f9e8c0a5e70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9e90005ab0 tx=0x7f9e90000bc0 comp rx=0 tx=0).stop 2024-04-24T06:44:04.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.393+0000 7f9eadc00640 1 -- 172.21.15.5:0/2251351959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ea8100f40 msgr2=0x7f9ea819a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:04.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.393+0000 7f9eadc00640 1 --2- 172.21.15.5:0/2251351959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ea8100f40 0x7f9ea819a5f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9e9800ea60 tx=0x7f9e9800ef20 comp rx=0 tx=0).stop 2024-04-24T06:44:04.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.393+0000 7f9eadc00640 1 -- 172.21.15.5:0/2251351959 shutdown_connections 2024-04-24T06:44:04.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.393+0000 7f9eadc00640 1 --2- 172.21.15.5:0/2251351959 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f9e8c0a39b0 0x7f9e8c0a5e70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:04.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.393+0000 7f9eadc00640 1 --2- 172.21.15.5:0/2251351959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ea8102a50 0x7f9ea81a1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:04.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.393+0000 7f9eadc00640 1 --2- 172.21.15.5:0/2251351959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ea8102090 0x7f9ea819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.393+0000 7f9eadc00640 1 --2- 172.21.15.5:0/2251351959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ea8100f40 0x7f9ea819a5f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.393+0000 7f9eadc00640 1 -- 172.21.15.5:0/2251351959 >> 172.21.15.5:0/2251351959 conn(0x7f9ea80fc4d0 msgr2=0x7f9ea80fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.393+0000 7f9eadc00640 1 -- 172.21.15.5:0/2251351959 shutdown_connections 2024-04-24T06:44:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.393+0000 7f9eadc00640 1 -- 172.21.15.5:0/2251351959 wait complete. 2024-04-24T06:44:04.411 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:44:04.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.604+0000 7fb6cd200640 1 -- 172.21.15.5:0/4257191144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6c8075fc0 msgr2=0x7fb6c8074410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:04.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.604+0000 7fb6cd200640 1 --2- 172.21.15.5:0/4257191144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6c8075fc0 0x7fb6c8074410 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8009ca0 tx=0x7fb6b802a950 comp rx=0 tx=0).stop 2024-04-24T06:44:04.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.604+0000 7fb6cd200640 1 -- 172.21.15.5:0/4257191144 shutdown_connections 2024-04-24T06:44:04.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.604+0000 7fb6cd200640 1 --2- 172.21.15.5:0/4257191144 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6c8109d40 0x7fb6c810c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:04.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.604+0000 7fb6cd200640 1 --2- 172.21.15.5:0/4257191144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6c80749e0 0x7fb6c8074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:04.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.604+0000 7fb6cd200640 1 --2- 172.21.15.5:0/4257191144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6c8075fc0 0x7fb6c8074410 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:04.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.604+0000 7fb6cd200640 1 -- 172.21.15.5:0/4257191144 >> 172.21.15.5:0/4257191144 conn(0x7fb6c80fd400 msgr2=0x7fb6c80ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:04.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.604+0000 7fb6cd200640 1 -- 172.21.15.5:0/4257191144 shutdown_connections 2024-04-24T06:44:04.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.604+0000 7fb6cd200640 1 -- 172.21.15.5:0/4257191144 wait complete. 2024-04-24T06:44:04.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.605+0000 7fb6cd200640 1 Processor -- start 2024-04-24T06:44:04.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.606+0000 7fb6cd200640 1 -- start start 2024-04-24T06:44:04.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.606+0000 7fb6cd200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6c80749e0 0x7fb6c8196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:04.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.606+0000 7fb6cd200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6c8075fc0 0x7fb6c8196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:04.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.606+0000 7fb6cd200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6c8109d40 0x7fb6c819d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:04.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.606+0000 7fb6cd200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c810fd80 con 0x7fb6c8075fc0 2024-04-24T06:44:04.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.606+0000 7fb6cd200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c810fc00 con 0x7fb6c8109d40 2024-04-24T06:44:04.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.606+0000 7fb6cd200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c810ff00 con 0x7fb6c80749e0 2024-04-24T06:44:04.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.606+0000 7fb6c7400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6c80749e0 0x7fb6c8196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:04.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.606+0000 7fb6c6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6c8075fc0 0x7fb6c8196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:04.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.606+0000 7fb6c7400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6c80749e0 0x7fb6c8196000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:54160/0 (socket says 172.21.15.5:54160) 2024-04-24T06:44:04.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.606+0000 7fb6c7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6c8109d40 0x7fb6c819d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:04.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.606+0000 7fb6c7400640 1 -- 172.21.15.5:0/1447151142 learned_addr learned my addr 172.21.15.5:0/1447151142 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:04.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.607+0000 7fb6c7400640 1 -- 172.21.15.5:0/1447151142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6c8109d40 msgr2=0x7fb6c819d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:04.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.607+0000 7fb6c7400640 1 --2- 172.21.15.5:0/1447151142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6c8109d40 0x7fb6c819d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:04.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.607+0000 7fb6c7400640 1 -- 172.21.15.5:0/1447151142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6c8075fc0 msgr2=0x7fb6c8196540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:04.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.607+0000 7fb6c7400640 1 --2- 172.21.15.5:0/1447151142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6c8075fc0 0x7fb6c8196540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:04.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.607+0000 7fb6c7400640 1 -- 172.21.15.5:0/1447151142 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c81aec60 con 0x7fb6c80749e0 2024-04-24T06:44:04.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.607+0000 7fb6c7e00640 1 --2- 172.21.15.5:0/1447151142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6c8109d40 0x7fb6c819d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:44:04.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.607+0000 7fb6c7400640 1 --2- 172.21.15.5:0/1447151142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6c80749e0 0x7fb6c8196000 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb6b802f870 tx=0x7fb6b802fe30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:04.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.607+0000 7fb6bfe00640 1 -- 172.21.15.5:0/1447151142 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8034040 con 0x7fb6c80749e0 2024-04-24T06:44:04.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.607+0000 7fb6bfe00640 1 -- 172.21.15.5:0/1447151142 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6b8030920 con 0x7fb6c80749e0 2024-04-24T06:44:04.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.607+0000 7fb6bfe00640 1 -- 172.21.15.5:0/1447151142 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8045480 con 0x7fb6c80749e0 2024-04-24T06:44:04.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.607+0000 7fb6cd200640 1 -- 172.21.15.5:0/1447151142 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6c81aee30 con 0x7fb6c80749e0 2024-04-24T06:44:04.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.608+0000 7fb6cd200640 1 -- 172.21.15.5:0/1447151142 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6c81af240 con 0x7fb6c80749e0 2024-04-24T06:44:04.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.609+0000 7fb6cd200640 1 -- 172.21.15.5:0/1447151142 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6c81046f0 con 0x7fb6c80749e0 2024-04-24T06:44:04.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.611+0000 7fb6bfe00640 1 -- 172.21.15.5:0/1447151142 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 24) ==== 138816+0+0 (secure 0 0 0) 0x7fb6b8037020 con 0x7fb6c80749e0 2024-04-24T06:44:04.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.612+0000 7fb6bfe00640 1 --2- 172.21.15.5:0/1447151142 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb6ac0a3e30 0x7fb6ac0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:04.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.612+0000 7fb6bfe00640 1 -- 172.21.15.5:0/1447151142 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fb6b80f1750 con 0x7fb6c80749e0 2024-04-24T06:44:04.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.612+0000 7fb6c6a00640 1 --2- 172.21.15.5:0/1447151142 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb6ac0a3e30 0x7fb6ac0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:04.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.613+0000 7fb6c6a00640 1 --2- 172.21.15.5:0/1447151142 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb6ac0a3e30 0x7fb6ac0a62f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8197470 tx=0x7fb6b000b410 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:04.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.616+0000 7fb6bfe00640 1 -- 172.21.15.5:0/1447151142 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb6b80bdd40 con 0x7fb6c80749e0 2024-04-24T06:44:04.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:04 smithi005 ceph-mon[27798]: [24/Apr/2024:06:44:02] ENGINE Bus STARTING 2024-04-24T06:44:04.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:04 smithi005 ceph-mon[27798]: [24/Apr/2024:06:44:02] ENGINE Serving on https://172.21.15.5:7150 2024-04-24T06:44:04.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:04 smithi005 ceph-mon[27798]: [24/Apr/2024:06:44:03] ENGINE Serving on http://172.21.15.5:8765 2024-04-24T06:44:04.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:04 smithi005 ceph-mon[27798]: [24/Apr/2024:06:44:03] ENGINE Bus STARTED 2024-04-24T06:44:04.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:04 smithi005 ceph-mon[27798]: from='client.25285 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:44:04.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:04 smithi005 ceph-mon[27798]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:44:04.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:04 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:04.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:04 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:04.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:04 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/222020223' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:44:04.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:04 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/3186013690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T06:44:04.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:04 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:04.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:04 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:04.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:04 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:04.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:04 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:04.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:04 smithi137 ceph-mon[33017]: [24/Apr/2024:06:44:02] ENGINE Bus STARTING 2024-04-24T06:44:04.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:04 smithi137 ceph-mon[33017]: [24/Apr/2024:06:44:02] ENGINE Serving on https://172.21.15.5:7150 2024-04-24T06:44:04.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:04 smithi137 ceph-mon[33017]: [24/Apr/2024:06:44:03] ENGINE Serving on http://172.21.15.5:8765 2024-04-24T06:44:04.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:04 smithi137 ceph-mon[33017]: [24/Apr/2024:06:44:03] ENGINE Bus STARTED 2024-04-24T06:44:04.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:04 smithi137 ceph-mon[33017]: from='client.25285 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:44:04.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:04 smithi137 ceph-mon[33017]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:44:04.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:04 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:04.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:04 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:04.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:04 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/222020223' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:44:04.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:04 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/3186013690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T06:44:04.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:04 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:04.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:04 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:04.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:04 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:04.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:04 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:04.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:04 smithi037 ceph-mon[32864]: [24/Apr/2024:06:44:02] ENGINE Bus STARTING 2024-04-24T06:44:04.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:04 smithi037 ceph-mon[32864]: [24/Apr/2024:06:44:02] ENGINE Serving on https://172.21.15.5:7150 2024-04-24T06:44:04.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:04 smithi037 ceph-mon[32864]: [24/Apr/2024:06:44:03] ENGINE Serving on http://172.21.15.5:8765 2024-04-24T06:44:04.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:04 smithi037 ceph-mon[32864]: [24/Apr/2024:06:44:03] ENGINE Bus STARTED 2024-04-24T06:44:04.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:04 smithi037 ceph-mon[32864]: from='client.25285 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:44:04.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:04 smithi037 ceph-mon[32864]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:44:04.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:04 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:04.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:04 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:04.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:04 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/222020223' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:44:04.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:04 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/3186013690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T06:44:04.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:04 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:04.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:04 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:04.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:04 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:04.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:04 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:04.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.832+0000 7fb6cd200640 1 -- 172.21.15.5:0/1447151142 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb6c802d870 con 0x7fb6c80749e0 2024-04-24T06:44:04.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.832+0000 7fb6bfe00640 1 -- 172.21.15.5:0/1447151142 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7fb6b80b9070 con 0x7fb6c80749e0 2024-04-24T06:44:04.833 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-24T06:44:04.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.835+0000 7fb6cd200640 1 -- 172.21.15.5:0/1447151142 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb6ac0a3e30 msgr2=0x7fb6ac0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:04.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.835+0000 7fb6cd200640 1 --2- 172.21.15.5:0/1447151142 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb6ac0a3e30 0x7fb6ac0a62f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8197470 tx=0x7fb6b000b410 comp rx=0 tx=0).stop 2024-04-24T06:44:04.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.835+0000 7fb6cd200640 1 -- 172.21.15.5:0/1447151142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6c80749e0 msgr2=0x7fb6c8196000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:04.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.835+0000 7fb6cd200640 1 --2- 172.21.15.5:0/1447151142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6c80749e0 0x7fb6c8196000 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb6b802f870 tx=0x7fb6b802fe30 comp rx=0 tx=0).stop 2024-04-24T06:44:04.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.835+0000 7fb6cd200640 1 -- 172.21.15.5:0/1447151142 shutdown_connections 2024-04-24T06:44:04.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.835+0000 7fb6cd200640 1 --2- 172.21.15.5:0/1447151142 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb6ac0a3e30 0x7fb6ac0a62f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:04.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.835+0000 7fb6cd200640 1 --2- 172.21.15.5:0/1447151142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6c8109d40 0x7fb6c819d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:04.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.835+0000 7fb6cd200640 1 --2- 172.21.15.5:0/1447151142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6c8075fc0 0x7fb6c8196540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:04.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.835+0000 7fb6cd200640 1 --2- 172.21.15.5:0/1447151142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6c80749e0 0x7fb6c8196000 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:04.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.835+0000 7fb6cd200640 1 -- 172.21.15.5:0/1447151142 >> 172.21.15.5:0/1447151142 conn(0x7fb6c80fd400 msgr2=0x7fb6c81018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:04.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.835+0000 7fb6cd200640 1 -- 172.21.15.5:0/1447151142 shutdown_connections 2024-04-24T06:44:04.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:04.835+0000 7fb6cd200640 1 -- 172.21.15.5:0/1447151142 wait complete. 2024-04-24T06:44:04.855 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:44:05.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.053+0000 7f318dc00640 1 -- 172.21.15.5:0/3703561982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31881085a0 msgr2=0x7f31881089c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:05.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.053+0000 7f318dc00640 1 --2- 172.21.15.5:0/3703561982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31881085a0 0x7f31881089c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3178009900 tx=0x7f317802a3f0 comp rx=0 tx=0).stop 2024-04-24T06:44:05.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.053+0000 7f318dc00640 1 -- 172.21.15.5:0/3703561982 shutdown_connections 2024-04-24T06:44:05.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.053+0000 7f318dc00640 1 --2- 172.21.15.5:0/3703561982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f318810a100 0x7f3188110990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:05.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.053+0000 7f318dc00640 1 --2- 172.21.15.5:0/3703561982 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3188109740 0x7f3188109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:05.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.053+0000 7f318dc00640 1 --2- 172.21.15.5:0/3703561982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31881085a0 0x7f31881089c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:05.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.053+0000 7f318dc00640 1 -- 172.21.15.5:0/3703561982 >> 172.21.15.5:0/3703561982 conn(0x7f31880774f0 msgr2=0x7f3188079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:05.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.053+0000 7f318dc00640 1 -- 172.21.15.5:0/3703561982 shutdown_connections 2024-04-24T06:44:05.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.053+0000 7f318dc00640 1 -- 172.21.15.5:0/3703561982 wait complete. 2024-04-24T06:44:05.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.054+0000 7f318dc00640 1 Processor -- start 2024-04-24T06:44:05.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.054+0000 7f318dc00640 1 -- start start 2024-04-24T06:44:05.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.054+0000 7f318dc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31881085a0 0x7f318819e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:05.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.054+0000 7f318dc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3188109740 0x7f318819ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:05.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.054+0000 7f318dc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f318810a100 0x7f31881a5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:05.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.054+0000 7f318dc00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3188114230 con 0x7f3188109740 2024-04-24T06:44:05.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.054+0000 7f318dc00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f31881140b0 con 0x7f31881085a0 2024-04-24T06:44:05.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.054+0000 7f318dc00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f31881143b0 con 0x7f318810a100 2024-04-24T06:44:05.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.055+0000 7f3187e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31881085a0 0x7f318819e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:05.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.055+0000 7f3187400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3188109740 0x7f318819ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:05.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.055+0000 7f3187e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31881085a0 0x7f318819e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:46032/0 (socket says 172.21.15.5:46032) 2024-04-24T06:44:05.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.055+0000 7f3187e00640 1 -- 172.21.15.5:0/1015201619 learned_addr learned my addr 172.21.15.5:0/1015201619 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:05.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.055+0000 7f318d200640 1 --2- 172.21.15.5:0/1015201619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f318810a100 0x7f31881a5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:05.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.055+0000 7f3187e00640 1 -- 172.21.15.5:0/1015201619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f318810a100 msgr2=0x7f31881a5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:05.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.055+0000 7f3187e00640 1 --2- 172.21.15.5:0/1015201619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f318810a100 0x7f31881a5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:05.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.055+0000 7f3187e00640 1 -- 172.21.15.5:0/1015201619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3188109740 msgr2=0x7f318819ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:05.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.055+0000 7f3187e00640 1 --2- 172.21.15.5:0/1015201619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3188109740 0x7f318819ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:05.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.055+0000 7f3187e00640 1 -- 172.21.15.5:0/1015201619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31881a6470 con 0x7f31881085a0 2024-04-24T06:44:05.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.055+0000 7f3187e00640 1 --2- 172.21.15.5:0/1015201619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31881085a0 0x7f318819e850 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3178031440 tx=0x7f3178032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:05.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.056+0000 7f3184c00640 1 -- 172.21.15.5:0/1015201619 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317803f030 con 0x7f31881085a0 2024-04-24T06:44:05.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.056+0000 7f3184c00640 1 -- 172.21.15.5:0/1015201619 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3178003ce0 con 0x7f31881085a0 2024-04-24T06:44:05.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.056+0000 7f318dc00640 1 -- 172.21.15.5:0/1015201619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31881a6700 con 0x7f31881085a0 2024-04-24T06:44:05.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.056+0000 7f3184c00640 1 -- 172.21.15.5:0/1015201619 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3178035070 con 0x7f31881085a0 2024-04-24T06:44:05.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.056+0000 7f318dc00640 1 -- 172.21.15.5:0/1015201619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31881a6c40 con 0x7f31881085a0 2024-04-24T06:44:05.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.057+0000 7f318dc00640 1 -- 172.21.15.5:0/1015201619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f318810f870 con 0x7f31881085a0 2024-04-24T06:44:05.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.059+0000 7f3184c00640 1 -- 172.21.15.5:0/1015201619 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138816+0+0 (secure 0 0 0) 0x7f3178042030 con 0x7f31881085a0 2024-04-24T06:44:05.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.060+0000 7f3184c00640 1 --2- 172.21.15.5:0/1015201619 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f316c0a3e20 0x7f316c0a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:05.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.060+0000 7f3184c00640 1 -- 172.21.15.5:0/1015201619 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f31780f0e70 con 0x7f31881085a0 2024-04-24T06:44:05.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.060+0000 7f3187400640 1 --2- 172.21.15.5:0/1015201619 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f316c0a3e20 0x7f316c0a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:05.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.061+0000 7f3187400640 1 --2- 172.21.15.5:0/1015201619 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f316c0a3e20 0x7f316c0a62e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f318819fd60 tx=0x7f3174005f90 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:05.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.064+0000 7f3184c00640 1 -- 172.21.15.5:0/1015201619 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f31780bcd00 con 0x7f31881085a0 2024-04-24T06:44:05.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.286+0000 7f318dc00640 1 -- 172.21.15.5:0/1015201619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3188062c00 con 0x7f31881085a0 2024-04-24T06:44:05.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.288+0000 7f3184c00640 1 -- 172.21.15.5:0/1015201619 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22884 (secure 0 0 0) 0x7f31780c1000 con 0x7f31881085a0 2024-04-24T06:44:05.289 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:44:05.290 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":89,"fsid":"43e9bad0-0204-11ef-bc93-c7b262605968","created":"2024-04-24T06:32:26.340555+0000","modified":"2024-04-24T06:44:01.965991+0000","last_up_change":"2024-04-24T06:40:36.790934+0000","last_in_change":"2024-04-24T06:40:17.803257+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:36:27.320342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:41:32.254502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:41:33.739067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c2e974c-745c-4dd5-8c3d-8e8da2eb1ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6803","nonce":2155858515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6805","nonce":2155858515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6809","nonce":2155858515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6807","nonce":2155858515}]},"public_addr":"172.21.15.5:6803/2155858515","cluster_addr":"172.21.15.5:6805/2155858515","heartbeat_back_addr":"172.21.15.5:6809/2155858515","heartbeat_front_addr":"172.21.15.5:6807/2155858515","state":["exists","up"]},{"osd":1,"uuid":"50467744-db20-4721-bf2d-1c5db938297b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6801","nonce":1302362650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6803","nonce":1302362650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6807","nonce":1302362650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6805","nonce":1302362650}]},"public_addr":"172.21.15.37:6801/1302362650","cluster_addr":"172.21.15.37:6803/1302362650","heartbeat_back_addr":"172.21.15.37:6807/1302362650","heartbeat_front_addr":"172.21.15.37:6805/1302362650","state":["exists","up"]},{"osd":2,"uuid":"0e190b15-d04c-4288-8d48-ecf6664392fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6801","nonce":2885027256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6803","nonce":2885027256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6807","nonce":2885027256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6805","nonce":2885027256}]},"public_addr":"172.21.15.137:6801/2885027256","cluster_addr":"172.21.15.137:6803/2885027256","heartbeat_back_addr":"172.21.15.137:6807/2885027256","heartbeat_front_addr":"172.21.15.137:6805/2885027256","state":["exists","up"]},{"osd":3,"uuid":"e05592ba-d96a-472a-9f63-09a535d04dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6811","nonce":3701747260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6813","nonce":3701747260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6817","nonce":3701747260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6815","nonce":3701747260}]},"public_addr":"172.21.15.5:6811/3701747260","cluster_addr":"172.21.15.5:6813/3701747260","heartbeat_back_addr":"172.21.15.5:6817/3701747260","heartbeat_front_addr":"172.21.15.5:6815/3701747260","state":["exists","up"]},{"osd":4,"uuid":"c3f36e51-7a3b-4dd9-af4b-a10558830c4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6809","nonce":759253844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6811","nonce":759253844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6815","nonce":759253844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6813","nonce":759253844}]},"public_addr":"172.21.15.37:6809/759253844","cluster_addr":"172.21.15.37:6811/759253844","heartbeat_back_addr":"172.21.15.37:6815/759253844","heartbeat_front_addr":"172.21.15.37:6813/759253844","state":["exists","up"]},{"osd":5,"uuid":"97a0500c-cbb8-4ff5-b476-8634b26b3821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6809","nonce":3771921560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6811","nonce":3771921560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6815","nonce":3771921560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6813","nonce":3771921560}]},"public_addr":"172.21.15.137:6809/3771921560","cluster_addr":"172.21.15.137:6811/3771921560","heartbeat_back_addr":"172.21.15.137:6815/3771921560","heartbeat_front_addr":"172.21.15.137:6813/3771921560","state":["exists","up"]},{"osd":6,"uuid":"33a33d62-bbb3-4ed9-834e-19ae93a97670","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6819","nonce":2064636084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6821","nonce":2064636084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6825","nonce":2064636084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6823","nonce":2064636084}]},"public_addr":"172.21.15.5:6819/2064636084","cluster_addr":"172.21.15.5:6821/2064636084","heartbeat_back_addr":"172.21.15.5:6825/2064636084","heartbeat_front_addr":"172.21.15.5:6823/2064636084","state":["exists","up"]},{"osd":7,"uuid":"c0f4e036-4717-4514-bb59-ab11fbaab063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6817","nonce":2952750659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6819","nonce":2952750659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6823","nonce":2952750659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6821","nonce":2952750659}]},"public_addr":"172.21.15.37:6817/2952750659","cluster_addr":"172.21.15.37:6819/2952750659","heartbeat_back_addr":"172.21.15.37:6823/2952750659","heartbeat_front_addr":"172.21.15.37:6821/2952750659","state":["exists","up"]},{"osd":8,"uuid":"5391fd87-44ea-440f-80da-c54d0327c04c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6817","nonce":29724945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6819","nonce":29724945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6823","nonce":29724945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6821","nonce":29724945}]},"public_addr":"172.21.15.137:6817/29724945","cluster_addr":"172.21.15.137:6819/29724945","heartbeat_back_addr":"172.21.15.137:6823/29724945","heartbeat_front_addr":"172.21.15.137:6821/29724945","state":["exists","up"]},{"osd":9,"uuid":"d94c9dbc-f43d-4237-ac7f-47a6a5ede1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6827","nonce":2257102107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6829","nonce":2257102107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6833","nonce":2257102107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6831","nonce":2257102107}]},"public_addr":"172.21.15.5:6827/2257102107","cluster_addr":"172.21.15.5:6829/2257102107","heartbeat_back_addr":"172.21.15.5:6833/2257102107","heartbeat_front_addr":"172.21.15.5:6831/2257102107","state":["exists","up"]},{"osd":10,"uuid":"e2455332-2a5e-44b1-85d3-8e47da399f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6825","nonce":2451570637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6827","nonce":2451570637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6831","nonce":2451570637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6829","nonce":2451570637}]},"public_addr":"172.21.15.37:6825/2451570637","cluster_addr":"172.21.15.37:6827/2451570637","heartbeat_back_addr":"172.21.15.37:6831/2451570637","heartbeat_front_addr":"172.21.15.37:6829/2451570637","state":["exists","up"]},{"osd":11,"uuid":"49d9396b-704a-4cef-8210-75d30c28e924","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6825","nonce":90600596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6827","nonce":90600596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6831","nonce":90600596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6829","nonce":90600596}]},"public_addr":"172.21.15.137:6825/90600596","cluster_addr":"172.21.15.137:6827/90600596","heartbeat_back_addr":"172.21.15.137:6831/90600596","heartbeat_front_addr":"172.21.15.137:6829/90600596","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:35:35.591619+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:00.568817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:24.195643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:56.451724+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:22.109293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:46.161749+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:19.278305+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:45.620256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:10.041165+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:43.813697+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:10.736026+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:35.384377+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6800/4042936965":"2024-04-25T06:44:01.965881+0000","172.21.15.5:0/2912680052":"2024-04-25T06:44:01.965881+0000","172.21.15.5:6801/4042936965":"2024-04-25T06:44:01.965881+0000","172.21.15.5:0/3478347847":"2024-04-25T06:44:01.965881+0000","172.21.15.5:0/23307164":"2024-04-25T06:33:41.161852+0000","172.21.15.5:0/1012004002":"2024-04-25T06:44:01.965881+0000","172.21.15.5:6801/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:0/230785941":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/805658012":"2024-04-25T06:44:01.965881+0000","172.21.15.5:0/2940023597":"2024-04-25T06:33:07.040369+0000","172.21.15.5:6801/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3886613904":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/4199165437":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3459593499":"2024-04-25T06:33:41.161852+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:44:05.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.291+0000 7f318dc00640 1 -- 172.21.15.5:0/1015201619 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f316c0a3e20 msgr2=0x7f316c0a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:05.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.291+0000 7f318dc00640 1 --2- 172.21.15.5:0/1015201619 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f316c0a3e20 0x7f316c0a62e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f318819fd60 tx=0x7f3174005f90 comp rx=0 tx=0).stop 2024-04-24T06:44:05.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.291+0000 7f318dc00640 1 -- 172.21.15.5:0/1015201619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31881085a0 msgr2=0x7f318819e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:05.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.291+0000 7f318dc00640 1 --2- 172.21.15.5:0/1015201619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31881085a0 0x7f318819e850 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3178031440 tx=0x7f3178032ce0 comp rx=0 tx=0).stop 2024-04-24T06:44:05.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.291+0000 7f318dc00640 1 -- 172.21.15.5:0/1015201619 shutdown_connections 2024-04-24T06:44:05.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.291+0000 7f318dc00640 1 --2- 172.21.15.5:0/1015201619 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f316c0a3e20 0x7f316c0a62e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:05.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.291+0000 7f318dc00640 1 --2- 172.21.15.5:0/1015201619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f318810a100 0x7f31881a5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:05.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.291+0000 7f318dc00640 1 --2- 172.21.15.5:0/1015201619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3188109740 0x7f318819ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:05.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.291+0000 7f318dc00640 1 --2- 172.21.15.5:0/1015201619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31881085a0 0x7f318819e850 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:05.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.291+0000 7f318dc00640 1 -- 172.21.15.5:0/1015201619 >> 172.21.15.5:0/1015201619 conn(0x7f31880774f0 msgr2=0x7f318810ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:05.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.292+0000 7f318dc00640 1 -- 172.21.15.5:0/1015201619 shutdown_connections 2024-04-24T06:44:05.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.292+0000 7f318dc00640 1 -- 172.21.15.5:0/1015201619 wait complete. 2024-04-24T06:44:05.315 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:36:27.320342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:41:32.254502+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:41:33.739067+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:44:05.315 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:44:05.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.514+0000 7ff182400640 1 -- 172.21.15.5:0/1374008951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff17c105440 msgr2=0x7ff17c1058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:05.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.514+0000 7ff182400640 1 --2- 172.21.15.5:0/1374008951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff17c105440 0x7ff17c1058c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff168009900 tx=0x7ff16802a410 comp rx=0 tx=0).stop 2024-04-24T06:44:05.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.514+0000 7ff182400640 1 -- 172.21.15.5:0/1374008951 shutdown_connections 2024-04-24T06:44:05.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.514+0000 7ff182400640 1 --2- 172.21.15.5:0/1374008951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff17c105e00 0x7ff17c10c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:05.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.514+0000 7ff182400640 1 --2- 172.21.15.5:0/1374008951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff17c105440 0x7ff17c1058c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:05.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.514+0000 7ff182400640 1 --2- 172.21.15.5:0/1374008951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff17c104250 0x7ff17c104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:05.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.514+0000 7ff182400640 1 -- 172.21.15.5:0/1374008951 >> 172.21.15.5:0/1374008951 conn(0x7ff17c0ff7c0 msgr2=0x7ff17c101c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:05.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.514+0000 7ff182400640 1 -- 172.21.15.5:0/1374008951 shutdown_connections 2024-04-24T06:44:05.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.514+0000 7ff182400640 1 -- 172.21.15.5:0/1374008951 wait complete. 2024-04-24T06:44:05.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.515+0000 7ff182400640 1 Processor -- start 2024-04-24T06:44:05.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.515+0000 7ff182400640 1 -- start start 2024-04-24T06:44:05.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.516+0000 7ff182400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff17c104250 0x7ff17c19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:05.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.516+0000 7ff182400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff17c105440 0x7ff17c19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:05.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.516+0000 7ff182400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff17c105e00 0x7ff17c1a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:05.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.516+0000 7ff182400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c10ff30 con 0x7ff17c104250 2024-04-24T06:44:05.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.516+0000 7ff182400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c10fdb0 con 0x7ff17c105440 2024-04-24T06:44:05.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.516+0000 7ff182400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c1100b0 con 0x7ff17c105e00 2024-04-24T06:44:05.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.516+0000 7ff17b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff17c104250 0x7ff17c19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:05.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.516+0000 7ff17be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff17c105e00 0x7ff17c1a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:05.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.516+0000 7ff17aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff17c105440 0x7ff17c19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:05.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.516+0000 7ff17be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff17c105e00 0x7ff17c1a1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:54186/0 (socket says 172.21.15.5:54186) 2024-04-24T06:44:05.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.516+0000 7ff17b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff17c104250 0x7ff17c19a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48616/0 (socket says 172.21.15.5:48616) 2024-04-24T06:44:05.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.516+0000 7ff17be00640 1 -- 172.21.15.5:0/1830512179 learned_addr learned my addr 172.21.15.5:0/1830512179 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:05.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.516+0000 7ff17aa00640 1 -- 172.21.15.5:0/1830512179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff17c105e00 msgr2=0x7ff17c1a1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:05.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.516+0000 7ff17aa00640 1 --2- 172.21.15.5:0/1830512179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff17c105e00 0x7ff17c1a1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:05.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.517+0000 7ff17aa00640 1 -- 172.21.15.5:0/1830512179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff17c104250 msgr2=0x7ff17c19a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:05.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.517+0000 7ff17aa00640 1 --2- 172.21.15.5:0/1830512179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff17c104250 0x7ff17c19a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:05.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.517+0000 7ff17aa00640 1 -- 172.21.15.5:0/1830512179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff17c1a21b0 con 0x7ff17c105440 2024-04-24T06:44:05.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.517+0000 7ff17be00640 1 --2- 172.21.15.5:0/1830512179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff17c105e00 0x7ff17c1a1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:44:05.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.517+0000 7ff17aa00640 1 --2- 172.21.15.5:0/1830512179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff17c105440 0x7ff17c19aad0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff16802a930 tx=0x7ff168032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:05.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.517+0000 7ff173e00640 1 -- 172.21.15.5:0/1830512179 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16803f030 con 0x7ff17c105440 2024-04-24T06:44:05.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.517+0000 7ff173e00640 1 -- 172.21.15.5:0/1830512179 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff168003ce0 con 0x7ff17c105440 2024-04-24T06:44:05.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.517+0000 7ff173e00640 1 -- 172.21.15.5:0/1830512179 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff168035070 con 0x7ff17c105440 2024-04-24T06:44:05.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.517+0000 7ff182400640 1 -- 172.21.15.5:0/1830512179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff17c1a2440 con 0x7ff17c105440 2024-04-24T06:44:05.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.517+0000 7ff182400640 1 -- 172.21.15.5:0/1830512179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff17c1a2950 con 0x7ff17c105440 2024-04-24T06:44:05.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.519+0000 7ff182400640 1 -- 172.21.15.5:0/1830512179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff17c10b5f0 con 0x7ff17c105440 2024-04-24T06:44:05.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.522+0000 7ff173e00640 1 -- 172.21.15.5:0/1830512179 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138816+0+0 (secure 0 0 0) 0x7ff168042030 con 0x7ff17c105440 2024-04-24T06:44:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.523+0000 7ff173e00640 1 --2- 172.21.15.5:0/1830512179 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ff1600a3e30 0x7ff1600a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.523+0000 7ff173e00640 1 -- 172.21.15.5:0/1830512179 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7ff1680f0c90 con 0x7ff17c105440 2024-04-24T06:44:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.523+0000 7ff17b400640 1 --2- 172.21.15.5:0/1830512179 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ff1600a3e30 0x7ff1600a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.524+0000 7ff17b400640 1 --2- 172.21.15.5:0/1830512179 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ff1600a3e30 0x7ff1600a62f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff17c1052a0 tx=0x7ff16c005f90 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:05.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.527+0000 7ff173e00640 1 -- 172.21.15.5:0/1830512179 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff1680bcb80 con 0x7ff17c105440 2024-04-24T06:44:05.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:05 smithi137 ceph-mon[33017]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:44:05.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:05 smithi137 ceph-mon[33017]: mgrmap e24: x(active, since 2s), standbys: y, z 2024-04-24T06:44:05.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:05 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/2251351959' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T06:44:05.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:05 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/1447151142' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T06:44:05.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:05 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/1015201619' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:44:05.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:05 smithi005 ceph-mon[27798]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:44:05.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:05 smithi005 ceph-mon[27798]: mgrmap e24: x(active, since 2s), standbys: y, z 2024-04-24T06:44:05.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:05 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/2251351959' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T06:44:05.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:05 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1447151142' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T06:44:05.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:05 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1015201619' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:44:05.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.758+0000 7ff182400640 1 -- 172.21.15.5:0/1830512179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff17c02d620 con 0x7ff17c105440 2024-04-24T06:44:05.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.759+0000 7ff173e00640 1 -- 172.21.15.5:0/1830512179 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7ff1680c0e80 con 0x7ff17c105440 2024-04-24T06:44:05.761 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-24T06:44:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.762+0000 7ff182400640 1 -- 172.21.15.5:0/1830512179 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ff1600a3e30 msgr2=0x7ff1600a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.762+0000 7ff182400640 1 --2- 172.21.15.5:0/1830512179 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ff1600a3e30 0x7ff1600a62f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff17c1052a0 tx=0x7ff16c005f90 comp rx=0 tx=0).stop 2024-04-24T06:44:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.762+0000 7ff182400640 1 -- 172.21.15.5:0/1830512179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff17c105440 msgr2=0x7ff17c19aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.762+0000 7ff182400640 1 --2- 172.21.15.5:0/1830512179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff17c105440 0x7ff17c19aad0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff16802a930 tx=0x7ff168032ce0 comp rx=0 tx=0).stop 2024-04-24T06:44:05.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.762+0000 7ff182400640 1 -- 172.21.15.5:0/1830512179 shutdown_connections 2024-04-24T06:44:05.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.762+0000 7ff182400640 1 --2- 172.21.15.5:0/1830512179 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ff1600a3e30 0x7ff1600a62f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:05.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.762+0000 7ff182400640 1 --2- 172.21.15.5:0/1830512179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff17c105e00 0x7ff17c1a1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:05.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.762+0000 7ff182400640 1 --2- 172.21.15.5:0/1830512179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff17c105440 0x7ff17c19aad0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:05.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.762+0000 7ff182400640 1 --2- 172.21.15.5:0/1830512179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff17c104250 0x7ff17c19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:05.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.762+0000 7ff182400640 1 -- 172.21.15.5:0/1830512179 >> 172.21.15.5:0/1830512179 conn(0x7ff17c0ff7c0 msgr2=0x7ff17c1087c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:05.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.762+0000 7ff182400640 1 -- 172.21.15.5:0/1830512179 shutdown_connections 2024-04-24T06:44:05.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.762+0000 7ff182400640 1 -- 172.21.15.5:0/1830512179 wait complete. 2024-04-24T06:44:05.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:05 smithi037 ceph-mon[32864]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:44:05.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:05 smithi037 ceph-mon[32864]: mgrmap e24: x(active, since 2s), standbys: y, z 2024-04-24T06:44:05.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:05 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/2251351959' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T06:44:05.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:05 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/1447151142' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T06:44:05.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:05 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/1015201619' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:44:05.781 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:44:05.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.988+0000 7ff796a00640 1 -- 172.21.15.5:0/2627313904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff790102fe0 msgr2=0x7ff790103460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:05.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.988+0000 7ff796a00640 1 --2- 172.21.15.5:0/2627313904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff790102fe0 0x7ff790103460 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff77c009900 tx=0x7ff77c02a410 comp rx=0 tx=0).stop 2024-04-24T06:44:05.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.988+0000 7ff796a00640 1 -- 172.21.15.5:0/2627313904 shutdown_connections 2024-04-24T06:44:05.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.988+0000 7ff796a00640 1 --2- 172.21.15.5:0/2627313904 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7901039a0 0x7ff79010a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:05.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.988+0000 7ff796a00640 1 --2- 172.21.15.5:0/2627313904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff790102fe0 0x7ff790103460 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:05.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.988+0000 7ff796a00640 1 --2- 172.21.15.5:0/2627313904 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff790101e90 0x7ff7901022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:05.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.988+0000 7ff796a00640 1 -- 172.21.15.5:0/2627313904 >> 172.21.15.5:0/2627313904 conn(0x7ff7900fd400 msgr2=0x7ff7900ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:05.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.988+0000 7ff796a00640 1 -- 172.21.15.5:0/2627313904 shutdown_connections 2024-04-24T06:44:05.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.988+0000 7ff796a00640 1 -- 172.21.15.5:0/2627313904 wait complete. 2024-04-24T06:44:05.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.989+0000 7ff796a00640 1 Processor -- start 2024-04-24T06:44:05.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.990+0000 7ff796a00640 1 -- start start 2024-04-24T06:44:05.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.990+0000 7ff796a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff790101e90 0x7ff790075db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:05.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.990+0000 7ff796a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff790102fe0 0x7ff7900743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:05.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.990+0000 7ff796a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7901039a0 0x7ff790074930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:05.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.990+0000 7ff796a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff79010db70 con 0x7ff790101e90 2024-04-24T06:44:05.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.990+0000 7ff796a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff79010d9f0 con 0x7ff790102fe0 2024-04-24T06:44:05.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.990+0000 7ff796a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff79010dcf0 con 0x7ff7901039a0 2024-04-24T06:44:05.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.990+0000 7ff795600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff790101e90 0x7ff790075db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:05.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.990+0000 7ff795600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff790101e90 0x7ff790075db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48632/0 (socket says 172.21.15.5:48632) 2024-04-24T06:44:05.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.990+0000 7ff794c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff790102fe0 0x7ff7900743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:05.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.990+0000 7ff795600640 1 -- 172.21.15.5:0/823253701 learned_addr learned my addr 172.21.15.5:0/823253701 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:05.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.990+0000 7ff796000640 1 --2- 172.21.15.5:0/823253701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7901039a0 0x7ff790074930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:05.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.991+0000 7ff794c00640 1 -- 172.21.15.5:0/823253701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7901039a0 msgr2=0x7ff790074930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:05.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.991+0000 7ff794c00640 1 --2- 172.21.15.5:0/823253701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7901039a0 0x7ff790074930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:05.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.991+0000 7ff794c00640 1 -- 172.21.15.5:0/823253701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff790101e90 msgr2=0x7ff790075db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:05.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.991+0000 7ff794c00640 1 --2- 172.21.15.5:0/823253701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff790101e90 0x7ff790075db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:05.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.991+0000 7ff794c00640 1 -- 172.21.15.5:0/823253701 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff790074e70 con 0x7ff790102fe0 2024-04-24T06:44:05.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.991+0000 7ff795600640 1 --2- 172.21.15.5:0/823253701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff790101e90 0x7ff790075db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:44:05.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.991+0000 7ff794c00640 1 --2- 172.21.15.5:0/823253701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff790102fe0 0x7ff7900743f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff77c009230 tx=0x7ff77c032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:05.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.991+0000 7ff78e000640 1 -- 172.21.15.5:0/823253701 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77c03f030 con 0x7ff790102fe0 2024-04-24T06:44:05.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.992+0000 7ff78e000640 1 -- 172.21.15.5:0/823253701 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff77c003ce0 con 0x7ff790102fe0 2024-04-24T06:44:05.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.992+0000 7ff796a00640 1 -- 172.21.15.5:0/823253701 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff790075100 con 0x7ff790102fe0 2024-04-24T06:44:05.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.992+0000 7ff78e000640 1 -- 172.21.15.5:0/823253701 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77c049060 con 0x7ff790102fe0 2024-04-24T06:44:05.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.992+0000 7ff796a00640 1 -- 172.21.15.5:0/823253701 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7901a6be0 con 0x7ff790102fe0 2024-04-24T06:44:05.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.993+0000 7ff796a00640 1 -- 172.21.15.5:0/823253701 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff790109220 con 0x7ff790102fe0 2024-04-24T06:44:05.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.995+0000 7ff78e000640 1 -- 172.21.15.5:0/823253701 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138816+0+0 (secure 0 0 0) 0x7ff77c042030 con 0x7ff790102fe0 2024-04-24T06:44:05.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.996+0000 7ff78e000640 1 --2- 172.21.15.5:0/823253701 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ff7780a3db0 0x7ff7780a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:05.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.997+0000 7ff78e000640 1 -- 172.21.15.5:0/823253701 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7ff77c0f13c0 con 0x7ff790102fe0 2024-04-24T06:44:05.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.997+0000 7ff795600640 1 --2- 172.21.15.5:0/823253701 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ff7780a3db0 0x7ff7780a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:05.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.998+0000 7ff795600640 1 --2- 172.21.15.5:0/823253701 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ff7780a3db0 0x7ff7780a6270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff784011550 tx=0x7ff784005f90 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:05.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:05.999+0000 7ff78e000640 1 -- 172.21.15.5:0/823253701 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff77c0bd8a0 con 0x7ff790102fe0 2024-04-24T06:44:06.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.264+0000 7ff796a00640 1 -- 172.21.15.5:0/823253701 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff790062c00 con 0x7ff790102fe0 2024-04-24T06:44:06.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.265+0000 7ff78e000640 1 -- 172.21.15.5:0/823253701 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7ff77c0bd1a0 con 0x7ff790102fe0 2024-04-24T06:44:06.266 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-24T06:44:06.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.267+0000 7ff796a00640 1 -- 172.21.15.5:0/823253701 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ff7780a3db0 msgr2=0x7ff7780a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:06.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.267+0000 7ff796a00640 1 --2- 172.21.15.5:0/823253701 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ff7780a3db0 0x7ff7780a6270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff784011550 tx=0x7ff784005f90 comp rx=0 tx=0).stop 2024-04-24T06:44:06.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.267+0000 7ff796a00640 1 -- 172.21.15.5:0/823253701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff790102fe0 msgr2=0x7ff7900743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:06.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.267+0000 7ff796a00640 1 --2- 172.21.15.5:0/823253701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff790102fe0 0x7ff7900743f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff77c009230 tx=0x7ff77c032ce0 comp rx=0 tx=0).stop 2024-04-24T06:44:06.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.267+0000 7ff796a00640 1 -- 172.21.15.5:0/823253701 shutdown_connections 2024-04-24T06:44:06.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.267+0000 7ff796a00640 1 --2- 172.21.15.5:0/823253701 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ff7780a3db0 0x7ff7780a6270 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:06.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.267+0000 7ff796a00640 1 --2- 172.21.15.5:0/823253701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7901039a0 0x7ff790074930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:06.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.267+0000 7ff796a00640 1 --2- 172.21.15.5:0/823253701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff790102fe0 0x7ff7900743f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:06.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.267+0000 7ff796a00640 1 --2- 172.21.15.5:0/823253701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff790101e90 0x7ff790075db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:06.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.267+0000 7ff796a00640 1 -- 172.21.15.5:0/823253701 >> 172.21.15.5:0/823253701 conn(0x7ff7900fd400 msgr2=0x7ff7901063f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:06.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.267+0000 7ff796a00640 1 -- 172.21.15.5:0/823253701 shutdown_connections 2024-04-24T06:44:06.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.267+0000 7ff796a00640 1 -- 172.21.15.5:0/823253701 wait complete. 2024-04-24T06:44:06.291 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:44:06.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.480+0000 7fb151c00640 1 -- 172.21.15.5:0/1344267039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb14c105450 msgr2=0x7fb14c1058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:06.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.480+0000 7fb151c00640 1 --2- 172.21.15.5:0/1344267039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb14c105450 0x7fb14c1058d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb134009900 tx=0x7fb13402a410 comp rx=0 tx=0).stop 2024-04-24T06:44:06.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.480+0000 7fb151c00640 1 -- 172.21.15.5:0/1344267039 shutdown_connections 2024-04-24T06:44:06.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.480+0000 7fb151c00640 1 --2- 172.21.15.5:0/1344267039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb14c105e10 0x7fb14c10c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:06.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.480+0000 7fb151c00640 1 --2- 172.21.15.5:0/1344267039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb14c105450 0x7fb14c1058d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:06.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.480+0000 7fb151c00640 1 --2- 172.21.15.5:0/1344267039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb14c104260 0x7fb14c104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:06.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.480+0000 7fb151c00640 1 -- 172.21.15.5:0/1344267039 >> 172.21.15.5:0/1344267039 conn(0x7fb14c0ff7f0 msgr2=0x7fb14c101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:06.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.480+0000 7fb151c00640 1 -- 172.21.15.5:0/1344267039 shutdown_connections 2024-04-24T06:44:06.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.480+0000 7fb151c00640 1 -- 172.21.15.5:0/1344267039 wait complete. 2024-04-24T06:44:06.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.481+0000 7fb151c00640 1 Processor -- start 2024-04-24T06:44:06.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.481+0000 7fb151c00640 1 -- start start 2024-04-24T06:44:06.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.481+0000 7fb151c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb14c104260 0x7fb14c19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:06.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.481+0000 7fb151c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb14c105450 0x7fb14c19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:06.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.482+0000 7fb151c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb14c105e10 0x7fb14c1a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:06.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.482+0000 7fb151c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c10ff40 con 0x7fb14c105450 2024-04-24T06:44:06.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.482+0000 7fb151c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c10fdc0 con 0x7fb14c105e10 2024-04-24T06:44:06.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.482+0000 7fb151c00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c1100c0 con 0x7fb14c104260 2024-04-24T06:44:06.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.482+0000 7fb142000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb14c105450 0x7fb14c19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:06.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.482+0000 7fb14b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb14c105e10 0x7fb14c1a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:06.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.482+0000 7fb142000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb14c105450 0x7fb14c19aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48648/0 (socket says 172.21.15.5:48648) 2024-04-24T06:44:06.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.482+0000 7fb142000640 1 -- 172.21.15.5:0/244738049 learned_addr learned my addr 172.21.15.5:0/244738049 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:06.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.482+0000 7fb14aa00640 1 --2- 172.21.15.5:0/244738049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb14c104260 0x7fb14c19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:06.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.482+0000 7fb14b400640 1 -- 172.21.15.5:0/244738049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb14c104260 msgr2=0x7fb14c19a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:06.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.482+0000 7fb14b400640 1 --2- 172.21.15.5:0/244738049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb14c104260 0x7fb14c19a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:06.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.482+0000 7fb14b400640 1 -- 172.21.15.5:0/244738049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb14c105450 msgr2=0x7fb14c19aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:06.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.483+0000 7fb14b400640 1 --2- 172.21.15.5:0/244738049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb14c105450 0x7fb14c19aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:06.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.483+0000 7fb14b400640 1 -- 172.21.15.5:0/244738049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb14c1a21b0 con 0x7fb14c105e10 2024-04-24T06:44:06.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.483+0000 7fb14b400640 1 --2- 172.21.15.5:0/244738049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb14c105e10 0x7fb14c1a1b40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb144009fd0 tx=0x7fb14400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:06.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.496+0000 7fb143e00640 1 -- 172.21.15.5:0/244738049 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14400ce20 con 0x7fb14c105e10 2024-04-24T06:44:06.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.496+0000 7fb143e00640 1 -- 172.21.15.5:0/244738049 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb144005020 con 0x7fb14c105e10 2024-04-24T06:44:06.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.496+0000 7fb143e00640 1 -- 172.21.15.5:0/244738049 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb144010720 con 0x7fb14c105e10 2024-04-24T06:44:06.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.496+0000 7fb151c00640 1 -- 172.21.15.5:0/244738049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb14c1a24a0 con 0x7fb14c105e10 2024-04-24T06:44:06.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.496+0000 7fb151c00640 1 -- 172.21.15.5:0/244738049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb14c1a29a0 con 0x7fb14c105e10 2024-04-24T06:44:06.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.498+0000 7fb151c00640 1 -- 172.21.15.5:0/244738049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb14c10b580 con 0x7fb14c105e10 2024-04-24T06:44:06.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.500+0000 7fb143e00640 1 -- 172.21.15.5:0/244738049 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138872+0+0 (secure 0 0 0) 0x7fb144005540 con 0x7fb14c105e10 2024-04-24T06:44:06.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.502+0000 7fb143e00640 1 --2- 172.21.15.5:0/244738049 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb1300a3ec0 0x7fb1300a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:06.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.502+0000 7fb143e00640 1 -- 172.21.15.5:0/244738049 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fb1440d1130 con 0x7fb14c105e10 2024-04-24T06:44:06.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.502+0000 7fb14aa00640 1 --2- 172.21.15.5:0/244738049 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb1300a3ec0 0x7fb1300a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:06.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.503+0000 7fb14aa00640 1 --2- 172.21.15.5:0/244738049 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb1300a3ec0 0x7fb1300a6380 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb14c1052b0 tx=0x7fb13c005f90 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:06.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.506+0000 7fb143e00640 1 -- 172.21.15.5:0/244738049 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb14409d590 con 0x7fb14c105e10 2024-04-24T06:44:06.593 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:06 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:06.593 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:06 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:06.593 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:06 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.593 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:06 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.593 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:06 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.593 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:06 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.593 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:06 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.593 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:06 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.593 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:06 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.594 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:06 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.594 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:06 smithi005 ceph-mon[27798]: Adjusting osd_memory_target on smithi137 to 176.2M 2024-04-24T06:44:06.594 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:06 smithi005 ceph-mon[27798]: Unable to set osd_memory_target on smithi137 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-24T06:44:06.594 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:06 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:06.594 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:06 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:06.594 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:06 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.594 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:06 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.594 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:06 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.594 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:06 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.594 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:06 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.594 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:06 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.594 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:06 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.594 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:06 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.595 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:06 smithi005 ceph-mon[27798]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-04-24T06:44:06.595 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:06 smithi005 ceph-mon[27798]: Unable to set osd_memory_target on smithi037 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-24T06:44:06.595 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:06 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1830512179' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T06:44:06.595 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:06 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:06.595 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:06 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:06.595 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:06 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/823253701' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T06:44:06.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:06 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:06.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:06 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:06.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:06 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:06 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:06 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:06 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:06 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:06 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:06 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:06 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:06 smithi137 ceph-mon[33017]: Adjusting osd_memory_target on smithi137 to 176.2M 2024-04-24T06:44:06.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:06 smithi137 ceph-mon[33017]: Unable to set osd_memory_target on smithi137 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-24T06:44:06.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:06 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:06.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:06 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:06.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:06 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:06 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:06 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:06 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.698 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:06 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.698 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:06 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.698 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:06 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.698 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:06 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.698 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:06 smithi137 ceph-mon[33017]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-04-24T06:44:06.698 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:06 smithi137 ceph-mon[33017]: Unable to set osd_memory_target on smithi037 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-24T06:44:06.698 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:06 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/1830512179' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T06:44:06.698 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:06 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:06.698 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:06 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:06.698 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:06 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/823253701' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T06:44:06.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.738+0000 7fb151c00640 1 -- 172.21.15.5:0/244738049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb14c062c00 con 0x7fb14c105e10 2024-04-24T06:44:06.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.739+0000 7fb143e00640 1 -- 172.21.15.5:0/244738049 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7fb14409ce90 con 0x7fb14c105e10 2024-04-24T06:44:06.740 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-24T06:44:06.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.742+0000 7fb151c00640 1 -- 172.21.15.5:0/244738049 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb1300a3ec0 msgr2=0x7fb1300a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:06.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.742+0000 7fb151c00640 1 --2- 172.21.15.5:0/244738049 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb1300a3ec0 0x7fb1300a6380 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb14c1052b0 tx=0x7fb13c005f90 comp rx=0 tx=0).stop 2024-04-24T06:44:06.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.742+0000 7fb151c00640 1 -- 172.21.15.5:0/244738049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb14c105e10 msgr2=0x7fb14c1a1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:06.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.742+0000 7fb151c00640 1 --2- 172.21.15.5:0/244738049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb14c105e10 0x7fb14c1a1b40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb144009fd0 tx=0x7fb14400ef10 comp rx=0 tx=0).stop 2024-04-24T06:44:06.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.742+0000 7fb151c00640 1 -- 172.21.15.5:0/244738049 shutdown_connections 2024-04-24T06:44:06.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.742+0000 7fb151c00640 1 --2- 172.21.15.5:0/244738049 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb1300a3ec0 0x7fb1300a6380 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:06.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.742+0000 7fb151c00640 1 --2- 172.21.15.5:0/244738049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb14c105e10 0x7fb14c1a1b40 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:06.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.742+0000 7fb151c00640 1 --2- 172.21.15.5:0/244738049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb14c105450 0x7fb14c19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:06.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.742+0000 7fb151c00640 1 --2- 172.21.15.5:0/244738049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb14c104260 0x7fb14c19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:06.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.742+0000 7fb151c00640 1 -- 172.21.15.5:0/244738049 >> 172.21.15.5:0/244738049 conn(0x7fb14c0ff7f0 msgr2=0x7fb14c108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:06.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.742+0000 7fb151c00640 1 -- 172.21.15.5:0/244738049 shutdown_connections 2024-04-24T06:44:06.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.742+0000 7fb151c00640 1 -- 172.21.15.5:0/244738049 wait complete. 2024-04-24T06:44:06.764 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-04-24T06:44:06.765 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:44:06.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:06 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:06.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:06 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:06.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:06 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:06 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:06 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:06 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:06 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:06 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:06 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:06 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:06 smithi037 ceph-mon[32864]: Adjusting osd_memory_target on smithi137 to 176.2M 2024-04-24T06:44:06.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:06 smithi037 ceph-mon[32864]: Unable to set osd_memory_target on smithi137 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-24T06:44:06.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:06 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:06.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:06 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:06.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:06 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:06 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:06 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:06 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:06 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.771 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:06 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.771 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:06 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.771 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:06 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:06.771 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:06 smithi037 ceph-mon[32864]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-04-24T06:44:06.771 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:06 smithi037 ceph-mon[32864]: Unable to set osd_memory_target on smithi037 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-24T06:44:06.771 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:06 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/1830512179' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T06:44:06.771 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:06 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:06.771 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:06 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:06.771 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:06 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/823253701' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T06:44:06.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.958+0000 7f1603200640 1 -- 172.21.15.5:0/1125432665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1604101170 msgr2=0x7f16041015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:06.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.958+0000 7f1603200640 1 --2- 172.21.15.5:0/1125432665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1604101170 0x7f16041015d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f15f0009860 tx=0x7f15f002a340 comp rx=0 tx=0).stop 2024-04-24T06:44:06.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.958+0000 7f1603200640 1 -- 172.21.15.5:0/1125432665 shutdown_connections 2024-04-24T06:44:06.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.958+0000 7f1603200640 1 --2- 172.21.15.5:0/1125432665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1604101b10 0x7f1604101f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:06.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.958+0000 7f1603200640 1 --2- 172.21.15.5:0/1125432665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1604101170 0x7f16041015d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:06.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.958+0000 7f1603200640 1 --2- 172.21.15.5:0/1125432665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1604100810 0x7f1604100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:06.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.958+0000 7f1603200640 1 -- 172.21.15.5:0/1125432665 >> 172.21.15.5:0/1125432665 conn(0x7f16040fc3d0 msgr2=0x7f16040fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:06.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.958+0000 7f1603200640 1 -- 172.21.15.5:0/1125432665 shutdown_connections 2024-04-24T06:44:06.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.958+0000 7f1603200640 1 -- 172.21.15.5:0/1125432665 wait complete. 2024-04-24T06:44:06.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.959+0000 7f1603200640 1 Processor -- start 2024-04-24T06:44:06.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.959+0000 7f1603200640 1 -- start start 2024-04-24T06:44:06.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.959+0000 7f1603200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1604100810 0x7f160419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:06.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.959+0000 7f1603200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1604101170 0x7f160419a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:06.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.959+0000 7f1603200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1604101b10 0x7f16041a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:06.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.959+0000 7f1603200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f160410fde0 con 0x7f1604100810 2024-04-24T06:44:06.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.959+0000 7f1603200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f160410fc60 con 0x7f1604101170 2024-04-24T06:44:06.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.959+0000 7f1603200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f160410ff60 con 0x7f1604101b10 2024-04-24T06:44:06.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.960+0000 7f1601e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1604100810 0x7f160419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:06.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.960+0000 7f1601e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1604100810 0x7f160419a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48664/0 (socket says 172.21.15.5:48664) 2024-04-24T06:44:06.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.960+0000 7f1601e00640 1 -- 172.21.15.5:0/3534096658 learned_addr learned my addr 172.21.15.5:0/3534096658 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:06.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.960+0000 7f1601400640 1 --2- 172.21.15.5:0/3534096658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1604101170 0x7f160419a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:06.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.960+0000 7f1602800640 1 --2- 172.21.15.5:0/3534096658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1604101b10 0x7f16041a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:06.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.960+0000 7f1601400640 1 -- 172.21.15.5:0/3534096658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1604101b10 msgr2=0x7f16041a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:06.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.960+0000 7f1601400640 1 --2- 172.21.15.5:0/3534096658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1604101b10 0x7f16041a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:06.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.960+0000 7f1601400640 1 -- 172.21.15.5:0/3534096658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1604100810 msgr2=0x7f160419a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:06.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.960+0000 7f1601400640 1 --2- 172.21.15.5:0/3534096658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1604100810 0x7f160419a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:06.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.960+0000 7f1601400640 1 -- 172.21.15.5:0/3534096658 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16041a2040 con 0x7f1604101170 2024-04-24T06:44:06.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.960+0000 7f1601e00640 1 --2- 172.21.15.5:0/3534096658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1604100810 0x7f160419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:44:06.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.961+0000 7f1601400640 1 --2- 172.21.15.5:0/3534096658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1604101170 0x7f160419a960 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f15f0035040 tx=0x7f15f0033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:06.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.961+0000 7f15faa00640 1 -- 172.21.15.5:0/3534096658 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f0037040 con 0x7f1604101170 2024-04-24T06:44:06.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.961+0000 7f1603200640 1 -- 172.21.15.5:0/3534096658 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16041a22d0 con 0x7f1604101170 2024-04-24T06:44:06.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.961+0000 7f15faa00640 1 -- 172.21.15.5:0/3534096658 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15f0030850 con 0x7f1604101170 2024-04-24T06:44:06.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.961+0000 7f15faa00640 1 -- 172.21.15.5:0/3534096658 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f0004920 con 0x7f1604101170 2024-04-24T06:44:06.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.961+0000 7f1603200640 1 -- 172.21.15.5:0/3534096658 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16041a27e0 con 0x7f1604101170 2024-04-24T06:44:06.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.962+0000 7f1603200640 1 -- 172.21.15.5:0/3534096658 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f160410b560 con 0x7f1604101170 2024-04-24T06:44:06.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.964+0000 7f15faa00640 1 -- 172.21.15.5:0/3534096658 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138872+0+0 (secure 0 0 0) 0x7f15f0004050 con 0x7f1604101170 2024-04-24T06:44:06.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.966+0000 7f15faa00640 1 --2- 172.21.15.5:0/3534096658 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f15e80a3eb0 0x7f15e80a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:06.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.966+0000 7f15faa00640 1 -- 172.21.15.5:0/3534096658 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f15f00f0720 con 0x7f1604101170 2024-04-24T06:44:06.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.966+0000 7f1601e00640 1 --2- 172.21.15.5:0/3534096658 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f15e80a3eb0 0x7f15e80a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:06.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.967+0000 7f1601e00640 1 --2- 172.21.15.5:0/3534096658 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f15e80a3eb0 0x7f15e80a6370 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f15f4011550 tx=0x7f15f4005f90 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:06.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:06.970+0000 7f15faa00640 1 -- 172.21.15.5:0/3534096658 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f15f00bcc30 con 0x7f1604101170 2024-04-24T06:44:07.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.256+0000 7f1603200640 1 -- 172.21.15.5:0/3534096658 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f160402d620 con 0x7f1604101170 2024-04-24T06:44:07.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.257+0000 7f15faa00640 1 -- 172.21.15.5:0/3534096658 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11856 (secure 0 0 0) 0x7f15f00bc530 con 0x7f1604101170 2024-04-24T06:44:07.258 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:44:07.259 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1735462750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1735462750},{"type":"v1","addr":"172.21.15.5:6837","nonce":1735462750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1499469370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1499469370},{"type":"v1","addr":"172.21.15.5:6841","nonce":1499469370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3705638847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3705638847},{"type":"v1","addr":"172.21.15.37:6835","nonce":3705638847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T06:41:34.764490+0000","modified":"2024-04-24T06:44:01.711021+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24550,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6837/1390103296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1390103296},{"type":"v1","addr":"172.21.15.37:6837","nonce":1390103296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3217754215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3217754215},{"type":"v1","addr":"172.21.15.137:6835","nonce":3217754215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6837/3044222976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3044222976},{"type":"v1","addr":"172.21.15.137:6837","nonce":3044222976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.5:6839/3173201459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3173201459},{"type":"v1","addr":"172.21.15.5:6839","nonce":3173201459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6839/1889774232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1889774232},{"type":"v1","addr":"172.21.15.137:6839","nonce":1889774232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6839/3536721753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":3536721753},{"type":"v1","addr":"172.21.15.37:6839","nonce":3536721753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24569,24539,24550]},"id":1}]} 2024-04-24T06:44:07.259 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 23 2024-04-24T06:44:07.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.260+0000 7f1603200640 1 -- 172.21.15.5:0/3534096658 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f15e80a3eb0 msgr2=0x7f15e80a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:07.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.260+0000 7f1603200640 1 --2- 172.21.15.5:0/3534096658 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f15e80a3eb0 0x7f15e80a6370 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f15f4011550 tx=0x7f15f4005f90 comp rx=0 tx=0).stop 2024-04-24T06:44:07.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.260+0000 7f1603200640 1 -- 172.21.15.5:0/3534096658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1604101170 msgr2=0x7f160419a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:07.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.260+0000 7f1603200640 1 --2- 172.21.15.5:0/3534096658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1604101170 0x7f160419a960 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f15f0035040 tx=0x7f15f0033d90 comp rx=0 tx=0).stop 2024-04-24T06:44:07.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.260+0000 7f1603200640 1 -- 172.21.15.5:0/3534096658 shutdown_connections 2024-04-24T06:44:07.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.260+0000 7f1603200640 1 --2- 172.21.15.5:0/3534096658 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f15e80a3eb0 0x7f15e80a6370 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:07.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.260+0000 7f1603200640 1 --2- 172.21.15.5:0/3534096658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1604101b10 0x7f16041a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:07.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.260+0000 7f1603200640 1 --2- 172.21.15.5:0/3534096658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1604101170 0x7f160419a960 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:07.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.260+0000 7f1603200640 1 --2- 172.21.15.5:0/3534096658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1604100810 0x7f160419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:07.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.260+0000 7f1603200640 1 -- 172.21.15.5:0/3534096658 >> 172.21.15.5:0/3534096658 conn(0x7f16040fc3d0 msgr2=0x7f16040ff710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:07.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.260+0000 7f1603200640 1 -- 172.21.15.5:0/3534096658 shutdown_connections 2024-04-24T06:44:07.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.260+0000 7f1603200640 1 -- 172.21.15.5:0/3534096658 wait complete. 2024-04-24T06:44:07.280 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-04-24T06:44:07.280 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-04-24T06:44:07.280 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T06:44:07.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.473+0000 7f7eca600640 1 -- 172.21.15.5:0/1296770024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ec4105e80 msgr2=0x7f7ec410c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:07.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.473+0000 7f7eca600640 1 --2- 172.21.15.5:0/1296770024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ec4105e80 0x7f7ec410c710 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc009ca0 tx=0x7f7ebc02a8c0 comp rx=0 tx=0).stop 2024-04-24T06:44:07.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.473+0000 7f7eca600640 1 -- 172.21.15.5:0/1296770024 shutdown_connections 2024-04-24T06:44:07.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.473+0000 7f7eca600640 1 --2- 172.21.15.5:0/1296770024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ec4105e80 0x7f7ec410c710 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:07.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.473+0000 7f7eca600640 1 --2- 172.21.15.5:0/1296770024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ec41054c0 0x7f7ec4105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:07.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.473+0000 7f7eca600640 1 --2- 172.21.15.5:0/1296770024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ec41042d0 0x7f7ec41046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:07.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.473+0000 7f7eca600640 1 -- 172.21.15.5:0/1296770024 >> 172.21.15.5:0/1296770024 conn(0x7f7ec40ff880 msgr2=0x7f7ec4101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:07.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.473+0000 7f7eca600640 1 -- 172.21.15.5:0/1296770024 shutdown_connections 2024-04-24T06:44:07.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.473+0000 7f7eca600640 1 -- 172.21.15.5:0/1296770024 wait complete. 2024-04-24T06:44:07.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.474+0000 7f7eca600640 1 Processor -- start 2024-04-24T06:44:07.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.474+0000 7f7eca600640 1 -- start start 2024-04-24T06:44:07.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.475+0000 7f7eca600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ec41042d0 0x7f7ec419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:07.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.475+0000 7f7eca600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ec41054c0 0x7f7ec419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:07.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.475+0000 7f7eca600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ec4105e80 0x7f7ec41a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:07.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.475+0000 7f7eca600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec410ff40 con 0x7f7ec4105e80 2024-04-24T06:44:07.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.475+0000 7f7eca600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec410fdc0 con 0x7f7ec41042d0 2024-04-24T06:44:07.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.475+0000 7f7eca600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec41100c0 con 0x7f7ec41054c0 2024-04-24T06:44:07.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.475+0000 7f7ec3400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ec41042d0 0x7f7ec419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:07.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.475+0000 7f7ec3400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ec41042d0 0x7f7ec419a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:46106/0 (socket says 172.21.15.5:46106) 2024-04-24T06:44:07.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.475+0000 7f7ec3400640 1 -- 172.21.15.5:0/3114897639 learned_addr learned my addr 172.21.15.5:0/3114897639 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:07.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.475+0000 7f7ebbe00640 1 --2- 172.21.15.5:0/3114897639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ec41054c0 0x7f7ec419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:07.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.475+0000 7f7ec3e00640 1 --2- 172.21.15.5:0/3114897639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ec4105e80 0x7f7ec41a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:07.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.476+0000 7f7ebbe00640 1 -- 172.21.15.5:0/3114897639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ec41042d0 msgr2=0x7f7ec419a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:07.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.476+0000 7f7ebbe00640 1 --2- 172.21.15.5:0/3114897639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ec41042d0 0x7f7ec419a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:07.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.476+0000 7f7ebbe00640 1 -- 172.21.15.5:0/3114897639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ec4105e80 msgr2=0x7f7ec41a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:07.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.476+0000 7f7ebbe00640 1 --2- 172.21.15.5:0/3114897639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ec4105e80 0x7f7ec41a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:07.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.476+0000 7f7ebbe00640 1 -- 172.21.15.5:0/3114897639 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ec41a20d0 con 0x7f7ec41054c0 2024-04-24T06:44:07.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.476+0000 7f7ebbe00640 1 --2- 172.21.15.5:0/3114897639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ec41054c0 0x7f7ec419a9f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7eac005fd0 tx=0x7f7eac00dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:07.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.476+0000 7f7ec0c00640 1 -- 172.21.15.5:0/3114897639 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac009c30 con 0x7f7ec41054c0 2024-04-24T06:44:07.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.476+0000 7f7ec0c00640 1 -- 172.21.15.5:0/3114897639 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7eac009dd0 con 0x7f7ec41054c0 2024-04-24T06:44:07.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.476+0000 7f7eca600640 1 -- 172.21.15.5:0/3114897639 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ec41a23c0 con 0x7f7ec41054c0 2024-04-24T06:44:07.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.476+0000 7f7ec0c00640 1 -- 172.21.15.5:0/3114897639 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac010460 con 0x7f7ec41054c0 2024-04-24T06:44:07.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.476+0000 7f7eca600640 1 -- 172.21.15.5:0/3114897639 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ec41a2910 con 0x7f7ec41054c0 2024-04-24T06:44:07.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.478+0000 7f7eca600640 1 -- 172.21.15.5:0/3114897639 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ec410b590 con 0x7f7ec41054c0 2024-04-24T06:44:07.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.481+0000 7f7ec0c00640 1 -- 172.21.15.5:0/3114897639 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 138872+0+0 (secure 0 0 0) 0x7f7eac003f50 con 0x7f7ec41054c0 2024-04-24T06:44:07.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.482+0000 7f7ec0c00640 1 --2- 172.21.15.5:0/3114897639 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f7ea80a3e40 0x7f7ea80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:07.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.482+0000 7f7ec0c00640 1 -- 172.21.15.5:0/3114897639 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f7eac0d09b0 con 0x7f7ec41054c0 2024-04-24T06:44:07.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.483+0000 7f7ec3400640 1 --2- 172.21.15.5:0/3114897639 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f7ea80a3e40 0x7f7ea80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:07.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.483+0000 7f7ec3400640 1 --2- 172.21.15.5:0/3114897639 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f7ea80a3e40 0x7f7ea80a6300 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400bc80 tx=0x7f7eb400b410 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:07.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.487+0000 7f7ec0c00640 1 -- 172.21.15.5:0/3114897639 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7eac017020 con 0x7f7ec41054c0 2024-04-24T06:44:07.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.692+0000 7f7eca600640 1 -- 172.21.15.5:0/3114897639 --> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7ec4062440 con 0x7f7ea80a3e40 2024-04-24T06:44:07.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.718+0000 7f7ec0c00640 1 -- 172.21.15.5:0/3114897639 <== mgr.25202 v2:172.21.15.5:6800/3154624651 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257767 (secure 0 0 0) 0x7f7ec4062440 con 0x7f7ea80a3e40 2024-04-24T06:44:07.723 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:44:07.724 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-24T06:44:07.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.723+0000 7f7eca600640 1 -- 172.21.15.5:0/3114897639 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f7ea80a3e40 msgr2=0x7f7ea80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:07.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.723+0000 7f7eca600640 1 --2- 172.21.15.5:0/3114897639 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f7ea80a3e40 0x7f7ea80a6300 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400bc80 tx=0x7f7eb400b410 comp rx=0 tx=0).stop 2024-04-24T06:44:07.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.723+0000 7f7eca600640 1 -- 172.21.15.5:0/3114897639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ec41054c0 msgr2=0x7f7ec419a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:07.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.723+0000 7f7eca600640 1 --2- 172.21.15.5:0/3114897639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ec41054c0 0x7f7ec419a9f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7eac005fd0 tx=0x7f7eac00dde0 comp rx=0 tx=0).stop 2024-04-24T06:44:07.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.724+0000 7f7eca600640 1 -- 172.21.15.5:0/3114897639 shutdown_connections 2024-04-24T06:44:07.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.724+0000 7f7eca600640 1 --2- 172.21.15.5:0/3114897639 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f7ea80a3e40 0x7f7ea80a6300 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:07.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.724+0000 7f7eca600640 1 --2- 172.21.15.5:0/3114897639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ec4105e80 0x7f7ec41a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:07.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.724+0000 7f7eca600640 1 --2- 172.21.15.5:0/3114897639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ec41054c0 0x7f7ec419a9f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:07.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.724+0000 7f7eca600640 1 --2- 172.21.15.5:0/3114897639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ec41042d0 0x7f7ec419a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:07.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.724+0000 7f7eca600640 1 -- 172.21.15.5:0/3114897639 >> 172.21.15.5:0/3114897639 conn(0x7f7ec40ff880 msgr2=0x7f7ec4108760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:07.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.724+0000 7f7eca600640 1 -- 172.21.15.5:0/3114897639 shutdown_connections 2024-04-24T06:44:07.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.724+0000 7f7eca600640 1 -- 172.21.15.5:0/3114897639 wait complete. 2024-04-24T06:44:07.742 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:07 smithi005 ceph-mon[27798]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:44:07.742 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:07 smithi005 ceph-mon[27798]: mgrmap e25: x(active, since 4s), standbys: y, z 2024-04-24T06:44:07.742 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:07 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/244738049' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T06:44:07.742 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:07 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/3534096658' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:44:07.744 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-24T06:44:06.005075+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1186377,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1117,"num_read_kb":717,"num_write":465,"num_write_kb":4370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":859,"ondisk_log_size":859,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376268,"kb_used_data":14608,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565556,"statfs":{"total":1151940427776,"available":1151555129344,"internally_reserved":0,"allocated":14958592,"data_stored":8637983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.971498"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.528183+0000","last_change":"2024-04-24T06:41:35.296451+0000","last_active":"2024-04-24T06:44:02.528183+0000","last_peered":"2024-04-24T06:44:02.528183+0000","last_clean":"2024-04-24T06:44:02.528183+0000","last_became_active":"2024-04-24T06:41:35.295733+0000","last_became_peered":"2024-04-24T06:41:35.295733+0000","last_unstale":"2024-04-24T06:44:02.528183+0000","last_undegraded":"2024-04-24T06:44:02.528183+0000","last_fullsized":"2024-04-24T06:44:02.528183+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:04:15.072644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.325058+0000","last_change":"2024-04-24T06:41:35.293359+0000","last_active":"2024-04-24T06:44:02.325058+0000","last_peered":"2024-04-24T06:44:02.325058+0000","last_clean":"2024-04-24T06:44:02.325058+0000","last_became_active":"2024-04-24T06:41:35.287371+0000","last_became_peered":"2024-04-24T06:41:35.287371+0000","last_unstale":"2024-04-24T06:44:02.325058+0000","last_undegraded":"2024-04-24T06:44:02.325058+0000","last_fullsized":"2024-04-24T06:44:02.325058+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:03:39.376495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.325258+0000","last_change":"2024-04-24T06:41:35.272734+0000","last_active":"2024-04-24T06:44:02.325258+0000","last_peered":"2024-04-24T06:44:02.325258+0000","last_clean":"2024-04-24T06:44:02.325258+0000","last_became_active":"2024-04-24T06:41:35.271685+0000","last_became_peered":"2024-04-24T06:41:35.271685+0000","last_unstale":"2024-04-24T06:44:02.325258+0000","last_undegraded":"2024-04-24T06:44:02.325258+0000","last_fullsized":"2024-04-24T06:44:02.325258+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:08:15.057918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.323790+0000","last_change":"2024-04-24T06:41:35.288574+0000","last_active":"2024-04-24T06:44:02.323790+0000","last_peered":"2024-04-24T06:44:02.323790+0000","last_clean":"2024-04-24T06:44:02.323790+0000","last_became_active":"2024-04-24T06:41:35.288229+0000","last_became_peered":"2024-04-24T06:41:35.288229+0000","last_unstale":"2024-04-24T06:44:02.323790+0000","last_undegraded":"2024-04-24T06:44:02.323790+0000","last_fullsized":"2024-04-24T06:44:02.323790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:58:18.028067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.128022+0000","last_change":"2024-04-24T06:41:35.310226+0000","last_active":"2024-04-24T06:44:02.128022+0000","last_peered":"2024-04-24T06:44:02.128022+0000","last_clean":"2024-04-24T06:44:02.128022+0000","last_became_active":"2024-04-24T06:41:35.309853+0000","last_became_peered":"2024-04-24T06:41:35.309853+0000","last_unstale":"2024-04-24T06:44:02.128022+0000","last_undegraded":"2024-04-24T06:44:02.128022+0000","last_fullsized":"2024-04-24T06:44:02.128022+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:34:57.767158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.138791+0000","last_change":"2024-04-24T06:41:35.294259+0000","last_active":"2024-04-24T06:44:02.138791+0000","last_peered":"2024-04-24T06:44:02.138791+0000","last_clean":"2024-04-24T06:44:02.138791+0000","last_became_active":"2024-04-24T06:41:35.293827+0000","last_became_peered":"2024-04-24T06:41:35.293827+0000","last_unstale":"2024-04-24T06:44:02.138791+0000","last_undegraded":"2024-04-24T06:44:02.138791+0000","last_fullsized":"2024-04-24T06:44:02.138791+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:09:23.302827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.982776+0000","last_change":"2024-04-24T06:41:35.295095+0000","last_active":"2024-04-24T06:44:01.982776+0000","last_peered":"2024-04-24T06:44:01.982776+0000","last_clean":"2024-04-24T06:44:01.982776+0000","last_became_active":"2024-04-24T06:41:35.294743+0000","last_became_peered":"2024-04-24T06:41:35.294743+0000","last_unstale":"2024-04-24T06:44:01.982776+0000","last_undegraded":"2024-04-24T06:44:01.982776+0000","last_fullsized":"2024-04-24T06:44:01.982776+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:06:09.473123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.125405+0000","last_change":"2024-04-24T06:42:42.886072+0000","last_active":"2024-04-24T06:44:02.125405+0000","last_peered":"2024-04-24T06:44:02.125405+0000","last_clean":"2024-04-24T06:44:02.125405+0000","last_became_active":"2024-04-24T06:42:42.882624+0000","last_became_peered":"2024-04-24T06:42:42.882624+0000","last_unstale":"2024-04-24T06:44:02.125405+0000","last_undegraded":"2024-04-24T06:44:02.125405+0000","last_fullsized":"2024-04-24T06:44:02.125405+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:53:02.523012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.320256+0000","last_change":"2024-04-24T06:41:35.283275+0000","last_active":"2024-04-24T06:44:02.320256+0000","last_peered":"2024-04-24T06:44:02.320256+0000","last_clean":"2024-04-24T06:44:02.320256+0000","last_became_active":"2024-04-24T06:41:35.282586+0000","last_became_peered":"2024-04-24T06:41:35.282586+0000","last_unstale":"2024-04-24T06:44:02.320256+0000","last_undegraded":"2024-04-24T06:44:02.320256+0000","last_fullsized":"2024-04-24T06:44:02.320256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:42:30.716149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.139163+0000","last_change":"2024-04-24T06:41:35.284986+0000","last_active":"2024-04-24T06:44:02.139163+0000","last_peered":"2024-04-24T06:44:02.139163+0000","last_clean":"2024-04-24T06:44:02.139163+0000","last_became_active":"2024-04-24T06:41:35.284628+0000","last_became_peered":"2024-04-24T06:41:35.284628+0000","last_unstale":"2024-04-24T06:44:02.139163+0000","last_undegraded":"2024-04-24T06:44:02.139163+0000","last_fullsized":"2024-04-24T06:44:02.139163+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:29:38.479593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.983393+0000","last_change":"2024-04-24T06:41:35.290353+0000","last_active":"2024-04-24T06:44:01.983393+0000","last_peered":"2024-04-24T06:44:01.983393+0000","last_clean":"2024-04-24T06:44:01.983393+0000","last_became_active":"2024-04-24T06:41:35.289987+0000","last_became_peered":"2024-04-24T06:41:35.289987+0000","last_unstale":"2024-04-24T06:44:01.983393+0000","last_undegraded":"2024-04-24T06:44:01.983393+0000","last_fullsized":"2024-04-24T06:44:01.983393+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:56:54.914426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.323831+0000","last_change":"2024-04-24T06:41:35.289462+0000","last_active":"2024-04-24T06:44:02.323831+0000","last_peered":"2024-04-24T06:44:02.323831+0000","last_clean":"2024-04-24T06:44:02.323831+0000","last_became_active":"2024-04-24T06:41:35.289123+0000","last_became_peered":"2024-04-24T06:41:35.289123+0000","last_unstale":"2024-04-24T06:44:02.323831+0000","last_undegraded":"2024-04-24T06:44:02.323831+0000","last_fullsized":"2024-04-24T06:44:02.323831+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:51:06.792508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.981156+0000","last_change":"2024-04-24T06:41:35.298320+0000","last_active":"2024-04-24T06:44:01.981156+0000","last_peered":"2024-04-24T06:44:01.981156+0000","last_clean":"2024-04-24T06:44:01.981156+0000","last_became_active":"2024-04-24T06:41:35.297839+0000","last_became_peered":"2024-04-24T06:41:35.297839+0000","last_unstale":"2024-04-24T06:44:01.981156+0000","last_undegraded":"2024-04-24T06:44:01.981156+0000","last_fullsized":"2024-04-24T06:44:01.981156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:10:36.423514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.324512+0000","last_change":"2024-04-24T06:41:35.301924+0000","last_active":"2024-04-24T06:44:02.324512+0000","last_peered":"2024-04-24T06:44:02.324512+0000","last_clean":"2024-04-24T06:44:02.324512+0000","last_became_active":"2024-04-24T06:41:35.299114+0000","last_became_peered":"2024-04-24T06:41:35.299114+0000","last_unstale":"2024-04-24T06:44:02.324512+0000","last_undegraded":"2024-04-24T06:44:02.324512+0000","last_fullsized":"2024-04-24T06:44:02.324512+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:34:46.194266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.323875+0000","last_change":"2024-04-24T06:41:35.284885+0000","last_active":"2024-04-24T06:44:02.323875+0000","last_peered":"2024-04-24T06:44:02.323875+0000","last_clean":"2024-04-24T06:44:02.323875+0000","last_became_active":"2024-04-24T06:41:35.284244+0000","last_became_peered":"2024-04-24T06:41:35.284244+0000","last_unstale":"2024-04-24T06:44:02.323875+0000","last_undegraded":"2024-04-24T06:44:02.323875+0000","last_fullsized":"2024-04-24T06:44:02.323875+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:30:40.279905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.319433+0000","last_change":"2024-04-24T06:41:35.286363+0000","last_active":"2024-04-24T06:44:02.319433+0000","last_peered":"2024-04-24T06:44:02.319433+0000","last_clean":"2024-04-24T06:44:02.319433+0000","last_became_active":"2024-04-24T06:41:35.286024+0000","last_became_peered":"2024-04-24T06:41:35.286024+0000","last_unstale":"2024-04-24T06:44:02.319433+0000","last_undegraded":"2024-04-24T06:44:02.319433+0000","last_fullsized":"2024-04-24T06:44:02.319433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:43:35.728990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.528311+0000","last_change":"2024-04-24T06:41:35.304005+0000","last_active":"2024-04-24T06:44:02.528311+0000","last_peered":"2024-04-24T06:44:02.528311+0000","last_clean":"2024-04-24T06:44:02.528311+0000","last_became_active":"2024-04-24T06:41:35.303659+0000","last_became_peered":"2024-04-24T06:41:35.303659+0000","last_unstale":"2024-04-24T06:44:02.528311+0000","last_undegraded":"2024-04-24T06:44:02.528311+0000","last_fullsized":"2024-04-24T06:44:02.528311+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:05:07.215927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.126161+0000","last_change":"2024-04-24T06:41:35.296218+0000","last_active":"2024-04-24T06:44:02.126161+0000","last_peered":"2024-04-24T06:44:02.126161+0000","last_clean":"2024-04-24T06:44:02.126161+0000","last_became_active":"2024-04-24T06:41:35.295486+0000","last_became_peered":"2024-04-24T06:41:35.295486+0000","last_unstale":"2024-04-24T06:44:02.126161+0000","last_undegraded":"2024-04-24T06:44:02.126161+0000","last_fullsized":"2024-04-24T06:44:02.126161+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:17:25.844899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.980909+0000","last_change":"2024-04-24T06:42:42.886291+0000","last_active":"2024-04-24T06:44:01.980909+0000","last_peered":"2024-04-24T06:44:01.980909+0000","last_clean":"2024-04-24T06:44:01.980909+0000","last_became_active":"2024-04-24T06:42:42.885781+0000","last_became_peered":"2024-04-24T06:42:42.885781+0000","last_unstale":"2024-04-24T06:44:01.980909+0000","last_undegraded":"2024-04-24T06:44:01.980909+0000","last_fullsized":"2024-04-24T06:44:01.980909+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:01:44.377335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.125607+0000","last_change":"2024-04-24T06:41:34.282140+0000","last_active":"2024-04-24T06:44:02.125607+0000","last_peered":"2024-04-24T06:44:02.125607+0000","last_clean":"2024-04-24T06:44:02.125607+0000","last_became_active":"2024-04-24T06:41:34.281608+0000","last_became_peered":"2024-04-24T06:41:34.281608+0000","last_unstale":"2024-04-24T06:44:02.125607+0000","last_undegraded":"2024-04-24T06:44:02.125607+0000","last_fullsized":"2024-04-24T06:44:02.125607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:32:06.727639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.129957+0000","last_change":"2024-04-24T06:41:35.308847+0000","last_active":"2024-04-24T06:44:02.129957+0000","last_peered":"2024-04-24T06:44:02.129957+0000","last_clean":"2024-04-24T06:44:02.129957+0000","last_became_active":"2024-04-24T06:41:35.308256+0000","last_became_peered":"2024-04-24T06:41:35.308256+0000","last_unstale":"2024-04-24T06:44:02.129957+0000","last_undegraded":"2024-04-24T06:44:02.129957+0000","last_fullsized":"2024-04-24T06:44:02.129957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:21:49.338737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.130066+0000","last_change":"2024-04-24T06:41:34.285819+0000","last_active":"2024-04-24T06:44:02.130066+0000","last_peered":"2024-04-24T06:44:02.130066+0000","last_clean":"2024-04-24T06:44:02.130066+0000","last_became_active":"2024-04-24T06:41:34.282393+0000","last_became_peered":"2024-04-24T06:41:34.282393+0000","last_unstale":"2024-04-24T06:44:02.130066+0000","last_undegraded":"2024-04-24T06:44:02.130066+0000","last_fullsized":"2024-04-24T06:44:02.130066+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:11:18.969178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.325587+0000","last_change":"2024-04-24T06:41:35.297437+0000","last_active":"2024-04-24T06:44:02.325587+0000","last_peered":"2024-04-24T06:44:02.325587+0000","last_clean":"2024-04-24T06:44:02.325587+0000","last_became_active":"2024-04-24T06:41:35.295195+0000","last_became_peered":"2024-04-24T06:41:35.295195+0000","last_unstale":"2024-04-24T06:44:02.325587+0000","last_undegraded":"2024-04-24T06:44:02.325587+0000","last_fullsized":"2024-04-24T06:44:02.325587+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:21:51.219219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.128352+0000","last_change":"2024-04-24T06:41:34.275768+0000","last_active":"2024-04-24T06:44:02.128352+0000","last_peered":"2024-04-24T06:44:02.128352+0000","last_clean":"2024-04-24T06:44:02.128352+0000","last_became_active":"2024-04-24T06:41:34.275426+0000","last_became_peered":"2024-04-24T06:41:34.275426+0000","last_unstale":"2024-04-24T06:44:02.128352+0000","last_undegraded":"2024-04-24T06:44:02.128352+0000","last_fullsized":"2024-04-24T06:44:02.128352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:01:08.735480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.319850+0000","last_change":"2024-04-24T06:41:35.305734+0000","last_active":"2024-04-24T06:44:02.319850+0000","last_peered":"2024-04-24T06:44:02.319850+0000","last_clean":"2024-04-24T06:44:02.319850+0000","last_became_active":"2024-04-24T06:41:35.305365+0000","last_became_peered":"2024-04-24T06:41:35.305365+0000","last_unstale":"2024-04-24T06:44:02.319850+0000","last_undegraded":"2024-04-24T06:44:02.319850+0000","last_fullsized":"2024-04-24T06:44:02.319850+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:38:52.740107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.128296+0000","last_change":"2024-04-24T06:41:34.278317+0000","last_active":"2024-04-24T06:44:02.128296+0000","last_peered":"2024-04-24T06:44:02.128296+0000","last_clean":"2024-04-24T06:44:02.128296+0000","last_became_active":"2024-04-24T06:41:34.277862+0000","last_became_peered":"2024-04-24T06:41:34.277862+0000","last_unstale":"2024-04-24T06:44:02.128296+0000","last_undegraded":"2024-04-24T06:44:02.128296+0000","last_fullsized":"2024-04-24T06:44:02.128296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:19:50.444136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.022666+0000","last_change":"2024-04-24T06:41:35.282648+0000","last_active":"2024-04-24T06:44:02.022666+0000","last_peered":"2024-04-24T06:44:02.022666+0000","last_clean":"2024-04-24T06:44:02.022666+0000","last_became_active":"2024-04-24T06:41:35.282335+0000","last_became_peered":"2024-04-24T06:41:35.282335+0000","last_unstale":"2024-04-24T06:44:02.022666+0000","last_undegraded":"2024-04-24T06:44:02.022666+0000","last_fullsized":"2024-04-24T06:44:02.022666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:30:11.909067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":174,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.568985+0000","last_change":"2024-04-24T06:41:34.274405+0000","last_active":"2024-04-24T06:44:02.568985+0000","last_peered":"2024-04-24T06:44:02.568985+0000","last_clean":"2024-04-24T06:44:02.568985+0000","last_became_active":"2024-04-24T06:41:34.273820+0000","last_became_peered":"2024-04-24T06:41:34.273820+0000","last_unstale":"2024-04-24T06:44:02.568985+0000","last_undegraded":"2024-04-24T06:44:02.568985+0000","last_fullsized":"2024-04-24T06:44:02.568985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:11:26.119236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.128021+0000","last_change":"2024-04-24T06:41:35.294133+0000","last_active":"2024-04-24T06:44:02.128021+0000","last_peered":"2024-04-24T06:44:02.128021+0000","last_clean":"2024-04-24T06:44:02.128021+0000","last_became_active":"2024-04-24T06:41:35.293741+0000","last_became_peered":"2024-04-24T06:41:35.293741+0000","last_unstale":"2024-04-24T06:44:02.128021+0000","last_undegraded":"2024-04-24T06:44:02.128021+0000","last_fullsized":"2024-04-24T06:44:02.128021+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:09:12.495187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.138617+0000","last_change":"2024-04-24T06:42:42.927397+0000","last_active":"2024-04-24T06:44:02.138617+0000","last_peered":"2024-04-24T06:44:02.138617+0000","last_clean":"2024-04-24T06:44:02.138617+0000","last_became_active":"2024-04-24T06:42:42.924070+0000","last_became_peered":"2024-04-24T06:42:42.924070+0000","last_unstale":"2024-04-24T06:44:02.138617+0000","last_undegraded":"2024-04-24T06:44:02.138617+0000","last_fullsized":"2024-04-24T06:44:02.138617+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:19:47.319762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.022818+0000","last_change":"2024-04-24T06:42:43.166909+0000","last_active":"2024-04-24T06:44:02.022818+0000","last_peered":"2024-04-24T06:44:02.022818+0000","last_clean":"2024-04-24T06:44:02.022818+0000","last_became_active":"2024-04-24T06:42:43.166559+0000","last_became_peered":"2024-04-24T06:42:43.166559+0000","last_unstale":"2024-04-24T06:44:02.022818+0000","last_undegraded":"2024-04-24T06:44:02.022818+0000","last_fullsized":"2024-04-24T06:44:02.022818+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:26:42.537506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.980468+0000","last_change":"2024-04-24T06:41:34.284958+0000","last_active":"2024-04-24T06:44:01.980468+0000","last_peered":"2024-04-24T06:44:01.980468+0000","last_clean":"2024-04-24T06:44:01.980468+0000","last_became_active":"2024-04-24T06:41:34.284279+0000","last_became_peered":"2024-04-24T06:41:34.284279+0000","last_unstale":"2024-04-24T06:44:01.980468+0000","last_undegraded":"2024-04-24T06:44:01.980468+0000","last_fullsized":"2024-04-24T06:44:01.980468+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:58:19.316091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.128861+0000","last_change":"2024-04-24T06:41:35.307280+0000","last_active":"2024-04-24T06:44:02.128861+0000","last_peered":"2024-04-24T06:44:02.128861+0000","last_clean":"2024-04-24T06:44:02.128861+0000","last_became_active":"2024-04-24T06:41:35.306908+0000","last_became_peered":"2024-04-24T06:41:35.306908+0000","last_unstale":"2024-04-24T06:44:02.128861+0000","last_undegraded":"2024-04-24T06:44:02.128861+0000","last_fullsized":"2024-04-24T06:44:02.128861+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:54:31.306302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.022540+0000","last_change":"2024-04-24T06:41:34.272221+0000","last_active":"2024-04-24T06:44:02.022540+0000","last_peered":"2024-04-24T06:44:02.022540+0000","last_clean":"2024-04-24T06:44:02.022540+0000","last_became_active":"2024-04-24T06:41:34.271748+0000","last_became_peered":"2024-04-24T06:41:34.271748+0000","last_unstale":"2024-04-24T06:44:02.022540+0000","last_undegraded":"2024-04-24T06:44:02.022540+0000","last_fullsized":"2024-04-24T06:44:02.022540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:29:02.667285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.125592+0000","last_change":"2024-04-24T06:41:35.309608+0000","last_active":"2024-04-24T06:44:02.125592+0000","last_peered":"2024-04-24T06:44:02.125592+0000","last_clean":"2024-04-24T06:44:02.125592+0000","last_became_active":"2024-04-24T06:41:35.309227+0000","last_became_peered":"2024-04-24T06:41:35.309227+0000","last_unstale":"2024-04-24T06:44:02.125592+0000","last_undegraded":"2024-04-24T06:44:02.125592+0000","last_fullsized":"2024-04-24T06:44:02.125592+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:51:23.151426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":169,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.567739+0000","last_change":"2024-04-24T06:41:34.281885+0000","last_active":"2024-04-24T06:44:02.567739+0000","last_peered":"2024-04-24T06:44:02.567739+0000","last_clean":"2024-04-24T06:44:02.567739+0000","last_became_active":"2024-04-24T06:41:34.281466+0000","last_became_peered":"2024-04-24T06:41:34.281466+0000","last_unstale":"2024-04-24T06:44:02.567739+0000","last_undegraded":"2024-04-24T06:44:02.567739+0000","last_fullsized":"2024-04-24T06:44:02.567739+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:20:08.500343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":143,"num_read_kb":143,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.320075+0000","last_change":"2024-04-24T06:41:35.292714+0000","last_active":"2024-04-24T06:44:02.320075+0000","last_peered":"2024-04-24T06:44:02.320075+0000","last_clean":"2024-04-24T06:44:02.320075+0000","last_became_active":"2024-04-24T06:41:35.291950+0000","last_became_peered":"2024-04-24T06:41:35.291950+0000","last_unstale":"2024-04-24T06:44:02.320075+0000","last_undegraded":"2024-04-24T06:44:02.320075+0000","last_fullsized":"2024-04-24T06:44:02.320075+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:42:18.049746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.983538+0000","last_change":"2024-04-24T06:41:34.275218+0000","last_active":"2024-04-24T06:44:01.983538+0000","last_peered":"2024-04-24T06:44:01.983538+0000","last_clean":"2024-04-24T06:44:01.983538+0000","last_became_active":"2024-04-24T06:41:34.274824+0000","last_became_peered":"2024-04-24T06:41:34.274824+0000","last_unstale":"2024-04-24T06:44:01.983538+0000","last_undegraded":"2024-04-24T06:44:01.983538+0000","last_fullsized":"2024-04-24T06:44:01.983538+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:09:57.271304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.325532+0000","last_change":"2024-04-24T06:42:42.883478+0000","last_active":"2024-04-24T06:44:02.325532+0000","last_peered":"2024-04-24T06:44:02.325532+0000","last_clean":"2024-04-24T06:44:02.325532+0000","last_became_active":"2024-04-24T06:42:42.882808+0000","last_became_peered":"2024-04-24T06:42:42.882808+0000","last_unstale":"2024-04-24T06:44:02.325532+0000","last_undegraded":"2024-04-24T06:44:02.325532+0000","last_fullsized":"2024-04-24T06:44:02.325532+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:17:11.514041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.022807+0000","last_change":"2024-04-24T06:41:34.277372+0000","last_active":"2024-04-24T06:44:02.022807+0000","last_peered":"2024-04-24T06:44:02.022807+0000","last_clean":"2024-04-24T06:44:02.022807+0000","last_became_active":"2024-04-24T06:41:34.276997+0000","last_became_peered":"2024-04-24T06:41:34.276997+0000","last_unstale":"2024-04-24T06:44:02.022807+0000","last_undegraded":"2024-04-24T06:44:02.022807+0000","last_fullsized":"2024-04-24T06:44:02.022807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:52:35.695109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.320091+0000","last_change":"2024-04-24T06:41:35.290233+0000","last_active":"2024-04-24T06:44:02.320091+0000","last_peered":"2024-04-24T06:44:02.320091+0000","last_clean":"2024-04-24T06:44:02.320091+0000","last_became_active":"2024-04-24T06:41:35.289885+0000","last_became_peered":"2024-04-24T06:41:35.289885+0000","last_unstale":"2024-04-24T06:44:02.320091+0000","last_undegraded":"2024-04-24T06:44:02.320091+0000","last_fullsized":"2024-04-24T06:44:02.320091+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:35:54.293308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'33","reported_seq":352,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.569436+0000","last_change":"2024-04-24T06:41:34.267693+0000","last_active":"2024-04-24T06:44:02.569436+0000","last_peered":"2024-04-24T06:44:02.569436+0000","last_clean":"2024-04-24T06:44:02.569436+0000","last_became_active":"2024-04-24T06:41:34.267298+0000","last_became_peered":"2024-04-24T06:41:34.267298+0000","last_unstale":"2024-04-24T06:44:02.569436+0000","last_undegraded":"2024-04-24T06:44:02.569436+0000","last_fullsized":"2024-04-24T06:44:02.569436+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:21:16.015695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66567,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":297,"num_read_kb":213,"num_write":34,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.128194+0000","last_change":"2024-04-24T06:41:35.286587+0000","last_active":"2024-04-24T06:44:02.128194+0000","last_peered":"2024-04-24T06:44:02.128194+0000","last_clean":"2024-04-24T06:44:02.128194+0000","last_became_active":"2024-04-24T06:41:35.286239+0000","last_became_peered":"2024-04-24T06:41:35.286239+0000","last_unstale":"2024-04-24T06:44:02.128194+0000","last_undegraded":"2024-04-24T06:44:02.128194+0000","last_fullsized":"2024-04-24T06:44:02.128194+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:18:56.013425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.527938+0000","last_change":"2024-04-24T06:42:42.892090+0000","last_active":"2024-04-24T06:44:02.527938+0000","last_peered":"2024-04-24T06:44:02.527938+0000","last_clean":"2024-04-24T06:44:02.527938+0000","last_became_active":"2024-04-24T06:42:42.879904+0000","last_became_peered":"2024-04-24T06:42:42.879904+0000","last_unstale":"2024-04-24T06:44:02.527938+0000","last_undegraded":"2024-04-24T06:44:02.527938+0000","last_fullsized":"2024-04-24T06:44:02.527938+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:36:02.420438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.980663+0000","last_change":"2024-04-24T06:41:35.306175+0000","last_active":"2024-04-24T06:44:01.980663+0000","last_peered":"2024-04-24T06:44:01.980663+0000","last_clean":"2024-04-24T06:44:01.980663+0000","last_became_active":"2024-04-24T06:41:35.305840+0000","last_became_peered":"2024-04-24T06:41:35.305840+0000","last_unstale":"2024-04-24T06:44:01.980663+0000","last_undegraded":"2024-04-24T06:44:01.980663+0000","last_fullsized":"2024-04-24T06:44:01.980663+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:42:46.238490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.022530+0000","last_change":"2024-04-24T06:41:34.277212+0000","last_active":"2024-04-24T06:44:02.022530+0000","last_peered":"2024-04-24T06:44:02.022530+0000","last_clean":"2024-04-24T06:44:02.022530+0000","last_became_active":"2024-04-24T06:41:34.276695+0000","last_became_peered":"2024-04-24T06:41:34.276695+0000","last_unstale":"2024-04-24T06:44:02.022530+0000","last_undegraded":"2024-04-24T06:44:02.022530+0000","last_fullsized":"2024-04-24T06:44:02.022530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:00:47.169384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.983372+0000","last_change":"2024-04-24T06:41:35.305348+0000","last_active":"2024-04-24T06:44:01.983372+0000","last_peered":"2024-04-24T06:44:01.983372+0000","last_clean":"2024-04-24T06:44:01.983372+0000","last_became_active":"2024-04-24T06:41:35.304899+0000","last_became_peered":"2024-04-24T06:41:35.304899+0000","last_unstale":"2024-04-24T06:44:01.983372+0000","last_undegraded":"2024-04-24T06:44:01.983372+0000","last_fullsized":"2024-04-24T06:44:01.983372+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:49:35.737711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'4","reported_seq":172,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.603197+0000","last_change":"2024-04-24T06:41:34.272902+0000","last_active":"2024-04-24T06:44:02.603197+0000","last_peered":"2024-04-24T06:44:02.603197+0000","last_clean":"2024-04-24T06:44:02.603197+0000","last_became_active":"2024-04-24T06:41:34.271726+0000","last_became_peered":"2024-04-24T06:41:34.271726+0000","last_unstale":"2024-04-24T06:44:02.603197+0000","last_undegraded":"2024-04-24T06:44:02.603197+0000","last_fullsized":"2024-04-24T06:44:02.603197+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:07:38.318110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":147,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.982800+0000","last_change":"2024-04-24T06:41:35.283764+0000","last_active":"2024-04-24T06:44:01.982800+0000","last_peered":"2024-04-24T06:44:01.982800+0000","last_clean":"2024-04-24T06:44:01.982800+0000","last_became_active":"2024-04-24T06:41:35.283400+0000","last_became_peered":"2024-04-24T06:41:35.283400+0000","last_unstale":"2024-04-24T06:44:01.982800+0000","last_undegraded":"2024-04-24T06:44:01.982800+0000","last_fullsized":"2024-04-24T06:44:01.982800+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:36:47.034068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.127983+0000","last_change":"2024-04-24T06:41:34.280020+0000","last_active":"2024-04-24T06:44:02.127983+0000","last_peered":"2024-04-24T06:44:02.127983+0000","last_clean":"2024-04-24T06:44:02.127983+0000","last_became_active":"2024-04-24T06:41:34.279659+0000","last_became_peered":"2024-04-24T06:41:34.279659+0000","last_unstale":"2024-04-24T06:44:02.127983+0000","last_undegraded":"2024-04-24T06:44:02.127983+0000","last_fullsized":"2024-04-24T06:44:02.127983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:13:29.529123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.022523+0000","last_change":"2024-04-24T06:41:35.284500+0000","last_active":"2024-04-24T06:44:02.022523+0000","last_peered":"2024-04-24T06:44:02.022523+0000","last_clean":"2024-04-24T06:44:02.022523+0000","last_became_active":"2024-04-24T06:41:35.283171+0000","last_became_peered":"2024-04-24T06:41:35.283171+0000","last_unstale":"2024-04-24T06:44:02.022523+0000","last_undegraded":"2024-04-24T06:44:02.022523+0000","last_fullsized":"2024-04-24T06:44:02.022523+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:33:19.527226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.128449+0000","last_change":"2024-04-24T06:41:34.279306+0000","last_active":"2024-04-24T06:44:02.128449+0000","last_peered":"2024-04-24T06:44:02.128449+0000","last_clean":"2024-04-24T06:44:02.128449+0000","last_became_active":"2024-04-24T06:41:34.278946+0000","last_became_peered":"2024-04-24T06:41:34.278946+0000","last_unstale":"2024-04-24T06:44:02.128449+0000","last_undegraded":"2024-04-24T06:44:02.128449+0000","last_fullsized":"2024-04-24T06:44:02.128449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:13:40.845871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.528202+0000","last_change":"2024-04-24T06:41:35.283716+0000","last_active":"2024-04-24T06:44:02.528202+0000","last_peered":"2024-04-24T06:44:02.528202+0000","last_clean":"2024-04-24T06:44:02.528202+0000","last_became_active":"2024-04-24T06:41:35.283330+0000","last_became_peered":"2024-04-24T06:41:35.283330+0000","last_unstale":"2024-04-24T06:44:02.528202+0000","last_undegraded":"2024-04-24T06:44:02.528202+0000","last_fullsized":"2024-04-24T06:44:02.528202+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:18:13.403264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.125845+0000","last_change":"2024-04-24T06:42:42.887349+0000","last_active":"2024-04-24T06:44:02.125845+0000","last_peered":"2024-04-24T06:44:02.125845+0000","last_clean":"2024-04-24T06:44:02.125845+0000","last_became_active":"2024-04-24T06:42:42.886813+0000","last_became_peered":"2024-04-24T06:42:42.886813+0000","last_unstale":"2024-04-24T06:44:02.125845+0000","last_undegraded":"2024-04-24T06:44:02.125845+0000","last_fullsized":"2024-04-24T06:44:02.125845+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:05:40.577189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.139022+0000","last_change":"2024-04-24T06:41:35.306607+0000","last_active":"2024-04-24T06:44:02.139022+0000","last_peered":"2024-04-24T06:44:02.139022+0000","last_clean":"2024-04-24T06:44:02.139022+0000","last_became_active":"2024-04-24T06:41:35.306217+0000","last_became_peered":"2024-04-24T06:41:35.306217+0000","last_unstale":"2024-04-24T06:44:02.139022+0000","last_undegraded":"2024-04-24T06:44:02.139022+0000","last_fullsized":"2024-04-24T06:44:02.139022+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:07:46.068876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.983051+0000","last_change":"2024-04-24T06:41:34.285319+0000","last_active":"2024-04-24T06:44:01.983051+0000","last_peered":"2024-04-24T06:44:01.983051+0000","last_clean":"2024-04-24T06:44:01.983051+0000","last_became_active":"2024-04-24T06:41:34.284912+0000","last_became_peered":"2024-04-24T06:41:34.284912+0000","last_unstale":"2024-04-24T06:44:01.983051+0000","last_undegraded":"2024-04-24T06:44:01.983051+0000","last_fullsized":"2024-04-24T06:44:01.983051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:34:56.454587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.139027+0000","last_change":"2024-04-24T06:41:35.296294+0000","last_active":"2024-04-24T06:44:02.139027+0000","last_peered":"2024-04-24T06:44:02.139027+0000","last_clean":"2024-04-24T06:44:02.139027+0000","last_became_active":"2024-04-24T06:41:35.295792+0000","last_became_peered":"2024-04-24T06:41:35.295792+0000","last_unstale":"2024-04-24T06:44:02.139027+0000","last_undegraded":"2024-04-24T06:44:02.139027+0000","last_fullsized":"2024-04-24T06:44:02.139027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:31:35.573858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.022494+0000","last_change":"2024-04-24T06:41:35.282558+0000","last_active":"2024-04-24T06:44:02.022494+0000","last_peered":"2024-04-24T06:44:02.022494+0000","last_clean":"2024-04-24T06:44:02.022494+0000","last_became_active":"2024-04-24T06:41:35.282163+0000","last_became_peered":"2024-04-24T06:41:35.282163+0000","last_unstale":"2024-04-24T06:44:02.022494+0000","last_undegraded":"2024-04-24T06:44:02.022494+0000","last_fullsized":"2024-04-24T06:44:02.022494+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:12:23.569849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":170,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.326095+0000","last_change":"2024-04-24T06:41:34.284294+0000","last_active":"2024-04-24T06:44:02.326095+0000","last_peered":"2024-04-24T06:44:02.326095+0000","last_clean":"2024-04-24T06:44:02.326095+0000","last_became_active":"2024-04-24T06:41:34.283933+0000","last_became_peered":"2024-04-24T06:41:34.283933+0000","last_unstale":"2024-04-24T06:44:02.326095+0000","last_undegraded":"2024-04-24T06:44:02.326095+0000","last_fullsized":"2024-04-24T06:44:02.326095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:35:26.042614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":143,"num_read_kb":143,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.129137+0000","last_change":"2024-04-24T06:41:35.302361+0000","last_active":"2024-04-24T06:44:02.129137+0000","last_peered":"2024-04-24T06:44:02.129137+0000","last_clean":"2024-04-24T06:44:02.129137+0000","last_became_active":"2024-04-24T06:41:35.301652+0000","last_became_peered":"2024-04-24T06:41:35.301652+0000","last_unstale":"2024-04-24T06:44:02.129137+0000","last_undegraded":"2024-04-24T06:44:02.129137+0000","last_fullsized":"2024-04-24T06:44:02.129137+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:35:58.010853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.323724+0000","last_change":"2024-04-24T06:41:34.282156+0000","last_active":"2024-04-24T06:44:02.323724+0000","last_peered":"2024-04-24T06:44:02.323724+0000","last_clean":"2024-04-24T06:44:02.323724+0000","last_became_active":"2024-04-24T06:41:34.281771+0000","last_became_peered":"2024-04-24T06:41:34.281771+0000","last_unstale":"2024-04-24T06:44:02.323724+0000","last_undegraded":"2024-04-24T06:44:02.323724+0000","last_fullsized":"2024-04-24T06:44:02.323724+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:41:49.584620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":357,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.982840+0000","last_change":"2024-04-24T06:40:16.681695+0000","last_active":"2024-04-24T06:44:01.982840+0000","last_peered":"2024-04-24T06:44:01.982840+0000","last_clean":"2024-04-24T06:44:01.982840+0000","last_became_active":"2024-04-24T06:40:16.681208+0000","last_became_peered":"2024-04-24T06:40:16.681208+0000","last_unstale":"2024-04-24T06:44:01.982840+0000","last_undegraded":"2024-04-24T06:44:01.982840+0000","last_fullsized":"2024-04-24T06:44:01.982840+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:36:27.950574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:36:27.950574+0000","last_clean_scrub_stamp":"2024-04-24T06:36:27.950574+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:39:01.985893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.980224+0000","last_change":"2024-04-24T06:42:42.885312+0000","last_active":"2024-04-24T06:44:01.980224+0000","last_peered":"2024-04-24T06:44:01.980224+0000","last_clean":"2024-04-24T06:44:01.980224+0000","last_became_active":"2024-04-24T06:42:42.884702+0000","last_became_peered":"2024-04-24T06:42:42.884702+0000","last_unstale":"2024-04-24T06:44:01.980224+0000","last_undegraded":"2024-04-24T06:44:01.980224+0000","last_fullsized":"2024-04-24T06:44:01.980224+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:33:44.590555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.982969+0000","last_change":"2024-04-24T06:41:34.273872+0000","last_active":"2024-04-24T06:44:01.982969+0000","last_peered":"2024-04-24T06:44:01.982969+0000","last_clean":"2024-04-24T06:44:01.982969+0000","last_became_active":"2024-04-24T06:41:34.273473+0000","last_became_peered":"2024-04-24T06:41:34.273473+0000","last_unstale":"2024-04-24T06:44:01.982969+0000","last_undegraded":"2024-04-24T06:44:01.982969+0000","last_fullsized":"2024-04-24T06:44:01.982969+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:47:11.914985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.128221+0000","last_change":"2024-04-24T06:41:34.289293+0000","last_active":"2024-04-24T06:44:02.128221+0000","last_peered":"2024-04-24T06:44:02.128221+0000","last_clean":"2024-04-24T06:44:02.128221+0000","last_became_active":"2024-04-24T06:41:34.288939+0000","last_became_peered":"2024-04-24T06:41:34.288939+0000","last_unstale":"2024-04-24T06:44:02.128221+0000","last_undegraded":"2024-04-24T06:44:02.128221+0000","last_fullsized":"2024-04-24T06:44:02.128221+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:03:01.026792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.981615+0000","last_change":"2024-04-24T06:41:35.302153+0000","last_active":"2024-04-24T06:44:01.981615+0000","last_peered":"2024-04-24T06:44:01.981615+0000","last_clean":"2024-04-24T06:44:01.981615+0000","last_became_active":"2024-04-24T06:41:35.301666+0000","last_became_peered":"2024-04-24T06:41:35.301666+0000","last_unstale":"2024-04-24T06:44:01.981615+0000","last_undegraded":"2024-04-24T06:44:01.981615+0000","last_fullsized":"2024-04-24T06:44:01.981615+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:45:09.642482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'45","reported_seq":90,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.264503+0000","last_change":"2024-04-24T06:41:34.291570+0000","last_active":"2024-04-24T06:44:02.264503+0000","last_peered":"2024-04-24T06:44:02.264503+0000","last_clean":"2024-04-24T06:44:02.264503+0000","last_became_active":"2024-04-24T06:41:34.289700+0000","last_became_peered":"2024-04-24T06:41:34.289700+0000","last_unstale":"2024-04-24T06:44:02.264503+0000","last_undegraded":"2024-04-24T06:44:02.264503+0000","last_fullsized":"2024-04-24T06:44:02.264503+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:57:20.537299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":80,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.129846+0000","last_change":"2024-04-24T06:41:35.304703+0000","last_active":"2024-04-24T06:44:02.129846+0000","last_peered":"2024-04-24T06:44:02.129846+0000","last_clean":"2024-04-24T06:44:02.129846+0000","last_became_active":"2024-04-24T06:41:35.304319+0000","last_became_peered":"2024-04-24T06:41:35.304319+0000","last_unstale":"2024-04-24T06:44:02.129846+0000","last_undegraded":"2024-04-24T06:44:02.129846+0000","last_fullsized":"2024-04-24T06:44:02.129846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:00:56.069509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.324026+0000","last_change":"2024-04-24T06:41:34.276866+0000","last_active":"2024-04-24T06:44:02.324026+0000","last_peered":"2024-04-24T06:44:02.324026+0000","last_clean":"2024-04-24T06:44:02.324026+0000","last_became_active":"2024-04-24T06:41:34.276478+0000","last_became_peered":"2024-04-24T06:41:34.276478+0000","last_unstale":"2024-04-24T06:44:02.324026+0000","last_undegraded":"2024-04-24T06:44:02.324026+0000","last_fullsized":"2024-04-24T06:44:02.324026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:10:35.047960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.324893+0000","last_change":"2024-04-24T06:41:35.300951+0000","last_active":"2024-04-24T06:44:02.324893+0000","last_peered":"2024-04-24T06:44:02.324893+0000","last_clean":"2024-04-24T06:44:02.324893+0000","last_became_active":"2024-04-24T06:41:35.296176+0000","last_became_peered":"2024-04-24T06:41:35.296176+0000","last_unstale":"2024-04-24T06:44:02.324893+0000","last_undegraded":"2024-04-24T06:44:02.324893+0000","last_fullsized":"2024-04-24T06:44:02.324893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:53:58.497437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.319371+0000","last_change":"2024-04-24T06:41:34.276154+0000","last_active":"2024-04-24T06:44:02.319371+0000","last_peered":"2024-04-24T06:44:02.319371+0000","last_clean":"2024-04-24T06:44:02.319371+0000","last_became_active":"2024-04-24T06:41:34.275734+0000","last_became_peered":"2024-04-24T06:41:34.275734+0000","last_unstale":"2024-04-24T06:44:02.319371+0000","last_undegraded":"2024-04-24T06:44:02.319371+0000","last_fullsized":"2024-04-24T06:44:02.319371+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:28:54.618061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.127930+0000","last_change":"2024-04-24T06:41:35.285837+0000","last_active":"2024-04-24T06:44:02.127930+0000","last_peered":"2024-04-24T06:44:02.127930+0000","last_clean":"2024-04-24T06:44:02.127930+0000","last_became_active":"2024-04-24T06:41:35.285505+0000","last_became_peered":"2024-04-24T06:41:35.285505+0000","last_unstale":"2024-04-24T06:44:02.127930+0000","last_undegraded":"2024-04-24T06:44:02.127930+0000","last_fullsized":"2024-04-24T06:44:02.127930+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:36:53.460780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.129053+0000","last_change":"2024-04-24T06:41:34.290039+0000","last_active":"2024-04-24T06:44:02.129053+0000","last_peered":"2024-04-24T06:44:02.129053+0000","last_clean":"2024-04-24T06:44:02.129053+0000","last_became_active":"2024-04-24T06:41:34.289228+0000","last_became_peered":"2024-04-24T06:41:34.289228+0000","last_unstale":"2024-04-24T06:44:02.129053+0000","last_undegraded":"2024-04-24T06:44:02.129053+0000","last_fullsized":"2024-04-24T06:44:02.129053+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:07:20.342359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.527923+0000","last_change":"2024-04-24T06:41:35.296214+0000","last_active":"2024-04-24T06:44:02.527923+0000","last_peered":"2024-04-24T06:44:02.527923+0000","last_clean":"2024-04-24T06:44:02.527923+0000","last_became_active":"2024-04-24T06:41:35.295888+0000","last_became_peered":"2024-04-24T06:41:35.295888+0000","last_unstale":"2024-04-24T06:44:02.527923+0000","last_undegraded":"2024-04-24T06:44:02.527923+0000","last_fullsized":"2024-04-24T06:44:02.527923+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:55:19.520479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"89'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.980700+0000","last_change":"2024-04-24T06:41:34.275885+0000","last_active":"2024-04-24T06:44:01.980700+0000","last_peered":"2024-04-24T06:44:01.980700+0000","last_clean":"2024-04-24T06:44:01.980700+0000","last_became_active":"2024-04-24T06:41:34.275503+0000","last_became_peered":"2024-04-24T06:41:34.275503+0000","last_unstale":"2024-04-24T06:44:01.980700+0000","last_undegraded":"2024-04-24T06:44:01.980700+0000","last_fullsized":"2024-04-24T06:44:01.980700+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:50:58.216699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.129732+0000","last_change":"2024-04-24T06:41:35.305330+0000","last_active":"2024-04-24T06:44:02.129732+0000","last_peered":"2024-04-24T06:44:02.129732+0000","last_clean":"2024-04-24T06:44:02.129732+0000","last_became_active":"2024-04-24T06:41:35.301262+0000","last_became_peered":"2024-04-24T06:41:35.301262+0000","last_unstale":"2024-04-24T06:44:02.129732+0000","last_undegraded":"2024-04-24T06:44:02.129732+0000","last_fullsized":"2024-04-24T06:44:02.129732+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:58:41.029511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.128025+0000","last_change":"2024-04-24T06:41:34.276926+0000","last_active":"2024-04-24T06:44:02.128025+0000","last_peered":"2024-04-24T06:44:02.128025+0000","last_clean":"2024-04-24T06:44:02.128025+0000","last_became_active":"2024-04-24T06:41:34.276539+0000","last_became_peered":"2024-04-24T06:41:34.276539+0000","last_unstale":"2024-04-24T06:44:02.128025+0000","last_undegraded":"2024-04-24T06:44:02.128025+0000","last_fullsized":"2024-04-24T06:44:02.128025+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:28:56.652584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.983056+0000","last_change":"2024-04-24T06:41:35.301052+0000","last_active":"2024-04-24T06:44:01.983056+0000","last_peered":"2024-04-24T06:44:01.983056+0000","last_clean":"2024-04-24T06:44:01.983056+0000","last_became_active":"2024-04-24T06:41:35.300650+0000","last_became_peered":"2024-04-24T06:41:35.300650+0000","last_unstale":"2024-04-24T06:44:01.983056+0000","last_undegraded":"2024-04-24T06:44:01.983056+0000","last_fullsized":"2024-04-24T06:44:01.983056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:06:54.290353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.980975+0000","last_change":"2024-04-24T06:41:34.273773+0000","last_active":"2024-04-24T06:44:01.980975+0000","last_peered":"2024-04-24T06:44:01.980975+0000","last_clean":"2024-04-24T06:44:01.980975+0000","last_became_active":"2024-04-24T06:41:34.273324+0000","last_became_peered":"2024-04-24T06:41:34.273324+0000","last_unstale":"2024-04-24T06:44:01.980975+0000","last_undegraded":"2024-04-24T06:44:01.980975+0000","last_fullsized":"2024-04-24T06:44:01.980975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:30:30.760740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.983425+0000","last_change":"2024-04-24T06:41:35.289565+0000","last_active":"2024-04-24T06:44:01.983425+0000","last_peered":"2024-04-24T06:44:01.983425+0000","last_clean":"2024-04-24T06:44:01.983425+0000","last_became_active":"2024-04-24T06:41:35.289185+0000","last_became_peered":"2024-04-24T06:41:35.289185+0000","last_unstale":"2024-04-24T06:44:01.983425+0000","last_undegraded":"2024-04-24T06:44:01.983425+0000","last_fullsized":"2024-04-24T06:44:01.983425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:26:28.345162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'39","reported_seq":69,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.261132+0000","last_change":"2024-04-24T06:41:34.285804+0000","last_active":"2024-04-24T06:44:02.261132+0000","last_peered":"2024-04-24T06:44:02.261132+0000","last_clean":"2024-04-24T06:44:02.261132+0000","last_became_active":"2024-04-24T06:41:34.285501+0000","last_became_peered":"2024-04-24T06:41:34.285501+0000","last_unstale":"2024-04-24T06:44:02.261132+0000","last_undegraded":"2024-04-24T06:44:02.261132+0000","last_fullsized":"2024-04-24T06:44:02.261132+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:25:15.804221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.983619+0000","last_change":"2024-04-24T06:41:35.300856+0000","last_active":"2024-04-24T06:44:01.983619+0000","last_peered":"2024-04-24T06:44:01.983619+0000","last_clean":"2024-04-24T06:44:01.983619+0000","last_became_active":"2024-04-24T06:41:35.300344+0000","last_became_peered":"2024-04-24T06:41:35.300344+0000","last_unstale":"2024-04-24T06:44:01.983619+0000","last_undegraded":"2024-04-24T06:44:01.983619+0000","last_fullsized":"2024-04-24T06:44:01.983619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:10:01.866954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.981589+0000","last_change":"2024-04-24T06:41:34.284112+0000","last_active":"2024-04-24T06:44:01.981589+0000","last_peered":"2024-04-24T06:44:01.981589+0000","last_clean":"2024-04-24T06:44:01.981589+0000","last_became_active":"2024-04-24T06:41:34.279317+0000","last_became_peered":"2024-04-24T06:41:34.279317+0000","last_unstale":"2024-04-24T06:44:01.981589+0000","last_undegraded":"2024-04-24T06:44:01.981589+0000","last_fullsized":"2024-04-24T06:44:01.981589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:49:45.673353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.983686+0000","last_change":"2024-04-24T06:41:35.288360+0000","last_active":"2024-04-24T06:44:01.983686+0000","last_peered":"2024-04-24T06:44:01.983686+0000","last_clean":"2024-04-24T06:44:01.983686+0000","last_became_active":"2024-04-24T06:41:35.288019+0000","last_became_peered":"2024-04-24T06:41:35.288019+0000","last_unstale":"2024-04-24T06:44:01.983686+0000","last_undegraded":"2024-04-24T06:44:01.983686+0000","last_fullsized":"2024-04-24T06:44:01.983686+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:33:49.927591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.528008+0000","last_change":"2024-04-24T06:41:35.286137+0000","last_active":"2024-04-24T06:44:02.528008+0000","last_peered":"2024-04-24T06:44:02.528008+0000","last_clean":"2024-04-24T06:44:02.528008+0000","last_became_active":"2024-04-24T06:41:35.285773+0000","last_became_peered":"2024-04-24T06:41:35.285773+0000","last_unstale":"2024-04-24T06:44:02.528008+0000","last_undegraded":"2024-04-24T06:44:02.528008+0000","last_fullsized":"2024-04-24T06:44:02.528008+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:29:32.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.980786+0000","last_change":"2024-04-24T06:41:35.283977+0000","last_active":"2024-04-24T06:44:01.980786+0000","last_peered":"2024-04-24T06:44:01.980786+0000","last_clean":"2024-04-24T06:44:01.980786+0000","last_became_active":"2024-04-24T06:41:35.282312+0000","last_became_peered":"2024-04-24T06:41:35.282312+0000","last_unstale":"2024-04-24T06:44:01.980786+0000","last_undegraded":"2024-04-24T06:44:01.980786+0000","last_fullsized":"2024-04-24T06:44:01.980786+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:41:36.463803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.980619+0000","last_change":"2024-04-24T06:41:35.300372+0000","last_active":"2024-04-24T06:44:01.980619+0000","last_peered":"2024-04-24T06:44:01.980619+0000","last_clean":"2024-04-24T06:44:01.980619+0000","last_became_active":"2024-04-24T06:41:35.300034+0000","last_became_peered":"2024-04-24T06:41:35.300034+0000","last_unstale":"2024-04-24T06:44:01.980619+0000","last_undegraded":"2024-04-24T06:44:01.980619+0000","last_fullsized":"2024-04-24T06:44:01.980619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:56:23.732798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.324944+0000","last_change":"2024-04-24T06:41:35.301619+0000","last_active":"2024-04-24T06:44:02.324944+0000","last_peered":"2024-04-24T06:44:02.324944+0000","last_clean":"2024-04-24T06:44:02.324944+0000","last_became_active":"2024-04-24T06:41:35.298897+0000","last_became_peered":"2024-04-24T06:41:35.298897+0000","last_unstale":"2024-04-24T06:44:02.324944+0000","last_undegraded":"2024-04-24T06:44:02.324944+0000","last_fullsized":"2024-04-24T06:44:02.324944+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:26:11.583079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.981573+0000","last_change":"2024-04-24T06:41:35.304905+0000","last_active":"2024-04-24T06:44:01.981573+0000","last_peered":"2024-04-24T06:44:01.981573+0000","last_clean":"2024-04-24T06:44:01.981573+0000","last_became_active":"2024-04-24T06:41:35.304407+0000","last_became_peered":"2024-04-24T06:41:35.304407+0000","last_unstale":"2024-04-24T06:44:01.981573+0000","last_undegraded":"2024-04-24T06:44:01.981573+0000","last_fullsized":"2024-04-24T06:44:01.981573+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:38:07.417898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.528268+0000","last_change":"2024-04-24T06:41:35.288646+0000","last_active":"2024-04-24T06:44:02.528268+0000","last_peered":"2024-04-24T06:44:02.528268+0000","last_clean":"2024-04-24T06:44:02.528268+0000","last_became_active":"2024-04-24T06:41:35.288266+0000","last_became_peered":"2024-04-24T06:41:35.288266+0000","last_unstale":"2024-04-24T06:44:02.528268+0000","last_undegraded":"2024-04-24T06:44:02.528268+0000","last_fullsized":"2024-04-24T06:44:02.528268+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:59:16.526712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.981229+0000","last_change":"2024-04-24T06:41:35.285001+0000","last_active":"2024-04-24T06:44:01.981229+0000","last_peered":"2024-04-24T06:44:01.981229+0000","last_clean":"2024-04-24T06:44:01.981229+0000","last_became_active":"2024-04-24T06:41:35.284671+0000","last_became_peered":"2024-04-24T06:41:35.284671+0000","last_unstale":"2024-04-24T06:44:01.981229+0000","last_undegraded":"2024-04-24T06:44:01.981229+0000","last_fullsized":"2024-04-24T06:44:01.981229+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:37:43.534018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.129692+0000","last_change":"2024-04-24T06:41:35.299124+0000","last_active":"2024-04-24T06:44:02.129692+0000","last_peered":"2024-04-24T06:44:02.129692+0000","last_clean":"2024-04-24T06:44:02.129692+0000","last_became_active":"2024-04-24T06:41:35.298246+0000","last_became_peered":"2024-04-24T06:41:35.298246+0000","last_unstale":"2024-04-24T06:44:02.129692+0000","last_undegraded":"2024-04-24T06:44:02.129692+0000","last_fullsized":"2024-04-24T06:44:02.129692+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:07:22.649874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.980060+0000","last_change":"2024-04-24T06:42:42.888183+0000","last_active":"2024-04-24T06:44:01.980060+0000","last_peered":"2024-04-24T06:44:01.980060+0000","last_clean":"2024-04-24T06:44:01.980060+0000","last_became_active":"2024-04-24T06:42:42.887373+0000","last_became_peered":"2024-04-24T06:42:42.887373+0000","last_unstale":"2024-04-24T06:44:01.980060+0000","last_undegraded":"2024-04-24T06:44:01.980060+0000","last_fullsized":"2024-04-24T06:44:01.980060+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:44:51.456753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.128989+0000","last_change":"2024-04-24T06:41:35.301619+0000","last_active":"2024-04-24T06:44:02.128989+0000","last_peered":"2024-04-24T06:44:02.128989+0000","last_clean":"2024-04-24T06:44:02.128989+0000","last_became_active":"2024-04-24T06:41:35.297987+0000","last_became_peered":"2024-04-24T06:41:35.297987+0000","last_unstale":"2024-04-24T06:44:02.128989+0000","last_undegraded":"2024-04-24T06:44:02.128989+0000","last_fullsized":"2024-04-24T06:44:02.128989+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:06:01.235593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.319683+0000","last_change":"2024-04-24T06:41:35.304347+0000","last_active":"2024-04-24T06:44:02.319683+0000","last_peered":"2024-04-24T06:44:02.319683+0000","last_clean":"2024-04-24T06:44:02.319683+0000","last_became_active":"2024-04-24T06:41:35.303971+0000","last_became_peered":"2024-04-24T06:41:35.303971+0000","last_unstale":"2024-04-24T06:44:02.319683+0000","last_undegraded":"2024-04-24T06:44:02.319683+0000","last_fullsized":"2024-04-24T06:44:02.319683+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:31:54.813243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.981029+0000","last_change":"2024-04-24T06:41:35.308314+0000","last_active":"2024-04-24T06:44:01.981029+0000","last_peered":"2024-04-24T06:44:01.981029+0000","last_clean":"2024-04-24T06:44:01.981029+0000","last_became_active":"2024-04-24T06:41:35.306581+0000","last_became_peered":"2024-04-24T06:41:35.306581+0000","last_unstale":"2024-04-24T06:44:01.981029+0000","last_undegraded":"2024-04-24T06:44:01.981029+0000","last_fullsized":"2024-04-24T06:44:01.981029+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:34:43.095087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.528089+0000","last_change":"2024-04-24T06:41:35.289166+0000","last_active":"2024-04-24T06:44:02.528089+0000","last_peered":"2024-04-24T06:44:02.528089+0000","last_clean":"2024-04-24T06:44:02.528089+0000","last_became_active":"2024-04-24T06:41:35.287672+0000","last_became_peered":"2024-04-24T06:41:35.287672+0000","last_unstale":"2024-04-24T06:44:02.528089+0000","last_undegraded":"2024-04-24T06:44:02.528089+0000","last_fullsized":"2024-04-24T06:44:02.528089+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:31:45.403387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":595775,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":981,"num_read_kb":598,"num_write":282,"num_write_kb":1430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1787325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":220,"ondisk_log_size":220,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351017,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":460828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612654,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28792,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716360,"statfs":{"total":95995035648,"available":95965552640,"internally_reserved":0,"allocated":1818624,"data_stored":1259789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776180,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":978944,"data_stored":461030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037819,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28376,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716776,"statfs":{"total":95995035648,"available":95965978624,"internally_reserved":0,"allocated":1388544,"data_stored":863122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299455,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28572,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716580,"statfs":{"total":95995035648,"available":95965777920,"internally_reserved":0,"allocated":1589248,"data_stored":1062227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462982,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":469375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574945,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1052672,"data_stored":535735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738471,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":987136,"data_stored":461690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411423,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1073152,"data_stored":537442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182485,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28384,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716768,"statfs":{"total":95995035648,"available":95965970432,"internally_reserved":0,"allocated":1396736,"data_stored":863422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920849,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":470203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724620,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32872,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712280,"statfs":{"total":95995035648,"available":95961374720,"internally_reserved":0,"allocated":1736704,"data_stored":1193120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":68274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":133467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":393954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":200136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T06:44:07.749 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T06:44:07.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:07 smithi037 ceph-mon[32864]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:44:07.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:07 smithi037 ceph-mon[32864]: mgrmap e25: x(active, since 4s), standbys: y, z 2024-04-24T06:44:07.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:07 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/244738049' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T06:44:07.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:07 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/3534096658' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:44:07.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:07 smithi137 ceph-mon[33017]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:44:07.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:07 smithi137 ceph-mon[33017]: mgrmap e25: x(active, since 4s), standbys: y, z 2024-04-24T06:44:07.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:07 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/244738049' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T06:44:07.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:07 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/3534096658' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:44:07.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.991+0000 7fa980e00640 1 -- 172.21.15.5:0/933458708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa97c1030d0 msgr2=0x7fa97c103550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:07.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.991+0000 7fa980e00640 1 --2- 172.21.15.5:0/933458708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa97c1030d0 0x7fa97c103550 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa968009900 tx=0x7fa96802a3f0 comp rx=0 tx=0).stop 2024-04-24T06:44:07.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.991+0000 7fa980e00640 1 -- 172.21.15.5:0/933458708 shutdown_connections 2024-04-24T06:44:07.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.991+0000 7fa980e00640 1 --2- 172.21.15.5:0/933458708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa97c103a90 0x7fa97c10a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:07.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.991+0000 7fa980e00640 1 --2- 172.21.15.5:0/933458708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa97c1030d0 0x7fa97c103550 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:07.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.991+0000 7fa980e00640 1 --2- 172.21.15.5:0/933458708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa97c101f00 0x7fa97c102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:07.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.991+0000 7fa980e00640 1 -- 172.21.15.5:0/933458708 >> 172.21.15.5:0/933458708 conn(0x7fa97c0fd4c0 msgr2=0x7fa97c0ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:07.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.991+0000 7fa980e00640 1 -- 172.21.15.5:0/933458708 shutdown_connections 2024-04-24T06:44:07.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.991+0000 7fa980e00640 1 -- 172.21.15.5:0/933458708 wait complete. 2024-04-24T06:44:07.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.992+0000 7fa980e00640 1 Processor -- start 2024-04-24T06:44:07.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.992+0000 7fa980e00640 1 -- start start 2024-04-24T06:44:07.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.993+0000 7fa980e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa97c101f00 0x7fa97c19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:07.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.993+0000 7fa980e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa97c1030d0 0x7fa97c19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:07.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.993+0000 7fa980e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa97c103a90 0x7fa97c1a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:07.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.993+0000 7fa980e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa97c075030 con 0x7fa97c1030d0 2024-04-24T06:44:07.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.993+0000 7fa980e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa97c074eb0 con 0x7fa97c103a90 2024-04-24T06:44:07.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.993+0000 7fa980e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa97c0751b0 con 0x7fa97c101f00 2024-04-24T06:44:07.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.993+0000 7fa97be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa97c103a90 0x7fa97c1a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:07.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.993+0000 7fa97aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa97c1030d0 0x7fa97c19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:07.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.993+0000 7fa97b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa97c101f00 0x7fa97c19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:07.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.993+0000 7fa97be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa97c103a90 0x7fa97c1a19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:46112/0 (socket says 172.21.15.5:46112) 2024-04-24T06:44:07.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.993+0000 7fa97be00640 1 -- 172.21.15.5:0/1076299551 learned_addr learned my addr 172.21.15.5:0/1076299551 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:07.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.993+0000 7fa97be00640 1 -- 172.21.15.5:0/1076299551 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa97c101f00 msgr2=0x7fa97c19a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:07.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.994+0000 7fa97be00640 1 --2- 172.21.15.5:0/1076299551 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa97c101f00 0x7fa97c19a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:07.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.994+0000 7fa97be00640 1 -- 172.21.15.5:0/1076299551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa97c1030d0 msgr2=0x7fa97c19a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:07.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.994+0000 7fa97be00640 1 --2- 172.21.15.5:0/1076299551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa97c1030d0 0x7fa97c19a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:07.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.994+0000 7fa97be00640 1 -- 172.21.15.5:0/1076299551 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa97c1a2040 con 0x7fa97c103a90 2024-04-24T06:44:07.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.994+0000 7fa97b400640 1 --2- 172.21.15.5:0/1076299551 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa97c101f00 0x7fa97c19a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:44:07.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.994+0000 7fa97be00640 1 --2- 172.21.15.5:0/1076299551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa97c103a90 0x7fa97c1a19d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa97400e9b0 tx=0x7fa97400ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:07.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.994+0000 7fa973e00640 1 -- 172.21.15.5:0/1076299551 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97400cd80 con 0x7fa97c103a90 2024-04-24T06:44:07.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.994+0000 7fa980e00640 1 -- 172.21.15.5:0/1076299551 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa97c1a2330 con 0x7fa97c103a90 2024-04-24T06:44:07.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.994+0000 7fa973e00640 1 -- 172.21.15.5:0/1076299551 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9740035f0 con 0x7fa97c103a90 2024-04-24T06:44:07.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.994+0000 7fa973e00640 1 -- 172.21.15.5:0/1076299551 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa974010670 con 0x7fa97c103a90 2024-04-24T06:44:07.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.994+0000 7fa980e00640 1 -- 172.21.15.5:0/1076299551 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa97c1a2850 con 0x7fa97c103a90 2024-04-24T06:44:07.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.995+0000 7fa980e00640 1 -- 172.21.15.5:0/1076299551 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa97c108ab0 con 0x7fa97c103a90 2024-04-24T06:44:07.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.997+0000 7fa973e00640 1 -- 172.21.15.5:0/1076299551 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138872+0+0 (secure 0 0 0) 0x7fa9740037a0 con 0x7fa97c103a90 2024-04-24T06:44:08.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.999+0000 7fa973e00640 1 --2- 172.21.15.5:0/1076299551 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fa9600a3eb0 0x7fa9600a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:08.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.999+0000 7fa973e00640 1 -- 172.21.15.5:0/1076299551 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fa9740d0d50 con 0x7fa97c103a90 2024-04-24T06:44:08.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:07.999+0000 7fa97b400640 1 --2- 172.21.15.5:0/1076299551 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fa9600a3eb0 0x7fa9600a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:08.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.000+0000 7fa97b400640 1 --2- 172.21.15.5:0/1076299551 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fa9600a3eb0 0x7fa9600a6370 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa96c0099a0 tx=0x7fa96c009500 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:08.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.003+0000 7fa973e00640 1 -- 172.21.15.5:0/1076299551 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa97409d150 con 0x7fa97c103a90 2024-04-24T06:44:08.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.233+0000 7fa980e00640 1 -- 172.21.15.5:0/1076299551 --> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa97c062440 con 0x7fa9600a3eb0 2024-04-24T06:44:08.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.242+0000 7fa973e00640 1 -- 172.21.15.5:0/1076299551 <== mgr.25202 v2:172.21.15.5:6800/3154624651 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257767 (secure 0 0 0) 0x7fa97c062440 con 0x7fa9600a3eb0 2024-04-24T06:44:08.247 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:44:08.248 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-24T06:44:08.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.246+0000 7fa980e00640 1 -- 172.21.15.5:0/1076299551 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fa9600a3eb0 msgr2=0x7fa9600a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:08.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.246+0000 7fa980e00640 1 --2- 172.21.15.5:0/1076299551 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fa9600a3eb0 0x7fa9600a6370 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa96c0099a0 tx=0x7fa96c009500 comp rx=0 tx=0).stop 2024-04-24T06:44:08.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.246+0000 7fa980e00640 1 -- 172.21.15.5:0/1076299551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa97c103a90 msgr2=0x7fa97c1a19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:08.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.246+0000 7fa980e00640 1 --2- 172.21.15.5:0/1076299551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa97c103a90 0x7fa97c1a19d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa97400e9b0 tx=0x7fa97400ee70 comp rx=0 tx=0).stop 2024-04-24T06:44:08.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.246+0000 7fa980e00640 1 -- 172.21.15.5:0/1076299551 shutdown_connections 2024-04-24T06:44:08.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.246+0000 7fa980e00640 1 --2- 172.21.15.5:0/1076299551 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fa9600a3eb0 0x7fa9600a6370 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:08.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.246+0000 7fa980e00640 1 --2- 172.21.15.5:0/1076299551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa97c103a90 0x7fa97c1a19d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:08.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.246+0000 7fa980e00640 1 --2- 172.21.15.5:0/1076299551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa97c1030d0 0x7fa97c19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:08.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.246+0000 7fa980e00640 1 --2- 172.21.15.5:0/1076299551 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa97c101f00 0x7fa97c19a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:08.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.246+0000 7fa980e00640 1 -- 172.21.15.5:0/1076299551 >> 172.21.15.5:0/1076299551 conn(0x7fa97c0fd4c0 msgr2=0x7fa97c0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:08.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.246+0000 7fa980e00640 1 -- 172.21.15.5:0/1076299551 shutdown_connections 2024-04-24T06:44:08.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.246+0000 7fa980e00640 1 -- 172.21.15.5:0/1076299551 wait complete. 2024-04-24T06:44:08.268 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-04-24T06:44:08.005449+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1186377,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1117,"num_read_kb":717,"num_write":465,"num_write_kb":4370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":859,"ondisk_log_size":859,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376268,"kb_used_data":14608,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565556,"statfs":{"total":1151940427776,"available":1151555129344,"internally_reserved":0,"allocated":14958592,"data_stored":8637983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.971872"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.528183+0000","last_change":"2024-04-24T06:41:35.296451+0000","last_active":"2024-04-24T06:44:02.528183+0000","last_peered":"2024-04-24T06:44:02.528183+0000","last_clean":"2024-04-24T06:44:02.528183+0000","last_became_active":"2024-04-24T06:41:35.295733+0000","last_became_peered":"2024-04-24T06:41:35.295733+0000","last_unstale":"2024-04-24T06:44:02.528183+0000","last_undegraded":"2024-04-24T06:44:02.528183+0000","last_fullsized":"2024-04-24T06:44:02.528183+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:04:15.072644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.325058+0000","last_change":"2024-04-24T06:41:35.293359+0000","last_active":"2024-04-24T06:44:02.325058+0000","last_peered":"2024-04-24T06:44:02.325058+0000","last_clean":"2024-04-24T06:44:02.325058+0000","last_became_active":"2024-04-24T06:41:35.287371+0000","last_became_peered":"2024-04-24T06:41:35.287371+0000","last_unstale":"2024-04-24T06:44:02.325058+0000","last_undegraded":"2024-04-24T06:44:02.325058+0000","last_fullsized":"2024-04-24T06:44:02.325058+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:03:39.376495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.325258+0000","last_change":"2024-04-24T06:41:35.272734+0000","last_active":"2024-04-24T06:44:02.325258+0000","last_peered":"2024-04-24T06:44:02.325258+0000","last_clean":"2024-04-24T06:44:02.325258+0000","last_became_active":"2024-04-24T06:41:35.271685+0000","last_became_peered":"2024-04-24T06:41:35.271685+0000","last_unstale":"2024-04-24T06:44:02.325258+0000","last_undegraded":"2024-04-24T06:44:02.325258+0000","last_fullsized":"2024-04-24T06:44:02.325258+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:08:15.057918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.323790+0000","last_change":"2024-04-24T06:41:35.288574+0000","last_active":"2024-04-24T06:44:02.323790+0000","last_peered":"2024-04-24T06:44:02.323790+0000","last_clean":"2024-04-24T06:44:02.323790+0000","last_became_active":"2024-04-24T06:41:35.288229+0000","last_became_peered":"2024-04-24T06:41:35.288229+0000","last_unstale":"2024-04-24T06:44:02.323790+0000","last_undegraded":"2024-04-24T06:44:02.323790+0000","last_fullsized":"2024-04-24T06:44:02.323790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:58:18.028067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.128022+0000","last_change":"2024-04-24T06:41:35.310226+0000","last_active":"2024-04-24T06:44:02.128022+0000","last_peered":"2024-04-24T06:44:02.128022+0000","last_clean":"2024-04-24T06:44:02.128022+0000","last_became_active":"2024-04-24T06:41:35.309853+0000","last_became_peered":"2024-04-24T06:41:35.309853+0000","last_unstale":"2024-04-24T06:44:02.128022+0000","last_undegraded":"2024-04-24T06:44:02.128022+0000","last_fullsized":"2024-04-24T06:44:02.128022+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:34:57.767158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.138791+0000","last_change":"2024-04-24T06:41:35.294259+0000","last_active":"2024-04-24T06:44:02.138791+0000","last_peered":"2024-04-24T06:44:02.138791+0000","last_clean":"2024-04-24T06:44:02.138791+0000","last_became_active":"2024-04-24T06:41:35.293827+0000","last_became_peered":"2024-04-24T06:41:35.293827+0000","last_unstale":"2024-04-24T06:44:02.138791+0000","last_undegraded":"2024-04-24T06:44:02.138791+0000","last_fullsized":"2024-04-24T06:44:02.138791+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:09:23.302827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.982776+0000","last_change":"2024-04-24T06:41:35.295095+0000","last_active":"2024-04-24T06:44:01.982776+0000","last_peered":"2024-04-24T06:44:01.982776+0000","last_clean":"2024-04-24T06:44:01.982776+0000","last_became_active":"2024-04-24T06:41:35.294743+0000","last_became_peered":"2024-04-24T06:41:35.294743+0000","last_unstale":"2024-04-24T06:44:01.982776+0000","last_undegraded":"2024-04-24T06:44:01.982776+0000","last_fullsized":"2024-04-24T06:44:01.982776+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:06:09.473123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.125405+0000","last_change":"2024-04-24T06:42:42.886072+0000","last_active":"2024-04-24T06:44:02.125405+0000","last_peered":"2024-04-24T06:44:02.125405+0000","last_clean":"2024-04-24T06:44:02.125405+0000","last_became_active":"2024-04-24T06:42:42.882624+0000","last_became_peered":"2024-04-24T06:42:42.882624+0000","last_unstale":"2024-04-24T06:44:02.125405+0000","last_undegraded":"2024-04-24T06:44:02.125405+0000","last_fullsized":"2024-04-24T06:44:02.125405+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:53:02.523012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.320256+0000","last_change":"2024-04-24T06:41:35.283275+0000","last_active":"2024-04-24T06:44:02.320256+0000","last_peered":"2024-04-24T06:44:02.320256+0000","last_clean":"2024-04-24T06:44:02.320256+0000","last_became_active":"2024-04-24T06:41:35.282586+0000","last_became_peered":"2024-04-24T06:41:35.282586+0000","last_unstale":"2024-04-24T06:44:02.320256+0000","last_undegraded":"2024-04-24T06:44:02.320256+0000","last_fullsized":"2024-04-24T06:44:02.320256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:42:30.716149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.139163+0000","last_change":"2024-04-24T06:41:35.284986+0000","last_active":"2024-04-24T06:44:02.139163+0000","last_peered":"2024-04-24T06:44:02.139163+0000","last_clean":"2024-04-24T06:44:02.139163+0000","last_became_active":"2024-04-24T06:41:35.284628+0000","last_became_peered":"2024-04-24T06:41:35.284628+0000","last_unstale":"2024-04-24T06:44:02.139163+0000","last_undegraded":"2024-04-24T06:44:02.139163+0000","last_fullsized":"2024-04-24T06:44:02.139163+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:29:38.479593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.983393+0000","last_change":"2024-04-24T06:41:35.290353+0000","last_active":"2024-04-24T06:44:01.983393+0000","last_peered":"2024-04-24T06:44:01.983393+0000","last_clean":"2024-04-24T06:44:01.983393+0000","last_became_active":"2024-04-24T06:41:35.289987+0000","last_became_peered":"2024-04-24T06:41:35.289987+0000","last_unstale":"2024-04-24T06:44:01.983393+0000","last_undegraded":"2024-04-24T06:44:01.983393+0000","last_fullsized":"2024-04-24T06:44:01.983393+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:56:54.914426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.323831+0000","last_change":"2024-04-24T06:41:35.289462+0000","last_active":"2024-04-24T06:44:02.323831+0000","last_peered":"2024-04-24T06:44:02.323831+0000","last_clean":"2024-04-24T06:44:02.323831+0000","last_became_active":"2024-04-24T06:41:35.289123+0000","last_became_peered":"2024-04-24T06:41:35.289123+0000","last_unstale":"2024-04-24T06:44:02.323831+0000","last_undegraded":"2024-04-24T06:44:02.323831+0000","last_fullsized":"2024-04-24T06:44:02.323831+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:51:06.792508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.981156+0000","last_change":"2024-04-24T06:41:35.298320+0000","last_active":"2024-04-24T06:44:01.981156+0000","last_peered":"2024-04-24T06:44:01.981156+0000","last_clean":"2024-04-24T06:44:01.981156+0000","last_became_active":"2024-04-24T06:41:35.297839+0000","last_became_peered":"2024-04-24T06:41:35.297839+0000","last_unstale":"2024-04-24T06:44:01.981156+0000","last_undegraded":"2024-04-24T06:44:01.981156+0000","last_fullsized":"2024-04-24T06:44:01.981156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:10:36.423514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.324512+0000","last_change":"2024-04-24T06:41:35.301924+0000","last_active":"2024-04-24T06:44:02.324512+0000","last_peered":"2024-04-24T06:44:02.324512+0000","last_clean":"2024-04-24T06:44:02.324512+0000","last_became_active":"2024-04-24T06:41:35.299114+0000","last_became_peered":"2024-04-24T06:41:35.299114+0000","last_unstale":"2024-04-24T06:44:02.324512+0000","last_undegraded":"2024-04-24T06:44:02.324512+0000","last_fullsized":"2024-04-24T06:44:02.324512+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:34:46.194266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.323875+0000","last_change":"2024-04-24T06:41:35.284885+0000","last_active":"2024-04-24T06:44:02.323875+0000","last_peered":"2024-04-24T06:44:02.323875+0000","last_clean":"2024-04-24T06:44:02.323875+0000","last_became_active":"2024-04-24T06:41:35.284244+0000","last_became_peered":"2024-04-24T06:41:35.284244+0000","last_unstale":"2024-04-24T06:44:02.323875+0000","last_undegraded":"2024-04-24T06:44:02.323875+0000","last_fullsized":"2024-04-24T06:44:02.323875+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:30:40.279905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.319433+0000","last_change":"2024-04-24T06:41:35.286363+0000","last_active":"2024-04-24T06:44:02.319433+0000","last_peered":"2024-04-24T06:44:02.319433+0000","last_clean":"2024-04-24T06:44:02.319433+0000","last_became_active":"2024-04-24T06:41:35.286024+0000","last_became_peered":"2024-04-24T06:41:35.286024+0000","last_unstale":"2024-04-24T06:44:02.319433+0000","last_undegraded":"2024-04-24T06:44:02.319433+0000","last_fullsized":"2024-04-24T06:44:02.319433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:43:35.728990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.528311+0000","last_change":"2024-04-24T06:41:35.304005+0000","last_active":"2024-04-24T06:44:02.528311+0000","last_peered":"2024-04-24T06:44:02.528311+0000","last_clean":"2024-04-24T06:44:02.528311+0000","last_became_active":"2024-04-24T06:41:35.303659+0000","last_became_peered":"2024-04-24T06:41:35.303659+0000","last_unstale":"2024-04-24T06:44:02.528311+0000","last_undegraded":"2024-04-24T06:44:02.528311+0000","last_fullsized":"2024-04-24T06:44:02.528311+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:05:07.215927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.126161+0000","last_change":"2024-04-24T06:41:35.296218+0000","last_active":"2024-04-24T06:44:02.126161+0000","last_peered":"2024-04-24T06:44:02.126161+0000","last_clean":"2024-04-24T06:44:02.126161+0000","last_became_active":"2024-04-24T06:41:35.295486+0000","last_became_peered":"2024-04-24T06:41:35.295486+0000","last_unstale":"2024-04-24T06:44:02.126161+0000","last_undegraded":"2024-04-24T06:44:02.126161+0000","last_fullsized":"2024-04-24T06:44:02.126161+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:17:25.844899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.980909+0000","last_change":"2024-04-24T06:42:42.886291+0000","last_active":"2024-04-24T06:44:01.980909+0000","last_peered":"2024-04-24T06:44:01.980909+0000","last_clean":"2024-04-24T06:44:01.980909+0000","last_became_active":"2024-04-24T06:42:42.885781+0000","last_became_peered":"2024-04-24T06:42:42.885781+0000","last_unstale":"2024-04-24T06:44:01.980909+0000","last_undegraded":"2024-04-24T06:44:01.980909+0000","last_fullsized":"2024-04-24T06:44:01.980909+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:01:44.377335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.125607+0000","last_change":"2024-04-24T06:41:34.282140+0000","last_active":"2024-04-24T06:44:02.125607+0000","last_peered":"2024-04-24T06:44:02.125607+0000","last_clean":"2024-04-24T06:44:02.125607+0000","last_became_active":"2024-04-24T06:41:34.281608+0000","last_became_peered":"2024-04-24T06:41:34.281608+0000","last_unstale":"2024-04-24T06:44:02.125607+0000","last_undegraded":"2024-04-24T06:44:02.125607+0000","last_fullsized":"2024-04-24T06:44:02.125607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:32:06.727639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.129957+0000","last_change":"2024-04-24T06:41:35.308847+0000","last_active":"2024-04-24T06:44:02.129957+0000","last_peered":"2024-04-24T06:44:02.129957+0000","last_clean":"2024-04-24T06:44:02.129957+0000","last_became_active":"2024-04-24T06:41:35.308256+0000","last_became_peered":"2024-04-24T06:41:35.308256+0000","last_unstale":"2024-04-24T06:44:02.129957+0000","last_undegraded":"2024-04-24T06:44:02.129957+0000","last_fullsized":"2024-04-24T06:44:02.129957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:21:49.338737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.130066+0000","last_change":"2024-04-24T06:41:34.285819+0000","last_active":"2024-04-24T06:44:02.130066+0000","last_peered":"2024-04-24T06:44:02.130066+0000","last_clean":"2024-04-24T06:44:02.130066+0000","last_became_active":"2024-04-24T06:41:34.282393+0000","last_became_peered":"2024-04-24T06:41:34.282393+0000","last_unstale":"2024-04-24T06:44:02.130066+0000","last_undegraded":"2024-04-24T06:44:02.130066+0000","last_fullsized":"2024-04-24T06:44:02.130066+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:11:18.969178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.325587+0000","last_change":"2024-04-24T06:41:35.297437+0000","last_active":"2024-04-24T06:44:02.325587+0000","last_peered":"2024-04-24T06:44:02.325587+0000","last_clean":"2024-04-24T06:44:02.325587+0000","last_became_active":"2024-04-24T06:41:35.295195+0000","last_became_peered":"2024-04-24T06:41:35.295195+0000","last_unstale":"2024-04-24T06:44:02.325587+0000","last_undegraded":"2024-04-24T06:44:02.325587+0000","last_fullsized":"2024-04-24T06:44:02.325587+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:21:51.219219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.128352+0000","last_change":"2024-04-24T06:41:34.275768+0000","last_active":"2024-04-24T06:44:02.128352+0000","last_peered":"2024-04-24T06:44:02.128352+0000","last_clean":"2024-04-24T06:44:02.128352+0000","last_became_active":"2024-04-24T06:41:34.275426+0000","last_became_peered":"2024-04-24T06:41:34.275426+0000","last_unstale":"2024-04-24T06:44:02.128352+0000","last_undegraded":"2024-04-24T06:44:02.128352+0000","last_fullsized":"2024-04-24T06:44:02.128352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:01:08.735480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.319850+0000","last_change":"2024-04-24T06:41:35.305734+0000","last_active":"2024-04-24T06:44:02.319850+0000","last_peered":"2024-04-24T06:44:02.319850+0000","last_clean":"2024-04-24T06:44:02.319850+0000","last_became_active":"2024-04-24T06:41:35.305365+0000","last_became_peered":"2024-04-24T06:41:35.305365+0000","last_unstale":"2024-04-24T06:44:02.319850+0000","last_undegraded":"2024-04-24T06:44:02.319850+0000","last_fullsized":"2024-04-24T06:44:02.319850+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:38:52.740107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.128296+0000","last_change":"2024-04-24T06:41:34.278317+0000","last_active":"2024-04-24T06:44:02.128296+0000","last_peered":"2024-04-24T06:44:02.128296+0000","last_clean":"2024-04-24T06:44:02.128296+0000","last_became_active":"2024-04-24T06:41:34.277862+0000","last_became_peered":"2024-04-24T06:41:34.277862+0000","last_unstale":"2024-04-24T06:44:02.128296+0000","last_undegraded":"2024-04-24T06:44:02.128296+0000","last_fullsized":"2024-04-24T06:44:02.128296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:19:50.444136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.022666+0000","last_change":"2024-04-24T06:41:35.282648+0000","last_active":"2024-04-24T06:44:02.022666+0000","last_peered":"2024-04-24T06:44:02.022666+0000","last_clean":"2024-04-24T06:44:02.022666+0000","last_became_active":"2024-04-24T06:41:35.282335+0000","last_became_peered":"2024-04-24T06:41:35.282335+0000","last_unstale":"2024-04-24T06:44:02.022666+0000","last_undegraded":"2024-04-24T06:44:02.022666+0000","last_fullsized":"2024-04-24T06:44:02.022666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:30:11.909067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":174,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.568985+0000","last_change":"2024-04-24T06:41:34.274405+0000","last_active":"2024-04-24T06:44:02.568985+0000","last_peered":"2024-04-24T06:44:02.568985+0000","last_clean":"2024-04-24T06:44:02.568985+0000","last_became_active":"2024-04-24T06:41:34.273820+0000","last_became_peered":"2024-04-24T06:41:34.273820+0000","last_unstale":"2024-04-24T06:44:02.568985+0000","last_undegraded":"2024-04-24T06:44:02.568985+0000","last_fullsized":"2024-04-24T06:44:02.568985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:11:26.119236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.128021+0000","last_change":"2024-04-24T06:41:35.294133+0000","last_active":"2024-04-24T06:44:02.128021+0000","last_peered":"2024-04-24T06:44:02.128021+0000","last_clean":"2024-04-24T06:44:02.128021+0000","last_became_active":"2024-04-24T06:41:35.293741+0000","last_became_peered":"2024-04-24T06:41:35.293741+0000","last_unstale":"2024-04-24T06:44:02.128021+0000","last_undegraded":"2024-04-24T06:44:02.128021+0000","last_fullsized":"2024-04-24T06:44:02.128021+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:09:12.495187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.138617+0000","last_change":"2024-04-24T06:42:42.927397+0000","last_active":"2024-04-24T06:44:02.138617+0000","last_peered":"2024-04-24T06:44:02.138617+0000","last_clean":"2024-04-24T06:44:02.138617+0000","last_became_active":"2024-04-24T06:42:42.924070+0000","last_became_peered":"2024-04-24T06:42:42.924070+0000","last_unstale":"2024-04-24T06:44:02.138617+0000","last_undegraded":"2024-04-24T06:44:02.138617+0000","last_fullsized":"2024-04-24T06:44:02.138617+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:19:47.319762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.022818+0000","last_change":"2024-04-24T06:42:43.166909+0000","last_active":"2024-04-24T06:44:02.022818+0000","last_peered":"2024-04-24T06:44:02.022818+0000","last_clean":"2024-04-24T06:44:02.022818+0000","last_became_active":"2024-04-24T06:42:43.166559+0000","last_became_peered":"2024-04-24T06:42:43.166559+0000","last_unstale":"2024-04-24T06:44:02.022818+0000","last_undegraded":"2024-04-24T06:44:02.022818+0000","last_fullsized":"2024-04-24T06:44:02.022818+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:26:42.537506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.980468+0000","last_change":"2024-04-24T06:41:34.284958+0000","last_active":"2024-04-24T06:44:01.980468+0000","last_peered":"2024-04-24T06:44:01.980468+0000","last_clean":"2024-04-24T06:44:01.980468+0000","last_became_active":"2024-04-24T06:41:34.284279+0000","last_became_peered":"2024-04-24T06:41:34.284279+0000","last_unstale":"2024-04-24T06:44:01.980468+0000","last_undegraded":"2024-04-24T06:44:01.980468+0000","last_fullsized":"2024-04-24T06:44:01.980468+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:58:19.316091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.128861+0000","last_change":"2024-04-24T06:41:35.307280+0000","last_active":"2024-04-24T06:44:02.128861+0000","last_peered":"2024-04-24T06:44:02.128861+0000","last_clean":"2024-04-24T06:44:02.128861+0000","last_became_active":"2024-04-24T06:41:35.306908+0000","last_became_peered":"2024-04-24T06:41:35.306908+0000","last_unstale":"2024-04-24T06:44:02.128861+0000","last_undegraded":"2024-04-24T06:44:02.128861+0000","last_fullsized":"2024-04-24T06:44:02.128861+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:54:31.306302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.022540+0000","last_change":"2024-04-24T06:41:34.272221+0000","last_active":"2024-04-24T06:44:02.022540+0000","last_peered":"2024-04-24T06:44:02.022540+0000","last_clean":"2024-04-24T06:44:02.022540+0000","last_became_active":"2024-04-24T06:41:34.271748+0000","last_became_peered":"2024-04-24T06:41:34.271748+0000","last_unstale":"2024-04-24T06:44:02.022540+0000","last_undegraded":"2024-04-24T06:44:02.022540+0000","last_fullsized":"2024-04-24T06:44:02.022540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:29:02.667285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.125592+0000","last_change":"2024-04-24T06:41:35.309608+0000","last_active":"2024-04-24T06:44:02.125592+0000","last_peered":"2024-04-24T06:44:02.125592+0000","last_clean":"2024-04-24T06:44:02.125592+0000","last_became_active":"2024-04-24T06:41:35.309227+0000","last_became_peered":"2024-04-24T06:41:35.309227+0000","last_unstale":"2024-04-24T06:44:02.125592+0000","last_undegraded":"2024-04-24T06:44:02.125592+0000","last_fullsized":"2024-04-24T06:44:02.125592+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:51:23.151426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":169,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.567739+0000","last_change":"2024-04-24T06:41:34.281885+0000","last_active":"2024-04-24T06:44:02.567739+0000","last_peered":"2024-04-24T06:44:02.567739+0000","last_clean":"2024-04-24T06:44:02.567739+0000","last_became_active":"2024-04-24T06:41:34.281466+0000","last_became_peered":"2024-04-24T06:41:34.281466+0000","last_unstale":"2024-04-24T06:44:02.567739+0000","last_undegraded":"2024-04-24T06:44:02.567739+0000","last_fullsized":"2024-04-24T06:44:02.567739+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:20:08.500343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":143,"num_read_kb":143,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.320075+0000","last_change":"2024-04-24T06:41:35.292714+0000","last_active":"2024-04-24T06:44:02.320075+0000","last_peered":"2024-04-24T06:44:02.320075+0000","last_clean":"2024-04-24T06:44:02.320075+0000","last_became_active":"2024-04-24T06:41:35.291950+0000","last_became_peered":"2024-04-24T06:41:35.291950+0000","last_unstale":"2024-04-24T06:44:02.320075+0000","last_undegraded":"2024-04-24T06:44:02.320075+0000","last_fullsized":"2024-04-24T06:44:02.320075+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:42:18.049746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.983538+0000","last_change":"2024-04-24T06:41:34.275218+0000","last_active":"2024-04-24T06:44:01.983538+0000","last_peered":"2024-04-24T06:44:01.983538+0000","last_clean":"2024-04-24T06:44:01.983538+0000","last_became_active":"2024-04-24T06:41:34.274824+0000","last_became_peered":"2024-04-24T06:41:34.274824+0000","last_unstale":"2024-04-24T06:44:01.983538+0000","last_undegraded":"2024-04-24T06:44:01.983538+0000","last_fullsized":"2024-04-24T06:44:01.983538+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:09:57.271304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.325532+0000","last_change":"2024-04-24T06:42:42.883478+0000","last_active":"2024-04-24T06:44:02.325532+0000","last_peered":"2024-04-24T06:44:02.325532+0000","last_clean":"2024-04-24T06:44:02.325532+0000","last_became_active":"2024-04-24T06:42:42.882808+0000","last_became_peered":"2024-04-24T06:42:42.882808+0000","last_unstale":"2024-04-24T06:44:02.325532+0000","last_undegraded":"2024-04-24T06:44:02.325532+0000","last_fullsized":"2024-04-24T06:44:02.325532+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:17:11.514041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.022807+0000","last_change":"2024-04-24T06:41:34.277372+0000","last_active":"2024-04-24T06:44:02.022807+0000","last_peered":"2024-04-24T06:44:02.022807+0000","last_clean":"2024-04-24T06:44:02.022807+0000","last_became_active":"2024-04-24T06:41:34.276997+0000","last_became_peered":"2024-04-24T06:41:34.276997+0000","last_unstale":"2024-04-24T06:44:02.022807+0000","last_undegraded":"2024-04-24T06:44:02.022807+0000","last_fullsized":"2024-04-24T06:44:02.022807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:52:35.695109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.320091+0000","last_change":"2024-04-24T06:41:35.290233+0000","last_active":"2024-04-24T06:44:02.320091+0000","last_peered":"2024-04-24T06:44:02.320091+0000","last_clean":"2024-04-24T06:44:02.320091+0000","last_became_active":"2024-04-24T06:41:35.289885+0000","last_became_peered":"2024-04-24T06:41:35.289885+0000","last_unstale":"2024-04-24T06:44:02.320091+0000","last_undegraded":"2024-04-24T06:44:02.320091+0000","last_fullsized":"2024-04-24T06:44:02.320091+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:35:54.293308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'33","reported_seq":352,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.569436+0000","last_change":"2024-04-24T06:41:34.267693+0000","last_active":"2024-04-24T06:44:02.569436+0000","last_peered":"2024-04-24T06:44:02.569436+0000","last_clean":"2024-04-24T06:44:02.569436+0000","last_became_active":"2024-04-24T06:41:34.267298+0000","last_became_peered":"2024-04-24T06:41:34.267298+0000","last_unstale":"2024-04-24T06:44:02.569436+0000","last_undegraded":"2024-04-24T06:44:02.569436+0000","last_fullsized":"2024-04-24T06:44:02.569436+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:21:16.015695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66567,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":297,"num_read_kb":213,"num_write":34,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.128194+0000","last_change":"2024-04-24T06:41:35.286587+0000","last_active":"2024-04-24T06:44:02.128194+0000","last_peered":"2024-04-24T06:44:02.128194+0000","last_clean":"2024-04-24T06:44:02.128194+0000","last_became_active":"2024-04-24T06:41:35.286239+0000","last_became_peered":"2024-04-24T06:41:35.286239+0000","last_unstale":"2024-04-24T06:44:02.128194+0000","last_undegraded":"2024-04-24T06:44:02.128194+0000","last_fullsized":"2024-04-24T06:44:02.128194+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:18:56.013425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.527938+0000","last_change":"2024-04-24T06:42:42.892090+0000","last_active":"2024-04-24T06:44:02.527938+0000","last_peered":"2024-04-24T06:44:02.527938+0000","last_clean":"2024-04-24T06:44:02.527938+0000","last_became_active":"2024-04-24T06:42:42.879904+0000","last_became_peered":"2024-04-24T06:42:42.879904+0000","last_unstale":"2024-04-24T06:44:02.527938+0000","last_undegraded":"2024-04-24T06:44:02.527938+0000","last_fullsized":"2024-04-24T06:44:02.527938+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:36:02.420438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.980663+0000","last_change":"2024-04-24T06:41:35.306175+0000","last_active":"2024-04-24T06:44:01.980663+0000","last_peered":"2024-04-24T06:44:01.980663+0000","last_clean":"2024-04-24T06:44:01.980663+0000","last_became_active":"2024-04-24T06:41:35.305840+0000","last_became_peered":"2024-04-24T06:41:35.305840+0000","last_unstale":"2024-04-24T06:44:01.980663+0000","last_undegraded":"2024-04-24T06:44:01.980663+0000","last_fullsized":"2024-04-24T06:44:01.980663+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:42:46.238490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.022530+0000","last_change":"2024-04-24T06:41:34.277212+0000","last_active":"2024-04-24T06:44:02.022530+0000","last_peered":"2024-04-24T06:44:02.022530+0000","last_clean":"2024-04-24T06:44:02.022530+0000","last_became_active":"2024-04-24T06:41:34.276695+0000","last_became_peered":"2024-04-24T06:41:34.276695+0000","last_unstale":"2024-04-24T06:44:02.022530+0000","last_undegraded":"2024-04-24T06:44:02.022530+0000","last_fullsized":"2024-04-24T06:44:02.022530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:00:47.169384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.983372+0000","last_change":"2024-04-24T06:41:35.305348+0000","last_active":"2024-04-24T06:44:01.983372+0000","last_peered":"2024-04-24T06:44:01.983372+0000","last_clean":"2024-04-24T06:44:01.983372+0000","last_became_active":"2024-04-24T06:41:35.304899+0000","last_became_peered":"2024-04-24T06:41:35.304899+0000","last_unstale":"2024-04-24T06:44:01.983372+0000","last_undegraded":"2024-04-24T06:44:01.983372+0000","last_fullsized":"2024-04-24T06:44:01.983372+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:49:35.737711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'4","reported_seq":172,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.603197+0000","last_change":"2024-04-24T06:41:34.272902+0000","last_active":"2024-04-24T06:44:02.603197+0000","last_peered":"2024-04-24T06:44:02.603197+0000","last_clean":"2024-04-24T06:44:02.603197+0000","last_became_active":"2024-04-24T06:41:34.271726+0000","last_became_peered":"2024-04-24T06:41:34.271726+0000","last_unstale":"2024-04-24T06:44:02.603197+0000","last_undegraded":"2024-04-24T06:44:02.603197+0000","last_fullsized":"2024-04-24T06:44:02.603197+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:07:38.318110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":147,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.982800+0000","last_change":"2024-04-24T06:41:35.283764+0000","last_active":"2024-04-24T06:44:01.982800+0000","last_peered":"2024-04-24T06:44:01.982800+0000","last_clean":"2024-04-24T06:44:01.982800+0000","last_became_active":"2024-04-24T06:41:35.283400+0000","last_became_peered":"2024-04-24T06:41:35.283400+0000","last_unstale":"2024-04-24T06:44:01.982800+0000","last_undegraded":"2024-04-24T06:44:01.982800+0000","last_fullsized":"2024-04-24T06:44:01.982800+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:36:47.034068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.127983+0000","last_change":"2024-04-24T06:41:34.280020+0000","last_active":"2024-04-24T06:44:02.127983+0000","last_peered":"2024-04-24T06:44:02.127983+0000","last_clean":"2024-04-24T06:44:02.127983+0000","last_became_active":"2024-04-24T06:41:34.279659+0000","last_became_peered":"2024-04-24T06:41:34.279659+0000","last_unstale":"2024-04-24T06:44:02.127983+0000","last_undegraded":"2024-04-24T06:44:02.127983+0000","last_fullsized":"2024-04-24T06:44:02.127983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:13:29.529123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.022523+0000","last_change":"2024-04-24T06:41:35.284500+0000","last_active":"2024-04-24T06:44:02.022523+0000","last_peered":"2024-04-24T06:44:02.022523+0000","last_clean":"2024-04-24T06:44:02.022523+0000","last_became_active":"2024-04-24T06:41:35.283171+0000","last_became_peered":"2024-04-24T06:41:35.283171+0000","last_unstale":"2024-04-24T06:44:02.022523+0000","last_undegraded":"2024-04-24T06:44:02.022523+0000","last_fullsized":"2024-04-24T06:44:02.022523+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:33:19.527226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.128449+0000","last_change":"2024-04-24T06:41:34.279306+0000","last_active":"2024-04-24T06:44:02.128449+0000","last_peered":"2024-04-24T06:44:02.128449+0000","last_clean":"2024-04-24T06:44:02.128449+0000","last_became_active":"2024-04-24T06:41:34.278946+0000","last_became_peered":"2024-04-24T06:41:34.278946+0000","last_unstale":"2024-04-24T06:44:02.128449+0000","last_undegraded":"2024-04-24T06:44:02.128449+0000","last_fullsized":"2024-04-24T06:44:02.128449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:13:40.845871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.528202+0000","last_change":"2024-04-24T06:41:35.283716+0000","last_active":"2024-04-24T06:44:02.528202+0000","last_peered":"2024-04-24T06:44:02.528202+0000","last_clean":"2024-04-24T06:44:02.528202+0000","last_became_active":"2024-04-24T06:41:35.283330+0000","last_became_peered":"2024-04-24T06:41:35.283330+0000","last_unstale":"2024-04-24T06:44:02.528202+0000","last_undegraded":"2024-04-24T06:44:02.528202+0000","last_fullsized":"2024-04-24T06:44:02.528202+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:18:13.403264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.125845+0000","last_change":"2024-04-24T06:42:42.887349+0000","last_active":"2024-04-24T06:44:02.125845+0000","last_peered":"2024-04-24T06:44:02.125845+0000","last_clean":"2024-04-24T06:44:02.125845+0000","last_became_active":"2024-04-24T06:42:42.886813+0000","last_became_peered":"2024-04-24T06:42:42.886813+0000","last_unstale":"2024-04-24T06:44:02.125845+0000","last_undegraded":"2024-04-24T06:44:02.125845+0000","last_fullsized":"2024-04-24T06:44:02.125845+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:05:40.577189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.139022+0000","last_change":"2024-04-24T06:41:35.306607+0000","last_active":"2024-04-24T06:44:02.139022+0000","last_peered":"2024-04-24T06:44:02.139022+0000","last_clean":"2024-04-24T06:44:02.139022+0000","last_became_active":"2024-04-24T06:41:35.306217+0000","last_became_peered":"2024-04-24T06:41:35.306217+0000","last_unstale":"2024-04-24T06:44:02.139022+0000","last_undegraded":"2024-04-24T06:44:02.139022+0000","last_fullsized":"2024-04-24T06:44:02.139022+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:07:46.068876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.983051+0000","last_change":"2024-04-24T06:41:34.285319+0000","last_active":"2024-04-24T06:44:01.983051+0000","last_peered":"2024-04-24T06:44:01.983051+0000","last_clean":"2024-04-24T06:44:01.983051+0000","last_became_active":"2024-04-24T06:41:34.284912+0000","last_became_peered":"2024-04-24T06:41:34.284912+0000","last_unstale":"2024-04-24T06:44:01.983051+0000","last_undegraded":"2024-04-24T06:44:01.983051+0000","last_fullsized":"2024-04-24T06:44:01.983051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:34:56.454587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.139027+0000","last_change":"2024-04-24T06:41:35.296294+0000","last_active":"2024-04-24T06:44:02.139027+0000","last_peered":"2024-04-24T06:44:02.139027+0000","last_clean":"2024-04-24T06:44:02.139027+0000","last_became_active":"2024-04-24T06:41:35.295792+0000","last_became_peered":"2024-04-24T06:41:35.295792+0000","last_unstale":"2024-04-24T06:44:02.139027+0000","last_undegraded":"2024-04-24T06:44:02.139027+0000","last_fullsized":"2024-04-24T06:44:02.139027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:31:35.573858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.022494+0000","last_change":"2024-04-24T06:41:35.282558+0000","last_active":"2024-04-24T06:44:02.022494+0000","last_peered":"2024-04-24T06:44:02.022494+0000","last_clean":"2024-04-24T06:44:02.022494+0000","last_became_active":"2024-04-24T06:41:35.282163+0000","last_became_peered":"2024-04-24T06:41:35.282163+0000","last_unstale":"2024-04-24T06:44:02.022494+0000","last_undegraded":"2024-04-24T06:44:02.022494+0000","last_fullsized":"2024-04-24T06:44:02.022494+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:12:23.569849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":170,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.326095+0000","last_change":"2024-04-24T06:41:34.284294+0000","last_active":"2024-04-24T06:44:02.326095+0000","last_peered":"2024-04-24T06:44:02.326095+0000","last_clean":"2024-04-24T06:44:02.326095+0000","last_became_active":"2024-04-24T06:41:34.283933+0000","last_became_peered":"2024-04-24T06:41:34.283933+0000","last_unstale":"2024-04-24T06:44:02.326095+0000","last_undegraded":"2024-04-24T06:44:02.326095+0000","last_fullsized":"2024-04-24T06:44:02.326095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:35:26.042614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":143,"num_read_kb":143,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.129137+0000","last_change":"2024-04-24T06:41:35.302361+0000","last_active":"2024-04-24T06:44:02.129137+0000","last_peered":"2024-04-24T06:44:02.129137+0000","last_clean":"2024-04-24T06:44:02.129137+0000","last_became_active":"2024-04-24T06:41:35.301652+0000","last_became_peered":"2024-04-24T06:41:35.301652+0000","last_unstale":"2024-04-24T06:44:02.129137+0000","last_undegraded":"2024-04-24T06:44:02.129137+0000","last_fullsized":"2024-04-24T06:44:02.129137+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:35:58.010853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.323724+0000","last_change":"2024-04-24T06:41:34.282156+0000","last_active":"2024-04-24T06:44:02.323724+0000","last_peered":"2024-04-24T06:44:02.323724+0000","last_clean":"2024-04-24T06:44:02.323724+0000","last_became_active":"2024-04-24T06:41:34.281771+0000","last_became_peered":"2024-04-24T06:41:34.281771+0000","last_unstale":"2024-04-24T06:44:02.323724+0000","last_undegraded":"2024-04-24T06:44:02.323724+0000","last_fullsized":"2024-04-24T06:44:02.323724+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:41:49.584620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":357,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.982840+0000","last_change":"2024-04-24T06:40:16.681695+0000","last_active":"2024-04-24T06:44:01.982840+0000","last_peered":"2024-04-24T06:44:01.982840+0000","last_clean":"2024-04-24T06:44:01.982840+0000","last_became_active":"2024-04-24T06:40:16.681208+0000","last_became_peered":"2024-04-24T06:40:16.681208+0000","last_unstale":"2024-04-24T06:44:01.982840+0000","last_undegraded":"2024-04-24T06:44:01.982840+0000","last_fullsized":"2024-04-24T06:44:01.982840+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:36:27.950574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:36:27.950574+0000","last_clean_scrub_stamp":"2024-04-24T06:36:27.950574+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:39:01.985893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.980224+0000","last_change":"2024-04-24T06:42:42.885312+0000","last_active":"2024-04-24T06:44:01.980224+0000","last_peered":"2024-04-24T06:44:01.980224+0000","last_clean":"2024-04-24T06:44:01.980224+0000","last_became_active":"2024-04-24T06:42:42.884702+0000","last_became_peered":"2024-04-24T06:42:42.884702+0000","last_unstale":"2024-04-24T06:44:01.980224+0000","last_undegraded":"2024-04-24T06:44:01.980224+0000","last_fullsized":"2024-04-24T06:44:01.980224+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:33:44.590555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.982969+0000","last_change":"2024-04-24T06:41:34.273872+0000","last_active":"2024-04-24T06:44:01.982969+0000","last_peered":"2024-04-24T06:44:01.982969+0000","last_clean":"2024-04-24T06:44:01.982969+0000","last_became_active":"2024-04-24T06:41:34.273473+0000","last_became_peered":"2024-04-24T06:41:34.273473+0000","last_unstale":"2024-04-24T06:44:01.982969+0000","last_undegraded":"2024-04-24T06:44:01.982969+0000","last_fullsized":"2024-04-24T06:44:01.982969+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:47:11.914985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.128221+0000","last_change":"2024-04-24T06:41:34.289293+0000","last_active":"2024-04-24T06:44:02.128221+0000","last_peered":"2024-04-24T06:44:02.128221+0000","last_clean":"2024-04-24T06:44:02.128221+0000","last_became_active":"2024-04-24T06:41:34.288939+0000","last_became_peered":"2024-04-24T06:41:34.288939+0000","last_unstale":"2024-04-24T06:44:02.128221+0000","last_undegraded":"2024-04-24T06:44:02.128221+0000","last_fullsized":"2024-04-24T06:44:02.128221+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:03:01.026792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.981615+0000","last_change":"2024-04-24T06:41:35.302153+0000","last_active":"2024-04-24T06:44:01.981615+0000","last_peered":"2024-04-24T06:44:01.981615+0000","last_clean":"2024-04-24T06:44:01.981615+0000","last_became_active":"2024-04-24T06:41:35.301666+0000","last_became_peered":"2024-04-24T06:41:35.301666+0000","last_unstale":"2024-04-24T06:44:01.981615+0000","last_undegraded":"2024-04-24T06:44:01.981615+0000","last_fullsized":"2024-04-24T06:44:01.981615+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:45:09.642482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'45","reported_seq":90,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.264503+0000","last_change":"2024-04-24T06:41:34.291570+0000","last_active":"2024-04-24T06:44:02.264503+0000","last_peered":"2024-04-24T06:44:02.264503+0000","last_clean":"2024-04-24T06:44:02.264503+0000","last_became_active":"2024-04-24T06:41:34.289700+0000","last_became_peered":"2024-04-24T06:41:34.289700+0000","last_unstale":"2024-04-24T06:44:02.264503+0000","last_undegraded":"2024-04-24T06:44:02.264503+0000","last_fullsized":"2024-04-24T06:44:02.264503+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:57:20.537299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":80,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.129846+0000","last_change":"2024-04-24T06:41:35.304703+0000","last_active":"2024-04-24T06:44:02.129846+0000","last_peered":"2024-04-24T06:44:02.129846+0000","last_clean":"2024-04-24T06:44:02.129846+0000","last_became_active":"2024-04-24T06:41:35.304319+0000","last_became_peered":"2024-04-24T06:41:35.304319+0000","last_unstale":"2024-04-24T06:44:02.129846+0000","last_undegraded":"2024-04-24T06:44:02.129846+0000","last_fullsized":"2024-04-24T06:44:02.129846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:00:56.069509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.324026+0000","last_change":"2024-04-24T06:41:34.276866+0000","last_active":"2024-04-24T06:44:02.324026+0000","last_peered":"2024-04-24T06:44:02.324026+0000","last_clean":"2024-04-24T06:44:02.324026+0000","last_became_active":"2024-04-24T06:41:34.276478+0000","last_became_peered":"2024-04-24T06:41:34.276478+0000","last_unstale":"2024-04-24T06:44:02.324026+0000","last_undegraded":"2024-04-24T06:44:02.324026+0000","last_fullsized":"2024-04-24T06:44:02.324026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:10:35.047960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.324893+0000","last_change":"2024-04-24T06:41:35.300951+0000","last_active":"2024-04-24T06:44:02.324893+0000","last_peered":"2024-04-24T06:44:02.324893+0000","last_clean":"2024-04-24T06:44:02.324893+0000","last_became_active":"2024-04-24T06:41:35.296176+0000","last_became_peered":"2024-04-24T06:41:35.296176+0000","last_unstale":"2024-04-24T06:44:02.324893+0000","last_undegraded":"2024-04-24T06:44:02.324893+0000","last_fullsized":"2024-04-24T06:44:02.324893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:53:58.497437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.319371+0000","last_change":"2024-04-24T06:41:34.276154+0000","last_active":"2024-04-24T06:44:02.319371+0000","last_peered":"2024-04-24T06:44:02.319371+0000","last_clean":"2024-04-24T06:44:02.319371+0000","last_became_active":"2024-04-24T06:41:34.275734+0000","last_became_peered":"2024-04-24T06:41:34.275734+0000","last_unstale":"2024-04-24T06:44:02.319371+0000","last_undegraded":"2024-04-24T06:44:02.319371+0000","last_fullsized":"2024-04-24T06:44:02.319371+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:28:54.618061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.127930+0000","last_change":"2024-04-24T06:41:35.285837+0000","last_active":"2024-04-24T06:44:02.127930+0000","last_peered":"2024-04-24T06:44:02.127930+0000","last_clean":"2024-04-24T06:44:02.127930+0000","last_became_active":"2024-04-24T06:41:35.285505+0000","last_became_peered":"2024-04-24T06:41:35.285505+0000","last_unstale":"2024-04-24T06:44:02.127930+0000","last_undegraded":"2024-04-24T06:44:02.127930+0000","last_fullsized":"2024-04-24T06:44:02.127930+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:36:53.460780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.129053+0000","last_change":"2024-04-24T06:41:34.290039+0000","last_active":"2024-04-24T06:44:02.129053+0000","last_peered":"2024-04-24T06:44:02.129053+0000","last_clean":"2024-04-24T06:44:02.129053+0000","last_became_active":"2024-04-24T06:41:34.289228+0000","last_became_peered":"2024-04-24T06:41:34.289228+0000","last_unstale":"2024-04-24T06:44:02.129053+0000","last_undegraded":"2024-04-24T06:44:02.129053+0000","last_fullsized":"2024-04-24T06:44:02.129053+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:07:20.342359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.527923+0000","last_change":"2024-04-24T06:41:35.296214+0000","last_active":"2024-04-24T06:44:02.527923+0000","last_peered":"2024-04-24T06:44:02.527923+0000","last_clean":"2024-04-24T06:44:02.527923+0000","last_became_active":"2024-04-24T06:41:35.295888+0000","last_became_peered":"2024-04-24T06:41:35.295888+0000","last_unstale":"2024-04-24T06:44:02.527923+0000","last_undegraded":"2024-04-24T06:44:02.527923+0000","last_fullsized":"2024-04-24T06:44:02.527923+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:55:19.520479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"89'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.980700+0000","last_change":"2024-04-24T06:41:34.275885+0000","last_active":"2024-04-24T06:44:01.980700+0000","last_peered":"2024-04-24T06:44:01.980700+0000","last_clean":"2024-04-24T06:44:01.980700+0000","last_became_active":"2024-04-24T06:41:34.275503+0000","last_became_peered":"2024-04-24T06:41:34.275503+0000","last_unstale":"2024-04-24T06:44:01.980700+0000","last_undegraded":"2024-04-24T06:44:01.980700+0000","last_fullsized":"2024-04-24T06:44:01.980700+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:50:58.216699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.129732+0000","last_change":"2024-04-24T06:41:35.305330+0000","last_active":"2024-04-24T06:44:02.129732+0000","last_peered":"2024-04-24T06:44:02.129732+0000","last_clean":"2024-04-24T06:44:02.129732+0000","last_became_active":"2024-04-24T06:41:35.301262+0000","last_became_peered":"2024-04-24T06:41:35.301262+0000","last_unstale":"2024-04-24T06:44:02.129732+0000","last_undegraded":"2024-04-24T06:44:02.129732+0000","last_fullsized":"2024-04-24T06:44:02.129732+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:58:41.029511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.128025+0000","last_change":"2024-04-24T06:41:34.276926+0000","last_active":"2024-04-24T06:44:02.128025+0000","last_peered":"2024-04-24T06:44:02.128025+0000","last_clean":"2024-04-24T06:44:02.128025+0000","last_became_active":"2024-04-24T06:41:34.276539+0000","last_became_peered":"2024-04-24T06:41:34.276539+0000","last_unstale":"2024-04-24T06:44:02.128025+0000","last_undegraded":"2024-04-24T06:44:02.128025+0000","last_fullsized":"2024-04-24T06:44:02.128025+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:28:56.652584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.983056+0000","last_change":"2024-04-24T06:41:35.301052+0000","last_active":"2024-04-24T06:44:01.983056+0000","last_peered":"2024-04-24T06:44:01.983056+0000","last_clean":"2024-04-24T06:44:01.983056+0000","last_became_active":"2024-04-24T06:41:35.300650+0000","last_became_peered":"2024-04-24T06:41:35.300650+0000","last_unstale":"2024-04-24T06:44:01.983056+0000","last_undegraded":"2024-04-24T06:44:01.983056+0000","last_fullsized":"2024-04-24T06:44:01.983056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:06:54.290353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.980975+0000","last_change":"2024-04-24T06:41:34.273773+0000","last_active":"2024-04-24T06:44:01.980975+0000","last_peered":"2024-04-24T06:44:01.980975+0000","last_clean":"2024-04-24T06:44:01.980975+0000","last_became_active":"2024-04-24T06:41:34.273324+0000","last_became_peered":"2024-04-24T06:41:34.273324+0000","last_unstale":"2024-04-24T06:44:01.980975+0000","last_undegraded":"2024-04-24T06:44:01.980975+0000","last_fullsized":"2024-04-24T06:44:01.980975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:30:30.760740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.983425+0000","last_change":"2024-04-24T06:41:35.289565+0000","last_active":"2024-04-24T06:44:01.983425+0000","last_peered":"2024-04-24T06:44:01.983425+0000","last_clean":"2024-04-24T06:44:01.983425+0000","last_became_active":"2024-04-24T06:41:35.289185+0000","last_became_peered":"2024-04-24T06:41:35.289185+0000","last_unstale":"2024-04-24T06:44:01.983425+0000","last_undegraded":"2024-04-24T06:44:01.983425+0000","last_fullsized":"2024-04-24T06:44:01.983425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:26:28.345162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'39","reported_seq":69,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.261132+0000","last_change":"2024-04-24T06:41:34.285804+0000","last_active":"2024-04-24T06:44:02.261132+0000","last_peered":"2024-04-24T06:44:02.261132+0000","last_clean":"2024-04-24T06:44:02.261132+0000","last_became_active":"2024-04-24T06:41:34.285501+0000","last_became_peered":"2024-04-24T06:41:34.285501+0000","last_unstale":"2024-04-24T06:44:02.261132+0000","last_undegraded":"2024-04-24T06:44:02.261132+0000","last_fullsized":"2024-04-24T06:44:02.261132+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:25:15.804221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.983619+0000","last_change":"2024-04-24T06:41:35.300856+0000","last_active":"2024-04-24T06:44:01.983619+0000","last_peered":"2024-04-24T06:44:01.983619+0000","last_clean":"2024-04-24T06:44:01.983619+0000","last_became_active":"2024-04-24T06:41:35.300344+0000","last_became_peered":"2024-04-24T06:41:35.300344+0000","last_unstale":"2024-04-24T06:44:01.983619+0000","last_undegraded":"2024-04-24T06:44:01.983619+0000","last_fullsized":"2024-04-24T06:44:01.983619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:10:01.866954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.981589+0000","last_change":"2024-04-24T06:41:34.284112+0000","last_active":"2024-04-24T06:44:01.981589+0000","last_peered":"2024-04-24T06:44:01.981589+0000","last_clean":"2024-04-24T06:44:01.981589+0000","last_became_active":"2024-04-24T06:41:34.279317+0000","last_became_peered":"2024-04-24T06:41:34.279317+0000","last_unstale":"2024-04-24T06:44:01.981589+0000","last_undegraded":"2024-04-24T06:44:01.981589+0000","last_fullsized":"2024-04-24T06:44:01.981589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:49:45.673353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.983686+0000","last_change":"2024-04-24T06:41:35.288360+0000","last_active":"2024-04-24T06:44:01.983686+0000","last_peered":"2024-04-24T06:44:01.983686+0000","last_clean":"2024-04-24T06:44:01.983686+0000","last_became_active":"2024-04-24T06:41:35.288019+0000","last_became_peered":"2024-04-24T06:41:35.288019+0000","last_unstale":"2024-04-24T06:44:01.983686+0000","last_undegraded":"2024-04-24T06:44:01.983686+0000","last_fullsized":"2024-04-24T06:44:01.983686+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:33:49.927591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.528008+0000","last_change":"2024-04-24T06:41:35.286137+0000","last_active":"2024-04-24T06:44:02.528008+0000","last_peered":"2024-04-24T06:44:02.528008+0000","last_clean":"2024-04-24T06:44:02.528008+0000","last_became_active":"2024-04-24T06:41:35.285773+0000","last_became_peered":"2024-04-24T06:41:35.285773+0000","last_unstale":"2024-04-24T06:44:02.528008+0000","last_undegraded":"2024-04-24T06:44:02.528008+0000","last_fullsized":"2024-04-24T06:44:02.528008+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:29:32.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.980786+0000","last_change":"2024-04-24T06:41:35.283977+0000","last_active":"2024-04-24T06:44:01.980786+0000","last_peered":"2024-04-24T06:44:01.980786+0000","last_clean":"2024-04-24T06:44:01.980786+0000","last_became_active":"2024-04-24T06:41:35.282312+0000","last_became_peered":"2024-04-24T06:41:35.282312+0000","last_unstale":"2024-04-24T06:44:01.980786+0000","last_undegraded":"2024-04-24T06:44:01.980786+0000","last_fullsized":"2024-04-24T06:44:01.980786+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:41:36.463803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.980619+0000","last_change":"2024-04-24T06:41:35.300372+0000","last_active":"2024-04-24T06:44:01.980619+0000","last_peered":"2024-04-24T06:44:01.980619+0000","last_clean":"2024-04-24T06:44:01.980619+0000","last_became_active":"2024-04-24T06:41:35.300034+0000","last_became_peered":"2024-04-24T06:41:35.300034+0000","last_unstale":"2024-04-24T06:44:01.980619+0000","last_undegraded":"2024-04-24T06:44:01.980619+0000","last_fullsized":"2024-04-24T06:44:01.980619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:56:23.732798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.324944+0000","last_change":"2024-04-24T06:41:35.301619+0000","last_active":"2024-04-24T06:44:02.324944+0000","last_peered":"2024-04-24T06:44:02.324944+0000","last_clean":"2024-04-24T06:44:02.324944+0000","last_became_active":"2024-04-24T06:41:35.298897+0000","last_became_peered":"2024-04-24T06:41:35.298897+0000","last_unstale":"2024-04-24T06:44:02.324944+0000","last_undegraded":"2024-04-24T06:44:02.324944+0000","last_fullsized":"2024-04-24T06:44:02.324944+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:26:11.583079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.981573+0000","last_change":"2024-04-24T06:41:35.304905+0000","last_active":"2024-04-24T06:44:01.981573+0000","last_peered":"2024-04-24T06:44:01.981573+0000","last_clean":"2024-04-24T06:44:01.981573+0000","last_became_active":"2024-04-24T06:41:35.304407+0000","last_became_peered":"2024-04-24T06:41:35.304407+0000","last_unstale":"2024-04-24T06:44:01.981573+0000","last_undegraded":"2024-04-24T06:44:01.981573+0000","last_fullsized":"2024-04-24T06:44:01.981573+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:38:07.417898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.528268+0000","last_change":"2024-04-24T06:41:35.288646+0000","last_active":"2024-04-24T06:44:02.528268+0000","last_peered":"2024-04-24T06:44:02.528268+0000","last_clean":"2024-04-24T06:44:02.528268+0000","last_became_active":"2024-04-24T06:41:35.288266+0000","last_became_peered":"2024-04-24T06:41:35.288266+0000","last_unstale":"2024-04-24T06:44:02.528268+0000","last_undegraded":"2024-04-24T06:44:02.528268+0000","last_fullsized":"2024-04-24T06:44:02.528268+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:59:16.526712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.981229+0000","last_change":"2024-04-24T06:41:35.285001+0000","last_active":"2024-04-24T06:44:01.981229+0000","last_peered":"2024-04-24T06:44:01.981229+0000","last_clean":"2024-04-24T06:44:01.981229+0000","last_became_active":"2024-04-24T06:41:35.284671+0000","last_became_peered":"2024-04-24T06:41:35.284671+0000","last_unstale":"2024-04-24T06:44:01.981229+0000","last_undegraded":"2024-04-24T06:44:01.981229+0000","last_fullsized":"2024-04-24T06:44:01.981229+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:37:43.534018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.129692+0000","last_change":"2024-04-24T06:41:35.299124+0000","last_active":"2024-04-24T06:44:02.129692+0000","last_peered":"2024-04-24T06:44:02.129692+0000","last_clean":"2024-04-24T06:44:02.129692+0000","last_became_active":"2024-04-24T06:41:35.298246+0000","last_became_peered":"2024-04-24T06:41:35.298246+0000","last_unstale":"2024-04-24T06:44:02.129692+0000","last_undegraded":"2024-04-24T06:44:02.129692+0000","last_fullsized":"2024-04-24T06:44:02.129692+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:07:22.649874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.980060+0000","last_change":"2024-04-24T06:42:42.888183+0000","last_active":"2024-04-24T06:44:01.980060+0000","last_peered":"2024-04-24T06:44:01.980060+0000","last_clean":"2024-04-24T06:44:01.980060+0000","last_became_active":"2024-04-24T06:42:42.887373+0000","last_became_peered":"2024-04-24T06:42:42.887373+0000","last_unstale":"2024-04-24T06:44:01.980060+0000","last_undegraded":"2024-04-24T06:44:01.980060+0000","last_fullsized":"2024-04-24T06:44:01.980060+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:44:51.456753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.128989+0000","last_change":"2024-04-24T06:41:35.301619+0000","last_active":"2024-04-24T06:44:02.128989+0000","last_peered":"2024-04-24T06:44:02.128989+0000","last_clean":"2024-04-24T06:44:02.128989+0000","last_became_active":"2024-04-24T06:41:35.297987+0000","last_became_peered":"2024-04-24T06:41:35.297987+0000","last_unstale":"2024-04-24T06:44:02.128989+0000","last_undegraded":"2024-04-24T06:44:02.128989+0000","last_fullsized":"2024-04-24T06:44:02.128989+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:06:01.235593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.319683+0000","last_change":"2024-04-24T06:41:35.304347+0000","last_active":"2024-04-24T06:44:02.319683+0000","last_peered":"2024-04-24T06:44:02.319683+0000","last_clean":"2024-04-24T06:44:02.319683+0000","last_became_active":"2024-04-24T06:41:35.303971+0000","last_became_peered":"2024-04-24T06:41:35.303971+0000","last_unstale":"2024-04-24T06:44:02.319683+0000","last_undegraded":"2024-04-24T06:44:02.319683+0000","last_fullsized":"2024-04-24T06:44:02.319683+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:31:54.813243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.981029+0000","last_change":"2024-04-24T06:41:35.308314+0000","last_active":"2024-04-24T06:44:01.981029+0000","last_peered":"2024-04-24T06:44:01.981029+0000","last_clean":"2024-04-24T06:44:01.981029+0000","last_became_active":"2024-04-24T06:41:35.306581+0000","last_became_peered":"2024-04-24T06:41:35.306581+0000","last_unstale":"2024-04-24T06:44:01.981029+0000","last_undegraded":"2024-04-24T06:44:01.981029+0000","last_fullsized":"2024-04-24T06:44:01.981029+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:34:43.095087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.528089+0000","last_change":"2024-04-24T06:41:35.289166+0000","last_active":"2024-04-24T06:44:02.528089+0000","last_peered":"2024-04-24T06:44:02.528089+0000","last_clean":"2024-04-24T06:44:02.528089+0000","last_became_active":"2024-04-24T06:41:35.287672+0000","last_became_peered":"2024-04-24T06:41:35.287672+0000","last_unstale":"2024-04-24T06:44:02.528089+0000","last_undegraded":"2024-04-24T06:44:02.528089+0000","last_fullsized":"2024-04-24T06:44:02.528089+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:31:45.403387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":595775,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":981,"num_read_kb":598,"num_write":282,"num_write_kb":1430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1787325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":220,"ondisk_log_size":220,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351017,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":460828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612654,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28792,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716360,"statfs":{"total":95995035648,"available":95965552640,"internally_reserved":0,"allocated":1818624,"data_stored":1259789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776180,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":978944,"data_stored":461030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037819,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28376,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716776,"statfs":{"total":95995035648,"available":95965978624,"internally_reserved":0,"allocated":1388544,"data_stored":863122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299455,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28572,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716580,"statfs":{"total":95995035648,"available":95965777920,"internally_reserved":0,"allocated":1589248,"data_stored":1062227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462982,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":469375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574945,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1052672,"data_stored":535735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738471,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":987136,"data_stored":461690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411423,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1073152,"data_stored":537442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182485,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28384,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716768,"statfs":{"total":95995035648,"available":95965970432,"internally_reserved":0,"allocated":1396736,"data_stored":863422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920849,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":470203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724620,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32872,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712280,"statfs":{"total":95995035648,"available":95961374720,"internally_reserved":0,"allocated":1736704,"data_stored":1193120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":68274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":133467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":393954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":200136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T06:44:08.272 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-04-24T06:44:08.272 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T06:44:08.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.494+0000 7f230c400640 1 -- 172.21.15.5:0/3585700098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2304105e70 msgr2=0x7f230410c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:08.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.494+0000 7f230c400640 1 --2- 172.21.15.5:0/3585700098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2304105e70 0x7f230410c700 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f22fc009ca0 tx=0x7f22fc02a8b0 comp rx=0 tx=0).stop 2024-04-24T06:44:08.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.494+0000 7f230c400640 1 -- 172.21.15.5:0/3585700098 shutdown_connections 2024-04-24T06:44:08.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.494+0000 7f230c400640 1 --2- 172.21.15.5:0/3585700098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2304105e70 0x7f230410c700 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:08.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.494+0000 7f230c400640 1 --2- 172.21.15.5:0/3585700098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23041054b0 0x7f2304105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:08.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.494+0000 7f230c400640 1 --2- 172.21.15.5:0/3585700098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23041042c0 0x7f23041046e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:08.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.494+0000 7f230c400640 1 -- 172.21.15.5:0/3585700098 >> 172.21.15.5:0/3585700098 conn(0x7f23040ff850 msgr2=0x7f2304101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:08.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.494+0000 7f230c400640 1 -- 172.21.15.5:0/3585700098 shutdown_connections 2024-04-24T06:44:08.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.494+0000 7f230c400640 1 -- 172.21.15.5:0/3585700098 wait complete. 2024-04-24T06:44:08.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.495+0000 7f230c400640 1 Processor -- start 2024-04-24T06:44:08.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.495+0000 7f230c400640 1 -- start start 2024-04-24T06:44:08.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.495+0000 7f230c400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23041042c0 0x7f230419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:08.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.496+0000 7f230c400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23041054b0 0x7f230419aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:08.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.496+0000 7f230c400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2304105e70 0x7f23041a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:08.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.496+0000 7f230c400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f230410ff30 con 0x7f2304105e70 2024-04-24T06:44:08.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.496+0000 7f230c400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f230410fdb0 con 0x7f23041042c0 2024-04-24T06:44:08.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.496+0000 7f230c400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f23041100b0 con 0x7f23041054b0 2024-04-24T06:44:08.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.496+0000 7f2309a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23041042c0 0x7f230419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.496+0000 7f2309a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23041042c0 0x7f230419a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:46130/0 (socket says 172.21.15.5:46130) 2024-04-24T06:44:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.496+0000 7f2309a00640 1 -- 172.21.15.5:0/3290790371 learned_addr learned my addr 172.21.15.5:0/3290790371 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.496+0000 7f230a400640 1 --2- 172.21.15.5:0/3290790371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2304105e70 0x7f23041a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.496+0000 7f2309000640 1 --2- 172.21.15.5:0/3290790371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23041054b0 0x7f230419aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.496+0000 7f2309a00640 1 -- 172.21.15.5:0/3290790371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23041054b0 msgr2=0x7f230419aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.496+0000 7f2309a00640 1 --2- 172.21.15.5:0/3290790371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23041054b0 0x7f230419aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.496+0000 7f2309a00640 1 -- 172.21.15.5:0/3290790371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2304105e70 msgr2=0x7f23041a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.496+0000 7f2309a00640 1 --2- 172.21.15.5:0/3290790371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2304105e70 0x7f23041a1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.496+0000 7f2309a00640 1 -- 172.21.15.5:0/3290790371 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23041a2130 con 0x7f23041042c0 2024-04-24T06:44:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.497+0000 7f2309a00640 1 --2- 172.21.15.5:0/3290790371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23041042c0 0x7f230419a510 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f22f800e9c0 tx=0x7f22f800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.497+0000 7f2302000640 1 -- 172.21.15.5:0/3290790371 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f800cde0 con 0x7f23041042c0 2024-04-24T06:44:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.497+0000 7f2302000640 1 -- 172.21.15.5:0/3290790371 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22f8010910 con 0x7f23041042c0 2024-04-24T06:44:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.497+0000 7f2302000640 1 -- 172.21.15.5:0/3290790371 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f8018a00 con 0x7f23041042c0 2024-04-24T06:44:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.497+0000 7f230c400640 1 -- 172.21.15.5:0/3290790371 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23041a2420 con 0x7f23041042c0 2024-04-24T06:44:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.497+0000 7f230c400640 1 -- 172.21.15.5:0/3290790371 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23041a29c0 con 0x7f23041042c0 2024-04-24T06:44:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.499+0000 7f230c400640 1 -- 172.21.15.5:0/3290790371 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f230410b580 con 0x7f23041042c0 2024-04-24T06:44:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.500+0000 7f2302000640 1 -- 172.21.15.5:0/3290790371 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138872+0+0 (secure 0 0 0) 0x7f22f801f020 con 0x7f23041042c0 2024-04-24T06:44:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.502+0000 7f2302000640 1 --2- 172.21.15.5:0/3290790371 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f22ec0a3ec0 0x7f22ec0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.502+0000 7f2302000640 1 -- 172.21.15.5:0/3290790371 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f22f80d0c60 con 0x7f23041042c0 2024-04-24T06:44:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.502+0000 7f2309000640 1 --2- 172.21.15.5:0/3290790371 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f22ec0a3ec0 0x7f22ec0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.503+0000 7f2309000640 1 --2- 172.21.15.5:0/3290790371 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f22ec0a3ec0 0x7f22ec0a6380 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f230419ba20 tx=0x7f22f0005f90 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.506+0000 7f2302000640 1 -- 172.21.15.5:0/3290790371 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f22f809d060 con 0x7f23041042c0 2024-04-24T06:44:08.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.727+0000 7f230c400640 1 -- 172.21.15.5:0/3290790371 --> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2304062440 con 0x7f22ec0a3ec0 2024-04-24T06:44:08.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.737+0000 7f2302000640 1 -- 172.21.15.5:0/3290790371 <== mgr.25202 v2:172.21.15.5:6800/3154624651 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257767 (secure 0 0 0) 0x7f2304062440 con 0x7f22ec0a3ec0 2024-04-24T06:44:08.739 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:44:08.745 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-24T06:44:08.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.741+0000 7f230c400640 1 -- 172.21.15.5:0/3290790371 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f22ec0a3ec0 msgr2=0x7f22ec0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:08.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.741+0000 7f230c400640 1 --2- 172.21.15.5:0/3290790371 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f22ec0a3ec0 0x7f22ec0a6380 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f230419ba20 tx=0x7f22f0005f90 comp rx=0 tx=0).stop 2024-04-24T06:44:08.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.741+0000 7f230c400640 1 -- 172.21.15.5:0/3290790371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23041042c0 msgr2=0x7f230419a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:08.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.741+0000 7f230c400640 1 --2- 172.21.15.5:0/3290790371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23041042c0 0x7f230419a510 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f22f800e9c0 tx=0x7f22f800ee80 comp rx=0 tx=0).stop 2024-04-24T06:44:08.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.741+0000 7f230c400640 1 -- 172.21.15.5:0/3290790371 shutdown_connections 2024-04-24T06:44:08.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.741+0000 7f230c400640 1 --2- 172.21.15.5:0/3290790371 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f22ec0a3ec0 0x7f22ec0a6380 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:08.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.741+0000 7f230c400640 1 --2- 172.21.15.5:0/3290790371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2304105e70 0x7f23041a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:08.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.741+0000 7f230c400640 1 --2- 172.21.15.5:0/3290790371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23041054b0 0x7f230419aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:08.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.741+0000 7f230c400640 1 --2- 172.21.15.5:0/3290790371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23041042c0 0x7f230419a510 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:08.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.741+0000 7f230c400640 1 -- 172.21.15.5:0/3290790371 >> 172.21.15.5:0/3290790371 conn(0x7f23040ff850 msgr2=0x7f2304108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:08.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.741+0000 7f230c400640 1 -- 172.21.15.5:0/3290790371 shutdown_connections 2024-04-24T06:44:08.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.741+0000 7f230c400640 1 -- 172.21.15.5:0/3290790371 wait complete. 2024-04-24T06:44:08.761 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-04-24T06:44:08.005449+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1186377,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1117,"num_read_kb":717,"num_write":465,"num_write_kb":4370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":859,"ondisk_log_size":859,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376268,"kb_used_data":14608,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565556,"statfs":{"total":1151940427776,"available":1151555129344,"internally_reserved":0,"allocated":14958592,"data_stored":8637983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.971872"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.528183+0000","last_change":"2024-04-24T06:41:35.296451+0000","last_active":"2024-04-24T06:44:02.528183+0000","last_peered":"2024-04-24T06:44:02.528183+0000","last_clean":"2024-04-24T06:44:02.528183+0000","last_became_active":"2024-04-24T06:41:35.295733+0000","last_became_peered":"2024-04-24T06:41:35.295733+0000","last_unstale":"2024-04-24T06:44:02.528183+0000","last_undegraded":"2024-04-24T06:44:02.528183+0000","last_fullsized":"2024-04-24T06:44:02.528183+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:04:15.072644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.325058+0000","last_change":"2024-04-24T06:41:35.293359+0000","last_active":"2024-04-24T06:44:02.325058+0000","last_peered":"2024-04-24T06:44:02.325058+0000","last_clean":"2024-04-24T06:44:02.325058+0000","last_became_active":"2024-04-24T06:41:35.287371+0000","last_became_peered":"2024-04-24T06:41:35.287371+0000","last_unstale":"2024-04-24T06:44:02.325058+0000","last_undegraded":"2024-04-24T06:44:02.325058+0000","last_fullsized":"2024-04-24T06:44:02.325058+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:03:39.376495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.325258+0000","last_change":"2024-04-24T06:41:35.272734+0000","last_active":"2024-04-24T06:44:02.325258+0000","last_peered":"2024-04-24T06:44:02.325258+0000","last_clean":"2024-04-24T06:44:02.325258+0000","last_became_active":"2024-04-24T06:41:35.271685+0000","last_became_peered":"2024-04-24T06:41:35.271685+0000","last_unstale":"2024-04-24T06:44:02.325258+0000","last_undegraded":"2024-04-24T06:44:02.325258+0000","last_fullsized":"2024-04-24T06:44:02.325258+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:08:15.057918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.323790+0000","last_change":"2024-04-24T06:41:35.288574+0000","last_active":"2024-04-24T06:44:02.323790+0000","last_peered":"2024-04-24T06:44:02.323790+0000","last_clean":"2024-04-24T06:44:02.323790+0000","last_became_active":"2024-04-24T06:41:35.288229+0000","last_became_peered":"2024-04-24T06:41:35.288229+0000","last_unstale":"2024-04-24T06:44:02.323790+0000","last_undegraded":"2024-04-24T06:44:02.323790+0000","last_fullsized":"2024-04-24T06:44:02.323790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:58:18.028067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.128022+0000","last_change":"2024-04-24T06:41:35.310226+0000","last_active":"2024-04-24T06:44:02.128022+0000","last_peered":"2024-04-24T06:44:02.128022+0000","last_clean":"2024-04-24T06:44:02.128022+0000","last_became_active":"2024-04-24T06:41:35.309853+0000","last_became_peered":"2024-04-24T06:41:35.309853+0000","last_unstale":"2024-04-24T06:44:02.128022+0000","last_undegraded":"2024-04-24T06:44:02.128022+0000","last_fullsized":"2024-04-24T06:44:02.128022+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:34:57.767158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.138791+0000","last_change":"2024-04-24T06:41:35.294259+0000","last_active":"2024-04-24T06:44:02.138791+0000","last_peered":"2024-04-24T06:44:02.138791+0000","last_clean":"2024-04-24T06:44:02.138791+0000","last_became_active":"2024-04-24T06:41:35.293827+0000","last_became_peered":"2024-04-24T06:41:35.293827+0000","last_unstale":"2024-04-24T06:44:02.138791+0000","last_undegraded":"2024-04-24T06:44:02.138791+0000","last_fullsized":"2024-04-24T06:44:02.138791+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:09:23.302827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.982776+0000","last_change":"2024-04-24T06:41:35.295095+0000","last_active":"2024-04-24T06:44:01.982776+0000","last_peered":"2024-04-24T06:44:01.982776+0000","last_clean":"2024-04-24T06:44:01.982776+0000","last_became_active":"2024-04-24T06:41:35.294743+0000","last_became_peered":"2024-04-24T06:41:35.294743+0000","last_unstale":"2024-04-24T06:44:01.982776+0000","last_undegraded":"2024-04-24T06:44:01.982776+0000","last_fullsized":"2024-04-24T06:44:01.982776+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:06:09.473123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.125405+0000","last_change":"2024-04-24T06:42:42.886072+0000","last_active":"2024-04-24T06:44:02.125405+0000","last_peered":"2024-04-24T06:44:02.125405+0000","last_clean":"2024-04-24T06:44:02.125405+0000","last_became_active":"2024-04-24T06:42:42.882624+0000","last_became_peered":"2024-04-24T06:42:42.882624+0000","last_unstale":"2024-04-24T06:44:02.125405+0000","last_undegraded":"2024-04-24T06:44:02.125405+0000","last_fullsized":"2024-04-24T06:44:02.125405+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:53:02.523012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.320256+0000","last_change":"2024-04-24T06:41:35.283275+0000","last_active":"2024-04-24T06:44:02.320256+0000","last_peered":"2024-04-24T06:44:02.320256+0000","last_clean":"2024-04-24T06:44:02.320256+0000","last_became_active":"2024-04-24T06:41:35.282586+0000","last_became_peered":"2024-04-24T06:41:35.282586+0000","last_unstale":"2024-04-24T06:44:02.320256+0000","last_undegraded":"2024-04-24T06:44:02.320256+0000","last_fullsized":"2024-04-24T06:44:02.320256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:42:30.716149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.139163+0000","last_change":"2024-04-24T06:41:35.284986+0000","last_active":"2024-04-24T06:44:02.139163+0000","last_peered":"2024-04-24T06:44:02.139163+0000","last_clean":"2024-04-24T06:44:02.139163+0000","last_became_active":"2024-04-24T06:41:35.284628+0000","last_became_peered":"2024-04-24T06:41:35.284628+0000","last_unstale":"2024-04-24T06:44:02.139163+0000","last_undegraded":"2024-04-24T06:44:02.139163+0000","last_fullsized":"2024-04-24T06:44:02.139163+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:29:38.479593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.983393+0000","last_change":"2024-04-24T06:41:35.290353+0000","last_active":"2024-04-24T06:44:01.983393+0000","last_peered":"2024-04-24T06:44:01.983393+0000","last_clean":"2024-04-24T06:44:01.983393+0000","last_became_active":"2024-04-24T06:41:35.289987+0000","last_became_peered":"2024-04-24T06:41:35.289987+0000","last_unstale":"2024-04-24T06:44:01.983393+0000","last_undegraded":"2024-04-24T06:44:01.983393+0000","last_fullsized":"2024-04-24T06:44:01.983393+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:56:54.914426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.323831+0000","last_change":"2024-04-24T06:41:35.289462+0000","last_active":"2024-04-24T06:44:02.323831+0000","last_peered":"2024-04-24T06:44:02.323831+0000","last_clean":"2024-04-24T06:44:02.323831+0000","last_became_active":"2024-04-24T06:41:35.289123+0000","last_became_peered":"2024-04-24T06:41:35.289123+0000","last_unstale":"2024-04-24T06:44:02.323831+0000","last_undegraded":"2024-04-24T06:44:02.323831+0000","last_fullsized":"2024-04-24T06:44:02.323831+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:51:06.792508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.981156+0000","last_change":"2024-04-24T06:41:35.298320+0000","last_active":"2024-04-24T06:44:01.981156+0000","last_peered":"2024-04-24T06:44:01.981156+0000","last_clean":"2024-04-24T06:44:01.981156+0000","last_became_active":"2024-04-24T06:41:35.297839+0000","last_became_peered":"2024-04-24T06:41:35.297839+0000","last_unstale":"2024-04-24T06:44:01.981156+0000","last_undegraded":"2024-04-24T06:44:01.981156+0000","last_fullsized":"2024-04-24T06:44:01.981156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:10:36.423514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.324512+0000","last_change":"2024-04-24T06:41:35.301924+0000","last_active":"2024-04-24T06:44:02.324512+0000","last_peered":"2024-04-24T06:44:02.324512+0000","last_clean":"2024-04-24T06:44:02.324512+0000","last_became_active":"2024-04-24T06:41:35.299114+0000","last_became_peered":"2024-04-24T06:41:35.299114+0000","last_unstale":"2024-04-24T06:44:02.324512+0000","last_undegraded":"2024-04-24T06:44:02.324512+0000","last_fullsized":"2024-04-24T06:44:02.324512+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:34:46.194266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.323875+0000","last_change":"2024-04-24T06:41:35.284885+0000","last_active":"2024-04-24T06:44:02.323875+0000","last_peered":"2024-04-24T06:44:02.323875+0000","last_clean":"2024-04-24T06:44:02.323875+0000","last_became_active":"2024-04-24T06:41:35.284244+0000","last_became_peered":"2024-04-24T06:41:35.284244+0000","last_unstale":"2024-04-24T06:44:02.323875+0000","last_undegraded":"2024-04-24T06:44:02.323875+0000","last_fullsized":"2024-04-24T06:44:02.323875+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:30:40.279905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.319433+0000","last_change":"2024-04-24T06:41:35.286363+0000","last_active":"2024-04-24T06:44:02.319433+0000","last_peered":"2024-04-24T06:44:02.319433+0000","last_clean":"2024-04-24T06:44:02.319433+0000","last_became_active":"2024-04-24T06:41:35.286024+0000","last_became_peered":"2024-04-24T06:41:35.286024+0000","last_unstale":"2024-04-24T06:44:02.319433+0000","last_undegraded":"2024-04-24T06:44:02.319433+0000","last_fullsized":"2024-04-24T06:44:02.319433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:43:35.728990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.528311+0000","last_change":"2024-04-24T06:41:35.304005+0000","last_active":"2024-04-24T06:44:02.528311+0000","last_peered":"2024-04-24T06:44:02.528311+0000","last_clean":"2024-04-24T06:44:02.528311+0000","last_became_active":"2024-04-24T06:41:35.303659+0000","last_became_peered":"2024-04-24T06:41:35.303659+0000","last_unstale":"2024-04-24T06:44:02.528311+0000","last_undegraded":"2024-04-24T06:44:02.528311+0000","last_fullsized":"2024-04-24T06:44:02.528311+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:05:07.215927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.126161+0000","last_change":"2024-04-24T06:41:35.296218+0000","last_active":"2024-04-24T06:44:02.126161+0000","last_peered":"2024-04-24T06:44:02.126161+0000","last_clean":"2024-04-24T06:44:02.126161+0000","last_became_active":"2024-04-24T06:41:35.295486+0000","last_became_peered":"2024-04-24T06:41:35.295486+0000","last_unstale":"2024-04-24T06:44:02.126161+0000","last_undegraded":"2024-04-24T06:44:02.126161+0000","last_fullsized":"2024-04-24T06:44:02.126161+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:17:25.844899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.980909+0000","last_change":"2024-04-24T06:42:42.886291+0000","last_active":"2024-04-24T06:44:01.980909+0000","last_peered":"2024-04-24T06:44:01.980909+0000","last_clean":"2024-04-24T06:44:01.980909+0000","last_became_active":"2024-04-24T06:42:42.885781+0000","last_became_peered":"2024-04-24T06:42:42.885781+0000","last_unstale":"2024-04-24T06:44:01.980909+0000","last_undegraded":"2024-04-24T06:44:01.980909+0000","last_fullsized":"2024-04-24T06:44:01.980909+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:01:44.377335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.125607+0000","last_change":"2024-04-24T06:41:34.282140+0000","last_active":"2024-04-24T06:44:02.125607+0000","last_peered":"2024-04-24T06:44:02.125607+0000","last_clean":"2024-04-24T06:44:02.125607+0000","last_became_active":"2024-04-24T06:41:34.281608+0000","last_became_peered":"2024-04-24T06:41:34.281608+0000","last_unstale":"2024-04-24T06:44:02.125607+0000","last_undegraded":"2024-04-24T06:44:02.125607+0000","last_fullsized":"2024-04-24T06:44:02.125607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:32:06.727639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.129957+0000","last_change":"2024-04-24T06:41:35.308847+0000","last_active":"2024-04-24T06:44:02.129957+0000","last_peered":"2024-04-24T06:44:02.129957+0000","last_clean":"2024-04-24T06:44:02.129957+0000","last_became_active":"2024-04-24T06:41:35.308256+0000","last_became_peered":"2024-04-24T06:41:35.308256+0000","last_unstale":"2024-04-24T06:44:02.129957+0000","last_undegraded":"2024-04-24T06:44:02.129957+0000","last_fullsized":"2024-04-24T06:44:02.129957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:21:49.338737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.130066+0000","last_change":"2024-04-24T06:41:34.285819+0000","last_active":"2024-04-24T06:44:02.130066+0000","last_peered":"2024-04-24T06:44:02.130066+0000","last_clean":"2024-04-24T06:44:02.130066+0000","last_became_active":"2024-04-24T06:41:34.282393+0000","last_became_peered":"2024-04-24T06:41:34.282393+0000","last_unstale":"2024-04-24T06:44:02.130066+0000","last_undegraded":"2024-04-24T06:44:02.130066+0000","last_fullsized":"2024-04-24T06:44:02.130066+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:11:18.969178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.325587+0000","last_change":"2024-04-24T06:41:35.297437+0000","last_active":"2024-04-24T06:44:02.325587+0000","last_peered":"2024-04-24T06:44:02.325587+0000","last_clean":"2024-04-24T06:44:02.325587+0000","last_became_active":"2024-04-24T06:41:35.295195+0000","last_became_peered":"2024-04-24T06:41:35.295195+0000","last_unstale":"2024-04-24T06:44:02.325587+0000","last_undegraded":"2024-04-24T06:44:02.325587+0000","last_fullsized":"2024-04-24T06:44:02.325587+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:21:51.219219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.128352+0000","last_change":"2024-04-24T06:41:34.275768+0000","last_active":"2024-04-24T06:44:02.128352+0000","last_peered":"2024-04-24T06:44:02.128352+0000","last_clean":"2024-04-24T06:44:02.128352+0000","last_became_active":"2024-04-24T06:41:34.275426+0000","last_became_peered":"2024-04-24T06:41:34.275426+0000","last_unstale":"2024-04-24T06:44:02.128352+0000","last_undegraded":"2024-04-24T06:44:02.128352+0000","last_fullsized":"2024-04-24T06:44:02.128352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:01:08.735480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.319850+0000","last_change":"2024-04-24T06:41:35.305734+0000","last_active":"2024-04-24T06:44:02.319850+0000","last_peered":"2024-04-24T06:44:02.319850+0000","last_clean":"2024-04-24T06:44:02.319850+0000","last_became_active":"2024-04-24T06:41:35.305365+0000","last_became_peered":"2024-04-24T06:41:35.305365+0000","last_unstale":"2024-04-24T06:44:02.319850+0000","last_undegraded":"2024-04-24T06:44:02.319850+0000","last_fullsized":"2024-04-24T06:44:02.319850+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:38:52.740107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.128296+0000","last_change":"2024-04-24T06:41:34.278317+0000","last_active":"2024-04-24T06:44:02.128296+0000","last_peered":"2024-04-24T06:44:02.128296+0000","last_clean":"2024-04-24T06:44:02.128296+0000","last_became_active":"2024-04-24T06:41:34.277862+0000","last_became_peered":"2024-04-24T06:41:34.277862+0000","last_unstale":"2024-04-24T06:44:02.128296+0000","last_undegraded":"2024-04-24T06:44:02.128296+0000","last_fullsized":"2024-04-24T06:44:02.128296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:19:50.444136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.022666+0000","last_change":"2024-04-24T06:41:35.282648+0000","last_active":"2024-04-24T06:44:02.022666+0000","last_peered":"2024-04-24T06:44:02.022666+0000","last_clean":"2024-04-24T06:44:02.022666+0000","last_became_active":"2024-04-24T06:41:35.282335+0000","last_became_peered":"2024-04-24T06:41:35.282335+0000","last_unstale":"2024-04-24T06:44:02.022666+0000","last_undegraded":"2024-04-24T06:44:02.022666+0000","last_fullsized":"2024-04-24T06:44:02.022666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:30:11.909067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":174,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.568985+0000","last_change":"2024-04-24T06:41:34.274405+0000","last_active":"2024-04-24T06:44:02.568985+0000","last_peered":"2024-04-24T06:44:02.568985+0000","last_clean":"2024-04-24T06:44:02.568985+0000","last_became_active":"2024-04-24T06:41:34.273820+0000","last_became_peered":"2024-04-24T06:41:34.273820+0000","last_unstale":"2024-04-24T06:44:02.568985+0000","last_undegraded":"2024-04-24T06:44:02.568985+0000","last_fullsized":"2024-04-24T06:44:02.568985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:11:26.119236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.128021+0000","last_change":"2024-04-24T06:41:35.294133+0000","last_active":"2024-04-24T06:44:02.128021+0000","last_peered":"2024-04-24T06:44:02.128021+0000","last_clean":"2024-04-24T06:44:02.128021+0000","last_became_active":"2024-04-24T06:41:35.293741+0000","last_became_peered":"2024-04-24T06:41:35.293741+0000","last_unstale":"2024-04-24T06:44:02.128021+0000","last_undegraded":"2024-04-24T06:44:02.128021+0000","last_fullsized":"2024-04-24T06:44:02.128021+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:09:12.495187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.138617+0000","last_change":"2024-04-24T06:42:42.927397+0000","last_active":"2024-04-24T06:44:02.138617+0000","last_peered":"2024-04-24T06:44:02.138617+0000","last_clean":"2024-04-24T06:44:02.138617+0000","last_became_active":"2024-04-24T06:42:42.924070+0000","last_became_peered":"2024-04-24T06:42:42.924070+0000","last_unstale":"2024-04-24T06:44:02.138617+0000","last_undegraded":"2024-04-24T06:44:02.138617+0000","last_fullsized":"2024-04-24T06:44:02.138617+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:19:47.319762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.022818+0000","last_change":"2024-04-24T06:42:43.166909+0000","last_active":"2024-04-24T06:44:02.022818+0000","last_peered":"2024-04-24T06:44:02.022818+0000","last_clean":"2024-04-24T06:44:02.022818+0000","last_became_active":"2024-04-24T06:42:43.166559+0000","last_became_peered":"2024-04-24T06:42:43.166559+0000","last_unstale":"2024-04-24T06:44:02.022818+0000","last_undegraded":"2024-04-24T06:44:02.022818+0000","last_fullsized":"2024-04-24T06:44:02.022818+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:26:42.537506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.980468+0000","last_change":"2024-04-24T06:41:34.284958+0000","last_active":"2024-04-24T06:44:01.980468+0000","last_peered":"2024-04-24T06:44:01.980468+0000","last_clean":"2024-04-24T06:44:01.980468+0000","last_became_active":"2024-04-24T06:41:34.284279+0000","last_became_peered":"2024-04-24T06:41:34.284279+0000","last_unstale":"2024-04-24T06:44:01.980468+0000","last_undegraded":"2024-04-24T06:44:01.980468+0000","last_fullsized":"2024-04-24T06:44:01.980468+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:58:19.316091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.128861+0000","last_change":"2024-04-24T06:41:35.307280+0000","last_active":"2024-04-24T06:44:02.128861+0000","last_peered":"2024-04-24T06:44:02.128861+0000","last_clean":"2024-04-24T06:44:02.128861+0000","last_became_active":"2024-04-24T06:41:35.306908+0000","last_became_peered":"2024-04-24T06:41:35.306908+0000","last_unstale":"2024-04-24T06:44:02.128861+0000","last_undegraded":"2024-04-24T06:44:02.128861+0000","last_fullsized":"2024-04-24T06:44:02.128861+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:54:31.306302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.022540+0000","last_change":"2024-04-24T06:41:34.272221+0000","last_active":"2024-04-24T06:44:02.022540+0000","last_peered":"2024-04-24T06:44:02.022540+0000","last_clean":"2024-04-24T06:44:02.022540+0000","last_became_active":"2024-04-24T06:41:34.271748+0000","last_became_peered":"2024-04-24T06:41:34.271748+0000","last_unstale":"2024-04-24T06:44:02.022540+0000","last_undegraded":"2024-04-24T06:44:02.022540+0000","last_fullsized":"2024-04-24T06:44:02.022540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:29:02.667285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.125592+0000","last_change":"2024-04-24T06:41:35.309608+0000","last_active":"2024-04-24T06:44:02.125592+0000","last_peered":"2024-04-24T06:44:02.125592+0000","last_clean":"2024-04-24T06:44:02.125592+0000","last_became_active":"2024-04-24T06:41:35.309227+0000","last_became_peered":"2024-04-24T06:41:35.309227+0000","last_unstale":"2024-04-24T06:44:02.125592+0000","last_undegraded":"2024-04-24T06:44:02.125592+0000","last_fullsized":"2024-04-24T06:44:02.125592+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:51:23.151426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":169,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.567739+0000","last_change":"2024-04-24T06:41:34.281885+0000","last_active":"2024-04-24T06:44:02.567739+0000","last_peered":"2024-04-24T06:44:02.567739+0000","last_clean":"2024-04-24T06:44:02.567739+0000","last_became_active":"2024-04-24T06:41:34.281466+0000","last_became_peered":"2024-04-24T06:41:34.281466+0000","last_unstale":"2024-04-24T06:44:02.567739+0000","last_undegraded":"2024-04-24T06:44:02.567739+0000","last_fullsized":"2024-04-24T06:44:02.567739+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:20:08.500343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":143,"num_read_kb":143,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.320075+0000","last_change":"2024-04-24T06:41:35.292714+0000","last_active":"2024-04-24T06:44:02.320075+0000","last_peered":"2024-04-24T06:44:02.320075+0000","last_clean":"2024-04-24T06:44:02.320075+0000","last_became_active":"2024-04-24T06:41:35.291950+0000","last_became_peered":"2024-04-24T06:41:35.291950+0000","last_unstale":"2024-04-24T06:44:02.320075+0000","last_undegraded":"2024-04-24T06:44:02.320075+0000","last_fullsized":"2024-04-24T06:44:02.320075+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:42:18.049746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.983538+0000","last_change":"2024-04-24T06:41:34.275218+0000","last_active":"2024-04-24T06:44:01.983538+0000","last_peered":"2024-04-24T06:44:01.983538+0000","last_clean":"2024-04-24T06:44:01.983538+0000","last_became_active":"2024-04-24T06:41:34.274824+0000","last_became_peered":"2024-04-24T06:41:34.274824+0000","last_unstale":"2024-04-24T06:44:01.983538+0000","last_undegraded":"2024-04-24T06:44:01.983538+0000","last_fullsized":"2024-04-24T06:44:01.983538+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:09:57.271304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.325532+0000","last_change":"2024-04-24T06:42:42.883478+0000","last_active":"2024-04-24T06:44:02.325532+0000","last_peered":"2024-04-24T06:44:02.325532+0000","last_clean":"2024-04-24T06:44:02.325532+0000","last_became_active":"2024-04-24T06:42:42.882808+0000","last_became_peered":"2024-04-24T06:42:42.882808+0000","last_unstale":"2024-04-24T06:44:02.325532+0000","last_undegraded":"2024-04-24T06:44:02.325532+0000","last_fullsized":"2024-04-24T06:44:02.325532+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:17:11.514041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.022807+0000","last_change":"2024-04-24T06:41:34.277372+0000","last_active":"2024-04-24T06:44:02.022807+0000","last_peered":"2024-04-24T06:44:02.022807+0000","last_clean":"2024-04-24T06:44:02.022807+0000","last_became_active":"2024-04-24T06:41:34.276997+0000","last_became_peered":"2024-04-24T06:41:34.276997+0000","last_unstale":"2024-04-24T06:44:02.022807+0000","last_undegraded":"2024-04-24T06:44:02.022807+0000","last_fullsized":"2024-04-24T06:44:02.022807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:52:35.695109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.320091+0000","last_change":"2024-04-24T06:41:35.290233+0000","last_active":"2024-04-24T06:44:02.320091+0000","last_peered":"2024-04-24T06:44:02.320091+0000","last_clean":"2024-04-24T06:44:02.320091+0000","last_became_active":"2024-04-24T06:41:35.289885+0000","last_became_peered":"2024-04-24T06:41:35.289885+0000","last_unstale":"2024-04-24T06:44:02.320091+0000","last_undegraded":"2024-04-24T06:44:02.320091+0000","last_fullsized":"2024-04-24T06:44:02.320091+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:35:54.293308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'33","reported_seq":352,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.569436+0000","last_change":"2024-04-24T06:41:34.267693+0000","last_active":"2024-04-24T06:44:02.569436+0000","last_peered":"2024-04-24T06:44:02.569436+0000","last_clean":"2024-04-24T06:44:02.569436+0000","last_became_active":"2024-04-24T06:41:34.267298+0000","last_became_peered":"2024-04-24T06:41:34.267298+0000","last_unstale":"2024-04-24T06:44:02.569436+0000","last_undegraded":"2024-04-24T06:44:02.569436+0000","last_fullsized":"2024-04-24T06:44:02.569436+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:21:16.015695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66567,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":297,"num_read_kb":213,"num_write":34,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.128194+0000","last_change":"2024-04-24T06:41:35.286587+0000","last_active":"2024-04-24T06:44:02.128194+0000","last_peered":"2024-04-24T06:44:02.128194+0000","last_clean":"2024-04-24T06:44:02.128194+0000","last_became_active":"2024-04-24T06:41:35.286239+0000","last_became_peered":"2024-04-24T06:41:35.286239+0000","last_unstale":"2024-04-24T06:44:02.128194+0000","last_undegraded":"2024-04-24T06:44:02.128194+0000","last_fullsized":"2024-04-24T06:44:02.128194+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:18:56.013425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.527938+0000","last_change":"2024-04-24T06:42:42.892090+0000","last_active":"2024-04-24T06:44:02.527938+0000","last_peered":"2024-04-24T06:44:02.527938+0000","last_clean":"2024-04-24T06:44:02.527938+0000","last_became_active":"2024-04-24T06:42:42.879904+0000","last_became_peered":"2024-04-24T06:42:42.879904+0000","last_unstale":"2024-04-24T06:44:02.527938+0000","last_undegraded":"2024-04-24T06:44:02.527938+0000","last_fullsized":"2024-04-24T06:44:02.527938+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:36:02.420438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.980663+0000","last_change":"2024-04-24T06:41:35.306175+0000","last_active":"2024-04-24T06:44:01.980663+0000","last_peered":"2024-04-24T06:44:01.980663+0000","last_clean":"2024-04-24T06:44:01.980663+0000","last_became_active":"2024-04-24T06:41:35.305840+0000","last_became_peered":"2024-04-24T06:41:35.305840+0000","last_unstale":"2024-04-24T06:44:01.980663+0000","last_undegraded":"2024-04-24T06:44:01.980663+0000","last_fullsized":"2024-04-24T06:44:01.980663+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:42:46.238490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.022530+0000","last_change":"2024-04-24T06:41:34.277212+0000","last_active":"2024-04-24T06:44:02.022530+0000","last_peered":"2024-04-24T06:44:02.022530+0000","last_clean":"2024-04-24T06:44:02.022530+0000","last_became_active":"2024-04-24T06:41:34.276695+0000","last_became_peered":"2024-04-24T06:41:34.276695+0000","last_unstale":"2024-04-24T06:44:02.022530+0000","last_undegraded":"2024-04-24T06:44:02.022530+0000","last_fullsized":"2024-04-24T06:44:02.022530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:00:47.169384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.983372+0000","last_change":"2024-04-24T06:41:35.305348+0000","last_active":"2024-04-24T06:44:01.983372+0000","last_peered":"2024-04-24T06:44:01.983372+0000","last_clean":"2024-04-24T06:44:01.983372+0000","last_became_active":"2024-04-24T06:41:35.304899+0000","last_became_peered":"2024-04-24T06:41:35.304899+0000","last_unstale":"2024-04-24T06:44:01.983372+0000","last_undegraded":"2024-04-24T06:44:01.983372+0000","last_fullsized":"2024-04-24T06:44:01.983372+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:49:35.737711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'4","reported_seq":172,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.603197+0000","last_change":"2024-04-24T06:41:34.272902+0000","last_active":"2024-04-24T06:44:02.603197+0000","last_peered":"2024-04-24T06:44:02.603197+0000","last_clean":"2024-04-24T06:44:02.603197+0000","last_became_active":"2024-04-24T06:41:34.271726+0000","last_became_peered":"2024-04-24T06:41:34.271726+0000","last_unstale":"2024-04-24T06:44:02.603197+0000","last_undegraded":"2024-04-24T06:44:02.603197+0000","last_fullsized":"2024-04-24T06:44:02.603197+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:07:38.318110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":147,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.982800+0000","last_change":"2024-04-24T06:41:35.283764+0000","last_active":"2024-04-24T06:44:01.982800+0000","last_peered":"2024-04-24T06:44:01.982800+0000","last_clean":"2024-04-24T06:44:01.982800+0000","last_became_active":"2024-04-24T06:41:35.283400+0000","last_became_peered":"2024-04-24T06:41:35.283400+0000","last_unstale":"2024-04-24T06:44:01.982800+0000","last_undegraded":"2024-04-24T06:44:01.982800+0000","last_fullsized":"2024-04-24T06:44:01.982800+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:36:47.034068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.127983+0000","last_change":"2024-04-24T06:41:34.280020+0000","last_active":"2024-04-24T06:44:02.127983+0000","last_peered":"2024-04-24T06:44:02.127983+0000","last_clean":"2024-04-24T06:44:02.127983+0000","last_became_active":"2024-04-24T06:41:34.279659+0000","last_became_peered":"2024-04-24T06:41:34.279659+0000","last_unstale":"2024-04-24T06:44:02.127983+0000","last_undegraded":"2024-04-24T06:44:02.127983+0000","last_fullsized":"2024-04-24T06:44:02.127983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:13:29.529123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.022523+0000","last_change":"2024-04-24T06:41:35.284500+0000","last_active":"2024-04-24T06:44:02.022523+0000","last_peered":"2024-04-24T06:44:02.022523+0000","last_clean":"2024-04-24T06:44:02.022523+0000","last_became_active":"2024-04-24T06:41:35.283171+0000","last_became_peered":"2024-04-24T06:41:35.283171+0000","last_unstale":"2024-04-24T06:44:02.022523+0000","last_undegraded":"2024-04-24T06:44:02.022523+0000","last_fullsized":"2024-04-24T06:44:02.022523+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:33:19.527226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.128449+0000","last_change":"2024-04-24T06:41:34.279306+0000","last_active":"2024-04-24T06:44:02.128449+0000","last_peered":"2024-04-24T06:44:02.128449+0000","last_clean":"2024-04-24T06:44:02.128449+0000","last_became_active":"2024-04-24T06:41:34.278946+0000","last_became_peered":"2024-04-24T06:41:34.278946+0000","last_unstale":"2024-04-24T06:44:02.128449+0000","last_undegraded":"2024-04-24T06:44:02.128449+0000","last_fullsized":"2024-04-24T06:44:02.128449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:13:40.845871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.528202+0000","last_change":"2024-04-24T06:41:35.283716+0000","last_active":"2024-04-24T06:44:02.528202+0000","last_peered":"2024-04-24T06:44:02.528202+0000","last_clean":"2024-04-24T06:44:02.528202+0000","last_became_active":"2024-04-24T06:41:35.283330+0000","last_became_peered":"2024-04-24T06:41:35.283330+0000","last_unstale":"2024-04-24T06:44:02.528202+0000","last_undegraded":"2024-04-24T06:44:02.528202+0000","last_fullsized":"2024-04-24T06:44:02.528202+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:18:13.403264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.125845+0000","last_change":"2024-04-24T06:42:42.887349+0000","last_active":"2024-04-24T06:44:02.125845+0000","last_peered":"2024-04-24T06:44:02.125845+0000","last_clean":"2024-04-24T06:44:02.125845+0000","last_became_active":"2024-04-24T06:42:42.886813+0000","last_became_peered":"2024-04-24T06:42:42.886813+0000","last_unstale":"2024-04-24T06:44:02.125845+0000","last_undegraded":"2024-04-24T06:44:02.125845+0000","last_fullsized":"2024-04-24T06:44:02.125845+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:05:40.577189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.139022+0000","last_change":"2024-04-24T06:41:35.306607+0000","last_active":"2024-04-24T06:44:02.139022+0000","last_peered":"2024-04-24T06:44:02.139022+0000","last_clean":"2024-04-24T06:44:02.139022+0000","last_became_active":"2024-04-24T06:41:35.306217+0000","last_became_peered":"2024-04-24T06:41:35.306217+0000","last_unstale":"2024-04-24T06:44:02.139022+0000","last_undegraded":"2024-04-24T06:44:02.139022+0000","last_fullsized":"2024-04-24T06:44:02.139022+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:07:46.068876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.983051+0000","last_change":"2024-04-24T06:41:34.285319+0000","last_active":"2024-04-24T06:44:01.983051+0000","last_peered":"2024-04-24T06:44:01.983051+0000","last_clean":"2024-04-24T06:44:01.983051+0000","last_became_active":"2024-04-24T06:41:34.284912+0000","last_became_peered":"2024-04-24T06:41:34.284912+0000","last_unstale":"2024-04-24T06:44:01.983051+0000","last_undegraded":"2024-04-24T06:44:01.983051+0000","last_fullsized":"2024-04-24T06:44:01.983051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:34:56.454587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.139027+0000","last_change":"2024-04-24T06:41:35.296294+0000","last_active":"2024-04-24T06:44:02.139027+0000","last_peered":"2024-04-24T06:44:02.139027+0000","last_clean":"2024-04-24T06:44:02.139027+0000","last_became_active":"2024-04-24T06:41:35.295792+0000","last_became_peered":"2024-04-24T06:41:35.295792+0000","last_unstale":"2024-04-24T06:44:02.139027+0000","last_undegraded":"2024-04-24T06:44:02.139027+0000","last_fullsized":"2024-04-24T06:44:02.139027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:31:35.573858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.022494+0000","last_change":"2024-04-24T06:41:35.282558+0000","last_active":"2024-04-24T06:44:02.022494+0000","last_peered":"2024-04-24T06:44:02.022494+0000","last_clean":"2024-04-24T06:44:02.022494+0000","last_became_active":"2024-04-24T06:41:35.282163+0000","last_became_peered":"2024-04-24T06:41:35.282163+0000","last_unstale":"2024-04-24T06:44:02.022494+0000","last_undegraded":"2024-04-24T06:44:02.022494+0000","last_fullsized":"2024-04-24T06:44:02.022494+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:12:23.569849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":170,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.326095+0000","last_change":"2024-04-24T06:41:34.284294+0000","last_active":"2024-04-24T06:44:02.326095+0000","last_peered":"2024-04-24T06:44:02.326095+0000","last_clean":"2024-04-24T06:44:02.326095+0000","last_became_active":"2024-04-24T06:41:34.283933+0000","last_became_peered":"2024-04-24T06:41:34.283933+0000","last_unstale":"2024-04-24T06:44:02.326095+0000","last_undegraded":"2024-04-24T06:44:02.326095+0000","last_fullsized":"2024-04-24T06:44:02.326095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:35:26.042614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":143,"num_read_kb":143,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.129137+0000","last_change":"2024-04-24T06:41:35.302361+0000","last_active":"2024-04-24T06:44:02.129137+0000","last_peered":"2024-04-24T06:44:02.129137+0000","last_clean":"2024-04-24T06:44:02.129137+0000","last_became_active":"2024-04-24T06:41:35.301652+0000","last_became_peered":"2024-04-24T06:41:35.301652+0000","last_unstale":"2024-04-24T06:44:02.129137+0000","last_undegraded":"2024-04-24T06:44:02.129137+0000","last_fullsized":"2024-04-24T06:44:02.129137+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:35:58.010853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.323724+0000","last_change":"2024-04-24T06:41:34.282156+0000","last_active":"2024-04-24T06:44:02.323724+0000","last_peered":"2024-04-24T06:44:02.323724+0000","last_clean":"2024-04-24T06:44:02.323724+0000","last_became_active":"2024-04-24T06:41:34.281771+0000","last_became_peered":"2024-04-24T06:41:34.281771+0000","last_unstale":"2024-04-24T06:44:02.323724+0000","last_undegraded":"2024-04-24T06:44:02.323724+0000","last_fullsized":"2024-04-24T06:44:02.323724+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:41:49.584620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":357,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.982840+0000","last_change":"2024-04-24T06:40:16.681695+0000","last_active":"2024-04-24T06:44:01.982840+0000","last_peered":"2024-04-24T06:44:01.982840+0000","last_clean":"2024-04-24T06:44:01.982840+0000","last_became_active":"2024-04-24T06:40:16.681208+0000","last_became_peered":"2024-04-24T06:40:16.681208+0000","last_unstale":"2024-04-24T06:44:01.982840+0000","last_undegraded":"2024-04-24T06:44:01.982840+0000","last_fullsized":"2024-04-24T06:44:01.982840+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:36:27.950574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:36:27.950574+0000","last_clean_scrub_stamp":"2024-04-24T06:36:27.950574+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:39:01.985893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.980224+0000","last_change":"2024-04-24T06:42:42.885312+0000","last_active":"2024-04-24T06:44:01.980224+0000","last_peered":"2024-04-24T06:44:01.980224+0000","last_clean":"2024-04-24T06:44:01.980224+0000","last_became_active":"2024-04-24T06:42:42.884702+0000","last_became_peered":"2024-04-24T06:42:42.884702+0000","last_unstale":"2024-04-24T06:44:01.980224+0000","last_undegraded":"2024-04-24T06:44:01.980224+0000","last_fullsized":"2024-04-24T06:44:01.980224+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:33:44.590555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.982969+0000","last_change":"2024-04-24T06:41:34.273872+0000","last_active":"2024-04-24T06:44:01.982969+0000","last_peered":"2024-04-24T06:44:01.982969+0000","last_clean":"2024-04-24T06:44:01.982969+0000","last_became_active":"2024-04-24T06:41:34.273473+0000","last_became_peered":"2024-04-24T06:41:34.273473+0000","last_unstale":"2024-04-24T06:44:01.982969+0000","last_undegraded":"2024-04-24T06:44:01.982969+0000","last_fullsized":"2024-04-24T06:44:01.982969+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:47:11.914985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.128221+0000","last_change":"2024-04-24T06:41:34.289293+0000","last_active":"2024-04-24T06:44:02.128221+0000","last_peered":"2024-04-24T06:44:02.128221+0000","last_clean":"2024-04-24T06:44:02.128221+0000","last_became_active":"2024-04-24T06:41:34.288939+0000","last_became_peered":"2024-04-24T06:41:34.288939+0000","last_unstale":"2024-04-24T06:44:02.128221+0000","last_undegraded":"2024-04-24T06:44:02.128221+0000","last_fullsized":"2024-04-24T06:44:02.128221+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:03:01.026792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.981615+0000","last_change":"2024-04-24T06:41:35.302153+0000","last_active":"2024-04-24T06:44:01.981615+0000","last_peered":"2024-04-24T06:44:01.981615+0000","last_clean":"2024-04-24T06:44:01.981615+0000","last_became_active":"2024-04-24T06:41:35.301666+0000","last_became_peered":"2024-04-24T06:41:35.301666+0000","last_unstale":"2024-04-24T06:44:01.981615+0000","last_undegraded":"2024-04-24T06:44:01.981615+0000","last_fullsized":"2024-04-24T06:44:01.981615+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:45:09.642482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'45","reported_seq":90,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.264503+0000","last_change":"2024-04-24T06:41:34.291570+0000","last_active":"2024-04-24T06:44:02.264503+0000","last_peered":"2024-04-24T06:44:02.264503+0000","last_clean":"2024-04-24T06:44:02.264503+0000","last_became_active":"2024-04-24T06:41:34.289700+0000","last_became_peered":"2024-04-24T06:41:34.289700+0000","last_unstale":"2024-04-24T06:44:02.264503+0000","last_undegraded":"2024-04-24T06:44:02.264503+0000","last_fullsized":"2024-04-24T06:44:02.264503+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:57:20.537299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":80,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.129846+0000","last_change":"2024-04-24T06:41:35.304703+0000","last_active":"2024-04-24T06:44:02.129846+0000","last_peered":"2024-04-24T06:44:02.129846+0000","last_clean":"2024-04-24T06:44:02.129846+0000","last_became_active":"2024-04-24T06:41:35.304319+0000","last_became_peered":"2024-04-24T06:41:35.304319+0000","last_unstale":"2024-04-24T06:44:02.129846+0000","last_undegraded":"2024-04-24T06:44:02.129846+0000","last_fullsized":"2024-04-24T06:44:02.129846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:00:56.069509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.324026+0000","last_change":"2024-04-24T06:41:34.276866+0000","last_active":"2024-04-24T06:44:02.324026+0000","last_peered":"2024-04-24T06:44:02.324026+0000","last_clean":"2024-04-24T06:44:02.324026+0000","last_became_active":"2024-04-24T06:41:34.276478+0000","last_became_peered":"2024-04-24T06:41:34.276478+0000","last_unstale":"2024-04-24T06:44:02.324026+0000","last_undegraded":"2024-04-24T06:44:02.324026+0000","last_fullsized":"2024-04-24T06:44:02.324026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:10:35.047960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.324893+0000","last_change":"2024-04-24T06:41:35.300951+0000","last_active":"2024-04-24T06:44:02.324893+0000","last_peered":"2024-04-24T06:44:02.324893+0000","last_clean":"2024-04-24T06:44:02.324893+0000","last_became_active":"2024-04-24T06:41:35.296176+0000","last_became_peered":"2024-04-24T06:41:35.296176+0000","last_unstale":"2024-04-24T06:44:02.324893+0000","last_undegraded":"2024-04-24T06:44:02.324893+0000","last_fullsized":"2024-04-24T06:44:02.324893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:53:58.497437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.319371+0000","last_change":"2024-04-24T06:41:34.276154+0000","last_active":"2024-04-24T06:44:02.319371+0000","last_peered":"2024-04-24T06:44:02.319371+0000","last_clean":"2024-04-24T06:44:02.319371+0000","last_became_active":"2024-04-24T06:41:34.275734+0000","last_became_peered":"2024-04-24T06:41:34.275734+0000","last_unstale":"2024-04-24T06:44:02.319371+0000","last_undegraded":"2024-04-24T06:44:02.319371+0000","last_fullsized":"2024-04-24T06:44:02.319371+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:28:54.618061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.127930+0000","last_change":"2024-04-24T06:41:35.285837+0000","last_active":"2024-04-24T06:44:02.127930+0000","last_peered":"2024-04-24T06:44:02.127930+0000","last_clean":"2024-04-24T06:44:02.127930+0000","last_became_active":"2024-04-24T06:41:35.285505+0000","last_became_peered":"2024-04-24T06:41:35.285505+0000","last_unstale":"2024-04-24T06:44:02.127930+0000","last_undegraded":"2024-04-24T06:44:02.127930+0000","last_fullsized":"2024-04-24T06:44:02.127930+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:36:53.460780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.129053+0000","last_change":"2024-04-24T06:41:34.290039+0000","last_active":"2024-04-24T06:44:02.129053+0000","last_peered":"2024-04-24T06:44:02.129053+0000","last_clean":"2024-04-24T06:44:02.129053+0000","last_became_active":"2024-04-24T06:41:34.289228+0000","last_became_peered":"2024-04-24T06:41:34.289228+0000","last_unstale":"2024-04-24T06:44:02.129053+0000","last_undegraded":"2024-04-24T06:44:02.129053+0000","last_fullsized":"2024-04-24T06:44:02.129053+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:07:20.342359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.527923+0000","last_change":"2024-04-24T06:41:35.296214+0000","last_active":"2024-04-24T06:44:02.527923+0000","last_peered":"2024-04-24T06:44:02.527923+0000","last_clean":"2024-04-24T06:44:02.527923+0000","last_became_active":"2024-04-24T06:41:35.295888+0000","last_became_peered":"2024-04-24T06:41:35.295888+0000","last_unstale":"2024-04-24T06:44:02.527923+0000","last_undegraded":"2024-04-24T06:44:02.527923+0000","last_fullsized":"2024-04-24T06:44:02.527923+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:55:19.520479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"89'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.980700+0000","last_change":"2024-04-24T06:41:34.275885+0000","last_active":"2024-04-24T06:44:01.980700+0000","last_peered":"2024-04-24T06:44:01.980700+0000","last_clean":"2024-04-24T06:44:01.980700+0000","last_became_active":"2024-04-24T06:41:34.275503+0000","last_became_peered":"2024-04-24T06:41:34.275503+0000","last_unstale":"2024-04-24T06:44:01.980700+0000","last_undegraded":"2024-04-24T06:44:01.980700+0000","last_fullsized":"2024-04-24T06:44:01.980700+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:50:58.216699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.129732+0000","last_change":"2024-04-24T06:41:35.305330+0000","last_active":"2024-04-24T06:44:02.129732+0000","last_peered":"2024-04-24T06:44:02.129732+0000","last_clean":"2024-04-24T06:44:02.129732+0000","last_became_active":"2024-04-24T06:41:35.301262+0000","last_became_peered":"2024-04-24T06:41:35.301262+0000","last_unstale":"2024-04-24T06:44:02.129732+0000","last_undegraded":"2024-04-24T06:44:02.129732+0000","last_fullsized":"2024-04-24T06:44:02.129732+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:58:41.029511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.128025+0000","last_change":"2024-04-24T06:41:34.276926+0000","last_active":"2024-04-24T06:44:02.128025+0000","last_peered":"2024-04-24T06:44:02.128025+0000","last_clean":"2024-04-24T06:44:02.128025+0000","last_became_active":"2024-04-24T06:41:34.276539+0000","last_became_peered":"2024-04-24T06:41:34.276539+0000","last_unstale":"2024-04-24T06:44:02.128025+0000","last_undegraded":"2024-04-24T06:44:02.128025+0000","last_fullsized":"2024-04-24T06:44:02.128025+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:28:56.652584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.983056+0000","last_change":"2024-04-24T06:41:35.301052+0000","last_active":"2024-04-24T06:44:01.983056+0000","last_peered":"2024-04-24T06:44:01.983056+0000","last_clean":"2024-04-24T06:44:01.983056+0000","last_became_active":"2024-04-24T06:41:35.300650+0000","last_became_peered":"2024-04-24T06:41:35.300650+0000","last_unstale":"2024-04-24T06:44:01.983056+0000","last_undegraded":"2024-04-24T06:44:01.983056+0000","last_fullsized":"2024-04-24T06:44:01.983056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:06:54.290353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.980975+0000","last_change":"2024-04-24T06:41:34.273773+0000","last_active":"2024-04-24T06:44:01.980975+0000","last_peered":"2024-04-24T06:44:01.980975+0000","last_clean":"2024-04-24T06:44:01.980975+0000","last_became_active":"2024-04-24T06:41:34.273324+0000","last_became_peered":"2024-04-24T06:41:34.273324+0000","last_unstale":"2024-04-24T06:44:01.980975+0000","last_undegraded":"2024-04-24T06:44:01.980975+0000","last_fullsized":"2024-04-24T06:44:01.980975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:30:30.760740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.983425+0000","last_change":"2024-04-24T06:41:35.289565+0000","last_active":"2024-04-24T06:44:01.983425+0000","last_peered":"2024-04-24T06:44:01.983425+0000","last_clean":"2024-04-24T06:44:01.983425+0000","last_became_active":"2024-04-24T06:41:35.289185+0000","last_became_peered":"2024-04-24T06:41:35.289185+0000","last_unstale":"2024-04-24T06:44:01.983425+0000","last_undegraded":"2024-04-24T06:44:01.983425+0000","last_fullsized":"2024-04-24T06:44:01.983425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:26:28.345162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'39","reported_seq":69,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.261132+0000","last_change":"2024-04-24T06:41:34.285804+0000","last_active":"2024-04-24T06:44:02.261132+0000","last_peered":"2024-04-24T06:44:02.261132+0000","last_clean":"2024-04-24T06:44:02.261132+0000","last_became_active":"2024-04-24T06:41:34.285501+0000","last_became_peered":"2024-04-24T06:41:34.285501+0000","last_unstale":"2024-04-24T06:44:02.261132+0000","last_undegraded":"2024-04-24T06:44:02.261132+0000","last_fullsized":"2024-04-24T06:44:02.261132+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:25:15.804221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.983619+0000","last_change":"2024-04-24T06:41:35.300856+0000","last_active":"2024-04-24T06:44:01.983619+0000","last_peered":"2024-04-24T06:44:01.983619+0000","last_clean":"2024-04-24T06:44:01.983619+0000","last_became_active":"2024-04-24T06:41:35.300344+0000","last_became_peered":"2024-04-24T06:41:35.300344+0000","last_unstale":"2024-04-24T06:44:01.983619+0000","last_undegraded":"2024-04-24T06:44:01.983619+0000","last_fullsized":"2024-04-24T06:44:01.983619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:10:01.866954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.981589+0000","last_change":"2024-04-24T06:41:34.284112+0000","last_active":"2024-04-24T06:44:01.981589+0000","last_peered":"2024-04-24T06:44:01.981589+0000","last_clean":"2024-04-24T06:44:01.981589+0000","last_became_active":"2024-04-24T06:41:34.279317+0000","last_became_peered":"2024-04-24T06:41:34.279317+0000","last_unstale":"2024-04-24T06:44:01.981589+0000","last_undegraded":"2024-04-24T06:44:01.981589+0000","last_fullsized":"2024-04-24T06:44:01.981589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:33.230781+0000","last_clean_scrub_stamp":"2024-04-24T06:41:33.230781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:49:45.673353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.983686+0000","last_change":"2024-04-24T06:41:35.288360+0000","last_active":"2024-04-24T06:44:01.983686+0000","last_peered":"2024-04-24T06:44:01.983686+0000","last_clean":"2024-04-24T06:44:01.983686+0000","last_became_active":"2024-04-24T06:41:35.288019+0000","last_became_peered":"2024-04-24T06:41:35.288019+0000","last_unstale":"2024-04-24T06:44:01.983686+0000","last_undegraded":"2024-04-24T06:44:01.983686+0000","last_fullsized":"2024-04-24T06:44:01.983686+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:33:49.927591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.528008+0000","last_change":"2024-04-24T06:41:35.286137+0000","last_active":"2024-04-24T06:44:02.528008+0000","last_peered":"2024-04-24T06:44:02.528008+0000","last_clean":"2024-04-24T06:44:02.528008+0000","last_became_active":"2024-04-24T06:41:35.285773+0000","last_became_peered":"2024-04-24T06:41:35.285773+0000","last_unstale":"2024-04-24T06:44:02.528008+0000","last_undegraded":"2024-04-24T06:44:02.528008+0000","last_fullsized":"2024-04-24T06:44:02.528008+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:29:32.556342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.980786+0000","last_change":"2024-04-24T06:41:35.283977+0000","last_active":"2024-04-24T06:44:01.980786+0000","last_peered":"2024-04-24T06:44:01.980786+0000","last_clean":"2024-04-24T06:44:01.980786+0000","last_became_active":"2024-04-24T06:41:35.282312+0000","last_became_peered":"2024-04-24T06:41:35.282312+0000","last_unstale":"2024-04-24T06:44:01.980786+0000","last_undegraded":"2024-04-24T06:44:01.980786+0000","last_fullsized":"2024-04-24T06:44:01.980786+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:41:36.463803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.980619+0000","last_change":"2024-04-24T06:41:35.300372+0000","last_active":"2024-04-24T06:44:01.980619+0000","last_peered":"2024-04-24T06:44:01.980619+0000","last_clean":"2024-04-24T06:44:01.980619+0000","last_became_active":"2024-04-24T06:41:35.300034+0000","last_became_peered":"2024-04-24T06:41:35.300034+0000","last_unstale":"2024-04-24T06:44:01.980619+0000","last_undegraded":"2024-04-24T06:44:01.980619+0000","last_fullsized":"2024-04-24T06:44:01.980619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:56:23.732798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.324944+0000","last_change":"2024-04-24T06:41:35.301619+0000","last_active":"2024-04-24T06:44:02.324944+0000","last_peered":"2024-04-24T06:44:02.324944+0000","last_clean":"2024-04-24T06:44:02.324944+0000","last_became_active":"2024-04-24T06:41:35.298897+0000","last_became_peered":"2024-04-24T06:41:35.298897+0000","last_unstale":"2024-04-24T06:44:02.324944+0000","last_undegraded":"2024-04-24T06:44:02.324944+0000","last_fullsized":"2024-04-24T06:44:02.324944+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:26:11.583079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.981573+0000","last_change":"2024-04-24T06:41:35.304905+0000","last_active":"2024-04-24T06:44:01.981573+0000","last_peered":"2024-04-24T06:44:01.981573+0000","last_clean":"2024-04-24T06:44:01.981573+0000","last_became_active":"2024-04-24T06:41:35.304407+0000","last_became_peered":"2024-04-24T06:41:35.304407+0000","last_unstale":"2024-04-24T06:44:01.981573+0000","last_undegraded":"2024-04-24T06:44:01.981573+0000","last_fullsized":"2024-04-24T06:44:01.981573+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:38:07.417898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.528268+0000","last_change":"2024-04-24T06:41:35.288646+0000","last_active":"2024-04-24T06:44:02.528268+0000","last_peered":"2024-04-24T06:44:02.528268+0000","last_clean":"2024-04-24T06:44:02.528268+0000","last_became_active":"2024-04-24T06:41:35.288266+0000","last_became_peered":"2024-04-24T06:41:35.288266+0000","last_unstale":"2024-04-24T06:44:02.528268+0000","last_undegraded":"2024-04-24T06:44:02.528268+0000","last_fullsized":"2024-04-24T06:44:02.528268+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:59:16.526712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.981229+0000","last_change":"2024-04-24T06:41:35.285001+0000","last_active":"2024-04-24T06:44:01.981229+0000","last_peered":"2024-04-24T06:44:01.981229+0000","last_clean":"2024-04-24T06:44:01.981229+0000","last_became_active":"2024-04-24T06:41:35.284671+0000","last_became_peered":"2024-04-24T06:41:35.284671+0000","last_unstale":"2024-04-24T06:44:01.981229+0000","last_undegraded":"2024-04-24T06:44:01.981229+0000","last_fullsized":"2024-04-24T06:44:01.981229+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:37:43.534018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.129692+0000","last_change":"2024-04-24T06:41:35.299124+0000","last_active":"2024-04-24T06:44:02.129692+0000","last_peered":"2024-04-24T06:44:02.129692+0000","last_clean":"2024-04-24T06:44:02.129692+0000","last_became_active":"2024-04-24T06:41:35.298246+0000","last_became_peered":"2024-04-24T06:41:35.298246+0000","last_unstale":"2024-04-24T06:44:02.129692+0000","last_undegraded":"2024-04-24T06:44:02.129692+0000","last_fullsized":"2024-04-24T06:44:02.129692+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:07:22.649874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.980060+0000","last_change":"2024-04-24T06:42:42.888183+0000","last_active":"2024-04-24T06:44:01.980060+0000","last_peered":"2024-04-24T06:44:01.980060+0000","last_clean":"2024-04-24T06:44:01.980060+0000","last_became_active":"2024-04-24T06:42:42.887373+0000","last_became_peered":"2024-04-24T06:42:42.887373+0000","last_unstale":"2024-04-24T06:44:01.980060+0000","last_undegraded":"2024-04-24T06:44:01.980060+0000","last_fullsized":"2024-04-24T06:44:01.980060+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:44:51.456753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.128989+0000","last_change":"2024-04-24T06:41:35.301619+0000","last_active":"2024-04-24T06:44:02.128989+0000","last_peered":"2024-04-24T06:44:02.128989+0000","last_clean":"2024-04-24T06:44:02.128989+0000","last_became_active":"2024-04-24T06:41:35.297987+0000","last_became_peered":"2024-04-24T06:41:35.297987+0000","last_unstale":"2024-04-24T06:44:02.128989+0000","last_undegraded":"2024-04-24T06:44:02.128989+0000","last_fullsized":"2024-04-24T06:44:02.128989+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:06:01.235593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.319683+0000","last_change":"2024-04-24T06:41:35.304347+0000","last_active":"2024-04-24T06:44:02.319683+0000","last_peered":"2024-04-24T06:44:02.319683+0000","last_clean":"2024-04-24T06:44:02.319683+0000","last_became_active":"2024-04-24T06:41:35.303971+0000","last_became_peered":"2024-04-24T06:41:35.303971+0000","last_unstale":"2024-04-24T06:44:02.319683+0000","last_undegraded":"2024-04-24T06:44:02.319683+0000","last_fullsized":"2024-04-24T06:44:02.319683+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:31:54.813243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:01.981029+0000","last_change":"2024-04-24T06:41:35.308314+0000","last_active":"2024-04-24T06:44:01.981029+0000","last_peered":"2024-04-24T06:44:01.981029+0000","last_clean":"2024-04-24T06:44:01.981029+0000","last_became_active":"2024-04-24T06:41:35.306581+0000","last_became_peered":"2024-04-24T06:41:35.306581+0000","last_unstale":"2024-04-24T06:44:01.981029+0000","last_undegraded":"2024-04-24T06:44:01.981029+0000","last_fullsized":"2024-04-24T06:44:01.981029+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:34:43.095087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T06:44:02.528089+0000","last_change":"2024-04-24T06:41:35.289166+0000","last_active":"2024-04-24T06:44:02.528089+0000","last_peered":"2024-04-24T06:44:02.528089+0000","last_clean":"2024-04-24T06:44:02.528089+0000","last_became_active":"2024-04-24T06:41:35.287672+0000","last_became_peered":"2024-04-24T06:41:35.287672+0000","last_unstale":"2024-04-24T06:44:02.528089+0000","last_undegraded":"2024-04-24T06:44:02.528089+0000","last_fullsized":"2024-04-24T06:44:02.528089+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T06:41:34.245207+0000","last_clean_scrub_stamp":"2024-04-24T06:41:34.245207+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:31:45.403387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":595775,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":981,"num_read_kb":598,"num_write":282,"num_write_kb":1430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1787325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":220,"ondisk_log_size":220,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351017,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":460828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612654,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28792,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716360,"statfs":{"total":95995035648,"available":95965552640,"internally_reserved":0,"allocated":1818624,"data_stored":1259789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776180,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":978944,"data_stored":461030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037819,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28376,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716776,"statfs":{"total":95995035648,"available":95965978624,"internally_reserved":0,"allocated":1388544,"data_stored":863122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299455,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28572,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716580,"statfs":{"total":95995035648,"available":95965777920,"internally_reserved":0,"allocated":1589248,"data_stored":1062227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462982,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":469375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574945,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1052672,"data_stored":535735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738471,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":987136,"data_stored":461690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411423,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1073152,"data_stored":537442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182485,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28384,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716768,"statfs":{"total":95995035648,"available":95965970432,"internally_reserved":0,"allocated":1396736,"data_stored":863422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920849,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":470203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724620,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32872,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712280,"statfs":{"total":95995035648,"available":95961374720,"internally_reserved":0,"allocated":1736704,"data_stored":1193120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":68274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":133467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":393954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":200136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T06:44:08.765 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:44:08.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.996+0000 7f176ec00640 1 -- 172.21.15.5:0/2321746260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17680769c0 msgr2=0x7f1768076de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:08.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.996+0000 7f176ec00640 1 --2- 172.21.15.5:0/2321746260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17680769c0 0x7f1768076de0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f175c009900 tx=0x7f175c02a400 comp rx=0 tx=0).stop 2024-04-24T06:44:08.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.996+0000 7f176ec00640 1 -- 172.21.15.5:0/2321746260 shutdown_connections 2024-04-24T06:44:08.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.996+0000 7f176ec00640 1 --2- 172.21.15.5:0/2321746260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17680785a0 0x7f1768078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:08.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.996+0000 7f176ec00640 1 --2- 172.21.15.5:0/2321746260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1768077c00 0x7f1768078060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:08.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.996+0000 7f176ec00640 1 --2- 172.21.15.5:0/2321746260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17680769c0 0x7f1768076de0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:08.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.996+0000 7f176ec00640 1 -- 172.21.15.5:0/2321746260 >> 172.21.15.5:0/2321746260 conn(0x7f17680ff6d0 msgr2=0x7f1768101b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:08.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.996+0000 7f176ec00640 1 -- 172.21.15.5:0/2321746260 shutdown_connections 2024-04-24T06:44:08.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.996+0000 7f176ec00640 1 -- 172.21.15.5:0/2321746260 wait complete. 2024-04-24T06:44:09.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.997+0000 7f176ec00640 1 Processor -- start 2024-04-24T06:44:09.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.998+0000 7f176ec00640 1 -- start start 2024-04-24T06:44:09.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.998+0000 7f176ec00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17680769c0 0x7f176819e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:09.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.998+0000 7f176ec00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1768077c00 0x7f176819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:09.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.998+0000 7f176ec00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17680785a0 0x7f17681a5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:09.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.998+0000 7f176ec00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1768114230 con 0x7f1768077c00 2024-04-24T06:44:09.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.998+0000 7f176ec00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f17681140b0 con 0x7f17680769c0 2024-04-24T06:44:09.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.998+0000 7f176ec00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f17681143b0 con 0x7f17680785a0 2024-04-24T06:44:09.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.998+0000 7f176e200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17680785a0 0x7f17681a5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:09.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.998+0000 7f176ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1768077c00 0x7f176819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:09.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.998+0000 7f176d800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17680769c0 0x7f176819e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:09.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.998+0000 7f176e200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17680785a0 0x7f17681a5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:54322/0 (socket says 172.21.15.5:54322) 2024-04-24T06:44:09.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.998+0000 7f176e200640 1 -- 172.21.15.5:0/1091637997 learned_addr learned my addr 172.21.15.5:0/1091637997 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:09.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.999+0000 7f176d800640 1 -- 172.21.15.5:0/1091637997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17680785a0 msgr2=0x7f17681a5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:09.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.999+0000 7f176d800640 1 --2- 172.21.15.5:0/1091637997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17680785a0 0x7f17681a5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:09.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.999+0000 7f176d800640 1 -- 172.21.15.5:0/1091637997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1768077c00 msgr2=0x7f176819ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:09.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.999+0000 7f176d800640 1 --2- 172.21.15.5:0/1091637997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1768077c00 0x7f176819ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:09.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.999+0000 7f176d800640 1 -- 172.21.15.5:0/1091637997 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17681a6430 con 0x7f17680769c0 2024-04-24T06:44:09.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.999+0000 7f176e200640 1 --2- 172.21.15.5:0/1091637997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17680785a0 0x7f17681a5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:44:09.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:08.999+0000 7f176d800640 1 --2- 172.21.15.5:0/1091637997 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17680769c0 0x7f176819e810 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f175c02a920 tx=0x7f175c032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:09.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.000+0000 7f1766000640 1 -- 172.21.15.5:0/1091637997 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175c03f030 con 0x7f17680769c0 2024-04-24T06:44:09.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.000+0000 7f1766000640 1 -- 172.21.15.5:0/1091637997 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f175c003ce0 con 0x7f17680769c0 2024-04-24T06:44:09.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.000+0000 7f176ec00640 1 -- 172.21.15.5:0/1091637997 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17681a66c0 con 0x7f17680769c0 2024-04-24T06:44:09.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.000+0000 7f1766000640 1 -- 172.21.15.5:0/1091637997 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175c035070 con 0x7f17680769c0 2024-04-24T06:44:09.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.000+0000 7f176ec00640 1 -- 172.21.15.5:0/1091637997 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17681a6bd0 con 0x7f17680769c0 2024-04-24T06:44:09.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.001+0000 7f176ec00640 1 -- 172.21.15.5:0/1091637997 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f176804fe00 con 0x7f17680769c0 2024-04-24T06:44:09.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.003+0000 7f1766000640 1 -- 172.21.15.5:0/1091637997 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138872+0+0 (secure 0 0 0) 0x7f175c042030 con 0x7f17680769c0 2024-04-24T06:44:09.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.004+0000 7f1766000640 1 --2- 172.21.15.5:0/1091637997 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f17500a3eb0 0x7f17500a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:09.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.004+0000 7f1766000640 1 -- 172.21.15.5:0/1091637997 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f175c0f0ce0 con 0x7f17680769c0 2024-04-24T06:44:09.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.005+0000 7f176ce00640 1 --2- 172.21.15.5:0/1091637997 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f17500a3eb0 0x7f17500a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:09.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.005+0000 7f176ce00640 1 --2- 172.21.15.5:0/1091637997 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f17500a3eb0 0x7f17500a6370 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f176819fd20 tx=0x7f1754005f90 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:09.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.008+0000 7f1766000640 1 -- 172.21.15.5:0/1091637997 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f175c0bcbc0 con 0x7f17680769c0 2024-04-24T06:44:09.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:09 smithi037 ceph-mon[32864]: from='client.25354 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T06:44:09.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:09 smithi037 ceph-mon[32864]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:44:09.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:09 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:09.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:09 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:09.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:09 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:09.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:09 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:09.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:09 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:09.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:09 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:09.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:09 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:09.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:09 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:09.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:09 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:09.269 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:09 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:09.270 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:09 smithi037 ceph-mon[32864]: Adjusting osd_memory_target on smithi005 to 177.5M 2024-04-24T06:44:09.270 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:09 smithi037 ceph-mon[32864]: Unable to set osd_memory_target on smithi005 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-24T06:44:09.270 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:09 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:44:09.270 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:09 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:44:09.270 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:09 smithi037 ceph-mon[32864]: Updating smithi005:/etc/ceph/ceph.conf 2024-04-24T06:44:09.270 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:09 smithi037 ceph-mon[32864]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-24T06:44:09.270 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:09 smithi037 ceph-mon[32864]: Updating smithi137:/etc/ceph/ceph.conf 2024-04-24T06:44:09.270 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:09 smithi037 ceph-mon[32864]: from='client.25340 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T06:44:09.270 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:09 smithi037 ceph-mon[32864]: Updating smithi137:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T06:44:09.270 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:09 smithi037 ceph-mon[32864]: Updating smithi037:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T06:44:09.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.277+0000 7f176ec00640 1 -- 172.21.15.5:0/1091637997 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1768062c00 con 0x7f17680769c0 2024-04-24T06:44:09.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.278+0000 7f1766000640 1 -- 172.21.15.5:0/1091637997 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22884 (secure 0 0 0) 0x7f175c0c0ec0 con 0x7f17680769c0 2024-04-24T06:44:09.280 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:44:09.280 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":89,"fsid":"43e9bad0-0204-11ef-bc93-c7b262605968","created":"2024-04-24T06:32:26.340555+0000","modified":"2024-04-24T06:44:01.965991+0000","last_up_change":"2024-04-24T06:40:36.790934+0000","last_in_change":"2024-04-24T06:40:17.803257+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:36:27.320342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:41:32.254502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:41:33.739067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c2e974c-745c-4dd5-8c3d-8e8da2eb1ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6803","nonce":2155858515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6805","nonce":2155858515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6809","nonce":2155858515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6807","nonce":2155858515}]},"public_addr":"172.21.15.5:6803/2155858515","cluster_addr":"172.21.15.5:6805/2155858515","heartbeat_back_addr":"172.21.15.5:6809/2155858515","heartbeat_front_addr":"172.21.15.5:6807/2155858515","state":["exists","up"]},{"osd":1,"uuid":"50467744-db20-4721-bf2d-1c5db938297b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6801","nonce":1302362650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6803","nonce":1302362650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6807","nonce":1302362650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6805","nonce":1302362650}]},"public_addr":"172.21.15.37:6801/1302362650","cluster_addr":"172.21.15.37:6803/1302362650","heartbeat_back_addr":"172.21.15.37:6807/1302362650","heartbeat_front_addr":"172.21.15.37:6805/1302362650","state":["exists","up"]},{"osd":2,"uuid":"0e190b15-d04c-4288-8d48-ecf6664392fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6801","nonce":2885027256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6803","nonce":2885027256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6807","nonce":2885027256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6805","nonce":2885027256}]},"public_addr":"172.21.15.137:6801/2885027256","cluster_addr":"172.21.15.137:6803/2885027256","heartbeat_back_addr":"172.21.15.137:6807/2885027256","heartbeat_front_addr":"172.21.15.137:6805/2885027256","state":["exists","up"]},{"osd":3,"uuid":"e05592ba-d96a-472a-9f63-09a535d04dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6811","nonce":3701747260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6813","nonce":3701747260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6817","nonce":3701747260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6815","nonce":3701747260}]},"public_addr":"172.21.15.5:6811/3701747260","cluster_addr":"172.21.15.5:6813/3701747260","heartbeat_back_addr":"172.21.15.5:6817/3701747260","heartbeat_front_addr":"172.21.15.5:6815/3701747260","state":["exists","up"]},{"osd":4,"uuid":"c3f36e51-7a3b-4dd9-af4b-a10558830c4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6809","nonce":759253844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6811","nonce":759253844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6815","nonce":759253844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6813","nonce":759253844}]},"public_addr":"172.21.15.37:6809/759253844","cluster_addr":"172.21.15.37:6811/759253844","heartbeat_back_addr":"172.21.15.37:6815/759253844","heartbeat_front_addr":"172.21.15.37:6813/759253844","state":["exists","up"]},{"osd":5,"uuid":"97a0500c-cbb8-4ff5-b476-8634b26b3821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6809","nonce":3771921560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6811","nonce":3771921560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6815","nonce":3771921560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6813","nonce":3771921560}]},"public_addr":"172.21.15.137:6809/3771921560","cluster_addr":"172.21.15.137:6811/3771921560","heartbeat_back_addr":"172.21.15.137:6815/3771921560","heartbeat_front_addr":"172.21.15.137:6813/3771921560","state":["exists","up"]},{"osd":6,"uuid":"33a33d62-bbb3-4ed9-834e-19ae93a97670","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6819","nonce":2064636084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6821","nonce":2064636084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6825","nonce":2064636084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6823","nonce":2064636084}]},"public_addr":"172.21.15.5:6819/2064636084","cluster_addr":"172.21.15.5:6821/2064636084","heartbeat_back_addr":"172.21.15.5:6825/2064636084","heartbeat_front_addr":"172.21.15.5:6823/2064636084","state":["exists","up"]},{"osd":7,"uuid":"c0f4e036-4717-4514-bb59-ab11fbaab063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6817","nonce":2952750659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6819","nonce":2952750659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6823","nonce":2952750659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6821","nonce":2952750659}]},"public_addr":"172.21.15.37:6817/2952750659","cluster_addr":"172.21.15.37:6819/2952750659","heartbeat_back_addr":"172.21.15.37:6823/2952750659","heartbeat_front_addr":"172.21.15.37:6821/2952750659","state":["exists","up"]},{"osd":8,"uuid":"5391fd87-44ea-440f-80da-c54d0327c04c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6817","nonce":29724945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6819","nonce":29724945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6823","nonce":29724945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6821","nonce":29724945}]},"public_addr":"172.21.15.137:6817/29724945","cluster_addr":"172.21.15.137:6819/29724945","heartbeat_back_addr":"172.21.15.137:6823/29724945","heartbeat_front_addr":"172.21.15.137:6821/29724945","state":["exists","up"]},{"osd":9,"uuid":"d94c9dbc-f43d-4237-ac7f-47a6a5ede1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6827","nonce":2257102107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6829","nonce":2257102107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6833","nonce":2257102107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6831","nonce":2257102107}]},"public_addr":"172.21.15.5:6827/2257102107","cluster_addr":"172.21.15.5:6829/2257102107","heartbeat_back_addr":"172.21.15.5:6833/2257102107","heartbeat_front_addr":"172.21.15.5:6831/2257102107","state":["exists","up"]},{"osd":10,"uuid":"e2455332-2a5e-44b1-85d3-8e47da399f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6825","nonce":2451570637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6827","nonce":2451570637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6831","nonce":2451570637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6829","nonce":2451570637}]},"public_addr":"172.21.15.37:6825/2451570637","cluster_addr":"172.21.15.37:6827/2451570637","heartbeat_back_addr":"172.21.15.37:6831/2451570637","heartbeat_front_addr":"172.21.15.37:6829/2451570637","state":["exists","up"]},{"osd":11,"uuid":"49d9396b-704a-4cef-8210-75d30c28e924","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6825","nonce":90600596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6827","nonce":90600596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6831","nonce":90600596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6829","nonce":90600596}]},"public_addr":"172.21.15.137:6825/90600596","cluster_addr":"172.21.15.137:6827/90600596","heartbeat_back_addr":"172.21.15.137:6831/90600596","heartbeat_front_addr":"172.21.15.137:6829/90600596","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:35:35.591619+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:00.568817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:24.195643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:56.451724+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:22.109293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:46.161749+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:19.278305+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:45.620256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:10.041165+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:43.813697+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:10.736026+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:35.384377+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6800/4042936965":"2024-04-25T06:44:01.965881+0000","172.21.15.5:0/2912680052":"2024-04-25T06:44:01.965881+0000","172.21.15.5:6801/4042936965":"2024-04-25T06:44:01.965881+0000","172.21.15.5:0/3478347847":"2024-04-25T06:44:01.965881+0000","172.21.15.5:0/23307164":"2024-04-25T06:33:41.161852+0000","172.21.15.5:0/1012004002":"2024-04-25T06:44:01.965881+0000","172.21.15.5:6801/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:0/230785941":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/805658012":"2024-04-25T06:44:01.965881+0000","172.21.15.5:0/2940023597":"2024-04-25T06:33:07.040369+0000","172.21.15.5:6801/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3886613904":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/4199165437":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3459593499":"2024-04-25T06:33:41.161852+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:44:09.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.281+0000 7f176ec00640 1 -- 172.21.15.5:0/1091637997 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f17500a3eb0 msgr2=0x7f17500a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:09.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.281+0000 7f176ec00640 1 --2- 172.21.15.5:0/1091637997 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f17500a3eb0 0x7f17500a6370 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f176819fd20 tx=0x7f1754005f90 comp rx=0 tx=0).stop 2024-04-24T06:44:09.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.281+0000 7f176ec00640 1 -- 172.21.15.5:0/1091637997 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17680769c0 msgr2=0x7f176819e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:09.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.281+0000 7f176ec00640 1 --2- 172.21.15.5:0/1091637997 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17680769c0 0x7f176819e810 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f175c02a920 tx=0x7f175c032ce0 comp rx=0 tx=0).stop 2024-04-24T06:44:09.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.281+0000 7f176ec00640 1 -- 172.21.15.5:0/1091637997 shutdown_connections 2024-04-24T06:44:09.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.281+0000 7f176ec00640 1 --2- 172.21.15.5:0/1091637997 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f17500a3eb0 0x7f17500a6370 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:09.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.281+0000 7f176ec00640 1 --2- 172.21.15.5:0/1091637997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17680785a0 0x7f17681a5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:09.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.281+0000 7f176ec00640 1 --2- 172.21.15.5:0/1091637997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1768077c00 0x7f176819ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:09.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.281+0000 7f176ec00640 1 --2- 172.21.15.5:0/1091637997 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17680769c0 0x7f176819e810 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:09.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.281+0000 7f176ec00640 1 -- 172.21.15.5:0/1091637997 >> 172.21.15.5:0/1091637997 conn(0x7f17680ff6d0 msgr2=0x7f1768100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:09.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.281+0000 7f176ec00640 1 -- 172.21.15.5:0/1091637997 shutdown_connections 2024-04-24T06:44:09.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.281+0000 7f176ec00640 1 -- 172.21.15.5:0/1091637997 wait complete. 2024-04-24T06:44:09.303 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:36:27.320342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:41:32.254502+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:41:33.739067+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:44:09.303 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:44:09.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:09 smithi137 ceph-mon[33017]: from='client.25354 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T06:44:09.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:09 smithi137 ceph-mon[33017]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:44:09.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:09 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:09.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:09 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:09.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:09 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:09.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:09 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:09.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:09 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:09.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:09 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:09.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:09 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:09.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:09 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:09.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:09 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:09.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:09 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:09.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:09 smithi137 ceph-mon[33017]: Adjusting osd_memory_target on smithi005 to 177.5M 2024-04-24T06:44:09.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:09 smithi137 ceph-mon[33017]: Unable to set osd_memory_target on smithi005 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-24T06:44:09.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:09 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:44:09.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:09 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:44:09.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:09 smithi137 ceph-mon[33017]: Updating smithi005:/etc/ceph/ceph.conf 2024-04-24T06:44:09.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:09 smithi137 ceph-mon[33017]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-24T06:44:09.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:09 smithi137 ceph-mon[33017]: Updating smithi137:/etc/ceph/ceph.conf 2024-04-24T06:44:09.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:09 smithi137 ceph-mon[33017]: from='client.25340 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T06:44:09.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:09 smithi137 ceph-mon[33017]: Updating smithi137:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T06:44:09.448 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:09 smithi137 ceph-mon[33017]: Updating smithi037:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T06:44:09.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.492+0000 7f1a3f800640 1 -- 172.21.15.5:0/3283258585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a38105450 msgr2=0x7f1a381058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:09.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.492+0000 7f1a3f800640 1 --2- 172.21.15.5:0/3283258585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a38105450 0x7f1a381058d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009900 tx=0x7f1a2402a3f0 comp rx=0 tx=0).stop 2024-04-24T06:44:09.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.493+0000 7f1a3f800640 1 -- 172.21.15.5:0/3283258585 shutdown_connections 2024-04-24T06:44:09.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.493+0000 7f1a3f800640 1 --2- 172.21.15.5:0/3283258585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a38105e10 0x7f1a3810c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:09.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.493+0000 7f1a3f800640 1 --2- 172.21.15.5:0/3283258585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a38105450 0x7f1a381058d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:09.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.493+0000 7f1a3f800640 1 --2- 172.21.15.5:0/3283258585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a38104260 0x7f1a38104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:09.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.493+0000 7f1a3f800640 1 -- 172.21.15.5:0/3283258585 >> 172.21.15.5:0/3283258585 conn(0x7f1a380ff7f0 msgr2=0x7f1a38101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:09.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.493+0000 7f1a3f800640 1 -- 172.21.15.5:0/3283258585 shutdown_connections 2024-04-24T06:44:09.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.493+0000 7f1a3f800640 1 -- 172.21.15.5:0/3283258585 wait complete. 2024-04-24T06:44:09.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.494+0000 7f1a3f800640 1 Processor -- start 2024-04-24T06:44:09.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.494+0000 7f1a3f800640 1 -- start start 2024-04-24T06:44:09.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.494+0000 7f1a3f800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a38104260 0x7f1a3819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:09.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.494+0000 7f1a3f800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a38105450 0x7f1a3819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:09.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.494+0000 7f1a3f800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a38105e10 0x7f1a381a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:09.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.494+0000 7f1a3f800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3810ff40 con 0x7f1a38104260 2024-04-24T06:44:09.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.494+0000 7f1a3f800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3810fdc0 con 0x7f1a38105e10 2024-04-24T06:44:09.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.494+0000 7f1a3f800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1a381100c0 con 0x7f1a38105450 2024-04-24T06:44:09.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.494+0000 7f1a3ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a38104260 0x7f1a3819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:09.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.494+0000 7f1a3d800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a38105e10 0x7f1a381a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:09.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.494+0000 7f1a3ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a38104260 0x7f1a3819a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48764/0 (socket says 172.21.15.5:48764) 2024-04-24T06:44:09.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.494+0000 7f1a3ce00640 1 -- 172.21.15.5:0/1506927762 learned_addr learned my addr 172.21.15.5:0/1506927762 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:09.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.495+0000 7f1a37e00640 1 --2- 172.21.15.5:0/1506927762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a38105450 0x7f1a3819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:09.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.495+0000 7f1a3d800640 1 -- 172.21.15.5:0/1506927762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a38105450 msgr2=0x7f1a3819aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:09.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.495+0000 7f1a3d800640 1 --2- 172.21.15.5:0/1506927762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a38105450 0x7f1a3819aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:09.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.495+0000 7f1a3d800640 1 -- 172.21.15.5:0/1506927762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a38104260 msgr2=0x7f1a3819a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:09.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.495+0000 7f1a3d800640 1 --2- 172.21.15.5:0/1506927762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a38104260 0x7f1a3819a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:09.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.495+0000 7f1a3d800640 1 -- 172.21.15.5:0/1506927762 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a381a2160 con 0x7f1a38105e10 2024-04-24T06:44:09.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.495+0000 7f1a37e00640 1 --2- 172.21.15.5:0/1506927762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a38105450 0x7f1a3819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:44:09.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.495+0000 7f1a3d800640 1 --2- 172.21.15.5:0/1506927762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a38105e10 0x7f1a381a1af0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1a30009fd0 tx=0x7f1a3000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:09.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.495+0000 7f1a35600640 1 -- 172.21.15.5:0/1506927762 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3000ce20 con 0x7f1a38105e10 2024-04-24T06:44:09.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.495+0000 7f1a35600640 1 -- 172.21.15.5:0/1506927762 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a30005020 con 0x7f1a38105e10 2024-04-24T06:44:09.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.495+0000 7f1a35600640 1 -- 172.21.15.5:0/1506927762 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a30010720 con 0x7f1a38105e10 2024-04-24T06:44:09.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.495+0000 7f1a3f800640 1 -- 172.21.15.5:0/1506927762 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a381a2450 con 0x7f1a38105e10 2024-04-24T06:44:09.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.496+0000 7f1a3f800640 1 -- 172.21.15.5:0/1506927762 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a381a2920 con 0x7f1a38105e10 2024-04-24T06:44:09.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.497+0000 7f1a3f800640 1 -- 172.21.15.5:0/1506927762 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a3810b580 con 0x7f1a38105e10 2024-04-24T06:44:09.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.498+0000 7f1a35600640 1 -- 172.21.15.5:0/1506927762 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138872+0+0 (secure 0 0 0) 0x7f1a30005540 con 0x7f1a38105e10 2024-04-24T06:44:09.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.500+0000 7f1a35600640 1 --2- 172.21.15.5:0/1506927762 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1a200a3ec0 0x7f1a200a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:09.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.500+0000 7f1a35600640 1 -- 172.21.15.5:0/1506927762 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f1a300d1120 con 0x7f1a38105e10 2024-04-24T06:44:09.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.500+0000 7f1a3ce00640 1 --2- 172.21.15.5:0/1506927762 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1a200a3ec0 0x7f1a200a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:09.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.501+0000 7f1a3ce00640 1 --2- 172.21.15.5:0/1506927762 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1a200a3ec0 0x7f1a200a6380 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c0058c0 tx=0x7f1a2c005f90 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:09.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.504+0000 7f1a35600640 1 -- 172.21.15.5:0/1506927762 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1a3009d520 con 0x7f1a38105e10 2024-04-24T06:44:09.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:09 smithi005 ceph-mon[27798]: from='client.25354 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T06:44:09.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:09 smithi005 ceph-mon[27798]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T06:44:09.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:09 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:09.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:09 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:09.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:09 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:09.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:09 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:09.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:09 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:09.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:09 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:09.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:09 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:09.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:09 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:09.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:09 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:09.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:09 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:44:09.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:09 smithi005 ceph-mon[27798]: Adjusting osd_memory_target on smithi005 to 177.5M 2024-04-24T06:44:09.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:09 smithi005 ceph-mon[27798]: Unable to set osd_memory_target on smithi005 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-24T06:44:09.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:09 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:44:09.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:09 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:44:09.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:09 smithi005 ceph-mon[27798]: Updating smithi005:/etc/ceph/ceph.conf 2024-04-24T06:44:09.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:09 smithi005 ceph-mon[27798]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-24T06:44:09.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:09 smithi005 ceph-mon[27798]: Updating smithi137:/etc/ceph/ceph.conf 2024-04-24T06:44:09.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:09 smithi005 ceph-mon[27798]: from='client.25340 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T06:44:09.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:09 smithi005 ceph-mon[27798]: Updating smithi137:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T06:44:09.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:09 smithi005 ceph-mon[27798]: Updating smithi037:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T06:44:09.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.725+0000 7f1a3f800640 1 -- 172.21.15.5:0/1506927762 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1a3802d860 con 0x7f1a38105e10 2024-04-24T06:44:09.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.725+0000 7f1a35600640 1 -- 172.21.15.5:0/1506927762 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f1a3009ce20 con 0x7f1a38105e10 2024-04-24T06:44:09.726 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-24T06:44:09.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.727+0000 7f1a3f800640 1 -- 172.21.15.5:0/1506927762 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1a200a3ec0 msgr2=0x7f1a200a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:09.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.727+0000 7f1a3f800640 1 --2- 172.21.15.5:0/1506927762 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1a200a3ec0 0x7f1a200a6380 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c0058c0 tx=0x7f1a2c005f90 comp rx=0 tx=0).stop 2024-04-24T06:44:09.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.727+0000 7f1a3f800640 1 -- 172.21.15.5:0/1506927762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a38105e10 msgr2=0x7f1a381a1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:09.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.727+0000 7f1a3f800640 1 --2- 172.21.15.5:0/1506927762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a38105e10 0x7f1a381a1af0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1a30009fd0 tx=0x7f1a3000ef10 comp rx=0 tx=0).stop 2024-04-24T06:44:09.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.728+0000 7f1a3f800640 1 -- 172.21.15.5:0/1506927762 shutdown_connections 2024-04-24T06:44:09.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.728+0000 7f1a3f800640 1 --2- 172.21.15.5:0/1506927762 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1a200a3ec0 0x7f1a200a6380 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:09.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.728+0000 7f1a3f800640 1 --2- 172.21.15.5:0/1506927762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a38105e10 0x7f1a381a1af0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:09.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.728+0000 7f1a3f800640 1 --2- 172.21.15.5:0/1506927762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a38105450 0x7f1a3819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:09.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.728+0000 7f1a3f800640 1 --2- 172.21.15.5:0/1506927762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a38104260 0x7f1a3819a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:09.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.728+0000 7f1a3f800640 1 -- 172.21.15.5:0/1506927762 >> 172.21.15.5:0/1506927762 conn(0x7f1a380ff7f0 msgr2=0x7f1a38108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:09.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.728+0000 7f1a3f800640 1 -- 172.21.15.5:0/1506927762 shutdown_connections 2024-04-24T06:44:09.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.728+0000 7f1a3f800640 1 -- 172.21.15.5:0/1506927762 wait complete. 2024-04-24T06:44:09.749 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:44:09.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.938+0000 7fac17e00640 1 -- 172.21.15.5:0/3601089336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac10104260 msgr2=0x7fac10104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:09.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.938+0000 7fac17e00640 1 --2- 172.21.15.5:0/3601089336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac10104260 0x7fac10104680 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fac04009900 tx=0x7fac0402a410 comp rx=0 tx=0).stop 2024-04-24T06:44:09.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.938+0000 7fac17e00640 1 -- 172.21.15.5:0/3601089336 shutdown_connections 2024-04-24T06:44:09.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.938+0000 7fac17e00640 1 --2- 172.21.15.5:0/3601089336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac10105e10 0x7fac1010c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:09.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.938+0000 7fac17e00640 1 --2- 172.21.15.5:0/3601089336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac10105450 0x7fac101058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:09.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.938+0000 7fac17e00640 1 --2- 172.21.15.5:0/3601089336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac10104260 0x7fac10104680 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:09.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.938+0000 7fac17e00640 1 -- 172.21.15.5:0/3601089336 >> 172.21.15.5:0/3601089336 conn(0x7fac100ff7f0 msgr2=0x7fac10101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:09.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.938+0000 7fac17e00640 1 -- 172.21.15.5:0/3601089336 shutdown_connections 2024-04-24T06:44:09.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.938+0000 7fac17e00640 1 -- 172.21.15.5:0/3601089336 wait complete. 2024-04-24T06:44:09.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.939+0000 7fac17e00640 1 Processor -- start 2024-04-24T06:44:09.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.939+0000 7fac17e00640 1 -- start start 2024-04-24T06:44:09.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.939+0000 7fac17e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac10104260 0x7fac1019a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:09.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.939+0000 7fac17e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac10105450 0x7fac1019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:09.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.939+0000 7fac17e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac10105e10 0x7fac101a1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:09.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.939+0000 7fac17e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fac1010ff10 con 0x7fac10104260 2024-04-24T06:44:09.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.939+0000 7fac17e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fac1010fd90 con 0x7fac10105450 2024-04-24T06:44:09.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.939+0000 7fac17e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fac10110090 con 0x7fac10105e10 2024-04-24T06:44:09.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.940+0000 7fac15400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac10104260 0x7fac1019a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:09.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.940+0000 7fac15400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac10104260 0x7fac1019a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48784/0 (socket says 172.21.15.5:48784) 2024-04-24T06:44:09.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.940+0000 7fac15400640 1 -- 172.21.15.5:0/865229232 learned_addr learned my addr 172.21.15.5:0/865229232 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:09.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.940+0000 7fac14a00640 1 --2- 172.21.15.5:0/865229232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac10105450 0x7fac1019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:09.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.940+0000 7fac15e00640 1 --2- 172.21.15.5:0/865229232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac10105e10 0x7fac101a1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:09.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.940+0000 7fac14a00640 1 -- 172.21.15.5:0/865229232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac10105e10 msgr2=0x7fac101a1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:09.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.940+0000 7fac14a00640 1 --2- 172.21.15.5:0/865229232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac10105e10 0x7fac101a1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:09.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.940+0000 7fac14a00640 1 -- 172.21.15.5:0/865229232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac10104260 msgr2=0x7fac1019a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:09.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.940+0000 7fac14a00640 1 --2- 172.21.15.5:0/865229232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac10104260 0x7fac1019a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:09.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.940+0000 7fac14a00640 1 -- 172.21.15.5:0/865229232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac101a21d0 con 0x7fac10105450 2024-04-24T06:44:09.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.940+0000 7fac15400640 1 --2- 172.21.15.5:0/865229232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac10104260 0x7fac1019a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:44:09.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.940+0000 7fac15e00640 1 --2- 172.21.15.5:0/865229232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac10105e10 0x7fac101a1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:44:09.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.941+0000 7fac14a00640 1 --2- 172.21.15.5:0/865229232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac10105450 0x7fac1019aaf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fabfc00e9f0 tx=0x7fabfc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:09.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.941+0000 7fac0e000640 1 -- 172.21.15.5:0/865229232 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabfc00ccc0 con 0x7fac10105450 2024-04-24T06:44:09.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.941+0000 7fac17e00640 1 -- 172.21.15.5:0/865229232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac101a24c0 con 0x7fac10105450 2024-04-24T06:44:09.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.941+0000 7fac0e000640 1 -- 172.21.15.5:0/865229232 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabfc010910 con 0x7fac10105450 2024-04-24T06:44:09.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.941+0000 7fac0e000640 1 -- 172.21.15.5:0/865229232 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabfc018960 con 0x7fac10105450 2024-04-24T06:44:09.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.941+0000 7fac17e00640 1 -- 172.21.15.5:0/865229232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac101a2a60 con 0x7fac10105450 2024-04-24T06:44:09.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.942+0000 7fac17e00640 1 -- 172.21.15.5:0/865229232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac1010b580 con 0x7fac10105450 2024-04-24T06:44:09.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.945+0000 7fac0e000640 1 -- 172.21.15.5:0/865229232 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138872+0+0 (secure 0 0 0) 0x7fabfc010ac0 con 0x7fac10105450 2024-04-24T06:44:09.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.946+0000 7fac0e000640 1 --2- 172.21.15.5:0/865229232 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fabf80a3ec0 0x7fabf80a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:09.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.946+0000 7fac0e000640 1 -- 172.21.15.5:0/865229232 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fabfc0d0e00 con 0x7fac10105450 2024-04-24T06:44:09.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.946+0000 7fac15400640 1 --2- 172.21.15.5:0/865229232 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fabf80a3ec0 0x7fabf80a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:09.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.947+0000 7fac15400640 1 --2- 172.21.15.5:0/865229232 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fabf80a3ec0 0x7fabf80a6380 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fac04000c00 tx=0x7fac040315b0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:09.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:09.948+0000 7fac0e000640 1 -- 172.21.15.5:0/865229232 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fabfc09ccc0 con 0x7fac10105450 2024-04-24T06:44:10.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.160+0000 7fac17e00640 1 -- 172.21.15.5:0/865229232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fac1002d7b0 con 0x7fac10105450 2024-04-24T06:44:10.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.161+0000 7fac0e000640 1 -- 172.21.15.5:0/865229232 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7fabfc0a0fc0 con 0x7fac10105450 2024-04-24T06:44:10.163 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-24T06:44:10.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.163+0000 7fac17e00640 1 -- 172.21.15.5:0/865229232 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fabf80a3ec0 msgr2=0x7fabf80a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:10.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.163+0000 7fac17e00640 1 --2- 172.21.15.5:0/865229232 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fabf80a3ec0 0x7fabf80a6380 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fac04000c00 tx=0x7fac040315b0 comp rx=0 tx=0).stop 2024-04-24T06:44:10.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.163+0000 7fac17e00640 1 -- 172.21.15.5:0/865229232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac10105450 msgr2=0x7fac1019aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:10.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.163+0000 7fac17e00640 1 --2- 172.21.15.5:0/865229232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac10105450 0x7fac1019aaf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fabfc00e9f0 tx=0x7fabfc00eeb0 comp rx=0 tx=0).stop 2024-04-24T06:44:10.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.164+0000 7fac17e00640 1 -- 172.21.15.5:0/865229232 shutdown_connections 2024-04-24T06:44:10.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.164+0000 7fac17e00640 1 --2- 172.21.15.5:0/865229232 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fabf80a3ec0 0x7fabf80a6380 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:10.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.164+0000 7fac17e00640 1 --2- 172.21.15.5:0/865229232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac10105e10 0x7fac101a1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:10.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.164+0000 7fac17e00640 1 --2- 172.21.15.5:0/865229232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac10105450 0x7fac1019aaf0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:10.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.164+0000 7fac17e00640 1 --2- 172.21.15.5:0/865229232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac10104260 0x7fac1019a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:10.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.164+0000 7fac17e00640 1 -- 172.21.15.5:0/865229232 >> 172.21.15.5:0/865229232 conn(0x7fac100ff7f0 msgr2=0x7fac10108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:10.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.164+0000 7fac17e00640 1 -- 172.21.15.5:0/865229232 shutdown_connections 2024-04-24T06:44:10.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.164+0000 7fac17e00640 1 -- 172.21.15.5:0/865229232 wait complete. 2024-04-24T06:44:10.185 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:44:10.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.373+0000 7fb9fe200640 1 -- 172.21.15.5:0/1227661910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9f80743d0 msgr2=0x7fb9f80747b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:10.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.373+0000 7fb9fe200640 1 --2- 172.21.15.5:0/1227661910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9f80743d0 0x7fb9f80747b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb9e8009900 tx=0x7fb9e802a480 comp rx=0 tx=0).stop 2024-04-24T06:44:10.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.373+0000 7fb9fe200640 1 -- 172.21.15.5:0/1227661910 shutdown_connections 2024-04-24T06:44:10.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.373+0000 7fb9fe200640 1 --2- 172.21.15.5:0/1227661910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9f810e860 0x7fb9f8110c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:10.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.374+0000 7fb9fe200640 1 --2- 172.21.15.5:0/1227661910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9f8074cf0 0x7fb9f810e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:10.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.374+0000 7fb9fe200640 1 --2- 172.21.15.5:0/1227661910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9f80743d0 0x7fb9f80747b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:10.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.374+0000 7fb9fe200640 1 -- 172.21.15.5:0/1227661910 >> 172.21.15.5:0/1227661910 conn(0x7fb9f80fdab0 msgr2=0x7fb9f80ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:10.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.374+0000 7fb9fe200640 1 -- 172.21.15.5:0/1227661910 shutdown_connections 2024-04-24T06:44:10.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.374+0000 7fb9fe200640 1 -- 172.21.15.5:0/1227661910 wait complete. 2024-04-24T06:44:10.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.375+0000 7fb9fe200640 1 Processor -- start 2024-04-24T06:44:10.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.375+0000 7fb9fe200640 1 -- start start 2024-04-24T06:44:10.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.376+0000 7fb9fe200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9f80743d0 0x7fb9f8101f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:10.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.376+0000 7fb9fe200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9f8074cf0 0x7fb9f81024c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:10.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.376+0000 7fb9fe200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9f810e860 0x7fb9f8102a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:10.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.376+0000 7fb9fe200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f8114620 con 0x7fb9f810e860 2024-04-24T06:44:10.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.376+0000 7fb9fe200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f81144a0 con 0x7fb9f80743d0 2024-04-24T06:44:10.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.376+0000 7fb9fe200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f81147a0 con 0x7fb9f8074cf0 2024-04-24T06:44:10.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.376+0000 7fb9f7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9f810e860 0x7fb9f8102a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:10.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.376+0000 7fb9f7400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9f80743d0 0x7fb9f8101f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:10.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.376+0000 7fb9f7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9f810e860 0x7fb9f8102a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48796/0 (socket says 172.21.15.5:48796) 2024-04-24T06:44:10.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.376+0000 7fb9f7e00640 1 -- 172.21.15.5:0/1017145709 learned_addr learned my addr 172.21.15.5:0/1017145709 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:10.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.376+0000 7fb9f6a00640 1 --2- 172.21.15.5:0/1017145709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9f8074cf0 0x7fb9f81024c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:10.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.377+0000 7fb9f7400640 1 -- 172.21.15.5:0/1017145709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9f8074cf0 msgr2=0x7fb9f81024c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:10.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.377+0000 7fb9f7400640 1 --2- 172.21.15.5:0/1017145709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9f8074cf0 0x7fb9f81024c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:10.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.377+0000 7fb9f7400640 1 -- 172.21.15.5:0/1017145709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9f810e860 msgr2=0x7fb9f8102a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:10.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.377+0000 7fb9f7400640 1 --2- 172.21.15.5:0/1017145709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9f810e860 0x7fb9f8102a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:10.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.377+0000 7fb9f7400640 1 -- 172.21.15.5:0/1017145709 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9f81ab200 con 0x7fb9f80743d0 2024-04-24T06:44:10.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.377+0000 7fb9f7400640 1 --2- 172.21.15.5:0/1017145709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9f80743d0 0x7fb9f8101f80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb9e8009230 tx=0x7fb9e8003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:10.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.377+0000 7fb9efe00640 1 -- 172.21.15.5:0/1017145709 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e803f070 con 0x7fb9f80743d0 2024-04-24T06:44:10.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.377+0000 7fb9efe00640 1 -- 172.21.15.5:0/1017145709 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9e8031c90 con 0x7fb9f80743d0 2024-04-24T06:44:10.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.377+0000 7fb9fe200640 1 -- 172.21.15.5:0/1017145709 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9f81ab490 con 0x7fb9f80743d0 2024-04-24T06:44:10.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.377+0000 7fb9efe00640 1 -- 172.21.15.5:0/1017145709 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e8035030 con 0x7fb9f80743d0 2024-04-24T06:44:10.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.378+0000 7fb9fe200640 1 -- 172.21.15.5:0/1017145709 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9f81ab980 con 0x7fb9f80743d0 2024-04-24T06:44:10.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.379+0000 7fb9fe200640 1 -- 172.21.15.5:0/1017145709 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9f8075a80 con 0x7fb9f80743d0 2024-04-24T06:44:10.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.380+0000 7fb9efe00640 1 -- 172.21.15.5:0/1017145709 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138872+0+0 (secure 0 0 0) 0x7fb9e8042070 con 0x7fb9f80743d0 2024-04-24T06:44:10.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.382+0000 7fb9efe00640 1 --2- 172.21.15.5:0/1017145709 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb9dc0a3e40 0x7fb9dc0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:10.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.382+0000 7fb9efe00640 1 -- 172.21.15.5:0/1017145709 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fb9e80f1260 con 0x7fb9f80743d0 2024-04-24T06:44:10.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.382+0000 7fb9f6a00640 1 --2- 172.21.15.5:0/1017145709 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb9dc0a3e40 0x7fb9dc0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:10.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.383+0000 7fb9f6a00640 1 --2- 172.21.15.5:0/1017145709 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb9dc0a3e40 0x7fb9dc0a6300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb9f81036f0 tx=0x7fb9e4005f90 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:10.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.386+0000 7fb9efe00640 1 -- 172.21.15.5:0/1017145709 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb9e80bd140 con 0x7fb9f80743d0 2024-04-24T06:44:10.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:10 smithi137 ceph-mon[33017]: from='client.25346 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T06:44:10.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:10 smithi137 ceph-mon[33017]: Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-04-24T06:44:10.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:10 smithi137 ceph-mon[33017]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-04-24T06:44:10.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:10 smithi137 ceph-mon[33017]: Updating smithi005:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T06:44:10.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:10 smithi137 ceph-mon[33017]: Updating smithi137:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T06:44:10.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:10 smithi137 ceph-mon[33017]: Updating smithi037:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T06:44:10.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:10 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/1091637997' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:44:10.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:10 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:10.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:10 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:10.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:10 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:10.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:10 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:10.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:10 smithi137 ceph-mon[33017]: Updating smithi005:/etc/ceph/ceph.client.admin.keyring 2024-04-24T06:44:10.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:10 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/1506927762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T06:44:10.447 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:10 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/865229232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T06:44:10.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:10 smithi005 ceph-mon[27798]: from='client.25346 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T06:44:10.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:10 smithi005 ceph-mon[27798]: Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-04-24T06:44:10.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:10 smithi005 ceph-mon[27798]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-04-24T06:44:10.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:10 smithi005 ceph-mon[27798]: Updating smithi005:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T06:44:10.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:10 smithi005 ceph-mon[27798]: Updating smithi137:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T06:44:10.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:10 smithi005 ceph-mon[27798]: Updating smithi037:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T06:44:10.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:10 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1091637997' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:44:10.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:10 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:10.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:10 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:10.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:10 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:10.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:10 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:10.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:10 smithi005 ceph-mon[27798]: Updating smithi005:/etc/ceph/ceph.client.admin.keyring 2024-04-24T06:44:10.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:10 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1506927762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T06:44:10.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:10 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/865229232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T06:44:10.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:10 smithi037 ceph-mon[32864]: from='client.25346 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T06:44:10.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:10 smithi037 ceph-mon[32864]: Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-04-24T06:44:10.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:10 smithi037 ceph-mon[32864]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-04-24T06:44:10.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:10 smithi037 ceph-mon[32864]: Updating smithi005:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T06:44:10.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:10 smithi037 ceph-mon[32864]: Updating smithi137:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T06:44:10.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:10 smithi037 ceph-mon[32864]: Updating smithi037:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T06:44:10.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:10 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/1091637997' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:44:10.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:10 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:10.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:10 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:10.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:10 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:10.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:10 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:10.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:10 smithi037 ceph-mon[32864]: Updating smithi005:/etc/ceph/ceph.client.admin.keyring 2024-04-24T06:44:10.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:10 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/1506927762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T06:44:10.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:10 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/865229232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T06:44:10.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.598+0000 7fb9fe200640 1 -- 172.21.15.5:0/1017145709 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb9f8109a40 con 0x7fb9f80743d0 2024-04-24T06:44:10.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.599+0000 7fb9efe00640 1 -- 172.21.15.5:0/1017145709 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7fb9e80c1440 con 0x7fb9f80743d0 2024-04-24T06:44:10.600 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-24T06:44:10.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.601+0000 7fb9fe200640 1 -- 172.21.15.5:0/1017145709 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb9dc0a3e40 msgr2=0x7fb9dc0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:10.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.601+0000 7fb9fe200640 1 --2- 172.21.15.5:0/1017145709 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb9dc0a3e40 0x7fb9dc0a6300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb9f81036f0 tx=0x7fb9e4005f90 comp rx=0 tx=0).stop 2024-04-24T06:44:10.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.601+0000 7fb9fe200640 1 -- 172.21.15.5:0/1017145709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9f80743d0 msgr2=0x7fb9f8101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:10.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.601+0000 7fb9fe200640 1 --2- 172.21.15.5:0/1017145709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9f80743d0 0x7fb9f8101f80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb9e8009230 tx=0x7fb9e8003b20 comp rx=0 tx=0).stop 2024-04-24T06:44:10.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.601+0000 7fb9fe200640 1 -- 172.21.15.5:0/1017145709 shutdown_connections 2024-04-24T06:44:10.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.601+0000 7fb9fe200640 1 --2- 172.21.15.5:0/1017145709 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb9dc0a3e40 0x7fb9dc0a6300 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:10.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.601+0000 7fb9fe200640 1 --2- 172.21.15.5:0/1017145709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9f810e860 0x7fb9f8102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:10.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.601+0000 7fb9fe200640 1 --2- 172.21.15.5:0/1017145709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9f8074cf0 0x7fb9f81024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:10.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.601+0000 7fb9fe200640 1 --2- 172.21.15.5:0/1017145709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9f80743d0 0x7fb9f8101f80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:10.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.601+0000 7fb9fe200640 1 -- 172.21.15.5:0/1017145709 >> 172.21.15.5:0/1017145709 conn(0x7fb9f80fdab0 msgr2=0x7fb9f80ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:10.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.601+0000 7fb9fe200640 1 -- 172.21.15.5:0/1017145709 shutdown_connections 2024-04-24T06:44:10.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.601+0000 7fb9fe200640 1 -- 172.21.15.5:0/1017145709 wait complete. 2024-04-24T06:44:10.621 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:44:10.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.813+0000 7feba3e00640 1 -- 172.21.15.5:0/3248516834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feba4103090 msgr2=0x7feba4103470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:10.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.813+0000 7feba3e00640 1 --2- 172.21.15.5:0/3248516834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feba4103090 0x7feba4103470 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7feb94009900 tx=0x7feb9402a480 comp rx=0 tx=0).stop 2024-04-24T06:44:10.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.813+0000 7feba3e00640 1 -- 172.21.15.5:0/3248516834 shutdown_connections 2024-04-24T06:44:10.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.813+0000 7feba3e00640 1 --2- 172.21.15.5:0/3248516834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feba410c4b0 0x7feba410e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:10.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.813+0000 7feba3e00640 1 --2- 172.21.15.5:0/3248516834 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feba41039b0 0x7feba410be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:10.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.813+0000 7feba3e00640 1 --2- 172.21.15.5:0/3248516834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feba4103090 0x7feba4103470 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:10.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.813+0000 7feba3e00640 1 -- 172.21.15.5:0/3248516834 >> 172.21.15.5:0/3248516834 conn(0x7feba40fc890 msgr2=0x7feba40fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:10.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.813+0000 7feba3e00640 1 -- 172.21.15.5:0/3248516834 shutdown_connections 2024-04-24T06:44:10.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.813+0000 7feba3e00640 1 -- 172.21.15.5:0/3248516834 wait complete. 2024-04-24T06:44:10.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.814+0000 7feba3e00640 1 Processor -- start 2024-04-24T06:44:10.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.814+0000 7feba3e00640 1 -- start start 2024-04-24T06:44:10.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.814+0000 7feba3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feba4103090 0x7feba41a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:10.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.814+0000 7feba3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feba41039b0 0x7feba41a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:10.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.814+0000 7feba3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feba410c4b0 0x7feba419a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:10.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.814+0000 7feba3e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feba41122e0 con 0x7feba410c4b0 2024-04-24T06:44:10.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.814+0000 7feba3e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7feba4112160 con 0x7feba4103090 2024-04-24T06:44:10.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.814+0000 7feba3e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7feba4112460 con 0x7feba41039b0 2024-04-24T06:44:10.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.815+0000 7feba3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feba410c4b0 0x7feba419a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:10.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.815+0000 7feba2000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feba41039b0 0x7feba41a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:10.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.815+0000 7feba3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feba410c4b0 0x7feba419a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48802/0 (socket says 172.21.15.5:48802) 2024-04-24T06:44:10.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.815+0000 7feba3400640 1 -- 172.21.15.5:0/3696510880 learned_addr learned my addr 172.21.15.5:0/3696510880 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:10.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.815+0000 7feba2a00640 1 --2- 172.21.15.5:0/3696510880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feba4103090 0x7feba41a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:10.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.815+0000 7feba3400640 1 -- 172.21.15.5:0/3696510880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feba41039b0 msgr2=0x7feba41a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:10.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.815+0000 7feba3400640 1 --2- 172.21.15.5:0/3696510880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feba41039b0 0x7feba41a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:10.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.815+0000 7feba3400640 1 -- 172.21.15.5:0/3696510880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feba4103090 msgr2=0x7feba41a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:10.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.815+0000 7feba3400640 1 --2- 172.21.15.5:0/3696510880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feba4103090 0x7feba41a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:10.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.815+0000 7feba3400640 1 -- 172.21.15.5:0/3696510880 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba419af90 con 0x7feba410c4b0 2024-04-24T06:44:10.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.816+0000 7feba3400640 1 --2- 172.21.15.5:0/3696510880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feba410c4b0 0x7feba419a7c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7feb9c00eb10 tx=0x7feb9c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:10.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.816+0000 7feb9b400640 1 -- 172.21.15.5:0/3696510880 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c004b20 con 0x7feba410c4b0 2024-04-24T06:44:10.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.816+0000 7feb9b400640 1 -- 172.21.15.5:0/3696510880 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb9c004cc0 con 0x7feba410c4b0 2024-04-24T06:44:10.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.816+0000 7feb9b400640 1 -- 172.21.15.5:0/3696510880 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c010760 con 0x7feba410c4b0 2024-04-24T06:44:10.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.816+0000 7feba3e00640 1 -- 172.21.15.5:0/3696510880 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feba419b280 con 0x7feba410c4b0 2024-04-24T06:44:10.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.816+0000 7feba3e00640 1 -- 172.21.15.5:0/3696510880 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feba41a74a0 con 0x7feba410c4b0 2024-04-24T06:44:10.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.817+0000 7feba3e00640 1 -- 172.21.15.5:0/3696510880 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feba4069be0 con 0x7feba410c4b0 2024-04-24T06:44:10.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.820+0000 7feb9b400640 1 -- 172.21.15.5:0/3696510880 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 138872+0+0 (secure 0 0 0) 0x7feb9c005000 con 0x7feba410c4b0 2024-04-24T06:44:10.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.821+0000 7feb9b400640 1 --2- 172.21.15.5:0/3696510880 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7feb880a3eb0 0x7feb880a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:10.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.821+0000 7feb9b400640 1 -- 172.21.15.5:0/3696510880 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7feb9c0d1450 con 0x7feba410c4b0 2024-04-24T06:44:10.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.821+0000 7feba2a00640 1 --2- 172.21.15.5:0/3696510880 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7feb880a3eb0 0x7feb880a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:10.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.822+0000 7feba2a00640 1 --2- 172.21.15.5:0/3696510880 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7feb880a3eb0 0x7feb880a6370 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7feb94000c00 tx=0x7feb94005b50 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:10.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:10.824+0000 7feb9b400640 1 -- 172.21.15.5:0/3696510880 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feb9c09d850 con 0x7feba410c4b0 2024-04-24T06:44:11.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.077+0000 7feba3e00640 1 -- 172.21.15.5:0/3696510880 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feba41a7760 con 0x7feba410c4b0 2024-04-24T06:44:11.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.079+0000 7feb9b400640 1 -- 172.21.15.5:0/3696510880 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11856 (secure 0 0 0) 0x7feb9c09d150 con 0x7feba410c4b0 2024-04-24T06:44:11.080 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:44:11.080 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1735462750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1735462750},{"type":"v1","addr":"172.21.15.5:6837","nonce":1735462750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1499469370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1499469370},{"type":"v1","addr":"172.21.15.5:6841","nonce":1499469370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3705638847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3705638847},{"type":"v1","addr":"172.21.15.37:6835","nonce":3705638847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T06:41:34.764490+0000","modified":"2024-04-24T06:44:01.711021+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24550,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6837/1390103296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1390103296},{"type":"v1","addr":"172.21.15.37:6837","nonce":1390103296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3217754215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3217754215},{"type":"v1","addr":"172.21.15.137:6835","nonce":3217754215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6837/3044222976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3044222976},{"type":"v1","addr":"172.21.15.137:6837","nonce":3044222976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.5:6839/3173201459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3173201459},{"type":"v1","addr":"172.21.15.5:6839","nonce":3173201459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6839/1889774232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1889774232},{"type":"v1","addr":"172.21.15.137:6839","nonce":1889774232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6839/3536721753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":3536721753},{"type":"v1","addr":"172.21.15.37:6839","nonce":3536721753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24569,24539,24550]},"id":1}]} 2024-04-24T06:44:11.081 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 23 2024-04-24T06:44:11.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.082+0000 7feba3e00640 1 -- 172.21.15.5:0/3696510880 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7feb880a3eb0 msgr2=0x7feb880a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:11.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.082+0000 7feba3e00640 1 --2- 172.21.15.5:0/3696510880 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7feb880a3eb0 0x7feb880a6370 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7feb94000c00 tx=0x7feb94005b50 comp rx=0 tx=0).stop 2024-04-24T06:44:11.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.082+0000 7feba3e00640 1 -- 172.21.15.5:0/3696510880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feba410c4b0 msgr2=0x7feba419a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:11.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.082+0000 7feba3e00640 1 --2- 172.21.15.5:0/3696510880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feba410c4b0 0x7feba419a7c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7feb9c00eb10 tx=0x7feb9c00c2d0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.082+0000 7feba3e00640 1 -- 172.21.15.5:0/3696510880 shutdown_connections 2024-04-24T06:44:11.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.082+0000 7feba3e00640 1 --2- 172.21.15.5:0/3696510880 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7feb880a3eb0 0x7feb880a6370 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.082+0000 7feba3e00640 1 --2- 172.21.15.5:0/3696510880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feba410c4b0 0x7feba419a7c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.082+0000 7feba3e00640 1 --2- 172.21.15.5:0/3696510880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feba41039b0 0x7feba41a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.082+0000 7feba3e00640 1 --2- 172.21.15.5:0/3696510880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feba4103090 0x7feba41a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.082+0000 7feba3e00640 1 -- 172.21.15.5:0/3696510880 >> 172.21.15.5:0/3696510880 conn(0x7feba40fc890 msgr2=0x7feba40fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:11.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.082+0000 7feba3e00640 1 -- 172.21.15.5:0/3696510880 shutdown_connections 2024-04-24T06:44:11.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.082+0000 7feba3e00640 1 -- 172.21.15.5:0/3696510880 wait complete. 2024-04-24T06:44:11.104 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:44:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.297+0000 7efdfd800640 1 -- 172.21.15.5:0/654089929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdf81087b0 msgr2=0x7efdf8108b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.297+0000 7efdfd800640 1 --2- 172.21.15.5:0/654089929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdf81087b0 0x7efdf8108b90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7efde8009930 tx=0x7efde802a440 comp rx=0 tx=0).stop 2024-04-24T06:44:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.297+0000 7efdfd800640 1 -- 172.21.15.5:0/654089929 shutdown_connections 2024-04-24T06:44:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.297+0000 7efdfd800640 1 --2- 172.21.15.5:0/654089929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdf810d8a0 0x7efdf8115420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.298+0000 7efdfd800640 1 --2- 172.21.15.5:0/654089929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdf8109160 0x7efdf810d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.298+0000 7efdfd800640 1 --2- 172.21.15.5:0/654089929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdf81087b0 0x7efdf8108b90 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.298+0000 7efdfd800640 1 -- 172.21.15.5:0/654089929 >> 172.21.15.5:0/654089929 conn(0x7efdf8077dd0 msgr2=0x7efdf8106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.298+0000 7efdfd800640 1 -- 172.21.15.5:0/654089929 shutdown_connections 2024-04-24T06:44:11.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.298+0000 7efdfd800640 1 -- 172.21.15.5:0/654089929 wait complete. 2024-04-24T06:44:11.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.299+0000 7efdfd800640 1 Processor -- start 2024-04-24T06:44:11.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.299+0000 7efdfd800640 1 -- start start 2024-04-24T06:44:11.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.299+0000 7efdfd800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdf81087b0 0x7efdf819edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:11.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.299+0000 7efdfd800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdf810d8a0 0x7efdf819f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:11.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.299+0000 7efdfd800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdf81a3680 0x7efdf81a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:11.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.299+0000 7efdfd800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efdf81188c0 con 0x7efdf81087b0 2024-04-24T06:44:11.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.299+0000 7efdfd800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efdf8118740 con 0x7efdf81a3680 2024-04-24T06:44:11.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.299+0000 7efdfd800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efdf8118a40 con 0x7efdf810d8a0 2024-04-24T06:44:11.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.299+0000 7efdf7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdf81087b0 0x7efdf819edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:11.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.299+0000 7efdf7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdf81087b0 0x7efdf819edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48814/0 (socket says 172.21.15.5:48814) 2024-04-24T06:44:11.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.299+0000 7efdf7e00640 1 -- 172.21.15.5:0/1602288035 learned_addr learned my addr 172.21.15.5:0/1602288035 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:11.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.299+0000 7efdfce00640 1 --2- 172.21.15.5:0/1602288035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdf81a3680 0x7efdf81a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:11.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.300+0000 7efdf7400640 1 --2- 172.21.15.5:0/1602288035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdf810d8a0 0x7efdf819f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:11.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.300+0000 7efdf7e00640 1 -- 172.21.15.5:0/1602288035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdf810d8a0 msgr2=0x7efdf819f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:11.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.300+0000 7efdf7e00640 1 --2- 172.21.15.5:0/1602288035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdf810d8a0 0x7efdf819f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.300+0000 7efdf7e00640 1 -- 172.21.15.5:0/1602288035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdf81a3680 msgr2=0x7efdf81a3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:11.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.300+0000 7efdf7e00640 1 --2- 172.21.15.5:0/1602288035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdf81a3680 0x7efdf81a3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.300+0000 7efdf7e00640 1 -- 172.21.15.5:0/1602288035 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf81a4170 con 0x7efdf81087b0 2024-04-24T06:44:11.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.300+0000 7efdfce00640 1 --2- 172.21.15.5:0/1602288035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdf81a3680 0x7efdf81a3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:44:11.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.300+0000 7efdf7400640 1 --2- 172.21.15.5:0/1602288035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdf810d8a0 0x7efdf819f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:44:11.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.300+0000 7efdf7e00640 1 --2- 172.21.15.5:0/1602288035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdf81087b0 0x7efdf819edc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efde8005ab0 tx=0x7efde8032d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:11.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.301+0000 7efdf4c00640 1 -- 172.21.15.5:0/1602288035 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde803f030 con 0x7efdf81087b0 2024-04-24T06:44:11.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.301+0000 7efdf4c00640 1 -- 172.21.15.5:0/1602288035 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efde8003ce0 con 0x7efdf81087b0 2024-04-24T06:44:11.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.301+0000 7efdf4c00640 1 -- 172.21.15.5:0/1602288035 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde8035070 con 0x7efdf81087b0 2024-04-24T06:44:11.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.301+0000 7efdfd800640 1 -- 172.21.15.5:0/1602288035 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdf81a4400 con 0x7efdf81087b0 2024-04-24T06:44:11.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.301+0000 7efdfd800640 1 -- 172.21.15.5:0/1602288035 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdf81abbc0 con 0x7efdf81087b0 2024-04-24T06:44:11.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.302+0000 7efdfd800640 1 -- 172.21.15.5:0/1602288035 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdf804fe00 con 0x7efdf81087b0 2024-04-24T06:44:11.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.304+0000 7efdf4c00640 1 -- 172.21.15.5:0/1602288035 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 138872+0+0 (secure 0 0 0) 0x7efde8042030 con 0x7efdf81087b0 2024-04-24T06:44:11.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.305+0000 7efdf4c00640 1 --2- 172.21.15.5:0/1602288035 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7efdd80a3e30 0x7efdd80a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:11.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.305+0000 7efdf4c00640 1 -- 172.21.15.5:0/1602288035 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7efde80f0f10 con 0x7efdf81087b0 2024-04-24T06:44:11.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.306+0000 7efdf7400640 1 --2- 172.21.15.5:0/1602288035 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7efdd80a3e30 0x7efdd80a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:11.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.306+0000 7efdf7400640 1 --2- 172.21.15.5:0/1602288035 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7efdd80a3e30 0x7efdd80a62f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efdf810eac0 tx=0x7efde0009520 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:11.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.310+0000 7efdf4c00640 1 -- 172.21.15.5:0/1602288035 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efde80bd3e0 con 0x7efdf81087b0 2024-04-24T06:44:11.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:11 smithi005 ceph-mon[27798]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.0 KiB/s wr, 22 op/s 2024-04-24T06:44:11.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:11 smithi005 ceph-mon[27798]: Updating smithi005:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T06:44:11.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:11 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1017145709' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T06:44:11.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:11 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:11.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:11 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:11.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:11 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:11.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:11 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:44:11.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:11 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/3696510880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:44:11.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.520+0000 7efdfd800640 1 -- 172.21.15.5:0/1602288035 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efdf8109160 con 0x7efdf81087b0 2024-04-24T06:44:11.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.524+0000 7efdf4c00640 1 -- 172.21.15.5:0/1602288035 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22884 (secure 0 0 0) 0x7efde80bcce0 con 0x7efdf81087b0 2024-04-24T06:44:11.526 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:44:11.526 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":89,"fsid":"43e9bad0-0204-11ef-bc93-c7b262605968","created":"2024-04-24T06:32:26.340555+0000","modified":"2024-04-24T06:44:01.965991+0000","last_up_change":"2024-04-24T06:40:36.790934+0000","last_in_change":"2024-04-24T06:40:17.803257+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:36:27.320342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:41:32.254502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:41:33.739067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c2e974c-745c-4dd5-8c3d-8e8da2eb1ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6803","nonce":2155858515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6805","nonce":2155858515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6809","nonce":2155858515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6807","nonce":2155858515}]},"public_addr":"172.21.15.5:6803/2155858515","cluster_addr":"172.21.15.5:6805/2155858515","heartbeat_back_addr":"172.21.15.5:6809/2155858515","heartbeat_front_addr":"172.21.15.5:6807/2155858515","state":["exists","up"]},{"osd":1,"uuid":"50467744-db20-4721-bf2d-1c5db938297b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6801","nonce":1302362650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6803","nonce":1302362650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6807","nonce":1302362650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6805","nonce":1302362650}]},"public_addr":"172.21.15.37:6801/1302362650","cluster_addr":"172.21.15.37:6803/1302362650","heartbeat_back_addr":"172.21.15.37:6807/1302362650","heartbeat_front_addr":"172.21.15.37:6805/1302362650","state":["exists","up"]},{"osd":2,"uuid":"0e190b15-d04c-4288-8d48-ecf6664392fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6801","nonce":2885027256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6803","nonce":2885027256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6807","nonce":2885027256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6805","nonce":2885027256}]},"public_addr":"172.21.15.137:6801/2885027256","cluster_addr":"172.21.15.137:6803/2885027256","heartbeat_back_addr":"172.21.15.137:6807/2885027256","heartbeat_front_addr":"172.21.15.137:6805/2885027256","state":["exists","up"]},{"osd":3,"uuid":"e05592ba-d96a-472a-9f63-09a535d04dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6811","nonce":3701747260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6813","nonce":3701747260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6817","nonce":3701747260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6815","nonce":3701747260}]},"public_addr":"172.21.15.5:6811/3701747260","cluster_addr":"172.21.15.5:6813/3701747260","heartbeat_back_addr":"172.21.15.5:6817/3701747260","heartbeat_front_addr":"172.21.15.5:6815/3701747260","state":["exists","up"]},{"osd":4,"uuid":"c3f36e51-7a3b-4dd9-af4b-a10558830c4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6809","nonce":759253844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6811","nonce":759253844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6815","nonce":759253844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6813","nonce":759253844}]},"public_addr":"172.21.15.37:6809/759253844","cluster_addr":"172.21.15.37:6811/759253844","heartbeat_back_addr":"172.21.15.37:6815/759253844","heartbeat_front_addr":"172.21.15.37:6813/759253844","state":["exists","up"]},{"osd":5,"uuid":"97a0500c-cbb8-4ff5-b476-8634b26b3821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6809","nonce":3771921560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6811","nonce":3771921560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6815","nonce":3771921560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6813","nonce":3771921560}]},"public_addr":"172.21.15.137:6809/3771921560","cluster_addr":"172.21.15.137:6811/3771921560","heartbeat_back_addr":"172.21.15.137:6815/3771921560","heartbeat_front_addr":"172.21.15.137:6813/3771921560","state":["exists","up"]},{"osd":6,"uuid":"33a33d62-bbb3-4ed9-834e-19ae93a97670","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6819","nonce":2064636084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6821","nonce":2064636084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6825","nonce":2064636084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6823","nonce":2064636084}]},"public_addr":"172.21.15.5:6819/2064636084","cluster_addr":"172.21.15.5:6821/2064636084","heartbeat_back_addr":"172.21.15.5:6825/2064636084","heartbeat_front_addr":"172.21.15.5:6823/2064636084","state":["exists","up"]},{"osd":7,"uuid":"c0f4e036-4717-4514-bb59-ab11fbaab063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6817","nonce":2952750659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6819","nonce":2952750659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6823","nonce":2952750659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6821","nonce":2952750659}]},"public_addr":"172.21.15.37:6817/2952750659","cluster_addr":"172.21.15.37:6819/2952750659","heartbeat_back_addr":"172.21.15.37:6823/2952750659","heartbeat_front_addr":"172.21.15.37:6821/2952750659","state":["exists","up"]},{"osd":8,"uuid":"5391fd87-44ea-440f-80da-c54d0327c04c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6817","nonce":29724945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6819","nonce":29724945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6823","nonce":29724945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6821","nonce":29724945}]},"public_addr":"172.21.15.137:6817/29724945","cluster_addr":"172.21.15.137:6819/29724945","heartbeat_back_addr":"172.21.15.137:6823/29724945","heartbeat_front_addr":"172.21.15.137:6821/29724945","state":["exists","up"]},{"osd":9,"uuid":"d94c9dbc-f43d-4237-ac7f-47a6a5ede1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6827","nonce":2257102107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6829","nonce":2257102107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6833","nonce":2257102107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6831","nonce":2257102107}]},"public_addr":"172.21.15.5:6827/2257102107","cluster_addr":"172.21.15.5:6829/2257102107","heartbeat_back_addr":"172.21.15.5:6833/2257102107","heartbeat_front_addr":"172.21.15.5:6831/2257102107","state":["exists","up"]},{"osd":10,"uuid":"e2455332-2a5e-44b1-85d3-8e47da399f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6825","nonce":2451570637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6827","nonce":2451570637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6831","nonce":2451570637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6829","nonce":2451570637}]},"public_addr":"172.21.15.37:6825/2451570637","cluster_addr":"172.21.15.37:6827/2451570637","heartbeat_back_addr":"172.21.15.37:6831/2451570637","heartbeat_front_addr":"172.21.15.37:6829/2451570637","state":["exists","up"]},{"osd":11,"uuid":"49d9396b-704a-4cef-8210-75d30c28e924","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6825","nonce":90600596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6827","nonce":90600596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6831","nonce":90600596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6829","nonce":90600596}]},"public_addr":"172.21.15.137:6825/90600596","cluster_addr":"172.21.15.137:6827/90600596","heartbeat_back_addr":"172.21.15.137:6831/90600596","heartbeat_front_addr":"172.21.15.137:6829/90600596","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:35:35.591619+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:00.568817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:24.195643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:56.451724+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:22.109293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:46.161749+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:19.278305+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:45.620256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:10.041165+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:43.813697+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:10.736026+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:35.384377+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6800/4042936965":"2024-04-25T06:44:01.965881+0000","172.21.15.5:0/2912680052":"2024-04-25T06:44:01.965881+0000","172.21.15.5:6801/4042936965":"2024-04-25T06:44:01.965881+0000","172.21.15.5:0/3478347847":"2024-04-25T06:44:01.965881+0000","172.21.15.5:0/23307164":"2024-04-25T06:33:41.161852+0000","172.21.15.5:0/1012004002":"2024-04-25T06:44:01.965881+0000","172.21.15.5:6801/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:0/230785941":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/805658012":"2024-04-25T06:44:01.965881+0000","172.21.15.5:0/2940023597":"2024-04-25T06:33:07.040369+0000","172.21.15.5:6801/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3886613904":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/4199165437":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3459593499":"2024-04-25T06:33:41.161852+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:44:11.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.527+0000 7efdfd800640 1 -- 172.21.15.5:0/1602288035 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7efdd80a3e30 msgr2=0x7efdd80a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:11.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.527+0000 7efdfd800640 1 --2- 172.21.15.5:0/1602288035 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7efdd80a3e30 0x7efdd80a62f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efdf810eac0 tx=0x7efde0009520 comp rx=0 tx=0).stop 2024-04-24T06:44:11.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.527+0000 7efdfd800640 1 -- 172.21.15.5:0/1602288035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdf81087b0 msgr2=0x7efdf819edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:11.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.527+0000 7efdfd800640 1 --2- 172.21.15.5:0/1602288035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdf81087b0 0x7efdf819edc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efde8005ab0 tx=0x7efde8032d90 comp rx=0 tx=0).stop 2024-04-24T06:44:11.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.527+0000 7efdfd800640 1 -- 172.21.15.5:0/1602288035 shutdown_connections 2024-04-24T06:44:11.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.527+0000 7efdfd800640 1 --2- 172.21.15.5:0/1602288035 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7efdd80a3e30 0x7efdd80a62f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.527+0000 7efdfd800640 1 --2- 172.21.15.5:0/1602288035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdf81a3680 0x7efdf81a3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.527+0000 7efdfd800640 1 --2- 172.21.15.5:0/1602288035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdf810d8a0 0x7efdf819f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.527+0000 7efdfd800640 1 --2- 172.21.15.5:0/1602288035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdf81087b0 0x7efdf819edc0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.527+0000 7efdfd800640 1 -- 172.21.15.5:0/1602288035 >> 172.21.15.5:0/1602288035 conn(0x7efdf8077dd0 msgr2=0x7efdf8079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:11.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.528+0000 7efdfd800640 1 -- 172.21.15.5:0/1602288035 shutdown_connections 2024-04-24T06:44:11.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.528+0000 7efdfd800640 1 -- 172.21.15.5:0/1602288035 wait complete. 2024-04-24T06:44:11.549 DEBUG:tasks.fwd_scrub:Yielding 2024-04-24T06:44:11.549 INFO:teuthology.run_tasks:Running task exec... 2024-04-24T06:44:11.558 INFO:teuthology.task.exec:Executing custom commands... 2024-04-24T06:44:11.558 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-04-24T06:44:11.558 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-24T06:44:11.558 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-04-24T06:44:11.559 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T06:44:11.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:44:11.559 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T06:44:11.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:11 smithi137 ceph-mon[33017]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.0 KiB/s wr, 22 op/s 2024-04-24T06:44:11.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:11 smithi137 ceph-mon[33017]: Updating smithi005:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T06:44:11.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:11 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/1017145709' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T06:44:11.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:11 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:11.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:11 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:11.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:11 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:11.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:11 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:44:11.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:11 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/3696510880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:44:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.751+0000 7f0436000640 1 -- 172.21.15.5:0/661373854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0430102a40 msgr2=0x7f0430106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.751+0000 7f0436000640 1 --2- 172.21.15.5:0/661373854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0430102a40 0x7f0430106a80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f041c009900 tx=0x7f041c02a410 comp rx=0 tx=0).stop 2024-04-24T06:44:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.751+0000 7f0436000640 1 -- 172.21.15.5:0/661373854 shutdown_connections 2024-04-24T06:44:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.751+0000 7f0436000640 1 --2- 172.21.15.5:0/661373854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0430107160 0x7f043010ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.751+0000 7f0436000640 1 --2- 172.21.15.5:0/661373854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0430102a40 0x7f0430106a80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.751+0000 7f0436000640 1 --2- 172.21.15.5:0/661373854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0430102090 0x7f0430102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.751+0000 7f0436000640 1 -- 172.21.15.5:0/661373854 >> 172.21.15.5:0/661373854 conn(0x7f04300fd910 msgr2=0x7f04300ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.751+0000 7f0436000640 1 -- 172.21.15.5:0/661373854 shutdown_connections 2024-04-24T06:44:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.751+0000 7f0436000640 1 -- 172.21.15.5:0/661373854 wait complete. 2024-04-24T06:44:11.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.752+0000 7f0436000640 1 Processor -- start 2024-04-24T06:44:11.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.753+0000 7f0436000640 1 -- start start 2024-04-24T06:44:11.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.753+0000 7f0436000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0430102090 0x7f04301a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:11.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.753+0000 7f0436000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0430107160 0x7f04301a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:11.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.753+0000 7f0436000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f043019a820 0x7f043019acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:11.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.753+0000 7f0436000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0430112240 con 0x7f043019a820 2024-04-24T06:44:11.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.753+0000 7f0436000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f04301120c0 con 0x7f0430107160 2024-04-24T06:44:11.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.753+0000 7f0436000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f04301123c0 con 0x7f0430102090 2024-04-24T06:44:11.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.753+0000 7f0435600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f043019a820 0x7f043019acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:11.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.753+0000 7f0435600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f043019a820 0x7f043019acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48840/0 (socket says 172.21.15.5:48840) 2024-04-24T06:44:11.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.753+0000 7f0435600640 1 -- 172.21.15.5:0/1201414261 learned_addr learned my addr 172.21.15.5:0/1201414261 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:11.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.753+0000 7f042fe00640 1 --2- 172.21.15.5:0/1201414261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0430107160 0x7f04301a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:11.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.753+0000 7f0434c00640 1 --2- 172.21.15.5:0/1201414261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0430102090 0x7f04301a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:11.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.754+0000 7f0435600640 1 -- 172.21.15.5:0/1201414261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0430102090 msgr2=0x7f04301a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:11.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.754+0000 7f0435600640 1 --2- 172.21.15.5:0/1201414261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0430102090 0x7f04301a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.754+0000 7f0435600640 1 -- 172.21.15.5:0/1201414261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0430107160 msgr2=0x7f04301a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:11.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.754+0000 7f0435600640 1 --2- 172.21.15.5:0/1201414261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0430107160 0x7f04301a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.754+0000 7f0435600640 1 -- 172.21.15.5:0/1201414261 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f043019b3b0 con 0x7f043019a820 2024-04-24T06:44:11.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.754+0000 7f0435600640 1 --2- 172.21.15.5:0/1201414261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f043019a820 0x7f043019acd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f042800eb10 tx=0x7f042800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:11.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.754+0000 7f042d600640 1 -- 172.21.15.5:0/1201414261 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0428004b20 con 0x7f043019a820 2024-04-24T06:44:11.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.754+0000 7f0436000640 1 -- 172.21.15.5:0/1201414261 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f043019b6a0 con 0x7f043019a820 2024-04-24T06:44:11.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.754+0000 7f042d600640 1 -- 172.21.15.5:0/1201414261 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0428004cc0 con 0x7f043019a820 2024-04-24T06:44:11.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.755+0000 7f042d600640 1 -- 172.21.15.5:0/1201414261 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0428010760 con 0x7f043019a820 2024-04-24T06:44:11.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.755+0000 7f0436000640 1 -- 172.21.15.5:0/1201414261 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04301a7800 con 0x7f043019a820 2024-04-24T06:44:11.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.756+0000 7f0436000640 1 -- 172.21.15.5:0/1201414261 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0430103560 con 0x7f043019a820 2024-04-24T06:44:11.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.758+0000 7f042d600640 1 -- 172.21.15.5:0/1201414261 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 138872+0+0 (secure 0 0 0) 0x7f0428005020 con 0x7f043019a820 2024-04-24T06:44:11.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.759+0000 7f042d600640 1 --2- 172.21.15.5:0/1201414261 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f04180a3e20 0x7f04180a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:11.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.759+0000 7f042d600640 1 -- 172.21.15.5:0/1201414261 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f04280d1070 con 0x7f043019a820 2024-04-24T06:44:11.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.760+0000 7f0434c00640 1 --2- 172.21.15.5:0/1201414261 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f04180a3e20 0x7f04180a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:11.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.760+0000 7f0434c00640 1 --2- 172.21.15.5:0/1201414261 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f04180a3e20 0x7f04180a62e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0424009b60 tx=0x7f0424005f90 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:11.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.764+0000 7f042d600640 1 -- 172.21.15.5:0/1201414261 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f043019b6a0 con 0x7f043019a820 2024-04-24T06:44:11.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:11 smithi037 ceph-mon[32864]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.0 KiB/s wr, 22 op/s 2024-04-24T06:44:11.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:11 smithi037 ceph-mon[32864]: Updating smithi005:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T06:44:11.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:11 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/1017145709' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T06:44:11.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:11 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:11.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:11 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:11.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:11 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:11.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:11 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:44:11.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:11 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/3696510880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:44:11.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.768+0000 7fea6a400640 1 -- 172.21.15.5:0/3102906862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea64104e00 msgr2=0x7fea64108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:11.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.768+0000 7fea6a400640 1 --2- 172.21.15.5:0/3102906862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea64104e00 0x7fea64108e40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fea50009900 tx=0x7fea5002a410 comp rx=0 tx=0).stop 2024-04-24T06:44:11.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.768+0000 7fea6a400640 1 -- 172.21.15.5:0/3102906862 shutdown_connections 2024-04-24T06:44:11.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.768+0000 7fea6a400640 1 --2- 172.21.15.5:0/3102906862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea64109520 0x7fea641110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.768+0000 7fea6a400640 1 --2- 172.21.15.5:0/3102906862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea64104e00 0x7fea64108e40 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.768+0000 7fea6a400640 1 --2- 172.21.15.5:0/3102906862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea64104450 0x7fea64104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.768+0000 7fea6a400640 1 -- 172.21.15.5:0/3102906862 >> 172.21.15.5:0/3102906862 conn(0x7fea640ffc90 msgr2=0x7fea641020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:11.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.768+0000 7fea6a400640 1 -- 172.21.15.5:0/3102906862 shutdown_connections 2024-04-24T06:44:11.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.768+0000 7fea6a400640 1 -- 172.21.15.5:0/3102906862 wait complete. 2024-04-24T06:44:11.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.769+0000 7fea6a400640 1 Processor -- start 2024-04-24T06:44:11.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.769+0000 7fea6a400640 1 -- start start 2024-04-24T06:44:11.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.769+0000 7fea6a400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea64104450 0x7fea641a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:11.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.769+0000 7fea6a400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea64109520 0x7fea641a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:11.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.769+0000 7fea6a400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea6419a980 0x7fea6419ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:11.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.769+0000 7fea6a400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fea641145e0 con 0x7fea64109520 2024-04-24T06:44:11.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.769+0000 7fea6a400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fea64114460 con 0x7fea6419a980 2024-04-24T06:44:11.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.769+0000 7fea6a400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fea64114760 con 0x7fea64104450 2024-04-24T06:44:11.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.770+0000 7fea62a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea64109520 0x7fea641a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:11.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.770+0000 7fea62a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea64109520 0x7fea641a0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48858/0 (socket says 172.21.15.5:48858) 2024-04-24T06:44:11.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.770+0000 7fea62a00640 1 -- 172.21.15.5:0/4157256777 learned_addr learned my addr 172.21.15.5:0/4157256777 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:11.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.770+0000 7fea63400640 1 --2- 172.21.15.5:0/4157256777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea64104450 0x7fea641a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:11.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.770+0000 7fea63e00640 1 --2- 172.21.15.5:0/4157256777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea6419a980 0x7fea6419ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:11.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.770+0000 7fea62a00640 1 -- 172.21.15.5:0/4157256777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea64104450 msgr2=0x7fea641a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:11.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.770+0000 7fea62a00640 1 --2- 172.21.15.5:0/4157256777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea64104450 0x7fea641a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.770+0000 7fea62a00640 1 -- 172.21.15.5:0/4157256777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea6419a980 msgr2=0x7fea6419ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:11.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.770+0000 7fea62a00640 1 --2- 172.21.15.5:0/4157256777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea6419a980 0x7fea6419ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.770+0000 7fea62a00640 1 -- 172.21.15.5:0/4157256777 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea6419b4a0 con 0x7fea64109520 2024-04-24T06:44:11.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.771+0000 7fea62a00640 1 --2- 172.21.15.5:0/4157256777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea64109520 0x7fea641a0cc0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fea5002a930 tx=0x7fea50003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:11.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.771+0000 7fea5be00640 1 -- 172.21.15.5:0/4157256777 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5003f070 con 0x7fea64109520 2024-04-24T06:44:11.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.771+0000 7fea6a400640 1 -- 172.21.15.5:0/4157256777 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea6419b730 con 0x7fea64109520 2024-04-24T06:44:11.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.771+0000 7fea5be00640 1 -- 172.21.15.5:0/4157256777 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea50031c90 con 0x7fea64109520 2024-04-24T06:44:11.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.771+0000 7fea5be00640 1 -- 172.21.15.5:0/4157256777 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50035030 con 0x7fea64109520 2024-04-24T06:44:11.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.771+0000 7fea6a400640 1 -- 172.21.15.5:0/4157256777 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea641a78d0 con 0x7fea64109520 2024-04-24T06:44:11.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.775+0000 7fea6d1d2740 1 Processor -- start 2024-04-24T06:44:11.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.776+0000 7fea6d1d2740 1 -- start start 2024-04-24T06:44:11.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.776+0000 7fea6d1d2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559b1c9b090 0x5559b1c9b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:11.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.776+0000 7fea6d1d2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559b1c9b9b0 0x5559b1ca5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:11.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.776+0000 7fea6d1d2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559b1ca6080 0x5559b1ca8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:11.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.776+0000 7fea6d1d2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5559b1c00e70 con 0x5559b1c9b9b0 2024-04-24T06:44:11.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.776+0000 7fea6d1d2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5559b1ba7380 con 0x5559b1ca6080 2024-04-24T06:44:11.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.776+0000 7fea6d1d2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5559b1bfce30 con 0x5559b1c9b090 2024-04-24T06:44:11.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.777+0000 7fea63400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559b1c9b9b0 0x5559b1ca5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:11.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.777+0000 7fea63e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559b1c9b090 0x5559b1c9b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:11.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.777+0000 7fea63e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559b1c9b090 0x5559b1c9b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:54466/0 (socket says 172.21.15.5:54466) 2024-04-24T06:44:11.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.777+0000 7fea63e00640 1 -- 172.21.15.5:0/3373432573 learned_addr learned my addr 172.21.15.5:0/3373432573 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:11.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.777+0000 7fea5be00640 1 -- 172.21.15.5:0/4157256777 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 138872+0+0 (secure 0 0 0) 0x7fea50042070 con 0x7fea64109520 2024-04-24T06:44:11.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.777+0000 7fea63e00640 1 -- 172.21.15.5:0/3373432573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559b1ca6080 msgr2=0x5559b1ca8470 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T06:44:11.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.777+0000 7fea63e00640 1 --2- 172.21.15.5:0/3373432573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559b1ca6080 0x5559b1ca8470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.777+0000 7fea63e00640 1 -- 172.21.15.5:0/3373432573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559b1c9b9b0 msgr2=0x5559b1ca5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:11.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.777+0000 7fea63e00640 1 --2- 172.21.15.5:0/3373432573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559b1c9b9b0 0x5559b1ca5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.777+0000 7fea63e00640 1 -- 172.21.15.5:0/3373432573 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559b19db7e0 con 0x5559b1c9b090 2024-04-24T06:44:11.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.777+0000 7fea63400640 1 --2- 172.21.15.5:0/3373432573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559b1c9b9b0 0x5559b1ca5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:44:11.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.777+0000 7fea63e00640 1 --2- 172.21.15.5:0/3373432573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559b1c9b090 0x5559b1c9b470 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fea5c009f90 tx=0x7fea5c010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89e7c94cf904f042 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.778+0000 7fea58c00640 1 -- 172.21.15.5:0/3373432573 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c011700 con 0x5559b1c9b090 2024-04-24T06:44:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.778+0000 7fea58c00640 1 -- 172.21.15.5:0/3373432573 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea5c011d80 con 0x5559b1c9b090 2024-04-24T06:44:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.778+0000 7fea58c00640 1 -- 172.21.15.5:0/3373432573 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c018380 con 0x5559b1c9b090 2024-04-24T06:44:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.778+0000 7fea5be00640 1 --2- 172.21.15.5:0/4157256777 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fea480a3e40 0x7fea480a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.779+0000 7fea5be00640 1 -- 172.21.15.5:0/4157256777 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fea500f0d30 con 0x7fea64109520 2024-04-24T06:44:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.779+0000 7fea6d1d2740 1 -- 172.21.15.5:0/3373432573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559b1c9b090 msgr2=0x5559b1c9b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.779+0000 7fea6d1d2740 1 --2- 172.21.15.5:0/3373432573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559b1c9b090 0x5559b1c9b470 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fea5c009f90 tx=0x7fea5c010860 comp rx=0 tx=0).stop 2024-04-24T06:44:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.779+0000 7fea63400640 1 --2- 172.21.15.5:0/4157256777 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fea480a3e40 0x7fea480a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.779+0000 7fea6d1d2740 1 -- 172.21.15.5:0/3373432573 shutdown_connections 2024-04-24T06:44:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.779+0000 7fea6d1d2740 1 --2- 172.21.15.5:0/3373432573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559b1ca6080 0x5559b1ca8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.779+0000 7fea6d1d2740 1 --2- 172.21.15.5:0/3373432573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559b1c9b9b0 0x5559b1ca5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.779+0000 7fea6d1d2740 1 --2- 172.21.15.5:0/3373432573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559b1c9b090 0x5559b1c9b470 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.779+0000 7fea6d1d2740 1 -- 172.21.15.5:0/3373432573 >> 172.21.15.5:0/3373432573 conn(0x5559b1c973b0 msgr2=0x5559b1c97790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:11.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.779+0000 7fea6d1d2740 1 -- 172.21.15.5:0/3373432573 shutdown_connections 2024-04-24T06:44:11.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.779+0000 7fea6d1d2740 1 -- 172.21.15.5:0/3373432573 wait complete. 2024-04-24T06:44:11.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.780+0000 7fea63400640 1 --2- 172.21.15.5:0/4157256777 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fea480a3e40 0x7fea480a6300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x5559b1c9c650 tx=0x7fea54005f90 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:11.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.780+0000 7fea6d1d2740 1 -- start start 2024-04-24T06:44:11.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.780+0000 7fea6d1d2740 1 Processor -- start 2024-04-24T06:44:11.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.780+0000 7fea6d1d2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559b1c9b090 0x5559b1ca3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:11.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.780+0000 7fea6d1d2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559b1c9b9b0 0x5559b1ca35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:11.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.780+0000 7fea6d1d2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559b1ca6080 0x5559b1ca3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:11.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.780+0000 7fea6d1d2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5559b1ca8e70 con 0x5559b1c9b9b0 2024-04-24T06:44:11.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.780+0000 7fea6d1d2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5559b1bba660 con 0x5559b1c9b090 2024-04-24T06:44:11.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.780+0000 7fea6d1d2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5559b1ca8ff0 con 0x5559b1ca6080 2024-04-24T06:44:11.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.780+0000 7fea6d1d2740 10 client.0 _ll_register_callbacks cb 0x5559b1ba8aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:44:11.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.781+0000 7fea63400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559b1c9b090 0x5559b1ca3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:11.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.781+0000 7fea63400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559b1c9b090 0x5559b1ca3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:46274/0 (socket says 172.21.15.5:46274) 2024-04-24T06:44:11.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.781+0000 7fea63400640 1 -- 172.21.15.5:0/1912989430 learned_addr learned my addr 172.21.15.5:0/1912989430 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:11.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.781+0000 7fea63e00640 1 --2- 172.21.15.5:0/1912989430 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559b1ca6080 0x5559b1ca3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:11.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.781+0000 7fea62a00640 1 --2- 172.21.15.5:0/1912989430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559b1c9b9b0 0x5559b1ca35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:11.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.781+0000 7fea63400640 1 -- 172.21.15.5:0/1912989430 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559b1ca6080 msgr2=0x5559b1ca3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:11.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.781+0000 7fea63400640 1 --2- 172.21.15.5:0/1912989430 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559b1ca6080 0x5559b1ca3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.781+0000 7fea63400640 1 -- 172.21.15.5:0/1912989430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559b1c9b9b0 msgr2=0x5559b1ca35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:11.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.781+0000 7fea63400640 1 --2- 172.21.15.5:0/1912989430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559b1c9b9b0 0x5559b1ca35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.781+0000 7fea63400640 1 -- 172.21.15.5:0/1912989430 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559b1a3bf70 con 0x5559b1c9b090 2024-04-24T06:44:11.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.781+0000 7fea62a00640 1 --2- 172.21.15.5:0/1912989430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559b1c9b9b0 0x5559b1ca35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:44:11.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.782+0000 7fea63400640 1 --2- 172.21.15.5:0/1912989430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559b1c9b090 0x5559b1ca3070 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fea54004630 tx=0x7fea54004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20dd402b37cb3270 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:11.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.782+0000 7fea4d600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:44:11.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.782+0000 7fea4d600640 1 -- 172.21.15.5:0/1912989430 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5401f630 con 0x5559b1c9b090 2024-04-24T06:44:11.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.782+0000 7fea4d600640 1 -- 172.21.15.5:0/1912989430 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fea5401f7d0 con 0x5559b1c9b090 2024-04-24T06:44:11.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.782+0000 7fea6d1d2740 1 -- 172.21.15.5:0/1912989430 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea5c011d80 con 0x5559b1c9b090 2024-04-24T06:44:11.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.782+0000 7fea4d600640 1 -- 172.21.15.5:0/1912989430 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fea5401f9d0 con 0x5559b1c9b090 2024-04-24T06:44:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.782+0000 7fea6d1d2740 10 client.25397 fetch_fsmap learned FSMap version 23 2024-04-24T06:44:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.782+0000 7fea6d1d2740 1 -- 172.21.15.5:0/1912989430 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5559b18d3170 con 0x5559b1c9b090 2024-04-24T06:44:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.782+0000 7fea4d600640 1 -- 172.21.15.5:0/1912989430 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea5c011d80 con 0x5559b1c9b090 2024-04-24T06:44:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.783+0000 7fea4d600640 1 -- 172.21.15.5:0/1912989430 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fea54027070 con 0x5559b1c9b090 2024-04-24T06:44:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.783+0000 7fea6d1d2740 10 client.25397 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T06:44:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.783+0000 7fea6d1d2740 10 client.25397 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:44:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.783+0000 7fea6d1d2740 20 client.25397 populate_metadata read hostname 'smithi005' 2024-04-24T06:44:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.783+0000 7fea6d1d2740 1 --2- 172.21.15.5:0/1912989430 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5559b1cb64c0 0x5559b1cb8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.783+0000 7fea6d1d2740 4 client.25397 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:44:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.783+0000 7fea6d1d2740 1 -- 172.21.15.5:0/1912989430 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559b1c0e890 con 0x5559b1cb64c0 2024-04-24T06:44:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.783+0000 7fea62a00640 1 --2- 172.21.15.5:0/1912989430 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5559b1cb64c0 0x5559b1cb8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.784+0000 7fea62a00640 1 --2- 172.21.15.5:0/1912989430 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5559b1cb64c0 0x5559b1cb8930 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=315daed7358d4515 server_cookie=9ab16aa000db1c78 in_seq=0 out_seq=0 2024-04-24T06:44:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.784+0000 7fea4d600640 10 client.25397 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:44:11.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.784+0000 7fea4d600640 1 -- 172.21.15.5:0/1912989430 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5559b1c0e890 con 0x5559b1cb64c0 2024-04-24T06:44:11.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.784+0000 7fea4d600640 10 client.25397 handle_command_reply: tid=0 2024-04-24T06:44:11.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.784+0000 7fea6d1d2740 1 client.25397 shutdown 2024-04-24T06:44:11.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.785+0000 7fea6d1d2740 1 -- 172.21.15.5:0/1912989430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559b1c9b090 msgr2=0x5559b1ca3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:11.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.785+0000 7fea6d1d2740 1 --2- 172.21.15.5:0/1912989430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559b1c9b090 0x5559b1ca3070 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fea54004630 tx=0x7fea54004730 comp rx=0 tx=0).stop 2024-04-24T06:44:11.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.785+0000 7fea6d1d2740 1 -- 172.21.15.5:0/1912989430 shutdown_connections 2024-04-24T06:44:11.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.785+0000 7fea6d1d2740 1 --2- 172.21.15.5:0/1912989430 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5559b1cb64c0 0x5559b1cb8930 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.785+0000 7fea6d1d2740 1 --2- 172.21.15.5:0/1912989430 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559b1ca6080 0x5559b1ca3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.785+0000 7fea4d600640 0 client.25397 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:44:11.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.785+0000 7fea6d1d2740 1 --2- 172.21.15.5:0/1912989430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559b1c9b9b0 0x5559b1ca35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.785+0000 7fea6d1d2740 1 --2- 172.21.15.5:0/1912989430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559b1c9b090 0x5559b1ca3070 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.785+0000 7fea6d1d2740 1 -- 172.21.15.5:0/1912989430 >> 172.21.15.5:0/1912989430 conn(0x5559b1c973b0 msgr2=0x5559b1c9d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:11.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.785+0000 7fea6d1d2740 1 -- 172.21.15.5:0/1912989430 shutdown_connections 2024-04-24T06:44:11.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.785+0000 7fea6d1d2740 1 -- 172.21.15.5:0/1912989430 wait complete. 2024-04-24T06:44:11.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.785+0000 7fea6d1d2740 20 client.25397 trim_cache size 0 max 16384 2024-04-24T06:44:11.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.805+0000 7fea6d1d2740 1 Processor -- start 2024-04-24T06:44:11.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.805+0000 7fea6d1d2740 1 -- start start 2024-04-24T06:44:11.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.805+0000 7fea6d1d2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559b1c9b090 0x5559b1cb1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:11.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.805+0000 7fea6d1d2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559b1c9b9b0 0x5559b1c9d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:11.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.805+0000 7fea6d1d2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559b1ca6080 0x5559b1cbe200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:11.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.805+0000 7fea6d1d2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fea641b2880 con 0x5559b1c9b090 2024-04-24T06:44:11.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.805+0000 7fea6d1d2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fea641b0150 con 0x5559b1c9b9b0 2024-04-24T06:44:11.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.805+0000 7fea6d1d2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fea641adfa0 con 0x5559b1ca6080 2024-04-24T06:44:11.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.806+0000 7fea63400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559b1c9b090 0x5559b1cb1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:11.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.806+0000 7fea62a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559b1c9b9b0 0x5559b1c9d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:11.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.806+0000 7fea63400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559b1c9b090 0x5559b1cb1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48882/0 (socket says 172.21.15.5:48882) 2024-04-24T06:44:11.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.806+0000 7fea63400640 1 -- 172.21.15.5:0/69510729 learned_addr learned my addr 172.21.15.5:0/69510729 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:11.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.806+0000 7fea63e00640 1 --2- 172.21.15.5:0/69510729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559b1ca6080 0x5559b1cbe200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:11.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.806+0000 7fea62a00640 1 -- 172.21.15.5:0/69510729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559b1ca6080 msgr2=0x5559b1cbe200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:11.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.806+0000 7fea62a00640 1 --2- 172.21.15.5:0/69510729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559b1ca6080 0x5559b1cbe200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.806+0000 7fea62a00640 1 -- 172.21.15.5:0/69510729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559b1c9b090 msgr2=0x5559b1cb1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:11.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.806+0000 7fea62a00640 1 --2- 172.21.15.5:0/69510729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559b1c9b090 0x5559b1cb1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.806+0000 7fea62a00640 1 -- 172.21.15.5:0/69510729 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559b1c9e880 con 0x5559b1c9b9b0 2024-04-24T06:44:11.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.806+0000 7fea63400640 1 --2- 172.21.15.5:0/69510729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559b1c9b090 0x5559b1cb1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:44:11.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.806+0000 7fea62a00640 1 --2- 172.21.15.5:0/69510729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559b1c9b9b0 0x5559b1c9d900 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fea500ee2c0 tx=0x7fea50003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ed45939930ce73b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:11.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.807+0000 7fea4fe00640 1 -- 172.21.15.5:0/69510729 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50049040 con 0x5559b1c9b9b0 2024-04-24T06:44:11.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.807+0000 7fea4fe00640 1 -- 172.21.15.5:0/69510729 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea500baef0 con 0x5559b1c9b9b0 2024-04-24T06:44:11.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.807+0000 7fea4fe00640 1 -- 172.21.15.5:0/69510729 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea500b93f0 con 0x5559b1c9b9b0 2024-04-24T06:44:11.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.808+0000 7fea6d1d2740 1 -- 172.21.15.5:0/69510729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559b1c9b9b0 msgr2=0x5559b1c9d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:11.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.808+0000 7fea6d1d2740 1 --2- 172.21.15.5:0/69510729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559b1c9b9b0 0x5559b1c9d900 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fea500ee2c0 tx=0x7fea50003f90 comp rx=0 tx=0).stop 2024-04-24T06:44:11.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.808+0000 7fea6d1d2740 1 -- 172.21.15.5:0/69510729 shutdown_connections 2024-04-24T06:44:11.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.808+0000 7fea6d1d2740 1 --2- 172.21.15.5:0/69510729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559b1ca6080 0x5559b1cbe200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.808+0000 7fea6d1d2740 1 --2- 172.21.15.5:0/69510729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559b1c9b9b0 0x5559b1c9d900 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.808+0000 7fea6d1d2740 1 --2- 172.21.15.5:0/69510729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559b1c9b090 0x5559b1cb1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.808+0000 7fea6d1d2740 1 -- 172.21.15.5:0/69510729 >> 172.21.15.5:0/69510729 conn(0x5559b1c973b0 msgr2=0x5559b1c96510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:11.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.808+0000 7fea6d1d2740 1 -- 172.21.15.5:0/69510729 shutdown_connections 2024-04-24T06:44:11.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.808+0000 7fea6d1d2740 1 -- 172.21.15.5:0/69510729 wait complete. 2024-04-24T06:44:11.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.808+0000 7fea6d1d2740 1 -- start start 2024-04-24T06:44:11.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.809+0000 7fea6d1d2740 1 Processor -- start 2024-04-24T06:44:11.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.809+0000 7fea6d1d2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559b1c9b090 0x5559b1ca9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:11.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.809+0000 7fea6d1d2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559b1c9b9b0 0x5559b1caa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:11.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.809+0000 7fea6d1d2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559b1ca6080 0x5559b1caa830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:11.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.809+0000 7fea6d1d2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fea641ae610 con 0x5559b1ca6080 2024-04-24T06:44:11.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.809+0000 7fea6d1d2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fea641afa80 con 0x5559b1c9b9b0 2024-04-24T06:44:11.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.809+0000 7fea6d1d2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fea641aece0 con 0x5559b1c9b090 2024-04-24T06:44:11.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.809+0000 7fea6d1d2740 10 client.0 _ll_register_callbacks cb 0x7fea54012f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:44:11.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.809+0000 7fea63400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559b1c9b090 0x5559b1ca9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:11.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.809+0000 7fea63e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559b1ca6080 0x5559b1caa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:11.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.809+0000 7fea63400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559b1c9b090 0x5559b1ca9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:54488/0 (socket says 172.21.15.5:54488) 2024-04-24T06:44:11.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.809+0000 7fea63400640 1 -- 172.21.15.5:0/2716418256 learned_addr learned my addr 172.21.15.5:0/2716418256 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:11.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.809+0000 7fea62a00640 1 --2- 172.21.15.5:0/2716418256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559b1c9b9b0 0x5559b1caa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:11.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.810+0000 7fea62a00640 1 -- 172.21.15.5:0/2716418256 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559b1c9b090 msgr2=0x5559b1ca9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:11.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.810+0000 7fea62a00640 1 --2- 172.21.15.5:0/2716418256 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559b1c9b090 0x5559b1ca9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.810+0000 7fea62a00640 1 -- 172.21.15.5:0/2716418256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559b1ca6080 msgr2=0x5559b1caa830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:11.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.810+0000 7fea62a00640 1 --2- 172.21.15.5:0/2716418256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559b1ca6080 0x5559b1caa830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.810+0000 7fea62a00640 1 -- 172.21.15.5:0/2716418256 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559b1d1bbd0 con 0x5559b1c9b9b0 2024-04-24T06:44:11.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.810+0000 7fea63400640 1 --2- 172.21.15.5:0/2716418256 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559b1c9b090 0x5559b1ca9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:44:11.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.810+0000 7fea62a00640 1 --2- 172.21.15.5:0/2716418256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559b1c9b9b0 0x5559b1caa2f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fea500b95c0 tx=0x7fea500c1070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de1c0bf88e3f554c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:11.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.810+0000 7fea4d600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:44:11.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.810+0000 7fea4d600640 1 -- 172.21.15.5:0/2716418256 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50049070 con 0x5559b1c9b9b0 2024-04-24T06:44:11.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.810+0000 7fea4d600640 1 -- 172.21.15.5:0/2716418256 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fea5003b460 con 0x5559b1c9b9b0 2024-04-24T06:44:11.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.810+0000 7fea6d1d2740 1 -- 172.21.15.5:0/2716418256 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea500baef0 con 0x5559b1c9b9b0 2024-04-24T06:44:11.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.811+0000 7fea4d600640 1 -- 172.21.15.5:0/2716418256 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fea500b9260 con 0x5559b1c9b9b0 2024-04-24T06:44:11.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.811+0000 7fea6d1d2740 10 client.25403 fetch_fsmap learned FSMap version 23 2024-04-24T06:44:11.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.811+0000 7fea6d1d2740 1 -- 172.21.15.5:0/2716418256 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5559b1d1cef0 con 0x5559b1c9b9b0 2024-04-24T06:44:11.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.811+0000 7fea4d600640 1 -- 172.21.15.5:0/2716418256 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea500baef0 con 0x5559b1c9b9b0 2024-04-24T06:44:11.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.811+0000 7fea4d600640 1 -- 172.21.15.5:0/2716418256 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fea5004c060 con 0x5559b1c9b9b0 2024-04-24T06:44:11.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.811+0000 7fea6d1d2740 10 client.25403 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T06:44:11.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.811+0000 7fea6d1d2740 10 client.25403 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:44:11.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.811+0000 7fea6d1d2740 20 client.25403 populate_metadata read hostname 'smithi005' 2024-04-24T06:44:11.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.811+0000 7fea6d1d2740 1 --2- 172.21.15.5:0/2716418256 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5559b1d1d5a0 0x5559b1d1f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:11.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.811+0000 7fea6d1d2740 4 client.25403 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T06:44:11.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.811+0000 7fea6d1d2740 1 -- 172.21.15.5:0/2716418256 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5559b1c08700 con 0x5559b1d1d5a0 2024-04-24T06:44:11.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.812+0000 7fea63400640 1 --2- 172.21.15.5:0/2716418256 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5559b1d1d5a0 0x5559b1d1f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:11.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.812+0000 7fea63400640 1 --2- 172.21.15.5:0/2716418256 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5559b1d1d5a0 0x5559b1d1f9c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2acd4eb6eea1151a server_cookie=13ccc91d841c90fd in_seq=0 out_seq=0 2024-04-24T06:44:11.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.812+0000 7fea4d600640 10 client.25403 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:44:11.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.813+0000 7fea4d600640 1 -- 172.21.15.5:0/2716418256 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5559b1c08700 con 0x5559b1d1d5a0 2024-04-24T06:44:11.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.813+0000 7fea4d600640 10 client.25403 handle_command_reply: tid=0 2024-04-24T06:44:11.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.813+0000 7fea6d1d2740 1 client.25403 shutdown 2024-04-24T06:44:11.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.813+0000 7fea6d1d2740 1 -- 172.21.15.5:0/2716418256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559b1c9b9b0 msgr2=0x5559b1caa2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:11.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.813+0000 7fea6d1d2740 1 --2- 172.21.15.5:0/2716418256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559b1c9b9b0 0x5559b1caa2f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fea500b95c0 tx=0x7fea500c1070 comp rx=0 tx=0).stop 2024-04-24T06:44:11.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.813+0000 7fea6d1d2740 1 -- 172.21.15.5:0/2716418256 shutdown_connections 2024-04-24T06:44:11.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.813+0000 7fea6d1d2740 1 --2- 172.21.15.5:0/2716418256 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5559b1d1d5a0 0x5559b1d1f9c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.813+0000 7fea6d1d2740 1 --2- 172.21.15.5:0/2716418256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559b1ca6080 0x5559b1caa830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.813+0000 7fea6d1d2740 1 --2- 172.21.15.5:0/2716418256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559b1c9b9b0 0x5559b1caa2f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.813+0000 7fea6d1d2740 1 --2- 172.21.15.5:0/2716418256 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559b1c9b090 0x5559b1ca9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.813+0000 7fea6d1d2740 1 -- 172.21.15.5:0/2716418256 >> 172.21.15.5:0/2716418256 conn(0x5559b1c973b0 msgr2=0x5559b1ca48a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:11.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.813+0000 7fea4d600640 0 client.25403 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:44:11.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.814+0000 7fea6d1d2740 1 -- 172.21.15.5:0/2716418256 shutdown_connections 2024-04-24T06:44:11.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.814+0000 7fea6d1d2740 1 -- 172.21.15.5:0/2716418256 wait complete. 2024-04-24T06:44:11.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.814+0000 7fea6d1d2740 20 client.25403 trim_cache size 0 max 16384 2024-04-24T06:44:11.818 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:44:11.818 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-24T06:44:11.818 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "3efd49f2-f0ae-491d-b034-6ef4d5dc44a1", 2024-04-24T06:44:11.818 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-24T06:44:11.819 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:44:11.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.814+0000 7fea6a400640 1 -- 172.21.15.5:0/4157256777 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fea480a3e40 msgr2=0x7fea480a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:11.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.814+0000 7fea6a400640 1 --2- 172.21.15.5:0/4157256777 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fea480a3e40 0x7fea480a6300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x5559b1c9c650 tx=0x7fea54005f90 comp rx=0 tx=0).stop 2024-04-24T06:44:11.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.814+0000 7fea6a400640 1 -- 172.21.15.5:0/4157256777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea64109520 msgr2=0x7fea641a0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:11.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.814+0000 7fea6a400640 1 --2- 172.21.15.5:0/4157256777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea64109520 0x7fea641a0cc0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fea5002a930 tx=0x7fea50003b90 comp rx=0 tx=0).stop 2024-04-24T06:44:11.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.815+0000 7fea6a400640 1 -- 172.21.15.5:0/4157256777 shutdown_connections 2024-04-24T06:44:11.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.815+0000 7fea6a400640 1 --2- 172.21.15.5:0/4157256777 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fea480a3e40 0x7fea480a6300 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.815+0000 7fea6a400640 1 --2- 172.21.15.5:0/4157256777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea6419a980 0x7fea6419ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.815+0000 7fea6a400640 1 --2- 172.21.15.5:0/4157256777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea64109520 0x7fea641a0cc0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.815+0000 7fea6a400640 1 --2- 172.21.15.5:0/4157256777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea64104450 0x7fea641a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:11.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.815+0000 7fea6a400640 1 -- 172.21.15.5:0/4157256777 >> 172.21.15.5:0/4157256777 conn(0x7fea640ffc90 msgr2=0x7fea64100fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:11.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.815+0000 7fea6a400640 1 -- 172.21.15.5:0/4157256777 shutdown_connections 2024-04-24T06:44:11.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:11.815+0000 7fea6a400640 1 -- 172.21.15.5:0/4157256777 wait complete. 2024-04-24T06:44:11.834 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T06:44:11.834 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T06:44:12.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.009+0000 7f0436000640 1 -- 172.21.15.5:0/1201414261 --> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f0430102b50 con 0x7f04180a3e20 2024-04-24T06:44:12.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.027+0000 7f4926a00640 1 -- 172.21.15.5:0/1790614820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4920104280 msgr2=0x7f4920104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:12.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.027+0000 7f4926a00640 1 --2- 172.21.15.5:0/1790614820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4920104280 0x7f4920104660 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4914009900 tx=0x7f491402a3f0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.027+0000 7f4926a00640 1 -- 172.21.15.5:0/1790614820 shutdown_connections 2024-04-24T06:44:12.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.028+0000 7f4926a00640 1 --2- 172.21.15.5:0/1790614820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4920109350 0x7f4920110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.028+0000 7f4926a00640 1 --2- 172.21.15.5:0/1790614820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4920104c30 0x7f4920108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.028+0000 7f4926a00640 1 --2- 172.21.15.5:0/1790614820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4920104280 0x7f4920104660 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.028+0000 7f4926a00640 1 -- 172.21.15.5:0/1790614820 >> 172.21.15.5:0/1790614820 conn(0x7f49200ffae0 msgr2=0x7f4920101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:12.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.028+0000 7f4926a00640 1 -- 172.21.15.5:0/1790614820 shutdown_connections 2024-04-24T06:44:12.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.028+0000 7f4926a00640 1 -- 172.21.15.5:0/1790614820 wait complete. 2024-04-24T06:44:12.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.029+0000 7f4926a00640 1 Processor -- start 2024-04-24T06:44:12.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.029+0000 7f4926a00640 1 -- start start 2024-04-24T06:44:12.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.029+0000 7f4926a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4920104280 0x7f49201a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:12.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.029+0000 7f4926a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4920109350 0x7f49201a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:12.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.029+0000 7f4926a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f492019a7f0 0x7f492019aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:12.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.029+0000 7f4926a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4920114450 con 0x7f4920104280 2024-04-24T06:44:12.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.029+0000 7f4926a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f49201142d0 con 0x7f492019a7f0 2024-04-24T06:44:12.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.029+0000 7f4926a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f49201145d0 con 0x7f4920109350 2024-04-24T06:44:12.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.030+0000 7f4925600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4920104280 0x7f49201a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:12.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.030+0000 7f4924c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4920109350 0x7f49201a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:12.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.030+0000 7f4925600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4920104280 0x7f49201a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48918/0 (socket says 172.21.15.5:48918) 2024-04-24T06:44:12.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.030+0000 7f4925600640 1 -- 172.21.15.5:0/3086467023 learned_addr learned my addr 172.21.15.5:0/3086467023 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:12.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.030+0000 7f4926000640 1 --2- 172.21.15.5:0/3086467023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f492019a7f0 0x7f492019aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:12.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.030+0000 7f4924c00640 1 -- 172.21.15.5:0/3086467023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f492019a7f0 msgr2=0x7f492019aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:12.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.030+0000 7f4924c00640 1 --2- 172.21.15.5:0/3086467023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f492019a7f0 0x7f492019aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.030+0000 7f4924c00640 1 -- 172.21.15.5:0/3086467023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4920104280 msgr2=0x7f49201a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:12.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.030+0000 7f4924c00640 1 --2- 172.21.15.5:0/3086467023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4920104280 0x7f49201a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.030+0000 7f4924c00640 1 -- 172.21.15.5:0/3086467023 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f492019b380 con 0x7f4920109350 2024-04-24T06:44:12.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.031+0000 7f4924c00640 1 --2- 172.21.15.5:0/3086467023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4920109350 0x7f49201a0bc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f490c00e9c0 tx=0x7f490c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:12.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.031+0000 7f491e000640 1 -- 172.21.15.5:0/3086467023 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c00ccc0 con 0x7f4920109350 2024-04-24T06:44:12.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.031+0000 7f4926a00640 1 -- 172.21.15.5:0/3086467023 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f492019b670 con 0x7f4920109350 2024-04-24T06:44:12.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.031+0000 7f491e000640 1 -- 172.21.15.5:0/3086467023 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f490c005020 con 0x7f4920109350 2024-04-24T06:44:12.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.031+0000 7f491e000640 1 -- 172.21.15.5:0/3086467023 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c004a00 con 0x7f4920109350 2024-04-24T06:44:12.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.031+0000 7f4926a00640 1 -- 172.21.15.5:0/3086467023 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49201a7850 con 0x7f4920109350 2024-04-24T06:44:12.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.036+0000 7f492ad77740 1 Processor -- start 2024-04-24T06:44:12.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.036+0000 7f492ad77740 1 -- start start 2024-04-24T06:44:12.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.036+0000 7f491e000640 1 -- 172.21.15.5:0/3086467023 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 138872+0+0 (secure 0 0 0) 0x7f490c004ba0 con 0x7f4920109350 2024-04-24T06:44:12.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.037+0000 7f492ad77740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560abd21c780 0x560abd21cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:12.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.037+0000 7f492ad77740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560abd21d130 0x560abd22fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:12.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.037+0000 7f492ad77740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560abd230230 0x560abd232620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:12.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.037+0000 7f492ad77740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560abd1821f0 con 0x560abd21c780 2024-04-24T06:44:12.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.037+0000 7f492ad77740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560abd128350 con 0x560abd230230 2024-04-24T06:44:12.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.037+0000 7f492ad77740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560abd17e1b0 con 0x560abd21d130 2024-04-24T06:44:12.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.037+0000 7f4924c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560abd230230 0x560abd232620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:12.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.037+0000 7f4925600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560abd21d130 0x560abd22fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:12.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.037+0000 7f4924c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560abd230230 0x560abd232620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:46320/0 (socket says 172.21.15.5:46320) 2024-04-24T06:44:12.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.037+0000 7f4924c00640 1 -- 172.21.15.5:0/2600739460 learned_addr learned my addr 172.21.15.5:0/2600739460 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:12.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.037+0000 7f4926000640 1 --2- 172.21.15.5:0/2600739460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560abd21c780 0x560abd21cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:12.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.038+0000 7f4924c00640 1 -- 172.21.15.5:0/2600739460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560abd21d130 msgr2=0x560abd22fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:12.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.038+0000 7f4924c00640 1 --2- 172.21.15.5:0/2600739460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560abd21d130 0x560abd22fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.038+0000 7f4924c00640 1 -- 172.21.15.5:0/2600739460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560abd21c780 msgr2=0x560abd21cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:12.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.038+0000 7f4924c00640 1 --2- 172.21.15.5:0/2600739460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560abd21c780 0x560abd21cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.038+0000 7f4924c00640 1 -- 172.21.15.5:0/2600739460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560abd17cf50 con 0x560abd230230 2024-04-24T06:44:12.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.038+0000 7f4925600640 1 --2- 172.21.15.5:0/2600739460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560abd21d130 0x560abd22fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:44:12.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.038+0000 7f4926000640 1 --2- 172.21.15.5:0/2600739460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560abd21c780 0x560abd21cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:44:12.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.038+0000 7f4924c00640 1 --2- 172.21.15.5:0/2600739460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560abd230230 0x560abd232620 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f490c0cdef0 tx=0x7f490c0d6cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce2d97416d97a169 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:12.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.038+0000 7f491e000640 1 --2- 172.21.15.5:0/3086467023 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f49080a3f40 0x7f49080a6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:12.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.039+0000 7f4926000640 1 --2- 172.21.15.5:0/3086467023 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f49080a3f40 0x7f49080a6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:12.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.039+0000 7f4912600640 1 -- 172.21.15.5:0/2600739460 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c101070 con 0x560abd230230 2024-04-24T06:44:12.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.039+0000 7f491e000640 1 -- 172.21.15.5:0/3086467023 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f490c0d09d0 con 0x7f4920109350 2024-04-24T06:44:12.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.039+0000 7f4912600640 1 -- 172.21.15.5:0/2600739460 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f490c09cdd0 con 0x560abd230230 2024-04-24T06:44:12.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.039+0000 7f4912600640 1 -- 172.21.15.5:0/2600739460 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c09c210 con 0x560abd230230 2024-04-24T06:44:12.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.040+0000 7f492ad77740 1 -- 172.21.15.5:0/2600739460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560abd230230 msgr2=0x560abd232620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:12.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.040+0000 7f492ad77740 1 --2- 172.21.15.5:0/2600739460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560abd230230 0x560abd232620 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f490c0cdef0 tx=0x7f490c0d6cf0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.040+0000 7f4926000640 1 --2- 172.21.15.5:0/3086467023 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f49080a3f40 0x7f49080a6400 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4918005fd0 tx=0x7f49180096a0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:12.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.040+0000 7f492ad77740 1 -- 172.21.15.5:0/2600739460 shutdown_connections 2024-04-24T06:44:12.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.040+0000 7f492ad77740 1 --2- 172.21.15.5:0/2600739460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560abd230230 0x560abd232620 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.040+0000 7f492ad77740 1 --2- 172.21.15.5:0/2600739460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560abd21d130 0x560abd22fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.040+0000 7f492ad77740 1 --2- 172.21.15.5:0/2600739460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560abd21c780 0x560abd21cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.040+0000 7f492ad77740 1 -- 172.21.15.5:0/2600739460 >> 172.21.15.5:0/2600739460 conn(0x560abd218d50 msgr2=0x560abd219130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:12.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.040+0000 7f492ad77740 1 -- 172.21.15.5:0/2600739460 shutdown_connections 2024-04-24T06:44:12.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.040+0000 7f492ad77740 1 -- 172.21.15.5:0/2600739460 wait complete. 2024-04-24T06:44:12.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.041+0000 7f492ad77740 1 -- start start 2024-04-24T06:44:12.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.041+0000 7f492ad77740 1 Processor -- start 2024-04-24T06:44:12.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.041+0000 7f492ad77740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560abd21c780 0x560abd229ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:12.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.041+0000 7f492ad77740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560abd21d130 0x560abd22fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:12.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.041+0000 7f492ad77740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560abd230230 0x560abd22a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:12.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.042+0000 7f492ad77740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560abd21f880 con 0x560abd21c780 2024-04-24T06:44:12.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.042+0000 7f492ad77740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560abd13b690 con 0x560abd21d130 2024-04-24T06:44:12.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.042+0000 7f492ad77740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560abd21fa00 con 0x560abd230230 2024-04-24T06:44:12.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.042+0000 7f492ad77740 10 client.0 _ll_register_callbacks cb 0x560abd177650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:44:12.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.042+0000 7f4926000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560abd230230 0x560abd22a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:12.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.042+0000 7f4926000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560abd230230 0x560abd22a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:54534/0 (socket says 172.21.15.5:54534) 2024-04-24T06:44:12.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.042+0000 7f4926000640 1 -- 172.21.15.5:0/707379994 learned_addr learned my addr 172.21.15.5:0/707379994 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:12.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.042+0000 7f4924c00640 1 --2- 172.21.15.5:0/707379994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560abd21d130 0x560abd22fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:12.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.042+0000 7f4925600640 1 --2- 172.21.15.5:0/707379994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560abd21c780 0x560abd229ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:12.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.042+0000 7f4926000640 1 -- 172.21.15.5:0/707379994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560abd21d130 msgr2=0x560abd22fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:12.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.042+0000 7f4926000640 1 --2- 172.21.15.5:0/707379994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560abd21d130 0x560abd22fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.042+0000 7f4926000640 1 -- 172.21.15.5:0/707379994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560abd21c780 msgr2=0x560abd229ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:12.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.042+0000 7f4926000640 1 --2- 172.21.15.5:0/707379994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560abd21c780 0x560abd229ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.042+0000 7f4926000640 1 -- 172.21.15.5:0/707379994 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560abd13a920 con 0x560abd230230 2024-04-24T06:44:12.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.042+0000 7f4925600640 1 --2- 172.21.15.5:0/707379994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560abd21c780 0x560abd229ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:44:12.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.043+0000 7f4926000640 1 --2- 172.21.15.5:0/707379994 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560abd230230 0x560abd22a510 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4918009fd0 tx=0x7f4918004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ca2583b9a15fd6f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:12.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.043+0000 7f4906a00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:44:12.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.043+0000 7f4906a00640 1 -- 172.21.15.5:0/707379994 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f491802c060 con 0x560abd230230 2024-04-24T06:44:12.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.043+0000 7f492ad77740 1 -- 172.21.15.5:0/707379994 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f490c09cdd0 con 0x560abd230230 2024-04-24T06:44:12.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.043+0000 7f4906a00640 1 -- 172.21.15.5:0/707379994 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f4918017030 con 0x560abd230230 2024-04-24T06:44:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.043+0000 7f4906a00640 1 -- 172.21.15.5:0/707379994 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f491801f8b0 con 0x560abd230230 2024-04-24T06:44:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.043+0000 7f492ad77740 10 client.25438 fetch_fsmap learned FSMap version 23 2024-04-24T06:44:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.043+0000 7f492ad77740 1 -- 172.21.15.5:0/707379994 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x560abcf5e7d0 con 0x560abd230230 2024-04-24T06:44:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.044+0000 7f4906a00640 1 -- 172.21.15.5:0/707379994 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f490c09cdd0 con 0x560abd230230 2024-04-24T06:44:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.044+0000 7f4906a00640 1 -- 172.21.15.5:0/707379994 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f491801c3f0 con 0x560abd230230 2024-04-24T06:44:12.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.044+0000 7f492ad77740 10 client.25438 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T06:44:12.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.044+0000 7f492ad77740 10 client.25438 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:44:12.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.044+0000 7f492ad77740 20 client.25438 populate_metadata read hostname 'smithi005' 2024-04-24T06:44:12.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.044+0000 7f492ad77740 1 --2- 172.21.15.5:0/707379994 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x560abd237dd0 0x560abd23a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:12.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.044+0000 7f492ad77740 4 client.25438 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:44:12.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.044+0000 7f492ad77740 1 -- 172.21.15.5:0/707379994 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560abd18fc10 con 0x560abd237dd0 2024-04-24T06:44:12.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.044+0000 7f4925600640 1 --2- 172.21.15.5:0/707379994 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x560abd237dd0 0x560abd23a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:12.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.045+0000 7f4906a00640 10 client.25438 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:44:12.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.045+0000 7f4925600640 1 --2- 172.21.15.5:0/707379994 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x560abd237dd0 0x560abd23a1c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d2f0a13e1c27b8d server_cookie=331ea2c77427dbe7 in_seq=0 out_seq=0 2024-04-24T06:44:12.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.046+0000 7f4906a00640 1 -- 172.21.15.5:0/707379994 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560abd18fc10 con 0x560abd237dd0 2024-04-24T06:44:12.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.046+0000 7f4906a00640 10 client.25438 handle_command_reply: tid=0 2024-04-24T06:44:12.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.046+0000 7f492ad77740 1 client.25438 shutdown 2024-04-24T06:44:12.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.046+0000 7f492ad77740 1 -- 172.21.15.5:0/707379994 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560abd230230 msgr2=0x560abd22a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:12.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.046+0000 7f492ad77740 1 --2- 172.21.15.5:0/707379994 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560abd230230 0x560abd22a510 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4918009fd0 tx=0x7f4918004ab0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.046+0000 7f492ad77740 1 -- 172.21.15.5:0/707379994 shutdown_connections 2024-04-24T06:44:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.046+0000 7f492ad77740 1 --2- 172.21.15.5:0/707379994 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x560abd237dd0 0x560abd23a1c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.046+0000 7f492ad77740 1 --2- 172.21.15.5:0/707379994 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560abd230230 0x560abd22a510 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.046+0000 7f492ad77740 1 --2- 172.21.15.5:0/707379994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560abd21d130 0x560abd22fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.046+0000 7f492ad77740 1 --2- 172.21.15.5:0/707379994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560abd21c780 0x560abd229ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.046+0000 7f4906a00640 0 client.25438 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:44:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.046+0000 7f492ad77740 1 -- 172.21.15.5:0/707379994 >> 172.21.15.5:0/707379994 conn(0x560abd218d50 msgr2=0x560abd21ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.046+0000 7f492ad77740 1 -- 172.21.15.5:0/707379994 shutdown_connections 2024-04-24T06:44:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.047+0000 7f492ad77740 1 -- 172.21.15.5:0/707379994 wait complete. 2024-04-24T06:44:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.047+0000 7f492ad77740 20 client.25438 trim_cache size 0 max 16384 2024-04-24T06:44:12.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.071+0000 7f492ad77740 1 Processor -- start 2024-04-24T06:44:12.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.071+0000 7f492ad77740 1 -- start start 2024-04-24T06:44:12.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.071+0000 7f492ad77740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560abd21c780 0x560abd22fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:12.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.071+0000 7f492ad77740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560abd21d130 0x560abd229a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:12.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.071+0000 7f492ad77740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560abd230230 0x560abd23fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:12.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.071+0000 7f492ad77740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f49201ae2b0 con 0x560abd21c780 2024-04-24T06:44:12.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.071+0000 7f492ad77740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f49201abb80 con 0x560abd230230 2024-04-24T06:44:12.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.071+0000 7f492ad77740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f49201a99d0 con 0x560abd21d130 2024-04-24T06:44:12.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.072+0000 7f4924c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560abd21d130 0x560abd229a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:12.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.072+0000 7f4925600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560abd21c780 0x560abd22fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:12.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.072+0000 7f4924c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560abd21d130 0x560abd229a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:54540/0 (socket says 172.21.15.5:54540) 2024-04-24T06:44:12.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.072+0000 7f4924c00640 1 -- 172.21.15.5:0/1031313501 learned_addr learned my addr 172.21.15.5:0/1031313501 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:12.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.072+0000 7f4926000640 1 --2- 172.21.15.5:0/1031313501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560abd230230 0x560abd23fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:12.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.072+0000 7f4924c00640 1 -- 172.21.15.5:0/1031313501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560abd230230 msgr2=0x560abd23fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:12.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.072+0000 7f4924c00640 1 --2- 172.21.15.5:0/1031313501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560abd230230 0x560abd23fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.072+0000 7f4924c00640 1 -- 172.21.15.5:0/1031313501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560abd21c780 msgr2=0x560abd22fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:12.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.072+0000 7f4924c00640 1 --2- 172.21.15.5:0/1031313501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560abd21c780 0x560abd22fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.072+0000 7f4924c00640 1 -- 172.21.15.5:0/1031313501 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560abce943f0 con 0x560abd21d130 2024-04-24T06:44:12.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.073+0000 7f4924c00640 1 --2- 172.21.15.5:0/1031313501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560abd21d130 0x560abd229a90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f490c00c6a0 tx=0x7f490c0096e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca84875fdb98a1f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:12.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.073+0000 7f4911c00640 1 -- 172.21.15.5:0/1031313501 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c0b2070 con 0x560abd21d130 2024-04-24T06:44:12.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.073+0000 7f4911c00640 1 -- 172.21.15.5:0/1031313501 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f490c0d6300 con 0x560abd21d130 2024-04-24T06:44:12.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.073+0000 7f4911c00640 1 -- 172.21.15.5:0/1031313501 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c01c070 con 0x560abd21d130 2024-04-24T06:44:12.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.074+0000 7f492ad77740 1 -- 172.21.15.5:0/1031313501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560abd21d130 msgr2=0x560abd229a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:12.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.074+0000 7f492ad77740 1 --2- 172.21.15.5:0/1031313501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560abd21d130 0x560abd229a90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f490c00c6a0 tx=0x7f490c0096e0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.074+0000 7f492ad77740 1 -- 172.21.15.5:0/1031313501 shutdown_connections 2024-04-24T06:44:12.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.074+0000 7f492ad77740 1 --2- 172.21.15.5:0/1031313501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560abd230230 0x560abd23fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.074+0000 7f492ad77740 1 --2- 172.21.15.5:0/1031313501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560abd21d130 0x560abd229a90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.074+0000 7f492ad77740 1 --2- 172.21.15.5:0/1031313501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560abd21c780 0x560abd22fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.074+0000 7f492ad77740 1 -- 172.21.15.5:0/1031313501 >> 172.21.15.5:0/1031313501 conn(0x560abd218d50 msgr2=0x560abd219130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:12.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.074+0000 7f492ad77740 1 -- 172.21.15.5:0/1031313501 shutdown_connections 2024-04-24T06:44:12.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.074+0000 7f492ad77740 1 -- 172.21.15.5:0/1031313501 wait complete. 2024-04-24T06:44:12.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.075+0000 7f492ad77740 1 -- start start 2024-04-24T06:44:12.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.075+0000 7f492ad77740 1 Processor -- start 2024-04-24T06:44:12.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.075+0000 7f492ad77740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560abd21c780 0x560abd246cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:12.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.075+0000 7f492ad77740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560abd21d130 0x560abd247210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:12.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.075+0000 7f492ad77740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560abd230230 0x560abd247750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:12.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.075+0000 7f492ad77740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f49201aa040 con 0x560abd21d130 2024-04-24T06:44:12.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.075+0000 7f492ad77740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f49201ab4b0 con 0x560abd230230 2024-04-24T06:44:12.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.075+0000 7f492ad77740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f49201aa710 con 0x560abd21c780 2024-04-24T06:44:12.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.075+0000 7f492ad77740 10 client.0 _ll_register_callbacks cb 0x7f4918009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:44:12.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.076+0000 7f4925600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560abd21c780 0x560abd246cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:12.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.076+0000 7f4924c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560abd21d130 0x560abd247210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:12.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.076+0000 7f4925600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560abd21c780 0x560abd246cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:54554/0 (socket says 172.21.15.5:54554) 2024-04-24T06:44:12.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.076+0000 7f4925600640 1 -- 172.21.15.5:0/2652873968 learned_addr learned my addr 172.21.15.5:0/2652873968 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:12.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.076+0000 7f4926000640 1 --2- 172.21.15.5:0/2652873968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560abd230230 0x560abd247750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:12.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.076+0000 7f4925600640 1 -- 172.21.15.5:0/2652873968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560abd230230 msgr2=0x560abd247750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:12.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.076+0000 7f4925600640 1 --2- 172.21.15.5:0/2652873968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560abd230230 0x560abd247750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.076+0000 7f4925600640 1 -- 172.21.15.5:0/2652873968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560abd21d130 msgr2=0x560abd247210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:12.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.076+0000 7f4925600640 1 --2- 172.21.15.5:0/2652873968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560abd21d130 0x560abd247210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.076+0000 7f4925600640 1 -- 172.21.15.5:0/2652873968 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560abce540f0 con 0x560abd21c780 2024-04-24T06:44:12.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.076+0000 7f4926000640 1 --2- 172.21.15.5:0/2652873968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560abd230230 0x560abd247750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:44:12.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.076+0000 7f4925600640 1 --2- 172.21.15.5:0/2652873968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560abd21c780 0x560abd246cd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4914004770 tx=0x7f4914004550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8179ec51d12e5469 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:12.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.076+0000 7f4906a00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:44:12.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.077+0000 7f4906a00640 1 -- 172.21.15.5:0/2652873968 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4914049070 con 0x560abd21c780 2024-04-24T06:44:12.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.077+0000 7f4906a00640 1 -- 172.21.15.5:0/2652873968 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f4914005220 con 0x560abd21c780 2024-04-24T06:44:12.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.077+0000 7f492ad77740 1 -- 172.21.15.5:0/2652873968 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f490c0d6300 con 0x560abd21c780 2024-04-24T06:44:12.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.077+0000 7f4906a00640 1 -- 172.21.15.5:0/2652873968 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f4914035040 con 0x560abd21c780 2024-04-24T06:44:12.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.077+0000 7f492ad77740 10 client.25444 fetch_fsmap learned FSMap version 23 2024-04-24T06:44:12.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.077+0000 7f492ad77740 1 -- 172.21.15.5:0/2652873968 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x560abd147810 con 0x560abd21c780 2024-04-24T06:44:12.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.078+0000 7f4906a00640 1 -- 172.21.15.5:0/2652873968 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f490c0d6300 con 0x560abd21c780 2024-04-24T06:44:12.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.078+0000 7f4906a00640 1 -- 172.21.15.5:0/2652873968 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f4914004830 con 0x560abd21c780 2024-04-24T06:44:12.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.078+0000 7f492ad77740 10 client.25444 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T06:44:12.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.078+0000 7f492ad77740 10 client.25444 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:44:12.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.078+0000 7f492ad77740 20 client.25444 populate_metadata read hostname 'smithi005' 2024-04-24T06:44:12.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.078+0000 7f492ad77740 1 --2- 172.21.15.5:0/2652873968 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x560abd29e700 0x560abd2a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:12.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.078+0000 7f492ad77740 4 client.25444 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:44:12.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.078+0000 7f492ad77740 1 -- 172.21.15.5:0/2652873968 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560abd189a80 con 0x560abd29e700 2024-04-24T06:44:12.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.078+0000 7f4924c00640 1 --2- 172.21.15.5:0/2652873968 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x560abd29e700 0x560abd2a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:12.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.079+0000 7f4924c00640 1 --2- 172.21.15.5:0/2652873968 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x560abd29e700 0x560abd2a0b40 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed154f10efa2533d server_cookie=de4a86d8e48f7a3b in_seq=0 out_seq=0 2024-04-24T06:44:12.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.079+0000 7f4906a00640 10 client.25444 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:44:12.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.079+0000 7f4906a00640 1 -- 172.21.15.5:0/2652873968 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560abd189a80 con 0x560abd29e700 2024-04-24T06:44:12.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.079+0000 7f4906a00640 10 client.25444 handle_command_reply: tid=0 2024-04-24T06:44:12.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.079+0000 7f492ad77740 1 client.25444 shutdown 2024-04-24T06:44:12.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.079+0000 7f492ad77740 1 -- 172.21.15.5:0/2652873968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560abd21c780 msgr2=0x560abd246cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:12.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.079+0000 7f492ad77740 1 --2- 172.21.15.5:0/2652873968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560abd21c780 0x560abd246cd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4914004770 tx=0x7f4914004550 comp rx=0 tx=0).stop 2024-04-24T06:44:12.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.079+0000 7f492ad77740 1 -- 172.21.15.5:0/2652873968 shutdown_connections 2024-04-24T06:44:12.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.080+0000 7f492ad77740 1 --2- 172.21.15.5:0/2652873968 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x560abd29e700 0x560abd2a0b40 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.080+0000 7f492ad77740 1 --2- 172.21.15.5:0/2652873968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560abd230230 0x560abd247750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.080+0000 7f492ad77740 1 --2- 172.21.15.5:0/2652873968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560abd21d130 0x560abd247210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.080+0000 7f492ad77740 1 --2- 172.21.15.5:0/2652873968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560abd21c780 0x560abd246cd0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.080+0000 7f4906a00640 0 client.25444 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:44:12.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.080+0000 7f492ad77740 1 -- 172.21.15.5:0/2652873968 >> 172.21.15.5:0/2652873968 conn(0x560abd218d50 msgr2=0x560abd217740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:12.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.080+0000 7f492ad77740 1 -- 172.21.15.5:0/2652873968 shutdown_connections 2024-04-24T06:44:12.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.080+0000 7f492ad77740 1 -- 172.21.15.5:0/2652873968 wait complete. 2024-04-24T06:44:12.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.080+0000 7f492ad77740 20 client.25444 trim_cache size 0 max 16384 2024-04-24T06:44:12.086 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:44:12.086 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T06:44:12.086 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-24T06:44:12.086 INFO:teuthology.orchestra.run.smithi005.stdout: "3efd49f2-f0ae-491d-b034-6ef4d5dc44a1": { 2024-04-24T06:44:12.086 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-24T06:44:12.086 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "3efd49f2-f0ae-491d-b034-6ef4d5dc44a1", 2024-04-24T06:44:12.086 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-04-24T06:44:12.086 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:44:12.086 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:44:12.086 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:44:12.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.081+0000 7f4926a00640 1 -- 172.21.15.5:0/3086467023 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f49080a3f40 msgr2=0x7f49080a6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:12.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.081+0000 7f4926a00640 1 --2- 172.21.15.5:0/3086467023 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f49080a3f40 0x7f49080a6400 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4918005fd0 tx=0x7f49180096a0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.081+0000 7f4926a00640 1 -- 172.21.15.5:0/3086467023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4920109350 msgr2=0x7f49201a0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:12.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.081+0000 7f4926a00640 1 --2- 172.21.15.5:0/3086467023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4920109350 0x7f49201a0bc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f490c00e9c0 tx=0x7f490c00ee80 comp rx=0 tx=0).stop 2024-04-24T06:44:12.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.081+0000 7f4926a00640 1 -- 172.21.15.5:0/3086467023 shutdown_connections 2024-04-24T06:44:12.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.081+0000 7f4926a00640 1 --2- 172.21.15.5:0/3086467023 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f49080a3f40 0x7f49080a6400 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.081+0000 7f4926a00640 1 --2- 172.21.15.5:0/3086467023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f492019a7f0 0x7f492019aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.081+0000 7f4926a00640 1 --2- 172.21.15.5:0/3086467023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4920109350 0x7f49201a0bc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.081+0000 7f4926a00640 1 --2- 172.21.15.5:0/3086467023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4920104280 0x7f49201a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.081+0000 7f4926a00640 1 -- 172.21.15.5:0/3086467023 >> 172.21.15.5:0/3086467023 conn(0x7f49200ffae0 msgr2=0x7f4920100df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:12.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.081+0000 7f4926a00640 1 -- 172.21.15.5:0/3086467023 shutdown_connections 2024-04-24T06:44:12.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.081+0000 7f4926a00640 1 -- 172.21.15.5:0/3086467023 wait complete. 2024-04-24T06:44:12.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.097+0000 7f042d600640 1 -- 172.21.15.5:0/1201414261 <== mgr.25202 v2:172.21.15.5:6800/3154624651 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0430102b50 con 0x7f04180a3e20 2024-04-24T06:44:12.098 INFO:teuthology.orchestra.run.smithi005.stdout:Schedule set for path / 2024-04-24T06:44:12.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.099+0000 7f0436000640 1 -- 172.21.15.5:0/1201414261 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f04180a3e20 msgr2=0x7f04180a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:12.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.099+0000 7f0436000640 1 --2- 172.21.15.5:0/1201414261 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f04180a3e20 0x7f04180a62e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0424009b60 tx=0x7f0424005f90 comp rx=0 tx=0).stop 2024-04-24T06:44:12.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.099+0000 7f0436000640 1 -- 172.21.15.5:0/1201414261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f043019a820 msgr2=0x7f043019acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:12.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.099+0000 7f0436000640 1 --2- 172.21.15.5:0/1201414261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f043019a820 0x7f043019acd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f042800eb10 tx=0x7f042800c2d0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.099+0000 7f0436000640 1 -- 172.21.15.5:0/1201414261 shutdown_connections 2024-04-24T06:44:12.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.099+0000 7f0436000640 1 --2- 172.21.15.5:0/1201414261 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f04180a3e20 0x7f04180a62e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.099+0000 7f0436000640 1 --2- 172.21.15.5:0/1201414261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f043019a820 0x7f043019acd0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.099+0000 7f0436000640 1 --2- 172.21.15.5:0/1201414261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0430107160 0x7f04301a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.099+0000 7f0436000640 1 --2- 172.21.15.5:0/1201414261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0430102090 0x7f04301a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.099+0000 7f0436000640 1 -- 172.21.15.5:0/1201414261 >> 172.21.15.5:0/1201414261 conn(0x7f04300fd910 msgr2=0x7f04300fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:12.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.100+0000 7f0436000640 1 -- 172.21.15.5:0/1201414261 shutdown_connections 2024-04-24T06:44:12.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.100+0000 7f0436000640 1 -- 172.21.15.5:0/1201414261 wait complete. 2024-04-24T06:44:12.104 INFO:tasks.cephfs.filesystem:scrub status for tag:3efd49f2-f0ae-491d-b034-6ef4d5dc44a1 - {'path': '/', 'tag': '3efd49f2-f0ae-491d-b034-6ef4d5dc44a1', 'options': 'recursive,force'} 2024-04-24T06:44:12.118 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-24T06:44:12.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.362+0000 7f302ce00640 1 -- 172.21.15.5:0/1058598696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30281042d0 msgr2=0x7f30281046b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:12.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.363+0000 7f302ce00640 1 --2- 172.21.15.5:0/1058598696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30281042d0 0x7f30281046b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3018009860 tx=0x7f301802a340 comp rx=0 tx=0).stop 2024-04-24T06:44:12.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.363+0000 7f302ce00640 1 -- 172.21.15.5:0/1058598696 shutdown_connections 2024-04-24T06:44:12.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.363+0000 7f302ce00640 1 --2- 172.21.15.5:0/1058598696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30281093a0 0x7f3028110f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.363+0000 7f302ce00640 1 --2- 172.21.15.5:0/1058598696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3028104c80 0x7f3028108cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.363+0000 7f302ce00640 1 --2- 172.21.15.5:0/1058598696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30281042d0 0x7f30281046b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.363+0000 7f302ce00640 1 -- 172.21.15.5:0/1058598696 >> 172.21.15.5:0/1058598696 conn(0x7f30280ffb10 msgr2=0x7f3028101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:12.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.363+0000 7f302ce00640 1 -- 172.21.15.5:0/1058598696 shutdown_connections 2024-04-24T06:44:12.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.363+0000 7f302ce00640 1 -- 172.21.15.5:0/1058598696 wait complete. 2024-04-24T06:44:12.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.364+0000 7f302ce00640 1 Processor -- start 2024-04-24T06:44:12.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.364+0000 7f302ce00640 1 -- start start 2024-04-24T06:44:12.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.365+0000 7f302ce00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30281042d0 0x7f30281a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:12.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.365+0000 7f302ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30281093a0 0x7f30281a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:12.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.365+0000 7f302ce00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f302819a7f0 0x7f302819aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:12.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.365+0000 7f302ce00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3028114430 con 0x7f30281093a0 2024-04-24T06:44:12.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.365+0000 7f302ce00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f30281142b0 con 0x7f302819a7f0 2024-04-24T06:44:12.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.365+0000 7f302ce00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f30281145b0 con 0x7f30281042d0 2024-04-24T06:44:12.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.365+0000 7f3027e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f302819a7f0 0x7f302819aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:12.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.365+0000 7f3027400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30281042d0 0x7f30281a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:12.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.365+0000 7f301fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30281093a0 0x7f30281a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:12.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.365+0000 7f3027e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f302819a7f0 0x7f302819aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:46378/0 (socket says 172.21.15.5:46378) 2024-04-24T06:44:12.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.365+0000 7f3027e00640 1 -- 172.21.15.5:0/547847535 learned_addr learned my addr 172.21.15.5:0/547847535 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:12.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.366+0000 7f3027e00640 1 -- 172.21.15.5:0/547847535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30281042d0 msgr2=0x7f30281a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:12.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.366+0000 7f3027e00640 1 --2- 172.21.15.5:0/547847535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30281042d0 0x7f30281a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.366+0000 7f3027e00640 1 -- 172.21.15.5:0/547847535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30281093a0 msgr2=0x7f30281a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:12.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.366+0000 7f3027e00640 1 --2- 172.21.15.5:0/547847535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30281093a0 0x7f30281a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.366+0000 7f3027e00640 1 -- 172.21.15.5:0/547847535 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f302819b380 con 0x7f302819a7f0 2024-04-24T06:44:12.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.366+0000 7f3027e00640 1 --2- 172.21.15.5:0/547847535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f302819a7f0 0x7f302819aca0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3020009fd0 tx=0x7f302000c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:12.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.366+0000 7f3024c00640 1 -- 172.21.15.5:0/547847535 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3020004b20 con 0x7f302819a7f0 2024-04-24T06:44:12.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.366+0000 7f3024c00640 1 -- 172.21.15.5:0/547847535 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3020003e50 con 0x7f302819a7f0 2024-04-24T06:44:12.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.366+0000 7f302ce00640 1 -- 172.21.15.5:0/547847535 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f302819b670 con 0x7f302819a7f0 2024-04-24T06:44:12.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.366+0000 7f3024c00640 1 -- 172.21.15.5:0/547847535 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302000b630 con 0x7f302819a7f0 2024-04-24T06:44:12.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.366+0000 7f302ce00640 1 -- 172.21.15.5:0/547847535 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30281a7800 con 0x7f302819a7f0 2024-04-24T06:44:12.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.368+0000 7f302ce00640 1 -- 172.21.15.5:0/547847535 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30281057a0 con 0x7f302819a7f0 2024-04-24T06:44:12.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.370+0000 7f3024c00640 1 -- 172.21.15.5:0/547847535 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138872+0+0 (secure 0 0 0) 0x7f302001d020 con 0x7f302819a7f0 2024-04-24T06:44:12.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.371+0000 7f3024c00640 1 --2- 172.21.15.5:0/547847535 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f300c0a3ea0 0x7f300c0a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:12.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.372+0000 7f3024c00640 1 -- 172.21.15.5:0/547847535 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f30200d09c0 con 0x7f302819a7f0 2024-04-24T06:44:12.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.372+0000 7f3027400640 1 --2- 172.21.15.5:0/547847535 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f300c0a3ea0 0x7f300c0a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:12.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.373+0000 7f3027400640 1 --2- 172.21.15.5:0/547847535 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f300c0a3ea0 0x7f300c0a6360 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f301802a930 tx=0x7f30180056d0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:12.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.376+0000 7f3024c00640 1 -- 172.21.15.5:0/547847535 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f302819b670 con 0x7f302819a7f0 2024-04-24T06:44:12.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:12 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1602288035' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:44:12.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:12 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:12.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:12 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:12.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:12 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:44:12.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:12 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:44:12.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:12 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:12.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.610+0000 7f302ce00640 1 -- 172.21.15.5:0/547847535 --> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f3028104c80 con 0x7f300c0a3ea0 2024-04-24T06:44:12.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:12 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/1602288035' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:44:12.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:12 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:12.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:12 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:12.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:12 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:44:12.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:12 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:44:12.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:12 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:12.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.755+0000 7f3024c00640 1 -- 172.21.15.5:0/547847535 <== mgr.25202 v2:172.21.15.5:6800/3154624651 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f3028104c80 con 0x7f300c0a3ea0 2024-04-24T06:44:12.757 INFO:teuthology.orchestra.run.smithi005.stdout:Retention added to path / 2024-04-24T06:44:12.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.759+0000 7f302ce00640 1 -- 172.21.15.5:0/547847535 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f300c0a3ea0 msgr2=0x7f300c0a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:12.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.759+0000 7f302ce00640 1 --2- 172.21.15.5:0/547847535 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f300c0a3ea0 0x7f300c0a6360 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f301802a930 tx=0x7f30180056d0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.759+0000 7f302ce00640 1 -- 172.21.15.5:0/547847535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f302819a7f0 msgr2=0x7f302819aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:12.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.759+0000 7f302ce00640 1 --2- 172.21.15.5:0/547847535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f302819a7f0 0x7f302819aca0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3020009fd0 tx=0x7f302000c800 comp rx=0 tx=0).stop 2024-04-24T06:44:12.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.760+0000 7f302ce00640 1 -- 172.21.15.5:0/547847535 shutdown_connections 2024-04-24T06:44:12.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.760+0000 7f302ce00640 1 --2- 172.21.15.5:0/547847535 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f300c0a3ea0 0x7f300c0a6360 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.760+0000 7f302ce00640 1 --2- 172.21.15.5:0/547847535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f302819a7f0 0x7f302819aca0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.760+0000 7f302ce00640 1 --2- 172.21.15.5:0/547847535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30281093a0 0x7f30281a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.760+0000 7f302ce00640 1 --2- 172.21.15.5:0/547847535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30281042d0 0x7f30281a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:12.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.760+0000 7f302ce00640 1 -- 172.21.15.5:0/547847535 >> 172.21.15.5:0/547847535 conn(0x7f30280ffb10 msgr2=0x7f3028100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:12.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.760+0000 7f302ce00640 1 -- 172.21.15.5:0/547847535 shutdown_connections 2024-04-24T06:44:12.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.760+0000 7f302ce00640 1 -- 172.21.15.5:0/547847535 wait complete. 2024-04-24T06:44:12.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:12 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/1602288035' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:44:12.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:12 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:12.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:12 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:12.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:12 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:44:12.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:12 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:44:12.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:12 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:44:12.786 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-24T06:44:12.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.997+0000 7f83e2a00640 1 -- 172.21.15.5:0/139428919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83dc074cc0 msgr2=0x7f83dc1128d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:12.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.997+0000 7f83e2a00640 1 --2- 172.21.15.5:0/139428919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83dc074cc0 0x7f83dc1128d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009900 tx=0x7f83c802a480 comp rx=0 tx=0).stop 2024-04-24T06:44:12.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.997+0000 7f83e2a00640 1 -- 172.21.15.5:0/139428919 shutdown_connections 2024-04-24T06:44:12.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.997+0000 7f83e2a00640 1 --2- 172.21.15.5:0/139428919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83dc112e10 0x7f83dc115200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:13.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.997+0000 7f83e2a00640 1 --2- 172.21.15.5:0/139428919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83dc074cc0 0x7f83dc1128d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:13.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.997+0000 7f83e2a00640 1 --2- 172.21.15.5:0/139428919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83dc0743a0 0x7f83dc074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:13.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.997+0000 7f83e2a00640 1 -- 172.21.15.5:0/139428919 >> 172.21.15.5:0/139428919 conn(0x7f83dc0fd930 msgr2=0x7f83dc0ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:13.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.997+0000 7f83e2a00640 1 -- 172.21.15.5:0/139428919 shutdown_connections 2024-04-24T06:44:13.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.997+0000 7f83e2a00640 1 -- 172.21.15.5:0/139428919 wait complete. 2024-04-24T06:44:13.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.998+0000 7f83e2a00640 1 Processor -- start 2024-04-24T06:44:13.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.998+0000 7f83e2a00640 1 -- start start 2024-04-24T06:44:13.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.999+0000 7f83e2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83dc0743a0 0x7f83dc19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:13.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.999+0000 7f83e2a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83dc074cc0 0x7f83dc19f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:13.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.999+0000 7f83e2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83dc112e10 0x7f83dc1a3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:13.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.999+0000 7f83e2a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc1187e0 con 0x7f83dc112e10 2024-04-24T06:44:13.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.999+0000 7f83e2a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc118660 con 0x7f83dc0743a0 2024-04-24T06:44:13.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.999+0000 7f83e2a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc118960 con 0x7f83dc074cc0 2024-04-24T06:44:13.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.999+0000 7f83e1600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83dc0743a0 0x7f83dc19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:13.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.999+0000 7f83e1600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83dc0743a0 0x7f83dc19eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:46398/0 (socket says 172.21.15.5:46398) 2024-04-24T06:44:13.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.999+0000 7f83e1600640 1 -- 172.21.15.5:0/2850052090 learned_addr learned my addr 172.21.15.5:0/2850052090 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:13.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.999+0000 7f83e2000640 1 --2- 172.21.15.5:0/2850052090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83dc112e10 0x7f83dc1a3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:13.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:12.999+0000 7f83e0c00640 1 --2- 172.21.15.5:0/2850052090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83dc074cc0 0x7f83dc19f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:13.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.000+0000 7f83e1600640 1 -- 172.21.15.5:0/2850052090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83dc074cc0 msgr2=0x7f83dc19f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:13.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.000+0000 7f83e1600640 1 --2- 172.21.15.5:0/2850052090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83dc074cc0 0x7f83dc19f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:13.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.000+0000 7f83e1600640 1 -- 172.21.15.5:0/2850052090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83dc112e10 msgr2=0x7f83dc1a3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:13.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.000+0000 7f83e1600640 1 --2- 172.21.15.5:0/2850052090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83dc112e10 0x7f83dc1a3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:13.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.000+0000 7f83e1600640 1 -- 172.21.15.5:0/2850052090 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83dc1a3d40 con 0x7f83dc0743a0 2024-04-24T06:44:13.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.000+0000 7f83e1600640 1 --2- 172.21.15.5:0/2850052090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83dc0743a0 0x7f83dc19eda0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f83d000e9f0 tx=0x7f83d000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:13.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.000+0000 7f83da000640 1 -- 172.21.15.5:0/2850052090 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d000cd80 con 0x7f83dc0743a0 2024-04-24T06:44:13.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.000+0000 7f83da000640 1 -- 172.21.15.5:0/2850052090 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83d0010910 con 0x7f83dc0743a0 2024-04-24T06:44:13.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.000+0000 7f83e2a00640 1 -- 172.21.15.5:0/2850052090 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83dc1a4030 con 0x7f83dc0743a0 2024-04-24T06:44:13.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.000+0000 7f83da000640 1 -- 172.21.15.5:0/2850052090 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d00188f0 con 0x7f83dc0743a0 2024-04-24T06:44:13.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.000+0000 7f83e2a00640 1 -- 172.21.15.5:0/2850052090 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83dc1ab8c0 con 0x7f83dc0743a0 2024-04-24T06:44:13.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.001+0000 7f83e2a00640 1 -- 172.21.15.5:0/2850052090 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83dc1a00a0 con 0x7f83dc0743a0 2024-04-24T06:44:13.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.004+0000 7f83da000640 1 -- 172.21.15.5:0/2850052090 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138872+0+0 (secure 0 0 0) 0x7f83d0010ac0 con 0x7f83dc0743a0 2024-04-24T06:44:13.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.005+0000 7f83da000640 1 --2- 172.21.15.5:0/2850052090 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f83c40a3e20 0x7f83c40a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:13.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.005+0000 7f83da000640 1 -- 172.21.15.5:0/2850052090 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f83d00d0ff0 con 0x7f83dc0743a0 2024-04-24T06:44:13.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.005+0000 7f83e0c00640 1 --2- 172.21.15.5:0/2850052090 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f83c40a3e20 0x7f83c40a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:13.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.006+0000 7f83e0c00640 1 --2- 172.21.15.5:0/2850052090 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f83c40a3e20 0x7f83c40a62e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009230 tx=0x7f83c80058d0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:13.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.008+0000 7f83da000640 1 -- 172.21.15.5:0/2850052090 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f83d009ce80 con 0x7f83dc0743a0 2024-04-24T06:44:13.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.242+0000 7f83e2a00640 1 -- 172.21.15.5:0/2850052090 --> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f83dc062440 con 0x7f83c40a3e20 2024-04-24T06:44:13.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.254+0000 7f83da000640 1 -- 172.21.15.5:0/2850052090 <== mgr.25202 v2:172.21.15.5:6800/3154624651 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f83dc062440 con 0x7f83c40a3e20 2024-04-24T06:44:13.256 INFO:teuthology.orchestra.run.smithi005.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-24T00:00:00", "created": "2024-04-24T06:44:12", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-24T06:44:13.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.258+0000 7f83e2a00640 1 -- 172.21.15.5:0/2850052090 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f83c40a3e20 msgr2=0x7f83c40a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:13.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.258+0000 7f83e2a00640 1 --2- 172.21.15.5:0/2850052090 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f83c40a3e20 0x7f83c40a62e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009230 tx=0x7f83c80058d0 comp rx=0 tx=0).stop 2024-04-24T06:44:13.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.258+0000 7f83e2a00640 1 -- 172.21.15.5:0/2850052090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83dc0743a0 msgr2=0x7f83dc19eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:13.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.258+0000 7f83e2a00640 1 --2- 172.21.15.5:0/2850052090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83dc0743a0 0x7f83dc19eda0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f83d000e9f0 tx=0x7f83d000eeb0 comp rx=0 tx=0).stop 2024-04-24T06:44:13.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.258+0000 7f83e2a00640 1 -- 172.21.15.5:0/2850052090 shutdown_connections 2024-04-24T06:44:13.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.258+0000 7f83e2a00640 1 --2- 172.21.15.5:0/2850052090 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f83c40a3e20 0x7f83c40a62e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:13.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.258+0000 7f83e2a00640 1 --2- 172.21.15.5:0/2850052090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83dc112e10 0x7f83dc1a3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:13.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.258+0000 7f83e2a00640 1 --2- 172.21.15.5:0/2850052090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83dc074cc0 0x7f83dc19f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:13.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.258+0000 7f83e2a00640 1 --2- 172.21.15.5:0/2850052090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83dc0743a0 0x7f83dc19eda0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:13.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.258+0000 7f83e2a00640 1 -- 172.21.15.5:0/2850052090 >> 172.21.15.5:0/2850052090 conn(0x7f83dc0fd930 msgr2=0x7f83dc0ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:13.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.258+0000 7f83e2a00640 1 -- 172.21.15.5:0/2850052090 shutdown_connections 2024-04-24T06:44:13.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.258+0000 7f83e2a00640 1 -- 172.21.15.5:0/2850052090 wait complete. 2024-04-24T06:44:13.280 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-24T06:44:13.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.490+0000 7f70dc400640 1 -- 172.21.15.5:0/2039527461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70d4112ec0 msgr2=0x7f70d41152b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:13.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.490+0000 7f70dc400640 1 --2- 172.21.15.5:0/2039527461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70d4112ec0 0x7f70d41152b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f70cc009900 tx=0x7f70cc02a3f0 comp rx=0 tx=0).stop 2024-04-24T06:44:13.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.490+0000 7f70dc400640 1 -- 172.21.15.5:0/2039527461 shutdown_connections 2024-04-24T06:44:13.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.491+0000 7f70dc400640 1 --2- 172.21.15.5:0/2039527461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70d4112ec0 0x7f70d41152b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:13.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.491+0000 7f70dc400640 1 --2- 172.21.15.5:0/2039527461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70d4076e50 0x7f70d4112980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:13.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.491+0000 7f70dc400640 1 --2- 172.21.15.5:0/2039527461 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70d4076530 0x7f70d4076910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:13.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.491+0000 7f70dc400640 1 -- 172.21.15.5:0/2039527461 >> 172.21.15.5:0/2039527461 conn(0x7f70d40ffcb0 msgr2=0x7f70d41020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:13.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.491+0000 7f70dc400640 1 -- 172.21.15.5:0/2039527461 shutdown_connections 2024-04-24T06:44:13.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.491+0000 7f70dc400640 1 -- 172.21.15.5:0/2039527461 wait complete. 2024-04-24T06:44:13.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.492+0000 7f70dc400640 1 Processor -- start 2024-04-24T06:44:13.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.492+0000 7f70dc400640 1 -- start start 2024-04-24T06:44:13.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.492+0000 7f70dc400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70d4076530 0x7f70d419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:13.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.492+0000 7f70dc400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70d4076e50 0x7f70d419f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:13.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.492+0000 7f70dc400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70d4112ec0 0x7f70d41a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:13.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.492+0000 7f70dc400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f70d4118a70 con 0x7f70d4076530 2024-04-24T06:44:13.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.492+0000 7f70dc400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f70d41188f0 con 0x7f70d4112ec0 2024-04-24T06:44:13.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.492+0000 7f70dc400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f70d4118bf0 con 0x7f70d4076e50 2024-04-24T06:44:13.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.493+0000 7f70d9a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70d4076530 0x7f70d419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:13.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.493+0000 7f70da400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70d4112ec0 0x7f70d41a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:13.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.493+0000 7f70da400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70d4112ec0 0x7f70d41a3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:46422/0 (socket says 172.21.15.5:46422) 2024-04-24T06:44:13.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.493+0000 7f70da400640 1 -- 172.21.15.5:0/1537702569 learned_addr learned my addr 172.21.15.5:0/1537702569 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:13.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.493+0000 7f70d9000640 1 --2- 172.21.15.5:0/1537702569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70d4076e50 0x7f70d419f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:13.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.493+0000 7f70d9a00640 1 -- 172.21.15.5:0/1537702569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70d4076e50 msgr2=0x7f70d419f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:13.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.493+0000 7f70d9a00640 1 --2- 172.21.15.5:0/1537702569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70d4076e50 0x7f70d419f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:13.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.493+0000 7f70d9a00640 1 -- 172.21.15.5:0/1537702569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70d4112ec0 msgr2=0x7f70d41a3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:13.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.493+0000 7f70d9a00640 1 --2- 172.21.15.5:0/1537702569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70d4112ec0 0x7f70d41a3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:13.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.493+0000 7f70d9a00640 1 -- 172.21.15.5:0/1537702569 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d41a3ea0 con 0x7f70d4076530 2024-04-24T06:44:13.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.493+0000 7f70da400640 1 --2- 172.21.15.5:0/1537702569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70d4112ec0 0x7f70d41a3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:44:13.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.494+0000 7f70d9a00640 1 --2- 172.21.15.5:0/1537702569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70d4076530 0x7f70d419eeb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f70c800eb00 tx=0x7f70c800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:13.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.494+0000 7f70d2000640 1 -- 172.21.15.5:0/1537702569 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c8010430 con 0x7f70d4076530 2024-04-24T06:44:13.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.494+0000 7f70d2000640 1 -- 172.21.15.5:0/1537702569 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70c8010ab0 con 0x7f70d4076530 2024-04-24T06:44:13.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.494+0000 7f70d2000640 1 -- 172.21.15.5:0/1537702569 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c8017bb0 con 0x7f70d4076530 2024-04-24T06:44:13.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.494+0000 7f70dc400640 1 -- 172.21.15.5:0/1537702569 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70d41a4190 con 0x7f70d4076530 2024-04-24T06:44:13.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.494+0000 7f70dc400640 1 -- 172.21.15.5:0/1537702569 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70d41aba20 con 0x7f70d4076530 2024-04-24T06:44:13.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.495+0000 7f70dc400640 1 -- 172.21.15.5:0/1537702569 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70d40505e0 con 0x7f70d4076530 2024-04-24T06:44:13.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.498+0000 7f70d2000640 1 -- 172.21.15.5:0/1537702569 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 138872+0+0 (secure 0 0 0) 0x7f70c8010c60 con 0x7f70d4076530 2024-04-24T06:44:13.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.499+0000 7f70d2000640 1 --2- 172.21.15.5:0/1537702569 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f70bc0a3eb0 0x7f70bc0a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:13.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.499+0000 7f70d2000640 1 -- 172.21.15.5:0/1537702569 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f70c80d13e0 con 0x7f70d4076530 2024-04-24T06:44:13.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.499+0000 7f70d9000640 1 --2- 172.21.15.5:0/1537702569 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f70bc0a3eb0 0x7f70bc0a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:13.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.500+0000 7f70d9000640 1 --2- 172.21.15.5:0/1537702569 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f70bc0a3eb0 0x7f70bc0a6370 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f70d4102180 tx=0x7f70c0005f30 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:13.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.504+0000 7f70d2000640 1 -- 172.21.15.5:0/1537702569 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f70c809d7e0 con 0x7f70d4076530 2024-04-24T06:44:13.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:13 smithi137 ceph-mon[33017]: scrub queued for path: / 2024-04-24T06:44:13.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:13 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:44:13.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:13 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:44:13.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:13 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:44:13.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:13 smithi137 ceph-mon[33017]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 798 B/s wr, 17 op/s 2024-04-24T06:44:13.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:13 smithi137 ceph-mon[33017]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:44:13.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.735+0000 7f70dc400640 1 -- 172.21.15.5:0/1537702569 --> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f70d4062440 con 0x7f70bc0a3eb0 2024-04-24T06:44:13.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.749+0000 7f70d2000640 1 -- 172.21.15.5:0/1537702569 <== mgr.25202 v2:172.21.15.5:6800/3154624651 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f70d4062440 con 0x7f70bc0a3eb0 2024-04-24T06:44:13.750 INFO:teuthology.orchestra.run.smithi005.stdout:/ 1m 6m3h 2024-04-24T06:44:13.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.751+0000 7f70dc400640 1 -- 172.21.15.5:0/1537702569 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f70bc0a3eb0 msgr2=0x7f70bc0a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:13.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.751+0000 7f70dc400640 1 --2- 172.21.15.5:0/1537702569 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f70bc0a3eb0 0x7f70bc0a6370 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f70d4102180 tx=0x7f70c0005f30 comp rx=0 tx=0).stop 2024-04-24T06:44:13.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.752+0000 7f70dc400640 1 -- 172.21.15.5:0/1537702569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70d4076530 msgr2=0x7f70d419eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:13.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.752+0000 7f70dc400640 1 --2- 172.21.15.5:0/1537702569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70d4076530 0x7f70d419eeb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f70c800eb00 tx=0x7f70c800c2d0 comp rx=0 tx=0).stop 2024-04-24T06:44:13.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.752+0000 7f70dc400640 1 -- 172.21.15.5:0/1537702569 shutdown_connections 2024-04-24T06:44:13.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.752+0000 7f70dc400640 1 --2- 172.21.15.5:0/1537702569 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f70bc0a3eb0 0x7f70bc0a6370 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:13.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.752+0000 7f70dc400640 1 --2- 172.21.15.5:0/1537702569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70d4112ec0 0x7f70d41a3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:13.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.752+0000 7f70dc400640 1 --2- 172.21.15.5:0/1537702569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70d4076e50 0x7f70d419f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:13.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.752+0000 7f70dc400640 1 --2- 172.21.15.5:0/1537702569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70d4076530 0x7f70d419eeb0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:13.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.752+0000 7f70dc400640 1 -- 172.21.15.5:0/1537702569 >> 172.21.15.5:0/1537702569 conn(0x7f70d40ffcb0 msgr2=0x7f70d41144d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:13.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.752+0000 7f70dc400640 1 -- 172.21.15.5:0/1537702569 shutdown_connections 2024-04-24T06:44:13.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:13.752+0000 7f70dc400640 1 -- 172.21.15.5:0/1537702569 wait complete. 2024-04-24T06:44:13.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:13 smithi005 ceph-mon[27798]: scrub queued for path: / 2024-04-24T06:44:13.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:13 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:44:13.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:13 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:44:13.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:13 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:44:13.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:13 smithi005 ceph-mon[27798]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 798 B/s wr, 17 op/s 2024-04-24T06:44:13.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:13 smithi005 ceph-mon[27798]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:44:13.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:13 smithi037 ceph-mon[32864]: scrub queued for path: / 2024-04-24T06:44:13.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:13 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:44:13.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:13 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:44:13.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:13 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:44:13.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:13 smithi037 ceph-mon[32864]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 798 B/s wr, 17 op/s 2024-04-24T06:44:13.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:13 smithi037 ceph-mon[32864]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:44:13.773 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-24T06:44:13.865 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-24T06:44:13.875 INFO:teuthology.run_tasks:Running task workunit... 2024-04-24T06:44:13.888 INFO:tasks.workunit:Pulling workunits from ref a3c8a22bd5efe18ab230a8e2d072db8f381a0411 2024-04-24T06:44:13.889 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-24T06:44:13.889 INFO:tasks.workunit:timeout=3h 2024-04-24T06:44:13.889 INFO:tasks.workunit:cleanup=True 2024-04-24T06:44:13.889 DEBUG:teuthology.orchestra.run.smithi005:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-24T06:44:13.949 INFO:teuthology.orchestra.run.smithi005.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-24T06:44:13.949 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-24T06:44:13.950 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 72h/114d Inode: 1099511628283 Links: 2 2024-04-24T06:44:13.950 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T06:44:13.950 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-24T06:44:13.950 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-04-24 06:41:42.384374887 +0000 2024-04-24T06:44:13.950 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-04-24 06:41:42.384374887 +0000 2024-04-24T06:44:13.950 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-04-24 06:44:00.657363697 +0000 2024-04-24T06:44:13.950 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-04-24 06:41:42.384374887 +0000 2024-04-24T06:44:13.950 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-24T06:44:13.950 DEBUG:teuthology.orchestra.run.smithi005:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-24T06:44:14.056 DEBUG:teuthology.orchestra.run.smithi005:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout a3c8a22bd5efe18ab230a8e2d072db8f381a0411 2024-04-24T06:44:14.118 INFO:tasks.workunit.client.0.smithi005.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-24T06:44:14.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:14 smithi137 ceph-mon[33017]: from='client.25427 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:44:14.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:14 smithi137 ceph-mon[33017]: from='client.25433 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:44:14.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:14 smithi005 ceph-mon[27798]: from='client.25427 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:44:14.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:14 smithi005 ceph-mon[27798]: from='client.25433 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:44:14.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:14 smithi037 ceph-mon[32864]: from='client.25427 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:44:14.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:14 smithi037 ceph-mon[32864]: from='client.25433 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:44:15.658 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:15 smithi005 ceph-mon[27798]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:44:15.658 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:15 smithi005 ceph-mon[27798]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 49 op/s 2024-04-24T06:44:15.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:15 smithi137 ceph-mon[33017]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:44:15.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:15 smithi137 ceph-mon[33017]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 49 op/s 2024-04-24T06:44:15.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:15 smithi037 ceph-mon[32864]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:44:15.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:15 smithi037 ceph-mon[32864]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 49 op/s 2024-04-24T06:44:17.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:17 smithi137 ceph-mon[33017]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-04-24T06:44:17.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:17 smithi005 ceph-mon[27798]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-04-24T06:44:17.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:17 smithi037 ceph-mon[32864]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-04-24T06:44:19.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:19 smithi137 ceph-mon[33017]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-04-24T06:44:19.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:19 smithi005 ceph-mon[27798]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-04-24T06:44:19.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:19 smithi037 ceph-mon[32864]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-04-24T06:44:21.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:21 smithi137 ceph-mon[33017]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-04-24T06:44:21.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:21 smithi005 ceph-mon[27798]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-04-24T06:44:21.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:21 smithi037 ceph-mon[32864]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-04-24T06:44:23.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:23 smithi137 ceph-mon[33017]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-04-24T06:44:23.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:23 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:44:23.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:23 smithi005 ceph-mon[27798]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-04-24T06:44:23.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:23 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:44:23.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:23 smithi037 ceph-mon[32864]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-04-24T06:44:23.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:23 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:44:25.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:25 smithi137 ceph-mon[33017]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 39 op/s 2024-04-24T06:44:25.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:25 smithi005 ceph-mon[27798]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 39 op/s 2024-04-24T06:44:25.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:25 smithi037 ceph-mon[32864]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 39 op/s 2024-04-24T06:44:27.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:27 smithi137 ceph-mon[33017]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-24T06:44:27.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:27 smithi005 ceph-mon[27798]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-24T06:44:27.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:27 smithi037 ceph-mon[32864]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-24T06:44:29.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:29 smithi137 ceph-mon[33017]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-24T06:44:29.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:29 smithi005 ceph-mon[27798]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-24T06:44:29.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:29 smithi037 ceph-mon[32864]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-24T06:44:31.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:31 smithi137 ceph-mon[33017]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-24T06:44:31.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:31 smithi005 ceph-mon[27798]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-24T06:44:31.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:31 smithi037 ceph-mon[32864]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-24T06:44:33.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:33 smithi137 ceph-mon[33017]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:44:33.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:33 smithi005 ceph-mon[27798]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:44:33.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:33 smithi037 ceph-mon[32864]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:44:35.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:35 smithi137 ceph-mon[33017]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:44:35.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:35 smithi005 ceph-mon[27798]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:44:35.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:35 smithi037 ceph-mon[32864]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:44:37.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:37 smithi137 ceph-mon[33017]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:44:37.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:37 smithi005 ceph-mon[27798]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:44:37.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:37 smithi037 ceph-mon[32864]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:44:39.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:39 smithi137 ceph-mon[33017]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T06:44:39.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:39 smithi005 ceph-mon[27798]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T06:44:39.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:39 smithi037 ceph-mon[32864]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-24T06:44:41.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:41 smithi137 ceph-mon[33017]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:44:41.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:41 smithi005 ceph-mon[27798]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:44:41.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:41 smithi037 ceph-mon[32864]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:44:42.104 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T06:44:42.105 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T06:44:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.333+0000 7f4d61400640 1 -- 172.21.15.5:0/2182205554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d5c074cc0 msgr2=0x7f4d5c1128d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.333+0000 7f4d61400640 1 --2- 172.21.15.5:0/2182205554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d5c074cc0 0x7f4d5c1128d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009be0 tx=0x7f4d4402a750 comp rx=0 tx=0).stop 2024-04-24T06:44:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.334+0000 7f4d61400640 1 -- 172.21.15.5:0/2182205554 shutdown_connections 2024-04-24T06:44:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.334+0000 7f4d61400640 1 --2- 172.21.15.5:0/2182205554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d5c112e10 0x7f4d5c115200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.334+0000 7f4d61400640 1 --2- 172.21.15.5:0/2182205554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d5c074cc0 0x7f4d5c1128d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.334+0000 7f4d61400640 1 --2- 172.21.15.5:0/2182205554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d5c0743a0 0x7f4d5c074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.334+0000 7f4d61400640 1 -- 172.21.15.5:0/2182205554 >> 172.21.15.5:0/2182205554 conn(0x7f4d5c0fd950 msgr2=0x7f4d5c0ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:42.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.334+0000 7f4d61400640 1 -- 172.21.15.5:0/2182205554 shutdown_connections 2024-04-24T06:44:42.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.334+0000 7f4d61400640 1 -- 172.21.15.5:0/2182205554 wait complete. 2024-04-24T06:44:42.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.335+0000 7f4d61400640 1 Processor -- start 2024-04-24T06:44:42.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.335+0000 7f4d61400640 1 -- start start 2024-04-24T06:44:42.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.336+0000 7f4d61400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d5c0743a0 0x7f4d5c19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:42.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.336+0000 7f4d61400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d5c074cc0 0x7f4d5c19f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:42.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.336+0000 7f4d61400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d5c112e10 0x7f4d5c1a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:42.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.336+0000 7f4d61400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5c118790 con 0x7f4d5c112e10 2024-04-24T06:44:42.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.336+0000 7f4d61400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5c118610 con 0x7f4d5c074cc0 2024-04-24T06:44:42.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.336+0000 7f4d61400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5c118910 con 0x7f4d5c0743a0 2024-04-24T06:44:42.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.336+0000 7f4d60a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d5c112e10 0x7f4d5c1a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:42.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.336+0000 7f4d60a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d5c112e10 0x7f4d5c1a3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55116/0 (socket says 172.21.15.5:55116) 2024-04-24T06:44:42.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.336+0000 7f4d60a00640 1 -- 172.21.15.5:0/937857062 learned_addr learned my addr 172.21.15.5:0/937857062 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:42.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.336+0000 7f4d5be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d5c0743a0 0x7f4d5c19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:42.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.336+0000 7f4d5b400640 1 --2- 172.21.15.5:0/937857062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d5c074cc0 0x7f4d5c19f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:42.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.337+0000 7f4d5be00640 1 -- 172.21.15.5:0/937857062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d5c074cc0 msgr2=0x7f4d5c19f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:42.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.337+0000 7f4d5be00640 1 --2- 172.21.15.5:0/937857062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d5c074cc0 0x7f4d5c19f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.337+0000 7f4d5be00640 1 -- 172.21.15.5:0/937857062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d5c112e10 msgr2=0x7f4d5c1a3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:42.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.337+0000 7f4d5be00640 1 --2- 172.21.15.5:0/937857062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d5c112e10 0x7f4d5c1a3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.337+0000 7f4d5be00640 1 -- 172.21.15.5:0/937857062 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d5c1a3d60 con 0x7f4d5c0743a0 2024-04-24T06:44:42.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.337+0000 7f4d60a00640 1 --2- 172.21.15.5:0/937857062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d5c112e10 0x7f4d5c1a3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:44:42.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.337+0000 7f4d5be00640 1 --2- 172.21.15.5:0/937857062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d5c0743a0 0x7f4d5c19ed70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c005fd0 tx=0x7f4d4c00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:42.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.337+0000 7f4d58c00640 1 -- 172.21.15.5:0/937857062 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c00cd00 con 0x7f4d5c0743a0 2024-04-24T06:44:42.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.337+0000 7f4d61400640 1 -- 172.21.15.5:0/937857062 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d5c1a4050 con 0x7f4d5c0743a0 2024-04-24T06:44:42.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.337+0000 7f4d58c00640 1 -- 172.21.15.5:0/937857062 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d4c0035f0 con 0x7f4d5c0743a0 2024-04-24T06:44:42.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.337+0000 7f4d58c00640 1 -- 172.21.15.5:0/937857062 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c010690 con 0x7f4d5c0743a0 2024-04-24T06:44:42.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.338+0000 7f4d61400640 1 -- 172.21.15.5:0/937857062 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d5c1ab8e0 con 0x7f4d5c0743a0 2024-04-24T06:44:42.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.341+0000 7f4d659a0740 1 Processor -- start 2024-04-24T06:44:42.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.341+0000 7f4d659a0740 1 -- start start 2024-04-24T06:44:42.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.341+0000 7f4d58c00640 1 -- 172.21.15.5:0/937857062 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 138872+0+0 (secure 0 0 0) 0x7f4d4c0037a0 con 0x7f4d5c0743a0 2024-04-24T06:44:42.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.341+0000 7f4d659a0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffcf448a30 0x55ffcf448e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:42.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.341+0000 7f4d659a0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffcf449350 0x55ffcf4534e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:42.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.341+0000 7f4d659a0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffcf453a20 0x55ffcf455e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:42.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.341+0000 7f4d659a0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ffcf3ae1f0 con 0x55ffcf453a20 2024-04-24T06:44:42.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.341+0000 7f4d659a0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ffcf354350 con 0x55ffcf448a30 2024-04-24T06:44:42.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.341+0000 7f4d659a0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ffcf3aa1b0 con 0x55ffcf449350 2024-04-24T06:44:42.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.342+0000 7f4d5b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffcf453a20 0x55ffcf455e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:42.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.342+0000 7f4d60a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffcf448a30 0x55ffcf448e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:42.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.342+0000 7f4d5b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffcf453a20 0x55ffcf455e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55122/0 (socket says 172.21.15.5:55122) 2024-04-24T06:44:42.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.342+0000 7f4d60a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffcf448a30 0x55ffcf448e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:56678/0 (socket says 172.21.15.5:56678) 2024-04-24T06:44:42.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.342+0000 7f4d5b400640 1 -- 172.21.15.5:0/2048776741 learned_addr learned my addr 172.21.15.5:0/2048776741 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:42.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.342+0000 7f4d5be00640 1 --2- 172.21.15.5:0/2048776741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffcf449350 0x55ffcf4534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:42.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.342+0000 7f4d60a00640 1 -- 172.21.15.5:0/2048776741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffcf449350 msgr2=0x55ffcf4534e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:42.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.342+0000 7f4d60a00640 1 --2- 172.21.15.5:0/2048776741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffcf449350 0x55ffcf4534e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.342+0000 7f4d60a00640 1 -- 172.21.15.5:0/2048776741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffcf453a20 msgr2=0x55ffcf455e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:42.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.342+0000 7f4d60a00640 1 --2- 172.21.15.5:0/2048776741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffcf453a20 0x55ffcf455e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.342+0000 7f4d60a00640 1 -- 172.21.15.5:0/2048776741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffcf3a8f50 con 0x55ffcf448a30 2024-04-24T06:44:42.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.343+0000 7f4d60a00640 1 --2- 172.21.15.5:0/2048776741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffcf448a30 0x55ffcf448e10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4d54009f70 tx=0x7f4d540118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a67585f841d649 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:42.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.343+0000 7f4d58c00640 1 --2- 172.21.15.5:0/937857062 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f4d3c0a3e40 0x7f4d3c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:42.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.343+0000 7f4d51200640 1 -- 172.21.15.5:0/2048776741 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5401e070 con 0x55ffcf448a30 2024-04-24T06:44:42.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.343+0000 7f4d51200640 1 -- 172.21.15.5:0/2048776741 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d5400fa20 con 0x55ffcf448a30 2024-04-24T06:44:42.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.343+0000 7f4d51200640 1 -- 172.21.15.5:0/2048776741 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d54019b80 con 0x55ffcf448a30 2024-04-24T06:44:42.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.343+0000 7f4d5be00640 1 --2- 172.21.15.5:0/937857062 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f4d3c0a3e40 0x7f4d3c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:42.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.343+0000 7f4d58c00640 1 -- 172.21.15.5:0/937857062 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f4d4c0d0d30 con 0x7f4d5c0743a0 2024-04-24T06:44:42.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.344+0000 7f4d659a0740 1 -- 172.21.15.5:0/2048776741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffcf448a30 msgr2=0x55ffcf448e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:42.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.344+0000 7f4d659a0740 1 --2- 172.21.15.5:0/2048776741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffcf448a30 0x55ffcf448e10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4d54009f70 tx=0x7f4d540118d0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.344+0000 7f4d5be00640 1 --2- 172.21.15.5:0/937857062 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f4d3c0a3e40 0x7f4d3c0a6300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c0ce310 tx=0x7f4d4c009f70 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:42.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.344+0000 7f4d659a0740 1 -- 172.21.15.5:0/2048776741 shutdown_connections 2024-04-24T06:44:42.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.344+0000 7f4d659a0740 1 --2- 172.21.15.5:0/2048776741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffcf453a20 0x55ffcf455e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.344+0000 7f4d659a0740 1 --2- 172.21.15.5:0/2048776741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffcf449350 0x55ffcf4534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.344+0000 7f4d659a0740 1 --2- 172.21.15.5:0/2048776741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffcf448a30 0x55ffcf448e10 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.344+0000 7f4d659a0740 1 -- 172.21.15.5:0/2048776741 >> 172.21.15.5:0/2048776741 conn(0x55ffcf444d50 msgr2=0x55ffcf445130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:42.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.344+0000 7f4d659a0740 1 -- 172.21.15.5:0/2048776741 shutdown_connections 2024-04-24T06:44:42.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.344+0000 7f4d659a0740 1 -- 172.21.15.5:0/2048776741 wait complete. 2024-04-24T06:44:42.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.344+0000 7f4d659a0740 1 -- start start 2024-04-24T06:44:42.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.345+0000 7f4d659a0740 1 Processor -- start 2024-04-24T06:44:42.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.345+0000 7f4d659a0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffcf448a30 0x55ffcf4509e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:42.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.345+0000 7f4d659a0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffcf449350 0x55ffcf450f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:42.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.345+0000 7f4d659a0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffcf453a20 0x55ffcf451460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:42.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.345+0000 7f4d659a0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ffcf456810 con 0x55ffcf453a20 2024-04-24T06:44:42.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.345+0000 7f4d659a0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ffcf367690 con 0x55ffcf449350 2024-04-24T06:44:42.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.345+0000 7f4d659a0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ffcf456990 con 0x55ffcf448a30 2024-04-24T06:44:42.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.345+0000 7f4d659a0740 10 client.0 _ll_register_callbacks cb 0x55ffcf3a3650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:44:42.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.346+0000 7f4d5b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffcf449350 0x55ffcf450f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:42.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.346+0000 7f4d60a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffcf453a20 0x55ffcf451460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:42.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.346+0000 7f4d5b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffcf449350 0x55ffcf450f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:56686/0 (socket says 172.21.15.5:56686) 2024-04-24T06:44:42.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.346+0000 7f4d60a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffcf453a20 0x55ffcf451460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55132/0 (socket says 172.21.15.5:55132) 2024-04-24T06:44:42.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.346+0000 7f4d5b400640 1 -- 172.21.15.5:0/1671978420 learned_addr learned my addr 172.21.15.5:0/1671978420 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:42.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.346+0000 7f4d60a00640 1 --2- 172.21.15.5:0/1671978420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffcf448a30 0x55ffcf4509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:42.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.346+0000 7f4d5b400640 1 -- 172.21.15.5:0/1671978420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffcf448a30 msgr2=0x55ffcf4509e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:42.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.346+0000 7f4d5b400640 1 --2- 172.21.15.5:0/1671978420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffcf448a30 0x55ffcf4509e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.346+0000 7f4d5b400640 1 -- 172.21.15.5:0/1671978420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffcf453a20 msgr2=0x55ffcf451460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:42.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.346+0000 7f4d5b400640 1 --2- 172.21.15.5:0/1671978420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffcf453a20 0x55ffcf451460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.346+0000 7f4d5b400640 1 -- 172.21.15.5:0/1671978420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffcf366920 con 0x55ffcf449350 2024-04-24T06:44:42.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.346+0000 7f4d5b400640 1 --2- 172.21.15.5:0/1671978420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffcf449350 0x55ffcf450f20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4d44033000 tx=0x7f4d44030c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa10c0583b475d12 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:42.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.346+0000 7f4d49600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:44:42.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.347+0000 7f4d49600640 1 -- 172.21.15.5:0/1671978420 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d44036070 con 0x55ffcf449350 2024-04-24T06:44:42.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.347+0000 7f4d49600640 1 -- 172.21.15.5:0/1671978420 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f4d440303a0 con 0x55ffcf449350 2024-04-24T06:44:42.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.347+0000 7f4d659a0740 1 -- 172.21.15.5:0/1671978420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d5400fa20 con 0x55ffcf449350 2024-04-24T06:44:42.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.347+0000 7f4d49600640 1 -- 172.21.15.5:0/1671978420 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f4d440305a0 con 0x55ffcf449350 2024-04-24T06:44:42.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.347+0000 7f4d659a0740 10 client.25451 fetch_fsmap learned FSMap version 23 2024-04-24T06:44:42.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.347+0000 7f4d659a0740 1 -- 172.21.15.5:0/1671978420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55ffcf18a7d0 con 0x55ffcf449350 2024-04-24T06:44:42.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.348+0000 7f4d49600640 1 -- 172.21.15.5:0/1671978420 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d5400fa20 con 0x55ffcf449350 2024-04-24T06:44:42.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.348+0000 7f4d49600640 1 -- 172.21.15.5:0/1671978420 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f4d4403c040 con 0x55ffcf449350 2024-04-24T06:44:42.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.348+0000 7f4d659a0740 10 client.25451 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T06:44:42.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.348+0000 7f4d659a0740 10 client.25451 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:44:42.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.348+0000 7f4d659a0740 20 client.25451 populate_metadata read hostname 'smithi005' 2024-04-24T06:44:42.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.348+0000 7f4d659a0740 1 --2- 172.21.15.5:0/1671978420 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55ffcf463e20 0x55ffcf466260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:42.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.348+0000 7f4d659a0740 4 client.25451 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:44:42.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.348+0000 7f4d659a0740 1 -- 172.21.15.5:0/1671978420 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffcf3bbc10 con 0x55ffcf463e20 2024-04-24T06:44:42.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.348+0000 7f4d60a00640 1 --2- 172.21.15.5:0/1671978420 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55ffcf463e20 0x55ffcf466260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:42.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.349+0000 7f4d60a00640 1 --2- 172.21.15.5:0/1671978420 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55ffcf463e20 0x55ffcf466260 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=662d18359c6e1104 server_cookie=90725a0ac52407e4 in_seq=0 out_seq=0 2024-04-24T06:44:42.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.349+0000 7f4d49600640 10 client.25451 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:44:42.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.350+0000 7f4d49600640 1 -- 172.21.15.5:0/1671978420 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ffcf3bbc10 con 0x55ffcf463e20 2024-04-24T06:44:42.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.350+0000 7f4d49600640 10 client.25451 handle_command_reply: tid=0 2024-04-24T06:44:42.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.350+0000 7f4d659a0740 1 client.25451 shutdown 2024-04-24T06:44:42.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.350+0000 7f4d659a0740 1 -- 172.21.15.5:0/1671978420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffcf449350 msgr2=0x55ffcf450f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:42.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.350+0000 7f4d659a0740 1 --2- 172.21.15.5:0/1671978420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffcf449350 0x55ffcf450f20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4d44033000 tx=0x7f4d44030c60 comp rx=0 tx=0).stop 2024-04-24T06:44:42.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.350+0000 7f4d659a0740 1 -- 172.21.15.5:0/1671978420 shutdown_connections 2024-04-24T06:44:42.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.350+0000 7f4d659a0740 1 --2- 172.21.15.5:0/1671978420 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55ffcf463e20 0x55ffcf466260 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.350+0000 7f4d659a0740 1 --2- 172.21.15.5:0/1671978420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffcf453a20 0x55ffcf451460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.350+0000 7f4d49600640 0 client.25451 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:44:42.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.350+0000 7f4d659a0740 1 --2- 172.21.15.5:0/1671978420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffcf449350 0x55ffcf450f20 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.350+0000 7f4d659a0740 1 --2- 172.21.15.5:0/1671978420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffcf448a30 0x55ffcf4509e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.350+0000 7f4d659a0740 1 -- 172.21.15.5:0/1671978420 >> 172.21.15.5:0/1671978420 conn(0x55ffcf444d50 msgr2=0x55ffcf44ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:42.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.350+0000 7f4d659a0740 1 -- 172.21.15.5:0/1671978420 shutdown_connections 2024-04-24T06:44:42.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.350+0000 7f4d659a0740 1 -- 172.21.15.5:0/1671978420 wait complete. 2024-04-24T06:44:42.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.351+0000 7f4d659a0740 20 client.25451 trim_cache size 0 max 16384 2024-04-24T06:44:42.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.370+0000 7f4d659a0740 1 Processor -- start 2024-04-24T06:44:42.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.370+0000 7f4d659a0740 1 -- start start 2024-04-24T06:44:42.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.370+0000 7f4d659a0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffcf448a30 0x55ffcf451280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:42.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.370+0000 7f4d659a0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffcf449350 0x55ffcf46bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:42.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.370+0000 7f4d659a0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffcf453a20 0x55ffcf46c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:42.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.370+0000 7f4d659a0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5c1b6b00 con 0x55ffcf449350 2024-04-24T06:44:42.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.370+0000 7f4d659a0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5c1b43d0 con 0x55ffcf453a20 2024-04-24T06:44:42.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.370+0000 7f4d659a0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5c1b2220 con 0x55ffcf448a30 2024-04-24T06:44:42.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.371+0000 7f4d5b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffcf449350 0x55ffcf46bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:42.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.371+0000 7f4d60a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffcf453a20 0x55ffcf46c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:42.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.371+0000 7f4d5b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffcf449350 0x55ffcf46bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55136/0 (socket says 172.21.15.5:55136) 2024-04-24T06:44:42.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.371+0000 7f4d5b400640 1 -- 172.21.15.5:0/362426691 learned_addr learned my addr 172.21.15.5:0/362426691 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:42.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.371+0000 7f4d60a00640 1 --2- 172.21.15.5:0/362426691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffcf448a30 0x55ffcf451280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:42.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.371+0000 7f4d60a00640 1 -- 172.21.15.5:0/362426691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffcf448a30 msgr2=0x55ffcf451280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:42.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.371+0000 7f4d60a00640 1 --2- 172.21.15.5:0/362426691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffcf448a30 0x55ffcf451280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.371+0000 7f4d60a00640 1 -- 172.21.15.5:0/362426691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffcf449350 msgr2=0x55ffcf46bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:42.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.371+0000 7f4d60a00640 1 --2- 172.21.15.5:0/362426691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffcf449350 0x55ffcf46bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.371+0000 7f4d60a00640 1 -- 172.21.15.5:0/362426691 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffcf0c03f0 con 0x55ffcf453a20 2024-04-24T06:44:42.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.371+0000 7f4d60a00640 1 --2- 172.21.15.5:0/362426691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffcf453a20 0x55ffcf46c070 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4d54009db0 tx=0x7f4d54019760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a526ee0fa9157593 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:42.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.372+0000 7f4d4be00640 1 -- 172.21.15.5:0/362426691 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5401e040 con 0x55ffcf453a20 2024-04-24T06:44:42.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.372+0000 7f4d4be00640 1 -- 172.21.15.5:0/362426691 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d5402c030 con 0x55ffcf453a20 2024-04-24T06:44:42.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.372+0000 7f4d4be00640 1 -- 172.21.15.5:0/362426691 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d54005040 con 0x55ffcf453a20 2024-04-24T06:44:42.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.372+0000 7f4d659a0740 1 -- 172.21.15.5:0/362426691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffcf453a20 msgr2=0x55ffcf46c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:42.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.372+0000 7f4d659a0740 1 --2- 172.21.15.5:0/362426691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffcf453a20 0x55ffcf46c070 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4d54009db0 tx=0x7f4d54019760 comp rx=0 tx=0).stop 2024-04-24T06:44:42.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.372+0000 7f4d659a0740 1 -- 172.21.15.5:0/362426691 shutdown_connections 2024-04-24T06:44:42.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.372+0000 7f4d659a0740 1 --2- 172.21.15.5:0/362426691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffcf453a20 0x55ffcf46c070 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.372+0000 7f4d659a0740 1 --2- 172.21.15.5:0/362426691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffcf449350 0x55ffcf46bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.372+0000 7f4d659a0740 1 --2- 172.21.15.5:0/362426691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffcf448a30 0x55ffcf451280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.372+0000 7f4d659a0740 1 -- 172.21.15.5:0/362426691 >> 172.21.15.5:0/362426691 conn(0x55ffcf444d50 msgr2=0x55ffcf443740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:42.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.373+0000 7f4d659a0740 1 -- 172.21.15.5:0/362426691 shutdown_connections 2024-04-24T06:44:42.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.373+0000 7f4d659a0740 1 -- 172.21.15.5:0/362426691 wait complete. 2024-04-24T06:44:42.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.373+0000 7f4d659a0740 1 -- start start 2024-04-24T06:44:42.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.373+0000 7f4d659a0740 1 Processor -- start 2024-04-24T06:44:42.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.374+0000 7f4d659a0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffcf448a30 0x55ffcf4579a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:42.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.374+0000 7f4d659a0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffcf449350 0x55ffcf457ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:42.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.374+0000 7f4d659a0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffcf453a20 0x55ffcf4c93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:42.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.374+0000 7f4d659a0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5c1b2890 con 0x55ffcf449350 2024-04-24T06:44:42.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.374+0000 7f4d659a0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5c1b3d00 con 0x55ffcf453a20 2024-04-24T06:44:42.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.374+0000 7f4d659a0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5c1b2f60 con 0x55ffcf448a30 2024-04-24T06:44:42.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.374+0000 7f4d659a0740 10 client.0 _ll_register_callbacks cb 0x7f4d54005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:44:42.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.374+0000 7f4d5b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffcf449350 0x55ffcf457ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:42.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.374+0000 7f4d5b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffcf449350 0x55ffcf457ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55142/0 (socket says 172.21.15.5:55142) 2024-04-24T06:44:42.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.374+0000 7f4d5b400640 1 -- 172.21.15.5:0/2868818033 learned_addr learned my addr 172.21.15.5:0/2868818033 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:42.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.374+0000 7f4d60a00640 1 --2- 172.21.15.5:0/2868818033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffcf448a30 0x55ffcf4579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:42.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.374+0000 7f4d60a00640 1 --2- 172.21.15.5:0/2868818033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffcf453a20 0x55ffcf4c93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:42.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.374+0000 7f4d5b400640 1 -- 172.21.15.5:0/2868818033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffcf448a30 msgr2=0x55ffcf4579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:42.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.374+0000 7f4d5b400640 1 --2- 172.21.15.5:0/2868818033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffcf448a30 0x55ffcf4579a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.374+0000 7f4d5b400640 1 -- 172.21.15.5:0/2868818033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffcf453a20 msgr2=0x55ffcf4c93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:42.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.374+0000 7f4d5b400640 1 --2- 172.21.15.5:0/2868818033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffcf453a20 0x55ffcf4c93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.374+0000 7f4d5b400640 1 -- 172.21.15.5:0/2868818033 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffcf0800f0 con 0x55ffcf449350 2024-04-24T06:44:42.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.375+0000 7f4d5b400640 1 --2- 172.21.15.5:0/2868818033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffcf449350 0x55ffcf457ee0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009b00 tx=0x7f4d44031670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d97ea6dab33b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:42.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.375+0000 7f4d49600640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:44:42.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.375+0000 7f4d49600640 1 -- 172.21.15.5:0/2868818033 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d44036070 con 0x55ffcf449350 2024-04-24T06:44:42.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.375+0000 7f4d659a0740 1 -- 172.21.15.5:0/2868818033 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d5402c030 con 0x55ffcf449350 2024-04-24T06:44:42.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.375+0000 7f4d49600640 1 -- 172.21.15.5:0/2868818033 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f4d440303a0 con 0x55ffcf449350 2024-04-24T06:44:42.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.375+0000 7f4d49600640 1 -- 172.21.15.5:0/2868818033 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f4d4405f050 con 0x55ffcf449350 2024-04-24T06:44:42.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.375+0000 7f4d659a0740 10 client.15357 fetch_fsmap learned FSMap version 23 2024-04-24T06:44:42.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.375+0000 7f4d659a0740 1 -- 172.21.15.5:0/2868818033 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55ffcf373810 con 0x55ffcf449350 2024-04-24T06:44:42.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.376+0000 7f4d49600640 1 -- 172.21.15.5:0/2868818033 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d5402c030 con 0x55ffcf449350 2024-04-24T06:44:42.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.376+0000 7f4d49600640 1 -- 172.21.15.5:0/2868818033 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f4d44048430 con 0x55ffcf449350 2024-04-24T06:44:42.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.376+0000 7f4d659a0740 10 client.15357 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T06:44:42.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.376+0000 7f4d659a0740 10 client.15357 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:44:42.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.376+0000 7f4d659a0740 20 client.15357 populate_metadata read hostname 'smithi005' 2024-04-24T06:44:42.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.376+0000 7f4d659a0740 1 --2- 172.21.15.5:0/2868818033 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55ffcf4cae40 0x55ffcf4cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:42.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.376+0000 7f4d659a0740 4 client.15357 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:44:42.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.376+0000 7f4d659a0740 1 -- 172.21.15.5:0/2868818033 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ffcf3b5a80 con 0x55ffcf4cae40 2024-04-24T06:44:42.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.376+0000 7f4d60a00640 1 --2- 172.21.15.5:0/2868818033 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55ffcf4cae40 0x55ffcf4cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:42.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.377+0000 7f4d60a00640 1 --2- 172.21.15.5:0/2868818033 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55ffcf4cae40 0x55ffcf4cd260 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c4f66b89088732 server_cookie=c2f29e5b012dafc3 in_seq=0 out_seq=0 2024-04-24T06:44:42.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.377+0000 7f4d49600640 10 client.15357 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:44:42.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.377+0000 7f4d49600640 1 -- 172.21.15.5:0/2868818033 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ffcf3b5a80 con 0x55ffcf4cae40 2024-04-24T06:44:42.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.377+0000 7f4d49600640 10 client.15357 handle_command_reply: tid=0 2024-04-24T06:44:42.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.377+0000 7f4d659a0740 1 client.15357 shutdown 2024-04-24T06:44:42.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.377+0000 7f4d659a0740 1 -- 172.21.15.5:0/2868818033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffcf449350 msgr2=0x55ffcf457ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:42.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.377+0000 7f4d659a0740 1 --2- 172.21.15.5:0/2868818033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffcf449350 0x55ffcf457ee0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009b00 tx=0x7f4d44031670 comp rx=0 tx=0).stop 2024-04-24T06:44:42.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.377+0000 7f4d659a0740 1 -- 172.21.15.5:0/2868818033 shutdown_connections 2024-04-24T06:44:42.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.377+0000 7f4d659a0740 1 --2- 172.21.15.5:0/2868818033 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55ffcf4cae40 0x55ffcf4cd260 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.377+0000 7f4d659a0740 1 --2- 172.21.15.5:0/2868818033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ffcf453a20 0x55ffcf4c93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.377+0000 7f4d659a0740 1 --2- 172.21.15.5:0/2868818033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffcf449350 0x55ffcf457ee0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.377+0000 7f4d659a0740 1 --2- 172.21.15.5:0/2868818033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffcf448a30 0x55ffcf4579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.378+0000 7f4d49600640 0 client.15357 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:44:42.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.378+0000 7f4d659a0740 1 -- 172.21.15.5:0/2868818033 >> 172.21.15.5:0/2868818033 conn(0x55ffcf444d50 msgr2=0x55ffcf443fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:42.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.378+0000 7f4d659a0740 1 -- 172.21.15.5:0/2868818033 shutdown_connections 2024-04-24T06:44:42.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.378+0000 7f4d659a0740 1 -- 172.21.15.5:0/2868818033 wait complete. 2024-04-24T06:44:42.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.378+0000 7f4d659a0740 20 client.15357 trim_cache size 0 max 16384 2024-04-24T06:44:42.389 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:44:42.389 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-24T06:44:42.389 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-24T06:44:42.389 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:44:42.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.378+0000 7f4d61400640 1 -- 172.21.15.5:0/937857062 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f4d3c0a3e40 msgr2=0x7f4d3c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:42.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.378+0000 7f4d61400640 1 --2- 172.21.15.5:0/937857062 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f4d3c0a3e40 0x7f4d3c0a6300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c0ce310 tx=0x7f4d4c009f70 comp rx=0 tx=0).stop 2024-04-24T06:44:42.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.378+0000 7f4d61400640 1 -- 172.21.15.5:0/937857062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d5c0743a0 msgr2=0x7f4d5c19ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:42.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.378+0000 7f4d61400640 1 --2- 172.21.15.5:0/937857062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d5c0743a0 0x7f4d5c19ed70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c005fd0 tx=0x7f4d4c00ee50 comp rx=0 tx=0).stop 2024-04-24T06:44:42.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.379+0000 7f4d61400640 1 -- 172.21.15.5:0/937857062 shutdown_connections 2024-04-24T06:44:42.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.379+0000 7f4d61400640 1 --2- 172.21.15.5:0/937857062 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f4d3c0a3e40 0x7f4d3c0a6300 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.379+0000 7f4d61400640 1 --2- 172.21.15.5:0/937857062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d5c112e10 0x7f4d5c1a3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.379+0000 7f4d61400640 1 --2- 172.21.15.5:0/937857062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d5c074cc0 0x7f4d5c19f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.379+0000 7f4d61400640 1 --2- 172.21.15.5:0/937857062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d5c0743a0 0x7f4d5c19ed70 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.379+0000 7f4d61400640 1 -- 172.21.15.5:0/937857062 >> 172.21.15.5:0/937857062 conn(0x7f4d5c0fd950 msgr2=0x7f4d5c0fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:42.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.379+0000 7f4d61400640 1 -- 172.21.15.5:0/937857062 shutdown_connections 2024-04-24T06:44:42.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.379+0000 7f4d61400640 1 -- 172.21.15.5:0/937857062 wait complete. 2024-04-24T06:44:42.399 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T06:44:42.400 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:44:42.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.612+0000 7f74f5a00640 1 -- 172.21.15.5:0/1536257603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74f01039b0 msgr2=0x7f74f010be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:42.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.612+0000 7f74f5a00640 1 --2- 172.21.15.5:0/1536257603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74f01039b0 0x7f74f010be90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f74d8009ca0 tx=0x7f74d802a940 comp rx=0 tx=0).stop 2024-04-24T06:44:42.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.613+0000 7f74f5a00640 1 -- 172.21.15.5:0/1536257603 shutdown_connections 2024-04-24T06:44:42.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.613+0000 7f74f5a00640 1 --2- 172.21.15.5:0/1536257603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74f010c4b0 0x7f74f010e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.613+0000 7f74f5a00640 1 --2- 172.21.15.5:0/1536257603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74f01039b0 0x7f74f010be90 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.613+0000 7f74f5a00640 1 --2- 172.21.15.5:0/1536257603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74f0103090 0x7f74f0103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.613+0000 7f74f5a00640 1 -- 172.21.15.5:0/1536257603 >> 172.21.15.5:0/1536257603 conn(0x7f74f00fc890 msgr2=0x7f74f00fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:42.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.613+0000 7f74f5a00640 1 -- 172.21.15.5:0/1536257603 shutdown_connections 2024-04-24T06:44:42.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.613+0000 7f74f5a00640 1 -- 172.21.15.5:0/1536257603 wait complete. 2024-04-24T06:44:42.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.614+0000 7f74f5a00640 1 Processor -- start 2024-04-24T06:44:42.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.614+0000 7f74f5a00640 1 -- start start 2024-04-24T06:44:42.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.615+0000 7f74f5a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74f0103090 0x7f74f01a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:42.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.615+0000 7f74f5a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74f01039b0 0x7f74f01a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:42.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.615+0000 7f74f5a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74f010c4b0 0x7f74f019a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:42.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.615+0000 7f74f5a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f74f01122c0 con 0x7f74f010c4b0 2024-04-24T06:44:42.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.615+0000 7f74f5a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f74f0112140 con 0x7f74f01039b0 2024-04-24T06:44:42.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.615+0000 7f74f5a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f74f0112440 con 0x7f74f0103090 2024-04-24T06:44:42.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.615+0000 7f74efe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74f0103090 0x7f74f01a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:42.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.615+0000 7f74efe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74f0103090 0x7f74f01a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:37472/0 (socket says 172.21.15.5:37472) 2024-04-24T06:44:42.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.615+0000 7f74efe00640 1 -- 172.21.15.5:0/1097170729 learned_addr learned my addr 172.21.15.5:0/1097170729 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:42.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.615+0000 7f74ef400640 1 --2- 172.21.15.5:0/1097170729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74f01039b0 0x7f74f01a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:42.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.615+0000 7f74f5000640 1 --2- 172.21.15.5:0/1097170729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74f010c4b0 0x7f74f019a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:42.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.615+0000 7f74efe00640 1 -- 172.21.15.5:0/1097170729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74f01039b0 msgr2=0x7f74f01a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:42.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.615+0000 7f74efe00640 1 --2- 172.21.15.5:0/1097170729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74f01039b0 0x7f74f01a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.615+0000 7f74efe00640 1 -- 172.21.15.5:0/1097170729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74f010c4b0 msgr2=0x7f74f019a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:42.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.615+0000 7f74efe00640 1 --2- 172.21.15.5:0/1097170729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74f010c4b0 0x7f74f019a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.615+0000 7f74efe00640 1 -- 172.21.15.5:0/1097170729 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74f019aff0 con 0x7f74f0103090 2024-04-24T06:44:42.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.615+0000 7f74f5000640 1 --2- 172.21.15.5:0/1097170729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74f010c4b0 0x7f74f019a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:44:42.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.616+0000 7f74efe00640 1 --2- 172.21.15.5:0/1097170729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74f0103090 0x7f74f01a06b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f74e000e9f0 tx=0x7f74e000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:42.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.616+0000 7f74ecc00640 1 -- 172.21.15.5:0/1097170729 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e000cd80 con 0x7f74f0103090 2024-04-24T06:44:42.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.616+0000 7f74ecc00640 1 -- 172.21.15.5:0/1097170729 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74e0010910 con 0x7f74f0103090 2024-04-24T06:44:42.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.616+0000 7f74ecc00640 1 -- 172.21.15.5:0/1097170729 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e0018990 con 0x7f74f0103090 2024-04-24T06:44:42.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.616+0000 7f74f5a00640 1 -- 172.21.15.5:0/1097170729 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74f019b2e0 con 0x7f74f0103090 2024-04-24T06:44:42.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.616+0000 7f74f5a00640 1 -- 172.21.15.5:0/1097170729 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74f01a7500 con 0x7f74f0103090 2024-04-24T06:44:42.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.618+0000 7f74f5a00640 1 -- 172.21.15.5:0/1097170729 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74f0069be0 con 0x7f74f0103090 2024-04-24T06:44:42.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.620+0000 7f74ecc00640 1 -- 172.21.15.5:0/1097170729 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 138872+0+0 (secure 0 0 0) 0x7f74e0010ac0 con 0x7f74f0103090 2024-04-24T06:44:42.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.621+0000 7f74ecc00640 1 --2- 172.21.15.5:0/1097170729 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f74d00a3eb0 0x7f74d00a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:42.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.621+0000 7f74ecc00640 1 -- 172.21.15.5:0/1097170729 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f74e00d0e30 con 0x7f74f0103090 2024-04-24T06:44:42.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.621+0000 7f74ef400640 1 --2- 172.21.15.5:0/1097170729 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f74d00a3eb0 0x7f74d00a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:42.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.622+0000 7f74ef400640 1 --2- 172.21.15.5:0/1097170729 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f74d00a3eb0 0x7f74d00a6370 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f74d8006010 tx=0x7f74d8005d80 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:42.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.625+0000 7f74ecc00640 1 -- 172.21.15.5:0/1097170729 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f74e009d230 con 0x7f74f0103090 2024-04-24T06:44:42.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.875+0000 7f74f5a00640 1 -- 172.21.15.5:0/1097170729 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74f01a76d0 con 0x7f74f0103090 2024-04-24T06:44:42.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.877+0000 7f74ecc00640 1 -- 172.21.15.5:0/1097170729 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11856 (secure 0 0 0) 0x7f74e009cb30 con 0x7f74f0103090 2024-04-24T06:44:42.878 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:44:42.879 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1735462750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1735462750},{"type":"v1","addr":"172.21.15.5:6837","nonce":1735462750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1499469370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1499469370},{"type":"v1","addr":"172.21.15.5:6841","nonce":1499469370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3705638847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3705638847},{"type":"v1","addr":"172.21.15.37:6835","nonce":3705638847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T06:41:34.764490+0000","modified":"2024-04-24T06:44:01.711021+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24550,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6837/1390103296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1390103296},{"type":"v1","addr":"172.21.15.37:6837","nonce":1390103296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3217754215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3217754215},{"type":"v1","addr":"172.21.15.137:6835","nonce":3217754215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6837/3044222976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3044222976},{"type":"v1","addr":"172.21.15.137:6837","nonce":3044222976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.5:6839/3173201459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3173201459},{"type":"v1","addr":"172.21.15.5:6839","nonce":3173201459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6839/1889774232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1889774232},{"type":"v1","addr":"172.21.15.137:6839","nonce":1889774232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6839/3536721753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":3536721753},{"type":"v1","addr":"172.21.15.37:6839","nonce":3536721753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24569,24539,24550]},"id":1}]} 2024-04-24T06:44:42.879 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 23 2024-04-24T06:44:42.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.880+0000 7f74f5a00640 1 -- 172.21.15.5:0/1097170729 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f74d00a3eb0 msgr2=0x7f74d00a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:42.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.880+0000 7f74f5a00640 1 --2- 172.21.15.5:0/1097170729 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f74d00a3eb0 0x7f74d00a6370 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f74d8006010 tx=0x7f74d8005d80 comp rx=0 tx=0).stop 2024-04-24T06:44:42.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.880+0000 7f74f5a00640 1 -- 172.21.15.5:0/1097170729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74f0103090 msgr2=0x7f74f01a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:42.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.880+0000 7f74f5a00640 1 --2- 172.21.15.5:0/1097170729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74f0103090 0x7f74f01a06b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f74e000e9f0 tx=0x7f74e000eeb0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.880+0000 7f74f5a00640 1 -- 172.21.15.5:0/1097170729 shutdown_connections 2024-04-24T06:44:42.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.880+0000 7f74f5a00640 1 --2- 172.21.15.5:0/1097170729 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f74d00a3eb0 0x7f74d00a6370 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.880+0000 7f74f5a00640 1 --2- 172.21.15.5:0/1097170729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74f010c4b0 0x7f74f019a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.880+0000 7f74f5a00640 1 --2- 172.21.15.5:0/1097170729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74f01039b0 0x7f74f01a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.880+0000 7f74f5a00640 1 --2- 172.21.15.5:0/1097170729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74f0103090 0x7f74f01a06b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:42.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.880+0000 7f74f5a00640 1 -- 172.21.15.5:0/1097170729 >> 172.21.15.5:0/1097170729 conn(0x7f74f00fc890 msgr2=0x7f74f00fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:42.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.880+0000 7f74f5a00640 1 -- 172.21.15.5:0/1097170729 shutdown_connections 2024-04-24T06:44:42.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:42.880+0000 7f74f5a00640 1 -- 172.21.15.5:0/1097170729 wait complete. 2024-04-24T06:44:42.903 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T06:44:43.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.125+0000 7efc9be00640 1 -- 172.21.15.5:0/80711376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc9c109360 msgr2=0x7efc9c110f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:43.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.125+0000 7efc9be00640 1 --2- 172.21.15.5:0/80711376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc9c109360 0x7efc9c110f30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efc94009ca0 tx=0x7efc9402a8b0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.125+0000 7efc9be00640 1 -- 172.21.15.5:0/80711376 shutdown_connections 2024-04-24T06:44:43.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.125+0000 7efc9be00640 1 --2- 172.21.15.5:0/80711376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc9c109360 0x7efc9c110f30 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.125+0000 7efc9be00640 1 --2- 172.21.15.5:0/80711376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc9c104c40 0x7efc9c108c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.125+0000 7efc9be00640 1 --2- 172.21.15.5:0/80711376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc9c104290 0x7efc9c104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.125+0000 7efc9be00640 1 -- 172.21.15.5:0/80711376 >> 172.21.15.5:0/80711376 conn(0x7efc9c0ffaf0 msgr2=0x7efc9c101f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:43.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.125+0000 7efc9be00640 1 -- 172.21.15.5:0/80711376 shutdown_connections 2024-04-24T06:44:43.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.125+0000 7efc9be00640 1 -- 172.21.15.5:0/80711376 wait complete. 2024-04-24T06:44:43.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.127+0000 7efc9be00640 1 Processor -- start 2024-04-24T06:44:43.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.127+0000 7efc9be00640 1 -- start start 2024-04-24T06:44:43.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.127+0000 7efc9be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc9c104290 0x7efc9c1a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.127+0000 7efc9be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc9c109360 0x7efc9c1a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.127+0000 7efc9be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc9c19a7f0 0x7efc9c19aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.127+0000 7efc9be00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c114450 con 0x7efc9c104290 2024-04-24T06:44:43.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.127+0000 7efc9be00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c1142d0 con 0x7efc9c109360 2024-04-24T06:44:43.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.127+0000 7efc9be00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c1145d0 con 0x7efc9c19a7f0 2024-04-24T06:44:43.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.128+0000 7efc9aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc9c104290 0x7efc9c1a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.128+0000 7efc9aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc9c104290 0x7efc9c1a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55160/0 (socket says 172.21.15.5:55160) 2024-04-24T06:44:43.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.128+0000 7efc9aa00640 1 -- 172.21.15.5:0/3722990484 learned_addr learned my addr 172.21.15.5:0/3722990484 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:43.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.128+0000 7efc9b400640 1 --2- 172.21.15.5:0/3722990484 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc9c19a7f0 0x7efc9c19aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.128+0000 7efc9a000640 1 --2- 172.21.15.5:0/3722990484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc9c109360 0x7efc9c1a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.128+0000 7efc9aa00640 1 -- 172.21.15.5:0/3722990484 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc9c19a7f0 msgr2=0x7efc9c19aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:43.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.128+0000 7efc9aa00640 1 --2- 172.21.15.5:0/3722990484 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc9c19a7f0 0x7efc9c19aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.128+0000 7efc9aa00640 1 -- 172.21.15.5:0/3722990484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc9c109360 msgr2=0x7efc9c1a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:43.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.128+0000 7efc9aa00640 1 --2- 172.21.15.5:0/3722990484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc9c109360 0x7efc9c1a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.128+0000 7efc9aa00640 1 -- 172.21.15.5:0/3722990484 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc9c19b380 con 0x7efc9c104290 2024-04-24T06:44:43.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.129+0000 7efc9aa00640 1 --2- 172.21.15.5:0/3722990484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc9c104290 0x7efc9c1a0630 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7efc8c00e9f0 tx=0x7efc8c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:43.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.129+0000 7efc93400640 1 -- 172.21.15.5:0/3722990484 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8c00cd80 con 0x7efc9c104290 2024-04-24T06:44:43.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.129+0000 7efc9be00640 1 -- 172.21.15.5:0/3722990484 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc9c19b670 con 0x7efc9c104290 2024-04-24T06:44:43.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.129+0000 7efc93400640 1 -- 172.21.15.5:0/3722990484 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc8c010910 con 0x7efc9c104290 2024-04-24T06:44:43.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.129+0000 7efc93400640 1 -- 172.21.15.5:0/3722990484 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8c0188f0 con 0x7efc9c104290 2024-04-24T06:44:43.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.129+0000 7efc9be00640 1 -- 172.21.15.5:0/3722990484 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc9c1a7800 con 0x7efc9c104290 2024-04-24T06:44:43.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.132+0000 7efca437a740 1 Processor -- start 2024-04-24T06:44:43.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.132+0000 7efca437a740 1 -- start start 2024-04-24T06:44:43.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.133+0000 7efca437a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640d7db7b40 0x5640d7db7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.133+0000 7efc93400640 1 -- 172.21.15.5:0/3722990484 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 138872+0+0 (secure 0 0 0) 0x7efc8c010ac0 con 0x7efc9c104290 2024-04-24T06:44:43.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.133+0000 7efca437a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640d7db8460 0x5640d7dc25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.133+0000 7efca437a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d7dc2b30 0x5640d7dc4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.133+0000 7efca437a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5640d7d1d1d0 con 0x5640d7db7b40 2024-04-24T06:44:43.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.133+0000 7efca437a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5640d7cc3450 con 0x5640d7dc2b30 2024-04-24T06:44:43.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.133+0000 7efca437a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5640d7d19190 con 0x5640d7db8460 2024-04-24T06:44:43.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.133+0000 7efc9b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640d7db7b40 0x5640d7db7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.133+0000 7efc9b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640d7db7b40 0x5640d7db7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55176/0 (socket says 172.21.15.5:55176) 2024-04-24T06:44:43.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.133+0000 7efc9b400640 1 -- 172.21.15.5:0/1111183847 learned_addr learned my addr 172.21.15.5:0/1111183847 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:43.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.133+0000 7efc9a000640 1 --2- 172.21.15.5:0/1111183847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d7dc2b30 0x5640d7dc4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.133+0000 7efc9aa00640 1 --2- 172.21.15.5:0/1111183847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640d7db8460 0x5640d7dc25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.134+0000 7efc9a000640 1 -- 172.21.15.5:0/1111183847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640d7db8460 msgr2=0x5640d7dc25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:43.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.134+0000 7efc9a000640 1 --2- 172.21.15.5:0/1111183847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640d7db8460 0x5640d7dc25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.134+0000 7efc9a000640 1 -- 172.21.15.5:0/1111183847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640d7db7b40 msgr2=0x5640d7db7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:43.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.134+0000 7efc9a000640 1 --2- 172.21.15.5:0/1111183847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640d7db7b40 0x5640d7db7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.134+0000 7efc9a000640 1 -- 172.21.15.5:0/1111183847 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640d7d80310 con 0x5640d7dc2b30 2024-04-24T06:44:43.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.134+0000 7efc9b400640 1 --2- 172.21.15.5:0/1111183847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640d7db7b40 0x5640d7db7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:44:43.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.134+0000 7efc9a000640 1 --2- 172.21.15.5:0/1111183847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d7dc2b30 0x5640d7dc4f20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efc9c19baf0 tx=0x7efc880103d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5294d0e606126a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:43.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.134+0000 7efc87400640 1 -- 172.21.15.5:0/1111183847 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc88010e90 con 0x5640d7dc2b30 2024-04-24T06:44:43.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.134+0000 7efc87400640 1 -- 172.21.15.5:0/1111183847 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc880035f0 con 0x5640d7dc2b30 2024-04-24T06:44:43.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.134+0000 7efc87400640 1 -- 172.21.15.5:0/1111183847 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc880116a0 con 0x5640d7dc2b30 2024-04-24T06:44:43.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.135+0000 7efc93400640 1 --2- 172.21.15.5:0/3722990484 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7efc800a3e40 0x7efc800a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.135+0000 7efc93400640 1 -- 172.21.15.5:0/3722990484 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7efc8c0d0ea0 con 0x7efc9c104290 2024-04-24T06:44:43.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.135+0000 7efc9b400640 1 --2- 172.21.15.5:0/3722990484 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7efc800a3e40 0x7efc800a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.135+0000 7efca437a740 1 -- 172.21.15.5:0/1111183847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d7dc2b30 msgr2=0x5640d7dc4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:43.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.135+0000 7efca437a740 1 --2- 172.21.15.5:0/1111183847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d7dc2b30 0x5640d7dc4f20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efc9c19baf0 tx=0x7efc880103d0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.135+0000 7efca437a740 1 -- 172.21.15.5:0/1111183847 shutdown_connections 2024-04-24T06:44:43.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.135+0000 7efca437a740 1 --2- 172.21.15.5:0/1111183847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d7dc2b30 0x5640d7dc4f20 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.135+0000 7efca437a740 1 --2- 172.21.15.5:0/1111183847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640d7db8460 0x5640d7dc25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.135+0000 7efca437a740 1 --2- 172.21.15.5:0/1111183847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640d7db7b40 0x5640d7db7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.135+0000 7efca437a740 1 -- 172.21.15.5:0/1111183847 >> 172.21.15.5:0/1111183847 conn(0x5640d7db3e60 msgr2=0x5640d7db4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:43.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.136+0000 7efca437a740 1 -- 172.21.15.5:0/1111183847 shutdown_connections 2024-04-24T06:44:43.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.136+0000 7efc9b400640 1 --2- 172.21.15.5:0/3722990484 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7efc800a3e40 0x7efc800a6300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efc9402aea0 tx=0x7efc94005d80 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:43.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.136+0000 7efca437a740 1 -- 172.21.15.5:0/1111183847 wait complete. 2024-04-24T06:44:43.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.136+0000 7efca437a740 1 -- start start 2024-04-24T06:44:43.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.136+0000 7efca437a740 1 Processor -- start 2024-04-24T06:44:43.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.137+0000 7efca437a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d7db7b40 0x5640d7dc25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.137+0000 7efca437a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640d7db8460 0x5640d7dbf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.137+0000 7efca437a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640d7dc2b30 0x5640d7dbffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.137+0000 7efca437a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5640d7dc59c0 con 0x5640d7dc2b30 2024-04-24T06:44:43.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.137+0000 7efca437a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5640d7cd6810 con 0x5640d7db7b40 2024-04-24T06:44:43.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.137+0000 7efca437a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5640d7dc5b40 con 0x5640d7db8460 2024-04-24T06:44:43.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.137+0000 7efca437a740 10 client.0 _ll_register_callbacks cb 0x5640d7ca4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:44:43.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.137+0000 7efc9b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640d7dc2b30 0x5640d7dbffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.137+0000 7efc9aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d7db7b40 0x5640d7dc25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.137+0000 7efc9aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d7db7b40 0x5640d7dc25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:56760/0 (socket says 172.21.15.5:56760) 2024-04-24T06:44:43.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.137+0000 7efc9aa00640 1 -- 172.21.15.5:0/326153365 learned_addr learned my addr 172.21.15.5:0/326153365 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:43.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.137+0000 7efc9b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640d7dc2b30 0x5640d7dbffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55184/0 (socket says 172.21.15.5:55184) 2024-04-24T06:44:43.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.137+0000 7efc9a000640 1 --2- 172.21.15.5:0/326153365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640d7db8460 0x5640d7dbf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.138+0000 7efc9aa00640 1 -- 172.21.15.5:0/326153365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640d7db8460 msgr2=0x5640d7dbf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:43.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.138+0000 7efc9aa00640 1 --2- 172.21.15.5:0/326153365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640d7db8460 0x5640d7dbf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.138+0000 7efc9aa00640 1 -- 172.21.15.5:0/326153365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640d7dc2b30 msgr2=0x5640d7dbffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:43.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.138+0000 7efc9aa00640 1 --2- 172.21.15.5:0/326153365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640d7dc2b30 0x5640d7dbffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.138+0000 7efc9aa00640 1 -- 172.21.15.5:0/326153365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640d7ce73f0 con 0x5640d7db7b40 2024-04-24T06:44:43.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.138+0000 7efc9aa00640 1 --2- 172.21.15.5:0/326153365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d7db7b40 0x5640d7dc25a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7efc8c00c6a0 tx=0x7efc8c098ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9559214347cb8cc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:43.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.138+0000 7efc7be00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:44:43.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.138+0000 7efc7be00640 1 -- 172.21.15.5:0/326153365 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8c09c5b0 con 0x5640d7db7b40 2024-04-24T06:44:43.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.138+0000 7efc7be00640 1 -- 172.21.15.5:0/326153365 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7efc8c0d4da0 con 0x5640d7db7b40 2024-04-24T06:44:43.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.138+0000 7efca437a740 1 -- 172.21.15.5:0/326153365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc880035f0 con 0x5640d7db7b40 2024-04-24T06:44:43.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.139+0000 7efc7be00640 1 -- 172.21.15.5:0/326153365 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7efc8c09cc50 con 0x5640d7db7b40 2024-04-24T06:44:43.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.139+0000 7efca437a740 10 client.25475 fetch_fsmap learned FSMap version 23 2024-04-24T06:44:43.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.139+0000 7efca437a740 1 -- 172.21.15.5:0/326153365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5640d7a2f3f0 con 0x5640d7db7b40 2024-04-24T06:44:43.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.139+0000 7efc7be00640 1 -- 172.21.15.5:0/326153365 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc880035f0 con 0x5640d7db7b40 2024-04-24T06:44:43.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.139+0000 7efc7be00640 1 -- 172.21.15.5:0/326153365 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7efc8c09c0a0 con 0x5640d7db7b40 2024-04-24T06:44:43.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.139+0000 7efca437a740 10 client.25475 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T06:44:43.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.139+0000 7efca437a740 10 client.25475 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-24T06:44:43.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.139+0000 7efca437a740 20 client.25475 populate_metadata read hostname 'smithi005' 2024-04-24T06:44:43.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.139+0000 7efca437a740 1 --2- 172.21.15.5:0/326153365 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x5640d7dd2c90 0x5640d7dd50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.139+0000 7efca437a740 4 client.25475 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:44:43.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.139+0000 7efca437a740 1 -- 172.21.15.5:0/326153365 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640d7d2abf0 con 0x5640d7dd2c90 2024-04-24T06:44:43.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.140+0000 7efc9a000640 1 --2- 172.21.15.5:0/326153365 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x5640d7dd2c90 0x5640d7dd50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.140+0000 7efc9a000640 1 --2- 172.21.15.5:0/326153365 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x5640d7dd2c90 0x5640d7dd50e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d348688d2071c8f8 server_cookie=699c7b621a6084e7 in_seq=0 out_seq=0 2024-04-24T06:44:43.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.140+0000 7efc7be00640 10 client.25475 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:44:43.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.141+0000 7efc7be00640 1 -- 172.21.15.5:0/326153365 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5640d7d2abf0 con 0x5640d7dd2c90 2024-04-24T06:44:43.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.141+0000 7efc7be00640 10 client.25475 handle_command_reply: tid=0 2024-04-24T06:44:43.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.141+0000 7efca437a740 1 client.25475 shutdown 2024-04-24T06:44:43.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.141+0000 7efca437a740 1 -- 172.21.15.5:0/326153365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d7db7b40 msgr2=0x5640d7dc25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:43.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.141+0000 7efca437a740 1 --2- 172.21.15.5:0/326153365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d7db7b40 0x5640d7dc25a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7efc8c00c6a0 tx=0x7efc8c098ef0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.141+0000 7efca437a740 1 -- 172.21.15.5:0/326153365 shutdown_connections 2024-04-24T06:44:43.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.141+0000 7efca437a740 1 --2- 172.21.15.5:0/326153365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640d7dc2b30 0x5640d7dbffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.141+0000 7efca437a740 1 --2- 172.21.15.5:0/326153365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640d7db8460 0x5640d7dbf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.141+0000 7efca437a740 1 --2- 172.21.15.5:0/326153365 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x5640d7dd2c90 0x5640d7dd50e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.141+0000 7efca437a740 1 --2- 172.21.15.5:0/326153365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d7db7b40 0x5640d7dc25a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.141+0000 7efc7be00640 0 client.25475 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:44:43.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.141+0000 7efca437a740 1 -- 172.21.15.5:0/326153365 >> 172.21.15.5:0/326153365 conn(0x5640d7db3e60 msgr2=0x5640d7db9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:43.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.142+0000 7efca437a740 1 -- 172.21.15.5:0/326153365 shutdown_connections 2024-04-24T06:44:43.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.142+0000 7efca437a740 1 -- 172.21.15.5:0/326153365 wait complete. 2024-04-24T06:44:43.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.142+0000 7efca437a740 20 client.25475 trim_cache size 0 max 16384 2024-04-24T06:44:43.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.157+0000 7efca437a740 1 Processor -- start 2024-04-24T06:44:43.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.157+0000 7efca437a740 1 -- start start 2024-04-24T06:44:43.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.157+0000 7efca437a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640d7db7b40 0x5640d7dce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.157+0000 7efca437a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640d7db8460 0x5640d7db3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.157+0000 7efca437a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d7dc2b30 0x5640d7dc20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.157+0000 7efca437a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c1b26f0 con 0x5640d7db7b40 2024-04-24T06:44:43.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.157+0000 7efca437a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c1affc0 con 0x5640d7dc2b30 2024-04-24T06:44:43.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.157+0000 7efca437a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c1ade10 con 0x5640d7db8460 2024-04-24T06:44:43.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.157+0000 7efc9aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640d7db7b40 0x5640d7dce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.157+0000 7efc9aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640d7db7b40 0x5640d7dce880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55192/0 (socket says 172.21.15.5:55192) 2024-04-24T06:44:43.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.157+0000 7efc9aa00640 1 -- 172.21.15.5:0/382678173 learned_addr learned my addr 172.21.15.5:0/382678173 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:43.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.157+0000 7efc9b400640 1 --2- 172.21.15.5:0/382678173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d7dc2b30 0x5640d7dc20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.157+0000 7efc9a000640 1 --2- 172.21.15.5:0/382678173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640d7db8460 0x5640d7db3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.158+0000 7efc9aa00640 1 -- 172.21.15.5:0/382678173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640d7db8460 msgr2=0x5640d7db3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:43.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.158+0000 7efc9aa00640 1 --2- 172.21.15.5:0/382678173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640d7db8460 0x5640d7db3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.158+0000 7efc9aa00640 1 -- 172.21.15.5:0/382678173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d7dc2b30 msgr2=0x5640d7dc20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:43.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.158+0000 7efc9aa00640 1 --2- 172.21.15.5:0/382678173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d7dc2b30 0x5640d7dc20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.158+0000 7efc9aa00640 1 -- 172.21.15.5:0/382678173 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640d7af97d0 con 0x5640d7db7b40 2024-04-24T06:44:43.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.158+0000 7efc9a000640 1 --2- 172.21.15.5:0/382678173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640d7db8460 0x5640d7db3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:44:43.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.158+0000 7efc9aa00640 1 --2- 172.21.15.5:0/382678173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640d7db7b40 0x5640d7dce880 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efc8c0a52e0 tx=0x7efc8c0d1120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4055f43014ce6e77 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:43.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.158+0000 7efc86a00640 1 -- 172.21.15.5:0/382678173 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8c0b6070 con 0x5640d7db7b40 2024-04-24T06:44:43.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.158+0000 7efc86a00640 1 -- 172.21.15.5:0/382678173 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc8c0ce360 con 0x5640d7db7b40 2024-04-24T06:44:43.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.158+0000 7efc86a00640 1 -- 172.21.15.5:0/382678173 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8c0d8070 con 0x5640d7db7b40 2024-04-24T06:44:43.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.159+0000 7efca437a740 1 -- 172.21.15.5:0/382678173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640d7db7b40 msgr2=0x5640d7dce880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:43.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.159+0000 7efca437a740 1 --2- 172.21.15.5:0/382678173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640d7db7b40 0x5640d7dce880 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efc8c0a52e0 tx=0x7efc8c0d1120 comp rx=0 tx=0).stop 2024-04-24T06:44:43.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.159+0000 7efca437a740 1 -- 172.21.15.5:0/382678173 shutdown_connections 2024-04-24T06:44:43.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.159+0000 7efca437a740 1 --2- 172.21.15.5:0/382678173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d7dc2b30 0x5640d7dc20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.159+0000 7efca437a740 1 --2- 172.21.15.5:0/382678173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640d7db8460 0x5640d7db3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.159+0000 7efca437a740 1 --2- 172.21.15.5:0/382678173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640d7db7b40 0x5640d7dce880 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.159+0000 7efca437a740 1 -- 172.21.15.5:0/382678173 >> 172.21.15.5:0/382678173 conn(0x5640d7db3e60 msgr2=0x5640d7db9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:43.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.159+0000 7efca437a740 1 -- 172.21.15.5:0/382678173 shutdown_connections 2024-04-24T06:44:43.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.159+0000 7efca437a740 1 -- 172.21.15.5:0/382678173 wait complete. 2024-04-24T06:44:43.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.160+0000 7efca437a740 1 -- start start 2024-04-24T06:44:43.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.160+0000 7efca437a740 1 Processor -- start 2024-04-24T06:44:43.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.160+0000 7efca437a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640d7db7b40 0x5640d7dcad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.160+0000 7efca437a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d7db8460 0x5640d7dcb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.160+0000 7efca437a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640d7dc2b30 0x5640d7dc3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.160+0000 7efca437a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c1ae480 con 0x5640d7db7b40 2024-04-24T06:44:43.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.160+0000 7efca437a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c1af8f0 con 0x5640d7db8460 2024-04-24T06:44:43.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.160+0000 7efca437a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c1aeb50 con 0x5640d7dc2b30 2024-04-24T06:44:43.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.160+0000 7efca437a740 10 client.0 _ll_register_callbacks cb 0x7efc8c009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:44:43.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.160+0000 7efc9aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640d7db7b40 0x5640d7dcad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.161+0000 7efc9aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640d7db7b40 0x5640d7dcad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55198/0 (socket says 172.21.15.5:55198) 2024-04-24T06:44:43.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.161+0000 7efc9aa00640 1 -- 172.21.15.5:0/823151919 learned_addr learned my addr 172.21.15.5:0/823151919 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:43.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.161+0000 7efc9a000640 1 --2- 172.21.15.5:0/823151919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d7db8460 0x5640d7dcb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.161+0000 7efc9b400640 1 --2- 172.21.15.5:0/823151919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640d7dc2b30 0x5640d7dc3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.161+0000 7efc9aa00640 1 -- 172.21.15.5:0/823151919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640d7dc2b30 msgr2=0x5640d7dc3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:43.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.161+0000 7efc9aa00640 1 --2- 172.21.15.5:0/823151919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640d7dc2b30 0x5640d7dc3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.161+0000 7efc9aa00640 1 -- 172.21.15.5:0/823151919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d7db8460 msgr2=0x5640d7dcb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:43.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.161+0000 7efc9aa00640 1 --2- 172.21.15.5:0/823151919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d7db8460 0x5640d7dcb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.161+0000 7efc9aa00640 1 -- 172.21.15.5:0/823151919 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640d7dc42e0 con 0x5640d7db7b40 2024-04-24T06:44:43.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.161+0000 7efc9aa00640 1 --2- 172.21.15.5:0/823151919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640d7db7b40 0x5640d7dcad90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7efc8c09cbe0 tx=0x7efc8c00e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=345b9a780286c7b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:43.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.161+0000 7efc7be00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:44:43.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.162+0000 7efc7be00640 1 -- 172.21.15.5:0/823151919 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8c0d8070 con 0x5640d7db7b40 2024-04-24T06:44:43.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.162+0000 7efc7be00640 1 -- 172.21.15.5:0/823151919 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7efc8c0ce900 con 0x5640d7db7b40 2024-04-24T06:44:43.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.162+0000 7efca437a740 1 -- 172.21.15.5:0/823151919 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc8c0ce360 con 0x5640d7db7b40 2024-04-24T06:44:43.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.162+0000 7efc7be00640 1 -- 172.21.15.5:0/823151919 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7efc8c0b6040 con 0x5640d7db7b40 2024-04-24T06:44:43.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.162+0000 7efca437a740 10 client.15381 fetch_fsmap learned FSMap version 23 2024-04-24T06:44:43.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.162+0000 7efca437a740 1 -- 172.21.15.5:0/823151919 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5640d7e39a40 con 0x5640d7db7b40 2024-04-24T06:44:43.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.162+0000 7efc7be00640 1 -- 172.21.15.5:0/823151919 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc8c0ce360 con 0x5640d7db7b40 2024-04-24T06:44:43.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.162+0000 7efc7be00640 1 -- 172.21.15.5:0/823151919 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7efc8c0d3050 con 0x5640d7db7b40 2024-04-24T06:44:43.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.162+0000 7efca437a740 10 client.15381 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T06:44:43.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.162+0000 7efca437a740 10 client.15381 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-24T06:44:43.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.162+0000 7efca437a740 20 client.15381 populate_metadata read hostname 'smithi005' 2024-04-24T06:44:43.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.162+0000 7efca437a740 1 --2- 172.21.15.5:0/823151919 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x5640d7e3a0d0 0x5640d7e3c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.162+0000 7efca437a740 4 client.15381 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:44:43.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.162+0000 7efca437a740 1 -- 172.21.15.5:0/823151919 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5640d7d24a60 con 0x5640d7e3a0d0 2024-04-24T06:44:43.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.163+0000 7efc9a000640 1 --2- 172.21.15.5:0/823151919 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x5640d7e3a0d0 0x5640d7e3c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.163+0000 7efc9a000640 1 --2- 172.21.15.5:0/823151919 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x5640d7e3a0d0 0x5640d7e3c4f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d38d99221c4e1351 server_cookie=9d30870d45255f2f in_seq=0 out_seq=0 2024-04-24T06:44:43.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.163+0000 7efc7be00640 10 client.15381 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:44:43.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.163+0000 7efc7be00640 1 -- 172.21.15.5:0/823151919 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5640d7d24a60 con 0x5640d7e3a0d0 2024-04-24T06:44:43.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.163+0000 7efc7be00640 10 client.15381 handle_command_reply: tid=0 2024-04-24T06:44:43.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.163+0000 7efca437a740 1 client.15381 shutdown 2024-04-24T06:44:43.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.164+0000 7efca437a740 1 -- 172.21.15.5:0/823151919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640d7db7b40 msgr2=0x5640d7dcad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:43.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.164+0000 7efca437a740 1 --2- 172.21.15.5:0/823151919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640d7db7b40 0x5640d7dcad90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7efc8c09cbe0 tx=0x7efc8c00e940 comp rx=0 tx=0).stop 2024-04-24T06:44:43.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.164+0000 7efca437a740 1 -- 172.21.15.5:0/823151919 shutdown_connections 2024-04-24T06:44:43.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.164+0000 7efca437a740 1 --2- 172.21.15.5:0/823151919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640d7dc2b30 0x5640d7dc3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.164+0000 7efca437a740 1 --2- 172.21.15.5:0/823151919 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x5640d7e3a0d0 0x5640d7e3c4f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.164+0000 7efca437a740 1 --2- 172.21.15.5:0/823151919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d7db8460 0x5640d7dcb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.164+0000 7efca437a740 1 --2- 172.21.15.5:0/823151919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640d7db7b40 0x5640d7dcad90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.164+0000 7efc7be00640 0 client.15381 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:44:43.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.164+0000 7efca437a740 1 -- 172.21.15.5:0/823151919 >> 172.21.15.5:0/823151919 conn(0x5640d7db3e60 msgr2=0x5640d7dcd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:43.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.164+0000 7efca437a740 1 -- 172.21.15.5:0/823151919 shutdown_connections 2024-04-24T06:44:43.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.164+0000 7efca437a740 1 -- 172.21.15.5:0/823151919 wait complete. 2024-04-24T06:44:43.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.164+0000 7efca437a740 20 client.15381 trim_cache size 0 max 16384 2024-04-24T06:44:43.171 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:44:43.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.165+0000 7efc9be00640 1 -- 172.21.15.5:0/3722990484 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7efc800a3e40 msgr2=0x7efc800a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:43.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.165+0000 7efc9be00640 1 --2- 172.21.15.5:0/3722990484 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7efc800a3e40 0x7efc800a6300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efc9402aea0 tx=0x7efc94005d80 comp rx=0 tx=0).stop 2024-04-24T06:44:43.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.165+0000 7efc9be00640 1 -- 172.21.15.5:0/3722990484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc9c104290 msgr2=0x7efc9c1a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:43.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.165+0000 7efc9be00640 1 --2- 172.21.15.5:0/3722990484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc9c104290 0x7efc9c1a0630 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7efc8c00e9f0 tx=0x7efc8c00eeb0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.165+0000 7efc9be00640 1 -- 172.21.15.5:0/3722990484 shutdown_connections 2024-04-24T06:44:43.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.165+0000 7efc9be00640 1 --2- 172.21.15.5:0/3722990484 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7efc800a3e40 0x7efc800a6300 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.165+0000 7efc9be00640 1 --2- 172.21.15.5:0/3722990484 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc9c19a7f0 0x7efc9c19aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.165+0000 7efc9be00640 1 --2- 172.21.15.5:0/3722990484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc9c109360 0x7efc9c1a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.165+0000 7efc9be00640 1 --2- 172.21.15.5:0/3722990484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc9c104290 0x7efc9c1a0630 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.165+0000 7efc9be00640 1 -- 172.21.15.5:0/3722990484 >> 172.21.15.5:0/3722990484 conn(0x7efc9c0ffaf0 msgr2=0x7efc9c101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:43.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.165+0000 7efc9be00640 1 -- 172.21.15.5:0/3722990484 shutdown_connections 2024-04-24T06:44:43.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.165+0000 7efc9be00640 1 -- 172.21.15.5:0/3722990484 wait complete. 2024-04-24T06:44:43.186 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T06:44:43.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.398+0000 7ff9fb600640 1 -- 172.21.15.5:0/1339368429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9f410e860 msgr2=0x7ff9f4110c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:43.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.398+0000 7ff9fb600640 1 --2- 172.21.15.5:0/1339368429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9f410e860 0x7ff9f4110c70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec009be0 tx=0x7ff9ec02a750 comp rx=0 tx=0).stop 2024-04-24T06:44:43.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.398+0000 7ff9fb600640 1 -- 172.21.15.5:0/1339368429 shutdown_connections 2024-04-24T06:44:43.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.398+0000 7ff9fb600640 1 --2- 172.21.15.5:0/1339368429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9f410e860 0x7ff9f4110c70 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.398+0000 7ff9fb600640 1 --2- 172.21.15.5:0/1339368429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9f4074cf0 0x7ff9f410e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.398+0000 7ff9fb600640 1 --2- 172.21.15.5:0/1339368429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9f40743d0 0x7ff9f40747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.398+0000 7ff9fb600640 1 -- 172.21.15.5:0/1339368429 >> 172.21.15.5:0/1339368429 conn(0x7ff9f40fdab0 msgr2=0x7ff9f40ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:43.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.398+0000 7ff9fb600640 1 -- 172.21.15.5:0/1339368429 shutdown_connections 2024-04-24T06:44:43.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.398+0000 7ff9fb600640 1 -- 172.21.15.5:0/1339368429 wait complete. 2024-04-24T06:44:43.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.399+0000 7ff9fb600640 1 Processor -- start 2024-04-24T06:44:43.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.399+0000 7ff9fb600640 1 -- start start 2024-04-24T06:44:43.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.399+0000 7ff9fb600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9f40743d0 0x7ff9f41a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.399+0000 7ff9fb600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9f4074cf0 0x7ff9f41a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.399+0000 7ff9fb600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9f410e860 0x7ff9f419a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.399+0000 7ff9fb600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f41145a0 con 0x7ff9f40743d0 2024-04-24T06:44:43.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.399+0000 7ff9fb600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f4114420 con 0x7ff9f4074cf0 2024-04-24T06:44:43.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.399+0000 7ff9fb600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f4114720 con 0x7ff9f410e860 2024-04-24T06:44:43.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.400+0000 7ff9f8c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9f40743d0 0x7ff9f41a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.400+0000 7ff9f3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9f4074cf0 0x7ff9f41a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.400+0000 7ff9f3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9f4074cf0 0x7ff9f41a0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:56816/0 (socket says 172.21.15.5:56816) 2024-04-24T06:44:43.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.400+0000 7ff9f3e00640 1 -- 172.21.15.5:0/1715342414 learned_addr learned my addr 172.21.15.5:0/1715342414 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:43.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.400+0000 7ff9f9600640 1 --2- 172.21.15.5:0/1715342414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9f410e860 0x7ff9f419a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.400+0000 7ff9f3e00640 1 -- 172.21.15.5:0/1715342414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9f410e860 msgr2=0x7ff9f419a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:43.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.400+0000 7ff9f3e00640 1 --2- 172.21.15.5:0/1715342414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9f410e860 0x7ff9f419a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.400+0000 7ff9f3e00640 1 -- 172.21.15.5:0/1715342414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9f40743d0 msgr2=0x7ff9f41a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:43.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.400+0000 7ff9f3e00640 1 --2- 172.21.15.5:0/1715342414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9f40743d0 0x7ff9f41a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.400+0000 7ff9f3e00640 1 -- 172.21.15.5:0/1715342414 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f419b0a0 con 0x7ff9f4074cf0 2024-04-24T06:44:43.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.400+0000 7ff9f8c00640 1 --2- 172.21.15.5:0/1715342414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9f40743d0 0x7ff9f41a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:44:43.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.400+0000 7ff9f3e00640 1 --2- 172.21.15.5:0/1715342414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9f4074cf0 0x7ff9f41a0cb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc00e9f0 tx=0x7ff9dc00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:43.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.401+0000 7ff9f1600640 1 -- 172.21.15.5:0/1715342414 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc00cd80 con 0x7ff9f4074cf0 2024-04-24T06:44:43.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.401+0000 7ff9f1600640 1 -- 172.21.15.5:0/1715342414 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9dc005500 con 0x7ff9f4074cf0 2024-04-24T06:44:43.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.401+0000 7ff9f1600640 1 -- 172.21.15.5:0/1715342414 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc010690 con 0x7ff9f4074cf0 2024-04-24T06:44:43.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.401+0000 7ff9fb600640 1 -- 172.21.15.5:0/1715342414 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9f419b390 con 0x7ff9f4074cf0 2024-04-24T06:44:43.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.401+0000 7ff9fb600640 1 -- 172.21.15.5:0/1715342414 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9f41a7630 con 0x7ff9f4074cf0 2024-04-24T06:44:43.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.404+0000 7ff9fe5af740 1 Processor -- start 2024-04-24T06:44:43.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.404+0000 7ff9fe5af740 1 -- start start 2024-04-24T06:44:43.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.404+0000 7ff9f1600640 1 -- 172.21.15.5:0/1715342414 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138872+0+0 (secure 0 0 0) 0x7ff9dc0046c0 con 0x7ff9f4074cf0 2024-04-24T06:44:43.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.405+0000 7ff9fe5af740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563383796950 0x563383796d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.405+0000 7ff9fe5af740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563383797270 0x5633837a1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.405+0000 7ff9fe5af740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633837a1940 0x5633837a3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.405+0000 7ff9fe5af740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5633836fc1d0 con 0x5633837a1940 2024-04-24T06:44:43.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.405+0000 7ff9fe5af740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5633836a2450 con 0x563383797270 2024-04-24T06:44:43.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.405+0000 7ff9fe5af740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5633836f8190 con 0x563383796950 2024-04-24T06:44:43.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.405+0000 7ff9f9600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563383796950 0x563383796d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.405+0000 7ff9f8c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563383797270 0x5633837a1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.405+0000 7ff9f9600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563383796950 0x563383796d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:37550/0 (socket says 172.21.15.5:37550) 2024-04-24T06:44:43.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.405+0000 7ff9f3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633837a1940 0x5633837a3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.405+0000 7ff9f9600640 1 -- 172.21.15.5:0/178951481 learned_addr learned my addr 172.21.15.5:0/178951481 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:43.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.405+0000 7ff9f8c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563383797270 0x5633837a1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:56830/0 (socket says 172.21.15.5:56830) 2024-04-24T06:44:43.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.405+0000 7ff9f8c00640 1 -- 172.21.15.5:0/178951481 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563383796950 msgr2=0x563383796d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:43.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.405+0000 7ff9f8c00640 1 --2- 172.21.15.5:0/178951481 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563383796950 0x563383796d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.405+0000 7ff9f8c00640 1 -- 172.21.15.5:0/178951481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633837a1940 msgr2=0x5633837a3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:43.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.405+0000 7ff9f8c00640 1 --2- 172.21.15.5:0/178951481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633837a1940 0x5633837a3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.405+0000 7ff9f8c00640 1 -- 172.21.15.5:0/178951481 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56338375f310 con 0x563383797270 2024-04-24T06:44:43.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.406+0000 7ff9f8c00640 1 --2- 172.21.15.5:0/178951481 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563383797270 0x5633837a1400 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4009530 tx=0x7ff9e40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a3e2e6ac7bb3d09 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:43.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.406+0000 7ff9e9c00640 1 -- 172.21.15.5:0/178951481 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e4005020 con 0x563383797270 2024-04-24T06:44:43.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.406+0000 7ff9e9c00640 1 -- 172.21.15.5:0/178951481 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9e40047a0 con 0x563383797270 2024-04-24T06:44:43.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.406+0000 7ff9e9c00640 1 -- 172.21.15.5:0/178951481 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e400f810 con 0x563383797270 2024-04-24T06:44:43.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.406+0000 7ff9f1600640 1 --2- 172.21.15.5:0/1715342414 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ff9d80a3ec0 0x7ff9d80a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.407+0000 7ff9f1600640 1 -- 172.21.15.5:0/1715342414 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7ff9dc0d0df0 con 0x7ff9f4074cf0 2024-04-24T06:44:43.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.407+0000 7ff9fe5af740 1 -- 172.21.15.5:0/178951481 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563383797270 msgr2=0x5633837a1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:43.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.407+0000 7ff9fe5af740 1 --2- 172.21.15.5:0/178951481 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563383797270 0x5633837a1400 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4009530 tx=0x7ff9e40118d0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.407+0000 7ff9f9600640 1 --2- 172.21.15.5:0/1715342414 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ff9d80a3ec0 0x7ff9d80a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.407+0000 7ff9fe5af740 1 -- 172.21.15.5:0/178951481 shutdown_connections 2024-04-24T06:44:43.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.407+0000 7ff9fe5af740 1 --2- 172.21.15.5:0/178951481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633837a1940 0x5633837a3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.407+0000 7ff9fe5af740 1 --2- 172.21.15.5:0/178951481 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563383797270 0x5633837a1400 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.407+0000 7ff9fe5af740 1 --2- 172.21.15.5:0/178951481 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563383796950 0x563383796d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.407+0000 7ff9fe5af740 1 -- 172.21.15.5:0/178951481 >> 172.21.15.5:0/178951481 conn(0x563383792c70 msgr2=0x563383793050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:43.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.407+0000 7ff9fe5af740 1 -- 172.21.15.5:0/178951481 shutdown_connections 2024-04-24T06:44:43.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.407+0000 7ff9fe5af740 1 -- 172.21.15.5:0/178951481 wait complete. 2024-04-24T06:44:43.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.407+0000 7ff9fe5af740 1 -- start start 2024-04-24T06:44:43.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.408+0000 7ff9f9600640 1 --2- 172.21.15.5:0/1715342414 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ff9d80a3ec0 0x7ff9d80a6380 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec0099a0 tx=0x7ff9ec005fd0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:43.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.408+0000 7ff9fe5af740 1 Processor -- start 2024-04-24T06:44:43.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.408+0000 7ff9fe5af740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563383796950 0x5633837a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.408+0000 7ff9fe5af740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563383797270 0x56338379e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.408+0000 7ff9fe5af740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633837a1940 0x56338379ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.408+0000 7ff9fe5af740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5633837a47d0 con 0x563383797270 2024-04-24T06:44:43.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.408+0000 7ff9fe5af740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5633836b5810 con 0x5633837a1940 2024-04-24T06:44:43.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.408+0000 7ff9fe5af740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5633837a4950 con 0x563383796950 2024-04-24T06:44:43.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.408+0000 7ff9fe5af740 10 client.0 _ll_register_callbacks cb 0x563383683140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:44:43.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.409+0000 7ff9f3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563383797270 0x56338379e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.409+0000 7ff9f9600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633837a1940 0x56338379ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.409+0000 7ff9f3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563383797270 0x56338379e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55244/0 (socket says 172.21.15.5:55244) 2024-04-24T06:44:43.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.409+0000 7ff9f3e00640 1 -- 172.21.15.5:0/1364541559 learned_addr learned my addr 172.21.15.5:0/1364541559 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:43.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.409+0000 7ff9f8c00640 1 --2- 172.21.15.5:0/1364541559 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563383796950 0x5633837a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.409+0000 7ff9f3e00640 1 -- 172.21.15.5:0/1364541559 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563383796950 msgr2=0x5633837a13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:43.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.409+0000 7ff9f3e00640 1 --2- 172.21.15.5:0/1364541559 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563383796950 0x5633837a13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.409+0000 7ff9f3e00640 1 -- 172.21.15.5:0/1364541559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633837a1940 msgr2=0x56338379ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:43.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.409+0000 7ff9f3e00640 1 --2- 172.21.15.5:0/1364541559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633837a1940 0x56338379ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.409+0000 7ff9f3e00640 1 -- 172.21.15.5:0/1364541559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633836c63f0 con 0x563383797270 2024-04-24T06:44:43.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.409+0000 7ff9f8c00640 1 --2- 172.21.15.5:0/1364541559 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563383796950 0x5633837a13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:44:43.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.409+0000 7ff9f3e00640 1 --2- 172.21.15.5:0/1364541559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563383797270 0x56338379e6a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc09c7f0 tx=0x7ff9dc0056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b67386b3db6b5ee server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:43.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.409+0000 7ff9e2000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:44:43.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.410+0000 7ff9e2000640 1 -- 172.21.15.5:0/1364541559 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc09c8c0 con 0x563383797270 2024-04-24T06:44:43.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.410+0000 7ff9e2000640 1 -- 172.21.15.5:0/1364541559 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7ff9dc0a52d0 con 0x563383797270 2024-04-24T06:44:43.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.410+0000 7ff9fe5af740 1 -- 172.21.15.5:0/1364541559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9e40047a0 con 0x563383797270 2024-04-24T06:44:43.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.410+0000 7ff9e2000640 1 -- 172.21.15.5:0/1364541559 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ff9dc09c450 con 0x563383797270 2024-04-24T06:44:43.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.410+0000 7ff9fe5af740 10 client.15393 fetch_fsmap learned FSMap version 23 2024-04-24T06:44:43.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.410+0000 7ff9fe5af740 1 -- 172.21.15.5:0/1364541559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x56338340e3f0 con 0x563383797270 2024-04-24T06:44:43.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.410+0000 7ff9e2000640 1 -- 172.21.15.5:0/1364541559 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9e40047a0 con 0x563383797270 2024-04-24T06:44:43.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.410+0000 7ff9e2000640 1 -- 172.21.15.5:0/1364541559 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7ff9dc0d4b80 con 0x563383797270 2024-04-24T06:44:43.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.410+0000 7ff9fe5af740 10 client.15393 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T06:44:43.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.410+0000 7ff9fe5af740 10 client.15393 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T06:44:43.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.410+0000 7ff9fe5af740 20 client.15393 populate_metadata read hostname 'smithi005' 2024-04-24T06:44:43.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.410+0000 7ff9fe5af740 1 --2- 172.21.15.5:0/1364541559 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x5633837b1c00 0x5633837b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.410+0000 7ff9fe5af740 4 client.15393 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:44:43.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.411+0000 7ff9fe5af740 1 -- 172.21.15.5:0/1364541559 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563383709bf0 con 0x5633837b1c00 2024-04-24T06:44:43.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.411+0000 7ff9f8c00640 1 --2- 172.21.15.5:0/1364541559 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x5633837b1c00 0x5633837b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.411+0000 7ff9f8c00640 1 --2- 172.21.15.5:0/1364541559 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x5633837b1c00 0x5633837b3ff0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8d8eebed8e0d17e6 server_cookie=6af17884fa7a9b63 in_seq=0 out_seq=0 2024-04-24T06:44:43.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.411+0000 7ff9e2000640 10 client.15393 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:44:43.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.412+0000 7ff9e2000640 1 -- 172.21.15.5:0/1364541559 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563383709bf0 con 0x5633837b1c00 2024-04-24T06:44:43.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.412+0000 7ff9e2000640 10 client.15393 handle_command_reply: tid=0 2024-04-24T06:44:43.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.412+0000 7ff9fe5af740 1 client.15393 shutdown 2024-04-24T06:44:43.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.412+0000 7ff9fe5af740 1 -- 172.21.15.5:0/1364541559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563383797270 msgr2=0x56338379e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:43.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.412+0000 7ff9fe5af740 1 --2- 172.21.15.5:0/1364541559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563383797270 0x56338379e6a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc09c7f0 tx=0x7ff9dc0056b0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.413+0000 7ff9fe5af740 1 -- 172.21.15.5:0/1364541559 shutdown_connections 2024-04-24T06:44:43.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.413+0000 7ff9fe5af740 1 --2- 172.21.15.5:0/1364541559 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x5633837b1c00 0x5633837b3ff0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.413+0000 7ff9fe5af740 1 --2- 172.21.15.5:0/1364541559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633837a1940 0x56338379ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.413+0000 7ff9fe5af740 1 --2- 172.21.15.5:0/1364541559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563383797270 0x56338379e6a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.413+0000 7ff9fe5af740 1 --2- 172.21.15.5:0/1364541559 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563383796950 0x5633837a13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.413+0000 7ff9e2000640 0 client.15393 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:44:43.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.413+0000 7ff9fe5af740 1 -- 172.21.15.5:0/1364541559 >> 172.21.15.5:0/1364541559 conn(0x563383792c70 msgr2=0x563383798a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:43.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.413+0000 7ff9fe5af740 1 -- 172.21.15.5:0/1364541559 shutdown_connections 2024-04-24T06:44:43.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.413+0000 7ff9fe5af740 1 -- 172.21.15.5:0/1364541559 wait complete. 2024-04-24T06:44:43.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.413+0000 7ff9fe5af740 20 client.15393 trim_cache size 0 max 16384 2024-04-24T06:44:43.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.428+0000 7ff9fe5af740 1 Processor -- start 2024-04-24T06:44:43.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.428+0000 7ff9fe5af740 1 -- start start 2024-04-24T06:44:43.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.429+0000 7ff9fe5af740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563383796950 0x5633837ad690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.429+0000 7ff9fe5af740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563383797270 0x56338379d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.429+0000 7ff9fe5af740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633837a1940 0x5633837a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.429+0000 7ff9fe5af740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f41b2860 con 0x563383796950 2024-04-24T06:44:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.429+0000 7ff9fe5af740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f41b0130 con 0x5633837a1940 2024-04-24T06:44:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.429+0000 7ff9fe5af740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f41adf80 con 0x563383797270 2024-04-24T06:44:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.429+0000 7ff9f9600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633837a1940 0x5633837a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.429+0000 7ff9f8c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563383796950 0x5633837ad690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.429+0000 7ff9f9600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633837a1940 0x5633837a0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:56854/0 (socket says 172.21.15.5:56854) 2024-04-24T06:44:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.429+0000 7ff9f9600640 1 -- 172.21.15.5:0/3920225597 learned_addr learned my addr 172.21.15.5:0/3920225597 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.429+0000 7ff9f3e00640 1 --2- 172.21.15.5:0/3920225597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563383797270 0x56338379d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.430+0000 7ff9f9600640 1 -- 172.21.15.5:0/3920225597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563383797270 msgr2=0x56338379d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.430+0000 7ff9f9600640 1 --2- 172.21.15.5:0/3920225597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563383797270 0x56338379d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.430+0000 7ff9f9600640 1 -- 172.21.15.5:0/3920225597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563383796950 msgr2=0x5633837ad690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.430+0000 7ff9f9600640 1 --2- 172.21.15.5:0/3920225597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563383796950 0x5633837ad690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.430+0000 7ff9f9600640 1 -- 172.21.15.5:0/3920225597 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633834d87d0 con 0x5633837a1940 2024-04-24T06:44:43.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.430+0000 7ff9f9600640 1 --2- 172.21.15.5:0/3920225597 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633837a1940 0x5633837a0ee0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec009be0 tx=0x7ff9ec040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a9d6a84df038875 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:43.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.436+0000 7ff9e9200640 1 -- 172.21.15.5:0/3920225597 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec036040 con 0x5633837a1940 2024-04-24T06:44:43.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.436+0000 7ff9e9200640 1 -- 172.21.15.5:0/3920225597 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9ec050070 con 0x5633837a1940 2024-04-24T06:44:43.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.436+0000 7ff9e9200640 1 -- 172.21.15.5:0/3920225597 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec055810 con 0x5633837a1940 2024-04-24T06:44:43.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.436+0000 7ff9fe5af740 1 -- 172.21.15.5:0/3920225597 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633837a1940 msgr2=0x5633837a0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:43.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.436+0000 7ff9fe5af740 1 --2- 172.21.15.5:0/3920225597 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633837a1940 0x5633837a0ee0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec009be0 tx=0x7ff9ec040000 comp rx=0 tx=0).stop 2024-04-24T06:44:43.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.437+0000 7ff9fe5af740 1 -- 172.21.15.5:0/3920225597 shutdown_connections 2024-04-24T06:44:43.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.437+0000 7ff9fe5af740 1 --2- 172.21.15.5:0/3920225597 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633837a1940 0x5633837a0ee0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.437+0000 7ff9fe5af740 1 --2- 172.21.15.5:0/3920225597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563383797270 0x56338379d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.437+0000 7ff9fe5af740 1 --2- 172.21.15.5:0/3920225597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563383796950 0x5633837ad690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.437+0000 7ff9fe5af740 1 -- 172.21.15.5:0/3920225597 >> 172.21.15.5:0/3920225597 conn(0x563383792c70 msgr2=0x563383798180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:43.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.437+0000 7ff9fe5af740 1 -- 172.21.15.5:0/3920225597 shutdown_connections 2024-04-24T06:44:43.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.437+0000 7ff9fe5af740 1 -- 172.21.15.5:0/3920225597 wait complete. 2024-04-24T06:44:43.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.437+0000 7ff9fe5af740 1 -- start start 2024-04-24T06:44:43.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.437+0000 7ff9fe5af740 1 Processor -- start 2024-04-24T06:44:43.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.438+0000 7ff9fe5af740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563383796950 0x5633837a2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.438+0000 7ff9fe5af740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563383797270 0x5633837a32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.438+0000 7ff9fe5af740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5633837a1940 0x5633837a81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.438+0000 7ff9fe5af740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f41ae5f0 con 0x563383797270 2024-04-24T06:44:43.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.438+0000 7ff9fe5af740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f41afa60 con 0x563383796950 2024-04-24T06:44:43.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.438+0000 7ff9fe5af740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f41aecc0 con 0x5633837a1940 2024-04-24T06:44:43.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.438+0000 7ff9fe5af740 10 client.0 _ll_register_callbacks cb 0x7ff9dc00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:44:43.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.438+0000 7ff9f3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563383797270 0x5633837a32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.438+0000 7ff9f3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563383797270 0x5633837a32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55266/0 (socket says 172.21.15.5:55266) 2024-04-24T06:44:43.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.438+0000 7ff9f3e00640 1 -- 172.21.15.5:0/67829391 learned_addr learned my addr 172.21.15.5:0/67829391 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:43.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.438+0000 7ff9f8c00640 1 --2- 172.21.15.5:0/67829391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563383796950 0x5633837a2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.438+0000 7ff9f9600640 1 --2- 172.21.15.5:0/67829391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5633837a1940 0x5633837a81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.439+0000 7ff9f3e00640 1 -- 172.21.15.5:0/67829391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5633837a1940 msgr2=0x5633837a81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:43.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.439+0000 7ff9f3e00640 1 --2- 172.21.15.5:0/67829391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5633837a1940 0x5633837a81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.439+0000 7ff9f3e00640 1 -- 172.21.15.5:0/67829391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563383796950 msgr2=0x5633837a2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:43.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.439+0000 7ff9f3e00640 1 --2- 172.21.15.5:0/67829391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563383796950 0x5633837a2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.439+0000 7ff9f3e00640 1 -- 172.21.15.5:0/67829391 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633837a86f0 con 0x563383797270 2024-04-24T06:44:43.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.439+0000 7ff9f8c00640 1 --2- 172.21.15.5:0/67829391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563383796950 0x5633837a2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:44:43.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.439+0000 7ff9f3e00640 1 --2- 172.21.15.5:0/67829391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563383797270 0x5633837a32a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc0ce510 tx=0x7ff9dc0b6110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dc0987827b21c81 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:43.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.439+0000 7ff9e2000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:44:43.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.439+0000 7ff9e2000640 1 -- 172.21.15.5:0/67829391 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc09c630 con 0x563383797270 2024-04-24T06:44:43.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.439+0000 7ff9e2000640 1 -- 172.21.15.5:0/67829391 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7ff9dc0a54d0 con 0x563383797270 2024-04-24T06:44:43.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.439+0000 7ff9fe5af740 1 -- 172.21.15.5:0/67829391 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9ec050070 con 0x563383797270 2024-04-24T06:44:43.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.439+0000 7ff9e2000640 1 -- 172.21.15.5:0/67829391 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ff9dc09d1f0 con 0x563383797270 2024-04-24T06:44:43.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.439+0000 7ff9fe5af740 10 client.15399 fetch_fsmap learned FSMap version 23 2024-04-24T06:44:43.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.440+0000 7ff9fe5af740 1 -- 172.21.15.5:0/67829391 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5633838188a0 con 0x563383797270 2024-04-24T06:44:43.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.440+0000 7ff9e2000640 1 -- 172.21.15.5:0/67829391 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9ec050070 con 0x563383797270 2024-04-24T06:44:43.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.440+0000 7ff9e2000640 1 -- 172.21.15.5:0/67829391 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7ff9dc0ac080 con 0x563383797270 2024-04-24T06:44:43.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.440+0000 7ff9fe5af740 10 client.15399 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T06:44:43.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.440+0000 7ff9fe5af740 10 client.15399 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T06:44:43.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.440+0000 7ff9fe5af740 20 client.15399 populate_metadata read hostname 'smithi005' 2024-04-24T06:44:43.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.440+0000 7ff9fe5af740 1 --2- 172.21.15.5:0/67829391 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x563383818ff0 0x56338381b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.440+0000 7ff9fe5af740 4 client.15399 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:44:43.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.440+0000 7ff9fe5af740 1 -- 172.21.15.5:0/67829391 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563383703a60 con 0x563383818ff0 2024-04-24T06:44:43.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.440+0000 7ff9f8c00640 1 --2- 172.21.15.5:0/67829391 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x563383818ff0 0x56338381b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.441+0000 7ff9f8c00640 1 --2- 172.21.15.5:0/67829391 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x563383818ff0 0x56338381b410 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=95a4f72d2c2e20b2 server_cookie=ed5544debce78965 in_seq=0 out_seq=0 2024-04-24T06:44:43.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.441+0000 7ff9e2000640 10 client.15399 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:44:43.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.441+0000 7ff9e2000640 1 -- 172.21.15.5:0/67829391 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563383703a60 con 0x563383818ff0 2024-04-24T06:44:43.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.441+0000 7ff9e2000640 10 client.15399 handle_command_reply: tid=0 2024-04-24T06:44:43.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.441+0000 7ff9fe5af740 1 client.15399 shutdown 2024-04-24T06:44:43.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.441+0000 7ff9fe5af740 1 -- 172.21.15.5:0/67829391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563383797270 msgr2=0x5633837a32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:43.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.441+0000 7ff9fe5af740 1 --2- 172.21.15.5:0/67829391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563383797270 0x5633837a32a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc0ce510 tx=0x7ff9dc0b6110 comp rx=0 tx=0).stop 2024-04-24T06:44:43.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.441+0000 7ff9fe5af740 1 -- 172.21.15.5:0/67829391 shutdown_connections 2024-04-24T06:44:43.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.441+0000 7ff9fe5af740 1 --2- 172.21.15.5:0/67829391 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x563383818ff0 0x56338381b410 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.442+0000 7ff9fe5af740 1 --2- 172.21.15.5:0/67829391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5633837a1940 0x5633837a81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.442+0000 7ff9fe5af740 1 --2- 172.21.15.5:0/67829391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563383797270 0x5633837a32a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.442+0000 7ff9e2000640 0 client.15399 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:44:43.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.442+0000 7ff9fe5af740 1 --2- 172.21.15.5:0/67829391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563383796950 0x5633837a2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.442+0000 7ff9fe5af740 1 -- 172.21.15.5:0/67829391 >> 172.21.15.5:0/67829391 conn(0x563383792c70 msgr2=0x56338379d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:43.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.442+0000 7ff9fe5af740 1 -- 172.21.15.5:0/67829391 shutdown_connections 2024-04-24T06:44:43.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.442+0000 7ff9fe5af740 1 -- 172.21.15.5:0/67829391 wait complete. 2024-04-24T06:44:43.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.442+0000 7ff9fe5af740 20 client.15399 trim_cache size 0 max 16384 2024-04-24T06:44:43.446 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:44:43.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.442+0000 7ff9fb600640 1 -- 172.21.15.5:0/1715342414 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ff9d80a3ec0 msgr2=0x7ff9d80a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:43.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.442+0000 7ff9fb600640 1 --2- 172.21.15.5:0/1715342414 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ff9d80a3ec0 0x7ff9d80a6380 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec0099a0 tx=0x7ff9ec005fd0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.443+0000 7ff9fb600640 1 -- 172.21.15.5:0/1715342414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9f4074cf0 msgr2=0x7ff9f41a0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:43.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.443+0000 7ff9fb600640 1 --2- 172.21.15.5:0/1715342414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9f4074cf0 0x7ff9f41a0cb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc00e9f0 tx=0x7ff9dc00edf0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.443+0000 7ff9fb600640 1 -- 172.21.15.5:0/1715342414 shutdown_connections 2024-04-24T06:44:43.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.443+0000 7ff9fb600640 1 --2- 172.21.15.5:0/1715342414 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ff9d80a3ec0 0x7ff9d80a6380 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.443+0000 7ff9fb600640 1 --2- 172.21.15.5:0/1715342414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9f410e860 0x7ff9f419a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.443+0000 7ff9fb600640 1 --2- 172.21.15.5:0/1715342414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9f4074cf0 0x7ff9f41a0cb0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.443+0000 7ff9fb600640 1 --2- 172.21.15.5:0/1715342414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9f40743d0 0x7ff9f41a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.443+0000 7ff9fb600640 1 -- 172.21.15.5:0/1715342414 >> 172.21.15.5:0/1715342414 conn(0x7ff9f40fdab0 msgr2=0x7ff9f40ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:43.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.443+0000 7ff9fb600640 1 -- 172.21.15.5:0/1715342414 shutdown_connections 2024-04-24T06:44:43.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.443+0000 7ff9fb600640 1 -- 172.21.15.5:0/1715342414 wait complete. 2024-04-24T06:44:43.459 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:43 smithi005 ceph-mon[27798]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:44:43.459 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:43 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1097170729' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:44:43.461 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T06:44:43.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.668+0000 7fb516c00640 1 -- 172.21.15.5:0/3924120486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb510105e10 msgr2=0x7fb510106270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:43.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.668+0000 7fb516c00640 1 --2- 172.21.15.5:0/3924120486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb510105e10 0x7fb510106270 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8009890 tx=0x7fb4f802a370 comp rx=0 tx=0).stop 2024-04-24T06:44:43.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.668+0000 7fb516c00640 1 -- 172.21.15.5:0/3924120486 shutdown_connections 2024-04-24T06:44:43.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.668+0000 7fb516c00640 1 --2- 172.21.15.5:0/3924120486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5101067b0 0x7fb5101111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.668+0000 7fb516c00640 1 --2- 172.21.15.5:0/3924120486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb510105e10 0x7fb510106270 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.668+0000 7fb516c00640 1 --2- 172.21.15.5:0/3924120486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb510104530 0x7fb510104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.668+0000 7fb516c00640 1 -- 172.21.15.5:0/3924120486 >> 172.21.15.5:0/3924120486 conn(0x7fb5100ffc90 msgr2=0x7fb5101020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:43.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.668+0000 7fb516c00640 1 -- 172.21.15.5:0/3924120486 shutdown_connections 2024-04-24T06:44:43.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.668+0000 7fb516c00640 1 -- 172.21.15.5:0/3924120486 wait complete. 2024-04-24T06:44:43.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.669+0000 7fb516c00640 1 Processor -- start 2024-04-24T06:44:43.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.669+0000 7fb516c00640 1 -- start start 2024-04-24T06:44:43.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.670+0000 7fb516c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb510104530 0x7fb5101a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.670+0000 7fb516c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb510105e10 0x7fb5101a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.670+0000 7fb516c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5101067b0 0x7fb51019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.670+0000 7fb516c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb510114680 con 0x7fb5101067b0 2024-04-24T06:44:43.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.670+0000 7fb516c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb510114500 con 0x7fb510104530 2024-04-24T06:44:43.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.670+0000 7fb516c00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb510114800 con 0x7fb510105e10 2024-04-24T06:44:43.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.670+0000 7fb507e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb510105e10 0x7fb5101a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.670+0000 7fb50fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb510104530 0x7fb5101a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.670+0000 7fb507e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb510105e10 0x7fb5101a0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:37590/0 (socket says 172.21.15.5:37590) 2024-04-24T06:44:43.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.670+0000 7fb507e00640 1 -- 172.21.15.5:0/3961037521 learned_addr learned my addr 172.21.15.5:0/3961037521 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:43.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.670+0000 7fb514c00640 1 --2- 172.21.15.5:0/3961037521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5101067b0 0x7fb51019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.670+0000 7fb507e00640 1 -- 172.21.15.5:0/3961037521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb510104530 msgr2=0x7fb5101a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:43.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.670+0000 7fb507e00640 1 --2- 172.21.15.5:0/3961037521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb510104530 0x7fb5101a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.670+0000 7fb507e00640 1 -- 172.21.15.5:0/3961037521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5101067b0 msgr2=0x7fb51019a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:43.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.670+0000 7fb507e00640 1 --2- 172.21.15.5:0/3961037521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5101067b0 0x7fb51019a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.670+0000 7fb507e00640 1 -- 172.21.15.5:0/3961037521 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb51019b160 con 0x7fb510105e10 2024-04-24T06:44:43.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.670+0000 7fb50fe00640 1 --2- 172.21.15.5:0/3961037521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb510104530 0x7fb5101a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:44:43.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.671+0000 7fb514c00640 1 --2- 172.21.15.5:0/3961037521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5101067b0 0x7fb51019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:44:43.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.671+0000 7fb507e00640 1 --2- 172.21.15.5:0/3961037521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb510105e10 0x7fb5101a0d20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8030510 tx=0x7fb4f8033ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:43.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.671+0000 7fb50d600640 1 -- 172.21.15.5:0/3961037521 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f8037040 con 0x7fb510105e10 2024-04-24T06:44:43.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.671+0000 7fb516c00640 1 -- 172.21.15.5:0/3961037521 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb51019b3f0 con 0x7fb510105e10 2024-04-24T06:44:43.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.671+0000 7fb50d600640 1 -- 172.21.15.5:0/3961037521 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4f8040070 con 0x7fb510105e10 2024-04-24T06:44:43.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.671+0000 7fb50d600640 1 -- 172.21.15.5:0/3961037521 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f8003a10 con 0x7fb510105e10 2024-04-24T06:44:43.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.671+0000 7fb516c00640 1 -- 172.21.15.5:0/3961037521 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5101a76a0 con 0x7fb510105e10 2024-04-24T06:44:43.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.674+0000 7fb519b89740 1 Processor -- start 2024-04-24T06:44:43.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.675+0000 7fb519b89740 1 -- start start 2024-04-24T06:44:43.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.675+0000 7fb50d600640 1 -- 172.21.15.5:0/3961037521 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 138872+0+0 (secure 0 0 0) 0x7fb4f8004170 con 0x7fb510105e10 2024-04-24T06:44:43.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.675+0000 7fb519b89740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae18ceb950 0x55ae18cebd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.675+0000 7fb519b89740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae18cec270 0x55ae18cf6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.675+0000 7fb519b89740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae18cf6940 0x55ae18cf8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.675+0000 7fb519b89740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ae18c511d0 con 0x55ae18ceb950 2024-04-24T06:44:43.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.675+0000 7fb519b89740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ae18bf7450 con 0x55ae18cec270 2024-04-24T06:44:43.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.675+0000 7fb519b89740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ae18c4d190 con 0x55ae18cf6940 2024-04-24T06:44:43.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.675+0000 7fb514c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae18ceb950 0x55ae18cebd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.675+0000 7fb514c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae18ceb950 0x55ae18cebd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55290/0 (socket says 172.21.15.5:55290) 2024-04-24T06:44:43.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.675+0000 7fb514c00640 1 -- 172.21.15.5:0/2705472346 learned_addr learned my addr 172.21.15.5:0/2705472346 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:43.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.675+0000 7fb507e00640 1 --2- 172.21.15.5:0/2705472346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae18cf6940 0x55ae18cf8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.675+0000 7fb50fe00640 1 --2- 172.21.15.5:0/2705472346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae18cec270 0x55ae18cf6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.676+0000 7fb50fe00640 1 -- 172.21.15.5:0/2705472346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae18cf6940 msgr2=0x55ae18cf8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:43.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.676+0000 7fb50fe00640 1 --2- 172.21.15.5:0/2705472346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae18cf6940 0x55ae18cf8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.676+0000 7fb50fe00640 1 -- 172.21.15.5:0/2705472346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae18ceb950 msgr2=0x55ae18cebd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:43.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.676+0000 7fb50fe00640 1 --2- 172.21.15.5:0/2705472346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae18ceb950 0x55ae18cebd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.676+0000 7fb50fe00640 1 -- 172.21.15.5:0/2705472346 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae18cb4310 con 0x55ae18cec270 2024-04-24T06:44:43.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.676+0000 7fb514c00640 1 --2- 172.21.15.5:0/2705472346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae18ceb950 0x55ae18cebd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:44:43.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.676+0000 7fb50fe00640 1 --2- 172.21.15.5:0/2705472346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae18cec270 0x55ae18cf6400 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb500009fa0 tx=0x7fb5000118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11f2d341724be97f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:43.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.676+0000 7fb505200640 1 -- 172.21.15.5:0/2705472346 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb500004030 con 0x55ae18cec270 2024-04-24T06:44:43.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.676+0000 7fb505200640 1 -- 172.21.15.5:0/2705472346 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb500005500 con 0x55ae18cec270 2024-04-24T06:44:43.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.676+0000 7fb505200640 1 -- 172.21.15.5:0/2705472346 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb500004db0 con 0x55ae18cec270 2024-04-24T06:44:43.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.677+0000 7fb50d600640 1 --2- 172.21.15.5:0/3961037521 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb4f40a3ec0 0x7fb4f40a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.677+0000 7fb50d600640 1 -- 172.21.15.5:0/3961037521 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fb4f80f09e0 con 0x7fb510105e10 2024-04-24T06:44:43.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.677+0000 7fb519b89740 1 -- 172.21.15.5:0/2705472346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae18cec270 msgr2=0x55ae18cf6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:43.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.677+0000 7fb519b89740 1 --2- 172.21.15.5:0/2705472346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae18cec270 0x55ae18cf6400 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb500009fa0 tx=0x7fb5000118d0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.677+0000 7fb514c00640 1 --2- 172.21.15.5:0/3961037521 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb4f40a3ec0 0x7fb4f40a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.677+0000 7fb519b89740 1 -- 172.21.15.5:0/2705472346 shutdown_connections 2024-04-24T06:44:43.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.677+0000 7fb519b89740 1 --2- 172.21.15.5:0/2705472346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae18cf6940 0x55ae18cf8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.677+0000 7fb519b89740 1 --2- 172.21.15.5:0/2705472346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae18cec270 0x55ae18cf6400 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.677+0000 7fb519b89740 1 --2- 172.21.15.5:0/2705472346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae18ceb950 0x55ae18cebd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.677+0000 7fb519b89740 1 -- 172.21.15.5:0/2705472346 >> 172.21.15.5:0/2705472346 conn(0x55ae18ce7c70 msgr2=0x55ae18ce8050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:43.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.677+0000 7fb519b89740 1 -- 172.21.15.5:0/2705472346 shutdown_connections 2024-04-24T06:44:43.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.677+0000 7fb519b89740 1 -- 172.21.15.5:0/2705472346 wait complete. 2024-04-24T06:44:43.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.678+0000 7fb519b89740 1 -- start start 2024-04-24T06:44:43.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.678+0000 7fb514c00640 1 --2- 172.21.15.5:0/3961037521 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb4f40a3ec0 0x7fb4f40a6380 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb508009a60 tx=0x7fb5080095c0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:43.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.678+0000 7fb519b89740 1 Processor -- start 2024-04-24T06:44:43.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.679+0000 7fb519b89740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae18ceb950 0x55ae18cf6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.679+0000 7fb519b89740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae18cec270 0x55ae18cf3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.679+0000 7fb519b89740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae18cf6940 0x55ae18cf3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.679+0000 7fb519b89740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ae18cf97d0 con 0x55ae18cec270 2024-04-24T06:44:43.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.679+0000 7fb519b89740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ae18c0a810 con 0x55ae18ceb950 2024-04-24T06:44:43.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.679+0000 7fb519b89740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ae18cf9950 con 0x55ae18cf6940 2024-04-24T06:44:43.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.679+0000 7fb519b89740 10 client.0 _ll_register_callbacks cb 0x55ae18bd8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:44:43.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.679+0000 7fb514c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae18cf6940 0x55ae18cf3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.679+0000 7fb50fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae18ceb950 0x55ae18cf6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.679+0000 7fb514c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae18cf6940 0x55ae18cf3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:37594/0 (socket says 172.21.15.5:37594) 2024-04-24T06:44:43.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.679+0000 7fb514c00640 1 -- 172.21.15.5:0/810511596 learned_addr learned my addr 172.21.15.5:0/810511596 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:43.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.679+0000 7fb507e00640 1 --2- 172.21.15.5:0/810511596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae18cec270 0x55ae18cf3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.679+0000 7fb514c00640 1 -- 172.21.15.5:0/810511596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae18ceb950 msgr2=0x55ae18cf6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:43.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.679+0000 7fb514c00640 1 --2- 172.21.15.5:0/810511596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae18ceb950 0x55ae18cf6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.679+0000 7fb514c00640 1 -- 172.21.15.5:0/810511596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae18cec270 msgr2=0x55ae18cf3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:43.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.679+0000 7fb514c00640 1 --2- 172.21.15.5:0/810511596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae18cec270 0x55ae18cf3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.679+0000 7fb514c00640 1 -- 172.21.15.5:0/810511596 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae18c1b3f0 con 0x55ae18cf6940 2024-04-24T06:44:43.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.679+0000 7fb507e00640 1 --2- 172.21.15.5:0/810511596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae18cec270 0x55ae18cf3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:44:43.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.680+0000 7fb514c00640 1 --2- 172.21.15.5:0/810511596 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae18cf6940 0x55ae18cf3da0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb50801cc00 tx=0x7fb508004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31d13d67f8e8ae76 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:43.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.680+0000 7fb4fd600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:44:43.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.680+0000 7fb4fd600640 1 -- 172.21.15.5:0/810511596 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50802c060 con 0x55ae18cf6940 2024-04-24T06:44:43.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.680+0000 7fb4fd600640 1 -- 172.21.15.5:0/810511596 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fb508016030 con 0x55ae18cf6940 2024-04-24T06:44:43.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.680+0000 7fb519b89740 1 -- 172.21.15.5:0/810511596 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb500005500 con 0x55ae18cf6940 2024-04-24T06:44:43.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.680+0000 7fb4fd600640 1 -- 172.21.15.5:0/810511596 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb50801cdb0 con 0x55ae18cf6940 2024-04-24T06:44:43.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.680+0000 7fb519b89740 10 client.25534 fetch_fsmap learned FSMap version 23 2024-04-24T06:44:43.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.680+0000 7fb519b89740 1 -- 172.21.15.5:0/810511596 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55ae189633f0 con 0x55ae18cf6940 2024-04-24T06:44:43.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.681+0000 7fb4fd600640 1 -- 172.21.15.5:0/810511596 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb500005500 con 0x55ae18cf6940 2024-04-24T06:44:43.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.681+0000 7fb4fd600640 1 -- 172.21.15.5:0/810511596 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fb50800b040 con 0x55ae18cf6940 2024-04-24T06:44:43.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.681+0000 7fb519b89740 10 client.25534 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T06:44:43.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.681+0000 7fb519b89740 10 client.25534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:44:43.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.681+0000 7fb519b89740 20 client.25534 populate_metadata read hostname 'smithi005' 2024-04-24T06:44:43.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.681+0000 7fb519b89740 1 --2- 172.21.15.5:0/810511596 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55ae18d06b40 0x55ae18d08f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.681+0000 7fb519b89740 4 client.25534 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:44:43.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.681+0000 7fb519b89740 1 -- 172.21.15.5:0/810511596 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae18c5ebf0 con 0x55ae18d06b40 2024-04-24T06:44:43.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.682+0000 7fb50fe00640 1 --2- 172.21.15.5:0/810511596 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55ae18d06b40 0x55ae18d08f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.682+0000 7fb50fe00640 1 --2- 172.21.15.5:0/810511596 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55ae18d06b40 0x55ae18d08f90 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a943140640a8aa6 server_cookie=e6a7a939f813577f in_seq=0 out_seq=0 2024-04-24T06:44:43.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.682+0000 7fb4fd600640 10 client.25534 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:44:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.683+0000 7fb4fd600640 1 -- 172.21.15.5:0/810511596 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ae18c5ebf0 con 0x55ae18d06b40 2024-04-24T06:44:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.683+0000 7fb4fd600640 10 client.25534 handle_command_reply: tid=0 2024-04-24T06:44:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.683+0000 7fb519b89740 1 client.25534 shutdown 2024-04-24T06:44:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.683+0000 7fb519b89740 1 -- 172.21.15.5:0/810511596 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae18cf6940 msgr2=0x55ae18cf3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.683+0000 7fb519b89740 1 --2- 172.21.15.5:0/810511596 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae18cf6940 0x55ae18cf3da0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb50801cc00 tx=0x7fb508004ab0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.684+0000 7fb519b89740 1 -- 172.21.15.5:0/810511596 shutdown_connections 2024-04-24T06:44:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.684+0000 7fb519b89740 1 --2- 172.21.15.5:0/810511596 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55ae18d06b40 0x55ae18d08f90 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.684+0000 7fb519b89740 1 --2- 172.21.15.5:0/810511596 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae18cf6940 0x55ae18cf3da0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.684+0000 7fb519b89740 1 --2- 172.21.15.5:0/810511596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae18cec270 0x55ae18cf3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.684+0000 7fb4fd600640 0 client.25534 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:44:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.684+0000 7fb519b89740 1 --2- 172.21.15.5:0/810511596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae18ceb950 0x55ae18cf6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.684+0000 7fb519b89740 1 -- 172.21.15.5:0/810511596 >> 172.21.15.5:0/810511596 conn(0x55ae18ce7c70 msgr2=0x55ae18ceda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.684+0000 7fb519b89740 1 -- 172.21.15.5:0/810511596 shutdown_connections 2024-04-24T06:44:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.684+0000 7fb519b89740 1 -- 172.21.15.5:0/810511596 wait complete. 2024-04-24T06:44:43.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.684+0000 7fb519b89740 20 client.25534 trim_cache size 0 max 16384 2024-04-24T06:44:43.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:43 smithi137 ceph-mon[33017]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:44:43.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:43 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/1097170729' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:44:43.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.699+0000 7fb519b89740 1 Processor -- start 2024-04-24T06:44:43.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.699+0000 7fb519b89740 1 -- start start 2024-04-24T06:44:43.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.699+0000 7fb519b89740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae18ceb950 0x55ae18d02690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.699+0000 7fb519b89740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae18cec270 0x55ae18cf5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.699+0000 7fb519b89740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae18cf6940 0x55ae18cf63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.699+0000 7fb519b89740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb5101b2790 con 0x55ae18cf6940 2024-04-24T06:44:43.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.699+0000 7fb519b89740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb5101b0060 con 0x55ae18ceb950 2024-04-24T06:44:43.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.699+0000 7fb519b89740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb5101adeb0 con 0x55ae18cec270 2024-04-24T06:44:43.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.700+0000 7fb50fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae18ceb950 0x55ae18d02690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.700+0000 7fb514c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae18cf6940 0x55ae18cf63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.700+0000 7fb50fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae18ceb950 0x55ae18d02690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:56926/0 (socket says 172.21.15.5:56926) 2024-04-24T06:44:43.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.700+0000 7fb50fe00640 1 -- 172.21.15.5:0/224967543 learned_addr learned my addr 172.21.15.5:0/224967543 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:43.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.700+0000 7fb507e00640 1 --2- 172.21.15.5:0/224967543 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae18cec270 0x55ae18cf5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.700+0000 7fb507e00640 1 -- 172.21.15.5:0/224967543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae18ceb950 msgr2=0x55ae18d02690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:43.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.700+0000 7fb507e00640 1 --2- 172.21.15.5:0/224967543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae18ceb950 0x55ae18d02690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.700+0000 7fb507e00640 1 -- 172.21.15.5:0/224967543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae18cf6940 msgr2=0x55ae18cf63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:43.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.700+0000 7fb507e00640 1 --2- 172.21.15.5:0/224967543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae18cf6940 0x55ae18cf63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.700+0000 7fb507e00640 1 -- 172.21.15.5:0/224967543 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae18a2d7d0 con 0x55ae18cec270 2024-04-24T06:44:43.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.700+0000 7fb50fe00640 1 --2- 172.21.15.5:0/224967543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae18ceb950 0x55ae18d02690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:44:43.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.700+0000 7fb507e00640 1 --2- 172.21.15.5:0/224967543 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae18cec270 0x55ae18cf5ea0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb4f80ee070 tx=0x7fb4f80f64d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9153f37af7deb32b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:43.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.701+0000 7fb4ffe00640 1 -- 172.21.15.5:0/224967543 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f80b8a90 con 0x55ae18cec270 2024-04-24T06:44:43.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.701+0000 7fb4ffe00640 1 -- 172.21.15.5:0/224967543 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4f80f60d0 con 0x55ae18cec270 2024-04-24T06:44:43.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.701+0000 7fb4ffe00640 1 -- 172.21.15.5:0/224967543 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f80bc150 con 0x55ae18cec270 2024-04-24T06:44:43.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.701+0000 7fb519b89740 1 -- 172.21.15.5:0/224967543 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae18cec270 msgr2=0x55ae18cf5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:43.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.701+0000 7fb519b89740 1 --2- 172.21.15.5:0/224967543 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae18cec270 0x55ae18cf5ea0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb4f80ee070 tx=0x7fb4f80f64d0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.701+0000 7fb519b89740 1 -- 172.21.15.5:0/224967543 shutdown_connections 2024-04-24T06:44:43.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.701+0000 7fb519b89740 1 --2- 172.21.15.5:0/224967543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae18cf6940 0x55ae18cf63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.701+0000 7fb519b89740 1 --2- 172.21.15.5:0/224967543 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae18cec270 0x55ae18cf5ea0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.701+0000 7fb519b89740 1 --2- 172.21.15.5:0/224967543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae18ceb950 0x55ae18d02690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.701+0000 7fb519b89740 1 -- 172.21.15.5:0/224967543 >> 172.21.15.5:0/224967543 conn(0x55ae18ce7c70 msgr2=0x55ae18cee940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.702+0000 7fb519b89740 1 -- 172.21.15.5:0/224967543 shutdown_connections 2024-04-24T06:44:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.702+0000 7fb519b89740 1 -- 172.21.15.5:0/224967543 wait complete. 2024-04-24T06:44:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.702+0000 7fb519b89740 1 -- start start 2024-04-24T06:44:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.702+0000 7fb519b89740 1 Processor -- start 2024-04-24T06:44:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.703+0000 7fb519b89740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae18ceb950 0x55ae18cf7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.703+0000 7fb519b89740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae18cec270 0x55ae18cf7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.703+0000 7fb519b89740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae18cf6940 0x55ae18cfd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.703+0000 7fb519b89740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb5101ae520 con 0x55ae18ceb950 2024-04-24T06:44:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.703+0000 7fb519b89740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb5101af990 con 0x55ae18cec270 2024-04-24T06:44:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.703+0000 7fb519b89740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb5101aebf0 con 0x55ae18cf6940 2024-04-24T06:44:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.703+0000 7fb519b89740 10 client.0 _ll_register_callbacks cb 0x7fb508009d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:44:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.703+0000 7fb50fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae18ceb950 0x55ae18cf7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.703+0000 7fb50fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae18ceb950 0x55ae18cf7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55326/0 (socket says 172.21.15.5:55326) 2024-04-24T06:44:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.703+0000 7fb50fe00640 1 -- 172.21.15.5:0/2696996312 learned_addr learned my addr 172.21.15.5:0/2696996312 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.703+0000 7fb514c00640 1 --2- 172.21.15.5:0/2696996312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae18cf6940 0x55ae18cfd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.703+0000 7fb507e00640 1 --2- 172.21.15.5:0/2696996312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae18cec270 0x55ae18cf7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.703+0000 7fb50fe00640 1 -- 172.21.15.5:0/2696996312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae18cf6940 msgr2=0x55ae18cfd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.703+0000 7fb50fe00640 1 --2- 172.21.15.5:0/2696996312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae18cf6940 0x55ae18cfd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.703+0000 7fb50fe00640 1 -- 172.21.15.5:0/2696996312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae18cec270 msgr2=0x55ae18cf7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.703+0000 7fb50fe00640 1 --2- 172.21.15.5:0/2696996312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae18cec270 0x55ae18cf7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.703+0000 7fb50fe00640 1 -- 172.21.15.5:0/2696996312 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae18cf8840 con 0x55ae18ceb950 2024-04-24T06:44:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.704+0000 7fb50fe00640 1 --2- 172.21.15.5:0/2696996312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae18ceb950 0x55ae18cf7a00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb500000c00 tx=0x7fb5000045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97b67d533ce45bcd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.704+0000 7fb4fd600640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:44:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.704+0000 7fb4fd600640 1 -- 172.21.15.5:0/2696996312 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50001d040 con 0x55ae18ceb950 2024-04-24T06:44:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.704+0000 7fb4fd600640 1 -- 172.21.15.5:0/2696996312 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fb50002cb50 con 0x55ae18ceb950 2024-04-24T06:44:43.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.704+0000 7fb519b89740 1 -- 172.21.15.5:0/2696996312 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4f80f60d0 con 0x55ae18ceb950 2024-04-24T06:44:43.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.704+0000 7fb4fd600640 1 -- 172.21.15.5:0/2696996312 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb50003d050 con 0x55ae18ceb950 2024-04-24T06:44:43.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.704+0000 7fb519b89740 10 client.15414 fetch_fsmap learned FSMap version 23 2024-04-24T06:44:43.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.704+0000 7fb519b89740 1 -- 172.21.15.5:0/2696996312 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55ae18d6d700 con 0x55ae18ceb950 2024-04-24T06:44:43.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.704+0000 7fb4fd600640 1 -- 172.21.15.5:0/2696996312 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4f80f60d0 con 0x55ae18ceb950 2024-04-24T06:44:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.705+0000 7fb4fd600640 1 -- 172.21.15.5:0/2696996312 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fb500023040 con 0x55ae18ceb950 2024-04-24T06:44:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.705+0000 7fb519b89740 10 client.15414 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T06:44:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.705+0000 7fb519b89740 10 client.15414 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:44:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.705+0000 7fb519b89740 20 client.15414 populate_metadata read hostname 'smithi005' 2024-04-24T06:44:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.705+0000 7fb519b89740 1 --2- 172.21.15.5:0/2696996312 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55ae18d6dd40 0x55ae18d70160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.705+0000 7fb519b89740 4 client.15414 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:44:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.705+0000 7fb519b89740 1 -- 172.21.15.5:0/2696996312 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ae18c58a60 con 0x55ae18d6dd40 2024-04-24T06:44:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.705+0000 7fb507e00640 1 --2- 172.21.15.5:0/2696996312 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55ae18d6dd40 0x55ae18d70160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.706+0000 7fb507e00640 1 --2- 172.21.15.5:0/2696996312 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55ae18d6dd40 0x55ae18d70160 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71b9196a17845389 server_cookie=e4825ae9c2378086 in_seq=0 out_seq=0 2024-04-24T06:44:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.706+0000 7fb4fd600640 10 client.15414 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:44:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.706+0000 7fb4fd600640 1 -- 172.21.15.5:0/2696996312 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ae18c58a60 con 0x55ae18d6dd40 2024-04-24T06:44:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.706+0000 7fb4fd600640 10 client.15414 handle_command_reply: tid=0 2024-04-24T06:44:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.706+0000 7fb519b89740 1 client.15414 shutdown 2024-04-24T06:44:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.706+0000 7fb519b89740 1 -- 172.21.15.5:0/2696996312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae18ceb950 msgr2=0x55ae18cf7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:43.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.706+0000 7fb519b89740 1 --2- 172.21.15.5:0/2696996312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae18ceb950 0x55ae18cf7a00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb500000c00 tx=0x7fb5000045f0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.706+0000 7fb519b89740 1 -- 172.21.15.5:0/2696996312 shutdown_connections 2024-04-24T06:44:43.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.706+0000 7fb519b89740 1 --2- 172.21.15.5:0/2696996312 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55ae18d6dd40 0x55ae18d70160 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.707+0000 7fb519b89740 1 --2- 172.21.15.5:0/2696996312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae18cf6940 0x55ae18cfd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.707+0000 7fb519b89740 1 --2- 172.21.15.5:0/2696996312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae18cec270 0x55ae18cf7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.707+0000 7fb4fd600640 0 client.15414 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:44:43.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.707+0000 7fb519b89740 1 --2- 172.21.15.5:0/2696996312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae18ceb950 0x55ae18cf7a00 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.707+0000 7fb519b89740 1 -- 172.21.15.5:0/2696996312 >> 172.21.15.5:0/2696996312 conn(0x55ae18ce7c70 msgr2=0x55ae18ce6560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:43.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.707+0000 7fb519b89740 1 -- 172.21.15.5:0/2696996312 shutdown_connections 2024-04-24T06:44:43.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.707+0000 7fb519b89740 1 -- 172.21.15.5:0/2696996312 wait complete. 2024-04-24T06:44:43.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.707+0000 7fb519b89740 20 client.15414 trim_cache size 0 max 16384 2024-04-24T06:44:43.711 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:44:43.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.707+0000 7fb516c00640 1 -- 172.21.15.5:0/3961037521 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb4f40a3ec0 msgr2=0x7fb4f40a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:43.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.707+0000 7fb516c00640 1 --2- 172.21.15.5:0/3961037521 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb4f40a3ec0 0x7fb4f40a6380 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb508009a60 tx=0x7fb5080095c0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.707+0000 7fb516c00640 1 -- 172.21.15.5:0/3961037521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb510105e10 msgr2=0x7fb5101a0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:43.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.707+0000 7fb516c00640 1 --2- 172.21.15.5:0/3961037521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb510105e10 0x7fb5101a0d20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8030510 tx=0x7fb4f8033ae0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.708+0000 7fb516c00640 1 -- 172.21.15.5:0/3961037521 shutdown_connections 2024-04-24T06:44:43.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.708+0000 7fb516c00640 1 --2- 172.21.15.5:0/3961037521 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb4f40a3ec0 0x7fb4f40a6380 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.708+0000 7fb516c00640 1 --2- 172.21.15.5:0/3961037521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5101067b0 0x7fb51019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.708+0000 7fb516c00640 1 --2- 172.21.15.5:0/3961037521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb510105e10 0x7fb5101a0d20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.708+0000 7fb516c00640 1 --2- 172.21.15.5:0/3961037521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb510104530 0x7fb5101a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:43.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.708+0000 7fb516c00640 1 -- 172.21.15.5:0/3961037521 >> 172.21.15.5:0/3961037521 conn(0x7fb5100ffc90 msgr2=0x7fb510100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:43.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.708+0000 7fb516c00640 1 -- 172.21.15.5:0/3961037521 shutdown_connections 2024-04-24T06:44:43.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:43.708+0000 7fb516c00640 1 -- 172.21.15.5:0/3961037521 wait complete. 2024-04-24T06:44:43.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:43 smithi037 ceph-mon[32864]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:44:43.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:43 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/1097170729' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:44:44.727 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T06:44:44.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:44:44.728 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T06:44:44.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.958+0000 7f9d4c000640 1 -- 172.21.15.5:0/1439602723 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d44108720 msgr2=0x7f9d44108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:44.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.958+0000 7f9d4c000640 1 --2- 172.21.15.5:0/1439602723 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d44108720 0x7f9d44108b80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9d30009900 tx=0x7f9d3002a490 comp rx=0 tx=0).stop 2024-04-24T06:44:44.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.958+0000 7f9d4c000640 1 -- 172.21.15.5:0/1439602723 shutdown_connections 2024-04-24T06:44:44.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.958+0000 7f9d4c000640 1 --2- 172.21.15.5:0/1439602723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d441090c0 0x7f9d44115570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:44.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.958+0000 7f9d4c000640 1 --2- 172.21.15.5:0/1439602723 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d44108720 0x7f9d44108b80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:44.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.958+0000 7f9d4c000640 1 --2- 172.21.15.5:0/1439602723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d4410a590 0x7f9d4410a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:44.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.958+0000 7f9d4c000640 1 -- 172.21.15.5:0/1439602723 >> 172.21.15.5:0/1439602723 conn(0x7f9d44077e70 msgr2=0x7f9d44106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:44.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.958+0000 7f9d4c000640 1 -- 172.21.15.5:0/1439602723 shutdown_connections 2024-04-24T06:44:44.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.958+0000 7f9d4c000640 1 -- 172.21.15.5:0/1439602723 wait complete. 2024-04-24T06:44:44.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.959+0000 7f9d4c000640 1 Processor -- start 2024-04-24T06:44:44.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.960+0000 7f9d4c000640 1 -- start start 2024-04-24T06:44:44.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.960+0000 7f9d4c000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d44108720 0x7f9d4419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:44.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.960+0000 7f9d4c000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d441090c0 0x7f9d4419f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:44.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.960+0000 7f9d4c000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d4410a590 0x7f9d441a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:44.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.960+0000 7f9d4c000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9d44118b40 con 0x7f9d44108720 2024-04-24T06:44:44.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.960+0000 7f9d4c000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9d441189c0 con 0x7f9d4410a590 2024-04-24T06:44:44.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.960+0000 7f9d4c000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9d44118cc0 con 0x7f9d441090c0 2024-04-24T06:44:44.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.960+0000 7f9d4a000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d4410a590 0x7f9d441a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:44.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.960+0000 7f9d48c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d441090c0 0x7f9d4419f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:44.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.960+0000 7f9d49600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d44108720 0x7f9d4419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:44.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.961+0000 7f9d4a000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d4410a590 0x7f9d441a3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:60236/0 (socket says 172.21.15.5:60236) 2024-04-24T06:44:44.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.961+0000 7f9d4a000640 1 -- 172.21.15.5:0/1169979752 learned_addr learned my addr 172.21.15.5:0/1169979752 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:44.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.961+0000 7f9d48c00640 1 --2- 172.21.15.5:0/1169979752 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d441090c0 0x7f9d4419f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:34420/0 (socket says 172.21.15.5:34420) 2024-04-24T06:44:44.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.961+0000 7f9d4a000640 1 -- 172.21.15.5:0/1169979752 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d441090c0 msgr2=0x7f9d4419f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:44.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.961+0000 7f9d4a000640 1 --2- 172.21.15.5:0/1169979752 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d441090c0 0x7f9d4419f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:44.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.961+0000 7f9d4a000640 1 -- 172.21.15.5:0/1169979752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d44108720 msgr2=0x7f9d4419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:44.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.961+0000 7f9d4a000640 1 --2- 172.21.15.5:0/1169979752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d44108720 0x7f9d4419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:44.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.961+0000 7f9d4a000640 1 -- 172.21.15.5:0/1169979752 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d441a3f70 con 0x7f9d4410a590 2024-04-24T06:44:44.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.961+0000 7f9d4a000640 1 --2- 172.21.15.5:0/1169979752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d4410a590 0x7f9d441a3840 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c00da40 tx=0x7f9d3c00df00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:44.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.962+0000 7f9d42000640 1 -- 172.21.15.5:0/1169979752 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c017070 con 0x7f9d4410a590 2024-04-24T06:44:44.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.962+0000 7f9d42000640 1 -- 172.21.15.5:0/1169979752 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d3c003e50 con 0x7f9d4410a590 2024-04-24T06:44:44.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.962+0000 7f9d4c000640 1 -- 172.21.15.5:0/1169979752 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d441a4260 con 0x7f9d4410a590 2024-04-24T06:44:44.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.962+0000 7f9d42000640 1 -- 172.21.15.5:0/1169979752 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c010460 con 0x7f9d4410a590 2024-04-24T06:44:44.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.962+0000 7f9d4c000640 1 -- 172.21.15.5:0/1169979752 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d441abaf0 con 0x7f9d4410a590 2024-04-24T06:44:44.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.965+0000 7f9d4ed66740 1 Processor -- start 2024-04-24T06:44:44.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.965+0000 7f9d42000640 1 -- 172.21.15.5:0/1169979752 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138872+0+0 (secure 0 0 0) 0x7f9d3c003970 con 0x7f9d4410a590 2024-04-24T06:44:44.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.965+0000 7f9d4ed66740 1 -- start start 2024-04-24T06:44:44.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.966+0000 7f9d4ed66740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7a2c97090 0x55c7a2c97470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:44.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.966+0000 7f9d4ed66740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7a2c979b0 0x55c7a2ca1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:44.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.966+0000 7f9d4ed66740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7a2ca2080 0x55c7a2ca4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:44.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.966+0000 7f9d4ed66740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c7a2bfce70 con 0x55c7a2c97090 2024-04-24T06:44:44.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.966+0000 7f9d4ed66740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c7a2ba3380 con 0x55c7a2ca2080 2024-04-24T06:44:44.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.966+0000 7f9d4ed66740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c7a2bf8e30 con 0x55c7a2c979b0 2024-04-24T06:44:44.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.966+0000 7f9d48c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7a2c97090 0x55c7a2c97470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:44.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.966+0000 7f9d49600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7a2c979b0 0x55c7a2ca1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:44.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.966+0000 7f9d48c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7a2c97090 0x55c7a2c97470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51954/0 (socket says 172.21.15.5:51954) 2024-04-24T06:44:44.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.966+0000 7f9d48c00640 1 -- 172.21.15.5:0/2534531615 learned_addr learned my addr 172.21.15.5:0/2534531615 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:44.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.967+0000 7f9d48c00640 1 --2- 172.21.15.5:0/2534531615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7a2ca2080 0x55c7a2ca4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:44.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.967+0000 7f9d49600640 1 -- 172.21.15.5:0/2534531615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7a2ca2080 msgr2=0x55c7a2ca4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:44.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.967+0000 7f9d49600640 1 --2- 172.21.15.5:0/2534531615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7a2ca2080 0x55c7a2ca4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:44.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.967+0000 7f9d49600640 1 -- 172.21.15.5:0/2534531615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7a2c97090 msgr2=0x55c7a2c97470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:44.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.967+0000 7f9d49600640 1 --2- 172.21.15.5:0/2534531615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7a2c97090 0x55c7a2c97470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:44.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.967+0000 7f9d49600640 1 -- 172.21.15.5:0/2534531615 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7a29d77e0 con 0x55c7a2c979b0 2024-04-24T06:44:44.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.967+0000 7f9d48c00640 1 --2- 172.21.15.5:0/2534531615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7a2ca2080 0x55c7a2ca4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:44:44.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.967+0000 7f9d42000640 1 --2- 172.21.15.5:0/1169979752 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f9d2c0a3e90 0x7f9d2c0a6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:44.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.967+0000 7f9d42000640 1 -- 172.21.15.5:0/1169979752 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f9d3c0d06f0 con 0x7f9d4410a590 2024-04-24T06:44:44.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.967+0000 7f9d49600640 1 --2- 172.21.15.5:0/2534531615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7a2c979b0 0x55c7a2ca1b40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9d38009c20 tx=0x7f9d38011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68b5d4ae506637f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:44.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.967+0000 7f9d48c00640 1 --2- 172.21.15.5:0/1169979752 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f9d2c0a3e90 0x7f9d2c0a6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:44.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.968+0000 7f9d36600640 1 -- 172.21.15.5:0/2534531615 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d38004050 con 0x55c7a2c979b0 2024-04-24T06:44:44.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.968+0000 7f9d36600640 1 -- 172.21.15.5:0/2534531615 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d38005500 con 0x55c7a2c979b0 2024-04-24T06:44:44.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.968+0000 7f9d36600640 1 -- 172.21.15.5:0/2534531615 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d38004d60 con 0x55c7a2c979b0 2024-04-24T06:44:44.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.968+0000 7f9d4ed66740 1 -- 172.21.15.5:0/2534531615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7a2c979b0 msgr2=0x55c7a2ca1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:44.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.968+0000 7f9d48c00640 1 --2- 172.21.15.5:0/1169979752 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f9d2c0a3e90 0x7f9d2c0a6350 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9d30005c40 tx=0x7f9d30005b50 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:44.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.968+0000 7f9d4ed66740 1 --2- 172.21.15.5:0/2534531615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7a2c979b0 0x55c7a2ca1b40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9d38009c20 tx=0x7f9d38011810 comp rx=0 tx=0).stop 2024-04-24T06:44:44.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.969+0000 7f9d4ed66740 1 -- 172.21.15.5:0/2534531615 shutdown_connections 2024-04-24T06:44:44.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.969+0000 7f9d4ed66740 1 --2- 172.21.15.5:0/2534531615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7a2ca2080 0x55c7a2ca4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:44.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.969+0000 7f9d4ed66740 1 --2- 172.21.15.5:0/2534531615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7a2c979b0 0x55c7a2ca1b40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:44.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.969+0000 7f9d4ed66740 1 --2- 172.21.15.5:0/2534531615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7a2c97090 0x55c7a2c97470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:44.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.969+0000 7f9d4ed66740 1 -- 172.21.15.5:0/2534531615 >> 172.21.15.5:0/2534531615 conn(0x55c7a2c933b0 msgr2=0x55c7a2c93790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:44.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.969+0000 7f9d4ed66740 1 -- 172.21.15.5:0/2534531615 shutdown_connections 2024-04-24T06:44:44.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.969+0000 7f9d4ed66740 1 -- 172.21.15.5:0/2534531615 wait complete. 2024-04-24T06:44:44.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.969+0000 7f9d4ed66740 1 -- start start 2024-04-24T06:44:44.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.969+0000 7f9d4ed66740 1 Processor -- start 2024-04-24T06:44:44.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.970+0000 7f9d4ed66740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7a2c97090 0x55c7a2c9efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:44.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.970+0000 7f9d4ed66740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7a2c979b0 0x55c7a2c9f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:44.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.970+0000 7f9d4ed66740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7a2ca2080 0x55c7a2c9fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:44.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.970+0000 7f9d4ed66740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c7a2ca4e70 con 0x55c7a2c97090 2024-04-24T06:44:44.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.970+0000 7f9d4ed66740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c7a2bb6660 con 0x55c7a2ca2080 2024-04-24T06:44:44.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.970+0000 7f9d4ed66740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c7a2ca4ff0 con 0x55c7a2c979b0 2024-04-24T06:44:44.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.970+0000 7f9d4ed66740 10 client.0 _ll_register_callbacks cb 0x55c7a2ba4aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:44:44.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.970+0000 7f9d49600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7a2c97090 0x55c7a2c9efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:44.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.970+0000 7f9d4a000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7a2ca2080 0x55c7a2c9fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:44.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.970+0000 7f9d49600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7a2c97090 0x55c7a2c9efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51968/0 (socket says 172.21.15.5:51968) 2024-04-24T06:44:44.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.970+0000 7f9d49600640 1 -- 172.21.15.5:0/4267261330 learned_addr learned my addr 172.21.15.5:0/4267261330 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:44.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.970+0000 7f9d48c00640 1 --2- 172.21.15.5:0/4267261330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7a2c979b0 0x55c7a2c9f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:44.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.971+0000 7f9d4a000640 1 -- 172.21.15.5:0/4267261330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7a2c979b0 msgr2=0x55c7a2c9f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:44.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.971+0000 7f9d4a000640 1 --2- 172.21.15.5:0/4267261330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7a2c979b0 0x55c7a2c9f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:44.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.971+0000 7f9d4a000640 1 -- 172.21.15.5:0/4267261330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7a2c97090 msgr2=0x55c7a2c9efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:44.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.971+0000 7f9d4a000640 1 --2- 172.21.15.5:0/4267261330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7a2c97090 0x55c7a2c9efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:44.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.971+0000 7f9d4a000640 1 -- 172.21.15.5:0/4267261330 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7a2a37f70 con 0x55c7a2ca2080 2024-04-24T06:44:44.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.971+0000 7f9d49600640 1 --2- 172.21.15.5:0/4267261330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7a2c97090 0x55c7a2c9efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:44:44.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.971+0000 7f9d4a000640 1 --2- 172.21.15.5:0/4267261330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7a2ca2080 0x55c7a2c9fa50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c0cdc70 tx=0x7f9d3c09c0e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c142f70d21ca677c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:44.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.971+0000 7f9d2aa00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:44:44.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.971+0000 7f9d2aa00640 1 -- 172.21.15.5:0/4267261330 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c020040 con 0x55c7a2ca2080 2024-04-24T06:44:44.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.971+0000 7f9d2aa00640 1 -- 172.21.15.5:0/4267261330 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f9d3c0cdf50 con 0x55c7a2ca2080 2024-04-24T06:44:44.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.971+0000 7f9d4ed66740 1 -- 172.21.15.5:0/4267261330 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d38005500 con 0x55c7a2ca2080 2024-04-24T06:44:44.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.972+0000 7f9d2aa00640 1 -- 172.21.15.5:0/4267261330 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f9d3c0a5060 con 0x55c7a2ca2080 2024-04-24T06:44:44.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.972+0000 7f9d4ed66740 10 client.25526 fetch_fsmap learned FSMap version 23 2024-04-24T06:44:44.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.972+0000 7f9d4ed66740 1 -- 172.21.15.5:0/4267261330 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55c7a28cf170 con 0x55c7a2ca2080 2024-04-24T06:44:44.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.972+0000 7f9d2aa00640 1 -- 172.21.15.5:0/4267261330 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d38005500 con 0x55c7a2ca2080 2024-04-24T06:44:44.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.972+0000 7f9d2aa00640 1 -- 172.21.15.5:0/4267261330 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f9d3c0b5b70 con 0x55c7a2ca2080 2024-04-24T06:44:44.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.972+0000 7f9d4ed66740 10 client.25526 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T06:44:44.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.972+0000 7f9d4ed66740 10 client.25526 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:44:44.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.972+0000 7f9d4ed66740 20 client.25526 populate_metadata read hostname 'smithi005' 2024-04-24T06:44:44.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.972+0000 7f9d4ed66740 1 --2- 172.21.15.5:0/4267261330 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55c7a2cb22a0 0x55c7a2cb4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:44.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.972+0000 7f9d4ed66740 4 client.25526 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:44:44.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.972+0000 7f9d4ed66740 1 -- 172.21.15.5:0/4267261330 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7a2c0a890 con 0x55c7a2cb22a0 2024-04-24T06:44:44.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.973+0000 7f9d49600640 1 --2- 172.21.15.5:0/4267261330 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55c7a2cb22a0 0x55c7a2cb4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:44.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.973+0000 7f9d49600640 1 --2- 172.21.15.5:0/4267261330 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55c7a2cb22a0 0x55c7a2cb4710 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cdb44008b2e48e5 server_cookie=32e6667d9d0c9049 in_seq=0 out_seq=0 2024-04-24T06:44:44.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.973+0000 7f9d2aa00640 10 client.25526 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:44:44.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.974+0000 7f9d2aa00640 1 -- 172.21.15.5:0/4267261330 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c7a2c0a890 con 0x55c7a2cb22a0 2024-04-24T06:44:44.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.974+0000 7f9d2aa00640 10 client.25526 handle_command_reply: tid=0 2024-04-24T06:44:44.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.974+0000 7f9d4ed66740 1 client.25526 shutdown 2024-04-24T06:44:44.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.975+0000 7f9d4ed66740 1 -- 172.21.15.5:0/4267261330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7a2ca2080 msgr2=0x55c7a2c9fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:44.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.975+0000 7f9d4ed66740 1 --2- 172.21.15.5:0/4267261330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7a2ca2080 0x55c7a2c9fa50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c0cdc70 tx=0x7f9d3c09c0e0 comp rx=0 tx=0).stop 2024-04-24T06:44:44.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.975+0000 7f9d4ed66740 1 -- 172.21.15.5:0/4267261330 shutdown_connections 2024-04-24T06:44:44.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.975+0000 7f9d4ed66740 1 --2- 172.21.15.5:0/4267261330 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55c7a2cb22a0 0x55c7a2cb4710 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:44.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.975+0000 7f9d4ed66740 1 --2- 172.21.15.5:0/4267261330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7a2ca2080 0x55c7a2c9fa50 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:44.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.975+0000 7f9d4ed66740 1 --2- 172.21.15.5:0/4267261330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7a2c979b0 0x55c7a2c9f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:44.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.975+0000 7f9d2aa00640 0 client.25526 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:44:44.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.975+0000 7f9d4ed66740 1 --2- 172.21.15.5:0/4267261330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7a2c97090 0x55c7a2c9efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:44.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.975+0000 7f9d4ed66740 1 -- 172.21.15.5:0/4267261330 >> 172.21.15.5:0/4267261330 conn(0x55c7a2c933b0 msgr2=0x55c7a2c990b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:44.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.975+0000 7f9d4ed66740 1 -- 172.21.15.5:0/4267261330 shutdown_connections 2024-04-24T06:44:44.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.975+0000 7f9d4ed66740 1 -- 172.21.15.5:0/4267261330 wait complete. 2024-04-24T06:44:44.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.975+0000 7f9d4ed66740 20 client.25526 trim_cache size 0 max 16384 2024-04-24T06:44:44.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.995+0000 7f9d4ed66740 1 Processor -- start 2024-04-24T06:44:44.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.995+0000 7f9d4ed66740 1 -- start start 2024-04-24T06:44:44.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.995+0000 7f9d4ed66740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7a2c97090 0x55c7a2cadd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:44.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.995+0000 7f9d4ed66740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7a2c979b0 0x55c7a2ca0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:44.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.995+0000 7f9d4ed66740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7a2ca2080 0x55c7a2cba000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:44.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.995+0000 7f9d4ed66740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9d441b6ad0 con 0x55c7a2ca2080 2024-04-24T06:44:44.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.995+0000 7f9d4ed66740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9d441b43a0 con 0x55c7a2c979b0 2024-04-24T06:44:44.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.995+0000 7f9d4ed66740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9d441b21f0 con 0x55c7a2c97090 2024-04-24T06:44:44.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.995+0000 7f9d48c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7a2c979b0 0x55c7a2ca0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:44.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.995+0000 7f9d4a000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7a2ca2080 0x55c7a2cba000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:44.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.996+0000 7f9d48c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7a2c979b0 0x55c7a2ca0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:60264/0 (socket says 172.21.15.5:60264) 2024-04-24T06:44:44.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.996+0000 7f9d48c00640 1 -- 172.21.15.5:0/167057367 learned_addr learned my addr 172.21.15.5:0/167057367 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:44.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.996+0000 7f9d4a000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7a2ca2080 0x55c7a2cba000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51972/0 (socket says 172.21.15.5:51972) 2024-04-24T06:44:44.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.996+0000 7f9d49600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7a2c97090 0x55c7a2cadd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:44.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.996+0000 7f9d48c00640 1 -- 172.21.15.5:0/167057367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7a2c97090 msgr2=0x55c7a2cadd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:44.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.996+0000 7f9d48c00640 1 --2- 172.21.15.5:0/167057367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7a2c97090 0x55c7a2cadd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:44.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.996+0000 7f9d48c00640 1 -- 172.21.15.5:0/167057367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7a2ca2080 msgr2=0x55c7a2cba000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:44.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.996+0000 7f9d48c00640 1 --2- 172.21.15.5:0/167057367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7a2ca2080 0x55c7a2cba000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:44.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.996+0000 7f9d48c00640 1 -- 172.21.15.5:0/167057367 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7a2c98550 con 0x55c7a2c979b0 2024-04-24T06:44:44.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.996+0000 7f9d4a000640 1 --2- 172.21.15.5:0/167057367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7a2ca2080 0x55c7a2cba000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:44:44.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.996+0000 7f9d48c00640 1 --2- 172.21.15.5:0/167057367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7a2c979b0 0x55c7a2ca0e00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9d30009900 tx=0x7f9d30040070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35bb841dfcd04516 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:45.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.997+0000 7f9d35c00640 1 -- 172.21.15.5:0/167057367 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d30035040 con 0x55c7a2c979b0 2024-04-24T06:44:45.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.997+0000 7f9d35c00640 1 -- 172.21.15.5:0/167057367 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d30050070 con 0x55c7a2c979b0 2024-04-24T06:44:45.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.997+0000 7f9d35c00640 1 -- 172.21.15.5:0/167057367 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d30003850 con 0x55c7a2c979b0 2024-04-24T06:44:45.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.998+0000 7f9d4ed66740 1 -- 172.21.15.5:0/167057367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7a2c979b0 msgr2=0x55c7a2ca0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:45.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.998+0000 7f9d4ed66740 1 --2- 172.21.15.5:0/167057367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7a2c979b0 0x55c7a2ca0e00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9d30009900 tx=0x7f9d30040070 comp rx=0 tx=0).stop 2024-04-24T06:44:45.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.998+0000 7f9d4ed66740 1 -- 172.21.15.5:0/167057367 shutdown_connections 2024-04-24T06:44:45.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.998+0000 7f9d4ed66740 1 --2- 172.21.15.5:0/167057367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7a2ca2080 0x55c7a2cba000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.998+0000 7f9d4ed66740 1 --2- 172.21.15.5:0/167057367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7a2c979b0 0x55c7a2ca0e00 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.998+0000 7f9d4ed66740 1 --2- 172.21.15.5:0/167057367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7a2c97090 0x55c7a2cadd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.998+0000 7f9d4ed66740 1 -- 172.21.15.5:0/167057367 >> 172.21.15.5:0/167057367 conn(0x55c7a2c933b0 msgr2=0x55c7a2ca06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:45.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.998+0000 7f9d4ed66740 1 -- 172.21.15.5:0/167057367 shutdown_connections 2024-04-24T06:44:45.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.998+0000 7f9d4ed66740 1 -- 172.21.15.5:0/167057367 wait complete. 2024-04-24T06:44:45.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.998+0000 7f9d4ed66740 1 -- start start 2024-04-24T06:44:45.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.998+0000 7f9d4ed66740 1 Processor -- start 2024-04-24T06:44:45.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.999+0000 7f9d4ed66740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7a2c97090 0x55c7a2c9c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:45.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.999+0000 7f9d4ed66740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7a2c979b0 0x55c7a2c9c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:45.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.999+0000 7f9d4ed66740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7a2ca2080 0x55c7a2d17870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:45.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.999+0000 7f9d4ed66740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9d441b2860 con 0x55c7a2ca2080 2024-04-24T06:44:45.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.999+0000 7f9d4ed66740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9d441b3cd0 con 0x55c7a2c979b0 2024-04-24T06:44:45.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.999+0000 7f9d4ed66740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9d441b2f30 con 0x55c7a2c97090 2024-04-24T06:44:45.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.999+0000 7f9d4ed66740 10 client.0 _ll_register_callbacks cb 0x7f9d3c00bdd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:44:45.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.999+0000 7f9d4a000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7a2ca2080 0x55c7a2d17870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:45.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.999+0000 7f9d48c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7a2c979b0 0x55c7a2c9c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:45.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.999+0000 7f9d4a000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7a2ca2080 0x55c7a2d17870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51974/0 (socket says 172.21.15.5:51974) 2024-04-24T06:44:45.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.999+0000 7f9d4a000640 1 -- 172.21.15.5:0/82923807 learned_addr learned my addr 172.21.15.5:0/82923807 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:45.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:44.999+0000 7f9d49600640 1 --2- 172.21.15.5:0/82923807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7a2c97090 0x55c7a2c9c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:45.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.000+0000 7f9d4a000640 1 -- 172.21.15.5:0/82923807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7a2c97090 msgr2=0x55c7a2c9c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:45.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.000+0000 7f9d4a000640 1 --2- 172.21.15.5:0/82923807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7a2c97090 0x55c7a2c9c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.000+0000 7f9d4a000640 1 -- 172.21.15.5:0/82923807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7a2c979b0 msgr2=0x55c7a2c9c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:45.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.000+0000 7f9d4a000640 1 --2- 172.21.15.5:0/82923807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7a2c979b0 0x55c7a2c9c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.000+0000 7f9d4a000640 1 -- 172.21.15.5:0/82923807 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7a2d17db0 con 0x55c7a2ca2080 2024-04-24T06:44:45.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.000+0000 7f9d49600640 1 --2- 172.21.15.5:0/82923807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7a2c97090 0x55c7a2c9c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:44:45.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.000+0000 7f9d48c00640 1 --2- 172.21.15.5:0/82923807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7a2c979b0 0x55c7a2c9c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:44:45.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.000+0000 7f9d4a000640 1 --2- 172.21.15.5:0/82923807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7a2ca2080 0x55c7a2d17870 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c098900 tx=0x7f9d3c09c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5464e2850514e69c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:45.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.000+0000 7f9d2aa00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:44:45.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.000+0000 7f9d2aa00640 1 -- 172.21.15.5:0/82923807 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c020070 con 0x55c7a2ca2080 2024-04-24T06:44:45.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.000+0000 7f9d2aa00640 1 -- 172.21.15.5:0/82923807 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f9d3c0cdf50 con 0x55c7a2ca2080 2024-04-24T06:44:45.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.000+0000 7f9d4ed66740 1 -- 172.21.15.5:0/82923807 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d30050070 con 0x55c7a2ca2080 2024-04-24T06:44:45.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.001+0000 7f9d2aa00640 1 -- 172.21.15.5:0/82923807 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f9d3c003c60 con 0x55c7a2ca2080 2024-04-24T06:44:45.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.001+0000 7f9d4ed66740 10 client.15432 fetch_fsmap learned FSMap version 23 2024-04-24T06:44:45.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.001+0000 7f9d4ed66740 1 -- 172.21.15.5:0/82923807 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55c7a2d18f90 con 0x55c7a2ca2080 2024-04-24T06:44:45.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.001+0000 7f9d2aa00640 1 -- 172.21.15.5:0/82923807 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d30050070 con 0x55c7a2ca2080 2024-04-24T06:44:45.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.001+0000 7f9d2aa00640 1 -- 172.21.15.5:0/82923807 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f9d3c09b480 con 0x55c7a2ca2080 2024-04-24T06:44:45.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.001+0000 7f9d4ed66740 10 client.15432 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T06:44:45.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.001+0000 7f9d4ed66740 10 client.15432 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:44:45.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.001+0000 7f9d4ed66740 20 client.15432 populate_metadata read hostname 'smithi005' 2024-04-24T06:44:45.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.001+0000 7f9d4ed66740 1 --2- 172.21.15.5:0/82923807 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55c7a2d19520 0x55c7a2d1b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:45.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.001+0000 7f9d4ed66740 4 client.15432 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T06:44:45.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.001+0000 7f9d4ed66740 1 -- 172.21.15.5:0/82923807 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c7a2c04700 con 0x55c7a2d19520 2024-04-24T06:44:45.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.002+0000 7f9d49600640 1 --2- 172.21.15.5:0/82923807 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55c7a2d19520 0x55c7a2d1b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:45.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.002+0000 7f9d49600640 1 --2- 172.21.15.5:0/82923807 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55c7a2d19520 0x55c7a2d1b910 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26035021aef6f879 server_cookie=9f992558006342b0 in_seq=0 out_seq=0 2024-04-24T06:44:45.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.002+0000 7f9d2aa00640 10 client.15432 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:44:45.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.003+0000 7f9d2aa00640 1 -- 172.21.15.5:0/82923807 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c7a2c04700 con 0x55c7a2d19520 2024-04-24T06:44:45.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.003+0000 7f9d2aa00640 10 client.15432 handle_command_reply: tid=0 2024-04-24T06:44:45.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.003+0000 7f9d4ed66740 1 client.15432 shutdown 2024-04-24T06:44:45.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.004+0000 7f9d4ed66740 1 -- 172.21.15.5:0/82923807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7a2ca2080 msgr2=0x55c7a2d17870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:45.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.004+0000 7f9d4ed66740 1 --2- 172.21.15.5:0/82923807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7a2ca2080 0x55c7a2d17870 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c098900 tx=0x7f9d3c09c070 comp rx=0 tx=0).stop 2024-04-24T06:44:45.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.004+0000 7f9d4ed66740 1 -- 172.21.15.5:0/82923807 shutdown_connections 2024-04-24T06:44:45.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.004+0000 7f9d4ed66740 1 --2- 172.21.15.5:0/82923807 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55c7a2d19520 0x55c7a2d1b910 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.004+0000 7f9d4ed66740 1 --2- 172.21.15.5:0/82923807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7a2ca2080 0x55c7a2d17870 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.004+0000 7f9d4ed66740 1 --2- 172.21.15.5:0/82923807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7a2c979b0 0x55c7a2c9c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.004+0000 7f9d4ed66740 1 --2- 172.21.15.5:0/82923807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7a2c97090 0x55c7a2c9c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.004+0000 7f9d4ed66740 1 -- 172.21.15.5:0/82923807 >> 172.21.15.5:0/82923807 conn(0x55c7a2c933b0 msgr2=0x55c7a2ca0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:45.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.004+0000 7f9d2aa00640 0 client.15432 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:44:45.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.004+0000 7f9d4ed66740 1 -- 172.21.15.5:0/82923807 shutdown_connections 2024-04-24T06:44:45.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.004+0000 7f9d4ed66740 1 -- 172.21.15.5:0/82923807 wait complete. 2024-04-24T06:44:45.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.004+0000 7f9d4ed66740 20 client.15432 trim_cache size 0 max 16384 2024-04-24T06:44:45.008 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:44:45.008 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-24T06:44:45.008 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "d7b1d815-4ca6-4170-9bb7-e49c4aa68f3e", 2024-04-24T06:44:45.008 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-24T06:44:45.008 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:44:45.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.005+0000 7f9d4c000640 1 -- 172.21.15.5:0/1169979752 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f9d2c0a3e90 msgr2=0x7f9d2c0a6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:45.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.005+0000 7f9d4c000640 1 --2- 172.21.15.5:0/1169979752 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f9d2c0a3e90 0x7f9d2c0a6350 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9d30005c40 tx=0x7f9d30005b50 comp rx=0 tx=0).stop 2024-04-24T06:44:45.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.005+0000 7f9d4c000640 1 -- 172.21.15.5:0/1169979752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d4410a590 msgr2=0x7f9d441a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:45.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.005+0000 7f9d4c000640 1 --2- 172.21.15.5:0/1169979752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d4410a590 0x7f9d441a3840 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c00da40 tx=0x7f9d3c00df00 comp rx=0 tx=0).stop 2024-04-24T06:44:45.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.005+0000 7f9d4c000640 1 -- 172.21.15.5:0/1169979752 shutdown_connections 2024-04-24T06:44:45.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.005+0000 7f9d4c000640 1 --2- 172.21.15.5:0/1169979752 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f9d2c0a3e90 0x7f9d2c0a6350 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.005+0000 7f9d4c000640 1 --2- 172.21.15.5:0/1169979752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d4410a590 0x7f9d441a3840 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.005+0000 7f9d4c000640 1 --2- 172.21.15.5:0/1169979752 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d441090c0 0x7f9d4419f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.005+0000 7f9d4c000640 1 --2- 172.21.15.5:0/1169979752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d44108720 0x7f9d4419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.005+0000 7f9d4c000640 1 -- 172.21.15.5:0/1169979752 >> 172.21.15.5:0/1169979752 conn(0x7f9d44077e70 msgr2=0x7f9d44078430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:45.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.005+0000 7f9d4c000640 1 -- 172.21.15.5:0/1169979752 shutdown_connections 2024-04-24T06:44:45.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.005+0000 7f9d4c000640 1 -- 172.21.15.5:0/1169979752 wait complete. 2024-04-24T06:44:45.024 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T06:44:45.024 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T06:44:45.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.243+0000 7f7f72e00640 1 -- 172.21.15.5:0/1446873488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f6c104450 msgr2=0x7f7f6c104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:45.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.243+0000 7f7f72e00640 1 --2- 172.21.15.5:0/1446873488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f6c104450 0x7f7f6c104830 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c009860 tx=0x7f7f5c02a340 comp rx=0 tx=0).stop 2024-04-24T06:44:45.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.243+0000 7f7f72e00640 1 -- 172.21.15.5:0/1446873488 shutdown_connections 2024-04-24T06:44:45.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.243+0000 7f7f72e00640 1 --2- 172.21.15.5:0/1446873488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f6c109520 0x7f7f6c1110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.243+0000 7f7f72e00640 1 --2- 172.21.15.5:0/1446873488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f6c104e00 0x7f7f6c108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.243+0000 7f7f72e00640 1 --2- 172.21.15.5:0/1446873488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f6c104450 0x7f7f6c104830 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.243+0000 7f7f72e00640 1 -- 172.21.15.5:0/1446873488 >> 172.21.15.5:0/1446873488 conn(0x7f7f6c0ffc90 msgr2=0x7f7f6c1020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:45.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.244+0000 7f7f72e00640 1 -- 172.21.15.5:0/1446873488 shutdown_connections 2024-04-24T06:44:45.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.244+0000 7f7f72e00640 1 -- 172.21.15.5:0/1446873488 wait complete. 2024-04-24T06:44:45.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.245+0000 7f7f72e00640 1 Processor -- start 2024-04-24T06:44:45.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.245+0000 7f7f72e00640 1 -- start start 2024-04-24T06:44:45.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.245+0000 7f7f72e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f6c104450 0x7f7f6c1a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:45.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.245+0000 7f7f72e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f6c109520 0x7f7f6c1a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:45.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.245+0000 7f7f72e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f6c19a970 0x7f7f6c19ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:45.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.245+0000 7f7f72e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1145d0 con 0x7f7f6c19a970 2024-04-24T06:44:45.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.245+0000 7f7f72e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c114450 con 0x7f7f6c104450 2024-04-24T06:44:45.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.245+0000 7f7f72e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c114750 con 0x7f7f6c109520 2024-04-24T06:44:45.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.245+0000 7f7f70e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f6c19a970 0x7f7f6c19ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:45.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.245+0000 7f7f70e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f6c19a970 0x7f7f6c19ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51988/0 (socket says 172.21.15.5:51988) 2024-04-24T06:44:45.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.245+0000 7f7f70e00640 1 -- 172.21.15.5:0/2834863576 learned_addr learned my addr 172.21.15.5:0/2834863576 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:45.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.246+0000 7f7f6be00640 1 --2- 172.21.15.5:0/2834863576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f6c104450 0x7f7f6c1a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:45.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.246+0000 7f7f63400640 1 --2- 172.21.15.5:0/2834863576 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f6c109520 0x7f7f6c1a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:45.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.246+0000 7f7f70e00640 1 -- 172.21.15.5:0/2834863576 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f6c109520 msgr2=0x7f7f6c1a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:45.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.246+0000 7f7f70e00640 1 --2- 172.21.15.5:0/2834863576 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f6c109520 0x7f7f6c1a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.246+0000 7f7f70e00640 1 -- 172.21.15.5:0/2834863576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f6c104450 msgr2=0x7f7f6c1a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:45.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.246+0000 7f7f70e00640 1 --2- 172.21.15.5:0/2834863576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f6c104450 0x7f7f6c1a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.246+0000 7f7f70e00640 1 -- 172.21.15.5:0/2834863576 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f6c19b490 con 0x7f7f6c19a970 2024-04-24T06:44:45.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.246+0000 7f7f70e00640 1 --2- 172.21.15.5:0/2834863576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f6c19a970 0x7f7f6c19ae20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7f6400eb10 tx=0x7f7f6400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:45.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.246+0000 7f7f69600640 1 -- 172.21.15.5:0/2834863576 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6400ce20 con 0x7f7f6c19a970 2024-04-24T06:44:45.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.247+0000 7f7f72e00640 1 -- 172.21.15.5:0/2834863576 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f6c19b780 con 0x7f7f6c19a970 2024-04-24T06:44:45.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.247+0000 7f7f69600640 1 -- 172.21.15.5:0/2834863576 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f64005020 con 0x7f7f6c19a970 2024-04-24T06:44:45.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.247+0000 7f7f69600640 1 -- 172.21.15.5:0/2834863576 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f64010690 con 0x7f7f6c19a970 2024-04-24T06:44:45.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.247+0000 7f7f72e00640 1 -- 172.21.15.5:0/2834863576 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f6c1a7930 con 0x7f7f6c19a970 2024-04-24T06:44:45.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.250+0000 7f7f75d6a740 1 Processor -- start 2024-04-24T06:44:45.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.250+0000 7f7f69600640 1 -- 172.21.15.5:0/2834863576 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 138872+0+0 (secure 0 0 0) 0x7f7f64005540 con 0x7f7f6c19a970 2024-04-24T06:44:45.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.251+0000 7f7f75d6a740 1 -- start start 2024-04-24T06:44:45.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.251+0000 7f7f75d6a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b255a82860 0x55b255a82c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:45.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.251+0000 7f7f75d6a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b255a83180 0x55b255a8d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:45.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.251+0000 7f7f75d6a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b255a8d850 0x55b255a8fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:45.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.251+0000 7f7f75d6a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b2559e81f0 con 0x55b255a82860 2024-04-24T06:44:45.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.251+0000 7f7f75d6a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b25598e350 con 0x55b255a83180 2024-04-24T06:44:45.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.251+0000 7f7f75d6a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b2559e41b0 con 0x55b255a8d850 2024-04-24T06:44:45.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.251+0000 7f7f63400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b255a82860 0x55b255a82c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:45.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.251+0000 7f7f63400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b255a82860 0x55b255a82c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51992/0 (socket says 172.21.15.5:51992) 2024-04-24T06:44:45.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.251+0000 7f7f63400640 1 -- 172.21.15.5:0/352423799 learned_addr learned my addr 172.21.15.5:0/352423799 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:45.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.251+0000 7f7f6be00640 1 --2- 172.21.15.5:0/352423799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b255a83180 0x55b255a8d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:45.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.252+0000 7f7f63400640 1 --2- 172.21.15.5:0/352423799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b255a8d850 0x55b255a8fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:45.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.252+0000 7f7f69600640 1 --2- 172.21.15.5:0/2834863576 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f7f500a3e40 0x7f7f500a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:45.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.252+0000 7f7f63400640 1 -- 172.21.15.5:0/352423799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b255a8d850 msgr2=0x55b255a8fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:45.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.252+0000 7f7f63400640 1 --2- 172.21.15.5:0/352423799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b255a8d850 0x55b255a8fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.252+0000 7f7f63400640 1 -- 172.21.15.5:0/352423799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b255a83180 msgr2=0x55b255a8d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:45.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.252+0000 7f7f63400640 1 --2- 172.21.15.5:0/352423799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b255a83180 0x55b255a8d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.252+0000 7f7f63400640 1 -- 172.21.15.5:0/352423799 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2559e2f50 con 0x55b255a82860 2024-04-24T06:44:45.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.252+0000 7f7f6be00640 1 --2- 172.21.15.5:0/352423799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b255a83180 0x55b255a8d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:44:45.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.252+0000 7f7f69600640 1 -- 172.21.15.5:0/2834863576 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f7f640d0ff0 con 0x7f7f6c19a970 2024-04-24T06:44:45.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.252+0000 7f7f70e00640 1 --2- 172.21.15.5:0/2834863576 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f7f500a3e40 0x7f7f500a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:45.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.252+0000 7f7f63400640 1 --2- 172.21.15.5:0/352423799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b255a82860 0x55b255a82c40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7f54013530 tx=0x7f7f54005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf9678ac017e4ce server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:45.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.253+0000 7f7f61600640 1 -- 172.21.15.5:0/352423799 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f54004050 con 0x55b255a82860 2024-04-24T06:44:45.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.253+0000 7f7f61600640 1 -- 172.21.15.5:0/352423799 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f54005500 con 0x55b255a82860 2024-04-24T06:44:45.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.253+0000 7f7f61600640 1 -- 172.21.15.5:0/352423799 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f54004c00 con 0x55b255a82860 2024-04-24T06:44:45.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.253+0000 7f7f75d6a740 1 -- 172.21.15.5:0/352423799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b255a82860 msgr2=0x55b255a82c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:45.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.253+0000 7f7f75d6a740 1 --2- 172.21.15.5:0/352423799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b255a82860 0x55b255a82c40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7f54013530 tx=0x7f7f54005850 comp rx=0 tx=0).stop 2024-04-24T06:44:45.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.253+0000 7f7f70e00640 1 --2- 172.21.15.5:0/2834863576 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f7f500a3e40 0x7f7f500a6300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7f640ce5f0 tx=0x7f7f64005940 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:45.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.254+0000 7f7f75d6a740 1 -- 172.21.15.5:0/352423799 shutdown_connections 2024-04-24T06:44:45.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.254+0000 7f7f75d6a740 1 --2- 172.21.15.5:0/352423799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b255a8d850 0x55b255a8fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.254+0000 7f7f75d6a740 1 --2- 172.21.15.5:0/352423799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b255a83180 0x55b255a8d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.254+0000 7f7f75d6a740 1 --2- 172.21.15.5:0/352423799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b255a82860 0x55b255a82c40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.254+0000 7f7f75d6a740 1 -- 172.21.15.5:0/352423799 >> 172.21.15.5:0/352423799 conn(0x55b255a7eb80 msgr2=0x55b255a7ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:45.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.254+0000 7f7f75d6a740 1 -- 172.21.15.5:0/352423799 shutdown_connections 2024-04-24T06:44:45.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.254+0000 7f7f75d6a740 1 -- 172.21.15.5:0/352423799 wait complete. 2024-04-24T06:44:45.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.254+0000 7f7f75d6a740 1 -- start start 2024-04-24T06:44:45.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.254+0000 7f7f75d6a740 1 Processor -- start 2024-04-24T06:44:45.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.255+0000 7f7f75d6a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b255a82860 0x55b255a8a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:45.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.255+0000 7f7f75d6a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b255a83180 0x55b255a8ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:45.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.255+0000 7f7f75d6a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b255a8d850 0x55b255a8b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:45.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.255+0000 7f7f75d6a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b255a90640 con 0x55b255a83180 2024-04-24T06:44:45.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.255+0000 7f7f75d6a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b2559a1690 con 0x55b255a8d850 2024-04-24T06:44:45.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.255+0000 7f7f75d6a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b255a907c0 con 0x55b255a82860 2024-04-24T06:44:45.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.255+0000 7f7f75d6a740 10 client.0 _ll_register_callbacks cb 0x55b2559dd650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:44:45.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.255+0000 7f7f6be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b255a83180 0x55b255a8ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:45.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.255+0000 7f7f6be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b255a83180 0x55b255a8ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52008/0 (socket says 172.21.15.5:52008) 2024-04-24T06:44:45.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.255+0000 7f7f6be00640 1 -- 172.21.15.5:0/938423106 learned_addr learned my addr 172.21.15.5:0/938423106 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:45.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.255+0000 7f7f63400640 1 --2- 172.21.15.5:0/938423106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b255a82860 0x55b255a8a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:45.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.255+0000 7f7f63400640 1 --2- 172.21.15.5:0/938423106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b255a8d850 0x55b255a8b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:45.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.255+0000 7f7f6be00640 1 -- 172.21.15.5:0/938423106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b255a82860 msgr2=0x55b255a8a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:45.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.255+0000 7f7f6be00640 1 --2- 172.21.15.5:0/938423106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b255a82860 0x55b255a8a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.256+0000 7f7f6be00640 1 -- 172.21.15.5:0/938423106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b255a8d850 msgr2=0x55b255a8b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:45.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.256+0000 7f7f6be00640 1 --2- 172.21.15.5:0/938423106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b255a8d850 0x55b255a8b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.256+0000 7f7f6be00640 1 -- 172.21.15.5:0/938423106 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2559a0920 con 0x55b255a83180 2024-04-24T06:44:45.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.256+0000 7f7f63400640 1 --2- 172.21.15.5:0/938423106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b255a8d850 0x55b255a8b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:44:45.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.256+0000 7f7f6be00640 1 --2- 172.21.15.5:0/938423106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b255a83180 0x55b255a8ad20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c02a930 tx=0x7f7f5c0303d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db15244425aefa84 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:45.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.256+0000 7f7f5a000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:44:45.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.256+0000 7f7f5a000640 1 -- 172.21.15.5:0/938423106 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c037040 con 0x55b255a83180 2024-04-24T06:44:45.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.256+0000 7f7f5a000640 1 -- 172.21.15.5:0/938423106 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f7f5c030550 con 0x55b255a83180 2024-04-24T06:44:45.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.256+0000 7f7f75d6a740 1 -- 172.21.15.5:0/938423106 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f54005500 con 0x55b255a83180 2024-04-24T06:44:45.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.256+0000 7f7f5a000640 1 -- 172.21.15.5:0/938423106 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f7f5c04c050 con 0x55b255a83180 2024-04-24T06:44:45.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.257+0000 7f7f75d6a740 10 client.15444 fetch_fsmap learned FSMap version 23 2024-04-24T06:44:45.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.257+0000 7f7f75d6a740 1 -- 172.21.15.5:0/938423106 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55b2557c47d0 con 0x55b255a83180 2024-04-24T06:44:45.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.257+0000 7f7f5a000640 1 -- 172.21.15.5:0/938423106 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f54005500 con 0x55b255a83180 2024-04-24T06:44:45.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.257+0000 7f7f5a000640 1 -- 172.21.15.5:0/938423106 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f7f5c03d500 con 0x55b255a83180 2024-04-24T06:44:45.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.257+0000 7f7f75d6a740 10 client.15444 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T06:44:45.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.257+0000 7f7f75d6a740 10 client.15444 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:44:45.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.257+0000 7f7f75d6a740 20 client.15444 populate_metadata read hostname 'smithi005' 2024-04-24T06:44:45.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.257+0000 7f7f75d6a740 1 --2- 172.21.15.5:0/938423106 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55b255a9dc20 0x55b255aa0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:45.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.257+0000 7f7f75d6a740 4 client.15444 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:44:45.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.257+0000 7f7f75d6a740 1 -- 172.21.15.5:0/938423106 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2559f5c10 con 0x55b255a9dc20 2024-04-24T06:44:45.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.258+0000 7f7f63400640 1 --2- 172.21.15.5:0/938423106 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55b255a9dc20 0x55b255aa0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:45.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.258+0000 7f7f63400640 1 --2- 172.21.15.5:0/938423106 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55b255a9dc20 0x55b255aa0060 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b86f6f52d630715e server_cookie=f043a3f01076444 in_seq=0 out_seq=0 2024-04-24T06:44:45.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.258+0000 7f7f5a000640 10 client.15444 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:44:45.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.259+0000 7f7f5a000640 1 -- 172.21.15.5:0/938423106 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b2559f5c10 con 0x55b255a9dc20 2024-04-24T06:44:45.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.259+0000 7f7f5a000640 10 client.15444 handle_command_reply: tid=0 2024-04-24T06:44:45.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.259+0000 7f7f75d6a740 1 client.15444 shutdown 2024-04-24T06:44:45.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.259+0000 7f7f75d6a740 1 -- 172.21.15.5:0/938423106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b255a83180 msgr2=0x55b255a8ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:45.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.259+0000 7f7f75d6a740 1 --2- 172.21.15.5:0/938423106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b255a83180 0x55b255a8ad20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c02a930 tx=0x7f7f5c0303d0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.259+0000 7f7f75d6a740 1 -- 172.21.15.5:0/938423106 shutdown_connections 2024-04-24T06:44:45.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.259+0000 7f7f75d6a740 1 --2- 172.21.15.5:0/938423106 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55b255a9dc20 0x55b255aa0060 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.259+0000 7f7f75d6a740 1 --2- 172.21.15.5:0/938423106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b255a8d850 0x55b255a8b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.259+0000 7f7f75d6a740 1 --2- 172.21.15.5:0/938423106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b255a83180 0x55b255a8ad20 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.259+0000 7f7f75d6a740 1 --2- 172.21.15.5:0/938423106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b255a82860 0x55b255a8a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.259+0000 7f7f5a000640 0 client.15444 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:44:45.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.259+0000 7f7f75d6a740 1 -- 172.21.15.5:0/938423106 >> 172.21.15.5:0/938423106 conn(0x55b255a7eb80 msgr2=0x55b255a84980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:45.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.259+0000 7f7f75d6a740 1 -- 172.21.15.5:0/938423106 shutdown_connections 2024-04-24T06:44:45.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.259+0000 7f7f75d6a740 1 -- 172.21.15.5:0/938423106 wait complete. 2024-04-24T06:44:45.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.259+0000 7f7f75d6a740 20 client.15444 trim_cache size 0 max 16384 2024-04-24T06:44:45.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.279+0000 7f7f75d6a740 1 Processor -- start 2024-04-24T06:44:45.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.279+0000 7f7f75d6a740 1 -- start start 2024-04-24T06:44:45.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.279+0000 7f7f75d6a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b255a82860 0x55b255a99500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:45.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.279+0000 7f7f75d6a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b255a83180 0x55b255a89720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:45.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.279+0000 7f7f75d6a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b255a8d850 0x55b255a8b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:45.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.279+0000 7f7f75d6a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1b27d0 con 0x55b255a83180 2024-04-24T06:44:45.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.279+0000 7f7f75d6a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1b00a0 con 0x55b255a82860 2024-04-24T06:44:45.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.279+0000 7f7f75d6a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1adef0 con 0x55b255a8d850 2024-04-24T06:44:45.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.280+0000 7f7f6be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b255a83180 0x55b255a89720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:45.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.280+0000 7f7f6be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b255a83180 0x55b255a89720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52024/0 (socket says 172.21.15.5:52024) 2024-04-24T06:44:45.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.280+0000 7f7f6be00640 1 -- 172.21.15.5:0/113248141 learned_addr learned my addr 172.21.15.5:0/113248141 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:45.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.280+0000 7f7f63400640 1 --2- 172.21.15.5:0/113248141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b255a82860 0x55b255a99500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:45.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.280+0000 7f7f63400640 1 --2- 172.21.15.5:0/113248141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b255a8d850 0x55b255a8b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:45.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.280+0000 7f7f6be00640 1 -- 172.21.15.5:0/113248141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b255a8d850 msgr2=0x55b255a8b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:45.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.280+0000 7f7f6be00640 1 --2- 172.21.15.5:0/113248141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b255a8d850 0x55b255a8b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.280+0000 7f7f6be00640 1 -- 172.21.15.5:0/113248141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b255a82860 msgr2=0x55b255a99500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:45.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.280+0000 7f7f6be00640 1 --2- 172.21.15.5:0/113248141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b255a82860 0x55b255a99500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.280+0000 7f7f6be00640 1 -- 172.21.15.5:0/113248141 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2556fa3f0 con 0x55b255a83180 2024-04-24T06:44:45.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.281+0000 7f7f6be00640 1 --2- 172.21.15.5:0/113248141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b255a83180 0x55b255a89720 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c03dab0 tx=0x7f7f5c02a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d1c4bcea4f998d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:45.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.281+0000 7f7f60c00640 1 -- 172.21.15.5:0/113248141 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c037040 con 0x55b255a83180 2024-04-24T06:44:45.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.281+0000 7f7f60c00640 1 -- 172.21.15.5:0/113248141 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f5c03db30 con 0x55b255a83180 2024-04-24T06:44:45.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.281+0000 7f7f60c00640 1 -- 172.21.15.5:0/113248141 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c0035f0 con 0x55b255a83180 2024-04-24T06:44:45.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.281+0000 7f7f75d6a740 1 -- 172.21.15.5:0/113248141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b255a83180 msgr2=0x55b255a89720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:45.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.281+0000 7f7f75d6a740 1 --2- 172.21.15.5:0/113248141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b255a83180 0x55b255a89720 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c03dab0 tx=0x7f7f5c02a340 comp rx=0 tx=0).stop 2024-04-24T06:44:45.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.281+0000 7f7f75d6a740 1 -- 172.21.15.5:0/113248141 shutdown_connections 2024-04-24T06:44:45.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.282+0000 7f7f75d6a740 1 --2- 172.21.15.5:0/113248141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b255a8d850 0x55b255a8b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.282+0000 7f7f75d6a740 1 --2- 172.21.15.5:0/113248141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b255a83180 0x55b255a89720 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.282+0000 7f7f75d6a740 1 --2- 172.21.15.5:0/113248141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b255a82860 0x55b255a99500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.282+0000 7f7f75d6a740 1 -- 172.21.15.5:0/113248141 >> 172.21.15.5:0/113248141 conn(0x55b255a7eb80 msgr2=0x55b255a84950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:45.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.282+0000 7f7f75d6a740 1 -- 172.21.15.5:0/113248141 shutdown_connections 2024-04-24T06:44:45.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.282+0000 7f7f75d6a740 1 -- 172.21.15.5:0/113248141 wait complete. 2024-04-24T06:44:45.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.282+0000 7f7f75d6a740 1 -- start start 2024-04-24T06:44:45.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.282+0000 7f7f75d6a740 1 Processor -- start 2024-04-24T06:44:45.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.283+0000 7f7f75d6a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b255a82860 0x55b255a91780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:45.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.283+0000 7f7f75d6a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b255a83180 0x55b255a91cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:45.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.283+0000 7f7f75d6a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b255a8d850 0x55b255b031a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:45.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.283+0000 7f7f75d6a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1ae560 con 0x55b255a82860 2024-04-24T06:44:45.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.283+0000 7f7f75d6a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1af9d0 con 0x55b255a83180 2024-04-24T06:44:45.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.283+0000 7f7f75d6a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1aec30 con 0x55b255a8d850 2024-04-24T06:44:45.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.283+0000 7f7f75d6a740 10 client.0 _ll_register_callbacks cb 0x7f7f54005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:44:45.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.283+0000 7f7f63400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b255a82860 0x55b255a91780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:45.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.283+0000 7f7f6be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b255a83180 0x55b255a91cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:45.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.283+0000 7f7f63400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b255a82860 0x55b255a91780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52038/0 (socket says 172.21.15.5:52038) 2024-04-24T06:44:45.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.283+0000 7f7f63400640 1 -- 172.21.15.5:0/671492867 learned_addr learned my addr 172.21.15.5:0/671492867 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:44:45.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.283+0000 7f7f6be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b255a83180 0x55b255a91cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:60332/0 (socket says 172.21.15.5:60332) 2024-04-24T06:44:45.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.283+0000 7f7f63400640 1 --2- 172.21.15.5:0/671492867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b255a8d850 0x55b255b031a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:45.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.284+0000 7f7f6be00640 1 -- 172.21.15.5:0/671492867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b255a8d850 msgr2=0x55b255b031a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:45.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.284+0000 7f7f6be00640 1 --2- 172.21.15.5:0/671492867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b255a8d850 0x55b255b031a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.284+0000 7f7f6be00640 1 -- 172.21.15.5:0/671492867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b255a82860 msgr2=0x55b255a91780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:44:45.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.284+0000 7f7f6be00640 1 --2- 172.21.15.5:0/671492867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b255a82860 0x55b255a91780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.284+0000 7f7f6be00640 1 -- 172.21.15.5:0/671492867 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2556ba0f0 con 0x55b255a83180 2024-04-24T06:44:45.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.284+0000 7f7f63400640 1 --2- 172.21.15.5:0/671492867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b255a82860 0x55b255a91780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:44:45.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.284+0000 7f7f6be00640 1 --2- 172.21.15.5:0/671492867 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b255a83180 0x55b255a91cc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x55b255a7e7e0 tx=0x7f7f5c009570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aaebd0d7940b614e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:44:45.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.284+0000 7f7f5a000640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:44:45.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.284+0000 7f7f5a000640 1 -- 172.21.15.5:0/671492867 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c037040 con 0x55b255a83180 2024-04-24T06:44:45.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.284+0000 7f7f75d6a740 1 -- 172.21.15.5:0/671492867 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f5c03db30 con 0x55b255a83180 2024-04-24T06:44:45.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.284+0000 7f7f5a000640 1 -- 172.21.15.5:0/671492867 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f7f5c030550 con 0x55b255a83180 2024-04-24T06:44:45.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.284+0000 7f7f5a000640 1 -- 172.21.15.5:0/671492867 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f7f5c030750 con 0x55b255a83180 2024-04-24T06:44:45.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.284+0000 7f7f75d6a740 10 client.25547 fetch_fsmap learned FSMap version 23 2024-04-24T06:44:45.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.284+0000 7f7f75d6a740 1 -- 172.21.15.5:0/671492867 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55b2559ad810 con 0x55b255a83180 2024-04-24T06:44:45.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.285+0000 7f7f5a000640 1 -- 172.21.15.5:0/671492867 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f5c03db30 con 0x55b255a83180 2024-04-24T06:44:45.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.285+0000 7f7f5a000640 1 -- 172.21.15.5:0/671492867 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f7f5c056030 con 0x55b255a83180 2024-04-24T06:44:45.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.285+0000 7f7f75d6a740 10 client.25547 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T06:44:45.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.285+0000 7f7f75d6a740 10 client.25547 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:44:45.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.285+0000 7f7f75d6a740 20 client.25547 populate_metadata read hostname 'smithi005' 2024-04-24T06:44:45.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.285+0000 7f7f75d6a740 1 --2- 172.21.15.5:0/671492867 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55b255b04a10 0x55b255b06e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:44:45.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.285+0000 7f7f75d6a740 4 client.25547 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:44:45.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.285+0000 7f7f75d6a740 1 -- 172.21.15.5:0/671492867 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b2559efa80 con 0x55b255b04a10 2024-04-24T06:44:45.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.285+0000 7f7f63400640 1 --2- 172.21.15.5:0/671492867 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55b255b04a10 0x55b255b06e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:44:45.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.286+0000 7f7f63400640 1 --2- 172.21.15.5:0/671492867 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55b255b04a10 0x55b255b06e00 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44a26576f0de8666 server_cookie=107541d4c295591b in_seq=0 out_seq=0 2024-04-24T06:44:45.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.286+0000 7f7f5a000640 10 client.25547 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:44:45.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.286+0000 7f7f5a000640 1 -- 172.21.15.5:0/671492867 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b2559efa80 con 0x55b255b04a10 2024-04-24T06:44:45.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.286+0000 7f7f5a000640 10 client.25547 handle_command_reply: tid=0 2024-04-24T06:44:45.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.286+0000 7f7f75d6a740 1 client.25547 shutdown 2024-04-24T06:44:45.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.286+0000 7f7f75d6a740 1 -- 172.21.15.5:0/671492867 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b255a83180 msgr2=0x55b255a91cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:45.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.286+0000 7f7f75d6a740 1 --2- 172.21.15.5:0/671492867 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b255a83180 0x55b255a91cc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x55b255a7e7e0 tx=0x7f7f5c009570 comp rx=0 tx=0).stop 2024-04-24T06:44:45.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.286+0000 7f7f75d6a740 1 -- 172.21.15.5:0/671492867 shutdown_connections 2024-04-24T06:44:45.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.286+0000 7f7f75d6a740 1 --2- 172.21.15.5:0/671492867 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55b255b04a10 0x55b255b06e00 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.286+0000 7f7f75d6a740 1 --2- 172.21.15.5:0/671492867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b255a8d850 0x55b255b031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.286+0000 7f7f75d6a740 1 --2- 172.21.15.5:0/671492867 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b255a83180 0x55b255a91cc0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.286+0000 7f7f5a000640 0 client.25547 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:44:45.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.286+0000 7f7f75d6a740 1 --2- 172.21.15.5:0/671492867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b255a82860 0x55b255a91780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.286+0000 7f7f75d6a740 1 -- 172.21.15.5:0/671492867 >> 172.21.15.5:0/671492867 conn(0x55b255a7eb80 msgr2=0x55b255a83fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:45.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.287+0000 7f7f75d6a740 1 -- 172.21.15.5:0/671492867 shutdown_connections 2024-04-24T06:44:45.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.287+0000 7f7f75d6a740 1 -- 172.21.15.5:0/671492867 wait complete. 2024-04-24T06:44:45.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.287+0000 7f7f75d6a740 20 client.25547 trim_cache size 0 max 16384 2024-04-24T06:44:45.292 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:44:45.292 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T06:44:45.292 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-24T06:44:45.293 INFO:teuthology.orchestra.run.smithi005.stdout: "d7b1d815-4ca6-4170-9bb7-e49c4aa68f3e": { 2024-04-24T06:44:45.293 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-24T06:44:45.293 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "d7b1d815-4ca6-4170-9bb7-e49c4aa68f3e", 2024-04-24T06:44:45.293 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-04-24T06:44:45.293 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:44:45.293 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:44:45.293 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:44:45.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.287+0000 7f7f72e00640 1 -- 172.21.15.5:0/2834863576 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f7f500a3e40 msgr2=0x7f7f500a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:45.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.287+0000 7f7f72e00640 1 --2- 172.21.15.5:0/2834863576 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f7f500a3e40 0x7f7f500a6300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7f640ce5f0 tx=0x7f7f64005940 comp rx=0 tx=0).stop 2024-04-24T06:44:45.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.287+0000 7f7f72e00640 1 -- 172.21.15.5:0/2834863576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f6c19a970 msgr2=0x7f7f6c19ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:44:45.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.287+0000 7f7f72e00640 1 --2- 172.21.15.5:0/2834863576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f6c19a970 0x7f7f6c19ae20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7f6400eb10 tx=0x7f7f6400ef10 comp rx=0 tx=0).stop 2024-04-24T06:44:45.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.288+0000 7f7f72e00640 1 -- 172.21.15.5:0/2834863576 shutdown_connections 2024-04-24T06:44:45.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.288+0000 7f7f72e00640 1 --2- 172.21.15.5:0/2834863576 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f7f500a3e40 0x7f7f500a6300 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.288+0000 7f7f72e00640 1 --2- 172.21.15.5:0/2834863576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f6c19a970 0x7f7f6c19ae20 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.288+0000 7f7f72e00640 1 --2- 172.21.15.5:0/2834863576 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f6c109520 0x7f7f6c1a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.288+0000 7f7f72e00640 1 --2- 172.21.15.5:0/2834863576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f6c104450 0x7f7f6c1a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:44:45.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.288+0000 7f7f72e00640 1 -- 172.21.15.5:0/2834863576 >> 172.21.15.5:0/2834863576 conn(0x7f7f6c0ffc90 msgr2=0x7f7f6c100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:44:45.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.288+0000 7f7f72e00640 1 -- 172.21.15.5:0/2834863576 shutdown_connections 2024-04-24T06:44:45.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:44:45.288+0000 7f7f72e00640 1 -- 172.21.15.5:0/2834863576 wait complete. 2024-04-24T06:44:45.308 INFO:tasks.cephfs.filesystem:scrub status for tag:d7b1d815-4ca6-4170-9bb7-e49c4aa68f3e - {'path': '/', 'tag': 'd7b1d815-4ca6-4170-9bb7-e49c4aa68f3e', 'options': 'recursive,force'} 2024-04-24T06:44:45.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:45 smithi137 ceph-mon[33017]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:44:45.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:45 smithi005 ceph-mon[27798]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:44:45.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:45 smithi037 ceph-mon[32864]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:44:46.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:46 smithi137 ceph-mon[33017]: scrub queued for path: / 2024-04-24T06:44:46.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:46 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:44:46.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:46 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:44:46.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:46 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:44:46.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:46 smithi005 ceph-mon[27798]: scrub queued for path: / 2024-04-24T06:44:46.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:46 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:44:46.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:46 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:44:46.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:46 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:44:46.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:46 smithi037 ceph-mon[32864]: scrub queued for path: / 2024-04-24T06:44:46.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:46 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:44:46.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:46 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:44:46.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:46 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:44:47.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:47 smithi005 ceph-mon[27798]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:44:47.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:47 smithi037 ceph-mon[32864]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:44:47.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:47 smithi137 ceph-mon[33017]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:44:48.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:48 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:44:48.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:48 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:44:48.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:48 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:44:49.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:49 smithi005 ceph-mon[27798]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-04-24T06:44:49.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:49 smithi037 ceph-mon[32864]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-04-24T06:44:49.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:49 smithi137 ceph-mon[33017]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-04-24T06:44:51.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:51 smithi005 ceph-mon[27798]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-24T06:44:51.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:51 smithi037 ceph-mon[32864]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-24T06:44:51.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:51 smithi137 ceph-mon[33017]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-24T06:44:53.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:53 smithi005 ceph-mon[27798]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:44:53.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:53 smithi037 ceph-mon[32864]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:44:53.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:53 smithi137 ceph-mon[33017]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:44:55.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:55 smithi005 ceph-mon[27798]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 8 op/s 2024-04-24T06:44:55.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:55 smithi037 ceph-mon[32864]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 8 op/s 2024-04-24T06:44:55.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:55 smithi137 ceph-mon[33017]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 8 op/s 2024-04-24T06:44:57.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:57 smithi005 ceph-mon[27798]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-24T06:44:57.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:57 smithi037 ceph-mon[32864]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-24T06:44:57.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:57 smithi137 ceph-mon[33017]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-24T06:44:59.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:44:59 smithi005 ceph-mon[27798]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-24T06:44:59.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:44:59 smithi037 ceph-mon[32864]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-24T06:44:59.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:44:59 smithi137 ceph-mon[33017]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-24T06:45:01.257 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:45:00 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:45:00.782+0000 7fd623c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T06:45:01.257 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:45:00 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:45:00.782+0000 7fd623c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T06:45:01.257 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:45:00 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:45:00.783+0000 7fd623c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T06:45:01.257 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:45:00 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:45:00.783+0000 7fd623c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T06:45:01.257 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:45:00 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:45:00.783+0000 7fd623c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T06:45:01.748 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:01 smithi137 ceph-mon[33017]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-24T06:45:01.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:01 smithi005 ceph-mon[27798]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-24T06:45:01.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:01 smithi037 ceph-mon[32864]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-24T06:45:03.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:03 smithi037 ceph-mon[32864]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-24T06:45:03.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:03 smithi037 ceph-mon[32864]: mgrmap e26: x(active, since 60s), standbys: y, z 2024-04-24T06:45:03.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:03 smithi137 ceph-mon[33017]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-24T06:45:03.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:03 smithi137 ceph-mon[33017]: mgrmap e26: x(active, since 60s), standbys: y, z 2024-04-24T06:45:04.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:03 smithi005 ceph-mon[27798]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-24T06:45:04.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:03 smithi005 ceph-mon[27798]: mgrmap e26: x(active, since 60s), standbys: y, z 2024-04-24T06:45:05.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:05 smithi037 ceph-mon[32864]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-24T06:45:05.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:05 smithi137 ceph-mon[33017]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-24T06:45:06.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:05 smithi005 ceph-mon[27798]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-24T06:45:07.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:07 smithi137 ceph-mon[33017]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-24T06:45:08.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:07 smithi005 ceph-mon[27798]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-24T06:45:08.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:07 smithi037 ceph-mon[32864]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-24T06:45:09.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:09 smithi137 ceph-mon[33017]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-24T06:45:10.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:09 smithi005 ceph-mon[27798]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-24T06:45:10.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:09 smithi037 ceph-mon[32864]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-24T06:45:11.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:11 smithi137 ceph-mon[33017]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-04-24T06:45:12.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:11 smithi005 ceph-mon[27798]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-04-24T06:45:12.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:11 smithi037 ceph-mon[32864]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-04-24T06:45:12.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:12 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:45:12.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:12 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:45:12.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:12 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:45:13.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:12 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:45:13.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:12 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:45:13.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:12 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:45:13.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:12 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:45:13.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:12 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:45:13.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:12 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:45:13.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:13 smithi137 ceph-mon[33017]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-24T06:45:13.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:13 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:45:13.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:13 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:45:13.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:13 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:45:14.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:13 smithi005 ceph-mon[27798]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-24T06:45:14.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:13 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:45:14.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:13 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:45:14.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:13 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:45:14.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:13 smithi037 ceph-mon[32864]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-24T06:45:14.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:13 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:45:14.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:13 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:45:14.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:13 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:45:15.309 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T06:45:15.309 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T06:45:15.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.499+0000 7f577b400640 1 -- 172.21.15.5:0/771257715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5774103b90 msgr2=0x7f577410c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:15.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.499+0000 7f577b400640 1 --2- 172.21.15.5:0/771257715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5774103b90 0x7f577410c070 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5760009900 tx=0x7f576002a480 comp rx=0 tx=0).stop 2024-04-24T06:45:15.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.500+0000 7f577b400640 1 -- 172.21.15.5:0/771257715 shutdown_connections 2024-04-24T06:45:15.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.500+0000 7f577b400640 1 --2- 172.21.15.5:0/771257715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f577410c5b0 0x7f577410ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:15.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.500+0000 7f577b400640 1 --2- 172.21.15.5:0/771257715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5774103b90 0x7f577410c070 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:15.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.500+0000 7f577b400640 1 --2- 172.21.15.5:0/771257715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5774103270 0x7f5774103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:15.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.500+0000 7f577b400640 1 -- 172.21.15.5:0/771257715 >> 172.21.15.5:0/771257715 conn(0x7f57740780b0 msgr2=0x7f57740784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:15.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.500+0000 7f577b400640 1 -- 172.21.15.5:0/771257715 shutdown_connections 2024-04-24T06:45:15.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.500+0000 7f577b400640 1 -- 172.21.15.5:0/771257715 wait complete. 2024-04-24T06:45:15.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.501+0000 7f577b400640 1 Processor -- start 2024-04-24T06:45:15.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.501+0000 7f577b400640 1 -- start start 2024-04-24T06:45:15.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.501+0000 7f577b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5774103270 0x7f5774100ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:15.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.501+0000 7f577b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5774103b90 0x7f5774101400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:15.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.501+0000 7f577b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f577410c5b0 0x7f5774101940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:15.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.501+0000 7f577b400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f57741123e0 con 0x7f5774103270 2024-04-24T06:45:15.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.501+0000 7f577b400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5774112260 con 0x7f577410c5b0 2024-04-24T06:45:15.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.501+0000 7f577b400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5774112560 con 0x7f5774103b90 2024-04-24T06:45:15.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.502+0000 7f5778a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5774103270 0x7f5774100ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:15.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.502+0000 7f5778a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5774103270 0x7f5774100ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40178/0 (socket says 172.21.15.5:40178) 2024-04-24T06:45:15.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.502+0000 7f5778a00640 1 -- 172.21.15.5:0/1512576196 learned_addr learned my addr 172.21.15.5:0/1512576196 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:15.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.502+0000 7f5779400640 1 --2- 172.21.15.5:0/1512576196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f577410c5b0 0x7f5774101940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:15.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.502+0000 7f5773e00640 1 --2- 172.21.15.5:0/1512576196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5774103b90 0x7f5774101400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:15.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.502+0000 7f5778a00640 1 -- 172.21.15.5:0/1512576196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5774103b90 msgr2=0x7f5774101400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:15.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.502+0000 7f5778a00640 1 --2- 172.21.15.5:0/1512576196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5774103b90 0x7f5774101400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:15.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.502+0000 7f5778a00640 1 -- 172.21.15.5:0/1512576196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f577410c5b0 msgr2=0x7f5774101940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:15.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.502+0000 7f5778a00640 1 --2- 172.21.15.5:0/1512576196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f577410c5b0 0x7f5774101940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:15.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.502+0000 7f5778a00640 1 -- 172.21.15.5:0/1512576196 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57741a6de0 con 0x7f5774103270 2024-04-24T06:45:15.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.502+0000 7f5778a00640 1 --2- 172.21.15.5:0/1512576196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5774103270 0x7f5774100ec0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f576800e9f0 tx=0x7f576800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:15.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.503+0000 7f5771600640 1 -- 172.21.15.5:0/1512576196 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576800cd80 con 0x7f5774103270 2024-04-24T06:45:15.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.503+0000 7f5771600640 1 -- 172.21.15.5:0/1512576196 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5768005500 con 0x7f5774103270 2024-04-24T06:45:15.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.503+0000 7f5771600640 1 -- 172.21.15.5:0/1512576196 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5768010690 con 0x7f5774103270 2024-04-24T06:45:15.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.504+0000 7f577b400640 1 -- 172.21.15.5:0/1512576196 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57741a70d0 con 0x7f5774103270 2024-04-24T06:45:15.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.504+0000 7f577b400640 1 -- 172.21.15.5:0/1512576196 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57741a75a0 con 0x7f5774103270 2024-04-24T06:45:15.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.507+0000 7f5771600640 1 -- 172.21.15.5:0/1512576196 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138929+0+0 (secure 0 0 0) 0x7f57680046c0 con 0x7f5774103270 2024-04-24T06:45:15.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.508+0000 7f577e1fa740 1 Processor -- start 2024-04-24T06:45:15.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.508+0000 7f577e1fa740 1 -- start start 2024-04-24T06:45:15.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.509+0000 7f577e1fa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0b7c00860 0x55a0b7c00c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:15.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.509+0000 7f577e1fa740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0b7c01180 0x55a0b7c0b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:15.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.509+0000 7f577e1fa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0b7c0b850 0x55a0b7c0dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:15.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.509+0000 7f577e1fa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a0b7b661f0 con 0x55a0b7c0b850 2024-04-24T06:45:15.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.509+0000 7f577e1fa740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a0b7b0c350 con 0x55a0b7c01180 2024-04-24T06:45:15.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.509+0000 7f577e1fa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a0b7b621b0 con 0x55a0b7c00860 2024-04-24T06:45:15.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.509+0000 7f5779400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0b7c00860 0x55a0b7c00c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:15.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.509+0000 7f5779400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0b7c00860 0x55a0b7c00c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:58690/0 (socket says 172.21.15.5:58690) 2024-04-24T06:45:15.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.509+0000 7f5779400640 1 -- 172.21.15.5:0/2980976315 learned_addr learned my addr 172.21.15.5:0/2980976315 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:15.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.509+0000 7f5778a00640 1 --2- 172.21.15.5:0/2980976315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0b7c01180 0x55a0b7c0b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:15.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.510+0000 7f5773e00640 1 --2- 172.21.15.5:0/2980976315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0b7c0b850 0x55a0b7c0dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:15.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.510+0000 7f5779400640 1 -- 172.21.15.5:0/2980976315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0b7c01180 msgr2=0x55a0b7c0b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:15.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.510+0000 7f5779400640 1 --2- 172.21.15.5:0/2980976315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0b7c01180 0x55a0b7c0b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:15.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.510+0000 7f5779400640 1 -- 172.21.15.5:0/2980976315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0b7c0b850 msgr2=0x55a0b7c0dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:15.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.510+0000 7f5779400640 1 --2- 172.21.15.5:0/2980976315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0b7c0b850 0x55a0b7c0dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:15.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.510+0000 7f5779400640 1 -- 172.21.15.5:0/2980976315 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0b7b60f50 con 0x55a0b7c00860 2024-04-24T06:45:15.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.510+0000 7f5773e00640 1 --2- 172.21.15.5:0/2980976315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0b7c0b850 0x55a0b7c0dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:45:15.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.510+0000 7f5771600640 1 --2- 172.21.15.5:0/1512576196 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f575c0a4020 0x7f575c0a64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:15.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.510+0000 7f5779400640 1 --2- 172.21.15.5:0/2980976315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0b7c00860 0x55a0b7c00c40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f576c009f80 tx=0x7f576c010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acefb908a8ff602 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:15.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.510+0000 7f5771600640 1 -- 172.21.15.5:0/1512576196 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f57680d0e20 con 0x7f5774103270 2024-04-24T06:45:15.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.510+0000 7f5778a00640 1 --2- 172.21.15.5:0/1512576196 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f575c0a4020 0x7f575c0a64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:15.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.510+0000 7f5765c00640 1 -- 172.21.15.5:0/2980976315 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c011700 con 0x55a0b7c00860 2024-04-24T06:45:15.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.510+0000 7f5765c00640 1 -- 172.21.15.5:0/2980976315 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f576c011d80 con 0x55a0b7c00860 2024-04-24T06:45:15.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.510+0000 7f5765c00640 1 -- 172.21.15.5:0/2980976315 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c018380 con 0x55a0b7c00860 2024-04-24T06:45:15.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.511+0000 7f577e1fa740 1 -- 172.21.15.5:0/2980976315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0b7c00860 msgr2=0x55a0b7c00c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:15.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.511+0000 7f5778a00640 1 --2- 172.21.15.5:0/1512576196 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f575c0a4020 0x7f575c0a64e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f57680ce3a0 tx=0x7f5768004a40 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:15.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.511+0000 7f577e1fa740 1 --2- 172.21.15.5:0/2980976315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0b7c00860 0x55a0b7c00c40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f576c009f80 tx=0x7f576c010860 comp rx=0 tx=0).stop 2024-04-24T06:45:15.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.511+0000 7f577e1fa740 1 -- 172.21.15.5:0/2980976315 shutdown_connections 2024-04-24T06:45:15.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.511+0000 7f577e1fa740 1 --2- 172.21.15.5:0/2980976315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0b7c0b850 0x55a0b7c0dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:15.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.511+0000 7f577e1fa740 1 --2- 172.21.15.5:0/2980976315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0b7c01180 0x55a0b7c0b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:15.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.512+0000 7f577e1fa740 1 --2- 172.21.15.5:0/2980976315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0b7c00860 0x55a0b7c00c40 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:15.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.512+0000 7f577e1fa740 1 -- 172.21.15.5:0/2980976315 >> 172.21.15.5:0/2980976315 conn(0x55a0b7bfcb80 msgr2=0x55a0b7bfcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:15.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.512+0000 7f577e1fa740 1 -- 172.21.15.5:0/2980976315 shutdown_connections 2024-04-24T06:45:15.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.512+0000 7f577e1fa740 1 -- 172.21.15.5:0/2980976315 wait complete. 2024-04-24T06:45:15.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.512+0000 7f577e1fa740 1 -- start start 2024-04-24T06:45:15.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.513+0000 7f577e1fa740 1 Processor -- start 2024-04-24T06:45:15.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.513+0000 7f577e1fa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0b7c00860 0x55a0b7c08870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:15.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.513+0000 7f577e1fa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0b7c01180 0x55a0b7c08db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:15.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.513+0000 7f577e1fa740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0b7c0b850 0x55a0b7c092f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:15.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.513+0000 7f577e1fa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a0b7c0e640 con 0x55a0b7c01180 2024-04-24T06:45:15.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.513+0000 7f577e1fa740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a0b7b1f690 con 0x55a0b7c0b850 2024-04-24T06:45:15.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.513+0000 7f577e1fa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a0b7c0e7c0 con 0x55a0b7c00860 2024-04-24T06:45:15.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.513+0000 7f577e1fa740 10 client.0 _ll_register_callbacks cb 0x55a0b7b5b650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:45:15.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.514+0000 7f5779400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0b7c0b850 0x55a0b7c092f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:15.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.514+0000 7f5779400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0b7c0b850 0x55a0b7c092f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:47544/0 (socket says 172.21.15.5:47544) 2024-04-24T06:45:15.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.514+0000 7f5779400640 1 -- 172.21.15.5:0/1325551532 learned_addr learned my addr 172.21.15.5:0/1325551532 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:15.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.514+0000 7f5779400640 1 --2- 172.21.15.5:0/1325551532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0b7c00860 0x55a0b7c08870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:15.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.514+0000 7f5773e00640 1 --2- 172.21.15.5:0/1325551532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0b7c01180 0x55a0b7c08db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:15.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.514+0000 7f5779400640 1 -- 172.21.15.5:0/1325551532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0b7c00860 msgr2=0x55a0b7c08870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:15.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.514+0000 7f5779400640 1 --2- 172.21.15.5:0/1325551532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0b7c00860 0x55a0b7c08870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:15.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.514+0000 7f5779400640 1 -- 172.21.15.5:0/1325551532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0b7c01180 msgr2=0x55a0b7c08db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:15.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.514+0000 7f5779400640 1 --2- 172.21.15.5:0/1325551532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0b7c01180 0x55a0b7c08db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:15.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.514+0000 7f5779400640 1 -- 172.21.15.5:0/1325551532 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0b7b1e920 con 0x55a0b7c0b850 2024-04-24T06:45:15.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.515+0000 7f5779400640 1 --2- 172.21.15.5:0/1325551532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0b7c0b850 0x55a0b7c092f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f576c019b70 tx=0x7f576c013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c560fe6bb042c7e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:15.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.515+0000 7f575a000640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:45:15.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.515+0000 7f575a000640 1 -- 172.21.15.5:0/1325551532 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c02e060 con 0x55a0b7c0b850 2024-04-24T06:45:15.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.515+0000 7f577e1fa740 1 -- 172.21.15.5:0/1325551532 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f576c011d80 con 0x55a0b7c0b850 2024-04-24T06:45:15.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.515+0000 7f575a000640 1 -- 172.21.15.5:0/1325551532 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f576c003b70 con 0x55a0b7c0b850 2024-04-24T06:45:15.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.515+0000 7f575a000640 1 -- 172.21.15.5:0/1325551532 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f576c003d70 con 0x55a0b7c0b850 2024-04-24T06:45:15.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.515+0000 7f577e1fa740 10 client.25565 fetch_fsmap learned FSMap version 23 2024-04-24T06:45:15.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.515+0000 7f577e1fa740 1 -- 172.21.15.5:0/1325551532 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55a0b79427d0 con 0x55a0b7c0b850 2024-04-24T06:45:15.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.516+0000 7f575a000640 1 -- 172.21.15.5:0/1325551532 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f576c011d80 con 0x55a0b7c0b850 2024-04-24T06:45:15.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.517+0000 7f575a000640 1 -- 172.21.15.5:0/1325551532 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f576c0043d0 con 0x55a0b7c0b850 2024-04-24T06:45:15.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.517+0000 7f577e1fa740 10 client.25565 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T06:45:15.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.517+0000 7f577e1fa740 10 client.25565 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:45:15.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.517+0000 7f577e1fa740 20 client.25565 populate_metadata read hostname 'smithi005' 2024-04-24T06:45:15.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.517+0000 7f577e1fa740 1 --2- 172.21.15.5:0/1325551532 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a0b7c1bac0 0x55a0b7c1deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:15.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.517+0000 7f577e1fa740 4 client.25565 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:45:15.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.517+0000 7f577e1fa740 1 -- 172.21.15.5:0/1325551532 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0b7b73c10 con 0x55a0b7c1bac0 2024-04-24T06:45:15.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.517+0000 7f5773e00640 1 --2- 172.21.15.5:0/1325551532 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a0b7c1bac0 0x55a0b7c1deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:15.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.518+0000 7f5773e00640 1 --2- 172.21.15.5:0/1325551532 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a0b7c1bac0 0x55a0b7c1deb0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c29f4945c4a7808 server_cookie=aa1f6f5f43168a27 in_seq=0 out_seq=0 2024-04-24T06:45:15.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.518+0000 7f575a000640 10 client.25565 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:45:15.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.519+0000 7f575a000640 1 -- 172.21.15.5:0/1325551532 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a0b7b73c10 con 0x55a0b7c1bac0 2024-04-24T06:45:15.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.519+0000 7f575a000640 10 client.25565 handle_command_reply: tid=0 2024-04-24T06:45:15.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.519+0000 7f577e1fa740 1 client.25565 shutdown 2024-04-24T06:45:15.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.519+0000 7f577e1fa740 1 -- 172.21.15.5:0/1325551532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0b7c0b850 msgr2=0x55a0b7c092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:15.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.519+0000 7f577e1fa740 1 --2- 172.21.15.5:0/1325551532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0b7c0b850 0x55a0b7c092f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f576c019b70 tx=0x7f576c013040 comp rx=0 tx=0).stop 2024-04-24T06:45:15.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.519+0000 7f577e1fa740 1 -- 172.21.15.5:0/1325551532 shutdown_connections 2024-04-24T06:45:15.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.519+0000 7f577e1fa740 1 --2- 172.21.15.5:0/1325551532 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a0b7c1bac0 0x55a0b7c1deb0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:15.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.519+0000 7f577e1fa740 1 --2- 172.21.15.5:0/1325551532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0b7c0b850 0x55a0b7c092f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:15.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.519+0000 7f577e1fa740 1 --2- 172.21.15.5:0/1325551532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0b7c01180 0x55a0b7c08db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:15.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.519+0000 7f577e1fa740 1 --2- 172.21.15.5:0/1325551532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0b7c00860 0x55a0b7c08870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:15.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.519+0000 7f577e1fa740 1 -- 172.21.15.5:0/1325551532 >> 172.21.15.5:0/1325551532 conn(0x55a0b7bfcb80 msgr2=0x55a0b7c02980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:15.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.519+0000 7f575a000640 0 client.25565 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:45:15.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.519+0000 7f577e1fa740 1 -- 172.21.15.5:0/1325551532 shutdown_connections 2024-04-24T06:45:15.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.519+0000 7f577e1fa740 1 -- 172.21.15.5:0/1325551532 wait complete. 2024-04-24T06:45:15.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.520+0000 7f577e1fa740 20 client.25565 trim_cache size 0 max 16384 2024-04-24T06:45:15.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.544+0000 7f577e1fa740 1 Processor -- start 2024-04-24T06:45:15.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.544+0000 7f577e1fa740 1 -- start start 2024-04-24T06:45:15.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.545+0000 7f577e1fa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0b7c00860 0x55a0b7c05af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:15.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.545+0000 7f577e1fa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0b7c01180 0x55a0b7c06030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:15.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.545+0000 7f577e1fa740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0b7c0b850 0x55a0b7bfb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:15.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.545+0000 7f577e1fa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f57741b27c0 con 0x55a0b7c00860 2024-04-24T06:45:15.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.545+0000 7f577e1fa740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f57741b0090 con 0x55a0b7c0b850 2024-04-24T06:45:15.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.545+0000 7f577e1fa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f57741adee0 con 0x55a0b7c01180 2024-04-24T06:45:15.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.545+0000 7f5779400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0b7c00860 0x55a0b7c05af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:15.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.545+0000 7f5779400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0b7c00860 0x55a0b7c05af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40204/0 (socket says 172.21.15.5:40204) 2024-04-24T06:45:15.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.545+0000 7f5779400640 1 -- 172.21.15.5:0/3388671968 learned_addr learned my addr 172.21.15.5:0/3388671968 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:15.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.545+0000 7f5779400640 1 --2- 172.21.15.5:0/3388671968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0b7c0b850 0x55a0b7bfb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:15.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.545+0000 7f5773e00640 1 --2- 172.21.15.5:0/3388671968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0b7c01180 0x55a0b7c06030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:15.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.546+0000 7f5779400640 1 -- 172.21.15.5:0/3388671968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0b7c01180 msgr2=0x55a0b7c06030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:15.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.546+0000 7f5779400640 1 --2- 172.21.15.5:0/3388671968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0b7c01180 0x55a0b7c06030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:15.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.546+0000 7f5779400640 1 -- 172.21.15.5:0/3388671968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0b7c0b850 msgr2=0x55a0b7bfb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:15.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.546+0000 7f5779400640 1 --2- 172.21.15.5:0/3388671968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0b7c0b850 0x55a0b7bfb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:15.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.546+0000 7f5779400640 1 -- 172.21.15.5:0/3388671968 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0b78783f0 con 0x55a0b7c00860 2024-04-24T06:45:15.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.546+0000 7f5779400640 1 --2- 172.21.15.5:0/3388671968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0b7c00860 0x55a0b7c05af0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f576c019b40 tx=0x7f576c0099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b644f209f7cbdd0b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:15.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.547+0000 7f5765200640 1 -- 172.21.15.5:0/3388671968 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c02e070 con 0x55a0b7c00860 2024-04-24T06:45:15.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.547+0000 7f5765200640 1 -- 172.21.15.5:0/3388671968 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f576c020040 con 0x55a0b7c00860 2024-04-24T06:45:15.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.547+0000 7f5765200640 1 -- 172.21.15.5:0/3388671968 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c003970 con 0x55a0b7c00860 2024-04-24T06:45:15.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.548+0000 7f577e1fa740 1 -- 172.21.15.5:0/3388671968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0b7c00860 msgr2=0x55a0b7c05af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:15.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.548+0000 7f577e1fa740 1 --2- 172.21.15.5:0/3388671968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0b7c00860 0x55a0b7c05af0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f576c019b40 tx=0x7f576c0099f0 comp rx=0 tx=0).stop 2024-04-24T06:45:15.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.548+0000 7f577e1fa740 1 -- 172.21.15.5:0/3388671968 shutdown_connections 2024-04-24T06:45:15.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.548+0000 7f577e1fa740 1 --2- 172.21.15.5:0/3388671968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0b7c0b850 0x55a0b7bfb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:15.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.548+0000 7f577e1fa740 1 --2- 172.21.15.5:0/3388671968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0b7c01180 0x55a0b7c06030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:15.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.548+0000 7f577e1fa740 1 --2- 172.21.15.5:0/3388671968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0b7c00860 0x55a0b7c05af0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:15.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.548+0000 7f577e1fa740 1 -- 172.21.15.5:0/3388671968 >> 172.21.15.5:0/3388671968 conn(0x55a0b7bfcb80 msgr2=0x55a0b7c172a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:15.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.548+0000 7f577e1fa740 1 -- 172.21.15.5:0/3388671968 shutdown_connections 2024-04-24T06:45:15.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.549+0000 7f577e1fa740 1 -- 172.21.15.5:0/3388671968 wait complete. 2024-04-24T06:45:15.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.549+0000 7f577e1fa740 1 -- start start 2024-04-24T06:45:15.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.549+0000 7f577e1fa740 1 Processor -- start 2024-04-24T06:45:15.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.550+0000 7f577e1fa740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0b7c00860 0x55a0b7c20dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:15.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.550+0000 7f577e1fa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0b7c01180 0x55a0b7c1e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:15.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.550+0000 7f577e1fa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0b7c0b850 0x55a0b7c1e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:15.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.550+0000 7f577e1fa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f57741ae550 con 0x55a0b7c0b850 2024-04-24T06:45:15.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.550+0000 7f577e1fa740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f57741af9c0 con 0x55a0b7c00860 2024-04-24T06:45:15.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.550+0000 7f577e1fa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f57741aec20 con 0x55a0b7c01180 2024-04-24T06:45:15.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.550+0000 7f577e1fa740 10 client.0 _ll_register_callbacks cb 0x7f576c009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:45:15.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.550+0000 7f5779400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0b7c00860 0x55a0b7c20dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:15.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.551+0000 7f5779400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0b7c00860 0x55a0b7c20dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:47560/0 (socket says 172.21.15.5:47560) 2024-04-24T06:45:15.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.551+0000 7f5779400640 1 -- 172.21.15.5:0/1894636320 learned_addr learned my addr 172.21.15.5:0/1894636320 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:15.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.551+0000 7f5779400640 1 --2- 172.21.15.5:0/1894636320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0b7c0b850 0x55a0b7c1e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:15.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.551+0000 7f5773e00640 1 --2- 172.21.15.5:0/1894636320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0b7c01180 0x55a0b7c1e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:15.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.551+0000 7f5773e00640 1 -- 172.21.15.5:0/1894636320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0b7c00860 msgr2=0x55a0b7c20dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:15.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.551+0000 7f5773e00640 1 --2- 172.21.15.5:0/1894636320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0b7c00860 0x55a0b7c20dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:15.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.551+0000 7f5773e00640 1 -- 172.21.15.5:0/1894636320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0b7c0b850 msgr2=0x55a0b7c1e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:15.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.551+0000 7f5773e00640 1 --2- 172.21.15.5:0/1894636320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0b7c0b850 0x55a0b7c1e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:15.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.551+0000 7f5773e00640 1 -- 172.21.15.5:0/1894636320 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0b78380f0 con 0x55a0b7c01180 2024-04-24T06:45:15.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.552+0000 7f5773e00640 1 --2- 172.21.15.5:0/1894636320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0b7c01180 0x55a0b7c1e1b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f576002aa70 tx=0x7f57600043f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=849759e4c6c23d19 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:15.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.552+0000 7f575a000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:45:15.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.552+0000 7f575a000640 1 -- 172.21.15.5:0/1894636320 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5760035040 con 0x55a0b7c01180 2024-04-24T06:45:15.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.552+0000 7f577e1fa740 1 -- 172.21.15.5:0/1894636320 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f576c020040 con 0x55a0b7c01180 2024-04-24T06:45:15.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.552+0000 7f575a000640 1 -- 172.21.15.5:0/1894636320 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f5760005040 con 0x55a0b7c01180 2024-04-24T06:45:15.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.552+0000 7f575a000640 1 -- 172.21.15.5:0/1894636320 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f5760054050 con 0x55a0b7c01180 2024-04-24T06:45:15.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.552+0000 7f577e1fa740 10 client.25600 fetch_fsmap learned FSMap version 23 2024-04-24T06:45:15.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.552+0000 7f577e1fa740 1 -- 172.21.15.5:0/1894636320 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55a0b7b2b810 con 0x55a0b7c01180 2024-04-24T06:45:15.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.553+0000 7f575a000640 1 -- 172.21.15.5:0/1894636320 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f576c020040 con 0x55a0b7c01180 2024-04-24T06:45:15.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.553+0000 7f575a000640 1 -- 172.21.15.5:0/1894636320 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f57600046c0 con 0x55a0b7c01180 2024-04-24T06:45:15.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.553+0000 7f577e1fa740 10 client.25600 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T06:45:15.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.553+0000 7f577e1fa740 10 client.25600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:45:15.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.553+0000 7f577e1fa740 20 client.25600 populate_metadata read hostname 'smithi005' 2024-04-24T06:45:15.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.553+0000 7f577e1fa740 1 --2- 172.21.15.5:0/1894636320 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a0b7c81f90 0x55a0b7c843b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:15.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.553+0000 7f577e1fa740 4 client.25600 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:45:15.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.553+0000 7f577e1fa740 1 -- 172.21.15.5:0/1894636320 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a0b7b6da80 con 0x55a0b7c81f90 2024-04-24T06:45:15.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.554+0000 7f5779400640 1 --2- 172.21.15.5:0/1894636320 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a0b7c81f90 0x55a0b7c843b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:15.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.554+0000 7f5779400640 1 --2- 172.21.15.5:0/1894636320 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a0b7c81f90 0x55a0b7c843b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9c29a18a71dcfe7 server_cookie=d3cca11b6f9b0e31 in_seq=0 out_seq=0 2024-04-24T06:45:15.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.554+0000 7f575a000640 10 client.25600 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:45:15.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.555+0000 7f575a000640 1 -- 172.21.15.5:0/1894636320 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a0b7b6da80 con 0x55a0b7c81f90 2024-04-24T06:45:15.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.555+0000 7f575a000640 10 client.25600 handle_command_reply: tid=0 2024-04-24T06:45:15.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.555+0000 7f577e1fa740 1 client.25600 shutdown 2024-04-24T06:45:15.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.555+0000 7f577e1fa740 1 -- 172.21.15.5:0/1894636320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0b7c01180 msgr2=0x55a0b7c1e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:15.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.555+0000 7f577e1fa740 1 --2- 172.21.15.5:0/1894636320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0b7c01180 0x55a0b7c1e1b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f576002aa70 tx=0x7f57600043f0 comp rx=0 tx=0).stop 2024-04-24T06:45:15.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.555+0000 7f577e1fa740 1 -- 172.21.15.5:0/1894636320 shutdown_connections 2024-04-24T06:45:15.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.555+0000 7f577e1fa740 1 --2- 172.21.15.5:0/1894636320 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a0b7c81f90 0x55a0b7c843b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:15.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.555+0000 7f577e1fa740 1 --2- 172.21.15.5:0/1894636320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0b7c0b850 0x55a0b7c1e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:15.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.555+0000 7f575a000640 0 client.25600 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:45:15.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.555+0000 7f577e1fa740 1 --2- 172.21.15.5:0/1894636320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0b7c01180 0x55a0b7c1e1b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:15.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.555+0000 7f577e1fa740 1 --2- 172.21.15.5:0/1894636320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0b7c00860 0x55a0b7c20dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:15.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.555+0000 7f577e1fa740 1 -- 172.21.15.5:0/1894636320 >> 172.21.15.5:0/1894636320 conn(0x55a0b7bfcb80 msgr2=0x55a0b7c16a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:15.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.555+0000 7f577e1fa740 1 -- 172.21.15.5:0/1894636320 shutdown_connections 2024-04-24T06:45:15.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.555+0000 7f577e1fa740 1 -- 172.21.15.5:0/1894636320 wait complete. 2024-04-24T06:45:15.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.555+0000 7f577e1fa740 20 client.25600 trim_cache size 0 max 16384 2024-04-24T06:45:15.563 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:45:15.563 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-24T06:45:15.563 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-24T06:45:15.563 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:45:15.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.556+0000 7f577b400640 1 -- 172.21.15.5:0/1512576196 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f575c0a4020 msgr2=0x7f575c0a64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:15.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.556+0000 7f577b400640 1 --2- 172.21.15.5:0/1512576196 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f575c0a4020 0x7f575c0a64e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f57680ce3a0 tx=0x7f5768004a40 comp rx=0 tx=0).stop 2024-04-24T06:45:15.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.556+0000 7f577b400640 1 -- 172.21.15.5:0/1512576196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5774103270 msgr2=0x7f5774100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:15.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.556+0000 7f577b400640 1 --2- 172.21.15.5:0/1512576196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5774103270 0x7f5774100ec0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f576800e9f0 tx=0x7f576800edf0 comp rx=0 tx=0).stop 2024-04-24T06:45:15.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.556+0000 7f577b400640 1 -- 172.21.15.5:0/1512576196 shutdown_connections 2024-04-24T06:45:15.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.556+0000 7f577b400640 1 --2- 172.21.15.5:0/1512576196 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f575c0a4020 0x7f575c0a64e0 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f57680ce3a0 tx=0x7f5768004a40 comp rx=0 tx=0).stop 2024-04-24T06:45:15.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.556+0000 7f577b400640 1 --2- 172.21.15.5:0/1512576196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f577410c5b0 0x7f5774101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:15.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.556+0000 7f577b400640 1 --2- 172.21.15.5:0/1512576196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5774103b90 0x7f5774101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:15.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.556+0000 7f577b400640 1 --2- 172.21.15.5:0/1512576196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5774103270 0x7f5774100ec0 secure :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f576800e9f0 tx=0x7f576800edf0 comp rx=0 tx=0).stop 2024-04-24T06:45:15.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.556+0000 7f577b400640 1 -- 172.21.15.5:0/1512576196 >> 172.21.15.5:0/1512576196 conn(0x7f57740780b0 msgr2=0x7f57740fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:15.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.557+0000 7f577b400640 1 -- 172.21.15.5:0/1512576196 shutdown_connections 2024-04-24T06:45:15.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.557+0000 7f577b400640 1 -- 172.21.15.5:0/1512576196 wait complete. 2024-04-24T06:45:15.580 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T06:45:15.580 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:45:15.638 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:15 smithi005 ceph-mon[27798]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-24T06:45:15.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.756+0000 7f82ce600640 1 -- 172.21.15.5:0/881077030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82c81039b0 msgr2=0x7f82c810be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:15.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.756+0000 7f82ce600640 1 --2- 172.21.15.5:0/881077030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82c81039b0 0x7f82c810be90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f82b4009900 tx=0x7f82b402a480 comp rx=0 tx=0).stop 2024-04-24T06:45:15.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.756+0000 7f82ce600640 1 -- 172.21.15.5:0/881077030 shutdown_connections 2024-04-24T06:45:15.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.756+0000 7f82ce600640 1 --2- 172.21.15.5:0/881077030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82c810c4b0 0x7f82c810e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:15.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.756+0000 7f82ce600640 1 --2- 172.21.15.5:0/881077030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82c81039b0 0x7f82c810be90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:15.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.756+0000 7f82ce600640 1 --2- 172.21.15.5:0/881077030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82c8103090 0x7f82c8103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:15.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.756+0000 7f82ce600640 1 -- 172.21.15.5:0/881077030 >> 172.21.15.5:0/881077030 conn(0x7f82c80fc890 msgr2=0x7f82c80fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:15.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.757+0000 7f82ce600640 1 -- 172.21.15.5:0/881077030 shutdown_connections 2024-04-24T06:45:15.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.757+0000 7f82ce600640 1 -- 172.21.15.5:0/881077030 wait complete. 2024-04-24T06:45:15.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.758+0000 7f82ce600640 1 Processor -- start 2024-04-24T06:45:15.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.758+0000 7f82ce600640 1 -- start start 2024-04-24T06:45:15.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.758+0000 7f82ce600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82c8103090 0x7f82c81a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:15.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.758+0000 7f82ce600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82c81039b0 0x7f82c81a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:15.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.758+0000 7f82ce600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82c810c4b0 0x7f82c819a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:15.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.758+0000 7f82ce600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f82c81122e0 con 0x7f82c810c4b0 2024-04-24T06:45:15.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.758+0000 7f82ce600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f82c8112160 con 0x7f82c81039b0 2024-04-24T06:45:15.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.758+0000 7f82ce600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f82c8112460 con 0x7f82c8103090 2024-04-24T06:45:15.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.759+0000 7f82cd200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82c8103090 0x7f82c81a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:15.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.759+0000 7f82cd200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82c8103090 0x7f82c81a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:58746/0 (socket says 172.21.15.5:58746) 2024-04-24T06:45:15.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.759+0000 7f82cd200640 1 -- 172.21.15.5:0/2920533384 learned_addr learned my addr 172.21.15.5:0/2920533384 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:15.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.759+0000 7f82cdc00640 1 --2- 172.21.15.5:0/2920533384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82c810c4b0 0x7f82c819a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:15.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.759+0000 7f82c7e00640 1 --2- 172.21.15.5:0/2920533384 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82c81039b0 0x7f82c81a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:15.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.759+0000 7f82cd200640 1 -- 172.21.15.5:0/2920533384 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82c81039b0 msgr2=0x7f82c81a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:15.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.759+0000 7f82cd200640 1 --2- 172.21.15.5:0/2920533384 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82c81039b0 0x7f82c81a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:15.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.759+0000 7f82cd200640 1 -- 172.21.15.5:0/2920533384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82c810c4b0 msgr2=0x7f82c819a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:15.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.759+0000 7f82cd200640 1 --2- 172.21.15.5:0/2920533384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82c810c4b0 0x7f82c819a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:15.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.759+0000 7f82cd200640 1 -- 172.21.15.5:0/2920533384 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82c819aff0 con 0x7f82c8103090 2024-04-24T06:45:15.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.760+0000 7f82cd200640 1 --2- 172.21.15.5:0/2920533384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82c8103090 0x7f82c81a06b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00e9f0 tx=0x7f82bc00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:15.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.760+0000 7f82c5600640 1 -- 172.21.15.5:0/2920533384 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc00cd80 con 0x7f82c8103090 2024-04-24T06:45:15.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.760+0000 7f82c5600640 1 -- 172.21.15.5:0/2920533384 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82bc005500 con 0x7f82c8103090 2024-04-24T06:45:15.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.760+0000 7f82c5600640 1 -- 172.21.15.5:0/2920533384 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc010690 con 0x7f82c8103090 2024-04-24T06:45:15.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.760+0000 7f82ce600640 1 -- 172.21.15.5:0/2920533384 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82c819b2e0 con 0x7f82c8103090 2024-04-24T06:45:15.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.760+0000 7f82ce600640 1 -- 172.21.15.5:0/2920533384 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82c81a7500 con 0x7f82c8103090 2024-04-24T06:45:15.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.762+0000 7f82ce600640 1 -- 172.21.15.5:0/2920533384 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82c8069be0 con 0x7f82c8103090 2024-04-24T06:45:15.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.765+0000 7f82c5600640 1 -- 172.21.15.5:0/2920533384 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 26) ==== 138929+0+0 (secure 0 0 0) 0x7f82bc0046c0 con 0x7f82c8103090 2024-04-24T06:45:15.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.767+0000 7f82c5600640 1 --2- 172.21.15.5:0/2920533384 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f82b00a3f10 0x7f82b00a63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:15.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.768+0000 7f82c5600640 1 -- 172.21.15.5:0/2920533384 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f82bc0d0ee0 con 0x7f82c8103090 2024-04-24T06:45:15.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.768+0000 7f82c7e00640 1 --2- 172.21.15.5:0/2920533384 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f82b00a3f10 0x7f82b00a63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:15.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.768+0000 7f82c7e00640 1 --2- 172.21.15.5:0/2920533384 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f82b00a3f10 0x7f82b00a63d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f82b4005ab0 tx=0x7f82b402a980 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:15.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:15.771+0000 7f82c5600640 1 -- 172.21.15.5:0/2920533384 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f82bc09d350 con 0x7f82c8103090 2024-04-24T06:45:15.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:15 smithi137 ceph-mon[33017]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-24T06:45:16.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:15 smithi037 ceph-mon[32864]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-24T06:45:16.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.088+0000 7f82ce600640 1 -- 172.21.15.5:0/2920533384 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82c81a76d0 con 0x7f82c8103090 2024-04-24T06:45:16.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.089+0000 7f82c5600640 1 -- 172.21.15.5:0/2920533384 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11856 (secure 0 0 0) 0x7f82bc09cc50 con 0x7f82c8103090 2024-04-24T06:45:16.092 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:45:16.092 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1735462750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1735462750},{"type":"v1","addr":"172.21.15.5:6837","nonce":1735462750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1499469370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1499469370},{"type":"v1","addr":"172.21.15.5:6841","nonce":1499469370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3705638847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3705638847},{"type":"v1","addr":"172.21.15.37:6835","nonce":3705638847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T06:41:34.764490+0000","modified":"2024-04-24T06:44:01.711021+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24550,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6837/1390103296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1390103296},{"type":"v1","addr":"172.21.15.37:6837","nonce":1390103296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3217754215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3217754215},{"type":"v1","addr":"172.21.15.137:6835","nonce":3217754215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6837/3044222976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3044222976},{"type":"v1","addr":"172.21.15.137:6837","nonce":3044222976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.5:6839/3173201459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3173201459},{"type":"v1","addr":"172.21.15.5:6839","nonce":3173201459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6839/1889774232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1889774232},{"type":"v1","addr":"172.21.15.137:6839","nonce":1889774232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6839/3536721753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":3536721753},{"type":"v1","addr":"172.21.15.37:6839","nonce":3536721753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24569,24539,24550]},"id":1}]} 2024-04-24T06:45:16.092 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 23 2024-04-24T06:45:16.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.092+0000 7f82ce600640 1 -- 172.21.15.5:0/2920533384 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f82b00a3f10 msgr2=0x7f82b00a63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:16.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.092+0000 7f82ce600640 1 --2- 172.21.15.5:0/2920533384 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f82b00a3f10 0x7f82b00a63d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f82b4005ab0 tx=0x7f82b402a980 comp rx=0 tx=0).stop 2024-04-24T06:45:16.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.092+0000 7f82ce600640 1 -- 172.21.15.5:0/2920533384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82c8103090 msgr2=0x7f82c81a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:16.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.092+0000 7f82ce600640 1 --2- 172.21.15.5:0/2920533384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82c8103090 0x7f82c81a06b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00e9f0 tx=0x7f82bc00edf0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.092+0000 7f82ce600640 1 -- 172.21.15.5:0/2920533384 shutdown_connections 2024-04-24T06:45:16.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.092+0000 7f82ce600640 1 --2- 172.21.15.5:0/2920533384 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f82b00a3f10 0x7f82b00a63d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.092+0000 7f82ce600640 1 --2- 172.21.15.5:0/2920533384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82c810c4b0 0x7f82c819a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.092+0000 7f82ce600640 1 --2- 172.21.15.5:0/2920533384 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82c81039b0 0x7f82c81a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.092+0000 7f82ce600640 1 --2- 172.21.15.5:0/2920533384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82c8103090 0x7f82c81a06b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.092+0000 7f82ce600640 1 -- 172.21.15.5:0/2920533384 >> 172.21.15.5:0/2920533384 conn(0x7f82c80fc890 msgr2=0x7f82c80fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:16.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.092+0000 7f82ce600640 1 -- 172.21.15.5:0/2920533384 shutdown_connections 2024-04-24T06:45:16.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.092+0000 7f82ce600640 1 -- 172.21.15.5:0/2920533384 wait complete. 2024-04-24T06:45:16.113 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T06:45:16.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.285+0000 7fd5ad200640 1 -- 172.21.15.5:0/2893899642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5a810e6c0 msgr2=0x7fd5a8110ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:16.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.285+0000 7fd5ad200640 1 --2- 172.21.15.5:0/2893899642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5a810e6c0 0x7fd5a8110ab0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0009ca0 tx=0x7fd5a002a950 comp rx=0 tx=0).stop 2024-04-24T06:45:16.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.285+0000 7fd5ad200640 1 -- 172.21.15.5:0/2893899642 shutdown_connections 2024-04-24T06:45:16.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.285+0000 7fd5ad200640 1 --2- 172.21.15.5:0/2893899642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5a810e6c0 0x7fd5a8110ab0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.285+0000 7fd5ad200640 1 --2- 172.21.15.5:0/2893899642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5a8074cd0 0x7fd5a810e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.285+0000 7fd5ad200640 1 --2- 172.21.15.5:0/2893899642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5a80743b0 0x7fd5a8074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.285+0000 7fd5ad200640 1 -- 172.21.15.5:0/2893899642 >> 172.21.15.5:0/2893899642 conn(0x7fd5a80fd950 msgr2=0x7fd5a80ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:16.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.285+0000 7fd5ad200640 1 -- 172.21.15.5:0/2893899642 shutdown_connections 2024-04-24T06:45:16.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.285+0000 7fd5ad200640 1 -- 172.21.15.5:0/2893899642 wait complete. 2024-04-24T06:45:16.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.287+0000 7fd5ad200640 1 Processor -- start 2024-04-24T06:45:16.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.287+0000 7fd5ad200640 1 -- start start 2024-04-24T06:45:16.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.287+0000 7fd5ad200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5a80743b0 0x7fd5a81a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.287+0000 7fd5ad200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5a8074cd0 0x7fd5a81a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.287+0000 7fd5ad200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5a810e6c0 0x7fd5a819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.287+0000 7fd5ad200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a8114410 con 0x7fd5a810e6c0 2024-04-24T06:45:16.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.287+0000 7fd5ad200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a8114290 con 0x7fd5a80743b0 2024-04-24T06:45:16.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.287+0000 7fd5ad200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a8114590 con 0x7fd5a8074cd0 2024-04-24T06:45:16.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.288+0000 7fd5a7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5a810e6c0 0x7fd5a819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.288+0000 7fd5a7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5a810e6c0 0x7fd5a819a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40262/0 (socket says 172.21.15.5:40262) 2024-04-24T06:45:16.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.288+0000 7fd5a7e00640 1 -- 172.21.15.5:0/1894704783 learned_addr learned my addr 172.21.15.5:0/1894704783 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:16.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.288+0000 7fd5a6a00640 1 --2- 172.21.15.5:0/1894704783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5a8074cd0 0x7fd5a81a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.288+0000 7fd5a7400640 1 --2- 172.21.15.5:0/1894704783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5a80743b0 0x7fd5a81a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.288+0000 7fd5a7e00640 1 -- 172.21.15.5:0/1894704783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5a8074cd0 msgr2=0x7fd5a81a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:16.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.288+0000 7fd5a7e00640 1 --2- 172.21.15.5:0/1894704783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5a8074cd0 0x7fd5a81a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.288+0000 7fd5a7e00640 1 -- 172.21.15.5:0/1894704783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5a80743b0 msgr2=0x7fd5a81a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:16.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.288+0000 7fd5a7e00640 1 --2- 172.21.15.5:0/1894704783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5a80743b0 0x7fd5a81a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.288+0000 7fd5a7e00640 1 -- 172.21.15.5:0/1894704783 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5a819aff0 con 0x7fd5a810e6c0 2024-04-24T06:45:16.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.289+0000 7fd5a7e00640 1 --2- 172.21.15.5:0/1894704783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5a810e6c0 0x7fd5a819a730 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0009dc0 tx=0x7fd5a0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:16.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.289+0000 7fd59fe00640 1 -- 172.21.15.5:0/1894704783 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0034040 con 0x7fd5a810e6c0 2024-04-24T06:45:16.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.289+0000 7fd59fe00640 1 -- 172.21.15.5:0/1894704783 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5a002f940 con 0x7fd5a810e6c0 2024-04-24T06:45:16.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.289+0000 7fd59fe00640 1 -- 172.21.15.5:0/1894704783 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a00046a0 con 0x7fd5a810e6c0 2024-04-24T06:45:16.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.289+0000 7fd5ad200640 1 -- 172.21.15.5:0/1894704783 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5a819b280 con 0x7fd5a810e6c0 2024-04-24T06:45:16.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.289+0000 7fd5ad200640 1 -- 172.21.15.5:0/1894704783 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5a81a73c0 con 0x7fd5a810e6c0 2024-04-24T06:45:16.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.294+0000 7fd59fe00640 1 -- 172.21.15.5:0/1894704783 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138929+0+0 (secure 0 0 0) 0x7fd5a0037020 con 0x7fd5a810e6c0 2024-04-24T06:45:16.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.294+0000 7fd5b15a6740 1 Processor -- start 2024-04-24T06:45:16.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.294+0000 7fd5b15a6740 1 -- start start 2024-04-24T06:45:16.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.294+0000 7fd5b15a6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e92530b40 0x561e92530f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.294+0000 7fd5b15a6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e92531460 0x561e9253b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.294+0000 7fd5b15a6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e9253bb30 0x561e9253df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.294+0000 7fd5b15a6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561e924961d0 con 0x561e9253bb30 2024-04-24T06:45:16.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.294+0000 7fd5b15a6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561e9243c450 con 0x561e92531460 2024-04-24T06:45:16.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.294+0000 7fd5b15a6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561e92492190 con 0x561e92530b40 2024-04-24T06:45:16.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.295+0000 7fd5a7400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e92531460 0x561e9253b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.295+0000 7fd5a7400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e92531460 0x561e9253b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:47606/0 (socket says 172.21.15.5:47606) 2024-04-24T06:45:16.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.295+0000 7fd5a7400640 1 -- 172.21.15.5:0/2119215130 learned_addr learned my addr 172.21.15.5:0/2119215130 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:16.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.295+0000 7fd5a6a00640 1 --2- 172.21.15.5:0/2119215130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e9253bb30 0x561e9253df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.295+0000 7fd5a6a00640 1 --2- 172.21.15.5:0/2119215130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e92530b40 0x561e92530f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.295+0000 7fd5a7400640 1 -- 172.21.15.5:0/2119215130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e92530b40 msgr2=0x561e92530f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:16.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.295+0000 7fd5a7400640 1 --2- 172.21.15.5:0/2119215130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e92530b40 0x561e92530f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.295+0000 7fd5a7400640 1 -- 172.21.15.5:0/2119215130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e9253bb30 msgr2=0x561e9253df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:16.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.295+0000 7fd5a7400640 1 --2- 172.21.15.5:0/2119215130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e9253bb30 0x561e9253df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.295+0000 7fd5a7400640 1 -- 172.21.15.5:0/2119215130 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e924f9310 con 0x561e92531460 2024-04-24T06:45:16.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.295+0000 7fd5a6a00640 1 --2- 172.21.15.5:0/2119215130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e9253bb30 0x561e9253df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:45:16.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.296+0000 7fd59fe00640 1 --2- 172.21.15.5:0/1894704783 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fd58c0a3f50 0x7fd58c0a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.296+0000 7fd5a7400640 1 --2- 172.21.15.5:0/2119215130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e92531460 0x561e9253b5f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd598009d20 tx=0x7fd598011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97bb05e790179299 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:16.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.296+0000 7fd59fe00640 1 -- 172.21.15.5:0/1894704783 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fd5a00f1740 con 0x7fd5a810e6c0 2024-04-24T06:45:16.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.296+0000 7fd59cc00640 1 -- 172.21.15.5:0/2119215130 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59801e070 con 0x561e92531460 2024-04-24T06:45:16.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.296+0000 7fd59cc00640 1 -- 172.21.15.5:0/2119215130 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd598019b10 con 0x561e92531460 2024-04-24T06:45:16.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.296+0000 7fd59cc00640 1 -- 172.21.15.5:0/2119215130 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59800ebf0 con 0x561e92531460 2024-04-24T06:45:16.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.296+0000 7fd5a6a00640 1 --2- 172.21.15.5:0/1894704783 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fd58c0a3f50 0x7fd58c0a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.297+0000 7fd5b15a6740 1 -- 172.21.15.5:0/2119215130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e92531460 msgr2=0x561e9253b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:16.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.297+0000 7fd5b15a6740 1 --2- 172.21.15.5:0/2119215130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e92531460 0x561e9253b5f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd598009d20 tx=0x7fd598011810 comp rx=0 tx=0).stop 2024-04-24T06:45:16.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.297+0000 7fd5a6a00640 1 --2- 172.21.15.5:0/1894704783 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fd58c0a3f50 0x7fd58c0a6410 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x561e9252dff0 tx=0x7fd594005f90 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:16.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.297+0000 7fd5b15a6740 1 -- 172.21.15.5:0/2119215130 shutdown_connections 2024-04-24T06:45:16.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.297+0000 7fd5b15a6740 1 --2- 172.21.15.5:0/2119215130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e9253bb30 0x561e9253df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.297+0000 7fd5b15a6740 1 --2- 172.21.15.5:0/2119215130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e92531460 0x561e9253b5f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.297+0000 7fd5b15a6740 1 --2- 172.21.15.5:0/2119215130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e92530b40 0x561e92530f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.297+0000 7fd5b15a6740 1 -- 172.21.15.5:0/2119215130 >> 172.21.15.5:0/2119215130 conn(0x561e9252ce60 msgr2=0x561e9252d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:16.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.297+0000 7fd5b15a6740 1 -- 172.21.15.5:0/2119215130 shutdown_connections 2024-04-24T06:45:16.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.297+0000 7fd5b15a6740 1 -- 172.21.15.5:0/2119215130 wait complete. 2024-04-24T06:45:16.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.298+0000 7fd5b15a6740 1 -- start start 2024-04-24T06:45:16.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.298+0000 7fd5b15a6740 1 Processor -- start 2024-04-24T06:45:16.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.299+0000 7fd5b15a6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e92530b40 0x561e9253b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.299+0000 7fd5b15a6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e92531460 0x561e92538850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.299+0000 7fd5b15a6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e9253bb30 0x561e92538f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.299+0000 7fd5b15a6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561e9253e9c0 con 0x561e92531460 2024-04-24T06:45:16.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.299+0000 7fd5b15a6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561e9244f810 con 0x561e92530b40 2024-04-24T06:45:16.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.299+0000 7fd5b15a6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561e9253eb40 con 0x561e9253bb30 2024-04-24T06:45:16.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.299+0000 7fd5b15a6740 10 client.0 _ll_register_callbacks cb 0x561e9241d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:45:16.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.299+0000 7fd5a7400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e92530b40 0x561e9253b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.299+0000 7fd5a7400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e92530b40 0x561e9253b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:47614/0 (socket says 172.21.15.5:47614) 2024-04-24T06:45:16.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.299+0000 7fd5a7400640 1 -- 172.21.15.5:0/1750203418 learned_addr learned my addr 172.21.15.5:0/1750203418 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:16.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.299+0000 7fd5a6a00640 1 --2- 172.21.15.5:0/1750203418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e92531460 0x561e92538850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.299+0000 7fd5a7e00640 1 --2- 172.21.15.5:0/1750203418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e9253bb30 0x561e92538f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.300+0000 7fd5a7400640 1 -- 172.21.15.5:0/1750203418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e9253bb30 msgr2=0x561e92538f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:16.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.300+0000 7fd5a7400640 1 --2- 172.21.15.5:0/1750203418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e9253bb30 0x561e92538f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.300+0000 7fd5a7400640 1 -- 172.21.15.5:0/1750203418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e92531460 msgr2=0x561e92538850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:16.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.300+0000 7fd5a7400640 1 --2- 172.21.15.5:0/1750203418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e92531460 0x561e92538850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.300+0000 7fd5a7400640 1 -- 172.21.15.5:0/1750203418 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e924603f0 con 0x561e92530b40 2024-04-24T06:45:16.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.300+0000 7fd5a7e00640 1 --2- 172.21.15.5:0/1750203418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e9253bb30 0x561e92538f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:45:16.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.300+0000 7fd5a7400640 1 --2- 172.21.15.5:0/1750203418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e92530b40 0x561e9253b560 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd598005890 tx=0x7fd598009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85e1c6fb3c184a6d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:16.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.300+0000 7fd591600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:45:16.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.300+0000 7fd591600640 1 -- 172.21.15.5:0/1750203418 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd598026020 con 0x561e92530b40 2024-04-24T06:45:16.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.301+0000 7fd5b15a6740 1 -- 172.21.15.5:0/1750203418 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd598019b10 con 0x561e92530b40 2024-04-24T06:45:16.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.301+0000 7fd591600640 1 -- 172.21.15.5:0/1750203418 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fd598019840 con 0x561e92530b40 2024-04-24T06:45:16.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.301+0000 7fd591600640 1 -- 172.21.15.5:0/1750203418 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd598031050 con 0x561e92530b40 2024-04-24T06:45:16.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.301+0000 7fd5b15a6740 10 client.25589 fetch_fsmap learned FSMap version 23 2024-04-24T06:45:16.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.301+0000 7fd5b15a6740 1 -- 172.21.15.5:0/1750203418 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x561e921a83f0 con 0x561e92530b40 2024-04-24T06:45:16.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.301+0000 7fd591600640 1 -- 172.21.15.5:0/1750203418 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd598019b10 con 0x561e92530b40 2024-04-24T06:45:16.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.301+0000 7fd591600640 1 -- 172.21.15.5:0/1750203418 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fd5980046c0 con 0x561e92530b40 2024-04-24T06:45:16.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.301+0000 7fd5b15a6740 10 client.25589 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T06:45:16.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.301+0000 7fd5b15a6740 10 client.25589 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-24T06:45:16.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.301+0000 7fd5b15a6740 20 client.25589 populate_metadata read hostname 'smithi005' 2024-04-24T06:45:16.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.302+0000 7fd5b15a6740 1 --2- 172.21.15.5:0/1750203418 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x561e9254bc70 0x561e9254e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.302+0000 7fd5b15a6740 4 client.25589 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:45:16.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.302+0000 7fd5b15a6740 1 -- 172.21.15.5:0/1750203418 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e924a3bf0 con 0x561e9254bc70 2024-04-24T06:45:16.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.302+0000 7fd5a6a00640 1 --2- 172.21.15.5:0/1750203418 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x561e9254bc70 0x561e9254e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.303+0000 7fd5a6a00640 1 --2- 172.21.15.5:0/1750203418 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x561e9254bc70 0x561e9254e060 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a3a9558f75bddf0c server_cookie=4c6d59e28c9c37f7 in_seq=0 out_seq=0 2024-04-24T06:45:16.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.303+0000 7fd591600640 10 client.25589 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:45:16.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.303+0000 7fd591600640 1 -- 172.21.15.5:0/1750203418 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561e924a3bf0 con 0x561e9254bc70 2024-04-24T06:45:16.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.303+0000 7fd591600640 10 client.25589 handle_command_reply: tid=0 2024-04-24T06:45:16.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.303+0000 7fd5b15a6740 1 client.25589 shutdown 2024-04-24T06:45:16.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.304+0000 7fd5b15a6740 1 -- 172.21.15.5:0/1750203418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e92530b40 msgr2=0x561e9253b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:16.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.304+0000 7fd5b15a6740 1 --2- 172.21.15.5:0/1750203418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e92530b40 0x561e9253b560 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd598005890 tx=0x7fd598009ed0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.304+0000 7fd5b15a6740 1 -- 172.21.15.5:0/1750203418 shutdown_connections 2024-04-24T06:45:16.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.304+0000 7fd5b15a6740 1 --2- 172.21.15.5:0/1750203418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e9253bb30 0x561e92538f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.304+0000 7fd5b15a6740 1 --2- 172.21.15.5:0/1750203418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e92531460 0x561e92538850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.304+0000 7fd5b15a6740 1 --2- 172.21.15.5:0/1750203418 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x561e9254bc70 0x561e9254e060 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.304+0000 7fd5b15a6740 1 --2- 172.21.15.5:0/1750203418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e92530b40 0x561e9253b560 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.304+0000 7fd5b15a6740 1 -- 172.21.15.5:0/1750203418 >> 172.21.15.5:0/1750203418 conn(0x561e9252ce60 msgr2=0x561e92532c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:16.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.304+0000 7fd591600640 0 client.25589 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:45:16.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.304+0000 7fd5b15a6740 1 -- 172.21.15.5:0/1750203418 shutdown_connections 2024-04-24T06:45:16.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.304+0000 7fd5b15a6740 1 -- 172.21.15.5:0/1750203418 wait complete. 2024-04-24T06:45:16.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.304+0000 7fd5b15a6740 20 client.25589 trim_cache size 0 max 16384 2024-04-24T06:45:16.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.325+0000 7fd5b15a6740 1 Processor -- start 2024-04-24T06:45:16.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.325+0000 7fd5b15a6740 1 -- start start 2024-04-24T06:45:16.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.325+0000 7fd5b15a6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e92530b40 0x561e92547880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.325+0000 7fd5b15a6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e92531460 0x561e9253b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.325+0000 7fd5b15a6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e9253bb30 0x561e9253b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.325+0000 7fd5b15a6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a81b25d0 con 0x561e9253bb30 2024-04-24T06:45:16.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.325+0000 7fd5b15a6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a81afea0 con 0x561e92530b40 2024-04-24T06:45:16.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.325+0000 7fd5b15a6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a81adcf0 con 0x561e92531460 2024-04-24T06:45:16.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.326+0000 7fd5a7400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e92530b40 0x561e92547880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.326+0000 7fd5a7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e9253bb30 0x561e9253b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.326+0000 7fd5a6a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e92531460 0x561e9253b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.326+0000 7fd5a7400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e92530b40 0x561e92547880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:47624/0 (socket says 172.21.15.5:47624) 2024-04-24T06:45:16.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.326+0000 7fd5a7400640 1 -- 172.21.15.5:0/4265469264 learned_addr learned my addr 172.21.15.5:0/4265469264 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:16.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.326+0000 7fd5a7400640 1 -- 172.21.15.5:0/4265469264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e92531460 msgr2=0x561e9253b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:16.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.326+0000 7fd5a7400640 1 --2- 172.21.15.5:0/4265469264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e92531460 0x561e9253b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.326+0000 7fd5a7400640 1 -- 172.21.15.5:0/4265469264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e9253bb30 msgr2=0x561e9253b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:16.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.326+0000 7fd5a7400640 1 --2- 172.21.15.5:0/4265469264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e9253bb30 0x561e9253b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.326+0000 7fd5a7400640 1 -- 172.21.15.5:0/4265469264 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e92539300 con 0x561e92530b40 2024-04-24T06:45:16.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.326+0000 7fd5a7400640 1 --2- 172.21.15.5:0/4265469264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e92530b40 0x561e92547880 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd598011850 tx=0x7fd5980096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b91fbec141ccf384 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:16.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.327+0000 7fd593e00640 1 -- 172.21.15.5:0/4265469264 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd598026070 con 0x561e92530b40 2024-04-24T06:45:16.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.327+0000 7fd593e00640 1 -- 172.21.15.5:0/4265469264 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5980098f0 con 0x561e92530b40 2024-04-24T06:45:16.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.327+0000 7fd593e00640 1 -- 172.21.15.5:0/4265469264 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59801e040 con 0x561e92530b40 2024-04-24T06:45:16.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.328+0000 7fd5b15a6740 1 -- 172.21.15.5:0/4265469264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e92530b40 msgr2=0x561e92547880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:16.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.328+0000 7fd5b15a6740 1 --2- 172.21.15.5:0/4265469264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e92530b40 0x561e92547880 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd598011850 tx=0x7fd5980096a0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.328+0000 7fd5b15a6740 1 -- 172.21.15.5:0/4265469264 shutdown_connections 2024-04-24T06:45:16.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.328+0000 7fd5b15a6740 1 --2- 172.21.15.5:0/4265469264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e9253bb30 0x561e9253b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.328+0000 7fd5b15a6740 1 --2- 172.21.15.5:0/4265469264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e92531460 0x561e9253b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.328+0000 7fd5b15a6740 1 --2- 172.21.15.5:0/4265469264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e92530b40 0x561e92547880 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.328+0000 7fd5b15a6740 1 -- 172.21.15.5:0/4265469264 >> 172.21.15.5:0/4265469264 conn(0x561e9252ce60 msgr2=0x561e9253de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:16.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.328+0000 7fd5b15a6740 1 -- 172.21.15.5:0/4265469264 shutdown_connections 2024-04-24T06:45:16.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.328+0000 7fd5b15a6740 1 -- 172.21.15.5:0/4265469264 wait complete. 2024-04-24T06:45:16.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.328+0000 7fd5b15a6740 1 -- start start 2024-04-24T06:45:16.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.329+0000 7fd5b15a6740 1 Processor -- start 2024-04-24T06:45:16.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.329+0000 7fd5b15a6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e92530b40 0x561e92534c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.329+0000 7fd5b15a6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e92531460 0x561e92535180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.329+0000 7fd5b15a6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e9253bb30 0x561e925356c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.329+0000 7fd5b15a6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a81ae360 con 0x561e92530b40 2024-04-24T06:45:16.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.329+0000 7fd5b15a6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a81af7d0 con 0x561e92531460 2024-04-24T06:45:16.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.329+0000 7fd5b15a6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a81aea30 con 0x561e9253bb30 2024-04-24T06:45:16.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.329+0000 7fd5b15a6740 10 client.0 _ll_register_callbacks cb 0x7fd59801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:45:16.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.329+0000 7fd5a7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e92530b40 0x561e92534c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.329+0000 7fd5a6a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e92531460 0x561e92535180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.330+0000 7fd5a6a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e92531460 0x561e92535180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:47626/0 (socket says 172.21.15.5:47626) 2024-04-24T06:45:16.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.330+0000 7fd5a6a00640 1 -- 172.21.15.5:0/3192587919 learned_addr learned my addr 172.21.15.5:0/3192587919 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:16.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.330+0000 7fd5a7e00640 1 --2- 172.21.15.5:0/3192587919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e9253bb30 0x561e925356c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.330+0000 7fd5a6a00640 1 -- 172.21.15.5:0/3192587919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e9253bb30 msgr2=0x561e925356c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:16.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.330+0000 7fd5a6a00640 1 --2- 172.21.15.5:0/3192587919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e9253bb30 0x561e925356c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.330+0000 7fd5a6a00640 1 -- 172.21.15.5:0/3192587919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e92530b40 msgr2=0x561e92534c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:16.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.330+0000 7fd5a6a00640 1 --2- 172.21.15.5:0/3192587919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e92530b40 0x561e92534c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.330+0000 7fd5a6a00640 1 -- 172.21.15.5:0/3192587919 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e922727d0 con 0x561e92531460 2024-04-24T06:45:16.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.330+0000 7fd5a6a00640 1 --2- 172.21.15.5:0/3192587919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e92531460 0x561e92535180 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x561e925380b0 tx=0x7fd5940046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5675449e00040ef4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:16.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.330+0000 7fd591600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:45:16.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.331+0000 7fd591600640 1 -- 172.21.15.5:0/3192587919 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd594030070 con 0x561e92531460 2024-04-24T06:45:16.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.331+0000 7fd591600640 1 -- 172.21.15.5:0/3192587919 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fd594004020 con 0x561e92531460 2024-04-24T06:45:16.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.331+0000 7fd5b15a6740 1 -- 172.21.15.5:0/3192587919 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5980098f0 con 0x561e92531460 2024-04-24T06:45:16.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.331+0000 7fd591600640 1 -- 172.21.15.5:0/3192587919 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd594004220 con 0x561e92531460 2024-04-24T06:45:16.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.331+0000 7fd5b15a6740 10 client.25595 fetch_fsmap learned FSMap version 23 2024-04-24T06:45:16.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.331+0000 7fd5b15a6740 1 -- 172.21.15.5:0/3192587919 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x561e92544130 con 0x561e92531460 2024-04-24T06:45:16.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.331+0000 7fd591600640 1 -- 172.21.15.5:0/3192587919 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5980098f0 con 0x561e92531460 2024-04-24T06:45:16.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.332+0000 7fd591600640 1 -- 172.21.15.5:0/3192587919 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fd59402c300 con 0x561e92531460 2024-04-24T06:45:16.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.332+0000 7fd5b15a6740 10 client.25595 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T06:45:16.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.332+0000 7fd5b15a6740 10 client.25595 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-24T06:45:16.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.332+0000 7fd5b15a6740 20 client.25595 populate_metadata read hostname 'smithi005' 2024-04-24T06:45:16.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.332+0000 7fd5b15a6740 1 --2- 172.21.15.5:0/3192587919 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x561e925ba980 0x561e925bcd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.332+0000 7fd5b15a6740 4 client.25595 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:45:16.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.332+0000 7fd5b15a6740 1 -- 172.21.15.5:0/3192587919 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561e9249da60 con 0x561e925ba980 2024-04-24T06:45:16.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.333+0000 7fd5a7400640 1 --2- 172.21.15.5:0/3192587919 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x561e925ba980 0x561e925bcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.333+0000 7fd5a7400640 1 --2- 172.21.15.5:0/3192587919 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x561e925ba980 0x561e925bcd70 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=646b14ecd2aac11c server_cookie=ed2de7a00ccf99ac in_seq=0 out_seq=0 2024-04-24T06:45:16.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.333+0000 7fd591600640 10 client.25595 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:45:16.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.334+0000 7fd591600640 1 -- 172.21.15.5:0/3192587919 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561e9249da60 con 0x561e925ba980 2024-04-24T06:45:16.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.334+0000 7fd591600640 10 client.25595 handle_command_reply: tid=0 2024-04-24T06:45:16.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.334+0000 7fd5b15a6740 1 client.25595 shutdown 2024-04-24T06:45:16.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.334+0000 7fd5b15a6740 1 -- 172.21.15.5:0/3192587919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e92531460 msgr2=0x561e92535180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:16.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.334+0000 7fd5b15a6740 1 --2- 172.21.15.5:0/3192587919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e92531460 0x561e92535180 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x561e925380b0 tx=0x7fd5940046c0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.334+0000 7fd5b15a6740 1 -- 172.21.15.5:0/3192587919 shutdown_connections 2024-04-24T06:45:16.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.334+0000 7fd5b15a6740 1 --2- 172.21.15.5:0/3192587919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e9253bb30 0x561e925356c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.334+0000 7fd5b15a6740 1 --2- 172.21.15.5:0/3192587919 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x561e925ba980 0x561e925bcd70 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.334+0000 7fd5b15a6740 1 --2- 172.21.15.5:0/3192587919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e92531460 0x561e92535180 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.334+0000 7fd5b15a6740 1 --2- 172.21.15.5:0/3192587919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e92530b40 0x561e92534c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.334+0000 7fd591600640 0 client.25595 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:45:16.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.334+0000 7fd5b15a6740 1 -- 172.21.15.5:0/3192587919 >> 172.21.15.5:0/3192587919 conn(0x561e9252ce60 msgr2=0x561e9253df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:16.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.334+0000 7fd5b15a6740 1 -- 172.21.15.5:0/3192587919 shutdown_connections 2024-04-24T06:45:16.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.334+0000 7fd5b15a6740 1 -- 172.21.15.5:0/3192587919 wait complete. 2024-04-24T06:45:16.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.335+0000 7fd5b15a6740 20 client.25595 trim_cache size 0 max 16384 2024-04-24T06:45:16.339 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:45:16.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.335+0000 7fd5ad200640 1 -- 172.21.15.5:0/1894704783 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fd58c0a3f50 msgr2=0x7fd58c0a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:16.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.335+0000 7fd5ad200640 1 --2- 172.21.15.5:0/1894704783 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fd58c0a3f50 0x7fd58c0a6410 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x561e9252dff0 tx=0x7fd594005f90 comp rx=0 tx=0).stop 2024-04-24T06:45:16.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.335+0000 7fd5ad200640 1 -- 172.21.15.5:0/1894704783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5a810e6c0 msgr2=0x7fd5a819a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:16.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.335+0000 7fd5ad200640 1 --2- 172.21.15.5:0/1894704783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5a810e6c0 0x7fd5a819a730 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0009dc0 tx=0x7fd5a0003c10 comp rx=0 tx=0).stop 2024-04-24T06:45:16.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.336+0000 7fd5ad200640 1 -- 172.21.15.5:0/1894704783 shutdown_connections 2024-04-24T06:45:16.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.336+0000 7fd5ad200640 1 --2- 172.21.15.5:0/1894704783 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fd58c0a3f50 0x7fd58c0a6410 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.336+0000 7fd5ad200640 1 --2- 172.21.15.5:0/1894704783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5a810e6c0 0x7fd5a819a730 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.336+0000 7fd5ad200640 1 --2- 172.21.15.5:0/1894704783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5a8074cd0 0x7fd5a81a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.336+0000 7fd5ad200640 1 --2- 172.21.15.5:0/1894704783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5a80743b0 0x7fd5a81a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.336+0000 7fd5ad200640 1 -- 172.21.15.5:0/1894704783 >> 172.21.15.5:0/1894704783 conn(0x7fd5a80fd950 msgr2=0x7fd5a80fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:16.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.336+0000 7fd5ad200640 1 -- 172.21.15.5:0/1894704783 shutdown_connections 2024-04-24T06:45:16.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.336+0000 7fd5ad200640 1 -- 172.21.15.5:0/1894704783 wait complete. 2024-04-24T06:45:16.355 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T06:45:16.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.533+0000 7fe43fe00640 1 -- 172.21.15.5:0/2134275823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe440104c30 msgr2=0x7fe440108c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:16.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.533+0000 7fe43fe00640 1 --2- 172.21.15.5:0/2134275823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe440104c30 0x7fe440108c70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe42c009860 tx=0x7fe42c02a340 comp rx=0 tx=0).stop 2024-04-24T06:45:16.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.533+0000 7fe43fe00640 1 -- 172.21.15.5:0/2134275823 shutdown_connections 2024-04-24T06:45:16.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.533+0000 7fe43fe00640 1 --2- 172.21.15.5:0/2134275823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe440109350 0x7fe440110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.533+0000 7fe43fe00640 1 --2- 172.21.15.5:0/2134275823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe440104c30 0x7fe440108c70 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.533+0000 7fe43fe00640 1 --2- 172.21.15.5:0/2134275823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe440104280 0x7fe440104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.533+0000 7fe43fe00640 1 -- 172.21.15.5:0/2134275823 >> 172.21.15.5:0/2134275823 conn(0x7fe4400ffae0 msgr2=0x7fe440101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:16.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.533+0000 7fe43fe00640 1 -- 172.21.15.5:0/2134275823 shutdown_connections 2024-04-24T06:45:16.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.533+0000 7fe43fe00640 1 -- 172.21.15.5:0/2134275823 wait complete. 2024-04-24T06:45:16.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.534+0000 7fe43fe00640 1 Processor -- start 2024-04-24T06:45:16.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.534+0000 7fe43fe00640 1 -- start start 2024-04-24T06:45:16.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.535+0000 7fe43fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe440104280 0x7fe4401a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.535+0000 7fe43fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe440109350 0x7fe4401a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.535+0000 7fe43fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe44019a800 0x7fe44019acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.535+0000 7fe43fe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe4401143c0 con 0x7fe44019a800 2024-04-24T06:45:16.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.535+0000 7fe43fe00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe440114240 con 0x7fe440104280 2024-04-24T06:45:16.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.535+0000 7fe43fe00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe440114540 con 0x7fe440109350 2024-04-24T06:45:16.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.535+0000 7fe43ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe440104280 0x7fe4401a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.535+0000 7fe43ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe440104280 0x7fe4401a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:47638/0 (socket says 172.21.15.5:47638) 2024-04-24T06:45:16.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.535+0000 7fe43ea00640 1 -- 172.21.15.5:0/2370360872 learned_addr learned my addr 172.21.15.5:0/2370360872 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:16.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.535+0000 7fe43e000640 1 --2- 172.21.15.5:0/2370360872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe440109350 0x7fe4401a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.535+0000 7fe43f400640 1 --2- 172.21.15.5:0/2370360872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe44019a800 0x7fe44019acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.536+0000 7fe43ea00640 1 -- 172.21.15.5:0/2370360872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe440109350 msgr2=0x7fe4401a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:16.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.536+0000 7fe43ea00640 1 --2- 172.21.15.5:0/2370360872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe440109350 0x7fe4401a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.536+0000 7fe43ea00640 1 -- 172.21.15.5:0/2370360872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe44019a800 msgr2=0x7fe44019acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:16.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.536+0000 7fe43ea00640 1 --2- 172.21.15.5:0/2370360872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe44019a800 0x7fe44019acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.536+0000 7fe43ea00640 1 -- 172.21.15.5:0/2370360872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe44019b390 con 0x7fe440104280 2024-04-24T06:45:16.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.536+0000 7fe43e000640 1 --2- 172.21.15.5:0/2370360872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe440109350 0x7fe4401a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:45:16.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.536+0000 7fe43ea00640 1 --2- 172.21.15.5:0/2370360872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe440104280 0x7fe4401a0690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe43000e9c0 tx=0x7fe43000edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:16.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.537+0000 7fe437400640 1 -- 172.21.15.5:0/2370360872 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43000cd80 con 0x7fe440104280 2024-04-24T06:45:16.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.537+0000 7fe437400640 1 -- 172.21.15.5:0/2370360872 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4300035f0 con 0x7fe440104280 2024-04-24T06:45:16.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.537+0000 7fe43fe00640 1 -- 172.21.15.5:0/2370360872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe44019b680 con 0x7fe440104280 2024-04-24T06:45:16.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.537+0000 7fe437400640 1 -- 172.21.15.5:0/2370360872 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe430010670 con 0x7fe440104280 2024-04-24T06:45:16.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.537+0000 7fe43fe00640 1 -- 172.21.15.5:0/2370360872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4401a7880 con 0x7fe440104280 2024-04-24T06:45:16.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.540+0000 7fe44847f740 1 Processor -- start 2024-04-24T06:45:16.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.540+0000 7fe44847f740 1 -- start start 2024-04-24T06:45:16.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.540+0000 7fe44847f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636bf8adb40 0x5636bf8adf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.540+0000 7fe44847f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636bf8ae460 0x5636bf8b85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.540+0000 7fe44847f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636bf8b8b30 0x5636bf8baf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.540+0000 7fe44847f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5636bf8131d0 con 0x5636bf8adb40 2024-04-24T06:45:16.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.540+0000 7fe44847f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5636bf7b9450 con 0x5636bf8ae460 2024-04-24T06:45:16.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.540+0000 7fe44847f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5636bf80f190 con 0x5636bf8b8b30 2024-04-24T06:45:16.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.541+0000 7fe43f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636bf8adb40 0x5636bf8adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.541+0000 7fe43f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636bf8adb40 0x5636bf8adf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40316/0 (socket says 172.21.15.5:40316) 2024-04-24T06:45:16.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.541+0000 7fe43f400640 1 -- 172.21.15.5:0/3063683676 learned_addr learned my addr 172.21.15.5:0/3063683676 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:16.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.541+0000 7fe43e000640 1 --2- 172.21.15.5:0/3063683676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636bf8b8b30 0x5636bf8baf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.541+0000 7fe43f400640 1 -- 172.21.15.5:0/3063683676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636bf8b8b30 msgr2=0x5636bf8baf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:16.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.541+0000 7fe43f400640 1 --2- 172.21.15.5:0/3063683676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636bf8b8b30 0x5636bf8baf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.541+0000 7fe43f400640 1 -- 172.21.15.5:0/3063683676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636bf8ae460 msgr2=0x5636bf8b85f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T06:45:16.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.541+0000 7fe43f400640 1 --2- 172.21.15.5:0/3063683676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636bf8ae460 0x5636bf8b85f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.541+0000 7fe43f400640 1 -- 172.21.15.5:0/3063683676 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636bf876310 con 0x5636bf8adb40 2024-04-24T06:45:16.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.542+0000 7fe43f400640 1 --2- 172.21.15.5:0/3063683676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636bf8adb40 0x5636bf8adf20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe438009f60 tx=0x7fe438009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b32d846515eab92d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:16.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.542+0000 7fe437400640 1 -- 172.21.15.5:0/2370360872 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138929+0+0 (secure 0 0 0) 0x7fe4300037a0 con 0x7fe440104280 2024-04-24T06:45:16.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.543+0000 7fe42b400640 1 -- 172.21.15.5:0/3063683676 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43801e070 con 0x5636bf8adb40 2024-04-24T06:45:16.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.543+0000 7fe42b400640 1 -- 172.21.15.5:0/3063683676 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe438003c30 con 0x5636bf8adb40 2024-04-24T06:45:16.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.543+0000 7fe42b400640 1 -- 172.21.15.5:0/3063683676 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43800f990 con 0x5636bf8adb40 2024-04-24T06:45:16.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.543+0000 7fe44847f740 1 -- 172.21.15.5:0/3063683676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636bf8adb40 msgr2=0x5636bf8adf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:16.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.543+0000 7fe44847f740 1 --2- 172.21.15.5:0/3063683676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636bf8adb40 0x5636bf8adf20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe438009f60 tx=0x7fe438009f90 comp rx=0 tx=0).stop 2024-04-24T06:45:16.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.544+0000 7fe44847f740 1 -- 172.21.15.5:0/3063683676 shutdown_connections 2024-04-24T06:45:16.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.544+0000 7fe44847f740 1 --2- 172.21.15.5:0/3063683676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636bf8b8b30 0x5636bf8baf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.544+0000 7fe44847f740 1 --2- 172.21.15.5:0/3063683676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636bf8ae460 0x5636bf8b85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.544+0000 7fe44847f740 1 --2- 172.21.15.5:0/3063683676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636bf8adb40 0x5636bf8adf20 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.544+0000 7fe44847f740 1 -- 172.21.15.5:0/3063683676 >> 172.21.15.5:0/3063683676 conn(0x5636bf8a9e60 msgr2=0x5636bf8aa240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:16.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.544+0000 7fe44847f740 1 -- 172.21.15.5:0/3063683676 shutdown_connections 2024-04-24T06:45:16.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.544+0000 7fe44847f740 1 -- 172.21.15.5:0/3063683676 wait complete. 2024-04-24T06:45:16.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.545+0000 7fe44847f740 1 -- start start 2024-04-24T06:45:16.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.545+0000 7fe437400640 1 --2- 172.21.15.5:0/2370360872 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fe4240a3ef0 0x7fe4240a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.545+0000 7fe44847f740 1 Processor -- start 2024-04-24T06:45:16.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.545+0000 7fe437400640 1 -- 172.21.15.5:0/2370360872 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fe4300d0c70 con 0x7fe440104280 2024-04-24T06:45:16.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.546+0000 7fe43f400640 1 --2- 172.21.15.5:0/2370360872 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fe4240a3ef0 0x7fe4240a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.546+0000 7fe44847f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636bf8adb40 0x5636bf8b5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.546+0000 7fe44847f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636bf8ae460 0x5636bf8b5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.546+0000 7fe44847f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636bf8b8b30 0x5636bf8b63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.546+0000 7fe44847f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5636bf8bb9c0 con 0x5636bf8adb40 2024-04-24T06:45:16.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.546+0000 7fe44847f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5636bf7cc810 con 0x5636bf8ae460 2024-04-24T06:45:16.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.546+0000 7fe44847f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5636bf8bbb40 con 0x5636bf8b8b30 2024-04-24T06:45:16.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.546+0000 7fe44847f740 10 client.0 _ll_register_callbacks cb 0x5636bf79a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:45:16.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.547+0000 7fe43e000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636bf8ae460 0x5636bf8b5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.547+0000 7fe43e000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636bf8ae460 0x5636bf8b5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:47652/0 (socket says 172.21.15.5:47652) 2024-04-24T06:45:16.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.547+0000 7fe43e000640 1 -- 172.21.15.5:0/2322248943 learned_addr learned my addr 172.21.15.5:0/2322248943 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:16.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.547+0000 7fe43f400640 1 --2- 172.21.15.5:0/2370360872 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fe4240a3ef0 0x7fe4240a63b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe438006010 tx=0x7fe438000f40 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:16.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.547+0000 7fe43ea00640 1 --2- 172.21.15.5:0/2322248943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636bf8adb40 0x5636bf8b5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.547+0000 7fe43e000640 1 -- 172.21.15.5:0/2322248943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636bf8b8b30 msgr2=0x5636bf8b63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:16.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.547+0000 7fe43f400640 1 --2- 172.21.15.5:0/2322248943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636bf8b8b30 0x5636bf8b63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.547+0000 7fe43e000640 1 --2- 172.21.15.5:0/2322248943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636bf8b8b30 0x5636bf8b63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.547+0000 7fe43e000640 1 -- 172.21.15.5:0/2322248943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636bf8adb40 msgr2=0x5636bf8b5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:16.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.547+0000 7fe43e000640 1 --2- 172.21.15.5:0/2322248943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636bf8adb40 0x5636bf8b5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.548+0000 7fe43e000640 1 -- 172.21.15.5:0/2322248943 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636bf7dd3f0 con 0x5636bf8ae460 2024-04-24T06:45:16.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.548+0000 7fe43ea00640 1 --2- 172.21.15.5:0/2322248943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636bf8adb40 0x5636bf8b5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:45:16.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.548+0000 7fe43f400640 1 --2- 172.21.15.5:0/2322248943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636bf8b8b30 0x5636bf8b63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:45:16.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.548+0000 7fe43e000640 1 --2- 172.21.15.5:0/2322248943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636bf8ae460 0x5636bf8b5e60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe42c000c00 tx=0x7fe42c009490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a4a229a692462f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:16.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.548+0000 7fe41fe00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:45:16.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.548+0000 7fe41fe00640 1 -- 172.21.15.5:0/2322248943 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c037040 con 0x5636bf8ae460 2024-04-24T06:45:16.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.548+0000 7fe41fe00640 1 -- 172.21.15.5:0/2322248943 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fe42c030960 con 0x5636bf8ae460 2024-04-24T06:45:16.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.548+0000 7fe44847f740 1 -- 172.21.15.5:0/2322248943 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe438003c30 con 0x5636bf8ae460 2024-04-24T06:45:16.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.549+0000 7fe41fe00640 1 -- 172.21.15.5:0/2322248943 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe42c04c050 con 0x5636bf8ae460 2024-04-24T06:45:16.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.549+0000 7fe44847f740 10 client.25604 fetch_fsmap learned FSMap version 23 2024-04-24T06:45:16.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.549+0000 7fe44847f740 1 -- 172.21.15.5:0/2322248943 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5636bf5253f0 con 0x5636bf8ae460 2024-04-24T06:45:16.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.549+0000 7fe41fe00640 1 -- 172.21.15.5:0/2322248943 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe438003c30 con 0x5636bf8ae460 2024-04-24T06:45:16.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.550+0000 7fe41fe00640 1 -- 172.21.15.5:0/2322248943 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fe42c03d7c0 con 0x5636bf8ae460 2024-04-24T06:45:16.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.550+0000 7fe44847f740 10 client.25604 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T06:45:16.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.550+0000 7fe44847f740 10 client.25604 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T06:45:16.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.550+0000 7fe44847f740 20 client.25604 populate_metadata read hostname 'smithi005' 2024-04-24T06:45:16.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.550+0000 7fe44847f740 1 --2- 172.21.15.5:0/2322248943 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x5636bf8d15b0 0x5636bf8d39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.550+0000 7fe44847f740 4 client.25604 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:45:16.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.550+0000 7fe44847f740 1 -- 172.21.15.5:0/2322248943 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636bf820bf0 con 0x5636bf8d15b0 2024-04-24T06:45:16.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.551+0000 7fe43ea00640 1 --2- 172.21.15.5:0/2322248943 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x5636bf8d15b0 0x5636bf8d39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.551+0000 7fe43ea00640 1 --2- 172.21.15.5:0/2322248943 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x5636bf8d15b0 0x5636bf8d39a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ced23cd35de93446 server_cookie=3568cf65126d5c47 in_seq=0 out_seq=0 2024-04-24T06:45:16.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.551+0000 7fe41fe00640 10 client.25604 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:45:16.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.552+0000 7fe41fe00640 1 -- 172.21.15.5:0/2322248943 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5636bf820bf0 con 0x5636bf8d15b0 2024-04-24T06:45:16.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.552+0000 7fe41fe00640 10 client.25604 handle_command_reply: tid=0 2024-04-24T06:45:16.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.552+0000 7fe44847f740 1 client.25604 shutdown 2024-04-24T06:45:16.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.553+0000 7fe44847f740 1 -- 172.21.15.5:0/2322248943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636bf8ae460 msgr2=0x5636bf8b5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:16.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.553+0000 7fe44847f740 1 --2- 172.21.15.5:0/2322248943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636bf8ae460 0x5636bf8b5e60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe42c000c00 tx=0x7fe42c009490 comp rx=0 tx=0).stop 2024-04-24T06:45:16.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.553+0000 7fe44847f740 1 -- 172.21.15.5:0/2322248943 shutdown_connections 2024-04-24T06:45:16.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.553+0000 7fe44847f740 1 --2- 172.21.15.5:0/2322248943 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x5636bf8d15b0 0x5636bf8d39a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.553+0000 7fe44847f740 1 --2- 172.21.15.5:0/2322248943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636bf8b8b30 0x5636bf8b63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.553+0000 7fe44847f740 1 --2- 172.21.15.5:0/2322248943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636bf8ae460 0x5636bf8b5e60 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.553+0000 7fe44847f740 1 --2- 172.21.15.5:0/2322248943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636bf8adb40 0x5636bf8b5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.553+0000 7fe44847f740 1 -- 172.21.15.5:0/2322248943 >> 172.21.15.5:0/2322248943 conn(0x5636bf8a9e60 msgr2=0x5636bf8afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:16.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.553+0000 7fe41fe00640 0 client.25604 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:45:16.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.553+0000 7fe44847f740 1 -- 172.21.15.5:0/2322248943 shutdown_connections 2024-04-24T06:45:16.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.553+0000 7fe44847f740 1 -- 172.21.15.5:0/2322248943 wait complete. 2024-04-24T06:45:16.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.554+0000 7fe44847f740 20 client.25604 trim_cache size 0 max 16384 2024-04-24T06:45:16.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.568+0000 7fe44847f740 1 Processor -- start 2024-04-24T06:45:16.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.568+0000 7fe44847f740 1 -- start start 2024-04-24T06:45:16.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.569+0000 7fe44847f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636bf8adb40 0x5636bf8b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.569+0000 7fe44847f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636bf8ae460 0x5636bf8b4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.569+0000 7fe44847f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636bf8b8b30 0x5636bf8b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.569+0000 7fe44847f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe4401ae250 con 0x5636bf8ae460 2024-04-24T06:45:16.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.569+0000 7fe44847f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe4401abb20 con 0x5636bf8adb40 2024-04-24T06:45:16.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.569+0000 7fe44847f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe4401a99b0 con 0x5636bf8b8b30 2024-04-24T06:45:16.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.569+0000 7fe43ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636bf8adb40 0x5636bf8b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.569+0000 7fe43e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636bf8ae460 0x5636bf8b4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.569+0000 7fe43ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636bf8adb40 0x5636bf8b02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:47660/0 (socket says 172.21.15.5:47660) 2024-04-24T06:45:16.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.569+0000 7fe43ea00640 1 -- 172.21.15.5:0/2751741982 learned_addr learned my addr 172.21.15.5:0/2751741982 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:16.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.569+0000 7fe43ea00640 1 -- 172.21.15.5:0/2751741982 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636bf8b8b30 msgr2=0x5636bf8b61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:16.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.569+0000 7fe43ea00640 1 --2- 172.21.15.5:0/2751741982 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636bf8b8b30 0x5636bf8b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.569+0000 7fe43ea00640 1 -- 172.21.15.5:0/2751741982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636bf8ae460 msgr2=0x5636bf8b4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:16.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.569+0000 7fe43ea00640 1 --2- 172.21.15.5:0/2751741982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636bf8ae460 0x5636bf8b4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.569+0000 7fe43ea00640 1 -- 172.21.15.5:0/2751741982 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636bf5ef7d0 con 0x5636bf8adb40 2024-04-24T06:45:16.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.570+0000 7fe43ea00640 1 --2- 172.21.15.5:0/2751741982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636bf8adb40 0x5636bf8b02a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe43009c6e0 tx=0x7fe4300046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a46bf6cac681abe1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:16.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.570+0000 7fe42aa00640 1 -- 172.21.15.5:0/2751741982 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4300a0d90 con 0x5636bf8adb40 2024-04-24T06:45:16.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.570+0000 7fe42aa00640 1 -- 172.21.15.5:0/2751741982 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4300a5650 con 0x5636bf8adb40 2024-04-24T06:45:16.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.570+0000 7fe42aa00640 1 -- 172.21.15.5:0/2751741982 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4300a5420 con 0x5636bf8adb40 2024-04-24T06:45:16.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.571+0000 7fe44847f740 1 -- 172.21.15.5:0/2751741982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636bf8adb40 msgr2=0x5636bf8b02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:16.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.571+0000 7fe44847f740 1 --2- 172.21.15.5:0/2751741982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636bf8adb40 0x5636bf8b02a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe43009c6e0 tx=0x7fe4300046c0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.571+0000 7fe44847f740 1 -- 172.21.15.5:0/2751741982 shutdown_connections 2024-04-24T06:45:16.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.571+0000 7fe44847f740 1 --2- 172.21.15.5:0/2751741982 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636bf8b8b30 0x5636bf8b61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.571+0000 7fe44847f740 1 --2- 172.21.15.5:0/2751741982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636bf8ae460 0x5636bf8b4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.571+0000 7fe44847f740 1 --2- 172.21.15.5:0/2751741982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636bf8adb40 0x5636bf8b02a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.571+0000 7fe44847f740 1 -- 172.21.15.5:0/2751741982 >> 172.21.15.5:0/2751741982 conn(0x5636bf8a9e60 msgr2=0x5636bf8a86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:16.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.571+0000 7fe44847f740 1 -- 172.21.15.5:0/2751741982 shutdown_connections 2024-04-24T06:45:16.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.571+0000 7fe44847f740 1 -- 172.21.15.5:0/2751741982 wait complete. 2024-04-24T06:45:16.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.571+0000 7fe44847f740 1 -- start start 2024-04-24T06:45:16.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.572+0000 7fe44847f740 1 Processor -- start 2024-04-24T06:45:16.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.572+0000 7fe44847f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636bf8adb40 0x5636bf8b9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.572+0000 7fe44847f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636bf8ae460 0x5636bf8b9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.572+0000 7fe44847f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636bf8b8b30 0x5636bf8b9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.572+0000 7fe44847f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe4401aa000 con 0x5636bf8b8b30 2024-04-24T06:45:16.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.572+0000 7fe44847f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe4401ab450 con 0x5636bf8adb40 2024-04-24T06:45:16.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.572+0000 7fe44847f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe4401aa6b0 con 0x5636bf8ae460 2024-04-24T06:45:16.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.572+0000 7fe44847f740 10 client.0 _ll_register_callbacks cb 0x5636bf8b8a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:45:16.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.573+0000 7fe43ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636bf8adb40 0x5636bf8b9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.573+0000 7fe43ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636bf8adb40 0x5636bf8b9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:47674/0 (socket says 172.21.15.5:47674) 2024-04-24T06:45:16.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.573+0000 7fe43ea00640 1 -- 172.21.15.5:0/1229712480 learned_addr learned my addr 172.21.15.5:0/1229712480 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:16.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.573+0000 7fe43f400640 1 --2- 172.21.15.5:0/1229712480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636bf8b8b30 0x5636bf8b9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.573+0000 7fe43e000640 1 --2- 172.21.15.5:0/1229712480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636bf8ae460 0x5636bf8b9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.573+0000 7fe43ea00640 1 -- 172.21.15.5:0/1229712480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636bf8ae460 msgr2=0x5636bf8b9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:16.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.573+0000 7fe43ea00640 1 --2- 172.21.15.5:0/1229712480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636bf8ae460 0x5636bf8b9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.573+0000 7fe43ea00640 1 -- 172.21.15.5:0/1229712480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636bf8b8b30 msgr2=0x5636bf8b9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:16.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.573+0000 7fe43ea00640 1 --2- 172.21.15.5:0/1229712480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636bf8b8b30 0x5636bf8b9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.573+0000 7fe43ea00640 1 -- 172.21.15.5:0/1229712480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636bf8c88c0 con 0x5636bf8adb40 2024-04-24T06:45:16.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.573+0000 7fe43ea00640 1 --2- 172.21.15.5:0/1229712480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636bf8adb40 0x5636bf8b9110 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe43009c970 tx=0x7fe430004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a75c166480169f69 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:16.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.573+0000 7fe41fe00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:45:16.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.574+0000 7fe41fe00640 1 -- 172.21.15.5:0/1229712480 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe430099050 con 0x5636bf8adb40 2024-04-24T06:45:16.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.574+0000 7fe41fe00640 1 -- 172.21.15.5:0/1229712480 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fe4300a51c0 con 0x5636bf8adb40 2024-04-24T06:45:16.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.574+0000 7fe44847f740 1 -- 172.21.15.5:0/1229712480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4300a5650 con 0x5636bf8adb40 2024-04-24T06:45:16.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.574+0000 7fe41fe00640 1 -- 172.21.15.5:0/1229712480 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe4300991f0 con 0x5636bf8adb40 2024-04-24T06:45:16.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.574+0000 7fe44847f740 10 client.25610 fetch_fsmap learned FSMap version 23 2024-04-24T06:45:16.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.574+0000 7fe44847f740 1 -- 172.21.15.5:0/1229712480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5636bf8c9bc0 con 0x5636bf8adb40 2024-04-24T06:45:16.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.574+0000 7fe41fe00640 1 -- 172.21.15.5:0/1229712480 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4300a5650 con 0x5636bf8adb40 2024-04-24T06:45:16.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.574+0000 7fe41fe00640 1 -- 172.21.15.5:0/1229712480 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fe4300871d0 con 0x5636bf8adb40 2024-04-24T06:45:16.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.575+0000 7fe44847f740 10 client.25610 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T06:45:16.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.575+0000 7fe44847f740 10 client.25610 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T06:45:16.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.575+0000 7fe44847f740 20 client.25610 populate_metadata read hostname 'smithi005' 2024-04-24T06:45:16.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.575+0000 7fe44847f740 1 --2- 172.21.15.5:0/1229712480 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x5636bf937d40 0x5636bf93a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.575+0000 7fe44847f740 4 client.25610 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:45:16.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.575+0000 7fe44847f740 1 -- 172.21.15.5:0/1229712480 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5636bf81aa60 con 0x5636bf937d40 2024-04-24T06:45:16.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.575+0000 7fe43e000640 1 --2- 172.21.15.5:0/1229712480 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x5636bf937d40 0x5636bf93a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.575+0000 7fe43e000640 1 --2- 172.21.15.5:0/1229712480 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x5636bf937d40 0x5636bf93a160 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f208f8b48de00c14 server_cookie=5b7594f79b395c1a in_seq=0 out_seq=0 2024-04-24T06:45:16.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.575+0000 7fe41fe00640 10 client.25610 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:45:16.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.576+0000 7fe41fe00640 1 -- 172.21.15.5:0/1229712480 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5636bf81aa60 con 0x5636bf937d40 2024-04-24T06:45:16.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.576+0000 7fe41fe00640 10 client.25610 handle_command_reply: tid=0 2024-04-24T06:45:16.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.576+0000 7fe44847f740 1 client.25610 shutdown 2024-04-24T06:45:16.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.576+0000 7fe44847f740 1 -- 172.21.15.5:0/1229712480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636bf8adb40 msgr2=0x5636bf8b9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:16.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.576+0000 7fe44847f740 1 --2- 172.21.15.5:0/1229712480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636bf8adb40 0x5636bf8b9110 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe43009c970 tx=0x7fe430004730 comp rx=0 tx=0).stop 2024-04-24T06:45:16.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.576+0000 7fe44847f740 1 -- 172.21.15.5:0/1229712480 shutdown_connections 2024-04-24T06:45:16.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.576+0000 7fe44847f740 1 --2- 172.21.15.5:0/1229712480 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x5636bf937d40 0x5636bf93a160 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.576+0000 7fe44847f740 1 --2- 172.21.15.5:0/1229712480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636bf8b8b30 0x5636bf8b9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.576+0000 7fe41fe00640 0 client.25610 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:45:16.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.576+0000 7fe44847f740 1 --2- 172.21.15.5:0/1229712480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636bf8ae460 0x5636bf8b9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.576+0000 7fe44847f740 1 --2- 172.21.15.5:0/1229712480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636bf8adb40 0x5636bf8b9110 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.576+0000 7fe44847f740 1 -- 172.21.15.5:0/1229712480 >> 172.21.15.5:0/1229712480 conn(0x5636bf8a9e60 msgr2=0x5636bf8b4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:16.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.576+0000 7fe44847f740 1 -- 172.21.15.5:0/1229712480 shutdown_connections 2024-04-24T06:45:16.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.576+0000 7fe44847f740 1 -- 172.21.15.5:0/1229712480 wait complete. 2024-04-24T06:45:16.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.576+0000 7fe44847f740 20 client.25610 trim_cache size 0 max 16384 2024-04-24T06:45:16.582 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:45:16.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.577+0000 7fe43fe00640 1 -- 172.21.15.5:0/2370360872 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fe4240a3ef0 msgr2=0x7fe4240a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:16.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.577+0000 7fe43fe00640 1 --2- 172.21.15.5:0/2370360872 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fe4240a3ef0 0x7fe4240a63b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe438006010 tx=0x7fe438000f40 comp rx=0 tx=0).stop 2024-04-24T06:45:16.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.577+0000 7fe43fe00640 1 -- 172.21.15.5:0/2370360872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe440104280 msgr2=0x7fe4401a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:16.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.577+0000 7fe43fe00640 1 --2- 172.21.15.5:0/2370360872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe440104280 0x7fe4401a0690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe43000e9c0 tx=0x7fe43000edc0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.577+0000 7fe43fe00640 1 -- 172.21.15.5:0/2370360872 shutdown_connections 2024-04-24T06:45:16.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.577+0000 7fe43fe00640 1 --2- 172.21.15.5:0/2370360872 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fe4240a3ef0 0x7fe4240a63b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.577+0000 7fe43fe00640 1 --2- 172.21.15.5:0/2370360872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe44019a800 0x7fe44019acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.577+0000 7fe43fe00640 1 --2- 172.21.15.5:0/2370360872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe440109350 0x7fe4401a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.577+0000 7fe43fe00640 1 --2- 172.21.15.5:0/2370360872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe440104280 0x7fe4401a0690 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.577+0000 7fe43fe00640 1 -- 172.21.15.5:0/2370360872 >> 172.21.15.5:0/2370360872 conn(0x7fe4400ffae0 msgr2=0x7fe440100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:16.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.577+0000 7fe43fe00640 1 -- 172.21.15.5:0/2370360872 shutdown_connections 2024-04-24T06:45:16.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.578+0000 7fe43fe00640 1 -- 172.21.15.5:0/2370360872 wait complete. 2024-04-24T06:45:16.602 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T06:45:16.663 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:16 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/2920533384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:45:16.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.817+0000 7f56f5e00640 1 -- 172.21.15.5:0/2905440073 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56f0102070 msgr2=0x7f56f0102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:16.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.817+0000 7f56f5e00640 1 --2- 172.21.15.5:0/2905440073 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56f0102070 0x7f56f0102450 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f56e4009900 tx=0x7f56e402a3f0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.818+0000 7f56f5e00640 1 -- 172.21.15.5:0/2905440073 shutdown_connections 2024-04-24T06:45:16.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.818+0000 7f56f5e00640 1 --2- 172.21.15.5:0/2905440073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56f0107140 0x7f56f010ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.818+0000 7f56f5e00640 1 --2- 172.21.15.5:0/2905440073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56f0102a20 0x7f56f0106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.818+0000 7f56f5e00640 1 --2- 172.21.15.5:0/2905440073 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56f0102070 0x7f56f0102450 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.818+0000 7f56f5e00640 1 -- 172.21.15.5:0/2905440073 >> 172.21.15.5:0/2905440073 conn(0x7f56f00fd950 msgr2=0x7f56f00ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:16.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.818+0000 7f56f5e00640 1 -- 172.21.15.5:0/2905440073 shutdown_connections 2024-04-24T06:45:16.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.818+0000 7f56f5e00640 1 -- 172.21.15.5:0/2905440073 wait complete. 2024-04-24T06:45:16.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.819+0000 7f56f5e00640 1 Processor -- start 2024-04-24T06:45:16.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.819+0000 7f56f5e00640 1 -- start start 2024-04-24T06:45:16.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.820+0000 7f56f5e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56f0102070 0x7f56f0198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.820+0000 7f56f5e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56f0107140 0x7f56f0198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.820+0000 7f56f5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56f019d010 0x7f56f019d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.820+0000 7f56f5e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f56f01121f0 con 0x7f56f019d010 2024-04-24T06:45:16.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.820+0000 7f56f5e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f56f0112070 con 0x7f56f0102070 2024-04-24T06:45:16.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.820+0000 7f56f5e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f56f0112370 con 0x7f56f0107140 2024-04-24T06:45:16.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.820+0000 7f56f4a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56f0102070 0x7f56f0198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.820+0000 7f56f5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56f019d010 0x7f56f019d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.820+0000 7f56f5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56f019d010 0x7f56f019d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40372/0 (socket says 172.21.15.5:40372) 2024-04-24T06:45:16.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.820+0000 7f56f5400640 1 -- 172.21.15.5:0/404346712 learned_addr learned my addr 172.21.15.5:0/404346712 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:16.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.820+0000 7f56efe00640 1 --2- 172.21.15.5:0/404346712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56f0107140 0x7f56f0198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.821+0000 7f56f5400640 1 -- 172.21.15.5:0/404346712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56f0107140 msgr2=0x7f56f0198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:16.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.821+0000 7f56f5400640 1 --2- 172.21.15.5:0/404346712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56f0107140 0x7f56f0198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.821+0000 7f56f5400640 1 -- 172.21.15.5:0/404346712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56f0102070 msgr2=0x7f56f0198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:16.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.821+0000 7f56f5400640 1 --2- 172.21.15.5:0/404346712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56f0102070 0x7f56f0198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.821+0000 7f56f5400640 1 -- 172.21.15.5:0/404346712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f019db00 con 0x7f56f019d010 2024-04-24T06:45:16.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.821+0000 7f56efe00640 1 --2- 172.21.15.5:0/404346712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56f0107140 0x7f56f0198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:45:16.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.821+0000 7f56f4a00640 1 --2- 172.21.15.5:0/404346712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56f0102070 0x7f56f0198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:45:16.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.821+0000 7f56f5400640 1 --2- 172.21.15.5:0/404346712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56f019d010 0x7f56f019d4c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f56e800eb10 tx=0x7f56e800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:16.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.822+0000 7f56ed600640 1 -- 172.21.15.5:0/404346712 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e8004b20 con 0x7f56f019d010 2024-04-24T06:45:16.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.822+0000 7f56ed600640 1 -- 172.21.15.5:0/404346712 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56e8004cc0 con 0x7f56f019d010 2024-04-24T06:45:16.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.822+0000 7f56f5e00640 1 -- 172.21.15.5:0/404346712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56f019ddf0 con 0x7f56f019d010 2024-04-24T06:45:16.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.822+0000 7f56ed600640 1 -- 172.21.15.5:0/404346712 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e8010760 con 0x7f56f019d010 2024-04-24T06:45:16.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.822+0000 7f56f5e00640 1 -- 172.21.15.5:0/404346712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56f0074410 con 0x7f56f019d010 2024-04-24T06:45:16.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.826+0000 7f56fa1f2740 1 Processor -- start 2024-04-24T06:45:16.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.827+0000 7f56ed600640 1 -- 172.21.15.5:0/404346712 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138929+0+0 (secure 0 0 0) 0x7f56e8005000 con 0x7f56f019d010 2024-04-24T06:45:16.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.827+0000 7f56fa1f2740 1 -- start start 2024-04-24T06:45:16.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.827+0000 7f56fa1f2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb414bbb40 0x55cb414bbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.827+0000 7f56fa1f2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb414bc460 0x55cb414c65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.828+0000 7f56fa1f2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb414c6b30 0x55cb414c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.828+0000 7f56fa1f2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cb414211d0 con 0x55cb414bc460 2024-04-24T06:45:16.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.828+0000 7f56fa1f2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55cb413c7450 con 0x55cb414bbb40 2024-04-24T06:45:16.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.828+0000 7f56fa1f2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55cb4141d190 con 0x55cb414c6b30 2024-04-24T06:45:16.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.828+0000 7f56efe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb414bbb40 0x55cb414bbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.828+0000 7f56efe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb414bbb40 0x55cb414bbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:47714/0 (socket says 172.21.15.5:47714) 2024-04-24T06:45:16.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.828+0000 7f56efe00640 1 -- 172.21.15.5:0/1532842128 learned_addr learned my addr 172.21.15.5:0/1532842128 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:16.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.828+0000 7f56efe00640 1 --2- 172.21.15.5:0/1532842128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb414c6b30 0x55cb414c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.828+0000 7f56f4a00640 1 --2- 172.21.15.5:0/1532842128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb414bc460 0x55cb414c65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.829+0000 7f56efe00640 1 -- 172.21.15.5:0/1532842128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb414c6b30 msgr2=0x55cb414c8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:16.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.829+0000 7f56efe00640 1 --2- 172.21.15.5:0/1532842128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb414c6b30 0x55cb414c8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.829+0000 7f56efe00640 1 -- 172.21.15.5:0/1532842128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb414bc460 msgr2=0x55cb414c65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:16.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.829+0000 7f56efe00640 1 --2- 172.21.15.5:0/1532842128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb414bc460 0x55cb414c65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.829+0000 7f56efe00640 1 -- 172.21.15.5:0/1532842128 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb41484310 con 0x55cb414bbb40 2024-04-24T06:45:16.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.829+0000 7f56efe00640 1 --2- 172.21.15.5:0/1532842128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb414bbb40 0x55cb414bbf20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f56dc0127f0 tx=0x7f56dc009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cb75d38a1b789c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:16.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.829+0000 7f56e1c00640 1 -- 172.21.15.5:0/1532842128 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56dc012e30 con 0x55cb414bbb40 2024-04-24T06:45:16.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.830+0000 7f56e1c00640 1 -- 172.21.15.5:0/1532842128 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56dc010cb0 con 0x55cb414bbb40 2024-04-24T06:45:16.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.830+0000 7f56e1c00640 1 -- 172.21.15.5:0/1532842128 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56dc01ec20 con 0x55cb414bbb40 2024-04-24T06:45:16.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.830+0000 7f56ed600640 1 --2- 172.21.15.5:0/404346712 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f56d80a82e0 0x7f56d80aa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.830+0000 7f56f4a00640 1 --2- 172.21.15.5:0/404346712 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f56d80a82e0 0x7f56d80aa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.830+0000 7f56ed600640 1 -- 172.21.15.5:0/404346712 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f56e80d10f0 con 0x7f56f019d010 2024-04-24T06:45:16.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.830+0000 7f56fa1f2740 1 -- 172.21.15.5:0/1532842128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb414bbb40 msgr2=0x55cb414bbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:16.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.830+0000 7f56fa1f2740 1 --2- 172.21.15.5:0/1532842128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb414bbb40 0x55cb414bbf20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f56dc0127f0 tx=0x7f56dc009700 comp rx=0 tx=0).stop 2024-04-24T06:45:16.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.831+0000 7f56fa1f2740 1 -- 172.21.15.5:0/1532842128 shutdown_connections 2024-04-24T06:45:16.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.831+0000 7f56fa1f2740 1 --2- 172.21.15.5:0/1532842128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb414c6b30 0x55cb414c8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.831+0000 7f56fa1f2740 1 --2- 172.21.15.5:0/1532842128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb414bc460 0x55cb414c65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.831+0000 7f56fa1f2740 1 --2- 172.21.15.5:0/1532842128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb414bbb40 0x55cb414bbf20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.831+0000 7f56fa1f2740 1 -- 172.21.15.5:0/1532842128 >> 172.21.15.5:0/1532842128 conn(0x55cb414b7e60 msgr2=0x55cb414b8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:16.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.831+0000 7f56fa1f2740 1 -- 172.21.15.5:0/1532842128 shutdown_connections 2024-04-24T06:45:16.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.831+0000 7f56fa1f2740 1 -- 172.21.15.5:0/1532842128 wait complete. 2024-04-24T06:45:16.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.831+0000 7f56f4a00640 1 --2- 172.21.15.5:0/404346712 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f56d80a82e0 0x7f56d80aa7a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f56e4009230 tx=0x7f56e40058d0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:16.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.831+0000 7f56fa1f2740 1 -- start start 2024-04-24T06:45:16.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.832+0000 7f56fa1f2740 1 Processor -- start 2024-04-24T06:45:16.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.832+0000 7f56fa1f2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb414bbb40 0x55cb414c65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.832+0000 7f56fa1f2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb414bc460 0x55cb414c38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.832+0000 7f56fa1f2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb414c6b30 0x55cb414c3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.832+0000 7f56fa1f2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cb414c99c0 con 0x55cb414bc460 2024-04-24T06:45:16.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.832+0000 7f56fa1f2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55cb413da810 con 0x55cb414c6b30 2024-04-24T06:45:16.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.832+0000 7f56fa1f2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55cb414c9b40 con 0x55cb414bbb40 2024-04-24T06:45:16.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.832+0000 7f56fa1f2740 10 client.0 _ll_register_callbacks cb 0x55cb413a8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:45:16.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.833+0000 7f56f4a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb414bbb40 0x55cb414c65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.833+0000 7f56efe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb414bc460 0x55cb414c38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.833+0000 7f56f4a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb414bbb40 0x55cb414c65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:58902/0 (socket says 172.21.15.5:58902) 2024-04-24T06:45:16.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.833+0000 7f56f4a00640 1 -- 172.21.15.5:0/3195503718 learned_addr learned my addr 172.21.15.5:0/3195503718 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:16.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.833+0000 7f56f5400640 1 --2- 172.21.15.5:0/3195503718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb414c6b30 0x55cb414c3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.833+0000 7f56f4a00640 1 -- 172.21.15.5:0/3195503718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb414c6b30 msgr2=0x55cb414c3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:16.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.833+0000 7f56f4a00640 1 --2- 172.21.15.5:0/3195503718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb414c6b30 0x55cb414c3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.833+0000 7f56f4a00640 1 -- 172.21.15.5:0/3195503718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb414bc460 msgr2=0x55cb414c38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:16.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.833+0000 7f56f4a00640 1 --2- 172.21.15.5:0/3195503718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb414bc460 0x55cb414c38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.833+0000 7f56f4a00640 1 -- 172.21.15.5:0/3195503718 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb413eb3f0 con 0x55cb414bbb40 2024-04-24T06:45:16.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.833+0000 7f56efe00640 1 --2- 172.21.15.5:0/3195503718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb414bc460 0x55cb414c38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:45:16.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.834+0000 7f56f4a00640 1 --2- 172.21.15.5:0/3195503718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb414bbb40 0x55cb414c65b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f56e4009900 tx=0x7f56e4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d87c9d0a58653316 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:16.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.834+0000 7f56d6000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:45:16.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.834+0000 7f56d6000640 1 -- 172.21.15.5:0/3195503718 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e404d070 con 0x55cb414bbb40 2024-04-24T06:45:16.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.834+0000 7f56fa1f2740 1 -- 172.21.15.5:0/3195503718 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56dc010cb0 con 0x55cb414bbb40 2024-04-24T06:45:16.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.834+0000 7f56d6000640 1 -- 172.21.15.5:0/3195503718 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f56e4003ac0 con 0x55cb414bbb40 2024-04-24T06:45:16.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.834+0000 7f56d6000640 1 -- 172.21.15.5:0/3195503718 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f56e4043070 con 0x55cb414bbb40 2024-04-24T06:45:16.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.834+0000 7f56fa1f2740 10 client.25645 fetch_fsmap learned FSMap version 23 2024-04-24T06:45:16.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.834+0000 7f56fa1f2740 1 -- 172.21.15.5:0/3195503718 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55cb411333f0 con 0x55cb414bbb40 2024-04-24T06:45:16.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.835+0000 7f56d6000640 1 -- 172.21.15.5:0/3195503718 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56dc010cb0 con 0x55cb414bbb40 2024-04-24T06:45:16.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.835+0000 7f56d6000640 1 -- 172.21.15.5:0/3195503718 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f56e4049420 con 0x55cb414bbb40 2024-04-24T06:45:16.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.835+0000 7f56fa1f2740 10 client.25645 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T06:45:16.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.835+0000 7f56fa1f2740 10 client.25645 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:45:16.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.835+0000 7f56fa1f2740 20 client.25645 populate_metadata read hostname 'smithi005' 2024-04-24T06:45:16.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.835+0000 7f56fa1f2740 1 --2- 172.21.15.5:0/3195503718 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55cb414d6ce0 0x55cb414d9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.835+0000 7f56fa1f2740 4 client.25645 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:45:16.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.835+0000 7f56fa1f2740 1 -- 172.21.15.5:0/3195503718 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb4142ebf0 con 0x55cb414d6ce0 2024-04-24T06:45:16.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.835+0000 7f56efe00640 1 --2- 172.21.15.5:0/3195503718 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55cb414d6ce0 0x55cb414d9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.836+0000 7f56efe00640 1 --2- 172.21.15.5:0/3195503718 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55cb414d6ce0 0x55cb414d9180 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c461dd18d870cc82 server_cookie=42fb4ff3f69043d6 in_seq=0 out_seq=0 2024-04-24T06:45:16.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.836+0000 7f56d6000640 10 client.25645 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:45:16.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.837+0000 7f56d6000640 1 -- 172.21.15.5:0/3195503718 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cb4142ebf0 con 0x55cb414d6ce0 2024-04-24T06:45:16.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.837+0000 7f56d6000640 10 client.25645 handle_command_reply: tid=0 2024-04-24T06:45:16.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.837+0000 7f56fa1f2740 1 client.25645 shutdown 2024-04-24T06:45:16.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.837+0000 7f56fa1f2740 1 -- 172.21.15.5:0/3195503718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb414bbb40 msgr2=0x55cb414c65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:16.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.837+0000 7f56fa1f2740 1 --2- 172.21.15.5:0/3195503718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb414bbb40 0x55cb414c65b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f56e4009900 tx=0x7f56e4009260 comp rx=0 tx=0).stop 2024-04-24T06:45:16.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.838+0000 7f56fa1f2740 1 -- 172.21.15.5:0/3195503718 shutdown_connections 2024-04-24T06:45:16.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.838+0000 7f56fa1f2740 1 --2- 172.21.15.5:0/3195503718 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55cb414d6ce0 0x55cb414d9180 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.838+0000 7f56fa1f2740 1 --2- 172.21.15.5:0/3195503718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb414c6b30 0x55cb414c3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.838+0000 7f56fa1f2740 1 --2- 172.21.15.5:0/3195503718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb414bc460 0x55cb414c38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.838+0000 7f56d6000640 0 client.25645 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:45:16.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.838+0000 7f56fa1f2740 1 --2- 172.21.15.5:0/3195503718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb414bbb40 0x55cb414c65b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.838+0000 7f56fa1f2740 1 -- 172.21.15.5:0/3195503718 >> 172.21.15.5:0/3195503718 conn(0x55cb414b7e60 msgr2=0x55cb414bdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:16.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.838+0000 7f56fa1f2740 1 -- 172.21.15.5:0/3195503718 shutdown_connections 2024-04-24T06:45:16.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.838+0000 7f56fa1f2740 1 -- 172.21.15.5:0/3195503718 wait complete. 2024-04-24T06:45:16.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.838+0000 7f56fa1f2740 20 client.25645 trim_cache size 0 max 16384 2024-04-24T06:45:16.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.867+0000 7f56fa1f2740 1 Processor -- start 2024-04-24T06:45:16.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.867+0000 7f56fa1f2740 1 -- start start 2024-04-24T06:45:16.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.867+0000 7f56fa1f2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb414bbb40 0x55cb414d2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.867+0000 7f56fa1f2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb414bc460 0x55cb414c2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.867+0000 7f56fa1f2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb414c6b30 0x55cb414c60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.867+0000 7f56fa1f2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f56f01ae1f0 con 0x55cb414bbb40 2024-04-24T06:45:16.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.867+0000 7f56fa1f2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f56f01abac0 con 0x55cb414c6b30 2024-04-24T06:45:16.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.867+0000 7f56fa1f2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f56f01a9910 con 0x55cb414bc460 2024-04-24T06:45:16.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.868+0000 7f56f4a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb414bbb40 0x55cb414d2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.868+0000 7f56f4a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb414bbb40 0x55cb414d2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40388/0 (socket says 172.21.15.5:40388) 2024-04-24T06:45:16.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.868+0000 7f56f5400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb414c6b30 0x55cb414c60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.868+0000 7f56f4a00640 1 -- 172.21.15.5:0/1796030960 learned_addr learned my addr 172.21.15.5:0/1796030960 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:16.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.868+0000 7f56efe00640 1 --2- 172.21.15.5:0/1796030960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb414bc460 0x55cb414c2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.868+0000 7f56f5400640 1 -- 172.21.15.5:0/1796030960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb414bc460 msgr2=0x55cb414c2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:16.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.868+0000 7f56f5400640 1 --2- 172.21.15.5:0/1796030960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb414bc460 0x55cb414c2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.868+0000 7f56f5400640 1 -- 172.21.15.5:0/1796030960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb414bbb40 msgr2=0x55cb414d2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:16.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.868+0000 7f56f5400640 1 --2- 172.21.15.5:0/1796030960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb414bbb40 0x55cb414d2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.868+0000 7f56f5400640 1 -- 172.21.15.5:0/1796030960 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb411fd7d0 con 0x55cb414c6b30 2024-04-24T06:45:16.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.868+0000 7f56f4a00640 1 --2- 172.21.15.5:0/1796030960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb414bbb40 0x55cb414d2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:45:16.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.869+0000 7f56f5400640 1 --2- 172.21.15.5:0/1796030960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb414c6b30 0x55cb414c60e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f56e80ce610 tx=0x7f56e809ccf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=341d58d894f330d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:16.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.869+0000 7f56e1200640 1 -- 172.21.15.5:0/1796030960 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e80b5070 con 0x55cb414c6b30 2024-04-24T06:45:16.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.869+0000 7f56e1200640 1 -- 172.21.15.5:0/1796030960 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56e809ce60 con 0x55cb414c6b30 2024-04-24T06:45:16.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.869+0000 7f56e1200640 1 -- 172.21.15.5:0/1796030960 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e80a5640 con 0x55cb414c6b30 2024-04-24T06:45:16.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.870+0000 7f56fa1f2740 1 -- 172.21.15.5:0/1796030960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb414c6b30 msgr2=0x55cb414c60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:16.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.870+0000 7f56fa1f2740 1 --2- 172.21.15.5:0/1796030960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb414c6b30 0x55cb414c60e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f56e80ce610 tx=0x7f56e809ccf0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.870+0000 7f56fa1f2740 1 -- 172.21.15.5:0/1796030960 shutdown_connections 2024-04-24T06:45:16.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.870+0000 7f56fa1f2740 1 --2- 172.21.15.5:0/1796030960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb414c6b30 0x55cb414c60e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.870+0000 7f56fa1f2740 1 --2- 172.21.15.5:0/1796030960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb414bc460 0x55cb414c2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.870+0000 7f56fa1f2740 1 --2- 172.21.15.5:0/1796030960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb414bbb40 0x55cb414d2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.870+0000 7f56fa1f2740 1 -- 172.21.15.5:0/1796030960 >> 172.21.15.5:0/1796030960 conn(0x55cb414b7e60 msgr2=0x55cb414c2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:16.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.870+0000 7f56fa1f2740 1 -- 172.21.15.5:0/1796030960 shutdown_connections 2024-04-24T06:45:16.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.870+0000 7f56fa1f2740 1 -- 172.21.15.5:0/1796030960 wait complete. 2024-04-24T06:45:16.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.871+0000 7f56fa1f2740 1 -- start start 2024-04-24T06:45:16.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.871+0000 7f56fa1f2740 1 Processor -- start 2024-04-24T06:45:16.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.871+0000 7f56fa1f2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb414bbb40 0x55cb414c7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.871+0000 7f56fa1f2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb414bc460 0x55cb414c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.871+0000 7f56fa1f2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb414c6b30 0x55cb414cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.872+0000 7f56fa1f2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f56f01a9f80 con 0x55cb414c6b30 2024-04-24T06:45:16.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.872+0000 7f56fa1f2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f56f01ab3f0 con 0x55cb414bc460 2024-04-24T06:45:16.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.872+0000 7f56fa1f2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f56f01aa650 con 0x55cb414bbb40 2024-04-24T06:45:16.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.872+0000 7f56fa1f2740 10 client.0 _ll_register_callbacks cb 0x7f56e4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:45:16.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.872+0000 7f56f5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb414c6b30 0x55cb414cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.872+0000 7f56f4a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb414bbb40 0x55cb414c7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.872+0000 7f56f5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb414c6b30 0x55cb414cdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40396/0 (socket says 172.21.15.5:40396) 2024-04-24T06:45:16.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.872+0000 7f56f4a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb414bbb40 0x55cb414c7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:58918/0 (socket says 172.21.15.5:58918) 2024-04-24T06:45:16.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.872+0000 7f56f5400640 1 -- 172.21.15.5:0/2010557669 learned_addr learned my addr 172.21.15.5:0/2010557669 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:16.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.872+0000 7f56efe00640 1 --2- 172.21.15.5:0/2010557669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb414bc460 0x55cb414c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.872+0000 7f56f4a00640 1 -- 172.21.15.5:0/2010557669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb414bc460 msgr2=0x55cb414c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:16.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.872+0000 7f56f4a00640 1 --2- 172.21.15.5:0/2010557669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb414bc460 0x55cb414c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.872+0000 7f56f4a00640 1 -- 172.21.15.5:0/2010557669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb414c6b30 msgr2=0x55cb414cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:16.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.872+0000 7f56f4a00640 1 --2- 172.21.15.5:0/2010557669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb414c6b30 0x55cb414cdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.872+0000 7f56f4a00640 1 -- 172.21.15.5:0/2010557669 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb414ce2d0 con 0x55cb414bbb40 2024-04-24T06:45:16.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.872+0000 7f56efe00640 1 --2- 172.21.15.5:0/2010557669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb414bc460 0x55cb414c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:45:16.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.873+0000 7f56f5400640 1 --2- 172.21.15.5:0/2010557669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb414c6b30 0x55cb414cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:45:16.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.873+0000 7f56f4a00640 1 --2- 172.21.15.5:0/2010557669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb414bbb40 0x55cb414c7110 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f56e4005780 tx=0x7f56e4004410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d11960cdede2cc85 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:16.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.873+0000 7f56d6000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:45:16.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.873+0000 7f56d6000640 1 -- 172.21.15.5:0/2010557669 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e405f040 con 0x55cb414bbb40 2024-04-24T06:45:16.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.873+0000 7f56fa1f2740 1 -- 172.21.15.5:0/2010557669 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56e809ce60 con 0x55cb414bbb40 2024-04-24T06:45:16.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.873+0000 7f56d6000640 1 -- 172.21.15.5:0/2010557669 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f56e4005030 con 0x55cb414bbb40 2024-04-24T06:45:16.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.873+0000 7f56d6000640 1 -- 172.21.15.5:0/2010557669 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f56e4072050 con 0x55cb414bbb40 2024-04-24T06:45:16.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.873+0000 7f56fa1f2740 10 client.25651 fetch_fsmap learned FSMap version 23 2024-04-24T06:45:16.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.873+0000 7f56fa1f2740 1 -- 172.21.15.5:0/2010557669 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55cb414cf5d0 con 0x55cb414bbb40 2024-04-24T06:45:16.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.874+0000 7f56d6000640 1 -- 172.21.15.5:0/2010557669 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56e809ce60 con 0x55cb414bbb40 2024-04-24T06:45:16.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.874+0000 7f56d6000640 1 -- 172.21.15.5:0/2010557669 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f56e4004750 con 0x55cb414bbb40 2024-04-24T06:45:16.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.874+0000 7f56fa1f2740 10 client.25651 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T06:45:16.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.874+0000 7f56fa1f2740 10 client.25651 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:45:16.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.874+0000 7f56fa1f2740 20 client.25651 populate_metadata read hostname 'smithi005' 2024-04-24T06:45:16.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.874+0000 7f56fa1f2740 1 --2- 172.21.15.5:0/2010557669 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55cb4153d560 0x55cb4153f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:16.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.874+0000 7f56fa1f2740 4 client.25651 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:45:16.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.874+0000 7f56fa1f2740 1 -- 172.21.15.5:0/2010557669 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cb41428a60 con 0x55cb4153d560 2024-04-24T06:45:16.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.874+0000 7f56efe00640 1 --2- 172.21.15.5:0/2010557669 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55cb4153d560 0x55cb4153f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:16.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.875+0000 7f56efe00640 1 --2- 172.21.15.5:0/2010557669 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55cb4153d560 0x55cb4153f950 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e9ca414a13e9e89 server_cookie=f38cfb9b8b851968 in_seq=0 out_seq=0 2024-04-24T06:45:16.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.875+0000 7f56d6000640 10 client.25651 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:45:16.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.875+0000 7f56d6000640 1 -- 172.21.15.5:0/2010557669 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cb41428a60 con 0x55cb4153d560 2024-04-24T06:45:16.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.875+0000 7f56d6000640 10 client.25651 handle_command_reply: tid=0 2024-04-24T06:45:16.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.875+0000 7f56fa1f2740 1 client.25651 shutdown 2024-04-24T06:45:16.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.876+0000 7f56fa1f2740 1 -- 172.21.15.5:0/2010557669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb414bbb40 msgr2=0x55cb414c7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:16.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.876+0000 7f56fa1f2740 1 --2- 172.21.15.5:0/2010557669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb414bbb40 0x55cb414c7110 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f56e4005780 tx=0x7f56e4004410 comp rx=0 tx=0).stop 2024-04-24T06:45:16.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.876+0000 7f56fa1f2740 1 -- 172.21.15.5:0/2010557669 shutdown_connections 2024-04-24T06:45:16.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.876+0000 7f56fa1f2740 1 --2- 172.21.15.5:0/2010557669 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55cb4153d560 0x55cb4153f950 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.876+0000 7f56fa1f2740 1 --2- 172.21.15.5:0/2010557669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb414c6b30 0x55cb414cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.876+0000 7f56fa1f2740 1 --2- 172.21.15.5:0/2010557669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb414bc460 0x55cb414c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.876+0000 7f56fa1f2740 1 --2- 172.21.15.5:0/2010557669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb414bbb40 0x55cb414c7110 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.876+0000 7f56fa1f2740 1 -- 172.21.15.5:0/2010557669 >> 172.21.15.5:0/2010557669 conn(0x55cb414b7e60 msgr2=0x55cb414b6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:16.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.876+0000 7f56d6000640 0 client.25651 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:45:16.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.876+0000 7f56fa1f2740 1 -- 172.21.15.5:0/2010557669 shutdown_connections 2024-04-24T06:45:16.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.876+0000 7f56fa1f2740 1 -- 172.21.15.5:0/2010557669 wait complete. 2024-04-24T06:45:16.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.876+0000 7f56fa1f2740 20 client.25651 trim_cache size 0 max 16384 2024-04-24T06:45:16.882 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:45:16.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.877+0000 7f56f5e00640 1 -- 172.21.15.5:0/404346712 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f56d80a82e0 msgr2=0x7f56d80aa7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:16.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.877+0000 7f56f5e00640 1 --2- 172.21.15.5:0/404346712 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f56d80a82e0 0x7f56d80aa7a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f56e4009230 tx=0x7f56e40058d0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.877+0000 7f56f5e00640 1 -- 172.21.15.5:0/404346712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56f019d010 msgr2=0x7f56f019d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:16.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.877+0000 7f56f5e00640 1 --2- 172.21.15.5:0/404346712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56f019d010 0x7f56f019d4c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f56e800eb10 tx=0x7f56e800c2d0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.877+0000 7f56f5e00640 1 -- 172.21.15.5:0/404346712 shutdown_connections 2024-04-24T06:45:16.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.877+0000 7f56f5e00640 1 --2- 172.21.15.5:0/404346712 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f56d80a82e0 0x7f56d80aa7a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.877+0000 7f56f5e00640 1 --2- 172.21.15.5:0/404346712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56f019d010 0x7f56f019d4c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.877+0000 7f56f5e00640 1 --2- 172.21.15.5:0/404346712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56f0107140 0x7f56f0198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.877+0000 7f56f5e00640 1 --2- 172.21.15.5:0/404346712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56f0102070 0x7f56f0198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:16.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.877+0000 7f56f5e00640 1 -- 172.21.15.5:0/404346712 >> 172.21.15.5:0/404346712 conn(0x7f56f00fd950 msgr2=0x7f56f00feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:16.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.878+0000 7f56f5e00640 1 -- 172.21.15.5:0/404346712 shutdown_connections 2024-04-24T06:45:16.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:16.878+0000 7f56f5e00640 1 -- 172.21.15.5:0/404346712 wait complete. 2024-04-24T06:45:16.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:16 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/2920533384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:45:17.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:16 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/2920533384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:45:17.898 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T06:45:17.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:45:17.899 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T06:45:17.945 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:17 smithi137 ceph-mon[33017]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-24T06:45:17.967 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:17 smithi005 ceph-mon[27798]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-24T06:45:18.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:17 smithi037 ceph-mon[32864]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-24T06:45:18.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.110+0000 7fc319a00640 1 -- 172.21.15.5:0/2516093080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc31410e6c0 msgr2=0x7fc314110ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:18.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.110+0000 7fc319a00640 1 --2- 172.21.15.5:0/2516093080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc31410e6c0 0x7fc314110ab0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc30c009ca0 tx=0x7fc30c02a950 comp rx=0 tx=0).stop 2024-04-24T06:45:18.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.110+0000 7fc319a00640 1 -- 172.21.15.5:0/2516093080 shutdown_connections 2024-04-24T06:45:18.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.110+0000 7fc319a00640 1 --2- 172.21.15.5:0/2516093080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc31410e6c0 0x7fc314110ab0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.110+0000 7fc319a00640 1 --2- 172.21.15.5:0/2516093080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc314074cd0 0x7fc31410e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.110+0000 7fc319a00640 1 --2- 172.21.15.5:0/2516093080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3140743b0 0x7fc314074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.110+0000 7fc319a00640 1 -- 172.21.15.5:0/2516093080 >> 172.21.15.5:0/2516093080 conn(0x7fc3140fd950 msgr2=0x7fc3140ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:18.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.111+0000 7fc319a00640 1 -- 172.21.15.5:0/2516093080 shutdown_connections 2024-04-24T06:45:18.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.111+0000 7fc319a00640 1 -- 172.21.15.5:0/2516093080 wait complete. 2024-04-24T06:45:18.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.112+0000 7fc319a00640 1 Processor -- start 2024-04-24T06:45:18.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.112+0000 7fc319a00640 1 -- start start 2024-04-24T06:45:18.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.113+0000 7fc319a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3140743b0 0x7fc3141a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:18.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.113+0000 7fc319a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc314074cd0 0x7fc3141a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:18.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.113+0000 7fc319a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc31410e6c0 0x7fc31419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:18.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.113+0000 7fc319a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc3141143c0 con 0x7fc3140743b0 2024-04-24T06:45:18.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.113+0000 7fc319a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc314114240 con 0x7fc31410e6c0 2024-04-24T06:45:18.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.113+0000 7fc319a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc314114540 con 0x7fc314074cd0 2024-04-24T06:45:18.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.113+0000 7fc313e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3140743b0 0x7fc3141a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:18.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.113+0000 7fc313e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3140743b0 0x7fc3141a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40414/0 (socket says 172.21.15.5:40414) 2024-04-24T06:45:18.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.113+0000 7fc313e00640 1 -- 172.21.15.5:0/3373385439 learned_addr learned my addr 172.21.15.5:0/3373385439 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:18.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.113+0000 7fc313400640 1 --2- 172.21.15.5:0/3373385439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc314074cd0 0x7fc3141a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:18.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.113+0000 7fc319000640 1 --2- 172.21.15.5:0/3373385439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc31410e6c0 0x7fc31419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:18.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.114+0000 7fc313e00640 1 -- 172.21.15.5:0/3373385439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc314074cd0 msgr2=0x7fc3141a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:18.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.114+0000 7fc313e00640 1 --2- 172.21.15.5:0/3373385439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc314074cd0 0x7fc3141a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.114+0000 7fc313e00640 1 -- 172.21.15.5:0/3373385439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc31410e6c0 msgr2=0x7fc31419a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:18.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.114+0000 7fc313e00640 1 --2- 172.21.15.5:0/3373385439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc31410e6c0 0x7fc31419a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.114+0000 7fc313e00640 1 -- 172.21.15.5:0/3373385439 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc31419b000 con 0x7fc3140743b0 2024-04-24T06:45:18.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.114+0000 7fc319000640 1 --2- 172.21.15.5:0/3373385439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc31410e6c0 0x7fc31419a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:45:18.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.114+0000 7fc313e00640 1 --2- 172.21.15.5:0/3373385439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3140743b0 0x7fc3141a0630 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc30400e9f0 tx=0x7fc30400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:18.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.114+0000 7fc310c00640 1 -- 172.21.15.5:0/3373385439 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30400cd80 con 0x7fc3140743b0 2024-04-24T06:45:18.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.114+0000 7fc310c00640 1 -- 172.21.15.5:0/3373385439 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc304010910 con 0x7fc3140743b0 2024-04-24T06:45:18.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.114+0000 7fc310c00640 1 -- 172.21.15.5:0/3373385439 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc304018990 con 0x7fc3140743b0 2024-04-24T06:45:18.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.114+0000 7fc319a00640 1 -- 172.21.15.5:0/3373385439 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc31419b2f0 con 0x7fc3140743b0 2024-04-24T06:45:18.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.115+0000 7fc319a00640 1 -- 172.21.15.5:0/3373385439 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3141a75b0 con 0x7fc3140743b0 2024-04-24T06:45:18.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.118+0000 7fc31dfe2740 1 Processor -- start 2024-04-24T06:45:18.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.118+0000 7fc31dfe2740 1 -- start start 2024-04-24T06:45:18.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.119+0000 7fc31dfe2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd4ed2fa80 0x55dd4ed2fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:18.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.119+0000 7fc31dfe2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd4ed303a0 0x55dd4ed3a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:18.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.119+0000 7fc31dfe2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd4ed3aa70 0x55dd4ed3ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:18.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.119+0000 7fc31dfe2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dd4ec8ce70 con 0x55dd4ed303a0 2024-04-24T06:45:18.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.119+0000 7fc31dfe2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55dd4ec33380 con 0x55dd4ed3aa70 2024-04-24T06:45:18.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.119+0000 7fc31dfe2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55dd4ec88e30 con 0x55dd4ed2fa80 2024-04-24T06:45:18.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.119+0000 7fc319000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd4ed2fa80 0x55dd4ed2fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:18.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.119+0000 7fc319000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd4ed2fa80 0x55dd4ed2fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:58940/0 (socket says 172.21.15.5:58940) 2024-04-24T06:45:18.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.119+0000 7fc319000640 1 -- 172.21.15.5:0/2416730679 learned_addr learned my addr 172.21.15.5:0/2416730679 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:18.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.119+0000 7fc313400640 1 --2- 172.21.15.5:0/2416730679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd4ed3aa70 0x55dd4ed3ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:18.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.120+0000 7fc319000640 1 -- 172.21.15.5:0/2416730679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd4ed3aa70 msgr2=0x55dd4ed3ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:18.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.120+0000 7fc319000640 1 --2- 172.21.15.5:0/2416730679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd4ed3aa70 0x55dd4ed3ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.120+0000 7fc319000640 1 -- 172.21.15.5:0/2416730679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd4ed303a0 msgr2=0x55dd4ed3a530 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T06:45:18.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.120+0000 7fc319000640 1 --2- 172.21.15.5:0/2416730679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd4ed303a0 0x55dd4ed3a530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.120+0000 7fc319000640 1 -- 172.21.15.5:0/2416730679 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd4ea677e0 con 0x55dd4ed2fa80 2024-04-24T06:45:18.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.120+0000 7fc310c00640 1 -- 172.21.15.5:0/3373385439 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138929+0+0 (secure 0 0 0) 0x7fc304018b30 con 0x7fc3140743b0 2024-04-24T06:45:18.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.120+0000 7fc319000640 1 --2- 172.21.15.5:0/2416730679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd4ed2fa80 0x55dd4ed2fe60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc30c009ca0 tx=0x7fc30c009360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb2811771a8906ce server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:18.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.120+0000 7fc309600640 1 -- 172.21.15.5:0/2416730679 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30c048070 con 0x55dd4ed2fa80 2024-04-24T06:45:18.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.120+0000 7fc309600640 1 -- 172.21.15.5:0/2416730679 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc30c009820 con 0x55dd4ed2fa80 2024-04-24T06:45:18.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.120+0000 7fc309600640 1 -- 172.21.15.5:0/2416730679 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30c034040 con 0x55dd4ed2fa80 2024-04-24T06:45:18.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.121+0000 7fc31dfe2740 1 -- 172.21.15.5:0/2416730679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd4ed2fa80 msgr2=0x55dd4ed2fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:18.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.121+0000 7fc31dfe2740 1 --2- 172.21.15.5:0/2416730679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd4ed2fa80 0x55dd4ed2fe60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc30c009ca0 tx=0x7fc30c009360 comp rx=0 tx=0).stop 2024-04-24T06:45:18.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.121+0000 7fc31dfe2740 1 -- 172.21.15.5:0/2416730679 shutdown_connections 2024-04-24T06:45:18.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.121+0000 7fc31dfe2740 1 --2- 172.21.15.5:0/2416730679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd4ed3aa70 0x55dd4ed3ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.121+0000 7fc31dfe2740 1 --2- 172.21.15.5:0/2416730679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd4ed303a0 0x55dd4ed3a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.121+0000 7fc31dfe2740 1 --2- 172.21.15.5:0/2416730679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd4ed2fa80 0x55dd4ed2fe60 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.121+0000 7fc31dfe2740 1 -- 172.21.15.5:0/2416730679 >> 172.21.15.5:0/2416730679 conn(0x55dd4ed2bda0 msgr2=0x55dd4ed2c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:18.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.121+0000 7fc31dfe2740 1 -- 172.21.15.5:0/2416730679 shutdown_connections 2024-04-24T06:45:18.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.121+0000 7fc31dfe2740 1 -- 172.21.15.5:0/2416730679 wait complete. 2024-04-24T06:45:18.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.122+0000 7fc31dfe2740 1 -- start start 2024-04-24T06:45:18.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.122+0000 7fc31dfe2740 1 Processor -- start 2024-04-24T06:45:18.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.122+0000 7fc310c00640 1 --2- 172.21.15.5:0/3373385439 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc2f80a3e60 0x7fc2f80a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:18.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.122+0000 7fc310c00640 1 -- 172.21.15.5:0/3373385439 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fc3040d0df0 con 0x7fc3140743b0 2024-04-24T06:45:18.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.123+0000 7fc319000640 1 --2- 172.21.15.5:0/3373385439 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc2f80a3e60 0x7fc2f80a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:18.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.123+0000 7fc31dfe2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd4ed2fa80 0x55dd4ed37ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:18.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.123+0000 7fc31dfe2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd4ed303a0 0x55dd4ed37ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:18.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.123+0000 7fc31dfe2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd4ed3aa70 0x55dd4ed38530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:18.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.123+0000 7fc31dfe2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dd4ed3d800 con 0x55dd4ed2fa80 2024-04-24T06:45:18.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.123+0000 7fc31dfe2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55dd4ec46660 con 0x55dd4ed3aa70 2024-04-24T06:45:18.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.123+0000 7fc31dfe2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55dd4ed3d980 con 0x55dd4ed303a0 2024-04-24T06:45:18.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.123+0000 7fc31dfe2740 10 client.0 _ll_register_callbacks cb 0x55dd4ec34aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:45:18.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.123+0000 7fc313e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd4ed2fa80 0x55dd4ed37ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:18.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.123+0000 7fc313e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd4ed2fa80 0x55dd4ed37ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40418/0 (socket says 172.21.15.5:40418) 2024-04-24T06:45:18.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.123+0000 7fc313e00640 1 -- 172.21.15.5:0/161446044 learned_addr learned my addr 172.21.15.5:0/161446044 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:18.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.123+0000 7fc313400640 1 --2- 172.21.15.5:0/161446044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd4ed303a0 0x55dd4ed37ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:18.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.124+0000 7fc313e00640 1 -- 172.21.15.5:0/161446044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd4ed303a0 msgr2=0x55dd4ed37ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:18.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.124+0000 7fc313e00640 1 --2- 172.21.15.5:0/161446044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd4ed303a0 0x55dd4ed37ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.124+0000 7fc313e00640 1 -- 172.21.15.5:0/161446044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd4ed3aa70 msgr2=0x55dd4ed38530 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T06:45:18.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.124+0000 7fc313e00640 1 --2- 172.21.15.5:0/161446044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd4ed3aa70 0x55dd4ed38530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.124+0000 7fc313e00640 1 -- 172.21.15.5:0/161446044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd4eac7f70 con 0x55dd4ed2fa80 2024-04-24T06:45:18.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.124+0000 7fc313e00640 1 --2- 172.21.15.5:0/161446044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd4ed2fa80 0x55dd4ed37ab0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc3040ce400 tx=0x7fc30409c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e182ef35304efe1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:18.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.124+0000 7fc2fe000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:45:18.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.124+0000 7fc2fe000640 1 -- 172.21.15.5:0/161446044 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30409c060 con 0x55dd4ed2fa80 2024-04-24T06:45:18.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.124+0000 7fc31dfe2740 1 -- 172.21.15.5:0/161446044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc30c009820 con 0x55dd4ed2fa80 2024-04-24T06:45:18.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.124+0000 7fc319000640 1 --2- 172.21.15.5:0/3373385439 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc2f80a3e60 0x7fc2f80a6320 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc30c005e30 tx=0x7fc30c02a950 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:18.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.124+0000 7fc2fe000640 1 -- 172.21.15.5:0/161446044 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fc30409c200 con 0x55dd4ed2fa80 2024-04-24T06:45:18.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.125+0000 7fc2fe000640 1 -- 172.21.15.5:0/161446044 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc304010430 con 0x55dd4ed2fa80 2024-04-24T06:45:18.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.125+0000 7fc31dfe2740 10 client.15537 fetch_fsmap learned FSMap version 23 2024-04-24T06:45:18.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.125+0000 7fc31dfe2740 1 -- 172.21.15.5:0/161446044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55dd4e95f170 con 0x55dd4ed2fa80 2024-04-24T06:45:18.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.125+0000 7fc2fe000640 1 -- 172.21.15.5:0/161446044 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc30c009820 con 0x55dd4ed2fa80 2024-04-24T06:45:18.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.125+0000 7fc2fe000640 1 -- 172.21.15.5:0/161446044 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fc30409bc60 con 0x55dd4ed2fa80 2024-04-24T06:45:18.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.125+0000 7fc31dfe2740 10 client.15537 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T06:45:18.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.125+0000 7fc31dfe2740 10 client.15537 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:45:18.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.125+0000 7fc31dfe2740 20 client.15537 populate_metadata read hostname 'smithi005' 2024-04-24T06:45:18.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.125+0000 7fc31dfe2740 1 --2- 172.21.15.5:0/161446044 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55dd4ed4adf0 0x55dd4ed4d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:18.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.125+0000 7fc31dfe2740 4 client.15537 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:45:18.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.125+0000 7fc31dfe2740 1 -- 172.21.15.5:0/161446044 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd4ec9a890 con 0x55dd4ed4adf0 2024-04-24T06:45:18.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.126+0000 7fc313400640 1 --2- 172.21.15.5:0/161446044 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55dd4ed4adf0 0x55dd4ed4d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:18.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.126+0000 7fc313400640 1 --2- 172.21.15.5:0/161446044 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55dd4ed4adf0 0x55dd4ed4d1e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ade4a2d2c36b87e8 server_cookie=23afb0a4382da34f in_seq=0 out_seq=0 2024-04-24T06:45:18.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.126+0000 7fc2fe000640 10 client.15537 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:45:18.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.127+0000 7fc2fe000640 1 -- 172.21.15.5:0/161446044 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dd4ec9a890 con 0x55dd4ed4adf0 2024-04-24T06:45:18.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.127+0000 7fc2fe000640 10 client.15537 handle_command_reply: tid=0 2024-04-24T06:45:18.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.127+0000 7fc31dfe2740 1 client.15537 shutdown 2024-04-24T06:45:18.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.127+0000 7fc31dfe2740 1 -- 172.21.15.5:0/161446044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd4ed2fa80 msgr2=0x55dd4ed37ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:18.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.127+0000 7fc31dfe2740 1 --2- 172.21.15.5:0/161446044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd4ed2fa80 0x55dd4ed37ab0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc3040ce400 tx=0x7fc30409c670 comp rx=0 tx=0).stop 2024-04-24T06:45:18.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.127+0000 7fc31dfe2740 1 -- 172.21.15.5:0/161446044 shutdown_connections 2024-04-24T06:45:18.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.127+0000 7fc31dfe2740 1 --2- 172.21.15.5:0/161446044 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55dd4ed4adf0 0x55dd4ed4d1e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.127+0000 7fc31dfe2740 1 --2- 172.21.15.5:0/161446044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd4ed3aa70 0x55dd4ed38530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.127+0000 7fc31dfe2740 1 --2- 172.21.15.5:0/161446044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd4ed303a0 0x55dd4ed37ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.127+0000 7fc31dfe2740 1 --2- 172.21.15.5:0/161446044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd4ed2fa80 0x55dd4ed37ab0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.127+0000 7fc31dfe2740 1 -- 172.21.15.5:0/161446044 >> 172.21.15.5:0/161446044 conn(0x55dd4ed2bda0 msgr2=0x55dd4ed31ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:18.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.127+0000 7fc2fe000640 0 client.15537 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:45:18.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.128+0000 7fc31dfe2740 1 -- 172.21.15.5:0/161446044 shutdown_connections 2024-04-24T06:45:18.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.128+0000 7fc31dfe2740 1 -- 172.21.15.5:0/161446044 wait complete. 2024-04-24T06:45:18.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.128+0000 7fc31dfe2740 20 client.15537 trim_cache size 0 max 16384 2024-04-24T06:45:18.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.147+0000 7fc31dfe2740 1 Processor -- start 2024-04-24T06:45:18.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.147+0000 7fc31dfe2740 1 -- start start 2024-04-24T06:45:18.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.147+0000 7fc31dfe2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd4ed2fa80 0x55dd4ed466c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:18.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.147+0000 7fc31dfe2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd4ed303a0 0x55dd4ed52a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:18.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.147+0000 7fc31dfe2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd4ed3aa70 0x55dd4ed52fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:18.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.147+0000 7fc31dfe2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc3141ae260 con 0x55dd4ed3aa70 2024-04-24T06:45:18.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.147+0000 7fc31dfe2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc3141abb30 con 0x55dd4ed303a0 2024-04-24T06:45:18.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.147+0000 7fc31dfe2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc3141a9940 con 0x55dd4ed2fa80 2024-04-24T06:45:18.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.147+0000 7fc319000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd4ed3aa70 0x55dd4ed52fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:18.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.147+0000 7fc319000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd4ed3aa70 0x55dd4ed52fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40430/0 (socket says 172.21.15.5:40430) 2024-04-24T06:45:18.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.147+0000 7fc319000640 1 -- 172.21.15.5:0/3253993927 learned_addr learned my addr 172.21.15.5:0/3253993927 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:18.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.148+0000 7fc313e00640 1 --2- 172.21.15.5:0/3253993927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd4ed2fa80 0x55dd4ed466c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:18.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.148+0000 7fc313400640 1 --2- 172.21.15.5:0/3253993927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd4ed303a0 0x55dd4ed52a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:18.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.148+0000 7fc313400640 1 -- 172.21.15.5:0/3253993927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd4ed2fa80 msgr2=0x55dd4ed466c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:18.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.148+0000 7fc313400640 1 --2- 172.21.15.5:0/3253993927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd4ed2fa80 0x55dd4ed466c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.148+0000 7fc313400640 1 -- 172.21.15.5:0/3253993927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd4ed3aa70 msgr2=0x55dd4ed52fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:18.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.148+0000 7fc313400640 1 --2- 172.21.15.5:0/3253993927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd4ed3aa70 0x55dd4ed52fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.148+0000 7fc313400640 1 -- 172.21.15.5:0/3253993927 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd4ed36000 con 0x55dd4ed303a0 2024-04-24T06:45:18.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.148+0000 7fc319000640 1 --2- 172.21.15.5:0/3253993927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd4ed3aa70 0x55dd4ed52fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:45:18.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.148+0000 7fc313e00640 1 --2- 172.21.15.5:0/3253993927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd4ed2fa80 0x55dd4ed466c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:45:18.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.149+0000 7fc313400640 1 --2- 172.21.15.5:0/3253993927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd4ed303a0 0x55dd4ed52a60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x55dd4ed4d9d0 tx=0x7fc300005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a64342227a578ab server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:18.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.149+0000 7fc308c00640 1 -- 172.21.15.5:0/3253993927 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc300024030 con 0x55dd4ed303a0 2024-04-24T06:45:18.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.149+0000 7fc308c00640 1 -- 172.21.15.5:0/3253993927 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc300005500 con 0x55dd4ed303a0 2024-04-24T06:45:18.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.149+0000 7fc308c00640 1 -- 172.21.15.5:0/3253993927 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc300020350 con 0x55dd4ed303a0 2024-04-24T06:45:18.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.150+0000 7fc31dfe2740 1 -- 172.21.15.5:0/3253993927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd4ed303a0 msgr2=0x55dd4ed52a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:18.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.150+0000 7fc31dfe2740 1 --2- 172.21.15.5:0/3253993927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd4ed303a0 0x55dd4ed52a60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x55dd4ed4d9d0 tx=0x7fc300005f90 comp rx=0 tx=0).stop 2024-04-24T06:45:18.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.150+0000 7fc31dfe2740 1 -- 172.21.15.5:0/3253993927 shutdown_connections 2024-04-24T06:45:18.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.150+0000 7fc31dfe2740 1 --2- 172.21.15.5:0/3253993927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd4ed3aa70 0x55dd4ed52fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.150+0000 7fc31dfe2740 1 --2- 172.21.15.5:0/3253993927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd4ed303a0 0x55dd4ed52a60 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.150+0000 7fc31dfe2740 1 --2- 172.21.15.5:0/3253993927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd4ed2fa80 0x55dd4ed466c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.150+0000 7fc31dfe2740 1 -- 172.21.15.5:0/3253993927 >> 172.21.15.5:0/3253993927 conn(0x55dd4ed2bda0 msgr2=0x55dd4ed226f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:18.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.150+0000 7fc31dfe2740 1 -- 172.21.15.5:0/3253993927 shutdown_connections 2024-04-24T06:45:18.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.150+0000 7fc31dfe2740 1 -- 172.21.15.5:0/3253993927 wait complete. 2024-04-24T06:45:18.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.150+0000 7fc31dfe2740 1 -- start start 2024-04-24T06:45:18.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.151+0000 7fc31dfe2740 1 Processor -- start 2024-04-24T06:45:18.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.151+0000 7fc31dfe2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd4ed2fa80 0x55dd4ed42d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:18.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.151+0000 7fc31dfe2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd4ed303a0 0x55dd4ed3f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:18.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.151+0000 7fc31dfe2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd4ed3aa70 0x55dd4ed3f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:18.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.151+0000 7fc31dfe2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc3141a9ff0 con 0x55dd4ed303a0 2024-04-24T06:45:18.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.151+0000 7fc31dfe2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc3141ab460 con 0x55dd4ed2fa80 2024-04-24T06:45:18.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.151+0000 7fc31dfe2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc3141aa6c0 con 0x55dd4ed3aa70 2024-04-24T06:45:18.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.151+0000 7fc31dfe2740 10 client.0 _ll_register_callbacks cb 0x7fc30400cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:45:18.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.151+0000 7fc313e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd4ed2fa80 0x55dd4ed42d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:18.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.151+0000 7fc313400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd4ed303a0 0x55dd4ed3f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:18.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.151+0000 7fc313e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd4ed2fa80 0x55dd4ed42d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:47786/0 (socket says 172.21.15.5:47786) 2024-04-24T06:45:18.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.151+0000 7fc313e00640 1 -- 172.21.15.5:0/3104110587 learned_addr learned my addr 172.21.15.5:0/3104110587 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:18.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.151+0000 7fc319000640 1 --2- 172.21.15.5:0/3104110587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd4ed3aa70 0x55dd4ed3f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:18.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.152+0000 7fc313e00640 1 -- 172.21.15.5:0/3104110587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd4ed3aa70 msgr2=0x55dd4ed3f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:18.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.152+0000 7fc313e00640 1 --2- 172.21.15.5:0/3104110587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd4ed3aa70 0x55dd4ed3f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.152+0000 7fc313e00640 1 -- 172.21.15.5:0/3104110587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd4ed303a0 msgr2=0x55dd4ed3f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:18.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.152+0000 7fc313e00640 1 --2- 172.21.15.5:0/3104110587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd4ed303a0 0x55dd4ed3f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.152+0000 7fc313e00640 1 -- 172.21.15.5:0/3104110587 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd4ed3fb30 con 0x55dd4ed2fa80 2024-04-24T06:45:18.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.152+0000 7fc313e00640 1 --2- 172.21.15.5:0/3104110587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd4ed2fa80 0x55dd4ed42d00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc3040b6250 tx=0x7fc304018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7f28c2329d62b2c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:18.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.152+0000 7fc2fe000640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:45:18.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.152+0000 7fc2fe000640 1 -- 172.21.15.5:0/3104110587 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc304010430 con 0x55dd4ed2fa80 2024-04-24T06:45:18.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.152+0000 7fc31dfe2740 1 -- 172.21.15.5:0/3104110587 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc300005500 con 0x55dd4ed2fa80 2024-04-24T06:45:18.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.152+0000 7fc2fe000640 1 -- 172.21.15.5:0/3104110587 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fc3040105d0 con 0x55dd4ed2fa80 2024-04-24T06:45:18.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.153+0000 7fc2fe000640 1 -- 172.21.15.5:0/3104110587 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc30409c9e0 con 0x55dd4ed2fa80 2024-04-24T06:45:18.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.153+0000 7fc31dfe2740 10 client.25643 fetch_fsmap learned FSMap version 23 2024-04-24T06:45:18.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.153+0000 7fc31dfe2740 1 -- 172.21.15.5:0/3104110587 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55dd4edb2080 con 0x55dd4ed2fa80 2024-04-24T06:45:18.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.153+0000 7fc2fe000640 1 -- 172.21.15.5:0/3104110587 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc300005500 con 0x55dd4ed2fa80 2024-04-24T06:45:18.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.153+0000 7fc2fe000640 1 -- 172.21.15.5:0/3104110587 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fc30409bbf0 con 0x55dd4ed2fa80 2024-04-24T06:45:18.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.153+0000 7fc31dfe2740 10 client.25643 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T06:45:18.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.153+0000 7fc31dfe2740 10 client.25643 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:45:18.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.153+0000 7fc31dfe2740 20 client.25643 populate_metadata read hostname 'smithi005' 2024-04-24T06:45:18.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.153+0000 7fc31dfe2740 1 --2- 172.21.15.5:0/3104110587 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55dd4edb27f0 0x55dd4edb4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:18.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.153+0000 7fc31dfe2740 4 client.25643 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T06:45:18.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.153+0000 7fc31dfe2740 1 -- 172.21.15.5:0/3104110587 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55dd4ec94700 con 0x55dd4edb27f0 2024-04-24T06:45:18.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.154+0000 7fc313400640 1 --2- 172.21.15.5:0/3104110587 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55dd4edb27f0 0x55dd4edb4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:18.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.154+0000 7fc313400640 1 --2- 172.21.15.5:0/3104110587 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55dd4edb27f0 0x55dd4edb4c10 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a4d02a1829cca68 server_cookie=8b8591b3ae2d530b in_seq=0 out_seq=0 2024-04-24T06:45:18.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.154+0000 7fc2fe000640 10 client.25643 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:45:18.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.155+0000 7fc2fe000640 1 -- 172.21.15.5:0/3104110587 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55dd4ec94700 con 0x55dd4edb27f0 2024-04-24T06:45:18.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.155+0000 7fc2fe000640 10 client.25643 handle_command_reply: tid=0 2024-04-24T06:45:18.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.155+0000 7fc31dfe2740 1 client.25643 shutdown 2024-04-24T06:45:18.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.156+0000 7fc31dfe2740 1 -- 172.21.15.5:0/3104110587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd4ed2fa80 msgr2=0x55dd4ed42d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:18.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.156+0000 7fc31dfe2740 1 --2- 172.21.15.5:0/3104110587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd4ed2fa80 0x55dd4ed42d00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc3040b6250 tx=0x7fc304018420 comp rx=0 tx=0).stop 2024-04-24T06:45:18.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.156+0000 7fc31dfe2740 1 -- 172.21.15.5:0/3104110587 shutdown_connections 2024-04-24T06:45:18.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.156+0000 7fc31dfe2740 1 --2- 172.21.15.5:0/3104110587 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55dd4edb27f0 0x55dd4edb4c10 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.156+0000 7fc31dfe2740 1 --2- 172.21.15.5:0/3104110587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd4ed3aa70 0x55dd4ed3f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.156+0000 7fc31dfe2740 1 --2- 172.21.15.5:0/3104110587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd4ed303a0 0x55dd4ed3f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.156+0000 7fc31dfe2740 1 --2- 172.21.15.5:0/3104110587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd4ed2fa80 0x55dd4ed42d00 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.156+0000 7fc31dfe2740 1 -- 172.21.15.5:0/3104110587 >> 172.21.15.5:0/3104110587 conn(0x55dd4ed2bda0 msgr2=0x55dd4ed21e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:18.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.156+0000 7fc2fe000640 0 client.25643 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:45:18.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.156+0000 7fc31dfe2740 1 -- 172.21.15.5:0/3104110587 shutdown_connections 2024-04-24T06:45:18.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.156+0000 7fc31dfe2740 1 -- 172.21.15.5:0/3104110587 wait complete. 2024-04-24T06:45:18.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.156+0000 7fc31dfe2740 20 client.25643 trim_cache size 0 max 16384 2024-04-24T06:45:18.165 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:45:18.165 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-24T06:45:18.165 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "e0208b86-07ac-4827-bafa-dec018d55907", 2024-04-24T06:45:18.165 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-24T06:45:18.165 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:45:18.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.157+0000 7fc319a00640 1 -- 172.21.15.5:0/3373385439 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc2f80a3e60 msgr2=0x7fc2f80a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:18.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.157+0000 7fc319a00640 1 --2- 172.21.15.5:0/3373385439 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc2f80a3e60 0x7fc2f80a6320 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc30c005e30 tx=0x7fc30c02a950 comp rx=0 tx=0).stop 2024-04-24T06:45:18.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.157+0000 7fc319a00640 1 -- 172.21.15.5:0/3373385439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3140743b0 msgr2=0x7fc3141a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:18.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.157+0000 7fc319a00640 1 --2- 172.21.15.5:0/3373385439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3140743b0 0x7fc3141a0630 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc30400e9f0 tx=0x7fc30400eeb0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.158+0000 7fc319a00640 1 -- 172.21.15.5:0/3373385439 shutdown_connections 2024-04-24T06:45:18.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.158+0000 7fc319a00640 1 --2- 172.21.15.5:0/3373385439 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc2f80a3e60 0x7fc2f80a6320 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.158+0000 7fc319a00640 1 --2- 172.21.15.5:0/3373385439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc31410e6c0 0x7fc31419a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.158+0000 7fc319a00640 1 --2- 172.21.15.5:0/3373385439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc314074cd0 0x7fc3141a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.158+0000 7fc319a00640 1 --2- 172.21.15.5:0/3373385439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3140743b0 0x7fc3141a0630 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.158+0000 7fc319a00640 1 -- 172.21.15.5:0/3373385439 >> 172.21.15.5:0/3373385439 conn(0x7fc3140fd950 msgr2=0x7fc3140fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:18.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.158+0000 7fc319a00640 1 -- 172.21.15.5:0/3373385439 shutdown_connections 2024-04-24T06:45:18.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.158+0000 7fc319a00640 1 -- 172.21.15.5:0/3373385439 wait complete. 2024-04-24T06:45:18.182 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T06:45:18.182 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T06:45:18.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.384+0000 7f340fe00640 1 -- 172.21.15.5:0/1575912137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3410101910 msgr2=0x7f3410105950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:18.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.384+0000 7f340fe00640 1 --2- 172.21.15.5:0/1575912137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3410101910 0x7f3410105950 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f33fc009900 tx=0x7f33fc02a3f0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.384+0000 7f340fe00640 1 -- 172.21.15.5:0/1575912137 shutdown_connections 2024-04-24T06:45:18.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.384+0000 7f340fe00640 1 --2- 172.21.15.5:0/1575912137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3410106030 0x7f341010dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.384+0000 7f340fe00640 1 --2- 172.21.15.5:0/1575912137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3410101910 0x7f3410105950 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.384+0000 7f340fe00640 1 --2- 172.21.15.5:0/1575912137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3410100f60 0x7f3410101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.384+0000 7f340fe00640 1 -- 172.21.15.5:0/1575912137 >> 172.21.15.5:0/1575912137 conn(0x7f3410077ab0 msgr2=0x7f34100fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:18.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.384+0000 7f340fe00640 1 -- 172.21.15.5:0/1575912137 shutdown_connections 2024-04-24T06:45:18.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.384+0000 7f340fe00640 1 -- 172.21.15.5:0/1575912137 wait complete. 2024-04-24T06:45:18.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.385+0000 7f340fe00640 1 Processor -- start 2024-04-24T06:45:18.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.385+0000 7f340fe00640 1 -- start start 2024-04-24T06:45:18.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.385+0000 7f340fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3410100f60 0x7f34100fcf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:18.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.385+0000 7f340fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3410106030 0x7f34100fd470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:18.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.385+0000 7f340fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34100fd9b0 0x7f34101a6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:18.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.386+0000 7f340fe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f34101110f0 con 0x7f3410100f60 2024-04-24T06:45:18.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.386+0000 7f340fe00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3410110f70 con 0x7f34100fd9b0 2024-04-24T06:45:18.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.386+0000 7f340fe00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3410111270 con 0x7f3410106030 2024-04-24T06:45:18.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.386+0000 7f340ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3410100f60 0x7f34100fcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:18.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.386+0000 7f340ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3410100f60 0x7f34100fcf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40462/0 (socket says 172.21.15.5:40462) 2024-04-24T06:45:18.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.386+0000 7f340ea00640 1 -- 172.21.15.5:0/3639702415 learned_addr learned my addr 172.21.15.5:0/3639702415 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:18.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.386+0000 7f340e000640 1 --2- 172.21.15.5:0/3639702415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3410106030 0x7f34100fd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:18.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.386+0000 7f340f400640 1 --2- 172.21.15.5:0/3639702415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34100fd9b0 0x7f34101a6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:18.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.386+0000 7f340ea00640 1 -- 172.21.15.5:0/3639702415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3410106030 msgr2=0x7f34100fd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:18.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.386+0000 7f340ea00640 1 --2- 172.21.15.5:0/3639702415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3410106030 0x7f34100fd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.386+0000 7f340ea00640 1 -- 172.21.15.5:0/3639702415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34100fd9b0 msgr2=0x7f34101a6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:18.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.386+0000 7f340ea00640 1 --2- 172.21.15.5:0/3639702415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34100fd9b0 0x7f34101a6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.386+0000 7f340ea00640 1 -- 172.21.15.5:0/3639702415 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34101a7060 con 0x7f3410100f60 2024-04-24T06:45:18.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.387+0000 7f340ea00640 1 --2- 172.21.15.5:0/3639702415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3410100f60 0x7f34100fcf30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f340000e9f0 tx=0x7f340000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:18.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.387+0000 7f3414a00640 1 -- 172.21.15.5:0/3639702415 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340000cd80 con 0x7f3410100f60 2024-04-24T06:45:18.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.387+0000 7f3414a00640 1 -- 172.21.15.5:0/3639702415 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3400010910 con 0x7f3410100f60 2024-04-24T06:45:18.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.387+0000 7f340fe00640 1 -- 172.21.15.5:0/3639702415 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34101a7350 con 0x7f3410100f60 2024-04-24T06:45:18.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.387+0000 7f3414a00640 1 -- 172.21.15.5:0/3639702415 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3400018990 con 0x7f3410100f60 2024-04-24T06:45:18.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.387+0000 7f340fe00640 1 -- 172.21.15.5:0/3639702415 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34101a78c0 con 0x7f3410100f60 2024-04-24T06:45:18.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.392+0000 7f3418b6b740 1 Processor -- start 2024-04-24T06:45:18.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.392+0000 7f3418b6b740 1 -- start start 2024-04-24T06:45:18.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.392+0000 7f3418b6b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587a55e5a30 0x5587a55e5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:18.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.392+0000 7f3418b6b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a55e6350 0x5587a55f04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:18.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.392+0000 7f3418b6b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587a55f0a20 0x5587a55f2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:18.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.392+0000 7f3418b6b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5587a554b1f0 con 0x5587a55e6350 2024-04-24T06:45:18.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.392+0000 7f3418b6b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5587a54f1350 con 0x5587a55e5a30 2024-04-24T06:45:18.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.392+0000 7f3418b6b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5587a55471b0 con 0x5587a55f0a20 2024-04-24T06:45:18.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.393+0000 7f3414a00640 1 -- 172.21.15.5:0/3639702415 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138929+0+0 (secure 0 0 0) 0x7f3400010ac0 con 0x7f3410100f60 2024-04-24T06:45:18.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.393+0000 7f340f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587a55e5a30 0x5587a55e5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:18.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.393+0000 7f340f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587a55e5a30 0x5587a55e5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:47812/0 (socket says 172.21.15.5:47812) 2024-04-24T06:45:18.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.393+0000 7f340f400640 1 -- 172.21.15.5:0/2976288038 learned_addr learned my addr 172.21.15.5:0/2976288038 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:18.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.393+0000 7f340e000640 1 --2- 172.21.15.5:0/2976288038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587a55f0a20 0x5587a55f2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:18.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.393+0000 7f340f400640 1 -- 172.21.15.5:0/2976288038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587a55f0a20 msgr2=0x5587a55f2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:18.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.393+0000 7f340f400640 1 --2- 172.21.15.5:0/2976288038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587a55f0a20 0x5587a55f2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.393+0000 7f340f400640 1 -- 172.21.15.5:0/2976288038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a55e6350 msgr2=0x5587a55f04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:18.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.393+0000 7f340f400640 1 --2- 172.21.15.5:0/2976288038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a55e6350 0x5587a55f04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.393+0000 7f340f400640 1 -- 172.21.15.5:0/2976288038 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587a5545f50 con 0x5587a55e5a30 2024-04-24T06:45:18.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.393+0000 7f340e000640 1 --2- 172.21.15.5:0/2976288038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587a55f0a20 0x5587a55f2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:45:18.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.394+0000 7f340f400640 1 --2- 172.21.15.5:0/2976288038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587a55e5a30 0x5587a55e5e10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3408009f20 tx=0x7f340800f7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=681470c68b9fdfe2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:18.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.394+0000 7f33fbe00640 1 -- 172.21.15.5:0/2976288038 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3408015070 con 0x5587a55e5a30 2024-04-24T06:45:18.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.394+0000 7f33fbe00640 1 -- 172.21.15.5:0/2976288038 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f340801ab40 con 0x5587a55e5a30 2024-04-24T06:45:18.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.394+0000 7f33fbe00640 1 -- 172.21.15.5:0/2976288038 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3408011bb0 con 0x5587a55e5a30 2024-04-24T06:45:18.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.395+0000 7f3418b6b740 1 -- 172.21.15.5:0/2976288038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587a55e5a30 msgr2=0x5587a55e5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:18.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.395+0000 7f3418b6b740 1 --2- 172.21.15.5:0/2976288038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587a55e5a30 0x5587a55e5e10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3408009f20 tx=0x7f340800f7f0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.395+0000 7f3414a00640 1 --2- 172.21.15.5:0/3639702415 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f33f40a4000 0x7f33f40a64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:18.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.395+0000 7f3414a00640 1 -- 172.21.15.5:0/3639702415 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f34000d0fd0 con 0x7f3410100f60 2024-04-24T06:45:18.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.395+0000 7f3418b6b740 1 -- 172.21.15.5:0/2976288038 shutdown_connections 2024-04-24T06:45:18.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.395+0000 7f3418b6b740 1 --2- 172.21.15.5:0/2976288038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587a55f0a20 0x5587a55f2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.395+0000 7f3418b6b740 1 --2- 172.21.15.5:0/2976288038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a55e6350 0x5587a55f04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.395+0000 7f3418b6b740 1 --2- 172.21.15.5:0/2976288038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587a55e5a30 0x5587a55e5e10 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.395+0000 7f3418b6b740 1 -- 172.21.15.5:0/2976288038 >> 172.21.15.5:0/2976288038 conn(0x5587a55e1d50 msgr2=0x5587a55e2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:18.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.395+0000 7f340ea00640 1 --2- 172.21.15.5:0/3639702415 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f33f40a4000 0x7f33f40a64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:18.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.395+0000 7f3418b6b740 1 -- 172.21.15.5:0/2976288038 shutdown_connections 2024-04-24T06:45:18.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.396+0000 7f3418b6b740 1 -- 172.21.15.5:0/2976288038 wait complete. 2024-04-24T06:45:18.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.396+0000 7f3418b6b740 1 -- start start 2024-04-24T06:45:18.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.396+0000 7f340ea00640 1 --2- 172.21.15.5:0/3639702415 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f33f40a4000 0x7f33f40a64c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f34000ce550 tx=0x7f340009c8d0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:18.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.396+0000 7f3418b6b740 1 Processor -- start 2024-04-24T06:45:18.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.397+0000 7f3418b6b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587a55e5a30 0x5587a55ed9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:18.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.397+0000 7f3418b6b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587a55e6350 0x5587a55edef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:18.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.397+0000 7f3418b6b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a55f0a20 0x5587a55ee430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:18.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.397+0000 7f3418b6b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5587a55f3810 con 0x5587a55f0a20 2024-04-24T06:45:18.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.397+0000 7f3418b6b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5587a5504690 con 0x5587a55e5a30 2024-04-24T06:45:18.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.397+0000 7f3418b6b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5587a55f3990 con 0x5587a55e6350 2024-04-24T06:45:18.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.397+0000 7f3418b6b740 10 client.0 _ll_register_callbacks cb 0x5587a5540650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:45:18.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.397+0000 7f340f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587a55e5a30 0x5587a55ed9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:18.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.398+0000 7f340f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587a55e5a30 0x5587a55ed9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:47816/0 (socket says 172.21.15.5:47816) 2024-04-24T06:45:18.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.398+0000 7f340f400640 1 -- 172.21.15.5:0/1043693013 learned_addr learned my addr 172.21.15.5:0/1043693013 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:18.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.398+0000 7f340f400640 1 --2- 172.21.15.5:0/1043693013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a55f0a20 0x5587a55ee430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:18.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.398+0000 7f340e000640 1 --2- 172.21.15.5:0/1043693013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587a55e6350 0x5587a55edef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:18.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.398+0000 7f340f400640 1 -- 172.21.15.5:0/1043693013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587a55e6350 msgr2=0x5587a55edef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:18.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.398+0000 7f340f400640 1 --2- 172.21.15.5:0/1043693013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587a55e6350 0x5587a55edef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.398+0000 7f340f400640 1 -- 172.21.15.5:0/1043693013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a55f0a20 msgr2=0x5587a55ee430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:18.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.398+0000 7f340f400640 1 --2- 172.21.15.5:0/1043693013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a55f0a20 0x5587a55ee430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.398+0000 7f340f400640 1 -- 172.21.15.5:0/1043693013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587a5503920 con 0x5587a55e5a30 2024-04-24T06:45:18.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.398+0000 7f340f400640 1 --2- 172.21.15.5:0/1043693013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587a55e5a30 0x5587a55ed9b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3408011690 tx=0x7f3408011790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d98ce497fbb74b94 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:18.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.398+0000 7f33f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:45:18.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.399+0000 7f33f8c00640 1 -- 172.21.15.5:0/1043693013 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3408015040 con 0x5587a55e5a30 2024-04-24T06:45:18.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.399+0000 7f33f8c00640 1 -- 172.21.15.5:0/1043693013 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f3408003970 con 0x5587a55e5a30 2024-04-24T06:45:18.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.399+0000 7f3418b6b740 1 -- 172.21.15.5:0/1043693013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f340801ab40 con 0x5587a55e5a30 2024-04-24T06:45:18.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.400+0000 7f33f8c00640 1 -- 172.21.15.5:0/1043693013 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f3408003b70 con 0x5587a55e5a30 2024-04-24T06:45:18.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.400+0000 7f3418b6b740 10 client.25655 fetch_fsmap learned FSMap version 23 2024-04-24T06:45:18.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.400+0000 7f3418b6b740 1 -- 172.21.15.5:0/1043693013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5587a53277d0 con 0x5587a55e5a30 2024-04-24T06:45:18.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.400+0000 7f33f8c00640 1 -- 172.21.15.5:0/1043693013 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f340801ab40 con 0x5587a55e5a30 2024-04-24T06:45:18.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.401+0000 7f33f8c00640 1 -- 172.21.15.5:0/1043693013 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f3408028030 con 0x5587a55e5a30 2024-04-24T06:45:18.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.401+0000 7f3418b6b740 10 client.25655 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T06:45:18.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.401+0000 7f3418b6b740 10 client.25655 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:45:18.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.401+0000 7f3418b6b740 20 client.25655 populate_metadata read hostname 'smithi005' 2024-04-24T06:45:18.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.401+0000 7f3418b6b740 1 --2- 172.21.15.5:0/1043693013 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5587a56094f0 0x5587a560b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:18.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.401+0000 7f3418b6b740 4 client.25655 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:45:18.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.401+0000 7f3418b6b740 1 -- 172.21.15.5:0/1043693013 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587a5558c10 con 0x5587a56094f0 2024-04-24T06:45:18.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.401+0000 7f340e000640 1 --2- 172.21.15.5:0/1043693013 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5587a56094f0 0x5587a560b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:18.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.402+0000 7f340e000640 1 --2- 172.21.15.5:0/1043693013 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5587a56094f0 0x5587a560b8e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2623ab1646dc60c7 server_cookie=f55cdabff0cb4951 in_seq=0 out_seq=0 2024-04-24T06:45:18.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.402+0000 7f33f8c00640 10 client.25655 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:45:18.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.403+0000 7f33f8c00640 1 -- 172.21.15.5:0/1043693013 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5587a5558c10 con 0x5587a56094f0 2024-04-24T06:45:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.403+0000 7f33f8c00640 10 client.25655 handle_command_reply: tid=0 2024-04-24T06:45:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.403+0000 7f3418b6b740 1 client.25655 shutdown 2024-04-24T06:45:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.403+0000 7f3418b6b740 1 -- 172.21.15.5:0/1043693013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587a55e5a30 msgr2=0x5587a55ed9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.403+0000 7f3418b6b740 1 --2- 172.21.15.5:0/1043693013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587a55e5a30 0x5587a55ed9b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3408011690 tx=0x7f3408011790 comp rx=0 tx=0).stop 2024-04-24T06:45:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.403+0000 7f3418b6b740 1 -- 172.21.15.5:0/1043693013 shutdown_connections 2024-04-24T06:45:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.403+0000 7f3418b6b740 1 --2- 172.21.15.5:0/1043693013 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5587a56094f0 0x5587a560b8e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.403+0000 7f3418b6b740 1 --2- 172.21.15.5:0/1043693013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a55f0a20 0x5587a55ee430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.403+0000 7f3418b6b740 1 --2- 172.21.15.5:0/1043693013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587a55e6350 0x5587a55edef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.403+0000 7f3418b6b740 1 --2- 172.21.15.5:0/1043693013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587a55e5a30 0x5587a55ed9b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.403+0000 7f3418b6b740 1 -- 172.21.15.5:0/1043693013 >> 172.21.15.5:0/1043693013 conn(0x5587a55e1d50 msgr2=0x5587a55e0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.403+0000 7f33f8c00640 0 client.25655 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:45:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.403+0000 7f3418b6b740 1 -- 172.21.15.5:0/1043693013 shutdown_connections 2024-04-24T06:45:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.403+0000 7f3418b6b740 1 -- 172.21.15.5:0/1043693013 wait complete. 2024-04-24T06:45:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.403+0000 7f3418b6b740 20 client.25655 trim_cache size 0 max 16384 2024-04-24T06:45:18.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.426+0000 7f3418b6b740 1 Processor -- start 2024-04-24T06:45:18.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.426+0000 7f3418b6b740 1 -- start start 2024-04-24T06:45:18.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.427+0000 7f3418b6b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587a55e5a30 0x5587a55fc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:18.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.427+0000 7f3418b6b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a55e6350 0x5587a55e8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:18.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.427+0000 7f3418b6b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587a55f0a20 0x5587a55ed2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:18.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.427+0000 7f3418b6b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f34101ae3b0 con 0x5587a55e6350 2024-04-24T06:45:18.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.427+0000 7f3418b6b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f34101abc80 con 0x5587a55e5a30 2024-04-24T06:45:18.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.427+0000 7f3418b6b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f34101a9b50 con 0x5587a55f0a20 2024-04-24T06:45:18.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.427+0000 7f340e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a55e6350 0x5587a55e8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:18.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.427+0000 7f340f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587a55e5a30 0x5587a55fc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:18.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.427+0000 7f340e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a55e6350 0x5587a55e8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40478/0 (socket says 172.21.15.5:40478) 2024-04-24T06:45:18.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.427+0000 7f340e000640 1 -- 172.21.15.5:0/3875170285 learned_addr learned my addr 172.21.15.5:0/3875170285 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:18.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.427+0000 7f340f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587a55e5a30 0x5587a55fc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:47820/0 (socket says 172.21.15.5:47820) 2024-04-24T06:45:18.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.427+0000 7f340f400640 1 --2- 172.21.15.5:0/3875170285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587a55f0a20 0x5587a55ed2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:18.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.428+0000 7f340e000640 1 -- 172.21.15.5:0/3875170285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587a55f0a20 msgr2=0x5587a55ed2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:18.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.428+0000 7f340e000640 1 --2- 172.21.15.5:0/3875170285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587a55f0a20 0x5587a55ed2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.428+0000 7f340e000640 1 -- 172.21.15.5:0/3875170285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587a55e5a30 msgr2=0x5587a55fc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:18.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.428+0000 7f340e000640 1 --2- 172.21.15.5:0/3875170285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587a55e5a30 0x5587a55fc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.428+0000 7f340e000640 1 -- 172.21.15.5:0/3875170285 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587a525d3f0 con 0x5587a55e6350 2024-04-24T06:45:18.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.428+0000 7f340e000640 1 --2- 172.21.15.5:0/3875170285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a55e6350 0x5587a55e8f70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f33fc031550 tx=0x7f33fc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7dec58a257dc355 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:18.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.428+0000 7f33fb400640 1 -- 172.21.15.5:0/3875170285 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33fc035040 con 0x5587a55e6350 2024-04-24T06:45:18.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.428+0000 7f33fb400640 1 -- 172.21.15.5:0/3875170285 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33fc04a070 con 0x5587a55e6350 2024-04-24T06:45:18.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.428+0000 7f33fb400640 1 -- 172.21.15.5:0/3875170285 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33fc0055c0 con 0x5587a55e6350 2024-04-24T06:45:18.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.429+0000 7f3418b6b740 1 -- 172.21.15.5:0/3875170285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a55e6350 msgr2=0x5587a55e8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:18.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.429+0000 7f3418b6b740 1 --2- 172.21.15.5:0/3875170285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a55e6350 0x5587a55e8f70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f33fc031550 tx=0x7f33fc000bc0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.429+0000 7f3418b6b740 1 -- 172.21.15.5:0/3875170285 shutdown_connections 2024-04-24T06:45:18.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.429+0000 7f3418b6b740 1 --2- 172.21.15.5:0/3875170285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587a55f0a20 0x5587a55ed2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.429+0000 7f3418b6b740 1 --2- 172.21.15.5:0/3875170285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a55e6350 0x5587a55e8f70 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.429+0000 7f3418b6b740 1 --2- 172.21.15.5:0/3875170285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587a55e5a30 0x5587a55fc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.429+0000 7f3418b6b740 1 -- 172.21.15.5:0/3875170285 >> 172.21.15.5:0/3875170285 conn(0x5587a55e1d50 msgr2=0x5587a55e7a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:18.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.429+0000 7f3418b6b740 1 -- 172.21.15.5:0/3875170285 shutdown_connections 2024-04-24T06:45:18.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.430+0000 7f3418b6b740 1 -- 172.21.15.5:0/3875170285 wait complete. 2024-04-24T06:45:18.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.430+0000 7f3418b6b740 1 -- start start 2024-04-24T06:45:18.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.430+0000 7f3418b6b740 1 Processor -- start 2024-04-24T06:45:18.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.431+0000 7f3418b6b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587a55e5a30 0x5587a55f8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:18.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.431+0000 7f3418b6b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587a55e6350 0x5587a55f9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:18.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.431+0000 7f3418b6b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a55f0a20 0x5587a55e9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:18.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.431+0000 7f3418b6b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f34101aa1a0 con 0x5587a55f0a20 2024-04-24T06:45:18.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.431+0000 7f3418b6b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f34101ab5b0 con 0x5587a55e5a30 2024-04-24T06:45:18.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.431+0000 7f3418b6b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f34101aa810 con 0x5587a55e6350 2024-04-24T06:45:18.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.431+0000 7f3418b6b740 10 client.0 _ll_register_callbacks cb 0x7f33fc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:45:18.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.431+0000 7f340f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587a55e5a30 0x5587a55f8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:18.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.431+0000 7f340f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a55f0a20 0x5587a55e9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:18.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.431+0000 7f340e000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587a55e6350 0x5587a55f9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:18.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.431+0000 7f340f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587a55e5a30 0x5587a55f8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:47822/0 (socket says 172.21.15.5:47822) 2024-04-24T06:45:18.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.431+0000 7f340e000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587a55e6350 0x5587a55f9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:59026/0 (socket says 172.21.15.5:59026) 2024-04-24T06:45:18.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.431+0000 7f340f400640 1 -- 172.21.15.5:0/3679071488 learned_addr learned my addr 172.21.15.5:0/3679071488 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:18.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.432+0000 7f340e000640 1 -- 172.21.15.5:0/3679071488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587a55e5a30 msgr2=0x5587a55f8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:18.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.432+0000 7f340e000640 1 --2- 172.21.15.5:0/3679071488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587a55e5a30 0x5587a55f8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.432+0000 7f340e000640 1 -- 172.21.15.5:0/3679071488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a55f0a20 msgr2=0x5587a55e9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:18.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.432+0000 7f340e000640 1 --2- 172.21.15.5:0/3679071488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a55f0a20 0x5587a55e9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.432+0000 7f340e000640 1 -- 172.21.15.5:0/3679071488 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587a521d0f0 con 0x5587a55e6350 2024-04-24T06:45:18.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.432+0000 7f340e000640 1 --2- 172.21.15.5:0/3679071488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587a55e6350 0x5587a55f9120 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f33fc009230 tx=0x7f33fc0095b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8453580ee65e89e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:18.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.432+0000 7f33f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:45:18.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.433+0000 7f33f8c00640 1 -- 172.21.15.5:0/3679071488 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33fc035050 con 0x5587a55e6350 2024-04-24T06:45:18.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.433+0000 7f3418b6b740 1 -- 172.21.15.5:0/3679071488 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33fc04a070 con 0x5587a55e6350 2024-04-24T06:45:18.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.433+0000 7f33f8c00640 1 -- 172.21.15.5:0/3679071488 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f33fc005670 con 0x5587a55e6350 2024-04-24T06:45:18.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.433+0000 7f33f8c00640 1 -- 172.21.15.5:0/3679071488 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f33fc0046c0 con 0x5587a55e6350 2024-04-24T06:45:18.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.433+0000 7f3418b6b740 10 client.25687 fetch_fsmap learned FSMap version 23 2024-04-24T06:45:18.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.433+0000 7f3418b6b740 1 -- 172.21.15.5:0/3679071488 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5587a5510810 con 0x5587a55e6350 2024-04-24T06:45:18.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.434+0000 7f33f8c00640 1 -- 172.21.15.5:0/3679071488 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33fc04a070 con 0x5587a55e6350 2024-04-24T06:45:18.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.434+0000 7f33f8c00640 1 -- 172.21.15.5:0/3679071488 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f33fc031c80 con 0x5587a55e6350 2024-04-24T06:45:18.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.434+0000 7f3418b6b740 10 client.25687 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T06:45:18.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.434+0000 7f3418b6b740 10 client.25687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:45:18.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.434+0000 7f3418b6b740 20 client.25687 populate_metadata read hostname 'smithi005' 2024-04-24T06:45:18.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.434+0000 7f3418b6b740 1 --2- 172.21.15.5:0/3679071488 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5587a566f6d0 0x5587a5671af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:18.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.434+0000 7f3418b6b740 4 client.25687 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:45:18.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.434+0000 7f3418b6b740 1 -- 172.21.15.5:0/3679071488 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5587a5552a80 con 0x5587a566f6d0 2024-04-24T06:45:18.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.434+0000 7f340f400640 1 --2- 172.21.15.5:0/3679071488 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5587a566f6d0 0x5587a5671af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:18.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.435+0000 7f340f400640 1 --2- 172.21.15.5:0/3679071488 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5587a566f6d0 0x5587a5671af0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4524ca7dd042500 server_cookie=8b465d90443a1607 in_seq=0 out_seq=0 2024-04-24T06:45:18.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.435+0000 7f33f8c00640 10 client.25687 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:45:18.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.435+0000 7f33f8c00640 1 -- 172.21.15.5:0/3679071488 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5587a5552a80 con 0x5587a566f6d0 2024-04-24T06:45:18.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.435+0000 7f33f8c00640 10 client.25687 handle_command_reply: tid=0 2024-04-24T06:45:18.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.435+0000 7f3418b6b740 1 client.25687 shutdown 2024-04-24T06:45:18.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.435+0000 7f3418b6b740 1 -- 172.21.15.5:0/3679071488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587a55e6350 msgr2=0x5587a55f9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:18.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.435+0000 7f3418b6b740 1 --2- 172.21.15.5:0/3679071488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587a55e6350 0x5587a55f9120 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f33fc009230 tx=0x7f33fc0095b0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.435+0000 7f3418b6b740 1 -- 172.21.15.5:0/3679071488 shutdown_connections 2024-04-24T06:45:18.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.435+0000 7f3418b6b740 1 --2- 172.21.15.5:0/3679071488 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5587a566f6d0 0x5587a5671af0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.435+0000 7f3418b6b740 1 --2- 172.21.15.5:0/3679071488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a55f0a20 0x5587a55e9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.435+0000 7f3418b6b740 1 --2- 172.21.15.5:0/3679071488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587a55e6350 0x5587a55f9120 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.435+0000 7f33f8c00640 0 client.25687 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:45:18.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.435+0000 7f3418b6b740 1 --2- 172.21.15.5:0/3679071488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587a55e5a30 0x5587a55f8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.435+0000 7f3418b6b740 1 -- 172.21.15.5:0/3679071488 >> 172.21.15.5:0/3679071488 conn(0x5587a55e1d50 msgr2=0x5587a55e7a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:18.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.436+0000 7f3418b6b740 1 -- 172.21.15.5:0/3679071488 shutdown_connections 2024-04-24T06:45:18.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.436+0000 7f3418b6b740 1 -- 172.21.15.5:0/3679071488 wait complete. 2024-04-24T06:45:18.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.436+0000 7f3418b6b740 20 client.25687 trim_cache size 0 max 16384 2024-04-24T06:45:18.440 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:45:18.440 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T06:45:18.440 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-24T06:45:18.440 INFO:teuthology.orchestra.run.smithi005.stdout: "e0208b86-07ac-4827-bafa-dec018d55907": { 2024-04-24T06:45:18.440 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-24T06:45:18.440 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "e0208b86-07ac-4827-bafa-dec018d55907", 2024-04-24T06:45:18.440 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-04-24T06:45:18.441 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:45:18.441 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:45:18.441 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:45:18.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.436+0000 7f340fe00640 1 -- 172.21.15.5:0/3639702415 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f33f40a4000 msgr2=0x7f33f40a64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:18.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.436+0000 7f340fe00640 1 --2- 172.21.15.5:0/3639702415 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f33f40a4000 0x7f33f40a64c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f34000ce550 tx=0x7f340009c8d0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.436+0000 7f340fe00640 1 -- 172.21.15.5:0/3639702415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3410100f60 msgr2=0x7f34100fcf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:18.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.436+0000 7f340fe00640 1 --2- 172.21.15.5:0/3639702415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3410100f60 0x7f34100fcf30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f340000e9f0 tx=0x7f340000eeb0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.437+0000 7f340fe00640 1 -- 172.21.15.5:0/3639702415 shutdown_connections 2024-04-24T06:45:18.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.437+0000 7f340fe00640 1 --2- 172.21.15.5:0/3639702415 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f33f40a4000 0x7f33f40a64c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.437+0000 7f340fe00640 1 --2- 172.21.15.5:0/3639702415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34100fd9b0 0x7f34101a6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.437+0000 7f340fe00640 1 --2- 172.21.15.5:0/3639702415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3410106030 0x7f34100fd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.437+0000 7f340fe00640 1 --2- 172.21.15.5:0/3639702415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3410100f60 0x7f34100fcf30 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:18.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.437+0000 7f340fe00640 1 -- 172.21.15.5:0/3639702415 >> 172.21.15.5:0/3639702415 conn(0x7f3410077ab0 msgr2=0x7f3410069540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:18.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.437+0000 7f340fe00640 1 -- 172.21.15.5:0/3639702415 shutdown_connections 2024-04-24T06:45:18.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:18.437+0000 7f340fe00640 1 -- 172.21.15.5:0/3639702415 wait complete. 2024-04-24T06:45:18.454 INFO:tasks.cephfs.filesystem:scrub status for tag:e0208b86-07ac-4827-bafa-dec018d55907 - {'path': '/', 'tag': 'e0208b86-07ac-4827-bafa-dec018d55907', 'options': 'recursive,force'} 2024-04-24T06:45:19.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:19 smithi137 ceph-mon[33017]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-24T06:45:19.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:19 smithi137 ceph-mon[33017]: scrub queued for path: / 2024-04-24T06:45:19.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:19 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:45:19.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:19 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:45:19.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:19 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:45:20.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:19 smithi005 ceph-mon[27798]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-24T06:45:20.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:19 smithi005 ceph-mon[27798]: scrub queued for path: / 2024-04-24T06:45:20.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:19 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:45:20.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:19 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:45:20.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:19 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:45:20.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:19 smithi037 ceph-mon[32864]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-24T06:45:20.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:19 smithi037 ceph-mon[32864]: scrub queued for path: / 2024-04-24T06:45:20.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:19 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:45:20.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:19 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:45:20.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:19 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:45:21.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:21 smithi137 ceph-mon[33017]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-24T06:45:22.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:21 smithi005 ceph-mon[27798]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-24T06:45:22.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:21 smithi037 ceph-mon[32864]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-24T06:45:23.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:23 smithi137 ceph-mon[33017]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-24T06:45:23.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:23 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:45:24.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:23 smithi005 ceph-mon[27798]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-24T06:45:24.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:23 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:45:24.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:23 smithi037 ceph-mon[32864]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-24T06:45:24.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:23 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:45:25.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:25 smithi137 ceph-mon[33017]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-04-24T06:45:26.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:25 smithi005 ceph-mon[27798]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-04-24T06:45:26.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:25 smithi037 ceph-mon[32864]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-04-24T06:45:27.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:27 smithi137 ceph-mon[33017]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:45:28.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:27 smithi005 ceph-mon[27798]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:45:28.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:27 smithi037 ceph-mon[32864]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:45:29.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:29 smithi137 ceph-mon[33017]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:45:30.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:29 smithi005 ceph-mon[27798]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:45:30.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:29 smithi037 ceph-mon[32864]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:45:31.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:31 smithi137 ceph-mon[33017]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:45:32.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:31 smithi005 ceph-mon[27798]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:45:32.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:31 smithi037 ceph-mon[32864]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:45:33.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:33 smithi137 ceph-mon[33017]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:45:34.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:33 smithi005 ceph-mon[27798]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:45:34.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:33 smithi037 ceph-mon[32864]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:45:35.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:35 smithi137 ceph-mon[33017]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 9 op/s 2024-04-24T06:45:35.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:35 smithi137 ceph-mon[33017]: mgrmap e27: x(active, since 92s), standbys: y, z 2024-04-24T06:45:36.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:35 smithi005 ceph-mon[27798]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 9 op/s 2024-04-24T06:45:36.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:35 smithi005 ceph-mon[27798]: mgrmap e27: x(active, since 92s), standbys: y, z 2024-04-24T06:45:36.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:35 smithi037 ceph-mon[32864]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 9 op/s 2024-04-24T06:45:36.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:35 smithi037 ceph-mon[32864]: mgrmap e27: x(active, since 92s), standbys: y, z 2024-04-24T06:45:37.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:37 smithi137 ceph-mon[33017]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 853 B/s wr, 6 op/s 2024-04-24T06:45:38.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:37 smithi005 ceph-mon[27798]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 853 B/s wr, 6 op/s 2024-04-24T06:45:38.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:37 smithi037 ceph-mon[32864]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 853 B/s wr, 6 op/s 2024-04-24T06:45:39.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:39 smithi137 ceph-mon[33017]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 853 B/s wr, 8 op/s 2024-04-24T06:45:40.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:39 smithi005 ceph-mon[27798]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 853 B/s wr, 8 op/s 2024-04-24T06:45:40.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:39 smithi037 ceph-mon[32864]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 853 B/s wr, 8 op/s 2024-04-24T06:45:41.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:41 smithi137 ceph-mon[33017]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 853 B/s wr, 6 op/s 2024-04-24T06:45:42.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:41 smithi005 ceph-mon[27798]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 853 B/s wr, 6 op/s 2024-04-24T06:45:42.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:41 smithi037 ceph-mon[32864]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 853 B/s wr, 6 op/s 2024-04-24T06:45:43.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:43 smithi137 ceph-mon[33017]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 853 B/s wr, 6 op/s 2024-04-24T06:45:44.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:43 smithi005 ceph-mon[27798]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 853 B/s wr, 6 op/s 2024-04-24T06:45:44.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:43 smithi037 ceph-mon[32864]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 853 B/s wr, 6 op/s 2024-04-24T06:45:44.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:44 smithi137 ceph-mon[33017]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 853 B/s wr, 8 op/s 2024-04-24T06:45:45.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:44 smithi005 ceph-mon[27798]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 853 B/s wr, 8 op/s 2024-04-24T06:45:45.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:44 smithi037 ceph-mon[32864]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 853 B/s wr, 8 op/s 2024-04-24T06:45:47.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:47 smithi137 ceph-mon[33017]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:45:47.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:47 smithi005 ceph-mon[27798]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:45:47.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:47 smithi037 ceph-mon[32864]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:45:48.455 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T06:45:48.455 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T06:45:48.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.660+0000 7f21da400640 1 -- 172.21.15.5:0/3741730065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21d4102a40 msgr2=0x7f21d4106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:48.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.660+0000 7f21da400640 1 --2- 172.21.15.5:0/3741730065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21d4102a40 0x7f21d4106a80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f21c0009900 tx=0x7f21c002a3f0 comp rx=0 tx=0).stop 2024-04-24T06:45:48.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.660+0000 7f21da400640 1 -- 172.21.15.5:0/3741730065 shutdown_connections 2024-04-24T06:45:48.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.660+0000 7f21da400640 1 --2- 172.21.15.5:0/3741730065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21d4107160 0x7f21d410ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:48.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.660+0000 7f21da400640 1 --2- 172.21.15.5:0/3741730065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21d4102a40 0x7f21d4106a80 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:48.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.660+0000 7f21da400640 1 --2- 172.21.15.5:0/3741730065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f21d4102090 0x7f21d4102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:48.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.660+0000 7f21da400640 1 -- 172.21.15.5:0/3741730065 >> 172.21.15.5:0/3741730065 conn(0x7f21d40fd910 msgr2=0x7f21d40ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:48.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.661+0000 7f21da400640 1 -- 172.21.15.5:0/3741730065 shutdown_connections 2024-04-24T06:45:48.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.661+0000 7f21da400640 1 -- 172.21.15.5:0/3741730065 wait complete. 2024-04-24T06:45:48.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.662+0000 7f21da400640 1 Processor -- start 2024-04-24T06:45:48.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.662+0000 7f21da400640 1 -- start start 2024-04-24T06:45:48.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.662+0000 7f21da400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f21d4102090 0x7f21d41a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:48.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.662+0000 7f21da400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21d4107160 0x7f21d41a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:48.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.662+0000 7f21da400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21d419a790 0x7f21d419ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:48.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.662+0000 7f21da400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f21d4112240 con 0x7f21d4107160 2024-04-24T06:45:48.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.662+0000 7f21da400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f21d41120c0 con 0x7f21d419a790 2024-04-24T06:45:48.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.662+0000 7f21da400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f21d41123c0 con 0x7f21d4102090 2024-04-24T06:45:48.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.662+0000 7f21d9000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f21d4102090 0x7f21d41a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:48.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.663+0000 7f21d3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21d4107160 0x7f21d41a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:48.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.663+0000 7f21d9000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f21d4102090 0x7f21d41a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:59648/0 (socket says 172.21.15.5:59648) 2024-04-24T06:45:48.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.663+0000 7f21d9000640 1 -- 172.21.15.5:0/2939128687 learned_addr learned my addr 172.21.15.5:0/2939128687 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:48.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.663+0000 7f21d9a00640 1 --2- 172.21.15.5:0/2939128687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21d419a790 0x7f21d419ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:48.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.663+0000 7f21d9000640 1 -- 172.21.15.5:0/2939128687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21d419a790 msgr2=0x7f21d419ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:48.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.663+0000 7f21d9000640 1 --2- 172.21.15.5:0/2939128687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21d419a790 0x7f21d419ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:48.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.663+0000 7f21d9000640 1 -- 172.21.15.5:0/2939128687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21d4107160 msgr2=0x7f21d41a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:48.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.663+0000 7f21d9000640 1 --2- 172.21.15.5:0/2939128687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21d4107160 0x7f21d41a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:48.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.663+0000 7f21d9000640 1 -- 172.21.15.5:0/2939128687 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21d419b320 con 0x7f21d4102090 2024-04-24T06:45:48.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.663+0000 7f21d9000640 1 --2- 172.21.15.5:0/2939128687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f21d4102090 0x7f21d41a0620 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f21c800e9f0 tx=0x7f21c800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:48.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.664+0000 7f21d1600640 1 -- 172.21.15.5:0/2939128687 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c800ccc0 con 0x7f21d4102090 2024-04-24T06:45:48.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.664+0000 7f21d1600640 1 -- 172.21.15.5:0/2939128687 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21c800ce60 con 0x7f21d4102090 2024-04-24T06:45:48.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.664+0000 7f21da400640 1 -- 172.21.15.5:0/2939128687 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21d419b610 con 0x7f21d4102090 2024-04-24T06:45:48.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.664+0000 7f21d1600640 1 -- 172.21.15.5:0/2939128687 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c8010690 con 0x7f21d4102090 2024-04-24T06:45:48.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.664+0000 7f21da400640 1 -- 172.21.15.5:0/2939128687 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21d41a7770 con 0x7f21d4102090 2024-04-24T06:45:48.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.667+0000 7f21de9ab740 1 Processor -- start 2024-04-24T06:45:48.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.667+0000 7f21de9ab740 1 -- start start 2024-04-24T06:45:48.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.667+0000 7f21d1600640 1 -- 172.21.15.5:0/2939128687 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f21c80046c0 con 0x7f21d4102090 2024-04-24T06:45:48.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.667+0000 7f21de9ab740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559859266a30 0x559859266e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:48.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.667+0000 7f21de9ab740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559859267350 0x5598592714e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:48.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.667+0000 7f21de9ab740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559859271a20 0x559859273e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:48.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.667+0000 7f21de9ab740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5598591cc1f0 con 0x559859271a20 2024-04-24T06:45:48.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.667+0000 7f21de9ab740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559859172350 con 0x559859266a30 2024-04-24T06:45:48.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.667+0000 7f21de9ab740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5598591c81b0 con 0x559859267350 2024-04-24T06:45:48.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.667+0000 7f21d3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559859271a20 0x559859273e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:48.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.667+0000 7f21d3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559859271a20 0x559859273e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52466/0 (socket says 172.21.15.5:52466) 2024-04-24T06:45:48.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.667+0000 7f21d3e00640 1 -- 172.21.15.5:0/1374388445 learned_addr learned my addr 172.21.15.5:0/1374388445 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:48.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.668+0000 7f21d9000640 1 --2- 172.21.15.5:0/1374388445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559859267350 0x5598592714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:48.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.668+0000 7f21d9a00640 1 --2- 172.21.15.5:0/1374388445 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559859266a30 0x559859266e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:48.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.668+0000 7f21d3e00640 1 -- 172.21.15.5:0/1374388445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559859267350 msgr2=0x5598592714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:48.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.668+0000 7f21d3e00640 1 --2- 172.21.15.5:0/1374388445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559859267350 0x5598592714e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:48.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.668+0000 7f21d3e00640 1 -- 172.21.15.5:0/1374388445 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559859266a30 msgr2=0x559859266e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:48.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.668+0000 7f21d3e00640 1 --2- 172.21.15.5:0/1374388445 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559859266a30 0x559859266e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:48.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.668+0000 7f21d3e00640 1 -- 172.21.15.5:0/1374388445 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598591c6f50 con 0x559859271a20 2024-04-24T06:45:48.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.668+0000 7f21d9000640 1 --2- 172.21.15.5:0/1374388445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559859267350 0x5598592714e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:45:48.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.668+0000 7f21d3e00640 1 --2- 172.21.15.5:0/1374388445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559859271a20 0x559859273e10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f21c0009900 tx=0x7f21c0005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5418541bc21400f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.669+0000 7f21c6000640 1 -- 172.21.15.5:0/1374388445 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c0035040 con 0x559859271a20 2024-04-24T06:45:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.669+0000 7f21c6000640 1 -- 172.21.15.5:0/1374388445 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21c0005520 con 0x559859271a20 2024-04-24T06:45:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.669+0000 7f21c6000640 1 -- 172.21.15.5:0/1374388445 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c00035f0 con 0x559859271a20 2024-04-24T06:45:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.669+0000 7f21d1600640 1 --2- 172.21.15.5:0/2939128687 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f21bc0a3d20 0x7f21bc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.669+0000 7f21d1600640 1 -- 172.21.15.5:0/2939128687 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f21c80d0bc0 con 0x7f21d4102090 2024-04-24T06:45:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.669+0000 7f21d9a00640 1 --2- 172.21.15.5:0/2939128687 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f21bc0a3d20 0x7f21bc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.669+0000 7f21de9ab740 1 -- 172.21.15.5:0/1374388445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559859271a20 msgr2=0x559859273e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.669+0000 7f21de9ab740 1 --2- 172.21.15.5:0/1374388445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559859271a20 0x559859273e10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f21c0009900 tx=0x7f21c0005de0 comp rx=0 tx=0).stop 2024-04-24T06:45:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.670+0000 7f21de9ab740 1 -- 172.21.15.5:0/1374388445 shutdown_connections 2024-04-24T06:45:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.670+0000 7f21de9ab740 1 --2- 172.21.15.5:0/1374388445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559859271a20 0x559859273e10 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.670+0000 7f21de9ab740 1 --2- 172.21.15.5:0/1374388445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559859267350 0x5598592714e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.670+0000 7f21de9ab740 1 --2- 172.21.15.5:0/1374388445 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559859266a30 0x559859266e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:48.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.670+0000 7f21de9ab740 1 -- 172.21.15.5:0/1374388445 >> 172.21.15.5:0/1374388445 conn(0x559859262d50 msgr2=0x559859263130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:48.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.670+0000 7f21de9ab740 1 -- 172.21.15.5:0/1374388445 shutdown_connections 2024-04-24T06:45:48.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.670+0000 7f21d9a00640 1 --2- 172.21.15.5:0/2939128687 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f21bc0a3d20 0x7f21bc0a61e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f21cc005fd0 tx=0x7f21cc009500 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:48.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.670+0000 7f21de9ab740 1 -- 172.21.15.5:0/1374388445 wait complete. 2024-04-24T06:45:48.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.670+0000 7f21de9ab740 1 -- start start 2024-04-24T06:45:48.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.670+0000 7f21de9ab740 1 Processor -- start 2024-04-24T06:45:48.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.671+0000 7f21de9ab740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559859266a30 0x55985926e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:48.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.671+0000 7f21de9ab740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559859267350 0x55985926eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:48.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.671+0000 7f21de9ab740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559859271a20 0x55985926f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:48.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.671+0000 7f21de9ab740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5598592747b0 con 0x559859267350 2024-04-24T06:45:48.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.671+0000 7f21de9ab740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559859185690 con 0x559859266a30 2024-04-24T06:45:48.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.671+0000 7f21de9ab740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559859274930 con 0x559859271a20 2024-04-24T06:45:48.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.671+0000 7f21de9ab740 10 client.0 _ll_register_callbacks cb 0x5598591c1650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:45:48.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.671+0000 7f21d3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559859267350 0x55985926eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:48.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.671+0000 7f21d9000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559859266a30 0x55985926e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:48.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.671+0000 7f21d3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559859267350 0x55985926eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52482/0 (socket says 172.21.15.5:52482) 2024-04-24T06:45:48.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.671+0000 7f21d3e00640 1 -- 172.21.15.5:0/3207690228 learned_addr learned my addr 172.21.15.5:0/3207690228 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:48.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.671+0000 7f21d9a00640 1 --2- 172.21.15.5:0/3207690228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559859271a20 0x55985926f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:48.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.672+0000 7f21d9000640 1 -- 172.21.15.5:0/3207690228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559859271a20 msgr2=0x55985926f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:48.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.672+0000 7f21d9000640 1 --2- 172.21.15.5:0/3207690228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559859271a20 0x55985926f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:48.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.672+0000 7f21d9000640 1 -- 172.21.15.5:0/3207690228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559859267350 msgr2=0x55985926eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:48.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.672+0000 7f21d9000640 1 --2- 172.21.15.5:0/3207690228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559859267350 0x55985926eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:48.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.672+0000 7f21d9000640 1 -- 172.21.15.5:0/3207690228 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559859184920 con 0x559859266a30 2024-04-24T06:45:48.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.672+0000 7f21d9000640 1 --2- 172.21.15.5:0/3207690228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559859266a30 0x55985926e990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f21c80ce2f0 tx=0x7f21c809c770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3574f98a4f850a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:48.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.672+0000 7f21baa00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:45:48.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.672+0000 7f21baa00640 1 -- 172.21.15.5:0/3207690228 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c80a0f20 con 0x559859266a30 2024-04-24T06:45:48.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.672+0000 7f21baa00640 1 -- 172.21.15.5:0/3207690228 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f21c80a5350 con 0x559859266a30 2024-04-24T06:45:48.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.672+0000 7f21de9ab740 1 -- 172.21.15.5:0/3207690228 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21c0005520 con 0x559859266a30 2024-04-24T06:45:48.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.672+0000 7f21baa00640 1 -- 172.21.15.5:0/3207690228 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f21c80a57e0 con 0x559859266a30 2024-04-24T06:45:48.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.672+0000 7f21de9ab740 10 client.25673 fetch_fsmap learned FSMap version 23 2024-04-24T06:45:48.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.673+0000 7f21de9ab740 1 -- 172.21.15.5:0/3207690228 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x559858fa87d0 con 0x559859266a30 2024-04-24T06:45:48.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.673+0000 7f21baa00640 1 -- 172.21.15.5:0/3207690228 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21c0005520 con 0x559859266a30 2024-04-24T06:45:48.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.673+0000 7f21baa00640 1 -- 172.21.15.5:0/3207690228 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f21c80b7020 con 0x559859266a30 2024-04-24T06:45:48.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.673+0000 7f21de9ab740 10 client.25673 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T06:45:48.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.673+0000 7f21de9ab740 10 client.25673 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:45:48.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.673+0000 7f21de9ab740 20 client.25673 populate_metadata read hostname 'smithi005' 2024-04-24T06:45:48.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.673+0000 7f21de9ab740 1 --2- 172.21.15.5:0/3207690228 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x559859281dd0 0x5598592841c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:48.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.673+0000 7f21de9ab740 4 client.25673 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:45:48.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.673+0000 7f21de9ab740 1 -- 172.21.15.5:0/3207690228 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598591d9c10 con 0x559859281dd0 2024-04-24T06:45:48.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.674+0000 7f21d3e00640 1 --2- 172.21.15.5:0/3207690228 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x559859281dd0 0x5598592841c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:48.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.674+0000 7f21d3e00640 1 --2- 172.21.15.5:0/3207690228 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x559859281dd0 0x5598592841c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dc116b3a8707545 server_cookie=73318ce0276b1727 in_seq=0 out_seq=0 2024-04-24T06:45:48.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.674+0000 7f21baa00640 10 client.25673 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:45:48.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.675+0000 7f21baa00640 1 -- 172.21.15.5:0/3207690228 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5598591d9c10 con 0x559859281dd0 2024-04-24T06:45:48.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.675+0000 7f21baa00640 10 client.25673 handle_command_reply: tid=0 2024-04-24T06:45:48.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.675+0000 7f21de9ab740 1 client.25673 shutdown 2024-04-24T06:45:48.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.675+0000 7f21de9ab740 1 -- 172.21.15.5:0/3207690228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559859266a30 msgr2=0x55985926e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:48.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.675+0000 7f21de9ab740 1 --2- 172.21.15.5:0/3207690228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559859266a30 0x55985926e990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f21c80ce2f0 tx=0x7f21c809c770 comp rx=0 tx=0).stop 2024-04-24T06:45:48.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.676+0000 7f21de9ab740 1 -- 172.21.15.5:0/3207690228 shutdown_connections 2024-04-24T06:45:48.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.676+0000 7f21de9ab740 1 --2- 172.21.15.5:0/3207690228 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x559859281dd0 0x5598592841c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:48.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.676+0000 7f21de9ab740 1 --2- 172.21.15.5:0/3207690228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559859271a20 0x55985926f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:48.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.676+0000 7f21de9ab740 1 --2- 172.21.15.5:0/3207690228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559859267350 0x55985926eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:48.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.676+0000 7f21de9ab740 1 --2- 172.21.15.5:0/3207690228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559859266a30 0x55985926e990 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:48.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.676+0000 7f21baa00640 0 client.25673 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:45:48.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.676+0000 7f21de9ab740 1 -- 172.21.15.5:0/3207690228 >> 172.21.15.5:0/3207690228 conn(0x559859262d50 msgr2=0x559859261fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:48.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.676+0000 7f21de9ab740 1 -- 172.21.15.5:0/3207690228 shutdown_connections 2024-04-24T06:45:48.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.676+0000 7f21de9ab740 1 -- 172.21.15.5:0/3207690228 wait complete. 2024-04-24T06:45:48.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.676+0000 7f21de9ab740 20 client.25673 trim_cache size 0 max 16384 2024-04-24T06:45:48.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.695+0000 7f21de9ab740 1 Processor -- start 2024-04-24T06:45:48.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.695+0000 7f21de9ab740 1 -- start start 2024-04-24T06:45:48.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.695+0000 7f21de9ab740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559859266a30 0x55985927d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:48.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.695+0000 7f21de9ab740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559859267350 0x559859289a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:48.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.695+0000 7f21de9ab740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559859271a20 0x559859289fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:48.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.695+0000 7f21de9ab740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f21d41b25d0 con 0x559859271a20 2024-04-24T06:45:48.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.695+0000 7f21de9ab740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f21d41afea0 con 0x559859267350 2024-04-24T06:45:48.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.695+0000 7f21de9ab740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f21d41adcf0 con 0x559859266a30 2024-04-24T06:45:48.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.696+0000 7f21d9a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559859271a20 0x559859289fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:48.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.696+0000 7f21d9000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559859266a30 0x55985927d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:48.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.696+0000 7f21d9000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559859266a30 0x55985927d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:59680/0 (socket says 172.21.15.5:59680) 2024-04-24T06:45:48.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.696+0000 7f21d9a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559859271a20 0x559859289fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52498/0 (socket says 172.21.15.5:52498) 2024-04-24T06:45:48.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.696+0000 7f21d9000640 1 -- 172.21.15.5:0/2592825174 learned_addr learned my addr 172.21.15.5:0/2592825174 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:48.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.696+0000 7f21d3e00640 1 --2- 172.21.15.5:0/2592825174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559859267350 0x559859289a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:48.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.696+0000 7f21d9000640 1 -- 172.21.15.5:0/2592825174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559859267350 msgr2=0x559859289a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:48.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.696+0000 7f21d9000640 1 --2- 172.21.15.5:0/2592825174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559859267350 0x559859289a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:48.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.696+0000 7f21d9000640 1 -- 172.21.15.5:0/2592825174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559859271a20 msgr2=0x559859289fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:48.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.696+0000 7f21d9000640 1 --2- 172.21.15.5:0/2592825174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559859271a20 0x559859289fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:48.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.696+0000 7f21d9000640 1 -- 172.21.15.5:0/2592825174 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559859267180 con 0x559859266a30 2024-04-24T06:45:48.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.696+0000 7f21d9000640 1 --2- 172.21.15.5:0/2592825174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559859266a30 0x55985927d670 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f21c8098fd0 tx=0x7f21c80bd2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be8efeba41b1e211 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:48.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.696+0000 7f21c5600640 1 -- 172.21.15.5:0/2592825174 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c80a0f20 con 0x559859266a30 2024-04-24T06:45:48.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.697+0000 7f21c5600640 1 -- 172.21.15.5:0/2592825174 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21c809bc70 con 0x559859266a30 2024-04-24T06:45:48.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.697+0000 7f21c5600640 1 -- 172.21.15.5:0/2592825174 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c80a5350 con 0x559859266a30 2024-04-24T06:45:48.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.697+0000 7f21de9ab740 1 -- 172.21.15.5:0/2592825174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559859266a30 msgr2=0x55985927d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:48.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.697+0000 7f21de9ab740 1 --2- 172.21.15.5:0/2592825174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559859266a30 0x55985927d670 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f21c8098fd0 tx=0x7f21c80bd2d0 comp rx=0 tx=0).stop 2024-04-24T06:45:48.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.697+0000 7f21de9ab740 1 -- 172.21.15.5:0/2592825174 shutdown_connections 2024-04-24T06:45:48.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.697+0000 7f21de9ab740 1 --2- 172.21.15.5:0/2592825174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559859271a20 0x559859289fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:48.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.697+0000 7f21de9ab740 1 --2- 172.21.15.5:0/2592825174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559859267350 0x559859289a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:48.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.697+0000 7f21de9ab740 1 --2- 172.21.15.5:0/2592825174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559859266a30 0x55985927d670 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:48.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.697+0000 7f21de9ab740 1 -- 172.21.15.5:0/2592825174 >> 172.21.15.5:0/2592825174 conn(0x559859262d50 msgr2=0x55985926f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:48.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.697+0000 7f21de9ab740 1 -- 172.21.15.5:0/2592825174 shutdown_connections 2024-04-24T06:45:48.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.698+0000 7f21de9ab740 1 -- 172.21.15.5:0/2592825174 wait complete. 2024-04-24T06:45:48.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.698+0000 7f21de9ab740 1 -- start start 2024-04-24T06:45:48.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.698+0000 7f21de9ab740 1 Processor -- start 2024-04-24T06:45:48.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.699+0000 7f21de9ab740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559859266a30 0x559859275580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:48.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.699+0000 7f21de9ab740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559859267350 0x559859275ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:48.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.699+0000 7f21de9ab740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559859271a20 0x559859276000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:48.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.699+0000 7f21de9ab740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f21d41ae360 con 0x559859267350 2024-04-24T06:45:48.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.699+0000 7f21de9ab740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f21d41af7d0 con 0x559859266a30 2024-04-24T06:45:48.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.699+0000 7f21de9ab740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f21d41aea30 con 0x559859271a20 2024-04-24T06:45:48.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.699+0000 7f21de9ab740 10 client.0 _ll_register_callbacks cb 0x7f21c8098e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:45:48.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.699+0000 7f21d3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559859267350 0x559859275ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:48.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.699+0000 7f21d9a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559859271a20 0x559859276000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:48.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.699+0000 7f21d9a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559859271a20 0x559859276000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:59686/0 (socket says 172.21.15.5:59686) 2024-04-24T06:45:48.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.699+0000 7f21d9a00640 1 -- 172.21.15.5:0/944055015 learned_addr learned my addr 172.21.15.5:0/944055015 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:48.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.699+0000 7f21d9000640 1 --2- 172.21.15.5:0/944055015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559859266a30 0x559859275580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:48.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.699+0000 7f21d9a00640 1 -- 172.21.15.5:0/944055015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559859266a30 msgr2=0x559859275580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:48.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.699+0000 7f21d9a00640 1 --2- 172.21.15.5:0/944055015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559859266a30 0x559859275580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:48.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.699+0000 7f21d9a00640 1 -- 172.21.15.5:0/944055015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559859267350 msgr2=0x559859275ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:48.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.699+0000 7f21d9a00640 1 --2- 172.21.15.5:0/944055015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559859267350 0x559859275ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:48.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.699+0000 7f21d9a00640 1 -- 172.21.15.5:0/944055015 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559858ede3f0 con 0x559859271a20 2024-04-24T06:45:48.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.700+0000 7f21d9a00640 1 --2- 172.21.15.5:0/944055015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559859271a20 0x559859276000 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00fc50 tx=0x7f21cc004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f853cfd62b98a85d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:48.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.700+0000 7f21baa00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:45:48.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.700+0000 7f21baa00640 1 -- 172.21.15.5:0/944055015 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21cc01db20 con 0x559859271a20 2024-04-24T06:45:48.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.700+0000 7f21de9ab740 1 -- 172.21.15.5:0/944055015 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21c809bc70 con 0x559859271a20 2024-04-24T06:45:48.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.700+0000 7f21baa00640 1 -- 172.21.15.5:0/944055015 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f21cc01dcc0 con 0x559859271a20 2024-04-24T06:45:48.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.700+0000 7f21baa00640 1 -- 172.21.15.5:0/944055015 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f21cc039050 con 0x559859271a20 2024-04-24T06:45:48.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.700+0000 7f21de9ab740 10 client.25705 fetch_fsmap learned FSMap version 23 2024-04-24T06:45:48.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.700+0000 7f21de9ab740 1 -- 172.21.15.5:0/944055015 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x559858e9e0f0 con 0x559859271a20 2024-04-24T06:45:48.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.701+0000 7f21baa00640 1 -- 172.21.15.5:0/944055015 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21c809bc70 con 0x559859271a20 2024-04-24T06:45:48.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.701+0000 7f21baa00640 1 -- 172.21.15.5:0/944055015 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f21cc0323e0 con 0x559859271a20 2024-04-24T06:45:48.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.701+0000 7f21de9ab740 10 client.25705 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T06:45:48.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.701+0000 7f21de9ab740 10 client.25705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:45:48.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.701+0000 7f21de9ab740 20 client.25705 populate_metadata read hostname 'smithi005' 2024-04-24T06:45:48.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.701+0000 7f21de9ab740 1 --2- 172.21.15.5:0/944055015 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5598592e8a50 0x5598592eae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:48.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.701+0000 7f21de9ab740 4 client.25705 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:45:48.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.701+0000 7f21de9ab740 1 -- 172.21.15.5:0/944055015 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5598591d3a80 con 0x5598592e8a50 2024-04-24T06:45:48.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.701+0000 7f21d9000640 1 --2- 172.21.15.5:0/944055015 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5598592e8a50 0x5598592eae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:48.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.702+0000 7f21d9000640 1 --2- 172.21.15.5:0/944055015 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5598592e8a50 0x5598592eae70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=799b015c78827f38 server_cookie=ec9c900702f8fbdd in_seq=0 out_seq=0 2024-04-24T06:45:48.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.702+0000 7f21baa00640 10 client.25705 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:45:48.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.702+0000 7f21baa00640 1 -- 172.21.15.5:0/944055015 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5598591d3a80 con 0x5598592e8a50 2024-04-24T06:45:48.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.702+0000 7f21baa00640 10 client.25705 handle_command_reply: tid=0 2024-04-24T06:45:48.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.702+0000 7f21de9ab740 1 client.25705 shutdown 2024-04-24T06:45:48.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.702+0000 7f21de9ab740 1 -- 172.21.15.5:0/944055015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559859271a20 msgr2=0x559859276000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:48.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.702+0000 7f21de9ab740 1 --2- 172.21.15.5:0/944055015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559859271a20 0x559859276000 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00fc50 tx=0x7f21cc004ab0 comp rx=0 tx=0).stop 2024-04-24T06:45:48.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.702+0000 7f21de9ab740 1 -- 172.21.15.5:0/944055015 shutdown_connections 2024-04-24T06:45:48.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.702+0000 7f21de9ab740 1 --2- 172.21.15.5:0/944055015 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5598592e8a50 0x5598592eae70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:48.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.702+0000 7f21de9ab740 1 --2- 172.21.15.5:0/944055015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559859271a20 0x559859276000 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:48.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.702+0000 7f21de9ab740 1 --2- 172.21.15.5:0/944055015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559859267350 0x559859275ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:48.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.702+0000 7f21baa00640 0 client.25705 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:45:48.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.702+0000 7f21de9ab740 1 --2- 172.21.15.5:0/944055015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559859266a30 0x559859275580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:48.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.702+0000 7f21de9ab740 1 -- 172.21.15.5:0/944055015 >> 172.21.15.5:0/944055015 conn(0x559859262d50 msgr2=0x559859268180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:48.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.703+0000 7f21de9ab740 1 -- 172.21.15.5:0/944055015 shutdown_connections 2024-04-24T06:45:48.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.703+0000 7f21de9ab740 1 -- 172.21.15.5:0/944055015 wait complete. 2024-04-24T06:45:48.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.703+0000 7f21de9ab740 20 client.25705 trim_cache size 0 max 16384 2024-04-24T06:45:48.708 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:45:48.708 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-24T06:45:48.708 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-24T06:45:48.708 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:45:48.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.703+0000 7f21da400640 1 -- 172.21.15.5:0/2939128687 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f21bc0a3d20 msgr2=0x7f21bc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:48.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.703+0000 7f21da400640 1 --2- 172.21.15.5:0/2939128687 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f21bc0a3d20 0x7f21bc0a61e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f21cc005fd0 tx=0x7f21cc009500 comp rx=0 tx=0).stop 2024-04-24T06:45:48.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.703+0000 7f21da400640 1 -- 172.21.15.5:0/2939128687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f21d4102090 msgr2=0x7f21d41a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:48.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.703+0000 7f21da400640 1 --2- 172.21.15.5:0/2939128687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f21d4102090 0x7f21d41a0620 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f21c800e9f0 tx=0x7f21c800edf0 comp rx=0 tx=0).stop 2024-04-24T06:45:48.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.703+0000 7f21da400640 1 -- 172.21.15.5:0/2939128687 shutdown_connections 2024-04-24T06:45:48.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.703+0000 7f21da400640 1 --2- 172.21.15.5:0/2939128687 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f21bc0a3d20 0x7f21bc0a61e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:48.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.703+0000 7f21da400640 1 --2- 172.21.15.5:0/2939128687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21d419a790 0x7f21d419ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:48.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.703+0000 7f21da400640 1 --2- 172.21.15.5:0/2939128687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21d4107160 0x7f21d41a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:48.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.703+0000 7f21da400640 1 --2- 172.21.15.5:0/2939128687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f21d4102090 0x7f21d41a0620 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:48.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.703+0000 7f21da400640 1 -- 172.21.15.5:0/2939128687 >> 172.21.15.5:0/2939128687 conn(0x7f21d40fd910 msgr2=0x7f21d40ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:48.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.704+0000 7f21da400640 1 -- 172.21.15.5:0/2939128687 shutdown_connections 2024-04-24T06:45:48.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.704+0000 7f21da400640 1 -- 172.21.15.5:0/2939128687 wait complete. 2024-04-24T06:45:48.721 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T06:45:48.721 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:45:48.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.915+0000 7f04dbe00640 1 -- 172.21.15.5:0/3964645813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04dc103a30 msgr2=0x7f04dc10bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:48.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.915+0000 7f04dbe00640 1 --2- 172.21.15.5:0/3964645813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04dc103a30 0x7f04dc10bf10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f04c4009900 tx=0x7f04c402a480 comp rx=0 tx=0).stop 2024-04-24T06:45:48.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.915+0000 7f04dbe00640 1 -- 172.21.15.5:0/3964645813 shutdown_connections 2024-04-24T06:45:48.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.915+0000 7f04dbe00640 1 --2- 172.21.15.5:0/3964645813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04dc10c4e0 0x7f04dc10e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:48.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.915+0000 7f04dbe00640 1 --2- 172.21.15.5:0/3964645813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04dc103a30 0x7f04dc10bf10 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:48.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.915+0000 7f04dbe00640 1 --2- 172.21.15.5:0/3964645813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04dc103110 0x7f04dc1034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:48.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.915+0000 7f04dbe00640 1 -- 172.21.15.5:0/3964645813 >> 172.21.15.5:0/3964645813 conn(0x7f04dc0fc890 msgr2=0x7f04dc0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:48.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.915+0000 7f04dbe00640 1 -- 172.21.15.5:0/3964645813 shutdown_connections 2024-04-24T06:45:48.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.915+0000 7f04dbe00640 1 -- 172.21.15.5:0/3964645813 wait complete. 2024-04-24T06:45:48.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.916+0000 7f04dbe00640 1 Processor -- start 2024-04-24T06:45:48.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.917+0000 7f04dbe00640 1 -- start start 2024-04-24T06:45:48.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.917+0000 7f04dbe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04dc103110 0x7f04dc1a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:48.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.917+0000 7f04dbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04dc103a30 0x7f04dc1a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:48.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.917+0000 7f04dbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04dc10c4e0 0x7f04dc19a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:48.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.917+0000 7f04dbe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc1122c0 con 0x7f04dc10c4e0 2024-04-24T06:45:48.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.917+0000 7f04dbe00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc112140 con 0x7f04dc103a30 2024-04-24T06:45:48.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.917+0000 7f04dbe00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc112440 con 0x7f04dc103110 2024-04-24T06:45:48.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.917+0000 7f04db400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04dc10c4e0 0x7f04dc19a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:48.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.917+0000 7f04d2000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04dc103a30 0x7f04dc1a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:48.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.917+0000 7f04db400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04dc10c4e0 0x7f04dc19a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52530/0 (socket says 172.21.15.5:52530) 2024-04-24T06:45:48.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.917+0000 7f04db400640 1 -- 172.21.15.5:0/4077283676 learned_addr learned my addr 172.21.15.5:0/4077283676 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:48.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.917+0000 7f04daa00640 1 --2- 172.21.15.5:0/4077283676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04dc103110 0x7f04dc1a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:48.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.918+0000 7f04d2000640 1 -- 172.21.15.5:0/4077283676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04dc103110 msgr2=0x7f04dc1a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:48.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.918+0000 7f04d2000640 1 --2- 172.21.15.5:0/4077283676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04dc103110 0x7f04dc1a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:48.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.918+0000 7f04d2000640 1 -- 172.21.15.5:0/4077283676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04dc10c4e0 msgr2=0x7f04dc19a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:48.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.918+0000 7f04d2000640 1 --2- 172.21.15.5:0/4077283676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04dc10c4e0 0x7f04dc19a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:48.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.918+0000 7f04d2000640 1 -- 172.21.15.5:0/4077283676 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04dc19b010 con 0x7f04dc103a30 2024-04-24T06:45:48.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.918+0000 7f04d2000640 1 --2- 172.21.15.5:0/4077283676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04dc103a30 0x7f04dc1a0c10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f04c402a9a0 tx=0x7f04c4003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:48.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.918+0000 7f04d3e00640 1 -- 172.21.15.5:0/4077283676 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c403f070 con 0x7f04dc103a30 2024-04-24T06:45:48.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.919+0000 7f04dbe00640 1 -- 172.21.15.5:0/4077283676 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04dc19b2a0 con 0x7f04dc103a30 2024-04-24T06:45:48.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.919+0000 7f04d3e00640 1 -- 172.21.15.5:0/4077283676 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04c4031c90 con 0x7f04dc103a30 2024-04-24T06:45:48.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.919+0000 7f04d3e00640 1 -- 172.21.15.5:0/4077283676 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c4035030 con 0x7f04dc103a30 2024-04-24T06:45:48.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.919+0000 7f04dbe00640 1 -- 172.21.15.5:0/4077283676 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04dc1a7520 con 0x7f04dc103a30 2024-04-24T06:45:48.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.920+0000 7f04dbe00640 1 -- 172.21.15.5:0/4077283676 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04dc069b60 con 0x7f04dc103a30 2024-04-24T06:45:48.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.922+0000 7f04d3e00640 1 -- 172.21.15.5:0/4077283676 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f04c4042070 con 0x7f04dc103a30 2024-04-24T06:45:48.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.923+0000 7f04d3e00640 1 --2- 172.21.15.5:0/4077283676 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f04c00a3d10 0x7f04c00a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:48.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.923+0000 7f04d3e00640 1 -- 172.21.15.5:0/4077283676 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f04c40f1130 con 0x7f04dc103a30 2024-04-24T06:45:48.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.924+0000 7f04daa00640 1 --2- 172.21.15.5:0/4077283676 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f04c00a3d10 0x7f04c00a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:48.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.924+0000 7f04daa00640 1 --2- 172.21.15.5:0/4077283676 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f04c00a3d10 0x7f04c00a61d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f04cc005fd0 tx=0x7f04cc009500 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:48.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:48.928+0000 7f04d3e00640 1 -- 172.21.15.5:0/4077283676 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f04c40bd130 con 0x7f04dc103a30 2024-04-24T06:45:49.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.193+0000 7f04dbe00640 1 -- 172.21.15.5:0/4077283676 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f04dc1a76f0 con 0x7f04dc103a30 2024-04-24T06:45:49.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.194+0000 7f04d3e00640 1 -- 172.21.15.5:0/4077283676 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11856 (secure 0 0 0) 0x7f04c40c1430 con 0x7f04dc103a30 2024-04-24T06:45:49.195 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:45:49.195 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1735462750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1735462750},{"type":"v1","addr":"172.21.15.5:6837","nonce":1735462750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1499469370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1499469370},{"type":"v1","addr":"172.21.15.5:6841","nonce":1499469370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3705638847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3705638847},{"type":"v1","addr":"172.21.15.37:6835","nonce":3705638847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T06:41:34.764490+0000","modified":"2024-04-24T06:44:01.711021+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24550,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6837/1390103296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1390103296},{"type":"v1","addr":"172.21.15.37:6837","nonce":1390103296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3217754215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3217754215},{"type":"v1","addr":"172.21.15.137:6835","nonce":3217754215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6837/3044222976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3044222976},{"type":"v1","addr":"172.21.15.137:6837","nonce":3044222976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.5:6839/3173201459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3173201459},{"type":"v1","addr":"172.21.15.5:6839","nonce":3173201459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6839/1889774232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1889774232},{"type":"v1","addr":"172.21.15.137:6839","nonce":1889774232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6839/3536721753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":3536721753},{"type":"v1","addr":"172.21.15.37:6839","nonce":3536721753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24569,24539,24550]},"id":1}]} 2024-04-24T06:45:49.196 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 23 2024-04-24T06:45:49.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.196+0000 7f04dbe00640 1 -- 172.21.15.5:0/4077283676 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f04c00a3d10 msgr2=0x7f04c00a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:49.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.196+0000 7f04dbe00640 1 --2- 172.21.15.5:0/4077283676 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f04c00a3d10 0x7f04c00a61d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f04cc005fd0 tx=0x7f04cc009500 comp rx=0 tx=0).stop 2024-04-24T06:45:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.196+0000 7f04dbe00640 1 -- 172.21.15.5:0/4077283676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04dc103a30 msgr2=0x7f04dc1a0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.196+0000 7f04dbe00640 1 --2- 172.21.15.5:0/4077283676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04dc103a30 0x7f04dc1a0c10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f04c402a9a0 tx=0x7f04c4003b20 comp rx=0 tx=0).stop 2024-04-24T06:45:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.196+0000 7f04dbe00640 1 -- 172.21.15.5:0/4077283676 shutdown_connections 2024-04-24T06:45:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.196+0000 7f04dbe00640 1 --2- 172.21.15.5:0/4077283676 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f04c00a3d10 0x7f04c00a61d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.196+0000 7f04dbe00640 1 --2- 172.21.15.5:0/4077283676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04dc10c4e0 0x7f04dc19a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.196+0000 7f04dbe00640 1 --2- 172.21.15.5:0/4077283676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04dc103a30 0x7f04dc1a0c10 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.196+0000 7f04dbe00640 1 --2- 172.21.15.5:0/4077283676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04dc103110 0x7f04dc1a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.196+0000 7f04dbe00640 1 -- 172.21.15.5:0/4077283676 >> 172.21.15.5:0/4077283676 conn(0x7f04dc0fc890 msgr2=0x7f04dc10d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.196+0000 7f04dbe00640 1 -- 172.21.15.5:0/4077283676 shutdown_connections 2024-04-24T06:45:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.197+0000 7f04dbe00640 1 -- 172.21.15.5:0/4077283676 wait complete. 2024-04-24T06:45:49.216 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T06:45:49.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.414+0000 7fbec2600640 1 -- 172.21.15.5:0/1970836201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbebc076530 msgr2=0x7fbebc076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:49.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.414+0000 7fbec2600640 1 --2- 172.21.15.5:0/1970836201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbebc076530 0x7fbebc076910 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbeac009900 tx=0x7fbeac02a410 comp rx=0 tx=0).stop 2024-04-24T06:45:49.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.415+0000 7fbec2600640 1 -- 172.21.15.5:0/1970836201 shutdown_connections 2024-04-24T06:45:49.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.415+0000 7fbec2600640 1 --2- 172.21.15.5:0/1970836201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbebc112e80 0x7fbebc115270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.415+0000 7fbec2600640 1 --2- 172.21.15.5:0/1970836201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbebc076e50 0x7fbebc112940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.415+0000 7fbec2600640 1 --2- 172.21.15.5:0/1970836201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbebc076530 0x7fbebc076910 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.415+0000 7fbec2600640 1 -- 172.21.15.5:0/1970836201 >> 172.21.15.5:0/1970836201 conn(0x7fbebc0ffc90 msgr2=0x7fbebc1020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:49.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.415+0000 7fbec2600640 1 -- 172.21.15.5:0/1970836201 shutdown_connections 2024-04-24T06:45:49.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.415+0000 7fbec2600640 1 -- 172.21.15.5:0/1970836201 wait complete. 2024-04-24T06:45:49.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.416+0000 7fbec2600640 1 Processor -- start 2024-04-24T06:45:49.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.416+0000 7fbec2600640 1 -- start start 2024-04-24T06:45:49.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.417+0000 7fbec2600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbebc076530 0x7fbebc19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.417+0000 7fbec2600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbebc076e50 0x7fbebc19f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.417+0000 7fbec2600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbebc112e80 0x7fbebc1a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.417+0000 7fbec2600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbebc118a30 con 0x7fbebc076e50 2024-04-24T06:45:49.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.417+0000 7fbec2600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbebc1188b0 con 0x7fbebc076530 2024-04-24T06:45:49.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.417+0000 7fbec2600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbebc118bb0 con 0x7fbebc112e80 2024-04-24T06:45:49.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.417+0000 7fbebb400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbebc076530 0x7fbebc19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.417+0000 7fbebaa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbebc076e50 0x7fbebc19f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.417+0000 7fbebb400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbebc076530 0x7fbebc19ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:37004/0 (socket says 172.21.15.5:37004) 2024-04-24T06:45:49.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.417+0000 7fbebb400640 1 -- 172.21.15.5:0/1776765871 learned_addr learned my addr 172.21.15.5:0/1776765871 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:49.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.417+0000 7fbebbe00640 1 --2- 172.21.15.5:0/1776765871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbebc112e80 0x7fbebc1a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.418+0000 7fbebb400640 1 -- 172.21.15.5:0/1776765871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbebc112e80 msgr2=0x7fbebc1a37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:49.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.418+0000 7fbebb400640 1 --2- 172.21.15.5:0/1776765871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbebc112e80 0x7fbebc1a37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.418+0000 7fbebb400640 1 -- 172.21.15.5:0/1776765871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbebc076e50 msgr2=0x7fbebc19f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:49.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.418+0000 7fbebb400640 1 --2- 172.21.15.5:0/1776765871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbebc076e50 0x7fbebc19f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.418+0000 7fbebb400640 1 -- 172.21.15.5:0/1776765871 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbebc1a3f20 con 0x7fbebc076530 2024-04-24T06:45:49.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.418+0000 7fbebb400640 1 --2- 172.21.15.5:0/1776765871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbebc076530 0x7fbebc19ef30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbeac02aa00 tx=0x7fbeac003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:49.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.418+0000 7fbeb3e00640 1 -- 172.21.15.5:0/1776765871 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac03f070 con 0x7fbebc076530 2024-04-24T06:45:49.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.418+0000 7fbeb3e00640 1 -- 172.21.15.5:0/1776765871 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbeac031c30 con 0x7fbebc076530 2024-04-24T06:45:49.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.418+0000 7fbec2600640 1 -- 172.21.15.5:0/1776765871 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbebc1a41b0 con 0x7fbebc076530 2024-04-24T06:45:49.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.418+0000 7fbeb3e00640 1 -- 172.21.15.5:0/1776765871 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac035030 con 0x7fbebc076530 2024-04-24T06:45:49.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.418+0000 7fbec2600640 1 -- 172.21.15.5:0/1776765871 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbebc1abaa0 con 0x7fbebc076530 2024-04-24T06:45:49.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.421+0000 7fbeb3e00640 1 -- 172.21.15.5:0/1776765871 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fbeac042070 con 0x7fbebc076530 2024-04-24T06:45:49.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.422+0000 7fbec53a5740 1 Processor -- start 2024-04-24T06:45:49.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.422+0000 7fbec53a5740 1 -- start start 2024-04-24T06:45:49.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.422+0000 7fbec53a5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2cdaaf950 0x55f2cdaafd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.422+0000 7fbec53a5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2cdab0270 0x55f2cdaba400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.422+0000 7fbec53a5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f2cdaba940 0x55f2cdabcd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.422+0000 7fbec53a5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f2cda151d0 con 0x55f2cdaaf950 2024-04-24T06:45:49.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.422+0000 7fbec53a5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f2cd9bb450 con 0x55f2cdaba940 2024-04-24T06:45:49.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.422+0000 7fbec53a5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f2cda11190 con 0x55f2cdab0270 2024-04-24T06:45:49.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.423+0000 7fbebb400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2cdab0270 0x55f2cdaba400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.423+0000 7fbebbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2cdaaf950 0x55f2cdaafd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.423+0000 7fbebb400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2cdab0270 0x55f2cdaba400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:59728/0 (socket says 172.21.15.5:59728) 2024-04-24T06:45:49.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.423+0000 7fbebb400640 1 -- 172.21.15.5:0/4076017182 learned_addr learned my addr 172.21.15.5:0/4076017182 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:49.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.423+0000 7fbebaa00640 1 --2- 172.21.15.5:0/4076017182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f2cdaba940 0x55f2cdabcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.423+0000 7fbebb400640 1 -- 172.21.15.5:0/4076017182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f2cdaba940 msgr2=0x55f2cdabcd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:49.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.423+0000 7fbebb400640 1 --2- 172.21.15.5:0/4076017182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f2cdaba940 0x55f2cdabcd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.423+0000 7fbebb400640 1 -- 172.21.15.5:0/4076017182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2cdaaf950 msgr2=0x55f2cdaafd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:49.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.423+0000 7fbebb400640 1 --2- 172.21.15.5:0/4076017182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2cdaaf950 0x55f2cdaafd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.423+0000 7fbebb400640 1 -- 172.21.15.5:0/4076017182 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2cda78310 con 0x55f2cdab0270 2024-04-24T06:45:49.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.423+0000 7fbebaa00640 1 --2- 172.21.15.5:0/4076017182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f2cdaba940 0x55f2cdabcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:45:49.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.423+0000 7fbebb400640 1 --2- 172.21.15.5:0/4076017182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2cdab0270 0x55f2cdaba400 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbeac0ee450 tx=0x7fbeac0f90e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a7b3f3258f8d191 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:49.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.424+0000 7fbeb0c00640 1 -- 172.21.15.5:0/4076017182 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac0f9950 con 0x55f2cdab0270 2024-04-24T06:45:49.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.424+0000 7fbeb0c00640 1 -- 172.21.15.5:0/4076017182 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbeac0fe070 con 0x55f2cdab0270 2024-04-24T06:45:49.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.424+0000 7fbeb3e00640 1 --2- 172.21.15.5:0/1776765871 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fbea00a3d20 0x7fbea00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.424+0000 7fbeb0c00640 1 -- 172.21.15.5:0/4076017182 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac101370 con 0x55f2cdab0270 2024-04-24T06:45:49.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.424+0000 7fbeb3e00640 1 -- 172.21.15.5:0/1776765871 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fbeac0f0e10 con 0x7fbebc076530 2024-04-24T06:45:49.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.424+0000 7fbebbe00640 1 --2- 172.21.15.5:0/1776765871 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fbea00a3d20 0x7fbea00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.425+0000 7fbec53a5740 1 -- 172.21.15.5:0/4076017182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2cdab0270 msgr2=0x55f2cdaba400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:49.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.425+0000 7fbec53a5740 1 --2- 172.21.15.5:0/4076017182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2cdab0270 0x55f2cdaba400 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbeac0ee450 tx=0x7fbeac0f90e0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.425+0000 7fbebbe00640 1 --2- 172.21.15.5:0/1776765871 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fbea00a3d20 0x7fbea00a61e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbebc06b380 tx=0x7fbeb4009500 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:49.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.425+0000 7fbec53a5740 1 -- 172.21.15.5:0/4076017182 shutdown_connections 2024-04-24T06:45:49.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.425+0000 7fbec53a5740 1 --2- 172.21.15.5:0/4076017182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f2cdaba940 0x55f2cdabcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.425+0000 7fbec53a5740 1 --2- 172.21.15.5:0/4076017182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2cdab0270 0x55f2cdaba400 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.425+0000 7fbec53a5740 1 --2- 172.21.15.5:0/4076017182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2cdaaf950 0x55f2cdaafd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.425+0000 7fbec53a5740 1 -- 172.21.15.5:0/4076017182 >> 172.21.15.5:0/4076017182 conn(0x55f2cdaabc70 msgr2=0x55f2cdaac050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:49.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.425+0000 7fbec53a5740 1 -- 172.21.15.5:0/4076017182 shutdown_connections 2024-04-24T06:45:49.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.425+0000 7fbec53a5740 1 -- 172.21.15.5:0/4076017182 wait complete. 2024-04-24T06:45:49.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.425+0000 7fbec53a5740 1 -- start start 2024-04-24T06:45:49.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.425+0000 7fbec53a5740 1 Processor -- start 2024-04-24T06:45:49.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.426+0000 7fbec53a5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f2cdaaf950 0x55f2cdaba3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.426+0000 7fbec53a5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2cdab0270 0x55f2cdab76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.426+0000 7fbec53a5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2cdaba940 0x55f2cdab7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.426+0000 7fbec53a5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f2cdabd4f0 con 0x55f2cdaba940 2024-04-24T06:45:49.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.426+0000 7fbec53a5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f2cd9ce810 con 0x55f2cdaaf950 2024-04-24T06:45:49.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.426+0000 7fbec53a5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f2cdabd670 con 0x55f2cdab0270 2024-04-24T06:45:49.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.426+0000 7fbec53a5740 10 client.0 _ll_register_callbacks cb 0x55f2cd99c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:45:49.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.426+0000 7fbebbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2cdaba940 0x55f2cdab7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.426+0000 7fbebb400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f2cdaaf950 0x55f2cdaba3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.426+0000 7fbebaa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2cdab0270 0x55f2cdab76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.426+0000 7fbebbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2cdaba940 0x55f2cdab7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52570/0 (socket says 172.21.15.5:52570) 2024-04-24T06:45:49.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.426+0000 7fbebb400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f2cdaaf950 0x55f2cdaba3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:37032/0 (socket says 172.21.15.5:37032) 2024-04-24T06:45:49.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.426+0000 7fbebbe00640 1 -- 172.21.15.5:0/655987776 learned_addr learned my addr 172.21.15.5:0/655987776 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:49.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.427+0000 7fbebaa00640 1 -- 172.21.15.5:0/655987776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f2cdaaf950 msgr2=0x55f2cdaba3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:49.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.427+0000 7fbebaa00640 1 --2- 172.21.15.5:0/655987776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f2cdaaf950 0x55f2cdaba3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.427+0000 7fbebaa00640 1 -- 172.21.15.5:0/655987776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2cdaba940 msgr2=0x55f2cdab7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:49.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.427+0000 7fbebaa00640 1 --2- 172.21.15.5:0/655987776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2cdaba940 0x55f2cdab7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.427+0000 7fbebaa00640 1 -- 172.21.15.5:0/655987776 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2cd9df3f0 con 0x55f2cdab0270 2024-04-24T06:45:49.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.427+0000 7fbebaa00640 1 --2- 172.21.15.5:0/655987776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2cdab0270 0x55f2cdab76a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbebc102160 tx=0x7fbea8012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87f664a162ba1231 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:49.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.427+0000 7fbea5600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:45:49.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.427+0000 7fbea5600640 1 -- 172.21.15.5:0/655987776 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea8004030 con 0x55f2cdab0270 2024-04-24T06:45:49.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.427+0000 7fbec53a5740 1 -- 172.21.15.5:0/655987776 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbeac0fe070 con 0x55f2cdab0270 2024-04-24T06:45:49.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.428+0000 7fbea5600640 1 -- 172.21.15.5:0/655987776 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fbea80041d0 con 0x55f2cdab0270 2024-04-24T06:45:49.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.428+0000 7fbea5600640 1 -- 172.21.15.5:0/655987776 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fbea80043d0 con 0x55f2cdab0270 2024-04-24T06:45:49.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.428+0000 7fbec53a5740 10 client.25723 fetch_fsmap learned FSMap version 23 2024-04-24T06:45:49.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.428+0000 7fbec53a5740 1 -- 172.21.15.5:0/655987776 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55f2cd7273f0 con 0x55f2cdab0270 2024-04-24T06:45:49.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.428+0000 7fbea5600640 1 -- 172.21.15.5:0/655987776 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbeac0fe070 con 0x55f2cdab0270 2024-04-24T06:45:49.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.429+0000 7fbea5600640 1 -- 172.21.15.5:0/655987776 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fbea8003bc0 con 0x55f2cdab0270 2024-04-24T06:45:49.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.429+0000 7fbec53a5740 10 client.25723 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T06:45:49.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.429+0000 7fbec53a5740 10 client.25723 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-24T06:45:49.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.429+0000 7fbec53a5740 20 client.25723 populate_metadata read hostname 'smithi005' 2024-04-24T06:45:49.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.429+0000 7fbec53a5740 1 --2- 172.21.15.5:0/655987776 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55f2cdaca940 0x55f2cdaccd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.429+0000 7fbec53a5740 4 client.25723 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:45:49.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.429+0000 7fbec53a5740 1 -- 172.21.15.5:0/655987776 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2cda22bf0 con 0x55f2cdaca940 2024-04-24T06:45:49.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.429+0000 7fbebb400640 1 --2- 172.21.15.5:0/655987776 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55f2cdaca940 0x55f2cdaccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.430+0000 7fbebb400640 1 --2- 172.21.15.5:0/655987776 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55f2cdaca940 0x55f2cdaccd60 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=42ab2d00dd838ea0 server_cookie=3e11d2ee1017c769 in_seq=0 out_seq=0 2024-04-24T06:45:49.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.430+0000 7fbea5600640 10 client.25723 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:45:49.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.430+0000 7fbea5600640 1 -- 172.21.15.5:0/655987776 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f2cda22bf0 con 0x55f2cdaca940 2024-04-24T06:45:49.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.430+0000 7fbea5600640 10 client.25723 handle_command_reply: tid=0 2024-04-24T06:45:49.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.431+0000 7fbec53a5740 1 client.25723 shutdown 2024-04-24T06:45:49.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.431+0000 7fbec53a5740 1 -- 172.21.15.5:0/655987776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2cdab0270 msgr2=0x55f2cdab76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:49.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.431+0000 7fbec53a5740 1 --2- 172.21.15.5:0/655987776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2cdab0270 0x55f2cdab76a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbebc102160 tx=0x7fbea8012b60 comp rx=0 tx=0).stop 2024-04-24T06:45:49.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.431+0000 7fbec53a5740 1 -- 172.21.15.5:0/655987776 shutdown_connections 2024-04-24T06:45:49.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.431+0000 7fbec53a5740 1 --2- 172.21.15.5:0/655987776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2cdaba940 0x55f2cdab7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.431+0000 7fbec53a5740 1 --2- 172.21.15.5:0/655987776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2cdab0270 0x55f2cdab76a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.431+0000 7fbec53a5740 1 --2- 172.21.15.5:0/655987776 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55f2cdaca940 0x55f2cdaccd60 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.431+0000 7fbec53a5740 1 --2- 172.21.15.5:0/655987776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f2cdaaf950 0x55f2cdaba3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.431+0000 7fbea5600640 0 client.25723 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:45:49.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.431+0000 7fbec53a5740 1 -- 172.21.15.5:0/655987776 >> 172.21.15.5:0/655987776 conn(0x55f2cdaabc70 msgr2=0x55f2cdaaadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:49.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.431+0000 7fbec53a5740 1 -- 172.21.15.5:0/655987776 shutdown_connections 2024-04-24T06:45:49.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.431+0000 7fbec53a5740 1 -- 172.21.15.5:0/655987776 wait complete. 2024-04-24T06:45:49.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.431+0000 7fbec53a5740 20 client.25723 trim_cache size 0 max 16384 2024-04-24T06:45:49.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:49 smithi137 ceph-mon[33017]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:45:49.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.446+0000 7fbec53a5740 1 Processor -- start 2024-04-24T06:45:49.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.446+0000 7fbec53a5740 1 -- start start 2024-04-24T06:45:49.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.446+0000 7fbec53a5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2cdaaf950 0x55f2cdaab570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.446+0000 7fbec53a5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2cdab0270 0x55f2cdab6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.446+0000 7fbec53a5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f2cdaba940 0x55f2cdab9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.446+0000 7fbec53a5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbebc1b6c50 con 0x55f2cdab0270 2024-04-24T06:45:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.446+0000 7fbec53a5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbebc1b4520 con 0x55f2cdaba940 2024-04-24T06:45:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.446+0000 7fbec53a5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbebc1b2370 con 0x55f2cdaaf950 2024-04-24T06:45:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.447+0000 7fbebbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f2cdaba940 0x55f2cdab9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.447+0000 7fbebbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f2cdaba940 0x55f2cdab9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:37038/0 (socket says 172.21.15.5:37038) 2024-04-24T06:45:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.447+0000 7fbebbe00640 1 -- 172.21.15.5:0/2281608339 learned_addr learned my addr 172.21.15.5:0/2281608339 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.447+0000 7fbebaa00640 1 --2- 172.21.15.5:0/2281608339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2cdab0270 0x55f2cdab6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.447+0000 7fbebb400640 1 --2- 172.21.15.5:0/2281608339 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2cdaaf950 0x55f2cdaab570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.447+0000 7fbebbe00640 1 -- 172.21.15.5:0/2281608339 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2cdaaf950 msgr2=0x55f2cdaab570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.447+0000 7fbebbe00640 1 --2- 172.21.15.5:0/2281608339 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2cdaaf950 0x55f2cdaab570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.447+0000 7fbebbe00640 1 -- 172.21.15.5:0/2281608339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2cdab0270 msgr2=0x55f2cdab6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.447+0000 7fbebbe00640 1 --2- 172.21.15.5:0/2281608339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2cdab0270 0x55f2cdab6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.447+0000 7fbebbe00640 1 -- 172.21.15.5:0/2281608339 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2cd7f17d0 con 0x55f2cdaba940 2024-04-24T06:45:49.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.448+0000 7fbebbe00640 1 --2- 172.21.15.5:0/2281608339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f2cdaba940 0x55f2cdab9ee0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbeb400fa00 tx=0x7fbeb4017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f3c46c6dfa4ac37 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:49.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.448+0000 7fbea7e00640 1 -- 172.21.15.5:0/2281608339 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb401d610 con 0x55f2cdaba940 2024-04-24T06:45:49.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.448+0000 7fbea7e00640 1 -- 172.21.15.5:0/2281608339 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbeb402e070 con 0x55f2cdaba940 2024-04-24T06:45:49.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.448+0000 7fbea7e00640 1 -- 172.21.15.5:0/2281608339 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb4029c00 con 0x55f2cdaba940 2024-04-24T06:45:49.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.449+0000 7fbec53a5740 1 -- 172.21.15.5:0/2281608339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f2cdaba940 msgr2=0x55f2cdab9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:49.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.449+0000 7fbec53a5740 1 --2- 172.21.15.5:0/2281608339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f2cdaba940 0x55f2cdab9ee0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbeb400fa00 tx=0x7fbeb4017000 comp rx=0 tx=0).stop 2024-04-24T06:45:49.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.449+0000 7fbec53a5740 1 -- 172.21.15.5:0/2281608339 shutdown_connections 2024-04-24T06:45:49.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.449+0000 7fbec53a5740 1 --2- 172.21.15.5:0/2281608339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f2cdaba940 0x55f2cdab9ee0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.449+0000 7fbec53a5740 1 --2- 172.21.15.5:0/2281608339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2cdab0270 0x55f2cdab6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.449+0000 7fbec53a5740 1 --2- 172.21.15.5:0/2281608339 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2cdaaf950 0x55f2cdaab570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.449+0000 7fbec53a5740 1 -- 172.21.15.5:0/2281608339 >> 172.21.15.5:0/2281608339 conn(0x55f2cdaabc70 msgr2=0x55f2cdaac050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:49.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.449+0000 7fbec53a5740 1 -- 172.21.15.5:0/2281608339 shutdown_connections 2024-04-24T06:45:49.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.449+0000 7fbec53a5740 1 -- 172.21.15.5:0/2281608339 wait complete. 2024-04-24T06:45:49.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.449+0000 7fbec53a5740 1 -- start start 2024-04-24T06:45:49.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.449+0000 7fbec53a5740 1 Processor -- start 2024-04-24T06:45:49.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.450+0000 7fbec53a5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2cdaaf950 0x55f2cdac2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.450+0000 7fbec53a5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2cdab0270 0x55f2cdab4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.450+0000 7fbec53a5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f2cdaba940 0x55f2cdab4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.450+0000 7fbec53a5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbebc1b29e0 con 0x55f2cdaaf950 2024-04-24T06:45:49.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.450+0000 7fbec53a5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbebc1b3e50 con 0x55f2cdaba940 2024-04-24T06:45:49.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.450+0000 7fbec53a5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbebc1b30b0 con 0x55f2cdab0270 2024-04-24T06:45:49.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.450+0000 7fbec53a5740 10 client.0 _ll_register_callbacks cb 0x7fbea8009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:45:49.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.450+0000 7fbebb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2cdaaf950 0x55f2cdac2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.450+0000 7fbebbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f2cdaba940 0x55f2cdab4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.450+0000 7fbebb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2cdaaf950 0x55f2cdac2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52592/0 (socket says 172.21.15.5:52592) 2024-04-24T06:45:49.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.450+0000 7fbebb400640 1 -- 172.21.15.5:0/1244118171 learned_addr learned my addr 172.21.15.5:0/1244118171 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:49.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.450+0000 7fbebaa00640 1 --2- 172.21.15.5:0/1244118171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2cdab0270 0x55f2cdab4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.451+0000 7fbebb400640 1 -- 172.21.15.5:0/1244118171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2cdab0270 msgr2=0x55f2cdab4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:49.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.451+0000 7fbebb400640 1 --2- 172.21.15.5:0/1244118171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2cdab0270 0x55f2cdab4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.451+0000 7fbebb400640 1 -- 172.21.15.5:0/1244118171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f2cdaba940 msgr2=0x55f2cdab4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:49.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.451+0000 7fbebb400640 1 --2- 172.21.15.5:0/1244118171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f2cdaba940 0x55f2cdab4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.451+0000 7fbebb400640 1 -- 172.21.15.5:0/1244118171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2cdabb4d0 con 0x55f2cdaaf950 2024-04-24T06:45:49.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.451+0000 7fbebbe00640 1 --2- 172.21.15.5:0/1244118171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f2cdaba940 0x55f2cdab4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:45:49.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.451+0000 7fbebb400640 1 --2- 172.21.15.5:0/1244118171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2cdaaf950 0x55f2cdac2dd0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbeac0f9210 tx=0x7fbeac009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb968f3f850a674 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:49.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.451+0000 7fbea5600640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:45:49.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.451+0000 7fbea5600640 1 -- 172.21.15.5:0/1244118171 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac049040 con 0x55f2cdaaf950 2024-04-24T06:45:49.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.451+0000 7fbea5600640 1 -- 172.21.15.5:0/1244118171 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fbeac0bae00 con 0x55f2cdaaf950 2024-04-24T06:45:49.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.451+0000 7fbec53a5740 1 -- 172.21.15.5:0/1244118171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbeb402e070 con 0x55f2cdaaf950 2024-04-24T06:45:49.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.451+0000 7fbea5600640 1 -- 172.21.15.5:0/1244118171 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fbeac0b9000 con 0x55f2cdaaf950 2024-04-24T06:45:49.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.451+0000 7fbec53a5740 10 client.15600 fetch_fsmap learned FSMap version 23 2024-04-24T06:45:49.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.451+0000 7fbec53a5740 1 -- 172.21.15.5:0/1244118171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55f2cdb310c0 con 0x55f2cdaaf950 2024-04-24T06:45:49.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.452+0000 7fbea5600640 1 -- 172.21.15.5:0/1244118171 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbeb402e070 con 0x55f2cdaaf950 2024-04-24T06:45:49.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.452+0000 7fbea5600640 1 -- 172.21.15.5:0/1244118171 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fbeac0b93d0 con 0x55f2cdaaf950 2024-04-24T06:45:49.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.452+0000 7fbec53a5740 10 client.15600 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T06:45:49.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.452+0000 7fbec53a5740 10 client.15600 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-24T06:45:49.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.452+0000 7fbec53a5740 20 client.15600 populate_metadata read hostname 'smithi005' 2024-04-24T06:45:49.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.452+0000 7fbec53a5740 1 --2- 172.21.15.5:0/1244118171 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55f2cdb315e0 0x55f2cdb33a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.452+0000 7fbec53a5740 4 client.15600 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:45:49.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.452+0000 7fbec53a5740 1 -- 172.21.15.5:0/1244118171 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f2cda1ca60 con 0x55f2cdb315e0 2024-04-24T06:45:49.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.452+0000 7fbebaa00640 1 --2- 172.21.15.5:0/1244118171 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55f2cdb315e0 0x55f2cdb33a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.453+0000 7fbebaa00640 1 --2- 172.21.15.5:0/1244118171 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55f2cdb315e0 0x55f2cdb33a00 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a4f17dc0412a4fe1 server_cookie=5eefa681137bcdea in_seq=0 out_seq=0 2024-04-24T06:45:49.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.453+0000 7fbea5600640 10 client.15600 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:45:49.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.453+0000 7fbea5600640 1 -- 172.21.15.5:0/1244118171 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f2cda1ca60 con 0x55f2cdb315e0 2024-04-24T06:45:49.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.453+0000 7fbea5600640 10 client.15600 handle_command_reply: tid=0 2024-04-24T06:45:49.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.453+0000 7fbec53a5740 1 client.15600 shutdown 2024-04-24T06:45:49.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.453+0000 7fbec53a5740 1 -- 172.21.15.5:0/1244118171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2cdaaf950 msgr2=0x55f2cdac2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:49.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.453+0000 7fbec53a5740 1 --2- 172.21.15.5:0/1244118171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2cdaaf950 0x55f2cdac2dd0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbeac0f9210 tx=0x7fbeac009260 comp rx=0 tx=0).stop 2024-04-24T06:45:49.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.453+0000 7fbec53a5740 1 -- 172.21.15.5:0/1244118171 shutdown_connections 2024-04-24T06:45:49.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.453+0000 7fbec53a5740 1 --2- 172.21.15.5:0/1244118171 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55f2cdb315e0 0x55f2cdb33a00 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.453+0000 7fbec53a5740 1 --2- 172.21.15.5:0/1244118171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f2cdaba940 0x55f2cdab4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.453+0000 7fbec53a5740 1 --2- 172.21.15.5:0/1244118171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2cdab0270 0x55f2cdab4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.453+0000 7fbec53a5740 1 --2- 172.21.15.5:0/1244118171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2cdaaf950 0x55f2cdac2dd0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.453+0000 7fbea5600640 0 client.15600 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:45:49.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.453+0000 7fbec53a5740 1 -- 172.21.15.5:0/1244118171 >> 172.21.15.5:0/1244118171 conn(0x55f2cdaabc70 msgr2=0x55f2cdab10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:49.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.454+0000 7fbec53a5740 1 -- 172.21.15.5:0/1244118171 shutdown_connections 2024-04-24T06:45:49.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.454+0000 7fbec53a5740 1 -- 172.21.15.5:0/1244118171 wait complete. 2024-04-24T06:45:49.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.454+0000 7fbec53a5740 20 client.15600 trim_cache size 0 max 16384 2024-04-24T06:45:49.460 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:45:49.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.454+0000 7fbec2600640 1 -- 172.21.15.5:0/1776765871 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fbea00a3d20 msgr2=0x7fbea00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:49.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.454+0000 7fbec2600640 1 --2- 172.21.15.5:0/1776765871 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fbea00a3d20 0x7fbea00a61e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbebc06b380 tx=0x7fbeb4009500 comp rx=0 tx=0).stop 2024-04-24T06:45:49.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.454+0000 7fbec2600640 1 -- 172.21.15.5:0/1776765871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbebc076530 msgr2=0x7fbebc19ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:49.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.454+0000 7fbec2600640 1 --2- 172.21.15.5:0/1776765871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbebc076530 0x7fbebc19ef30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbeac02aa00 tx=0x7fbeac003c10 comp rx=0 tx=0).stop 2024-04-24T06:45:49.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.455+0000 7fbec2600640 1 -- 172.21.15.5:0/1776765871 shutdown_connections 2024-04-24T06:45:49.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.455+0000 7fbec2600640 1 --2- 172.21.15.5:0/1776765871 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fbea00a3d20 0x7fbea00a61e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.455+0000 7fbec2600640 1 --2- 172.21.15.5:0/1776765871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbebc112e80 0x7fbebc1a37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.455+0000 7fbec2600640 1 --2- 172.21.15.5:0/1776765871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbebc076e50 0x7fbebc19f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.455+0000 7fbec2600640 1 --2- 172.21.15.5:0/1776765871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbebc076530 0x7fbebc19ef30 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.455+0000 7fbec2600640 1 -- 172.21.15.5:0/1776765871 >> 172.21.15.5:0/1776765871 conn(0x7fbebc0ffc90 msgr2=0x7fbebc1144d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:49.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.455+0000 7fbec2600640 1 -- 172.21.15.5:0/1776765871 shutdown_connections 2024-04-24T06:45:49.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.455+0000 7fbec2600640 1 -- 172.21.15.5:0/1776765871 wait complete. 2024-04-24T06:45:49.470 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:49 smithi005 ceph-mon[27798]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:45:49.473 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T06:45:49.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:49 smithi037 ceph-mon[32864]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:45:49.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.714+0000 7f1a1ce00640 1 -- 172.21.15.5:0/2885557992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a18103210 msgr2=0x7f1a181035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:49.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.714+0000 7f1a1ce00640 1 --2- 172.21.15.5:0/2885557992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a18103210 0x7f1a181035f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1a08009900 tx=0x7f1a0802a480 comp rx=0 tx=0).stop 2024-04-24T06:45:49.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.714+0000 7f1a1ce00640 1 -- 172.21.15.5:0/2885557992 shutdown_connections 2024-04-24T06:45:49.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.714+0000 7f1a1ce00640 1 --2- 172.21.15.5:0/2885557992 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a1810c5e0 0x7f1a1810ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.714+0000 7f1a1ce00640 1 --2- 172.21.15.5:0/2885557992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a18103b30 0x7f1a1810c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.714+0000 7f1a1ce00640 1 --2- 172.21.15.5:0/2885557992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a18103210 0x7f1a181035f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.714+0000 7f1a1ce00640 1 -- 172.21.15.5:0/2885557992 >> 172.21.15.5:0/2885557992 conn(0x7f1a18078110 msgr2=0x7f1a18078520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:49.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.715+0000 7f1a1ce00640 1 -- 172.21.15.5:0/2885557992 shutdown_connections 2024-04-24T06:45:49.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.715+0000 7f1a1ce00640 1 -- 172.21.15.5:0/2885557992 wait complete. 2024-04-24T06:45:49.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.716+0000 7f1a1ce00640 1 Processor -- start 2024-04-24T06:45:49.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.716+0000 7f1a1ce00640 1 -- start start 2024-04-24T06:45:49.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.716+0000 7f1a1ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a18103210 0x7f1a181a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.716+0000 7f1a1ce00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a18103b30 0x7f1a181a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.716+0000 7f1a1ce00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a1810c5e0 0x7f1a1819a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.716+0000 7f1a1ce00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1a181123c0 con 0x7f1a18103210 2024-04-24T06:45:49.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.716+0000 7f1a1ce00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1a18112240 con 0x7f1a18103b30 2024-04-24T06:45:49.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.716+0000 7f1a1ce00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1a18112540 con 0x7f1a1810c5e0 2024-04-24T06:45:49.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.716+0000 7f1a15e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a18103210 0x7f1a181a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.717+0000 7f1a15e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a18103210 0x7f1a181a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52618/0 (socket says 172.21.15.5:52618) 2024-04-24T06:45:49.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.717+0000 7f1a15e00640 1 -- 172.21.15.5:0/1983272624 learned_addr learned my addr 172.21.15.5:0/1983272624 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:49.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.717+0000 7f1a16800640 1 --2- 172.21.15.5:0/1983272624 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a1810c5e0 0x7f1a1819a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.717+0000 7f1a0fe00640 1 --2- 172.21.15.5:0/1983272624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a18103b30 0x7f1a181a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.717+0000 7f1a15e00640 1 -- 172.21.15.5:0/1983272624 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a1810c5e0 msgr2=0x7f1a1819a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:49.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.717+0000 7f1a15e00640 1 --2- 172.21.15.5:0/1983272624 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a1810c5e0 0x7f1a1819a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.717+0000 7f1a15e00640 1 -- 172.21.15.5:0/1983272624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a18103b30 msgr2=0x7f1a181a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:49.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.717+0000 7f1a15e00640 1 --2- 172.21.15.5:0/1983272624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a18103b30 0x7f1a181a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.717+0000 7f1a15e00640 1 -- 172.21.15.5:0/1983272624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a1819b020 con 0x7f1a18103210 2024-04-24T06:45:49.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.717+0000 7f1a16800640 1 --2- 172.21.15.5:0/1983272624 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a1810c5e0 0x7f1a1819a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:45:49.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.717+0000 7f1a15e00640 1 --2- 172.21.15.5:0/1983272624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a18103210 0x7f1a181a06f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1a08004770 tx=0x7f1a08003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:49.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.718+0000 7f1a0ea00640 1 -- 172.21.15.5:0/1983272624 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0803f070 con 0x7f1a18103210 2024-04-24T06:45:49.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.718+0000 7f1a1ce00640 1 -- 172.21.15.5:0/1983272624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a1819b2b0 con 0x7f1a18103210 2024-04-24T06:45:49.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.718+0000 7f1a0ea00640 1 -- 172.21.15.5:0/1983272624 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a08031ca0 con 0x7f1a18103210 2024-04-24T06:45:49.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.718+0000 7f1a0ea00640 1 -- 172.21.15.5:0/1983272624 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a08035030 con 0x7f1a18103210 2024-04-24T06:45:49.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.718+0000 7f1a1ce00640 1 -- 172.21.15.5:0/1983272624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a181a75b0 con 0x7f1a18103210 2024-04-24T06:45:49.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.721+0000 7f1a1fc58740 1 Processor -- start 2024-04-24T06:45:49.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.721+0000 7f1a0ea00640 1 -- 172.21.15.5:0/1983272624 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f1a08042070 con 0x7f1a18103210 2024-04-24T06:45:49.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.721+0000 7f1a1fc58740 1 -- start start 2024-04-24T06:45:49.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.721+0000 7f1a1fc58740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f92e9ef950 0x55f92e9efd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.721+0000 7f1a1fc58740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f92e9f0270 0x55f92e9fa400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.722+0000 7f1a1fc58740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f92e9fa940 0x55f92e9fcd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.722+0000 7f1a1fc58740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f92e9551d0 con 0x55f92e9ef950 2024-04-24T06:45:49.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.722+0000 7f1a1fc58740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f92e8fb450 con 0x55f92e9fa940 2024-04-24T06:45:49.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.722+0000 7f1a1fc58740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f92e951190 con 0x55f92e9f0270 2024-04-24T06:45:49.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.722+0000 7f1a16800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f92e9ef950 0x55f92e9efd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.722+0000 7f1a16800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f92e9ef950 0x55f92e9efd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52630/0 (socket says 172.21.15.5:52630) 2024-04-24T06:45:49.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.722+0000 7f1a0fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f92e9fa940 0x55f92e9fcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.722+0000 7f1a16800640 1 -- 172.21.15.5:0/1842323405 learned_addr learned my addr 172.21.15.5:0/1842323405 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:49.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.722+0000 7f1a15e00640 1 --2- 172.21.15.5:0/1842323405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f92e9f0270 0x55f92e9fa400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.722+0000 7f1a0fe00640 1 -- 172.21.15.5:0/1842323405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f92e9f0270 msgr2=0x55f92e9fa400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:49.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.722+0000 7f1a0fe00640 1 --2- 172.21.15.5:0/1842323405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f92e9f0270 0x55f92e9fa400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.722+0000 7f1a0fe00640 1 -- 172.21.15.5:0/1842323405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f92e9ef950 msgr2=0x55f92e9efd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:49.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.722+0000 7f1a0fe00640 1 --2- 172.21.15.5:0/1842323405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f92e9ef950 0x55f92e9efd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.722+0000 7f1a0fe00640 1 -- 172.21.15.5:0/1842323405 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f92e9b8310 con 0x55f92e9fa940 2024-04-24T06:45:49.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.723+0000 7f1a0fe00640 1 --2- 172.21.15.5:0/1842323405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f92e9fa940 0x55f92e9fcd30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1a1819c090 tx=0x7f1a00011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f8b5fd81056f719 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:49.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.723+0000 7f1a07400640 1 -- 172.21.15.5:0/1842323405 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a00004050 con 0x55f92e9fa940 2024-04-24T06:45:49.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.723+0000 7f1a07400640 1 -- 172.21.15.5:0/1842323405 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a00005500 con 0x55f92e9fa940 2024-04-24T06:45:49.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.723+0000 7f1a07400640 1 -- 172.21.15.5:0/1842323405 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a00004d90 con 0x55f92e9fa940 2024-04-24T06:45:49.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.723+0000 7f1a0ea00640 1 --2- 172.21.15.5:0/1983272624 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f19fc0a3d20 0x7f19fc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.724+0000 7f1a1fc58740 1 -- 172.21.15.5:0/1842323405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f92e9fa940 msgr2=0x55f92e9fcd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:49.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.724+0000 7f1a1fc58740 1 --2- 172.21.15.5:0/1842323405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f92e9fa940 0x55f92e9fcd30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1a1819c090 tx=0x7f1a00011810 comp rx=0 tx=0).stop 2024-04-24T06:45:49.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.724+0000 7f1a0ea00640 1 -- 172.21.15.5:0/1983272624 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f1a080f0e90 con 0x7f1a18103210 2024-04-24T06:45:49.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.724+0000 7f1a16800640 1 --2- 172.21.15.5:0/1983272624 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f19fc0a3d20 0x7f19fc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.724+0000 7f1a1fc58740 1 -- 172.21.15.5:0/1842323405 shutdown_connections 2024-04-24T06:45:49.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.724+0000 7f1a1fc58740 1 --2- 172.21.15.5:0/1842323405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f92e9fa940 0x55f92e9fcd30 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.724+0000 7f1a1fc58740 1 --2- 172.21.15.5:0/1842323405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f92e9f0270 0x55f92e9fa400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.724+0000 7f1a1fc58740 1 --2- 172.21.15.5:0/1842323405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f92e9ef950 0x55f92e9efd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.724+0000 7f1a1fc58740 1 -- 172.21.15.5:0/1842323405 >> 172.21.15.5:0/1842323405 conn(0x55f92e9ebc70 msgr2=0x55f92e9ec050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:49.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.724+0000 7f1a1fc58740 1 -- 172.21.15.5:0/1842323405 shutdown_connections 2024-04-24T06:45:49.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.724+0000 7f1a1fc58740 1 -- 172.21.15.5:0/1842323405 wait complete. 2024-04-24T06:45:49.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.725+0000 7f1a1fc58740 1 -- start start 2024-04-24T06:45:49.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.725+0000 7f1a16800640 1 --2- 172.21.15.5:0/1983272624 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f19fc0a3d20 0x7f19fc0a61e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1a100092f0 tx=0x7f1a10009600 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:49.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.725+0000 7f1a1fc58740 1 Processor -- start 2024-04-24T06:45:49.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.725+0000 7f1a1fc58740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f92e9ef950 0x55f92e9fa300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.725+0000 7f1a1fc58740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f92e9f0270 0x55f92e9f75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.725+0000 7f1a1fc58740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f92e9fa940 0x55f92e9f7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.725+0000 7f1a1fc58740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f92e9fd7d0 con 0x55f92e9ef950 2024-04-24T06:45:49.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.725+0000 7f1a1fc58740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f92e90e810 con 0x55f92e9fa940 2024-04-24T06:45:49.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.725+0000 7f1a1fc58740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f92e9fd950 con 0x55f92e9f0270 2024-04-24T06:45:49.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.725+0000 7f1a1fc58740 10 client.0 _ll_register_callbacks cb 0x55f92e8dc140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:45:49.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.726+0000 7f1a15e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f92e9ef950 0x55f92e9fa300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.726+0000 7f1a15e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f92e9ef950 0x55f92e9fa300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52640/0 (socket says 172.21.15.5:52640) 2024-04-24T06:45:49.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.726+0000 7f1a15e00640 1 -- 172.21.15.5:0/1478318850 learned_addr learned my addr 172.21.15.5:0/1478318850 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:49.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.726+0000 7f1a0fe00640 1 --2- 172.21.15.5:0/1478318850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f92e9f0270 0x55f92e9f75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.726+0000 7f1a16800640 1 --2- 172.21.15.5:0/1478318850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f92e9fa940 0x55f92e9f7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.726+0000 7f1a15e00640 1 -- 172.21.15.5:0/1478318850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f92e9f0270 msgr2=0x55f92e9f75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:49.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.726+0000 7f1a15e00640 1 --2- 172.21.15.5:0/1478318850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f92e9f0270 0x55f92e9f75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.726+0000 7f1a15e00640 1 -- 172.21.15.5:0/1478318850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f92e9fa940 msgr2=0x55f92e9f7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:49.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.726+0000 7f1a15e00640 1 --2- 172.21.15.5:0/1478318850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f92e9fa940 0x55f92e9f7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.726+0000 7f1a15e00640 1 -- 172.21.15.5:0/1478318850 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f92e91f3f0 con 0x55f92e9ef950 2024-04-24T06:45:49.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.726+0000 7f1a16800640 1 --2- 172.21.15.5:0/1478318850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f92e9fa940 0x55f92e9f7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:45:49.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.726+0000 7f1a15e00640 1 --2- 172.21.15.5:0/1478318850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f92e9ef950 0x55f92e9fa300 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1a08031550 tx=0x7f1a08009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c48df6a776472f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:49.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.726+0000 7f19fbe00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:45:49.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.726+0000 7f19fbe00640 1 -- 172.21.15.5:0/1478318850 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a08049050 con 0x55f92e9ef950 2024-04-24T06:45:49.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.726+0000 7f19fbe00640 1 -- 172.21.15.5:0/1478318850 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f1a0803b4f0 con 0x55f92e9ef950 2024-04-24T06:45:49.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.727+0000 7f1a1fc58740 1 -- 172.21.15.5:0/1478318850 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a00005500 con 0x55f92e9ef950 2024-04-24T06:45:49.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.727+0000 7f19fbe00640 1 -- 172.21.15.5:0/1478318850 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f1a080bc7f0 con 0x55f92e9ef950 2024-04-24T06:45:49.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.727+0000 7f1a1fc58740 10 client.15612 fetch_fsmap learned FSMap version 23 2024-04-24T06:45:49.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.727+0000 7f1a1fc58740 1 -- 172.21.15.5:0/1478318850 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55f92e6673f0 con 0x55f92e9ef950 2024-04-24T06:45:49.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.727+0000 7f19fbe00640 1 -- 172.21.15.5:0/1478318850 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a00005500 con 0x55f92e9ef950 2024-04-24T06:45:49.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.727+0000 7f19fbe00640 1 -- 172.21.15.5:0/1478318850 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f1a080eece0 con 0x55f92e9ef950 2024-04-24T06:45:49.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.727+0000 7f1a1fc58740 10 client.15612 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T06:45:49.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.727+0000 7f1a1fc58740 10 client.15612 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T06:45:49.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.727+0000 7f1a1fc58740 20 client.15612 populate_metadata read hostname 'smithi005' 2024-04-24T06:45:49.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.727+0000 7f1a1fc58740 1 --2- 172.21.15.5:0/1478318850 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55f92ea0a950 0x55f92ea0cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.727+0000 7f1a1fc58740 4 client.15612 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:45:49.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.727+0000 7f1a1fc58740 1 -- 172.21.15.5:0/1478318850 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f92e962bf0 con 0x55f92ea0a950 2024-04-24T06:45:49.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.728+0000 7f1a0fe00640 1 --2- 172.21.15.5:0/1478318850 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55f92ea0a950 0x55f92ea0cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.728+0000 7f1a0fe00640 1 --2- 172.21.15.5:0/1478318850 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55f92ea0a950 0x55f92ea0cd40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=920e69d798564027 server_cookie=ca94ad3ba3a2e5fc in_seq=0 out_seq=0 2024-04-24T06:45:49.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.728+0000 7f19fbe00640 10 client.15612 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:45:49.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.729+0000 7f19fbe00640 1 -- 172.21.15.5:0/1478318850 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f92e962bf0 con 0x55f92ea0a950 2024-04-24T06:45:49.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.729+0000 7f19fbe00640 10 client.15612 handle_command_reply: tid=0 2024-04-24T06:45:49.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.729+0000 7f1a1fc58740 1 client.15612 shutdown 2024-04-24T06:45:49.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.730+0000 7f1a1fc58740 1 -- 172.21.15.5:0/1478318850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f92e9ef950 msgr2=0x55f92e9fa300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:49.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.730+0000 7f1a1fc58740 1 --2- 172.21.15.5:0/1478318850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f92e9ef950 0x55f92e9fa300 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1a08031550 tx=0x7f1a08009620 comp rx=0 tx=0).stop 2024-04-24T06:45:49.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.730+0000 7f1a1fc58740 1 -- 172.21.15.5:0/1478318850 shutdown_connections 2024-04-24T06:45:49.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.730+0000 7f1a1fc58740 1 --2- 172.21.15.5:0/1478318850 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55f92ea0a950 0x55f92ea0cd40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.730+0000 7f1a1fc58740 1 --2- 172.21.15.5:0/1478318850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f92e9fa940 0x55f92e9f7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.730+0000 7f1a1fc58740 1 --2- 172.21.15.5:0/1478318850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f92e9f0270 0x55f92e9f75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.730+0000 7f1a1fc58740 1 --2- 172.21.15.5:0/1478318850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f92e9ef950 0x55f92e9fa300 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.730+0000 7f1a1fc58740 1 -- 172.21.15.5:0/1478318850 >> 172.21.15.5:0/1478318850 conn(0x55f92e9ebc70 msgr2=0x55f92e9f1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:49.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.730+0000 7f19fbe00640 0 client.15612 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:45:49.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.730+0000 7f1a1fc58740 1 -- 172.21.15.5:0/1478318850 shutdown_connections 2024-04-24T06:45:49.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.730+0000 7f1a1fc58740 1 -- 172.21.15.5:0/1478318850 wait complete. 2024-04-24T06:45:49.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.730+0000 7f1a1fc58740 20 client.15612 trim_cache size 0 max 16384 2024-04-24T06:45:49.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.745+0000 7f1a1fc58740 1 Processor -- start 2024-04-24T06:45:49.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.745+0000 7f1a1fc58740 1 -- start start 2024-04-24T06:45:49.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.745+0000 7f1a1fc58740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f92e9ef950 0x55f92e9f20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.745+0000 7f1a1fc58740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f92e9f0270 0x55f92e9f6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.745+0000 7f1a1fc58740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f92e9fa940 0x55f92e9f9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.745+0000 7f1a1fc58740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1a181b27d0 con 0x55f92e9ef950 2024-04-24T06:45:49.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.745+0000 7f1a1fc58740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1a181b00a0 con 0x55f92e9f0270 2024-04-24T06:45:49.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.745+0000 7f1a1fc58740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1a181adef0 con 0x55f92e9fa940 2024-04-24T06:45:49.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.746+0000 7f1a15e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f92e9ef950 0x55f92e9f20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.746+0000 7f1a15e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f92e9ef950 0x55f92e9f20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52646/0 (socket says 172.21.15.5:52646) 2024-04-24T06:45:49.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.746+0000 7f1a15e00640 1 -- 172.21.15.5:0/3065767124 learned_addr learned my addr 172.21.15.5:0/3065767124 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:49.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.746+0000 7f1a0fe00640 1 --2- 172.21.15.5:0/3065767124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f92e9f0270 0x55f92e9f6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.746+0000 7f1a16800640 1 --2- 172.21.15.5:0/3065767124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f92e9fa940 0x55f92e9f9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.746+0000 7f1a15e00640 1 -- 172.21.15.5:0/3065767124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f92e9fa940 msgr2=0x55f92e9f9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:49.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.746+0000 7f1a15e00640 1 --2- 172.21.15.5:0/3065767124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f92e9fa940 0x55f92e9f9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.746+0000 7f1a15e00640 1 -- 172.21.15.5:0/3065767124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f92e9f0270 msgr2=0x55f92e9f6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:49.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.746+0000 7f1a15e00640 1 --2- 172.21.15.5:0/3065767124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f92e9f0270 0x55f92e9f6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.746+0000 7f1a15e00640 1 -- 172.21.15.5:0/3065767124 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f92e9f7f70 con 0x55f92e9ef950 2024-04-24T06:45:49.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.746+0000 7f1a0fe00640 1 --2- 172.21.15.5:0/3065767124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f92e9f0270 0x55f92e9f6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:45:49.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.747+0000 7f1a15e00640 1 --2- 172.21.15.5:0/3065767124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f92e9ef950 0x55f92e9f20b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1a08046040 tx=0x7f1a080ee8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c01301d4bf719a6f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:49.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.747+0000 7f1a06a00640 1 -- 172.21.15.5:0/3065767124 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a08049070 con 0x55f92e9ef950 2024-04-24T06:45:49.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.747+0000 7f1a06a00640 1 -- 172.21.15.5:0/3065767124 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a080c5a50 con 0x55f92e9ef950 2024-04-24T06:45:49.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.747+0000 7f1a06a00640 1 -- 172.21.15.5:0/3065767124 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a08005d50 con 0x55f92e9ef950 2024-04-24T06:45:49.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.748+0000 7f1a1fc58740 1 -- 172.21.15.5:0/3065767124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f92e9ef950 msgr2=0x55f92e9f20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:49.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.748+0000 7f1a1fc58740 1 --2- 172.21.15.5:0/3065767124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f92e9ef950 0x55f92e9f20b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1a08046040 tx=0x7f1a080ee8b0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.748+0000 7f1a1fc58740 1 -- 172.21.15.5:0/3065767124 shutdown_connections 2024-04-24T06:45:49.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.748+0000 7f1a1fc58740 1 --2- 172.21.15.5:0/3065767124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f92e9fa940 0x55f92e9f9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.748+0000 7f1a1fc58740 1 --2- 172.21.15.5:0/3065767124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f92e9f0270 0x55f92e9f6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.748+0000 7f1a1fc58740 1 --2- 172.21.15.5:0/3065767124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f92e9ef950 0x55f92e9f20b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.748+0000 7f1a1fc58740 1 -- 172.21.15.5:0/3065767124 >> 172.21.15.5:0/3065767124 conn(0x55f92e9ebc70 msgr2=0x55f92e9ea560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:49.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.748+0000 7f1a1fc58740 1 -- 172.21.15.5:0/3065767124 shutdown_connections 2024-04-24T06:45:49.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.748+0000 7f1a1fc58740 1 -- 172.21.15.5:0/3065767124 wait complete. 2024-04-24T06:45:49.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.748+0000 7f1a1fc58740 1 -- start start 2024-04-24T06:45:49.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.749+0000 7f1a1fc58740 1 Processor -- start 2024-04-24T06:45:49.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.749+0000 7f1a1fc58740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f92e9ef950 0x55f92ea030a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.749+0000 7f1a1fc58740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f92e9f0270 0x55f92e9f4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.749+0000 7f1a1fc58740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f92e9fa940 0x55f92e9f5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.749+0000 7f1a1fc58740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1a181ae560 con 0x55f92e9fa940 2024-04-24T06:45:49.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.749+0000 7f1a1fc58740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1a181af9d0 con 0x55f92e9f0270 2024-04-24T06:45:49.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.749+0000 7f1a1fc58740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1a181aec30 con 0x55f92e9ef950 2024-04-24T06:45:49.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.749+0000 7f1a1fc58740 10 client.0 _ll_register_callbacks cb 0x7f1a080f4630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:45:49.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.749+0000 7f1a16800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f92e9fa940 0x55f92e9f5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.749+0000 7f1a16800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f92e9fa940 0x55f92e9f5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52660/0 (socket says 172.21.15.5:52660) 2024-04-24T06:45:49.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.749+0000 7f1a16800640 1 -- 172.21.15.5:0/463261893 learned_addr learned my addr 172.21.15.5:0/463261893 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:49.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.749+0000 7f1a15e00640 1 --2- 172.21.15.5:0/463261893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f92e9ef950 0x55f92ea030a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.749+0000 7f1a0fe00640 1 --2- 172.21.15.5:0/463261893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f92e9f0270 0x55f92e9f4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.750+0000 7f1a16800640 1 -- 172.21.15.5:0/463261893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f92e9ef950 msgr2=0x55f92ea030a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:49.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.750+0000 7f1a16800640 1 --2- 172.21.15.5:0/463261893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f92e9ef950 0x55f92ea030a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.750+0000 7f1a16800640 1 -- 172.21.15.5:0/463261893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f92e9f0270 msgr2=0x55f92e9f4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:49.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.750+0000 7f1a16800640 1 --2- 172.21.15.5:0/463261893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f92e9f0270 0x55f92e9f4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.750+0000 7f1a16800640 1 -- 172.21.15.5:0/463261893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f92e7317d0 con 0x55f92e9fa940 2024-04-24T06:45:49.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.750+0000 7f1a0fe00640 1 --2- 172.21.15.5:0/463261893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f92e9f0270 0x55f92e9f4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:45:49.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.750+0000 7f1a16800640 1 --2- 172.21.15.5:0/463261893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f92e9fa940 0x55f92e9f5260 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1a1001d620 tx=0x7f1a10004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd8f5ebf2913536 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:49.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.750+0000 7f19fbe00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:45:49.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.751+0000 7f19fbe00640 1 -- 172.21.15.5:0/463261893 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a10004d20 con 0x55f92e9fa940 2024-04-24T06:45:49.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.751+0000 7f19fbe00640 1 -- 172.21.15.5:0/463261893 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f1a1001c660 con 0x55f92e9fa940 2024-04-24T06:45:49.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.751+0000 7f1a1fc58740 1 -- 172.21.15.5:0/463261893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a080c5a50 con 0x55f92e9fa940 2024-04-24T06:45:49.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.751+0000 7f19fbe00640 1 -- 172.21.15.5:0/463261893 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f1a1001c860 con 0x55f92e9fa940 2024-04-24T06:45:49.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.751+0000 7f1a1fc58740 10 client.15618 fetch_fsmap learned FSMap version 23 2024-04-24T06:45:49.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.751+0000 7f1a1fc58740 1 -- 172.21.15.5:0/463261893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55f92ea71200 con 0x55f92e9fa940 2024-04-24T06:45:49.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.751+0000 7f19fbe00640 1 -- 172.21.15.5:0/463261893 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a080c5a50 con 0x55f92e9fa940 2024-04-24T06:45:49.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.751+0000 7f19fbe00640 1 -- 172.21.15.5:0/463261893 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f1a10027070 con 0x55f92e9fa940 2024-04-24T06:45:49.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.751+0000 7f1a1fc58740 10 client.15618 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T06:45:49.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.751+0000 7f1a1fc58740 10 client.15618 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T06:45:49.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.751+0000 7f1a1fc58740 20 client.15618 populate_metadata read hostname 'smithi005' 2024-04-24T06:45:49.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.751+0000 7f1a1fc58740 1 --2- 172.21.15.5:0/463261893 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55f92ea71a00 0x55f92ea73e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.751+0000 7f1a1fc58740 4 client.15618 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:45:49.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.751+0000 7f1a1fc58740 1 -- 172.21.15.5:0/463261893 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f92e95ca60 con 0x55f92ea71a00 2024-04-24T06:45:49.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.752+0000 7f1a15e00640 1 --2- 172.21.15.5:0/463261893 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55f92ea71a00 0x55f92ea73e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.753+0000 7f19fbe00640 10 client.15618 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:45:49.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.753+0000 7f1a15e00640 1 --2- 172.21.15.5:0/463261893 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55f92ea71a00 0x55f92ea73e20 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fd941a7c96617561 server_cookie=be9be5b75fa01e40 in_seq=0 out_seq=0 2024-04-24T06:45:49.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.753+0000 7f19fbe00640 1 -- 172.21.15.5:0/463261893 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f92e95ca60 con 0x55f92ea71a00 2024-04-24T06:45:49.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.753+0000 7f19fbe00640 10 client.15618 handle_command_reply: tid=0 2024-04-24T06:45:49.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.753+0000 7f1a1fc58740 1 client.15618 shutdown 2024-04-24T06:45:49.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.753+0000 7f1a1fc58740 1 -- 172.21.15.5:0/463261893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f92e9fa940 msgr2=0x55f92e9f5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:49.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.753+0000 7f1a1fc58740 1 --2- 172.21.15.5:0/463261893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f92e9fa940 0x55f92e9f5260 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1a1001d620 tx=0x7f1a10004a40 comp rx=0 tx=0).stop 2024-04-24T06:45:49.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.753+0000 7f1a1fc58740 1 -- 172.21.15.5:0/463261893 shutdown_connections 2024-04-24T06:45:49.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.753+0000 7f1a1fc58740 1 --2- 172.21.15.5:0/463261893 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55f92ea71a00 0x55f92ea73e20 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.753+0000 7f1a1fc58740 1 --2- 172.21.15.5:0/463261893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f92e9fa940 0x55f92e9f5260 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.753+0000 7f1a1fc58740 1 --2- 172.21.15.5:0/463261893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f92e9f0270 0x55f92e9f4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.753+0000 7f1a1fc58740 1 --2- 172.21.15.5:0/463261893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f92e9ef950 0x55f92ea030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.753+0000 7f19fbe00640 0 client.15618 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:45:49.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.753+0000 7f1a1fc58740 1 -- 172.21.15.5:0/463261893 >> 172.21.15.5:0/463261893 conn(0x55f92e9ebc70 msgr2=0x55f92e9ea560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:49.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.753+0000 7f1a1fc58740 1 -- 172.21.15.5:0/463261893 shutdown_connections 2024-04-24T06:45:49.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.753+0000 7f1a1fc58740 1 -- 172.21.15.5:0/463261893 wait complete. 2024-04-24T06:45:49.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.754+0000 7f1a1fc58740 20 client.15618 trim_cache size 0 max 16384 2024-04-24T06:45:49.760 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:45:49.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.754+0000 7f1a1ce00640 1 -- 172.21.15.5:0/1983272624 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f19fc0a3d20 msgr2=0x7f19fc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:49.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.754+0000 7f1a1ce00640 1 --2- 172.21.15.5:0/1983272624 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f19fc0a3d20 0x7f19fc0a61e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1a100092f0 tx=0x7f1a10009600 comp rx=0 tx=0).stop 2024-04-24T06:45:49.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.754+0000 7f1a1ce00640 1 -- 172.21.15.5:0/1983272624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a18103210 msgr2=0x7f1a181a06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:49.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.754+0000 7f1a1ce00640 1 --2- 172.21.15.5:0/1983272624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a18103210 0x7f1a181a06f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1a08004770 tx=0x7f1a08003c10 comp rx=0 tx=0).stop 2024-04-24T06:45:49.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.754+0000 7f1a1ce00640 1 -- 172.21.15.5:0/1983272624 shutdown_connections 2024-04-24T06:45:49.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.754+0000 7f1a1ce00640 1 --2- 172.21.15.5:0/1983272624 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f19fc0a3d20 0x7f19fc0a61e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.754+0000 7f1a1ce00640 1 --2- 172.21.15.5:0/1983272624 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a1810c5e0 0x7f1a1819a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.754+0000 7f1a1ce00640 1 --2- 172.21.15.5:0/1983272624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a18103b30 0x7f1a181a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.754+0000 7f1a1ce00640 1 --2- 172.21.15.5:0/1983272624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a18103210 0x7f1a181a06f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.754+0000 7f1a1ce00640 1 -- 172.21.15.5:0/1983272624 >> 172.21.15.5:0/1983272624 conn(0x7f1a18078110 msgr2=0x7f1a18108420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:49.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.754+0000 7f1a1ce00640 1 -- 172.21.15.5:0/1983272624 shutdown_connections 2024-04-24T06:45:49.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.755+0000 7f1a1ce00640 1 -- 172.21.15.5:0/1983272624 wait complete. 2024-04-24T06:45:49.772 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T06:45:49.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.965+0000 7fc052800640 1 -- 172.21.15.5:0/1089973752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc04c10c400 msgr2=0x7fc04c10e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:49.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.965+0000 7fc052800640 1 --2- 172.21.15.5:0/1089973752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc04c10c400 0x7fc04c10e870 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc044009be0 tx=0x7fc04402a750 comp rx=0 tx=0).stop 2024-04-24T06:45:49.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.965+0000 7fc052800640 1 -- 172.21.15.5:0/1089973752 shutdown_connections 2024-04-24T06:45:49.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.965+0000 7fc052800640 1 --2- 172.21.15.5:0/1089973752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc04c10c400 0x7fc04c10e870 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.965+0000 7fc052800640 1 --2- 172.21.15.5:0/1089973752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc04c103900 0x7fc04c10bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.965+0000 7fc052800640 1 --2- 172.21.15.5:0/1089973752 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc04c102fe0 0x7fc04c1033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.965+0000 7fc052800640 1 -- 172.21.15.5:0/1089973752 >> 172.21.15.5:0/1089973752 conn(0x7fc04c077ab0 msgr2=0x7fc04c0fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:49.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.965+0000 7fc052800640 1 -- 172.21.15.5:0/1089973752 shutdown_connections 2024-04-24T06:45:49.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.965+0000 7fc052800640 1 -- 172.21.15.5:0/1089973752 wait complete. 2024-04-24T06:45:49.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.966+0000 7fc052800640 1 Processor -- start 2024-04-24T06:45:49.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.966+0000 7fc052800640 1 -- start start 2024-04-24T06:45:49.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.966+0000 7fc052800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc04c102fe0 0x7fc04c1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.966+0000 7fc052800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc04c103900 0x7fc04c1a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.966+0000 7fc052800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc04c10c400 0x7fc04c19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.966+0000 7fc052800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c112190 con 0x7fc04c103900 2024-04-24T06:45:49.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.966+0000 7fc052800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c112010 con 0x7fc04c102fe0 2024-04-24T06:45:49.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.966+0000 7fc052800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c112310 con 0x7fc04c10c400 2024-04-24T06:45:49.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.967+0000 7fc051400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc04c102fe0 0x7fc04c1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.967+0000 7fc051400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc04c102fe0 0x7fc04c1a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:37132/0 (socket says 172.21.15.5:37132) 2024-04-24T06:45:49.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.967+0000 7fc051400640 1 -- 172.21.15.5:0/518016131 learned_addr learned my addr 172.21.15.5:0/518016131 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:49.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.967+0000 7fc050a00640 1 --2- 172.21.15.5:0/518016131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc04c103900 0x7fc04c1a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.967+0000 7fc051e00640 1 --2- 172.21.15.5:0/518016131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc04c10c400 0x7fc04c19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.967+0000 7fc051400640 1 -- 172.21.15.5:0/518016131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc04c10c400 msgr2=0x7fc04c19a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:49.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.967+0000 7fc051400640 1 --2- 172.21.15.5:0/518016131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc04c10c400 0x7fc04c19a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.967+0000 7fc051400640 1 -- 172.21.15.5:0/518016131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc04c103900 msgr2=0x7fc04c1a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:49.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.967+0000 7fc051400640 1 --2- 172.21.15.5:0/518016131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc04c103900 0x7fc04c1a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.967+0000 7fc051400640 1 -- 172.21.15.5:0/518016131 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc04c19afc0 con 0x7fc04c102fe0 2024-04-24T06:45:49.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.968+0000 7fc051400640 1 --2- 172.21.15.5:0/518016131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc04c102fe0 0x7fc04c1a0640 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc04000e9f0 tx=0x7fc04000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:49.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.968+0000 7fc04a000640 1 -- 172.21.15.5:0/518016131 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04000ccc0 con 0x7fc04c102fe0 2024-04-24T06:45:49.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.968+0000 7fc04a000640 1 -- 172.21.15.5:0/518016131 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc040010910 con 0x7fc04c102fe0 2024-04-24T06:45:49.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.968+0000 7fc052800640 1 -- 172.21.15.5:0/518016131 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc04c19b2b0 con 0x7fc04c102fe0 2024-04-24T06:45:49.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.968+0000 7fc04a000640 1 -- 172.21.15.5:0/518016131 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0400188a0 con 0x7fc04c102fe0 2024-04-24T06:45:49.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.968+0000 7fc052800640 1 -- 172.21.15.5:0/518016131 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc04c1a75c0 con 0x7fc04c102fe0 2024-04-24T06:45:49.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.972+0000 7fc056bb1740 1 Processor -- start 2024-04-24T06:45:49.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.972+0000 7fc056bb1740 1 -- start start 2024-04-24T06:45:49.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.973+0000 7fc056bb1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f93c5e7b40 0x55f93c5e7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.973+0000 7fc056bb1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f93c5e8460 0x55f93c5f25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.973+0000 7fc056bb1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f93c5f2b30 0x55f93c5f4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.973+0000 7fc056bb1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f93c54d1d0 con 0x55f93c5e7b40 2024-04-24T06:45:49.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.973+0000 7fc056bb1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f93c4f3450 con 0x55f93c5f2b30 2024-04-24T06:45:49.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.973+0000 7fc056bb1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f93c549190 con 0x55f93c5e8460 2024-04-24T06:45:49.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.973+0000 7fc04a000640 1 -- 172.21.15.5:0/518016131 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fc040010ac0 con 0x7fc04c102fe0 2024-04-24T06:45:49.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.973+0000 7fc051e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f93c5e7b40 0x55f93c5e7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.973+0000 7fc050a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f93c5f2b30 0x55f93c5f4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.973+0000 7fc050a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f93c5f2b30 0x55f93c5f4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:37136/0 (socket says 172.21.15.5:37136) 2024-04-24T06:45:49.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.973+0000 7fc050a00640 1 -- 172.21.15.5:0/2228399769 learned_addr learned my addr 172.21.15.5:0/2228399769 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:49.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.973+0000 7fc051400640 1 --2- 172.21.15.5:0/2228399769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f93c5e8460 0x55f93c5f25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.974+0000 7fc050a00640 1 -- 172.21.15.5:0/2228399769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f93c5e8460 msgr2=0x55f93c5f25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:49.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.974+0000 7fc050a00640 1 --2- 172.21.15.5:0/2228399769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f93c5e8460 0x55f93c5f25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.974+0000 7fc050a00640 1 -- 172.21.15.5:0/2228399769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f93c5e7b40 msgr2=0x55f93c5e7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:49.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.974+0000 7fc050a00640 1 --2- 172.21.15.5:0/2228399769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f93c5e7b40 0x55f93c5e7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.974+0000 7fc050a00640 1 -- 172.21.15.5:0/2228399769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f93c5b0310 con 0x55f93c5f2b30 2024-04-24T06:45:49.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.974+0000 7fc050a00640 1 --2- 172.21.15.5:0/2228399769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f93c5f2b30 0x55f93c5f4f20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc038009c60 tx=0x7fc038011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b227df6266d8bde8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:49.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.974+0000 7fc03e600640 1 -- 172.21.15.5:0/2228399769 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc038004250 con 0x55f93c5f2b30 2024-04-24T06:45:49.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.974+0000 7fc03e600640 1 -- 172.21.15.5:0/2228399769 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc038009e10 con 0x55f93c5f2b30 2024-04-24T06:45:49.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.974+0000 7fc03e600640 1 -- 172.21.15.5:0/2228399769 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc038019800 con 0x55f93c5f2b30 2024-04-24T06:45:49.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.975+0000 7fc04a000640 1 --2- 172.21.15.5:0/518016131 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc0340a3d20 0x7fc0340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.975+0000 7fc04a000640 1 -- 172.21.15.5:0/518016131 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fc0400d0c90 con 0x7fc04c102fe0 2024-04-24T06:45:49.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.975+0000 7fc051e00640 1 --2- 172.21.15.5:0/518016131 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc0340a3d20 0x7fc0340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.975+0000 7fc056bb1740 1 -- 172.21.15.5:0/2228399769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f93c5f2b30 msgr2=0x55f93c5f4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:49.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.975+0000 7fc056bb1740 1 --2- 172.21.15.5:0/2228399769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f93c5f2b30 0x55f93c5f4f20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc038009c60 tx=0x7fc038011810 comp rx=0 tx=0).stop 2024-04-24T06:45:49.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.975+0000 7fc056bb1740 1 -- 172.21.15.5:0/2228399769 shutdown_connections 2024-04-24T06:45:49.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.975+0000 7fc056bb1740 1 --2- 172.21.15.5:0/2228399769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f93c5f2b30 0x55f93c5f4f20 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.975+0000 7fc056bb1740 1 --2- 172.21.15.5:0/2228399769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f93c5e8460 0x55f93c5f25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.975+0000 7fc056bb1740 1 --2- 172.21.15.5:0/2228399769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f93c5e7b40 0x55f93c5e7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.975+0000 7fc056bb1740 1 -- 172.21.15.5:0/2228399769 >> 172.21.15.5:0/2228399769 conn(0x55f93c5e3e60 msgr2=0x55f93c5e4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:49.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.976+0000 7fc056bb1740 1 -- 172.21.15.5:0/2228399769 shutdown_connections 2024-04-24T06:45:49.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.976+0000 7fc051e00640 1 --2- 172.21.15.5:0/518016131 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc0340a3d20 0x7fc0340a61e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc044006010 tx=0x7fc0440098c0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:49.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.976+0000 7fc056bb1740 1 -- 172.21.15.5:0/2228399769 wait complete. 2024-04-24T06:45:49.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.976+0000 7fc056bb1740 1 -- start start 2024-04-24T06:45:49.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.976+0000 7fc056bb1740 1 Processor -- start 2024-04-24T06:45:49.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.977+0000 7fc056bb1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f93c5e7b40 0x55f93c5f2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.977+0000 7fc056bb1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f93c5e8460 0x55f93c5ef850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.977+0000 7fc056bb1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f93c5f2b30 0x55f93c5eff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.977+0000 7fc056bb1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f93c5f59c0 con 0x55f93c5e7b40 2024-04-24T06:45:49.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.977+0000 7fc056bb1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f93c506810 con 0x55f93c5f2b30 2024-04-24T06:45:49.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.977+0000 7fc056bb1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f93c5f5b40 con 0x55f93c5e8460 2024-04-24T06:45:49.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.977+0000 7fc056bb1740 10 client.0 _ll_register_callbacks cb 0x55f93c4d4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:45:49.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.977+0000 7fc051400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f93c5e7b40 0x55f93c5f2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.977+0000 7fc051e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f93c5f2b30 0x55f93c5eff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.977+0000 7fc050a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f93c5e8460 0x55f93c5ef850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.977+0000 7fc050a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f93c5e8460 0x55f93c5ef850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:59840/0 (socket says 172.21.15.5:59840) 2024-04-24T06:45:49.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.977+0000 7fc051e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f93c5f2b30 0x55f93c5eff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:37148/0 (socket says 172.21.15.5:37148) 2024-04-24T06:45:49.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.977+0000 7fc051400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f93c5e7b40 0x55f93c5f2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52684/0 (socket says 172.21.15.5:52684) 2024-04-24T06:45:49.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.977+0000 7fc050a00640 1 -- 172.21.15.5:0/360276139 learned_addr learned my addr 172.21.15.5:0/360276139 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:49.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.977+0000 7fc050a00640 1 -- 172.21.15.5:0/360276139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f93c5f2b30 msgr2=0x55f93c5eff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:49.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.977+0000 7fc050a00640 1 --2- 172.21.15.5:0/360276139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f93c5f2b30 0x55f93c5eff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.977+0000 7fc050a00640 1 -- 172.21.15.5:0/360276139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f93c5e7b40 msgr2=0x55f93c5f2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:49.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.977+0000 7fc050a00640 1 --2- 172.21.15.5:0/360276139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f93c5e7b40 0x55f93c5f2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.977+0000 7fc050a00640 1 -- 172.21.15.5:0/360276139 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f93c5173f0 con 0x55f93c5e8460 2024-04-24T06:45:49.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.977+0000 7fc051400640 1 --2- 172.21.15.5:0/360276139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f93c5e7b40 0x55f93c5f2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:45:49.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.978+0000 7fc050a00640 1 --2- 172.21.15.5:0/360276139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f93c5e8460 0x55f93c5ef850 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc038000c00 tx=0x7fc038003b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df2b26929739d4e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:49.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.978+0000 7fc032a00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:45:49.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.978+0000 7fc032a00640 1 -- 172.21.15.5:0/360276139 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc038028020 con 0x55f93c5e8460 2024-04-24T06:45:49.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.978+0000 7fc056bb1740 1 -- 172.21.15.5:0/360276139 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc038009e10 con 0x55f93c5e8460 2024-04-24T06:45:49.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.978+0000 7fc032a00640 1 -- 172.21.15.5:0/360276139 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fc0380052d0 con 0x55f93c5e8460 2024-04-24T06:45:49.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.978+0000 7fc032a00640 1 -- 172.21.15.5:0/360276139 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc038033050 con 0x55f93c5e8460 2024-04-24T06:45:49.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.978+0000 7fc056bb1740 10 client.25759 fetch_fsmap learned FSMap version 23 2024-04-24T06:45:49.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.978+0000 7fc056bb1740 1 -- 172.21.15.5:0/360276139 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55f93c25f3f0 con 0x55f93c5e8460 2024-04-24T06:45:49.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.979+0000 7fc032a00640 1 -- 172.21.15.5:0/360276139 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc038009e10 con 0x55f93c5e8460 2024-04-24T06:45:49.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.979+0000 7fc032a00640 1 -- 172.21.15.5:0/360276139 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fc038018070 con 0x55f93c5e8460 2024-04-24T06:45:49.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.979+0000 7fc056bb1740 10 client.25759 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T06:45:49.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.979+0000 7fc056bb1740 10 client.25759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:45:49.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.979+0000 7fc056bb1740 20 client.25759 populate_metadata read hostname 'smithi005' 2024-04-24T06:45:49.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.979+0000 7fc056bb1740 1 --2- 172.21.15.5:0/360276139 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55f93c602b90 0x55f93c604f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:49.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.979+0000 7fc056bb1740 4 client.25759 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:45:49.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.979+0000 7fc056bb1740 1 -- 172.21.15.5:0/360276139 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f93c55abf0 con 0x55f93c602b90 2024-04-24T06:45:49.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.979+0000 7fc051400640 1 --2- 172.21.15.5:0/360276139 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55f93c602b90 0x55f93c604f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:49.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.980+0000 7fc051400640 1 --2- 172.21.15.5:0/360276139 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55f93c602b90 0x55f93c604f80 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17be4d673e9eb06d server_cookie=1d36f3fb90bcda77 in_seq=0 out_seq=0 2024-04-24T06:45:49.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.980+0000 7fc032a00640 10 client.25759 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:45:49.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.981+0000 7fc032a00640 1 -- 172.21.15.5:0/360276139 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f93c55abf0 con 0x55f93c602b90 2024-04-24T06:45:49.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.981+0000 7fc032a00640 10 client.25759 handle_command_reply: tid=0 2024-04-24T06:45:49.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.981+0000 7fc056bb1740 1 client.25759 shutdown 2024-04-24T06:45:49.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.981+0000 7fc056bb1740 1 -- 172.21.15.5:0/360276139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f93c5e8460 msgr2=0x55f93c5ef850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:49.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.981+0000 7fc056bb1740 1 --2- 172.21.15.5:0/360276139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f93c5e8460 0x55f93c5ef850 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc038000c00 tx=0x7fc038003b10 comp rx=0 tx=0).stop 2024-04-24T06:45:49.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.981+0000 7fc056bb1740 1 -- 172.21.15.5:0/360276139 shutdown_connections 2024-04-24T06:45:49.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.981+0000 7fc056bb1740 1 --2- 172.21.15.5:0/360276139 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55f93c602b90 0x55f93c604f80 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.981+0000 7fc056bb1740 1 --2- 172.21.15.5:0/360276139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f93c5f2b30 0x55f93c5eff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.981+0000 7fc056bb1740 1 --2- 172.21.15.5:0/360276139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f93c5e8460 0x55f93c5ef850 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.981+0000 7fc032a00640 0 client.25759 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:45:49.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.981+0000 7fc056bb1740 1 --2- 172.21.15.5:0/360276139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f93c5e7b40 0x55f93c5f2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:49.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.981+0000 7fc056bb1740 1 -- 172.21.15.5:0/360276139 >> 172.21.15.5:0/360276139 conn(0x55f93c5e3e60 msgr2=0x55f93c5e9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:49.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.981+0000 7fc056bb1740 1 -- 172.21.15.5:0/360276139 shutdown_connections 2024-04-24T06:45:49.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.981+0000 7fc056bb1740 1 -- 172.21.15.5:0/360276139 wait complete. 2024-04-24T06:45:49.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:49.981+0000 7fc056bb1740 20 client.25759 trim_cache size 0 max 16384 2024-04-24T06:45:50.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.001+0000 7fc056bb1740 1 Processor -- start 2024-04-24T06:45:50.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.001+0000 7fc056bb1740 1 -- start start 2024-04-24T06:45:50.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.001+0000 7fc056bb1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f93c5e7b40 0x55f93c5fe8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:50.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.001+0000 7fc056bb1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f93c5e8460 0x55f93c5f2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:50.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.001+0000 7fc056bb1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f93c5f2b30 0x55f93c5f25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:50.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.001+0000 7fc056bb1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c1b26e0 con 0x55f93c5e7b40 2024-04-24T06:45:50.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.001+0000 7fc056bb1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c1affb0 con 0x55f93c5f2b30 2024-04-24T06:45:50.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.001+0000 7fc056bb1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c1ade00 con 0x55f93c5e8460 2024-04-24T06:45:50.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.002+0000 7fc050a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f93c5e8460 0x55f93c5f2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:50.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.002+0000 7fc051400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f93c5e7b40 0x55f93c5fe8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:50.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.002+0000 7fc051e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f93c5f2b30 0x55f93c5f25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:50.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.002+0000 7fc050a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f93c5e8460 0x55f93c5f2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:59842/0 (socket says 172.21.15.5:59842) 2024-04-24T06:45:50.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.002+0000 7fc050a00640 1 -- 172.21.15.5:0/461748089 learned_addr learned my addr 172.21.15.5:0/461748089 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:50.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.002+0000 7fc051e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f93c5f2b30 0x55f93c5f25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:37160/0 (socket says 172.21.15.5:37160) 2024-04-24T06:45:50.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.002+0000 7fc050a00640 1 -- 172.21.15.5:0/461748089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f93c5f2b30 msgr2=0x55f93c5f25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:50.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.002+0000 7fc050a00640 1 --2- 172.21.15.5:0/461748089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f93c5f2b30 0x55f93c5f25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:50.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.002+0000 7fc050a00640 1 -- 172.21.15.5:0/461748089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f93c5e7b40 msgr2=0x55f93c5fe8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:50.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.002+0000 7fc050a00640 1 --2- 172.21.15.5:0/461748089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f93c5e7b40 0x55f93c5fe8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:50.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.002+0000 7fc050a00640 1 -- 172.21.15.5:0/461748089 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f93c3297d0 con 0x55f93c5e8460 2024-04-24T06:45:50.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.002+0000 7fc051400640 1 --2- 172.21.15.5:0/461748089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f93c5e7b40 0x55f93c5fe8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:45:50.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.002+0000 7fc050a00640 1 --2- 172.21.15.5:0/461748089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f93c5e8460 0x55f93c5f2090 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc038019ec0 tx=0x7fc038005050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ea3f55b41f426a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:50.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.003+0000 7fc03dc00640 1 -- 172.21.15.5:0/461748089 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc038028040 con 0x55f93c5e8460 2024-04-24T06:45:50.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.003+0000 7fc03dc00640 1 -- 172.21.15.5:0/461748089 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc038009ab0 con 0x55f93c5e8460 2024-04-24T06:45:50.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.003+0000 7fc03dc00640 1 -- 172.21.15.5:0/461748089 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc038024cd0 con 0x55f93c5e8460 2024-04-24T06:45:50.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.003+0000 7fc056bb1740 1 -- 172.21.15.5:0/461748089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f93c5e8460 msgr2=0x55f93c5f2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:50.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.003+0000 7fc056bb1740 1 --2- 172.21.15.5:0/461748089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f93c5e8460 0x55f93c5f2090 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc038019ec0 tx=0x7fc038005050 comp rx=0 tx=0).stop 2024-04-24T06:45:50.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.003+0000 7fc056bb1740 1 -- 172.21.15.5:0/461748089 shutdown_connections 2024-04-24T06:45:50.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.003+0000 7fc056bb1740 1 --2- 172.21.15.5:0/461748089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f93c5f2b30 0x55f93c5f25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:50.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.003+0000 7fc056bb1740 1 --2- 172.21.15.5:0/461748089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f93c5e8460 0x55f93c5f2090 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:50.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.003+0000 7fc056bb1740 1 --2- 172.21.15.5:0/461748089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f93c5e7b40 0x55f93c5fe8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:50.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.003+0000 7fc056bb1740 1 -- 172.21.15.5:0/461748089 >> 172.21.15.5:0/461748089 conn(0x55f93c5e3e60 msgr2=0x55f93c5eaad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:50.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.004+0000 7fc056bb1740 1 -- 172.21.15.5:0/461748089 shutdown_connections 2024-04-24T06:45:50.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.004+0000 7fc056bb1740 1 -- 172.21.15.5:0/461748089 wait complete. 2024-04-24T06:45:50.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.004+0000 7fc056bb1740 1 -- start start 2024-04-24T06:45:50.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.004+0000 7fc056bb1740 1 Processor -- start 2024-04-24T06:45:50.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.004+0000 7fc056bb1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f93c5e7b40 0x55f93c5fb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:50.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.005+0000 7fc056bb1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f93c5e8460 0x55f93c5f34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:50.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.005+0000 7fc056bb1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f93c5f2b30 0x55f93c5f3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:50.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.005+0000 7fc056bb1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c1ae470 con 0x55f93c5e7b40 2024-04-24T06:45:50.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.005+0000 7fc056bb1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c1af8e0 con 0x55f93c5e8460 2024-04-24T06:45:50.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.005+0000 7fc056bb1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c1aeb40 con 0x55f93c5f2b30 2024-04-24T06:45:50.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.005+0000 7fc056bb1740 10 client.0 _ll_register_callbacks cb 0x7fc038005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:45:50.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.005+0000 7fc051400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f93c5e7b40 0x55f93c5fb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:50.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.005+0000 7fc050a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f93c5e8460 0x55f93c5f34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:50.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.005+0000 7fc051400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f93c5e7b40 0x55f93c5fb030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52690/0 (socket says 172.21.15.5:52690) 2024-04-24T06:45:50.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.005+0000 7fc051400640 1 -- 172.21.15.5:0/3931837173 learned_addr learned my addr 172.21.15.5:0/3931837173 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:50.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.005+0000 7fc051e00640 1 --2- 172.21.15.5:0/3931837173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f93c5f2b30 0x55f93c5f3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:50.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.005+0000 7fc051400640 1 -- 172.21.15.5:0/3931837173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f93c5f2b30 msgr2=0x55f93c5f3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:50.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.005+0000 7fc051400640 1 --2- 172.21.15.5:0/3931837173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f93c5f2b30 0x55f93c5f3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:50.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.005+0000 7fc051400640 1 -- 172.21.15.5:0/3931837173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f93c5e8460 msgr2=0x55f93c5f34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:50.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.005+0000 7fc051400640 1 --2- 172.21.15.5:0/3931837173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f93c5e8460 0x55f93c5f34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:50.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.005+0000 7fc051400640 1 -- 172.21.15.5:0/3931837173 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f93c5f4050 con 0x55f93c5e7b40 2024-04-24T06:45:50.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.005+0000 7fc051400640 1 --2- 172.21.15.5:0/3931837173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f93c5e7b40 0x55f93c5fb030 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc04000c6a0 tx=0x7fc040009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aefec05ff2177c71 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:50.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.005+0000 7fc032a00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:45:50.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.006+0000 7fc032a00640 1 -- 172.21.15.5:0/3931837173 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0400a5850 con 0x55f93c5e7b40 2024-04-24T06:45:50.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.006+0000 7fc032a00640 1 -- 172.21.15.5:0/3931837173 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fc0400ce610 con 0x55f93c5e7b40 2024-04-24T06:45:50.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.006+0000 7fc056bb1740 1 -- 172.21.15.5:0/3931837173 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc038009ab0 con 0x55f93c5e7b40 2024-04-24T06:45:50.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.006+0000 7fc032a00640 1 -- 172.21.15.5:0/3931837173 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc04009c030 con 0x55f93c5e7b40 2024-04-24T06:45:50.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.006+0000 7fc056bb1740 10 client.15636 fetch_fsmap learned FSMap version 23 2024-04-24T06:45:50.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.006+0000 7fc056bb1740 1 -- 172.21.15.5:0/3931837173 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55f93c668fd0 con 0x55f93c5e7b40 2024-04-24T06:45:50.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.006+0000 7fc032a00640 1 -- 172.21.15.5:0/3931837173 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc038009ab0 con 0x55f93c5e7b40 2024-04-24T06:45:50.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.006+0000 7fc032a00640 1 -- 172.21.15.5:0/3931837173 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fc0400cec70 con 0x55f93c5e7b40 2024-04-24T06:45:50.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.007+0000 7fc056bb1740 10 client.15636 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T06:45:50.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.007+0000 7fc056bb1740 10 client.15636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:45:50.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.007+0000 7fc056bb1740 20 client.15636 populate_metadata read hostname 'smithi005' 2024-04-24T06:45:50.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.007+0000 7fc056bb1740 1 --2- 172.21.15.5:0/3931837173 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55f93c6695b0 0x55f93c66b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:50.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.007+0000 7fc056bb1740 4 client.15636 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:45:50.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.007+0000 7fc056bb1740 1 -- 172.21.15.5:0/3931837173 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f93c554a60 con 0x55f93c6695b0 2024-04-24T06:45:50.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.007+0000 7fc050a00640 1 --2- 172.21.15.5:0/3931837173 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55f93c6695b0 0x55f93c66b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:50.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.007+0000 7fc050a00640 1 --2- 172.21.15.5:0/3931837173 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55f93c6695b0 0x55f93c66b9d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1030bdd81f3cbab2 server_cookie=b336703da92eaf4b in_seq=0 out_seq=0 2024-04-24T06:45:50.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.007+0000 7fc032a00640 10 client.15636 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:45:50.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.008+0000 7fc032a00640 1 -- 172.21.15.5:0/3931837173 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f93c554a60 con 0x55f93c6695b0 2024-04-24T06:45:50.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.008+0000 7fc032a00640 10 client.15636 handle_command_reply: tid=0 2024-04-24T06:45:50.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.008+0000 7fc056bb1740 1 client.15636 shutdown 2024-04-24T06:45:50.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.008+0000 7fc056bb1740 1 -- 172.21.15.5:0/3931837173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f93c5e7b40 msgr2=0x55f93c5fb030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:50.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.008+0000 7fc056bb1740 1 --2- 172.21.15.5:0/3931837173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f93c5e7b40 0x55f93c5fb030 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc04000c6a0 tx=0x7fc040009700 comp rx=0 tx=0).stop 2024-04-24T06:45:50.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.008+0000 7fc056bb1740 1 -- 172.21.15.5:0/3931837173 shutdown_connections 2024-04-24T06:45:50.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.008+0000 7fc056bb1740 1 --2- 172.21.15.5:0/3931837173 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55f93c6695b0 0x55f93c66b9d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:50.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.008+0000 7fc056bb1740 1 --2- 172.21.15.5:0/3931837173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f93c5f2b30 0x55f93c5f3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:50.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.008+0000 7fc056bb1740 1 --2- 172.21.15.5:0/3931837173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f93c5e8460 0x55f93c5f34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:50.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.008+0000 7fc056bb1740 1 --2- 172.21.15.5:0/3931837173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f93c5e7b40 0x55f93c5fb030 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:50.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.008+0000 7fc032a00640 0 client.15636 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:45:50.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.008+0000 7fc056bb1740 1 -- 172.21.15.5:0/3931837173 >> 172.21.15.5:0/3931837173 conn(0x55f93c5e3e60 msgr2=0x55f93c5e26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:50.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.008+0000 7fc056bb1740 1 -- 172.21.15.5:0/3931837173 shutdown_connections 2024-04-24T06:45:50.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.008+0000 7fc056bb1740 1 -- 172.21.15.5:0/3931837173 wait complete. 2024-04-24T06:45:50.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.008+0000 7fc056bb1740 20 client.15636 trim_cache size 0 max 16384 2024-04-24T06:45:50.014 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:45:50.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.009+0000 7fc052800640 1 -- 172.21.15.5:0/518016131 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc0340a3d20 msgr2=0x7fc0340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:50.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.009+0000 7fc052800640 1 --2- 172.21.15.5:0/518016131 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc0340a3d20 0x7fc0340a61e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc044006010 tx=0x7fc0440098c0 comp rx=0 tx=0).stop 2024-04-24T06:45:50.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.009+0000 7fc052800640 1 -- 172.21.15.5:0/518016131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc04c102fe0 msgr2=0x7fc04c1a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:50.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.009+0000 7fc052800640 1 --2- 172.21.15.5:0/518016131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc04c102fe0 0x7fc04c1a0640 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc04000e9f0 tx=0x7fc04000edf0 comp rx=0 tx=0).stop 2024-04-24T06:45:50.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.009+0000 7fc052800640 1 -- 172.21.15.5:0/518016131 shutdown_connections 2024-04-24T06:45:50.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.009+0000 7fc052800640 1 --2- 172.21.15.5:0/518016131 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc0340a3d20 0x7fc0340a61e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:50.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.009+0000 7fc052800640 1 --2- 172.21.15.5:0/518016131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc04c10c400 0x7fc04c19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:50.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.009+0000 7fc052800640 1 --2- 172.21.15.5:0/518016131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc04c103900 0x7fc04c1a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:50.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.009+0000 7fc052800640 1 --2- 172.21.15.5:0/518016131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc04c102fe0 0x7fc04c1a0640 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:50.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.009+0000 7fc052800640 1 -- 172.21.15.5:0/518016131 >> 172.21.15.5:0/518016131 conn(0x7fc04c077ab0 msgr2=0x7fc04c0fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:50.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.009+0000 7fc052800640 1 -- 172.21.15.5:0/518016131 shutdown_connections 2024-04-24T06:45:50.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:50.009+0000 7fc052800640 1 -- 172.21.15.5:0/518016131 wait complete. 2024-04-24T06:45:50.118 INFO:tasks.workunit.client.0.smithi005.stderr:Updating files: 56% (6960/12271) Updating files: 57% (6995/12271) Updating files: 58% (7118/12271) Updating files: 59% (7240/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 92% (11370/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-24T06:45:50.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:50 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/4077283676' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:45:50.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:50 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/4077283676' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:45:50.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:50 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/4077283676' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:45:50.705 INFO:tasks.workunit.client.0.smithi005.stderr:Note: switching to 'a3c8a22bd5efe18ab230a8e2d072db8f381a0411'. 2024-04-24T06:45:50.705 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-04-24T06:45:50.705 INFO:tasks.workunit.client.0.smithi005.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-24T06:45:50.705 INFO:tasks.workunit.client.0.smithi005.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-24T06:45:50.705 INFO:tasks.workunit.client.0.smithi005.stderr:state without impacting any branches by switching back to a branch. 2024-04-24T06:45:50.705 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-04-24T06:45:50.705 INFO:tasks.workunit.client.0.smithi005.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-24T06:45:50.705 INFO:tasks.workunit.client.0.smithi005.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-24T06:45:50.706 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-04-24T06:45:50.706 INFO:tasks.workunit.client.0.smithi005.stderr: git switch -c 2024-04-24T06:45:50.706 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-04-24T06:45:50.706 INFO:tasks.workunit.client.0.smithi005.stderr:Or undo this operation with: 2024-04-24T06:45:50.706 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-04-24T06:45:50.706 INFO:tasks.workunit.client.0.smithi005.stderr: git switch - 2024-04-24T06:45:50.706 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-04-24T06:45:50.706 INFO:tasks.workunit.client.0.smithi005.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-24T06:45:50.706 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-04-24T06:45:50.706 INFO:tasks.workunit.client.0.smithi005.stderr:HEAD is now at a3c8a22bd5e DNM: trigger builds 2024-04-24T06:45:50.718 DEBUG:teuthology.orchestra.run.smithi005:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-24T06:45:50.829 INFO:tasks.workunit.client.0.smithi005.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-24T06:45:50.837 INFO:tasks.workunit.client.0.smithi005.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-24T06:45:50.838 INFO:tasks.workunit.client.0.smithi005.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-24T06:45:51.028 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T06:45:51.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:45:51.029 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T06:45:51.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.257+0000 7fcde4a00640 1 -- 172.21.15.5:0/471714382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcde00743a0 msgr2=0x7fcde0074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:51.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.257+0000 7fcde4a00640 1 --2- 172.21.15.5:0/471714382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcde00743a0 0x7fcde0074780 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0009900 tx=0x7fcdd002a480 comp rx=0 tx=0).stop 2024-04-24T06:45:51.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.257+0000 7fcde4a00640 1 -- 172.21.15.5:0/471714382 shutdown_connections 2024-04-24T06:45:51.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.257+0000 7fcde4a00640 1 --2- 172.21.15.5:0/471714382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcde0112de0 0x7fcde01151d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.257+0000 7fcde4a00640 1 --2- 172.21.15.5:0/471714382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcde0074cc0 0x7fcde01128a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.257+0000 7fcde4a00640 1 --2- 172.21.15.5:0/471714382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcde00743a0 0x7fcde0074780 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.257+0000 7fcde4a00640 1 -- 172.21.15.5:0/471714382 >> 172.21.15.5:0/471714382 conn(0x7fcde00fd8e0 msgr2=0x7fcde00ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:51.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.257+0000 7fcde4a00640 1 -- 172.21.15.5:0/471714382 shutdown_connections 2024-04-24T06:45:51.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.257+0000 7fcde4a00640 1 -- 172.21.15.5:0/471714382 wait complete. 2024-04-24T06:45:51.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.259+0000 7fcde4a00640 1 Processor -- start 2024-04-24T06:45:51.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.259+0000 7fcde4a00640 1 -- start start 2024-04-24T06:45:51.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.259+0000 7fcde4a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcde00743a0 0x7fcde019edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:51.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.259+0000 7fcde4a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcde0074cc0 0x7fcde019f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:51.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.259+0000 7fcde4a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcde0112de0 0x7fcde01a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:51.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.259+0000 7fcde4a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcde0118800 con 0x7fcde0074cc0 2024-04-24T06:45:51.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.259+0000 7fcde4a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcde0118680 con 0x7fcde0112de0 2024-04-24T06:45:51.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.259+0000 7fcde4a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcde0118980 con 0x7fcde00743a0 2024-04-24T06:45:51.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.259+0000 7fcddea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcde0074cc0 0x7fcde019f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:51.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.260+0000 7fcddea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcde0074cc0 0x7fcde019f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52706/0 (socket says 172.21.15.5:52706) 2024-04-24T06:45:51.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.260+0000 7fcddea00640 1 -- 172.21.15.5:0/1274740588 learned_addr learned my addr 172.21.15.5:0/1274740588 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:51.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.260+0000 7fcddf400640 1 --2- 172.21.15.5:0/1274740588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcde00743a0 0x7fcde019edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:51.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.260+0000 7fcddfe00640 1 --2- 172.21.15.5:0/1274740588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcde0112de0 0x7fcde01a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:51.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.260+0000 7fcddea00640 1 -- 172.21.15.5:0/1274740588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcde00743a0 msgr2=0x7fcde019edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:51.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.260+0000 7fcddea00640 1 --2- 172.21.15.5:0/1274740588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcde00743a0 0x7fcde019edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.260+0000 7fcddea00640 1 -- 172.21.15.5:0/1274740588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcde0112de0 msgr2=0x7fcde01a3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:51.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.260+0000 7fcddea00640 1 --2- 172.21.15.5:0/1274740588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcde0112de0 0x7fcde01a3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.260+0000 7fcddea00640 1 -- 172.21.15.5:0/1274740588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcde01a3d60 con 0x7fcde0074cc0 2024-04-24T06:45:51.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.260+0000 7fcddea00640 1 --2- 172.21.15.5:0/1274740588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcde0074cc0 0x7fcde019f300 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00e9f0 tx=0x7fcdcc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:51.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.261+0000 7fcdd7e00640 1 -- 172.21.15.5:0/1274740588 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc00cd80 con 0x7fcde0074cc0 2024-04-24T06:45:51.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.261+0000 7fcdd7e00640 1 -- 172.21.15.5:0/1274740588 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdcc010910 con 0x7fcde0074cc0 2024-04-24T06:45:51.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.261+0000 7fcdd7e00640 1 -- 172.21.15.5:0/1274740588 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc018990 con 0x7fcde0074cc0 2024-04-24T06:45:51.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.261+0000 7fcde4a00640 1 -- 172.21.15.5:0/1274740588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcde01a4050 con 0x7fcde0074cc0 2024-04-24T06:45:51.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.261+0000 7fcde4a00640 1 -- 172.21.15.5:0/1274740588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcde01ab8e0 con 0x7fcde0074cc0 2024-04-24T06:45:51.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.264+0000 7fcde8fcd740 1 Processor -- start 2024-04-24T06:45:51.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.264+0000 7fcde8fcd740 1 -- start start 2024-04-24T06:45:51.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.264+0000 7fcde8fcd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561264307270 0x561264307650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:51.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.264+0000 7fcde8fcd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561264307b90 0x561264311d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:51.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.264+0000 7fcdd7e00640 1 -- 172.21.15.5:0/1274740588 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fcdcc010ac0 con 0x7fcde0074cc0 2024-04-24T06:45:51.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.264+0000 7fcde8fcd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561264312260 0x561264314650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:51.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.265+0000 7fcde8fcd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56126426ce70 con 0x561264312260 2024-04-24T06:45:51.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.265+0000 7fcde8fcd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561264213380 con 0x561264307270 2024-04-24T06:45:51.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.265+0000 7fcde8fcd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561264268e30 con 0x561264307b90 2024-04-24T06:45:51.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.265+0000 7fcddea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561264312260 0x561264314650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:51.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.265+0000 7fcddfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561264307270 0x561264307650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:51.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.265+0000 7fcddf400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561264307b90 0x561264311d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:51.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.265+0000 7fcddfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561264307270 0x561264307650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:37186/0 (socket says 172.21.15.5:37186) 2024-04-24T06:45:51.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.265+0000 7fcddea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561264312260 0x561264314650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52708/0 (socket says 172.21.15.5:52708) 2024-04-24T06:45:51.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.265+0000 7fcddf400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561264307b90 0x561264311d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:59888/0 (socket says 172.21.15.5:59888) 2024-04-24T06:45:51.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.265+0000 7fcddfe00640 1 -- 172.21.15.5:0/1728712609 learned_addr learned my addr 172.21.15.5:0/1728712609 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:51.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.265+0000 7fcddfe00640 1 -- 172.21.15.5:0/1728712609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561264307b90 msgr2=0x561264311d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:51.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.265+0000 7fcddfe00640 1 --2- 172.21.15.5:0/1728712609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561264307b90 0x561264311d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.265+0000 7fcddfe00640 1 -- 172.21.15.5:0/1728712609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561264312260 msgr2=0x561264314650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:51.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.265+0000 7fcddfe00640 1 --2- 172.21.15.5:0/1728712609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561264312260 0x561264314650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.265+0000 7fcddfe00640 1 -- 172.21.15.5:0/1728712609 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612640477e0 con 0x561264307270 2024-04-24T06:45:51.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.266+0000 7fcddfe00640 1 --2- 172.21.15.5:0/1728712609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561264307270 0x561264307650 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8009f30 tx=0x7fcdd80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af80a3d78f096401 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:51.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.266+0000 7fcdcbe00640 1 -- 172.21.15.5:0/1728712609 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8004d20 con 0x561264307270 2024-04-24T06:45:51.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.266+0000 7fcdcbe00640 1 -- 172.21.15.5:0/1728712609 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdd8004050 con 0x561264307270 2024-04-24T06:45:51.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.266+0000 7fcdcbe00640 1 -- 172.21.15.5:0/1728712609 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8005280 con 0x561264307270 2024-04-24T06:45:51.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.266+0000 7fcdd7e00640 1 --2- 172.21.15.5:0/1274740588 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fcdc40a3da0 0x7fcdc40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:51.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.267+0000 7fcdd7e00640 1 -- 172.21.15.5:0/1274740588 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fcdcc0d0ad0 con 0x7fcde0074cc0 2024-04-24T06:45:51.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.267+0000 7fcddf400640 1 --2- 172.21.15.5:0/1274740588 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fcdc40a3da0 0x7fcdc40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:51.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.267+0000 7fcde8fcd740 1 -- 172.21.15.5:0/1728712609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561264307270 msgr2=0x561264307650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:51.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.267+0000 7fcde8fcd740 1 --2- 172.21.15.5:0/1728712609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561264307270 0x561264307650 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8009f30 tx=0x7fcdd80118d0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.267+0000 7fcde8fcd740 1 -- 172.21.15.5:0/1728712609 shutdown_connections 2024-04-24T06:45:51.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.267+0000 7fcde8fcd740 1 --2- 172.21.15.5:0/1728712609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561264312260 0x561264314650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.267+0000 7fcde8fcd740 1 --2- 172.21.15.5:0/1728712609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561264307b90 0x561264311d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.267+0000 7fcde8fcd740 1 --2- 172.21.15.5:0/1728712609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561264307270 0x561264307650 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.267+0000 7fcde8fcd740 1 -- 172.21.15.5:0/1728712609 >> 172.21.15.5:0/1728712609 conn(0x561264303590 msgr2=0x561264303970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:51.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.267+0000 7fcddf400640 1 --2- 172.21.15.5:0/1274740588 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fcdc40a3da0 0x7fcdc40a6260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0000c00 tx=0x7fcdd0005b50 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:51.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.267+0000 7fcde8fcd740 1 -- 172.21.15.5:0/1728712609 shutdown_connections 2024-04-24T06:45:51.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.267+0000 7fcde8fcd740 1 -- 172.21.15.5:0/1728712609 wait complete. 2024-04-24T06:45:51.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.268+0000 7fcde8fcd740 1 -- start start 2024-04-24T06:45:51.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.268+0000 7fcde8fcd740 1 Processor -- start 2024-04-24T06:45:51.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.269+0000 7fcde8fcd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561264307270 0x56126430f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:51.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.269+0000 7fcde8fcd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561264307b90 0x56126430f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:51.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.269+0000 7fcde8fcd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561264312260 0x56126430fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:51.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.269+0000 7fcde8fcd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561264315050 con 0x561264312260 2024-04-24T06:45:51.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.269+0000 7fcde8fcd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561264226660 con 0x561264307b90 2024-04-24T06:45:51.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.269+0000 7fcde8fcd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5612643151d0 con 0x561264307270 2024-04-24T06:45:51.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.269+0000 7fcde8fcd740 10 client.0 _ll_register_callbacks cb 0x561264214aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:45:51.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.269+0000 7fcddfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561264312260 0x56126430fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:51.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.269+0000 7fcddfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561264312260 0x56126430fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52712/0 (socket says 172.21.15.5:52712) 2024-04-24T06:45:51.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.269+0000 7fcddfe00640 1 -- 172.21.15.5:0/1276470628 learned_addr learned my addr 172.21.15.5:0/1276470628 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:51.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.269+0000 7fcddf400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561264307270 0x56126430f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:51.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.269+0000 7fcddea00640 1 --2- 172.21.15.5:0/1276470628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561264307b90 0x56126430f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:51.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.269+0000 7fcddfe00640 1 -- 172.21.15.5:0/1276470628 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561264307270 msgr2=0x56126430f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:51.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.269+0000 7fcddfe00640 1 --2- 172.21.15.5:0/1276470628 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561264307270 0x56126430f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.269+0000 7fcddfe00640 1 -- 172.21.15.5:0/1276470628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561264307b90 msgr2=0x56126430f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:51.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.269+0000 7fcddfe00640 1 --2- 172.21.15.5:0/1276470628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561264307b90 0x56126430f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.269+0000 7fcddfe00640 1 -- 172.21.15.5:0/1276470628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612640a7f70 con 0x561264312260 2024-04-24T06:45:51.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.269+0000 7fcddea00640 1 --2- 172.21.15.5:0/1276470628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561264307b90 0x56126430f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:45:51.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.270+0000 7fcddfe00640 1 --2- 172.21.15.5:0/1276470628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561264312260 0x56126430fc40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fcdd801b040 tx=0x7fcdd8004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a79b78500184703d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:51.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.270+0000 7fcdc8c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:45:51.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.270+0000 7fcdc8c00640 1 -- 172.21.15.5:0/1276470628 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8029020 con 0x561264312260 2024-04-24T06:45:51.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.270+0000 7fcdc8c00640 1 -- 172.21.15.5:0/1276470628 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fcdd8005040 con 0x561264312260 2024-04-24T06:45:51.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.270+0000 7fcde8fcd740 1 -- 172.21.15.5:0/1276470628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdd8004050 con 0x561264312260 2024-04-24T06:45:51.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.271+0000 7fcdc8c00640 1 -- 172.21.15.5:0/1276470628 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fcdd8034050 con 0x561264312260 2024-04-24T06:45:51.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.271+0000 7fcde8fcd740 10 client.15648 fetch_fsmap learned FSMap version 23 2024-04-24T06:45:51.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.271+0000 7fcde8fcd740 1 -- 172.21.15.5:0/1276470628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x561263f3f170 con 0x561264312260 2024-04-24T06:45:51.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.271+0000 7fcdc8c00640 1 -- 172.21.15.5:0/1276470628 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdd8004050 con 0x561264312260 2024-04-24T06:45:51.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.271+0000 7fcdc8c00640 1 -- 172.21.15.5:0/1276470628 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fcdd8025490 con 0x561264312260 2024-04-24T06:45:51.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.271+0000 7fcde8fcd740 10 client.15648 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T06:45:51.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.271+0000 7fcde8fcd740 10 client.15648 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:45:51.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.271+0000 7fcde8fcd740 20 client.15648 populate_metadata read hostname 'smithi005' 2024-04-24T06:45:51.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.271+0000 7fcde8fcd740 1 --2- 172.21.15.5:0/1276470628 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5612643225d0 0x5612643249c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:51.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.271+0000 7fcde8fcd740 4 client.15648 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:45:51.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.271+0000 7fcde8fcd740 1 -- 172.21.15.5:0/1276470628 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56126427a890 con 0x5612643225d0 2024-04-24T06:45:51.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.272+0000 7fcddf400640 1 --2- 172.21.15.5:0/1276470628 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5612643225d0 0x5612643249c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:51.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.272+0000 7fcddf400640 1 --2- 172.21.15.5:0/1276470628 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5612643225d0 0x5612643249c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6854f0001f5f1343 server_cookie=52cda78aac2fa911 in_seq=0 out_seq=0 2024-04-24T06:45:51.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.272+0000 7fcdc8c00640 10 client.15648 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:45:51.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.273+0000 7fcdc8c00640 1 -- 172.21.15.5:0/1276470628 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56126427a890 con 0x5612643225d0 2024-04-24T06:45:51.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.273+0000 7fcdc8c00640 10 client.15648 handle_command_reply: tid=0 2024-04-24T06:45:51.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.273+0000 7fcde8fcd740 1 client.15648 shutdown 2024-04-24T06:45:51.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.273+0000 7fcde8fcd740 1 -- 172.21.15.5:0/1276470628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561264312260 msgr2=0x56126430fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:51.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.273+0000 7fcde8fcd740 1 --2- 172.21.15.5:0/1276470628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561264312260 0x56126430fc40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fcdd801b040 tx=0x7fcdd8004cf0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.273+0000 7fcde8fcd740 1 -- 172.21.15.5:0/1276470628 shutdown_connections 2024-04-24T06:45:51.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.274+0000 7fcde8fcd740 1 --2- 172.21.15.5:0/1276470628 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5612643225d0 0x5612643249c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.274+0000 7fcde8fcd740 1 --2- 172.21.15.5:0/1276470628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561264312260 0x56126430fc40 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.274+0000 7fcde8fcd740 1 --2- 172.21.15.5:0/1276470628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561264307b90 0x56126430f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.274+0000 7fcde8fcd740 1 --2- 172.21.15.5:0/1276470628 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561264307270 0x56126430f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.274+0000 7fcde8fcd740 1 -- 172.21.15.5:0/1276470628 >> 172.21.15.5:0/1276470628 conn(0x561264303590 msgr2=0x561264309390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:51.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.274+0000 7fcdc8c00640 0 client.15648 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:45:51.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.274+0000 7fcde8fcd740 1 -- 172.21.15.5:0/1276470628 shutdown_connections 2024-04-24T06:45:51.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.274+0000 7fcde8fcd740 1 -- 172.21.15.5:0/1276470628 wait complete. 2024-04-24T06:45:51.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.274+0000 7fcde8fcd740 20 client.15648 trim_cache size 0 max 16384 2024-04-24T06:45:51.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.293+0000 7fcde8fcd740 1 Processor -- start 2024-04-24T06:45:51.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.294+0000 7fcde8fcd740 1 -- start start 2024-04-24T06:45:51.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.294+0000 7fcde8fcd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561264307270 0x56126431df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:51.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.294+0000 7fcde8fcd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561264307b90 0x561264302570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:51.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.294+0000 7fcde8fcd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561264312260 0x56126432a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:51.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.294+0000 7fcde8fcd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcde01b6b50 con 0x561264307270 2024-04-24T06:45:51.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.294+0000 7fcde8fcd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcde01b4420 con 0x561264312260 2024-04-24T06:45:51.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.294+0000 7fcde8fcd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcde01b2270 con 0x561264307b90 2024-04-24T06:45:51.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.294+0000 7fcddfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561264312260 0x56126432a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:51.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.294+0000 7fcddf400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561264307270 0x56126431df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:51.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.294+0000 7fcddfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561264312260 0x56126432a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:37216/0 (socket says 172.21.15.5:37216) 2024-04-24T06:45:51.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.294+0000 7fcddfe00640 1 -- 172.21.15.5:0/2980634459 learned_addr learned my addr 172.21.15.5:0/2980634459 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:51.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.294+0000 7fcddea00640 1 --2- 172.21.15.5:0/2980634459 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561264307b90 0x561264302570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:51.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.295+0000 7fcddfe00640 1 -- 172.21.15.5:0/2980634459 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561264307b90 msgr2=0x561264302570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:51.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.295+0000 7fcddfe00640 1 --2- 172.21.15.5:0/2980634459 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561264307b90 0x561264302570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.295+0000 7fcddfe00640 1 -- 172.21.15.5:0/2980634459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561264307270 msgr2=0x56126431df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:51.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.295+0000 7fcddfe00640 1 --2- 172.21.15.5:0/2980634459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561264307270 0x56126431df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.295+0000 7fcddfe00640 1 -- 172.21.15.5:0/2980634459 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561264324f80 con 0x561264312260 2024-04-24T06:45:51.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.295+0000 7fcddf400640 1 --2- 172.21.15.5:0/2980634459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561264307270 0x56126431df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:45:51.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.295+0000 7fcddfe00640 1 --2- 172.21.15.5:0/2980634459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561264312260 0x56126432a1b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8000f80 tx=0x7fcdd8005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc9885af10834878 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:51.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.295+0000 7fcdcb400640 1 -- 172.21.15.5:0/2980634459 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8029040 con 0x561264312260 2024-04-24T06:45:51.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.295+0000 7fcdcb400640 1 -- 172.21.15.5:0/2980634459 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdd8004580 con 0x561264312260 2024-04-24T06:45:51.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.295+0000 7fcdcb400640 1 -- 172.21.15.5:0/2980634459 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8005040 con 0x561264312260 2024-04-24T06:45:51.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.296+0000 7fcde8fcd740 1 -- 172.21.15.5:0/2980634459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561264312260 msgr2=0x56126432a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:51.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.296+0000 7fcde8fcd740 1 --2- 172.21.15.5:0/2980634459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561264312260 0x56126432a1b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8000f80 tx=0x7fcdd8005f90 comp rx=0 tx=0).stop 2024-04-24T06:45:51.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.296+0000 7fcde8fcd740 1 -- 172.21.15.5:0/2980634459 shutdown_connections 2024-04-24T06:45:51.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.296+0000 7fcde8fcd740 1 --2- 172.21.15.5:0/2980634459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561264312260 0x56126432a1b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.296+0000 7fcde8fcd740 1 --2- 172.21.15.5:0/2980634459 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561264307b90 0x561264302570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.296+0000 7fcde8fcd740 1 --2- 172.21.15.5:0/2980634459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561264307270 0x56126431df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.296+0000 7fcde8fcd740 1 -- 172.21.15.5:0/2980634459 >> 172.21.15.5:0/2980634459 conn(0x561264303590 msgr2=0x561264301e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:51.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.296+0000 7fcde8fcd740 1 -- 172.21.15.5:0/2980634459 shutdown_connections 2024-04-24T06:45:51.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.296+0000 7fcde8fcd740 1 -- 172.21.15.5:0/2980634459 wait complete. 2024-04-24T06:45:51.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.297+0000 7fcde8fcd740 1 -- start start 2024-04-24T06:45:51.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.297+0000 7fcde8fcd740 1 Processor -- start 2024-04-24T06:45:51.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.297+0000 7fcde8fcd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561264307270 0x56126431a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:51.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.297+0000 7fcde8fcd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561264307b90 0x56126431a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:51.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.297+0000 7fcde8fcd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561264312260 0x561264317420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:51.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.297+0000 7fcde8fcd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcde01b28e0 con 0x561264307b90 2024-04-24T06:45:51.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.297+0000 7fcde8fcd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcde01b3d50 con 0x561264307270 2024-04-24T06:45:51.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.297+0000 7fcde8fcd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcde01b2fb0 con 0x561264312260 2024-04-24T06:45:51.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.297+0000 7fcde8fcd740 10 client.0 _ll_register_callbacks cb 0x7fcdcc0d4450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:45:51.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.298+0000 7fcddf400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561264307270 0x56126431a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:51.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.298+0000 7fcddf400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561264307270 0x56126431a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:37224/0 (socket says 172.21.15.5:37224) 2024-04-24T06:45:51.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.298+0000 7fcddf400640 1 -- 172.21.15.5:0/3684850597 learned_addr learned my addr 172.21.15.5:0/3684850597 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:51.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.298+0000 7fcddfe00640 1 --2- 172.21.15.5:0/3684850597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561264312260 0x561264317420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:51.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.298+0000 7fcddea00640 1 --2- 172.21.15.5:0/3684850597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561264307b90 0x56126431a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:51.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.298+0000 7fcddea00640 1 -- 172.21.15.5:0/3684850597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561264312260 msgr2=0x561264317420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:51.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.298+0000 7fcddea00640 1 --2- 172.21.15.5:0/3684850597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561264312260 0x561264317420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.298+0000 7fcddea00640 1 -- 172.21.15.5:0/3684850597 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561264307270 msgr2=0x56126431a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:51.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.298+0000 7fcddea00640 1 --2- 172.21.15.5:0/3684850597 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561264307270 0x56126431a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.298+0000 7fcddea00640 1 -- 172.21.15.5:0/3684850597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561264302fb0 con 0x561264307b90 2024-04-24T06:45:51.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.298+0000 7fcddf400640 1 --2- 172.21.15.5:0/3684850597 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561264307270 0x56126431a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:45:51.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.298+0000 7fcddea00640 1 --2- 172.21.15.5:0/3684850597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561264307b90 0x56126431a990 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc0ce1d0 tx=0x7fcdcc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f50892643aa05 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:51.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.298+0000 7fcdc8c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:45:51.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.299+0000 7fcdc8c00640 1 -- 172.21.15.5:0/3684850597 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc09c8a0 con 0x561264307b90 2024-04-24T06:45:51.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.299+0000 7fcdc8c00640 1 -- 172.21.15.5:0/3684850597 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fcdcc018b30 con 0x561264307b90 2024-04-24T06:45:51.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.299+0000 7fcde8fcd740 1 -- 172.21.15.5:0/3684850597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdd8004580 con 0x561264307b90 2024-04-24T06:45:51.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.299+0000 7fcdc8c00640 1 -- 172.21.15.5:0/3684850597 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fcdcc09c430 con 0x561264307b90 2024-04-24T06:45:51.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.299+0000 7fcde8fcd740 10 client.15654 fetch_fsmap learned FSMap version 23 2024-04-24T06:45:51.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.299+0000 7fcde8fcd740 1 -- 172.21.15.5:0/3684850597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5612643033c0 con 0x561264307b90 2024-04-24T06:45:51.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.299+0000 7fcdc8c00640 1 -- 172.21.15.5:0/3684850597 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdd8004580 con 0x561264307b90 2024-04-24T06:45:51.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.299+0000 7fcdc8c00640 1 -- 172.21.15.5:0/3684850597 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fcdcc09bfb0 con 0x561264307b90 2024-04-24T06:45:51.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.299+0000 7fcde8fcd740 10 client.15654 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T06:45:51.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.299+0000 7fcde8fcd740 10 client.15654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:45:51.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.299+0000 7fcde8fcd740 20 client.15654 populate_metadata read hostname 'smithi005' 2024-04-24T06:45:51.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.299+0000 7fcde8fcd740 1 --2- 172.21.15.5:0/3684850597 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x561264389970 0x56126438bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:51.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.299+0000 7fcde8fcd740 4 client.15654 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T06:45:51.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.299+0000 7fcde8fcd740 1 -- 172.21.15.5:0/3684850597 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561264274700 con 0x561264389970 2024-04-24T06:45:51.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.300+0000 7fcddf400640 1 --2- 172.21.15.5:0/3684850597 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x561264389970 0x56126438bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:51.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.300+0000 7fcddf400640 1 --2- 172.21.15.5:0/3684850597 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x561264389970 0x56126438bd90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=568a0f2c29ad87bb server_cookie=d0e5fb921239d4f8 in_seq=0 out_seq=0 2024-04-24T06:45:51.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.300+0000 7fcdc8c00640 10 client.15654 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:45:51.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.302+0000 7fcdc8c00640 1 -- 172.21.15.5:0/3684850597 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561264274700 con 0x561264389970 2024-04-24T06:45:51.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.302+0000 7fcdc8c00640 10 client.15654 handle_command_reply: tid=0 2024-04-24T06:45:51.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.302+0000 7fcde8fcd740 1 client.15654 shutdown 2024-04-24T06:45:51.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.302+0000 7fcde8fcd740 1 -- 172.21.15.5:0/3684850597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561264307b90 msgr2=0x56126431a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:51.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.302+0000 7fcde8fcd740 1 --2- 172.21.15.5:0/3684850597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561264307b90 0x56126431a990 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc0ce1d0 tx=0x7fcdcc000bc0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.302+0000 7fcde8fcd740 1 -- 172.21.15.5:0/3684850597 shutdown_connections 2024-04-24T06:45:51.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.302+0000 7fcde8fcd740 1 --2- 172.21.15.5:0/3684850597 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x561264389970 0x56126438bd90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.302+0000 7fcde8fcd740 1 --2- 172.21.15.5:0/3684850597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561264312260 0x561264317420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.302+0000 7fcde8fcd740 1 --2- 172.21.15.5:0/3684850597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561264307b90 0x56126431a990 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.302+0000 7fcde8fcd740 1 --2- 172.21.15.5:0/3684850597 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561264307270 0x56126431a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.302+0000 7fcdc8c00640 0 client.15654 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:45:51.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.302+0000 7fcde8fcd740 1 -- 172.21.15.5:0/3684850597 >> 172.21.15.5:0/3684850597 conn(0x561264303590 msgr2=0x56126430f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:51.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.303+0000 7fcde8fcd740 1 -- 172.21.15.5:0/3684850597 shutdown_connections 2024-04-24T06:45:51.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.303+0000 7fcde8fcd740 1 -- 172.21.15.5:0/3684850597 wait complete. 2024-04-24T06:45:51.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.303+0000 7fcde8fcd740 20 client.15654 trim_cache size 0 max 16384 2024-04-24T06:45:51.307 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:45:51.307 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-24T06:45:51.307 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "29bf836a-da1f-4aa3-9396-b02440210d6b", 2024-04-24T06:45:51.307 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-24T06:45:51.307 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:45:51.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.303+0000 7fcde4a00640 1 -- 172.21.15.5:0/1274740588 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fcdc40a3da0 msgr2=0x7fcdc40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:51.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.303+0000 7fcde4a00640 1 --2- 172.21.15.5:0/1274740588 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fcdc40a3da0 0x7fcdc40a6260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0000c00 tx=0x7fcdd0005b50 comp rx=0 tx=0).stop 2024-04-24T06:45:51.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.303+0000 7fcde4a00640 1 -- 172.21.15.5:0/1274740588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcde0074cc0 msgr2=0x7fcde019f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:51.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.303+0000 7fcde4a00640 1 --2- 172.21.15.5:0/1274740588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcde0074cc0 0x7fcde019f300 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00e9f0 tx=0x7fcdcc00eeb0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.304+0000 7fcde4a00640 1 -- 172.21.15.5:0/1274740588 shutdown_connections 2024-04-24T06:45:51.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.304+0000 7fcde4a00640 1 --2- 172.21.15.5:0/1274740588 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fcdc40a3da0 0x7fcdc40a6260 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.304+0000 7fcde4a00640 1 --2- 172.21.15.5:0/1274740588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcde0112de0 0x7fcde01a3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.304+0000 7fcde4a00640 1 --2- 172.21.15.5:0/1274740588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcde0074cc0 0x7fcde019f300 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.304+0000 7fcde4a00640 1 --2- 172.21.15.5:0/1274740588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcde00743a0 0x7fcde019edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.304+0000 7fcde4a00640 1 -- 172.21.15.5:0/1274740588 >> 172.21.15.5:0/1274740588 conn(0x7fcde00fd8e0 msgr2=0x7fcde00ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:51.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.304+0000 7fcde4a00640 1 -- 172.21.15.5:0/1274740588 shutdown_connections 2024-04-24T06:45:51.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.304+0000 7fcde4a00640 1 -- 172.21.15.5:0/1274740588 wait complete. 2024-04-24T06:45:51.324 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T06:45:51.324 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T06:45:51.387 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:51 smithi005 ceph-mon[27798]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:45:51.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:51 smithi137 ceph-mon[33017]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:45:51.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:51 smithi037 ceph-mon[32864]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:45:51.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.523+0000 7fe60b200640 1 -- 172.21.15.5:0/1740607568 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe60c100f60 msgr2=0x7fe60c101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:51.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.523+0000 7fe60b200640 1 --2- 172.21.15.5:0/1740607568 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe60c100f60 0x7fe60c101340 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc009900 tx=0x7fe5fc02a3f0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.523+0000 7fe60b200640 1 -- 172.21.15.5:0/1740607568 shutdown_connections 2024-04-24T06:45:51.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.523+0000 7fe60b200640 1 --2- 172.21.15.5:0/1740607568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe60c106030 0x7fe60c10dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.523+0000 7fe60b200640 1 --2- 172.21.15.5:0/1740607568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe60c101910 0x7fe60c105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.523+0000 7fe60b200640 1 --2- 172.21.15.5:0/1740607568 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe60c100f60 0x7fe60c101340 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.523+0000 7fe60b200640 1 -- 172.21.15.5:0/1740607568 >> 172.21.15.5:0/1740607568 conn(0x7fe60c077ab0 msgr2=0x7fe60c0fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:51.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.523+0000 7fe60b200640 1 -- 172.21.15.5:0/1740607568 shutdown_connections 2024-04-24T06:45:51.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.523+0000 7fe60b200640 1 -- 172.21.15.5:0/1740607568 wait complete. 2024-04-24T06:45:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.524+0000 7fe60b200640 1 Processor -- start 2024-04-24T06:45:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.525+0000 7fe60b200640 1 -- start start 2024-04-24T06:45:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.525+0000 7fe60b200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe60c100f60 0x7fe60c1a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.525+0000 7fe60b200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe60c106030 0x7fe60c1a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.525+0000 7fe60b200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe60c19a790 0x7fe60c19ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.525+0000 7fe60b200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c069e90 con 0x7fe60c19a790 2024-04-24T06:45:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.525+0000 7fe60b200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c069d10 con 0x7fe60c100f60 2024-04-24T06:45:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.525+0000 7fe60b200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c06a010 con 0x7fe60c106030 2024-04-24T06:45:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.525+0000 7fe60a800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe60c19a790 0x7fe60c19ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.526+0000 7fe60a800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe60c19a790 0x7fe60c19ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52760/0 (socket says 172.21.15.5:52760) 2024-04-24T06:45:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.526+0000 7fe60a800640 1 -- 172.21.15.5:0/1551648348 learned_addr learned my addr 172.21.15.5:0/1551648348 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.526+0000 7fe603e00640 1 --2- 172.21.15.5:0/1551648348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe60c106030 0x7fe60c1a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.526+0000 7fe609e00640 1 --2- 172.21.15.5:0/1551648348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe60c100f60 0x7fe60c1a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.526+0000 7fe60a800640 1 -- 172.21.15.5:0/1551648348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe60c106030 msgr2=0x7fe60c1a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.526+0000 7fe60a800640 1 --2- 172.21.15.5:0/1551648348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe60c106030 0x7fe60c1a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.526+0000 7fe60a800640 1 -- 172.21.15.5:0/1551648348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe60c100f60 msgr2=0x7fe60c1a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.526+0000 7fe60a800640 1 --2- 172.21.15.5:0/1551648348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe60c100f60 0x7fe60c1a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.526+0000 7fe60a800640 1 -- 172.21.15.5:0/1551648348 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe60c19b320 con 0x7fe60c19a790 2024-04-24T06:45:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.526+0000 7fe60a800640 1 --2- 172.21.15.5:0/1551648348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe60c19a790 0x7fe60c19ac40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe60400eb10 tx=0x7fe60400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:51.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.527+0000 7fe602a00640 1 -- 172.21.15.5:0/1551648348 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe604004b20 con 0x7fe60c19a790 2024-04-24T06:45:51.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.527+0000 7fe602a00640 1 -- 172.21.15.5:0/1551648348 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe604004cc0 con 0x7fe60c19a790 2024-04-24T06:45:51.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.527+0000 7fe60b200640 1 -- 172.21.15.5:0/1551648348 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe60c19b610 con 0x7fe60c19a790 2024-04-24T06:45:51.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.527+0000 7fe602a00640 1 -- 172.21.15.5:0/1551648348 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe604010760 con 0x7fe60c19a790 2024-04-24T06:45:51.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.527+0000 7fe60b200640 1 -- 172.21.15.5:0/1551648348 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe60c1a7840 con 0x7fe60c19a790 2024-04-24T06:45:51.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.530+0000 7fe6139d0740 1 Processor -- start 2024-04-24T06:45:51.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.530+0000 7fe602a00640 1 -- 172.21.15.5:0/1551648348 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fe604005020 con 0x7fe60c19a790 2024-04-24T06:45:51.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.531+0000 7fe6139d0740 1 -- start start 2024-04-24T06:45:51.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.531+0000 7fe6139d0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c0f097a30 0x560c0f097e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:51.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.531+0000 7fe6139d0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0f098350 0x560c0f0a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:51.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.531+0000 7fe6139d0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0f0a2a20 0x560c0f0a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:51.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.531+0000 7fe6139d0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560c0effd1f0 con 0x560c0f098350 2024-04-24T06:45:51.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.531+0000 7fe6139d0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560c0efa3350 con 0x560c0f097a30 2024-04-24T06:45:51.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.531+0000 7fe6139d0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560c0eff91b0 con 0x560c0f0a2a20 2024-04-24T06:45:51.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.531+0000 7fe603e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c0f097a30 0x560c0f097e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:51.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.531+0000 7fe609e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0f098350 0x560c0f0a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:51.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.531+0000 7fe603e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c0f097a30 0x560c0f097e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:37264/0 (socket says 172.21.15.5:37264) 2024-04-24T06:45:51.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.531+0000 7fe603e00640 1 -- 172.21.15.5:0/3168469604 learned_addr learned my addr 172.21.15.5:0/3168469604 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:51.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.532+0000 7fe603e00640 1 --2- 172.21.15.5:0/3168469604 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0f0a2a20 0x560c0f0a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:51.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.532+0000 7fe609e00640 1 -- 172.21.15.5:0/3168469604 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0f0a2a20 msgr2=0x560c0f0a4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:51.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.532+0000 7fe609e00640 1 --2- 172.21.15.5:0/3168469604 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0f0a2a20 0x560c0f0a4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.532+0000 7fe609e00640 1 -- 172.21.15.5:0/3168469604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c0f097a30 msgr2=0x560c0f097e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:51.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.532+0000 7fe609e00640 1 --2- 172.21.15.5:0/3168469604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c0f097a30 0x560c0f097e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.532+0000 7fe609e00640 1 -- 172.21.15.5:0/3168469604 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c0eff7f50 con 0x560c0f098350 2024-04-24T06:45:51.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.532+0000 7fe603e00640 1 --2- 172.21.15.5:0/3168469604 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0f0a2a20 0x560c0f0a4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:45:51.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.532+0000 7fe602a00640 1 --2- 172.21.15.5:0/1551648348 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fe5f00a3da0 0x7fe5f00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:51.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.532+0000 7fe602a00640 1 -- 172.21.15.5:0/1551648348 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fe6040d0e20 con 0x7fe60c19a790 2024-04-24T06:45:51.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.532+0000 7fe609e00640 1 --2- 172.21.15.5:0/3168469604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0f098350 0x560c0f0a24e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc02a9e0 tx=0x7fe5fc005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6915afbd79347950 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:51.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.532+0000 7fe603e00640 1 --2- 172.21.15.5:0/1551648348 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fe5f00a3da0 0x7fe5f00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:51.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.533+0000 7fe5fb000640 1 -- 172.21.15.5:0/3168469604 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc035040 con 0x560c0f098350 2024-04-24T06:45:51.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.533+0000 7fe5fb000640 1 -- 172.21.15.5:0/3168469604 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5fc004400 con 0x560c0f098350 2024-04-24T06:45:51.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.533+0000 7fe5fb000640 1 -- 172.21.15.5:0/3168469604 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc0035f0 con 0x560c0f098350 2024-04-24T06:45:51.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.534+0000 7fe6139d0740 1 -- 172.21.15.5:0/3168469604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0f098350 msgr2=0x560c0f0a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:51.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.534+0000 7fe603e00640 1 --2- 172.21.15.5:0/1551648348 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fe5f00a3da0 0x7fe5f00a6260 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x560c0f094ee0 tx=0x7fe5f4009770 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:51.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.534+0000 7fe6139d0740 1 --2- 172.21.15.5:0/3168469604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0f098350 0x560c0f0a24e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc02a9e0 tx=0x7fe5fc005de0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.534+0000 7fe6139d0740 1 -- 172.21.15.5:0/3168469604 shutdown_connections 2024-04-24T06:45:51.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.534+0000 7fe6139d0740 1 --2- 172.21.15.5:0/3168469604 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0f0a2a20 0x560c0f0a4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.534+0000 7fe6139d0740 1 --2- 172.21.15.5:0/3168469604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0f098350 0x560c0f0a24e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.534+0000 7fe6139d0740 1 --2- 172.21.15.5:0/3168469604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c0f097a30 0x560c0f097e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.534+0000 7fe6139d0740 1 -- 172.21.15.5:0/3168469604 >> 172.21.15.5:0/3168469604 conn(0x560c0f093d50 msgr2=0x560c0f094130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:51.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.534+0000 7fe6139d0740 1 -- 172.21.15.5:0/3168469604 shutdown_connections 2024-04-24T06:45:51.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.534+0000 7fe6139d0740 1 -- 172.21.15.5:0/3168469604 wait complete. 2024-04-24T06:45:51.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.534+0000 7fe6139d0740 1 -- start start 2024-04-24T06:45:51.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.535+0000 7fe6139d0740 1 Processor -- start 2024-04-24T06:45:51.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.535+0000 7fe6139d0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0f097a30 0x560c0f09fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:51.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.535+0000 7fe6139d0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0f098350 0x560c0f09ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:51.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.535+0000 7fe6139d0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c0f0a2a20 0x560c0f0a0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:51.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.535+0000 7fe6139d0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560c0f0a57b0 con 0x560c0f097a30 2024-04-24T06:45:51.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.535+0000 7fe6139d0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560c0efb6690 con 0x560c0f0a2a20 2024-04-24T06:45:51.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.535+0000 7fe6139d0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560c0f0a5930 con 0x560c0f098350 2024-04-24T06:45:51.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.535+0000 7fe6139d0740 10 client.0 _ll_register_callbacks cb 0x560c0eff2650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:45:51.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.535+0000 7fe609e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0f097a30 0x560c0f09fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:51.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.536+0000 7fe609e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0f097a30 0x560c0f09fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52778/0 (socket says 172.21.15.5:52778) 2024-04-24T06:45:51.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.536+0000 7fe609e00640 1 -- 172.21.15.5:0/4237346130 learned_addr learned my addr 172.21.15.5:0/4237346130 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:51.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.536+0000 7fe603e00640 1 --2- 172.21.15.5:0/4237346130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0f098350 0x560c0f09ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:51.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.536+0000 7fe60a800640 1 --2- 172.21.15.5:0/4237346130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c0f0a2a20 0x560c0f0a0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:51.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.536+0000 7fe609e00640 1 -- 172.21.15.5:0/4237346130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0f098350 msgr2=0x560c0f09ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:51.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.536+0000 7fe609e00640 1 --2- 172.21.15.5:0/4237346130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0f098350 0x560c0f09ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.536+0000 7fe609e00640 1 -- 172.21.15.5:0/4237346130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c0f0a2a20 msgr2=0x560c0f0a0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:51.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.536+0000 7fe609e00640 1 --2- 172.21.15.5:0/4237346130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c0f0a2a20 0x560c0f0a0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.536+0000 7fe609e00640 1 -- 172.21.15.5:0/4237346130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c0efb5920 con 0x560c0f097a30 2024-04-24T06:45:51.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.536+0000 7fe603e00640 1 --2- 172.21.15.5:0/4237346130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0f098350 0x560c0f09ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:45:51.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.536+0000 7fe60a800640 1 --2- 172.21.15.5:0/4237346130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c0f0a2a20 0x560c0f0a0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:45:51.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.536+0000 7fe609e00640 1 --2- 172.21.15.5:0/4237346130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0f097a30 0x560c0f09fa00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc0048f0 tx=0x7fe5fc004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84c8bb919a04004e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:51.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.536+0000 7fe5ef400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:45:51.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.537+0000 7fe5ef400640 1 -- 172.21.15.5:0/4237346130 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc052060 con 0x560c0f097a30 2024-04-24T06:45:51.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.537+0000 7fe6139d0740 1 -- 172.21.15.5:0/4237346130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5fc004400 con 0x560c0f097a30 2024-04-24T06:45:51.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.537+0000 7fe5ef400640 1 -- 172.21.15.5:0/4237346130 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fe5fc004200 con 0x560c0f097a30 2024-04-24T06:45:51.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.537+0000 7fe5ef400640 1 -- 172.21.15.5:0/4237346130 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe5fc035070 con 0x560c0f097a30 2024-04-24T06:45:51.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.537+0000 7fe6139d0740 10 client.15666 fetch_fsmap learned FSMap version 23 2024-04-24T06:45:51.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.537+0000 7fe6139d0740 1 -- 172.21.15.5:0/4237346130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x560c0edd97d0 con 0x560c0f097a30 2024-04-24T06:45:51.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.537+0000 7fe5ef400640 1 -- 172.21.15.5:0/4237346130 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5fc004400 con 0x560c0f097a30 2024-04-24T06:45:51.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.538+0000 7fe5ef400640 1 -- 172.21.15.5:0/4237346130 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fe5fc031a00 con 0x560c0f097a30 2024-04-24T06:45:51.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.538+0000 7fe6139d0740 10 client.15666 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T06:45:51.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.538+0000 7fe6139d0740 10 client.15666 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:45:51.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.538+0000 7fe6139d0740 20 client.15666 populate_metadata read hostname 'smithi005' 2024-04-24T06:45:51.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.538+0000 7fe6139d0740 1 --2- 172.21.15.5:0/4237346130 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x560c0f0b2c20 0x560c0f0b5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:51.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.538+0000 7fe6139d0740 4 client.15666 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:45:51.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.538+0000 7fe6139d0740 1 -- 172.21.15.5:0/4237346130 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c0f00ac10 con 0x560c0f0b2c20 2024-04-24T06:45:51.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.538+0000 7fe603e00640 1 --2- 172.21.15.5:0/4237346130 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x560c0f0b2c20 0x560c0f0b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:51.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.539+0000 7fe603e00640 1 --2- 172.21.15.5:0/4237346130 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x560c0f0b2c20 0x560c0f0b5010 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a92983bf246d8ceb server_cookie=aec4f690cf1e3960 in_seq=0 out_seq=0 2024-04-24T06:45:51.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.539+0000 7fe5ef400640 10 client.15666 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:45:51.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.539+0000 7fe5ef400640 1 -- 172.21.15.5:0/4237346130 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560c0f00ac10 con 0x560c0f0b2c20 2024-04-24T06:45:51.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.539+0000 7fe5ef400640 10 client.15666 handle_command_reply: tid=0 2024-04-24T06:45:51.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.540+0000 7fe6139d0740 1 client.15666 shutdown 2024-04-24T06:45:51.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.540+0000 7fe6139d0740 1 -- 172.21.15.5:0/4237346130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0f097a30 msgr2=0x560c0f09fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:51.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.540+0000 7fe6139d0740 1 --2- 172.21.15.5:0/4237346130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0f097a30 0x560c0f09fa00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc0048f0 tx=0x7fe5fc004920 comp rx=0 tx=0).stop 2024-04-24T06:45:51.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.540+0000 7fe6139d0740 1 -- 172.21.15.5:0/4237346130 shutdown_connections 2024-04-24T06:45:51.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.540+0000 7fe6139d0740 1 --2- 172.21.15.5:0/4237346130 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x560c0f0b2c20 0x560c0f0b5010 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.540+0000 7fe6139d0740 1 --2- 172.21.15.5:0/4237346130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c0f0a2a20 0x560c0f0a0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.540+0000 7fe6139d0740 1 --2- 172.21.15.5:0/4237346130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0f098350 0x560c0f09ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.540+0000 7fe6139d0740 1 --2- 172.21.15.5:0/4237346130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0f097a30 0x560c0f09fa00 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.540+0000 7fe5ef400640 0 client.15666 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:45:51.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.540+0000 7fe6139d0740 1 -- 172.21.15.5:0/4237346130 >> 172.21.15.5:0/4237346130 conn(0x560c0f093d50 msgr2=0x560c0f099b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:51.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.540+0000 7fe6139d0740 1 -- 172.21.15.5:0/4237346130 shutdown_connections 2024-04-24T06:45:51.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.540+0000 7fe6139d0740 1 -- 172.21.15.5:0/4237346130 wait complete. 2024-04-24T06:45:51.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.540+0000 7fe6139d0740 20 client.15666 trim_cache size 0 max 16384 2024-04-24T06:45:51.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.559+0000 7fe6139d0740 1 Processor -- start 2024-04-24T06:45:51.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.559+0000 7fe6139d0740 1 -- start start 2024-04-24T06:45:51.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.559+0000 7fe6139d0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0f097a30 0x560c0f0ae670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:51.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.559+0000 7fe6139d0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0f098350 0x560c0f094130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:51.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.559+0000 7fe6139d0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c0f0a2a20 0x560c0f099960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:51.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.559+0000 7fe6139d0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c1b25f0 con 0x560c0f098350 2024-04-24T06:45:51.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.559+0000 7fe6139d0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c1afec0 con 0x560c0f0a2a20 2024-04-24T06:45:51.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.559+0000 7fe6139d0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c1add10 con 0x560c0f097a30 2024-04-24T06:45:51.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.560+0000 7fe603e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0f098350 0x560c0f094130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:51.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.560+0000 7fe603e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0f098350 0x560c0f094130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52784/0 (socket says 172.21.15.5:52784) 2024-04-24T06:45:51.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.560+0000 7fe603e00640 1 -- 172.21.15.5:0/1953943416 learned_addr learned my addr 172.21.15.5:0/1953943416 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:51.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.560+0000 7fe60a800640 1 --2- 172.21.15.5:0/1953943416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c0f0a2a20 0x560c0f099960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:51.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.560+0000 7fe609e00640 1 --2- 172.21.15.5:0/1953943416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0f097a30 0x560c0f0ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:51.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.560+0000 7fe603e00640 1 -- 172.21.15.5:0/1953943416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0f097a30 msgr2=0x560c0f0ae670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:51.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.560+0000 7fe603e00640 1 --2- 172.21.15.5:0/1953943416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0f097a30 0x560c0f0ae670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.560+0000 7fe603e00640 1 -- 172.21.15.5:0/1953943416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c0f0a2a20 msgr2=0x560c0f099960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:51.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.560+0000 7fe603e00640 1 --2- 172.21.15.5:0/1953943416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c0f0a2a20 0x560c0f099960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.560+0000 7fe603e00640 1 -- 172.21.15.5:0/1953943416 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c0f09a7f0 con 0x560c0f098350 2024-04-24T06:45:51.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.560+0000 7fe603e00640 1 --2- 172.21.15.5:0/1953943416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0f098350 0x560c0f094130 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe5f40097e0 tx=0x7fe5f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d23f223994c6b945 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:51.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.561+0000 7fe5fa600640 1 -- 172.21.15.5:0/1953943416 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f402b410 con 0x560c0f098350 2024-04-24T06:45:51.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.561+0000 7fe5fa600640 1 -- 172.21.15.5:0/1953943416 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5f4005020 con 0x560c0f098350 2024-04-24T06:45:51.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.561+0000 7fe5fa600640 1 -- 172.21.15.5:0/1953943416 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f4004890 con 0x560c0f098350 2024-04-24T06:45:51.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.561+0000 7fe6139d0740 1 -- 172.21.15.5:0/1953943416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0f098350 msgr2=0x560c0f094130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:51.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.561+0000 7fe6139d0740 1 --2- 172.21.15.5:0/1953943416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0f098350 0x560c0f094130 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe5f40097e0 tx=0x7fe5f40046c0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.562+0000 7fe6139d0740 1 -- 172.21.15.5:0/1953943416 shutdown_connections 2024-04-24T06:45:51.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.562+0000 7fe6139d0740 1 --2- 172.21.15.5:0/1953943416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c0f0a2a20 0x560c0f099960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.562+0000 7fe6139d0740 1 --2- 172.21.15.5:0/1953943416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0f098350 0x560c0f094130 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.562+0000 7fe6139d0740 1 --2- 172.21.15.5:0/1953943416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0f097a30 0x560c0f0ae670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.562+0000 7fe6139d0740 1 -- 172.21.15.5:0/1953943416 >> 172.21.15.5:0/1953943416 conn(0x560c0f093d50 msgr2=0x560c0f099250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:51.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.562+0000 7fe6139d0740 1 -- 172.21.15.5:0/1953943416 shutdown_connections 2024-04-24T06:45:51.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.562+0000 7fe6139d0740 1 -- 172.21.15.5:0/1953943416 wait complete. 2024-04-24T06:45:51.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.562+0000 7fe6139d0740 1 -- start start 2024-04-24T06:45:51.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.562+0000 7fe6139d0740 1 Processor -- start 2024-04-24T06:45:51.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.563+0000 7fe6139d0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0f097a30 0x560c0f0aad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:51.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.563+0000 7fe6139d0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c0f098350 0x560c0f09c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:51.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.563+0000 7fe6139d0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0f0a2a20 0x560c0f09c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:51.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.563+0000 7fe6139d0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c1ae380 con 0x560c0f0a2a20 2024-04-24T06:45:51.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.563+0000 7fe6139d0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c1af7f0 con 0x560c0f098350 2024-04-24T06:45:51.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.563+0000 7fe6139d0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c1aea50 con 0x560c0f097a30 2024-04-24T06:45:51.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.563+0000 7fe6139d0740 10 client.0 _ll_register_callbacks cb 0x7fe5f4009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:45:51.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.563+0000 7fe60a800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0f0a2a20 0x560c0f09c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:51.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.563+0000 7fe603e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c0f098350 0x560c0f09c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:51.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.563+0000 7fe60a800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0f0a2a20 0x560c0f09c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52794/0 (socket says 172.21.15.5:52794) 2024-04-24T06:45:51.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.563+0000 7fe60a800640 1 -- 172.21.15.5:0/2283771584 learned_addr learned my addr 172.21.15.5:0/2283771584 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:45:51.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.563+0000 7fe609e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0f097a30 0x560c0f0aad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:51.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.564+0000 7fe60a800640 1 -- 172.21.15.5:0/2283771584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0f097a30 msgr2=0x560c0f0aad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:51.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.564+0000 7fe60a800640 1 --2- 172.21.15.5:0/2283771584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0f097a30 0x560c0f0aad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.564+0000 7fe60a800640 1 -- 172.21.15.5:0/2283771584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c0f098350 msgr2=0x560c0f09c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:45:51.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.564+0000 7fe60a800640 1 --2- 172.21.15.5:0/2283771584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c0f098350 0x560c0f09c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.564+0000 7fe60a800640 1 -- 172.21.15.5:0/2283771584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c0ed0f3f0 con 0x560c0f0a2a20 2024-04-24T06:45:51.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.564+0000 7fe609e00640 1 --2- 172.21.15.5:0/2283771584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0f097a30 0x560c0f0aad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:45:51.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.564+0000 7fe603e00640 1 --2- 172.21.15.5:0/2283771584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c0f098350 0x560c0f09c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:45:51.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.564+0000 7fe60a800640 1 --2- 172.21.15.5:0/2283771584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0f0a2a20 0x560c0f09c6f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe604004af0 tx=0x7fe60409c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c6de40973a9f9b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:45:51.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.564+0000 7fe5ef400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:45:51.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.564+0000 7fe5ef400640 1 -- 172.21.15.5:0/2283771584 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe604005510 con 0x560c0f0a2a20 2024-04-24T06:45:51.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.564+0000 7fe5ef400640 1 -- 172.21.15.5:0/2283771584 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fe60409c1c0 con 0x560c0f0a2a20 2024-04-24T06:45:51.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.564+0000 7fe6139d0740 1 -- 172.21.15.5:0/2283771584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5f4005020 con 0x560c0f0a2a20 2024-04-24T06:45:51.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.564+0000 7fe5ef400640 1 -- 172.21.15.5:0/2283771584 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe6040a1120 con 0x560c0f0a2a20 2024-04-24T06:45:51.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.564+0000 7fe6139d0740 10 client.15672 fetch_fsmap learned FSMap version 23 2024-04-24T06:45:51.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.564+0000 7fe6139d0740 1 -- 172.21.15.5:0/2283771584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x560c0eccf0f0 con 0x560c0f0a2a20 2024-04-24T06:45:51.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.565+0000 7fe5ef400640 1 -- 172.21.15.5:0/2283771584 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5f4005020 con 0x560c0f0a2a20 2024-04-24T06:45:51.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.565+0000 7fe5ef400640 1 -- 172.21.15.5:0/2283771584 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fe60407bc20 con 0x560c0f0a2a20 2024-04-24T06:45:51.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.565+0000 7fe6139d0740 10 client.15672 fetch_fsmap finished waiting for FSMap version 23 2024-04-24T06:45:51.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.565+0000 7fe6139d0740 10 client.15672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:45:51.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.565+0000 7fe6139d0740 20 client.15672 populate_metadata read hostname 'smithi005' 2024-04-24T06:45:51.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.565+0000 7fe6139d0740 1 --2- 172.21.15.5:0/2283771584 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x560c0f119030 0x560c0f11b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:45:51.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.565+0000 7fe6139d0740 4 client.15672 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:45:51.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.565+0000 7fe6139d0740 1 -- 172.21.15.5:0/2283771584 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560c0f004a80 con 0x560c0f119030 2024-04-24T06:45:51.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.565+0000 7fe609e00640 1 --2- 172.21.15.5:0/2283771584 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x560c0f119030 0x560c0f11b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:45:51.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.566+0000 7fe609e00640 1 --2- 172.21.15.5:0/2283771584 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x560c0f119030 0x560c0f11b450 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=119b9722b550ab2f server_cookie=e332a2d015f73b4f in_seq=0 out_seq=0 2024-04-24T06:45:51.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.566+0000 7fe5ef400640 10 client.15672 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:45:51.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.566+0000 7fe5ef400640 1 -- 172.21.15.5:0/2283771584 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560c0f004a80 con 0x560c0f119030 2024-04-24T06:45:51.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.566+0000 7fe5ef400640 10 client.15672 handle_command_reply: tid=0 2024-04-24T06:45:51.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.566+0000 7fe6139d0740 1 client.15672 shutdown 2024-04-24T06:45:51.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.566+0000 7fe6139d0740 1 -- 172.21.15.5:0/2283771584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0f0a2a20 msgr2=0x560c0f09c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:51.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.566+0000 7fe6139d0740 1 --2- 172.21.15.5:0/2283771584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0f0a2a20 0x560c0f09c6f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe604004af0 tx=0x7fe60409c6d0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.566+0000 7fe6139d0740 1 -- 172.21.15.5:0/2283771584 shutdown_connections 2024-04-24T06:45:51.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.566+0000 7fe6139d0740 1 --2- 172.21.15.5:0/2283771584 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x560c0f119030 0x560c0f11b450 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.566+0000 7fe6139d0740 1 --2- 172.21.15.5:0/2283771584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0f0a2a20 0x560c0f09c6f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.566+0000 7fe6139d0740 1 --2- 172.21.15.5:0/2283771584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c0f098350 0x560c0f09c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.566+0000 7fe6139d0740 1 --2- 172.21.15.5:0/2283771584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560c0f097a30 0x560c0f0aad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.566+0000 7fe5ef400640 0 client.15672 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:45:51.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.566+0000 7fe6139d0740 1 -- 172.21.15.5:0/2283771584 >> 172.21.15.5:0/2283771584 conn(0x560c0f093d50 msgr2=0x560c0f092fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:51.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.567+0000 7fe6139d0740 1 -- 172.21.15.5:0/2283771584 shutdown_connections 2024-04-24T06:45:51.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.567+0000 7fe6139d0740 1 -- 172.21.15.5:0/2283771584 wait complete. 2024-04-24T06:45:51.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.567+0000 7fe6139d0740 20 client.15672 trim_cache size 0 max 16384 2024-04-24T06:45:51.572 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:45:51.572 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T06:45:51.572 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-24T06:45:51.572 INFO:teuthology.orchestra.run.smithi005.stdout: "29bf836a-da1f-4aa3-9396-b02440210d6b": { 2024-04-24T06:45:51.572 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-24T06:45:51.572 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "29bf836a-da1f-4aa3-9396-b02440210d6b", 2024-04-24T06:45:51.572 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-04-24T06:45:51.572 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:45:51.572 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:45:51.572 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:45:51.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.567+0000 7fe60b200640 1 -- 172.21.15.5:0/1551648348 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fe5f00a3da0 msgr2=0x7fe5f00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:51.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.567+0000 7fe60b200640 1 --2- 172.21.15.5:0/1551648348 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fe5f00a3da0 0x7fe5f00a6260 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x560c0f094ee0 tx=0x7fe5f4009770 comp rx=0 tx=0).stop 2024-04-24T06:45:51.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.567+0000 7fe60b200640 1 -- 172.21.15.5:0/1551648348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe60c19a790 msgr2=0x7fe60c19ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:45:51.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.567+0000 7fe60b200640 1 --2- 172.21.15.5:0/1551648348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe60c19a790 0x7fe60c19ac40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe60400eb10 tx=0x7fe60400c2d0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.567+0000 7fe60b200640 1 -- 172.21.15.5:0/1551648348 shutdown_connections 2024-04-24T06:45:51.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.567+0000 7fe60b200640 1 --2- 172.21.15.5:0/1551648348 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fe5f00a3da0 0x7fe5f00a6260 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.567+0000 7fe60b200640 1 --2- 172.21.15.5:0/1551648348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe60c19a790 0x7fe60c19ac40 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.567+0000 7fe60b200640 1 --2- 172.21.15.5:0/1551648348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe60c106030 0x7fe60c1a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.567+0000 7fe60b200640 1 --2- 172.21.15.5:0/1551648348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe60c100f60 0x7fe60c1a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:45:51.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.567+0000 7fe60b200640 1 -- 172.21.15.5:0/1551648348 >> 172.21.15.5:0/1551648348 conn(0x7fe60c077ab0 msgr2=0x7fe60c104830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:45:51.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.568+0000 7fe60b200640 1 -- 172.21.15.5:0/1551648348 shutdown_connections 2024-04-24T06:45:51.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:45:51.568+0000 7fe60b200640 1 -- 172.21.15.5:0/1551648348 wait complete. 2024-04-24T06:45:51.585 INFO:tasks.cephfs.filesystem:scrub status for tag:29bf836a-da1f-4aa3-9396-b02440210d6b - {'path': '/', 'tag': '29bf836a-da1f-4aa3-9396-b02440210d6b', 'options': 'recursive,force'} 2024-04-24T06:45:51.639 INFO:tasks.workunit.client.0.smithi005.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-24T06:45:51.744 INFO:tasks.workunit.client.0.smithi005.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-24T06:45:51.835 INFO:tasks.workunit.client.0.smithi005.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-24T06:45:51.842 INFO:tasks.workunit.client.0.smithi005.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-24T06:45:51.842 INFO:tasks.workunit.client.0.smithi005.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-24T06:45:51.929 INFO:tasks.workunit.client.0.smithi005.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-24T06:45:51.939 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-24T06:45:51.939 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-24T06:45:51.992 INFO:tasks.workunit:Running workunits matching suites/fsx.sh on client.0... 2024-04-24T06:45:51.993 INFO:tasks.workunit:Running workunit suites/fsx.sh... 2024-04-24T06:45:51.994 DEBUG:teuthology.orchestra.run.smithi005:workunit test suites/fsx.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a3c8a22bd5efe18ab230a8e2d072db8f381a0411 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh 2024-04-24T06:45:52.067 INFO:tasks.workunit.client.0.smithi005.stderr:+ set -e 2024-04-24T06:45:52.068 INFO:tasks.workunit.client.0.smithi005.stderr:+ git clone https://git.ceph.com/xfstests-dev.git 2024-04-24T06:45:52.074 INFO:tasks.workunit.client.0.smithi005.stderr:Cloning into 'xfstests-dev'... 2024-04-24T06:45:52.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:52 smithi137 ceph-mon[33017]: scrub queued for path: / 2024-04-24T06:45:52.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:52 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:45:52.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:52 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:45:52.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:52 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:45:52.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:52 smithi005 ceph-mon[27798]: scrub queued for path: / 2024-04-24T06:45:52.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:52 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:45:52.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:52 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:45:52.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:52 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:45:52.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:52 smithi037 ceph-mon[32864]: scrub queued for path: / 2024-04-24T06:45:52.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:52 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:45:52.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:52 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:45:52.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:52 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:45:53.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:53 smithi137 ceph-mon[33017]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:45:53.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:53 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:45:53.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:53 smithi005 ceph-mon[27798]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:45:53.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:53 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:45:53.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:53 smithi037 ceph-mon[32864]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:45:53.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:53 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:45:55.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:55 smithi137 ceph-mon[33017]: pgmap v59: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 8 op/s 2024-04-24T06:45:55.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:55 smithi005 ceph-mon[27798]: pgmap v59: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 8 op/s 2024-04-24T06:45:55.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:55 smithi037 ceph-mon[32864]: pgmap v59: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 8 op/s 2024-04-24T06:45:57.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:57 smithi137 ceph-mon[33017]: pgmap v60: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 6 op/s 2024-04-24T06:45:57.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:57 smithi005 ceph-mon[27798]: pgmap v60: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 6 op/s 2024-04-24T06:45:57.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:57 smithi037 ceph-mon[32864]: pgmap v60: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 6 op/s 2024-04-24T06:45:59.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:59 smithi137 ceph-mon[33017]: pgmap v61: 97 pgs: 97 active+clean; 15 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.2 MiB/s wr, 13 op/s 2024-04-24T06:45:59.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:45:59 smithi137 ceph-mon[33017]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T06:45:59.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:59 smithi005 ceph-mon[27798]: pgmap v61: 97 pgs: 97 active+clean; 15 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.2 MiB/s wr, 13 op/s 2024-04-24T06:45:59.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:45:59 smithi005 ceph-mon[27798]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T06:45:59.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:59 smithi037 ceph-mon[32864]: pgmap v61: 97 pgs: 97 active+clean; 15 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.2 MiB/s wr, 13 op/s 2024-04-24T06:45:59.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:45:59 smithi037 ceph-mon[32864]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T06:46:01.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:01 smithi137 ceph-mon[33017]: pgmap v62: 97 pgs: 97 active+clean; 15 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 1.2 MiB/s wr, 10 op/s 2024-04-24T06:46:01.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:01 smithi005 ceph-mon[27798]: pgmap v62: 97 pgs: 97 active+clean; 15 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 1.2 MiB/s wr, 10 op/s 2024-04-24T06:46:01.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:01 smithi037 ceph-mon[32864]: pgmap v62: 97 pgs: 97 active+clean; 15 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 1.2 MiB/s wr, 10 op/s 2024-04-24T06:46:03.445 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:03 smithi137 ceph-mon[33017]: pgmap v63: 97 pgs: 97 active+clean; 15 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 1.2 MiB/s wr, 10 op/s 2024-04-24T06:46:03.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:03 smithi005 ceph-mon[27798]: pgmap v63: 97 pgs: 97 active+clean; 15 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 1.2 MiB/s wr, 10 op/s 2024-04-24T06:46:03.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:03 smithi037 ceph-mon[32864]: pgmap v63: 97 pgs: 97 active+clean; 15 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 1.2 MiB/s wr, 10 op/s 2024-04-24T06:46:04.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:04 smithi137 ceph-mon[33017]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T06:46:04.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:04 smithi005 ceph-mon[27798]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T06:46:04.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:04 smithi037 ceph-mon[32864]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T06:46:05.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:05 smithi137 ceph-mon[33017]: pgmap v64: 97 pgs: 97 active+clean; 19 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 1.5 MiB/s wr, 35 op/s 2024-04-24T06:46:05.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:05 smithi005 ceph-mon[27798]: pgmap v64: 97 pgs: 97 active+clean; 19 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 1.5 MiB/s wr, 35 op/s 2024-04-24T06:46:05.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:05 smithi037 ceph-mon[32864]: pgmap v64: 97 pgs: 97 active+clean; 19 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 1.5 MiB/s wr, 35 op/s 2024-04-24T06:46:07.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:07 smithi005 ceph-mon[27798]: pgmap v65: 97 pgs: 97 active+clean; 19 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 1.5 MiB/s wr, 31 op/s 2024-04-24T06:46:07.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:07 smithi037 ceph-mon[32864]: pgmap v65: 97 pgs: 97 active+clean; 19 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 1.5 MiB/s wr, 31 op/s 2024-04-24T06:46:07.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:07 smithi137 ceph-mon[33017]: pgmap v65: 97 pgs: 97 active+clean; 19 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 1.5 MiB/s wr, 31 op/s 2024-04-24T06:46:09.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:09 smithi005 ceph-mon[27798]: pgmap v66: 97 pgs: 97 active+clean; 22 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 1.9 MiB/s wr, 56 op/s 2024-04-24T06:46:09.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:09 smithi037 ceph-mon[32864]: pgmap v66: 97 pgs: 97 active+clean; 22 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 1.9 MiB/s wr, 56 op/s 2024-04-24T06:46:09.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:09 smithi137 ceph-mon[33017]: pgmap v66: 97 pgs: 97 active+clean; 22 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 1.9 MiB/s wr, 56 op/s 2024-04-24T06:46:10.738 INFO:tasks.workunit.client.0.smithi005.stderr:Updating files: 7% (321/4290) Updating files: 8% (344/4290) Updating files: 9% (387/4290) Updating files: 10% (429/4290) Updating files: 11% (472/4290) Updating files: 12% (515/4290) Updating files: 13% (558/4290) Updating files: 14% (601/4290) Updating files: 15% (644/4290) Updating files: 16% (687/4290) Updating files: 17% (730/4290) Updating files: 18% (773/4290) Updating files: 19% (816/4290) Updating files: 19% (857/4290) Updating files: 20% (858/4290) Updating files: 21% (901/4290) Updating files: 22% (944/4290) Updating files: 23% (987/4290) Updating files: 24% (1030/4290) Updating files: 25% (1073/4290) Updating files: 26% (1116/4290) Updating files: 27% (1159/4290) Updating files: 28% (1202/4290) Updating files: 29% (1245/4290) Updating files: 30% (1287/4290) Updating files: 30% (1288/4290) Updating files: 31% (1330/4290) Updating files: 32% (1373/4290) Updating files: 33% (1416/4290) Updating files: 34% (1459/4290) Updating files: 35% (1502/4290) Updating files: 36% (1545/4290) Updating files: 37% (1588/4290) Updating files: 37% (1618/4290) Updating files: 38% (1631/4290) Updating files: 39% (1674/4290) Updating files: 40% (1716/4290) Updating files: 41% (1759/4290) Updating files: 42% (1802/4290) Updating files: 43% (1845/4290) Updating files: 44% (1888/4290) Updating files: 45% (1931/4290) Updating files: 46% (1974/4290) Updating files: 47% (2017/4290) Updating files: 48% (2060/4290) Updating files: 48% (2090/4290) Updating files: 49% (2103/4290) Updating files: 50% (2145/4290) Updating files: 51% (2188/4290) Updating files: 52% (2231/4290) Updating files: 53% (2274/4290) Updating files: 54% (2317/4290) Updating files: 55% (2360/4290) Updating files: 56% (2403/4290) Updating files: 57% (2446/4290) Updating files: 57% (2467/4290) Updating files: 58% (2489/4290) Updating files: 59% (2532/4290) Updating files: 60% (2574/4290) Updating files: 61% (2617/4290) Updating files: 62% (2660/4290) Updating files: 63% (2703/4290) Updating files: 64% (2746/4290) Updating files: 65% (2789/4290) Updating files: 66% (2832/4290) Updating files: 67% (2875/4290) Updating files: 68% (2918/4290) Updating files: 69% (2961/4290) Updating files: 69% (2973/4290) Updating files: 70% (3003/4290) Updating files: 70% (3022/4290) Updating files: 70% (3032/4290) Updating files: 71% (3046/4290) Updating files: 72% (3089/4290) Updating files: 73% (3132/4290) Updating files: 74% (3175/4290) Updating files: 75% (3218/4290) Updating files: 76% (3261/4290) Updating files: 77% (3304/4290) Updating files: 78% (3347/4290) Updating files: 79% (3390/4290) Updating files: 80% (3432/4290) Updating files: 81% (3475/4290) Updating files: 82% (3518/4290) Updating files: 83% (3561/4290) Updating files: 83% (3584/4290) Updating files: 84% (3604/4290) Updating files: 85% (3647/4290) Updating files: 86% (3690/4290) Updating files: 87% (3733/4290) Updating files: 88% (3776/4290) Updating files: 89% (3819/4290) Updating files: 90% (3861/4290) Updating files: 91% (3904/4290) Updating files: 92% (3947/4290) Updating files: 92% (3969/4290) Updating files: 93% (3990/4290) Updating files: 94% (4033/4290) Updating files: 95% (4076/4290) Updating files: 96% (4119/4290) Updating files: 97% (4162/4290) Updating files: 98% (4205/4290) Updating files: 99% (4248/4290) Updating files: 100% (4290/4290) Updating files: 100% (4290/4290), done. 2024-04-24T06:46:10.759 INFO:tasks.workunit.client.0.smithi005.stderr:+ cd xfstests-dev 2024-04-24T06:46:10.759 INFO:tasks.workunit.client.0.smithi005.stderr:+ git checkout 12973fc04fd10d4af086901e10ffa8e48866b735 2024-04-24T06:46:11.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:11 smithi005 ceph-mon[27798]: pgmap v67: 97 pgs: 97 active+clean; 22 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 760 KiB/s wr, 48 op/s 2024-04-24T06:46:11.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:11 smithi037 ceph-mon[32864]: pgmap v67: 97 pgs: 97 active+clean; 22 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 760 KiB/s wr, 48 op/s 2024-04-24T06:46:11.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:11 smithi137 ceph-mon[33017]: pgmap v67: 97 pgs: 97 active+clean; 22 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 760 KiB/s wr, 48 op/s 2024-04-24T06:46:13.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:13 smithi005 ceph-mon[27798]: pgmap v68: 97 pgs: 97 active+clean; 22 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 759 KiB/s wr, 48 op/s 2024-04-24T06:46:13.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:13 smithi037 ceph-mon[32864]: pgmap v68: 97 pgs: 97 active+clean; 22 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 759 KiB/s wr, 48 op/s 2024-04-24T06:46:13.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:13 smithi137 ceph-mon[33017]: pgmap v68: 97 pgs: 97 active+clean; 22 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 759 KiB/s wr, 48 op/s 2024-04-24T06:46:14.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:14 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:46:14.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:14 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:46:14.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:14 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:46:15.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:15 smithi005 ceph-mon[27798]: pgmap v69: 97 pgs: 97 active+clean; 28 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 1.3 MiB/s wr, 67 op/s 2024-04-24T06:46:15.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:15 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:46:15.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:15 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:46:15.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:15 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:46:15.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:15 smithi037 ceph-mon[32864]: pgmap v69: 97 pgs: 97 active+clean; 28 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 1.3 MiB/s wr, 67 op/s 2024-04-24T06:46:15.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:15 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:46:15.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:15 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:46:15.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:15 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:46:15.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:15 smithi137 ceph-mon[33017]: pgmap v69: 97 pgs: 97 active+clean; 28 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 1.3 MiB/s wr, 67 op/s 2024-04-24T06:46:15.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:15 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:46:15.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:15 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:46:15.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:15 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:46:17.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:17 smithi005 ceph-mon[27798]: pgmap v70: 97 pgs: 97 active+clean; 28 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 920 KiB/s wr, 43 op/s 2024-04-24T06:46:17.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:17 smithi037 ceph-mon[32864]: pgmap v70: 97 pgs: 97 active+clean; 28 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 920 KiB/s wr, 43 op/s 2024-04-24T06:46:17.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:17 smithi137 ceph-mon[33017]: pgmap v70: 97 pgs: 97 active+clean; 28 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 920 KiB/s wr, 43 op/s 2024-04-24T06:46:19.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:19 smithi005 ceph-mon[27798]: pgmap v71: 97 pgs: 97 active+clean; 37 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 50 op/s 2024-04-24T06:46:19.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:19 smithi037 ceph-mon[32864]: pgmap v71: 97 pgs: 97 active+clean; 37 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 50 op/s 2024-04-24T06:46:19.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:19 smithi137 ceph-mon[33017]: pgmap v71: 97 pgs: 97 active+clean; 37 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 50 op/s 2024-04-24T06:46:21.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:21 smithi005 ceph-mon[27798]: pgmap v72: 97 pgs: 97 active+clean; 37 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 26 op/s 2024-04-24T06:46:21.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:21 smithi037 ceph-mon[32864]: pgmap v72: 97 pgs: 97 active+clean; 37 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 26 op/s 2024-04-24T06:46:21.586 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T06:46:21.586 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T06:46:21.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:21 smithi137 ceph-mon[33017]: pgmap v72: 97 pgs: 97 active+clean; 37 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 26 op/s 2024-04-24T06:46:21.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.832+0000 7f36aaa00640 1 -- 172.21.15.5:0/1982929054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36a4104440 msgr2=0x7f36a4104820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:21.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.832+0000 7f36aaa00640 1 --2- 172.21.15.5:0/1982929054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36a4104440 0x7f36a4104820 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3694009900 tx=0x7f369402a410 comp rx=0 tx=0).stop 2024-04-24T06:46:21.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.832+0000 7f36aaa00640 1 -- 172.21.15.5:0/1982929054 shutdown_connections 2024-04-24T06:46:21.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.832+0000 7f36aaa00640 1 --2- 172.21.15.5:0/1982929054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36a4109510 0x7f36a41110e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:21.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.832+0000 7f36aaa00640 1 --2- 172.21.15.5:0/1982929054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36a4104df0 0x7f36a4108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:21.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.832+0000 7f36aaa00640 1 --2- 172.21.15.5:0/1982929054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36a4104440 0x7f36a4104820 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:21.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.832+0000 7f36aaa00640 1 -- 172.21.15.5:0/1982929054 >> 172.21.15.5:0/1982929054 conn(0x7f36a40ffc80 msgr2=0x7f36a41020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:21.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.832+0000 7f36aaa00640 1 -- 172.21.15.5:0/1982929054 shutdown_connections 2024-04-24T06:46:21.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.832+0000 7f36aaa00640 1 -- 172.21.15.5:0/1982929054 wait complete. 2024-04-24T06:46:21.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.833+0000 7f36aaa00640 1 Processor -- start 2024-04-24T06:46:21.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.834+0000 7f36aaa00640 1 -- start start 2024-04-24T06:46:21.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.834+0000 7f36aaa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36a4104440 0x7f36a41a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:21.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.834+0000 7f36aaa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36a4109510 0x7f36a41a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:21.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.834+0000 7f36aaa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36a419aa30 0x7f36a419aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:21.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.834+0000 7f36aaa00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f36a4114690 con 0x7f36a4104440 2024-04-24T06:46:21.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.834+0000 7f36aaa00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f36a4114510 con 0x7f36a4109510 2024-04-24T06:46:21.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.834+0000 7f36aaa00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f36a4114810 con 0x7f36a419aa30 2024-04-24T06:46:21.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.834+0000 7f36a3400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36a4109510 0x7f36a41a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:21.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.834+0000 7f36a3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36a4104440 0x7f36a41a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:21.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.834+0000 7f36a3400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36a4109510 0x7f36a41a0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:36986/0 (socket says 172.21.15.5:36986) 2024-04-24T06:46:21.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.834+0000 7f36a3400640 1 -- 172.21.15.5:0/962771010 learned_addr learned my addr 172.21.15.5:0/962771010 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:21.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.834+0000 7f36a8a00640 1 --2- 172.21.15.5:0/962771010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36a419aa30 0x7f36a419aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.835+0000 7f36a3400640 1 -- 172.21.15.5:0/962771010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36a419aa30 msgr2=0x7f36a419aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.835+0000 7f36a3400640 1 --2- 172.21.15.5:0/962771010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36a419aa30 0x7f36a419aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.835+0000 7f36a3400640 1 -- 172.21.15.5:0/962771010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36a4104440 msgr2=0x7f36a41a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.835+0000 7f36a3400640 1 --2- 172.21.15.5:0/962771010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36a4104440 0x7f36a41a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.835+0000 7f36a3400640 1 -- 172.21.15.5:0/962771010 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a419b550 con 0x7f36a4109510 2024-04-24T06:46:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.835+0000 7f36a3400640 1 --2- 172.21.15.5:0/962771010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36a4109510 0x7f36a41a0dc0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f369000e9f0 tx=0x7f369000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:21.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.835+0000 7f36a0c00640 1 -- 172.21.15.5:0/962771010 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369000ccc0 con 0x7f36a4109510 2024-04-24T06:46:21.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.835+0000 7f36aaa00640 1 -- 172.21.15.5:0/962771010 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36a419b840 con 0x7f36a4109510 2024-04-24T06:46:21.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.835+0000 7f36a0c00640 1 -- 172.21.15.5:0/962771010 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3690005500 con 0x7f36a4109510 2024-04-24T06:46:21.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.835+0000 7f36a0c00640 1 -- 172.21.15.5:0/962771010 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3690010690 con 0x7f36a4109510 2024-04-24T06:46:21.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.835+0000 7f36aaa00640 1 -- 172.21.15.5:0/962771010 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36a41a7a40 con 0x7f36a4109510 2024-04-24T06:46:21.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.839+0000 7f36ad779740 1 Processor -- start 2024-04-24T06:46:21.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.839+0000 7f36ad779740 1 -- start start 2024-04-24T06:46:21.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.839+0000 7f36a0c00640 1 -- 172.21.15.5:0/962771010 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f36900046c0 con 0x7f36a4109510 2024-04-24T06:46:21.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.839+0000 7f36ad779740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ca8a1860 0x5593ca8a1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:21.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.840+0000 7f36ad779740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ca8a2180 0x5593ca8ac310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:21.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.840+0000 7f36ad779740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593ca8ac850 0x5593ca8aec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:21.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.840+0000 7f36ad779740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5593ca8071f0 con 0x5593ca8a2180 2024-04-24T06:46:21.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.840+0000 7f36ad779740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5593ca7ad350 con 0x5593ca8ac850 2024-04-24T06:46:21.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.840+0000 7f36ad779740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5593ca8031b0 con 0x5593ca8a1860 2024-04-24T06:46:21.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.840+0000 7f36a8a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ca8a1860 0x5593ca8a1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:21.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.840+0000 7f36a8a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ca8a1860 0x5593ca8a1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:42708/0 (socket says 172.21.15.5:42708) 2024-04-24T06:46:21.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.840+0000 7f36a8a00640 1 -- 172.21.15.5:0/3793438902 learned_addr learned my addr 172.21.15.5:0/3793438902 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:21.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.840+0000 7f36a3400640 1 --2- 172.21.15.5:0/3793438902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593ca8ac850 0x5593ca8aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:21.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.840+0000 7f36a3e00640 1 --2- 172.21.15.5:0/3793438902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ca8a2180 0x5593ca8ac310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:21.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.840+0000 7f36a8a00640 1 -- 172.21.15.5:0/3793438902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593ca8ac850 msgr2=0x5593ca8aec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:21.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.840+0000 7f36a8a00640 1 --2- 172.21.15.5:0/3793438902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593ca8ac850 0x5593ca8aec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:21.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.840+0000 7f36a8a00640 1 -- 172.21.15.5:0/3793438902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ca8a2180 msgr2=0x5593ca8ac310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:21.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.840+0000 7f36a8a00640 1 --2- 172.21.15.5:0/3793438902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ca8a2180 0x5593ca8ac310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:21.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.840+0000 7f36a8a00640 1 -- 172.21.15.5:0/3793438902 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593ca801f50 con 0x5593ca8a1860 2024-04-24T06:46:21.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.840+0000 7f36a3400640 1 --2- 172.21.15.5:0/3793438902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593ca8ac850 0x5593ca8aec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:46:21.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.841+0000 7f36a8a00640 1 --2- 172.21.15.5:0/3793438902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ca8a1860 0x5593ca8a1c40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f369c00a430 tx=0x7f369c009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1eb1e516e7a75ec9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:21.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.841+0000 7f3699200640 1 -- 172.21.15.5:0/3793438902 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369c013790 con 0x5593ca8a1860 2024-04-24T06:46:21.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.841+0000 7f3699200640 1 -- 172.21.15.5:0/3793438902 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f369c019070 con 0x5593ca8a1860 2024-04-24T06:46:21.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.841+0000 7f3699200640 1 -- 172.21.15.5:0/3793438902 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369c014d80 con 0x5593ca8a1860 2024-04-24T06:46:21.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.841+0000 7f36a0c00640 1 --2- 172.21.15.5:0/962771010 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f36880a3d20 0x7f36880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:21.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.842+0000 7f36a0c00640 1 -- 172.21.15.5:0/962771010 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f36900d0ba0 con 0x7f36a4109510 2024-04-24T06:46:21.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.842+0000 7f36a3e00640 1 --2- 172.21.15.5:0/962771010 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f36880a3d20 0x7f36880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:21.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.842+0000 7f36ad779740 1 -- 172.21.15.5:0/3793438902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ca8a1860 msgr2=0x5593ca8a1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:21.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.842+0000 7f36ad779740 1 --2- 172.21.15.5:0/3793438902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ca8a1860 0x5593ca8a1c40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f369c00a430 tx=0x7f369c009650 comp rx=0 tx=0).stop 2024-04-24T06:46:21.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.842+0000 7f36ad779740 1 -- 172.21.15.5:0/3793438902 shutdown_connections 2024-04-24T06:46:21.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.842+0000 7f36ad779740 1 --2- 172.21.15.5:0/3793438902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593ca8ac850 0x5593ca8aec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:21.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.842+0000 7f36ad779740 1 --2- 172.21.15.5:0/3793438902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ca8a2180 0x5593ca8ac310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:21.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.842+0000 7f36ad779740 1 --2- 172.21.15.5:0/3793438902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ca8a1860 0x5593ca8a1c40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:21.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.842+0000 7f36ad779740 1 -- 172.21.15.5:0/3793438902 >> 172.21.15.5:0/3793438902 conn(0x5593ca89db80 msgr2=0x5593ca89df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:21.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.842+0000 7f36ad779740 1 -- 172.21.15.5:0/3793438902 shutdown_connections 2024-04-24T06:46:21.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.842+0000 7f36ad779740 1 -- 172.21.15.5:0/3793438902 wait complete. 2024-04-24T06:46:21.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.842+0000 7f36a3e00640 1 --2- 172.21.15.5:0/962771010 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f36880a3d20 0x7f36880a61e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3694006010 tx=0x7f3694000bc0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:21.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.843+0000 7f36ad779740 1 -- start start 2024-04-24T06:46:21.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.843+0000 7f36ad779740 1 Processor -- start 2024-04-24T06:46:21.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.844+0000 7f36ad779740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ca8a1860 0x5593ca8a9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:21.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.844+0000 7f36ad779740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593ca8a2180 0x5593ca8a9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:21.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.844+0000 7f36ad779740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ca8ac850 0x5593ca8aa210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:21.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.844+0000 7f36ad779740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5593ca8af640 con 0x5593ca8ac850 2024-04-24T06:46:21.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.844+0000 7f36ad779740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5593ca7c0690 con 0x5593ca8a2180 2024-04-24T06:46:21.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.844+0000 7f36ad779740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5593ca8af7c0 con 0x5593ca8a1860 2024-04-24T06:46:21.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.844+0000 7f36ad779740 10 client.0 _ll_register_callbacks cb 0x5593ca7fc650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:46:21.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.844+0000 7f36a3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ca8a1860 0x5593ca8a9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:21.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.844+0000 7f36a3400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593ca8a2180 0x5593ca8a9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:21.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.844+0000 7f36a3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ca8a1860 0x5593ca8a9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:42718/0 (socket says 172.21.15.5:42718) 2024-04-24T06:46:21.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.844+0000 7f36a3400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593ca8a2180 0x5593ca8a9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:37018/0 (socket says 172.21.15.5:37018) 2024-04-24T06:46:21.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.844+0000 7f36a3e00640 1 -- 172.21.15.5:0/942322797 learned_addr learned my addr 172.21.15.5:0/942322797 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:21.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.844+0000 7f36a8a00640 1 --2- 172.21.15.5:0/942322797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ca8ac850 0x5593ca8aa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:21.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.844+0000 7f36a3400640 1 -- 172.21.15.5:0/942322797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ca8a1860 msgr2=0x5593ca8a9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:21.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.844+0000 7f36a3400640 1 --2- 172.21.15.5:0/942322797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ca8a1860 0x5593ca8a9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:21.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.844+0000 7f36a3400640 1 -- 172.21.15.5:0/942322797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ca8ac850 msgr2=0x5593ca8aa210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:21.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.844+0000 7f36a3400640 1 --2- 172.21.15.5:0/942322797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ca8ac850 0x5593ca8aa210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:21.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.844+0000 7f36a3400640 1 -- 172.21.15.5:0/942322797 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593ca7bf920 con 0x5593ca8a2180 2024-04-24T06:46:21.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.845+0000 7f36a3400640 1 --2- 172.21.15.5:0/942322797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593ca8a2180 0x5593ca8a9cd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f36900ce2a0 tx=0x7f36900096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3252c275656f0d62 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:21.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.845+0000 7f368d600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:46:21.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.845+0000 7f368d600640 1 -- 172.21.15.5:0/942322797 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369009c630 con 0x5593ca8a2180 2024-04-24T06:46:21.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.845+0000 7f368d600640 1 -- 172.21.15.5:0/942322797 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f36900a52d0 con 0x5593ca8a2180 2024-04-24T06:46:21.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.845+0000 7f36ad779740 1 -- 172.21.15.5:0/942322797 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f369c019070 con 0x5593ca8a2180 2024-04-24T06:46:21.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.845+0000 7f368d600640 1 -- 172.21.15.5:0/942322797 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f36900056b0 con 0x5593ca8a2180 2024-04-24T06:46:21.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.845+0000 7f36ad779740 10 client.25787 fetch_fsmap learned FSMap version 25 2024-04-24T06:46:21.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.845+0000 7f36ad779740 1 -- 172.21.15.5:0/942322797 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5593ca5e37d0 con 0x5593ca8a2180 2024-04-24T06:46:21.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.845+0000 7f368d600640 1 -- 172.21.15.5:0/942322797 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f369c019070 con 0x5593ca8a2180 2024-04-24T06:46:21.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.846+0000 7f368d600640 1 -- 172.21.15.5:0/942322797 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f369009bf40 con 0x5593ca8a2180 2024-04-24T06:46:21.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.846+0000 7f36ad779740 10 client.25787 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:46:21.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.846+0000 7f36ad779740 10 client.25787 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:46:21.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.846+0000 7f36ad779740 20 client.25787 populate_metadata read hostname 'smithi005' 2024-04-24T06:46:21.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.846+0000 7f36ad779740 1 --2- 172.21.15.5:0/942322797 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5593ca8bcbd0 0x5593ca8bf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:21.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.846+0000 7f36ad779740 4 client.25787 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:46:21.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.846+0000 7f36ad779740 1 -- 172.21.15.5:0/942322797 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593ca814c10 con 0x5593ca8bcbd0 2024-04-24T06:46:21.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.846+0000 7f36a3e00640 1 --2- 172.21.15.5:0/942322797 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5593ca8bcbd0 0x5593ca8bf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:21.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.846+0000 7f36a3e00640 1 --2- 172.21.15.5:0/942322797 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5593ca8bcbd0 0x5593ca8bf010 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bbcfb73794d8a76 server_cookie=fd9df84525b92ae0 in_seq=0 out_seq=0 2024-04-24T06:46:21.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.847+0000 7f368d600640 10 client.25787 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:46:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.850+0000 7f368d600640 1 -- 172.21.15.5:0/942322797 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5593ca814c10 con 0x5593ca8bcbd0 2024-04-24T06:46:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.850+0000 7f368d600640 10 client.25787 handle_command_reply: tid=0 2024-04-24T06:46:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.850+0000 7f36ad779740 1 client.25787 shutdown 2024-04-24T06:46:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.850+0000 7f36ad779740 1 -- 172.21.15.5:0/942322797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593ca8a2180 msgr2=0x5593ca8a9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.850+0000 7f36ad779740 1 --2- 172.21.15.5:0/942322797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593ca8a2180 0x5593ca8a9cd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f36900ce2a0 tx=0x7f36900096c0 comp rx=0 tx=0).stop 2024-04-24T06:46:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.850+0000 7f36ad779740 1 -- 172.21.15.5:0/942322797 shutdown_connections 2024-04-24T06:46:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.850+0000 7f36ad779740 1 --2- 172.21.15.5:0/942322797 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5593ca8bcbd0 0x5593ca8bf010 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.850+0000 7f36ad779740 1 --2- 172.21.15.5:0/942322797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ca8ac850 0x5593ca8aa210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.850+0000 7f36ad779740 1 --2- 172.21.15.5:0/942322797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593ca8a2180 0x5593ca8a9cd0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:21.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.850+0000 7f36ad779740 1 --2- 172.21.15.5:0/942322797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ca8a1860 0x5593ca8a9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:21.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.850+0000 7f36ad779740 1 -- 172.21.15.5:0/942322797 >> 172.21.15.5:0/942322797 conn(0x5593ca89db80 msgr2=0x5593ca8a3980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:21.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.850+0000 7f368d600640 0 client.25787 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:46:21.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.850+0000 7f36ad779740 1 -- 172.21.15.5:0/942322797 shutdown_connections 2024-04-24T06:46:21.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.850+0000 7f36ad779740 1 -- 172.21.15.5:0/942322797 wait complete. 2024-04-24T06:46:21.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.850+0000 7f36ad779740 20 client.25787 trim_cache size 0 max 16384 2024-04-24T06:46:21.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.873+0000 7f36ad779740 1 Processor -- start 2024-04-24T06:46:21.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.873+0000 7f36ad779740 1 -- start start 2024-04-24T06:46:21.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.873+0000 7f36ad779740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ca8a1860 0x5593ca8a86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:21.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.873+0000 7f36ad779740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ca8a2180 0x5593ca8b8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:21.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.873+0000 7f36ad779740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593ca8ac850 0x5593ca8a3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:21.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.873+0000 7f36ad779740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f36a41b28a0 con 0x5593ca8a1860 2024-04-24T06:46:21.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.873+0000 7f36ad779740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f36a41b0170 con 0x5593ca8ac850 2024-04-24T06:46:21.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.873+0000 7f36ad779740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f36a41adfc0 con 0x5593ca8a2180 2024-04-24T06:46:21.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.873+0000 7f36a3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ca8a1860 0x5593ca8a86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:21.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.874+0000 7f36a3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ca8a2180 0x5593ca8b8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:21.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.874+0000 7f36a8a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593ca8ac850 0x5593ca8a3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:21.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.874+0000 7f36a3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ca8a1860 0x5593ca8a86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49422/0 (socket says 172.21.15.5:49422) 2024-04-24T06:46:21.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.874+0000 7f36a3e00640 1 -- 172.21.15.5:0/2892745209 learned_addr learned my addr 172.21.15.5:0/2892745209 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:21.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.874+0000 7f36a3e00640 1 -- 172.21.15.5:0/2892745209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ca8a2180 msgr2=0x5593ca8b8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:21.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.874+0000 7f36a3e00640 1 --2- 172.21.15.5:0/2892745209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ca8a2180 0x5593ca8b8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:21.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.874+0000 7f36a3e00640 1 -- 172.21.15.5:0/2892745209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593ca8ac850 msgr2=0x5593ca8a3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:21.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.874+0000 7f36a3e00640 1 --2- 172.21.15.5:0/2892745209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593ca8ac850 0x5593ca8a3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:21.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.874+0000 7f36a3e00640 1 -- 172.21.15.5:0/2892745209 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593ca5193f0 con 0x5593ca8a1860 2024-04-24T06:46:21.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.874+0000 7f36a8a00640 1 --2- 172.21.15.5:0/2892745209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593ca8ac850 0x5593ca8a3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:46:21.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.874+0000 7f36a3e00640 1 --2- 172.21.15.5:0/2892745209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ca8a1860 0x5593ca8a86f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f369402a930 tx=0x7f369403f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=899531f81db4b25b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:21.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.875+0000 7f368fe00640 1 -- 172.21.15.5:0/2892745209 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3694035040 con 0x5593ca8a1860 2024-04-24T06:46:21.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.875+0000 7f368fe00640 1 -- 172.21.15.5:0/2892745209 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f369405a070 con 0x5593ca8a1860 2024-04-24T06:46:21.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.875+0000 7f368fe00640 1 -- 172.21.15.5:0/2892745209 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3694032820 con 0x5593ca8a1860 2024-04-24T06:46:21.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.875+0000 7f36ad779740 1 -- 172.21.15.5:0/2892745209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ca8a1860 msgr2=0x5593ca8a86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:21.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.875+0000 7f36ad779740 1 --2- 172.21.15.5:0/2892745209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ca8a1860 0x5593ca8a86f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f369402a930 tx=0x7f369403f000 comp rx=0 tx=0).stop 2024-04-24T06:46:21.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.875+0000 7f36ad779740 1 -- 172.21.15.5:0/2892745209 shutdown_connections 2024-04-24T06:46:21.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.875+0000 7f36ad779740 1 --2- 172.21.15.5:0/2892745209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593ca8ac850 0x5593ca8a3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:21.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.875+0000 7f36ad779740 1 --2- 172.21.15.5:0/2892745209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ca8a2180 0x5593ca8b8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:21.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.875+0000 7f36ad779740 1 --2- 172.21.15.5:0/2892745209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ca8a1860 0x5593ca8a86f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:21.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.875+0000 7f36ad779740 1 -- 172.21.15.5:0/2892745209 >> 172.21.15.5:0/2892745209 conn(0x5593ca89db80 msgr2=0x5593ca89cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:21.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.875+0000 7f36ad779740 1 -- 172.21.15.5:0/2892745209 shutdown_connections 2024-04-24T06:46:21.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.876+0000 7f36ad779740 1 -- 172.21.15.5:0/2892745209 wait complete. 2024-04-24T06:46:21.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.876+0000 7f36ad779740 1 -- start start 2024-04-24T06:46:21.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.876+0000 7f36ad779740 1 Processor -- start 2024-04-24T06:46:21.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.877+0000 7f36ad779740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ca8a1860 0x5593ca8b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:21.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.877+0000 7f36ad779740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ca8a2180 0x5593ca8a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:21.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.877+0000 7f36ad779740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593ca8ac850 0x5593ca8a69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:21.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.877+0000 7f36ad779740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f36a41ae630 con 0x5593ca8a1860 2024-04-24T06:46:21.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.877+0000 7f36ad779740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f36a41afaa0 con 0x5593ca8ac850 2024-04-24T06:46:21.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.877+0000 7f36ad779740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f36a41aed00 con 0x5593ca8a2180 2024-04-24T06:46:21.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.877+0000 7f36ad779740 10 client.0 _ll_register_callbacks cb 0x7f369000ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:46:21.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.877+0000 7f36a3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ca8a1860 0x5593ca8b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:21.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.877+0000 7f36a8a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593ca8ac850 0x5593ca8a69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:21.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.877+0000 7f36a3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ca8a1860 0x5593ca8b4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49424/0 (socket says 172.21.15.5:49424) 2024-04-24T06:46:21.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.877+0000 7f36a3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ca8a2180 0x5593ca8a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:21.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.877+0000 7f36a3e00640 1 -- 172.21.15.5:0/2878710216 learned_addr learned my addr 172.21.15.5:0/2878710216 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:21.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.877+0000 7f36a8a00640 1 -- 172.21.15.5:0/2878710216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ca8a2180 msgr2=0x5593ca8a62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:21.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.877+0000 7f36a8a00640 1 --2- 172.21.15.5:0/2878710216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ca8a2180 0x5593ca8a62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:21.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.877+0000 7f36a8a00640 1 -- 172.21.15.5:0/2878710216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ca8a1860 msgr2=0x5593ca8b4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:21.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.877+0000 7f36a8a00640 1 --2- 172.21.15.5:0/2878710216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ca8a1860 0x5593ca8b4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:21.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.877+0000 7f36a8a00640 1 -- 172.21.15.5:0/2878710216 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593ca4d90f0 con 0x5593ca8ac850 2024-04-24T06:46:21.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.877+0000 7f36a3e00640 1 --2- 172.21.15.5:0/2878710216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ca8a1860 0x5593ca8b4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:46:21.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.878+0000 7f36a8a00640 1 --2- 172.21.15.5:0/2878710216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593ca8ac850 0x5593ca8a69c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f369c004b90 tx=0x7f369c014790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=791617b7ce87c7a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:21.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.878+0000 7f368d600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:46:21.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.878+0000 7f368d600640 1 -- 172.21.15.5:0/2878710216 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369c02d060 con 0x5593ca8ac850 2024-04-24T06:46:21.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.878+0000 7f36ad779740 1 -- 172.21.15.5:0/2878710216 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f369405a070 con 0x5593ca8ac850 2024-04-24T06:46:21.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.878+0000 7f368d600640 1 -- 172.21.15.5:0/2878710216 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f369c014c30 con 0x5593ca8ac850 2024-04-24T06:46:21.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.878+0000 7f368d600640 1 -- 172.21.15.5:0/2878710216 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f369c014e30 con 0x5593ca8ac850 2024-04-24T06:46:21.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.878+0000 7f36ad779740 10 client.25793 fetch_fsmap learned FSMap version 25 2024-04-24T06:46:21.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.878+0000 7f36ad779740 1 -- 172.21.15.5:0/2878710216 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5593ca7cc810 con 0x5593ca8ac850 2024-04-24T06:46:21.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.879+0000 7f368d600640 1 -- 172.21.15.5:0/2878710216 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f369405a070 con 0x5593ca8ac850 2024-04-24T06:46:21.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.879+0000 7f368d600640 1 -- 172.21.15.5:0/2878710216 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f369c0285d0 con 0x5593ca8ac850 2024-04-24T06:46:21.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.879+0000 7f36ad779740 10 client.25793 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:46:21.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.879+0000 7f36ad779740 10 client.25793 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:46:21.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.879+0000 7f36ad779740 20 client.25793 populate_metadata read hostname 'smithi005' 2024-04-24T06:46:21.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.879+0000 7f36ad779740 1 --2- 172.21.15.5:0/2878710216 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5593ca9234f0 0x5593ca925910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:21.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.879+0000 7f36ad779740 4 client.25793 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:46:21.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.879+0000 7f36ad779740 1 -- 172.21.15.5:0/2878710216 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5593ca80ea80 con 0x5593ca9234f0 2024-04-24T06:46:21.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.879+0000 7f36a3e00640 1 --2- 172.21.15.5:0/2878710216 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5593ca9234f0 0x5593ca925910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:21.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.880+0000 7f36a3e00640 1 --2- 172.21.15.5:0/2878710216 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5593ca9234f0 0x5593ca925910 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d524388c256b6796 server_cookie=faf9441819ea0c28 in_seq=0 out_seq=0 2024-04-24T06:46:21.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.880+0000 7f368d600640 10 client.25793 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:46:21.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.883+0000 7f368d600640 1 -- 172.21.15.5:0/2878710216 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5593ca80ea80 con 0x5593ca9234f0 2024-04-24T06:46:21.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.883+0000 7f368d600640 10 client.25793 handle_command_reply: tid=0 2024-04-24T06:46:21.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.883+0000 7f36ad779740 1 client.25793 shutdown 2024-04-24T06:46:21.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.883+0000 7f36ad779740 1 -- 172.21.15.5:0/2878710216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593ca8ac850 msgr2=0x5593ca8a69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:21.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.883+0000 7f36ad779740 1 --2- 172.21.15.5:0/2878710216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593ca8ac850 0x5593ca8a69c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f369c004b90 tx=0x7f369c014790 comp rx=0 tx=0).stop 2024-04-24T06:46:21.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.883+0000 7f36ad779740 1 -- 172.21.15.5:0/2878710216 shutdown_connections 2024-04-24T06:46:21.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.883+0000 7f36ad779740 1 --2- 172.21.15.5:0/2878710216 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5593ca9234f0 0x5593ca925910 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:21.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.883+0000 7f36ad779740 1 --2- 172.21.15.5:0/2878710216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5593ca8ac850 0x5593ca8a69c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:21.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.883+0000 7f36ad779740 1 --2- 172.21.15.5:0/2878710216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593ca8a2180 0x5593ca8a62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:21.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.883+0000 7f36ad779740 1 --2- 172.21.15.5:0/2878710216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593ca8a1860 0x5593ca8b4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:21.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.883+0000 7f36ad779740 1 -- 172.21.15.5:0/2878710216 >> 172.21.15.5:0/2878710216 conn(0x5593ca89db80 msgr2=0x5593ca8a9750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:21.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.883+0000 7f368d600640 0 client.25793 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:46:21.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.883+0000 7f36ad779740 1 -- 172.21.15.5:0/2878710216 shutdown_connections 2024-04-24T06:46:21.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.883+0000 7f36ad779740 1 -- 172.21.15.5:0/2878710216 wait complete. 2024-04-24T06:46:21.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.883+0000 7f36ad779740 20 client.25793 trim_cache size 0 max 16384 2024-04-24T06:46:21.886 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:46:21.886 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-24T06:46:21.886 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-24T06:46:21.886 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:46:21.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.884+0000 7f36aaa00640 1 -- 172.21.15.5:0/962771010 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f36880a3d20 msgr2=0x7f36880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:21.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.884+0000 7f36aaa00640 1 --2- 172.21.15.5:0/962771010 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f36880a3d20 0x7f36880a61e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3694006010 tx=0x7f3694000bc0 comp rx=0 tx=0).stop 2024-04-24T06:46:21.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.884+0000 7f36aaa00640 1 -- 172.21.15.5:0/962771010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36a4109510 msgr2=0x7f36a41a0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:21.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.884+0000 7f36aaa00640 1 --2- 172.21.15.5:0/962771010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36a4109510 0x7f36a41a0dc0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f369000e9f0 tx=0x7f369000eeb0 comp rx=0 tx=0).stop 2024-04-24T06:46:21.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.884+0000 7f36aaa00640 1 -- 172.21.15.5:0/962771010 shutdown_connections 2024-04-24T06:46:21.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.884+0000 7f36aaa00640 1 --2- 172.21.15.5:0/962771010 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f36880a3d20 0x7f36880a61e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:21.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.884+0000 7f36aaa00640 1 --2- 172.21.15.5:0/962771010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36a419aa30 0x7f36a419aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:21.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.884+0000 7f36aaa00640 1 --2- 172.21.15.5:0/962771010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36a4109510 0x7f36a41a0dc0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:21.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.884+0000 7f36aaa00640 1 --2- 172.21.15.5:0/962771010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36a4104440 0x7f36a41a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:21.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.884+0000 7f36aaa00640 1 -- 172.21.15.5:0/962771010 >> 172.21.15.5:0/962771010 conn(0x7f36a40ffc80 msgr2=0x7f36a4101f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:21.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.884+0000 7f36aaa00640 1 -- 172.21.15.5:0/962771010 shutdown_connections 2024-04-24T06:46:21.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:21.884+0000 7f36aaa00640 1 -- 172.21.15.5:0/962771010 wait complete. 2024-04-24T06:46:21.906 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T06:46:21.906 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:46:22.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.101+0000 7fcd2fe00640 1 -- 172.21.15.5:0/1290199092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd30109920 msgr2=0x7fcd30114c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:22.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.101+0000 7fcd2fe00640 1 --2- 172.21.15.5:0/1290199092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd30109920 0x7fcd30114c10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c009900 tx=0x7fcd1c02a410 comp rx=0 tx=0).stop 2024-04-24T06:46:22.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.101+0000 7fcd2fe00640 1 -- 172.21.15.5:0/1290199092 shutdown_connections 2024-04-24T06:46:22.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.101+0000 7fcd2fe00640 1 --2- 172.21.15.5:0/1290199092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd30115150 0x7fcd30117540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.101+0000 7fcd2fe00640 1 --2- 172.21.15.5:0/1290199092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd30109920 0x7fcd30114c10 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.101+0000 7fcd2fe00640 1 --2- 172.21.15.5:0/1290199092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd30109000 0x7fcd301093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.101+0000 7fcd2fe00640 1 -- 172.21.15.5:0/1290199092 >> 172.21.15.5:0/1290199092 conn(0x7fcd3006cfc0 msgr2=0x7fcd3006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:22.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.101+0000 7fcd2fe00640 1 -- 172.21.15.5:0/1290199092 shutdown_connections 2024-04-24T06:46:22.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.101+0000 7fcd2fe00640 1 -- 172.21.15.5:0/1290199092 wait complete. 2024-04-24T06:46:22.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.103+0000 7fcd2fe00640 1 Processor -- start 2024-04-24T06:46:22.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.103+0000 7fcd2fe00640 1 -- start start 2024-04-24T06:46:22.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.103+0000 7fcd2fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd30109000 0x7fcd3010aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:22.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.103+0000 7fcd2fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd30109920 0x7fcd3010afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:22.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.104+0000 7fcd2fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd30115150 0x7fcd3010b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:22.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.104+0000 7fcd2fe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3011aa80 con 0x7fcd30109920 2024-04-24T06:46:22.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.104+0000 7fcd2fe00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3011a900 con 0x7fcd30115150 2024-04-24T06:46:22.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.104+0000 7fcd2fe00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3011ac00 con 0x7fcd30109000 2024-04-24T06:46:22.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.104+0000 7fcd2ea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd30109000 0x7fcd3010aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:22.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.104+0000 7fcd2ea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd30109000 0x7fcd3010aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:42748/0 (socket says 172.21.15.5:42748) 2024-04-24T06:46:22.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.104+0000 7fcd2ea00640 1 -- 172.21.15.5:0/3926400489 learned_addr learned my addr 172.21.15.5:0/3926400489 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:22.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.104+0000 7fcd2e000640 1 --2- 172.21.15.5:0/3926400489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd30109920 0x7fcd3010afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:22.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.104+0000 7fcd2f400640 1 --2- 172.21.15.5:0/3926400489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd30115150 0x7fcd3010b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:22.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.104+0000 7fcd2ea00640 1 -- 172.21.15.5:0/3926400489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd30115150 msgr2=0x7fcd3010b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:22.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.104+0000 7fcd2ea00640 1 --2- 172.21.15.5:0/3926400489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd30115150 0x7fcd3010b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.104+0000 7fcd2ea00640 1 -- 172.21.15.5:0/3926400489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd30109920 msgr2=0x7fcd3010afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:22.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.104+0000 7fcd2ea00640 1 --2- 172.21.15.5:0/3926400489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd30109920 0x7fcd3010afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.104+0000 7fcd2ea00640 1 -- 172.21.15.5:0/3926400489 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd301af500 con 0x7fcd30109000 2024-04-24T06:46:22.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.105+0000 7fcd2ea00640 1 --2- 172.21.15.5:0/3926400489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd30109000 0x7fcd3010aa70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000d950 tx=0x7fcd2000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:22.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.105+0000 7fcd27400640 1 -- 172.21.15.5:0/3926400489 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd20017070 con 0x7fcd30109000 2024-04-24T06:46:22.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.105+0000 7fcd27400640 1 -- 172.21.15.5:0/3926400489 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd20009c30 con 0x7fcd30109000 2024-04-24T06:46:22.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.105+0000 7fcd2fe00640 1 -- 172.21.15.5:0/3926400489 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd301af7f0 con 0x7fcd30109000 2024-04-24T06:46:22.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.105+0000 7fcd27400640 1 -- 172.21.15.5:0/3926400489 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2000f6b0 con 0x7fcd30109000 2024-04-24T06:46:22.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.105+0000 7fcd2fe00640 1 -- 172.21.15.5:0/3926400489 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd301afd60 con 0x7fcd30109000 2024-04-24T06:46:22.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.106+0000 7fcd2fe00640 1 -- 172.21.15.5:0/3926400489 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd3004fe00 con 0x7fcd30109000 2024-04-24T06:46:22.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.108+0000 7fcd27400640 1 -- 172.21.15.5:0/3926400489 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fcd200109d0 con 0x7fcd30109000 2024-04-24T06:46:22.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.110+0000 7fcd27400640 1 --2- 172.21.15.5:0/3926400489 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fcd140a3d90 0x7fcd140a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:22.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.110+0000 7fcd27400640 1 -- 172.21.15.5:0/3926400489 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fcd200d0360 con 0x7fcd30109000 2024-04-24T06:46:22.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.110+0000 7fcd2e000640 1 --2- 172.21.15.5:0/3926400489 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fcd140a3d90 0x7fcd140a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:22.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.111+0000 7fcd2e000640 1 --2- 172.21.15.5:0/3926400489 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fcd140a3d90 0x7fcd140a6250 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c009230 tx=0x7fcd1c0058d0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:22.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.112+0000 7fcd27400640 1 -- 172.21.15.5:0/3926400489 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcd20009de0 con 0x7fcd30109000 2024-04-24T06:46:22.339 INFO:tasks.workunit.client.0.smithi005.stderr:Note: switching to '12973fc04fd10d4af086901e10ffa8e48866b735'. 2024-04-24T06:46:22.339 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-04-24T06:46:22.339 INFO:tasks.workunit.client.0.smithi005.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-24T06:46:22.339 INFO:tasks.workunit.client.0.smithi005.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-24T06:46:22.340 INFO:tasks.workunit.client.0.smithi005.stderr:state without impacting any branches by switching back to a branch. 2024-04-24T06:46:22.340 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-04-24T06:46:22.340 INFO:tasks.workunit.client.0.smithi005.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-24T06:46:22.340 INFO:tasks.workunit.client.0.smithi005.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-24T06:46:22.340 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-04-24T06:46:22.340 INFO:tasks.workunit.client.0.smithi005.stderr: git switch -c 2024-04-24T06:46:22.340 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-04-24T06:46:22.340 INFO:tasks.workunit.client.0.smithi005.stderr:Or undo this operation with: 2024-04-24T06:46:22.340 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-04-24T06:46:22.340 INFO:tasks.workunit.client.0.smithi005.stderr: git switch - 2024-04-24T06:46:22.340 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-04-24T06:46:22.340 INFO:tasks.workunit.client.0.smithi005.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-24T06:46:22.340 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-04-24T06:46:22.340 INFO:tasks.workunit.client.0.smithi005.stderr:HEAD is now at 12973fc0 btrfs/022: Correct seq number in golden output 2024-04-24T06:46:22.358 INFO:tasks.workunit.client.0.smithi005.stderr:+ make -j4 2024-04-24T06:46:22.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.375+0000 7fcd2fe00640 1 -- 172.21.15.5:0/3926400489 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd301102d0 con 0x7fcd30109000 2024-04-24T06:46:22.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.376+0000 7fcd27400640 1 -- 172.21.15.5:0/3926400489 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11859 (secure 0 0 0) 0x7fcd200cf090 con 0x7fcd30109000 2024-04-24T06:46:22.377 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:46:22.377 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1735462750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1735462750},{"type":"v1","addr":"172.21.15.5:6837","nonce":1735462750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1499469370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1499469370},{"type":"v1","addr":"172.21.15.5:6841","nonce":1499469370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3705638847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3705638847},{"type":"v1","addr":"172.21.15.37:6835","nonce":3705638847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T06:41:34.764490+0000","modified":"2024-04-24T06:46:02.765290+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24550,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6837/1390103296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1390103296},{"type":"v1","addr":"172.21.15.37:6837","nonce":1390103296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3217754215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3217754215},{"type":"v1","addr":"172.21.15.137:6835","nonce":3217754215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6837/3044222976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3044222976},{"type":"v1","addr":"172.21.15.137:6837","nonce":3044222976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.5:6839/3173201459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3173201459},{"type":"v1","addr":"172.21.15.5:6839","nonce":3173201459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6839/1889774232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1889774232},{"type":"v1","addr":"172.21.15.137:6839","nonce":1889774232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6839/3536721753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":3536721753},{"type":"v1","addr":"172.21.15.37:6839","nonce":3536721753}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24539,24550,24569]},"id":1}]} 2024-04-24T06:46:22.378 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-04-24T06:46:22.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.379+0000 7fcd2fe00640 1 -- 172.21.15.5:0/3926400489 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fcd140a3d90 msgr2=0x7fcd140a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:22.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.379+0000 7fcd2fe00640 1 --2- 172.21.15.5:0/3926400489 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fcd140a3d90 0x7fcd140a6250 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c009230 tx=0x7fcd1c0058d0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.379+0000 7fcd2fe00640 1 -- 172.21.15.5:0/3926400489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd30109000 msgr2=0x7fcd3010aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:22.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.379+0000 7fcd2fe00640 1 --2- 172.21.15.5:0/3926400489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd30109000 0x7fcd3010aa70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000d950 tx=0x7fcd2000de10 comp rx=0 tx=0).stop 2024-04-24T06:46:22.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.379+0000 7fcd2fe00640 1 -- 172.21.15.5:0/3926400489 shutdown_connections 2024-04-24T06:46:22.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.379+0000 7fcd2fe00640 1 --2- 172.21.15.5:0/3926400489 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fcd140a3d90 0x7fcd140a6250 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.379+0000 7fcd2fe00640 1 --2- 172.21.15.5:0/3926400489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd30115150 0x7fcd3010b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.379+0000 7fcd2fe00640 1 --2- 172.21.15.5:0/3926400489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd30109920 0x7fcd3010afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.379+0000 7fcd2fe00640 1 --2- 172.21.15.5:0/3926400489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd30109000 0x7fcd3010aa70 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.379+0000 7fcd2fe00640 1 -- 172.21.15.5:0/3926400489 >> 172.21.15.5:0/3926400489 conn(0x7fcd3006cfc0 msgr2=0x7fcd30071ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:22.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.379+0000 7fcd2fe00640 1 -- 172.21.15.5:0/3926400489 shutdown_connections 2024-04-24T06:46:22.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.379+0000 7fcd2fe00640 1 -- 172.21.15.5:0/3926400489 wait complete. 2024-04-24T06:46:22.383 INFO:tasks.workunit.client.0.smithi005.stdout:libtoolize -cfi 2024-04-24T06:46:22.399 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T06:46:22.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.596+0000 7fc0fd400640 1 -- 172.21.15.5:0/796486186 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0f8074cc0 msgr2=0x7fc0f81128d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:22.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.596+0000 7fc0fd400640 1 --2- 172.21.15.5:0/796486186 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0f8074cc0 0x7fc0f81128d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0009900 tx=0x7fc0e002a480 comp rx=0 tx=0).stop 2024-04-24T06:46:22.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.596+0000 7fc0fd400640 1 -- 172.21.15.5:0/796486186 shutdown_connections 2024-04-24T06:46:22.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.596+0000 7fc0fd400640 1 --2- 172.21.15.5:0/796486186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0f8112e10 0x7fc0f8115200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.596+0000 7fc0fd400640 1 --2- 172.21.15.5:0/796486186 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0f8074cc0 0x7fc0f81128d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.596+0000 7fc0fd400640 1 --2- 172.21.15.5:0/796486186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0f80743a0 0x7fc0f8074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.596+0000 7fc0fd400640 1 -- 172.21.15.5:0/796486186 >> 172.21.15.5:0/796486186 conn(0x7fc0f80fd950 msgr2=0x7fc0f80ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:22.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.596+0000 7fc0fd400640 1 -- 172.21.15.5:0/796486186 shutdown_connections 2024-04-24T06:46:22.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.596+0000 7fc0fd400640 1 -- 172.21.15.5:0/796486186 wait complete. 2024-04-24T06:46:22.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.597+0000 7fc0fd400640 1 Processor -- start 2024-04-24T06:46:22.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.598+0000 7fc0fd400640 1 -- start start 2024-04-24T06:46:22.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.598+0000 7fc0fd400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0f80743a0 0x7fc0f819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:22.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.598+0000 7fc0fd400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0f8074cc0 0x7fc0f819f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:22.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.598+0000 7fc0fd400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0f8112e10 0x7fc0f81a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:22.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.598+0000 7fc0fd400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f8118840 con 0x7fc0f8074cc0 2024-04-24T06:46:22.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.598+0000 7fc0fd400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f81186c0 con 0x7fc0f8112e10 2024-04-24T06:46:22.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.598+0000 7fc0fd400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f81189c0 con 0x7fc0f80743a0 2024-04-24T06:46:22.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.598+0000 7fc0f7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0f80743a0 0x7fc0f819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:22.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.598+0000 7fc0f7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0f80743a0 0x7fc0f819ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:42774/0 (socket says 172.21.15.5:42774) 2024-04-24T06:46:22.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.598+0000 7fc0f7e00640 1 -- 172.21.15.5:0/3045983417 learned_addr learned my addr 172.21.15.5:0/3045983417 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:22.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.598+0000 7fc0efe00640 1 --2- 172.21.15.5:0/3045983417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0f8074cc0 0x7fc0f819f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:22.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.598+0000 7fc0fca00640 1 --2- 172.21.15.5:0/3045983417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0f8112e10 0x7fc0f81a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:22.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.599+0000 7fc0f7e00640 1 -- 172.21.15.5:0/3045983417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0f8112e10 msgr2=0x7fc0f81a3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:22.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.599+0000 7fc0f7e00640 1 --2- 172.21.15.5:0/3045983417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0f8112e10 0x7fc0f81a3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.599+0000 7fc0f7e00640 1 -- 172.21.15.5:0/3045983417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0f8074cc0 msgr2=0x7fc0f819f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:22.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.599+0000 7fc0f7e00640 1 --2- 172.21.15.5:0/3045983417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0f8074cc0 0x7fc0f819f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.599+0000 7fc0f7e00640 1 -- 172.21.15.5:0/3045983417 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f81a3d30 con 0x7fc0f80743a0 2024-04-24T06:46:22.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.599+0000 7fc0f7e00640 1 --2- 172.21.15.5:0/3045983417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0f80743a0 0x7fc0f819ed40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc0e800ea60 tx=0x7fc0e800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:22.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.600+0000 7fc0f5600640 1 -- 172.21.15.5:0/3045983417 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e800cde0 con 0x7fc0f80743a0 2024-04-24T06:46:22.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.600+0000 7fc0f5600640 1 -- 172.21.15.5:0/3045983417 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0e80035f0 con 0x7fc0f80743a0 2024-04-24T06:46:22.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.600+0000 7fc0f5600640 1 -- 172.21.15.5:0/3045983417 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8010670 con 0x7fc0f80743a0 2024-04-24T06:46:22.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.600+0000 7fc0fd400640 1 -- 172.21.15.5:0/3045983417 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0f81a4020 con 0x7fc0f80743a0 2024-04-24T06:46:22.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.600+0000 7fc0fd400640 1 -- 172.21.15.5:0/3045983417 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0f81ab8b0 con 0x7fc0f80743a0 2024-04-24T06:46:22.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.604+0000 7fc1019a1740 1 Processor -- start 2024-04-24T06:46:22.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.604+0000 7fc1019a1740 1 -- start start 2024-04-24T06:46:22.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.605+0000 7fc0f5600640 1 -- 172.21.15.5:0/3045983417 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fc0e80037a0 con 0x7fc0f80743a0 2024-04-24T06:46:22.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.605+0000 7fc1019a1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f02b5cb40 0x557f02b5cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:22.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.605+0000 7fc1019a1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f02b5d460 0x557f02b675f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:22.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.605+0000 7fc1019a1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f02b67b30 0x557f02b69f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:22.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.605+0000 7fc1019a1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557f02ac21d0 con 0x557f02b5d460 2024-04-24T06:46:22.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.605+0000 7fc1019a1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557f02a68450 con 0x557f02b67b30 2024-04-24T06:46:22.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.605+0000 7fc1019a1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557f02abe190 con 0x557f02b5cb40 2024-04-24T06:46:22.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.605+0000 7fc0efe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f02b67b30 0x557f02b69f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:22.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.605+0000 7fc0fca00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f02b5cb40 0x557f02b5cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:22.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.605+0000 7fc0f7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f02b5d460 0x557f02b675f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:22.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.605+0000 7fc0efe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f02b67b30 0x557f02b69f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:37070/0 (socket says 172.21.15.5:37070) 2024-04-24T06:46:22.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.605+0000 7fc0efe00640 1 -- 172.21.15.5:0/2193487800 learned_addr learned my addr 172.21.15.5:0/2193487800 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:22.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.606+0000 7fc0efe00640 1 -- 172.21.15.5:0/2193487800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f02b5cb40 msgr2=0x557f02b5cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:22.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.606+0000 7fc0efe00640 1 --2- 172.21.15.5:0/2193487800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f02b5cb40 0x557f02b5cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.606+0000 7fc0efe00640 1 -- 172.21.15.5:0/2193487800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f02b5d460 msgr2=0x557f02b675f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:22.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.606+0000 7fc0efe00640 1 --2- 172.21.15.5:0/2193487800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f02b5d460 0x557f02b675f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.606+0000 7fc0efe00640 1 -- 172.21.15.5:0/2193487800 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f02b25310 con 0x557f02b67b30 2024-04-24T06:46:22.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.606+0000 7fc0efe00640 1 --2- 172.21.15.5:0/2193487800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f02b67b30 0x557f02b69f20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0009900 tx=0x7fc0e0005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2766db3685e8ecf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:22.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.606+0000 7fc0ed200640 1 -- 172.21.15.5:0/2193487800 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e0035040 con 0x557f02b67b30 2024-04-24T06:46:22.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.607+0000 7fc0ed200640 1 -- 172.21.15.5:0/2193487800 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0e0004400 con 0x557f02b67b30 2024-04-24T06:46:22.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.607+0000 7fc0ed200640 1 -- 172.21.15.5:0/2193487800 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e00035f0 con 0x557f02b67b30 2024-04-24T06:46:22.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.607+0000 7fc0f5600640 1 --2- 172.21.15.5:0/3045983417 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc0dc0a3da0 0x7fc0dc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:22.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.607+0000 7fc1019a1740 1 -- 172.21.15.5:0/2193487800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f02b67b30 msgr2=0x557f02b69f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:22.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.607+0000 7fc1019a1740 1 --2- 172.21.15.5:0/2193487800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f02b67b30 0x557f02b69f20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0009900 tx=0x7fc0e0005de0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.607+0000 7fc0f5600640 1 -- 172.21.15.5:0/3045983417 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fc0e80d0aa0 con 0x7fc0f80743a0 2024-04-24T06:46:22.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.607+0000 7fc0fca00640 1 --2- 172.21.15.5:0/3045983417 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc0dc0a3da0 0x7fc0dc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:22.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.607+0000 7fc1019a1740 1 -- 172.21.15.5:0/2193487800 shutdown_connections 2024-04-24T06:46:22.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.607+0000 7fc1019a1740 1 --2- 172.21.15.5:0/2193487800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f02b67b30 0x557f02b69f20 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.607+0000 7fc1019a1740 1 --2- 172.21.15.5:0/2193487800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f02b5d460 0x557f02b675f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.607+0000 7fc1019a1740 1 --2- 172.21.15.5:0/2193487800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f02b5cb40 0x557f02b5cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.607+0000 7fc1019a1740 1 -- 172.21.15.5:0/2193487800 >> 172.21.15.5:0/2193487800 conn(0x557f02b58e60 msgr2=0x557f02b59240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:22.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.607+0000 7fc1019a1740 1 -- 172.21.15.5:0/2193487800 shutdown_connections 2024-04-24T06:46:22.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.608+0000 7fc1019a1740 1 -- 172.21.15.5:0/2193487800 wait complete. 2024-04-24T06:46:22.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.608+0000 7fc1019a1740 1 -- start start 2024-04-24T06:46:22.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.608+0000 7fc0fca00640 1 --2- 172.21.15.5:0/3045983417 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc0dc0a3da0 0x7fc0dc0a6260 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc0f806b470 tx=0x7fc0f0009500 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:22.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.608+0000 7fc1019a1740 1 Processor -- start 2024-04-24T06:46:22.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.609+0000 7fc1019a1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f02b5cb40 0x557f02b67540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:22.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.609+0000 7fc1019a1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f02b5d460 0x557f02b64830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:22.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.609+0000 7fc1019a1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f02b67b30 0x557f02b64e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:22.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.609+0000 7fc1019a1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557f02b6a960 con 0x557f02b5d460 2024-04-24T06:46:22.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.609+0000 7fc1019a1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557f02a7b810 con 0x557f02b5cb40 2024-04-24T06:46:22.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.609+0000 7fc1019a1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557f02b6aae0 con 0x557f02b67b30 2024-04-24T06:46:22.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.609+0000 7fc1019a1740 10 client.0 _ll_register_callbacks cb 0x557f02a49140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:46:22.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.609+0000 7fc0f7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f02b5cb40 0x557f02b67540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:22.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.609+0000 7fc0f7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f02b5cb40 0x557f02b67540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:37086/0 (socket says 172.21.15.5:37086) 2024-04-24T06:46:22.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.609+0000 7fc0f7e00640 1 -- 172.21.15.5:0/1661993639 learned_addr learned my addr 172.21.15.5:0/1661993639 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:22.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.609+0000 7fc0efe00640 1 --2- 172.21.15.5:0/1661993639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f02b5d460 0x557f02b64830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:22.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.610+0000 7fc0fca00640 1 --2- 172.21.15.5:0/1661993639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f02b67b30 0x557f02b64e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:22.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.610+0000 7fc0f7e00640 1 -- 172.21.15.5:0/1661993639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f02b67b30 msgr2=0x557f02b64e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:22.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.610+0000 7fc0f7e00640 1 --2- 172.21.15.5:0/1661993639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f02b67b30 0x557f02b64e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.610+0000 7fc0f7e00640 1 -- 172.21.15.5:0/1661993639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f02b5d460 msgr2=0x557f02b64830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:22.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.610+0000 7fc0f7e00640 1 --2- 172.21.15.5:0/1661993639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f02b5d460 0x557f02b64830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.610+0000 7fc0f7e00640 1 -- 172.21.15.5:0/1661993639 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f02a8c3f0 con 0x557f02b5cb40 2024-04-24T06:46:22.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.610+0000 7fc0f7e00640 1 --2- 172.21.15.5:0/1661993639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f02b5cb40 0x557f02b67540 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc0e800ea30 tx=0x7fc0e80d4080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a1354f9ae1443bb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:22.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.610+0000 7fc0e5600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:46:22.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.610+0000 7fc0e5600640 1 -- 172.21.15.5:0/1661993639 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e809c530 con 0x557f02b5cb40 2024-04-24T06:46:22.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.610+0000 7fc0e5600640 1 -- 172.21.15.5:0/1661993639 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fc0e80ce420 con 0x557f02b5cb40 2024-04-24T06:46:22.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.610+0000 7fc1019a1740 1 -- 172.21.15.5:0/1661993639 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0e0004400 con 0x557f02b5cb40 2024-04-24T06:46:22.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.611+0000 7fc0e5600640 1 -- 172.21.15.5:0/1661993639 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc0e809c350 con 0x557f02b5cb40 2024-04-24T06:46:22.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.611+0000 7fc1019a1740 10 client.25811 fetch_fsmap learned FSMap version 25 2024-04-24T06:46:22.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.611+0000 7fc1019a1740 1 -- 172.21.15.5:0/1661993639 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557f027d43f0 con 0x557f02b5cb40 2024-04-24T06:46:22.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.611+0000 7fc0e5600640 1 -- 172.21.15.5:0/1661993639 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0e0004400 con 0x557f02b5cb40 2024-04-24T06:46:22.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.611+0000 7fc0e5600640 1 -- 172.21.15.5:0/1661993639 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc0e80bbbb0 con 0x557f02b5cb40 2024-04-24T06:46:22.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.611+0000 7fc1019a1740 10 client.25811 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:46:22.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.611+0000 7fc1019a1740 10 client.25811 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-24T06:46:22.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.611+0000 7fc1019a1740 20 client.25811 populate_metadata read hostname 'smithi005' 2024-04-24T06:46:22.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.611+0000 7fc1019a1740 1 --2- 172.21.15.5:0/1661993639 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x557f02b77ce0 0x557f02b7a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:22.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.611+0000 7fc1019a1740 4 client.25811 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:46:22.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.611+0000 7fc1019a1740 1 -- 172.21.15.5:0/1661993639 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f02acfbf0 con 0x557f02b77ce0 2024-04-24T06:46:22.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.611+0000 7fc0efe00640 1 --2- 172.21.15.5:0/1661993639 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x557f02b77ce0 0x557f02b7a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:22.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.612+0000 7fc0efe00640 1 --2- 172.21.15.5:0/1661993639 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x557f02b77ce0 0x557f02b7a0d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5cf6227852cb0649 server_cookie=a2d3010d0a8aa8f7 in_seq=0 out_seq=0 2024-04-24T06:46:22.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.612+0000 7fc0e5600640 10 client.25811 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:46:22.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.613+0000 7fc0e5600640 1 -- 172.21.15.5:0/1661993639 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557f02acfbf0 con 0x557f02b77ce0 2024-04-24T06:46:22.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.613+0000 7fc0e5600640 10 client.25811 handle_command_reply: tid=0 2024-04-24T06:46:22.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.613+0000 7fc1019a1740 1 client.25811 shutdown 2024-04-24T06:46:22.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.613+0000 7fc1019a1740 1 -- 172.21.15.5:0/1661993639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f02b5cb40 msgr2=0x557f02b67540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:22.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.613+0000 7fc1019a1740 1 --2- 172.21.15.5:0/1661993639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f02b5cb40 0x557f02b67540 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc0e800ea30 tx=0x7fc0e80d4080 comp rx=0 tx=0).stop 2024-04-24T06:46:22.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.613+0000 7fc1019a1740 1 -- 172.21.15.5:0/1661993639 shutdown_connections 2024-04-24T06:46:22.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.613+0000 7fc1019a1740 1 --2- 172.21.15.5:0/1661993639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f02b67b30 0x557f02b64e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.613+0000 7fc1019a1740 1 --2- 172.21.15.5:0/1661993639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f02b5d460 0x557f02b64830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.613+0000 7fc1019a1740 1 --2- 172.21.15.5:0/1661993639 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x557f02b77ce0 0x557f02b7a0d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.613+0000 7fc1019a1740 1 --2- 172.21.15.5:0/1661993639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f02b5cb40 0x557f02b67540 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.613+0000 7fc1019a1740 1 -- 172.21.15.5:0/1661993639 >> 172.21.15.5:0/1661993639 conn(0x557f02b58e60 msgr2=0x557f02b5ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:22.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.613+0000 7fc0e5600640 0 client.25811 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:46:22.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.614+0000 7fc1019a1740 1 -- 172.21.15.5:0/1661993639 shutdown_connections 2024-04-24T06:46:22.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.614+0000 7fc1019a1740 1 -- 172.21.15.5:0/1661993639 wait complete. 2024-04-24T06:46:22.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.614+0000 7fc1019a1740 20 client.25811 trim_cache size 0 max 16384 2024-04-24T06:46:22.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.629+0000 7fc1019a1740 1 Processor -- start 2024-04-24T06:46:22.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.629+0000 7fc1019a1740 1 -- start start 2024-04-24T06:46:22.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.630+0000 7fc1019a1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f02b5cb40 0x557f02b5f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:22.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.630+0000 7fc1019a1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f02b5d460 0x557f02b57dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:22.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.630+0000 7fc1019a1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f02b67b30 0x557f02b67070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:22.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.630+0000 7fc1019a1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f81b6ac0 con 0x557f02b67b30 2024-04-24T06:46:22.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.630+0000 7fc1019a1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f81b4390 con 0x557f02b5cb40 2024-04-24T06:46:22.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.630+0000 7fc1019a1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f81b21e0 con 0x557f02b5d460 2024-04-24T06:46:22.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.630+0000 7fc0f7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f02b5cb40 0x557f02b5f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:22.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.630+0000 7fc0fca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f02b67b30 0x557f02b67070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:22.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.630+0000 7fc0f7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f02b5cb40 0x557f02b5f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:37090/0 (socket says 172.21.15.5:37090) 2024-04-24T06:46:22.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.630+0000 7fc0f7e00640 1 -- 172.21.15.5:0/3326578464 learned_addr learned my addr 172.21.15.5:0/3326578464 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:22.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.630+0000 7fc0efe00640 1 --2- 172.21.15.5:0/3326578464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f02b5d460 0x557f02b57dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:22.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.630+0000 7fc0f7e00640 1 -- 172.21.15.5:0/3326578464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f02b5d460 msgr2=0x557f02b57dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:22.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.630+0000 7fc0f7e00640 1 --2- 172.21.15.5:0/3326578464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f02b5d460 0x557f02b57dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.630+0000 7fc0f7e00640 1 -- 172.21.15.5:0/3326578464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f02b67b30 msgr2=0x557f02b67070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:22.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.630+0000 7fc0f7e00640 1 --2- 172.21.15.5:0/3326578464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f02b67b30 0x557f02b67070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.630+0000 7fc0f7e00640 1 -- 172.21.15.5:0/3326578464 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f02b651b0 con 0x557f02b5cb40 2024-04-24T06:46:22.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.630+0000 7fc0efe00640 1 --2- 172.21.15.5:0/3326578464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f02b5d460 0x557f02b57dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:46:22.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.631+0000 7fc0f7e00640 1 --2- 172.21.15.5:0/3326578464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f02b5cb40 0x557f02b5f2a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc0e801f000 tx=0x7fc0e809af00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc1f2e6475c4386b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:22.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.631+0000 7fc0e7e00640 1 -- 172.21.15.5:0/3326578464 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e80b7080 con 0x557f02b5cb40 2024-04-24T06:46:22.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.631+0000 7fc0e7e00640 1 -- 172.21.15.5:0/3326578464 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0e80d9030 con 0x557f02b5cb40 2024-04-24T06:46:22.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.631+0000 7fc0e7e00640 1 -- 172.21.15.5:0/3326578464 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e80ce420 con 0x557f02b5cb40 2024-04-24T06:46:22.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.632+0000 7fc1019a1740 1 -- 172.21.15.5:0/3326578464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f02b5cb40 msgr2=0x557f02b5f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:22.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.632+0000 7fc1019a1740 1 --2- 172.21.15.5:0/3326578464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f02b5cb40 0x557f02b5f2a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc0e801f000 tx=0x7fc0e809af00 comp rx=0 tx=0).stop 2024-04-24T06:46:22.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.632+0000 7fc1019a1740 1 -- 172.21.15.5:0/3326578464 shutdown_connections 2024-04-24T06:46:22.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.632+0000 7fc1019a1740 1 --2- 172.21.15.5:0/3326578464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f02b67b30 0x557f02b67070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.632+0000 7fc1019a1740 1 --2- 172.21.15.5:0/3326578464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f02b5d460 0x557f02b57dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.632+0000 7fc1019a1740 1 --2- 172.21.15.5:0/3326578464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f02b5cb40 0x557f02b5f2a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.632+0000 7fc1019a1740 1 -- 172.21.15.5:0/3326578464 >> 172.21.15.5:0/3326578464 conn(0x557f02b58e60 msgr2=0x557f02b576c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:22.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.632+0000 7fc1019a1740 1 -- 172.21.15.5:0/3326578464 shutdown_connections 2024-04-24T06:46:22.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.632+0000 7fc1019a1740 1 -- 172.21.15.5:0/3326578464 wait complete. 2024-04-24T06:46:22.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.632+0000 7fc1019a1740 1 -- start start 2024-04-24T06:46:22.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.632+0000 7fc1019a1740 1 Processor -- start 2024-04-24T06:46:22.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.633+0000 7fc1019a1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f02b5cb40 0x557f02b6fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:22.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.633+0000 7fc1019a1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f02b5d460 0x557f02b70270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:22.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.633+0000 7fc1019a1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f02b67b30 0x557f02b68dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:22.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.633+0000 7fc1019a1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f81b2850 con 0x557f02b67b30 2024-04-24T06:46:22.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.633+0000 7fc1019a1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f81b3cc0 con 0x557f02b5d460 2024-04-24T06:46:22.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.633+0000 7fc1019a1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f81b2f20 con 0x557f02b5cb40 2024-04-24T06:46:22.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.633+0000 7fc1019a1740 10 client.0 _ll_register_callbacks cb 0x7fc0e80d4240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:46:22.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.633+0000 7fc0efe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f02b5d460 0x557f02b70270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:22.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.633+0000 7fc0fca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f02b67b30 0x557f02b68dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:22.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.633+0000 7fc0efe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f02b5d460 0x557f02b70270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:37106/0 (socket says 172.21.15.5:37106) 2024-04-24T06:46:22.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.633+0000 7fc0efe00640 1 -- 172.21.15.5:0/1211496060 learned_addr learned my addr 172.21.15.5:0/1211496060 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:22.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.633+0000 7fc0f7e00640 1 --2- 172.21.15.5:0/1211496060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f02b5cb40 0x557f02b6fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:22.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.634+0000 7fc0efe00640 1 -- 172.21.15.5:0/1211496060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f02b5cb40 msgr2=0x557f02b6fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:22.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.634+0000 7fc0efe00640 1 --2- 172.21.15.5:0/1211496060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f02b5cb40 0x557f02b6fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.634+0000 7fc0efe00640 1 -- 172.21.15.5:0/1211496060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f02b67b30 msgr2=0x557f02b68dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:22.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.634+0000 7fc0efe00640 1 --2- 172.21.15.5:0/1211496060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f02b67b30 0x557f02b68dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.634+0000 7fc0efe00640 1 -- 172.21.15.5:0/1211496060 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f0289e7d0 con 0x557f02b5d460 2024-04-24T06:46:22.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.634+0000 7fc0efe00640 1 --2- 172.21.15.5:0/1211496060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f02b5d460 0x557f02b70270 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x557f02b59ff0 tx=0x7fc0e0009620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=211cbad045e71563 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:22.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.634+0000 7fc0e5600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:46:22.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.634+0000 7fc0e5600640 1 -- 172.21.15.5:0/1211496060 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e005a060 con 0x557f02b5d460 2024-04-24T06:46:22.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.634+0000 7fc0e5600640 1 -- 172.21.15.5:0/1211496060 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fc0e0004400 con 0x557f02b5d460 2024-04-24T06:46:22.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.634+0000 7fc1019a1740 1 -- 172.21.15.5:0/1211496060 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0e80d9030 con 0x557f02b5d460 2024-04-24T06:46:22.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.635+0000 7fc0e5600640 1 -- 172.21.15.5:0/1211496060 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc0e0035040 con 0x557f02b5d460 2024-04-24T06:46:22.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.635+0000 7fc1019a1740 10 client.25817 fetch_fsmap learned FSMap version 25 2024-04-24T06:46:22.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.635+0000 7fc1019a1740 1 -- 172.21.15.5:0/1211496060 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557f02bde830 con 0x557f02b5d460 2024-04-24T06:46:22.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.635+0000 7fc0e5600640 1 -- 172.21.15.5:0/1211496060 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0e80d9030 con 0x557f02b5d460 2024-04-24T06:46:22.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.635+0000 7fc0e5600640 1 -- 172.21.15.5:0/1211496060 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc0e00318b0 con 0x557f02b5d460 2024-04-24T06:46:22.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.635+0000 7fc1019a1740 10 client.25817 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:46:22.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.635+0000 7fc1019a1740 10 client.25817 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-24T06:46:22.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.635+0000 7fc1019a1740 20 client.25817 populate_metadata read hostname 'smithi005' 2024-04-24T06:46:22.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.635+0000 7fc1019a1740 1 --2- 172.21.15.5:0/1211496060 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x557f02bdebf0 0x557f02be0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:22.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.635+0000 7fc1019a1740 4 client.25817 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:46:22.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.635+0000 7fc1019a1740 1 -- 172.21.15.5:0/1211496060 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557f02ac9a60 con 0x557f02bdebf0 2024-04-24T06:46:22.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.636+0000 7fc0f7e00640 1 --2- 172.21.15.5:0/1211496060 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x557f02bdebf0 0x557f02be0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:22.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.636+0000 7fc0f7e00640 1 --2- 172.21.15.5:0/1211496060 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x557f02bdebf0 0x557f02be0fe0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=df83750e60e9b773 server_cookie=a7c239bc61b2df42 in_seq=0 out_seq=0 2024-04-24T06:46:22.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.636+0000 7fc0e5600640 10 client.25817 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:46:22.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.636+0000 7fc0e5600640 1 -- 172.21.15.5:0/1211496060 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557f02ac9a60 con 0x557f02bdebf0 2024-04-24T06:46:22.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.636+0000 7fc0e5600640 10 client.25817 handle_command_reply: tid=0 2024-04-24T06:46:22.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.636+0000 7fc1019a1740 1 client.25817 shutdown 2024-04-24T06:46:22.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.637+0000 7fc1019a1740 1 -- 172.21.15.5:0/1211496060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f02b5d460 msgr2=0x557f02b70270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:22.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.637+0000 7fc1019a1740 1 --2- 172.21.15.5:0/1211496060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f02b5d460 0x557f02b70270 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x557f02b59ff0 tx=0x7fc0e0009620 comp rx=0 tx=0).stop 2024-04-24T06:46:22.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.637+0000 7fc1019a1740 1 -- 172.21.15.5:0/1211496060 shutdown_connections 2024-04-24T06:46:22.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.637+0000 7fc1019a1740 1 --2- 172.21.15.5:0/1211496060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f02b67b30 0x557f02b68dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.637+0000 7fc1019a1740 1 --2- 172.21.15.5:0/1211496060 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x557f02bdebf0 0x557f02be0fe0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.637+0000 7fc1019a1740 1 --2- 172.21.15.5:0/1211496060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f02b5d460 0x557f02b70270 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.637+0000 7fc1019a1740 1 --2- 172.21.15.5:0/1211496060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f02b5cb40 0x557f02b6fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.637+0000 7fc0e5600640 0 client.25817 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:46:22.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.637+0000 7fc1019a1740 1 -- 172.21.15.5:0/1211496060 >> 172.21.15.5:0/1211496060 conn(0x557f02b58e60 msgr2=0x557f02b57f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:22.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.637+0000 7fc1019a1740 1 -- 172.21.15.5:0/1211496060 shutdown_connections 2024-04-24T06:46:22.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.637+0000 7fc1019a1740 1 -- 172.21.15.5:0/1211496060 wait complete. 2024-04-24T06:46:22.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.637+0000 7fc1019a1740 20 client.25817 trim_cache size 0 max 16384 2024-04-24T06:46:22.641 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:46:22.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.638+0000 7fc0fd400640 1 -- 172.21.15.5:0/3045983417 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc0dc0a3da0 msgr2=0x7fc0dc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:22.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.638+0000 7fc0fd400640 1 --2- 172.21.15.5:0/3045983417 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc0dc0a3da0 0x7fc0dc0a6260 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc0f806b470 tx=0x7fc0f0009500 comp rx=0 tx=0).stop 2024-04-24T06:46:22.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.638+0000 7fc0fd400640 1 -- 172.21.15.5:0/3045983417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0f80743a0 msgr2=0x7fc0f819ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:22.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.638+0000 7fc0fd400640 1 --2- 172.21.15.5:0/3045983417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0f80743a0 0x7fc0f819ed40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc0e800ea60 tx=0x7fc0e800ef20 comp rx=0 tx=0).stop 2024-04-24T06:46:22.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.638+0000 7fc0fd400640 1 -- 172.21.15.5:0/3045983417 shutdown_connections 2024-04-24T06:46:22.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.638+0000 7fc0fd400640 1 --2- 172.21.15.5:0/3045983417 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc0dc0a3da0 0x7fc0dc0a6260 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.638+0000 7fc0fd400640 1 --2- 172.21.15.5:0/3045983417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0f8112e10 0x7fc0f81a3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.638+0000 7fc0fd400640 1 --2- 172.21.15.5:0/3045983417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0f8074cc0 0x7fc0f819f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.638+0000 7fc0fd400640 1 --2- 172.21.15.5:0/3045983417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0f80743a0 0x7fc0f819ed40 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.638+0000 7fc0fd400640 1 -- 172.21.15.5:0/3045983417 >> 172.21.15.5:0/3045983417 conn(0x7fc0f80fd950 msgr2=0x7fc0f80fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:22.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.638+0000 7fc0fd400640 1 -- 172.21.15.5:0/3045983417 shutdown_connections 2024-04-24T06:46:22.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.638+0000 7fc0fd400640 1 -- 172.21.15.5:0/3045983417 wait complete. 2024-04-24T06:46:22.658 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T06:46:22.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.948+0000 7fc2f5c00640 1 -- 172.21.15.5:0/3183494244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2f0109b00 msgr2=0x7fc2f0114c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:22.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.948+0000 7fc2f5c00640 1 --2- 172.21.15.5:0/3183494244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2f0109b00 0x7fc2f0114c10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc009890 tx=0x7fc2dc02a360 comp rx=0 tx=0).stop 2024-04-24T06:46:22.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.948+0000 7fc2f5c00640 1 -- 172.21.15.5:0/3183494244 shutdown_connections 2024-04-24T06:46:22.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.948+0000 7fc2f5c00640 1 --2- 172.21.15.5:0/3183494244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2f0115150 0x7fc2f0117540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.948+0000 7fc2f5c00640 1 --2- 172.21.15.5:0/3183494244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2f0109b00 0x7fc2f0114c10 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.948+0000 7fc2f5c00640 1 --2- 172.21.15.5:0/3183494244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2f01091e0 0x7fc2f01095c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.948+0000 7fc2f5c00640 1 -- 172.21.15.5:0/3183494244 >> 172.21.15.5:0/3183494244 conn(0x7fc2f006d0a0 msgr2=0x7fc2f006d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:22.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.951+0000 7fc2f5c00640 1 -- 172.21.15.5:0/3183494244 shutdown_connections 2024-04-24T06:46:22.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.953+0000 7fc2f5c00640 1 -- 172.21.15.5:0/3183494244 wait complete. 2024-04-24T06:46:22.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.954+0000 7fc2f5c00640 1 Processor -- start 2024-04-24T06:46:22.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.955+0000 7fc2f5c00640 1 -- start start 2024-04-24T06:46:22.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.955+0000 7fc2f5c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2f01091e0 0x7fc2f0113170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:22.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.955+0000 7fc2f5c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2f0109b00 0x7fc2f0124020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:22.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.955+0000 7fc2f5c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2f0115150 0x7fc2f01136b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:22.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.956+0000 7fc2ef400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2f0115150 0x7fc2f01136b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:22.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.956+0000 7fc2ef400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2f0115150 0x7fc2f01136b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:42824/0 (socket says 172.21.15.5:42824) 2024-04-24T06:46:22.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.956+0000 7fc2ef400640 1 -- 172.21.15.5:0/3552832328 learned_addr learned my addr 172.21.15.5:0/3552832328 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:22.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.956+0000 7fc2f5c00640 1 -- 172.21.15.5:0/3552832328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f0118a00 con 0x7fc2f01091e0 2024-04-24T06:46:22.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.956+0000 7fc2f5c00640 1 -- 172.21.15.5:0/3552832328 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f0118880 con 0x7fc2f0109b00 2024-04-24T06:46:22.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.956+0000 7fc2f5c00640 1 -- 172.21.15.5:0/3552832328 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f0118b80 con 0x7fc2f0115150 2024-04-24T06:46:22.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.957+0000 7fc2ef400640 1 -- 172.21.15.5:0/3552832328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2f0109b00 msgr2=0x7fc2f0124020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:22.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.957+0000 7fc2ef400640 1 --2- 172.21.15.5:0/3552832328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2f0109b00 0x7fc2f0124020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.957+0000 7fc2ef400640 1 -- 172.21.15.5:0/3552832328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2f01091e0 msgr2=0x7fc2f0113170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:22.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.957+0000 7fc2ef400640 1 --2- 172.21.15.5:0/3552832328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2f01091e0 0x7fc2f0113170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.957+0000 7fc2ef400640 1 -- 172.21.15.5:0/3552832328 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f0113e30 con 0x7fc2f0115150 2024-04-24T06:46:22.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.961+0000 7fc2ef400640 1 --2- 172.21.15.5:0/3552832328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2f0115150 0x7fc2f01136b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc2e800c970 tx=0x7fc2e800ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:22.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.961+0000 7fc2e7400640 1 -- 172.21.15.5:0/3552832328 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e8007bb0 con 0x7fc2f0115150 2024-04-24T06:46:22.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.961+0000 7fc2f5c00640 1 -- 172.21.15.5:0/3552832328 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2f01ab540 con 0x7fc2f0115150 2024-04-24T06:46:22.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.961+0000 7fc2e7400640 1 -- 172.21.15.5:0/3552832328 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2e8007d50 con 0x7fc2f0115150 2024-04-24T06:46:22.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.961+0000 7fc2e7400640 1 -- 172.21.15.5:0/3552832328 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e8018800 con 0x7fc2f0115150 2024-04-24T06:46:22.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.963+0000 7fc2f5c00640 1 -- 172.21.15.5:0/3552832328 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2f01aba30 con 0x7fc2f0115150 2024-04-24T06:46:22.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.969+0000 7fc2e7400640 1 -- 172.21.15.5:0/3552832328 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fc2e8018a80 con 0x7fc2f0115150 2024-04-24T06:46:22.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.975+0000 7fc2e7400640 1 --2- 172.21.15.5:0/3552832328 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc2d40a3da0 0x7fc2d40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:22.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.975+0000 7fc2e7400640 1 -- 172.21.15.5:0/3552832328 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fc2e80d00e0 con 0x7fc2f0115150 2024-04-24T06:46:22.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.976+0000 7fc2eea00640 1 --2- 172.21.15.5:0/3552832328 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc2d40a3da0 0x7fc2d40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:22.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.977+0000 7fc2f8989740 1 Processor -- start 2024-04-24T06:46:22.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.978+0000 7fc2f8989740 1 -- start start 2024-04-24T06:46:22.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.980+0000 7fc2f8989740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584eca9c950 0x5584eca9cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:22.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.980+0000 7fc2f8989740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584eca9d270 0x5584ecaa7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:22.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.980+0000 7fc2f8989740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584ecaa7940 0x5584ecaa9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:22.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.980+0000 7fc2ef400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584ecaa7940 0x5584ecaa9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:22.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.981+0000 7fc2ef400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584ecaa7940 0x5584ecaa9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:37142/0 (socket says 172.21.15.5:37142) 2024-04-24T06:46:22.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.981+0000 7fc2ef400640 1 -- 172.21.15.5:0/2645104585 learned_addr learned my addr 172.21.15.5:0/2645104585 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:22.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.981+0000 7fc2f8989740 1 -- 172.21.15.5:0/2645104585 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5584eca021d0 con 0x5584eca9d270 2024-04-24T06:46:22.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.981+0000 7fc2f8989740 1 -- 172.21.15.5:0/2645104585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5584ec9a8450 con 0x5584ecaa7940 2024-04-24T06:46:22.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.981+0000 7fc2f8989740 1 -- 172.21.15.5:0/2645104585 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5584ec9fe190 con 0x5584eca9c950 2024-04-24T06:46:22.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.981+0000 7fc2ee000640 1 --2- 172.21.15.5:0/2645104585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584eca9d270 0x5584ecaa7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:22.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.982+0000 7fc2eea00640 1 --2- 172.21.15.5:0/2645104585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584eca9c950 0x5584eca9cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:22.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.982+0000 7fc2eea00640 1 --2- 172.21.15.5:0/3552832328 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc2d40a3da0 0x7fc2d40a6260 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc2e000ab90 tx=0x7fc2e0009250 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:22.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.982+0000 7fc2ef400640 1 -- 172.21.15.5:0/2645104585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584eca9c950 msgr2=0x5584eca9cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:22.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.982+0000 7fc2ef400640 1 --2- 172.21.15.5:0/2645104585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584eca9c950 0x5584eca9cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.982+0000 7fc2ef400640 1 -- 172.21.15.5:0/2645104585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584eca9d270 msgr2=0x5584ecaa7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:22.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.982+0000 7fc2ef400640 1 --2- 172.21.15.5:0/2645104585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584eca9d270 0x5584ecaa7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.982+0000 7fc2ef400640 1 -- 172.21.15.5:0/2645104585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584eca65310 con 0x5584ecaa7940 2024-04-24T06:46:22.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.983+0000 7fc2ef400640 1 --2- 172.21.15.5:0/2645104585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584ecaa7940 0x5584ecaa9d30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc2e80cd940 tx=0x7fc2e809bbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77a9f3cf0ee529d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:22.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.983+0000 7fc2dbe00640 1 -- 172.21.15.5:0/2645104585 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e80b4020 con 0x5584ecaa7940 2024-04-24T06:46:22.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.984+0000 7fc2dbe00640 1 -- 172.21.15.5:0/2645104585 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2e80a4da0 con 0x5584ecaa7940 2024-04-24T06:46:22.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.984+0000 7fc2dbe00640 1 -- 172.21.15.5:0/2645104585 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e80b4020 con 0x5584ecaa7940 2024-04-24T06:46:22.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.986+0000 7fc2f8989740 1 -- 172.21.15.5:0/2645104585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584ecaa7940 msgr2=0x5584ecaa9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:22.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.987+0000 7fc2f8989740 1 --2- 172.21.15.5:0/2645104585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584ecaa7940 0x5584ecaa9d30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc2e80cd940 tx=0x7fc2e809bbc0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.988+0000 7fc2f8989740 1 -- 172.21.15.5:0/2645104585 shutdown_connections 2024-04-24T06:46:22.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.988+0000 7fc2f8989740 1 --2- 172.21.15.5:0/2645104585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584ecaa7940 0x5584ecaa9d30 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.988+0000 7fc2f8989740 1 --2- 172.21.15.5:0/2645104585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584eca9d270 0x5584ecaa7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.988+0000 7fc2f8989740 1 --2- 172.21.15.5:0/2645104585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584eca9c950 0x5584eca9cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:22.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.988+0000 7fc2f8989740 1 -- 172.21.15.5:0/2645104585 >> 172.21.15.5:0/2645104585 conn(0x5584eca98c70 msgr2=0x5584eca99050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:22.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.990+0000 7fc2f8989740 1 -- 172.21.15.5:0/2645104585 shutdown_connections 2024-04-24T06:46:22.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.991+0000 7fc2f8989740 1 -- 172.21.15.5:0/2645104585 wait complete. 2024-04-24T06:46:22.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.992+0000 7fc2f8989740 1 -- start start 2024-04-24T06:46:22.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.994+0000 7fc2f8989740 1 Processor -- start 2024-04-24T06:46:22.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.996+0000 7fc2f8989740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584eca9c950 0x5584ecaa72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:22.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.996+0000 7fc2f8989740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584eca9d270 0x5584ecaa45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:22.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.997+0000 7fc2f8989740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584ecaa7940 0x5584ecaa4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:22.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.997+0000 7fc2f8989740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5584ecaaa7d0 con 0x5584ecaa7940 2024-04-24T06:46:22.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.997+0000 7fc2f8989740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5584ec9bb810 con 0x5584eca9d270 2024-04-24T06:46:22.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.997+0000 7fc2f8989740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5584ecaaa950 con 0x5584eca9c950 2024-04-24T06:46:22.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.997+0000 7fc2f8989740 10 client.0 _ll_register_callbacks cb 0x5584ec989140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:46:22.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.997+0000 7fc2eea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584eca9c950 0x5584ecaa72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:22.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.997+0000 7fc2eea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584eca9c950 0x5584ecaa72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:42844/0 (socket says 172.21.15.5:42844) 2024-04-24T06:46:22.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.997+0000 7fc2eea00640 1 -- 172.21.15.5:0/4126752088 learned_addr learned my addr 172.21.15.5:0/4126752088 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:22.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.997+0000 7fc2ef400640 1 --2- 172.21.15.5:0/4126752088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584ecaa7940 0x5584ecaa4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:22.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.998+0000 7fc2ee000640 1 --2- 172.21.15.5:0/4126752088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584eca9d270 0x5584ecaa45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:22.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.998+0000 7fc2eea00640 1 -- 172.21.15.5:0/4126752088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584eca9d270 msgr2=0x5584ecaa45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:23.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.998+0000 7fc2eea00640 1 --2- 172.21.15.5:0/4126752088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584eca9d270 0x5584ecaa45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.998+0000 7fc2eea00640 1 -- 172.21.15.5:0/4126752088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584ecaa7940 msgr2=0x5584ecaa4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:23.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.998+0000 7fc2eea00640 1 --2- 172.21.15.5:0/4126752088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584ecaa7940 0x5584ecaa4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.998+0000 7fc2eea00640 1 -- 172.21.15.5:0/4126752088 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584ec9cc3f0 con 0x5584eca9c950 2024-04-24T06:46:23.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:22.999+0000 7fc2eea00640 1 --2- 172.21.15.5:0/4126752088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584eca9c950 0x5584ecaa72c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0019e20 tx=0x7fc2e0004bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ef8d3dba858ee25 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:23.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.000+0000 7fc2d8c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:46:23.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.001+0000 7fc2d8c00640 1 -- 172.21.15.5:0/4126752088 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e0018740 con 0x5584eca9c950 2024-04-24T06:46:23.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.001+0000 7fc2d8c00640 1 -- 172.21.15.5:0/4126752088 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fc2e00188e0 con 0x5584eca9c950 2024-04-24T06:46:23.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.001+0000 7fc2f8989740 1 -- 172.21.15.5:0/4126752088 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2e80a4da0 con 0x5584eca9c950 2024-04-24T06:46:23.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.002+0000 7fc2d8c00640 1 -- 172.21.15.5:0/4126752088 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc2e0018ae0 con 0x5584eca9c950 2024-04-24T06:46:23.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.002+0000 7fc2f8989740 10 client.25852 fetch_fsmap learned FSMap version 25 2024-04-24T06:46:23.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.002+0000 7fc2f8989740 1 -- 172.21.15.5:0/4126752088 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5584ec7143f0 con 0x5584eca9c950 2024-04-24T06:46:23.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.003+0000 7fc2d8c00640 1 -- 172.21.15.5:0/4126752088 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2e80a4da0 con 0x5584eca9c950 2024-04-24T06:46:23.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.003+0000 7fc2d8c00640 1 -- 172.21.15.5:0/4126752088 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc2e0016430 con 0x5584eca9c950 2024-04-24T06:46:23.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.004+0000 7fc2f8989740 10 client.25852 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:46:23.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.004+0000 7fc2f8989740 10 client.25852 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T06:46:23.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.004+0000 7fc2f8989740 20 client.25852 populate_metadata read hostname 'smithi005' 2024-04-24T06:46:23.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.004+0000 7fc2f8989740 1 --2- 172.21.15.5:0/4126752088 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x5584ecab7920 0x5584ecab9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:23.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.004+0000 7fc2f8989740 4 client.25852 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:46:23.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.004+0000 7fc2f8989740 1 -- 172.21.15.5:0/4126752088 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584eca0fbf0 con 0x5584ecab7920 2024-04-24T06:46:23.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.005+0000 7fc2ee000640 1 --2- 172.21.15.5:0/4126752088 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x5584ecab7920 0x5584ecab9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:23.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.007+0000 7fc2d8c00640 10 client.25852 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:46:23.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.010+0000 7fc2ee000640 1 --2- 172.21.15.5:0/4126752088 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x5584ecab7920 0x5584ecab9d10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a9feccf5068b5962 server_cookie=f98e74d008f8063e in_seq=0 out_seq=0 2024-04-24T06:46:23.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.011+0000 7fc2d8c00640 1 -- 172.21.15.5:0/4126752088 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5584eca0fbf0 con 0x5584ecab7920 2024-04-24T06:46:23.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.011+0000 7fc2d8c00640 10 client.25852 handle_command_reply: tid=0 2024-04-24T06:46:23.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.012+0000 7fc2f8989740 1 client.25852 shutdown 2024-04-24T06:46:23.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.013+0000 7fc2f8989740 1 -- 172.21.15.5:0/4126752088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584eca9c950 msgr2=0x5584ecaa72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:23.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.013+0000 7fc2f8989740 1 --2- 172.21.15.5:0/4126752088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584eca9c950 0x5584ecaa72c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0019e20 tx=0x7fc2e0004bc0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.013+0000 7fc2f8989740 1 -- 172.21.15.5:0/4126752088 shutdown_connections 2024-04-24T06:46:23.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.013+0000 7fc2f8989740 1 --2- 172.21.15.5:0/4126752088 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x5584ecab7920 0x5584ecab9d10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.013+0000 7fc2f8989740 1 --2- 172.21.15.5:0/4126752088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584ecaa7940 0x5584ecaa4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.014+0000 7fc2f8989740 1 --2- 172.21.15.5:0/4126752088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584eca9d270 0x5584ecaa45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.014+0000 7fc2f8989740 1 --2- 172.21.15.5:0/4126752088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584eca9c950 0x5584ecaa72c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.014+0000 7fc2f8989740 1 -- 172.21.15.5:0/4126752088 >> 172.21.15.5:0/4126752088 conn(0x5584eca98c70 msgr2=0x5584eca9e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:23.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.014+0000 7fc2d8c00640 0 client.25852 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:46:23.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.014+0000 7fc2f8989740 1 -- 172.21.15.5:0/4126752088 shutdown_connections 2024-04-24T06:46:23.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.014+0000 7fc2f8989740 1 -- 172.21.15.5:0/4126752088 wait complete. 2024-04-24T06:46:23.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.017+0000 7fc2f8989740 20 client.25852 trim_cache size 0 max 16384 2024-04-24T06:46:23.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.051+0000 7fc2f8989740 1 Processor -- start 2024-04-24T06:46:23.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.052+0000 7fc2f8989740 1 -- start start 2024-04-24T06:46:23.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.052+0000 7fc2f8989740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584eca9c950 0x5584ecab36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:23.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.052+0000 7fc2f8989740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584eca9d270 0x5584ecaa6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:23.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.052+0000 7fc2f8989740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584ecaa7940 0x5584ecabf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:23.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.052+0000 7fc2f8989740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f01b6670 con 0x5584ecaa7940 2024-04-24T06:46:23.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.052+0000 7fc2f8989740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f01b6d40 con 0x5584eca9c950 2024-04-24T06:46:23.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.052+0000 7fc2f8989740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f01b4610 con 0x5584eca9d270 2024-04-24T06:46:23.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.053+0000 7fc2ee000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584eca9d270 0x5584ecaa6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:23.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.053+0000 7fc2ee000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584eca9d270 0x5584ecaa6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:42850/0 (socket says 172.21.15.5:42850) 2024-04-24T06:46:23.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.053+0000 7fc2ee000640 1 -- 172.21.15.5:0/314656071 learned_addr learned my addr 172.21.15.5:0/314656071 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:23.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.053+0000 7fc2ee000640 1 -- 172.21.15.5:0/314656071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584eca9c950 msgr2=0x5584ecab36c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T06:46:23.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.054+0000 7fc2ee000640 1 --2- 172.21.15.5:0/314656071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584eca9c950 0x5584ecab36c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.054+0000 7fc2ee000640 1 -- 172.21.15.5:0/314656071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584ecaa7940 msgr2=0x5584ecabf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:23.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.054+0000 7fc2ee000640 1 --2- 172.21.15.5:0/314656071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584ecaa7940 0x5584ecabf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.054+0000 7fc2ee000640 1 -- 172.21.15.5:0/314656071 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584ecaaa0a0 con 0x5584eca9d270 2024-04-24T06:46:23.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.054+0000 7fc2ee000640 1 --2- 172.21.15.5:0/314656071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584eca9d270 0x5584ecaa6df0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc035040 tx=0x7fc2dc0097b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8ab03d8646ab74a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:23.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.055+0000 7fc2db400640 1 -- 172.21.15.5:0/314656071 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc046070 con 0x5584eca9d270 2024-04-24T06:46:23.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.055+0000 7fc2db400640 1 -- 172.21.15.5:0/314656071 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2dc041e50 con 0x5584eca9d270 2024-04-24T06:46:23.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.055+0000 7fc2db400640 1 -- 172.21.15.5:0/314656071 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc037040 con 0x5584eca9d270 2024-04-24T06:46:23.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.056+0000 7fc2f8989740 1 -- 172.21.15.5:0/314656071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584eca9d270 msgr2=0x5584ecaa6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:23.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.056+0000 7fc2f8989740 1 --2- 172.21.15.5:0/314656071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584eca9d270 0x5584ecaa6df0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc035040 tx=0x7fc2dc0097b0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.057+0000 7fc2f8989740 1 -- 172.21.15.5:0/314656071 shutdown_connections 2024-04-24T06:46:23.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.057+0000 7fc2f8989740 1 --2- 172.21.15.5:0/314656071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584ecaa7940 0x5584ecabf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.058+0000 7fc2f8989740 1 --2- 172.21.15.5:0/314656071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584eca9d270 0x5584ecaa6df0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.058+0000 7fc2f8989740 1 --2- 172.21.15.5:0/314656071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584eca9c950 0x5584ecab36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.058+0000 7fc2f8989740 1 -- 172.21.15.5:0/314656071 >> 172.21.15.5:0/314656071 conn(0x5584eca98c70 msgr2=0x5584eca9f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:23.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.059+0000 7fc2f8989740 1 -- 172.21.15.5:0/314656071 shutdown_connections 2024-04-24T06:46:23.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.059+0000 7fc2f8989740 1 -- 172.21.15.5:0/314656071 wait complete. 2024-04-24T06:46:23.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.060+0000 7fc2f8989740 1 -- start start 2024-04-24T06:46:23.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.060+0000 7fc2f8989740 1 Processor -- start 2024-04-24T06:46:23.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.061+0000 7fc2f8989740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584eca9c950 0x5584ecaab7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:23.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.061+0000 7fc2f8989740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584eca9d270 0x5584ecaabd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:23.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.061+0000 7fc2f8989740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584ecaa7940 0x5584ecaac270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:23.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.061+0000 7fc2f8989740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f01b2af0 con 0x5584eca9c950 2024-04-24T06:46:23.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.061+0000 7fc2f8989740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f01b3f40 con 0x5584ecaa7940 2024-04-24T06:46:23.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.061+0000 7fc2f8989740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f01b31a0 con 0x5584eca9d270 2024-04-24T06:46:23.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.061+0000 7fc2ee000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584eca9d270 0x5584ecaabd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:23.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.061+0000 7fc2ee000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584eca9d270 0x5584ecaabd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:42854/0 (socket says 172.21.15.5:42854) 2024-04-24T06:46:23.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.061+0000 7fc2ee000640 1 -- 172.21.15.5:0/2952674748 learned_addr learned my addr 172.21.15.5:0/2952674748 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:23.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.062+0000 7fc2f8989740 10 client.0 _ll_register_callbacks cb 0x7fc2e00197c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:46:23.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.062+0000 7fc2ef400640 1 --2- 172.21.15.5:0/2952674748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584ecaa7940 0x5584ecaac270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:23.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.062+0000 7fc2ef400640 1 -- 172.21.15.5:0/2952674748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584eca9d270 msgr2=0x5584ecaabd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:23.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.062+0000 7fc2ef400640 1 --2- 172.21.15.5:0/2952674748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584eca9d270 0x5584ecaabd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.062+0000 7fc2eea00640 1 --2- 172.21.15.5:0/2952674748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584eca9c950 0x5584ecaab7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:23.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.062+0000 7fc2ef400640 1 -- 172.21.15.5:0/2952674748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584eca9c950 msgr2=0x5584ecaab7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:23.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.062+0000 7fc2ef400640 1 --2- 172.21.15.5:0/2952674748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584eca9c950 0x5584ecaab7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.062+0000 7fc2ef400640 1 -- 172.21.15.5:0/2952674748 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584ec7de7d0 con 0x5584ecaa7940 2024-04-24T06:46:23.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.062+0000 7fc2ee000640 1 --2- 172.21.15.5:0/2952674748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584eca9d270 0x5584ecaabd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:46:23.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.063+0000 7fc2eea00640 1 --2- 172.21.15.5:0/2952674748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584eca9c950 0x5584ecaab7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:46:23.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.063+0000 7fc2ef400640 1 --2- 172.21.15.5:0/2952674748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584ecaa7940 0x5584ecaac270 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc2e809a3e0 tx=0x7fc2e800fcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7183787e96ab4cbf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:23.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.063+0000 7fc2d8c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:46:23.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.063+0000 7fc2d8c00640 1 -- 172.21.15.5:0/2952674748 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e80b4040 con 0x5584ecaa7940 2024-04-24T06:46:23.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.063+0000 7fc2d8c00640 1 -- 172.21.15.5:0/2952674748 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fc2e80b0bd0 con 0x5584ecaa7940 2024-04-24T06:46:23.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.063+0000 7fc2f8989740 1 -- 172.21.15.5:0/2952674748 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2dc041e50 con 0x5584ecaa7940 2024-04-24T06:46:23.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.064+0000 7fc2d8c00640 1 -- 172.21.15.5:0/2952674748 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc2e8042040 con 0x5584ecaa7940 2024-04-24T06:46:23.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.064+0000 7fc2f8989740 10 client.25826 fetch_fsmap learned FSMap version 25 2024-04-24T06:46:23.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.064+0000 7fc2f8989740 1 -- 172.21.15.5:0/2952674748 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5584ecb1e0b0 con 0x5584ecaa7940 2024-04-24T06:46:23.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.064+0000 7fc2d8c00640 1 -- 172.21.15.5:0/2952674748 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2dc041e50 con 0x5584ecaa7940 2024-04-24T06:46:23.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.064+0000 7fc2d8c00640 1 -- 172.21.15.5:0/2952674748 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc2e80bc040 con 0x5584ecaa7940 2024-04-24T06:46:23.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.064+0000 7fc2f8989740 10 client.25826 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:46:23.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.064+0000 7fc2f8989740 10 client.25826 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T06:46:23.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.064+0000 7fc2f8989740 20 client.25826 populate_metadata read hostname 'smithi005' 2024-04-24T06:46:23.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.065+0000 7fc2f8989740 1 --2- 172.21.15.5:0/2952674748 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x5584ecb1e340 0x5584ecb20730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:23.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.065+0000 7fc2f8989740 4 client.25826 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:46:23.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.065+0000 7fc2f8989740 1 -- 172.21.15.5:0/2952674748 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5584eca09a60 con 0x5584ecb1e340 2024-04-24T06:46:23.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.066+0000 7fc2eea00640 1 --2- 172.21.15.5:0/2952674748 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x5584ecb1e340 0x5584ecb20730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:23.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.067+0000 7fc2eea00640 1 --2- 172.21.15.5:0/2952674748 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x5584ecb1e340 0x5584ecb20730 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5fb633bbe725911e server_cookie=516fed41d0c4b536 in_seq=0 out_seq=0 2024-04-24T06:46:23.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.067+0000 7fc2d8c00640 10 client.25826 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:46:23.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.068+0000 7fc2d8c00640 1 -- 172.21.15.5:0/2952674748 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5584eca09a60 con 0x5584ecb1e340 2024-04-24T06:46:23.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.068+0000 7fc2d8c00640 10 client.25826 handle_command_reply: tid=0 2024-04-24T06:46:23.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.069+0000 7fc2f8989740 1 client.25826 shutdown 2024-04-24T06:46:23.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.069+0000 7fc2f8989740 1 -- 172.21.15.5:0/2952674748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584ecaa7940 msgr2=0x5584ecaac270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:23.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.069+0000 7fc2f8989740 1 --2- 172.21.15.5:0/2952674748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584ecaa7940 0x5584ecaac270 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc2e809a3e0 tx=0x7fc2e800fcd0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.069+0000 7fc2f8989740 1 -- 172.21.15.5:0/2952674748 shutdown_connections 2024-04-24T06:46:23.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.069+0000 7fc2f8989740 1 --2- 172.21.15.5:0/2952674748 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x5584ecb1e340 0x5584ecb20730 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.069+0000 7fc2f8989740 1 --2- 172.21.15.5:0/2952674748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584ecaa7940 0x5584ecaac270 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.069+0000 7fc2f8989740 1 --2- 172.21.15.5:0/2952674748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5584eca9d270 0x5584ecaabd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.069+0000 7fc2f8989740 1 --2- 172.21.15.5:0/2952674748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584eca9c950 0x5584ecaab7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.069+0000 7fc2f8989740 1 -- 172.21.15.5:0/2952674748 >> 172.21.15.5:0/2952674748 conn(0x5584eca98c70 msgr2=0x5584eca97560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:23.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.069+0000 7fc2d8c00640 0 client.25826 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:46:23.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.070+0000 7fc2f8989740 1 -- 172.21.15.5:0/2952674748 shutdown_connections 2024-04-24T06:46:23.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.070+0000 7fc2f8989740 1 -- 172.21.15.5:0/2952674748 wait complete. 2024-04-24T06:46:23.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.070+0000 7fc2f8989740 20 client.25826 trim_cache size 0 max 16384 2024-04-24T06:46:23.072 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:46:23.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.071+0000 7fc2f5c00640 1 -- 172.21.15.5:0/3552832328 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc2d40a3da0 msgr2=0x7fc2d40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:23.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.071+0000 7fc2f5c00640 1 --2- 172.21.15.5:0/3552832328 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc2d40a3da0 0x7fc2d40a6260 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc2e000ab90 tx=0x7fc2e0009250 comp rx=0 tx=0).stop 2024-04-24T06:46:23.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.071+0000 7fc2f5c00640 1 -- 172.21.15.5:0/3552832328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2f0115150 msgr2=0x7fc2f01136b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:23.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.071+0000 7fc2f5c00640 1 --2- 172.21.15.5:0/3552832328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2f0115150 0x7fc2f01136b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc2e800c970 tx=0x7fc2e800ce30 comp rx=0 tx=0).stop 2024-04-24T06:46:23.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.071+0000 7fc2f5c00640 1 -- 172.21.15.5:0/3552832328 shutdown_connections 2024-04-24T06:46:23.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.071+0000 7fc2f5c00640 1 --2- 172.21.15.5:0/3552832328 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc2d40a3da0 0x7fc2d40a6260 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.071+0000 7fc2f5c00640 1 --2- 172.21.15.5:0/3552832328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2f0115150 0x7fc2f01136b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.071+0000 7fc2f5c00640 1 --2- 172.21.15.5:0/3552832328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2f0109b00 0x7fc2f0124020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.071+0000 7fc2f5c00640 1 --2- 172.21.15.5:0/3552832328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2f01091e0 0x7fc2f0113170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.071+0000 7fc2f5c00640 1 -- 172.21.15.5:0/3552832328 >> 172.21.15.5:0/3552832328 conn(0x7fc2f006d0a0 msgr2=0x7fc2f011a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:23.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.071+0000 7fc2f5c00640 1 -- 172.21.15.5:0/3552832328 shutdown_connections 2024-04-24T06:46:23.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.071+0000 7fc2f5c00640 1 -- 172.21.15.5:0/3552832328 wait complete. 2024-04-24T06:46:23.090 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T06:46:23.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.274+0000 7f9a93200640 1 -- 172.21.15.5:0/2390874074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a94109360 msgr2=0x7f9a94110f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:23.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.274+0000 7f9a93200640 1 --2- 172.21.15.5:0/2390874074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a94109360 0x7f9a94110f30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c009ca0 tx=0x7f9a8c02a8c0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.274+0000 7f9a93200640 1 -- 172.21.15.5:0/2390874074 shutdown_connections 2024-04-24T06:46:23.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.274+0000 7f9a93200640 1 --2- 172.21.15.5:0/2390874074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a94109360 0x7f9a94110f30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.274+0000 7f9a93200640 1 --2- 172.21.15.5:0/2390874074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a94104c40 0x7f9a94108c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.274+0000 7f9a93200640 1 --2- 172.21.15.5:0/2390874074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a94104290 0x7f9a94104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.274+0000 7f9a93200640 1 -- 172.21.15.5:0/2390874074 >> 172.21.15.5:0/2390874074 conn(0x7f9a940ffaf0 msgr2=0x7f9a94101f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:23.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.274+0000 7f9a93200640 1 -- 172.21.15.5:0/2390874074 shutdown_connections 2024-04-24T06:46:23.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.274+0000 7f9a93200640 1 -- 172.21.15.5:0/2390874074 wait complete. 2024-04-24T06:46:23.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.275+0000 7f9a93200640 1 Processor -- start 2024-04-24T06:46:23.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.275+0000 7f9a93200640 1 -- start start 2024-04-24T06:46:23.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.275+0000 7f9a93200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a94104290 0x7f9a941a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:23.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.275+0000 7f9a93200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a94109360 0x7f9a941a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:23.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.275+0000 7f9a93200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a9419a770 0x7f9a9419abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:23.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.275+0000 7f9a93200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9a94114470 con 0x7f9a9419a770 2024-04-24T06:46:23.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.275+0000 7f9a93200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9a941142f0 con 0x7f9a94109360 2024-04-24T06:46:23.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.275+0000 7f9a93200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9a941145f0 con 0x7f9a94104290 2024-04-24T06:46:23.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.276+0000 7f9a91400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a94109360 0x7f9a941a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:23.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.276+0000 7f9a91e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a94104290 0x7f9a941a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:23.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.276+0000 7f9a92800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a9419a770 0x7f9a9419abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:23.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.276+0000 7f9a91400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a94109360 0x7f9a941a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:37182/0 (socket says 172.21.15.5:37182) 2024-04-24T06:46:23.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.276+0000 7f9a91400640 1 -- 172.21.15.5:0/589031779 learned_addr learned my addr 172.21.15.5:0/589031779 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:23.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.276+0000 7f9a91400640 1 -- 172.21.15.5:0/589031779 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a94104290 msgr2=0x7f9a941a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:23.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.276+0000 7f9a91400640 1 --2- 172.21.15.5:0/589031779 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a94104290 0x7f9a941a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.276+0000 7f9a91400640 1 -- 172.21.15.5:0/589031779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a9419a770 msgr2=0x7f9a9419abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:23.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.276+0000 7f9a91400640 1 --2- 172.21.15.5:0/589031779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a9419a770 0x7f9a9419abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.276+0000 7f9a91400640 1 -- 172.21.15.5:0/589031779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a9419b370 con 0x7f9a94109360 2024-04-24T06:46:23.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.276+0000 7f9a91e00640 1 --2- 172.21.15.5:0/589031779 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a94104290 0x7f9a941a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:46:23.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.276+0000 7f9a91400640 1 --2- 172.21.15.5:0/589031779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a94109360 0x7f9a941a0bf0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9a8000e9f0 tx=0x7f9a8000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:23.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.277+0000 7f9a8aa00640 1 -- 172.21.15.5:0/589031779 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8000ccc0 con 0x7f9a94109360 2024-04-24T06:46:23.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.277+0000 7f9a93200640 1 -- 172.21.15.5:0/589031779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a941a7450 con 0x7f9a94109360 2024-04-24T06:46:23.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.277+0000 7f9a8aa00640 1 -- 172.21.15.5:0/589031779 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a80005500 con 0x7f9a94109360 2024-04-24T06:46:23.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.277+0000 7f9a8aa00640 1 -- 172.21.15.5:0/589031779 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a80010690 con 0x7f9a94109360 2024-04-24T06:46:23.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.277+0000 7f9a93200640 1 -- 172.21.15.5:0/589031779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a941a7800 con 0x7f9a94109360 2024-04-24T06:46:23.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.280+0000 7f9a9bb89740 1 Processor -- start 2024-04-24T06:46:23.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.280+0000 7f9a9bb89740 1 -- start start 2024-04-24T06:46:23.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.280+0000 7f9a9bb89740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fb4eaeb40 0x555fb4eaef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:23.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.280+0000 7f9a9bb89740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fb4eaf460 0x555fb4eb95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:23.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.280+0000 7f9a9bb89740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fb4eb9b30 0x555fb4ebbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:23.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.280+0000 7f9a9bb89740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555fb4e141d0 con 0x555fb4eb9b30 2024-04-24T06:46:23.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.280+0000 7f9a9bb89740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555fb4dba450 con 0x555fb4eaf460 2024-04-24T06:46:23.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.280+0000 7f9a9bb89740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555fb4e10190 con 0x555fb4eaeb40 2024-04-24T06:46:23.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.281+0000 7f9a91e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fb4eaf460 0x555fb4eb95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:23.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.281+0000 7f9a92800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fb4eaeb40 0x555fb4eaef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:23.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.281+0000 7f9a91e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fb4eaf460 0x555fb4eb95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:37184/0 (socket says 172.21.15.5:37184) 2024-04-24T06:46:23.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.281+0000 7f9a92800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fb4eaeb40 0x555fb4eaef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:42880/0 (socket says 172.21.15.5:42880) 2024-04-24T06:46:23.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.281+0000 7f9a91e00640 1 -- 172.21.15.5:0/3380887205 learned_addr learned my addr 172.21.15.5:0/3380887205 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:23.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.281+0000 7f9a91e00640 1 -- 172.21.15.5:0/3380887205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fb4eaeb40 msgr2=0x555fb4eaef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:23.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.281+0000 7f9a91e00640 1 --2- 172.21.15.5:0/3380887205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fb4eaeb40 0x555fb4eaef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.281+0000 7f9a91e00640 1 -- 172.21.15.5:0/3380887205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fb4eb9b30 msgr2=0x555fb4ebbf20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T06:46:23.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.281+0000 7f9a91e00640 1 --2- 172.21.15.5:0/3380887205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fb4eb9b30 0x555fb4ebbf20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.281+0000 7f9a91e00640 1 -- 172.21.15.5:0/3380887205 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fb4e77310 con 0x555fb4eaf460 2024-04-24T06:46:23.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.281+0000 7f9a8aa00640 1 -- 172.21.15.5:0/589031779 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f9a800046c0 con 0x7f9a94109360 2024-04-24T06:46:23.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.281+0000 7f9a91e00640 1 --2- 172.21.15.5:0/3380887205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fb4eaf460 0x555fb4eb95f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9a840115f0 tx=0x7f9a84011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54f0f25d59ad4541 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:23.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.281+0000 7f9a7f400640 1 -- 172.21.15.5:0/3380887205 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8401e070 con 0x555fb4eaf460 2024-04-24T06:46:23.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.281+0000 7f9a7f400640 1 -- 172.21.15.5:0/3380887205 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a8400fb20 con 0x555fb4eaf460 2024-04-24T06:46:23.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.282+0000 7f9a7f400640 1 -- 172.21.15.5:0/3380887205 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a84019c40 con 0x555fb4eaf460 2024-04-24T06:46:23.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.282+0000 7f9a9bb89740 1 -- 172.21.15.5:0/3380887205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fb4eaf460 msgr2=0x555fb4eb95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:23.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.282+0000 7f9a9bb89740 1 --2- 172.21.15.5:0/3380887205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fb4eaf460 0x555fb4eb95f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9a840115f0 tx=0x7f9a84011ab0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.282+0000 7f9a9bb89740 1 -- 172.21.15.5:0/3380887205 shutdown_connections 2024-04-24T06:46:23.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.282+0000 7f9a9bb89740 1 --2- 172.21.15.5:0/3380887205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fb4eb9b30 0x555fb4ebbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.282+0000 7f9a9bb89740 1 --2- 172.21.15.5:0/3380887205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fb4eaf460 0x555fb4eb95f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.282+0000 7f9a9bb89740 1 --2- 172.21.15.5:0/3380887205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fb4eaeb40 0x555fb4eaef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.282+0000 7f9a9bb89740 1 -- 172.21.15.5:0/3380887205 >> 172.21.15.5:0/3380887205 conn(0x555fb4eaae60 msgr2=0x555fb4eab240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:23.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.283+0000 7f9a9bb89740 1 -- 172.21.15.5:0/3380887205 shutdown_connections 2024-04-24T06:46:23.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.283+0000 7f9a9bb89740 1 -- 172.21.15.5:0/3380887205 wait complete. 2024-04-24T06:46:23.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.283+0000 7f9a9bb89740 1 -- start start 2024-04-24T06:46:23.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.283+0000 7f9a8aa00640 1 --2- 172.21.15.5:0/589031779 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f9a780a3c30 0x7f9a780a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:23.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.283+0000 7f9a8aa00640 1 -- 172.21.15.5:0/589031779 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f9a800d0ab0 con 0x7f9a94109360 2024-04-24T06:46:23.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.283+0000 7f9a9bb89740 1 Processor -- start 2024-04-24T06:46:23.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.283+0000 7f9a92800640 1 --2- 172.21.15.5:0/589031779 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f9a780a3c30 0x7f9a780a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:23.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.284+0000 7f9a9bb89740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fb4eaeb40 0x555fb4eb95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:23.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.284+0000 7f9a9bb89740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fb4eaf460 0x555fb4eb6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:23.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.284+0000 7f9a9bb89740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fb4eb9b30 0x555fb4eb6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:23.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.284+0000 7f9a9bb89740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555fb4ebc9c0 con 0x555fb4eaf460 2024-04-24T06:46:23.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.284+0000 7f9a9bb89740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555fb4dcd810 con 0x555fb4eaeb40 2024-04-24T06:46:23.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.284+0000 7f9a9bb89740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555fb4ebcb40 con 0x555fb4eb9b30 2024-04-24T06:46:23.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.284+0000 7f9a92800640 1 --2- 172.21.15.5:0/589031779 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f9a780a3c30 0x7f9a780a60f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c006010 tx=0x7f9a8c005d80 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:23.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.284+0000 7f9a9bb89740 10 client.0 _ll_register_callbacks cb 0x555fb4d9b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:46:23.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.284+0000 7f9a91e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fb4eaeb40 0x555fb4eb95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:23.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.284+0000 7f9a92800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fb4eb9b30 0x555fb4eb6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:23.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.284+0000 7f9a91400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fb4eaf460 0x555fb4eb6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:23.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.284+0000 7f9a91e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fb4eaeb40 0x555fb4eb95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:37194/0 (socket says 172.21.15.5:37194) 2024-04-24T06:46:23.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.284+0000 7f9a91e00640 1 -- 172.21.15.5:0/1084501846 learned_addr learned my addr 172.21.15.5:0/1084501846 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:23.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.285+0000 7f9a92800640 1 -- 172.21.15.5:0/1084501846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fb4eaeb40 msgr2=0x555fb4eb95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:23.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.285+0000 7f9a92800640 1 --2- 172.21.15.5:0/1084501846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fb4eaeb40 0x555fb4eb95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.285+0000 7f9a92800640 1 -- 172.21.15.5:0/1084501846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fb4eaf460 msgr2=0x555fb4eb6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:23.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.285+0000 7f9a92800640 1 --2- 172.21.15.5:0/1084501846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fb4eaf460 0x555fb4eb6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.285+0000 7f9a92800640 1 -- 172.21.15.5:0/1084501846 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fb4dde3f0 con 0x555fb4eb9b30 2024-04-24T06:46:23.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.285+0000 7f9a91e00640 1 --2- 172.21.15.5:0/1084501846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fb4eaeb40 0x555fb4eb95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:46:23.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.285+0000 7f9a92800640 1 --2- 172.21.15.5:0/1084501846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fb4eb9b30 0x555fb4eb6fd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c005e30 tx=0x7f9a8c02a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a451c1a0d5b9cfa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:23.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.285+0000 7f9a73e00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:46:23.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.285+0000 7f9a73e00640 1 -- 172.21.15.5:0/1084501846 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c031720 con 0x555fb4eb9b30 2024-04-24T06:46:23.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.285+0000 7f9a73e00640 1 -- 172.21.15.5:0/1084501846 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f9a8c0318c0 con 0x555fb4eb9b30 2024-04-24T06:46:23.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.286+0000 7f9a9bb89740 1 -- 172.21.15.5:0/1084501846 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a8400fb20 con 0x555fb4eb9b30 2024-04-24T06:46:23.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.286+0000 7f9a73e00640 1 -- 172.21.15.5:0/1084501846 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9a8c004040 con 0x555fb4eb9b30 2024-04-24T06:46:23.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.286+0000 7f9a9bb89740 10 client.25870 fetch_fsmap learned FSMap version 25 2024-04-24T06:46:23.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.286+0000 7f9a9bb89740 1 -- 172.21.15.5:0/1084501846 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555fb4b263f0 con 0x555fb4eb9b30 2024-04-24T06:46:23.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.286+0000 7f9a73e00640 1 -- 172.21.15.5:0/1084501846 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a8400fb20 con 0x555fb4eb9b30 2024-04-24T06:46:23.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.286+0000 7f9a73e00640 1 -- 172.21.15.5:0/1084501846 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9a8c051020 con 0x555fb4eb9b30 2024-04-24T06:46:23.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.286+0000 7f9a9bb89740 10 client.25870 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:46:23.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.286+0000 7f9a9bb89740 10 client.25870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:46:23.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.286+0000 7f9a9bb89740 20 client.25870 populate_metadata read hostname 'smithi005' 2024-04-24T06:46:23.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.287+0000 7f9a9bb89740 1 --2- 172.21.15.5:0/1084501846 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x555fb4ec9cc0 0x555fb4ecc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:23.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.287+0000 7f9a9bb89740 4 client.25870 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:46:23.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.287+0000 7f9a9bb89740 1 -- 172.21.15.5:0/1084501846 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fb4e21bf0 con 0x555fb4ec9cc0 2024-04-24T06:46:23.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.287+0000 7f9a91e00640 1 --2- 172.21.15.5:0/1084501846 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x555fb4ec9cc0 0x555fb4ecc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:23.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.287+0000 7f9a91e00640 1 --2- 172.21.15.5:0/1084501846 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x555fb4ec9cc0 0x555fb4ecc100 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d78ef17a7502de3 server_cookie=e623fbc670411915 in_seq=0 out_seq=0 2024-04-24T06:46:23.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.287+0000 7f9a73e00640 10 client.25870 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:46:23.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.289+0000 7f9a73e00640 1 -- 172.21.15.5:0/1084501846 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555fb4e21bf0 con 0x555fb4ec9cc0 2024-04-24T06:46:23.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.289+0000 7f9a73e00640 10 client.25870 handle_command_reply: tid=0 2024-04-24T06:46:23.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.289+0000 7f9a9bb89740 1 client.25870 shutdown 2024-04-24T06:46:23.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.289+0000 7f9a9bb89740 1 -- 172.21.15.5:0/1084501846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fb4eb9b30 msgr2=0x555fb4eb6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:23.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.289+0000 7f9a9bb89740 1 --2- 172.21.15.5:0/1084501846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fb4eb9b30 0x555fb4eb6fd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c005e30 tx=0x7f9a8c02a8c0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.289+0000 7f9a9bb89740 1 -- 172.21.15.5:0/1084501846 shutdown_connections 2024-04-24T06:46:23.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.289+0000 7f9a9bb89740 1 --2- 172.21.15.5:0/1084501846 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x555fb4ec9cc0 0x555fb4ecc100 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.289+0000 7f9a9bb89740 1 --2- 172.21.15.5:0/1084501846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fb4eb9b30 0x555fb4eb6fd0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.289+0000 7f9a9bb89740 1 --2- 172.21.15.5:0/1084501846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fb4eaf460 0x555fb4eb6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.289+0000 7f9a9bb89740 1 --2- 172.21.15.5:0/1084501846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fb4eaeb40 0x555fb4eb95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.289+0000 7f9a9bb89740 1 -- 172.21.15.5:0/1084501846 >> 172.21.15.5:0/1084501846 conn(0x555fb4eaae60 msgr2=0x555fb4eb0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:23.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.289+0000 7f9a73e00640 0 client.25870 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:46:23.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.289+0000 7f9a9bb89740 1 -- 172.21.15.5:0/1084501846 shutdown_connections 2024-04-24T06:46:23.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.289+0000 7f9a9bb89740 1 -- 172.21.15.5:0/1084501846 wait complete. 2024-04-24T06:46:23.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.289+0000 7f9a9bb89740 20 client.25870 trim_cache size 0 max 16384 2024-04-24T06:46:23.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.309+0000 7f9a9bb89740 1 Processor -- start 2024-04-24T06:46:23.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.310+0000 7f9a9bb89740 1 -- start start 2024-04-24T06:46:23.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.310+0000 7f9a9bb89740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fb4eaeb40 0x555fb4ec58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:23.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.310+0000 7f9a9bb89740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fb4eaf460 0x555fb4eb5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:23.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.310+0000 7f9a9bb89740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fb4eb9b30 0x555fb4eb90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:23.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.310+0000 7f9a9bb89740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9a941b2660 con 0x555fb4eaf460 2024-04-24T06:46:23.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.310+0000 7f9a9bb89740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9a941aff30 con 0x555fb4eb9b30 2024-04-24T06:46:23.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.310+0000 7f9a9bb89740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9a941adda0 con 0x555fb4eaeb40 2024-04-24T06:46:23.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.310+0000 7f9a91e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fb4eaeb40 0x555fb4ec58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:23.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.311+0000 7f9a91e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fb4eaeb40 0x555fb4ec58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:42898/0 (socket says 172.21.15.5:42898) 2024-04-24T06:46:23.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.311+0000 7f9a91e00640 1 -- 172.21.15.5:0/406870410 learned_addr learned my addr 172.21.15.5:0/406870410 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:23.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.311+0000 7f9a92800640 1 --2- 172.21.15.5:0/406870410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fb4eb9b30 0x555fb4eb90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:23.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.311+0000 7f9a91400640 1 --2- 172.21.15.5:0/406870410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fb4eaf460 0x555fb4eb5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:23.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.311+0000 7f9a91e00640 1 -- 172.21.15.5:0/406870410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fb4eb9b30 msgr2=0x555fb4eb90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:23.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.311+0000 7f9a91e00640 1 --2- 172.21.15.5:0/406870410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fb4eb9b30 0x555fb4eb90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.311+0000 7f9a91e00640 1 -- 172.21.15.5:0/406870410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fb4eaf460 msgr2=0x555fb4eb5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:23.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.311+0000 7f9a91e00640 1 --2- 172.21.15.5:0/406870410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fb4eaf460 0x555fb4eb5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.311+0000 7f9a91e00640 1 -- 172.21.15.5:0/406870410 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fb4bf07d0 con 0x555fb4eaeb40 2024-04-24T06:46:23.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.311+0000 7f9a92800640 1 --2- 172.21.15.5:0/406870410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fb4eb9b30 0x555fb4eb90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:46:23.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.312+0000 7f9a91e00640 1 --2- 172.21.15.5:0/406870410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fb4eaeb40 0x555fb4ec58b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9a84019920 tx=0x7f9a84019820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d476f9804c1699f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:23.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.312+0000 7f9a7ea00640 1 -- 172.21.15.5:0/406870410 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8401e040 con 0x555fb4eaeb40 2024-04-24T06:46:23.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.312+0000 7f9a7ea00640 1 -- 172.21.15.5:0/406870410 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a8402a070 con 0x555fb4eaeb40 2024-04-24T06:46:23.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.312+0000 7f9a7ea00640 1 -- 172.21.15.5:0/406870410 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a840052a0 con 0x555fb4eaeb40 2024-04-24T06:46:23.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.313+0000 7f9a9bb89740 1 -- 172.21.15.5:0/406870410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fb4eaeb40 msgr2=0x555fb4ec58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:23.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.313+0000 7f9a9bb89740 1 --2- 172.21.15.5:0/406870410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fb4eaeb40 0x555fb4ec58b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9a84019920 tx=0x7f9a84019820 comp rx=0 tx=0).stop 2024-04-24T06:46:23.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.313+0000 7f9a9bb89740 1 -- 172.21.15.5:0/406870410 shutdown_connections 2024-04-24T06:46:23.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.313+0000 7f9a9bb89740 1 --2- 172.21.15.5:0/406870410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fb4eb9b30 0x555fb4eb90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.313+0000 7f9a9bb89740 1 --2- 172.21.15.5:0/406870410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fb4eaf460 0x555fb4eb5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.313+0000 7f9a9bb89740 1 --2- 172.21.15.5:0/406870410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fb4eaeb40 0x555fb4ec58b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.313+0000 7f9a9bb89740 1 -- 172.21.15.5:0/406870410 >> 172.21.15.5:0/406870410 conn(0x555fb4eaae60 msgr2=0x555fb4eb5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:23.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.313+0000 7f9a9bb89740 1 -- 172.21.15.5:0/406870410 shutdown_connections 2024-04-24T06:46:23.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.313+0000 7f9a9bb89740 1 -- 172.21.15.5:0/406870410 wait complete. 2024-04-24T06:46:23.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.313+0000 7f9a9bb89740 1 -- start start 2024-04-24T06:46:23.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.314+0000 7f9a9bb89740 1 Processor -- start 2024-04-24T06:46:23.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.314+0000 7f9a9bb89740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fb4eaeb40 0x555fb4eb9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:23.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.314+0000 7f9a9bb89740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fb4eaf460 0x555fb4eba4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:23.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.314+0000 7f9a9bb89740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fb4eb9b30 0x555fb4ebaa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:23.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.314+0000 7f9a9bb89740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9a941ae3f0 con 0x555fb4eb9b30 2024-04-24T06:46:23.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.314+0000 7f9a9bb89740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9a941af860 con 0x555fb4eaeb40 2024-04-24T06:46:23.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.314+0000 7f9a9bb89740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9a941aeac0 con 0x555fb4eaf460 2024-04-24T06:46:23.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.314+0000 7f9a9bb89740 10 client.0 _ll_register_callbacks cb 0x7f9a8c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:46:23.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.314+0000 7f9a91e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fb4eaeb40 0x555fb4eb9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:23.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.314+0000 7f9a91400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fb4eaf460 0x555fb4eba4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:23.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.314+0000 7f9a91e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fb4eaeb40 0x555fb4eb9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:37218/0 (socket says 172.21.15.5:37218) 2024-04-24T06:46:23.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.314+0000 7f9a91e00640 1 -- 172.21.15.5:0/2413456395 learned_addr learned my addr 172.21.15.5:0/2413456395 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:23.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.315+0000 7f9a92800640 1 --2- 172.21.15.5:0/2413456395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fb4eb9b30 0x555fb4ebaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:23.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.315+0000 7f9a91e00640 1 -- 172.21.15.5:0/2413456395 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fb4eaf460 msgr2=0x555fb4eba4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:23.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.315+0000 7f9a91e00640 1 --2- 172.21.15.5:0/2413456395 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fb4eaf460 0x555fb4eba4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.315+0000 7f9a91e00640 1 -- 172.21.15.5:0/2413456395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fb4eb9b30 msgr2=0x555fb4ebaa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:23.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.315+0000 7f9a91e00640 1 --2- 172.21.15.5:0/2413456395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fb4eb9b30 0x555fb4ebaa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.315+0000 7f9a91e00640 1 -- 172.21.15.5:0/2413456395 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fb4ec1030 con 0x555fb4eaeb40 2024-04-24T06:46:23.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.315+0000 7f9a91e00640 1 --2- 172.21.15.5:0/2413456395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fb4eaeb40 0x555fb4eb9fb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9a840198f0 tx=0x7f9a84011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86d33ded91c2c4fe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:23.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.315+0000 7f9a73e00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:46:23.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.316+0000 7f9a73e00640 1 -- 172.21.15.5:0/2413456395 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8401e050 con 0x555fb4eaeb40 2024-04-24T06:46:23.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.316+0000 7f9a9bb89740 1 -- 172.21.15.5:0/2413456395 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a8402a070 con 0x555fb4eaeb40 2024-04-24T06:46:23.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.316+0000 7f9a73e00640 1 -- 172.21.15.5:0/2413456395 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f9a84019950 con 0x555fb4eaeb40 2024-04-24T06:46:23.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.316+0000 7f9a73e00640 1 -- 172.21.15.5:0/2413456395 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9a84019b90 con 0x555fb4eaeb40 2024-04-24T06:46:23.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.316+0000 7f9a9bb89740 10 client.25844 fetch_fsmap learned FSMap version 25 2024-04-24T06:46:23.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.316+0000 7f9a9bb89740 1 -- 172.21.15.5:0/2413456395 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555fb4ec22d0 con 0x555fb4eaeb40 2024-04-24T06:46:23.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.316+0000 7f9a73e00640 1 -- 172.21.15.5:0/2413456395 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a8402a070 con 0x555fb4eaeb40 2024-04-24T06:46:23.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.317+0000 7f9a73e00640 1 -- 172.21.15.5:0/2413456395 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9a84031020 con 0x555fb4eaeb40 2024-04-24T06:46:23.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.317+0000 7f9a9bb89740 10 client.25844 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:46:23.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.317+0000 7f9a9bb89740 10 client.25844 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:46:23.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.317+0000 7f9a9bb89740 20 client.25844 populate_metadata read hostname 'smithi005' 2024-04-24T06:46:23.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.317+0000 7f9a9bb89740 1 --2- 172.21.15.5:0/2413456395 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x555fb4f30290 0x555fb4f32680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:23.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.317+0000 7f9a9bb89740 4 client.25844 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:46:23.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.317+0000 7f9a9bb89740 1 -- 172.21.15.5:0/2413456395 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555fb4e1ba60 con 0x555fb4f30290 2024-04-24T06:46:23.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.317+0000 7f9a91400640 1 --2- 172.21.15.5:0/2413456395 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x555fb4f30290 0x555fb4f32680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:23.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.317+0000 7f9a91400640 1 --2- 172.21.15.5:0/2413456395 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x555fb4f30290 0x555fb4f32680 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85edf3f57aa17a63 server_cookie=ee66baf77b6278b8 in_seq=0 out_seq=0 2024-04-24T06:46:23.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.317+0000 7f9a73e00640 10 client.25844 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:46:23.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.319+0000 7f9a73e00640 1 -- 172.21.15.5:0/2413456395 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555fb4e1ba60 con 0x555fb4f30290 2024-04-24T06:46:23.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.319+0000 7f9a73e00640 10 client.25844 handle_command_reply: tid=0 2024-04-24T06:46:23.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.319+0000 7f9a9bb89740 1 client.25844 shutdown 2024-04-24T06:46:23.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.319+0000 7f9a9bb89740 1 -- 172.21.15.5:0/2413456395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fb4eaeb40 msgr2=0x555fb4eb9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:23.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.319+0000 7f9a9bb89740 1 --2- 172.21.15.5:0/2413456395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fb4eaeb40 0x555fb4eb9fb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9a840198f0 tx=0x7f9a84011ab0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.319+0000 7f9a9bb89740 1 -- 172.21.15.5:0/2413456395 shutdown_connections 2024-04-24T06:46:23.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.319+0000 7f9a9bb89740 1 --2- 172.21.15.5:0/2413456395 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x555fb4f30290 0x555fb4f32680 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.319+0000 7f9a9bb89740 1 --2- 172.21.15.5:0/2413456395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fb4eb9b30 0x555fb4ebaa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.319+0000 7f9a9bb89740 1 --2- 172.21.15.5:0/2413456395 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555fb4eaf460 0x555fb4eba4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.319+0000 7f9a73e00640 0 client.25844 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:46:23.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.319+0000 7f9a9bb89740 1 --2- 172.21.15.5:0/2413456395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fb4eaeb40 0x555fb4eb9fb0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.319+0000 7f9a9bb89740 1 -- 172.21.15.5:0/2413456395 >> 172.21.15.5:0/2413456395 conn(0x555fb4eaae60 msgr2=0x555fb4eb58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:23.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.319+0000 7f9a9bb89740 1 -- 172.21.15.5:0/2413456395 shutdown_connections 2024-04-24T06:46:23.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.319+0000 7f9a9bb89740 1 -- 172.21.15.5:0/2413456395 wait complete. 2024-04-24T06:46:23.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.319+0000 7f9a9bb89740 20 client.25844 trim_cache size 0 max 16384 2024-04-24T06:46:23.322 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:46:23.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.320+0000 7f9a93200640 1 -- 172.21.15.5:0/589031779 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f9a780a3c30 msgr2=0x7f9a780a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:23.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.320+0000 7f9a93200640 1 --2- 172.21.15.5:0/589031779 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f9a780a3c30 0x7f9a780a60f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c006010 tx=0x7f9a8c005d80 comp rx=0 tx=0).stop 2024-04-24T06:46:23.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.320+0000 7f9a93200640 1 -- 172.21.15.5:0/589031779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a94109360 msgr2=0x7f9a941a0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:23.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.320+0000 7f9a93200640 1 --2- 172.21.15.5:0/589031779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a94109360 0x7f9a941a0bf0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9a8000e9f0 tx=0x7f9a8000eeb0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.320+0000 7f9a93200640 1 -- 172.21.15.5:0/589031779 shutdown_connections 2024-04-24T06:46:23.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.320+0000 7f9a93200640 1 --2- 172.21.15.5:0/589031779 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f9a780a3c30 0x7f9a780a60f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.320+0000 7f9a93200640 1 --2- 172.21.15.5:0/589031779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a9419a770 0x7f9a9419abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.320+0000 7f9a93200640 1 --2- 172.21.15.5:0/589031779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a94109360 0x7f9a941a0bf0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.320+0000 7f9a93200640 1 --2- 172.21.15.5:0/589031779 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a94104290 0x7f9a941a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:23.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.320+0000 7f9a93200640 1 -- 172.21.15.5:0/589031779 >> 172.21.15.5:0/589031779 conn(0x7f9a940ffaf0 msgr2=0x7f9a94100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:23.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.320+0000 7f9a93200640 1 -- 172.21.15.5:0/589031779 shutdown_connections 2024-04-24T06:46:23.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:23.320+0000 7f9a93200640 1 -- 172.21.15.5:0/589031779 wait complete. 2024-04-24T06:46:23.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:23 smithi005 ceph-mon[27798]: pgmap v73: 97 pgs: 97 active+clean; 37 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 26 op/s 2024-04-24T06:46:23.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:23 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/3926400489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:46:23.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:23 smithi037 ceph-mon[32864]: pgmap v73: 97 pgs: 97 active+clean; 37 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 26 op/s 2024-04-24T06:46:23.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:23 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/3926400489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:46:23.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:23 smithi137 ceph-mon[33017]: pgmap v73: 97 pgs: 97 active+clean; 37 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 26 op/s 2024-04-24T06:46:23.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:23 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/3926400489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:46:23.835 INFO:tasks.workunit.client.0.smithi005.stdout:libtoolize: putting auxiliary files in '.'. 2024-04-24T06:46:23.835 INFO:tasks.workunit.client.0.smithi005.stdout:libtoolize: copying file './config.guess' 2024-04-24T06:46:23.869 INFO:tasks.workunit.client.0.smithi005.stdout:libtoolize: copying file './config.sub' 2024-04-24T06:46:23.887 INFO:tasks.workunit.client.0.smithi005.stdout:libtoolize: copying file './install-sh' 2024-04-24T06:46:23.908 INFO:tasks.workunit.client.0.smithi005.stdout:libtoolize: copying file './ltmain.sh' 2024-04-24T06:46:24.006 INFO:tasks.workunit.client.0.smithi005.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-04-24T06:46:24.006 INFO:tasks.workunit.client.0.smithi005.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-04-24T06:46:24.027 INFO:tasks.workunit.client.0.smithi005.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-04-24T06:46:24.048 INFO:tasks.workunit.client.0.smithi005.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-04-24T06:46:24.073 INFO:tasks.workunit.client.0.smithi005.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-04-24T06:46:24.098 INFO:tasks.workunit.client.0.smithi005.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-04-24T06:46:24.126 INFO:tasks.workunit.client.0.smithi005.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-04-24T06:46:24.131 INFO:tasks.workunit.client.0.smithi005.stdout:cp include/install-sh . 2024-04-24T06:46:24.261 INFO:tasks.workunit.client.0.smithi005.stdout:aclocal -I m4 2024-04-24T06:46:24.339 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T06:46:24.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:46:24.339 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T06:46:24.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.548+0000 7f71b1c00640 1 -- 172.21.15.5:0/3852627182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71ac102a20 msgr2=0x7f71ac106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:24.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.548+0000 7f71b1c00640 1 --2- 172.21.15.5:0/3852627182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71ac102a20 0x7f71ac106a60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f719c009ca0 tx=0x7f719c02a8c0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.548+0000 7f71b1c00640 1 -- 172.21.15.5:0/3852627182 shutdown_connections 2024-04-24T06:46:24.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.548+0000 7f71b1c00640 1 --2- 172.21.15.5:0/3852627182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71ac107140 0x7f71ac10ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.548+0000 7f71b1c00640 1 --2- 172.21.15.5:0/3852627182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71ac102a20 0x7f71ac106a60 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.548+0000 7f71b1c00640 1 --2- 172.21.15.5:0/3852627182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71ac102070 0x7f71ac102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.548+0000 7f71b1c00640 1 -- 172.21.15.5:0/3852627182 >> 172.21.15.5:0/3852627182 conn(0x7f71ac0fd950 msgr2=0x7f71ac0ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:24.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.548+0000 7f71b1c00640 1 -- 172.21.15.5:0/3852627182 shutdown_connections 2024-04-24T06:46:24.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.548+0000 7f71b1c00640 1 -- 172.21.15.5:0/3852627182 wait complete. 2024-04-24T06:46:24.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.550+0000 7f71b1c00640 1 Processor -- start 2024-04-24T06:46:24.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.550+0000 7f71b1c00640 1 -- start start 2024-04-24T06:46:24.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.550+0000 7f71b1c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71ac102070 0x7f71ac198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:24.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.550+0000 7f71b1c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71ac107140 0x7f71ac198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:24.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.550+0000 7f71b1c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71ac19cff0 0x7f71ac19d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:24.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.550+0000 7f71b1c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac112200 con 0x7f71ac102070 2024-04-24T06:46:24.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.550+0000 7f71b1c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac112080 con 0x7f71ac19cff0 2024-04-24T06:46:24.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.550+0000 7f71b1c00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac112380 con 0x7f71ac107140 2024-04-24T06:46:24.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.551+0000 7f71ab400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71ac107140 0x7f71ac198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:24.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.551+0000 7f71abe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71ac102070 0x7f71ac198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:24.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.551+0000 7f71ab400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71ac107140 0x7f71ac198cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:42248/0 (socket says 172.21.15.5:42248) 2024-04-24T06:46:24.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.551+0000 7f71ab400640 1 -- 172.21.15.5:0/3904926477 learned_addr learned my addr 172.21.15.5:0/3904926477 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:24.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.551+0000 7f71b1200640 1 --2- 172.21.15.5:0/3904926477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71ac19cff0 0x7f71ac19d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:24.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.551+0000 7f71ab400640 1 -- 172.21.15.5:0/3904926477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71ac19cff0 msgr2=0x7f71ac19d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:24.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.551+0000 7f71ab400640 1 --2- 172.21.15.5:0/3904926477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71ac19cff0 0x7f71ac19d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.551+0000 7f71ab400640 1 -- 172.21.15.5:0/3904926477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71ac102070 msgr2=0x7f71ac198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:24.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.551+0000 7f71ab400640 1 --2- 172.21.15.5:0/3904926477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71ac102070 0x7f71ac198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.551+0000 7f71ab400640 1 -- 172.21.15.5:0/3904926477 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71ac19dae0 con 0x7f71ac107140 2024-04-24T06:46:24.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.551+0000 7f71b1200640 1 --2- 172.21.15.5:0/3904926477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71ac19cff0 0x7f71ac19d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:46:24.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.551+0000 7f71ab400640 1 --2- 172.21.15.5:0/3904926477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71ac107140 0x7f71ac198cc0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f719c000f80 tx=0x7f719c003f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:24.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.552+0000 7f71a8c00640 1 -- 172.21.15.5:0/3904926477 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719c031650 con 0x7f71ac107140 2024-04-24T06:46:24.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.552+0000 7f71a8c00640 1 -- 172.21.15.5:0/3904926477 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f719c0317f0 con 0x7f71ac107140 2024-04-24T06:46:24.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.552+0000 7f71a8c00640 1 -- 172.21.15.5:0/3904926477 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719c02f980 con 0x7f71ac107140 2024-04-24T06:46:24.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.552+0000 7f71b1c00640 1 -- 172.21.15.5:0/3904926477 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71ac19dd70 con 0x7f71ac107140 2024-04-24T06:46:24.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.552+0000 7f71b1c00640 1 -- 172.21.15.5:0/3904926477 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71ac1a5590 con 0x7f71ac107140 2024-04-24T06:46:24.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.555+0000 7f71b61e3740 1 Processor -- start 2024-04-24T06:46:24.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.555+0000 7f71b61e3740 1 -- start start 2024-04-24T06:46:24.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.555+0000 7f71a8c00640 1 -- 172.21.15.5:0/3904926477 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f719c02fb60 con 0x7f71ac107140 2024-04-24T06:46:24.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.556+0000 7f71b61e3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f3c11270 0x5590f3c11650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:24.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.556+0000 7f71b61e3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590f3c11b90 0x5590f3c1bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:24.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.556+0000 7f71b61e3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f3c1c260 0x5590f3c1e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:24.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.556+0000 7f71b61e3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5590f3b76e70 con 0x5590f3c1c260 2024-04-24T06:46:24.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.556+0000 7f71b61e3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5590f3b1d380 con 0x5590f3c11270 2024-04-24T06:46:24.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.556+0000 7f71b61e3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5590f3b72e30 con 0x5590f3c11b90 2024-04-24T06:46:24.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.556+0000 7f71b1200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f3c11270 0x5590f3c11650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:24.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.556+0000 7f71abe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590f3c11b90 0x5590f3c1bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:24.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.556+0000 7f71b1200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f3c11270 0x5590f3c11650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:40468/0 (socket says 172.21.15.5:40468) 2024-04-24T06:46:24.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.556+0000 7f71abe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590f3c11b90 0x5590f3c1bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:42250/0 (socket says 172.21.15.5:42250) 2024-04-24T06:46:24.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.556+0000 7f71abe00640 1 -- 172.21.15.5:0/1233898389 learned_addr learned my addr 172.21.15.5:0/1233898389 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:24.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.556+0000 7f71ab400640 1 --2- 172.21.15.5:0/1233898389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f3c1c260 0x5590f3c1e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:24.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.556+0000 7f71b1200640 1 -- 172.21.15.5:0/1233898389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590f3c11b90 msgr2=0x5590f3c1bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:24.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.556+0000 7f71b1200640 1 --2- 172.21.15.5:0/1233898389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590f3c11b90 0x5590f3c1bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.556+0000 7f71b1200640 1 -- 172.21.15.5:0/1233898389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f3c1c260 msgr2=0x5590f3c1e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:24.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.556+0000 7f71b1200640 1 --2- 172.21.15.5:0/1233898389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f3c1c260 0x5590f3c1e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.556+0000 7f71b1200640 1 -- 172.21.15.5:0/1233898389 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590f39517e0 con 0x5590f3c11270 2024-04-24T06:46:24.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.557+0000 7f71b1200640 1 --2- 172.21.15.5:0/1233898389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f3c11270 0x5590f3c11650 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f71a4009f40 tx=0x7f71a40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=479f256824265820 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:24.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.557+0000 7f71a1600640 1 -- 172.21.15.5:0/1233898389 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a4005020 con 0x5590f3c11270 2024-04-24T06:46:24.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.557+0000 7f71a1600640 1 -- 172.21.15.5:0/1233898389 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71a40047a0 con 0x5590f3c11270 2024-04-24T06:46:24.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.557+0000 7f71a1600640 1 -- 172.21.15.5:0/1233898389 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a400f8c0 con 0x5590f3c11270 2024-04-24T06:46:24.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.557+0000 7f71a8c00640 1 --2- 172.21.15.5:0/3904926477 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f71900a8130 0x7f71900aa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:24.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.557+0000 7f71a8c00640 1 -- 172.21.15.5:0/3904926477 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f719c0f1ad0 con 0x7f71ac107140 2024-04-24T06:46:24.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.558+0000 7f71abe00640 1 --2- 172.21.15.5:0/3904926477 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f71900a8130 0x7f71900aa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:24.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.558+0000 7f71b61e3740 1 -- 172.21.15.5:0/1233898389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f3c11270 msgr2=0x5590f3c11650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:24.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.558+0000 7f71b61e3740 1 --2- 172.21.15.5:0/1233898389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f3c11270 0x5590f3c11650 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f71a4009f40 tx=0x7f71a40118d0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.558+0000 7f71b61e3740 1 -- 172.21.15.5:0/1233898389 shutdown_connections 2024-04-24T06:46:24.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.558+0000 7f71b61e3740 1 --2- 172.21.15.5:0/1233898389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f3c1c260 0x5590f3c1e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.558+0000 7f71b61e3740 1 --2- 172.21.15.5:0/1233898389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590f3c11b90 0x5590f3c1bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.558+0000 7f71b61e3740 1 --2- 172.21.15.5:0/1233898389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f3c11270 0x5590f3c11650 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.558+0000 7f71b61e3740 1 -- 172.21.15.5:0/1233898389 >> 172.21.15.5:0/1233898389 conn(0x5590f3c0d590 msgr2=0x5590f3c0d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:24.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.558+0000 7f71b61e3740 1 -- 172.21.15.5:0/1233898389 shutdown_connections 2024-04-24T06:46:24.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.558+0000 7f71b61e3740 1 -- 172.21.15.5:0/1233898389 wait complete. 2024-04-24T06:46:24.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.558+0000 7f71abe00640 1 --2- 172.21.15.5:0/3904926477 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f71900a8130 0x7f71900aa5f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x5590f3c12830 tx=0x7f7194005f90 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:24.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.559+0000 7f71b61e3740 1 -- start start 2024-04-24T06:46:24.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.559+0000 7f71b61e3740 1 Processor -- start 2024-04-24T06:46:24.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.559+0000 7f71b61e3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f3c11270 0x5590f3c19200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:24.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.560+0000 7f71b61e3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590f3c11b90 0x5590f3c19740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:24.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.560+0000 7f71b61e3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f3c1c260 0x5590f3c19c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:24.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.560+0000 7f71b61e3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5590f3c1f050 con 0x5590f3c1c260 2024-04-24T06:46:24.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.560+0000 7f71b61e3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5590f3b30660 con 0x5590f3c11270 2024-04-24T06:46:24.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.560+0000 7f71b61e3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5590f3c1f1d0 con 0x5590f3c11b90 2024-04-24T06:46:24.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.560+0000 7f71b61e3740 10 client.0 _ll_register_callbacks cb 0x5590f3b1eaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:46:24.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.560+0000 7f71abe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f3c11270 0x5590f3c19200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:24.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.560+0000 7f71ab400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590f3c11b90 0x5590f3c19740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:24.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.560+0000 7f71abe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f3c11270 0x5590f3c19200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:40480/0 (socket says 172.21.15.5:40480) 2024-04-24T06:46:24.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.560+0000 7f71abe00640 1 -- 172.21.15.5:0/2047428645 learned_addr learned my addr 172.21.15.5:0/2047428645 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:24.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.560+0000 7f71b1200640 1 --2- 172.21.15.5:0/2047428645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f3c1c260 0x5590f3c19c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:24.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.560+0000 7f71abe00640 1 -- 172.21.15.5:0/2047428645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590f3c11b90 msgr2=0x5590f3c19740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:24.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.560+0000 7f71abe00640 1 --2- 172.21.15.5:0/2047428645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590f3c11b90 0x5590f3c19740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.560+0000 7f71abe00640 1 -- 172.21.15.5:0/2047428645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f3c1c260 msgr2=0x5590f3c19c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:24.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.560+0000 7f71abe00640 1 --2- 172.21.15.5:0/2047428645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f3c1c260 0x5590f3c19c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.560+0000 7f71abe00640 1 -- 172.21.15.5:0/2047428645 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590f39b1f70 con 0x5590f3c11270 2024-04-24T06:46:24.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.561+0000 7f71abe00640 1 --2- 172.21.15.5:0/2047428645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f3c11270 0x5590f3c19200 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f719401ea00 tx=0x7f7194004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=149996e80c9eadd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:24.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.561+0000 7f719a000640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:46:24.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.561+0000 7f719a000640 1 -- 172.21.15.5:0/2047428645 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719402e060 con 0x5590f3c11270 2024-04-24T06:46:24.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.561+0000 7f719a000640 1 -- 172.21.15.5:0/2047428645 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f7194018030 con 0x5590f3c11270 2024-04-24T06:46:24.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.561+0000 7f71b61e3740 1 -- 172.21.15.5:0/2047428645 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71a40047a0 con 0x5590f3c11270 2024-04-24T06:46:24.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.561+0000 7f719a000640 1 -- 172.21.15.5:0/2047428645 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7194029420 con 0x5590f3c11270 2024-04-24T06:46:24.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.561+0000 7f71b61e3740 10 client.25856 fetch_fsmap learned FSMap version 25 2024-04-24T06:46:24.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.561+0000 7f71b61e3740 1 -- 172.21.15.5:0/2047428645 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5590f3849170 con 0x5590f3c11270 2024-04-24T06:46:24.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.561+0000 7f719a000640 1 -- 172.21.15.5:0/2047428645 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71a40047a0 con 0x5590f3c11270 2024-04-24T06:46:24.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.562+0000 7f719a000640 1 -- 172.21.15.5:0/2047428645 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f719401c6a0 con 0x5590f3c11270 2024-04-24T06:46:24.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.562+0000 7f71b61e3740 10 client.25856 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:46:24.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.562+0000 7f71b61e3740 10 client.25856 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:46:24.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.562+0000 7f71b61e3740 20 client.25856 populate_metadata read hostname 'smithi005' 2024-04-24T06:46:24.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.562+0000 7f71b61e3740 1 --2- 172.21.15.5:0/2047428645 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5590f3c2c700 0x5590f3c2eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:24.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.562+0000 7f71b61e3740 4 client.25856 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:46:24.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.562+0000 7f71b61e3740 1 -- 172.21.15.5:0/2047428645 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590f3b84890 con 0x5590f3c2c700 2024-04-24T06:46:24.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.562+0000 7f71ab400640 1 --2- 172.21.15.5:0/2047428645 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5590f3c2c700 0x5590f3c2eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:24.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.563+0000 7f71ab400640 1 --2- 172.21.15.5:0/2047428645 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5590f3c2c700 0x5590f3c2eb80 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5725618b51f19a9 server_cookie=bd45fbe886259a0b in_seq=0 out_seq=0 2024-04-24T06:46:24.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.563+0000 7f719a000640 10 client.25856 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:46:24.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.563+0000 7f719a000640 1 -- 172.21.15.5:0/2047428645 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5590f3b84890 con 0x5590f3c2c700 2024-04-24T06:46:24.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.563+0000 7f719a000640 10 client.25856 handle_command_reply: tid=0 2024-04-24T06:46:24.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.563+0000 7f71b61e3740 1 client.25856 shutdown 2024-04-24T06:46:24.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.563+0000 7f71b61e3740 1 -- 172.21.15.5:0/2047428645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f3c11270 msgr2=0x5590f3c19200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:24.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.564+0000 7f71b61e3740 1 --2- 172.21.15.5:0/2047428645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f3c11270 0x5590f3c19200 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f719401ea00 tx=0x7f7194004730 comp rx=0 tx=0).stop 2024-04-24T06:46:24.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.564+0000 7f71b61e3740 1 -- 172.21.15.5:0/2047428645 shutdown_connections 2024-04-24T06:46:24.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.564+0000 7f71b61e3740 1 --2- 172.21.15.5:0/2047428645 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5590f3c2c700 0x5590f3c2eb80 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.564+0000 7f71b61e3740 1 --2- 172.21.15.5:0/2047428645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f3c1c260 0x5590f3c19c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.564+0000 7f71b61e3740 1 --2- 172.21.15.5:0/2047428645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590f3c11b90 0x5590f3c19740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.564+0000 7f71b61e3740 1 --2- 172.21.15.5:0/2047428645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f3c11270 0x5590f3c19200 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.564+0000 7f719a000640 0 client.25856 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:46:24.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.564+0000 7f71b61e3740 1 -- 172.21.15.5:0/2047428645 >> 172.21.15.5:0/2047428645 conn(0x5590f3c0d590 msgr2=0x5590f3c13390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:24.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.564+0000 7f71b61e3740 1 -- 172.21.15.5:0/2047428645 shutdown_connections 2024-04-24T06:46:24.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.564+0000 7f71b61e3740 1 -- 172.21.15.5:0/2047428645 wait complete. 2024-04-24T06:46:24.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.564+0000 7f71b61e3740 20 client.25856 trim_cache size 0 max 16384 2024-04-24T06:46:24.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.584+0000 7f71b61e3740 1 Processor -- start 2024-04-24T06:46:24.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.585+0000 7f71b61e3740 1 -- start start 2024-04-24T06:46:24.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.585+0000 7f71b61e3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f3c11270 0x5590f3c27f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:24.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.585+0000 7f71b61e3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590f3c11b90 0x5590f3c139d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:24.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.585+0000 7f71b61e3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f3c1c260 0x5590f3c34450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:24.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.585+0000 7f71b61e3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac1ae280 con 0x5590f3c11270 2024-04-24T06:46:24.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.585+0000 7f71b61e3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac1abb50 con 0x5590f3c1c260 2024-04-24T06:46:24.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.585+0000 7f71b61e3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac1a99a0 con 0x5590f3c11b90 2024-04-24T06:46:24.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.585+0000 7f71b1200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f3c1c260 0x5590f3c34450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:24.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.585+0000 7f71b1200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f3c1c260 0x5590f3c34450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:40492/0 (socket says 172.21.15.5:40492) 2024-04-24T06:46:24.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.586+0000 7f71b1200640 1 -- 172.21.15.5:0/3050526478 learned_addr learned my addr 172.21.15.5:0/3050526478 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:24.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.586+0000 7f71ab400640 1 --2- 172.21.15.5:0/3050526478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590f3c11b90 0x5590f3c139d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:24.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.586+0000 7f71b1200640 1 -- 172.21.15.5:0/3050526478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590f3c11b90 msgr2=0x5590f3c139d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:24.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.586+0000 7f71b1200640 1 --2- 172.21.15.5:0/3050526478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590f3c11b90 0x5590f3c139d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.586+0000 7f71b1200640 1 -- 172.21.15.5:0/3050526478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f3c11270 msgr2=0x5590f3c27f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T06:46:24.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.586+0000 7f71abe00640 1 --2- 172.21.15.5:0/3050526478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f3c11270 0x5590f3c27f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:24.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.586+0000 7f71b1200640 1 --2- 172.21.15.5:0/3050526478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f3c11270 0x5590f3c27f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.586+0000 7f71b1200640 1 -- 172.21.15.5:0/3050526478 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590f3c1e650 con 0x5590f3c1c260 2024-04-24T06:46:24.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.586+0000 7f71abe00640 1 --2- 172.21.15.5:0/3050526478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f3c11270 0x5590f3c27f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:46:24.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.587+0000 7f71b1200640 1 --2- 172.21.15.5:0/3050526478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f3c1c260 0x5590f3c34450 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f71a4011dd0 tx=0x7f71a4005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6ca406df5724632 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:24.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.587+0000 7f71a0c00640 1 -- 172.21.15.5:0/3050526478 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a40051f0 con 0x5590f3c1c260 2024-04-24T06:46:24.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.587+0000 7f71a0c00640 1 -- 172.21.15.5:0/3050526478 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71a4005390 con 0x5590f3c1c260 2024-04-24T06:46:24.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.587+0000 7f71a0c00640 1 -- 172.21.15.5:0/3050526478 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a400f730 con 0x5590f3c1c260 2024-04-24T06:46:24.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.588+0000 7f71b61e3740 1 -- 172.21.15.5:0/3050526478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f3c1c260 msgr2=0x5590f3c34450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:24.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.588+0000 7f71b61e3740 1 --2- 172.21.15.5:0/3050526478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f3c1c260 0x5590f3c34450 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f71a4011dd0 tx=0x7f71a4005850 comp rx=0 tx=0).stop 2024-04-24T06:46:24.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.588+0000 7f71b61e3740 1 -- 172.21.15.5:0/3050526478 shutdown_connections 2024-04-24T06:46:24.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.588+0000 7f71b61e3740 1 --2- 172.21.15.5:0/3050526478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f3c1c260 0x5590f3c34450 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.588+0000 7f71b61e3740 1 --2- 172.21.15.5:0/3050526478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590f3c11b90 0x5590f3c139d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.588+0000 7f71b61e3740 1 --2- 172.21.15.5:0/3050526478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f3c11270 0x5590f3c27f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.588+0000 7f71b61e3740 1 -- 172.21.15.5:0/3050526478 >> 172.21.15.5:0/3050526478 conn(0x5590f3c0d590 msgr2=0x5590f3c0c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:24.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.588+0000 7f71b61e3740 1 -- 172.21.15.5:0/3050526478 shutdown_connections 2024-04-24T06:46:24.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.588+0000 7f71b61e3740 1 -- 172.21.15.5:0/3050526478 wait complete. 2024-04-24T06:46:24.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.588+0000 7f71b61e3740 1 -- start start 2024-04-24T06:46:24.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.589+0000 7f71b61e3740 1 Processor -- start 2024-04-24T06:46:24.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.589+0000 7f71b61e3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f3c11270 0x5590f3c15f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:24.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.589+0000 7f71b61e3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590f3c11b90 0x5590f3c164b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:24.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.589+0000 7f71b61e3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f3c1c260 0x5590f3c169f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:24.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.589+0000 7f71b61e3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac1aa010 con 0x5590f3c11270 2024-04-24T06:46:24.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.589+0000 7f71b61e3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac1ab480 con 0x5590f3c1c260 2024-04-24T06:46:24.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.589+0000 7f71b61e3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac1aa6e0 con 0x5590f3c11b90 2024-04-24T06:46:24.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.589+0000 7f71b61e3740 10 client.0 _ll_register_callbacks cb 0x7f7194012f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:46:24.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.589+0000 7f71b1200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f3c1c260 0x5590f3c169f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:24.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.589+0000 7f71abe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f3c11270 0x5590f3c15f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:24.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.589+0000 7f71b1200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f3c1c260 0x5590f3c169f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:40500/0 (socket says 172.21.15.5:40500) 2024-04-24T06:46:24.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.589+0000 7f71b1200640 1 -- 172.21.15.5:0/3451787365 learned_addr learned my addr 172.21.15.5:0/3451787365 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:24.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.590+0000 7f71ab400640 1 --2- 172.21.15.5:0/3451787365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590f3c11b90 0x5590f3c164b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:24.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.590+0000 7f71b1200640 1 -- 172.21.15.5:0/3451787365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590f3c11b90 msgr2=0x5590f3c164b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:24.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.590+0000 7f71b1200640 1 --2- 172.21.15.5:0/3451787365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590f3c11b90 0x5590f3c164b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.590+0000 7f71b1200640 1 -- 172.21.15.5:0/3451787365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f3c11270 msgr2=0x5590f3c15f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:24.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.590+0000 7f71b1200640 1 --2- 172.21.15.5:0/3451787365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f3c11270 0x5590f3c15f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.590+0000 7f71b1200640 1 -- 172.21.15.5:0/3451787365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590f3c91d70 con 0x5590f3c1c260 2024-04-24T06:46:24.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.590+0000 7f71abe00640 1 --2- 172.21.15.5:0/3451787365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f3c11270 0x5590f3c15f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:46:24.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.590+0000 7f71b1200640 1 --2- 172.21.15.5:0/3451787365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f3c1c260 0x5590f3c169f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f71a4005010 tx=0x7f71a4003e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b1d2b312389e2cf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:24.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.590+0000 7f719a000640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:46:24.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.590+0000 7f719a000640 1 -- 172.21.15.5:0/3451787365 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a4032020 con 0x5590f3c1c260 2024-04-24T06:46:24.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.591+0000 7f719a000640 1 -- 172.21.15.5:0/3451787365 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f71a400f6b0 con 0x5590f3c1c260 2024-04-24T06:46:24.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.591+0000 7f71b61e3740 1 -- 172.21.15.5:0/3451787365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71a4005390 con 0x5590f3c1c260 2024-04-24T06:46:24.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.591+0000 7f719a000640 1 -- 172.21.15.5:0/3451787365 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f71a403d050 con 0x5590f3c1c260 2024-04-24T06:46:24.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.591+0000 7f71b61e3740 10 client.25862 fetch_fsmap learned FSMap version 25 2024-04-24T06:46:24.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.591+0000 7f71b61e3740 1 -- 172.21.15.5:0/3451787365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5590f3c93000 con 0x5590f3c1c260 2024-04-24T06:46:24.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.591+0000 7f719a000640 1 -- 172.21.15.5:0/3451787365 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71a4005390 con 0x5590f3c1c260 2024-04-24T06:46:24.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.591+0000 7f719a000640 1 -- 172.21.15.5:0/3451787365 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f71a4026810 con 0x5590f3c1c260 2024-04-24T06:46:24.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.591+0000 7f71b61e3740 10 client.25862 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:46:24.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.591+0000 7f71b61e3740 10 client.25862 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:46:24.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.591+0000 7f71b61e3740 20 client.25862 populate_metadata read hostname 'smithi005' 2024-04-24T06:46:24.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.591+0000 7f71b61e3740 1 --2- 172.21.15.5:0/3451787365 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5590f3c93520 0x5590f3c95940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:24.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.591+0000 7f71b61e3740 4 client.25862 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T06:46:24.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.591+0000 7f71b61e3740 1 -- 172.21.15.5:0/3451787365 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5590f3b7e700 con 0x5590f3c93520 2024-04-24T06:46:24.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.592+0000 7f71abe00640 1 --2- 172.21.15.5:0/3451787365 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5590f3c93520 0x5590f3c95940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:24.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.592+0000 7f71abe00640 1 --2- 172.21.15.5:0/3451787365 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5590f3c93520 0x5590f3c95940 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de759608f9c76e47 server_cookie=aa9080f93fc8f91 in_seq=0 out_seq=0 2024-04-24T06:46:24.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.592+0000 7f719a000640 10 client.25862 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:46:24.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.597+0000 7f719a000640 1 -- 172.21.15.5:0/3451787365 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5590f3b7e700 con 0x5590f3c93520 2024-04-24T06:46:24.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.597+0000 7f719a000640 10 client.25862 handle_command_reply: tid=0 2024-04-24T06:46:24.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.597+0000 7f71b61e3740 1 client.25862 shutdown 2024-04-24T06:46:24.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.597+0000 7f71b61e3740 1 -- 172.21.15.5:0/3451787365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f3c1c260 msgr2=0x5590f3c169f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:24.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.597+0000 7f71b61e3740 1 --2- 172.21.15.5:0/3451787365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f3c1c260 0x5590f3c169f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f71a4005010 tx=0x7f71a4003e10 comp rx=0 tx=0).stop 2024-04-24T06:46:24.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.598+0000 7f71b61e3740 1 -- 172.21.15.5:0/3451787365 shutdown_connections 2024-04-24T06:46:24.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.598+0000 7f71b61e3740 1 --2- 172.21.15.5:0/3451787365 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5590f3c93520 0x5590f3c95940 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.598+0000 7f71b61e3740 1 --2- 172.21.15.5:0/3451787365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590f3c1c260 0x5590f3c169f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.598+0000 7f71b61e3740 1 --2- 172.21.15.5:0/3451787365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590f3c11b90 0x5590f3c164b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.598+0000 7f71b61e3740 1 --2- 172.21.15.5:0/3451787365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f3c11270 0x5590f3c15f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.598+0000 7f719a000640 0 client.25862 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:46:24.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.598+0000 7f71b61e3740 1 -- 172.21.15.5:0/3451787365 >> 172.21.15.5:0/3451787365 conn(0x5590f3c0d590 msgr2=0x5590f3c19030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:24.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.598+0000 7f71b61e3740 1 -- 172.21.15.5:0/3451787365 shutdown_connections 2024-04-24T06:46:24.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.598+0000 7f71b61e3740 1 -- 172.21.15.5:0/3451787365 wait complete. 2024-04-24T06:46:24.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.599+0000 7f71b61e3740 20 client.25862 trim_cache size 0 max 16384 2024-04-24T06:46:24.600 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:46:24.600 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-24T06:46:24.601 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "8980b241-9afa-4477-b6e1-e6075eefe28d", 2024-04-24T06:46:24.601 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-24T06:46:24.601 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:46:24.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.599+0000 7f71b1c00640 1 -- 172.21.15.5:0/3904926477 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f71900a8130 msgr2=0x7f71900aa5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:24.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.599+0000 7f71b1c00640 1 --2- 172.21.15.5:0/3904926477 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f71900a8130 0x7f71900aa5f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x5590f3c12830 tx=0x7f7194005f90 comp rx=0 tx=0).stop 2024-04-24T06:46:24.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.599+0000 7f71b1c00640 1 -- 172.21.15.5:0/3904926477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71ac107140 msgr2=0x7f71ac198cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:24.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.599+0000 7f71b1c00640 1 --2- 172.21.15.5:0/3904926477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71ac107140 0x7f71ac198cc0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f719c000f80 tx=0x7f719c003f10 comp rx=0 tx=0).stop 2024-04-24T06:46:24.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.600+0000 7f71b1c00640 1 -- 172.21.15.5:0/3904926477 shutdown_connections 2024-04-24T06:46:24.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.600+0000 7f71b1c00640 1 --2- 172.21.15.5:0/3904926477 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f71900a8130 0x7f71900aa5f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.600+0000 7f71b1c00640 1 --2- 172.21.15.5:0/3904926477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71ac19cff0 0x7f71ac19d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.600+0000 7f71b1c00640 1 --2- 172.21.15.5:0/3904926477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71ac107140 0x7f71ac198cc0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.600+0000 7f71b1c00640 1 --2- 172.21.15.5:0/3904926477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71ac102070 0x7f71ac198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.600+0000 7f71b1c00640 1 -- 172.21.15.5:0/3904926477 >> 172.21.15.5:0/3904926477 conn(0x7f71ac0fd950 msgr2=0x7f71ac0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:24.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.600+0000 7f71b1c00640 1 -- 172.21.15.5:0/3904926477 shutdown_connections 2024-04-24T06:46:24.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.600+0000 7f71b1c00640 1 -- 172.21.15.5:0/3904926477 wait complete. 2024-04-24T06:46:24.625 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T06:46:24.625 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T06:46:24.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.893+0000 7f824d800640 1 -- 172.21.15.5:0/1822344429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8248109c20 msgr2=0x7f824810d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:24.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.893+0000 7f824d800640 1 --2- 172.21.15.5:0/1822344429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8248109c20 0x7f824810d6b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8230009370 tx=0x7f823002a400 comp rx=0 tx=0).stop 2024-04-24T06:46:24.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.893+0000 7f824d800640 1 -- 172.21.15.5:0/1822344429 shutdown_connections 2024-04-24T06:46:24.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.893+0000 7f824d800640 1 --2- 172.21.15.5:0/1822344429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f824810dd90 0x7f82481159a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.893+0000 7f824d800640 1 --2- 172.21.15.5:0/1822344429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8248109c20 0x7f824810d6b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.893+0000 7f824d800640 1 --2- 172.21.15.5:0/1822344429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8248109270 0x7f8248109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.893+0000 7f824d800640 1 -- 172.21.15.5:0/1822344429 >> 172.21.15.5:0/1822344429 conn(0x7f824806d0a0 msgr2=0x7f824806d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:24.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.894+0000 7f824d800640 1 -- 172.21.15.5:0/1822344429 shutdown_connections 2024-04-24T06:46:24.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.894+0000 7f824d800640 1 -- 172.21.15.5:0/1822344429 wait complete. 2024-04-24T06:46:24.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.896+0000 7f824d800640 1 Processor -- start 2024-04-24T06:46:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.896+0000 7f824d800640 1 -- start start 2024-04-24T06:46:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.897+0000 7f824d800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8248109270 0x7f82481142e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.898+0000 7f824d800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f824810dd90 0x7f8248114820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.898+0000 7f824d800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82481b1380 0x7f82481b3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.898+0000 7f824d800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8248118050 con 0x7f82481b1380 2024-04-24T06:46:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.898+0000 7f824d800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8248117ed0 con 0x7f824810dd90 2024-04-24T06:46:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.898+0000 7f824d800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f82481181d0 con 0x7f8248109270 2024-04-24T06:46:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.898+0000 7f823fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f824810dd90 0x7f8248114820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.898+0000 7f8246a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8248109270 0x7f82481142e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.898+0000 7f823fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f824810dd90 0x7f8248114820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:40518/0 (socket says 172.21.15.5:40518) 2024-04-24T06:46:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.898+0000 7f823fe00640 1 -- 172.21.15.5:0/3965101660 learned_addr learned my addr 172.21.15.5:0/3965101660 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.899+0000 7f823fe00640 1 -- 172.21.15.5:0/3965101660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8248109270 msgr2=0x7f82481142e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.899+0000 7f823fe00640 1 --2- 172.21.15.5:0/3965101660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8248109270 0x7f82481142e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.899+0000 7f823fe00640 1 -- 172.21.15.5:0/3965101660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82481b1380 msgr2=0x7f82481b3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.899+0000 7f823fe00640 1 --2- 172.21.15.5:0/3965101660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82481b1380 0x7f82481b3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.899+0000 7f823fe00640 1 -- 172.21.15.5:0/3965101660 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82481b3de0 con 0x7f824810dd90 2024-04-24T06:46:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.899+0000 7f823fe00640 1 --2- 172.21.15.5:0/3965101660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f824810dd90 0x7f8248114820 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f82300058c0 tx=0x7f8230030650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.900+0000 7f823f400640 1 -- 172.21.15.5:0/3965101660 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8230040020 con 0x7f824810dd90 2024-04-24T06:46:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.900+0000 7f823f400640 1 -- 172.21.15.5:0/3965101660 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8230004460 con 0x7f824810dd90 2024-04-24T06:46:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.900+0000 7f823f400640 1 -- 172.21.15.5:0/3965101660 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8230035070 con 0x7f824810dd90 2024-04-24T06:46:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.900+0000 7f824d800640 1 -- 172.21.15.5:0/3965101660 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82481b4070 con 0x7f824810dd90 2024-04-24T06:46:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.901+0000 7f824d800640 1 -- 172.21.15.5:0/3965101660 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82481b4510 con 0x7f824810dd90 2024-04-24T06:46:24.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.908+0000 7f823f400640 1 -- 172.21.15.5:0/3965101660 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f8230043020 con 0x7f824810dd90 2024-04-24T06:46:24.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.911+0000 7f825067b740 1 Processor -- start 2024-04-24T06:46:24.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.911+0000 7f825067b740 1 -- start start 2024-04-24T06:46:24.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.912+0000 7f825067b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a614514860 0x55a614514c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:24.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.912+0000 7f825067b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a614515180 0x55a61451f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:24.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.912+0000 7f8247400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a614514860 0x55a614514c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:24.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.913+0000 7f8247400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a614514860 0x55a614514c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:40522/0 (socket says 172.21.15.5:40522) 2024-04-24T06:46:24.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.913+0000 7f825067b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a61451f850 0x55a614521c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:24.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.913+0000 7f825067b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a61447a1f0 con 0x55a614515180 2024-04-24T06:46:24.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.913+0000 7f825067b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a614420350 con 0x55a614514860 2024-04-24T06:46:24.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.913+0000 7f825067b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a6144761b0 con 0x55a61451f850 2024-04-24T06:46:24.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.913+0000 7f8247400640 1 -- 172.21.15.5:0/2489923939 learned_addr learned my addr 172.21.15.5:0/2489923939 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:24.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.913+0000 7f8247400640 1 --2- 172.21.15.5:0/2489923939 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a61451f850 0x55a614521c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:24.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.914+0000 7f823fe00640 1 --2- 172.21.15.5:0/2489923939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a614515180 0x55a61451f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:24.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.912+0000 7f823f400640 1 --2- 172.21.15.5:0/3965101660 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f822c0a3e20 0x7f822c0a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:24.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.914+0000 7f823f400640 1 -- 172.21.15.5:0/3965101660 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f82300f1230 con 0x7f824810dd90 2024-04-24T06:46:24.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.914+0000 7f8247400640 1 -- 172.21.15.5:0/2489923939 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a61451f850 msgr2=0x55a614521c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:24.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.915+0000 7f8247400640 1 --2- 172.21.15.5:0/2489923939 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a61451f850 0x55a614521c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.915+0000 7f8247400640 1 -- 172.21.15.5:0/2489923939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a614515180 msgr2=0x55a61451f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:24.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.915+0000 7f8247400640 1 --2- 172.21.15.5:0/2489923939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a614515180 0x55a61451f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.915+0000 7f8247400640 1 -- 172.21.15.5:0/2489923939 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a614474f50 con 0x55a614514860 2024-04-24T06:46:24.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.915+0000 7f8247400640 1 --2- 172.21.15.5:0/2489923939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a614514860 0x55a614514c40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8240007f80 tx=0x7f8240007950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b170b755f873060 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:24.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.915+0000 7f8246a00640 1 --2- 172.21.15.5:0/3965101660 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f822c0a3e20 0x7f822c0a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:24.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.916+0000 7f8237e00640 1 -- 172.21.15.5:0/2489923939 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8240003b80 con 0x55a614514860 2024-04-24T06:46:24.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.916+0000 7f8237e00640 1 -- 172.21.15.5:0/2489923939 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f824000c510 con 0x55a614514860 2024-04-24T06:46:24.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.916+0000 7f8237e00640 1 -- 172.21.15.5:0/2489923939 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8240005280 con 0x55a614514860 2024-04-24T06:46:24.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.917+0000 7f825067b740 1 -- 172.21.15.5:0/2489923939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a614514860 msgr2=0x55a614514c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:24.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.917+0000 7f825067b740 1 --2- 172.21.15.5:0/2489923939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a614514860 0x55a614514c40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8240007f80 tx=0x7f8240007950 comp rx=0 tx=0).stop 2024-04-24T06:46:24.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.918+0000 7f825067b740 1 -- 172.21.15.5:0/2489923939 shutdown_connections 2024-04-24T06:46:24.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.918+0000 7f825067b740 1 --2- 172.21.15.5:0/2489923939 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a61451f850 0x55a614521c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.918+0000 7f825067b740 1 --2- 172.21.15.5:0/2489923939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a614515180 0x55a61451f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.918+0000 7f825067b740 1 --2- 172.21.15.5:0/2489923939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a614514860 0x55a614514c40 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.918+0000 7f825067b740 1 -- 172.21.15.5:0/2489923939 >> 172.21.15.5:0/2489923939 conn(0x55a614510b80 msgr2=0x55a614510f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:24.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.919+0000 7f8246a00640 1 --2- 172.21.15.5:0/3965101660 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f822c0a3e20 0x7f822c0a62e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f823800abd0 tx=0x7f8238005f90 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:24.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.919+0000 7f825067b740 1 -- 172.21.15.5:0/2489923939 shutdown_connections 2024-04-24T06:46:24.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.919+0000 7f825067b740 1 -- 172.21.15.5:0/2489923939 wait complete. 2024-04-24T06:46:24.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.920+0000 7f825067b740 1 -- start start 2024-04-24T06:46:24.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.920+0000 7f825067b740 1 Processor -- start 2024-04-24T06:46:24.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.921+0000 7f825067b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a614514860 0x55a61451c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:24.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.922+0000 7f825067b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a614515180 0x55a61451cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:24.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.922+0000 7f825067b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a61451f850 0x55a61451d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:24.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.922+0000 7f825067b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a614522640 con 0x55a61451f850 2024-04-24T06:46:24.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.922+0000 7f825067b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a614433690 con 0x55a614514860 2024-04-24T06:46:24.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.922+0000 7f825067b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a6145227c0 con 0x55a614515180 2024-04-24T06:46:24.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.922+0000 7f825067b740 10 client.0 _ll_register_callbacks cb 0x55a61446f650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:46:24.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.922+0000 7f8246a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a614514860 0x55a61451c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:24.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.922+0000 7f8246a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a614514860 0x55a61451c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:40538/0 (socket says 172.21.15.5:40538) 2024-04-24T06:46:24.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.922+0000 7f8246a00640 1 -- 172.21.15.5:0/3756281430 learned_addr learned my addr 172.21.15.5:0/3756281430 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:24.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.922+0000 7f8246a00640 1 -- 172.21.15.5:0/3756281430 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a614515180 msgr2=0x55a61451cd90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T06:46:24.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.922+0000 7f8247400640 1 --2- 172.21.15.5:0/3756281430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a61451f850 0x55a61451d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:24.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.923+0000 7f823fe00640 1 --2- 172.21.15.5:0/3756281430 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a614515180 0x55a61451cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:24.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.923+0000 7f8246a00640 1 --2- 172.21.15.5:0/3756281430 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a614515180 0x55a61451cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.923+0000 7f8246a00640 1 -- 172.21.15.5:0/3756281430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a61451f850 msgr2=0x55a61451d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:24.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.923+0000 7f8246a00640 1 --2- 172.21.15.5:0/3756281430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a61451f850 0x55a61451d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.923+0000 7f8246a00640 1 -- 172.21.15.5:0/3756281430 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a614432920 con 0x55a614514860 2024-04-24T06:46:24.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.923+0000 7f8247400640 1 --2- 172.21.15.5:0/3756281430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a61451f850 0x55a61451d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:46:24.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.923+0000 7f823fe00640 1 --2- 172.21.15.5:0/3756281430 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a614515180 0x55a61451cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:46:24.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.923+0000 7f8246a00640 1 --2- 172.21.15.5:0/3756281430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a614514860 0x55a61451c850 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f823800a7b0 tx=0x7f82380046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7feedae035377cf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:24.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.923+0000 7f8234c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:46:24.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.924+0000 7f8234c00640 1 -- 172.21.15.5:0/3756281430 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823802a060 con 0x55a614514860 2024-04-24T06:46:24.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.924+0000 7f8234c00640 1 -- 172.21.15.5:0/3756281430 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f8238014030 con 0x55a614514860 2024-04-24T06:46:24.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.924+0000 7f825067b740 1 -- 172.21.15.5:0/3756281430 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f824000c510 con 0x55a614514860 2024-04-24T06:46:24.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.924+0000 7f8234c00640 1 -- 172.21.15.5:0/3756281430 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f823801ac30 con 0x55a614514860 2024-04-24T06:46:24.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.924+0000 7f825067b740 10 client.25874 fetch_fsmap learned FSMap version 25 2024-04-24T06:46:24.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.924+0000 7f825067b740 1 -- 172.21.15.5:0/3756281430 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a6142567d0 con 0x55a614514860 2024-04-24T06:46:24.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.925+0000 7f8234c00640 1 -- 172.21.15.5:0/3756281430 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f824000c510 con 0x55a614514860 2024-04-24T06:46:24.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.925+0000 7f8234c00640 1 -- 172.21.15.5:0/3756281430 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f823800b040 con 0x55a614514860 2024-04-24T06:46:24.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.925+0000 7f825067b740 10 client.25874 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:46:24.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.925+0000 7f825067b740 10 client.25874 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:46:24.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.925+0000 7f825067b740 20 client.25874 populate_metadata read hostname 'smithi005' 2024-04-24T06:46:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.925+0000 7f825067b740 1 --2- 172.21.15.5:0/3756281430 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a61452fc30 0x55a614532080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.925+0000 7f825067b740 4 client.25874 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:46:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.925+0000 7f825067b740 1 -- 172.21.15.5:0/3756281430 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a614487c10 con 0x55a61452fc30 2024-04-24T06:46:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.926+0000 7f823fe00640 1 --2- 172.21.15.5:0/3756281430 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a61452fc30 0x55a614532080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:24.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.927+0000 7f823fe00640 1 --2- 172.21.15.5:0/3756281430 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a61452fc30 0x55a614532080 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8db765814ff6f8d8 server_cookie=6630a7a76fdcddfc in_seq=0 out_seq=0 2024-04-24T06:46:24.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.927+0000 7f8234c00640 10 client.25874 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:46:24.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.928+0000 7f8234c00640 1 -- 172.21.15.5:0/3756281430 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a614487c10 con 0x55a61452fc30 2024-04-24T06:46:24.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.928+0000 7f8234c00640 10 client.25874 handle_command_reply: tid=0 2024-04-24T06:46:24.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.928+0000 7f825067b740 1 client.25874 shutdown 2024-04-24T06:46:24.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.929+0000 7f825067b740 1 -- 172.21.15.5:0/3756281430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a614514860 msgr2=0x55a61451c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:24.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.929+0000 7f825067b740 1 --2- 172.21.15.5:0/3756281430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a614514860 0x55a61451c850 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f823800a7b0 tx=0x7f82380046c0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.929+0000 7f825067b740 1 -- 172.21.15.5:0/3756281430 shutdown_connections 2024-04-24T06:46:24.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.929+0000 7f825067b740 1 --2- 172.21.15.5:0/3756281430 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a61452fc30 0x55a614532080 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.929+0000 7f825067b740 1 --2- 172.21.15.5:0/3756281430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a61451f850 0x55a61451d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.929+0000 7f825067b740 1 --2- 172.21.15.5:0/3756281430 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a614515180 0x55a61451cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.929+0000 7f825067b740 1 --2- 172.21.15.5:0/3756281430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a614514860 0x55a61451c850 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.929+0000 7f825067b740 1 -- 172.21.15.5:0/3756281430 >> 172.21.15.5:0/3756281430 conn(0x55a614510b80 msgr2=0x55a614516980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:24.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.929+0000 7f8234c00640 0 client.25874 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:46:24.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.929+0000 7f825067b740 1 -- 172.21.15.5:0/3756281430 shutdown_connections 2024-04-24T06:46:24.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.929+0000 7f825067b740 1 -- 172.21.15.5:0/3756281430 wait complete. 2024-04-24T06:46:24.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.930+0000 7f825067b740 20 client.25874 trim_cache size 0 max 16384 2024-04-24T06:46:24.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.957+0000 7f825067b740 1 Processor -- start 2024-04-24T06:46:24.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.957+0000 7f825067b740 1 -- start start 2024-04-24T06:46:24.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.957+0000 7f825067b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a614514860 0x55a61452b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:24.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.958+0000 7f825067b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a614515180 0x55a61450faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:24.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.958+0000 7f825067b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a61451f850 0x55a614537990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:24.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.958+0000 7f825067b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f82481bf460 con 0x55a614515180 2024-04-24T06:46:24.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.958+0000 7f825067b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f82481bcd30 con 0x55a61451f850 2024-04-24T06:46:24.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.958+0000 7f825067b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f82481bab80 con 0x55a614514860 2024-04-24T06:46:24.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.958+0000 7f8246a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a614514860 0x55a61452b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:24.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.958+0000 7f8246a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a614514860 0x55a61452b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:42326/0 (socket says 172.21.15.5:42326) 2024-04-24T06:46:24.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.958+0000 7f8246a00640 1 -- 172.21.15.5:0/3943403434 learned_addr learned my addr 172.21.15.5:0/3943403434 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:24.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.958+0000 7f823fe00640 1 --2- 172.21.15.5:0/3943403434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a614515180 0x55a61450faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:24.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.958+0000 7f8247400640 1 --2- 172.21.15.5:0/3943403434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a61451f850 0x55a614537990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:24.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.958+0000 7f8246a00640 1 -- 172.21.15.5:0/3943403434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a61451f850 msgr2=0x55a614537990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:24.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.958+0000 7f8246a00640 1 --2- 172.21.15.5:0/3943403434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a61451f850 0x55a614537990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.959+0000 7f8246a00640 1 -- 172.21.15.5:0/3943403434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a614515180 msgr2=0x55a61450faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:24.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.959+0000 7f8246a00640 1 --2- 172.21.15.5:0/3943403434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a614515180 0x55a61450faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.959+0000 7f8246a00640 1 -- 172.21.15.5:0/3943403434 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a61418c3f0 con 0x55a614514860 2024-04-24T06:46:24.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.959+0000 7f8247400640 1 --2- 172.21.15.5:0/3943403434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a61451f850 0x55a614537990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:46:24.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.959+0000 7f823fe00640 1 --2- 172.21.15.5:0/3943403434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a614515180 0x55a61450faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:46:24.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.959+0000 7f8246a00640 1 --2- 172.21.15.5:0/3943403434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a614514860 0x55a61452b500 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f823800e8f0 tx=0x7f8238000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4880a20ec84e8f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:24.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.959+0000 7f8237400640 1 -- 172.21.15.5:0/3943403434 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8238034080 con 0x55a614514860 2024-04-24T06:46:24.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.959+0000 7f8237400640 1 -- 172.21.15.5:0/3943403434 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f823800a310 con 0x55a614514860 2024-04-24T06:46:24.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.959+0000 7f8237400640 1 -- 172.21.15.5:0/3943403434 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8238004730 con 0x55a614514860 2024-04-24T06:46:24.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.961+0000 7f825067b740 1 -- 172.21.15.5:0/3943403434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a614514860 msgr2=0x55a61452b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:24.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.961+0000 7f825067b740 1 --2- 172.21.15.5:0/3943403434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a614514860 0x55a61452b500 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f823800e8f0 tx=0x7f8238000bc0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.961+0000 7f825067b740 1 -- 172.21.15.5:0/3943403434 shutdown_connections 2024-04-24T06:46:24.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.961+0000 7f825067b740 1 --2- 172.21.15.5:0/3943403434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a61451f850 0x55a614537990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.961+0000 7f825067b740 1 --2- 172.21.15.5:0/3943403434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a614515180 0x55a61450faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.961+0000 7f825067b740 1 --2- 172.21.15.5:0/3943403434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a614514860 0x55a61452b500 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.961+0000 7f825067b740 1 -- 172.21.15.5:0/3943403434 >> 172.21.15.5:0/3943403434 conn(0x55a614510b80 msgr2=0x55a61450f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:24.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.961+0000 7f825067b740 1 -- 172.21.15.5:0/3943403434 shutdown_connections 2024-04-24T06:46:24.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.961+0000 7f825067b740 1 -- 172.21.15.5:0/3943403434 wait complete. 2024-04-24T06:46:24.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.962+0000 7f825067b740 1 -- start start 2024-04-24T06:46:24.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.962+0000 7f825067b740 1 Processor -- start 2024-04-24T06:46:24.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.963+0000 7f825067b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a614514860 0x55a614527b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:24.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.963+0000 7f825067b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a614515180 0x55a614522f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:24.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.963+0000 7f825067b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a61451f850 0x55a6145234d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:24.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.963+0000 7f825067b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f82481bb1f0 con 0x55a61451f850 2024-04-24T06:46:24.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.963+0000 7f825067b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f82481bc660 con 0x55a614514860 2024-04-24T06:46:24.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.963+0000 7f825067b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f82481bb8c0 con 0x55a614515180 2024-04-24T06:46:24.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.963+0000 7f825067b740 10 client.0 _ll_register_callbacks cb 0x7f823800a9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:46:24.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.963+0000 7f8246a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a614514860 0x55a614527b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:24.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.963+0000 7f823fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a614515180 0x55a614522f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:24.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.963+0000 7f8246a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a614514860 0x55a614527b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:40548/0 (socket says 172.21.15.5:40548) 2024-04-24T06:46:24.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.963+0000 7f823fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a614515180 0x55a614522f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:42336/0 (socket says 172.21.15.5:42336) 2024-04-24T06:46:24.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.963+0000 7f8246a00640 1 -- 172.21.15.5:0/2679475201 learned_addr learned my addr 172.21.15.5:0/2679475201 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:24.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.964+0000 7f823fe00640 1 -- 172.21.15.5:0/2679475201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a614514860 msgr2=0x55a614527b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:24.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.964+0000 7f823fe00640 1 --2- 172.21.15.5:0/2679475201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a614514860 0x55a614527b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.964+0000 7f823fe00640 1 -- 172.21.15.5:0/2679475201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a61451f850 msgr2=0x55a6145234d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:24.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.964+0000 7f8247400640 1 --2- 172.21.15.5:0/2679475201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a61451f850 0x55a6145234d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:24.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.964+0000 7f823fe00640 1 --2- 172.21.15.5:0/2679475201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a61451f850 0x55a6145234d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.964+0000 7f823fe00640 1 -- 172.21.15.5:0/2679475201 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a61414c0f0 con 0x55a614515180 2024-04-24T06:46:24.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.964+0000 7f8247400640 1 --2- 172.21.15.5:0/2679475201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a61451f850 0x55a6145234d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:46:24.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.964+0000 7f823fe00640 1 --2- 172.21.15.5:0/2679475201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a614515180 0x55a614522f90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f82300ee930 tx=0x7f8230000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adf278475ba61ac9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:24.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.964+0000 7f8234c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:46:24.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.964+0000 7f8234c00640 1 -- 172.21.15.5:0/2679475201 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82300c5df0 con 0x55a614515180 2024-04-24T06:46:24.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.964+0000 7f8234c00640 1 -- 172.21.15.5:0/2679475201 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f82300c11b0 con 0x55a614515180 2024-04-24T06:46:24.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.964+0000 7f825067b740 1 -- 172.21.15.5:0/2679475201 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f823800a310 con 0x55a614515180 2024-04-24T06:46:24.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.965+0000 7f8234c00640 1 -- 172.21.15.5:0/2679475201 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f82300eebb0 con 0x55a614515180 2024-04-24T06:46:24.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.965+0000 7f825067b740 10 client.25909 fetch_fsmap learned FSMap version 25 2024-04-24T06:46:24.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.965+0000 7f825067b740 1 -- 172.21.15.5:0/2679475201 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a61443f810 con 0x55a614515180 2024-04-24T06:46:24.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.965+0000 7f8234c00640 1 -- 172.21.15.5:0/2679475201 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f823800a310 con 0x55a614515180 2024-04-24T06:46:24.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.965+0000 7f8234c00640 1 -- 172.21.15.5:0/2679475201 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f823003a040 con 0x55a614515180 2024-04-24T06:46:24.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.965+0000 7f825067b740 10 client.25909 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:46:24.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.965+0000 7f825067b740 10 client.25909 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:46:24.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.965+0000 7f825067b740 20 client.25909 populate_metadata read hostname 'smithi005' 2024-04-24T06:46:24.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.965+0000 7f825067b740 1 --2- 172.21.15.5:0/2679475201 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a614596380 0x55a6145987a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:24.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.966+0000 7f825067b740 4 client.25909 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:46:24.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.966+0000 7f825067b740 1 -- 172.21.15.5:0/2679475201 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a614481a80 con 0x55a614596380 2024-04-24T06:46:24.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.966+0000 7f8246a00640 1 --2- 172.21.15.5:0/2679475201 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a614596380 0x55a6145987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:24.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.967+0000 7f8246a00640 1 --2- 172.21.15.5:0/2679475201 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a614596380 0x55a6145987a0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4d524cab772f84d server_cookie=77a17864b54a3f27 in_seq=0 out_seq=0 2024-04-24T06:46:24.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.967+0000 7f8234c00640 10 client.25909 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:46:24.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.967+0000 7f8234c00640 1 -- 172.21.15.5:0/2679475201 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55a614481a80 con 0x55a614596380 2024-04-24T06:46:24.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.967+0000 7f8234c00640 10 client.25909 handle_command_reply: tid=0 2024-04-24T06:46:24.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.967+0000 7f825067b740 1 client.25909 shutdown 2024-04-24T06:46:24.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.968+0000 7f825067b740 1 -- 172.21.15.5:0/2679475201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a614515180 msgr2=0x55a614522f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:24.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.968+0000 7f825067b740 1 --2- 172.21.15.5:0/2679475201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a614515180 0x55a614522f90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f82300ee930 tx=0x7f8230000bc0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.968+0000 7f825067b740 1 -- 172.21.15.5:0/2679475201 shutdown_connections 2024-04-24T06:46:24.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.968+0000 7f825067b740 1 --2- 172.21.15.5:0/2679475201 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a614596380 0x55a6145987a0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.968+0000 7f825067b740 1 --2- 172.21.15.5:0/2679475201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a61451f850 0x55a6145234d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.968+0000 7f825067b740 1 --2- 172.21.15.5:0/2679475201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a614515180 0x55a614522f90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.968+0000 7f825067b740 1 --2- 172.21.15.5:0/2679475201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a614514860 0x55a614527b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.968+0000 7f825067b740 1 -- 172.21.15.5:0/2679475201 >> 172.21.15.5:0/2679475201 conn(0x55a614510b80 msgr2=0x55a61451d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.968+0000 7f8234c00640 0 client.25909 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:46:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.968+0000 7f825067b740 1 -- 172.21.15.5:0/2679475201 shutdown_connections 2024-04-24T06:46:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.968+0000 7f825067b740 1 -- 172.21.15.5:0/2679475201 wait complete. 2024-04-24T06:46:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.968+0000 7f825067b740 20 client.25909 trim_cache size 0 max 16384 2024-04-24T06:46:24.975 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:46:24.975 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1215 inodes in the stack)", 2024-04-24T06:46:24.975 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-24T06:46:24.975 INFO:teuthology.orchestra.run.smithi005.stdout: "8980b241-9afa-4477-b6e1-e6075eefe28d": { 2024-04-24T06:46:24.975 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-24T06:46:24.975 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "8980b241-9afa-4477-b6e1-e6075eefe28d", 2024-04-24T06:46:24.975 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-04-24T06:46:24.975 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:46:24.975 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:46:24.975 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:46:24.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.969+0000 7f824d800640 1 -- 172.21.15.5:0/3965101660 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f822c0a3e20 msgr2=0x7f822c0a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:24.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.969+0000 7f824d800640 1 --2- 172.21.15.5:0/3965101660 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f822c0a3e20 0x7f822c0a62e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f823800abd0 tx=0x7f8238005f90 comp rx=0 tx=0).stop 2024-04-24T06:46:24.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.969+0000 7f824d800640 1 -- 172.21.15.5:0/3965101660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f824810dd90 msgr2=0x7f8248114820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:24.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.969+0000 7f824d800640 1 --2- 172.21.15.5:0/3965101660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f824810dd90 0x7f8248114820 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f82300058c0 tx=0x7f8230030650 comp rx=0 tx=0).stop 2024-04-24T06:46:24.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.969+0000 7f824d800640 1 -- 172.21.15.5:0/3965101660 shutdown_connections 2024-04-24T06:46:24.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.969+0000 7f824d800640 1 --2- 172.21.15.5:0/3965101660 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f822c0a3e20 0x7f822c0a62e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.969+0000 7f824d800640 1 --2- 172.21.15.5:0/3965101660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82481b1380 0x7f82481b3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.969+0000 7f824d800640 1 --2- 172.21.15.5:0/3965101660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f824810dd90 0x7f8248114820 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.969+0000 7f824d800640 1 --2- 172.21.15.5:0/3965101660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8248109270 0x7f82481142e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:24.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.969+0000 7f824d800640 1 -- 172.21.15.5:0/3965101660 >> 172.21.15.5:0/3965101660 conn(0x7f824806d0a0 msgr2=0x7f8248073160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:24.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.969+0000 7f824d800640 1 -- 172.21.15.5:0/3965101660 shutdown_connections 2024-04-24T06:46:24.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:24.969+0000 7f824d800640 1 -- 172.21.15.5:0/3965101660 wait complete. 2024-04-24T06:46:24.991 INFO:tasks.cephfs.filesystem:scrub status for tag:8980b241-9afa-4477-b6e1-e6075eefe28d - {'path': '/', 'tag': '8980b241-9afa-4477-b6e1-e6075eefe28d', 'options': 'recursive,force'} 2024-04-24T06:46:25.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:25 smithi005 ceph-mon[27798]: pgmap v74: 97 pgs: 97 active+clean; 52 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 157 op/s 2024-04-24T06:46:25.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:25 smithi005 ceph-mon[27798]: scrub queued for path: / 2024-04-24T06:46:25.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:25 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:46:25.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:25 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:46:25.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:25 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:46:25.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:25 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:46:25.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:25 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:46:25.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:25 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:46:25.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:25 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:46:25.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:25 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:46:25.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:25 smithi037 ceph-mon[32864]: pgmap v74: 97 pgs: 97 active+clean; 52 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 157 op/s 2024-04-24T06:46:25.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:25 smithi037 ceph-mon[32864]: scrub queued for path: / 2024-04-24T06:46:25.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:25 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:46:25.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:25 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:46:25.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:25 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:46:25.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:25 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:46:25.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:25 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:46:25.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:25 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:46:25.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:25 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:46:25.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:25 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:46:25.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:25 smithi137 ceph-mon[33017]: pgmap v74: 97 pgs: 97 active+clean; 52 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 157 op/s 2024-04-24T06:46:25.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:25 smithi137 ceph-mon[33017]: scrub queued for path: / 2024-04-24T06:46:25.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:25 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:46:25.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:25 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:46:25.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:25 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:46:25.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:25 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:46:25.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:25 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:46:25.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:25 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:46:25.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:25 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:46:25.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:25 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:46:26.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:26 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:46:26.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:26 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:46:26.547 INFO:tasks.workunit.client.0.smithi005.stdout:autoheader 2024-04-24T06:46:26.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:26 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:46:26.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:26 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:46:26.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:26 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:46:26.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:26 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:46:27.009 INFO:tasks.workunit.client.0.smithi005.stdout:autoconf 2024-04-24T06:46:27.298 INFO:tasks.workunit.client.0.smithi005.stdout:./configure \ 2024-04-24T06:46:27.298 INFO:tasks.workunit.client.0.smithi005.stdout: --libexecdir=/usr/lib \ 2024-04-24T06:46:27.298 INFO:tasks.workunit.client.0.smithi005.stdout: --exec_prefix=/var/lib 2024-04-24T06:46:27.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:27 smithi037 ceph-mon[32864]: pgmap v75: 97 pgs: 97 active+clean; 52 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 138 op/s 2024-04-24T06:46:27.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:27 smithi137 ceph-mon[33017]: pgmap v75: 97 pgs: 97 active+clean; 52 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 138 op/s 2024-04-24T06:46:27.699 INFO:tasks.workunit.client.0.smithi005.stdout:checking build system type... x86_64-pc-linux-gnu 2024-04-24T06:46:27.699 INFO:tasks.workunit.client.0.smithi005.stdout:checking host system type... x86_64-pc-linux-gnu 2024-04-24T06:46:27.703 INFO:tasks.workunit.client.0.smithi005.stdout:checking how to print strings... printf 2024-04-24T06:46:27.703 INFO:tasks.workunit.client.0.smithi005.stdout:checking for gcc... gcc 2024-04-24T06:46:27.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:27 smithi005 ceph-mon[27798]: pgmap v75: 97 pgs: 97 active+clean; 52 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 138 op/s 2024-04-24T06:46:28.436 INFO:tasks.workunit.client.0.smithi005.stdout:checking whether the C compiler works... yes 2024-04-24T06:46:28.436 INFO:tasks.workunit.client.0.smithi005.stdout:checking for C compiler default output file name... a.out 2024-04-24T06:46:28.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:28 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:46:28.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:28 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:46:28.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:28 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:46:28.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:28 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:46:28.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:28 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:46:28.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:28 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:46:28.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:28 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:46:28.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:28 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:46:28.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:28 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:46:29.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:29 smithi137 ceph-mon[33017]: pgmap v76: 97 pgs: 97 active+clean; 54 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 157 op/s 2024-04-24T06:46:29.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:29 smithi005 ceph-mon[27798]: pgmap v76: 97 pgs: 97 active+clean; 54 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 157 op/s 2024-04-24T06:46:29.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:29 smithi037 ceph-mon[32864]: pgmap v76: 97 pgs: 97 active+clean; 54 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 157 op/s 2024-04-24T06:46:31.437 INFO:tasks.workunit.client.0.smithi005.stdout:checking for suffix of executables... 2024-04-24T06:46:31.535 INFO:tasks.workunit.client.0.smithi005.stdout:checking whether we are cross compiling... no 2024-04-24T06:46:31.612 INFO:tasks.workunit.client.0.smithi005.stdout:checking for suffix of object files... o 2024-04-24T06:46:31.680 INFO:tasks.workunit.client.0.smithi005.stdout:checking whether we are using the GNU C compiler... yes 2024-04-24T06:46:31.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:31 smithi137 ceph-mon[33017]: pgmap v77: 97 pgs: 97 active+clean; 61 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 392 op/s 2024-04-24T06:46:31.751 INFO:tasks.workunit.client.0.smithi005.stdout:checking whether gcc accepts -g... yes 2024-04-24T06:46:31.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:31 smithi005 ceph-mon[27798]: pgmap v77: 97 pgs: 97 active+clean; 61 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 392 op/s 2024-04-24T06:46:31.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:31 smithi037 ceph-mon[32864]: pgmap v77: 97 pgs: 97 active+clean; 61 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 392 op/s 2024-04-24T06:46:31.886 INFO:tasks.workunit.client.0.smithi005.stdout:checking for gcc option to accept ISO C89... none needed 2024-04-24T06:46:31.900 INFO:tasks.workunit.client.0.smithi005.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2024-04-24T06:46:31.911 INFO:tasks.workunit.client.0.smithi005.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-04-24T06:46:31.917 INFO:tasks.workunit.client.0.smithi005.stdout:checking for egrep... /usr/bin/grep -E 2024-04-24T06:46:31.922 INFO:tasks.workunit.client.0.smithi005.stdout:checking for fgrep... /usr/bin/grep -F 2024-04-24T06:46:31.936 INFO:tasks.workunit.client.0.smithi005.stdout:checking for ld used by gcc... /usr/bin/ld 2024-04-24T06:46:31.943 INFO:tasks.workunit.client.0.smithi005.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2024-04-24T06:46:31.955 INFO:tasks.workunit.client.0.smithi005.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2024-04-24T06:46:33.198 INFO:tasks.workunit.client.0.smithi005.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2024-04-24T06:46:33.198 INFO:tasks.workunit.client.0.smithi005.stdout:checking whether ln -s works... yes 2024-04-24T06:46:33.281 INFO:tasks.workunit.client.0.smithi005.stdout:checking the maximum length of command line arguments... 1572864 2024-04-24T06:46:33.292 INFO:tasks.workunit.client.0.smithi005.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-04-24T06:46:33.293 INFO:tasks.workunit.client.0.smithi005.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-04-24T06:46:33.293 INFO:tasks.workunit.client.0.smithi005.stdout:checking for /usr/bin/ld option to reload object files... -r 2024-04-24T06:46:33.294 INFO:tasks.workunit.client.0.smithi005.stdout:checking for objdump... objdump 2024-04-24T06:46:33.294 INFO:tasks.workunit.client.0.smithi005.stdout:checking how to recognize dependent libraries... pass_all 2024-04-24T06:46:33.295 INFO:tasks.workunit.client.0.smithi005.stdout:checking for dlltool... no 2024-04-24T06:46:33.296 INFO:tasks.workunit.client.0.smithi005.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-04-24T06:46:33.296 INFO:tasks.workunit.client.0.smithi005.stdout:checking for ar... ar 2024-04-24T06:46:33.450 INFO:tasks.workunit.client.0.smithi005.stdout:checking for archiver @FILE support... @ 2024-04-24T06:46:33.451 INFO:tasks.workunit.client.0.smithi005.stdout:checking for strip... strip 2024-04-24T06:46:33.451 INFO:tasks.workunit.client.0.smithi005.stdout:checking for ranlib... ranlib 2024-04-24T06:46:33.452 INFO:tasks.workunit.client.0.smithi005.stdout:checking for gawk... gawk 2024-04-24T06:46:33.685 INFO:tasks.workunit.client.0.smithi005.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2024-04-24T06:46:33.689 INFO:tasks.workunit.client.0.smithi005.stdout:checking for sysroot... no 2024-04-24T06:46:33.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:33 smithi137 ceph-mon[33017]: pgmap v78: 97 pgs: 97 active+clean; 61 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 392 op/s 2024-04-24T06:46:33.712 INFO:tasks.workunit.client.0.smithi005.stdout:checking for a working dd... /usr/bin/dd 2024-04-24T06:46:33.733 INFO:tasks.workunit.client.0.smithi005.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2024-04-24T06:46:33.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:33 smithi005 ceph-mon[27798]: pgmap v78: 97 pgs: 97 active+clean; 61 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 392 op/s 2024-04-24T06:46:33.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:33 smithi037 ceph-mon[32864]: pgmap v78: 97 pgs: 97 active+clean; 61 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 392 op/s 2024-04-24T06:46:33.804 INFO:tasks.workunit.client.0.smithi005.stdout:checking for mt... no 2024-04-24T06:46:33.829 INFO:tasks.workunit.client.0.smithi005.stdout:checking if : is a manifest tool... no 2024-04-24T06:46:33.979 INFO:tasks.workunit.client.0.smithi005.stdout:checking how to run the C preprocessor... gcc -E 2024-04-24T06:46:34.418 INFO:tasks.workunit.client.0.smithi005.stdout:checking for ANSI C header files... yes 2024-04-24T06:46:34.507 INFO:tasks.workunit.client.0.smithi005.stdout:checking for sys/types.h... yes 2024-04-24T06:46:34.602 INFO:tasks.workunit.client.0.smithi005.stdout:checking for sys/stat.h... yes 2024-04-24T06:46:34.700 INFO:tasks.workunit.client.0.smithi005.stdout:checking for stdlib.h... yes 2024-04-24T06:46:34.796 INFO:tasks.workunit.client.0.smithi005.stdout:checking for string.h... yes 2024-04-24T06:46:34.909 INFO:tasks.workunit.client.0.smithi005.stdout:checking for memory.h... yes 2024-04-24T06:46:35.002 INFO:tasks.workunit.client.0.smithi005.stdout:checking for strings.h... yes 2024-04-24T06:46:35.102 INFO:tasks.workunit.client.0.smithi005.stdout:checking for inttypes.h... yes 2024-04-24T06:46:35.205 INFO:tasks.workunit.client.0.smithi005.stdout:checking for stdint.h... yes 2024-04-24T06:46:35.303 INFO:tasks.workunit.client.0.smithi005.stdout:checking for unistd.h... yes 2024-04-24T06:46:35.416 INFO:tasks.workunit.client.0.smithi005.stdout:checking for dlfcn.h... yes 2024-04-24T06:46:35.435 INFO:tasks.workunit.client.0.smithi005.stdout:checking for objdir... .libs 2024-04-24T06:46:35.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:35 smithi137 ceph-mon[33017]: pgmap v79: 97 pgs: 97 active+clean; 78 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.1 MiB/s wr, 788 op/s 2024-04-24T06:46:35.704 INFO:tasks.workunit.client.0.smithi005.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-04-24T06:46:35.704 INFO:tasks.workunit.client.0.smithi005.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-04-24T06:46:35.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:35 smithi005 ceph-mon[27798]: pgmap v79: 97 pgs: 97 active+clean; 78 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.1 MiB/s wr, 788 op/s 2024-04-24T06:46:35.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:35 smithi037 ceph-mon[32864]: pgmap v79: 97 pgs: 97 active+clean; 78 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.1 MiB/s wr, 788 op/s 2024-04-24T06:46:35.791 INFO:tasks.workunit.client.0.smithi005.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-04-24T06:46:35.886 INFO:tasks.workunit.client.0.smithi005.stdout:checking if gcc static flag -static works... no 2024-04-24T06:46:36.018 INFO:tasks.workunit.client.0.smithi005.stdout:checking if gcc supports -c -o file.o... yes 2024-04-24T06:46:36.019 INFO:tasks.workunit.client.0.smithi005.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-04-24T06:46:36.046 INFO:tasks.workunit.client.0.smithi005.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-04-24T06:46:36.158 INFO:tasks.workunit.client.0.smithi005.stdout:checking whether -lc should be explicitly linked in... no 2024-04-24T06:46:36.323 INFO:tasks.workunit.client.0.smithi005.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-04-24T06:46:36.323 INFO:tasks.workunit.client.0.smithi005.stdout:checking how to hardcode library paths into programs... immediate 2024-04-24T06:46:36.328 INFO:tasks.workunit.client.0.smithi005.stdout:checking whether stripping libraries is possible... yes 2024-04-24T06:46:36.328 INFO:tasks.workunit.client.0.smithi005.stdout:checking if libtool supports shared libraries... yes 2024-04-24T06:46:36.328 INFO:tasks.workunit.client.0.smithi005.stdout:checking whether to build shared libraries... yes 2024-04-24T06:46:36.328 INFO:tasks.workunit.client.0.smithi005.stdout:checking whether to build static libraries... yes 2024-04-24T06:46:36.349 INFO:tasks.workunit.client.0.smithi005.stdout:checking for gcc... (cached) gcc 2024-04-24T06:46:36.435 INFO:tasks.workunit.client.0.smithi005.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-04-24T06:46:36.435 INFO:tasks.workunit.client.0.smithi005.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-24T06:46:36.437 INFO:tasks.workunit.client.0.smithi005.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-04-24T06:46:36.438 INFO:tasks.workunit.client.0.smithi005.stdout:checking for gmake... /usr/bin/gmake 2024-04-24T06:46:36.439 INFO:tasks.workunit.client.0.smithi005.stderr:./configure: line 11968: test: $(SHELL): binary operator expected 2024-04-24T06:46:36.439 INFO:tasks.workunit.client.0.smithi005.stdout:checking for tar... /bin/tar 2024-04-24T06:46:36.440 INFO:tasks.workunit.client.0.smithi005.stdout:checking for gzip... /bin/gzip 2024-04-24T06:46:36.487 INFO:tasks.workunit.client.0.smithi005.stdout:checking whether gcc -MM is supported... yes 2024-04-24T06:46:36.489 INFO:tasks.workunit.client.0.smithi005.stdout:checking for sort... /bin/sort 2024-04-24T06:46:36.489 INFO:tasks.workunit.client.0.smithi005.stdout:checking whether ln -s works... yes 2024-04-24T06:46:36.490 INFO:tasks.workunit.client.0.smithi005.stdout:checking for rpm... /bin/rpm 2024-04-24T06:46:36.503 INFO:tasks.workunit.client.0.smithi005.stdout:checking for rpmbuild... no 2024-04-24T06:46:36.503 INFO:tasks.workunit.client.0.smithi005.stdout:checking for ANSI C header files... (cached) yes 2024-04-24T06:46:36.614 INFO:tasks.workunit.client.0.smithi005.stdout:checking assert.h usability... yes 2024-04-24T06:46:36.653 INFO:tasks.workunit.client.0.smithi005.stdout:checking assert.h presence... yes 2024-04-24T06:46:36.654 INFO:tasks.workunit.client.0.smithi005.stdout:checking for assert.h... yes 2024-04-24T06:46:36.762 INFO:tasks.workunit.client.0.smithi005.stdout:checking bstring.h usability... no 2024-04-24T06:46:36.824 INFO:tasks.workunit.client.0.smithi005.stdout:checking bstring.h presence... no 2024-04-24T06:46:36.825 INFO:tasks.workunit.client.0.smithi005.stdout:checking for bstring.h... no 2024-04-24T06:46:36.918 INFO:tasks.workunit.client.0.smithi005.stdout:checking libgen.h usability... yes 2024-04-24T06:46:36.962 INFO:tasks.workunit.client.0.smithi005.stdout:checking libgen.h presence... yes 2024-04-24T06:46:36.963 INFO:tasks.workunit.client.0.smithi005.stdout:checking for libgen.h... yes 2024-04-24T06:46:37.089 INFO:tasks.workunit.client.0.smithi005.stdout:checking dirent.h usability... yes 2024-04-24T06:46:37.132 INFO:tasks.workunit.client.0.smithi005.stdout:checking dirent.h presence... yes 2024-04-24T06:46:37.132 INFO:tasks.workunit.client.0.smithi005.stdout:checking for dirent.h... yes 2024-04-24T06:46:37.244 INFO:tasks.workunit.client.0.smithi005.stdout:checking errno.h usability... yes 2024-04-24T06:46:37.290 INFO:tasks.workunit.client.0.smithi005.stdout:checking errno.h presence... yes 2024-04-24T06:46:37.290 INFO:tasks.workunit.client.0.smithi005.stdout:checking for errno.h... yes 2024-04-24T06:46:37.403 INFO:tasks.workunit.client.0.smithi005.stdout:checking malloc.h usability... yes 2024-04-24T06:46:37.452 INFO:tasks.workunit.client.0.smithi005.stdout:checking malloc.h presence... yes 2024-04-24T06:46:37.452 INFO:tasks.workunit.client.0.smithi005.stdout:checking for malloc.h... yes 2024-04-24T06:46:37.570 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:37 smithi037 ceph-mon[32864]: pgmap v80: 97 pgs: 97 active+clean; 78 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 658 op/s 2024-04-24T06:46:37.571 INFO:tasks.workunit.client.0.smithi005.stdout:checking uuid.h usability... no 2024-04-24T06:46:37.645 INFO:tasks.workunit.client.0.smithi005.stdout:checking uuid.h presence... no 2024-04-24T06:46:37.645 INFO:tasks.workunit.client.0.smithi005.stdout:checking for uuid.h... no 2024-04-24T06:46:37.698 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:37 smithi137 ceph-mon[33017]: pgmap v80: 97 pgs: 97 active+clean; 78 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 658 op/s 2024-04-24T06:46:37.752 INFO:tasks.workunit.client.0.smithi005.stdout:checking uuid/uuid.h usability... yes 2024-04-24T06:46:37.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:37 smithi005 ceph-mon[27798]: pgmap v80: 97 pgs: 97 active+clean; 78 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 658 op/s 2024-04-24T06:46:37.799 INFO:tasks.workunit.client.0.smithi005.stdout:checking uuid/uuid.h presence... yes 2024-04-24T06:46:37.799 INFO:tasks.workunit.client.0.smithi005.stdout:checking for uuid/uuid.h... yes 2024-04-24T06:46:37.907 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/uuid.h usability... no 2024-04-24T06:46:37.973 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/uuid.h presence... no 2024-04-24T06:46:37.973 INFO:tasks.workunit.client.0.smithi005.stdout:checking for sys/uuid.h... no 2024-04-24T06:46:38.071 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/file.h usability... yes 2024-04-24T06:46:38.112 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/file.h presence... yes 2024-04-24T06:46:38.112 INFO:tasks.workunit.client.0.smithi005.stdout:checking for sys/file.h... yes 2024-04-24T06:46:38.225 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/fcntl.h usability... yes 2024-04-24T06:46:38.267 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/fcntl.h presence... yes 2024-04-24T06:46:38.267 INFO:tasks.workunit.client.0.smithi005.stdout:checking for sys/fcntl.h... yes 2024-04-24T06:46:38.390 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/syssgi.h usability... no 2024-04-24T06:46:38.461 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/syssgi.h presence... no 2024-04-24T06:46:38.461 INFO:tasks.workunit.client.0.smithi005.stdout:checking for sys/syssgi.h... no 2024-04-24T06:46:38.582 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/param.h usability... yes 2024-04-24T06:46:38.630 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/param.h presence... yes 2024-04-24T06:46:38.630 INFO:tasks.workunit.client.0.smithi005.stdout:checking for sys/param.h... yes 2024-04-24T06:46:38.660 INFO:tasks.workunit.client.0.smithi005.stdout:checking for sys/stat.h... (cached) yes 2024-04-24T06:46:38.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:38 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:46:38.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:38 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:46:38.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:38 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:46:38.755 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/statvfs.h usability... yes 2024-04-24T06:46:38.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:38 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:46:38.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:38 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:46:38.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:38 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:46:38.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:38 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:46:38.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:38 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:46:38.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:38 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:46:38.796 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/statvfs.h presence... yes 2024-04-24T06:46:38.796 INFO:tasks.workunit.client.0.smithi005.stdout:checking for sys/statvfs.h... yes 2024-04-24T06:46:38.900 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/time.h usability... yes 2024-04-24T06:46:38.940 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/time.h presence... yes 2024-04-24T06:46:38.940 INFO:tasks.workunit.client.0.smithi005.stdout:checking for sys/time.h... yes 2024-04-24T06:46:39.039 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/ioctl.h usability... yes 2024-04-24T06:46:39.080 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/ioctl.h presence... yes 2024-04-24T06:46:39.080 INFO:tasks.workunit.client.0.smithi005.stdout:checking for sys/ioctl.h... yes 2024-04-24T06:46:39.204 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/wait.h usability... yes 2024-04-24T06:46:39.248 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/wait.h presence... yes 2024-04-24T06:46:39.248 INFO:tasks.workunit.client.0.smithi005.stdout:checking for sys/wait.h... yes 2024-04-24T06:46:39.280 INFO:tasks.workunit.client.0.smithi005.stdout:checking for sys/types.h... (cached) yes 2024-04-24T06:46:39.298 INFO:tasks.workunit.client.0.smithi005.stdout:checking for strings.h... (cached) yes 2024-04-24T06:46:39.395 INFO:tasks.workunit.client.0.smithi005.stdout:checking err.h usability... yes 2024-04-24T06:46:39.434 INFO:tasks.workunit.client.0.smithi005.stdout:checking err.h presence... yes 2024-04-24T06:46:39.434 INFO:tasks.workunit.client.0.smithi005.stdout:checking for err.h... yes 2024-04-24T06:46:39.533 INFO:tasks.workunit.client.0.smithi005.stdout:checking linux/falloc.h usability... yes 2024-04-24T06:46:39.573 INFO:tasks.workunit.client.0.smithi005.stdout:checking linux/falloc.h presence... yes 2024-04-24T06:46:39.573 INFO:tasks.workunit.client.0.smithi005.stdout:checking for linux/falloc.h... yes 2024-04-24T06:46:39.679 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-04-24T06:46:39.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:39 smithi137 ceph-mon[33017]: pgmap v81: 97 pgs: 97 active+clean; 78 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 659 op/s 2024-04-24T06:46:39.740 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-04-24T06:46:39.740 INFO:tasks.workunit.client.0.smithi005.stdout:checking for sys/fs/xfs_fsops.h... no 2024-04-24T06:46:39.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:39 smithi005 ceph-mon[27798]: pgmap v81: 97 pgs: 97 active+clean; 78 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 659 op/s 2024-04-24T06:46:39.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:39 smithi037 ceph-mon[32864]: pgmap v81: 97 pgs: 97 active+clean; 78 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 659 op/s 2024-04-24T06:46:39.840 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/fs/xfs_itable.h usability... no 2024-04-24T06:46:39.906 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/fs/xfs_itable.h presence... no 2024-04-24T06:46:39.906 INFO:tasks.workunit.client.0.smithi005.stdout:checking for sys/fs/xfs_itable.h... no 2024-04-24T06:46:40.010 INFO:tasks.workunit.client.0.smithi005.stdout:checking xfs/platform_defs.h usability... no 2024-04-24T06:46:40.075 INFO:tasks.workunit.client.0.smithi005.stdout:checking xfs/platform_defs.h presence... no 2024-04-24T06:46:40.076 INFO:tasks.workunit.client.0.smithi005.stdout:checking for xfs/platform_defs.h... no 2024-04-24T06:46:40.176 INFO:tasks.workunit.client.0.smithi005.stdout:checking btrfs/ioctl.h usability... no 2024-04-24T06:46:40.240 INFO:tasks.workunit.client.0.smithi005.stdout:checking btrfs/ioctl.h presence... no 2024-04-24T06:46:40.240 INFO:tasks.workunit.client.0.smithi005.stdout:checking for btrfs/ioctl.h... no 2024-04-24T06:46:40.341 INFO:tasks.workunit.client.0.smithi005.stdout:checking cifs/ioctl.h usability... no 2024-04-24T06:46:40.404 INFO:tasks.workunit.client.0.smithi005.stdout:checking cifs/ioctl.h presence... no 2024-04-24T06:46:40.404 INFO:tasks.workunit.client.0.smithi005.stdout:checking for cifs/ioctl.h... no 2024-04-24T06:46:40.520 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/mman.h usability... yes 2024-04-24T06:46:40.563 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/mman.h presence... yes 2024-04-24T06:46:40.563 INFO:tasks.workunit.client.0.smithi005.stdout:checking for sys/mman.h... yes 2024-04-24T06:46:40.658 INFO:tasks.workunit.client.0.smithi005.stdout:checking for xfs/xfs_log_format.h... no 2024-04-24T06:46:40.787 INFO:tasks.workunit.client.0.smithi005.stdout:checking for uuid_compare... no 2024-04-24T06:46:40.885 INFO:tasks.workunit.client.0.smithi005.stdout:checking for uuid_compare in -luuid... yes 2024-04-24T06:46:40.995 INFO:tasks.workunit.client.0.smithi005.stdout:checking for xfs/xfs.h... yes 2024-04-24T06:46:41.084 INFO:tasks.workunit.client.0.smithi005.stdout:checking for xfs/libxfs.h... no 2024-04-24T06:46:41.174 INFO:tasks.workunit.client.0.smithi005.stdout:checking whether xlog_assign_lsn is declared... no 2024-04-24T06:46:41.266 INFO:tasks.workunit.client.0.smithi005.stdout:checking for xfs/xqm.h... yes 2024-04-24T06:46:41.394 INFO:tasks.workunit.client.0.smithi005.stdout:checking xfsctl from xfs/xfs.h... ok 2024-04-24T06:46:41.476 INFO:tasks.workunit.client.0.smithi005.stdout:checking xfs/handle.h usability... yes 2024-04-24T06:46:41.515 INFO:tasks.workunit.client.0.smithi005.stdout:checking xfs/handle.h presence... yes 2024-04-24T06:46:41.515 INFO:tasks.workunit.client.0.smithi005.stdout:checking for xfs/handle.h... yes 2024-04-24T06:46:41.637 INFO:tasks.workunit.client.0.smithi005.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-04-24T06:46:41.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:41 smithi137 ceph-mon[33017]: pgmap v82: 97 pgs: 97 active+clean; 79 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 651 op/s 2024-04-24T06:46:41.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:41 smithi005 ceph-mon[27798]: pgmap v82: 97 pgs: 97 active+clean; 79 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 651 op/s 2024-04-24T06:46:41.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:41 smithi037 ceph-mon[32864]: pgmap v82: 97 pgs: 97 active+clean; 79 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 651 op/s 2024-04-24T06:46:41.786 INFO:tasks.workunit.client.0.smithi005.stdout:checking attr/attributes.h usability... yes 2024-04-24T06:46:41.826 INFO:tasks.workunit.client.0.smithi005.stdout:checking attr/attributes.h presence... yes 2024-04-24T06:46:41.826 INFO:tasks.workunit.client.0.smithi005.stdout:checking for attr/attributes.h... yes 2024-04-24T06:46:41.938 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/attributes.h usability... no 2024-04-24T06:46:42.005 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/attributes.h presence... no 2024-04-24T06:46:42.005 INFO:tasks.workunit.client.0.smithi005.stdout:checking for sys/attributes.h... no 2024-04-24T06:46:42.109 INFO:tasks.workunit.client.0.smithi005.stdout:checking for attr_list in -lattr... yes 2024-04-24T06:46:42.208 INFO:tasks.workunit.client.0.smithi005.stdout:checking for attr_set in -lattr... yes 2024-04-24T06:46:42.293 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/acl.h usability... yes 2024-04-24T06:46:42.335 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/acl.h presence... yes 2024-04-24T06:46:42.335 INFO:tasks.workunit.client.0.smithi005.stdout:checking for sys/acl.h... yes 2024-04-24T06:46:42.426 INFO:tasks.workunit.client.0.smithi005.stdout:checking acl/libacl.h usability... yes 2024-04-24T06:46:42.470 INFO:tasks.workunit.client.0.smithi005.stdout:checking acl/libacl.h presence... yes 2024-04-24T06:46:42.470 INFO:tasks.workunit.client.0.smithi005.stdout:checking for acl/libacl.h... yes 2024-04-24T06:46:42.577 INFO:tasks.workunit.client.0.smithi005.stdout:checking for acl_init in -lacl... yes 2024-04-24T06:46:42.701 INFO:tasks.workunit.client.0.smithi005.stdout:checking gdbm-ndbm.h usability... no 2024-04-24T06:46:42.771 INFO:tasks.workunit.client.0.smithi005.stdout:checking gdbm-ndbm.h presence... no 2024-04-24T06:46:42.771 INFO:tasks.workunit.client.0.smithi005.stdout:checking for gdbm-ndbm.h... no 2024-04-24T06:46:42.861 INFO:tasks.workunit.client.0.smithi005.stdout:checking gdbm/ndbm.h usability... no 2024-04-24T06:46:42.928 INFO:tasks.workunit.client.0.smithi005.stdout:checking gdbm/ndbm.h presence... no 2024-04-24T06:46:42.928 INFO:tasks.workunit.client.0.smithi005.stdout:checking for gdbm/ndbm.h... no 2024-04-24T06:46:43.014 INFO:tasks.workunit.client.0.smithi005.stdout:checking ndbm.h usability... no 2024-04-24T06:46:43.077 INFO:tasks.workunit.client.0.smithi005.stdout:checking ndbm.h presence... no 2024-04-24T06:46:43.077 INFO:tasks.workunit.client.0.smithi005.stdout:checking for ndbm.h... no 2024-04-24T06:46:43.157 INFO:tasks.workunit.client.0.smithi005.stdout:checking libaio.h usability... yes 2024-04-24T06:46:43.202 INFO:tasks.workunit.client.0.smithi005.stdout:checking libaio.h presence... yes 2024-04-24T06:46:43.202 INFO:tasks.workunit.client.0.smithi005.stdout:checking for libaio.h... yes 2024-04-24T06:46:43.320 INFO:tasks.workunit.client.0.smithi005.stdout:checking for dm_make_handle in -ldm... no 2024-04-24T06:46:43.320 INFO:tasks.workunit.client.0.smithi005.stdout: 2024-04-24T06:46:43.320 INFO:tasks.workunit.client.0.smithi005.stdout:WARNING: could not find a valid DMAPI base library. 2024-04-24T06:46:43.320 INFO:tasks.workunit.client.0.smithi005.stdout:If you want DMAPI support please install the data migration 2024-04-24T06:46:43.321 INFO:tasks.workunit.client.0.smithi005.stdout:API (dmapi) library package. Alternatively, run "make install" 2024-04-24T06:46:43.321 INFO:tasks.workunit.client.0.smithi005.stdout:from the dmapi source. 2024-04-24T06:46:43.321 INFO:tasks.workunit.client.0.smithi005.stdout: 2024-04-24T06:46:43.406 INFO:tasks.workunit.client.0.smithi005.stdout:checking linux/fiemap.h usability... yes 2024-04-24T06:46:43.446 INFO:tasks.workunit.client.0.smithi005.stdout:checking linux/fiemap.h presence... yes 2024-04-24T06:46:43.446 INFO:tasks.workunit.client.0.smithi005.stdout:checking for linux/fiemap.h... yes 2024-04-24T06:46:43.546 INFO:tasks.workunit.client.0.smithi005.stdout:checking for fallocate... yes 2024-04-24T06:46:43.658 INFO:tasks.workunit.client.0.smithi005.stdout:checking for open_by_handle_at... yes 2024-04-24T06:46:43.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:43 smithi137 ceph-mon[33017]: pgmap v83: 97 pgs: 97 active+clean; 79 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 408 op/s 2024-04-24T06:46:43.744 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/prctl.h usability... yes 2024-04-24T06:46:43.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:43 smithi005 ceph-mon[27798]: pgmap v83: 97 pgs: 97 active+clean; 79 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 408 op/s 2024-04-24T06:46:43.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:43 smithi037 ceph-mon[32864]: pgmap v83: 97 pgs: 97 active+clean; 79 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 408 op/s 2024-04-24T06:46:43.785 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/prctl.h presence... yes 2024-04-24T06:46:43.785 INFO:tasks.workunit.client.0.smithi005.stdout:checking for sys/prctl.h... yes 2024-04-24T06:46:43.865 INFO:tasks.workunit.client.0.smithi005.stdout:checking linux/fs.h usability... yes 2024-04-24T06:46:43.905 INFO:tasks.workunit.client.0.smithi005.stdout:checking linux/fs.h presence... yes 2024-04-24T06:46:43.905 INFO:tasks.workunit.client.0.smithi005.stdout:checking for linux/fs.h... yes 2024-04-24T06:46:44.039 INFO:tasks.workunit.client.0.smithi005.stdout:checking btrfsutil.h usability... no 2024-04-24T06:46:44.104 INFO:tasks.workunit.client.0.smithi005.stdout:checking btrfsutil.h presence... no 2024-04-24T06:46:44.104 INFO:tasks.workunit.client.0.smithi005.stdout:checking for btrfsutil.h... no 2024-04-24T06:46:44.217 INFO:tasks.workunit.client.0.smithi005.stdout:checking for copy_file_range... yes 2024-04-24T06:46:44.324 INFO:tasks.workunit.client.0.smithi005.stdout:checking for renameat2... yes 2024-04-24T06:46:44.369 INFO:tasks.workunit.client.0.smithi005.stdout:configure: creating ./config.status 2024-04-24T06:46:45.676 INFO:tasks.workunit.client.0.smithi005.stdout:config.status: creating include/builddefs 2024-04-24T06:46:45.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:45 smithi137 ceph-mon[33017]: pgmap v84: 97 pgs: 97 active+clean; 81 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 417 op/s 2024-04-24T06:46:45.726 INFO:tasks.workunit.client.0.smithi005.stdout:config.status: creating include/config.h 2024-04-24T06:46:45.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:45 smithi005 ceph-mon[27798]: pgmap v84: 97 pgs: 97 active+clean; 81 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 417 op/s 2024-04-24T06:46:45.764 INFO:tasks.workunit.client.0.smithi005.stdout:config.status: executing libtool commands 2024-04-24T06:46:45.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:45 smithi037 ceph-mon[32864]: pgmap v84: 97 pgs: 97 active+clean; 81 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 417 op/s 2024-04-24T06:46:45.936 INFO:tasks.workunit.client.0.smithi005.stdout:Building include 2024-04-24T06:46:45.937 INFO:tasks.workunit.client.0.smithi005.stdout:Building lib 2024-04-24T06:46:45.938 INFO:tasks.workunit.client.0.smithi005.stdout:Building tests 2024-04-24T06:46:45.949 INFO:tasks.workunit.client.0.smithi005.stdout: [LTDEP] 2024-04-24T06:46:46.411 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] dataascii.lo 2024-04-24T06:46:46.412 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] databin.lo 2024-04-24T06:46:46.413 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] datapid.lo 2024-04-24T06:46:46.414 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] file_lock.lo 2024-04-24T06:46:46.734 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] forker.lo 2024-04-24T06:46:46.742 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] pattern.lo 2024-04-24T06:46:46.755 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] open_flags.lo 2024-04-24T06:46:46.781 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] random_range.lo 2024-04-24T06:46:47.041 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] string_to_tokens.lo 2024-04-24T06:46:47.104 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] tlibio.lo 2024-04-24T06:46:47.104 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] str_to_bytes.lo 2024-04-24T06:46:47.212 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] write_log.lo 2024-04-24T06:46:47.320 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] random.lo 2024-04-24T06:46:47.687 INFO:tasks.workunit.client.0.smithi005.stdout: [LD] 2024-04-24T06:46:47.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:47 smithi137 ceph-mon[33017]: pgmap v85: 97 pgs: 97 active+clean; 81 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 311 KiB/s wr, 21 op/s 2024-04-24T06:46:47.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:47 smithi005 ceph-mon[27798]: pgmap v85: 97 pgs: 97 active+clean; 81 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 311 KiB/s wr, 21 op/s 2024-04-24T06:46:47.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:47 smithi037 ceph-mon[32864]: pgmap v85: 97 pgs: 97 active+clean; 81 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 311 KiB/s wr, 21 op/s 2024-04-24T06:46:48.141 INFO:tasks.workunit.client.0.smithi005.stdout:Building ltp 2024-04-24T06:46:48.142 INFO:tasks.workunit.client.0.smithi005.stdout:Building m4 2024-04-24T06:46:48.142 INFO:tasks.workunit.client.0.smithi005.stdout:Building src 2024-04-24T06:46:48.143 INFO:tasks.workunit.client.0.smithi005.stdout:Building common 2024-04-24T06:46:48.156 INFO:tasks.workunit.client.0.smithi005.stdout: [DEP] 2024-04-24T06:46:48.183 INFO:tasks.workunit.client.0.smithi005.stdout: [DEP] 2024-04-24T06:46:48.540 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] doio 2024-04-24T06:46:48.540 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] fsstress 2024-04-24T06:46:48.541 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] fsx 2024-04-24T06:46:48.823 INFO:tasks.workunit.client.0.smithi005.stderr:fsstress.c: In function 'attr_list_path': 2024-04-24T06:46:48.823 INFO:tasks.workunit.client.0.smithi005.stderr:fsstress.c:859:9: warning: 'attr_list' is deprecated: Use listxattr or llistxattr instead [-Wdeprecated-declarations] 2024-04-24T06:46:48.823 INFO:tasks.workunit.client.0.smithi005.stderr: 859 | rval = attr_list(name->path, buffer, buffersize, flags, cursor); 2024-04-24T06:46:48.823 INFO:tasks.workunit.client.0.smithi005.stderr: | ^~~~ 2024-04-24T06:46:48.823 INFO:tasks.workunit.client.0.smithi005.stderr:In file included from ../src/global.h:25, 2024-04-24T06:46:48.823 INFO:tasks.workunit.client.0.smithi005.stderr: from fsstress.c:12: 2024-04-24T06:46:48.824 INFO:tasks.workunit.client.0.smithi005.stderr:/usr/include/attr/attributes.h:165:12: note: declared here 2024-04-24T06:46:48.824 INFO:tasks.workunit.client.0.smithi005.stderr: 165 | extern int attr_list(const char *__path, char *__buffer, const int __buffersize, 2024-04-24T06:46:48.824 INFO:tasks.workunit.client.0.smithi005.stderr: | ^~~~~~~~~ 2024-04-24T06:46:48.824 INFO:tasks.workunit.client.0.smithi005.stderr:fsstress.c: In function 'attr_remove_path': 2024-04-24T06:46:48.824 INFO:tasks.workunit.client.0.smithi005.stderr:fsstress.c:878:9: warning: 'attr_remove' is deprecated: Use removexattr or lremovexattr instead [-Wdeprecated-declarations] 2024-04-24T06:46:48.824 INFO:tasks.workunit.client.0.smithi005.stderr: 878 | rval = attr_remove(name->path, attrname, flags); 2024-04-24T06:46:48.824 INFO:tasks.workunit.client.0.smithi005.stderr: | ^~~~ 2024-04-24T06:46:48.824 INFO:tasks.workunit.client.0.smithi005.stderr:In file included from ../src/global.h:25, 2024-04-24T06:46:48.824 INFO:tasks.workunit.client.0.smithi005.stderr: from fsstress.c:12: 2024-04-24T06:46:48.824 INFO:tasks.workunit.client.0.smithi005.stderr:/usr/include/attr/attributes.h:152:12: note: declared here 2024-04-24T06:46:48.824 INFO:tasks.workunit.client.0.smithi005.stderr: 152 | extern int attr_remove (const char *__path, const char *__attrname, 2024-04-24T06:46:48.824 INFO:tasks.workunit.client.0.smithi005.stderr: | ^~~~~~~~~~~ 2024-04-24T06:46:48.824 INFO:tasks.workunit.client.0.smithi005.stderr:fsstress.c: In function 'attr_set_path': 2024-04-24T06:46:48.825 INFO:tasks.workunit.client.0.smithi005.stderr:fsstress.c:898:9: warning: 'attr_set' is deprecated: Use setxattr or lsetxattr instead [-Wdeprecated-declarations] 2024-04-24T06:46:48.825 INFO:tasks.workunit.client.0.smithi005.stderr: 898 | rval = attr_set(name->path, attrname, attrvalue, valuelength, flags); 2024-04-24T06:46:48.825 INFO:tasks.workunit.client.0.smithi005.stderr: | ^~~~ 2024-04-24T06:46:48.825 INFO:tasks.workunit.client.0.smithi005.stderr:In file included from ../src/global.h:25, 2024-04-24T06:46:48.825 INFO:tasks.workunit.client.0.smithi005.stderr: from fsstress.c:12: 2024-04-24T06:46:48.825 INFO:tasks.workunit.client.0.smithi005.stderr:/usr/include/attr/attributes.h:139:12: note: declared here 2024-04-24T06:46:48.825 INFO:tasks.workunit.client.0.smithi005.stderr: 139 | extern int attr_set (const char *__path, const char *__attrname, 2024-04-24T06:46:48.825 INFO:tasks.workunit.client.0.smithi005.stderr: | ^~~~~~~~ 2024-04-24T06:46:48.836 INFO:tasks.workunit.client.0.smithi005.stderr:fsstress.c: In function 'allocsp_f': 2024-04-24T06:46:48.836 INFO:tasks.workunit.client.0.smithi005.stderr:fsstress.c:2047:32: error: 'XFS_IOC_ALLOCSP64' undeclared (first use in this function); did you mean 'XFS_IOC_RESVSP64'? 2024-04-24T06:46:48.836 INFO:tasks.workunit.client.0.smithi005.stderr: 2047 | e = xfsctl(f.path, fd, XFS_IOC_ALLOCSP64, &fl) < 0 ? errno : 0; 2024-04-24T06:46:48.836 INFO:tasks.workunit.client.0.smithi005.stderr: | ^~~~~~~~~~~~~~~~~ 2024-04-24T06:46:48.836 INFO:tasks.workunit.client.0.smithi005.stderr: | XFS_IOC_RESVSP64 2024-04-24T06:46:48.836 INFO:tasks.workunit.client.0.smithi005.stderr:fsstress.c:2047:32: note: each undeclared identifier is reported only once for each function it appears in 2024-04-24T06:46:48.856 INFO:tasks.workunit.client.0.smithi005.stderr:fsstress.c: In function 'freesp_f': 2024-04-24T06:46:48.856 INFO:tasks.workunit.client.0.smithi005.stderr:fsstress.c:3640:32: error: 'XFS_IOC_FREESP64' undeclared (first use in this function); did you mean 'XFS_IOC_RESVSP64'? 2024-04-24T06:46:48.856 INFO:tasks.workunit.client.0.smithi005.stderr: 3640 | e = xfsctl(f.path, fd, XFS_IOC_FREESP64, &fl) < 0 ? errno : 0; 2024-04-24T06:46:48.857 INFO:tasks.workunit.client.0.smithi005.stderr: | ^~~~~~~~~~~~~~~~ 2024-04-24T06:46:48.857 INFO:tasks.workunit.client.0.smithi005.stderr: | XFS_IOC_RESVSP64 2024-04-24T06:46:48.961 INFO:tasks.workunit.client.0.smithi005.stderr:gmake[3]: *** [Makefile:47: fsstress] Error 1 2024-04-24T06:46:48.961 INFO:tasks.workunit.client.0.smithi005.stderr:gmake[3]: *** Waiting for unfinished jobs.... 2024-04-24T06:46:49.378 INFO:tasks.workunit.client.0.smithi005.stderr:doio.c: In function 'parse_cmdline': 2024-04-24T06:46:49.378 INFO:tasks.workunit.client.0.smithi005.stderr:doio.c:3113:33: warning: '%s' directive argument is null [-Wformat-overflow=] 2024-04-24T06:46:49.379 INFO:tasks.workunit.client.0.smithi005.stderr: 3113 | fprintf(stderr, 2024-04-24T06:46:49.379 INFO:tasks.workunit.client.0.smithi005.stderr: | ^~~~~~~~~~~~~~~ 2024-04-24T06:46:49.379 INFO:tasks.workunit.client.0.smithi005.stderr: 3114 | "%s%s: Illegal -C arg (%s). Must be one of: ", 2024-04-24T06:46:49.379 INFO:tasks.workunit.client.0.smithi005.stderr: | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-24T06:46:49.379 INFO:tasks.workunit.client.0.smithi005.stderr: 3115 | Prog, TagName, tok); 2024-04-24T06:46:49.379 INFO:tasks.workunit.client.0.smithi005.stderr: | ~~~~~~~~~~~~~~~~~~~ 2024-04-24T06:46:49.413 INFO:tasks.workunit.client.0.smithi005.stderr:fsx.c: In function 'main': 2024-04-24T06:46:49.413 INFO:tasks.workunit.client.0.smithi005.stderr:fsx.c:2662:33: warning: 'strncpy' specified bound 4096 equals destination size [-Wstringop-truncation] 2024-04-24T06:46:49.413 INFO:tasks.workunit.client.0.smithi005.stderr: 2662 | strncpy(opsfile, optarg, sizeof(opsfile)); 2024-04-24T06:46:49.413 INFO:tasks.workunit.client.0.smithi005.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-24T06:46:49.413 INFO:tasks.workunit.client.0.smithi005.stderr:fsx.c:2632:25: warning: 'strncpy' specified bound 1024 equals destination size [-Wstringop-truncation] 2024-04-24T06:46:49.413 INFO:tasks.workunit.client.0.smithi005.stderr: 2632 | strncpy(dname, optarg, sizeof(dname)); 2024-04-24T06:46:49.413 INFO:tasks.workunit.client.0.smithi005.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-24T06:46:49.553 INFO:tasks.workunit.client.0.smithi005.stderr:gmake[2]: *** [include/buildrules:30: ltp] Error 2 2024-04-24T06:46:49.553 INFO:tasks.workunit.client.0.smithi005.stderr:gmake[2]: *** Waiting for unfinished jobs.... 2024-04-24T06:46:49.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:49 smithi137 ceph-mon[33017]: pgmap v86: 97 pgs: 97 active+clean; 82 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 383 KiB/s wr, 26 op/s 2024-04-24T06:46:49.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:49 smithi005 ceph-mon[27798]: pgmap v86: 97 pgs: 97 active+clean; 82 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 383 KiB/s wr, 26 op/s 2024-04-24T06:46:49.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:49 smithi037 ceph-mon[32864]: pgmap v86: 97 pgs: 97 active+clean; 82 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 383 KiB/s wr, 26 op/s 2024-04-24T06:46:51.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:51 smithi137 ceph-mon[33017]: pgmap v87: 97 pgs: 97 active+clean; 82 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 396 KiB/s wr, 32 op/s 2024-04-24T06:46:51.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:51 smithi005 ceph-mon[27798]: pgmap v87: 97 pgs: 97 active+clean; 82 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 396 KiB/s wr, 32 op/s 2024-04-24T06:46:51.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:51 smithi037 ceph-mon[32864]: pgmap v87: 97 pgs: 97 active+clean; 82 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 396 KiB/s wr, 32 op/s 2024-04-24T06:46:51.789 INFO:tasks.workunit.client.0.smithi005.stdout: [DEP] 2024-04-24T06:46:51.789 INFO:tasks.workunit.client.0.smithi005.stdout: [DEP] 2024-04-24T06:46:51.790 INFO:tasks.workunit.client.0.smithi005.stdout: [DEP] 2024-04-24T06:46:52.352 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] dirstress 2024-04-24T06:46:52.358 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] fill 2024-04-24T06:46:52.359 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] fill2 2024-04-24T06:46:52.360 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] getpagesize 2024-04-24T06:46:52.636 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] holes 2024-04-24T06:46:52.659 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] lstat64 2024-04-24T06:46:52.782 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] nametest 2024-04-24T06:46:52.786 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] permname 2024-04-24T06:46:53.021 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] randholes 2024-04-24T06:46:53.041 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] runas 2024-04-24T06:46:53.151 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] truncfile 2024-04-24T06:46:53.233 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] usemem 2024-04-24T06:46:53.383 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] mmapcat 2024-04-24T06:46:53.435 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] append_reader 2024-04-24T06:46:53.476 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] append_writer 2024-04-24T06:46:53.532 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] dirperf 2024-04-24T06:46:53.683 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] metaperf 2024-04-24T06:46:53.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:53 smithi137 ceph-mon[33017]: pgmap v88: 97 pgs: 97 active+clean; 82 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 250 KiB/s wr, 21 op/s 2024-04-24T06:46:53.734 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] devzero 2024-04-24T06:46:53.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:53 smithi005 ceph-mon[27798]: pgmap v88: 97 pgs: 97 active+clean; 82 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 250 KiB/s wr, 21 op/s 2024-04-24T06:46:53.757 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] feature 2024-04-24T06:46:53.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:53 smithi037 ceph-mon[32864]: pgmap v88: 97 pgs: 97 active+clean; 82 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 250 KiB/s wr, 21 op/s 2024-04-24T06:46:53.920 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] alloc 2024-04-24T06:46:54.066 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] fault 2024-04-24T06:46:54.122 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] fstest 2024-04-24T06:46:54.137 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] t_access_root 2024-04-24T06:46:54.212 INFO:tasks.workunit.client.0.smithi005.stderr:alloc.c: In function 'main': 2024-04-24T06:46:54.212 INFO:tasks.workunit.client.0.smithi005.stderr:alloc.c:154:1: error: 'bozo' undeclared (first use in this function) 2024-04-24T06:46:54.212 INFO:tasks.workunit.client.0.smithi005.stderr: 154 | bozo! 2024-04-24T06:46:54.212 INFO:tasks.workunit.client.0.smithi005.stderr: | ^~~~ 2024-04-24T06:46:54.213 INFO:tasks.workunit.client.0.smithi005.stderr:alloc.c:154:1: note: each undeclared identifier is reported only once for each function it appears in 2024-04-24T06:46:54.213 INFO:tasks.workunit.client.0.smithi005.stderr:alloc.c:154:5: error: expected ';' before '!' token 2024-04-24T06:46:54.213 INFO:tasks.workunit.client.0.smithi005.stderr: 154 | bozo! 2024-04-24T06:46:54.213 INFO:tasks.workunit.client.0.smithi005.stderr: | ^ 2024-04-24T06:46:54.213 INFO:tasks.workunit.client.0.smithi005.stderr: | ; 2024-04-24T06:46:54.213 INFO:tasks.workunit.client.0.smithi005.stderr:alloc.c:184:25: error: 'rflag' undeclared (first use in this function); did you mean 'nflag'? 2024-04-24T06:46:54.213 INFO:tasks.workunit.client.0.smithi005.stderr: 184 | rflag = 1; 2024-04-24T06:46:54.213 INFO:tasks.workunit.client.0.smithi005.stderr: | ^~~~~ 2024-04-24T06:46:54.213 INFO:tasks.workunit.client.0.smithi005.stderr: | nflag 2024-04-24T06:46:54.214 INFO:tasks.workunit.client.0.smithi005.stderr:alloc.c:320:5: error: expected ';' before '!' token 2024-04-24T06:46:54.214 INFO:tasks.workunit.client.0.smithi005.stderr: 320 | bozo! 2024-04-24T06:46:54.214 INFO:tasks.workunit.client.0.smithi005.stderr: | ^ 2024-04-24T06:46:54.214 INFO:tasks.workunit.client.0.smithi005.stderr: | ; 2024-04-24T06:46:54.214 INFO:tasks.workunit.client.0.smithi005.stderr:alloc.c:128:25: warning: variable 'f' set but not used [-Wunused-but-set-variable] 2024-04-24T06:46:54.214 INFO:tasks.workunit.client.0.smithi005.stderr: 128 | struct flock64 f; 2024-04-24T06:46:54.214 INFO:tasks.workunit.client.0.smithi005.stderr: | ^ 2024-04-24T06:46:54.220 INFO:tasks.workunit.client.0.smithi005.stderr:gmake[3]: *** [Makefile:83: alloc] Error 1 2024-04-24T06:46:54.220 INFO:tasks.workunit.client.0.smithi005.stderr:gmake[3]: *** Waiting for unfinished jobs.... 2024-04-24T06:46:54.388 INFO:tasks.workunit.client.0.smithi005.stderr:fstest.c: In function 'run_child': 2024-04-24T06:46:54.388 INFO:tasks.workunit.client.0.smithi005.stderr:fstest.c:95:27: warning: '/file' directive writing 5 bytes into a region of size between 1 and 1024 [-Wformat-overflow=] 2024-04-24T06:46:54.388 INFO:tasks.workunit.client.0.smithi005.stderr: 95 | sprintf(fname, "%s/file%d", dir, fnum); 2024-04-24T06:46:54.388 INFO:tasks.workunit.client.0.smithi005.stderr: | ^~~~~ 2024-04-24T06:46:54.388 INFO:tasks.workunit.client.0.smithi005.stderr:fstest.c:95:24: note: directive argument in the range [0, 2147483647] 2024-04-24T06:46:54.388 INFO:tasks.workunit.client.0.smithi005.stderr: 95 | sprintf(fname, "%s/file%d", dir, fnum); 2024-04-24T06:46:54.388 INFO:tasks.workunit.client.0.smithi005.stderr: | ^~~~~~~~~~~ 2024-04-24T06:46:54.389 INFO:tasks.workunit.client.0.smithi005.stderr:fstest.c:95:9: note: 'sprintf' output between 7 and 1039 bytes into a destination of size 1024 2024-04-24T06:46:54.389 INFO:tasks.workunit.client.0.smithi005.stderr: 95 | sprintf(fname, "%s/file%d", dir, fnum); 2024-04-24T06:46:54.389 INFO:tasks.workunit.client.0.smithi005.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-24T06:46:54.389 INFO:tasks.workunit.client.0.smithi005.stderr:fstest.c:166:27: warning: '/file' directive writing 5 bytes into a region of size between 1 and 1024 [-Wformat-overflow=] 2024-04-24T06:46:54.389 INFO:tasks.workunit.client.0.smithi005.stderr: 166 | sprintf(fname, "%s/file%d", dir, fnum); 2024-04-24T06:46:54.389 INFO:tasks.workunit.client.0.smithi005.stderr: | ^~~~~ 2024-04-24T06:46:54.389 INFO:tasks.workunit.client.0.smithi005.stderr:fstest.c:166:24: note: directive argument in the range [0, 2147483647] 2024-04-24T06:46:54.389 INFO:tasks.workunit.client.0.smithi005.stderr: 166 | sprintf(fname, "%s/file%d", dir, fnum); 2024-04-24T06:46:54.389 INFO:tasks.workunit.client.0.smithi005.stderr: | ^~~~~~~~~~~ 2024-04-24T06:46:54.389 INFO:tasks.workunit.client.0.smithi005.stderr:fstest.c:166:9: note: 'sprintf' output between 7 and 1039 bytes into a destination of size 1024 2024-04-24T06:46:54.389 INFO:tasks.workunit.client.0.smithi005.stderr: 166 | sprintf(fname, "%s/file%d", dir, fnum); 2024-04-24T06:46:54.389 INFO:tasks.workunit.client.0.smithi005.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-24T06:46:54.497 INFO:tasks.workunit.client.0.smithi005.stderr:gmake[2]: *** [include/buildrules:30: src] Error 2 2024-04-24T06:46:54.498 INFO:tasks.workunit.client.0.smithi005.stderr:make[1]: *** [Makefile:53: default] Error 2 2024-04-24T06:46:54.498 INFO:tasks.workunit.client.0.smithi005.stderr:make: *** [Makefile:51: default] Error 2 2024-04-24T06:46:54.500 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-24T06:46:54.501 INFO:tasks.workunit:Stopping ['suites/fsx.sh'] on client.0... 2024-04-24T06:46:54.501 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-24T06:46:54.991 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T06:46:54.991 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T06:46:55.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.194+0000 7f1b16400640 1 -- 172.21.15.5:0/1876968810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b10104450 msgr2=0x7f1b10104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:55.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.194+0000 7f1b16400640 1 --2- 172.21.15.5:0/1876968810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b10104450 0x7f1b10104830 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1b00009860 tx=0x7f1b0002a340 comp rx=0 tx=0).stop 2024-04-24T06:46:55.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.195+0000 7f1b16400640 1 -- 172.21.15.5:0/1876968810 shutdown_connections 2024-04-24T06:46:55.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.195+0000 7f1b16400640 1 --2- 172.21.15.5:0/1876968810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b10109520 0x7f1b101110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.195+0000 7f1b16400640 1 --2- 172.21.15.5:0/1876968810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b10104e00 0x7f1b10108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.195+0000 7f1b16400640 1 --2- 172.21.15.5:0/1876968810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b10104450 0x7f1b10104830 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.195+0000 7f1b16400640 1 -- 172.21.15.5:0/1876968810 >> 172.21.15.5:0/1876968810 conn(0x7f1b100ffc90 msgr2=0x7f1b101020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:55.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.195+0000 7f1b16400640 1 -- 172.21.15.5:0/1876968810 shutdown_connections 2024-04-24T06:46:55.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.195+0000 7f1b16400640 1 -- 172.21.15.5:0/1876968810 wait complete. 2024-04-24T06:46:55.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.196+0000 7f1b16400640 1 Processor -- start 2024-04-24T06:46:55.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.196+0000 7f1b16400640 1 -- start start 2024-04-24T06:46:55.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.197+0000 7f1b16400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b10104450 0x7f1b101a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:55.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.197+0000 7f1b16400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b10109520 0x7f1b101a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:55.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.197+0000 7f1b16400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b1019a950 0x7f1b1019ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:55.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.197+0000 7f1b16400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1b101145b0 con 0x7f1b10104450 2024-04-24T06:46:55.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.197+0000 7f1b16400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1b10114430 con 0x7f1b1019a950 2024-04-24T06:46:55.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.197+0000 7f1b16400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1b10114730 con 0x7f1b10109520 2024-04-24T06:46:55.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.197+0000 7f1b0f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b10104450 0x7f1b101a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:55.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.197+0000 7f1b0f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b10104450 0x7f1b101a0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39560/0 (socket says 172.21.15.5:39560) 2024-04-24T06:46:55.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.197+0000 7f1b0f400640 1 -- 172.21.15.5:0/2498086420 learned_addr learned my addr 172.21.15.5:0/2498086420 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:55.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.197+0000 7f1b0ea00640 1 --2- 172.21.15.5:0/2498086420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b10109520 0x7f1b101a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:55.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.198+0000 7f1b0fe00640 1 --2- 172.21.15.5:0/2498086420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b1019a950 0x7f1b1019ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:55.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.198+0000 7f1b0f400640 1 -- 172.21.15.5:0/2498086420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b10109520 msgr2=0x7f1b101a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:55.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.198+0000 7f1b0f400640 1 --2- 172.21.15.5:0/2498086420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b10109520 0x7f1b101a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.198+0000 7f1b0f400640 1 -- 172.21.15.5:0/2498086420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b1019a950 msgr2=0x7f1b1019ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:55.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.198+0000 7f1b0f400640 1 --2- 172.21.15.5:0/2498086420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b1019a950 0x7f1b1019ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.198+0000 7f1b0f400640 1 -- 172.21.15.5:0/2498086420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b1019b470 con 0x7f1b10104450 2024-04-24T06:46:55.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.198+0000 7f1b0f400640 1 --2- 172.21.15.5:0/2498086420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b10104450 0x7f1b101a0750 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1b00035040 tx=0x7f1b00033b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:55.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.198+0000 7f1b07e00640 1 -- 172.21.15.5:0/2498086420 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b00037040 con 0x7f1b10104450 2024-04-24T06:46:55.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.198+0000 7f1b07e00640 1 -- 172.21.15.5:0/2498086420 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b00040070 con 0x7f1b10104450 2024-04-24T06:46:55.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.198+0000 7f1b16400640 1 -- 172.21.15.5:0/2498086420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b1019b700 con 0x7f1b10104450 2024-04-24T06:46:55.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.198+0000 7f1b07e00640 1 -- 172.21.15.5:0/2498086420 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b000039b0 con 0x7f1b10104450 2024-04-24T06:46:55.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.198+0000 7f1b16400640 1 -- 172.21.15.5:0/2498086420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b101a78a0 con 0x7f1b10104450 2024-04-24T06:46:55.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.202+0000 7f1b191c7740 1 Processor -- start 2024-04-24T06:46:55.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.202+0000 7f1b07e00640 1 -- 172.21.15.5:0/2498086420 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f1b000305e0 con 0x7f1b10104450 2024-04-24T06:46:55.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.202+0000 7f1b191c7740 1 -- start start 2024-04-24T06:46:55.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.202+0000 7f1b191c7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598f67a3860 0x5598f67a3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:55.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.202+0000 7f1b191c7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598f67a4180 0x5598f67ae310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:55.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.202+0000 7f1b191c7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598f67ae850 0x5598f67b0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:55.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.202+0000 7f1b191c7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5598f67091f0 con 0x5598f67a3860 2024-04-24T06:46:55.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.202+0000 7f1b191c7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5598f66af350 con 0x5598f67ae850 2024-04-24T06:46:55.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.202+0000 7f1b191c7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5598f67051b0 con 0x5598f67a4180 2024-04-24T06:46:55.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.202+0000 7f1b0fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598f67a3860 0x5598f67a3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:55.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.202+0000 7f1b0f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598f67a4180 0x5598f67ae310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:55.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.202+0000 7f1b0fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598f67a3860 0x5598f67a3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39568/0 (socket says 172.21.15.5:39568) 2024-04-24T06:46:55.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.202+0000 7f1b0fe00640 1 -- 172.21.15.5:0/1630676739 learned_addr learned my addr 172.21.15.5:0/1630676739 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:55.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.202+0000 7f1b0ea00640 1 --2- 172.21.15.5:0/1630676739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598f67ae850 0x5598f67b0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:55.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.203+0000 7f1b0fe00640 1 -- 172.21.15.5:0/1630676739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598f67a4180 msgr2=0x5598f67ae310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:55.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.203+0000 7f1b0fe00640 1 --2- 172.21.15.5:0/1630676739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598f67a4180 0x5598f67ae310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.203+0000 7f1b0fe00640 1 -- 172.21.15.5:0/1630676739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598f67ae850 msgr2=0x5598f67b0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:55.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.203+0000 7f1b0fe00640 1 --2- 172.21.15.5:0/1630676739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598f67ae850 0x5598f67b0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.203+0000 7f1b0fe00640 1 -- 172.21.15.5:0/1630676739 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598f6703f50 con 0x5598f67a3860 2024-04-24T06:46:55.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.203+0000 7f1b0fe00640 1 --2- 172.21.15.5:0/1630676739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598f67a3860 0x5598f67a3c40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1b08009ee0 tx=0x7f1b0800f7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7175dbdb9b949c93 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:55.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.203+0000 7f1b07e00640 1 --2- 172.21.15.5:0/2498086420 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1af40a3d20 0x7f1af40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:55.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.203+0000 7f1b07e00640 1 -- 172.21.15.5:0/2498086420 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f1b000f0850 con 0x7f1b10104450 2024-04-24T06:46:55.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.203+0000 7f1b04c00640 1 -- 172.21.15.5:0/1630676739 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0801d070 con 0x5598f67a3860 2024-04-24T06:46:55.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.204+0000 7f1b0f400640 1 --2- 172.21.15.5:0/2498086420 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1af40a3d20 0x7f1af40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:55.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.204+0000 7f1b04c00640 1 -- 172.21.15.5:0/1630676739 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b0800dad0 con 0x5598f67a3860 2024-04-24T06:46:55.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.204+0000 7f1b04c00640 1 -- 172.21.15.5:0/1630676739 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b08018c70 con 0x5598f67a3860 2024-04-24T06:46:55.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.204+0000 7f1b0f400640 1 --2- 172.21.15.5:0/2498086420 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1af40a3d20 0x7f1af40a61e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x5598f67a4e20 tx=0x7f1b000f6530 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:55.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.204+0000 7f1b191c7740 1 -- 172.21.15.5:0/1630676739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598f67a3860 msgr2=0x5598f67a3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:55.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.204+0000 7f1b191c7740 1 --2- 172.21.15.5:0/1630676739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598f67a3860 0x5598f67a3c40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1b08009ee0 tx=0x7f1b0800f7d0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.204+0000 7f1b191c7740 1 -- 172.21.15.5:0/1630676739 shutdown_connections 2024-04-24T06:46:55.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.204+0000 7f1b191c7740 1 --2- 172.21.15.5:0/1630676739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598f67ae850 0x5598f67b0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.204+0000 7f1b191c7740 1 --2- 172.21.15.5:0/1630676739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598f67a4180 0x5598f67ae310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.204+0000 7f1b191c7740 1 --2- 172.21.15.5:0/1630676739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598f67a3860 0x5598f67a3c40 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.204+0000 7f1b191c7740 1 -- 172.21.15.5:0/1630676739 >> 172.21.15.5:0/1630676739 conn(0x5598f679fb80 msgr2=0x5598f679ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:55.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.205+0000 7f1b191c7740 1 -- 172.21.15.5:0/1630676739 shutdown_connections 2024-04-24T06:46:55.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.205+0000 7f1b191c7740 1 -- 172.21.15.5:0/1630676739 wait complete. 2024-04-24T06:46:55.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.205+0000 7f1b191c7740 1 -- start start 2024-04-24T06:46:55.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.205+0000 7f1b191c7740 1 Processor -- start 2024-04-24T06:46:55.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.206+0000 7f1b191c7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598f67a3860 0x5598f67ab820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:55.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.206+0000 7f1b191c7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598f67a4180 0x5598f67abd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:55.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.206+0000 7f1b191c7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598f67ae850 0x5598f67ac2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:55.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.206+0000 7f1b191c7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5598f67b1640 con 0x5598f67a3860 2024-04-24T06:46:55.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.206+0000 7f1b191c7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5598f66c2690 con 0x5598f67a4180 2024-04-24T06:46:55.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.206+0000 7f1b191c7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5598f67b17c0 con 0x5598f67ae850 2024-04-24T06:46:55.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.206+0000 7f1b191c7740 10 client.0 _ll_register_callbacks cb 0x5598f66fe650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:46:55.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.206+0000 7f1b0fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598f67a3860 0x5598f67ab820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:55.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.206+0000 7f1b0ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598f67a4180 0x5598f67abd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:55.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.206+0000 7f1b0ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598f67a4180 0x5598f67abd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:38422/0 (socket says 172.21.15.5:38422) 2024-04-24T06:46:55.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.206+0000 7f1b0fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598f67a3860 0x5598f67ab820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39584/0 (socket says 172.21.15.5:39584) 2024-04-24T06:46:55.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.206+0000 7f1b0ea00640 1 -- 172.21.15.5:0/931924742 learned_addr learned my addr 172.21.15.5:0/931924742 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:55.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.206+0000 7f1b0fe00640 1 --2- 172.21.15.5:0/931924742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598f67ae850 0x5598f67ac2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:55.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.206+0000 7f1b0ea00640 1 -- 172.21.15.5:0/931924742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598f67ae850 msgr2=0x5598f67ac2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:55.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.206+0000 7f1b0ea00640 1 --2- 172.21.15.5:0/931924742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598f67ae850 0x5598f67ac2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.206+0000 7f1b0ea00640 1 -- 172.21.15.5:0/931924742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598f67a3860 msgr2=0x5598f67ab820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:55.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.206+0000 7f1b0ea00640 1 --2- 172.21.15.5:0/931924742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598f67a3860 0x5598f67ab820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.206+0000 7f1b0ea00640 1 -- 172.21.15.5:0/931924742 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598f66c1920 con 0x5598f67a4180 2024-04-24T06:46:55.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.206+0000 7f1b0fe00640 1 --2- 172.21.15.5:0/931924742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598f67a3860 0x5598f67ab820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:46:55.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.207+0000 7f1b0ea00640 1 --2- 172.21.15.5:0/931924742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598f67a4180 0x5598f67abd60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1af8012730 tx=0x7f1af8012bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef37f039c61ce648 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:55.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.207+0000 7f1afd600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:46:55.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.207+0000 7f1afd600640 1 -- 172.21.15.5:0/931924742 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af8004320 con 0x5598f67a4180 2024-04-24T06:46:55.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.207+0000 7f1afd600640 1 -- 172.21.15.5:0/931924742 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f1af80119a0 con 0x5598f67a4180 2024-04-24T06:46:55.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.207+0000 7f1b191c7740 1 -- 172.21.15.5:0/931924742 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b0800dad0 con 0x5598f67a4180 2024-04-24T06:46:55.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.207+0000 7f1afd600640 1 -- 172.21.15.5:0/931924742 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1af80044c0 con 0x5598f67a4180 2024-04-24T06:46:55.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.207+0000 7f1b191c7740 10 client.25892 fetch_fsmap learned FSMap version 25 2024-04-24T06:46:55.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.207+0000 7f1b191c7740 1 -- 172.21.15.5:0/931924742 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5598f64e57d0 con 0x5598f67a4180 2024-04-24T06:46:55.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.208+0000 7f1afd600640 1 -- 172.21.15.5:0/931924742 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b0800dad0 con 0x5598f67a4180 2024-04-24T06:46:55.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.208+0000 7f1afd600640 1 -- 172.21.15.5:0/931924742 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1af800f440 con 0x5598f67a4180 2024-04-24T06:46:55.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.208+0000 7f1b191c7740 10 client.25892 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:46:55.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.208+0000 7f1b191c7740 10 client.25892 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:46:55.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.208+0000 7f1b191c7740 20 client.25892 populate_metadata read hostname 'smithi005' 2024-04-24T06:46:55.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.208+0000 7f1b191c7740 1 --2- 172.21.15.5:0/931924742 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5598f67bec80 0x5598f67c10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:55.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.208+0000 7f1b191c7740 4 client.25892 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:46:55.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.208+0000 7f1b191c7740 1 -- 172.21.15.5:0/931924742 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598f6716c10 con 0x5598f67bec80 2024-04-24T06:46:55.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.209+0000 7f1b0fe00640 1 --2- 172.21.15.5:0/931924742 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5598f67bec80 0x5598f67c10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:55.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.209+0000 7f1b0fe00640 1 --2- 172.21.15.5:0/931924742 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5598f67bec80 0x5598f67c10b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=864729b03ae97227 server_cookie=a125de6af5c57cc5 in_seq=0 out_seq=0 2024-04-24T06:46:55.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.209+0000 7f1afd600640 10 client.25892 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:46:55.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.210+0000 7f1afd600640 1 -- 172.21.15.5:0/931924742 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5598f6716c10 con 0x5598f67bec80 2024-04-24T06:46:55.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.210+0000 7f1afd600640 10 client.25892 handle_command_reply: tid=0 2024-04-24T06:46:55.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.210+0000 7f1b191c7740 1 client.25892 shutdown 2024-04-24T06:46:55.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.210+0000 7f1b191c7740 1 -- 172.21.15.5:0/931924742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598f67a4180 msgr2=0x5598f67abd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:55.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.210+0000 7f1b191c7740 1 --2- 172.21.15.5:0/931924742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598f67a4180 0x5598f67abd60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1af8012730 tx=0x7f1af8012bf0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.210+0000 7f1b191c7740 1 -- 172.21.15.5:0/931924742 shutdown_connections 2024-04-24T06:46:55.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.210+0000 7f1b191c7740 1 --2- 172.21.15.5:0/931924742 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5598f67bec80 0x5598f67c10b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.210+0000 7f1b191c7740 1 --2- 172.21.15.5:0/931924742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598f67ae850 0x5598f67ac2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.210+0000 7f1b191c7740 1 --2- 172.21.15.5:0/931924742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598f67a4180 0x5598f67abd60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.210+0000 7f1afd600640 0 client.25892 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:46:55.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.210+0000 7f1b191c7740 1 --2- 172.21.15.5:0/931924742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598f67a3860 0x5598f67ab820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.210+0000 7f1b191c7740 1 -- 172.21.15.5:0/931924742 >> 172.21.15.5:0/931924742 conn(0x5598f679fb80 msgr2=0x5598f67a5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:55.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.210+0000 7f1b191c7740 1 -- 172.21.15.5:0/931924742 shutdown_connections 2024-04-24T06:46:55.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.210+0000 7f1b191c7740 1 -- 172.21.15.5:0/931924742 wait complete. 2024-04-24T06:46:55.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.211+0000 7f1b191c7740 20 client.25892 trim_cache size 0 max 16384 2024-04-24T06:46:55.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.229+0000 7f1b191c7740 1 Processor -- start 2024-04-24T06:46:55.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.229+0000 7f1b191c7740 1 -- start start 2024-04-24T06:46:55.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.230+0000 7f1b191c7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598f67a3860 0x5598f67aa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:55.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.230+0000 7f1b191c7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598f67a4180 0x5598f67ba500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:55.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.230+0000 7f1b191c7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598f67ae850 0x5598f67c69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:55.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.230+0000 7f1b191c7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1b101b28e0 con 0x5598f67ae850 2024-04-24T06:46:55.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.230+0000 7f1b191c7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1b101b01b0 con 0x5598f67a4180 2024-04-24T06:46:55.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.230+0000 7f1b191c7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1b101ae000 con 0x5598f67a3860 2024-04-24T06:46:55.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.230+0000 7f1b0fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598f67ae850 0x5598f67c69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:55.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.230+0000 7f1b0fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598f67ae850 0x5598f67c69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39586/0 (socket says 172.21.15.5:39586) 2024-04-24T06:46:55.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.230+0000 7f1b0ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598f67a4180 0x5598f67ba500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:55.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.230+0000 7f1b0fe00640 1 -- 172.21.15.5:0/1398432675 learned_addr learned my addr 172.21.15.5:0/1398432675 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:55.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.230+0000 7f1b0fe00640 1 --2- 172.21.15.5:0/1398432675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598f67a3860 0x5598f67aa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:55.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.231+0000 7f1b0ea00640 1 -- 172.21.15.5:0/1398432675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598f67a3860 msgr2=0x5598f67aa760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:55.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.231+0000 7f1b0ea00640 1 --2- 172.21.15.5:0/1398432675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598f67a3860 0x5598f67aa760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.231+0000 7f1b0ea00640 1 -- 172.21.15.5:0/1398432675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598f67ae850 msgr2=0x5598f67c69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:55.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.231+0000 7f1b0ea00640 1 --2- 172.21.15.5:0/1398432675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598f67ae850 0x5598f67c69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.231+0000 7f1b0ea00640 1 -- 172.21.15.5:0/1398432675 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598f641b3f0 con 0x5598f67a4180 2024-04-24T06:46:55.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.231+0000 7f1b0ea00640 1 --2- 172.21.15.5:0/1398432675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598f67a4180 0x5598f67ba500 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1af800fa50 tx=0x7f1af8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e27d0a7f7499f54 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:55.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.231+0000 7f1affe00640 1 -- 172.21.15.5:0/1398432675 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af8031020 con 0x5598f67a4180 2024-04-24T06:46:55.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.231+0000 7f1affe00640 1 -- 172.21.15.5:0/1398432675 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1af8011e80 con 0x5598f67a4180 2024-04-24T06:46:55.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.231+0000 7f1affe00640 1 -- 172.21.15.5:0/1398432675 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af8029040 con 0x5598f67a4180 2024-04-24T06:46:55.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.232+0000 7f1b191c7740 1 -- 172.21.15.5:0/1398432675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598f67a4180 msgr2=0x5598f67ba500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:55.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.232+0000 7f1b191c7740 1 --2- 172.21.15.5:0/1398432675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598f67a4180 0x5598f67ba500 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1af800fa50 tx=0x7f1af8011810 comp rx=0 tx=0).stop 2024-04-24T06:46:55.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.232+0000 7f1b191c7740 1 -- 172.21.15.5:0/1398432675 shutdown_connections 2024-04-24T06:46:55.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.232+0000 7f1b191c7740 1 --2- 172.21.15.5:0/1398432675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598f67ae850 0x5598f67c69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.232+0000 7f1b191c7740 1 --2- 172.21.15.5:0/1398432675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598f67a4180 0x5598f67ba500 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.232+0000 7f1b191c7740 1 --2- 172.21.15.5:0/1398432675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598f67a3860 0x5598f67aa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.232+0000 7f1b191c7740 1 -- 172.21.15.5:0/1398432675 >> 172.21.15.5:0/1398432675 conn(0x5598f679fb80 msgr2=0x5598f679ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:55.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.232+0000 7f1b191c7740 1 -- 172.21.15.5:0/1398432675 shutdown_connections 2024-04-24T06:46:55.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.232+0000 7f1b191c7740 1 -- 172.21.15.5:0/1398432675 wait complete. 2024-04-24T06:46:55.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.232+0000 7f1b191c7740 1 -- start start 2024-04-24T06:46:55.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.233+0000 7f1b191c7740 1 Processor -- start 2024-04-24T06:46:55.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.233+0000 7f1b191c7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598f67a3860 0x5598f67b2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:55.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.233+0000 7f1b191c7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598f67a4180 0x5598f67b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:55.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.233+0000 7f1b191c7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598f67ae850 0x5598f681e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:55.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.233+0000 7f1b191c7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1b101ae670 con 0x5598f67a3860 2024-04-24T06:46:55.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.233+0000 7f1b191c7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1b101afae0 con 0x5598f67a4180 2024-04-24T06:46:55.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.233+0000 7f1b191c7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1b101aed40 con 0x5598f67ae850 2024-04-24T06:46:55.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.233+0000 7f1b191c7740 10 client.0 _ll_register_callbacks cb 0x7f1af80097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:46:55.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.233+0000 7f1b0fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598f67a3860 0x5598f67b2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:55.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.233+0000 7f1b0fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598f67a3860 0x5598f67b2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39588/0 (socket says 172.21.15.5:39588) 2024-04-24T06:46:55.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.233+0000 7f1b0fe00640 1 -- 172.21.15.5:0/791043157 learned_addr learned my addr 172.21.15.5:0/791043157 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:55.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.233+0000 7f1b0ea00640 1 --2- 172.21.15.5:0/791043157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598f67a4180 0x5598f67b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:55.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.234+0000 7f1b0fe00640 1 --2- 172.21.15.5:0/791043157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598f67ae850 0x5598f681e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:55.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.234+0000 7f1b0fe00640 1 -- 172.21.15.5:0/791043157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598f67ae850 msgr2=0x5598f681e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:55.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.234+0000 7f1b0fe00640 1 --2- 172.21.15.5:0/791043157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598f67ae850 0x5598f681e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.234+0000 7f1b0fe00640 1 -- 172.21.15.5:0/791043157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598f67a4180 msgr2=0x5598f67b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:55.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.234+0000 7f1b0fe00640 1 --2- 172.21.15.5:0/791043157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598f67a4180 0x5598f67b31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.234+0000 7f1b0fe00640 1 -- 172.21.15.5:0/791043157 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598f63db0f0 con 0x5598f67a3860 2024-04-24T06:46:55.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.234+0000 7f1b0ea00640 1 --2- 172.21.15.5:0/791043157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598f67a4180 0x5598f67b31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:46:55.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.234+0000 7f1b0fe00640 1 --2- 172.21.15.5:0/791043157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598f67a3860 0x5598f67b2c90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1b0801b040 tx=0x7f1b0800f7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ddcaf7d0faf25d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:55.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.234+0000 7f1afd600640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:46:55.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.234+0000 7f1afd600640 1 -- 172.21.15.5:0/791043157 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0801d040 con 0x5598f67a3860 2024-04-24T06:46:55.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.234+0000 7f1b191c7740 1 -- 172.21.15.5:0/791043157 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5598f6710a80 con 0x5598f67a3860 2024-04-24T06:46:55.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.235+0000 7f1afd600640 1 -- 172.21.15.5:0/791043157 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f1b0800d910 con 0x5598f67a3860 2024-04-24T06:46:55.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.235+0000 7f1afd600640 1 -- 172.21.15.5:0/791043157 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1b08004a00 con 0x5598f67a3860 2024-04-24T06:46:55.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.235+0000 7f1b191c7740 10 client.15771 fetch_fsmap learned FSMap version 25 2024-04-24T06:46:55.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.235+0000 7f1b191c7740 1 -- 172.21.15.5:0/791043157 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5598f66ce810 con 0x5598f67a3860 2024-04-24T06:46:55.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.235+0000 7f1afd600640 1 -- 172.21.15.5:0/791043157 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5598f6710a80 con 0x5598f67a3860 2024-04-24T06:46:55.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.235+0000 7f1afd600640 1 -- 172.21.15.5:0/791043157 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1b0800b040 con 0x5598f67a3860 2024-04-24T06:46:55.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.235+0000 7f1b191c7740 10 client.15771 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:46:55.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.235+0000 7f1b191c7740 10 client.15771 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:46:55.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.235+0000 7f1b191c7740 20 client.15771 populate_metadata read hostname 'smithi005' 2024-04-24T06:46:55.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.235+0000 7f1b191c7740 1 --2- 172.21.15.5:0/791043157 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5598f6825d20 0x5598f6828140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:55.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.235+0000 7f1b191c7740 4 client.15771 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:46:55.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.235+0000 7f1b191c7740 1 -- 172.21.15.5:0/791043157 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5598f668c360 con 0x5598f6825d20 2024-04-24T06:46:55.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.236+0000 7f1b0ea00640 1 --2- 172.21.15.5:0/791043157 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5598f6825d20 0x5598f6828140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:55.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.236+0000 7f1b0ea00640 1 --2- 172.21.15.5:0/791043157 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5598f6825d20 0x5598f6828140 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3e4ade2a5188466 server_cookie=6a3d274c00caf530 in_seq=0 out_seq=0 2024-04-24T06:46:55.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.236+0000 7f1afd600640 10 client.15771 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:46:55.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.236+0000 7f1afd600640 1 -- 172.21.15.5:0/791043157 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5598f668c360 con 0x5598f6825d20 2024-04-24T06:46:55.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.236+0000 7f1afd600640 10 client.15771 handle_command_reply: tid=0 2024-04-24T06:46:55.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.236+0000 7f1b191c7740 1 client.15771 shutdown 2024-04-24T06:46:55.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.237+0000 7f1b191c7740 1 -- 172.21.15.5:0/791043157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598f67a3860 msgr2=0x5598f67b2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:55.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.237+0000 7f1b191c7740 1 --2- 172.21.15.5:0/791043157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598f67a3860 0x5598f67b2c90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1b0801b040 tx=0x7f1b0800f7d0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.237+0000 7f1b191c7740 1 -- 172.21.15.5:0/791043157 shutdown_connections 2024-04-24T06:46:55.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.237+0000 7f1b191c7740 1 --2- 172.21.15.5:0/791043157 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5598f6825d20 0x5598f6828140 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.237+0000 7f1b191c7740 1 --2- 172.21.15.5:0/791043157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5598f67ae850 0x5598f681e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.237+0000 7f1b191c7740 1 --2- 172.21.15.5:0/791043157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598f67a4180 0x5598f67b31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.237+0000 7f1b191c7740 1 --2- 172.21.15.5:0/791043157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598f67a3860 0x5598f67b2c90 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.237+0000 7f1b191c7740 1 -- 172.21.15.5:0/791043157 >> 172.21.15.5:0/791043157 conn(0x5598f679fb80 msgr2=0x5598f67ab7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:55.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.237+0000 7f1afd600640 0 client.15771 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:46:55.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.237+0000 7f1b191c7740 1 -- 172.21.15.5:0/791043157 shutdown_connections 2024-04-24T06:46:55.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.237+0000 7f1b191c7740 1 -- 172.21.15.5:0/791043157 wait complete. 2024-04-24T06:46:55.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.237+0000 7f1b191c7740 20 client.15771 trim_cache size 0 max 16384 2024-04-24T06:46:55.243 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:46:55.243 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-24T06:46:55.243 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-24T06:46:55.243 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:46:55.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.238+0000 7f1b16400640 1 -- 172.21.15.5:0/2498086420 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1af40a3d20 msgr2=0x7f1af40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:55.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.238+0000 7f1b16400640 1 --2- 172.21.15.5:0/2498086420 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1af40a3d20 0x7f1af40a61e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x5598f67a4e20 tx=0x7f1b000f6530 comp rx=0 tx=0).stop 2024-04-24T06:46:55.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.238+0000 7f1b16400640 1 -- 172.21.15.5:0/2498086420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b10104450 msgr2=0x7f1b101a0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:55.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.238+0000 7f1b16400640 1 --2- 172.21.15.5:0/2498086420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b10104450 0x7f1b101a0750 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1b00035040 tx=0x7f1b00033b60 comp rx=0 tx=0).stop 2024-04-24T06:46:55.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.238+0000 7f1b16400640 1 -- 172.21.15.5:0/2498086420 shutdown_connections 2024-04-24T06:46:55.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.238+0000 7f1b16400640 1 --2- 172.21.15.5:0/2498086420 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1af40a3d20 0x7f1af40a61e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.238+0000 7f1b16400640 1 --2- 172.21.15.5:0/2498086420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b1019a950 0x7f1b1019ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.238+0000 7f1b16400640 1 --2- 172.21.15.5:0/2498086420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b10109520 0x7f1b101a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.238+0000 7f1b16400640 1 --2- 172.21.15.5:0/2498086420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b10104450 0x7f1b101a0750 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.238+0000 7f1b16400640 1 -- 172.21.15.5:0/2498086420 >> 172.21.15.5:0/2498086420 conn(0x7f1b100ffc90 msgr2=0x7f1b10101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:55.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.238+0000 7f1b16400640 1 -- 172.21.15.5:0/2498086420 shutdown_connections 2024-04-24T06:46:55.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.238+0000 7f1b16400640 1 -- 172.21.15.5:0/2498086420 wait complete. 2024-04-24T06:46:55.255 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T06:46:55.255 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:46:55.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.447+0000 7f7efa200640 1 -- 172.21.15.5:0/43569075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ef4102150 msgr2=0x7f7ef4102530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:55.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.447+0000 7f7efa200640 1 --2- 172.21.15.5:0/43569075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ef4102150 0x7f7ef4102530 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4009900 tx=0x7f7ee402a3f0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.447+0000 7f7efa200640 1 -- 172.21.15.5:0/43569075 shutdown_connections 2024-04-24T06:46:55.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.447+0000 7f7efa200640 1 --2- 172.21.15.5:0/43569075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ef4107220 0x7f7ef410edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.447+0000 7f7efa200640 1 --2- 172.21.15.5:0/43569075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef4102b00 0x7f7ef4106b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.447+0000 7f7efa200640 1 --2- 172.21.15.5:0/43569075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ef4102150 0x7f7ef4102530 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.447+0000 7f7efa200640 1 -- 172.21.15.5:0/43569075 >> 172.21.15.5:0/43569075 conn(0x7f7ef40fd9f0 msgr2=0x7f7ef40ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:55.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.447+0000 7f7efa200640 1 -- 172.21.15.5:0/43569075 shutdown_connections 2024-04-24T06:46:55.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.447+0000 7f7efa200640 1 -- 172.21.15.5:0/43569075 wait complete. 2024-04-24T06:46:55.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.448+0000 7f7efa200640 1 Processor -- start 2024-04-24T06:46:55.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.448+0000 7f7efa200640 1 -- start start 2024-04-24T06:46:55.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.448+0000 7f7efa200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ef4102150 0x7f7ef419e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:55.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.449+0000 7f7efa200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ef4107220 0x7f7ef419e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:55.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.449+0000 7f7efa200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef4198600 0x7f7ef4198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:55.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.449+0000 7f7efa200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef41122e0 con 0x7f7ef4107220 2024-04-24T06:46:55.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.449+0000 7f7efa200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef4112160 con 0x7f7ef4198600 2024-04-24T06:46:55.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.449+0000 7f7efa200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef4112460 con 0x7f7ef4102150 2024-04-24T06:46:55.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.449+0000 7f7eebe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ef4107220 0x7f7ef419e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:55.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.449+0000 7f7eebe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ef4107220 0x7f7ef419e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39596/0 (socket says 172.21.15.5:39596) 2024-04-24T06:46:55.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.449+0000 7f7eebe00640 1 -- 172.21.15.5:0/2427183470 learned_addr learned my addr 172.21.15.5:0/2427183470 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:55.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.449+0000 7f7ef9800640 1 --2- 172.21.15.5:0/2427183470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef4198600 0x7f7ef4198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:55.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.449+0000 7f7ef8e00640 1 --2- 172.21.15.5:0/2427183470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ef4102150 0x7f7ef419e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:55.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.449+0000 7f7eebe00640 1 -- 172.21.15.5:0/2427183470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ef4102150 msgr2=0x7f7ef419e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:55.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.449+0000 7f7eebe00640 1 --2- 172.21.15.5:0/2427183470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ef4102150 0x7f7ef419e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.449+0000 7f7eebe00640 1 -- 172.21.15.5:0/2427183470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef4198600 msgr2=0x7f7ef4198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:55.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.449+0000 7f7eebe00640 1 --2- 172.21.15.5:0/2427183470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef4198600 0x7f7ef4198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.449+0000 7f7eebe00640 1 -- 172.21.15.5:0/2427183470 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef4199120 con 0x7f7ef4107220 2024-04-24T06:46:55.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.449+0000 7f7ef9800640 1 --2- 172.21.15.5:0/2427183470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef4198600 0x7f7ef4198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:46:55.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.450+0000 7f7eebe00640 1 --2- 172.21.15.5:0/2427183470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ef4107220 0x7f7ef419e990 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7edc00e9a0 tx=0x7f7edc00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:55.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.450+0000 7f7ef2000640 1 -- 172.21.15.5:0/2427183470 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7edc00cd80 con 0x7f7ef4107220 2024-04-24T06:46:55.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.450+0000 7f7ef2000640 1 -- 172.21.15.5:0/2427183470 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7edc0035f0 con 0x7f7ef4107220 2024-04-24T06:46:55.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.450+0000 7f7efa200640 1 -- 172.21.15.5:0/2427183470 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ef4199410 con 0x7f7ef4107220 2024-04-24T06:46:55.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.450+0000 7f7ef2000640 1 -- 172.21.15.5:0/2427183470 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7edc010670 con 0x7f7ef4107220 2024-04-24T06:46:55.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.450+0000 7f7efa200640 1 -- 172.21.15.5:0/2427183470 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ef41a56b0 con 0x7f7ef4107220 2024-04-24T06:46:55.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.451+0000 7f7efa200640 1 -- 172.21.15.5:0/2427183470 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ef404fe00 con 0x7f7ef4107220 2024-04-24T06:46:55.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.453+0000 7f7ef2000640 1 -- 172.21.15.5:0/2427183470 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f7edc0037a0 con 0x7f7ef4107220 2024-04-24T06:46:55.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.455+0000 7f7ef2000640 1 --2- 172.21.15.5:0/2427183470 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f7ed80a3d90 0x7f7ed80a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:55.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.455+0000 7f7ef2000640 1 -- 172.21.15.5:0/2427183470 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f7edc0d0b00 con 0x7f7ef4107220 2024-04-24T06:46:55.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.455+0000 7f7ef8e00640 1 --2- 172.21.15.5:0/2427183470 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f7ed80a3d90 0x7f7ed80a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:55.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.456+0000 7f7ef8e00640 1 --2- 172.21.15.5:0/2427183470 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f7ed80a3d90 0x7f7ed80a6250 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4009230 tx=0x7f7ee40058d0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:55.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.458+0000 7f7ef2000640 1 -- 172.21.15.5:0/2427183470 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7edc09d1e0 con 0x7f7ef4107220 2024-04-24T06:46:55.626 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:55 smithi005 ceph-mon[27798]: pgmap v89: 97 pgs: 97 active+clean; 87 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 799 KiB/s wr, 92 op/s 2024-04-24T06:46:55.630 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a3c8a22bd5efe18ab230a8e2d072db8f381a0411/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_a3c8a22bd5efe18ab230a8e2d072db8f381a0411/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a3c8a22bd5efe18ab230a8e2d072db8f381a0411/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/fsx.sh) on smithi005 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a3c8a22bd5efe18ab230a8e2d072db8f381a0411 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' 2024-04-24T06:46:55.866 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=01cb0e02457d45849ab5f39f1f15e2e1 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a3c8a22bd5efe18ab230a8e2d072db8f381a0411/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_a3c8a22bd5efe18ab230a8e2d072db8f381a0411/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a3c8a22bd5efe18ab230a8e2d072db8f381a0411/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/fsx.sh) on smithi005 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a3c8a22bd5efe18ab230a8e2d072db8f381a0411 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' 2024-04-24T06:46:55.869 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-24T06:46:55.880 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-24T06:46:55.880 INFO:teuthology.task.exec:Executing custom commands... 2024-04-24T06:46:55.880 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-04-24T06:46:55.880 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-24T06:46:55.885 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:55 smithi037 ceph-mon[32864]: pgmap v89: 97 pgs: 97 active+clean; 87 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 799 KiB/s wr, 92 op/s 2024-04-24T06:46:55.885 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:55 smithi137 ceph-mon[33017]: pgmap v89: 97 pgs: 97 active+clean; 87 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 799 KiB/s wr, 92 op/s 2024-04-24T06:46:55.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.700+0000 7f7efa200640 1 -- 172.21.15.5:0/2427183470 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ef4199b30 con 0x7f7ef4107220 2024-04-24T06:46:55.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.701+0000 7f7ef2000640 1 -- 172.21.15.5:0/2427183470 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11859 (secure 0 0 0) 0x7f7ef4199b30 con 0x7f7ef4107220 2024-04-24T06:46:55.886 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:46:55.886 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1735462750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1735462750},{"type":"v1","addr":"172.21.15.5:6837","nonce":1735462750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1499469370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1499469370},{"type":"v1","addr":"172.21.15.5:6841","nonce":1499469370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3705638847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3705638847},{"type":"v1","addr":"172.21.15.37:6835","nonce":3705638847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T06:41:34.764490+0000","modified":"2024-04-24T06:46:02.765290+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24550,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6837/1390103296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1390103296},{"type":"v1","addr":"172.21.15.37:6837","nonce":1390103296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3217754215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3217754215},{"type":"v1","addr":"172.21.15.137:6835","nonce":3217754215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6837/3044222976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3044222976},{"type":"v1","addr":"172.21.15.137:6837","nonce":3044222976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.5:6839/3173201459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3173201459},{"type":"v1","addr":"172.21.15.5:6839","nonce":3173201459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6839/1889774232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1889774232},{"type":"v1","addr":"172.21.15.137:6839","nonce":1889774232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6839/3536721753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":3536721753},{"type":"v1","addr":"172.21.15.37:6839","nonce":3536721753}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24539,24550,24569]},"id":1}]} 2024-04-24T06:46:55.888 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-04-24T06:46:55.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.704+0000 7f7efa200640 1 -- 172.21.15.5:0/2427183470 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f7ed80a3d90 msgr2=0x7f7ed80a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:55.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.704+0000 7f7efa200640 1 --2- 172.21.15.5:0/2427183470 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f7ed80a3d90 0x7f7ed80a6250 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4009230 tx=0x7f7ee40058d0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.704+0000 7f7efa200640 1 -- 172.21.15.5:0/2427183470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ef4107220 msgr2=0x7f7ef419e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:55.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.704+0000 7f7efa200640 1 --2- 172.21.15.5:0/2427183470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ef4107220 0x7f7ef419e990 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7edc00e9a0 tx=0x7f7edc00ee60 comp rx=0 tx=0).stop 2024-04-24T06:46:55.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.704+0000 7f7efa200640 1 -- 172.21.15.5:0/2427183470 shutdown_connections 2024-04-24T06:46:55.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.705+0000 7f7efa200640 1 --2- 172.21.15.5:0/2427183470 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f7ed80a3d90 0x7f7ed80a6250 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.705+0000 7f7efa200640 1 --2- 172.21.15.5:0/2427183470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef4198600 0x7f7ef4198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.705+0000 7f7efa200640 1 --2- 172.21.15.5:0/2427183470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ef4107220 0x7f7ef419e990 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.705+0000 7f7efa200640 1 --2- 172.21.15.5:0/2427183470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ef4102150 0x7f7ef419e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:55.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.705+0000 7f7efa200640 1 -- 172.21.15.5:0/2427183470 >> 172.21.15.5:0/2427183470 conn(0x7f7ef40fd9f0 msgr2=0x7f7ef40ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:55.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.705+0000 7f7efa200640 1 -- 172.21.15.5:0/2427183470 shutdown_connections 2024-04-24T06:46:55.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:55.705+0000 7f7efa200640 1 -- 172.21.15.5:0/2427183470 wait complete. 2024-04-24T06:46:55.890 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T06:46:56.009 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-24T06:46:56.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.134+0000 7f420a000640 1 -- 172.21.15.5:0/3687961139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4204109320 msgr2=0x7f4204110ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:56.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.134+0000 7f420a000640 1 --2- 172.21.15.5:0/3687961139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4204109320 0x7f4204110ef0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f41fc009ca0 tx=0x7f41fc02a8c0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.134+0000 7f420a000640 1 -- 172.21.15.5:0/3687961139 shutdown_connections 2024-04-24T06:46:56.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.134+0000 7f420a000640 1 --2- 172.21.15.5:0/3687961139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4204109320 0x7f4204110ef0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.134+0000 7f420a000640 1 --2- 172.21.15.5:0/3687961139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4204104c00 0x7f4204108c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.134+0000 7f420a000640 1 --2- 172.21.15.5:0/3687961139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4204104250 0x7f4204104630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.134+0000 7f420a000640 1 -- 172.21.15.5:0/3687961139 >> 172.21.15.5:0/3687961139 conn(0x7f42040ffad0 msgr2=0x7f4204101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:56.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.134+0000 7f420a000640 1 -- 172.21.15.5:0/3687961139 shutdown_connections 2024-04-24T06:46:56.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.134+0000 7f420a000640 1 -- 172.21.15.5:0/3687961139 wait complete. 2024-04-24T06:46:56.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.135+0000 7f420a000640 1 Processor -- start 2024-04-24T06:46:56.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.135+0000 7f420a000640 1 -- start start 2024-04-24T06:46:56.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.136+0000 7f420a000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4204104250 0x7f42041a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.136+0000 7f420a000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4204109320 0x7f42041a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.136+0000 7f420a000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f420419a730 0x7f420419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.136+0000 7f420a000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4204114430 con 0x7f420419a730 2024-04-24T06:46:56.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.136+0000 7f420a000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f42041142b0 con 0x7f4204104250 2024-04-24T06:46:56.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.136+0000 7f420a000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f42041145b0 con 0x7f4204109320 2024-04-24T06:46:56.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.136+0000 7f4208c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4204104250 0x7f42041a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.136+0000 7f4209600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f420419a730 0x7f420419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.136+0000 7f4208c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4204104250 0x7f42041a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:38458/0 (socket says 172.21.15.5:38458) 2024-04-24T06:46:56.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.136+0000 7f4208c00640 1 -- 172.21.15.5:0/736107762 learned_addr learned my addr 172.21.15.5:0/736107762 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:56.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.137+0000 7f4203e00640 1 --2- 172.21.15.5:0/736107762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4204109320 0x7f42041a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.137+0000 7f4209600640 1 -- 172.21.15.5:0/736107762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4204109320 msgr2=0x7f42041a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:56.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.137+0000 7f4209600640 1 --2- 172.21.15.5:0/736107762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4204109320 0x7f42041a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.137+0000 7f4209600640 1 -- 172.21.15.5:0/736107762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4204104250 msgr2=0x7f42041a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:56.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.137+0000 7f4209600640 1 --2- 172.21.15.5:0/736107762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4204104250 0x7f42041a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.137+0000 7f4209600640 1 -- 172.21.15.5:0/736107762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f420419b330 con 0x7f420419a730 2024-04-24T06:46:56.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.137+0000 7f4208c00640 1 --2- 172.21.15.5:0/736107762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4204104250 0x7f42041a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:46:56.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.137+0000 7f4203e00640 1 --2- 172.21.15.5:0/736107762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4204109320 0x7f42041a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:46:56.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.137+0000 7f4209600640 1 --2- 172.21.15.5:0/736107762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f420419a730 0x7f420419abb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f41fc005e30 tx=0x7f41fc003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:56.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.137+0000 7f4201600640 1 -- 172.21.15.5:0/736107762 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41fc031650 con 0x7f420419a730 2024-04-24T06:46:56.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.137+0000 7f4201600640 1 -- 172.21.15.5:0/736107762 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41fc0317f0 con 0x7f420419a730 2024-04-24T06:46:56.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.137+0000 7f4201600640 1 -- 172.21.15.5:0/736107762 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41fc02f990 con 0x7f420419a730 2024-04-24T06:46:56.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.137+0000 7f420a000640 1 -- 172.21.15.5:0/736107762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f420419b5c0 con 0x7f420419a730 2024-04-24T06:46:56.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.138+0000 7f420a000640 1 -- 172.21.15.5:0/736107762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42041a7710 con 0x7f420419a730 2024-04-24T06:46:56.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.141+0000 7f4201600640 1 -- 172.21.15.5:0/736107762 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f41fc02fb50 con 0x7f420419a730 2024-04-24T06:46:56.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.141+0000 7f420e5ae740 1 Processor -- start 2024-04-24T06:46:56.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.141+0000 7f420e5ae740 1 -- start start 2024-04-24T06:46:56.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.141+0000 7f420e5ae740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f72ec22b40 0x55f72ec22f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.141+0000 7f420e5ae740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f72ec23460 0x55f72ec2d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.141+0000 7f420e5ae740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72ec2db30 0x55f72ec2ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.141+0000 7f420e5ae740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f72eb881d0 con 0x55f72ec2db30 2024-04-24T06:46:56.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.141+0000 7f420e5ae740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f72eb2e450 con 0x55f72ec22b40 2024-04-24T06:46:56.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.141+0000 7f420e5ae740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f72eb84190 con 0x55f72ec23460 2024-04-24T06:46:56.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.142+0000 7f4203e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f72ec22b40 0x55f72ec22f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.142+0000 7f4203e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f72ec22b40 0x55f72ec22f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:38470/0 (socket says 172.21.15.5:38470) 2024-04-24T06:46:56.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.142+0000 7f4203e00640 1 -- 172.21.15.5:0/994207841 learned_addr learned my addr 172.21.15.5:0/994207841 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:56.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.142+0000 7f4208c00640 1 --2- 172.21.15.5:0/994207841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f72ec23460 0x55f72ec2d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.142+0000 7f4203e00640 1 --2- 172.21.15.5:0/994207841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72ec2db30 0x55f72ec2ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.142+0000 7f4208c00640 1 -- 172.21.15.5:0/994207841 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f72ec22b40 msgr2=0x55f72ec22f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:56.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.142+0000 7f4208c00640 1 --2- 172.21.15.5:0/994207841 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f72ec22b40 0x55f72ec22f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.142+0000 7f4208c00640 1 -- 172.21.15.5:0/994207841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72ec2db30 msgr2=0x55f72ec2ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:56.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.142+0000 7f4208c00640 1 --2- 172.21.15.5:0/994207841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72ec2db30 0x55f72ec2ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.142+0000 7f4208c00640 1 -- 172.21.15.5:0/994207841 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f72ebeb310 con 0x55f72ec23460 2024-04-24T06:46:56.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.142+0000 7f4208c00640 1 --2- 172.21.15.5:0/994207841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f72ec23460 0x55f72ec2d5f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f41f8011510 tx=0x7f41f80119d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff292bb9534a67fd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:56.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.143+0000 7f41f5c00640 1 -- 172.21.15.5:0/994207841 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f801e070 con 0x55f72ec23460 2024-04-24T06:46:56.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.143+0000 7f41f5c00640 1 -- 172.21.15.5:0/994207841 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41f800faa0 con 0x55f72ec23460 2024-04-24T06:46:56.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.143+0000 7f41f5c00640 1 -- 172.21.15.5:0/994207841 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f8019b60 con 0x55f72ec23460 2024-04-24T06:46:56.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.143+0000 7f4201600640 1 --2- 172.21.15.5:0/736107762 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f41ec0a3d20 0x7f41ec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.143+0000 7f4201600640 1 -- 172.21.15.5:0/736107762 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f41fc0f19a0 con 0x7f420419a730 2024-04-24T06:46:56.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.143+0000 7f420e5ae740 1 -- 172.21.15.5:0/994207841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f72ec23460 msgr2=0x55f72ec2d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:56.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.143+0000 7f420e5ae740 1 --2- 172.21.15.5:0/994207841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f72ec23460 0x55f72ec2d5f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f41f8011510 tx=0x7f41f80119d0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.143+0000 7f4203e00640 1 --2- 172.21.15.5:0/736107762 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f41ec0a3d20 0x7f41ec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.144+0000 7f420e5ae740 1 -- 172.21.15.5:0/994207841 shutdown_connections 2024-04-24T06:46:56.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.144+0000 7f420e5ae740 1 --2- 172.21.15.5:0/994207841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72ec2db30 0x55f72ec2ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.144+0000 7f420e5ae740 1 --2- 172.21.15.5:0/994207841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f72ec23460 0x55f72ec2d5f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.144+0000 7f420e5ae740 1 --2- 172.21.15.5:0/994207841 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f72ec22b40 0x55f72ec22f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.144+0000 7f420e5ae740 1 -- 172.21.15.5:0/994207841 >> 172.21.15.5:0/994207841 conn(0x55f72ec1ee60 msgr2=0x55f72ec1f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:56.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.144+0000 7f420e5ae740 1 -- 172.21.15.5:0/994207841 shutdown_connections 2024-04-24T06:46:56.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.144+0000 7f420e5ae740 1 -- 172.21.15.5:0/994207841 wait complete. 2024-04-24T06:46:56.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.144+0000 7f420e5ae740 1 -- start start 2024-04-24T06:46:56.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.144+0000 7f4203e00640 1 --2- 172.21.15.5:0/736107762 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f41ec0a3d20 0x7f41ec0a61e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55f72ec1fff0 tx=0x7f41f00046c0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:56.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.144+0000 7f420e5ae740 1 Processor -- start 2024-04-24T06:46:56.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.145+0000 7f420e5ae740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f72ec22b40 0x55f72ec2d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.145+0000 7f420e5ae740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f72ec23460 0x55f72ec2a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.145+0000 7f420e5ae740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72ec2db30 0x55f72ec2af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.145+0000 7f420e5ae740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f72ec309c0 con 0x55f72ec2db30 2024-04-24T06:46:56.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.145+0000 7f420e5ae740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f72eb41810 con 0x55f72ec22b40 2024-04-24T06:46:56.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.145+0000 7f420e5ae740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f72ec30b40 con 0x55f72ec23460 2024-04-24T06:46:56.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.145+0000 7f420e5ae740 10 client.0 _ll_register_callbacks cb 0x55f72eb0f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:46:56.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.145+0000 7f4208c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f72ec22b40 0x55f72ec2d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.145+0000 7f4208c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f72ec22b40 0x55f72ec2d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:38472/0 (socket says 172.21.15.5:38472) 2024-04-24T06:46:56.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.145+0000 7f4208c00640 1 -- 172.21.15.5:0/1717529648 learned_addr learned my addr 172.21.15.5:0/1717529648 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:56.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.145+0000 7f4209600640 1 --2- 172.21.15.5:0/1717529648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72ec2db30 0x55f72ec2af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.145+0000 7f4203e00640 1 --2- 172.21.15.5:0/1717529648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f72ec23460 0x55f72ec2a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.145+0000 7f4208c00640 1 -- 172.21.15.5:0/1717529648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f72ec23460 msgr2=0x55f72ec2a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:56.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.145+0000 7f4208c00640 1 --2- 172.21.15.5:0/1717529648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f72ec23460 0x55f72ec2a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.145+0000 7f4208c00640 1 -- 172.21.15.5:0/1717529648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72ec2db30 msgr2=0x55f72ec2af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:56.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.145+0000 7f4208c00640 1 --2- 172.21.15.5:0/1717529648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72ec2db30 0x55f72ec2af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.145+0000 7f4208c00640 1 -- 172.21.15.5:0/1717529648 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f72eb523f0 con 0x55f72ec22b40 2024-04-24T06:46:56.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.145+0000 7f4203e00640 1 --2- 172.21.15.5:0/1717529648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f72ec23460 0x55f72ec2a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:46:56.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.145+0000 7f4209600640 1 --2- 172.21.15.5:0/1717529648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72ec2db30 0x55f72ec2af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:46:56.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.146+0000 7f4208c00640 1 --2- 172.21.15.5:0/1717529648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f72ec22b40 0x55f72ec2d540 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f41f80196e0 tx=0x7f41f800f7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89c37ed45c6bbfc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:56.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.146+0000 7f41ea000640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:46:56.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.146+0000 7f41ea000640 1 -- 172.21.15.5:0/1717529648 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f801e070 con 0x55f72ec22b40 2024-04-24T06:46:56.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.146+0000 7f420e5ae740 1 -- 172.21.15.5:0/1717529648 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41f800faa0 con 0x55f72ec22b40 2024-04-24T06:46:56.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.146+0000 7f41ea000640 1 -- 172.21.15.5:0/1717529648 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f41f8019ad0 con 0x55f72ec22b40 2024-04-24T06:46:56.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.146+0000 7f41ea000640 1 -- 172.21.15.5:0/1717529648 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f41f8031050 con 0x55f72ec22b40 2024-04-24T06:46:56.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.146+0000 7f420e5ae740 10 client.25916 fetch_fsmap learned FSMap version 25 2024-04-24T06:46:56.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.146+0000 7f420e5ae740 1 -- 172.21.15.5:0/1717529648 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f72e89a3f0 con 0x55f72ec22b40 2024-04-24T06:46:56.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.147+0000 7f41ea000640 1 -- 172.21.15.5:0/1717529648 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41f800faa0 con 0x55f72ec22b40 2024-04-24T06:46:56.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.147+0000 7f41ea000640 1 -- 172.21.15.5:0/1717529648 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f41f8004750 con 0x55f72ec22b40 2024-04-24T06:46:56.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.147+0000 7f420e5ae740 10 client.25916 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:46:56.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.147+0000 7f420e5ae740 10 client.25916 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-24T06:46:56.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.147+0000 7f420e5ae740 20 client.25916 populate_metadata read hostname 'smithi005' 2024-04-24T06:46:56.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.147+0000 7f420e5ae740 1 --2- 172.21.15.5:0/1717529648 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55f72ec3dc80 0x55f72ec40070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.147+0000 7f420e5ae740 4 client.25916 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:46:56.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.147+0000 7f420e5ae740 1 -- 172.21.15.5:0/1717529648 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f72eb95bf0 con 0x55f72ec3dc80 2024-04-24T06:46:56.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.147+0000 7f4203e00640 1 --2- 172.21.15.5:0/1717529648 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55f72ec3dc80 0x55f72ec40070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.148+0000 7f4203e00640 1 --2- 172.21.15.5:0/1717529648 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55f72ec3dc80 0x55f72ec40070 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=27488bb97a78fbb7 server_cookie=1ce493a851a2477f in_seq=0 out_seq=0 2024-04-24T06:46:56.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.148+0000 7f41ea000640 10 client.25916 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:46:56.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.149+0000 7f41ea000640 1 -- 172.21.15.5:0/1717529648 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f72eb95bf0 con 0x55f72ec3dc80 2024-04-24T06:46:56.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.149+0000 7f41ea000640 10 client.25916 handle_command_reply: tid=0 2024-04-24T06:46:56.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.149+0000 7f420e5ae740 1 client.25916 shutdown 2024-04-24T06:46:56.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.149+0000 7f420e5ae740 1 -- 172.21.15.5:0/1717529648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f72ec22b40 msgr2=0x55f72ec2d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:56.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.149+0000 7f420e5ae740 1 --2- 172.21.15.5:0/1717529648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f72ec22b40 0x55f72ec2d540 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f41f80196e0 tx=0x7f41f800f7f0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.149+0000 7f420e5ae740 1 -- 172.21.15.5:0/1717529648 shutdown_connections 2024-04-24T06:46:56.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.149+0000 7f420e5ae740 1 --2- 172.21.15.5:0/1717529648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72ec2db30 0x55f72ec2af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.149+0000 7f420e5ae740 1 --2- 172.21.15.5:0/1717529648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f72ec23460 0x55f72ec2a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.149+0000 7f420e5ae740 1 --2- 172.21.15.5:0/1717529648 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55f72ec3dc80 0x55f72ec40070 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.149+0000 7f420e5ae740 1 --2- 172.21.15.5:0/1717529648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f72ec22b40 0x55f72ec2d540 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.149+0000 7f420e5ae740 1 -- 172.21.15.5:0/1717529648 >> 172.21.15.5:0/1717529648 conn(0x55f72ec1ee60 msgr2=0x55f72ec24c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:56.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.149+0000 7f41ea000640 0 client.25916 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:46:56.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.149+0000 7f420e5ae740 1 -- 172.21.15.5:0/1717529648 shutdown_connections 2024-04-24T06:46:56.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.149+0000 7f420e5ae740 1 -- 172.21.15.5:0/1717529648 wait complete. 2024-04-24T06:46:56.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.149+0000 7f420e5ae740 20 client.25916 trim_cache size 0 max 16384 2024-04-24T06:46:56.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.164+0000 7f420e5ae740 1 Processor -- start 2024-04-24T06:46:56.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.164+0000 7f420e5ae740 1 -- start start 2024-04-24T06:46:56.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.164+0000 7f420e5ae740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f72ec22b40 0x55f72ec39880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.164+0000 7f420e5ae740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72ec23460 0x55f72ec29e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.164+0000 7f420e5ae740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f72ec2db30 0x55f72ec458d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.164+0000 7f420e5ae740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f42041b2620 con 0x55f72ec23460 2024-04-24T06:46:56.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.164+0000 7f420e5ae740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f42041afef0 con 0x55f72ec2db30 2024-04-24T06:46:56.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.164+0000 7f420e5ae740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f42041add40 con 0x55f72ec22b40 2024-04-24T06:46:56.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.164+0000 7f4203e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72ec23460 0x55f72ec29e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.164+0000 7f4203e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72ec23460 0x55f72ec29e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39646/0 (socket says 172.21.15.5:39646) 2024-04-24T06:46:56.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.164+0000 7f4203e00640 1 -- 172.21.15.5:0/3843450364 learned_addr learned my addr 172.21.15.5:0/3843450364 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:56.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.164+0000 7f4208c00640 1 --2- 172.21.15.5:0/3843450364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f72ec22b40 0x55f72ec39880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.164+0000 7f4209600640 1 --2- 172.21.15.5:0/3843450364 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f72ec2db30 0x55f72ec458d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.165+0000 7f4203e00640 1 -- 172.21.15.5:0/3843450364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f72ec22b40 msgr2=0x55f72ec39880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:56.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.165+0000 7f4203e00640 1 --2- 172.21.15.5:0/3843450364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f72ec22b40 0x55f72ec39880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.165+0000 7f4203e00640 1 -- 172.21.15.5:0/3843450364 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f72ec2db30 msgr2=0x55f72ec458d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:56.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.165+0000 7f4203e00640 1 --2- 172.21.15.5:0/3843450364 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f72ec2db30 0x55f72ec458d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.165+0000 7f4203e00640 1 -- 172.21.15.5:0/3843450364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f72e9647d0 con 0x55f72ec23460 2024-04-24T06:46:56.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.165+0000 7f4208c00640 1 --2- 172.21.15.5:0/3843450364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f72ec22b40 0x55f72ec39880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:46:56.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.165+0000 7f4203e00640 1 --2- 172.21.15.5:0/3843450364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72ec23460 0x55f72ec29e00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f41f0005f30 tx=0x7f41f0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca2f3c949de17f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:56.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.165+0000 7f41f5200640 1 -- 172.21.15.5:0/3843450364 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f002b470 con 0x55f72ec23460 2024-04-24T06:46:56.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.165+0000 7f41f5200640 1 -- 172.21.15.5:0/3843450364 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41f0016070 con 0x55f72ec23460 2024-04-24T06:46:56.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.165+0000 7f41f5200640 1 -- 172.21.15.5:0/3843450364 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f00049e0 con 0x55f72ec23460 2024-04-24T06:46:56.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.166+0000 7f420e5ae740 1 -- 172.21.15.5:0/3843450364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72ec23460 msgr2=0x55f72ec29e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:56.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.166+0000 7f420e5ae740 1 --2- 172.21.15.5:0/3843450364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72ec23460 0x55f72ec29e00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f41f0005f30 tx=0x7f41f0009f90 comp rx=0 tx=0).stop 2024-04-24T06:46:56.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.166+0000 7f420e5ae740 1 -- 172.21.15.5:0/3843450364 shutdown_connections 2024-04-24T06:46:56.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.166+0000 7f420e5ae740 1 --2- 172.21.15.5:0/3843450364 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f72ec2db30 0x55f72ec458d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.166+0000 7f420e5ae740 1 --2- 172.21.15.5:0/3843450364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72ec23460 0x55f72ec29e00 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.166+0000 7f420e5ae740 1 --2- 172.21.15.5:0/3843450364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f72ec22b40 0x55f72ec39880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.166+0000 7f420e5ae740 1 -- 172.21.15.5:0/3843450364 >> 172.21.15.5:0/3843450364 conn(0x55f72ec1ee60 msgr2=0x55f72ec24290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.166+0000 7f420e5ae740 1 -- 172.21.15.5:0/3843450364 shutdown_connections 2024-04-24T06:46:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.166+0000 7f420e5ae740 1 -- 172.21.15.5:0/3843450364 wait complete. 2024-04-24T06:46:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.166+0000 7f420e5ae740 1 -- start start 2024-04-24T06:46:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.167+0000 7f420e5ae740 1 Processor -- start 2024-04-24T06:46:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.167+0000 7f420e5ae740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f72ec22b40 0x55f72ec36150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.167+0000 7f420e5ae740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f72ec23460 0x55f72ec2e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.167+0000 7f420e5ae740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72ec2db30 0x55f72ec2ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.167+0000 7f420e5ae740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f42041ae3b0 con 0x55f72ec2db30 2024-04-24T06:46:56.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.167+0000 7f420e5ae740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f42041af820 con 0x55f72ec22b40 2024-04-24T06:46:56.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.167+0000 7f420e5ae740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f42041aea80 con 0x55f72ec23460 2024-04-24T06:46:56.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.167+0000 7f420e5ae740 10 client.0 _ll_register_callbacks cb 0x7f41f801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:46:56.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.167+0000 7f4209600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72ec2db30 0x55f72ec2ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.167+0000 7f4208c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f72ec22b40 0x55f72ec36150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.167+0000 7f4208c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f72ec22b40 0x55f72ec36150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:38496/0 (socket says 172.21.15.5:38496) 2024-04-24T06:46:56.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.167+0000 7f4208c00640 1 -- 172.21.15.5:0/3357749327 learned_addr learned my addr 172.21.15.5:0/3357749327 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:56.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.167+0000 7f4203e00640 1 --2- 172.21.15.5:0/3357749327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f72ec23460 0x55f72ec2e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.168+0000 7f4208c00640 1 -- 172.21.15.5:0/3357749327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f72ec23460 msgr2=0x55f72ec2e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:56.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.168+0000 7f4208c00640 1 --2- 172.21.15.5:0/3357749327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f72ec23460 0x55f72ec2e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.168+0000 7f4208c00640 1 -- 172.21.15.5:0/3357749327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72ec2db30 msgr2=0x55f72ec2ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:56.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.168+0000 7f4208c00640 1 --2- 172.21.15.5:0/3357749327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72ec2db30 0x55f72ec2ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.168+0000 7f4208c00640 1 -- 172.21.15.5:0/3357749327 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f72ec2f300 con 0x55f72ec22b40 2024-04-24T06:46:56.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.168+0000 7f4208c00640 1 --2- 172.21.15.5:0/3357749327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f72ec22b40 0x55f72ec36150 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f41f8023460 tx=0x7f41f8005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2c8b89e2481a05f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:56.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.168+0000 7f41ea000640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:46:56.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.168+0000 7f41ea000640 1 -- 172.21.15.5:0/3357749327 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f803a060 con 0x55f72ec22b40 2024-04-24T06:46:56.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.168+0000 7f420e5ae740 1 -- 172.21.15.5:0/3357749327 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41f0016070 con 0x55f72ec22b40 2024-04-24T06:46:56.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.168+0000 7f41ea000640 1 -- 172.21.15.5:0/3357749327 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f41f8004880 con 0x55f72ec22b40 2024-04-24T06:46:56.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.169+0000 7f41ea000640 1 -- 172.21.15.5:0/3357749327 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f41f801e040 con 0x55f72ec22b40 2024-04-24T06:46:56.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.169+0000 7f420e5ae740 10 client.25922 fetch_fsmap learned FSMap version 25 2024-04-24T06:46:56.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.169+0000 7f420e5ae740 1 -- 172.21.15.5:0/3357749327 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f72eca4140 con 0x55f72ec22b40 2024-04-24T06:46:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.169+0000 7f41ea000640 1 -- 172.21.15.5:0/3357749327 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41f0016070 con 0x55f72ec22b40 2024-04-24T06:46:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.169+0000 7f41ea000640 1 -- 172.21.15.5:0/3357749327 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f41f8004dc0 con 0x55f72ec22b40 2024-04-24T06:46:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.169+0000 7f420e5ae740 10 client.25922 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:46:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.169+0000 7f420e5ae740 10 client.25922 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-24T06:46:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.169+0000 7f420e5ae740 20 client.25922 populate_metadata read hostname 'smithi005' 2024-04-24T06:46:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.169+0000 7f420e5ae740 1 --2- 172.21.15.5:0/3357749327 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55f72eca47b0 0x55f72eca6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.169+0000 7f420e5ae740 4 client.25922 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:46:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.169+0000 7f420e5ae740 1 -- 172.21.15.5:0/3357749327 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f72eb8fa60 con 0x55f72eca47b0 2024-04-24T06:46:56.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.170+0000 7f4203e00640 1 --2- 172.21.15.5:0/3357749327 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55f72eca47b0 0x55f72eca6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.170+0000 7f4203e00640 1 --2- 172.21.15.5:0/3357749327 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55f72eca47b0 0x55f72eca6ba0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e393a3bd557ac734 server_cookie=c85e722a3fbbdc28 in_seq=0 out_seq=0 2024-04-24T06:46:56.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.170+0000 7f41ea000640 10 client.25922 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:46:56.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.170+0000 7f41ea000640 1 -- 172.21.15.5:0/3357749327 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f72eb8fa60 con 0x55f72eca47b0 2024-04-24T06:46:56.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.170+0000 7f41ea000640 10 client.25922 handle_command_reply: tid=0 2024-04-24T06:46:56.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.170+0000 7f420e5ae740 1 client.25922 shutdown 2024-04-24T06:46:56.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.170+0000 7f420e5ae740 1 -- 172.21.15.5:0/3357749327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f72ec22b40 msgr2=0x55f72ec36150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:56.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.170+0000 7f420e5ae740 1 --2- 172.21.15.5:0/3357749327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f72ec22b40 0x55f72ec36150 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f41f8023460 tx=0x7f41f8005d30 comp rx=0 tx=0).stop 2024-04-24T06:46:56.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.171+0000 7f420e5ae740 1 -- 172.21.15.5:0/3357749327 shutdown_connections 2024-04-24T06:46:56.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.171+0000 7f420e5ae740 1 --2- 172.21.15.5:0/3357749327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f72ec2db30 0x55f72ec2ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.171+0000 7f420e5ae740 1 --2- 172.21.15.5:0/3357749327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f72ec23460 0x55f72ec2e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.171+0000 7f420e5ae740 1 --2- 172.21.15.5:0/3357749327 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55f72eca47b0 0x55f72eca6ba0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.171+0000 7f420e5ae740 1 --2- 172.21.15.5:0/3357749327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f72ec22b40 0x55f72ec36150 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.171+0000 7f420e5ae740 1 -- 172.21.15.5:0/3357749327 >> 172.21.15.5:0/3357749327 conn(0x55f72ec1ee60 msgr2=0x55f72ec24b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:56.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.171+0000 7f41ea000640 0 client.25922 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:46:56.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.171+0000 7f420e5ae740 1 -- 172.21.15.5:0/3357749327 shutdown_connections 2024-04-24T06:46:56.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.171+0000 7f420e5ae740 1 -- 172.21.15.5:0/3357749327 wait complete. 2024-04-24T06:46:56.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.171+0000 7f420e5ae740 20 client.25922 trim_cache size 0 max 16384 2024-04-24T06:46:56.182 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:46:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.171+0000 7f420a000640 1 -- 172.21.15.5:0/736107762 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f41ec0a3d20 msgr2=0x7f41ec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.171+0000 7f420a000640 1 --2- 172.21.15.5:0/736107762 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f41ec0a3d20 0x7f41ec0a61e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55f72ec1fff0 tx=0x7f41f00046c0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.171+0000 7f420a000640 1 -- 172.21.15.5:0/736107762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f420419a730 msgr2=0x7f420419abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.171+0000 7f420a000640 1 --2- 172.21.15.5:0/736107762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f420419a730 0x7f420419abb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f41fc005e30 tx=0x7f41fc003f10 comp rx=0 tx=0).stop 2024-04-24T06:46:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.172+0000 7f420a000640 1 -- 172.21.15.5:0/736107762 shutdown_connections 2024-04-24T06:46:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.172+0000 7f420a000640 1 --2- 172.21.15.5:0/736107762 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f41ec0a3d20 0x7f41ec0a61e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.172+0000 7f420a000640 1 --2- 172.21.15.5:0/736107762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f420419a730 0x7f420419abb0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.172+0000 7f420a000640 1 --2- 172.21.15.5:0/736107762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4204109320 0x7f42041a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.172+0000 7f420a000640 1 --2- 172.21.15.5:0/736107762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4204104250 0x7f42041a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.172+0000 7f420a000640 1 -- 172.21.15.5:0/736107762 >> 172.21.15.5:0/736107762 conn(0x7f42040ffad0 msgr2=0x7f4204101dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.172+0000 7f420a000640 1 -- 172.21.15.5:0/736107762 shutdown_connections 2024-04-24T06:46:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.172+0000 7f420a000640 1 -- 172.21.15.5:0/736107762 wait complete. 2024-04-24T06:46:56.191 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T06:46:56.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.399+0000 7f1a33e00640 1 -- 172.21.15.5:0/2887398335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a3410c420 msgr2=0x7f1a3410e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:56.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.399+0000 7f1a33e00640 1 --2- 172.21.15.5:0/2887398335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a3410c420 0x7f1a3410e870 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c009900 tx=0x7f1a2c02a410 comp rx=0 tx=0).stop 2024-04-24T06:46:56.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.399+0000 7f1a33e00640 1 -- 172.21.15.5:0/2887398335 shutdown_connections 2024-04-24T06:46:56.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.399+0000 7f1a33e00640 1 --2- 172.21.15.5:0/2887398335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a3410c420 0x7f1a3410e870 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.399+0000 7f1a33e00640 1 --2- 172.21.15.5:0/2887398335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a341038d0 0x7f1a3410bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.399+0000 7f1a33e00640 1 --2- 172.21.15.5:0/2887398335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a34102fb0 0x7f1a34103390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.399+0000 7f1a33e00640 1 -- 172.21.15.5:0/2887398335 >> 172.21.15.5:0/2887398335 conn(0x7f1a340fc810 msgr2=0x7f1a340fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:56.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.399+0000 7f1a33e00640 1 -- 172.21.15.5:0/2887398335 shutdown_connections 2024-04-24T06:46:56.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.399+0000 7f1a33e00640 1 -- 172.21.15.5:0/2887398335 wait complete. 2024-04-24T06:46:56.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.400+0000 7f1a33e00640 1 Processor -- start 2024-04-24T06:46:56.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.400+0000 7f1a33e00640 1 -- start start 2024-04-24T06:46:56.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.401+0000 7f1a33e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a34102fb0 0x7f1a341a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.401+0000 7f1a33e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a341038d0 0x7f1a341a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.401+0000 7f1a33e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a3410c420 0x7f1a3419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.401+0000 7f1a33e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1a34112210 con 0x7f1a34102fb0 2024-04-24T06:46:56.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.401+0000 7f1a33e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1a34112090 con 0x7f1a341038d0 2024-04-24T06:46:56.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.401+0000 7f1a33e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1a34112390 con 0x7f1a3410c420 2024-04-24T06:46:56.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.401+0000 7f1a32a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a34102fb0 0x7f1a341a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.401+0000 7f1a32a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a34102fb0 0x7f1a341a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39674/0 (socket says 172.21.15.5:39674) 2024-04-24T06:46:56.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.401+0000 7f1a32a00640 1 -- 172.21.15.5:0/1384274360 learned_addr learned my addr 172.21.15.5:0/1384274360 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:56.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.401+0000 7f1a32000640 1 --2- 172.21.15.5:0/1384274360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a341038d0 0x7f1a341a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.401+0000 7f1a33400640 1 --2- 172.21.15.5:0/1384274360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a3410c420 0x7f1a3419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.401+0000 7f1a32a00640 1 -- 172.21.15.5:0/1384274360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a3410c420 msgr2=0x7f1a3419a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:56.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.401+0000 7f1a32a00640 1 --2- 172.21.15.5:0/1384274360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a3410c420 0x7f1a3419a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.402+0000 7f1a32a00640 1 -- 172.21.15.5:0/1384274360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a341038d0 msgr2=0x7f1a341a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:56.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.402+0000 7f1a32a00640 1 --2- 172.21.15.5:0/1384274360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a341038d0 0x7f1a341a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.402+0000 7f1a32a00640 1 -- 172.21.15.5:0/1384274360 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a3419b010 con 0x7f1a34102fb0 2024-04-24T06:46:56.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.402+0000 7f1a32a00640 1 --2- 172.21.15.5:0/1384274360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a34102fb0 0x7f1a341a0640 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1a2400e9f0 tx=0x7f1a2400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:56.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.402+0000 7f1a2b400640 1 -- 172.21.15.5:0/1384274360 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2400cd80 con 0x7f1a34102fb0 2024-04-24T06:46:56.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.402+0000 7f1a2b400640 1 -- 172.21.15.5:0/1384274360 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a24005500 con 0x7f1a34102fb0 2024-04-24T06:46:56.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.402+0000 7f1a2b400640 1 -- 172.21.15.5:0/1384274360 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a24010690 con 0x7f1a34102fb0 2024-04-24T06:46:56.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.402+0000 7f1a33e00640 1 -- 172.21.15.5:0/1384274360 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a3419b300 con 0x7f1a34102fb0 2024-04-24T06:46:56.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.402+0000 7f1a33e00640 1 -- 172.21.15.5:0/1384274360 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a341a75c0 con 0x7f1a34102fb0 2024-04-24T06:46:56.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.405+0000 7f1a3c791740 1 Processor -- start 2024-04-24T06:46:56.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.405+0000 7f1a2b400640 1 -- 172.21.15.5:0/1384274360 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f1a240046c0 con 0x7f1a34102fb0 2024-04-24T06:46:56.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.406+0000 7f1a3c791740 1 -- start start 2024-04-24T06:46:56.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.406+0000 7f1a3c791740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38d329b40 0x55a38d329f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.406+0000 7f1a3c791740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a38d32a460 0x55a38d3345f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.406+0000 7f1a3c791740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d334b30 0x55a38d336f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.406+0000 7f1a3c791740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a38d28f1d0 con 0x55a38d334b30 2024-04-24T06:46:56.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.406+0000 7f1a3c791740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a38d235450 con 0x55a38d329b40 2024-04-24T06:46:56.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.406+0000 7f1a3c791740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a38d28b190 con 0x55a38d32a460 2024-04-24T06:46:56.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.406+0000 7f1a32000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d334b30 0x55a38d336f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.406+0000 7f1a32a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a38d32a460 0x55a38d3345f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.406+0000 7f1a32000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d334b30 0x55a38d336f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39690/0 (socket says 172.21.15.5:39690) 2024-04-24T06:46:56.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.406+0000 7f1a32000640 1 -- 172.21.15.5:0/758476856 learned_addr learned my addr 172.21.15.5:0/758476856 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:56.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.407+0000 7f1a33400640 1 --2- 172.21.15.5:0/758476856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38d329b40 0x55a38d329f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.407+0000 7f1a32a00640 1 -- 172.21.15.5:0/758476856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38d329b40 msgr2=0x55a38d329f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:56.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.407+0000 7f1a32a00640 1 --2- 172.21.15.5:0/758476856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38d329b40 0x55a38d329f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.407+0000 7f1a32a00640 1 -- 172.21.15.5:0/758476856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d334b30 msgr2=0x55a38d336f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:56.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.407+0000 7f1a32a00640 1 --2- 172.21.15.5:0/758476856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d334b30 0x55a38d336f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.407+0000 7f1a32a00640 1 -- 172.21.15.5:0/758476856 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a38d2f2310 con 0x55a38d32a460 2024-04-24T06:46:56.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.407+0000 7f1a32a00640 1 --2- 172.21.15.5:0/758476856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a38d32a460 0x55a38d3345f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1a240ce390 tx=0x7f1a240d4d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b18c5c6aa5dff998 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:56.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.407+0000 7f1a2b400640 1 --2- 172.21.15.5:0/1384274360 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1a140a3da0 0x7f1a140a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.408+0000 7f1a33400640 1 --2- 172.21.15.5:0/1384274360 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1a140a3da0 0x7f1a140a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.408+0000 7f1a2b400640 1 -- 172.21.15.5:0/1384274360 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f1a240d0c90 con 0x7f1a34102fb0 2024-04-24T06:46:56.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.408+0000 7f1a23e00640 1 -- 172.21.15.5:0/758476856 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a24098e80 con 0x55a38d32a460 2024-04-24T06:46:56.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.408+0000 7f1a23e00640 1 -- 172.21.15.5:0/758476856 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a240de070 con 0x55a38d32a460 2024-04-24T06:46:56.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.408+0000 7f1a23e00640 1 -- 172.21.15.5:0/758476856 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2409c540 con 0x55a38d32a460 2024-04-24T06:46:56.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.408+0000 7f1a3c791740 1 -- 172.21.15.5:0/758476856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a38d32a460 msgr2=0x55a38d3345f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:56.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.408+0000 7f1a3c791740 1 --2- 172.21.15.5:0/758476856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a38d32a460 0x55a38d3345f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1a240ce390 tx=0x7f1a240d4d30 comp rx=0 tx=0).stop 2024-04-24T06:46:56.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.408+0000 7f1a33400640 1 --2- 172.21.15.5:0/1384274360 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1a140a3da0 0x7f1a140a6260 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c005c40 tx=0x7f1a2c005b70 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:56.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.409+0000 7f1a3c791740 1 -- 172.21.15.5:0/758476856 shutdown_connections 2024-04-24T06:46:56.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.409+0000 7f1a3c791740 1 --2- 172.21.15.5:0/758476856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d334b30 0x55a38d336f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.409+0000 7f1a3c791740 1 --2- 172.21.15.5:0/758476856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a38d32a460 0x55a38d3345f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.409+0000 7f1a3c791740 1 --2- 172.21.15.5:0/758476856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38d329b40 0x55a38d329f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.409+0000 7f1a3c791740 1 -- 172.21.15.5:0/758476856 >> 172.21.15.5:0/758476856 conn(0x55a38d325e60 msgr2=0x55a38d326240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:56.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.409+0000 7f1a3c791740 1 -- 172.21.15.5:0/758476856 shutdown_connections 2024-04-24T06:46:56.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.409+0000 7f1a3c791740 1 -- 172.21.15.5:0/758476856 wait complete. 2024-04-24T06:46:56.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.409+0000 7f1a3c791740 1 -- start start 2024-04-24T06:46:56.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.409+0000 7f1a3c791740 1 Processor -- start 2024-04-24T06:46:56.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.410+0000 7f1a3c791740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a38d329b40 0x55a38d3345d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.410+0000 7f1a3c791740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38d32a460 0x55a38d3318c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.410+0000 7f1a3c791740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d334b30 0x55a38d331fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.410+0000 7f1a3c791740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a38d3379c0 con 0x55a38d334b30 2024-04-24T06:46:56.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.410+0000 7f1a3c791740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a38d248810 con 0x55a38d32a460 2024-04-24T06:46:56.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.410+0000 7f1a3c791740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a38d337b40 con 0x55a38d329b40 2024-04-24T06:46:56.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.410+0000 7f1a3c791740 10 client.0 _ll_register_callbacks cb 0x55a38d216140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:46:56.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.410+0000 7f1a33400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d334b30 0x55a38d331fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.410+0000 7f1a32000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38d32a460 0x55a38d3318c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.410+0000 7f1a32000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38d32a460 0x55a38d3318c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:38536/0 (socket says 172.21.15.5:38536) 2024-04-24T06:46:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.410+0000 7f1a33400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d334b30 0x55a38d331fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39704/0 (socket says 172.21.15.5:39704) 2024-04-24T06:46:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.410+0000 7f1a32000640 1 -- 172.21.15.5:0/2395035293 learned_addr learned my addr 172.21.15.5:0/2395035293 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.410+0000 7f1a32a00640 1 --2- 172.21.15.5:0/2395035293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a38d329b40 0x55a38d3345d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.411+0000 7f1a32a00640 1 -- 172.21.15.5:0/2395035293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38d32a460 msgr2=0x55a38d3318c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.411+0000 7f1a32a00640 1 --2- 172.21.15.5:0/2395035293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38d32a460 0x55a38d3318c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.411+0000 7f1a32a00640 1 -- 172.21.15.5:0/2395035293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d334b30 msgr2=0x55a38d331fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.411+0000 7f1a32a00640 1 --2- 172.21.15.5:0/2395035293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d334b30 0x55a38d331fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.411+0000 7f1a32a00640 1 -- 172.21.15.5:0/2395035293 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a38d2593f0 con 0x55a38d329b40 2024-04-24T06:46:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.411+0000 7f1a32000640 1 --2- 172.21.15.5:0/2395035293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38d32a460 0x55a38d3318c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:46:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.411+0000 7f1a33400640 1 --2- 172.21.15.5:0/2395035293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d334b30 0x55a38d331fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:46:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.411+0000 7f1a32a00640 1 --2- 172.21.15.5:0/2395035293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a38d329b40 0x55a38d3345d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1a240d4340 tx=0x7f1a240ce330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ee41934a6d6d041 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.411+0000 7f1a20c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:46:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.411+0000 7f1a20c00640 1 -- 172.21.15.5:0/2395035293 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2409c720 con 0x55a38d329b40 2024-04-24T06:46:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.411+0000 7f1a3c791740 1 -- 172.21.15.5:0/2395035293 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a240de070 con 0x55a38d329b40 2024-04-24T06:46:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.411+0000 7f1a20c00640 1 -- 172.21.15.5:0/2395035293 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f1a240bd0b0 con 0x55a38d329b40 2024-04-24T06:46:56.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.411+0000 7f1a20c00640 1 -- 172.21.15.5:0/2395035293 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1a2409c9b0 con 0x55a38d329b40 2024-04-24T06:46:56.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.411+0000 7f1a3c791740 10 client.25960 fetch_fsmap learned FSMap version 25 2024-04-24T06:46:56.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.411+0000 7f1a3c791740 1 -- 172.21.15.5:0/2395035293 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a38cfa13f0 con 0x55a38d329b40 2024-04-24T06:46:56.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.412+0000 7f1a20c00640 1 -- 172.21.15.5:0/2395035293 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a240de070 con 0x55a38d329b40 2024-04-24T06:46:56.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.412+0000 7f1a20c00640 1 -- 172.21.15.5:0/2395035293 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1a240ce610 con 0x55a38d329b40 2024-04-24T06:46:56.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.412+0000 7f1a3c791740 10 client.25960 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:46:56.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.412+0000 7f1a3c791740 10 client.25960 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T06:46:56.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.412+0000 7f1a3c791740 20 client.25960 populate_metadata read hostname 'smithi005' 2024-04-24T06:46:56.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.412+0000 7f1a3c791740 1 --2- 172.21.15.5:0/2395035293 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55a38d344c00 0x55a38d346ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.412+0000 7f1a3c791740 4 client.25960 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:46:56.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.412+0000 7f1a3c791740 1 -- 172.21.15.5:0/2395035293 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a38d29cbf0 con 0x55a38d344c00 2024-04-24T06:46:56.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.412+0000 7f1a32000640 1 --2- 172.21.15.5:0/2395035293 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55a38d344c00 0x55a38d346ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.413+0000 7f1a32000640 1 --2- 172.21.15.5:0/2395035293 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55a38d344c00 0x55a38d346ff0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b72cc139d9d351d9 server_cookie=d46e9c8e82199715 in_seq=0 out_seq=0 2024-04-24T06:46:56.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.413+0000 7f1a20c00640 10 client.25960 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:46:56.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.414+0000 7f1a20c00640 1 -- 172.21.15.5:0/2395035293 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a38d29cbf0 con 0x55a38d344c00 2024-04-24T06:46:56.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.414+0000 7f1a20c00640 10 client.25960 handle_command_reply: tid=0 2024-04-24T06:46:56.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.414+0000 7f1a3c791740 1 client.25960 shutdown 2024-04-24T06:46:56.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.414+0000 7f1a3c791740 1 -- 172.21.15.5:0/2395035293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a38d329b40 msgr2=0x55a38d3345d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:56.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.414+0000 7f1a3c791740 1 --2- 172.21.15.5:0/2395035293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a38d329b40 0x55a38d3345d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1a240d4340 tx=0x7f1a240ce330 comp rx=0 tx=0).stop 2024-04-24T06:46:56.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.414+0000 7f1a3c791740 1 -- 172.21.15.5:0/2395035293 shutdown_connections 2024-04-24T06:46:56.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.414+0000 7f1a3c791740 1 --2- 172.21.15.5:0/2395035293 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55a38d344c00 0x55a38d346ff0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.414+0000 7f1a3c791740 1 --2- 172.21.15.5:0/2395035293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d334b30 0x55a38d331fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.414+0000 7f1a3c791740 1 --2- 172.21.15.5:0/2395035293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38d32a460 0x55a38d3318c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.414+0000 7f1a20c00640 0 client.25960 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:46:56.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.414+0000 7f1a3c791740 1 --2- 172.21.15.5:0/2395035293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a38d329b40 0x55a38d3345d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.414+0000 7f1a3c791740 1 -- 172.21.15.5:0/2395035293 >> 172.21.15.5:0/2395035293 conn(0x55a38d325e60 msgr2=0x55a38d32bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:56.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.414+0000 7f1a3c791740 1 -- 172.21.15.5:0/2395035293 shutdown_connections 2024-04-24T06:46:56.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.414+0000 7f1a3c791740 1 -- 172.21.15.5:0/2395035293 wait complete. 2024-04-24T06:46:56.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.414+0000 7f1a3c791740 20 client.25960 trim_cache size 0 max 16384 2024-04-24T06:46:56.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.429+0000 7f1a3c791740 1 Processor -- start 2024-04-24T06:46:56.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.429+0000 7f1a3c791740 1 -- start start 2024-04-24T06:46:56.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.430+0000 7f1a3c791740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a38d329b40 0x55a38d334100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.430+0000 7f1a3c791740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38d32a460 0x55a38d331150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.430+0000 7f1a3c791740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d334b30 0x55a38d331690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.430+0000 7f1a3c791740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1a341b25c0 con 0x55a38d334b30 2024-04-24T06:46:56.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.430+0000 7f1a3c791740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1a341afe90 con 0x55a38d32a460 2024-04-24T06:46:56.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.430+0000 7f1a3c791740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1a341adce0 con 0x55a38d329b40 2024-04-24T06:46:56.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.430+0000 7f1a33400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d334b30 0x55a38d331690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.430+0000 7f1a32a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a38d329b40 0x55a38d334100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.430+0000 7f1a32000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38d32a460 0x55a38d331150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.430+0000 7f1a33400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d334b30 0x55a38d331690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39712/0 (socket says 172.21.15.5:39712) 2024-04-24T06:46:56.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.430+0000 7f1a32a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a38d329b40 0x55a38d334100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:56272/0 (socket says 172.21.15.5:56272) 2024-04-24T06:46:56.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.430+0000 7f1a33400640 1 -- 172.21.15.5:0/3703400328 learned_addr learned my addr 172.21.15.5:0/3703400328 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:56.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.430+0000 7f1a32000640 1 -- 172.21.15.5:0/3703400328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a38d329b40 msgr2=0x55a38d334100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:56.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.430+0000 7f1a32000640 1 --2- 172.21.15.5:0/3703400328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a38d329b40 0x55a38d334100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.430+0000 7f1a32000640 1 -- 172.21.15.5:0/3703400328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d334b30 msgr2=0x55a38d331690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:56.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.430+0000 7f1a32000640 1 --2- 172.21.15.5:0/3703400328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d334b30 0x55a38d331690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.430+0000 7f1a32000640 1 -- 172.21.15.5:0/3703400328 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a38d06b7d0 con 0x55a38d32a460 2024-04-24T06:46:56.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.431+0000 7f1a32000640 1 --2- 172.21.15.5:0/3703400328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38d32a460 0x55a38d331150 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55a38d337b80 tx=0x7f1a1c009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f155de20b398384 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:56.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.431+0000 7f1a23400640 1 -- 172.21.15.5:0/3703400328 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c0201b0 con 0x55a38d32a460 2024-04-24T06:46:56.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.431+0000 7f1a23400640 1 -- 172.21.15.5:0/3703400328 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a1c020830 con 0x55a38d32a460 2024-04-24T06:46:56.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.431+0000 7f1a23400640 1 -- 172.21.15.5:0/3703400328 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c01d3f0 con 0x55a38d32a460 2024-04-24T06:46:56.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.432+0000 7f1a3c791740 1 -- 172.21.15.5:0/3703400328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38d32a460 msgr2=0x55a38d331150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:56.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.432+0000 7f1a3c791740 1 --2- 172.21.15.5:0/3703400328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38d32a460 0x55a38d331150 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55a38d337b80 tx=0x7f1a1c009500 comp rx=0 tx=0).stop 2024-04-24T06:46:56.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.432+0000 7f1a3c791740 1 -- 172.21.15.5:0/3703400328 shutdown_connections 2024-04-24T06:46:56.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.432+0000 7f1a3c791740 1 --2- 172.21.15.5:0/3703400328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d334b30 0x55a38d331690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.432+0000 7f1a3c791740 1 --2- 172.21.15.5:0/3703400328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38d32a460 0x55a38d331150 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.432+0000 7f1a3c791740 1 --2- 172.21.15.5:0/3703400328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a38d329b40 0x55a38d334100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.432+0000 7f1a3c791740 1 -- 172.21.15.5:0/3703400328 >> 172.21.15.5:0/3703400328 conn(0x55a38d325e60 msgr2=0x55a38d330190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:56.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.432+0000 7f1a3c791740 1 -- 172.21.15.5:0/3703400328 shutdown_connections 2024-04-24T06:46:56.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.432+0000 7f1a3c791740 1 -- 172.21.15.5:0/3703400328 wait complete. 2024-04-24T06:46:56.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.432+0000 7f1a3c791740 1 -- start start 2024-04-24T06:46:56.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.432+0000 7f1a3c791740 1 Processor -- start 2024-04-24T06:46:56.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.433+0000 7f1a3c791740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38d329b40 0x55a38d344fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.433+0000 7f1a3c791740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a38d32a460 0x55a38d345520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.433+0000 7f1a3c791740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d334b30 0x55a38d3a8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.433+0000 7f1a3c791740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1a341ae350 con 0x55a38d334b30 2024-04-24T06:46:56.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.433+0000 7f1a3c791740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1a341af7c0 con 0x55a38d329b40 2024-04-24T06:46:56.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.433+0000 7f1a3c791740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1a341aea20 con 0x55a38d32a460 2024-04-24T06:46:56.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.433+0000 7f1a3c791740 10 client.0 _ll_register_callbacks cb 0x7f1a240d2030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:46:56.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.433+0000 7f1a33400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d334b30 0x55a38d3a8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.433+0000 7f1a32a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38d329b40 0x55a38d344fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.433+0000 7f1a33400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d334b30 0x55a38d3a8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39714/0 (socket says 172.21.15.5:39714) 2024-04-24T06:46:56.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.433+0000 7f1a33400640 1 -- 172.21.15.5:0/1534048262 learned_addr learned my addr 172.21.15.5:0/1534048262 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:56.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.433+0000 7f1a32000640 1 --2- 172.21.15.5:0/1534048262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a38d32a460 0x55a38d345520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.434+0000 7f1a32a00640 1 -- 172.21.15.5:0/1534048262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a38d32a460 msgr2=0x55a38d345520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:56.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.434+0000 7f1a32a00640 1 --2- 172.21.15.5:0/1534048262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a38d32a460 0x55a38d345520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.434+0000 7f1a32a00640 1 -- 172.21.15.5:0/1534048262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d334b30 msgr2=0x55a38d3a8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:56.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.434+0000 7f1a32a00640 1 --2- 172.21.15.5:0/1534048262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d334b30 0x55a38d3a8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.434+0000 7f1a32a00640 1 -- 172.21.15.5:0/1534048262 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a38d3a9300 con 0x55a38d329b40 2024-04-24T06:46:56.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.434+0000 7f1a32a00640 1 --2- 172.21.15.5:0/1534048262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38d329b40 0x55a38d344fe0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1a2400c6a0 tx=0x7f1a2409cbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc85b29c290f296e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:56.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.434+0000 7f1a20c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:46:56.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.434+0000 7f1a20c00640 1 -- 172.21.15.5:0/1534048262 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2409c720 con 0x55a38d329b40 2024-04-24T06:46:56.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.434+0000 7f1a3c791740 1 -- 172.21.15.5:0/1534048262 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a1c020830 con 0x55a38d329b40 2024-04-24T06:46:56.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.434+0000 7f1a20c00640 1 -- 172.21.15.5:0/1534048262 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f1a240ce610 con 0x55a38d329b40 2024-04-24T06:46:56.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.434+0000 7f1a20c00640 1 -- 172.21.15.5:0/1534048262 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1a240aa050 con 0x55a38d329b40 2024-04-24T06:46:56.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.434+0000 7f1a3c791740 10 client.25940 fetch_fsmap learned FSMap version 25 2024-04-24T06:46:56.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.434+0000 7f1a3c791740 1 -- 172.21.15.5:0/1534048262 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a38d3aa410 con 0x55a38d329b40 2024-04-24T06:46:56.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.435+0000 7f1a20c00640 1 -- 172.21.15.5:0/1534048262 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a1c020830 con 0x55a38d329b40 2024-04-24T06:46:56.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.435+0000 7f1a20c00640 1 -- 172.21.15.5:0/1534048262 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1a240a53c0 con 0x55a38d329b40 2024-04-24T06:46:56.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.435+0000 7f1a3c791740 10 client.25940 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:46:56.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.435+0000 7f1a3c791740 10 client.25940 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T06:46:56.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.435+0000 7f1a3c791740 20 client.25940 populate_metadata read hostname 'smithi005' 2024-04-24T06:46:56.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.435+0000 7f1a3c791740 1 --2- 172.21.15.5:0/1534048262 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55a38d3aa970 0x55a38d3acd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.435+0000 7f1a3c791740 4 client.25940 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:46:56.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.435+0000 7f1a3c791740 1 -- 172.21.15.5:0/1534048262 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a38d296a60 con 0x55a38d3aa970 2024-04-24T06:46:56.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.435+0000 7f1a32000640 1 --2- 172.21.15.5:0/1534048262 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55a38d3aa970 0x55a38d3acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.436+0000 7f1a32000640 1 --2- 172.21.15.5:0/1534048262 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55a38d3aa970 0x55a38d3acd90 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=34fd132cd1f9525 server_cookie=ea1d7790c0eda2 in_seq=0 out_seq=0 2024-04-24T06:46:56.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.436+0000 7f1a20c00640 10 client.25940 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:46:56.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.436+0000 7f1a20c00640 1 -- 172.21.15.5:0/1534048262 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a38d296a60 con 0x55a38d3aa970 2024-04-24T06:46:56.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.436+0000 7f1a20c00640 10 client.25940 handle_command_reply: tid=0 2024-04-24T06:46:56.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.436+0000 7f1a3c791740 1 client.25940 shutdown 2024-04-24T06:46:56.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.436+0000 7f1a3c791740 1 -- 172.21.15.5:0/1534048262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38d329b40 msgr2=0x55a38d344fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:56.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.436+0000 7f1a3c791740 1 --2- 172.21.15.5:0/1534048262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38d329b40 0x55a38d344fe0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1a2400c6a0 tx=0x7f1a2409cbe0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.436+0000 7f1a3c791740 1 -- 172.21.15.5:0/1534048262 shutdown_connections 2024-04-24T06:46:56.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.436+0000 7f1a3c791740 1 --2- 172.21.15.5:0/1534048262 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55a38d3aa970 0x55a38d3acd90 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.436+0000 7f1a3c791740 1 --2- 172.21.15.5:0/1534048262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d334b30 0x55a38d3a8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.436+0000 7f1a3c791740 1 --2- 172.21.15.5:0/1534048262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a38d32a460 0x55a38d345520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.436+0000 7f1a3c791740 1 --2- 172.21.15.5:0/1534048262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38d329b40 0x55a38d344fe0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.436+0000 7f1a20c00640 0 client.25940 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:46:56.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.436+0000 7f1a3c791740 1 -- 172.21.15.5:0/1534048262 >> 172.21.15.5:0/1534048262 conn(0x55a38d325e60 msgr2=0x55a38d331150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:56.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.437+0000 7f1a3c791740 1 -- 172.21.15.5:0/1534048262 shutdown_connections 2024-04-24T06:46:56.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.437+0000 7f1a3c791740 1 -- 172.21.15.5:0/1534048262 wait complete. 2024-04-24T06:46:56.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.437+0000 7f1a3c791740 20 client.25940 trim_cache size 0 max 16384 2024-04-24T06:46:56.446 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:46:56.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.437+0000 7f1a33e00640 1 -- 172.21.15.5:0/1384274360 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1a140a3da0 msgr2=0x7f1a140a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:56.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.437+0000 7f1a33e00640 1 --2- 172.21.15.5:0/1384274360 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1a140a3da0 0x7f1a140a6260 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c005c40 tx=0x7f1a2c005b70 comp rx=0 tx=0).stop 2024-04-24T06:46:56.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.437+0000 7f1a33e00640 1 -- 172.21.15.5:0/1384274360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a34102fb0 msgr2=0x7f1a341a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:56.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.437+0000 7f1a33e00640 1 --2- 172.21.15.5:0/1384274360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a34102fb0 0x7f1a341a0640 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1a2400e9f0 tx=0x7f1a2400edf0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.438+0000 7f1a33e00640 1 -- 172.21.15.5:0/1384274360 shutdown_connections 2024-04-24T06:46:56.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.438+0000 7f1a33e00640 1 --2- 172.21.15.5:0/1384274360 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1a140a3da0 0x7f1a140a6260 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.438+0000 7f1a33e00640 1 --2- 172.21.15.5:0/1384274360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a3410c420 0x7f1a3419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.438+0000 7f1a33e00640 1 --2- 172.21.15.5:0/1384274360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a341038d0 0x7f1a341a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.438+0000 7f1a33e00640 1 --2- 172.21.15.5:0/1384274360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a34102fb0 0x7f1a341a0640 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.438+0000 7f1a33e00640 1 -- 172.21.15.5:0/1384274360 >> 172.21.15.5:0/1384274360 conn(0x7f1a340fc810 msgr2=0x7f1a34101c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:56.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.438+0000 7f1a33e00640 1 -- 172.21.15.5:0/1384274360 shutdown_connections 2024-04-24T06:46:56.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.438+0000 7f1a33e00640 1 -- 172.21.15.5:0/1384274360 wait complete. 2024-04-24T06:46:56.455 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T06:46:56.518 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:56 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/2427183470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:46:56.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.660+0000 7fefdc600640 1 -- 172.21.15.5:0/1977906811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fefd4104e00 msgr2=0x7fefd4108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:56.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.660+0000 7fefdc600640 1 --2- 172.21.15.5:0/1977906811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fefd4104e00 0x7fefd4108e40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fefc0009900 tx=0x7fefc002a3f0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.661+0000 7fefdc600640 1 -- 172.21.15.5:0/1977906811 shutdown_connections 2024-04-24T06:46:56.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.661+0000 7fefdc600640 1 --2- 172.21.15.5:0/1977906811 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefd4109520 0x7fefd41110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.661+0000 7fefdc600640 1 --2- 172.21.15.5:0/1977906811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fefd4104e00 0x7fefd4108e40 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.661+0000 7fefdc600640 1 --2- 172.21.15.5:0/1977906811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefd4104450 0x7fefd4104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.661+0000 7fefdc600640 1 -- 172.21.15.5:0/1977906811 >> 172.21.15.5:0/1977906811 conn(0x7fefd40ffc90 msgr2=0x7fefd41020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:56.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.661+0000 7fefdc600640 1 -- 172.21.15.5:0/1977906811 shutdown_connections 2024-04-24T06:46:56.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.661+0000 7fefdc600640 1 -- 172.21.15.5:0/1977906811 wait complete. 2024-04-24T06:46:56.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.662+0000 7fefdc600640 1 Processor -- start 2024-04-24T06:46:56.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.662+0000 7fefdc600640 1 -- start start 2024-04-24T06:46:56.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.663+0000 7fefdc600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefd4104450 0x7fefd41a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.663+0000 7fefdc600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fefd4109520 0x7fefd41a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.663+0000 7fefdc600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefd419a960 0x7fefd419ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.663+0000 7fefdc600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fefd41145e0 con 0x7fefd4109520 2024-04-24T06:46:56.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.663+0000 7fefdc600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fefd4114460 con 0x7fefd4104450 2024-04-24T06:46:56.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.663+0000 7fefdc600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fefd4114760 con 0x7fefd419a960 2024-04-24T06:46:56.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.663+0000 7fefd9200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fefd4109520 0x7fefd41a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.663+0000 7fefd9c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefd4104450 0x7fefd41a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.663+0000 7fefd9200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fefd4109520 0x7fefd41a0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39734/0 (socket says 172.21.15.5:39734) 2024-04-24T06:46:56.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.663+0000 7fefd9200640 1 -- 172.21.15.5:0/1210482761 learned_addr learned my addr 172.21.15.5:0/1210482761 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:56.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.663+0000 7fefda600640 1 --2- 172.21.15.5:0/1210482761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefd419a960 0x7fefd419ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.664+0000 7fefd9200640 1 -- 172.21.15.5:0/1210482761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefd419a960 msgr2=0x7fefd419ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:56.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.664+0000 7fefd9200640 1 --2- 172.21.15.5:0/1210482761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefd419a960 0x7fefd419ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.664+0000 7fefd9200640 1 -- 172.21.15.5:0/1210482761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefd4104450 msgr2=0x7fefd41a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:56.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.664+0000 7fefd9200640 1 --2- 172.21.15.5:0/1210482761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefd4104450 0x7fefd41a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.664+0000 7fefd9200640 1 -- 172.21.15.5:0/1210482761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd419b480 con 0x7fefd4109520 2024-04-24T06:46:56.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.664+0000 7fefd9c00640 1 --2- 172.21.15.5:0/1210482761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefd4104450 0x7fefd41a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:46:56.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.664+0000 7fefd9200640 1 --2- 172.21.15.5:0/1210482761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fefd4109520 0x7fefd41a0ca0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fefc002a910 tx=0x7fefc0003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:56.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.664+0000 7fefd2000640 1 -- 172.21.15.5:0/1210482761 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc003f070 con 0x7fefd4109520 2024-04-24T06:46:56.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.664+0000 7fefd2000640 1 -- 172.21.15.5:0/1210482761 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefc0031c90 con 0x7fefd4109520 2024-04-24T06:46:56.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.664+0000 7fefdc600640 1 -- 172.21.15.5:0/1210482761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefd419b710 con 0x7fefd4109520 2024-04-24T06:46:56.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.664+0000 7fefd2000640 1 -- 172.21.15.5:0/1210482761 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc0035030 con 0x7fefd4109520 2024-04-24T06:46:56.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.665+0000 7fefdc600640 1 -- 172.21.15.5:0/1210482761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefd41a78b0 con 0x7fefd4109520 2024-04-24T06:46:56.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.668+0000 7fefd2000640 1 -- 172.21.15.5:0/1210482761 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fefc0042070 con 0x7fefd4109520 2024-04-24T06:46:56.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.668+0000 7fefdf476740 1 Processor -- start 2024-04-24T06:46:56.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.668+0000 7fefdf476740 1 -- start start 2024-04-24T06:46:56.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.669+0000 7fefdf476740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5633c7088950 0x5633c7088d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.669+0000 7fefdf476740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633c7089270 0x5633c7093400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.669+0000 7fefdf476740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633c7093940 0x5633c7095d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.669+0000 7fefdf476740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5633c6fee1d0 con 0x5633c7089270 2024-04-24T06:46:56.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.669+0000 7fefdf476740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5633c6f94450 con 0x5633c7093940 2024-04-24T06:46:56.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.669+0000 7fefdf476740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5633c6fea190 con 0x5633c7088950 2024-04-24T06:46:56.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.669+0000 7fefd9c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633c7089270 0x5633c7093400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.669+0000 7fefda600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5633c7088950 0x5633c7088d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.669+0000 7fefd9c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633c7089270 0x5633c7093400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39744/0 (socket says 172.21.15.5:39744) 2024-04-24T06:46:56.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.669+0000 7fefd9c00640 1 -- 172.21.15.5:0/2609569450 learned_addr learned my addr 172.21.15.5:0/2609569450 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:56.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.669+0000 7fefd9200640 1 --2- 172.21.15.5:0/2609569450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633c7093940 0x5633c7095d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.670+0000 7fefda600640 1 -- 172.21.15.5:0/2609569450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633c7093940 msgr2=0x5633c7095d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:56.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.670+0000 7fefda600640 1 --2- 172.21.15.5:0/2609569450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633c7093940 0x5633c7095d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.670+0000 7fefda600640 1 -- 172.21.15.5:0/2609569450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633c7089270 msgr2=0x5633c7093400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:56.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.670+0000 7fefda600640 1 --2- 172.21.15.5:0/2609569450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633c7089270 0x5633c7093400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.670+0000 7fefda600640 1 -- 172.21.15.5:0/2609569450 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633c7051310 con 0x5633c7088950 2024-04-24T06:46:56.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.670+0000 7fefda600640 1 --2- 172.21.15.5:0/2609569450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5633c7088950 0x5633c7088d30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fefcc009b80 tx=0x7fefcc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1ead2abb7b1ff79 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:56.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.670+0000 7fefd2000640 1 --2- 172.21.15.5:0/1210482761 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fefbc0a3d20 0x7fefbc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.670+0000 7fefc6a00640 1 -- 172.21.15.5:0/2609569450 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefcc004050 con 0x5633c7088950 2024-04-24T06:46:56.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.670+0000 7fefc6a00640 1 -- 172.21.15.5:0/2609569450 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefcc005500 con 0x5633c7088950 2024-04-24T06:46:56.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.670+0000 7fefc6a00640 1 -- 172.21.15.5:0/2609569450 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefcc004cd0 con 0x5633c7088950 2024-04-24T06:46:56.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.670+0000 7fefd2000640 1 -- 172.21.15.5:0/1210482761 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fefc00f0e70 con 0x7fefd4109520 2024-04-24T06:46:56.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.671+0000 7fefd9c00640 1 --2- 172.21.15.5:0/1210482761 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fefbc0a3d20 0x7fefbc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.671+0000 7fefdf476740 1 -- 172.21.15.5:0/2609569450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5633c7088950 msgr2=0x5633c7088d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:56.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.671+0000 7fefdf476740 1 --2- 172.21.15.5:0/2609569450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5633c7088950 0x5633c7088d30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fefcc009b80 tx=0x7fefcc011810 comp rx=0 tx=0).stop 2024-04-24T06:46:56.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.671+0000 7fefdf476740 1 -- 172.21.15.5:0/2609569450 shutdown_connections 2024-04-24T06:46:56.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.671+0000 7fefdf476740 1 --2- 172.21.15.5:0/2609569450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633c7093940 0x5633c7095d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.671+0000 7fefdf476740 1 --2- 172.21.15.5:0/2609569450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633c7089270 0x5633c7093400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.671+0000 7fefdf476740 1 --2- 172.21.15.5:0/2609569450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5633c7088950 0x5633c7088d30 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.671+0000 7fefdf476740 1 -- 172.21.15.5:0/2609569450 >> 172.21.15.5:0/2609569450 conn(0x5633c7084c70 msgr2=0x5633c7085050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:56.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.671+0000 7fefd9c00640 1 --2- 172.21.15.5:0/1210482761 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fefbc0a3d20 0x7fefbc0a61e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fefc8009fd0 tx=0x7fefc8009500 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:56.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.671+0000 7fefdf476740 1 -- 172.21.15.5:0/2609569450 shutdown_connections 2024-04-24T06:46:56.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.671+0000 7fefdf476740 1 -- 172.21.15.5:0/2609569450 wait complete. 2024-04-24T06:46:56.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.672+0000 7fefdf476740 1 -- start start 2024-04-24T06:46:56.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.672+0000 7fefdf476740 1 Processor -- start 2024-04-24T06:46:56.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.673+0000 7fefdf476740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633c7088950 0x5633c7093390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.673+0000 7fefdf476740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633c7089270 0x5633c70906f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.673+0000 7fefdf476740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5633c7093940 0x5633c7090e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.673+0000 7fefdf476740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5633c70967d0 con 0x5633c7088950 2024-04-24T06:46:56.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.673+0000 7fefdf476740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5633c6fa7810 con 0x5633c7089270 2024-04-24T06:46:56.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.673+0000 7fefdf476740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5633c7096950 con 0x5633c7093940 2024-04-24T06:46:56.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.673+0000 7fefdf476740 10 client.0 _ll_register_callbacks cb 0x5633c6f75140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:46:56.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.673+0000 7fefd9200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633c7089270 0x5633c70906f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.673+0000 7fefd9200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633c7089270 0x5633c70906f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:38598/0 (socket says 172.21.15.5:38598) 2024-04-24T06:46:56.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.673+0000 7fefd9200640 1 -- 172.21.15.5:0/1553673979 learned_addr learned my addr 172.21.15.5:0/1553673979 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:56.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.673+0000 7fefd9c00640 1 --2- 172.21.15.5:0/1553673979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633c7088950 0x5633c7093390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.673+0000 7fefda600640 1 --2- 172.21.15.5:0/1553673979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5633c7093940 0x5633c7090e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.673+0000 7fefd9200640 1 -- 172.21.15.5:0/1553673979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5633c7093940 msgr2=0x5633c7090e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:56.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.673+0000 7fefd9200640 1 --2- 172.21.15.5:0/1553673979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5633c7093940 0x5633c7090e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.673+0000 7fefd9200640 1 -- 172.21.15.5:0/1553673979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633c7088950 msgr2=0x5633c7093390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:56.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.673+0000 7fefd9200640 1 --2- 172.21.15.5:0/1553673979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633c7088950 0x5633c7093390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.673+0000 7fefd9200640 1 -- 172.21.15.5:0/1553673979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633c6fb83f0 con 0x5633c7089270 2024-04-24T06:46:56.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.673+0000 7fefd9c00640 1 --2- 172.21.15.5:0/1553673979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633c7088950 0x5633c7093390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:46:56.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.674+0000 7fefd9200640 1 --2- 172.21.15.5:0/1553673979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633c7089270 0x5633c70906f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fefc00bcb70 tx=0x7fefc0003f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7c06f315689e69b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:56.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.674+0000 7fefbb400640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:46:56.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.674+0000 7fefbb400640 1 -- 172.21.15.5:0/1553673979 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc0049070 con 0x5633c7089270 2024-04-24T06:46:56.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.674+0000 7fefbb400640 1 -- 172.21.15.5:0/1553673979 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fefc00ee7f0 con 0x5633c7089270 2024-04-24T06:46:56.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.674+0000 7fefdf476740 1 -- 172.21.15.5:0/1553673979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefcc005500 con 0x5633c7089270 2024-04-24T06:46:56.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.674+0000 7fefbb400640 1 -- 172.21.15.5:0/1553673979 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fefc00bcc40 con 0x5633c7089270 2024-04-24T06:46:56.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.674+0000 7fefdf476740 10 client.25952 fetch_fsmap learned FSMap version 25 2024-04-24T06:46:56.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.674+0000 7fefdf476740 1 -- 172.21.15.5:0/1553673979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5633c6d003f0 con 0x5633c7089270 2024-04-24T06:46:56.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.675+0000 7fefbb400640 1 -- 172.21.15.5:0/1553673979 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefcc005500 con 0x5633c7089270 2024-04-24T06:46:56.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.675+0000 7fefbb400640 1 -- 172.21.15.5:0/1553673979 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fefc00bd0d0 con 0x5633c7089270 2024-04-24T06:46:56.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.675+0000 7fefdf476740 10 client.25952 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:46:56.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.675+0000 7fefdf476740 10 client.25952 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:46:56.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.675+0000 7fefdf476740 20 client.25952 populate_metadata read hostname 'smithi005' 2024-04-24T06:46:56.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.675+0000 7fefdf476740 1 --2- 172.21.15.5:0/1553673979 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5633c70a3b60 0x5633c70a5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.675+0000 7fefdf476740 4 client.25952 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:46:56.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.675+0000 7fefdf476740 1 -- 172.21.15.5:0/1553673979 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633c6ffbbf0 con 0x5633c70a3b60 2024-04-24T06:46:56.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.675+0000 7fefd9c00640 1 --2- 172.21.15.5:0/1553673979 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5633c70a3b60 0x5633c70a5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.676+0000 7fefbb400640 10 client.25952 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:46:56.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.676+0000 7fefd9c00640 1 --2- 172.21.15.5:0/1553673979 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5633c70a3b60 0x5633c70a5fe0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf3b766c34cc9236 server_cookie=fda2dfe2a9a9b6ec in_seq=0 out_seq=0 2024-04-24T06:46:56.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.677+0000 7fefbb400640 1 -- 172.21.15.5:0/1553673979 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5633c6ffbbf0 con 0x5633c70a3b60 2024-04-24T06:46:56.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.677+0000 7fefbb400640 10 client.25952 handle_command_reply: tid=0 2024-04-24T06:46:56.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.677+0000 7fefdf476740 1 client.25952 shutdown 2024-04-24T06:46:56.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.677+0000 7fefdf476740 1 -- 172.21.15.5:0/1553673979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633c7089270 msgr2=0x5633c70906f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:56.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.677+0000 7fefdf476740 1 --2- 172.21.15.5:0/1553673979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633c7089270 0x5633c70906f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fefc00bcb70 tx=0x7fefc0003f50 comp rx=0 tx=0).stop 2024-04-24T06:46:56.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.677+0000 7fefdf476740 1 -- 172.21.15.5:0/1553673979 shutdown_connections 2024-04-24T06:46:56.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.677+0000 7fefdf476740 1 --2- 172.21.15.5:0/1553673979 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5633c70a3b60 0x5633c70a5fe0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.677+0000 7fefdf476740 1 --2- 172.21.15.5:0/1553673979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5633c7093940 0x5633c7090e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.677+0000 7fefdf476740 1 --2- 172.21.15.5:0/1553673979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633c7089270 0x5633c70906f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.677+0000 7fefdf476740 1 --2- 172.21.15.5:0/1553673979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633c7088950 0x5633c7093390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.677+0000 7fefdf476740 1 -- 172.21.15.5:0/1553673979 >> 172.21.15.5:0/1553673979 conn(0x5633c7084c70 msgr2=0x5633c708aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:56.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.677+0000 7fefbb400640 0 client.25952 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:46:56.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.677+0000 7fefdf476740 1 -- 172.21.15.5:0/1553673979 shutdown_connections 2024-04-24T06:46:56.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.677+0000 7fefdf476740 1 -- 172.21.15.5:0/1553673979 wait complete. 2024-04-24T06:46:56.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.677+0000 7fefdf476740 20 client.25952 trim_cache size 0 max 16384 2024-04-24T06:46:56.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.692+0000 7fefdf476740 1 Processor -- start 2024-04-24T06:46:56.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.692+0000 7fefdf476740 1 -- start start 2024-04-24T06:46:56.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.692+0000 7fefdf476740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633c7088950 0x5633c708b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.693+0000 7fefdf476740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5633c7089270 0x5633c709f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.693+0000 7fefdf476740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633c7093940 0x5633c70ab8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.693+0000 7fefdf476740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fefd41b2850 con 0x5633c7088950 2024-04-24T06:46:56.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.693+0000 7fefdf476740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fefd41b0120 con 0x5633c7093940 2024-04-24T06:46:56.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.693+0000 7fefdf476740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fefd41adf70 con 0x5633c7089270 2024-04-24T06:46:56.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.693+0000 7fefd9c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633c7088950 0x5633c708b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.693+0000 7fefda600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633c7093940 0x5633c70ab8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.693+0000 7fefd9c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633c7088950 0x5633c708b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39762/0 (socket says 172.21.15.5:39762) 2024-04-24T06:46:56.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.693+0000 7fefd9c00640 1 -- 172.21.15.5:0/2918042312 learned_addr learned my addr 172.21.15.5:0/2918042312 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:56.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.693+0000 7fefd9200640 1 --2- 172.21.15.5:0/2918042312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5633c7089270 0x5633c709f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.693+0000 7fefda600640 1 -- 172.21.15.5:0/2918042312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5633c7089270 msgr2=0x5633c709f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:56.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.693+0000 7fefda600640 1 --2- 172.21.15.5:0/2918042312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5633c7089270 0x5633c709f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.693+0000 7fefda600640 1 -- 172.21.15.5:0/2918042312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633c7088950 msgr2=0x5633c708b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:56.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.693+0000 7fefda600640 1 --2- 172.21.15.5:0/2918042312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633c7088950 0x5633c708b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.693+0000 7fefda600640 1 -- 172.21.15.5:0/2918042312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633c6dca7d0 con 0x5633c7093940 2024-04-24T06:46:56.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:56 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/2427183470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:46:56.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.694+0000 7fefda600640 1 --2- 172.21.15.5:0/2918042312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633c7093940 0x5633c70ab8f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fefcc009710 tx=0x7fefcc009910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9673c9fcbe0e627 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:56.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.694+0000 7fefc6000640 1 -- 172.21.15.5:0/2918042312 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefcc004050 con 0x5633c7093940 2024-04-24T06:46:56.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.694+0000 7fefc6000640 1 -- 172.21.15.5:0/2918042312 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefcc005520 con 0x5633c7093940 2024-04-24T06:46:56.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.694+0000 7fefc6000640 1 -- 172.21.15.5:0/2918042312 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefcc003d40 con 0x5633c7093940 2024-04-24T06:46:56.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.695+0000 7fefdf476740 1 -- 172.21.15.5:0/2918042312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633c7093940 msgr2=0x5633c70ab8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:56.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.695+0000 7fefdf476740 1 --2- 172.21.15.5:0/2918042312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633c7093940 0x5633c70ab8f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fefcc009710 tx=0x7fefcc009910 comp rx=0 tx=0).stop 2024-04-24T06:46:56.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.695+0000 7fefdf476740 1 -- 172.21.15.5:0/2918042312 shutdown_connections 2024-04-24T06:46:56.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.695+0000 7fefdf476740 1 --2- 172.21.15.5:0/2918042312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633c7093940 0x5633c70ab8f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.695+0000 7fefdf476740 1 --2- 172.21.15.5:0/2918042312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5633c7089270 0x5633c709f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.695+0000 7fefdf476740 1 --2- 172.21.15.5:0/2918042312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633c7088950 0x5633c708b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.695+0000 7fefdf476740 1 -- 172.21.15.5:0/2918042312 >> 172.21.15.5:0/2918042312 conn(0x5633c7084c70 msgr2=0x5633c7085050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:56.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.695+0000 7fefdf476740 1 -- 172.21.15.5:0/2918042312 shutdown_connections 2024-04-24T06:46:56.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.695+0000 7fefdf476740 1 -- 172.21.15.5:0/2918042312 wait complete. 2024-04-24T06:46:56.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.695+0000 7fefdf476740 1 -- start start 2024-04-24T06:46:56.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.695+0000 7fefdf476740 1 Processor -- start 2024-04-24T06:46:56.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.696+0000 7fefdf476740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5633c7088950 0x5633c709be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.696+0000 7fefdf476740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633c7089270 0x5633c7094320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.696+0000 7fefdf476740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633c7093940 0x5633c7094860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.696+0000 7fefdf476740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fefd41ae5e0 con 0x5633c7093940 2024-04-24T06:46:56.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.696+0000 7fefdf476740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fefd41afa50 con 0x5633c7089270 2024-04-24T06:46:56.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.696+0000 7fefdf476740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fefd41aecb0 con 0x5633c7088950 2024-04-24T06:46:56.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.696+0000 7fefdf476740 10 client.0 _ll_register_callbacks cb 0x7fefc00f4610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:46:56.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.696+0000 7fefda600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633c7093940 0x5633c7094860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.696+0000 7fefda600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633c7093940 0x5633c7094860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39774/0 (socket says 172.21.15.5:39774) 2024-04-24T06:46:56.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.696+0000 7fefda600640 1 -- 172.21.15.5:0/1629994127 learned_addr learned my addr 172.21.15.5:0/1629994127 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:56.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.696+0000 7fefd9200640 1 --2- 172.21.15.5:0/1629994127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633c7089270 0x5633c7094320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.696+0000 7fefd9c00640 1 --2- 172.21.15.5:0/1629994127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5633c7088950 0x5633c709be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.697+0000 7fefda600640 1 -- 172.21.15.5:0/1629994127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5633c7088950 msgr2=0x5633c709be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:56.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.697+0000 7fefda600640 1 --2- 172.21.15.5:0/1629994127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5633c7088950 0x5633c709be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.697+0000 7fefda600640 1 -- 172.21.15.5:0/1629994127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633c7089270 msgr2=0x5633c7094320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:56.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.697+0000 7fefda600640 1 --2- 172.21.15.5:0/1629994127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633c7089270 0x5633c7094320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.697+0000 7fefda600640 1 -- 172.21.15.5:0/1629994127 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633c7094ea0 con 0x5633c7093940 2024-04-24T06:46:56.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.697+0000 7fefd9200640 1 --2- 172.21.15.5:0/1629994127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633c7089270 0x5633c7094320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:46:56.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.697+0000 7fefda600640 1 --2- 172.21.15.5:0/1629994127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633c7093940 0x5633c7094860 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fefcc003960 tx=0x7fefcc0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19bfa60c83604b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:56.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.697+0000 7fefbb400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:46:56.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.697+0000 7fefbb400640 1 -- 172.21.15.5:0/1629994127 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefcc033020 con 0x5633c7093940 2024-04-24T06:46:56.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.697+0000 7fefbb400640 1 -- 172.21.15.5:0/1629994127 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fefcc003dd0 con 0x5633c7093940 2024-04-24T06:46:56.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.697+0000 7fefdf476740 1 -- 172.21.15.5:0/1629994127 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefcc005520 con 0x5633c7093940 2024-04-24T06:46:56.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.698+0000 7fefbb400640 1 -- 172.21.15.5:0/1629994127 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fefcc03e050 con 0x5633c7093940 2024-04-24T06:46:56.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.698+0000 7fefdf476740 10 client.15831 fetch_fsmap learned FSMap version 25 2024-04-24T06:46:56.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.698+0000 7fefdf476740 1 -- 172.21.15.5:0/1629994127 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5633c710a160 con 0x5633c7093940 2024-04-24T06:46:56.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.698+0000 7fefbb400640 1 -- 172.21.15.5:0/1629994127 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefcc005520 con 0x5633c7093940 2024-04-24T06:46:56.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.698+0000 7fefbb400640 1 -- 172.21.15.5:0/1629994127 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fefcc02b400 con 0x5633c7093940 2024-04-24T06:46:56.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.698+0000 7fefdf476740 10 client.15831 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:46:56.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.698+0000 7fefdf476740 10 client.15831 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:46:56.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.698+0000 7fefdf476740 20 client.15831 populate_metadata read hostname 'smithi005' 2024-04-24T06:46:56.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.698+0000 7fefdf476740 1 --2- 172.21.15.5:0/1629994127 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5633c710a510 0x5633c710c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:56.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.698+0000 7fefdf476740 4 client.15831 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:46:56.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.698+0000 7fefdf476740 1 -- 172.21.15.5:0/1629994127 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5633c6ff5a60 con 0x5633c710a510 2024-04-24T06:46:56.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.699+0000 7fefd9c00640 1 --2- 172.21.15.5:0/1629994127 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5633c710a510 0x5633c710c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:56.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.699+0000 7fefd9c00640 1 --2- 172.21.15.5:0/1629994127 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5633c710a510 0x5633c710c930 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47d2947004d588a5 server_cookie=2c599f602e76e28a in_seq=0 out_seq=0 2024-04-24T06:46:56.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.699+0000 7fefbb400640 10 client.15831 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:46:56.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.699+0000 7fefbb400640 1 -- 172.21.15.5:0/1629994127 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5633c6ff5a60 con 0x5633c710a510 2024-04-24T06:46:56.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.699+0000 7fefbb400640 10 client.15831 handle_command_reply: tid=0 2024-04-24T06:46:56.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.699+0000 7fefdf476740 1 client.15831 shutdown 2024-04-24T06:46:56.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.699+0000 7fefdf476740 1 -- 172.21.15.5:0/1629994127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633c7093940 msgr2=0x5633c7094860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:56.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.700+0000 7fefdf476740 1 --2- 172.21.15.5:0/1629994127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633c7093940 0x5633c7094860 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fefcc003960 tx=0x7fefcc0056d0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.700+0000 7fefdf476740 1 -- 172.21.15.5:0/1629994127 shutdown_connections 2024-04-24T06:46:56.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.700+0000 7fefdf476740 1 --2- 172.21.15.5:0/1629994127 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5633c710a510 0x5633c710c930 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.700+0000 7fefdf476740 1 --2- 172.21.15.5:0/1629994127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633c7093940 0x5633c7094860 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.700+0000 7fefdf476740 1 --2- 172.21.15.5:0/1629994127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633c7089270 0x5633c7094320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.700+0000 7fefdf476740 1 --2- 172.21.15.5:0/1629994127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5633c7088950 0x5633c709be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.700+0000 7fefbb400640 0 client.15831 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:46:56.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.700+0000 7fefdf476740 1 -- 172.21.15.5:0/1629994127 >> 172.21.15.5:0/1629994127 conn(0x5633c7084c70 msgr2=0x5633c708b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:56.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.700+0000 7fefdf476740 1 -- 172.21.15.5:0/1629994127 shutdown_connections 2024-04-24T06:46:56.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.700+0000 7fefdf476740 1 -- 172.21.15.5:0/1629994127 wait complete. 2024-04-24T06:46:56.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.700+0000 7fefdf476740 20 client.15831 trim_cache size 0 max 16384 2024-04-24T06:46:56.712 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:46:56.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.701+0000 7fefdc600640 1 -- 172.21.15.5:0/1210482761 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fefbc0a3d20 msgr2=0x7fefbc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:56.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.701+0000 7fefdc600640 1 --2- 172.21.15.5:0/1210482761 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fefbc0a3d20 0x7fefbc0a61e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fefc8009fd0 tx=0x7fefc8009500 comp rx=0 tx=0).stop 2024-04-24T06:46:56.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.701+0000 7fefdc600640 1 -- 172.21.15.5:0/1210482761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fefd4109520 msgr2=0x7fefd41a0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:56.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.701+0000 7fefdc600640 1 --2- 172.21.15.5:0/1210482761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fefd4109520 0x7fefd41a0ca0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fefc002a910 tx=0x7fefc0003b20 comp rx=0 tx=0).stop 2024-04-24T06:46:56.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.701+0000 7fefdc600640 1 -- 172.21.15.5:0/1210482761 shutdown_connections 2024-04-24T06:46:56.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.701+0000 7fefdc600640 1 --2- 172.21.15.5:0/1210482761 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fefbc0a3d20 0x7fefbc0a61e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.701+0000 7fefdc600640 1 --2- 172.21.15.5:0/1210482761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefd419a960 0x7fefd419ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.701+0000 7fefdc600640 1 --2- 172.21.15.5:0/1210482761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fefd4109520 0x7fefd41a0ca0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.701+0000 7fefdc600640 1 --2- 172.21.15.5:0/1210482761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefd4104450 0x7fefd41a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:56.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.701+0000 7fefdc600640 1 -- 172.21.15.5:0/1210482761 >> 172.21.15.5:0/1210482761 conn(0x7fefd40ffc90 msgr2=0x7fefd4101fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:56.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.701+0000 7fefdc600640 1 -- 172.21.15.5:0/1210482761 shutdown_connections 2024-04-24T06:46:56.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:56.701+0000 7fefdc600640 1 -- 172.21.15.5:0/1210482761 wait complete. 2024-04-24T06:46:56.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:56 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/2427183470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:46:57.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:57 smithi137 ceph-mon[33017]: pgmap v90: 97 pgs: 97 active+clean; 87 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 633 KiB/s wr, 83 op/s 2024-04-24T06:46:57.721 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T06:46:57.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:46:57.722 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T06:46:57.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:57 smithi005 ceph-mon[27798]: pgmap v90: 97 pgs: 97 active+clean; 87 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 633 KiB/s wr, 83 op/s 2024-04-24T06:46:57.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:57 smithi037 ceph-mon[32864]: pgmap v90: 97 pgs: 97 active+clean; 87 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 633 KiB/s wr, 83 op/s 2024-04-24T06:46:57.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.951+0000 7fae72000640 1 -- 172.21.15.5:0/2893434191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae6c112df0 msgr2=0x7fae6c1151e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:57.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.951+0000 7fae72000640 1 --2- 172.21.15.5:0/2893434191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae6c112df0 0x7fae6c1151e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fae64009ca0 tx=0x7fae6402a950 comp rx=0 tx=0).stop 2024-04-24T06:46:57.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.951+0000 7fae72000640 1 -- 172.21.15.5:0/2893434191 shutdown_connections 2024-04-24T06:46:57.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.951+0000 7fae72000640 1 --2- 172.21.15.5:0/2893434191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae6c112df0 0x7fae6c1151e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:57.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.951+0000 7fae72000640 1 --2- 172.21.15.5:0/2893434191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae6c074cc0 0x7fae6c1128b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:57.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.951+0000 7fae72000640 1 --2- 172.21.15.5:0/2893434191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae6c0743a0 0x7fae6c074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:57.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.951+0000 7fae72000640 1 -- 172.21.15.5:0/2893434191 >> 172.21.15.5:0/2893434191 conn(0x7fae6c0fd8d0 msgr2=0x7fae6c0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:57.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.951+0000 7fae72000640 1 -- 172.21.15.5:0/2893434191 shutdown_connections 2024-04-24T06:46:57.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.951+0000 7fae72000640 1 -- 172.21.15.5:0/2893434191 wait complete. 2024-04-24T06:46:57.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.952+0000 7fae72000640 1 Processor -- start 2024-04-24T06:46:57.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.952+0000 7fae72000640 1 -- start start 2024-04-24T06:46:57.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.953+0000 7fae72000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae6c0743a0 0x7fae6c19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:57.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.953+0000 7fae72000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae6c074cc0 0x7fae6c19f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:57.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.953+0000 7fae72000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae6c112df0 0x7fae6c1a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:57.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.953+0000 7fae72000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c118800 con 0x7fae6c0743a0 2024-04-24T06:46:57.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.953+0000 7fae72000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c118680 con 0x7fae6c112df0 2024-04-24T06:46:57.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.953+0000 7fae72000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c118980 con 0x7fae6c074cc0 2024-04-24T06:46:57.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.953+0000 7fae70c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae6c0743a0 0x7fae6c19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:57.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.953+0000 7fae70c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae6c0743a0 0x7fae6c19ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39794/0 (socket says 172.21.15.5:39794) 2024-04-24T06:46:57.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.953+0000 7fae70c00640 1 -- 172.21.15.5:0/1732161344 learned_addr learned my addr 172.21.15.5:0/1732161344 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:57.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.953+0000 7fae71600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae6c112df0 0x7fae6c1a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:57.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.953+0000 7fae63e00640 1 --2- 172.21.15.5:0/1732161344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae6c074cc0 0x7fae6c19f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:57.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.954+0000 7fae70c00640 1 -- 172.21.15.5:0/1732161344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae6c074cc0 msgr2=0x7fae6c19f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:57.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.954+0000 7fae70c00640 1 --2- 172.21.15.5:0/1732161344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae6c074cc0 0x7fae6c19f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:57.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.954+0000 7fae70c00640 1 -- 172.21.15.5:0/1732161344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae6c112df0 msgr2=0x7fae6c1a35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:57.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.954+0000 7fae70c00640 1 --2- 172.21.15.5:0/1732161344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae6c112df0 0x7fae6c1a35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:57.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.954+0000 7fae70c00640 1 -- 172.21.15.5:0/1732161344 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae6c1a3d10 con 0x7fae6c0743a0 2024-04-24T06:46:57.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.954+0000 7fae71600640 1 --2- 172.21.15.5:0/1732161344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae6c112df0 0x7fae6c1a35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:46:57.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.954+0000 7fae70c00640 1 --2- 172.21.15.5:0/1732161344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae6c0743a0 0x7fae6c19ed20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00e9f0 tx=0x7fae5c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:57.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.954+0000 7fae6a000640 1 -- 172.21.15.5:0/1732161344 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c00cd80 con 0x7fae6c0743a0 2024-04-24T06:46:57.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.954+0000 7fae6a000640 1 -- 172.21.15.5:0/1732161344 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae5c005500 con 0x7fae6c0743a0 2024-04-24T06:46:57.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.954+0000 7fae72000640 1 -- 172.21.15.5:0/1732161344 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae6c1a4000 con 0x7fae6c0743a0 2024-04-24T06:46:57.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.955+0000 7fae6a000640 1 -- 172.21.15.5:0/1732161344 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c010690 con 0x7fae6c0743a0 2024-04-24T06:46:57.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.955+0000 7fae72000640 1 -- 172.21.15.5:0/1732161344 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae6c1ab890 con 0x7fae6c0743a0 2024-04-24T06:46:57.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.958+0000 7fae765d8740 1 Processor -- start 2024-04-24T06:46:57.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.958+0000 7fae6a000640 1 -- 172.21.15.5:0/1732161344 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fae5c0046c0 con 0x7fae6c0743a0 2024-04-24T06:46:57.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.958+0000 7fae765d8740 1 -- start start 2024-04-24T06:46:57.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.958+0000 7fae765d8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eaf4f2270 0x557eaf4f2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:57.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.958+0000 7fae765d8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eaf4f2b90 0x557eaf4fcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:57.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.958+0000 7fae765d8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eaf4fd260 0x557eaf4ff650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:57.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.958+0000 7fae765d8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557eaf457e70 con 0x557eaf4f2270 2024-04-24T06:46:57.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.958+0000 7fae765d8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557eaf3fe380 con 0x557eaf4f2b90 2024-04-24T06:46:57.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.958+0000 7fae765d8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557eaf453e30 con 0x557eaf4fd260 2024-04-24T06:46:57.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.959+0000 7fae71600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eaf4f2270 0x557eaf4f2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:57.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.959+0000 7fae63e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eaf4fd260 0x557eaf4ff650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:57.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.959+0000 7fae70c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eaf4f2b90 0x557eaf4fcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:57.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.959+0000 7fae63e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eaf4fd260 0x557eaf4ff650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:56360/0 (socket says 172.21.15.5:56360) 2024-04-24T06:46:57.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.959+0000 7fae63e00640 1 -- 172.21.15.5:0/3956078326 learned_addr learned my addr 172.21.15.5:0/3956078326 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:57.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.959+0000 7fae71600640 1 -- 172.21.15.5:0/3956078326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eaf4fd260 msgr2=0x557eaf4ff650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:57.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.959+0000 7fae71600640 1 --2- 172.21.15.5:0/3956078326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eaf4fd260 0x557eaf4ff650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:57.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.959+0000 7fae71600640 1 -- 172.21.15.5:0/3956078326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eaf4f2b90 msgr2=0x557eaf4fcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:57.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.959+0000 7fae71600640 1 --2- 172.21.15.5:0/3956078326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eaf4f2b90 0x557eaf4fcd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:57.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.959+0000 7fae71600640 1 -- 172.21.15.5:0/3956078326 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557eaf2327e0 con 0x557eaf4f2270 2024-04-24T06:46:57.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.960+0000 7fae6a000640 1 --2- 172.21.15.5:0/1732161344 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fae500a3d20 0x7fae500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:57.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.960+0000 7fae71600640 1 --2- 172.21.15.5:0/3956078326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eaf4f2270 0x557eaf4f2650 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fae6402af40 tx=0x7fae64005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c90872b9726f3f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:57.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.960+0000 7fae6a000640 1 -- 172.21.15.5:0/1732161344 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fae5c0d0c40 con 0x7fae6c0743a0 2024-04-24T06:46:57.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.960+0000 7fae70c00640 1 --2- 172.21.15.5:0/1732161344 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fae500a3d20 0x7fae500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:57.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.960+0000 7fae61c00640 1 -- 172.21.15.5:0/3956078326 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64048070 con 0x557eaf4f2270 2024-04-24T06:46:57.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.960+0000 7fae61c00640 1 -- 172.21.15.5:0/3956078326 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae64003e50 con 0x557eaf4f2270 2024-04-24T06:46:57.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.960+0000 7fae61c00640 1 -- 172.21.15.5:0/3956078326 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64034040 con 0x557eaf4f2270 2024-04-24T06:46:57.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.961+0000 7fae765d8740 1 -- 172.21.15.5:0/3956078326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eaf4f2270 msgr2=0x557eaf4f2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:57.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.961+0000 7fae70c00640 1 --2- 172.21.15.5:0/1732161344 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fae500a3d20 0x7fae500a61e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x557eaf4f3830 tx=0x7fae5c0d46f0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:57.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.961+0000 7fae765d8740 1 --2- 172.21.15.5:0/3956078326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eaf4f2270 0x557eaf4f2650 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fae6402af40 tx=0x7fae64005f80 comp rx=0 tx=0).stop 2024-04-24T06:46:57.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.961+0000 7fae765d8740 1 -- 172.21.15.5:0/3956078326 shutdown_connections 2024-04-24T06:46:57.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.961+0000 7fae765d8740 1 --2- 172.21.15.5:0/3956078326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eaf4fd260 0x557eaf4ff650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:57.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.961+0000 7fae765d8740 1 --2- 172.21.15.5:0/3956078326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eaf4f2b90 0x557eaf4fcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:57.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.961+0000 7fae765d8740 1 --2- 172.21.15.5:0/3956078326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eaf4f2270 0x557eaf4f2650 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:57.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.961+0000 7fae765d8740 1 -- 172.21.15.5:0/3956078326 >> 172.21.15.5:0/3956078326 conn(0x557eaf4ee590 msgr2=0x557eaf4ee970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:57.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.961+0000 7fae765d8740 1 -- 172.21.15.5:0/3956078326 shutdown_connections 2024-04-24T06:46:57.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.961+0000 7fae765d8740 1 -- 172.21.15.5:0/3956078326 wait complete. 2024-04-24T06:46:57.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.962+0000 7fae765d8740 1 -- start start 2024-04-24T06:46:57.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.962+0000 7fae765d8740 1 Processor -- start 2024-04-24T06:46:57.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.962+0000 7fae765d8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eaf4f2270 0x557eaf4fa190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:57.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.962+0000 7fae765d8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eaf4f2b90 0x557eaf4fa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:57.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.962+0000 7fae765d8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eaf4fd260 0x557eaf4fac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:57.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.962+0000 7fae765d8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557eaf4ffff0 con 0x557eaf4fd260 2024-04-24T06:46:57.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.962+0000 7fae765d8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557eaf411660 con 0x557eaf4f2270 2024-04-24T06:46:57.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.962+0000 7fae765d8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557eaf500170 con 0x557eaf4f2b90 2024-04-24T06:46:57.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.962+0000 7fae765d8740 10 client.0 _ll_register_callbacks cb 0x557eaf3ffaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:46:57.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.963+0000 7fae63e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eaf4f2b90 0x557eaf4fa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:57.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.963+0000 7fae71600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eaf4f2270 0x557eaf4fa190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:57.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.963+0000 7fae63e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eaf4f2b90 0x557eaf4fa6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:56362/0 (socket says 172.21.15.5:56362) 2024-04-24T06:46:57.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.963+0000 7fae63e00640 1 -- 172.21.15.5:0/721494664 learned_addr learned my addr 172.21.15.5:0/721494664 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:57.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.963+0000 7fae71600640 1 --2- 172.21.15.5:0/721494664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eaf4fd260 0x557eaf4fac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:57.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.963+0000 7fae63e00640 1 -- 172.21.15.5:0/721494664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eaf4f2270 msgr2=0x557eaf4fa190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:57.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.963+0000 7fae63e00640 1 --2- 172.21.15.5:0/721494664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eaf4f2270 0x557eaf4fa190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:57.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.963+0000 7fae63e00640 1 -- 172.21.15.5:0/721494664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eaf4fd260 msgr2=0x557eaf4fac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:57.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.963+0000 7fae63e00640 1 --2- 172.21.15.5:0/721494664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eaf4fd260 0x557eaf4fac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:57.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.963+0000 7fae63e00640 1 -- 172.21.15.5:0/721494664 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557eaf292f70 con 0x557eaf4f2b90 2024-04-24T06:46:57.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.963+0000 7fae71600640 1 --2- 172.21.15.5:0/721494664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eaf4fd260 0x557eaf4fac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:46:57.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.964+0000 7fae63e00640 1 --2- 172.21.15.5:0/721494664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eaf4f2b90 0x557eaf4fa6d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fae540126e0 tx=0x7fae54012ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb694bb38e2f89d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:57.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.964+0000 7fae5a000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:46:57.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.964+0000 7fae5a000640 1 -- 172.21.15.5:0/721494664 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae540121d0 con 0x557eaf4f2b90 2024-04-24T06:46:57.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.964+0000 7fae765d8740 1 -- 172.21.15.5:0/721494664 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae64003e50 con 0x557eaf4f2b90 2024-04-24T06:46:57.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.964+0000 7fae5a000640 1 -- 172.21.15.5:0/721494664 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fae54004030 con 0x557eaf4f2b90 2024-04-24T06:46:57.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.964+0000 7fae5a000640 1 -- 172.21.15.5:0/721494664 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fae54004230 con 0x557eaf4f2b90 2024-04-24T06:46:57.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.964+0000 7fae765d8740 10 client.25996 fetch_fsmap learned FSMap version 25 2024-04-24T06:46:57.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.964+0000 7fae765d8740 1 -- 172.21.15.5:0/721494664 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557eaf12a170 con 0x557eaf4f2b90 2024-04-24T06:46:57.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.965+0000 7fae5a000640 1 -- 172.21.15.5:0/721494664 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae64003e50 con 0x557eaf4f2b90 2024-04-24T06:46:57.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.965+0000 7fae5a000640 1 -- 172.21.15.5:0/721494664 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fae54003970 con 0x557eaf4f2b90 2024-04-24T06:46:57.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.965+0000 7fae765d8740 10 client.25996 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:46:57.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.965+0000 7fae765d8740 10 client.25996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:46:57.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.965+0000 7fae765d8740 20 client.25996 populate_metadata read hostname 'smithi005' 2024-04-24T06:46:57.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.965+0000 7fae765d8740 1 --2- 172.21.15.5:0/721494664 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x557eaf50d5f0 0x557eaf50fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:57.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.965+0000 7fae765d8740 4 client.25996 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:46:57.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.965+0000 7fae765d8740 1 -- 172.21.15.5:0/721494664 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557eaf465890 con 0x557eaf50d5f0 2024-04-24T06:46:57.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.965+0000 7fae71600640 1 --2- 172.21.15.5:0/721494664 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x557eaf50d5f0 0x557eaf50fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:57.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.966+0000 7fae71600640 1 --2- 172.21.15.5:0/721494664 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x557eaf50d5f0 0x557eaf50fa10 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8887f57d0b42a server_cookie=bee720c1f4958f16 in_seq=0 out_seq=0 2024-04-24T06:46:57.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.966+0000 7fae5a000640 10 client.25996 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:46:57.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.967+0000 7fae5a000640 1 -- 172.21.15.5:0/721494664 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557eaf465890 con 0x557eaf50d5f0 2024-04-24T06:46:57.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.967+0000 7fae5a000640 10 client.25996 handle_command_reply: tid=0 2024-04-24T06:46:57.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.967+0000 7fae765d8740 1 client.25996 shutdown 2024-04-24T06:46:57.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.967+0000 7fae765d8740 1 -- 172.21.15.5:0/721494664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eaf4f2b90 msgr2=0x557eaf4fa6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:57.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.967+0000 7fae765d8740 1 --2- 172.21.15.5:0/721494664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eaf4f2b90 0x557eaf4fa6d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fae540126e0 tx=0x7fae54012ba0 comp rx=0 tx=0).stop 2024-04-24T06:46:57.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.967+0000 7fae765d8740 1 -- 172.21.15.5:0/721494664 shutdown_connections 2024-04-24T06:46:57.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.967+0000 7fae765d8740 1 --2- 172.21.15.5:0/721494664 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x557eaf50d5f0 0x557eaf50fa10 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:57.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.967+0000 7fae765d8740 1 --2- 172.21.15.5:0/721494664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eaf4fd260 0x557eaf4fac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:57.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.967+0000 7fae765d8740 1 --2- 172.21.15.5:0/721494664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eaf4f2b90 0x557eaf4fa6d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:57.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.967+0000 7fae765d8740 1 --2- 172.21.15.5:0/721494664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eaf4f2270 0x557eaf4fa190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:57.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.967+0000 7fae5a000640 0 client.25996 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:46:57.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.967+0000 7fae765d8740 1 -- 172.21.15.5:0/721494664 >> 172.21.15.5:0/721494664 conn(0x557eaf4ee590 msgr2=0x557eaf4f4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:57.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.967+0000 7fae765d8740 1 -- 172.21.15.5:0/721494664 shutdown_connections 2024-04-24T06:46:57.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.967+0000 7fae765d8740 1 -- 172.21.15.5:0/721494664 wait complete. 2024-04-24T06:46:57.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.967+0000 7fae765d8740 20 client.25996 trim_cache size 0 max 16384 2024-04-24T06:46:57.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.987+0000 7fae765d8740 1 Processor -- start 2024-04-24T06:46:57.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.987+0000 7fae765d8740 1 -- start start 2024-04-24T06:46:57.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.987+0000 7fae765d8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eaf4f2270 0x557eaf508eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:57.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.987+0000 7fae765d8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eaf4f2b90 0x557eaf4fa860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:57.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.987+0000 7fae765d8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eaf4fd260 0x557eaf515320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:57.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.987+0000 7fae765d8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c1b69b0 con 0x557eaf4fd260 2024-04-24T06:46:57.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.987+0000 7fae765d8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c1b4280 con 0x557eaf4f2b90 2024-04-24T06:46:57.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.987+0000 7fae765d8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c1b20d0 con 0x557eaf4f2270 2024-04-24T06:46:57.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.988+0000 7fae71600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eaf4f2270 0x557eaf508eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:57.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.988+0000 7fae71600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eaf4f2270 0x557eaf508eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:56370/0 (socket says 172.21.15.5:56370) 2024-04-24T06:46:57.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.988+0000 7fae71600640 1 -- 172.21.15.5:0/1165024050 learned_addr learned my addr 172.21.15.5:0/1165024050 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:57.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.988+0000 7fae71600640 1 --2- 172.21.15.5:0/1165024050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eaf4fd260 0x557eaf515320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:57.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.988+0000 7fae63e00640 1 --2- 172.21.15.5:0/1165024050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eaf4f2b90 0x557eaf4fa860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:57.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.988+0000 7fae71600640 1 -- 172.21.15.5:0/1165024050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eaf4f2b90 msgr2=0x557eaf4fa860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:57.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.988+0000 7fae71600640 1 --2- 172.21.15.5:0/1165024050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eaf4f2b90 0x557eaf4fa860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:57.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.988+0000 7fae71600640 1 -- 172.21.15.5:0/1165024050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eaf4fd260 msgr2=0x557eaf515320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:57.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.988+0000 7fae71600640 1 --2- 172.21.15.5:0/1165024050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eaf4fd260 0x557eaf515320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:57.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.988+0000 7fae71600640 1 -- 172.21.15.5:0/1165024050 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557eaf4f29c0 con 0x557eaf4f2270 2024-04-24T06:46:57.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.988+0000 7fae63e00640 1 --2- 172.21.15.5:0/1165024050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eaf4f2b90 0x557eaf4fa860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:46:57.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.988+0000 7fae71600640 1 --2- 172.21.15.5:0/1165024050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eaf4f2270 0x557eaf508eb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fae6402ff60 tx=0x7fae64045040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6adea75854001d4d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:57.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.989+0000 7fae61200640 1 -- 172.21.15.5:0/1165024050 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64048020 con 0x557eaf4f2270 2024-04-24T06:46:57.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.989+0000 7fae61200640 1 -- 172.21.15.5:0/1165024050 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae640528e0 con 0x557eaf4f2270 2024-04-24T06:46:57.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.989+0000 7fae61200640 1 -- 172.21.15.5:0/1165024050 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6403a430 con 0x557eaf4f2270 2024-04-24T06:46:57.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.989+0000 7fae765d8740 1 -- 172.21.15.5:0/1165024050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eaf4f2270 msgr2=0x557eaf508eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:57.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.989+0000 7fae765d8740 1 --2- 172.21.15.5:0/1165024050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eaf4f2270 0x557eaf508eb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fae6402ff60 tx=0x7fae64045040 comp rx=0 tx=0).stop 2024-04-24T06:46:57.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.989+0000 7fae765d8740 1 -- 172.21.15.5:0/1165024050 shutdown_connections 2024-04-24T06:46:57.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.989+0000 7fae765d8740 1 --2- 172.21.15.5:0/1165024050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eaf4fd260 0x557eaf515320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:57.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.989+0000 7fae765d8740 1 --2- 172.21.15.5:0/1165024050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eaf4f2b90 0x557eaf4fa860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:57.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.989+0000 7fae765d8740 1 --2- 172.21.15.5:0/1165024050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eaf4f2270 0x557eaf508eb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:57.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.989+0000 7fae765d8740 1 -- 172.21.15.5:0/1165024050 >> 172.21.15.5:0/1165024050 conn(0x557eaf4ee590 msgr2=0x557eaf4fa150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:57.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.989+0000 7fae765d8740 1 -- 172.21.15.5:0/1165024050 shutdown_connections 2024-04-24T06:46:57.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.989+0000 7fae765d8740 1 -- 172.21.15.5:0/1165024050 wait complete. 2024-04-24T06:46:57.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.990+0000 7fae765d8740 1 -- start start 2024-04-24T06:46:57.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.990+0000 7fae765d8740 1 Processor -- start 2024-04-24T06:46:57.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.991+0000 7fae765d8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eaf4f2270 0x557eaf5014c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:57.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.991+0000 7fae765d8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eaf4f2b90 0x557eaf501a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:57.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.991+0000 7fae765d8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eaf4fd260 0x557eaf572b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:57.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.991+0000 7fae765d8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c1b2740 con 0x557eaf4fd260 2024-04-24T06:46:57.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.991+0000 7fae765d8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c1b3bb0 con 0x557eaf4f2270 2024-04-24T06:46:57.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.991+0000 7fae765d8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c1b2e10 con 0x557eaf4f2b90 2024-04-24T06:46:57.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.991+0000 7fae765d8740 10 client.0 _ll_register_callbacks cb 0x7fae54009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:46:57.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.991+0000 7fae63e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eaf4f2b90 0x557eaf501a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:57.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.991+0000 7fae71600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eaf4f2270 0x557eaf5014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:57.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.991+0000 7fae63e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eaf4f2b90 0x557eaf501a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:56376/0 (socket says 172.21.15.5:56376) 2024-04-24T06:46:57.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.991+0000 7fae63e00640 1 -- 172.21.15.5:0/282567129 learned_addr learned my addr 172.21.15.5:0/282567129 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:57.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.991+0000 7fae71600640 1 --2- 172.21.15.5:0/282567129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eaf4fd260 0x557eaf572b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:57.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.991+0000 7fae63e00640 1 -- 172.21.15.5:0/282567129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eaf4f2270 msgr2=0x557eaf5014c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:57.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.991+0000 7fae63e00640 1 --2- 172.21.15.5:0/282567129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eaf4f2270 0x557eaf5014c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:57.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.991+0000 7fae63e00640 1 -- 172.21.15.5:0/282567129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eaf4fd260 msgr2=0x557eaf572b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:57.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.991+0000 7fae63e00640 1 --2- 172.21.15.5:0/282567129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eaf4fd260 0x557eaf572b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:57.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.991+0000 7fae63e00640 1 -- 172.21.15.5:0/282567129 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557eaf5730d0 con 0x557eaf4f2b90 2024-04-24T06:46:57.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.992+0000 7fae63e00640 1 --2- 172.21.15.5:0/282567129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eaf4f2b90 0x557eaf501a00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fae540126b0 tx=0x7fae54004140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56137c2d5a2e8c8b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:57.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.992+0000 7fae5a000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:46:57.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.992+0000 7fae5a000640 1 -- 172.21.15.5:0/282567129 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae54029040 con 0x557eaf4f2b90 2024-04-24T06:46:57.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.992+0000 7fae765d8740 1 -- 172.21.15.5:0/282567129 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae640528e0 con 0x557eaf4f2b90 2024-04-24T06:46:57.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.992+0000 7fae5a000640 1 -- 172.21.15.5:0/282567129 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fae54005440 con 0x557eaf4f2b90 2024-04-24T06:46:57.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.992+0000 7fae5a000640 1 -- 172.21.15.5:0/282567129 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fae54009e80 con 0x557eaf4f2b90 2024-04-24T06:46:58.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.992+0000 7fae765d8740 10 client.26002 fetch_fsmap learned FSMap version 25 2024-04-24T06:46:58.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.992+0000 7fae765d8740 1 -- 172.21.15.5:0/282567129 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557eaf5743c0 con 0x557eaf4f2b90 2024-04-24T06:46:58.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.993+0000 7fae5a000640 1 -- 172.21.15.5:0/282567129 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae640528e0 con 0x557eaf4f2b90 2024-04-24T06:46:58.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.993+0000 7fae5a000640 1 -- 172.21.15.5:0/282567129 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fae540107e0 con 0x557eaf4f2b90 2024-04-24T06:46:58.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.993+0000 7fae765d8740 10 client.26002 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:46:58.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.993+0000 7fae765d8740 10 client.26002 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:46:58.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.993+0000 7fae765d8740 20 client.26002 populate_metadata read hostname 'smithi005' 2024-04-24T06:46:58.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.993+0000 7fae765d8740 1 --2- 172.21.15.5:0/282567129 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x557eaf574b20 0x557eaf576f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:58.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.993+0000 7fae765d8740 4 client.26002 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T06:46:58.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.993+0000 7fae765d8740 1 -- 172.21.15.5:0/282567129 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557eaf45f700 con 0x557eaf574b20 2024-04-24T06:46:58.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.993+0000 7fae71600640 1 --2- 172.21.15.5:0/282567129 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x557eaf574b20 0x557eaf576f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:58.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.994+0000 7fae71600640 1 --2- 172.21.15.5:0/282567129 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x557eaf574b20 0x557eaf576f40 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d433787badd0f0 server_cookie=2d671e1f6439857a in_seq=0 out_seq=0 2024-04-24T06:46:58.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.994+0000 7fae5a000640 10 client.26002 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:46:58.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.995+0000 7fae5a000640 1 -- 172.21.15.5:0/282567129 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557eaf45f700 con 0x557eaf574b20 2024-04-24T06:46:58.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.995+0000 7fae5a000640 10 client.26002 handle_command_reply: tid=0 2024-04-24T06:46:58.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.995+0000 7fae765d8740 1 client.26002 shutdown 2024-04-24T06:46:58.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.995+0000 7fae765d8740 1 -- 172.21.15.5:0/282567129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eaf4f2b90 msgr2=0x557eaf501a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:58.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.995+0000 7fae765d8740 1 --2- 172.21.15.5:0/282567129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eaf4f2b90 0x557eaf501a00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fae540126b0 tx=0x7fae54004140 comp rx=0 tx=0).stop 2024-04-24T06:46:58.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.995+0000 7fae765d8740 1 -- 172.21.15.5:0/282567129 shutdown_connections 2024-04-24T06:46:58.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.995+0000 7fae765d8740 1 --2- 172.21.15.5:0/282567129 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x557eaf574b20 0x557eaf576f40 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.995+0000 7fae765d8740 1 --2- 172.21.15.5:0/282567129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557eaf4fd260 0x557eaf572b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.995+0000 7fae765d8740 1 --2- 172.21.15.5:0/282567129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eaf4f2b90 0x557eaf501a00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.995+0000 7fae765d8740 1 --2- 172.21.15.5:0/282567129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eaf4f2270 0x557eaf5014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.995+0000 7fae5a000640 0 client.26002 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:46:58.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.995+0000 7fae765d8740 1 -- 172.21.15.5:0/282567129 >> 172.21.15.5:0/282567129 conn(0x557eaf4ee590 msgr2=0x557eaf4f4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:58.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.995+0000 7fae765d8740 1 -- 172.21.15.5:0/282567129 shutdown_connections 2024-04-24T06:46:58.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.995+0000 7fae765d8740 1 -- 172.21.15.5:0/282567129 wait complete. 2024-04-24T06:46:58.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.996+0000 7fae765d8740 20 client.26002 trim_cache size 0 max 16384 2024-04-24T06:46:58.003 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:46:58.003 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-24T06:46:58.004 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "944d95de-94e8-46b3-9811-d47c69fffb8a", 2024-04-24T06:46:58.004 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-24T06:46:58.004 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:46:58.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.996+0000 7fae72000640 1 -- 172.21.15.5:0/1732161344 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fae500a3d20 msgr2=0x7fae500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:58.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.996+0000 7fae72000640 1 --2- 172.21.15.5:0/1732161344 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fae500a3d20 0x7fae500a61e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x557eaf4f3830 tx=0x7fae5c0d46f0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.996+0000 7fae72000640 1 -- 172.21.15.5:0/1732161344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae6c0743a0 msgr2=0x7fae6c19ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:58.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.996+0000 7fae72000640 1 --2- 172.21.15.5:0/1732161344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae6c0743a0 0x7fae6c19ed20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00e9f0 tx=0x7fae5c00edf0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.996+0000 7fae72000640 1 -- 172.21.15.5:0/1732161344 shutdown_connections 2024-04-24T06:46:58.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.996+0000 7fae72000640 1 --2- 172.21.15.5:0/1732161344 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fae500a3d20 0x7fae500a61e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.996+0000 7fae72000640 1 --2- 172.21.15.5:0/1732161344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae6c112df0 0x7fae6c1a35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.996+0000 7fae72000640 1 --2- 172.21.15.5:0/1732161344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae6c074cc0 0x7fae6c19f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.996+0000 7fae72000640 1 --2- 172.21.15.5:0/1732161344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae6c0743a0 0x7fae6c19ed20 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.996+0000 7fae72000640 1 -- 172.21.15.5:0/1732161344 >> 172.21.15.5:0/1732161344 conn(0x7fae6c0fd8d0 msgr2=0x7fae6c0ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:58.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.997+0000 7fae72000640 1 -- 172.21.15.5:0/1732161344 shutdown_connections 2024-04-24T06:46:58.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:57.997+0000 7fae72000640 1 -- 172.21.15.5:0/1732161344 wait complete. 2024-04-24T06:46:58.018 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T06:46:58.019 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T06:46:58.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.286+0000 7f970f200640 1 -- 172.21.15.5:0/4251838832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9708103900 msgr2=0x7f970810bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:58.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.286+0000 7f970f200640 1 --2- 172.21.15.5:0/4251838832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9708103900 0x7f970810bde0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f96fc009900 tx=0x7f96fc02a490 comp rx=0 tx=0).stop 2024-04-24T06:46:58.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.286+0000 7f970f200640 1 -- 172.21.15.5:0/4251838832 shutdown_connections 2024-04-24T06:46:58.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.286+0000 7f970f200640 1 --2- 172.21.15.5:0/4251838832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f970810c400 0x7f970810e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.286+0000 7f970f200640 1 --2- 172.21.15.5:0/4251838832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9708103900 0x7f970810bde0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.286+0000 7f970f200640 1 --2- 172.21.15.5:0/4251838832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9708102fe0 0x7f97081033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.286+0000 7f970f200640 1 -- 172.21.15.5:0/4251838832 >> 172.21.15.5:0/4251838832 conn(0x7f9708077ab0 msgr2=0x7f97080fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:58.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.286+0000 7f970f200640 1 -- 172.21.15.5:0/4251838832 shutdown_connections 2024-04-24T06:46:58.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.286+0000 7f970f200640 1 -- 172.21.15.5:0/4251838832 wait complete. 2024-04-24T06:46:58.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.287+0000 7f970f200640 1 Processor -- start 2024-04-24T06:46:58.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.287+0000 7f970f200640 1 -- start start 2024-04-24T06:46:58.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.288+0000 7f970f200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9708102fe0 0x7f9708198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:58.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.288+0000 7f970f200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9708103900 0x7f9708198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:58.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.288+0000 7f970f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f970810c400 0x7f970819d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:58.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.288+0000 7f970f200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f97081121e0 con 0x7f970810c400 2024-04-24T06:46:58.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.288+0000 7f970f200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9708112060 con 0x7f9708103900 2024-04-24T06:46:58.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.288+0000 7f970f200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9708112360 con 0x7f9708102fe0 2024-04-24T06:46:58.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.289+0000 7f970de00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9708102fe0 0x7f9708198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:58.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.289+0000 7f970de00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9708102fe0 0x7f9708198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:56384/0 (socket says 172.21.15.5:56384) 2024-04-24T06:46:58.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.289+0000 7f970de00640 1 -- 172.21.15.5:0/564906773 learned_addr learned my addr 172.21.15.5:0/564906773 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:58.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.289+0000 7f970d400640 1 --2- 172.21.15.5:0/564906773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9708103900 0x7f9708198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:58.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.289+0000 7f970e800640 1 --2- 172.21.15.5:0/564906773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f970810c400 0x7f970819d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:58.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.289+0000 7f970de00640 1 -- 172.21.15.5:0/564906773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9708103900 msgr2=0x7f9708198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:58.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.289+0000 7f970de00640 1 --2- 172.21.15.5:0/564906773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9708103900 0x7f9708198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.289+0000 7f970de00640 1 -- 172.21.15.5:0/564906773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f970810c400 msgr2=0x7f970819d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:58.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.289+0000 7f970de00640 1 --2- 172.21.15.5:0/564906773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f970810c400 0x7f970819d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.289+0000 7f970de00640 1 -- 172.21.15.5:0/564906773 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f970819d750 con 0x7f9708102fe0 2024-04-24T06:46:58.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.289+0000 7f970de00640 1 --2- 172.21.15.5:0/564906773 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9708102fe0 0x7f9708198760 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f96f400e9f0 tx=0x7f96f400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:58.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.290+0000 7f9702a00640 1 -- 172.21.15.5:0/564906773 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f400cd80 con 0x7f9708102fe0 2024-04-24T06:46:58.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.290+0000 7f9702a00640 1 -- 172.21.15.5:0/564906773 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96f4010910 con 0x7f9708102fe0 2024-04-24T06:46:58.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.290+0000 7f9702a00640 1 -- 172.21.15.5:0/564906773 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f4018990 con 0x7f9708102fe0 2024-04-24T06:46:58.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.290+0000 7f970f200640 1 -- 172.21.15.5:0/564906773 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f970819da40 con 0x7f9708102fe0 2024-04-24T06:46:58.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.290+0000 7f970f200640 1 -- 172.21.15.5:0/564906773 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97081a52d0 con 0x7f9708102fe0 2024-04-24T06:46:58.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.293+0000 7f9702a00640 1 -- 172.21.15.5:0/564906773 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f96f4010ac0 con 0x7f9708102fe0 2024-04-24T06:46:58.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.294+0000 7f9713603740 1 Processor -- start 2024-04-24T06:46:58.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.294+0000 7f9713603740 1 -- start start 2024-04-24T06:46:58.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.295+0000 7f9713603740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597f933bbf0 0x5597f933bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:58.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.295+0000 7f9713603740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597f9336330 0x5597f9336790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:58.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.295+0000 7f9713603740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597f9336cd0 0x5597f9342a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:58.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.295+0000 7f9713603740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5597f92921f0 con 0x5597f9336330 2024-04-24T06:46:58.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.295+0000 7f9713603740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5597f9238350 con 0x5597f9336cd0 2024-04-24T06:46:58.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.295+0000 7f9713603740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5597f928e1b0 con 0x5597f933bbf0 2024-04-24T06:46:58.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.295+0000 7f970e800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597f933bbf0 0x5597f933bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:58.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.295+0000 7f970e800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597f933bbf0 0x5597f933bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:56396/0 (socket says 172.21.15.5:56396) 2024-04-24T06:46:58.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.295+0000 7f970e800640 1 -- 172.21.15.5:0/1180283034 learned_addr learned my addr 172.21.15.5:0/1180283034 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:58.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.295+0000 7f970d400640 1 --2- 172.21.15.5:0/1180283034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597f9336cd0 0x5597f9342a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:58.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.295+0000 7f9702a00640 1 --2- 172.21.15.5:0/564906773 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f96f00a3e70 0x7f96f00a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:58.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.295+0000 7f970de00640 1 --2- 172.21.15.5:0/1180283034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597f9336330 0x5597f9336790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:58.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.296+0000 7f970e800640 1 -- 172.21.15.5:0/1180283034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597f9336cd0 msgr2=0x5597f9342a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:58.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.296+0000 7f970e800640 1 --2- 172.21.15.5:0/1180283034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597f9336cd0 0x5597f9342a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.296+0000 7f970e800640 1 -- 172.21.15.5:0/1180283034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597f9336330 msgr2=0x5597f9336790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:58.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.296+0000 7f970e800640 1 --2- 172.21.15.5:0/1180283034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597f9336330 0x5597f9336790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.296+0000 7f970e800640 1 -- 172.21.15.5:0/1180283034 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597f928cf50 con 0x5597f933bbf0 2024-04-24T06:46:58.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.296+0000 7f9702a00640 1 -- 172.21.15.5:0/564906773 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f96f40d0ac0 con 0x7f9708102fe0 2024-04-24T06:46:58.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.296+0000 7f970e800640 1 --2- 172.21.15.5:0/564906773 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f96f00a3e70 0x7f96f00a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:58.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.296+0000 7f970e800640 1 --2- 172.21.15.5:0/1180283034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597f933bbf0 0x5597f933bfd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9704009f20 tx=0x7f9704010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1765624ea49d317d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:58.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.297+0000 7f96fb000640 1 -- 172.21.15.5:0/1180283034 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9704011e30 con 0x5597f933bbf0 2024-04-24T06:46:58.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.297+0000 7f96fb000640 1 -- 172.21.15.5:0/1180283034 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f970401ddd0 con 0x5597f933bbf0 2024-04-24T06:46:58.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.297+0000 7f96fb000640 1 -- 172.21.15.5:0/1180283034 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970401cc50 con 0x5597f933bbf0 2024-04-24T06:46:58.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.298+0000 7f9713603740 1 -- 172.21.15.5:0/1180283034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597f933bbf0 msgr2=0x5597f933bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:58.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.298+0000 7f9713603740 1 --2- 172.21.15.5:0/1180283034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597f933bbf0 0x5597f933bfd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9704009f20 tx=0x7f9704010860 comp rx=0 tx=0).stop 2024-04-24T06:46:58.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.298+0000 7f970e800640 1 --2- 172.21.15.5:0/564906773 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f96f00a3e70 0x7f96f00a6330 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9704021000 tx=0x7f9704022000 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:58.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.298+0000 7f9713603740 1 -- 172.21.15.5:0/1180283034 shutdown_connections 2024-04-24T06:46:58.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.298+0000 7f9713603740 1 --2- 172.21.15.5:0/1180283034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597f9336cd0 0x5597f9342a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.298+0000 7f9713603740 1 --2- 172.21.15.5:0/1180283034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597f9336330 0x5597f9336790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.298+0000 7f9713603740 1 --2- 172.21.15.5:0/1180283034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597f933bbf0 0x5597f933bfd0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.298+0000 7f9713603740 1 -- 172.21.15.5:0/1180283034 >> 172.21.15.5:0/1180283034 conn(0x5597f9328800 msgr2=0x5597f9328be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:58.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.299+0000 7f9713603740 1 -- 172.21.15.5:0/1180283034 shutdown_connections 2024-04-24T06:46:58.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.299+0000 7f9713603740 1 -- 172.21.15.5:0/1180283034 wait complete. 2024-04-24T06:46:58.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.299+0000 7f9713603740 1 -- start start 2024-04-24T06:46:58.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.299+0000 7f9713603740 1 Processor -- start 2024-04-24T06:46:58.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.300+0000 7f9713603740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597f9336330 0x5597f933ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:58.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.300+0000 7f9713603740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597f9336cd0 0x5597f9340460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:58.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.300+0000 7f9713603740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597f933bbf0 0x5597f93409a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:58.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.300+0000 7f9713603740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5597f9343070 con 0x5597f9336330 2024-04-24T06:46:58.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.300+0000 7f9713603740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5597f924b690 con 0x5597f9336cd0 2024-04-24T06:46:58.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.300+0000 7f9713603740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5597f93431f0 con 0x5597f933bbf0 2024-04-24T06:46:58.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.300+0000 7f9713603740 10 client.0 _ll_register_callbacks cb 0x5597f9287650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:46:58.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.300+0000 7f970d400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597f9336cd0 0x5597f9340460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:58.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.300+0000 7f970d400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597f9336cd0 0x5597f9340460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:38688/0 (socket says 172.21.15.5:38688) 2024-04-24T06:46:58.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.300+0000 7f970d400640 1 -- 172.21.15.5:0/4144411670 learned_addr learned my addr 172.21.15.5:0/4144411670 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:58.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.300+0000 7f970de00640 1 --2- 172.21.15.5:0/4144411670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597f9336330 0x5597f933ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:58.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.300+0000 7f970e800640 1 --2- 172.21.15.5:0/4144411670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597f933bbf0 0x5597f93409a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:58.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.301+0000 7f970d400640 1 -- 172.21.15.5:0/4144411670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597f933bbf0 msgr2=0x5597f93409a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:58.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.301+0000 7f970d400640 1 --2- 172.21.15.5:0/4144411670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597f933bbf0 0x5597f93409a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.301+0000 7f970d400640 1 -- 172.21.15.5:0/4144411670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597f9336330 msgr2=0x5597f933ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:58.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.301+0000 7f970d400640 1 --2- 172.21.15.5:0/4144411670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597f9336330 0x5597f933ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.301+0000 7f970d400640 1 -- 172.21.15.5:0/4144411670 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597f924a920 con 0x5597f9336cd0 2024-04-24T06:46:58.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.301+0000 7f970d400640 1 --2- 172.21.15.5:0/4144411670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597f9336cd0 0x5597f9340460 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f96fc033000 tx=0x7f96fc004440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37dd5e6e6d50940b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:58.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.301+0000 7f96ef400640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:46:58.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.302+0000 7f96ef400640 1 -- 172.21.15.5:0/4144411670 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc035040 con 0x5597f9336cd0 2024-04-24T06:46:58.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.302+0000 7f9713603740 1 -- 172.21.15.5:0/4144411670 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f970401ddd0 con 0x5597f9336cd0 2024-04-24T06:46:58.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.302+0000 7f96ef400640 1 -- 172.21.15.5:0/4144411670 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f96fc005510 con 0x5597f9336cd0 2024-04-24T06:46:58.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.302+0000 7f96ef400640 1 -- 172.21.15.5:0/4144411670 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f96fc0046c0 con 0x5597f9336cd0 2024-04-24T06:46:58.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.302+0000 7f9713603740 10 client.25988 fetch_fsmap learned FSMap version 25 2024-04-24T06:46:58.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.302+0000 7f9713603740 1 -- 172.21.15.5:0/4144411670 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5597f906e7d0 con 0x5597f9336cd0 2024-04-24T06:46:58.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.302+0000 7f96ef400640 1 -- 172.21.15.5:0/4144411670 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f970401ddd0 con 0x5597f9336cd0 2024-04-24T06:46:58.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.303+0000 7f96ef400640 1 -- 172.21.15.5:0/4144411670 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f96fc031b20 con 0x5597f9336cd0 2024-04-24T06:46:58.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.303+0000 7f9713603740 10 client.25988 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:46:58.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.303+0000 7f9713603740 10 client.25988 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:46:58.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.303+0000 7f9713603740 20 client.25988 populate_metadata read hostname 'smithi005' 2024-04-24T06:46:58.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.303+0000 7f9713603740 1 --2- 172.21.15.5:0/4144411670 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5597f93505b0 0x5597f9352a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:58.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.303+0000 7f9713603740 4 client.25988 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:46:58.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.303+0000 7f9713603740 1 -- 172.21.15.5:0/4144411670 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597f929fc10 con 0x5597f93505b0 2024-04-24T06:46:58.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.303+0000 7f970de00640 1 --2- 172.21.15.5:0/4144411670 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5597f93505b0 0x5597f9352a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:58.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.304+0000 7f970de00640 1 --2- 172.21.15.5:0/4144411670 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5597f93505b0 0x5597f9352a50 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=354e35206dcbb6df server_cookie=60305cde8d825eba in_seq=0 out_seq=0 2024-04-24T06:46:58.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.304+0000 7f96ef400640 10 client.25988 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:46:58.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.305+0000 7f96ef400640 1 -- 172.21.15.5:0/4144411670 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5597f929fc10 con 0x5597f93505b0 2024-04-24T06:46:58.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.305+0000 7f96ef400640 10 client.25988 handle_command_reply: tid=0 2024-04-24T06:46:58.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.305+0000 7f9713603740 1 client.25988 shutdown 2024-04-24T06:46:58.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.305+0000 7f9713603740 1 -- 172.21.15.5:0/4144411670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597f9336cd0 msgr2=0x5597f9340460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:58.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.305+0000 7f9713603740 1 --2- 172.21.15.5:0/4144411670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597f9336cd0 0x5597f9340460 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f96fc033000 tx=0x7f96fc004440 comp rx=0 tx=0).stop 2024-04-24T06:46:58.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.305+0000 7f9713603740 1 -- 172.21.15.5:0/4144411670 shutdown_connections 2024-04-24T06:46:58.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.305+0000 7f9713603740 1 --2- 172.21.15.5:0/4144411670 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5597f93505b0 0x5597f9352a50 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.306+0000 7f9713603740 1 --2- 172.21.15.5:0/4144411670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597f933bbf0 0x5597f93409a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.306+0000 7f9713603740 1 --2- 172.21.15.5:0/4144411670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597f9336cd0 0x5597f9340460 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.306+0000 7f9713603740 1 --2- 172.21.15.5:0/4144411670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597f9336330 0x5597f933ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.306+0000 7f96ef400640 0 client.25988 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:46:58.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.306+0000 7f9713603740 1 -- 172.21.15.5:0/4144411670 >> 172.21.15.5:0/4144411670 conn(0x5597f9328800 msgr2=0x5597f9335bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:58.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.306+0000 7f9713603740 1 -- 172.21.15.5:0/4144411670 shutdown_connections 2024-04-24T06:46:58.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.306+0000 7f9713603740 1 -- 172.21.15.5:0/4144411670 wait complete. 2024-04-24T06:46:58.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.306+0000 7f9713603740 20 client.25988 trim_cache size 0 max 16384 2024-04-24T06:46:58.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.329+0000 7f9713603740 1 Processor -- start 2024-04-24T06:46:58.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.329+0000 7f9713603740 1 -- start start 2024-04-24T06:46:58.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.330+0000 7f9713603740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597f9336330 0x5597f934bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:58.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.330+0000 7f9713603740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597f9336cd0 0x5597f9323e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:58.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.330+0000 7f9713603740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597f933bbf0 0x5597f933d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:58.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.330+0000 7f9713603740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f97081ae100 con 0x5597f933bbf0 2024-04-24T06:46:58.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.330+0000 7f9713603740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f97081ab9d0 con 0x5597f9336cd0 2024-04-24T06:46:58.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.330+0000 7f9713603740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f97081a9840 con 0x5597f9336330 2024-04-24T06:46:58.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.330+0000 7f970de00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597f9336330 0x5597f934bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:58.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.330+0000 7f970d400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597f9336cd0 0x5597f9323e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:58.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.330+0000 7f970de00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597f9336330 0x5597f934bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:56410/0 (socket says 172.21.15.5:56410) 2024-04-24T06:46:58.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.330+0000 7f970de00640 1 -- 172.21.15.5:0/2570075004 learned_addr learned my addr 172.21.15.5:0/2570075004 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:58.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.330+0000 7f970d400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597f9336cd0 0x5597f9323e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:38694/0 (socket says 172.21.15.5:38694) 2024-04-24T06:46:58.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.330+0000 7f970e800640 1 --2- 172.21.15.5:0/2570075004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597f933bbf0 0x5597f933d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:58.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.330+0000 7f970de00640 1 -- 172.21.15.5:0/2570075004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597f9336cd0 msgr2=0x5597f9323e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:58.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.330+0000 7f970de00640 1 --2- 172.21.15.5:0/2570075004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597f9336cd0 0x5597f9323e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.330+0000 7f970de00640 1 -- 172.21.15.5:0/2570075004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597f933bbf0 msgr2=0x5597f933d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:58.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.330+0000 7f970de00640 1 --2- 172.21.15.5:0/2570075004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597f933bbf0 0x5597f933d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.330+0000 7f970de00640 1 -- 172.21.15.5:0/2570075004 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597f933eef0 con 0x5597f9336330 2024-04-24T06:46:58.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.331+0000 7f970de00640 1 --2- 172.21.15.5:0/2570075004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597f9336330 0x5597f934bf30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f96f400c6a0 tx=0x7f96f40bd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2ccff8b1609ca5e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:58.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.331+0000 7f96fa600640 1 -- 172.21.15.5:0/2570075004 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f409ca70 con 0x5597f9336330 2024-04-24T06:46:58.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.331+0000 7f96fa600640 1 -- 172.21.15.5:0/2570075004 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96f409ad70 con 0x5597f9336330 2024-04-24T06:46:58.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.331+0000 7f96fa600640 1 -- 172.21.15.5:0/2570075004 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f4018b30 con 0x5597f9336330 2024-04-24T06:46:58.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.332+0000 7f9713603740 1 -- 172.21.15.5:0/2570075004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597f9336330 msgr2=0x5597f934bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:58.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.332+0000 7f9713603740 1 --2- 172.21.15.5:0/2570075004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597f9336330 0x5597f934bf30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f96f400c6a0 tx=0x7f96f40bd040 comp rx=0 tx=0).stop 2024-04-24T06:46:58.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.332+0000 7f9713603740 1 -- 172.21.15.5:0/2570075004 shutdown_connections 2024-04-24T06:46:58.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.332+0000 7f9713603740 1 --2- 172.21.15.5:0/2570075004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597f933bbf0 0x5597f933d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.332+0000 7f9713603740 1 --2- 172.21.15.5:0/2570075004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597f9336cd0 0x5597f9323e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.332+0000 7f9713603740 1 --2- 172.21.15.5:0/2570075004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597f9336330 0x5597f934bf30 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.332+0000 7f9713603740 1 -- 172.21.15.5:0/2570075004 >> 172.21.15.5:0/2570075004 conn(0x5597f9328800 msgr2=0x5597f9335320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:58.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.332+0000 7f9713603740 1 -- 172.21.15.5:0/2570075004 shutdown_connections 2024-04-24T06:46:58.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.332+0000 7f9713603740 1 -- 172.21.15.5:0/2570075004 wait complete. 2024-04-24T06:46:58.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.333+0000 7f9713603740 1 -- start start 2024-04-24T06:46:58.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.333+0000 7f9713603740 1 Processor -- start 2024-04-24T06:46:58.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.333+0000 7f9713603740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597f9336330 0x5597f93486a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:58.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.334+0000 7f9713603740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597f9336cd0 0x5597f9345410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:58.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.334+0000 7f9713603740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597f933bbf0 0x5597f9345950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:58.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.334+0000 7f9713603740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f97081a9e90 con 0x5597f9336330 2024-04-24T06:46:58.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.334+0000 7f9713603740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f97081ab300 con 0x5597f9336cd0 2024-04-24T06:46:58.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.334+0000 7f9713603740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f97081aa560 con 0x5597f933bbf0 2024-04-24T06:46:58.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.334+0000 7f9713603740 10 client.0 _ll_register_callbacks cb 0x7f96fc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:46:58.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.334+0000 7f970d400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597f9336cd0 0x5597f9345410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:58.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.334+0000 7f970e800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597f933bbf0 0x5597f9345950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:58.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.334+0000 7f970de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597f9336330 0x5597f93486a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:58.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.334+0000 7f970e800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597f933bbf0 0x5597f9345950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:56414/0 (socket says 172.21.15.5:56414) 2024-04-24T06:46:58.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.334+0000 7f970e800640 1 -- 172.21.15.5:0/2630341153 learned_addr learned my addr 172.21.15.5:0/2630341153 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:46:58.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.335+0000 7f970d400640 1 -- 172.21.15.5:0/2630341153 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597f933bbf0 msgr2=0x5597f9345950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:58.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.335+0000 7f970d400640 1 --2- 172.21.15.5:0/2630341153 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597f933bbf0 0x5597f9345950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.335+0000 7f970d400640 1 -- 172.21.15.5:0/2630341153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597f9336330 msgr2=0x5597f93486a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:46:58.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.335+0000 7f970d400640 1 --2- 172.21.15.5:0/2630341153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597f9336330 0x5597f93486a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.335+0000 7f970d400640 1 -- 172.21.15.5:0/2630341153 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597f8fa43f0 con 0x5597f9336cd0 2024-04-24T06:46:58.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.335+0000 7f970e800640 1 --2- 172.21.15.5:0/2630341153 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597f933bbf0 0x5597f9345950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:46:58.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.335+0000 7f970d400640 1 --2- 172.21.15.5:0/2630341153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597f9336cd0 0x5597f9345410 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f96fc009900 tx=0x7f96fc003d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20f8e030e2ad9a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:46:58.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.335+0000 7f96ef400640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:46:58.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.335+0000 7f96ef400640 1 -- 172.21.15.5:0/2630341153 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc035030 con 0x5597f9336cd0 2024-04-24T06:46:58.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.335+0000 7f9713603740 1 -- 172.21.15.5:0/2630341153 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96f409ad70 con 0x5597f9336cd0 2024-04-24T06:46:58.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.335+0000 7f96ef400640 1 -- 172.21.15.5:0/2630341153 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f96fc0057f0 con 0x5597f9336cd0 2024-04-24T06:46:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.336+0000 7f96ef400640 1 -- 172.21.15.5:0/2630341153 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f96fc053040 con 0x5597f9336cd0 2024-04-24T06:46:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.336+0000 7f9713603740 10 client.25994 fetch_fsmap learned FSMap version 25 2024-04-24T06:46:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.336+0000 7f9713603740 1 -- 172.21.15.5:0/2630341153 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5597f8f640f0 con 0x5597f9336cd0 2024-04-24T06:46:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.336+0000 7f96ef400640 1 -- 172.21.15.5:0/2630341153 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96f409ad70 con 0x5597f9336cd0 2024-04-24T06:46:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.336+0000 7f96ef400640 1 -- 172.21.15.5:0/2630341153 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f96fc056070 con 0x5597f9336cd0 2024-04-24T06:46:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.337+0000 7f9713603740 10 client.25994 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:46:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.337+0000 7f9713603740 10 client.25994 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:46:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.337+0000 7f9713603740 20 client.25994 populate_metadata read hostname 'smithi005' 2024-04-24T06:46:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.337+0000 7f9713603740 1 --2- 172.21.15.5:0/2630341153 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5597f93b7ec0 0x5597f93ba2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:46:58.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.337+0000 7f9713603740 4 client.25994 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:46:58.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.337+0000 7f9713603740 1 -- 172.21.15.5:0/2630341153 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5597f9299a80 con 0x5597f93b7ec0 2024-04-24T06:46:58.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.337+0000 7f970de00640 1 --2- 172.21.15.5:0/2630341153 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5597f93b7ec0 0x5597f93ba2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:46:58.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.337+0000 7f970de00640 1 --2- 172.21.15.5:0/2630341153 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5597f93b7ec0 0x5597f93ba2e0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82558e64749aa623 server_cookie=14eb336a674cb365 in_seq=0 out_seq=0 2024-04-24T06:46:58.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.337+0000 7f96ef400640 10 client.25994 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:46:58.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.338+0000 7f96ef400640 1 -- 172.21.15.5:0/2630341153 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x5597f9299a80 con 0x5597f93b7ec0 2024-04-24T06:46:58.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.338+0000 7f96ef400640 10 client.25994 handle_command_reply: tid=0 2024-04-24T06:46:58.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.338+0000 7f9713603740 1 client.25994 shutdown 2024-04-24T06:46:58.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.338+0000 7f9713603740 1 -- 172.21.15.5:0/2630341153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597f9336cd0 msgr2=0x5597f9345410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:58.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.338+0000 7f9713603740 1 --2- 172.21.15.5:0/2630341153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597f9336cd0 0x5597f9345410 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f96fc009900 tx=0x7f96fc003d10 comp rx=0 tx=0).stop 2024-04-24T06:46:58.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.338+0000 7f9713603740 1 -- 172.21.15.5:0/2630341153 shutdown_connections 2024-04-24T06:46:58.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.338+0000 7f9713603740 1 --2- 172.21.15.5:0/2630341153 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5597f93b7ec0 0x5597f93ba2e0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.338+0000 7f9713603740 1 --2- 172.21.15.5:0/2630341153 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597f933bbf0 0x5597f9345950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.338+0000 7f9713603740 1 --2- 172.21.15.5:0/2630341153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597f9336cd0 0x5597f9345410 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.338+0000 7f9713603740 1 --2- 172.21.15.5:0/2630341153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597f9336330 0x5597f93486a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.338+0000 7f9713603740 1 -- 172.21.15.5:0/2630341153 >> 172.21.15.5:0/2630341153 conn(0x5597f9328800 msgr2=0x5597f9335bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:58.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.338+0000 7f96ef400640 0 client.25994 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:46:58.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.338+0000 7f9713603740 1 -- 172.21.15.5:0/2630341153 shutdown_connections 2024-04-24T06:46:58.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.339+0000 7f9713603740 1 -- 172.21.15.5:0/2630341153 wait complete. 2024-04-24T06:46:58.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.339+0000 7f9713603740 20 client.25994 trim_cache size 0 max 16384 2024-04-24T06:46:58.350 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:46:58.351 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (645 inodes in the stack)", 2024-04-24T06:46:58.352 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-24T06:46:58.352 INFO:teuthology.orchestra.run.smithi005.stdout: "944d95de-94e8-46b3-9811-d47c69fffb8a": { 2024-04-24T06:46:58.353 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-24T06:46:58.353 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "944d95de-94e8-46b3-9811-d47c69fffb8a", 2024-04-24T06:46:58.353 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-04-24T06:46:58.353 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:46:58.354 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:46:58.354 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:46:58.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.339+0000 7f970f200640 1 -- 172.21.15.5:0/564906773 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f96f00a3e70 msgr2=0x7f96f00a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:58.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.339+0000 7f970f200640 1 --2- 172.21.15.5:0/564906773 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f96f00a3e70 0x7f96f00a6330 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9704021000 tx=0x7f9704022000 comp rx=0 tx=0).stop 2024-04-24T06:46:58.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.339+0000 7f970f200640 1 -- 172.21.15.5:0/564906773 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9708102fe0 msgr2=0x7f9708198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:46:58.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.339+0000 7f970f200640 1 --2- 172.21.15.5:0/564906773 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9708102fe0 0x7f9708198760 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f96f400e9f0 tx=0x7f96f400eeb0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.340+0000 7f970f200640 1 -- 172.21.15.5:0/564906773 shutdown_connections 2024-04-24T06:46:58.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.340+0000 7f970f200640 1 --2- 172.21.15.5:0/564906773 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f96f00a3e70 0x7f96f00a6330 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.340+0000 7f970f200640 1 --2- 172.21.15.5:0/564906773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f970810c400 0x7f970819d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.340+0000 7f970f200640 1 --2- 172.21.15.5:0/564906773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9708103900 0x7f9708198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.340+0000 7f970f200640 1 --2- 172.21.15.5:0/564906773 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9708102fe0 0x7f9708198760 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:46:58.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.340+0000 7f970f200640 1 -- 172.21.15.5:0/564906773 >> 172.21.15.5:0/564906773 conn(0x7f9708077ab0 msgr2=0x7f97080fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:46:58.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.340+0000 7f970f200640 1 -- 172.21.15.5:0/564906773 shutdown_connections 2024-04-24T06:46:58.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:46:58.340+0000 7f970f200640 1 -- 172.21.15.5:0/564906773 wait complete. 2024-04-24T06:46:58.361 INFO:tasks.cephfs.filesystem:scrub status for tag:944d95de-94e8-46b3-9811-d47c69fffb8a - {'path': '/', 'tag': '944d95de-94e8-46b3-9811-d47c69fffb8a', 'options': 'recursive,force'} 2024-04-24T06:46:59.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:59 smithi137 ceph-mon[33017]: scrub queued for path: / 2024-04-24T06:46:59.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:59 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:46:59.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:59 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:46:59.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:59 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:46:59.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:59 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:46:59.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:59 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:46:59.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:59 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:46:59.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:59 smithi137 ceph-mon[33017]: pgmap v91: 97 pgs: 97 active+clean; 87 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 644 KiB/s wr, 85 op/s 2024-04-24T06:46:59.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:59 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:46:59.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:59 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:46:59.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:59 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:46:59.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:59 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:46:59.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:46:59 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:46:59.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:59 smithi005 ceph-mon[27798]: scrub queued for path: / 2024-04-24T06:46:59.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:59 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:46:59.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:59 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:46:59.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:59 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:46:59.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:59 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:46:59.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:59 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:46:59.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:59 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:46:59.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:59 smithi005 ceph-mon[27798]: pgmap v91: 97 pgs: 97 active+clean; 87 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 644 KiB/s wr, 85 op/s 2024-04-24T06:46:59.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:59 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:46:59.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:59 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:46:59.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:59 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:46:59.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:59 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:46:59.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:46:59 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:46:59.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:59 smithi037 ceph-mon[32864]: scrub queued for path: / 2024-04-24T06:46:59.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:59 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:46:59.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:59 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:46:59.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:59 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:46:59.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:59 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:46:59.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:59 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:46:59.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:59 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:46:59.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:59 smithi037 ceph-mon[32864]: pgmap v91: 97 pgs: 97 active+clean; 87 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 644 KiB/s wr, 85 op/s 2024-04-24T06:46:59.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:59 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:46:59.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:59 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:46:59.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:59 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:46:59.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:59 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:46:59.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:46:59 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:47:01.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:01 smithi137 ceph-mon[33017]: pgmap v92: 97 pgs: 97 active+clean; 87 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 574 KiB/s wr, 83 op/s 2024-04-24T06:47:01.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:01 smithi005 ceph-mon[27798]: pgmap v92: 97 pgs: 97 active+clean; 87 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 574 KiB/s wr, 83 op/s 2024-04-24T06:47:01.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:01 smithi037 ceph-mon[32864]: pgmap v92: 97 pgs: 97 active+clean; 87 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 574 KiB/s wr, 83 op/s 2024-04-24T06:47:03.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:03 smithi137 ceph-mon[33017]: pgmap v93: 97 pgs: 97 active+clean; 87 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 561 KiB/s wr, 76 op/s 2024-04-24T06:47:03.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:03 smithi005 ceph-mon[27798]: pgmap v93: 97 pgs: 97 active+clean; 87 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 561 KiB/s wr, 76 op/s 2024-04-24T06:47:03.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:03 smithi037 ceph-mon[32864]: pgmap v93: 97 pgs: 97 active+clean; 87 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 561 KiB/s wr, 76 op/s 2024-04-24T06:47:05.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:05 smithi137 ceph-mon[33017]: pgmap v94: 97 pgs: 97 active+clean; 87 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 583 KiB/s wr, 94 op/s 2024-04-24T06:47:05.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:05 smithi005 ceph-mon[27798]: pgmap v94: 97 pgs: 97 active+clean; 87 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 583 KiB/s wr, 94 op/s 2024-04-24T06:47:05.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:05 smithi037 ceph-mon[32864]: pgmap v94: 97 pgs: 97 active+clean; 87 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 583 KiB/s wr, 94 op/s 2024-04-24T06:47:07.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:07 smithi137 ceph-mon[33017]: pgmap v95: 97 pgs: 97 active+clean; 87 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 34 KiB/s wr, 23 op/s 2024-04-24T06:47:07.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:07 smithi005 ceph-mon[27798]: pgmap v95: 97 pgs: 97 active+clean; 87 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 34 KiB/s wr, 23 op/s 2024-04-24T06:47:07.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:07 smithi037 ceph-mon[32864]: pgmap v95: 97 pgs: 97 active+clean; 87 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 34 KiB/s wr, 23 op/s 2024-04-24T06:47:09.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:09 smithi137 ceph-mon[33017]: pgmap v96: 97 pgs: 97 active+clean; 87 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 75 KiB/s wr, 25 op/s 2024-04-24T06:47:09.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:09 smithi005 ceph-mon[27798]: pgmap v96: 97 pgs: 97 active+clean; 87 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 75 KiB/s wr, 25 op/s 2024-04-24T06:47:09.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:09 smithi037 ceph-mon[32864]: pgmap v96: 97 pgs: 97 active+clean; 87 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 75 KiB/s wr, 25 op/s 2024-04-24T06:47:11.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:11 smithi137 ceph-mon[33017]: pgmap v97: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 118 KiB/s wr, 27 op/s 2024-04-24T06:47:11.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:11 smithi005 ceph-mon[27798]: pgmap v97: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 118 KiB/s wr, 27 op/s 2024-04-24T06:47:11.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:11 smithi037 ceph-mon[32864]: pgmap v97: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 118 KiB/s wr, 27 op/s 2024-04-24T06:47:13.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:13 smithi137 ceph-mon[33017]: pgmap v98: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 117 KiB/s wr, 23 op/s 2024-04-24T06:47:13.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:13 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:47:13.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:13 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:47:13.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:13 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:47:13.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:13 smithi005 ceph-mon[27798]: pgmap v98: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 117 KiB/s wr, 23 op/s 2024-04-24T06:47:13.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:13 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:47:13.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:13 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:47:13.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:13 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:47:13.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:13 smithi037 ceph-mon[32864]: pgmap v98: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 117 KiB/s wr, 23 op/s 2024-04-24T06:47:13.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:13 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:47:13.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:13 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:47:13.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:13 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:47:15.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:15 smithi137 ceph-mon[33017]: pgmap v99: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 119 KiB/s wr, 26 op/s 2024-04-24T06:47:15.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:15 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:47:15.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:15 smithi005 ceph-mon[27798]: pgmap v99: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 119 KiB/s wr, 26 op/s 2024-04-24T06:47:15.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:15 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:47:15.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:15 smithi037 ceph-mon[32864]: pgmap v99: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 119 KiB/s wr, 26 op/s 2024-04-24T06:47:15.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:15 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:47:17.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:17 smithi137 ceph-mon[33017]: pgmap v100: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 97 KiB/s wr, 8 op/s 2024-04-24T06:47:17.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:17 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:47:17.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:17 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:47:17.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:17 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:47:17.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:17 smithi005 ceph-mon[27798]: pgmap v100: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 97 KiB/s wr, 8 op/s 2024-04-24T06:47:17.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:17 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:47:17.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:17 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:47:17.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:17 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:47:17.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:17 smithi037 ceph-mon[32864]: pgmap v100: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 97 KiB/s wr, 8 op/s 2024-04-24T06:47:17.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:17 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:47:17.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:17 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:47:17.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:17 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:47:19.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:19 smithi137 ceph-mon[33017]: pgmap v101: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 98 KiB/s wr, 9 op/s 2024-04-24T06:47:19.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:19 smithi005 ceph-mon[27798]: pgmap v101: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 98 KiB/s wr, 9 op/s 2024-04-24T06:47:19.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:19 smithi037 ceph-mon[32864]: pgmap v101: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 98 KiB/s wr, 9 op/s 2024-04-24T06:47:21.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:21 smithi137 ceph-mon[33017]: pgmap v102: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 58 KiB/s wr, 9 op/s 2024-04-24T06:47:21.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:21 smithi005 ceph-mon[27798]: pgmap v102: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 58 KiB/s wr, 9 op/s 2024-04-24T06:47:21.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:21 smithi037 ceph-mon[32864]: pgmap v102: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 58 KiB/s wr, 9 op/s 2024-04-24T06:47:23.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:23 smithi137 ceph-mon[33017]: pgmap v103: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.4 KiB/s wr, 5 op/s 2024-04-24T06:47:23.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:23 smithi005 ceph-mon[27798]: pgmap v103: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.4 KiB/s wr, 5 op/s 2024-04-24T06:47:23.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:23 smithi037 ceph-mon[32864]: pgmap v103: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.4 KiB/s wr, 5 op/s 2024-04-24T06:47:25.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:25 smithi137 ceph-mon[33017]: pgmap v104: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-04-24T06:47:25.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:25 smithi005 ceph-mon[27798]: pgmap v104: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-04-24T06:47:25.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:25 smithi037 ceph-mon[32864]: pgmap v104: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-04-24T06:47:27.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:27 smithi137 ceph-mon[33017]: pgmap v105: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-24T06:47:27.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:27 smithi005 ceph-mon[27798]: pgmap v105: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-24T06:47:27.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:27 smithi037 ceph-mon[32864]: pgmap v105: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-24T06:47:28.362 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T06:47:28.363 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T06:47:28.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.588+0000 7f89cf200640 1 -- 172.21.15.5:0/4252111677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89d0102fb0 msgr2=0x7f89d0103390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:28.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.588+0000 7f89cf200640 1 --2- 172.21.15.5:0/4252111677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89d0102fb0 0x7f89d0103390 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f89c0009900 tx=0x7f89c002a410 comp rx=0 tx=0).stop 2024-04-24T06:47:28.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.589+0000 7f89cf200640 1 -- 172.21.15.5:0/4252111677 shutdown_connections 2024-04-24T06:47:28.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.589+0000 7f89cf200640 1 --2- 172.21.15.5:0/4252111677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89d010c420 0x7f89d010e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:28.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.589+0000 7f89cf200640 1 --2- 172.21.15.5:0/4252111677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89d01038d0 0x7f89d010bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:28.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.589+0000 7f89cf200640 1 --2- 172.21.15.5:0/4252111677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89d0102fb0 0x7f89d0103390 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:28.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.589+0000 7f89cf200640 1 -- 172.21.15.5:0/4252111677 >> 172.21.15.5:0/4252111677 conn(0x7f89d00fc7f0 msgr2=0x7f89d00fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:47:28.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.589+0000 7f89cf200640 1 -- 172.21.15.5:0/4252111677 shutdown_connections 2024-04-24T06:47:28.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.589+0000 7f89cf200640 1 -- 172.21.15.5:0/4252111677 wait complete. 2024-04-24T06:47:28.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.590+0000 7f89cf200640 1 Processor -- start 2024-04-24T06:47:28.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.590+0000 7f89cf200640 1 -- start start 2024-04-24T06:47:28.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.591+0000 7f89cf200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89d0102fb0 0x7f89d01a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:28.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.591+0000 7f89cf200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89d01038d0 0x7f89d01a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:28.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.591+0000 7f89cf200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89d010c420 0x7f89d019a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:28.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.591+0000 7f89cf200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f89d01121d0 con 0x7f89d0102fb0 2024-04-24T06:47:28.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.591+0000 7f89cf200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f89d0112050 con 0x7f89d010c420 2024-04-24T06:47:28.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.591+0000 7f89cf200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f89d0112350 con 0x7f89d01038d0 2024-04-24T06:47:28.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.591+0000 7f89cde00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89d0102fb0 0x7f89d01a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:28.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.591+0000 7f89ce800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89d010c420 0x7f89d019a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:28.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.591+0000 7f89cde00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89d0102fb0 0x7f89d01a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41364/0 (socket says 172.21.15.5:41364) 2024-04-24T06:47:28.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.591+0000 7f89cde00640 1 -- 172.21.15.5:0/3525275215 learned_addr learned my addr 172.21.15.5:0/3525275215 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:47:28.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.591+0000 7f89ce800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89d010c420 0x7f89d019a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:48570/0 (socket says 172.21.15.5:48570) 2024-04-24T06:47:28.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.591+0000 7f89c5400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89d01038d0 0x7f89d01a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:28.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.591+0000 7f89ce800640 1 -- 172.21.15.5:0/3525275215 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89d01038d0 msgr2=0x7f89d01a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:28.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.591+0000 7f89ce800640 1 --2- 172.21.15.5:0/3525275215 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89d01038d0 0x7f89d01a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:28.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.591+0000 7f89ce800640 1 -- 172.21.15.5:0/3525275215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89d0102fb0 msgr2=0x7f89d01a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:28.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.591+0000 7f89ce800640 1 --2- 172.21.15.5:0/3525275215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89d0102fb0 0x7f89d01a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:28.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.591+0000 7f89ce800640 1 -- 172.21.15.5:0/3525275215 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d019afa0 con 0x7f89d010c420 2024-04-24T06:47:28.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.592+0000 7f89ce800640 1 --2- 172.21.15.5:0/3525275215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89d010c420 0x7f89d019a7d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f89c8009fd0 tx=0x7f89c800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:47:28.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.592+0000 7f89c7400640 1 -- 172.21.15.5:0/3525275215 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c800ce20 con 0x7f89d010c420 2024-04-24T06:47:28.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.592+0000 7f89c7400640 1 -- 172.21.15.5:0/3525275215 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89c8005020 con 0x7f89d010c420 2024-04-24T06:47:28.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.592+0000 7f89cf200640 1 -- 172.21.15.5:0/3525275215 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89d019b290 con 0x7f89d010c420 2024-04-24T06:47:28.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.592+0000 7f89c7400640 1 -- 172.21.15.5:0/3525275215 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c8010720 con 0x7f89d010c420 2024-04-24T06:47:28.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.592+0000 7f89cf200640 1 -- 172.21.15.5:0/3525275215 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89d01a7460 con 0x7f89d010c420 2024-04-24T06:47:28.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.595+0000 7f89c7400640 1 -- 172.21.15.5:0/3525275215 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f89c8005540 con 0x7f89d010c420 2024-04-24T06:47:28.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.595+0000 7f89d7a18740 1 Processor -- start 2024-04-24T06:47:28.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.596+0000 7f89d7a18740 1 -- start start 2024-04-24T06:47:28.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.596+0000 7f89d7a18740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eed0817a30 0x55eed0817e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:28.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.596+0000 7f89d7a18740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eed0818350 0x55eed08224e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:28.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.596+0000 7f89d7a18740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed0822a20 0x55eed0824e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:28.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.596+0000 7f89d7a18740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55eed077d1f0 con 0x55eed0822a20 2024-04-24T06:47:28.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.596+0000 7f89d7a18740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55eed0723350 con 0x55eed0818350 2024-04-24T06:47:28.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.596+0000 7f89d7a18740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55eed07791b0 con 0x55eed0817a30 2024-04-24T06:47:28.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.596+0000 7f89cde00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eed0818350 0x55eed08224e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:28.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.597+0000 7f89c5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed0822a20 0x55eed0824e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:28.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.597+0000 7f89cde00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eed0818350 0x55eed08224e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:48580/0 (socket says 172.21.15.5:48580) 2024-04-24T06:47:28.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.597+0000 7f89cde00640 1 -- 172.21.15.5:0/2963235561 learned_addr learned my addr 172.21.15.5:0/2963235561 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:47:28.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.597+0000 7f89c5400640 1 --2- 172.21.15.5:0/2963235561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eed0817a30 0x55eed0817e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:28.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.597+0000 7f89cde00640 1 -- 172.21.15.5:0/2963235561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eed0817a30 msgr2=0x55eed0817e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:28.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.597+0000 7f89cde00640 1 --2- 172.21.15.5:0/2963235561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eed0817a30 0x55eed0817e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:28.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.597+0000 7f89cde00640 1 -- 172.21.15.5:0/2963235561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed0822a20 msgr2=0x55eed0824e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:28.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.597+0000 7f89cde00640 1 --2- 172.21.15.5:0/2963235561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed0822a20 0x55eed0824e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:28.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.597+0000 7f89cde00640 1 -- 172.21.15.5:0/2963235561 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eed0777f50 con 0x55eed0818350 2024-04-24T06:47:28.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.597+0000 7f89c5400640 1 --2- 172.21.15.5:0/2963235561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eed0817a30 0x55eed0817e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:47:28.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.597+0000 7f89c7400640 1 --2- 172.21.15.5:0/3525275215 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f89b40a3d20 0x7f89b40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:28.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.597+0000 7f89cde00640 1 --2- 172.21.15.5:0/2963235561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eed0818350 0x55eed08224e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f89c0009a20 tx=0x7f89c0005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56a45507a304ca23 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:47:28.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.598+0000 7f89c7400640 1 -- 172.21.15.5:0/3525275215 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f89c80d0ec0 con 0x7f89d010c420 2024-04-24T06:47:28.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.598+0000 7f89c5400640 1 --2- 172.21.15.5:0/3525275215 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f89b40a3d20 0x7f89b40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:28.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.598+0000 7f89bf000640 1 -- 172.21.15.5:0/2963235561 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c0040070 con 0x55eed0818350 2024-04-24T06:47:28.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.598+0000 7f89bf000640 1 -- 172.21.15.5:0/2963235561 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89c0004030 con 0x55eed0818350 2024-04-24T06:47:28.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.598+0000 7f89bf000640 1 -- 172.21.15.5:0/2963235561 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c0035040 con 0x55eed0818350 2024-04-24T06:47:28.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.599+0000 7f89d7a18740 1 -- 172.21.15.5:0/2963235561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eed0818350 msgr2=0x55eed08224e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:28.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.599+0000 7f89c5400640 1 --2- 172.21.15.5:0/3525275215 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f89b40a3d20 0x7f89b40a61e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x55eed0814ee0 tx=0x7f89b8015040 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:47:28.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.599+0000 7f89d7a18740 1 --2- 172.21.15.5:0/2963235561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eed0818350 0x55eed08224e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f89c0009a20 tx=0x7f89c0005dd0 comp rx=0 tx=0).stop 2024-04-24T06:47:28.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.599+0000 7f89d7a18740 1 -- 172.21.15.5:0/2963235561 shutdown_connections 2024-04-24T06:47:28.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.599+0000 7f89d7a18740 1 --2- 172.21.15.5:0/2963235561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed0822a20 0x55eed0824e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:28.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.599+0000 7f89d7a18740 1 --2- 172.21.15.5:0/2963235561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eed0818350 0x55eed08224e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:28.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.599+0000 7f89d7a18740 1 --2- 172.21.15.5:0/2963235561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eed0817a30 0x55eed0817e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:28.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.599+0000 7f89d7a18740 1 -- 172.21.15.5:0/2963235561 >> 172.21.15.5:0/2963235561 conn(0x55eed0813d50 msgr2=0x55eed0814130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:47:28.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.599+0000 7f89d7a18740 1 -- 172.21.15.5:0/2963235561 shutdown_connections 2024-04-24T06:47:28.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.599+0000 7f89d7a18740 1 -- 172.21.15.5:0/2963235561 wait complete. 2024-04-24T06:47:28.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.599+0000 7f89d7a18740 1 -- start start 2024-04-24T06:47:28.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.600+0000 7f89d7a18740 1 Processor -- start 2024-04-24T06:47:28.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.600+0000 7f89d7a18740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eed0817a30 0x55eed081f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:28.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.600+0000 7f89d7a18740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed0818350 0x55eed081ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:28.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.600+0000 7f89d7a18740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eed0822a20 0x55eed0820450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:28.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.600+0000 7f89d7a18740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55eed08257b0 con 0x55eed0818350 2024-04-24T06:47:28.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.600+0000 7f89d7a18740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55eed0736690 con 0x55eed0822a20 2024-04-24T06:47:28.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.600+0000 7f89d7a18740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55eed0825930 con 0x55eed0817a30 2024-04-24T06:47:28.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.600+0000 7f89d7a18740 10 client.0 _ll_register_callbacks cb 0x55eed0772650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:47:28.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.600+0000 7f89c5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed0818350 0x55eed081ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:28.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.600+0000 7f89c5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed0818350 0x55eed081ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41372/0 (socket says 172.21.15.5:41372) 2024-04-24T06:47:28.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.601+0000 7f89c5400640 1 -- 172.21.15.5:0/3740747231 learned_addr learned my addr 172.21.15.5:0/3740747231 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:47:28.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.601+0000 7f89ce800640 1 --2- 172.21.15.5:0/3740747231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eed0822a20 0x55eed0820450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:28.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.601+0000 7f89cde00640 1 --2- 172.21.15.5:0/3740747231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eed0817a30 0x55eed081f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:28.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.601+0000 7f89c5400640 1 -- 172.21.15.5:0/3740747231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eed0817a30 msgr2=0x55eed081f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:28.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.601+0000 7f89c5400640 1 --2- 172.21.15.5:0/3740747231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eed0817a30 0x55eed081f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:28.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.601+0000 7f89c5400640 1 -- 172.21.15.5:0/3740747231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eed0822a20 msgr2=0x55eed0820450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:28.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.601+0000 7f89c5400640 1 --2- 172.21.15.5:0/3740747231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eed0822a20 0x55eed0820450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:28.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.601+0000 7f89c5400640 1 -- 172.21.15.5:0/3740747231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eed0735920 con 0x55eed0818350 2024-04-24T06:47:28.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.601+0000 7f89c5400640 1 --2- 172.21.15.5:0/3740747231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed0818350 0x55eed081ff10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f89b80209e0 tx=0x7f89b8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520b09bbdd2d5f19 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:47:28.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.601+0000 7f89b3400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:47:28.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.602+0000 7f89b3400640 1 -- 172.21.15.5:0/3740747231 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b801eac0 con 0x55eed0818350 2024-04-24T06:47:28.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.602+0000 7f89b3400640 1 -- 172.21.15.5:0/3740747231 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f89b801ec60 con 0x55eed0818350 2024-04-24T06:47:28.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.602+0000 7f89d7a18740 1 -- 172.21.15.5:0/3740747231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89c0004030 con 0x55eed0818350 2024-04-24T06:47:28.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.602+0000 7f89b3400640 1 -- 172.21.15.5:0/3740747231 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f89b801ee60 con 0x55eed0818350 2024-04-24T06:47:28.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.602+0000 7f89d7a18740 10 client.15879 fetch_fsmap learned FSMap version 25 2024-04-24T06:47:28.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.602+0000 7f89d7a18740 1 -- 172.21.15.5:0/3740747231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55eed05597d0 con 0x55eed0818350 2024-04-24T06:47:28.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.602+0000 7f89b3400640 1 -- 172.21.15.5:0/3740747231 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89c0004030 con 0x55eed0818350 2024-04-24T06:47:28.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.603+0000 7f89b3400640 1 -- 172.21.15.5:0/3740747231 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f89b802a650 con 0x55eed0818350 2024-04-24T06:47:28.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.603+0000 7f89d7a18740 10 client.15879 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:47:28.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.603+0000 7f89d7a18740 10 client.15879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:47:28.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.603+0000 7f89d7a18740 20 client.15879 populate_metadata read hostname 'smithi005' 2024-04-24T06:47:28.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.603+0000 7f89d7a18740 1 --2- 172.21.15.5:0/3740747231 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55eed0832e80 0x55eed08352e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:28.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.603+0000 7f89d7a18740 4 client.15879 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:47:28.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.603+0000 7f89d7a18740 1 -- 172.21.15.5:0/3740747231 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eed078ac10 con 0x55eed0832e80 2024-04-24T06:47:28.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.603+0000 7f89cde00640 1 --2- 172.21.15.5:0/3740747231 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55eed0832e80 0x55eed08352e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:28.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.604+0000 7f89cde00640 1 --2- 172.21.15.5:0/3740747231 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55eed0832e80 0x55eed08352e0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb5aa6cc0b926eee server_cookie=f335e4bee1a5a7d2 in_seq=0 out_seq=0 2024-04-24T06:47:28.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.604+0000 7f89b3400640 10 client.15879 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:47:28.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.605+0000 7f89b3400640 1 -- 172.21.15.5:0/3740747231 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eed078ac10 con 0x55eed0832e80 2024-04-24T06:47:28.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.605+0000 7f89b3400640 10 client.15879 handle_command_reply: tid=0 2024-04-24T06:47:28.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.605+0000 7f89d7a18740 1 client.15879 shutdown 2024-04-24T06:47:28.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.605+0000 7f89d7a18740 1 -- 172.21.15.5:0/3740747231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed0818350 msgr2=0x55eed081ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:28.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.605+0000 7f89d7a18740 1 --2- 172.21.15.5:0/3740747231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed0818350 0x55eed081ff10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f89b80209e0 tx=0x7f89b8005f90 comp rx=0 tx=0).stop 2024-04-24T06:47:28.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.605+0000 7f89d7a18740 1 -- 172.21.15.5:0/3740747231 shutdown_connections 2024-04-24T06:47:28.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.605+0000 7f89d7a18740 1 --2- 172.21.15.5:0/3740747231 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55eed0832e80 0x55eed08352e0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:28.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.605+0000 7f89d7a18740 1 --2- 172.21.15.5:0/3740747231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eed0822a20 0x55eed0820450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:28.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.605+0000 7f89d7a18740 1 --2- 172.21.15.5:0/3740747231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed0818350 0x55eed081ff10 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:28.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.605+0000 7f89d7a18740 1 --2- 172.21.15.5:0/3740747231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eed0817a30 0x55eed081f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:28.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.605+0000 7f89b3400640 0 client.15879 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:47:28.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.605+0000 7f89d7a18740 1 -- 172.21.15.5:0/3740747231 >> 172.21.15.5:0/3740747231 conn(0x55eed0813d50 msgr2=0x55eed0819b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:47:28.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.605+0000 7f89d7a18740 1 -- 172.21.15.5:0/3740747231 shutdown_connections 2024-04-24T06:47:28.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.606+0000 7f89d7a18740 1 -- 172.21.15.5:0/3740747231 wait complete. 2024-04-24T06:47:28.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.606+0000 7f89d7a18740 20 client.15879 trim_cache size 0 max 16384 2024-04-24T06:47:28.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.625+0000 7f89d7a18740 1 Processor -- start 2024-04-24T06:47:28.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.625+0000 7f89d7a18740 1 -- start start 2024-04-24T06:47:28.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.626+0000 7f89d7a18740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed0817a30 0x55eed0820220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:28.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.626+0000 7f89d7a18740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eed0818350 0x55eed081a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:28.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.626+0000 7f89d7a18740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eed0822a20 0x55eed083abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:28.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.626+0000 7f89d7a18740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f89d01b2690 con 0x55eed0817a30 2024-04-24T06:47:28.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.626+0000 7f89d7a18740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f89d01aff60 con 0x55eed0818350 2024-04-24T06:47:28.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.626+0000 7f89d7a18740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f89d01addb0 con 0x55eed0822a20 2024-04-24T06:47:28.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.626+0000 7f89cde00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed0817a30 0x55eed0820220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:28.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.626+0000 7f89cde00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed0817a30 0x55eed0820220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41376/0 (socket says 172.21.15.5:41376) 2024-04-24T06:47:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.626+0000 7f89cde00640 1 -- 172.21.15.5:0/954285073 learned_addr learned my addr 172.21.15.5:0/954285073 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:47:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.626+0000 7f89c5400640 1 --2- 172.21.15.5:0/954285073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eed0818350 0x55eed081a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.626+0000 7f89ce800640 1 --2- 172.21.15.5:0/954285073 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eed0822a20 0x55eed083abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.626+0000 7f89cde00640 1 -- 172.21.15.5:0/954285073 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eed0822a20 msgr2=0x55eed083abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.626+0000 7f89cde00640 1 --2- 172.21.15.5:0/954285073 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eed0822a20 0x55eed083abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.626+0000 7f89cde00640 1 -- 172.21.15.5:0/954285073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eed0818350 msgr2=0x55eed081a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.626+0000 7f89cde00640 1 --2- 172.21.15.5:0/954285073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eed0818350 0x55eed081a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.626+0000 7f89cde00640 1 -- 172.21.15.5:0/954285073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eed048f3f0 con 0x55eed0817a30 2024-04-24T06:47:28.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.626+0000 7f89ce800640 1 --2- 172.21.15.5:0/954285073 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eed0822a20 0x55eed083abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:47:28.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.627+0000 7f89cde00640 1 --2- 172.21.15.5:0/954285073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed0817a30 0x55eed0820220 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f89c0005e40 tx=0x7f89c00050e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad305083c932b20 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:47:28.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.627+0000 7f89be600640 1 -- 172.21.15.5:0/954285073 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c0057070 con 0x55eed0817a30 2024-04-24T06:47:28.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.627+0000 7f89be600640 1 -- 172.21.15.5:0/954285073 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89c00055f0 con 0x55eed0817a30 2024-04-24T06:47:28.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.627+0000 7f89be600640 1 -- 172.21.15.5:0/954285073 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c0035040 con 0x55eed0817a30 2024-04-24T06:47:28.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.628+0000 7f89d7a18740 1 -- 172.21.15.5:0/954285073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed0817a30 msgr2=0x55eed0820220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:28.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.628+0000 7f89d7a18740 1 --2- 172.21.15.5:0/954285073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed0817a30 0x55eed0820220 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f89c0005e40 tx=0x7f89c00050e0 comp rx=0 tx=0).stop 2024-04-24T06:47:28.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.628+0000 7f89d7a18740 1 -- 172.21.15.5:0/954285073 shutdown_connections 2024-04-24T06:47:28.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.628+0000 7f89d7a18740 1 --2- 172.21.15.5:0/954285073 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eed0822a20 0x55eed083abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:28.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.628+0000 7f89d7a18740 1 --2- 172.21.15.5:0/954285073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eed0818350 0x55eed081a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:28.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.628+0000 7f89d7a18740 1 --2- 172.21.15.5:0/954285073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed0817a30 0x55eed0820220 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:28.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.628+0000 7f89d7a18740 1 -- 172.21.15.5:0/954285073 >> 172.21.15.5:0/954285073 conn(0x55eed0813d50 msgr2=0x55eed0812740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:47:28.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.628+0000 7f89d7a18740 1 -- 172.21.15.5:0/954285073 shutdown_connections 2024-04-24T06:47:28.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.628+0000 7f89d7a18740 1 -- 172.21.15.5:0/954285073 wait complete. 2024-04-24T06:47:28.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.628+0000 7f89d7a18740 1 -- start start 2024-04-24T06:47:28.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.629+0000 7f89d7a18740 1 Processor -- start 2024-04-24T06:47:28.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.629+0000 7f89d7a18740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed0817a30 0x55eed082b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:28.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.629+0000 7f89d7a18740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eed0818350 0x55eed081c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:28.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.629+0000 7f89d7a18740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eed0822a20 0x55eed081c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:28.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.629+0000 7f89d7a18740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f89d01ae420 con 0x55eed0817a30 2024-04-24T06:47:28.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.629+0000 7f89d7a18740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f89d01af890 con 0x55eed0818350 2024-04-24T06:47:28.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.629+0000 7f89d7a18740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f89d01aeaf0 con 0x55eed0822a20 2024-04-24T06:47:28.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.629+0000 7f89d7a18740 10 client.0 _ll_register_callbacks cb 0x7f89b8009a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:47:28.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.629+0000 7f89cde00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed0817a30 0x55eed082b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:28.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.629+0000 7f89cde00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed0817a30 0x55eed082b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41378/0 (socket says 172.21.15.5:41378) 2024-04-24T06:47:28.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.629+0000 7f89cde00640 1 -- 172.21.15.5:0/3160239718 learned_addr learned my addr 172.21.15.5:0/3160239718 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:47:28.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.629+0000 7f89ce800640 1 --2- 172.21.15.5:0/3160239718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eed0822a20 0x55eed081c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:28.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.630+0000 7f89c5400640 1 --2- 172.21.15.5:0/3160239718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eed0818350 0x55eed081c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:28.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.630+0000 7f89cde00640 1 -- 172.21.15.5:0/3160239718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eed0822a20 msgr2=0x55eed081c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:28.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.630+0000 7f89cde00640 1 --2- 172.21.15.5:0/3160239718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eed0822a20 0x55eed081c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:28.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.630+0000 7f89cde00640 1 -- 172.21.15.5:0/3160239718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eed0818350 msgr2=0x55eed081c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:28.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.630+0000 7f89cde00640 1 --2- 172.21.15.5:0/3160239718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eed0818350 0x55eed081c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:28.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.630+0000 7f89cde00640 1 -- 172.21.15.5:0/3160239718 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eed044f0f0 con 0x55eed0817a30 2024-04-24T06:47:28.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.630+0000 7f89cde00640 1 --2- 172.21.15.5:0/3160239718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed0817a30 0x55eed082b010 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f89c00057c0 tx=0x7f89c0032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=873b31db61a1d08f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:47:28.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.630+0000 7f89b3400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:47:28.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.630+0000 7f89b3400640 1 -- 172.21.15.5:0/3160239718 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c0060060 con 0x55eed0817a30 2024-04-24T06:47:28.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.630+0000 7f89b3400640 1 -- 172.21.15.5:0/3160239718 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f89c0005150 con 0x55eed0817a30 2024-04-24T06:47:28.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.630+0000 7f89d7a18740 1 -- 172.21.15.5:0/3160239718 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89c00055f0 con 0x55eed0817a30 2024-04-24T06:47:28.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.631+0000 7f89b3400640 1 -- 172.21.15.5:0/3160239718 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f89c0035070 con 0x55eed0817a30 2024-04-24T06:47:28.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.631+0000 7f89d7a18740 10 client.15885 fetch_fsmap learned FSMap version 25 2024-04-24T06:47:28.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.631+0000 7f89d7a18740 1 -- 172.21.15.5:0/3160239718 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55eed0742810 con 0x55eed0817a30 2024-04-24T06:47:28.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.631+0000 7f89b3400640 1 -- 172.21.15.5:0/3160239718 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89c00055f0 con 0x55eed0817a30 2024-04-24T06:47:28.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.631+0000 7f89b3400640 1 -- 172.21.15.5:0/3160239718 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f89c0052760 con 0x55eed0817a30 2024-04-24T06:47:28.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.631+0000 7f89d7a18740 10 client.15885 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:47:28.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.631+0000 7f89d7a18740 10 client.15885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:47:28.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.631+0000 7f89d7a18740 20 client.15885 populate_metadata read hostname 'smithi005' 2024-04-24T06:47:28.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.631+0000 7f89d7a18740 1 --2- 172.21.15.5:0/3160239718 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55eed08a1b60 0x55eed08a3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:28.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.631+0000 7f89d7a18740 4 client.15885 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:47:28.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.631+0000 7f89d7a18740 1 -- 172.21.15.5:0/3160239718 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55eed0784a80 con 0x55eed08a1b60 2024-04-24T06:47:28.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.632+0000 7f89c5400640 1 --2- 172.21.15.5:0/3160239718 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55eed08a1b60 0x55eed08a3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:28.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.632+0000 7f89c5400640 1 --2- 172.21.15.5:0/3160239718 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55eed08a1b60 0x55eed08a3f50 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ac2b88340b5b226 server_cookie=4e9fdb9c8797e7fc in_seq=0 out_seq=0 2024-04-24T06:47:28.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.632+0000 7f89b3400640 10 client.15885 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:47:28.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.632+0000 7f89b3400640 1 -- 172.21.15.5:0/3160239718 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55eed0784a80 con 0x55eed08a1b60 2024-04-24T06:47:28.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.632+0000 7f89b3400640 10 client.15885 handle_command_reply: tid=0 2024-04-24T06:47:28.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.632+0000 7f89d7a18740 1 client.15885 shutdown 2024-04-24T06:47:28.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.633+0000 7f89d7a18740 1 -- 172.21.15.5:0/3160239718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed0817a30 msgr2=0x55eed082b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:28.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.633+0000 7f89d7a18740 1 --2- 172.21.15.5:0/3160239718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed0817a30 0x55eed082b010 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f89c00057c0 tx=0x7f89c0032f90 comp rx=0 tx=0).stop 2024-04-24T06:47:28.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.633+0000 7f89d7a18740 1 -- 172.21.15.5:0/3160239718 shutdown_connections 2024-04-24T06:47:28.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.633+0000 7f89d7a18740 1 --2- 172.21.15.5:0/3160239718 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55eed08a1b60 0x55eed08a3f50 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:28.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.633+0000 7f89d7a18740 1 --2- 172.21.15.5:0/3160239718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eed0822a20 0x55eed081c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:28.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.633+0000 7f89d7a18740 1 --2- 172.21.15.5:0/3160239718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eed0818350 0x55eed081c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:28.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.633+0000 7f89d7a18740 1 --2- 172.21.15.5:0/3160239718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eed0817a30 0x55eed082b010 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:28.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.633+0000 7f89d7a18740 1 -- 172.21.15.5:0/3160239718 >> 172.21.15.5:0/3160239718 conn(0x55eed0813d50 msgr2=0x55eed0812740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:47:28.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.633+0000 7f89b3400640 0 client.15885 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:47:28.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.633+0000 7f89d7a18740 1 -- 172.21.15.5:0/3160239718 shutdown_connections 2024-04-24T06:47:28.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.633+0000 7f89d7a18740 1 -- 172.21.15.5:0/3160239718 wait complete. 2024-04-24T06:47:28.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.633+0000 7f89d7a18740 20 client.15885 trim_cache size 0 max 16384 2024-04-24T06:47:28.710 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:47:28.710 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-24T06:47:28.710 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-24T06:47:28.710 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:47:28.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.634+0000 7f89cf200640 1 -- 172.21.15.5:0/3525275215 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f89b40a3d20 msgr2=0x7f89b40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:28.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.634+0000 7f89cf200640 1 --2- 172.21.15.5:0/3525275215 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f89b40a3d20 0x7f89b40a61e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x55eed0814ee0 tx=0x7f89b8015040 comp rx=0 tx=0).stop 2024-04-24T06:47:28.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.634+0000 7f89cf200640 1 -- 172.21.15.5:0/3525275215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89d010c420 msgr2=0x7f89d019a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:28.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.634+0000 7f89cf200640 1 --2- 172.21.15.5:0/3525275215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89d010c420 0x7f89d019a7d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f89c8009fd0 tx=0x7f89c800ef10 comp rx=0 tx=0).stop 2024-04-24T06:47:28.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.634+0000 7f89cf200640 1 -- 172.21.15.5:0/3525275215 shutdown_connections 2024-04-24T06:47:28.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.634+0000 7f89cf200640 1 --2- 172.21.15.5:0/3525275215 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f89b40a3d20 0x7f89b40a61e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:28.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.634+0000 7f89cf200640 1 --2- 172.21.15.5:0/3525275215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89d010c420 0x7f89d019a7d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:28.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.634+0000 7f89cf200640 1 --2- 172.21.15.5:0/3525275215 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89d01038d0 0x7f89d01a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:28.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.634+0000 7f89cf200640 1 --2- 172.21.15.5:0/3525275215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89d0102fb0 0x7f89d01a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:28.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.634+0000 7f89cf200640 1 -- 172.21.15.5:0/3525275215 >> 172.21.15.5:0/3525275215 conn(0x7f89d00fc7f0 msgr2=0x7f89d0101c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:47:28.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.634+0000 7f89cf200640 1 -- 172.21.15.5:0/3525275215 shutdown_connections 2024-04-24T06:47:28.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.634+0000 7f89cf200640 1 -- 172.21.15.5:0/3525275215 wait complete. 2024-04-24T06:47:28.713 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T06:47:28.714 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:47:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.931+0000 7f4b1e400640 1 -- 172.21.15.5:0/2258479287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b18103080 msgr2=0x7f4b18103460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.931+0000 7f4b1e400640 1 --2- 172.21.15.5:0/2258479287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b18103080 0x7f4b18103460 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c009900 tx=0x7f4b0c02a480 comp rx=0 tx=0).stop 2024-04-24T06:47:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.932+0000 7f4b1e400640 1 -- 172.21.15.5:0/2258479287 shutdown_connections 2024-04-24T06:47:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.932+0000 7f4b1e400640 1 --2- 172.21.15.5:0/2258479287 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b1810c4a0 0x7f4b1810e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.932+0000 7f4b1e400640 1 --2- 172.21.15.5:0/2258479287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b181039a0 0x7f4b1810be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.932+0000 7f4b1e400640 1 --2- 172.21.15.5:0/2258479287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b18103080 0x7f4b18103460 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.932+0000 7f4b1e400640 1 -- 172.21.15.5:0/2258479287 >> 172.21.15.5:0/2258479287 conn(0x7f4b180fc880 msgr2=0x7f4b180feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:47:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.932+0000 7f4b1e400640 1 -- 172.21.15.5:0/2258479287 shutdown_connections 2024-04-24T06:47:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.932+0000 7f4b1e400640 1 -- 172.21.15.5:0/2258479287 wait complete. 2024-04-24T06:47:28.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.933+0000 7f4b1e400640 1 Processor -- start 2024-04-24T06:47:28.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.933+0000 7f4b1e400640 1 -- start start 2024-04-24T06:47:28.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.933+0000 7f4b1e400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b18103080 0x7f4b181a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.933+0000 7f4b1e400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b181039a0 0x7f4b181a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.933+0000 7f4b1e400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b1810c4a0 0x7f4b1819a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:28.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.933+0000 7f4b1e400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4b181122b0 con 0x7f4b1810c4a0 2024-04-24T06:47:28.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.933+0000 7f4b1e400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4b18112130 con 0x7f4b181039a0 2024-04-24T06:47:28.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.933+0000 7f4b1e400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4b18112430 con 0x7f4b18103080 2024-04-24T06:47:28.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.933+0000 7f4b1da00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b1810c4a0 0x7f4b1819a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:28.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.934+0000 7f4b17e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b181039a0 0x7f4b181a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:28.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.934+0000 7f4b1da00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b1810c4a0 0x7f4b1819a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41404/0 (socket says 172.21.15.5:41404) 2024-04-24T06:47:28.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.934+0000 7f4b17e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b181039a0 0x7f4b181a0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:48624/0 (socket says 172.21.15.5:48624) 2024-04-24T06:47:28.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.934+0000 7f4b1da00640 1 -- 172.21.15.5:0/4274274332 learned_addr learned my addr 172.21.15.5:0/4274274332 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:47:28.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.934+0000 7f4b1d000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b18103080 0x7f4b181a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:28.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.934+0000 7f4b1d000640 1 -- 172.21.15.5:0/4274274332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b181039a0 msgr2=0x7f4b181a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:28.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.934+0000 7f4b1d000640 1 --2- 172.21.15.5:0/4274274332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b181039a0 0x7f4b181a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:28.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.934+0000 7f4b1d000640 1 -- 172.21.15.5:0/4274274332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b1810c4a0 msgr2=0x7f4b1819a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:28.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.934+0000 7f4b1d000640 1 --2- 172.21.15.5:0/4274274332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b1810c4a0 0x7f4b1819a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:28.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.934+0000 7f4b1d000640 1 -- 172.21.15.5:0/4274274332 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b1819b010 con 0x7f4b18103080 2024-04-24T06:47:28.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.934+0000 7f4b1da00640 1 --2- 172.21.15.5:0/4274274332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b1810c4a0 0x7f4b1819a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:47:28.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.934+0000 7f4b1d000640 1 --2- 172.21.15.5:0/4274274332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b18103080 0x7f4b181a0640 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c02a9a0 tx=0x7f4b0c003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:47:28.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.935+0000 7f4b15600640 1 -- 172.21.15.5:0/4274274332 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c03f070 con 0x7f4b18103080 2024-04-24T06:47:28.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.935+0000 7f4b1e400640 1 -- 172.21.15.5:0/4274274332 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b1819b2a0 con 0x7f4b18103080 2024-04-24T06:47:28.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.935+0000 7f4b15600640 1 -- 172.21.15.5:0/4274274332 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b0c031c90 con 0x7f4b18103080 2024-04-24T06:47:28.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.935+0000 7f4b15600640 1 -- 172.21.15.5:0/4274274332 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c035030 con 0x7f4b18103080 2024-04-24T06:47:28.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.935+0000 7f4b1e400640 1 -- 172.21.15.5:0/4274274332 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b181a7500 con 0x7f4b18103080 2024-04-24T06:47:28.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.936+0000 7f4b1e400640 1 -- 172.21.15.5:0/4274274332 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b1804fe00 con 0x7f4b18103080 2024-04-24T06:47:28.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.938+0000 7f4b15600640 1 -- 172.21.15.5:0/4274274332 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f4b0c042070 con 0x7f4b18103080 2024-04-24T06:47:28.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.940+0000 7f4b15600640 1 --2- 172.21.15.5:0/4274274332 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f4b000a3d90 0x7f4b000a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:28.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.940+0000 7f4b15600640 1 -- 172.21.15.5:0/4274274332 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f4b0c0f1680 con 0x7f4b18103080 2024-04-24T06:47:28.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.940+0000 7f4b17e00640 1 --2- 172.21.15.5:0/4274274332 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f4b000a3d90 0x7f4b000a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:28.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.941+0000 7f4b17e00640 1 --2- 172.21.15.5:0/4274274332 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f4b000a3d90 0x7f4b000a6250 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4b180fed50 tx=0x7f4b04005f90 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:47:28.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:28.942+0000 7f4b15600640 1 -- 172.21.15.5:0/4274274332 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4b0c0bd680 con 0x7f4b18103080 2024-04-24T06:47:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.190+0000 7f4b1e400640 1 -- 172.21.15.5:0/4274274332 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b1819bdc0 con 0x7f4b18103080 2024-04-24T06:47:29.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.192+0000 7f4b15600640 1 -- 172.21.15.5:0/4274274332 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11859 (secure 0 0 0) 0x7f4b0c0c1980 con 0x7f4b18103080 2024-04-24T06:47:29.193 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:47:29.193 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1735462750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1735462750},{"type":"v1","addr":"172.21.15.5:6837","nonce":1735462750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1499469370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1499469370},{"type":"v1","addr":"172.21.15.5:6841","nonce":1499469370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3705638847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3705638847},{"type":"v1","addr":"172.21.15.37:6835","nonce":3705638847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T06:41:34.764490+0000","modified":"2024-04-24T06:46:02.765290+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24550,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6837/1390103296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1390103296},{"type":"v1","addr":"172.21.15.37:6837","nonce":1390103296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3217754215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3217754215},{"type":"v1","addr":"172.21.15.137:6835","nonce":3217754215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6837/3044222976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3044222976},{"type":"v1","addr":"172.21.15.137:6837","nonce":3044222976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.5:6839/3173201459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3173201459},{"type":"v1","addr":"172.21.15.5:6839","nonce":3173201459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6839/1889774232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1889774232},{"type":"v1","addr":"172.21.15.137:6839","nonce":1889774232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6839/3536721753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":3536721753},{"type":"v1","addr":"172.21.15.37:6839","nonce":3536721753}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24539,24550,24569]},"id":1}]} 2024-04-24T06:47:29.193 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-04-24T06:47:29.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.194+0000 7f4b1e400640 1 -- 172.21.15.5:0/4274274332 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f4b000a3d90 msgr2=0x7f4b000a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:29.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.194+0000 7f4b1e400640 1 --2- 172.21.15.5:0/4274274332 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f4b000a3d90 0x7f4b000a6250 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4b180fed50 tx=0x7f4b04005f90 comp rx=0 tx=0).stop 2024-04-24T06:47:29.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.194+0000 7f4b1e400640 1 -- 172.21.15.5:0/4274274332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b18103080 msgr2=0x7f4b181a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:29.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.194+0000 7f4b1e400640 1 --2- 172.21.15.5:0/4274274332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b18103080 0x7f4b181a0640 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c02a9a0 tx=0x7f4b0c003b20 comp rx=0 tx=0).stop 2024-04-24T06:47:29.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.194+0000 7f4b1e400640 1 -- 172.21.15.5:0/4274274332 shutdown_connections 2024-04-24T06:47:29.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.194+0000 7f4b1e400640 1 --2- 172.21.15.5:0/4274274332 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f4b000a3d90 0x7f4b000a6250 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.194+0000 7f4b1e400640 1 --2- 172.21.15.5:0/4274274332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b1810c4a0 0x7f4b1819a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.194+0000 7f4b1e400640 1 --2- 172.21.15.5:0/4274274332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b181039a0 0x7f4b181a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.194+0000 7f4b1e400640 1 --2- 172.21.15.5:0/4274274332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b18103080 0x7f4b181a0640 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.194+0000 7f4b1e400640 1 -- 172.21.15.5:0/4274274332 >> 172.21.15.5:0/4274274332 conn(0x7f4b180fc880 msgr2=0x7f4b18101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:47:29.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.194+0000 7f4b1e400640 1 -- 172.21.15.5:0/4274274332 shutdown_connections 2024-04-24T06:47:29.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.194+0000 7f4b1e400640 1 -- 172.21.15.5:0/4274274332 wait complete. 2024-04-24T06:47:29.216 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T06:47:29.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.429+0000 7efc97e00640 1 -- 172.21.15.5:0/2161166948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc98108540 msgr2=0x7efc981089a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:29.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.429+0000 7efc97e00640 1 --2- 172.21.15.5:0/2161166948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc98108540 0x7efc981089a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7efc84009930 tx=0x7efc8402a4d0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.429+0000 7efc97e00640 1 -- 172.21.15.5:0/2161166948 shutdown_connections 2024-04-24T06:47:29.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.429+0000 7efc97e00640 1 --2- 172.21.15.5:0/2161166948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc98108ee0 0x7efc981152b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.429+0000 7efc97e00640 1 --2- 172.21.15.5:0/2161166948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc98108540 0x7efc981089a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.429+0000 7efc97e00640 1 --2- 172.21.15.5:0/2161166948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc9810e4f0 0x7efc9810e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.429+0000 7efc97e00640 1 -- 172.21.15.5:0/2161166948 >> 172.21.15.5:0/2161166948 conn(0x7efc98077e10 msgr2=0x7efc98106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:47:29.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.429+0000 7efc97e00640 1 -- 172.21.15.5:0/2161166948 shutdown_connections 2024-04-24T06:47:29.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.429+0000 7efc97e00640 1 -- 172.21.15.5:0/2161166948 wait complete. 2024-04-24T06:47:29.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.430+0000 7efc97e00640 1 Processor -- start 2024-04-24T06:47:29.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.430+0000 7efc97e00640 1 -- start start 2024-04-24T06:47:29.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.431+0000 7efc97e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc98108540 0x7efc981a4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.431+0000 7efc97e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc98108ee0 0x7efc981a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.431+0000 7efc97e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc9810e4f0 0x7efc9819eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.431+0000 7efc97e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efc981187f0 con 0x7efc98108ee0 2024-04-24T06:47:29.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.431+0000 7efc97e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efc98118670 con 0x7efc98108540 2024-04-24T06:47:29.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.431+0000 7efc97e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efc98118970 con 0x7efc9810e4f0 2024-04-24T06:47:29.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.431+0000 7efc96a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc98108540 0x7efc981a4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.431+0000 7efc96000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc98108ee0 0x7efc981a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.431+0000 7efc97400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc9810e4f0 0x7efc9819eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.431+0000 7efc96a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc98108540 0x7efc981a4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:48634/0 (socket says 172.21.15.5:48634) 2024-04-24T06:47:29.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.431+0000 7efc96a00640 1 -- 172.21.15.5:0/1180226905 learned_addr learned my addr 172.21.15.5:0/1180226905 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:47:29.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.432+0000 7efc96a00640 1 -- 172.21.15.5:0/1180226905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc9810e4f0 msgr2=0x7efc9819eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:29.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.432+0000 7efc96a00640 1 --2- 172.21.15.5:0/1180226905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc9810e4f0 0x7efc9819eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.432+0000 7efc96a00640 1 -- 172.21.15.5:0/1180226905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc98108ee0 msgr2=0x7efc981a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:29.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.432+0000 7efc96a00640 1 --2- 172.21.15.5:0/1180226905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc98108ee0 0x7efc981a4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.432+0000 7efc96a00640 1 -- 172.21.15.5:0/1180226905 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc9819f3f0 con 0x7efc98108540 2024-04-24T06:47:29.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.432+0000 7efc97400640 1 --2- 172.21.15.5:0/1180226905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc9810e4f0 0x7efc9819eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:47:29.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.432+0000 7efc96a00640 1 --2- 172.21.15.5:0/1180226905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc98108540 0x7efc981a4a70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7efc8800d950 tx=0x7efc8800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:47:29.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.433+0000 7efc8f400640 1 -- 172.21.15.5:0/1180226905 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc88017070 con 0x7efc98108540 2024-04-24T06:47:29.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.433+0000 7efc8f400640 1 -- 172.21.15.5:0/1180226905 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc88009c30 con 0x7efc98108540 2024-04-24T06:47:29.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.433+0000 7efc8f400640 1 -- 172.21.15.5:0/1180226905 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8800f6b0 con 0x7efc98108540 2024-04-24T06:47:29.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.433+0000 7efc97e00640 1 -- 172.21.15.5:0/1180226905 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc9819f6e0 con 0x7efc98108540 2024-04-24T06:47:29.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.433+0000 7efc97e00640 1 -- 172.21.15.5:0/1180226905 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc981ab9f0 con 0x7efc98108540 2024-04-24T06:47:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.436+0000 7efc8f400640 1 -- 172.21.15.5:0/1180226905 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7efc880109d0 con 0x7efc98108540 2024-04-24T06:47:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.436+0000 7efca0bdd740 1 Processor -- start 2024-04-24T06:47:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.436+0000 7efca0bdd740 1 -- start start 2024-04-24T06:47:29.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.437+0000 7efca0bdd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5e5f4db40 0x55a5e5f4df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.437+0000 7efca0bdd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e5f4e460 0x55a5e5f585f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.437+0000 7efca0bdd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5e5f58b30 0x55a5e5f5af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.437+0000 7efca0bdd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a5e5eb31d0 con 0x55a5e5f4e460 2024-04-24T06:47:29.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.437+0000 7efca0bdd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a5e5e59450 con 0x55a5e5f4db40 2024-04-24T06:47:29.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.437+0000 7efca0bdd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a5e5eaf190 con 0x55a5e5f58b30 2024-04-24T06:47:29.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.437+0000 7efc96a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e5f4e460 0x55a5e5f585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.437+0000 7efc97400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5e5f4db40 0x55a5e5f4df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.437+0000 7efc97400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5e5f4db40 0x55a5e5f4df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:48646/0 (socket says 172.21.15.5:48646) 2024-04-24T06:47:29.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.437+0000 7efc97400640 1 -- 172.21.15.5:0/130927501 learned_addr learned my addr 172.21.15.5:0/130927501 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:47:29.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.437+0000 7efc96000640 1 --2- 172.21.15.5:0/130927501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5e5f58b30 0x55a5e5f5af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.438+0000 7efc97400640 1 -- 172.21.15.5:0/130927501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5e5f58b30 msgr2=0x55a5e5f5af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:29.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.438+0000 7efc97400640 1 --2- 172.21.15.5:0/130927501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5e5f58b30 0x55a5e5f5af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.438+0000 7efc97400640 1 -- 172.21.15.5:0/130927501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e5f4e460 msgr2=0x55a5e5f585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:29.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.438+0000 7efc97400640 1 --2- 172.21.15.5:0/130927501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e5f4e460 0x55a5e5f585f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.438+0000 7efc97400640 1 -- 172.21.15.5:0/130927501 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5e5f16310 con 0x55a5e5f4db40 2024-04-24T06:47:29.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.438+0000 7efc8f400640 1 --2- 172.21.15.5:0/1180226905 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7efc7c0a3da0 0x7efc7c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.438+0000 7efc97400640 1 --2- 172.21.15.5:0/130927501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5e5f4db40 0x55a5e5f4df20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7efc900117e0 tx=0x7efc90011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6142be6044b7d3c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:47:29.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.438+0000 7efc96a00640 1 --2- 172.21.15.5:0/1180226905 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7efc7c0a3da0 0x7efc7c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.438+0000 7efc8f400640 1 -- 172.21.15.5:0/1180226905 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7efc880d02d0 con 0x7efc98108540 2024-04-24T06:47:29.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.438+0000 7efc83e00640 1 -- 172.21.15.5:0/130927501 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc90011310 con 0x55a5e5f4db40 2024-04-24T06:47:29.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.438+0000 7efc83e00640 1 -- 172.21.15.5:0/130927501 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc90003e50 con 0x55a5e5f4db40 2024-04-24T06:47:29.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.438+0000 7efc83e00640 1 -- 172.21.15.5:0/130927501 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc90004620 con 0x55a5e5f4db40 2024-04-24T06:47:29.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.439+0000 7efca0bdd740 1 -- 172.21.15.5:0/130927501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5e5f4db40 msgr2=0x55a5e5f4df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:29.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.439+0000 7efc96a00640 1 --2- 172.21.15.5:0/1180226905 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7efc7c0a3da0 0x7efc7c0a6260 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efc880d3fd0 tx=0x7efc880d3d80 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:47:29.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.439+0000 7efca0bdd740 1 --2- 172.21.15.5:0/130927501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5e5f4db40 0x55a5e5f4df20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7efc900117e0 tx=0x7efc90011ca0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.439+0000 7efca0bdd740 1 -- 172.21.15.5:0/130927501 shutdown_connections 2024-04-24T06:47:29.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.439+0000 7efca0bdd740 1 --2- 172.21.15.5:0/130927501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5e5f58b30 0x55a5e5f5af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.439+0000 7efca0bdd740 1 --2- 172.21.15.5:0/130927501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e5f4e460 0x55a5e5f585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.439+0000 7efca0bdd740 1 --2- 172.21.15.5:0/130927501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5e5f4db40 0x55a5e5f4df20 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.439+0000 7efca0bdd740 1 -- 172.21.15.5:0/130927501 >> 172.21.15.5:0/130927501 conn(0x55a5e5f49e60 msgr2=0x55a5e5f4a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:47:29.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.439+0000 7efca0bdd740 1 -- 172.21.15.5:0/130927501 shutdown_connections 2024-04-24T06:47:29.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.439+0000 7efca0bdd740 1 -- 172.21.15.5:0/130927501 wait complete. 2024-04-24T06:47:29.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.440+0000 7efca0bdd740 1 -- start start 2024-04-24T06:47:29.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.440+0000 7efca0bdd740 1 Processor -- start 2024-04-24T06:47:29.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.440+0000 7efca0bdd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e5f4db40 0x55a5e5f585e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.440+0000 7efca0bdd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5e5f4e460 0x55a5e5f558d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.440+0000 7efca0bdd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5e5f58b30 0x55a5e5f56010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.440+0000 7efca0bdd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a5e5f5b9c0 con 0x55a5e5f4db40 2024-04-24T06:47:29.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.440+0000 7efca0bdd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a5e5e6c810 con 0x55a5e5f58b30 2024-04-24T06:47:29.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.440+0000 7efca0bdd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a5e5f5bb40 con 0x55a5e5f4e460 2024-04-24T06:47:29.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.440+0000 7efca0bdd740 10 client.0 _ll_register_callbacks cb 0x55a5e5e3a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:47:29.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.441+0000 7efc97400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e5f4db40 0x55a5e5f585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.441+0000 7efc97400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e5f4db40 0x55a5e5f585e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41424/0 (socket says 172.21.15.5:41424) 2024-04-24T06:47:29.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.441+0000 7efc97400640 1 -- 172.21.15.5:0/2193101177 learned_addr learned my addr 172.21.15.5:0/2193101177 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:47:29.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.441+0000 7efc96000640 1 --2- 172.21.15.5:0/2193101177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5e5f4e460 0x55a5e5f558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.441+0000 7efc97400640 1 --2- 172.21.15.5:0/2193101177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5e5f58b30 0x55a5e5f56010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.441+0000 7efc97400640 1 -- 172.21.15.5:0/2193101177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5e5f4e460 msgr2=0x55a5e5f558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:29.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.441+0000 7efc97400640 1 --2- 172.21.15.5:0/2193101177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5e5f4e460 0x55a5e5f558d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.441+0000 7efc97400640 1 -- 172.21.15.5:0/2193101177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5e5f58b30 msgr2=0x55a5e5f56010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:29.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.441+0000 7efc97400640 1 --2- 172.21.15.5:0/2193101177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5e5f58b30 0x55a5e5f56010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.441+0000 7efc97400640 1 -- 172.21.15.5:0/2193101177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5e5e7d3f0 con 0x55a5e5f4db40 2024-04-24T06:47:29.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.441+0000 7efc96000640 1 --2- 172.21.15.5:0/2193101177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5e5f4e460 0x55a5e5f558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:47:29.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.441+0000 7efc97400640 1 --2- 172.21.15.5:0/2193101177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e5f4db40 0x55a5e5f585e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7efc90010010 tx=0x7efc90004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd09d0b72acf90 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:47:29.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.442+0000 7efc80c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:47:29.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.442+0000 7efc80c00640 1 -- 172.21.15.5:0/2193101177 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9002f060 con 0x55a5e5f4db40 2024-04-24T06:47:29.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.442+0000 7efc80c00640 1 -- 172.21.15.5:0/2193101177 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7efc9000fc60 con 0x55a5e5f4db40 2024-04-24T06:47:29.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.442+0000 7efca0bdd740 1 -- 172.21.15.5:0/2193101177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc90003e50 con 0x55a5e5f4db40 2024-04-24T06:47:29.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.442+0000 7efc80c00640 1 -- 172.21.15.5:0/2193101177 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efc90022040 con 0x55a5e5f4db40 2024-04-24T06:47:29.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.442+0000 7efca0bdd740 10 client.15903 fetch_fsmap learned FSMap version 25 2024-04-24T06:47:29.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.442+0000 7efca0bdd740 1 -- 172.21.15.5:0/2193101177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a5e5bc53f0 con 0x55a5e5f4db40 2024-04-24T06:47:29.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.442+0000 7efc80c00640 1 -- 172.21.15.5:0/2193101177 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc90003e50 con 0x55a5e5f4db40 2024-04-24T06:47:29.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.443+0000 7efc80c00640 1 -- 172.21.15.5:0/2193101177 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7efc90028040 con 0x55a5e5f4db40 2024-04-24T06:47:29.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.443+0000 7efca0bdd740 10 client.15903 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:47:29.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.443+0000 7efca0bdd740 10 client.15903 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-24T06:47:29.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.443+0000 7efca0bdd740 20 client.15903 populate_metadata read hostname 'smithi005' 2024-04-24T06:47:29.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.443+0000 7efca0bdd740 1 --2- 172.21.15.5:0/2193101177 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55a5e5f68cb0 0x55a5e5f6b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.443+0000 7efca0bdd740 4 client.15903 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:47:29.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.443+0000 7efca0bdd740 1 -- 172.21.15.5:0/2193101177 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5e5ec0bf0 con 0x55a5e5f68cb0 2024-04-24T06:47:29.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.443+0000 7efc96000640 1 --2- 172.21.15.5:0/2193101177 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55a5e5f68cb0 0x55a5e5f6b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.444+0000 7efc96000640 1 --2- 172.21.15.5:0/2193101177 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55a5e5f68cb0 0x55a5e5f6b0a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1e387a4222e3bf60 server_cookie=23c5b91404dd63c6 in_seq=0 out_seq=0 2024-04-24T06:47:29.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.444+0000 7efc80c00640 10 client.15903 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:47:29.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.444+0000 7efc80c00640 1 -- 172.21.15.5:0/2193101177 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a5e5ec0bf0 con 0x55a5e5f68cb0 2024-04-24T06:47:29.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.444+0000 7efc80c00640 10 client.15903 handle_command_reply: tid=0 2024-04-24T06:47:29.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.444+0000 7efca0bdd740 1 client.15903 shutdown 2024-04-24T06:47:29.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.445+0000 7efca0bdd740 1 -- 172.21.15.5:0/2193101177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e5f4db40 msgr2=0x55a5e5f585e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:29.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.445+0000 7efca0bdd740 1 --2- 172.21.15.5:0/2193101177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e5f4db40 0x55a5e5f585e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7efc90010010 tx=0x7efc90004200 comp rx=0 tx=0).stop 2024-04-24T06:47:29.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.445+0000 7efca0bdd740 1 -- 172.21.15.5:0/2193101177 shutdown_connections 2024-04-24T06:47:29.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.445+0000 7efca0bdd740 1 --2- 172.21.15.5:0/2193101177 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55a5e5f68cb0 0x55a5e5f6b0a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.445+0000 7efca0bdd740 1 --2- 172.21.15.5:0/2193101177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5e5f58b30 0x55a5e5f56010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.445+0000 7efca0bdd740 1 --2- 172.21.15.5:0/2193101177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5e5f4e460 0x55a5e5f558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.445+0000 7efca0bdd740 1 --2- 172.21.15.5:0/2193101177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e5f4db40 0x55a5e5f585e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.445+0000 7efc80c00640 0 client.15903 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:47:29.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.445+0000 7efca0bdd740 1 -- 172.21.15.5:0/2193101177 >> 172.21.15.5:0/2193101177 conn(0x55a5e5f49e60 msgr2=0x55a5e5f4fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:47:29.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.445+0000 7efca0bdd740 1 -- 172.21.15.5:0/2193101177 shutdown_connections 2024-04-24T06:47:29.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.445+0000 7efca0bdd740 1 -- 172.21.15.5:0/2193101177 wait complete. 2024-04-24T06:47:29.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.445+0000 7efca0bdd740 20 client.15903 trim_cache size 0 max 16384 2024-04-24T06:47:29.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.460+0000 7efca0bdd740 1 Processor -- start 2024-04-24T06:47:29.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.460+0000 7efca0bdd740 1 -- start start 2024-04-24T06:47:29.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.460+0000 7efca0bdd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e5f4db40 0x55a5e5f64880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.460+0000 7efca0bdd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5e5f4e460 0x55a5e5f58110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.460+0000 7efca0bdd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5e5f58b30 0x55a5e5f707e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.460+0000 7efca0bdd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efc981b6b30 con 0x55a5e5f4db40 2024-04-24T06:47:29.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.460+0000 7efca0bdd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efc981b4400 con 0x55a5e5f58b30 2024-04-24T06:47:29.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.460+0000 7efca0bdd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efc981b2250 con 0x55a5e5f4e460 2024-04-24T06:47:29.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.460+0000 7efc97400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e5f4db40 0x55a5e5f64880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.461+0000 7efc97400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e5f4db40 0x55a5e5f64880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41430/0 (socket says 172.21.15.5:41430) 2024-04-24T06:47:29.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.461+0000 7efc97400640 1 -- 172.21.15.5:0/231916170 learned_addr learned my addr 172.21.15.5:0/231916170 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:47:29.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.461+0000 7efc96000640 1 --2- 172.21.15.5:0/231916170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5e5f4e460 0x55a5e5f58110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.461+0000 7efc97400640 1 --2- 172.21.15.5:0/231916170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5e5f58b30 0x55a5e5f707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.461+0000 7efc97400640 1 -- 172.21.15.5:0/231916170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5e5f4e460 msgr2=0x55a5e5f58110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:29.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.461+0000 7efc97400640 1 --2- 172.21.15.5:0/231916170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5e5f4e460 0x55a5e5f58110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.461+0000 7efc97400640 1 -- 172.21.15.5:0/231916170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5e5f58b30 msgr2=0x55a5e5f707e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:29.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.461+0000 7efc97400640 1 --2- 172.21.15.5:0/231916170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5e5f58b30 0x55a5e5f707e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.461+0000 7efc97400640 1 -- 172.21.15.5:0/231916170 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5e5c8f7d0 con 0x55a5e5f4db40 2024-04-24T06:47:29.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.461+0000 7efc97400640 1 --2- 172.21.15.5:0/231916170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e5f4db40 0x55a5e5f64880 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7efc90005d70 tx=0x7efc90009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=186dd020c13f6740 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:47:29.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.461+0000 7efc83400640 1 -- 172.21.15.5:0/231916170 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9002f030 con 0x55a5e5f4db40 2024-04-24T06:47:29.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.461+0000 7efc83400640 1 -- 172.21.15.5:0/231916170 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc90011490 con 0x55a5e5f4db40 2024-04-24T06:47:29.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.462+0000 7efc83400640 1 -- 172.21.15.5:0/231916170 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc90005380 con 0x55a5e5f4db40 2024-04-24T06:47:29.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.462+0000 7efca0bdd740 1 -- 172.21.15.5:0/231916170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e5f4db40 msgr2=0x55a5e5f64880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:29.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.462+0000 7efca0bdd740 1 --2- 172.21.15.5:0/231916170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e5f4db40 0x55a5e5f64880 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7efc90005d70 tx=0x7efc90009480 comp rx=0 tx=0).stop 2024-04-24T06:47:29.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.462+0000 7efca0bdd740 1 -- 172.21.15.5:0/231916170 shutdown_connections 2024-04-24T06:47:29.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.462+0000 7efca0bdd740 1 --2- 172.21.15.5:0/231916170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5e5f58b30 0x55a5e5f707e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.462+0000 7efca0bdd740 1 --2- 172.21.15.5:0/231916170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5e5f4e460 0x55a5e5f58110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.462+0000 7efca0bdd740 1 --2- 172.21.15.5:0/231916170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e5f4db40 0x55a5e5f64880 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.462+0000 7efca0bdd740 1 -- 172.21.15.5:0/231916170 >> 172.21.15.5:0/231916170 conn(0x55a5e5f49e60 msgr2=0x55a5e5f54900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:47:29.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.462+0000 7efca0bdd740 1 -- 172.21.15.5:0/231916170 shutdown_connections 2024-04-24T06:47:29.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.462+0000 7efca0bdd740 1 -- 172.21.15.5:0/231916170 wait complete. 2024-04-24T06:47:29.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.463+0000 7efca0bdd740 1 -- start start 2024-04-24T06:47:29.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.463+0000 7efca0bdd740 1 Processor -- start 2024-04-24T06:47:29.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.463+0000 7efca0bdd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e5f4db40 0x55a5e5f59870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.463+0000 7efca0bdd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5e5f4e460 0x55a5e5f59db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.463+0000 7efca0bdd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5e5f58b30 0x55a5e5f5a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.463+0000 7efca0bdd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efc981b28c0 con 0x55a5e5f4db40 2024-04-24T06:47:29.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.463+0000 7efca0bdd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efc981b3d30 con 0x55a5e5f4e460 2024-04-24T06:47:29.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.463+0000 7efca0bdd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efc981b2f90 con 0x55a5e5f58b30 2024-04-24T06:47:29.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.463+0000 7efca0bdd740 10 client.0 _ll_register_callbacks cb 0x7efc880d3c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:47:29.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.464+0000 7efc97400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e5f4db40 0x55a5e5f59870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.464+0000 7efc97400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e5f4db40 0x55a5e5f59870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41446/0 (socket says 172.21.15.5:41446) 2024-04-24T06:47:29.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.464+0000 7efc97400640 1 -- 172.21.15.5:0/3863397986 learned_addr learned my addr 172.21.15.5:0/3863397986 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:47:29.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.464+0000 7efc96000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5e5f4e460 0x55a5e5f59db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.464+0000 7efc97400640 1 --2- 172.21.15.5:0/3863397986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5e5f58b30 0x55a5e5f5a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.464+0000 7efc96000640 1 -- 172.21.15.5:0/3863397986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5e5f58b30 msgr2=0x55a5e5f5a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:29.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.464+0000 7efc96000640 1 --2- 172.21.15.5:0/3863397986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5e5f58b30 0x55a5e5f5a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.464+0000 7efc96000640 1 -- 172.21.15.5:0/3863397986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e5f4db40 msgr2=0x55a5e5f59870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:29.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.464+0000 7efc96000640 1 --2- 172.21.15.5:0/3863397986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e5f4db40 0x55a5e5f59870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.464+0000 7efc96000640 1 -- 172.21.15.5:0/3863397986 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5e5f5eed0 con 0x55a5e5f4e460 2024-04-24T06:47:29.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.464+0000 7efc97400640 1 --2- 172.21.15.5:0/3863397986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e5f4db40 0x55a5e5f59870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:47:29.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.464+0000 7efc96000640 1 --2- 172.21.15.5:0/3863397986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5e5f4e460 0x55a5e5f59db0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efc8402a9f0 tx=0x7efc84004460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a79544ebd382f3d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:47:29.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.464+0000 7efc80c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:47:29.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.465+0000 7efc80c00640 1 -- 172.21.15.5:0/3863397986 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc84035040 con 0x55a5e5f4e460 2024-04-24T06:47:29.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.465+0000 7efc80c00640 1 -- 172.21.15.5:0/3863397986 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7efc84005040 con 0x55a5e5f4e460 2024-04-24T06:47:29.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.465+0000 7efca0bdd740 1 -- 172.21.15.5:0/3863397986 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a5e5ebaa60 con 0x55a5e5f4e460 2024-04-24T06:47:29.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.465+0000 7efc80c00640 1 -- 172.21.15.5:0/3863397986 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efc84005240 con 0x55a5e5f4e460 2024-04-24T06:47:29.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.465+0000 7efca0bdd740 10 client.26036 fetch_fsmap learned FSMap version 25 2024-04-24T06:47:29.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.465+0000 7efca0bdd740 1 -- 172.21.15.5:0/3863397986 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a5e5fcf310 con 0x55a5e5f4e460 2024-04-24T06:47:29.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.465+0000 7efc80c00640 1 -- 172.21.15.5:0/3863397986 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a5e5ebaa60 con 0x55a5e5f4e460 2024-04-24T06:47:29.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.466+0000 7efc80c00640 1 -- 172.21.15.5:0/3863397986 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7efc84004750 con 0x55a5e5f4e460 2024-04-24T06:47:29.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.466+0000 7efca0bdd740 10 client.26036 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:47:29.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.466+0000 7efca0bdd740 10 client.26036 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-24T06:47:29.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.466+0000 7efca0bdd740 20 client.26036 populate_metadata read hostname 'smithi005' 2024-04-24T06:47:29.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.466+0000 7efca0bdd740 1 --2- 172.21.15.5:0/3863397986 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55a5e5fcfa20 0x55a5e5fd1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.466+0000 7efca0bdd740 4 client.26036 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:47:29.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.466+0000 7efca0bdd740 1 -- 172.21.15.5:0/3863397986 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a5e5e364d0 con 0x55a5e5fcfa20 2024-04-24T06:47:29.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.466+0000 7efc97400640 1 --2- 172.21.15.5:0/3863397986 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55a5e5fcfa20 0x55a5e5fd1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.466+0000 7efc97400640 1 --2- 172.21.15.5:0/3863397986 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55a5e5fcfa20 0x55a5e5fd1e40 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4a86d18d88354e72 server_cookie=33fe9e9852c29a37 in_seq=0 out_seq=0 2024-04-24T06:47:29.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.466+0000 7efc80c00640 10 client.26036 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:47:29.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.467+0000 7efc80c00640 1 -- 172.21.15.5:0/3863397986 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a5e5e364d0 con 0x55a5e5fcfa20 2024-04-24T06:47:29.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.467+0000 7efc80c00640 10 client.26036 handle_command_reply: tid=0 2024-04-24T06:47:29.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.467+0000 7efca0bdd740 1 client.26036 shutdown 2024-04-24T06:47:29.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.467+0000 7efca0bdd740 1 -- 172.21.15.5:0/3863397986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5e5f4e460 msgr2=0x55a5e5f59db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:29.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.467+0000 7efca0bdd740 1 --2- 172.21.15.5:0/3863397986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5e5f4e460 0x55a5e5f59db0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efc8402a9f0 tx=0x7efc84004460 comp rx=0 tx=0).stop 2024-04-24T06:47:29.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.467+0000 7efca0bdd740 1 -- 172.21.15.5:0/3863397986 shutdown_connections 2024-04-24T06:47:29.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.467+0000 7efca0bdd740 1 --2- 172.21.15.5:0/3863397986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5e5f58b30 0x55a5e5f5a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.467+0000 7efca0bdd740 1 --2- 172.21.15.5:0/3863397986 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55a5e5fcfa20 0x55a5e5fd1e40 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.467+0000 7efca0bdd740 1 --2- 172.21.15.5:0/3863397986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5e5f4e460 0x55a5e5f59db0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.467+0000 7efca0bdd740 1 --2- 172.21.15.5:0/3863397986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e5f4db40 0x55a5e5f59870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.467+0000 7efc80c00640 0 client.26036 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:47:29.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.467+0000 7efca0bdd740 1 -- 172.21.15.5:0/3863397986 >> 172.21.15.5:0/3863397986 conn(0x55a5e5f49e60 msgr2=0x55a5e5f486c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:47:29.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.467+0000 7efca0bdd740 1 -- 172.21.15.5:0/3863397986 shutdown_connections 2024-04-24T06:47:29.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.467+0000 7efca0bdd740 1 -- 172.21.15.5:0/3863397986 wait complete. 2024-04-24T06:47:29.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.467+0000 7efca0bdd740 20 client.26036 trim_cache size 0 max 16384 2024-04-24T06:47:29.476 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:47:29.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.468+0000 7efc97e00640 1 -- 172.21.15.5:0/1180226905 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7efc7c0a3da0 msgr2=0x7efc7c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:29.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.468+0000 7efc97e00640 1 --2- 172.21.15.5:0/1180226905 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7efc7c0a3da0 0x7efc7c0a6260 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efc880d3fd0 tx=0x7efc880d3d80 comp rx=0 tx=0).stop 2024-04-24T06:47:29.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.468+0000 7efc97e00640 1 -- 172.21.15.5:0/1180226905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc98108540 msgr2=0x7efc981a4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:29.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.468+0000 7efc97e00640 1 --2- 172.21.15.5:0/1180226905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc98108540 0x7efc981a4a70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7efc8800d950 tx=0x7efc8800de10 comp rx=0 tx=0).stop 2024-04-24T06:47:29.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.468+0000 7efc97e00640 1 -- 172.21.15.5:0/1180226905 shutdown_connections 2024-04-24T06:47:29.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.468+0000 7efc97e00640 1 --2- 172.21.15.5:0/1180226905 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7efc7c0a3da0 0x7efc7c0a6260 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.468+0000 7efc97e00640 1 --2- 172.21.15.5:0/1180226905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc9810e4f0 0x7efc9819eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.468+0000 7efc97e00640 1 --2- 172.21.15.5:0/1180226905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc98108ee0 0x7efc981a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.468+0000 7efc97e00640 1 --2- 172.21.15.5:0/1180226905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc98108540 0x7efc981a4a70 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.468+0000 7efc97e00640 1 -- 172.21.15.5:0/1180226905 >> 172.21.15.5:0/1180226905 conn(0x7efc98077e10 msgr2=0x7efc980784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:47:29.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.468+0000 7efc97e00640 1 -- 172.21.15.5:0/1180226905 shutdown_connections 2024-04-24T06:47:29.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.468+0000 7efc97e00640 1 -- 172.21.15.5:0/1180226905 wait complete. 2024-04-24T06:47:29.485 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:29 smithi005 ceph-mon[27798]: pgmap v106: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-04-24T06:47:29.485 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:29 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/4274274332' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:47:29.489 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T06:47:29.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:29 smithi137 ceph-mon[33017]: pgmap v106: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-04-24T06:47:29.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:29 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/4274274332' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:47:29.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.707+0000 7f1093200640 1 -- 172.21.15.5:0/3519038251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1094108540 msgr2=0x7f10941089a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:29.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.707+0000 7f1093200640 1 --2- 172.21.15.5:0/3519038251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1094108540 0x7f10941089a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f107c009890 tx=0x7f107c02a400 comp rx=0 tx=0).stop 2024-04-24T06:47:29.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.707+0000 7f1093200640 1 -- 172.21.15.5:0/3519038251 shutdown_connections 2024-04-24T06:47:29.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.707+0000 7f1093200640 1 --2- 172.21.15.5:0/3519038251 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1094108ee0 0x7f10941152b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.707+0000 7f1093200640 1 --2- 172.21.15.5:0/3519038251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1094108540 0x7f10941089a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.707+0000 7f1093200640 1 --2- 172.21.15.5:0/3519038251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f109410e4f0 0x7f109410e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.707+0000 7f1093200640 1 -- 172.21.15.5:0/3519038251 >> 172.21.15.5:0/3519038251 conn(0x7f1094077e10 msgr2=0x7f1094106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:47:29.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.707+0000 7f1093200640 1 -- 172.21.15.5:0/3519038251 shutdown_connections 2024-04-24T06:47:29.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.707+0000 7f1093200640 1 -- 172.21.15.5:0/3519038251 wait complete. 2024-04-24T06:47:29.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.708+0000 7f1093200640 1 Processor -- start 2024-04-24T06:47:29.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.708+0000 7f1093200640 1 -- start start 2024-04-24T06:47:29.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.709+0000 7f1093200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1094108540 0x7f109419edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.709+0000 7f1093200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1094108ee0 0x7f109419f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.709+0000 7f1093200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f109410e4f0 0x7f10941a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.709+0000 7f1093200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f10941187c0 con 0x7f109410e4f0 2024-04-24T06:47:29.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.709+0000 7f1093200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1094118640 con 0x7f1094108540 2024-04-24T06:47:29.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.709+0000 7f1093200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1094118940 con 0x7f1094108ee0 2024-04-24T06:47:29.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.709+0000 7f1092800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f109410e4f0 0x7f10941a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.709+0000 7f1091e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1094108540 0x7f109419edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.709+0000 7f1091e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1094108540 0x7f109419edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:48694/0 (socket says 172.21.15.5:48694) 2024-04-24T06:47:29.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.709+0000 7f1091e00640 1 -- 172.21.15.5:0/4192512065 learned_addr learned my addr 172.21.15.5:0/4192512065 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:47:29.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.709+0000 7f108be00640 1 --2- 172.21.15.5:0/4192512065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1094108ee0 0x7f109419f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.710+0000 7f1091e00640 1 -- 172.21.15.5:0/4192512065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1094108ee0 msgr2=0x7f109419f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:29.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.710+0000 7f1091e00640 1 --2- 172.21.15.5:0/4192512065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1094108ee0 0x7f109419f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.710+0000 7f1091e00640 1 -- 172.21.15.5:0/4192512065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f109410e4f0 msgr2=0x7f10941a36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.710+0000 7f1091e00640 1 --2- 172.21.15.5:0/4192512065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f109410e4f0 0x7f10941a36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.710+0000 7f1091e00640 1 -- 172.21.15.5:0/4192512065 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10941a3de0 con 0x7f1094108540 2024-04-24T06:47:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.710+0000 7f1092800640 1 --2- 172.21.15.5:0/4192512065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f109410e4f0 0x7f10941a36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:47:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.710+0000 7f1091e00640 1 --2- 172.21.15.5:0/4192512065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1094108540 0x7f109419edf0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f108400e9c0 tx=0x7f108400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:47:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.710+0000 7f108aa00640 1 -- 172.21.15.5:0/4192512065 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108400ccc0 con 0x7f1094108540 2024-04-24T06:47:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.710+0000 7f1093200640 1 -- 172.21.15.5:0/4192512065 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10941a40d0 con 0x7f1094108540 2024-04-24T06:47:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.710+0000 7f108aa00640 1 -- 172.21.15.5:0/4192512065 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1084005500 con 0x7f1094108540 2024-04-24T06:47:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.710+0000 7f108aa00640 1 -- 172.21.15.5:0/4192512065 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1084010430 con 0x7f1094108540 2024-04-24T06:47:29.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.710+0000 7f1093200640 1 -- 172.21.15.5:0/4192512065 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10941ab960 con 0x7f1094108540 2024-04-24T06:47:29.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.713+0000 7f109ba2e740 1 Processor -- start 2024-04-24T06:47:29.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.713+0000 7f109ba2e740 1 -- start start 2024-04-24T06:47:29.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.714+0000 7f109ba2e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d253d1b40 0x557d253d1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.714+0000 7f109ba2e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d253d2460 0x557d253dc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.714+0000 7f109ba2e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d253dcb30 0x557d253def20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.714+0000 7f109ba2e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557d253371d0 con 0x557d253d1b40 2024-04-24T06:47:29.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.714+0000 7f109ba2e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557d252dd450 con 0x557d253d2460 2024-04-24T06:47:29.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.714+0000 7f109ba2e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557d25333190 con 0x557d253dcb30 2024-04-24T06:47:29.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.714+0000 7f108aa00640 1 -- 172.21.15.5:0/4192512065 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f10840046c0 con 0x7f1094108540 2024-04-24T06:47:29.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.714+0000 7f1092800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d253d1b40 0x557d253d1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.714+0000 7f1092800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d253d1b40 0x557d253d1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41476/0 (socket says 172.21.15.5:41476) 2024-04-24T06:47:29.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.714+0000 7f1092800640 1 -- 172.21.15.5:0/2666108514 learned_addr learned my addr 172.21.15.5:0/2666108514 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:47:29.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.714+0000 7f108be00640 1 --2- 172.21.15.5:0/2666108514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d253dcb30 0x557d253def20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.714+0000 7f1091e00640 1 --2- 172.21.15.5:0/2666108514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d253d2460 0x557d253dc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.715+0000 7f1092800640 1 -- 172.21.15.5:0/2666108514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d253dcb30 msgr2=0x557d253def20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:29.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.715+0000 7f1092800640 1 --2- 172.21.15.5:0/2666108514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d253dcb30 0x557d253def20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.715+0000 7f1092800640 1 -- 172.21.15.5:0/2666108514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d253d2460 msgr2=0x557d253dc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:29.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.715+0000 7f1092800640 1 --2- 172.21.15.5:0/2666108514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d253d2460 0x557d253dc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.715+0000 7f1092800640 1 -- 172.21.15.5:0/2666108514 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d2539a310 con 0x557d253d1b40 2024-04-24T06:47:29.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.715+0000 7f1092800640 1 --2- 172.21.15.5:0/2666108514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d253d1b40 0x557d253d1f20 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f108c0117e0 tx=0x7f108c011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d3fd08a7c2a7d5f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:47:29.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.715+0000 7f1083000640 1 -- 172.21.15.5:0/2666108514 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108c011310 con 0x557d253d1b40 2024-04-24T06:47:29.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.715+0000 7f1083000640 1 -- 172.21.15.5:0/2666108514 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f108c003e50 con 0x557d253d1b40 2024-04-24T06:47:29.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.715+0000 7f1083000640 1 -- 172.21.15.5:0/2666108514 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108c0045c0 con 0x557d253d1b40 2024-04-24T06:47:29.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.716+0000 7f108aa00640 1 --2- 172.21.15.5:0/4192512065 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f10780a3d20 0x7f10780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.716+0000 7f109ba2e740 1 -- 172.21.15.5:0/2666108514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d253d1b40 msgr2=0x557d253d1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:29.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.716+0000 7f109ba2e740 1 --2- 172.21.15.5:0/2666108514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d253d1b40 0x557d253d1f20 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f108c0117e0 tx=0x7f108c011ca0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.716+0000 7f108aa00640 1 -- 172.21.15.5:0/4192512065 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f10840d0ab0 con 0x7f1094108540 2024-04-24T06:47:29.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.716+0000 7f1091e00640 1 --2- 172.21.15.5:0/4192512065 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f10780a3d20 0x7f10780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.716+0000 7f109ba2e740 1 -- 172.21.15.5:0/2666108514 shutdown_connections 2024-04-24T06:47:29.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.716+0000 7f109ba2e740 1 --2- 172.21.15.5:0/2666108514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d253dcb30 0x557d253def20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.716+0000 7f109ba2e740 1 --2- 172.21.15.5:0/2666108514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d253d2460 0x557d253dc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.716+0000 7f109ba2e740 1 --2- 172.21.15.5:0/2666108514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d253d1b40 0x557d253d1f20 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.716+0000 7f109ba2e740 1 -- 172.21.15.5:0/2666108514 >> 172.21.15.5:0/2666108514 conn(0x557d253cde60 msgr2=0x557d253ce240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:47:29.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.716+0000 7f109ba2e740 1 -- 172.21.15.5:0/2666108514 shutdown_connections 2024-04-24T06:47:29.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.716+0000 7f109ba2e740 1 -- 172.21.15.5:0/2666108514 wait complete. 2024-04-24T06:47:29.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.717+0000 7f1091e00640 1 --2- 172.21.15.5:0/4192512065 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f10780a3d20 0x7f10780a61e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x557d253d3100 tx=0x7f10840bd040 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:47:29.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.717+0000 7f109ba2e740 1 -- start start 2024-04-24T06:47:29.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.717+0000 7f109ba2e740 1 Processor -- start 2024-04-24T06:47:29.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.718+0000 7f109ba2e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d253d1b40 0x557d253d98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.718+0000 7f109ba2e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d253d2460 0x557d253d9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.718+0000 7f109ba2e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d253dcb30 0x557d253da370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.718+0000 7f109ba2e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557d253df9c0 con 0x557d253dcb30 2024-04-24T06:47:29.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.718+0000 7f109ba2e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557d252f0810 con 0x557d253d1b40 2024-04-24T06:47:29.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.718+0000 7f109ba2e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557d253dfb40 con 0x557d253d2460 2024-04-24T06:47:29.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.718+0000 7f109ba2e740 10 client.0 _ll_register_callbacks cb 0x557d252be140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:47:29.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.718+0000 7f1092800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d253d1b40 0x557d253d98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.718+0000 7f1092800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d253d1b40 0x557d253d98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:48718/0 (socket says 172.21.15.5:48718) 2024-04-24T06:47:29.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.718+0000 7f1092800640 1 -- 172.21.15.5:0/990151656 learned_addr learned my addr 172.21.15.5:0/990151656 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:47:29.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.718+0000 7f1092800640 1 --2- 172.21.15.5:0/990151656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d253dcb30 0x557d253da370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.718+0000 7f108be00640 1 --2- 172.21.15.5:0/990151656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d253d2460 0x557d253d9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.718+0000 7f1092800640 1 -- 172.21.15.5:0/990151656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d253d2460 msgr2=0x557d253d9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:29.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.718+0000 7f1092800640 1 --2- 172.21.15.5:0/990151656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d253d2460 0x557d253d9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.718+0000 7f1092800640 1 -- 172.21.15.5:0/990151656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d253dcb30 msgr2=0x557d253da370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:29.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.718+0000 7f1092800640 1 --2- 172.21.15.5:0/990151656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d253dcb30 0x557d253da370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.718+0000 7f1092800640 1 -- 172.21.15.5:0/990151656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d253013f0 con 0x557d253d1b40 2024-04-24T06:47:29.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.719+0000 7f1092800640 1 --2- 172.21.15.5:0/990151656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d253d1b40 0x557d253d98f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f108c004300 tx=0x7f108c004200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bcb7218c43234cd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:47:29.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.719+0000 7f1077400640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:47:29.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.719+0000 7f1077400640 1 -- 172.21.15.5:0/990151656 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108c01e070 con 0x557d253d1b40 2024-04-24T06:47:29.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.719+0000 7f1077400640 1 -- 172.21.15.5:0/990151656 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f108c0043a0 con 0x557d253d1b40 2024-04-24T06:47:29.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.719+0000 7f109ba2e740 1 -- 172.21.15.5:0/990151656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f108c003e50 con 0x557d253d1b40 2024-04-24T06:47:29.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.719+0000 7f1077400640 1 -- 172.21.15.5:0/990151656 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f108c025040 con 0x557d253d1b40 2024-04-24T06:47:29.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.719+0000 7f109ba2e740 10 client.26048 fetch_fsmap learned FSMap version 25 2024-04-24T06:47:29.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.719+0000 7f109ba2e740 1 -- 172.21.15.5:0/990151656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557d250493f0 con 0x557d253d1b40 2024-04-24T06:47:29.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.720+0000 7f1077400640 1 -- 172.21.15.5:0/990151656 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f108c003e50 con 0x557d253d1b40 2024-04-24T06:47:29.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.720+0000 7f1077400640 1 -- 172.21.15.5:0/990151656 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f108c018020 con 0x557d253d1b40 2024-04-24T06:47:29.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.720+0000 7f109ba2e740 10 client.26048 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:47:29.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.720+0000 7f109ba2e740 10 client.26048 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T06:47:29.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.720+0000 7f109ba2e740 20 client.26048 populate_metadata read hostname 'smithi005' 2024-04-24T06:47:29.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.720+0000 7f109ba2e740 1 --2- 172.21.15.5:0/990151656 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x557d253ecc10 0x557d253ef000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.720+0000 7f109ba2e740 4 client.26048 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:47:29.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.720+0000 7f109ba2e740 1 -- 172.21.15.5:0/990151656 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d25344bf0 con 0x557d253ecc10 2024-04-24T06:47:29.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.721+0000 7f108be00640 1 --2- 172.21.15.5:0/990151656 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x557d253ecc10 0x557d253ef000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.721+0000 7f108be00640 1 --2- 172.21.15.5:0/990151656 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x557d253ecc10 0x557d253ef000 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=917cd992716f3ca0 server_cookie=23b880c33d39a94e in_seq=0 out_seq=0 2024-04-24T06:47:29.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.721+0000 7f1077400640 10 client.26048 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:47:29.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.722+0000 7f1077400640 1 -- 172.21.15.5:0/990151656 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557d25344bf0 con 0x557d253ecc10 2024-04-24T06:47:29.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.722+0000 7f1077400640 10 client.26048 handle_command_reply: tid=0 2024-04-24T06:47:29.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.722+0000 7f109ba2e740 1 client.26048 shutdown 2024-04-24T06:47:29.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.723+0000 7f109ba2e740 1 -- 172.21.15.5:0/990151656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d253d1b40 msgr2=0x557d253d98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:29.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.723+0000 7f109ba2e740 1 --2- 172.21.15.5:0/990151656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d253d1b40 0x557d253d98f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f108c004300 tx=0x7f108c004200 comp rx=0 tx=0).stop 2024-04-24T06:47:29.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.723+0000 7f109ba2e740 1 -- 172.21.15.5:0/990151656 shutdown_connections 2024-04-24T06:47:29.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.723+0000 7f109ba2e740 1 --2- 172.21.15.5:0/990151656 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x557d253ecc10 0x557d253ef000 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.723+0000 7f109ba2e740 1 --2- 172.21.15.5:0/990151656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d253dcb30 0x557d253da370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.723+0000 7f109ba2e740 1 --2- 172.21.15.5:0/990151656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d253d2460 0x557d253d9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.723+0000 7f109ba2e740 1 --2- 172.21.15.5:0/990151656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d253d1b40 0x557d253d98f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.723+0000 7f109ba2e740 1 -- 172.21.15.5:0/990151656 >> 172.21.15.5:0/990151656 conn(0x557d253cde60 msgr2=0x557d253d3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:47:29.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.723+0000 7f1077400640 0 client.26048 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:47:29.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.723+0000 7f109ba2e740 1 -- 172.21.15.5:0/990151656 shutdown_connections 2024-04-24T06:47:29.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.723+0000 7f109ba2e740 1 -- 172.21.15.5:0/990151656 wait complete. 2024-04-24T06:47:29.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.723+0000 7f109ba2e740 20 client.26048 trim_cache size 0 max 16384 2024-04-24T06:47:29.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.737+0000 7f109ba2e740 1 Processor -- start 2024-04-24T06:47:29.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.738+0000 7f109ba2e740 1 -- start start 2024-04-24T06:47:29.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.738+0000 7f109ba2e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d253d1b40 0x557d253e8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.738+0000 7f109ba2e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d253d2460 0x557d253d3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.738+0000 7f109ba2e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d253dcb30 0x557d253d90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.738+0000 7f109ba2e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f10941b6b20 con 0x557d253d1b40 2024-04-24T06:47:29.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.738+0000 7f109ba2e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f10941b43f0 con 0x557d253d2460 2024-04-24T06:47:29.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.738+0000 7f109ba2e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f10941b2240 con 0x557d253dcb30 2024-04-24T06:47:29.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.738+0000 7f1092800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d253d1b40 0x557d253e8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.738+0000 7f108be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d253d2460 0x557d253d3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.738+0000 7f108be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d253d2460 0x557d253d3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:48732/0 (socket says 172.21.15.5:48732) 2024-04-24T06:47:29.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.738+0000 7f1092800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d253d1b40 0x557d253e8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41500/0 (socket says 172.21.15.5:41500) 2024-04-24T06:47:29.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.738+0000 7f108be00640 1 -- 172.21.15.5:0/495975351 learned_addr learned my addr 172.21.15.5:0/495975351 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:47:29.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.739+0000 7f1092800640 1 --2- 172.21.15.5:0/495975351 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d253dcb30 0x557d253d90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.739+0000 7f108be00640 1 -- 172.21.15.5:0/495975351 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d253dcb30 msgr2=0x557d253d90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:29.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.739+0000 7f108be00640 1 --2- 172.21.15.5:0/495975351 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d253dcb30 0x557d253d90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.739+0000 7f108be00640 1 -- 172.21.15.5:0/495975351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d253d1b40 msgr2=0x557d253e8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.739+0000 7f108be00640 1 --2- 172.21.15.5:0/495975351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d253d1b40 0x557d253e8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.739+0000 7f108be00640 1 -- 172.21.15.5:0/495975351 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d251137d0 con 0x557d253d2460 2024-04-24T06:47:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.739+0000 7f1092800640 1 --2- 172.21.15.5:0/495975351 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d253dcb30 0x557d253d90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:47:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.739+0000 7f108be00640 1 --2- 172.21.15.5:0/495975351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d253d2460 0x557d253d3350 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f107c0095f0 tx=0x7f107c005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc32d34828d29ed1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:47:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.739+0000 7f1082600640 1 -- 172.21.15.5:0/495975351 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107c048020 con 0x557d253d2460 2024-04-24T06:47:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.739+0000 7f1082600640 1 -- 172.21.15.5:0/495975351 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f107c004390 con 0x557d253d2460 2024-04-24T06:47:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.739+0000 7f1082600640 1 -- 172.21.15.5:0/495975351 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107c037040 con 0x557d253d2460 2024-04-24T06:47:29.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.740+0000 7f109ba2e740 1 -- 172.21.15.5:0/495975351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d253d2460 msgr2=0x557d253d3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:29.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.740+0000 7f109ba2e740 1 --2- 172.21.15.5:0/495975351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d253d2460 0x557d253d3350 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f107c0095f0 tx=0x7f107c005f10 comp rx=0 tx=0).stop 2024-04-24T06:47:29.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.740+0000 7f109ba2e740 1 -- 172.21.15.5:0/495975351 shutdown_connections 2024-04-24T06:47:29.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.740+0000 7f109ba2e740 1 --2- 172.21.15.5:0/495975351 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d253dcb30 0x557d253d90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.740+0000 7f109ba2e740 1 --2- 172.21.15.5:0/495975351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d253d2460 0x557d253d3350 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.740+0000 7f109ba2e740 1 --2- 172.21.15.5:0/495975351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d253d1b40 0x557d253e8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.740+0000 7f109ba2e740 1 -- 172.21.15.5:0/495975351 >> 172.21.15.5:0/495975351 conn(0x557d253cde60 msgr2=0x557d253da140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:47:29.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.740+0000 7f109ba2e740 1 -- 172.21.15.5:0/495975351 shutdown_connections 2024-04-24T06:47:29.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.740+0000 7f109ba2e740 1 -- 172.21.15.5:0/495975351 wait complete. 2024-04-24T06:47:29.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.740+0000 7f109ba2e740 1 -- start start 2024-04-24T06:47:29.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.741+0000 7f109ba2e740 1 Processor -- start 2024-04-24T06:47:29.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.741+0000 7f109ba2e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d253d1b40 0x557d253dd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.741+0000 7f109ba2e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d253d2460 0x557d253ddef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.741+0000 7f109ba2e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d253dcb30 0x557d253de430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.741+0000 7f109ba2e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f10941b28b0 con 0x557d253d2460 2024-04-24T06:47:29.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.741+0000 7f109ba2e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f10941b3d20 con 0x557d253d1b40 2024-04-24T06:47:29.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.741+0000 7f109ba2e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f10941b2f80 con 0x557d253dcb30 2024-04-24T06:47:29.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.741+0000 7f109ba2e740 10 client.0 _ll_register_callbacks cb 0x7f1084005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:47:29.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.741+0000 7f108be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d253d2460 0x557d253ddef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.741+0000 7f108be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d253d2460 0x557d253ddef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41508/0 (socket says 172.21.15.5:41508) 2024-04-24T06:47:29.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.741+0000 7f1092800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d253d1b40 0x557d253dd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.741+0000 7f108be00640 1 -- 172.21.15.5:0/199075603 learned_addr learned my addr 172.21.15.5:0/199075603 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:47:29.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.742+0000 7f1092800640 1 --2- 172.21.15.5:0/199075603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d253dcb30 0x557d253de430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.742+0000 7f108be00640 1 -- 172.21.15.5:0/199075603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d253dcb30 msgr2=0x557d253de430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:29.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.742+0000 7f108be00640 1 --2- 172.21.15.5:0/199075603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d253dcb30 0x557d253de430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.742+0000 7f108be00640 1 -- 172.21.15.5:0/199075603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d253d1b40 msgr2=0x557d253dd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:29.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.742+0000 7f108be00640 1 --2- 172.21.15.5:0/199075603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d253d1b40 0x557d253dd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.742+0000 7f108be00640 1 -- 172.21.15.5:0/199075603 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d253e3030 con 0x557d253d2460 2024-04-24T06:47:29.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.742+0000 7f1092800640 1 --2- 172.21.15.5:0/199075603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d253d1b40 0x557d253dd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:47:29.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.742+0000 7f108be00640 1 --2- 172.21.15.5:0/199075603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d253d2460 0x557d253ddef0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f107c0057c0 tx=0x7f107c003630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adacfc80b9397dfb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:47:29.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.742+0000 7f1077400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:47:29.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.742+0000 7f1077400640 1 -- 172.21.15.5:0/199075603 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107c048050 con 0x557d253d2460 2024-04-24T06:47:29.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.742+0000 7f1077400640 1 -- 172.21.15.5:0/199075603 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f107c0039c0 con 0x557d253d2460 2024-04-24T06:47:29.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.742+0000 7f109ba2e740 1 -- 172.21.15.5:0/199075603 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f107c004390 con 0x557d253d2460 2024-04-24T06:47:29.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.743+0000 7f1077400640 1 -- 172.21.15.5:0/199075603 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f107c037040 con 0x557d253d2460 2024-04-24T06:47:29.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.743+0000 7f109ba2e740 10 client.15927 fetch_fsmap learned FSMap version 25 2024-04-24T06:47:29.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.743+0000 7f109ba2e740 1 -- 172.21.15.5:0/199075603 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557d25453280 con 0x557d253d2460 2024-04-24T06:47:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.743+0000 7f1077400640 1 -- 172.21.15.5:0/199075603 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f107c004390 con 0x557d253d2460 2024-04-24T06:47:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.743+0000 7f1077400640 1 -- 172.21.15.5:0/199075603 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f107c054030 con 0x557d253d2460 2024-04-24T06:47:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.743+0000 7f109ba2e740 10 client.15927 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:47:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.743+0000 7f109ba2e740 10 client.15927 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T06:47:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.743+0000 7f109ba2e740 20 client.15927 populate_metadata read hostname 'smithi005' 2024-04-24T06:47:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.743+0000 7f109ba2e740 1 --2- 172.21.15.5:0/199075603 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x557d254539f0 0x557d25455e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.743+0000 7f109ba2e740 4 client.15927 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:47:29.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.743+0000 7f109ba2e740 1 -- 172.21.15.5:0/199075603 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557d2533ea60 con 0x557d254539f0 2024-04-24T06:47:29.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.744+0000 7f1092800640 1 --2- 172.21.15.5:0/199075603 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x557d254539f0 0x557d25455e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.744+0000 7f1092800640 1 --2- 172.21.15.5:0/199075603 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x557d254539f0 0x557d25455e10 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b64ab4406bd7e452 server_cookie=650278b8767efeb0 in_seq=0 out_seq=0 2024-04-24T06:47:29.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.744+0000 7f1077400640 10 client.15927 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:47:29.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.744+0000 7f1077400640 1 -- 172.21.15.5:0/199075603 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557d2533ea60 con 0x557d254539f0 2024-04-24T06:47:29.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.744+0000 7f1077400640 10 client.15927 handle_command_reply: tid=0 2024-04-24T06:47:29.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.744+0000 7f109ba2e740 1 client.15927 shutdown 2024-04-24T06:47:29.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.745+0000 7f109ba2e740 1 -- 172.21.15.5:0/199075603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d253d2460 msgr2=0x557d253ddef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:29.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.745+0000 7f109ba2e740 1 --2- 172.21.15.5:0/199075603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d253d2460 0x557d253ddef0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f107c0057c0 tx=0x7f107c003630 comp rx=0 tx=0).stop 2024-04-24T06:47:29.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.745+0000 7f109ba2e740 1 -- 172.21.15.5:0/199075603 shutdown_connections 2024-04-24T06:47:29.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.745+0000 7f109ba2e740 1 --2- 172.21.15.5:0/199075603 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x557d254539f0 0x557d25455e10 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.745+0000 7f109ba2e740 1 --2- 172.21.15.5:0/199075603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d253dcb30 0x557d253de430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.745+0000 7f109ba2e740 1 --2- 172.21.15.5:0/199075603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d253d2460 0x557d253ddef0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.745+0000 7f109ba2e740 1 --2- 172.21.15.5:0/199075603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557d253d1b40 0x557d253dd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.745+0000 7f1077400640 0 client.15927 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:47:29.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.745+0000 7f109ba2e740 1 -- 172.21.15.5:0/199075603 >> 172.21.15.5:0/199075603 conn(0x557d253cde60 msgr2=0x557d253d98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:47:29.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.745+0000 7f109ba2e740 1 -- 172.21.15.5:0/199075603 shutdown_connections 2024-04-24T06:47:29.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.745+0000 7f109ba2e740 1 -- 172.21.15.5:0/199075603 wait complete. 2024-04-24T06:47:29.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.745+0000 7f109ba2e740 20 client.15927 trim_cache size 0 max 16384 2024-04-24T06:47:29.755 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:47:29.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.746+0000 7f1093200640 1 -- 172.21.15.5:0/4192512065 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f10780a3d20 msgr2=0x7f10780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:29.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.746+0000 7f1093200640 1 --2- 172.21.15.5:0/4192512065 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f10780a3d20 0x7f10780a61e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x557d253d3100 tx=0x7f10840bd040 comp rx=0 tx=0).stop 2024-04-24T06:47:29.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.746+0000 7f1093200640 1 -- 172.21.15.5:0/4192512065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1094108540 msgr2=0x7f109419edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:29.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.746+0000 7f1093200640 1 --2- 172.21.15.5:0/4192512065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1094108540 0x7f109419edf0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f108400e9c0 tx=0x7f108400ee80 comp rx=0 tx=0).stop 2024-04-24T06:47:29.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.746+0000 7f1093200640 1 -- 172.21.15.5:0/4192512065 shutdown_connections 2024-04-24T06:47:29.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.746+0000 7f1093200640 1 --2- 172.21.15.5:0/4192512065 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f10780a3d20 0x7f10780a61e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.746+0000 7f1093200640 1 --2- 172.21.15.5:0/4192512065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f109410e4f0 0x7f10941a36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.746+0000 7f1093200640 1 --2- 172.21.15.5:0/4192512065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1094108ee0 0x7f109419f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.746+0000 7f1093200640 1 --2- 172.21.15.5:0/4192512065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1094108540 0x7f109419edf0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.746+0000 7f1093200640 1 -- 172.21.15.5:0/4192512065 >> 172.21.15.5:0/4192512065 conn(0x7f1094077e10 msgr2=0x7f1094078530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:47:29.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.746+0000 7f1093200640 1 -- 172.21.15.5:0/4192512065 shutdown_connections 2024-04-24T06:47:29.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.746+0000 7f1093200640 1 -- 172.21.15.5:0/4192512065 wait complete. 2024-04-24T06:47:29.767 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T06:47:29.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:29 smithi037 ceph-mon[32864]: pgmap v106: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-04-24T06:47:29.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:29 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/4274274332' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:47:29.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.973+0000 7f322b200640 1 -- 172.21.15.5:0/137951396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f322c102070 msgr2=0x7f322c102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:29.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.973+0000 7f322b200640 1 --2- 172.21.15.5:0/137951396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f322c102070 0x7f322c102450 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f321c009900 tx=0x7f321c02a3f0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.973+0000 7f322b200640 1 -- 172.21.15.5:0/137951396 shutdown_connections 2024-04-24T06:47:29.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.973+0000 7f322b200640 1 --2- 172.21.15.5:0/137951396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f322c107140 0x7f322c10ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.973+0000 7f322b200640 1 --2- 172.21.15.5:0/137951396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f322c102a20 0x7f322c106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.973+0000 7f322b200640 1 --2- 172.21.15.5:0/137951396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f322c102070 0x7f322c102450 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.973+0000 7f322b200640 1 -- 172.21.15.5:0/137951396 >> 172.21.15.5:0/137951396 conn(0x7f322c0fd950 msgr2=0x7f322c0ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:47:29.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.973+0000 7f322b200640 1 -- 172.21.15.5:0/137951396 shutdown_connections 2024-04-24T06:47:29.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.973+0000 7f322b200640 1 -- 172.21.15.5:0/137951396 wait complete. 2024-04-24T06:47:29.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.974+0000 7f322b200640 1 Processor -- start 2024-04-24T06:47:29.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.975+0000 7f322b200640 1 -- start start 2024-04-24T06:47:29.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.975+0000 7f322b200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f322c102070 0x7f322c1a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.975+0000 7f322b200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f322c107140 0x7f322c1a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.975+0000 7f322b200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f322c19a7b0 0x7f322c19ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.975+0000 7f322b200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f322c112250 con 0x7f322c102070 2024-04-24T06:47:29.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.975+0000 7f322b200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f322c1120d0 con 0x7f322c107140 2024-04-24T06:47:29.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.975+0000 7f322b200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f322c1123d0 con 0x7f322c19a7b0 2024-04-24T06:47:29.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.975+0000 7f3229e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f322c102070 0x7f322c1a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.975+0000 7f3229400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f322c107140 0x7f322c1a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.975+0000 7f3229e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f322c102070 0x7f322c1a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41518/0 (socket says 172.21.15.5:41518) 2024-04-24T06:47:29.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.975+0000 7f3229e00640 1 -- 172.21.15.5:0/952108586 learned_addr learned my addr 172.21.15.5:0/952108586 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:47:29.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.976+0000 7f322a800640 1 --2- 172.21.15.5:0/952108586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f322c19a7b0 0x7f322c19ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.976+0000 7f3229400640 1 -- 172.21.15.5:0/952108586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f322c19a7b0 msgr2=0x7f322c19ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:29.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.976+0000 7f3229400640 1 --2- 172.21.15.5:0/952108586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f322c19a7b0 0x7f322c19ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.976+0000 7f3229400640 1 -- 172.21.15.5:0/952108586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f322c102070 msgr2=0x7f322c1a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:29.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.976+0000 7f3229400640 1 --2- 172.21.15.5:0/952108586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f322c102070 0x7f322c1a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.976+0000 7f3229400640 1 -- 172.21.15.5:0/952108586 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f322c19b3b0 con 0x7f322c107140 2024-04-24T06:47:29.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.976+0000 7f322a800640 1 --2- 172.21.15.5:0/952108586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f322c19a7b0 0x7f322c19ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:47:29.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.976+0000 7f3229400640 1 --2- 172.21.15.5:0/952108586 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f322c107140 0x7f322c1a0be0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3214005fd0 tx=0x7f321400ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:47:29.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.976+0000 7f3222a00640 1 -- 172.21.15.5:0/952108586 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3214010430 con 0x7f322c107140 2024-04-24T06:47:29.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.976+0000 7f322b200640 1 -- 172.21.15.5:0/952108586 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f322c1a7440 con 0x7f322c107140 2024-04-24T06:47:29.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.976+0000 7f3222a00640 1 -- 172.21.15.5:0/952108586 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3214010ab0 con 0x7f322c107140 2024-04-24T06:47:29.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.976+0000 7f3222a00640 1 -- 172.21.15.5:0/952108586 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3214017b70 con 0x7f322c107140 2024-04-24T06:47:29.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.977+0000 7f322b200640 1 -- 172.21.15.5:0/952108586 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f322c1a7850 con 0x7f322c107140 2024-04-24T06:47:29.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.980+0000 7f32333fd740 1 Processor -- start 2024-04-24T06:47:29.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.980+0000 7f32333fd740 1 -- start start 2024-04-24T06:47:29.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.980+0000 7f3222a00640 1 -- 172.21.15.5:0/952108586 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f3214017d10 con 0x7f322c107140 2024-04-24T06:47:29.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.980+0000 7f32333fd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557437f1eb40 0x557437f1ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.980+0000 7f32333fd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557437f1f460 0x557437f295f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.980+0000 7f32333fd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557437f29b30 0x557437f2bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.980+0000 7f32333fd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557437e841d0 con 0x557437f29b30 2024-04-24T06:47:29.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.980+0000 7f32333fd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557437e2a450 con 0x557437f1f460 2024-04-24T06:47:29.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.980+0000 7f32333fd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557437e80190 con 0x557437f1eb40 2024-04-24T06:47:29.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.980+0000 7f3229400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557437f29b30 0x557437f2bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.980+0000 7f3229e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557437f1f460 0x557437f295f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.981+0000 7f3229e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557437f1f460 0x557437f295f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:48768/0 (socket says 172.21.15.5:48768) 2024-04-24T06:47:29.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.981+0000 7f3229e00640 1 -- 172.21.15.5:0/2193914153 learned_addr learned my addr 172.21.15.5:0/2193914153 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:47:29.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.981+0000 7f322a800640 1 --2- 172.21.15.5:0/2193914153 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557437f1eb40 0x557437f1ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.981+0000 7f3229e00640 1 -- 172.21.15.5:0/2193914153 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557437f1eb40 msgr2=0x557437f1ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:29.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.981+0000 7f3229e00640 1 --2- 172.21.15.5:0/2193914153 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557437f1eb40 0x557437f1ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.981+0000 7f3229e00640 1 -- 172.21.15.5:0/2193914153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557437f29b30 msgr2=0x557437f2bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:29.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.981+0000 7f3229e00640 1 --2- 172.21.15.5:0/2193914153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557437f29b30 0x557437f2bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.981+0000 7f3229e00640 1 -- 172.21.15.5:0/2193914153 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557437ee7310 con 0x557437f1f460 2024-04-24T06:47:29.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.981+0000 7f322a800640 1 --2- 172.21.15.5:0/2193914153 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557437f1eb40 0x557437f1ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:47:29.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.981+0000 7f3229e00640 1 --2- 172.21.15.5:0/2193914153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557437f1f460 0x557437f295f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f321c005e80 tx=0x7f321c005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50822380ddca9f78 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:47:29.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.982+0000 7f321b000640 1 -- 172.21.15.5:0/2193914153 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c035040 con 0x557437f1f460 2024-04-24T06:47:29.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.982+0000 7f321b000640 1 -- 172.21.15.5:0/2193914153 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f321c0044b0 con 0x557437f1f460 2024-04-24T06:47:29.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.982+0000 7f321b000640 1 -- 172.21.15.5:0/2193914153 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c003850 con 0x557437f1f460 2024-04-24T06:47:29.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.982+0000 7f3222a00640 1 --2- 172.21.15.5:0/952108586 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f320c0a3d20 0x7f320c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.982+0000 7f3222a00640 1 -- 172.21.15.5:0/952108586 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f32140d11b0 con 0x7f322c107140 2024-04-24T06:47:29.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.982+0000 7f322a800640 1 --2- 172.21.15.5:0/952108586 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f320c0a3d20 0x7f320c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.982+0000 7f32333fd740 1 -- 172.21.15.5:0/2193914153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557437f1f460 msgr2=0x557437f295f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:29.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.982+0000 7f32333fd740 1 --2- 172.21.15.5:0/2193914153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557437f1f460 0x557437f295f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f321c005e80 tx=0x7f321c005c00 comp rx=0 tx=0).stop 2024-04-24T06:47:29.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.982+0000 7f32333fd740 1 -- 172.21.15.5:0/2193914153 shutdown_connections 2024-04-24T06:47:29.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.982+0000 7f32333fd740 1 --2- 172.21.15.5:0/2193914153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557437f29b30 0x557437f2bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.982+0000 7f32333fd740 1 --2- 172.21.15.5:0/2193914153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557437f1f460 0x557437f295f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.982+0000 7f32333fd740 1 --2- 172.21.15.5:0/2193914153 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557437f1eb40 0x557437f1ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.982+0000 7f32333fd740 1 -- 172.21.15.5:0/2193914153 >> 172.21.15.5:0/2193914153 conn(0x557437f1ae60 msgr2=0x557437f1b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:47:29.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.983+0000 7f32333fd740 1 -- 172.21.15.5:0/2193914153 shutdown_connections 2024-04-24T06:47:29.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.983+0000 7f32333fd740 1 -- 172.21.15.5:0/2193914153 wait complete. 2024-04-24T06:47:29.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.983+0000 7f322a800640 1 --2- 172.21.15.5:0/952108586 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f320c0a3d20 0x7f320c0a61e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3224005fd0 tx=0x7f3224005cf0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:47:29.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.983+0000 7f32333fd740 1 -- start start 2024-04-24T06:47:29.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.983+0000 7f32333fd740 1 Processor -- start 2024-04-24T06:47:29.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.984+0000 7f32333fd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557437f1eb40 0x557437f29530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.984+0000 7f32333fd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557437f1f460 0x557437f26820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.984+0000 7f32333fd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557437f29b30 0x557437f26e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.984+0000 7f32333fd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557437f2c960 con 0x557437f29b30 2024-04-24T06:47:29.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.984+0000 7f32333fd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557437e3d810 con 0x557437f1eb40 2024-04-24T06:47:29.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.984+0000 7f32333fd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557437f2cae0 con 0x557437f1f460 2024-04-24T06:47:29.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.984+0000 7f32333fd740 10 client.0 _ll_register_callbacks cb 0x557437e0b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:47:29.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.984+0000 7f3229e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557437f1eb40 0x557437f29530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.984+0000 7f322a800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557437f29b30 0x557437f26e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.984+0000 7f3229e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557437f1eb40 0x557437f29530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:48784/0 (socket says 172.21.15.5:48784) 2024-04-24T06:47:29.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.984+0000 7f3229e00640 1 -- 172.21.15.5:0/239935322 learned_addr learned my addr 172.21.15.5:0/239935322 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:47:29.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.984+0000 7f3229400640 1 --2- 172.21.15.5:0/239935322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557437f1f460 0x557437f26820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.985+0000 7f3229e00640 1 -- 172.21.15.5:0/239935322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557437f1f460 msgr2=0x557437f26820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:29.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.985+0000 7f3229e00640 1 --2- 172.21.15.5:0/239935322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557437f1f460 0x557437f26820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.985+0000 7f3229e00640 1 -- 172.21.15.5:0/239935322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557437f29b30 msgr2=0x557437f26e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:29.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.985+0000 7f3229e00640 1 --2- 172.21.15.5:0/239935322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557437f29b30 0x557437f26e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.985+0000 7f3229e00640 1 -- 172.21.15.5:0/239935322 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557437e4e3f0 con 0x557437f1eb40 2024-04-24T06:47:29.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.985+0000 7f3229e00640 1 --2- 172.21.15.5:0/239935322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557437f1eb40 0x557437f29530 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f321c02a430 tx=0x7f321c005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f29c13ca7f422307 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:47:29.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.985+0000 7f3213400640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:47:29.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.985+0000 7f3213400640 1 -- 172.21.15.5:0/239935322 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c053060 con 0x557437f1eb40 2024-04-24T06:47:29.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.985+0000 7f3213400640 1 -- 172.21.15.5:0/239935322 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f321c0042b0 con 0x557437f1eb40 2024-04-24T06:47:29.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.985+0000 7f32333fd740 1 -- 172.21.15.5:0/239935322 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f321c0044b0 con 0x557437f1eb40 2024-04-24T06:47:29.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.986+0000 7f3213400640 1 -- 172.21.15.5:0/239935322 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f321c035050 con 0x557437f1eb40 2024-04-24T06:47:29.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.986+0000 7f32333fd740 10 client.26066 fetch_fsmap learned FSMap version 25 2024-04-24T06:47:29.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.986+0000 7f32333fd740 1 -- 172.21.15.5:0/239935322 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557437b963f0 con 0x557437f1eb40 2024-04-24T06:47:29.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.986+0000 7f3213400640 1 -- 172.21.15.5:0/239935322 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f321c0044b0 con 0x557437f1eb40 2024-04-24T06:47:29.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.986+0000 7f3213400640 1 -- 172.21.15.5:0/239935322 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f321c0039e0 con 0x557437f1eb40 2024-04-24T06:47:29.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.986+0000 7f32333fd740 10 client.26066 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:47:29.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.986+0000 7f32333fd740 10 client.26066 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:47:29.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.986+0000 7f32333fd740 20 client.26066 populate_metadata read hostname 'smithi005' 2024-04-24T06:47:29.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.986+0000 7f32333fd740 1 --2- 172.21.15.5:0/239935322 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x557437f39ad0 0x557437f3bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:29.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.986+0000 7f32333fd740 4 client.26066 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:47:29.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.986+0000 7f32333fd740 1 -- 172.21.15.5:0/239935322 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557437e91bf0 con 0x557437f39ad0 2024-04-24T06:47:29.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.987+0000 7f3229400640 1 --2- 172.21.15.5:0/239935322 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x557437f39ad0 0x557437f3bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:29.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.987+0000 7f3229400640 1 --2- 172.21.15.5:0/239935322 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x557437f39ad0 0x557437f3bec0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28369e4212c631a0 server_cookie=36c5dec89633baf0 in_seq=0 out_seq=0 2024-04-24T06:47:29.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.987+0000 7f3213400640 10 client.26066 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:47:29.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.988+0000 7f3213400640 1 -- 172.21.15.5:0/239935322 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557437e91bf0 con 0x557437f39ad0 2024-04-24T06:47:29.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.988+0000 7f3213400640 10 client.26066 handle_command_reply: tid=0 2024-04-24T06:47:29.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.988+0000 7f32333fd740 1 client.26066 shutdown 2024-04-24T06:47:29.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.988+0000 7f32333fd740 1 -- 172.21.15.5:0/239935322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557437f1eb40 msgr2=0x557437f29530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:29.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.988+0000 7f32333fd740 1 --2- 172.21.15.5:0/239935322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557437f1eb40 0x557437f29530 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f321c02a430 tx=0x7f321c005850 comp rx=0 tx=0).stop 2024-04-24T06:47:29.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.988+0000 7f32333fd740 1 -- 172.21.15.5:0/239935322 shutdown_connections 2024-04-24T06:47:29.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.988+0000 7f32333fd740 1 --2- 172.21.15.5:0/239935322 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x557437f39ad0 0x557437f3bec0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.988+0000 7f32333fd740 1 --2- 172.21.15.5:0/239935322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557437f29b30 0x557437f26e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.988+0000 7f32333fd740 1 --2- 172.21.15.5:0/239935322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557437f1f460 0x557437f26820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.988+0000 7f32333fd740 1 --2- 172.21.15.5:0/239935322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557437f1eb40 0x557437f29530 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:29.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.988+0000 7f3213400640 0 client.26066 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:47:29.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.988+0000 7f32333fd740 1 -- 172.21.15.5:0/239935322 >> 172.21.15.5:0/239935322 conn(0x557437f1ae60 msgr2=0x557437f20c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:47:29.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.988+0000 7f32333fd740 1 -- 172.21.15.5:0/239935322 shutdown_connections 2024-04-24T06:47:29.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.989+0000 7f32333fd740 1 -- 172.21.15.5:0/239935322 wait complete. 2024-04-24T06:47:29.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:29.989+0000 7f32333fd740 20 client.26066 trim_cache size 0 max 16384 2024-04-24T06:47:30.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.003+0000 7f32333fd740 1 Processor -- start 2024-04-24T06:47:30.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.003+0000 7f32333fd740 1 -- start start 2024-04-24T06:47:30.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.004+0000 7f32333fd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557437f1eb40 0x557437f35820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:30.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.004+0000 7f32333fd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557437f1f460 0x557437f29060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:30.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.004+0000 7f32333fd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557437f29b30 0x557437f41520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:30.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.004+0000 7f32333fd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f322c1b2560 con 0x557437f1eb40 2024-04-24T06:47:30.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.004+0000 7f32333fd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f322c1afe30 con 0x557437f1f460 2024-04-24T06:47:30.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.004+0000 7f32333fd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f322c1adc80 con 0x557437f29b30 2024-04-24T06:47:30.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.004+0000 7f3229e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557437f1eb40 0x557437f35820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:30.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.004+0000 7f322a800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557437f29b30 0x557437f41520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:30.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.004+0000 7f3229400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557437f1f460 0x557437f29060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:30.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.004+0000 7f322a800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557437f29b30 0x557437f41520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:53056/0 (socket says 172.21.15.5:53056) 2024-04-24T06:47:30.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.004+0000 7f322a800640 1 -- 172.21.15.5:0/3286023056 learned_addr learned my addr 172.21.15.5:0/3286023056 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:47:30.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.005+0000 7f322a800640 1 -- 172.21.15.5:0/3286023056 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557437f1f460 msgr2=0x557437f29060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:30.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.005+0000 7f322a800640 1 --2- 172.21.15.5:0/3286023056 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557437f1f460 0x557437f29060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:30.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.005+0000 7f322a800640 1 -- 172.21.15.5:0/3286023056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557437f1eb40 msgr2=0x557437f35820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:30.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.005+0000 7f322a800640 1 --2- 172.21.15.5:0/3286023056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557437f1eb40 0x557437f35820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:30.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.005+0000 7f322a800640 1 -- 172.21.15.5:0/3286023056 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557437f1f290 con 0x557437f29b30 2024-04-24T06:47:30.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.005+0000 7f322a800640 1 --2- 172.21.15.5:0/3286023056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557437f29b30 0x557437f41520 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f32240127d0 tx=0x7f3224014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33ac9edbc53137e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:47:30.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.005+0000 7f321a600640 1 -- 172.21.15.5:0/3286023056 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32240209b0 con 0x557437f29b30 2024-04-24T06:47:30.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.006+0000 7f321a600640 1 -- 172.21.15.5:0/3286023056 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3224005500 con 0x557437f29b30 2024-04-24T06:47:30.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.006+0000 7f321a600640 1 -- 172.21.15.5:0/3286023056 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3224004ce0 con 0x557437f29b30 2024-04-24T06:47:30.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.006+0000 7f32333fd740 1 -- 172.21.15.5:0/3286023056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557437f29b30 msgr2=0x557437f41520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:30.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.006+0000 7f32333fd740 1 --2- 172.21.15.5:0/3286023056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557437f29b30 0x557437f41520 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f32240127d0 tx=0x7f3224014000 comp rx=0 tx=0).stop 2024-04-24T06:47:30.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.006+0000 7f32333fd740 1 -- 172.21.15.5:0/3286023056 shutdown_connections 2024-04-24T06:47:30.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.006+0000 7f32333fd740 1 --2- 172.21.15.5:0/3286023056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557437f29b30 0x557437f41520 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:30.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.006+0000 7f32333fd740 1 --2- 172.21.15.5:0/3286023056 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557437f1f460 0x557437f29060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:30.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.006+0000 7f32333fd740 1 --2- 172.21.15.5:0/3286023056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557437f1eb40 0x557437f35820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:30.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.006+0000 7f32333fd740 1 -- 172.21.15.5:0/3286023056 >> 172.21.15.5:0/3286023056 conn(0x557437f1ae60 msgr2=0x557437f2be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:47:30.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.006+0000 7f32333fd740 1 -- 172.21.15.5:0/3286023056 shutdown_connections 2024-04-24T06:47:30.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.006+0000 7f32333fd740 1 -- 172.21.15.5:0/3286023056 wait complete. 2024-04-24T06:47:30.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.007+0000 7f32333fd740 1 -- start start 2024-04-24T06:47:30.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.007+0000 7f32333fd740 1 Processor -- start 2024-04-24T06:47:30.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.007+0000 7f32333fd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557437f1eb40 0x557437f32130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:30.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.007+0000 7f32333fd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557437f1f460 0x557437f2ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:30.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.007+0000 7f32333fd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557437f29b30 0x557437f2b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:30.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.007+0000 7f32333fd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f322c1ae2f0 con 0x557437f1eb40 2024-04-24T06:47:30.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.007+0000 7f32333fd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f322c1af760 con 0x557437f1f460 2024-04-24T06:47:30.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.007+0000 7f32333fd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f322c1ae9c0 con 0x557437f29b30 2024-04-24T06:47:30.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.007+0000 7f32333fd740 10 client.0 _ll_register_callbacks cb 0x7f32140d4bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:47:30.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.008+0000 7f3229e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557437f1eb40 0x557437f32130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:30.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.008+0000 7f3229e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557437f1eb40 0x557437f32130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41542/0 (socket says 172.21.15.5:41542) 2024-04-24T06:47:30.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.008+0000 7f3229e00640 1 -- 172.21.15.5:0/3559516800 learned_addr learned my addr 172.21.15.5:0/3559516800 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:47:30.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.008+0000 7f3229400640 1 --2- 172.21.15.5:0/3559516800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557437f1f460 0x557437f2ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:30.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.008+0000 7f322a800640 1 --2- 172.21.15.5:0/3559516800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557437f29b30 0x557437f2b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:30.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.008+0000 7f3229e00640 1 -- 172.21.15.5:0/3559516800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557437f29b30 msgr2=0x557437f2b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:30.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.008+0000 7f3229e00640 1 --2- 172.21.15.5:0/3559516800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557437f29b30 0x557437f2b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:30.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.008+0000 7f3229e00640 1 -- 172.21.15.5:0/3559516800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557437f1f460 msgr2=0x557437f2ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:30.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.008+0000 7f3229e00640 1 --2- 172.21.15.5:0/3559516800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557437f1f460 0x557437f2ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:30.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.008+0000 7f3229e00640 1 -- 172.21.15.5:0/3559516800 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557437c607d0 con 0x557437f1eb40 2024-04-24T06:47:30.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.008+0000 7f3229e00640 1 --2- 172.21.15.5:0/3559516800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557437f1eb40 0x557437f32130 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f321c004700 tx=0x7f321c004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ba2a9b73a1b5be server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:47:30.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.008+0000 7f3213400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:47:30.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.009+0000 7f3213400640 1 -- 172.21.15.5:0/3559516800 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c053070 con 0x557437f1eb40 2024-04-24T06:47:30.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.009+0000 7f3213400640 1 -- 172.21.15.5:0/3559516800 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f321c005230 con 0x557437f1eb40 2024-04-24T06:47:30.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.009+0000 7f32333fd740 1 -- 172.21.15.5:0/3559516800 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3224005500 con 0x557437f1eb40 2024-04-24T06:47:30.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.009+0000 7f3213400640 1 -- 172.21.15.5:0/3559516800 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f321c064050 con 0x557437f1eb40 2024-04-24T06:47:30.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.009+0000 7f32333fd740 10 client.15945 fetch_fsmap learned FSMap version 25 2024-04-24T06:47:30.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.009+0000 7f32333fd740 1 -- 172.21.15.5:0/3559516800 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557437f9fd90 con 0x557437f1eb40 2024-04-24T06:47:30.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.009+0000 7f3213400640 1 -- 172.21.15.5:0/3559516800 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3224005500 con 0x557437f1eb40 2024-04-24T06:47:30.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.009+0000 7f3213400640 1 -- 172.21.15.5:0/3559516800 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f321c031640 con 0x557437f1eb40 2024-04-24T06:47:30.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.009+0000 7f32333fd740 10 client.15945 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:47:30.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.009+0000 7f32333fd740 10 client.15945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:47:30.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.009+0000 7f32333fd740 20 client.15945 populate_metadata read hostname 'smithi005' 2024-04-24T06:47:30.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.010+0000 7f32333fd740 1 --2- 172.21.15.5:0/3559516800 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x557437fa03a0 0x557437fa27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:30.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.010+0000 7f32333fd740 4 client.15945 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:47:30.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.010+0000 7f32333fd740 1 -- 172.21.15.5:0/3559516800 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557437e8ba60 con 0x557437fa03a0 2024-04-24T06:47:30.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.010+0000 7f3229400640 1 --2- 172.21.15.5:0/3559516800 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x557437fa03a0 0x557437fa27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:30.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.010+0000 7f3229400640 1 --2- 172.21.15.5:0/3559516800 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x557437fa03a0 0x557437fa27c0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a93c1ad2ad3dcb2 server_cookie=1ad6babf23a2570a in_seq=0 out_seq=0 2024-04-24T06:47:30.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.010+0000 7f3213400640 10 client.15945 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:47:30.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.011+0000 7f3213400640 1 -- 172.21.15.5:0/3559516800 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557437e8ba60 con 0x557437fa03a0 2024-04-24T06:47:30.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.011+0000 7f3213400640 10 client.15945 handle_command_reply: tid=0 2024-04-24T06:47:30.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.011+0000 7f32333fd740 1 client.15945 shutdown 2024-04-24T06:47:30.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.011+0000 7f32333fd740 1 -- 172.21.15.5:0/3559516800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557437f1eb40 msgr2=0x557437f32130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:30.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.011+0000 7f32333fd740 1 --2- 172.21.15.5:0/3559516800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557437f1eb40 0x557437f32130 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f321c004700 tx=0x7f321c004f80 comp rx=0 tx=0).stop 2024-04-24T06:47:30.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.011+0000 7f32333fd740 1 -- 172.21.15.5:0/3559516800 shutdown_connections 2024-04-24T06:47:30.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.011+0000 7f32333fd740 1 --2- 172.21.15.5:0/3559516800 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x557437fa03a0 0x557437fa27c0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:30.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.011+0000 7f32333fd740 1 --2- 172.21.15.5:0/3559516800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557437f29b30 0x557437f2b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:30.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.011+0000 7f32333fd740 1 --2- 172.21.15.5:0/3559516800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557437f1f460 0x557437f2ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:30.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.011+0000 7f32333fd740 1 --2- 172.21.15.5:0/3559516800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557437f1eb40 0x557437f32130 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:30.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.011+0000 7f32333fd740 1 -- 172.21.15.5:0/3559516800 >> 172.21.15.5:0/3559516800 conn(0x557437f1ae60 msgr2=0x557437f196c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:47:30.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.011+0000 7f3213400640 0 client.15945 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:47:30.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.011+0000 7f32333fd740 1 -- 172.21.15.5:0/3559516800 shutdown_connections 2024-04-24T06:47:30.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.011+0000 7f32333fd740 1 -- 172.21.15.5:0/3559516800 wait complete. 2024-04-24T06:47:30.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.011+0000 7f32333fd740 20 client.15945 trim_cache size 0 max 16384 2024-04-24T06:47:30.021 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:47:30.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.012+0000 7f322b200640 1 -- 172.21.15.5:0/952108586 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f320c0a3d20 msgr2=0x7f320c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:30.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.012+0000 7f322b200640 1 --2- 172.21.15.5:0/952108586 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f320c0a3d20 0x7f320c0a61e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3224005fd0 tx=0x7f3224005cf0 comp rx=0 tx=0).stop 2024-04-24T06:47:30.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.012+0000 7f322b200640 1 -- 172.21.15.5:0/952108586 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f322c107140 msgr2=0x7f322c1a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:30.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.012+0000 7f322b200640 1 --2- 172.21.15.5:0/952108586 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f322c107140 0x7f322c1a0be0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3214005fd0 tx=0x7f321400ef90 comp rx=0 tx=0).stop 2024-04-24T06:47:30.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.012+0000 7f322b200640 1 -- 172.21.15.5:0/952108586 shutdown_connections 2024-04-24T06:47:30.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.012+0000 7f322b200640 1 --2- 172.21.15.5:0/952108586 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f320c0a3d20 0x7f320c0a61e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:30.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.012+0000 7f322b200640 1 --2- 172.21.15.5:0/952108586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f322c19a7b0 0x7f322c19ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:30.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.012+0000 7f322b200640 1 --2- 172.21.15.5:0/952108586 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f322c107140 0x7f322c1a0be0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:30.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.012+0000 7f322b200640 1 --2- 172.21.15.5:0/952108586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f322c102070 0x7f322c1a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:30.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.012+0000 7f322b200640 1 -- 172.21.15.5:0/952108586 >> 172.21.15.5:0/952108586 conn(0x7f322c0fd950 msgr2=0x7f322c0ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:47:30.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.012+0000 7f322b200640 1 -- 172.21.15.5:0/952108586 shutdown_connections 2024-04-24T06:47:30.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:30.012+0000 7f322b200640 1 -- 172.21.15.5:0/952108586 wait complete. 2024-04-24T06:47:31.032 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T06:47:31.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:47:31.033 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T06:47:31.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.261+0000 7f3bc8e00640 1 -- 172.21.15.5:0/1977058696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc410e4f0 msgr2=0x7f3bc410e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:31.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.261+0000 7f3bc8e00640 1 --2- 172.21.15.5:0/1977058696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc410e4f0 0x7f3bc410e8d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4009900 tx=0x7f3bb402a480 comp rx=0 tx=0).stop 2024-04-24T06:47:31.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.261+0000 7f3bc8e00640 1 -- 172.21.15.5:0/1977058696 shutdown_connections 2024-04-24T06:47:31.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.261+0000 7f3bc8e00640 1 --2- 172.21.15.5:0/1977058696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bc4108ee0 0x7f3bc41152b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.261+0000 7f3bc8e00640 1 --2- 172.21.15.5:0/1977058696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3bc4108540 0x7f3bc41089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.261+0000 7f3bc8e00640 1 --2- 172.21.15.5:0/1977058696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc410e4f0 0x7f3bc410e8d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.261+0000 7f3bc8e00640 1 -- 172.21.15.5:0/1977058696 >> 172.21.15.5:0/1977058696 conn(0x7f3bc4077e10 msgr2=0x7f3bc4106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:47:31.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.261+0000 7f3bc8e00640 1 -- 172.21.15.5:0/1977058696 shutdown_connections 2024-04-24T06:47:31.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.261+0000 7f3bc8e00640 1 -- 172.21.15.5:0/1977058696 wait complete. 2024-04-24T06:47:31.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.262+0000 7f3bc8e00640 1 Processor -- start 2024-04-24T06:47:31.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.262+0000 7f3bc8e00640 1 -- start start 2024-04-24T06:47:31.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.263+0000 7f3bc8e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bc4108540 0x7f3bc419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:31.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.263+0000 7f3bc8e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc4108ee0 0x7f3bc419f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:31.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.263+0000 7f3bc8e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3bc410e4f0 0x7f3bc41a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:31.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.263+0000 7f3bc8e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc4118810 con 0x7f3bc4108ee0 2024-04-24T06:47:31.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.263+0000 7f3bc8e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc4118690 con 0x7f3bc4108540 2024-04-24T06:47:31.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.263+0000 7f3bc8e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc4118990 con 0x7f3bc410e4f0 2024-04-24T06:47:31.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.263+0000 7f3bc2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc4108ee0 0x7f3bc419f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:31.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.263+0000 7f3bc2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc4108ee0 0x7f3bc419f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41562/0 (socket says 172.21.15.5:41562) 2024-04-24T06:47:31.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.263+0000 7f3bc2a00640 1 -- 172.21.15.5:0/3555142900 learned_addr learned my addr 172.21.15.5:0/3555142900 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:47:31.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.263+0000 7f3bc3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3bc410e4f0 0x7f3bc41a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:31.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.263+0000 7f3bc3400640 1 --2- 172.21.15.5:0/3555142900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bc4108540 0x7f3bc419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:31.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.264+0000 7f3bc2a00640 1 -- 172.21.15.5:0/3555142900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3bc410e4f0 msgr2=0x7f3bc41a35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:31.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.264+0000 7f3bc2a00640 1 --2- 172.21.15.5:0/3555142900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3bc410e4f0 0x7f3bc41a35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.264+0000 7f3bc2a00640 1 -- 172.21.15.5:0/3555142900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bc4108540 msgr2=0x7f3bc419ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:31.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.264+0000 7f3bc2a00640 1 --2- 172.21.15.5:0/3555142900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bc4108540 0x7f3bc419ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.264+0000 7f3bc2a00640 1 -- 172.21.15.5:0/3555142900 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bc41a3d20 con 0x7f3bc4108ee0 2024-04-24T06:47:31.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.264+0000 7f3bc2a00640 1 --2- 172.21.15.5:0/3555142900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc4108ee0 0x7f3bc419f270 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3bac00eb90 tx=0x7f3bac00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:47:31.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.264+0000 7f3bbbe00640 1 -- 172.21.15.5:0/3555142900 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac004b20 con 0x7f3bc4108ee0 2024-04-24T06:47:31.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.264+0000 7f3bbbe00640 1 -- 172.21.15.5:0/3555142900 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bac004cc0 con 0x7f3bc4108ee0 2024-04-24T06:47:31.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.264+0000 7f3bc8e00640 1 -- 172.21.15.5:0/3555142900 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bc41a4010 con 0x7f3bc4108ee0 2024-04-24T06:47:31.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.264+0000 7f3bbbe00640 1 -- 172.21.15.5:0/3555142900 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac010720 con 0x7f3bc4108ee0 2024-04-24T06:47:31.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.264+0000 7f3bc8e00640 1 -- 172.21.15.5:0/3555142900 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bc41ab8a0 con 0x7f3bc4108ee0 2024-04-24T06:47:31.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.268+0000 7f3bcd265740 1 Processor -- start 2024-04-24T06:47:31.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.268+0000 7f3bcd265740 1 -- start start 2024-04-24T06:47:31.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.268+0000 7f3bbbe00640 1 -- 172.21.15.5:0/3555142900 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f3bac005020 con 0x7f3bc4108ee0 2024-04-24T06:47:31.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.268+0000 7f3bcd265740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565e88f6270 0x5565e88f6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:31.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.268+0000 7f3bcd265740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e88f6b90 0x5565e8900d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:31.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.268+0000 7f3bcd265740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e8901260 0x5565e8903650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:31.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.268+0000 7f3bcd265740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5565e885be70 con 0x5565e88f6b90 2024-04-24T06:47:31.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.268+0000 7f3bcd265740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5565e8802380 con 0x5565e8901260 2024-04-24T06:47:31.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.268+0000 7f3bcd265740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5565e8857e30 con 0x5565e88f6270 2024-04-24T06:47:31.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.269+0000 7f3bc3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565e88f6270 0x5565e88f6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:31.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.269+0000 7f3bc3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e88f6b90 0x5565e8900d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:31.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.269+0000 7f3bc3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565e88f6270 0x5565e88f6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:53098/0 (socket says 172.21.15.5:53098) 2024-04-24T06:47:31.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.269+0000 7f3bc3e00640 1 -- 172.21.15.5:0/1865176243 learned_addr learned my addr 172.21.15.5:0/1865176243 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:47:31.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.269+0000 7f3bc2a00640 1 --2- 172.21.15.5:0/1865176243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e8901260 0x5565e8903650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:31.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.269+0000 7f3bc3e00640 1 -- 172.21.15.5:0/1865176243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e8901260 msgr2=0x5565e8903650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:31.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.269+0000 7f3bc3e00640 1 --2- 172.21.15.5:0/1865176243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e8901260 0x5565e8903650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.269+0000 7f3bc3e00640 1 -- 172.21.15.5:0/1865176243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e88f6b90 msgr2=0x5565e8900d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:31.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.269+0000 7f3bc3e00640 1 --2- 172.21.15.5:0/1865176243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e88f6b90 0x5565e8900d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.269+0000 7f3bc3e00640 1 -- 172.21.15.5:0/1865176243 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565e86367e0 con 0x5565e88f6270 2024-04-24T06:47:31.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.270+0000 7f3bc3e00640 1 --2- 172.21.15.5:0/1865176243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565e88f6270 0x5565e88f6650 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc009c10 tx=0x7f3bbc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=806cb2743ebf40ce server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:47:31.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.270+0000 7f3bb8c00640 1 -- 172.21.15.5:0/1865176243 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc004230 con 0x5565e88f6270 2024-04-24T06:47:31.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.270+0000 7f3bb8c00640 1 -- 172.21.15.5:0/1865176243 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bbc0043d0 con 0x5565e88f6270 2024-04-24T06:47:31.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.270+0000 7f3bbbe00640 1 --2- 172.21.15.5:0/3555142900 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f3ba80a3da0 0x7f3ba80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:31.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.270+0000 7f3bb8c00640 1 -- 172.21.15.5:0/1865176243 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc0197a0 con 0x5565e88f6270 2024-04-24T06:47:31.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.270+0000 7f3bbbe00640 1 -- 172.21.15.5:0/3555142900 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f3bac0d0d90 con 0x7f3bc4108ee0 2024-04-24T06:47:31.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.270+0000 7f3bc3400640 1 --2- 172.21.15.5:0/3555142900 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f3ba80a3da0 0x7f3ba80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:31.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.271+0000 7f3bcd265740 1 -- 172.21.15.5:0/1865176243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565e88f6270 msgr2=0x5565e88f6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:31.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.271+0000 7f3bcd265740 1 --2- 172.21.15.5:0/1865176243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565e88f6270 0x5565e88f6650 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc009c10 tx=0x7f3bbc011810 comp rx=0 tx=0).stop 2024-04-24T06:47:31.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.271+0000 7f3bcd265740 1 -- 172.21.15.5:0/1865176243 shutdown_connections 2024-04-24T06:47:31.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.271+0000 7f3bcd265740 1 --2- 172.21.15.5:0/1865176243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e8901260 0x5565e8903650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.271+0000 7f3bcd265740 1 --2- 172.21.15.5:0/1865176243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e88f6b90 0x5565e8900d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.271+0000 7f3bcd265740 1 --2- 172.21.15.5:0/1865176243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565e88f6270 0x5565e88f6650 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.271+0000 7f3bcd265740 1 -- 172.21.15.5:0/1865176243 >> 172.21.15.5:0/1865176243 conn(0x5565e88f2590 msgr2=0x5565e88f2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:47:31.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.271+0000 7f3bcd265740 1 -- 172.21.15.5:0/1865176243 shutdown_connections 2024-04-24T06:47:31.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.271+0000 7f3bc3400640 1 --2- 172.21.15.5:0/3555142900 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f3ba80a3da0 0x7f3ba80a6260 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4000c00 tx=0x7f3bb4005b50 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:47:31.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.271+0000 7f3bcd265740 1 -- 172.21.15.5:0/1865176243 wait complete. 2024-04-24T06:47:31.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.272+0000 7f3bcd265740 1 -- start start 2024-04-24T06:47:31.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.272+0000 7f3bcd265740 1 Processor -- start 2024-04-24T06:47:31.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.272+0000 7f3bcd265740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e88f6270 0x5565e88fe220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:31.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.272+0000 7f3bcd265740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e88f6b90 0x5565e88fe760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:31.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.272+0000 7f3bcd265740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565e8901260 0x5565e88feca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:31.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.272+0000 7f3bcd265740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5565e8904050 con 0x5565e88f6270 2024-04-24T06:47:31.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.272+0000 7f3bcd265740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5565e8815660 con 0x5565e88f6b90 2024-04-24T06:47:31.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.272+0000 7f3bcd265740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5565e89041d0 con 0x5565e8901260 2024-04-24T06:47:31.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.272+0000 7f3bcd265740 10 client.0 _ll_register_callbacks cb 0x5565e8803aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:47:31.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.273+0000 7f3bc3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e88f6270 0x5565e88fe220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:31.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.273+0000 7f3bc3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e88f6270 0x5565e88fe220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41578/0 (socket says 172.21.15.5:41578) 2024-04-24T06:47:31.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.273+0000 7f3bc3400640 1 -- 172.21.15.5:0/2023458864 learned_addr learned my addr 172.21.15.5:0/2023458864 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:47:31.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.273+0000 7f3bc3e00640 1 --2- 172.21.15.5:0/2023458864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565e8901260 0x5565e88feca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:31.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.273+0000 7f3bc2a00640 1 --2- 172.21.15.5:0/2023458864 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e88f6b90 0x5565e88fe760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:31.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.273+0000 7f3bc3400640 1 -- 172.21.15.5:0/2023458864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565e8901260 msgr2=0x5565e88feca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:31.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.273+0000 7f3bc3400640 1 --2- 172.21.15.5:0/2023458864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565e8901260 0x5565e88feca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.273+0000 7f3bc3400640 1 -- 172.21.15.5:0/2023458864 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e88f6b90 msgr2=0x5565e88fe760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:31.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.273+0000 7f3bc3400640 1 --2- 172.21.15.5:0/2023458864 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e88f6b90 0x5565e88fe760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.273+0000 7f3bc3400640 1 -- 172.21.15.5:0/2023458864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565e8696f70 con 0x5565e88f6270 2024-04-24T06:47:31.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.274+0000 7f3bc3400640 1 --2- 172.21.15.5:0/2023458864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e88f6270 0x5565e88fe220 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4009900 tx=0x7f3bb4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7871d7911262902 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:47:31.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.274+0000 7f3bb1600640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:47:31.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.274+0000 7f3bb1600640 1 -- 172.21.15.5:0/2023458864 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb404e070 con 0x5565e88f6270 2024-04-24T06:47:31.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.274+0000 7f3bb1600640 1 -- 172.21.15.5:0/2023458864 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f3bb4004bb0 con 0x5565e88f6270 2024-04-24T06:47:31.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.274+0000 7f3bcd265740 1 -- 172.21.15.5:0/2023458864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bbc0043d0 con 0x5565e88f6270 2024-04-24T06:47:31.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.274+0000 7f3bb1600640 1 -- 172.21.15.5:0/2023458864 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3bb405a050 con 0x5565e88f6270 2024-04-24T06:47:31.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.274+0000 7f3bcd265740 10 client.15957 fetch_fsmap learned FSMap version 25 2024-04-24T06:47:31.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.274+0000 7f3bcd265740 1 -- 172.21.15.5:0/2023458864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5565e852e170 con 0x5565e88f6270 2024-04-24T06:47:31.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.274+0000 7f3bb1600640 1 -- 172.21.15.5:0/2023458864 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bbc0043d0 con 0x5565e88f6270 2024-04-24T06:47:31.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.275+0000 7f3bb1600640 1 -- 172.21.15.5:0/2023458864 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3bb40326e0 con 0x5565e88f6270 2024-04-24T06:47:31.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.275+0000 7f3bcd265740 10 client.15957 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:47:31.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.275+0000 7f3bcd265740 10 client.15957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:47:31.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.275+0000 7f3bcd265740 20 client.15957 populate_metadata read hostname 'smithi005' 2024-04-24T06:47:31.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.275+0000 7f3bcd265740 1 --2- 172.21.15.5:0/2023458864 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5565e8911600 0x5565e8913a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:31.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.275+0000 7f3bcd265740 4 client.15957 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:47:31.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.275+0000 7f3bcd265740 1 -- 172.21.15.5:0/2023458864 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565e8869890 con 0x5565e8911600 2024-04-24T06:47:31.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.275+0000 7f3bc2a00640 1 --2- 172.21.15.5:0/2023458864 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5565e8911600 0x5565e8913a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:31.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.276+0000 7f3bc2a00640 1 --2- 172.21.15.5:0/2023458864 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5565e8911600 0x5565e8913a80 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62545c3a179603df server_cookie=1904dc0c38025c6e in_seq=0 out_seq=0 2024-04-24T06:47:31.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.276+0000 7f3bb1600640 10 client.15957 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:47:31.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.276+0000 7f3bb1600640 1 -- 172.21.15.5:0/2023458864 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5565e8869890 con 0x5565e8911600 2024-04-24T06:47:31.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.276+0000 7f3bb1600640 10 client.15957 handle_command_reply: tid=0 2024-04-24T06:47:31.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.277+0000 7f3bcd265740 1 client.15957 shutdown 2024-04-24T06:47:31.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.277+0000 7f3bcd265740 1 -- 172.21.15.5:0/2023458864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e88f6270 msgr2=0x5565e88fe220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:31.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.277+0000 7f3bcd265740 1 --2- 172.21.15.5:0/2023458864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e88f6270 0x5565e88fe220 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4009900 tx=0x7f3bb4009260 comp rx=0 tx=0).stop 2024-04-24T06:47:31.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.277+0000 7f3bcd265740 1 -- 172.21.15.5:0/2023458864 shutdown_connections 2024-04-24T06:47:31.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.277+0000 7f3bcd265740 1 --2- 172.21.15.5:0/2023458864 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5565e8911600 0x5565e8913a80 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.277+0000 7f3bcd265740 1 --2- 172.21.15.5:0/2023458864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565e8901260 0x5565e88feca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.277+0000 7f3bcd265740 1 --2- 172.21.15.5:0/2023458864 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e88f6b90 0x5565e88fe760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.277+0000 7f3bcd265740 1 --2- 172.21.15.5:0/2023458864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e88f6270 0x5565e88fe220 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.277+0000 7f3bb1600640 0 client.15957 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:47:31.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.277+0000 7f3bcd265740 1 -- 172.21.15.5:0/2023458864 >> 172.21.15.5:0/2023458864 conn(0x5565e88f2590 msgr2=0x5565e88f8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:47:31.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.277+0000 7f3bcd265740 1 -- 172.21.15.5:0/2023458864 shutdown_connections 2024-04-24T06:47:31.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.277+0000 7f3bcd265740 1 -- 172.21.15.5:0/2023458864 wait complete. 2024-04-24T06:47:31.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.277+0000 7f3bcd265740 20 client.15957 trim_cache size 0 max 16384 2024-04-24T06:47:31.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.298+0000 7f3bcd265740 1 Processor -- start 2024-04-24T06:47:31.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.298+0000 7f3bcd265740 1 -- start start 2024-04-24T06:47:31.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.298+0000 7f3bcd265740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565e88f6270 0x5565e88f89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:31.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.298+0000 7f3bcd265740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e88f6b90 0x5565e88fe8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:31.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.298+0000 7f3bcd265740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e8901260 0x5565e8919350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:31.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.298+0000 7f3bcd265740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc41b69e0 con 0x5565e8901260 2024-04-24T06:47:31.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.298+0000 7f3bcd265740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc41b42b0 con 0x5565e88f6b90 2024-04-24T06:47:31.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.298+0000 7f3bcd265740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc41b2100 con 0x5565e88f6270 2024-04-24T06:47:31.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.299+0000 7f3bc3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565e88f6270 0x5565e88f89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:31.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.299+0000 7f3bc2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e88f6b90 0x5565e88fe8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:31.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.299+0000 7f3bc3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565e88f6270 0x5565e88f89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:53104/0 (socket says 172.21.15.5:53104) 2024-04-24T06:47:31.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.299+0000 7f3bc3400640 1 -- 172.21.15.5:0/455768867 learned_addr learned my addr 172.21.15.5:0/455768867 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:47:31.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.299+0000 7f3bc3e00640 1 --2- 172.21.15.5:0/455768867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e8901260 0x5565e8919350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:31.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.299+0000 7f3bc3400640 1 -- 172.21.15.5:0/455768867 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e88f6b90 msgr2=0x5565e88fe8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:31.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.299+0000 7f3bc3400640 1 --2- 172.21.15.5:0/455768867 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e88f6b90 0x5565e88fe8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.299+0000 7f3bc3400640 1 -- 172.21.15.5:0/455768867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e8901260 msgr2=0x5565e8919350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:31.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.299+0000 7f3bc3400640 1 --2- 172.21.15.5:0/455768867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e8901260 0x5565e8919350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.299+0000 7f3bc3400640 1 -- 172.21.15.5:0/455768867 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565e88ede80 con 0x5565e88f6270 2024-04-24T06:47:31.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.299+0000 7f3bc2a00640 1 --2- 172.21.15.5:0/455768867 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e88f6b90 0x5565e88fe8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:47:31.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.300+0000 7f3bc3400640 1 --2- 172.21.15.5:0/455768867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565e88f6270 0x5565e88f89d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4006010 tx=0x7f3bb40044d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b24802c3623d9b13 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:47:31.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.300+0000 7f3bb3e00640 1 -- 172.21.15.5:0/455768867 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb404e070 con 0x5565e88f6270 2024-04-24T06:47:31.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.300+0000 7f3bb3e00640 1 -- 172.21.15.5:0/455768867 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bb4063070 con 0x5565e88f6270 2024-04-24T06:47:31.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.300+0000 7f3bb3e00640 1 -- 172.21.15.5:0/455768867 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4035040 con 0x5565e88f6270 2024-04-24T06:47:31.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.301+0000 7f3bcd265740 1 -- 172.21.15.5:0/455768867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565e88f6270 msgr2=0x5565e88f89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:31.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.301+0000 7f3bcd265740 1 --2- 172.21.15.5:0/455768867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565e88f6270 0x5565e88f89d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4006010 tx=0x7f3bb40044d0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.301+0000 7f3bcd265740 1 -- 172.21.15.5:0/455768867 shutdown_connections 2024-04-24T06:47:31.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.301+0000 7f3bcd265740 1 --2- 172.21.15.5:0/455768867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e8901260 0x5565e8919350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.301+0000 7f3bcd265740 1 --2- 172.21.15.5:0/455768867 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e88f6b90 0x5565e88fe8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.301+0000 7f3bcd265740 1 --2- 172.21.15.5:0/455768867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565e88f6270 0x5565e88f89d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.301+0000 7f3bcd265740 1 -- 172.21.15.5:0/455768867 >> 172.21.15.5:0/455768867 conn(0x5565e88f2590 msgr2=0x5565e88fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:47:31.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.301+0000 7f3bcd265740 1 -- 172.21.15.5:0/455768867 shutdown_connections 2024-04-24T06:47:31.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.301+0000 7f3bcd265740 1 -- 172.21.15.5:0/455768867 wait complete. 2024-04-24T06:47:31.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.302+0000 7f3bcd265740 1 -- start start 2024-04-24T06:47:31.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.302+0000 7f3bcd265740 1 Processor -- start 2024-04-24T06:47:31.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.302+0000 7f3bcd265740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565e88f6270 0x5565e89094d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:31.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.302+0000 7f3bcd265740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e88f6b90 0x5565e8905850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:31.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.302+0000 7f3bcd265740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e8901260 0x5565e8905d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:31.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.302+0000 7f3bcd265740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc41b2770 con 0x5565e8901260 2024-04-24T06:47:31.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.302+0000 7f3bcd265740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc41b3be0 con 0x5565e88f6b90 2024-04-24T06:47:31.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.302+0000 7f3bcd265740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc41b2e40 con 0x5565e88f6270 2024-04-24T06:47:31.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.302+0000 7f3bcd265740 10 client.0 _ll_register_callbacks cb 0x7f3bb4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:47:31.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.303+0000 7f3bc3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e8901260 0x5565e8905d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:31.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.303+0000 7f3bc3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565e88f6270 0x5565e89094d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:31.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.303+0000 7f3bc3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e8901260 0x5565e8905d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41586/0 (socket says 172.21.15.5:41586) 2024-04-24T06:47:31.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.303+0000 7f3bc3e00640 1 -- 172.21.15.5:0/2281418595 learned_addr learned my addr 172.21.15.5:0/2281418595 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:47:31.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.303+0000 7f3bc2a00640 1 --2- 172.21.15.5:0/2281418595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e88f6b90 0x5565e8905850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:31.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.303+0000 7f3bc3e00640 1 -- 172.21.15.5:0/2281418595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565e88f6270 msgr2=0x5565e89094d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:31.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.303+0000 7f3bc3e00640 1 --2- 172.21.15.5:0/2281418595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565e88f6270 0x5565e89094d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.303+0000 7f3bc3e00640 1 -- 172.21.15.5:0/2281418595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e88f6b90 msgr2=0x5565e8905850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:31.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.303+0000 7f3bc3e00640 1 --2- 172.21.15.5:0/2281418595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e88f6b90 0x5565e8905850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.303+0000 7f3bc3e00640 1 -- 172.21.15.5:0/2281418595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565e89062d0 con 0x5565e8901260 2024-04-24T06:47:31.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.303+0000 7f3bc2a00640 1 --2- 172.21.15.5:0/2281418595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e88f6b90 0x5565e8905850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:47:31.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.303+0000 7f3bc3400640 1 --2- 172.21.15.5:0/2281418595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565e88f6270 0x5565e89094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:47:31.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.304+0000 7f3bc3e00640 1 --2- 172.21.15.5:0/2281418595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e8901260 0x5565e8905d90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc009c10 tx=0x7f3bbc005580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a56d3000b122d885 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:47:31.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.304+0000 7f3bb1600640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:47:31.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.304+0000 7f3bb1600640 1 -- 172.21.15.5:0/2281418595 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc0035f0 con 0x5565e8901260 2024-04-24T06:47:31.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.304+0000 7f3bb1600640 1 -- 172.21.15.5:0/2281418595 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f3bbc003790 con 0x5565e8901260 2024-04-24T06:47:31.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.304+0000 7f3bcd265740 1 -- 172.21.15.5:0/2281418595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bb4063070 con 0x5565e8901260 2024-04-24T06:47:31.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.304+0000 7f3bb1600640 1 -- 172.21.15.5:0/2281418595 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3bbc003990 con 0x5565e8901260 2024-04-24T06:47:31.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.304+0000 7f3bcd265740 10 client.15963 fetch_fsmap learned FSMap version 25 2024-04-24T06:47:31.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.304+0000 7f3bcd265740 1 -- 172.21.15.5:0/2281418595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5565e8978900 con 0x5565e8901260 2024-04-24T06:47:31.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.305+0000 7f3bb1600640 1 -- 172.21.15.5:0/2281418595 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bb4063070 con 0x5565e8901260 2024-04-24T06:47:31.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.305+0000 7f3bb1600640 1 -- 172.21.15.5:0/2281418595 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3bbc02a020 con 0x5565e8901260 2024-04-24T06:47:31.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.305+0000 7f3bcd265740 10 client.15963 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:47:31.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.305+0000 7f3bcd265740 10 client.15963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:47:31.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.305+0000 7f3bcd265740 20 client.15963 populate_metadata read hostname 'smithi005' 2024-04-24T06:47:31.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.305+0000 7f3bcd265740 1 --2- 172.21.15.5:0/2281418595 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5565e8978ee0 0x5565e897b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:31.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.305+0000 7f3bcd265740 4 client.15963 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T06:47:31.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.305+0000 7f3bcd265740 1 -- 172.21.15.5:0/2281418595 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5565e8863700 con 0x5565e8978ee0 2024-04-24T06:47:31.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.305+0000 7f3bc3400640 1 --2- 172.21.15.5:0/2281418595 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5565e8978ee0 0x5565e897b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:31.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.306+0000 7f3bc3400640 1 --2- 172.21.15.5:0/2281418595 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5565e8978ee0 0x5565e897b300 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4565b8a31d1d03e server_cookie=f929cec2eae75f15 in_seq=0 out_seq=0 2024-04-24T06:47:31.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.306+0000 7f3bb1600640 10 client.15963 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:47:31.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.307+0000 7f3bb1600640 1 -- 172.21.15.5:0/2281418595 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5565e8863700 con 0x5565e8978ee0 2024-04-24T06:47:31.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.307+0000 7f3bb1600640 10 client.15963 handle_command_reply: tid=0 2024-04-24T06:47:31.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.307+0000 7f3bcd265740 1 client.15963 shutdown 2024-04-24T06:47:31.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.307+0000 7f3bcd265740 1 -- 172.21.15.5:0/2281418595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e8901260 msgr2=0x5565e8905d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:31.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.307+0000 7f3bcd265740 1 --2- 172.21.15.5:0/2281418595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e8901260 0x5565e8905d90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc009c10 tx=0x7f3bbc005580 comp rx=0 tx=0).stop 2024-04-24T06:47:31.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.308+0000 7f3bcd265740 1 -- 172.21.15.5:0/2281418595 shutdown_connections 2024-04-24T06:47:31.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.308+0000 7f3bcd265740 1 --2- 172.21.15.5:0/2281418595 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5565e8978ee0 0x5565e897b300 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.308+0000 7f3bcd265740 1 --2- 172.21.15.5:0/2281418595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e8901260 0x5565e8905d90 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.308+0000 7f3bcd265740 1 --2- 172.21.15.5:0/2281418595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e88f6b90 0x5565e8905850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.308+0000 7f3bcd265740 1 --2- 172.21.15.5:0/2281418595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565e88f6270 0x5565e89094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.308+0000 7f3bb1600640 0 client.15963 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:47:31.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.308+0000 7f3bcd265740 1 -- 172.21.15.5:0/2281418595 >> 172.21.15.5:0/2281418595 conn(0x5565e88f2590 msgr2=0x5565e88fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:47:31.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.308+0000 7f3bcd265740 1 -- 172.21.15.5:0/2281418595 shutdown_connections 2024-04-24T06:47:31.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.308+0000 7f3bcd265740 1 -- 172.21.15.5:0/2281418595 wait complete. 2024-04-24T06:47:31.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.308+0000 7f3bcd265740 20 client.15963 trim_cache size 0 max 16384 2024-04-24T06:47:31.316 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:47:31.316 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-24T06:47:31.317 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "04e0297c-9b5f-4cb3-af15-2af54f2aca0e", 2024-04-24T06:47:31.317 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-24T06:47:31.317 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:47:31.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.309+0000 7f3bc8e00640 1 -- 172.21.15.5:0/3555142900 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f3ba80a3da0 msgr2=0x7f3ba80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:31.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.309+0000 7f3bc8e00640 1 --2- 172.21.15.5:0/3555142900 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f3ba80a3da0 0x7f3ba80a6260 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4000c00 tx=0x7f3bb4005b50 comp rx=0 tx=0).stop 2024-04-24T06:47:31.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.309+0000 7f3bc8e00640 1 -- 172.21.15.5:0/3555142900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc4108ee0 msgr2=0x7f3bc419f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:31.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.309+0000 7f3bc8e00640 1 --2- 172.21.15.5:0/3555142900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc4108ee0 0x7f3bc419f270 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3bac00eb90 tx=0x7f3bac00ef90 comp rx=0 tx=0).stop 2024-04-24T06:47:31.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.309+0000 7f3bc8e00640 1 -- 172.21.15.5:0/3555142900 shutdown_connections 2024-04-24T06:47:31.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.309+0000 7f3bc8e00640 1 --2- 172.21.15.5:0/3555142900 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f3ba80a3da0 0x7f3ba80a6260 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.309+0000 7f3bc8e00640 1 --2- 172.21.15.5:0/3555142900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3bc410e4f0 0x7f3bc41a35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.309+0000 7f3bc8e00640 1 --2- 172.21.15.5:0/3555142900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc4108ee0 0x7f3bc419f270 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.309+0000 7f3bc8e00640 1 --2- 172.21.15.5:0/3555142900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bc4108540 0x7f3bc419ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.309+0000 7f3bc8e00640 1 -- 172.21.15.5:0/3555142900 >> 172.21.15.5:0/3555142900 conn(0x7f3bc4077e10 msgr2=0x7f3bc4078490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:47:31.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.309+0000 7f3bc8e00640 1 -- 172.21.15.5:0/3555142900 shutdown_connections 2024-04-24T06:47:31.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.309+0000 7f3bc8e00640 1 -- 172.21.15.5:0/3555142900 wait complete. 2024-04-24T06:47:31.334 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T06:47:31.335 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T06:47:31.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.570+0000 7f8f55400640 1 -- 172.21.15.5:0/2461222803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f50104e00 msgr2=0x7f8f50108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:31.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.570+0000 7f8f55400640 1 --2- 172.21.15.5:0/2461222803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f50104e00 0x7f8f50108e40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c009900 tx=0x7f8f3c02a400 comp rx=0 tx=0).stop 2024-04-24T06:47:31.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.570+0000 7f8f55400640 1 -- 172.21.15.5:0/2461222803 shutdown_connections 2024-04-24T06:47:31.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.570+0000 7f8f55400640 1 --2- 172.21.15.5:0/2461222803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f50109520 0x7f8f501110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.570+0000 7f8f55400640 1 --2- 172.21.15.5:0/2461222803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f50104e00 0x7f8f50108e40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.570+0000 7f8f55400640 1 --2- 172.21.15.5:0/2461222803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f50104450 0x7f8f50104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.570+0000 7f8f55400640 1 -- 172.21.15.5:0/2461222803 >> 172.21.15.5:0/2461222803 conn(0x7f8f500ffc90 msgr2=0x7f8f501020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:47:31.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.570+0000 7f8f55400640 1 -- 172.21.15.5:0/2461222803 shutdown_connections 2024-04-24T06:47:31.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.570+0000 7f8f55400640 1 -- 172.21.15.5:0/2461222803 wait complete. 2024-04-24T06:47:31.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.572+0000 7f8f55400640 1 Processor -- start 2024-04-24T06:47:31.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.572+0000 7f8f55400640 1 -- start start 2024-04-24T06:47:31.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.572+0000 7f8f55400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f50104450 0x7f8f501a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:31.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.572+0000 7f8f55400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f50109520 0x7f8f501a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:31.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.572+0000 7f8f55400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f5019a960 0x7f8f5019ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:31.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.572+0000 7f8f55400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8f501145e0 con 0x7f8f5019a960 2024-04-24T06:47:31.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.572+0000 7f8f55400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8f50114460 con 0x7f8f50104450 2024-04-24T06:47:31.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.572+0000 7f8f55400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8f50114760 con 0x7f8f50109520 2024-04-24T06:47:31.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.573+0000 7f8f4de00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f50109520 0x7f8f501a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:31.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.573+0000 7f8f4de00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f50109520 0x7f8f501a0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:53146/0 (socket says 172.21.15.5:53146) 2024-04-24T06:47:31.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.573+0000 7f8f4de00640 1 -- 172.21.15.5:0/1643556288 learned_addr learned my addr 172.21.15.5:0/1643556288 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:47:31.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.573+0000 7f8f4e800640 1 --2- 172.21.15.5:0/1643556288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f50104450 0x7f8f501a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:31.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.573+0000 7f8f4f200640 1 --2- 172.21.15.5:0/1643556288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f5019a960 0x7f8f5019ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:31.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.573+0000 7f8f4de00640 1 -- 172.21.15.5:0/1643556288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f50104450 msgr2=0x7f8f501a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:31.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.573+0000 7f8f4de00640 1 --2- 172.21.15.5:0/1643556288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f50104450 0x7f8f501a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.573+0000 7f8f4de00640 1 -- 172.21.15.5:0/1643556288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f5019a960 msgr2=0x7f8f5019ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:31.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.573+0000 7f8f4de00640 1 --2- 172.21.15.5:0/1643556288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f5019a960 0x7f8f5019ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.573+0000 7f8f4de00640 1 -- 172.21.15.5:0/1643556288 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f5019b480 con 0x7f8f50109520 2024-04-24T06:47:31.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.573+0000 7f8f4e800640 1 --2- 172.21.15.5:0/1643556288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f50104450 0x7f8f501a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:47:31.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.573+0000 7f8f4f200640 1 --2- 172.21.15.5:0/1643556288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f5019a960 0x7f8f5019ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:47:31.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.574+0000 7f8f4de00640 1 --2- 172.21.15.5:0/1643556288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f50109520 0x7f8f501a0ca0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c02a920 tx=0x7f8f3c003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:47:31.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.574+0000 7f8f47400640 1 -- 172.21.15.5:0/1643556288 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c03f070 con 0x7f8f50109520 2024-04-24T06:47:31.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.574+0000 7f8f47400640 1 -- 172.21.15.5:0/1643556288 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f3c031c90 con 0x7f8f50109520 2024-04-24T06:47:31.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.574+0000 7f8f55400640 1 -- 172.21.15.5:0/1643556288 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f5019b710 con 0x7f8f50109520 2024-04-24T06:47:31.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.574+0000 7f8f47400640 1 -- 172.21.15.5:0/1643556288 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c035030 con 0x7f8f50109520 2024-04-24T06:47:31.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.574+0000 7f8f55400640 1 -- 172.21.15.5:0/1643556288 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f501a78b0 con 0x7f8f50109520 2024-04-24T06:47:31.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.578+0000 7f8f5821f740 1 Processor -- start 2024-04-24T06:47:31.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.578+0000 7f8f47400640 1 -- 172.21.15.5:0/1643556288 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f8f3c042070 con 0x7f8f50109520 2024-04-24T06:47:31.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.578+0000 7f8f5821f740 1 -- start start 2024-04-24T06:47:31.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.579+0000 7f8f5821f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b9978860 0x55e0b9978c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:31.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.579+0000 7f8f5821f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0b9979180 0x55e0b9983310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:31.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.579+0000 7f8f5821f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b9983850 0x55e0b9985c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:31.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.579+0000 7f8f5821f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e0b98de1f0 con 0x55e0b9979180 2024-04-24T06:47:31.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.579+0000 7f8f5821f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e0b9884350 con 0x55e0b9983850 2024-04-24T06:47:31.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.579+0000 7f8f5821f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e0b98da1b0 con 0x55e0b9978860 2024-04-24T06:47:31.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.579+0000 7f8f4f200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b9978860 0x55e0b9978c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:31.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.579+0000 7f8f4f200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b9978860 0x55e0b9978c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:53152/0 (socket says 172.21.15.5:53152) 2024-04-24T06:47:31.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.579+0000 7f8f4f200640 1 -- 172.21.15.5:0/2412872959 learned_addr learned my addr 172.21.15.5:0/2412872959 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:47:31.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.579+0000 7f8f4de00640 1 --2- 172.21.15.5:0/2412872959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b9983850 0x55e0b9985c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:31.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.579+0000 7f8f4e800640 1 --2- 172.21.15.5:0/2412872959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0b9979180 0x55e0b9983310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:31.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.580+0000 7f8f4f200640 1 -- 172.21.15.5:0/2412872959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b9983850 msgr2=0x55e0b9985c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:31.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.580+0000 7f8f4f200640 1 --2- 172.21.15.5:0/2412872959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b9983850 0x55e0b9985c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.580+0000 7f8f4f200640 1 -- 172.21.15.5:0/2412872959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0b9979180 msgr2=0x55e0b9983310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:31.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.580+0000 7f8f4f200640 1 --2- 172.21.15.5:0/2412872959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0b9979180 0x55e0b9983310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.580+0000 7f8f4f200640 1 -- 172.21.15.5:0/2412872959 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0b98d8f50 con 0x55e0b9978860 2024-04-24T06:47:31.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.580+0000 7f8f4f200640 1 --2- 172.21.15.5:0/2412872959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b9978860 0x55e0b9978c40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8f48009ee0 tx=0x7f8f4800f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5f045894b0ce22f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:47:31.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.580+0000 7f8f47400640 1 --2- 172.21.15.5:0/1643556288 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f8f340a3d20 0x7f8f340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:31.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.580+0000 7f8f3b400640 1 -- 172.21.15.5:0/2412872959 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f48011640 con 0x55e0b9978860 2024-04-24T06:47:31.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.581+0000 7f8f3b400640 1 -- 172.21.15.5:0/2412872959 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f48011cc0 con 0x55e0b9978860 2024-04-24T06:47:31.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.581+0000 7f8f3b400640 1 -- 172.21.15.5:0/2412872959 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4801be40 con 0x55e0b9978860 2024-04-24T06:47:31.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.581+0000 7f8f47400640 1 -- 172.21.15.5:0/1643556288 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f8f3c0f0bc0 con 0x7f8f50109520 2024-04-24T06:47:31.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.581+0000 7f8f4e800640 1 --2- 172.21.15.5:0/1643556288 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f8f340a3d20 0x7f8f340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:31.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.581+0000 7f8f5821f740 1 -- 172.21.15.5:0/2412872959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b9978860 msgr2=0x55e0b9978c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:31.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.581+0000 7f8f5821f740 1 --2- 172.21.15.5:0/2412872959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b9978860 0x55e0b9978c40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8f48009ee0 tx=0x7f8f4800f730 comp rx=0 tx=0).stop 2024-04-24T06:47:31.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.582+0000 7f8f5821f740 1 -- 172.21.15.5:0/2412872959 shutdown_connections 2024-04-24T06:47:31.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.582+0000 7f8f5821f740 1 --2- 172.21.15.5:0/2412872959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b9983850 0x55e0b9985c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.582+0000 7f8f5821f740 1 --2- 172.21.15.5:0/2412872959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0b9979180 0x55e0b9983310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.582+0000 7f8f5821f740 1 --2- 172.21.15.5:0/2412872959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b9978860 0x55e0b9978c40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.582+0000 7f8f5821f740 1 -- 172.21.15.5:0/2412872959 >> 172.21.15.5:0/2412872959 conn(0x55e0b9974b80 msgr2=0x55e0b9974f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:47:31.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.582+0000 7f8f4e800640 1 --2- 172.21.15.5:0/1643556288 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f8f340a3d20 0x7f8f340a61e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x55e0b9979e20 tx=0x7f8f40005f90 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:47:31.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.582+0000 7f8f5821f740 1 -- 172.21.15.5:0/2412872959 shutdown_connections 2024-04-24T06:47:31.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.582+0000 7f8f5821f740 1 -- 172.21.15.5:0/2412872959 wait complete. 2024-04-24T06:47:31.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.583+0000 7f8f5821f740 1 -- start start 2024-04-24T06:47:31.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.583+0000 7f8f5821f740 1 Processor -- start 2024-04-24T06:47:31.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.583+0000 7f8f5821f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b9978860 0x55e0b9980830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:31.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.583+0000 7f8f5821f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0b9979180 0x55e0b9980d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:31.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.583+0000 7f8f5821f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b9983850 0x55e0b99812b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:31.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.583+0000 7f8f5821f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e0b9986640 con 0x55e0b9979180 2024-04-24T06:47:31.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.583+0000 7f8f5821f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e0b9897690 con 0x55e0b9978860 2024-04-24T06:47:31.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.583+0000 7f8f5821f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e0b99867c0 con 0x55e0b9983850 2024-04-24T06:47:31.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.584+0000 7f8f5821f740 10 client.0 _ll_register_callbacks cb 0x55e0b98d3650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:47:31.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.584+0000 7f8f4e800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b9978860 0x55e0b9980830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:31.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.584+0000 7f8f4e800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b9978860 0x55e0b9980830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:48912/0 (socket says 172.21.15.5:48912) 2024-04-24T06:47:31.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.584+0000 7f8f4e800640 1 -- 172.21.15.5:0/669243835 learned_addr learned my addr 172.21.15.5:0/669243835 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:47:31.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.584+0000 7f8f4de00640 1 --2- 172.21.15.5:0/669243835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0b9979180 0x55e0b9980d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:31.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.584+0000 7f8f4f200640 1 --2- 172.21.15.5:0/669243835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b9983850 0x55e0b99812b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:31.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.584+0000 7f8f4e800640 1 -- 172.21.15.5:0/669243835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b9983850 msgr2=0x55e0b99812b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:31.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.584+0000 7f8f4e800640 1 --2- 172.21.15.5:0/669243835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b9983850 0x55e0b99812b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.584+0000 7f8f4e800640 1 -- 172.21.15.5:0/669243835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0b9979180 msgr2=0x55e0b9980d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:31.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.584+0000 7f8f4e800640 1 --2- 172.21.15.5:0/669243835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0b9979180 0x55e0b9980d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.584+0000 7f8f4e800640 1 -- 172.21.15.5:0/669243835 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0b9896920 con 0x55e0b9978860 2024-04-24T06:47:31.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.584+0000 7f8f4de00640 1 --2- 172.21.15.5:0/669243835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0b9979180 0x55e0b9980d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:47:31.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.585+0000 7f8f4e800640 1 --2- 172.21.15.5:0/669243835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b9978860 0x55e0b9980830 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000fb40 tx=0x7f8f40004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c5f5a819ee11ebf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:47:31.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.585+0000 7f8f2fe00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:47:31.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.585+0000 7f8f2fe00640 1 -- 172.21.15.5:0/669243835 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f40005020 con 0x55e0b9978860 2024-04-24T06:47:31.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.585+0000 7f8f2fe00640 1 -- 172.21.15.5:0/669243835 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f8f40018030 con 0x55e0b9978860 2024-04-24T06:47:31.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.585+0000 7f8f5821f740 1 -- 172.21.15.5:0/669243835 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f48011cc0 con 0x55e0b9978860 2024-04-24T06:47:31.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.585+0000 7f8f2fe00640 1 -- 172.21.15.5:0/669243835 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8f400051c0 con 0x55e0b9978860 2024-04-24T06:47:31.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.585+0000 7f8f5821f740 10 client.26102 fetch_fsmap learned FSMap version 25 2024-04-24T06:47:31.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.585+0000 7f8f5821f740 1 -- 172.21.15.5:0/669243835 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e0b96ba7d0 con 0x55e0b9978860 2024-04-24T06:47:31.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.586+0000 7f8f2fe00640 1 -- 172.21.15.5:0/669243835 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f48011cc0 con 0x55e0b9978860 2024-04-24T06:47:31.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.586+0000 7f8f2fe00640 1 -- 172.21.15.5:0/669243835 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8f4001e6e0 con 0x55e0b9978860 2024-04-24T06:47:31.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.586+0000 7f8f5821f740 10 client.26102 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:47:31.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.586+0000 7f8f5821f740 10 client.26102 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:47:31.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.586+0000 7f8f5821f740 20 client.26102 populate_metadata read hostname 'smithi005' 2024-04-24T06:47:31.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.586+0000 7f8f5821f740 1 --2- 172.21.15.5:0/669243835 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55e0b9993d20 0x55e0b99961a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:31.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.586+0000 7f8f5821f740 4 client.26102 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:47:31.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.586+0000 7f8f5821f740 1 -- 172.21.15.5:0/669243835 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0b98ebc10 con 0x55e0b9993d20 2024-04-24T06:47:31.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.586+0000 7f8f4de00640 1 --2- 172.21.15.5:0/669243835 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55e0b9993d20 0x55e0b99961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:31.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.587+0000 7f8f4de00640 1 --2- 172.21.15.5:0/669243835 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55e0b9993d20 0x55e0b99961a0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21452314fef0887f server_cookie=48f763462f50d5ad in_seq=0 out_seq=0 2024-04-24T06:47:31.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.587+0000 7f8f2fe00640 10 client.26102 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:47:31.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.588+0000 7f8f2fe00640 1 -- 172.21.15.5:0/669243835 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e0b98ebc10 con 0x55e0b9993d20 2024-04-24T06:47:31.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.588+0000 7f8f2fe00640 10 client.26102 handle_command_reply: tid=0 2024-04-24T06:47:31.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.588+0000 7f8f5821f740 1 client.26102 shutdown 2024-04-24T06:47:31.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.588+0000 7f8f5821f740 1 -- 172.21.15.5:0/669243835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b9978860 msgr2=0x55e0b9980830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:31.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.588+0000 7f8f5821f740 1 --2- 172.21.15.5:0/669243835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b9978860 0x55e0b9980830 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000fb40 tx=0x7f8f40004730 comp rx=0 tx=0).stop 2024-04-24T06:47:31.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.588+0000 7f8f5821f740 1 -- 172.21.15.5:0/669243835 shutdown_connections 2024-04-24T06:47:31.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.588+0000 7f8f5821f740 1 --2- 172.21.15.5:0/669243835 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55e0b9993d20 0x55e0b99961a0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.588+0000 7f8f5821f740 1 --2- 172.21.15.5:0/669243835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b9983850 0x55e0b99812b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.588+0000 7f8f5821f740 1 --2- 172.21.15.5:0/669243835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0b9979180 0x55e0b9980d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.588+0000 7f8f5821f740 1 --2- 172.21.15.5:0/669243835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b9978860 0x55e0b9980830 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.588+0000 7f8f5821f740 1 -- 172.21.15.5:0/669243835 >> 172.21.15.5:0/669243835 conn(0x55e0b9974b80 msgr2=0x55e0b997a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:47:31.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.588+0000 7f8f2fe00640 0 client.26102 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:47:31.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.589+0000 7f8f5821f740 1 -- 172.21.15.5:0/669243835 shutdown_connections 2024-04-24T06:47:31.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.589+0000 7f8f5821f740 1 -- 172.21.15.5:0/669243835 wait complete. 2024-04-24T06:47:31.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.589+0000 7f8f5821f740 20 client.26102 trim_cache size 0 max 16384 2024-04-24T06:47:31.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.613+0000 7f8f5821f740 1 Processor -- start 2024-04-24T06:47:31.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.613+0000 7f8f5821f740 1 -- start start 2024-04-24T06:47:31.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.613+0000 7f8f5821f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0b9978860 0x55e0b998f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:31.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.613+0000 7f8f5821f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b9979180 0x55e0b997b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:31.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.613+0000 7f8f5821f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b9983850 0x55e0b9973aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:31.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.613+0000 7f8f5821f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8f501b2820 con 0x55e0b9978860 2024-04-24T06:47:31.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.613+0000 7f8f5821f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8f501b00f0 con 0x55e0b9979180 2024-04-24T06:47:31.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.613+0000 7f8f5821f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8f501adf40 con 0x55e0b9983850 2024-04-24T06:47:31.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.614+0000 7f8f4f200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b9983850 0x55e0b9973aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:31.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.614+0000 7f8f4de00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b9979180 0x55e0b997b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:31.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.614+0000 7f8f4f200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b9983850 0x55e0b9973aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:53168/0 (socket says 172.21.15.5:53168) 2024-04-24T06:47:31.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.614+0000 7f8f4e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0b9978860 0x55e0b998f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:31.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.614+0000 7f8f4de00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b9979180 0x55e0b997b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:48922/0 (socket says 172.21.15.5:48922) 2024-04-24T06:47:31.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.614+0000 7f8f4f200640 1 -- 172.21.15.5:0/607689339 learned_addr learned my addr 172.21.15.5:0/607689339 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:47:31.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.614+0000 7f8f4f200640 1 -- 172.21.15.5:0/607689339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b9979180 msgr2=0x55e0b997b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:31.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.614+0000 7f8f4f200640 1 --2- 172.21.15.5:0/607689339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b9979180 0x55e0b997b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.614+0000 7f8f4f200640 1 -- 172.21.15.5:0/607689339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0b9978860 msgr2=0x55e0b998f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:31.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.614+0000 7f8f4f200640 1 --2- 172.21.15.5:0/607689339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0b9978860 0x55e0b998f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.614+0000 7f8f4f200640 1 -- 172.21.15.5:0/607689339 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0b95f03f0 con 0x55e0b9983850 2024-04-24T06:47:31.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.614+0000 7f8f4de00640 1 --2- 172.21.15.5:0/607689339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b9979180 0x55e0b997b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:47:31.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.615+0000 7f8f4f200640 1 --2- 172.21.15.5:0/607689339 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b9983850 0x55e0b9973aa0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8f48009ee0 tx=0x7f8f4801b9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=913ec1a6193e4c83 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:47:31.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.615+0000 7f8f3aa00640 1 -- 172.21.15.5:0/607689339 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f48028070 con 0x55e0b9983850 2024-04-24T06:47:31.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.615+0000 7f8f3aa00640 1 -- 172.21.15.5:0/607689339 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f4801bd10 con 0x55e0b9983850 2024-04-24T06:47:31.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.615+0000 7f8f3aa00640 1 -- 172.21.15.5:0/607689339 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f48005370 con 0x55e0b9983850 2024-04-24T06:47:31.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.616+0000 7f8f5821f740 1 -- 172.21.15.5:0/607689339 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b9983850 msgr2=0x55e0b9973aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:31.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.616+0000 7f8f5821f740 1 --2- 172.21.15.5:0/607689339 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b9983850 0x55e0b9973aa0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8f48009ee0 tx=0x7f8f4801b9e0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.616+0000 7f8f5821f740 1 -- 172.21.15.5:0/607689339 shutdown_connections 2024-04-24T06:47:31.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.616+0000 7f8f5821f740 1 --2- 172.21.15.5:0/607689339 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b9983850 0x55e0b9973aa0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.616+0000 7f8f5821f740 1 --2- 172.21.15.5:0/607689339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b9979180 0x55e0b997b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.616+0000 7f8f5821f740 1 --2- 172.21.15.5:0/607689339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0b9978860 0x55e0b998f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.616+0000 7f8f5821f740 1 -- 172.21.15.5:0/607689339 >> 172.21.15.5:0/607689339 conn(0x55e0b9974b80 msgr2=0x55e0b9973390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:47:31.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.616+0000 7f8f5821f740 1 -- 172.21.15.5:0/607689339 shutdown_connections 2024-04-24T06:47:31.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.616+0000 7f8f5821f740 1 -- 172.21.15.5:0/607689339 wait complete. 2024-04-24T06:47:31.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.617+0000 7f8f5821f740 1 -- start start 2024-04-24T06:47:31.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.617+0000 7f8f5821f740 1 Processor -- start 2024-04-24T06:47:31.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.617+0000 7f8f5821f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0b9978860 0x55e0b997d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:31.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.618+0000 7f8f5821f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b9979180 0x55e0b997dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:31.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.618+0000 7f8f5821f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b9983850 0x55e0b997e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:31.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.618+0000 7f8f5821f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8f501ae5b0 con 0x55e0b9978860 2024-04-24T06:47:31.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.618+0000 7f8f5821f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8f501afa20 con 0x55e0b9979180 2024-04-24T06:47:31.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.618+0000 7f8f5821f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8f501aec80 con 0x55e0b9983850 2024-04-24T06:47:31.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.618+0000 7f8f5821f740 10 client.0 _ll_register_callbacks cb 0x7f8f40009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:47:31.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.618+0000 7f8f4e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0b9978860 0x55e0b997d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:31.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.618+0000 7f8f4e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0b9978860 0x55e0b997d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41660/0 (socket says 172.21.15.5:41660) 2024-04-24T06:47:31.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.618+0000 7f8f4e800640 1 -- 172.21.15.5:0/1741802404 learned_addr learned my addr 172.21.15.5:0/1741802404 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:47:31.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.618+0000 7f8f4de00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b9979180 0x55e0b997dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:31.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.618+0000 7f8f4f200640 1 --2- 172.21.15.5:0/1741802404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b9983850 0x55e0b997e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:31.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.618+0000 7f8f4de00640 1 -- 172.21.15.5:0/1741802404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b9983850 msgr2=0x55e0b997e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:31.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.618+0000 7f8f4de00640 1 --2- 172.21.15.5:0/1741802404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b9983850 0x55e0b997e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.618+0000 7f8f4de00640 1 -- 172.21.15.5:0/1741802404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0b9978860 msgr2=0x55e0b997d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:47:31.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.618+0000 7f8f4de00640 1 --2- 172.21.15.5:0/1741802404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0b9978860 0x55e0b997d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.618+0000 7f8f4de00640 1 -- 172.21.15.5:0/1741802404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0b95b00f0 con 0x55e0b9979180 2024-04-24T06:47:31.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.619+0000 7f8f4de00640 1 --2- 172.21.15.5:0/1741802404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b9979180 0x55e0b997dc60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c0ee2c0 tx=0x7f8f3c02a400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4883c03a61a8e01 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:47:31.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.619+0000 7f8f2fe00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:47:31.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.619+0000 7f8f2fe00640 1 -- 172.21.15.5:0/1741802404 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c049040 con 0x55e0b9979180 2024-04-24T06:47:31.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.619+0000 7f8f2fe00640 1 -- 172.21.15.5:0/1741802404 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f8f3c0dd070 con 0x55e0b9979180 2024-04-24T06:47:31.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.619+0000 7f8f5821f740 1 -- 172.21.15.5:0/1741802404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f4801bd10 con 0x55e0b9979180 2024-04-24T06:47:31.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.619+0000 7f8f2fe00640 1 -- 172.21.15.5:0/1741802404 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8f3c031e40 con 0x55e0b9979180 2024-04-24T06:47:31.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.619+0000 7f8f5821f740 10 client.26108 fetch_fsmap learned FSMap version 25 2024-04-24T06:47:31.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.619+0000 7f8f5821f740 1 -- 172.21.15.5:0/1741802404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e0b98a3810 con 0x55e0b9979180 2024-04-24T06:47:31.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.620+0000 7f8f2fe00640 1 -- 172.21.15.5:0/1741802404 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f4801bd10 con 0x55e0b9979180 2024-04-24T06:47:31.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.620+0000 7f8f2fe00640 1 -- 172.21.15.5:0/1741802404 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8f3c0b9180 con 0x55e0b9979180 2024-04-24T06:47:31.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.620+0000 7f8f5821f740 10 client.26108 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:47:31.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.620+0000 7f8f5821f740 10 client.26108 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:47:31.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.620+0000 7f8f5821f740 20 client.26108 populate_metadata read hostname 'smithi005' 2024-04-24T06:47:31.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.620+0000 7f8f5821f740 1 --2- 172.21.15.5:0/1741802404 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55e0b99faa60 0x55e0b99fce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:47:31.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.620+0000 7f8f5821f740 4 client.26108 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:47:31.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.620+0000 7f8f5821f740 1 -- 172.21.15.5:0/1741802404 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e0b98e5a80 con 0x55e0b99faa60 2024-04-24T06:47:31.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.620+0000 7f8f4e800640 1 --2- 172.21.15.5:0/1741802404 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55e0b99faa60 0x55e0b99fce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:47:31.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.621+0000 7f8f4e800640 1 --2- 172.21.15.5:0/1741802404 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55e0b99faa60 0x55e0b99fce50 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89168f672be980ef server_cookie=1c6b66932d444 in_seq=0 out_seq=0 2024-04-24T06:47:31.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.621+0000 7f8f2fe00640 10 client.26108 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:47:31.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.622+0000 7f8f2fe00640 1 -- 172.21.15.5:0/1741802404 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55e0b98e5a80 con 0x55e0b99faa60 2024-04-24T06:47:31.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.622+0000 7f8f2fe00640 10 client.26108 handle_command_reply: tid=0 2024-04-24T06:47:31.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.622+0000 7f8f5821f740 1 client.26108 shutdown 2024-04-24T06:47:31.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.622+0000 7f8f5821f740 1 -- 172.21.15.5:0/1741802404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b9979180 msgr2=0x55e0b997dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:31.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.622+0000 7f8f5821f740 1 --2- 172.21.15.5:0/1741802404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b9979180 0x55e0b997dc60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c0ee2c0 tx=0x7f8f3c02a400 comp rx=0 tx=0).stop 2024-04-24T06:47:31.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.622+0000 7f8f5821f740 1 -- 172.21.15.5:0/1741802404 shutdown_connections 2024-04-24T06:47:31.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.622+0000 7f8f5821f740 1 --2- 172.21.15.5:0/1741802404 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55e0b99faa60 0x55e0b99fce50 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.622+0000 7f8f5821f740 1 --2- 172.21.15.5:0/1741802404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b9983850 0x55e0b997e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.622+0000 7f8f5821f740 1 --2- 172.21.15.5:0/1741802404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b9979180 0x55e0b997dc60 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.622+0000 7f8f5821f740 1 --2- 172.21.15.5:0/1741802404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0b9978860 0x55e0b997d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.622+0000 7f8f5821f740 1 -- 172.21.15.5:0/1741802404 >> 172.21.15.5:0/1741802404 conn(0x55e0b9974b80 msgr2=0x55e0b9973c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:47:31.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.622+0000 7f8f2fe00640 0 client.26108 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:47:31.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.622+0000 7f8f5821f740 1 -- 172.21.15.5:0/1741802404 shutdown_connections 2024-04-24T06:47:31.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.622+0000 7f8f5821f740 1 -- 172.21.15.5:0/1741802404 wait complete. 2024-04-24T06:47:31.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.623+0000 7f8f5821f740 20 client.26108 trim_cache size 0 max 16384 2024-04-24T06:47:31.630 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:47:31.630 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (872 inodes in the stack)", 2024-04-24T06:47:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-24T06:47:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: "04e0297c-9b5f-4cb3-af15-2af54f2aca0e": { 2024-04-24T06:47:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-24T06:47:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "04e0297c-9b5f-4cb3-af15-2af54f2aca0e", 2024-04-24T06:47:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-04-24T06:47:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:47:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:47:31.631 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:47:31.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.624+0000 7f8f55400640 1 -- 172.21.15.5:0/1643556288 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f8f340a3d20 msgr2=0x7f8f340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:31.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.624+0000 7f8f55400640 1 --2- 172.21.15.5:0/1643556288 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f8f340a3d20 0x7f8f340a61e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x55e0b9979e20 tx=0x7f8f40005f90 comp rx=0 tx=0).stop 2024-04-24T06:47:31.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.624+0000 7f8f55400640 1 -- 172.21.15.5:0/1643556288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f50109520 msgr2=0x7f8f501a0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:47:31.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.624+0000 7f8f55400640 1 --2- 172.21.15.5:0/1643556288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f50109520 0x7f8f501a0ca0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c02a920 tx=0x7f8f3c003b20 comp rx=0 tx=0).stop 2024-04-24T06:47:31.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.624+0000 7f8f55400640 1 -- 172.21.15.5:0/1643556288 shutdown_connections 2024-04-24T06:47:31.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.624+0000 7f8f55400640 1 --2- 172.21.15.5:0/1643556288 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f8f340a3d20 0x7f8f340a61e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.624+0000 7f8f55400640 1 --2- 172.21.15.5:0/1643556288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f5019a960 0x7f8f5019ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.624+0000 7f8f55400640 1 --2- 172.21.15.5:0/1643556288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f50109520 0x7f8f501a0ca0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.624+0000 7f8f55400640 1 --2- 172.21.15.5:0/1643556288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f50104450 0x7f8f501a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:47:31.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.624+0000 7f8f55400640 1 -- 172.21.15.5:0/1643556288 >> 172.21.15.5:0/1643556288 conn(0x7f8f500ffc90 msgr2=0x7f8f50102010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:47:31.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.624+0000 7f8f55400640 1 -- 172.21.15.5:0/1643556288 shutdown_connections 2024-04-24T06:47:31.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:47:31.624+0000 7f8f55400640 1 -- 172.21.15.5:0/1643556288 wait complete. 2024-04-24T06:47:31.645 INFO:tasks.cephfs.filesystem:scrub status for tag:04e0297c-9b5f-4cb3-af15-2af54f2aca0e - {'path': '/', 'tag': '04e0297c-9b5f-4cb3-af15-2af54f2aca0e', 'options': 'recursive,force'} 2024-04-24T06:47:31.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:31 smithi137 ceph-mon[33017]: pgmap v107: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-24T06:47:31.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:31 smithi005 ceph-mon[27798]: pgmap v107: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-24T06:47:31.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:31 smithi037 ceph-mon[32864]: pgmap v107: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-24T06:47:32.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:32 smithi005 ceph-mon[27798]: scrub queued for path: / 2024-04-24T06:47:32.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:32 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:47:32.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:32 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:47:32.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:32 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:47:32.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:32 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:47:32.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:32 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:47:32.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:32 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:47:32.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:32 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:47:32.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:32 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:47:32.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:32 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:47:32.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:32 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:47:32.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:32 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:47:32.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:32 smithi037 ceph-mon[32864]: scrub queued for path: / 2024-04-24T06:47:32.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:32 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:47:32.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:32 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:47:32.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:32 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:47:32.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:32 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:47:32.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:32 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:47:32.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:32 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:47:32.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:32 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:47:32.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:32 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:47:32.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:32 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:47:32.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:32 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:47:32.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:32 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:47:32.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:32 smithi137 ceph-mon[33017]: scrub queued for path: / 2024-04-24T06:47:32.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:32 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:47:32.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:32 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:47:32.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:32 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:47:32.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:32 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:47:32.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:32 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:47:32.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:32 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:47:32.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:32 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:47:32.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:32 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:47:32.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:32 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:47:32.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:32 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:47:32.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:32 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:47:33.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:33 smithi005 ceph-mon[27798]: pgmap v108: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-24T06:47:33.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:33 smithi037 ceph-mon[32864]: pgmap v108: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-24T06:47:33.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:33 smithi137 ceph-mon[33017]: pgmap v108: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-24T06:47:35.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:35 smithi005 ceph-mon[27798]: pgmap v109: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-24T06:47:35.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:35 smithi037 ceph-mon[32864]: pgmap v109: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-24T06:47:35.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:35 smithi137 ceph-mon[33017]: pgmap v109: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-24T06:47:37.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:37 smithi005 ceph-mon[27798]: pgmap v110: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-24T06:47:37.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:37 smithi037 ceph-mon[32864]: pgmap v110: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-24T06:47:37.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:37 smithi137 ceph-mon[33017]: pgmap v110: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-24T06:47:39.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:39 smithi005 ceph-mon[27798]: pgmap v111: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-24T06:47:39.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:39 smithi037 ceph-mon[32864]: pgmap v111: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-24T06:47:39.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:39 smithi137 ceph-mon[33017]: pgmap v111: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-24T06:47:41.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:41 smithi005 ceph-mon[27798]: pgmap v112: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-24T06:47:41.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:41 smithi037 ceph-mon[32864]: pgmap v112: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-24T06:47:41.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:41 smithi137 ceph-mon[33017]: pgmap v112: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-24T06:47:43.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:43 smithi005 ceph-mon[27798]: pgmap v113: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-04-24T06:47:43.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:43 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:47:43.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:43 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:47:43.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:43 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:47:43.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:43 smithi037 ceph-mon[32864]: pgmap v113: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-04-24T06:47:43.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:43 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:47:43.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:43 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:47:43.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:43 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:47:43.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:43 smithi137 ceph-mon[33017]: pgmap v113: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-04-24T06:47:43.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:43 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:47:43.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:43 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:47:43.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:43 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:47:45.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:45 smithi005 ceph-mon[27798]: pgmap v114: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 7 op/s 2024-04-24T06:47:45.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:45 smithi037 ceph-mon[32864]: pgmap v114: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 7 op/s 2024-04-24T06:47:45.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:45 smithi137 ceph-mon[33017]: pgmap v114: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 7 op/s 2024-04-24T06:47:47.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:47 smithi005 ceph-mon[27798]: pgmap v115: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:47:47.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:47 smithi037 ceph-mon[32864]: pgmap v115: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:47:47.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:47 smithi137 ceph-mon[33017]: pgmap v115: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:47:49.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:49 smithi005 ceph-mon[27798]: pgmap v116: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:47:49.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:49 smithi037 ceph-mon[32864]: pgmap v116: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:47:49.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:49 smithi137 ceph-mon[33017]: pgmap v116: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:47:51.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:51 smithi005 ceph-mon[27798]: pgmap v117: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T06:47:51.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:51 smithi037 ceph-mon[32864]: pgmap v117: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T06:47:51.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:51 smithi137 ceph-mon[33017]: pgmap v117: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T06:47:53.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:53 smithi005 ceph-mon[27798]: pgmap v118: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:47:53.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:53 smithi037 ceph-mon[32864]: pgmap v118: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:47:53.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:53 smithi137 ceph-mon[33017]: pgmap v118: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:47:55.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:55 smithi005 ceph-mon[27798]: pgmap v119: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:47:55.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:55 smithi037 ceph-mon[32864]: pgmap v119: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:47:55.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:55 smithi137 ceph-mon[33017]: pgmap v119: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:47:57.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:57 smithi037 ceph-mon[32864]: pgmap v120: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:47:57.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:57 smithi137 ceph-mon[33017]: pgmap v120: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:47:58.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:57 smithi005 ceph-mon[27798]: pgmap v120: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:47:59.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:47:59 smithi037 ceph-mon[32864]: pgmap v121: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:47:59.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:47:59 smithi137 ceph-mon[33017]: pgmap v121: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:48:00.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:47:59 smithi005 ceph-mon[27798]: pgmap v121: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:48:01.646 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T06:48:01.646 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T06:48:01.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.871+0000 7f25f0c00640 1 -- 172.21.15.5:0/767298278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25ec104260 msgr2=0x7f25ec104640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:01.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.871+0000 7f25f0c00640 1 --2- 172.21.15.5:0/767298278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25ec104260 0x7f25ec104640 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f25dc009900 tx=0x7f25dc02a3f0 comp rx=0 tx=0).stop 2024-04-24T06:48:01.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.871+0000 7f25f0c00640 1 -- 172.21.15.5:0/767298278 shutdown_connections 2024-04-24T06:48:01.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.871+0000 7f25f0c00640 1 --2- 172.21.15.5:0/767298278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25ec109330 0x7f25ec110f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:01.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.871+0000 7f25f0c00640 1 --2- 172.21.15.5:0/767298278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25ec104c10 0x7f25ec108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:01.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.871+0000 7f25f0c00640 1 --2- 172.21.15.5:0/767298278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25ec104260 0x7f25ec104640 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:01.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.871+0000 7f25f0c00640 1 -- 172.21.15.5:0/767298278 >> 172.21.15.5:0/767298278 conn(0x7f25ec0ffaa0 msgr2=0x7f25ec101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:01.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.871+0000 7f25f0c00640 1 -- 172.21.15.5:0/767298278 shutdown_connections 2024-04-24T06:48:01.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.871+0000 7f25f0c00640 1 -- 172.21.15.5:0/767298278 wait complete. 2024-04-24T06:48:01.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.872+0000 7f25f0c00640 1 Processor -- start 2024-04-24T06:48:01.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.872+0000 7f25f0c00640 1 -- start start 2024-04-24T06:48:01.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.872+0000 7f25f0c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25ec104260 0x7f25ec1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:01.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.872+0000 7f25f0c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25ec109330 0x7f25ec1a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:01.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.872+0000 7f25f0c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25ec19a830 0x7f25ec19ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:01.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.873+0000 7f25f0c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec114410 con 0x7f25ec104260 2024-04-24T06:48:01.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.873+0000 7f25f0c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec114290 con 0x7f25ec19a830 2024-04-24T06:48:01.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.873+0000 7f25f0c00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec114590 con 0x7f25ec109330 2024-04-24T06:48:01.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.873+0000 7f25eb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25ec104260 0x7f25ec1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:01.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.873+0000 7f25eaa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25ec109330 0x7f25ec1a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:01.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.873+0000 7f25ebe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25ec19a830 0x7f25ec19ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:01.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.873+0000 7f25eb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25ec104260 0x7f25ec1a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58916/0 (socket says 172.21.15.5:58916) 2024-04-24T06:48:01.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.873+0000 7f25eb400640 1 -- 172.21.15.5:0/1965378193 learned_addr learned my addr 172.21.15.5:0/1965378193 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:01.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.873+0000 7f25eaa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25ec109330 0x7f25ec1a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:54212/0 (socket says 172.21.15.5:54212) 2024-04-24T06:48:01.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.873+0000 7f25ebe00640 1 -- 172.21.15.5:0/1965378193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25ec109330 msgr2=0x7f25ec1a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:01.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.873+0000 7f25ebe00640 1 --2- 172.21.15.5:0/1965378193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25ec109330 0x7f25ec1a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:01.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.873+0000 7f25ebe00640 1 -- 172.21.15.5:0/1965378193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25ec104260 msgr2=0x7f25ec1a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:01.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.873+0000 7f25ebe00640 1 --2- 172.21.15.5:0/1965378193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25ec104260 0x7f25ec1a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:01.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.873+0000 7f25ebe00640 1 -- 172.21.15.5:0/1965378193 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25ec19b3c0 con 0x7f25ec19a830 2024-04-24T06:48:01.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.873+0000 7f25eb400640 1 --2- 172.21.15.5:0/1965378193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25ec104260 0x7f25ec1a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:48:01.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.874+0000 7f25ebe00640 1 --2- 172.21.15.5:0/1965378193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25ec19a830 0x7f25ec19ace0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f25e400da10 tx=0x7f25e400ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:01.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.874+0000 7f25e3e00640 1 -- 172.21.15.5:0/1965378193 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e400bd70 con 0x7f25ec19a830 2024-04-24T06:48:01.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.874+0000 7f25f0c00640 1 -- 172.21.15.5:0/1965378193 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25ec19b6b0 con 0x7f25ec19a830 2024-04-24T06:48:01.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.874+0000 7f25e3e00640 1 -- 172.21.15.5:0/1965378193 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25e4003e50 con 0x7f25ec19a830 2024-04-24T06:48:01.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.874+0000 7f25e3e00640 1 -- 172.21.15.5:0/1965378193 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e40106a0 con 0x7f25ec19a830 2024-04-24T06:48:01.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.874+0000 7f25f0c00640 1 -- 172.21.15.5:0/1965378193 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25ec1a77f0 con 0x7f25ec19a830 2024-04-24T06:48:01.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.877+0000 7f25f5035740 1 Processor -- start 2024-04-24T06:48:01.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.878+0000 7f25e3e00640 1 -- 172.21.15.5:0/1965378193 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f25e4003970 con 0x7f25ec19a830 2024-04-24T06:48:01.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.878+0000 7f25f5035740 1 -- start start 2024-04-24T06:48:01.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.878+0000 7f25f5035740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfd42c2a30 0x55dfd42c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:01.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.878+0000 7f25f5035740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd42c3350 0x55dfd42cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:01.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.878+0000 7f25f5035740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd42cda20 0x55dfd42cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:01.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.878+0000 7f25f5035740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dfd42281f0 con 0x55dfd42c3350 2024-04-24T06:48:01.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.878+0000 7f25f5035740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55dfd41ce350 con 0x55dfd42c2a30 2024-04-24T06:48:01.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.878+0000 7f25f5035740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55dfd42241b0 con 0x55dfd42cda20 2024-04-24T06:48:01.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.879+0000 7f25eaa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfd42c2a30 0x55dfd42c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:01.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.879+0000 7f25eaa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfd42c2a30 0x55dfd42c2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:38170/0 (socket says 172.21.15.5:38170) 2024-04-24T06:48:01.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.879+0000 7f25eaa00640 1 -- 172.21.15.5:0/3658046655 learned_addr learned my addr 172.21.15.5:0/3658046655 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:01.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.879+0000 7f25eb400640 1 --2- 172.21.15.5:0/3658046655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd42c3350 0x55dfd42cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:01.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.879+0000 7f25eaa00640 1 --2- 172.21.15.5:0/3658046655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd42cda20 0x55dfd42cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:01.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.879+0000 7f25eaa00640 1 -- 172.21.15.5:0/3658046655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd42cda20 msgr2=0x55dfd42cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:01.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.879+0000 7f25eaa00640 1 --2- 172.21.15.5:0/3658046655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd42cda20 0x55dfd42cfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:01.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.879+0000 7f25eaa00640 1 -- 172.21.15.5:0/3658046655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd42c3350 msgr2=0x55dfd42cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:01.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.879+0000 7f25eaa00640 1 --2- 172.21.15.5:0/3658046655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd42c3350 0x55dfd42cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:01.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.879+0000 7f25eaa00640 1 -- 172.21.15.5:0/3658046655 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfd4222f50 con 0x55dfd42c2a30 2024-04-24T06:48:01.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.879+0000 7f25e3e00640 1 --2- 172.21.15.5:0/1965378193 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f25d00a3d20 0x7f25d00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:01.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.879+0000 7f25eaa00640 1 --2- 172.21.15.5:0/3658046655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfd42c2a30 0x55dfd42c2e10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f25d8011710 tx=0x7f25d8005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6eb4ba8a7bdd09d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:01.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.880+0000 7f25e3e00640 1 -- 172.21.15.5:0/1965378193 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f25e40d0b70 con 0x7f25ec19a830 2024-04-24T06:48:01.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.880+0000 7f25eb400640 1 --2- 172.21.15.5:0/1965378193 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f25d00a3d20 0x7f25d00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:01.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.880+0000 7f25d7e00640 1 -- 172.21.15.5:0/3658046655 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d800f840 con 0x55dfd42c2a30 2024-04-24T06:48:01.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.880+0000 7f25d7e00640 1 -- 172.21.15.5:0/3658046655 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25d801f440 con 0x55dfd42c2a30 2024-04-24T06:48:01.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.880+0000 7f25d7e00640 1 -- 172.21.15.5:0/3658046655 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d800e610 con 0x55dfd42c2a30 2024-04-24T06:48:01.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.880+0000 7f25f5035740 1 -- 172.21.15.5:0/3658046655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfd42c2a30 msgr2=0x55dfd42c2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:01.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.880+0000 7f25f5035740 1 --2- 172.21.15.5:0/3658046655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfd42c2a30 0x55dfd42c2e10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f25d8011710 tx=0x7f25d8005850 comp rx=0 tx=0).stop 2024-04-24T06:48:01.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.881+0000 7f25eb400640 1 --2- 172.21.15.5:0/1965378193 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f25d00a3d20 0x7f25d00a61e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f25dc0096c0 tx=0x7f25dc0046c0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:01.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.881+0000 7f25f5035740 1 -- 172.21.15.5:0/3658046655 shutdown_connections 2024-04-24T06:48:01.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.881+0000 7f25f5035740 1 --2- 172.21.15.5:0/3658046655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd42cda20 0x55dfd42cfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:01.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.881+0000 7f25f5035740 1 --2- 172.21.15.5:0/3658046655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd42c3350 0x55dfd42cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:01.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.881+0000 7f25f5035740 1 --2- 172.21.15.5:0/3658046655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfd42c2a30 0x55dfd42c2e10 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:01.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.881+0000 7f25f5035740 1 -- 172.21.15.5:0/3658046655 >> 172.21.15.5:0/3658046655 conn(0x55dfd42bed50 msgr2=0x55dfd42bf130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:01.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.881+0000 7f25f5035740 1 -- 172.21.15.5:0/3658046655 shutdown_connections 2024-04-24T06:48:01.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.881+0000 7f25f5035740 1 -- 172.21.15.5:0/3658046655 wait complete. 2024-04-24T06:48:01.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.881+0000 7f25f5035740 1 -- start start 2024-04-24T06:48:01.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.881+0000 7f25f5035740 1 Processor -- start 2024-04-24T06:48:01.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.882+0000 7f25f5035740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfd42c2a30 0x55dfd42ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:01.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.882+0000 7f25f5035740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd42c3350 0x55dfd42caf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:01.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.882+0000 7f25f5035740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd42cda20 0x55dfd42cb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:01.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.882+0000 7f25f5035740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dfd42d0810 con 0x55dfd42cda20 2024-04-24T06:48:01.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.882+0000 7f25f5035740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55dfd41e1690 con 0x55dfd42c2a30 2024-04-24T06:48:01.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.882+0000 7f25f5035740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55dfd42d0990 con 0x55dfd42c3350 2024-04-24T06:48:01.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.882+0000 7f25f5035740 10 client.0 _ll_register_callbacks cb 0x55dfd421d650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:48:01.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.882+0000 7f25ebe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd42cda20 0x55dfd42cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:01.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.882+0000 7f25eb400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfd42c2a30 0x55dfd42ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:01.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.882+0000 7f25ebe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd42cda20 0x55dfd42cb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58932/0 (socket says 172.21.15.5:58932) 2024-04-24T06:48:01.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.882+0000 7f25eb400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfd42c2a30 0x55dfd42ca9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:38174/0 (socket says 172.21.15.5:38174) 2024-04-24T06:48:01.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.882+0000 7f25ebe00640 1 -- 172.21.15.5:0/2242144763 learned_addr learned my addr 172.21.15.5:0/2242144763 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:01.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.882+0000 7f25eaa00640 1 --2- 172.21.15.5:0/2242144763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd42c3350 0x55dfd42caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:01.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.883+0000 7f25ebe00640 1 -- 172.21.15.5:0/2242144763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd42c3350 msgr2=0x55dfd42caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:01.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.883+0000 7f25ebe00640 1 --2- 172.21.15.5:0/2242144763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd42c3350 0x55dfd42caf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:01.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.883+0000 7f25ebe00640 1 -- 172.21.15.5:0/2242144763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfd42c2a30 msgr2=0x55dfd42ca9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:01.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.883+0000 7f25ebe00640 1 --2- 172.21.15.5:0/2242144763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfd42c2a30 0x55dfd42ca9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:01.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.883+0000 7f25ebe00640 1 -- 172.21.15.5:0/2242144763 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfd41e0920 con 0x55dfd42cda20 2024-04-24T06:48:01.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.883+0000 7f25eb400640 1 --2- 172.21.15.5:0/2242144763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfd42c2a30 0x55dfd42ca9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:48:01.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.883+0000 7f25eaa00640 1 --2- 172.21.15.5:0/2242144763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd42c3350 0x55dfd42caf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:48:01.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.883+0000 7f25ebe00640 1 --2- 172.21.15.5:0/2242144763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd42cda20 0x55dfd42cb440 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f25e409c7c0 tx=0x7f25e409d280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf7d3aca6f4b6fa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:01.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.883+0000 7f25d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:48:01.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.883+0000 7f25d4c00640 1 -- 172.21.15.5:0/2242144763 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e40a5730 con 0x55dfd42cda20 2024-04-24T06:48:01.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.883+0000 7f25d4c00640 1 -- 172.21.15.5:0/2242144763 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f25e40042e0 con 0x55dfd42cda20 2024-04-24T06:48:01.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.883+0000 7f25f5035740 1 -- 172.21.15.5:0/2242144763 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25d801f440 con 0x55dfd42cda20 2024-04-24T06:48:01.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.883+0000 7f25d4c00640 1 -- 172.21.15.5:0/2242144763 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f25e4003c60 con 0x55dfd42cda20 2024-04-24T06:48:01.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.884+0000 7f25f5035740 10 client.15990 fetch_fsmap learned FSMap version 25 2024-04-24T06:48:01.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.884+0000 7f25f5035740 1 -- 172.21.15.5:0/2242144763 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dfd40047d0 con 0x55dfd42cda20 2024-04-24T06:48:01.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.884+0000 7f25d4c00640 1 -- 172.21.15.5:0/2242144763 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25d801f440 con 0x55dfd42cda20 2024-04-24T06:48:01.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.884+0000 7f25d4c00640 1 -- 172.21.15.5:0/2242144763 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f25e4024060 con 0x55dfd42cda20 2024-04-24T06:48:01.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.884+0000 7f25f5035740 10 client.15990 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:48:01.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.884+0000 7f25f5035740 10 client.15990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:48:01.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.884+0000 7f25f5035740 20 client.15990 populate_metadata read hostname 'smithi005' 2024-04-24T06:48:01.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.884+0000 7f25f5035740 1 --2- 172.21.15.5:0/2242144763 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55dfd42ddea0 0x55dfd42e02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:01.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.884+0000 7f25f5035740 4 client.15990 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:48:01.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.884+0000 7f25f5035740 1 -- 172.21.15.5:0/2242144763 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfd4235c10 con 0x55dfd42ddea0 2024-04-24T06:48:01.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.885+0000 7f25eb400640 1 --2- 172.21.15.5:0/2242144763 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55dfd42ddea0 0x55dfd42e02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:01.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.885+0000 7f25eb400640 1 --2- 172.21.15.5:0/2242144763 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55dfd42ddea0 0x55dfd42e02d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c9651790f25cd22 server_cookie=ef7ae6409d5d57ed in_seq=0 out_seq=0 2024-04-24T06:48:01.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.885+0000 7f25d4c00640 10 client.15990 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:48:01.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.886+0000 7f25d4c00640 1 -- 172.21.15.5:0/2242144763 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dfd4235c10 con 0x55dfd42ddea0 2024-04-24T06:48:01.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.886+0000 7f25d4c00640 10 client.15990 handle_command_reply: tid=0 2024-04-24T06:48:01.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.887+0000 7f25f5035740 1 client.15990 shutdown 2024-04-24T06:48:01.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.887+0000 7f25f5035740 1 -- 172.21.15.5:0/2242144763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd42cda20 msgr2=0x55dfd42cb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:01.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.887+0000 7f25f5035740 1 --2- 172.21.15.5:0/2242144763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd42cda20 0x55dfd42cb440 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f25e409c7c0 tx=0x7f25e409d280 comp rx=0 tx=0).stop 2024-04-24T06:48:01.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.887+0000 7f25f5035740 1 -- 172.21.15.5:0/2242144763 shutdown_connections 2024-04-24T06:48:01.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.887+0000 7f25f5035740 1 --2- 172.21.15.5:0/2242144763 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55dfd42ddea0 0x55dfd42e02d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:01.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.887+0000 7f25f5035740 1 --2- 172.21.15.5:0/2242144763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd42cda20 0x55dfd42cb440 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:01.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.887+0000 7f25f5035740 1 --2- 172.21.15.5:0/2242144763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd42c3350 0x55dfd42caf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:01.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.887+0000 7f25f5035740 1 --2- 172.21.15.5:0/2242144763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfd42c2a30 0x55dfd42ca9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:01.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.887+0000 7f25d4c00640 0 client.15990 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:48:01.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.887+0000 7f25f5035740 1 -- 172.21.15.5:0/2242144763 >> 172.21.15.5:0/2242144763 conn(0x55dfd42bed50 msgr2=0x55dfd42bdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:01.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.887+0000 7f25f5035740 1 -- 172.21.15.5:0/2242144763 shutdown_connections 2024-04-24T06:48:01.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.887+0000 7f25f5035740 1 -- 172.21.15.5:0/2242144763 wait complete. 2024-04-24T06:48:01.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.887+0000 7f25f5035740 20 client.15990 trim_cache size 0 max 16384 2024-04-24T06:48:01.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.907+0000 7f25f5035740 1 Processor -- start 2024-04-24T06:48:01.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.907+0000 7f25f5035740 1 -- start start 2024-04-24T06:48:01.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.907+0000 7f25f5035740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd42c2a30 0x55dfd42d9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:01.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.907+0000 7f25f5035740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd42c3350 0x55dfd42b9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:01.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.907+0000 7f25f5035740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfd42cda20 0x55dfd42bf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:01.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.907+0000 7f25f5035740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec1b26a0 con 0x55dfd42c3350 2024-04-24T06:48:01.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.907+0000 7f25f5035740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec1aff70 con 0x55dfd42cda20 2024-04-24T06:48:01.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.907+0000 7f25f5035740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec1addc0 con 0x55dfd42c2a30 2024-04-24T06:48:01.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.907+0000 7f25eaa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd42c3350 0x55dfd42b9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:01.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.908+0000 7f25eaa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd42c3350 0x55dfd42b9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58936/0 (socket says 172.21.15.5:58936) 2024-04-24T06:48:01.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.908+0000 7f25eaa00640 1 -- 172.21.15.5:0/580173481 learned_addr learned my addr 172.21.15.5:0/580173481 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:01.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.908+0000 7f25eb400640 1 --2- 172.21.15.5:0/580173481 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd42c2a30 0x55dfd42d9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:01.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.908+0000 7f25ebe00640 1 --2- 172.21.15.5:0/580173481 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfd42cda20 0x55dfd42bf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:01.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.908+0000 7f25eaa00640 1 -- 172.21.15.5:0/580173481 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd42c2a30 msgr2=0x55dfd42d9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:01.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.908+0000 7f25eaa00640 1 --2- 172.21.15.5:0/580173481 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd42c2a30 0x55dfd42d9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:01.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.908+0000 7f25eaa00640 1 -- 172.21.15.5:0/580173481 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfd42cda20 msgr2=0x55dfd42bf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:01.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.908+0000 7f25eaa00640 1 --2- 172.21.15.5:0/580173481 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfd42cda20 0x55dfd42bf130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:01.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.908+0000 7f25eaa00640 1 -- 172.21.15.5:0/580173481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfd42c3180 con 0x55dfd42c3350 2024-04-24T06:48:01.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.908+0000 7f25ebe00640 1 --2- 172.21.15.5:0/580173481 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfd42cda20 0x55dfd42bf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:48:01.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.908+0000 7f25eb400640 1 --2- 172.21.15.5:0/580173481 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd42c2a30 0x55dfd42d9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:48:01.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.908+0000 7f25eaa00640 1 --2- 172.21.15.5:0/580173481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd42c3350 0x55dfd42b9e60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f25d80116e0 tx=0x7f25d8005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f9e4cc3a543497 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:01.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.909+0000 7f25d7400640 1 -- 172.21.15.5:0/580173481 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d802c070 con 0x55dfd42c3350 2024-04-24T06:48:01.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.909+0000 7f25d7400640 1 -- 172.21.15.5:0/580173481 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25d801f440 con 0x55dfd42c3350 2024-04-24T06:48:01.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.909+0000 7f25d7400640 1 -- 172.21.15.5:0/580173481 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d800faa0 con 0x55dfd42c3350 2024-04-24T06:48:01.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.909+0000 7f25f5035740 1 -- 172.21.15.5:0/580173481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd42c3350 msgr2=0x55dfd42b9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:01.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.909+0000 7f25f5035740 1 --2- 172.21.15.5:0/580173481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd42c3350 0x55dfd42b9e60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f25d80116e0 tx=0x7f25d8005040 comp rx=0 tx=0).stop 2024-04-24T06:48:01.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.909+0000 7f25f5035740 1 -- 172.21.15.5:0/580173481 shutdown_connections 2024-04-24T06:48:01.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.909+0000 7f25f5035740 1 --2- 172.21.15.5:0/580173481 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfd42cda20 0x55dfd42bf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:01.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.909+0000 7f25f5035740 1 --2- 172.21.15.5:0/580173481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd42c3350 0x55dfd42b9e60 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:01.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.909+0000 7f25f5035740 1 --2- 172.21.15.5:0/580173481 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd42c2a30 0x55dfd42d9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:01.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.909+0000 7f25f5035740 1 -- 172.21.15.5:0/580173481 >> 172.21.15.5:0/580173481 conn(0x55dfd42bed50 msgr2=0x55dfd42c4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:01.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.910+0000 7f25f5035740 1 -- 172.21.15.5:0/580173481 shutdown_connections 2024-04-24T06:48:01.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.910+0000 7f25f5035740 1 -- 172.21.15.5:0/580173481 wait complete. 2024-04-24T06:48:01.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.910+0000 7f25f5035740 1 -- start start 2024-04-24T06:48:01.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.910+0000 7f25f5035740 1 Processor -- start 2024-04-24T06:48:01.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.911+0000 7f25f5035740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfd42c2a30 0x55dfd42d5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:01.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.911+0000 7f25f5035740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd42c3350 0x55dfd42d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:01.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.911+0000 7f25f5035740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd42cda20 0x55dfd42d1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:01.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.911+0000 7f25f5035740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec1ae430 con 0x55dfd42c3350 2024-04-24T06:48:01.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.911+0000 7f25f5035740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec1af8a0 con 0x55dfd42c2a30 2024-04-24T06:48:01.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.911+0000 7f25f5035740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec1aeb00 con 0x55dfd42cda20 2024-04-24T06:48:01.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.911+0000 7f25f5035740 10 client.0 _ll_register_callbacks cb 0x7f25e40092a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:48:01.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.911+0000 7f25eaa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd42c3350 0x55dfd42d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:01.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.911+0000 7f25eb400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfd42c2a30 0x55dfd42d5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:01.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.911+0000 7f25eb400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfd42c2a30 0x55dfd42d5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:38194/0 (socket says 172.21.15.5:38194) 2024-04-24T06:48:01.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.911+0000 7f25eb400640 1 -- 172.21.15.5:0/1135607278 learned_addr learned my addr 172.21.15.5:0/1135607278 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:01.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.911+0000 7f25ebe00640 1 --2- 172.21.15.5:0/1135607278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd42cda20 0x55dfd42d1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:01.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.911+0000 7f25eaa00640 1 -- 172.21.15.5:0/1135607278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd42cda20 msgr2=0x55dfd42d1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:01.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.911+0000 7f25eaa00640 1 --2- 172.21.15.5:0/1135607278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd42cda20 0x55dfd42d1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:01.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.911+0000 7f25eaa00640 1 -- 172.21.15.5:0/1135607278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfd42c2a30 msgr2=0x55dfd42d5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:01.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.911+0000 7f25eaa00640 1 --2- 172.21.15.5:0/1135607278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfd42c2a30 0x55dfd42d5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:01.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.911+0000 7f25eaa00640 1 -- 172.21.15.5:0/1135607278 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfd3f3a3f0 con 0x55dfd42c3350 2024-04-24T06:48:01.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.911+0000 7f25eb400640 1 --2- 172.21.15.5:0/1135607278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfd42c2a30 0x55dfd42d5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:48:01.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.912+0000 7f25eaa00640 1 --2- 172.21.15.5:0/1135607278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd42c3350 0x55dfd42d12e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f25d8000c00 tx=0x7f25d80114c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edd84a256e7a6572 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:01.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.912+0000 7f25d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:48:01.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.912+0000 7f25d4c00640 1 -- 172.21.15.5:0/1135607278 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d802c040 con 0x55dfd42c3350 2024-04-24T06:48:01.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.912+0000 7f25d4c00640 1 -- 172.21.15.5:0/1135607278 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f25d801f900 con 0x55dfd42c3350 2024-04-24T06:48:01.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.912+0000 7f25f5035740 1 -- 172.21.15.5:0/1135607278 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25d801f440 con 0x55dfd42c3350 2024-04-24T06:48:01.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.912+0000 7f25d4c00640 1 -- 172.21.15.5:0/1135607278 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f25d801fb20 con 0x55dfd42c3350 2024-04-24T06:48:01.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.912+0000 7f25f5035740 10 client.15996 fetch_fsmap learned FSMap version 25 2024-04-24T06:48:01.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.912+0000 7f25f5035740 1 -- 172.21.15.5:0/1135607278 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dfd3efa0f0 con 0x55dfd42c3350 2024-04-24T06:48:01.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.913+0000 7f25d4c00640 1 -- 172.21.15.5:0/1135607278 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25d801f440 con 0x55dfd42c3350 2024-04-24T06:48:01.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.913+0000 7f25d4c00640 1 -- 172.21.15.5:0/1135607278 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f25d8037050 con 0x55dfd42c3350 2024-04-24T06:48:01.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.913+0000 7f25f5035740 10 client.15996 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:48:01.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.913+0000 7f25f5035740 10 client.15996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:48:01.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.913+0000 7f25f5035740 20 client.15996 populate_metadata read hostname 'smithi005' 2024-04-24T06:48:01.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.913+0000 7f25f5035740 1 --2- 172.21.15.5:0/1135607278 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55dfd43444e0 0x55dfd4346900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:01.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.913+0000 7f25f5035740 4 client.15996 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:48:01.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.913+0000 7f25f5035740 1 -- 172.21.15.5:0/1135607278 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dfd422fa80 con 0x55dfd43444e0 2024-04-24T06:48:01.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.913+0000 7f25eb400640 1 --2- 172.21.15.5:0/1135607278 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55dfd43444e0 0x55dfd4346900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:01.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.914+0000 7f25eb400640 1 --2- 172.21.15.5:0/1135607278 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55dfd43444e0 0x55dfd4346900 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cfa0492d09ceade server_cookie=22e20b23410ac832 in_seq=0 out_seq=0 2024-04-24T06:48:01.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.914+0000 7f25d4c00640 10 client.15996 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:48:01.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.914+0000 7f25d4c00640 1 -- 172.21.15.5:0/1135607278 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55dfd422fa80 con 0x55dfd43444e0 2024-04-24T06:48:01.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.914+0000 7f25d4c00640 10 client.15996 handle_command_reply: tid=0 2024-04-24T06:48:01.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.914+0000 7f25f5035740 1 client.15996 shutdown 2024-04-24T06:48:01.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.914+0000 7f25f5035740 1 -- 172.21.15.5:0/1135607278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd42c3350 msgr2=0x55dfd42d12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:01.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.914+0000 7f25f5035740 1 --2- 172.21.15.5:0/1135607278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd42c3350 0x55dfd42d12e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f25d8000c00 tx=0x7f25d80114c0 comp rx=0 tx=0).stop 2024-04-24T06:48:01.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.914+0000 7f25f5035740 1 -- 172.21.15.5:0/1135607278 shutdown_connections 2024-04-24T06:48:01.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.914+0000 7f25f5035740 1 --2- 172.21.15.5:0/1135607278 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55dfd43444e0 0x55dfd4346900 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:01.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.914+0000 7f25f5035740 1 --2- 172.21.15.5:0/1135607278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd42cda20 0x55dfd42d1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:01.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.914+0000 7f25f5035740 1 --2- 172.21.15.5:0/1135607278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd42c3350 0x55dfd42d12e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:01.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.914+0000 7f25f5035740 1 --2- 172.21.15.5:0/1135607278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfd42c2a30 0x55dfd42d5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:01.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.914+0000 7f25d4c00640 0 client.15996 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:48:01.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.914+0000 7f25f5035740 1 -- 172.21.15.5:0/1135607278 >> 172.21.15.5:0/1135607278 conn(0x55dfd42bed50 msgr2=0x55dfd42ca980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:01.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.914+0000 7f25f5035740 1 -- 172.21.15.5:0/1135607278 shutdown_connections 2024-04-24T06:48:01.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.914+0000 7f25f5035740 1 -- 172.21.15.5:0/1135607278 wait complete. 2024-04-24T06:48:01.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.914+0000 7f25f5035740 20 client.15996 trim_cache size 0 max 16384 2024-04-24T06:48:01.924 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:48:01.924 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-24T06:48:01.925 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-24T06:48:01.925 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:48:01.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.915+0000 7f25f0c00640 1 -- 172.21.15.5:0/1965378193 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f25d00a3d20 msgr2=0x7f25d00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:01.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.915+0000 7f25f0c00640 1 --2- 172.21.15.5:0/1965378193 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f25d00a3d20 0x7f25d00a61e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f25dc0096c0 tx=0x7f25dc0046c0 comp rx=0 tx=0).stop 2024-04-24T06:48:01.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.915+0000 7f25f0c00640 1 -- 172.21.15.5:0/1965378193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25ec19a830 msgr2=0x7f25ec19ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:01.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.915+0000 7f25f0c00640 1 --2- 172.21.15.5:0/1965378193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25ec19a830 0x7f25ec19ace0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f25e400da10 tx=0x7f25e400ded0 comp rx=0 tx=0).stop 2024-04-24T06:48:01.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.915+0000 7f25f0c00640 1 -- 172.21.15.5:0/1965378193 shutdown_connections 2024-04-24T06:48:01.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.915+0000 7f25f0c00640 1 --2- 172.21.15.5:0/1965378193 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f25d00a3d20 0x7f25d00a61e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:01.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.915+0000 7f25f0c00640 1 --2- 172.21.15.5:0/1965378193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25ec19a830 0x7f25ec19ace0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:01.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.915+0000 7f25f0c00640 1 --2- 172.21.15.5:0/1965378193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25ec109330 0x7f25ec1a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:01.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.915+0000 7f25f0c00640 1 --2- 172.21.15.5:0/1965378193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25ec104260 0x7f25ec1a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:01.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.915+0000 7f25f0c00640 1 -- 172.21.15.5:0/1965378193 >> 172.21.15.5:0/1965378193 conn(0x7f25ec0ffaa0 msgr2=0x7f25ec100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:01.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.915+0000 7f25f0c00640 1 -- 172.21.15.5:0/1965378193 shutdown_connections 2024-04-24T06:48:01.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:01.915+0000 7f25f0c00640 1 -- 172.21.15.5:0/1965378193 wait complete. 2024-04-24T06:48:01.932 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:01 smithi005 ceph-mon[27798]: pgmap v122: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T06:48:01.936 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T06:48:01.936 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:48:01.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:01 smithi137 ceph-mon[33017]: pgmap v122: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T06:48:02.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:01 smithi037 ceph-mon[32864]: pgmap v122: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T06:48:02.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.153+0000 7fad58a00640 1 -- 172.21.15.5:0/1084712876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad54104d00 msgr2=0x7fad54108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:02.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.153+0000 7fad58a00640 1 --2- 172.21.15.5:0/1084712876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad54104d00 0x7fad54108d40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fad3c009860 tx=0x7fad3c02a340 comp rx=0 tx=0).stop 2024-04-24T06:48:02.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.153+0000 7fad58a00640 1 -- 172.21.15.5:0/1084712876 shutdown_connections 2024-04-24T06:48:02.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.153+0000 7fad58a00640 1 --2- 172.21.15.5:0/1084712876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad54109420 0x7fad54110ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.153+0000 7fad58a00640 1 --2- 172.21.15.5:0/1084712876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad54104d00 0x7fad54108d40 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.153+0000 7fad58a00640 1 --2- 172.21.15.5:0/1084712876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad54104350 0x7fad54104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.153+0000 7fad58a00640 1 -- 172.21.15.5:0/1084712876 >> 172.21.15.5:0/1084712876 conn(0x7fad540ffb70 msgr2=0x7fad54101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:02.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.153+0000 7fad58a00640 1 -- 172.21.15.5:0/1084712876 shutdown_connections 2024-04-24T06:48:02.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.153+0000 7fad58a00640 1 -- 172.21.15.5:0/1084712876 wait complete. 2024-04-24T06:48:02.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.154+0000 7fad58a00640 1 Processor -- start 2024-04-24T06:48:02.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.155+0000 7fad58a00640 1 -- start start 2024-04-24T06:48:02.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.155+0000 7fad58a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad54104350 0x7fad541a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:02.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.155+0000 7fad58a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad54109420 0x7fad541a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:02.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.155+0000 7fad58a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad5419a850 0x7fad5419ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:02.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.155+0000 7fad58a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fad541144b0 con 0x7fad5419a850 2024-04-24T06:48:02.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.155+0000 7fad58a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fad54114330 con 0x7fad54104350 2024-04-24T06:48:02.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.155+0000 7fad58a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fad54114630 con 0x7fad54109420 2024-04-24T06:48:02.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.155+0000 7fad53e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad5419a850 0x7fad5419ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:02.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.155+0000 7fad53400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad54104350 0x7fad541a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:02.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.155+0000 7fad53e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad5419a850 0x7fad5419ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58948/0 (socket says 172.21.15.5:58948) 2024-04-24T06:48:02.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.155+0000 7fad53e00640 1 -- 172.21.15.5:0/4022639635 learned_addr learned my addr 172.21.15.5:0/4022639635 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:02.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.156+0000 7fad52a00640 1 --2- 172.21.15.5:0/4022639635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad54109420 0x7fad541a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:02.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.156+0000 7fad53e00640 1 -- 172.21.15.5:0/4022639635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad54109420 msgr2=0x7fad541a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:02.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.156+0000 7fad53e00640 1 --2- 172.21.15.5:0/4022639635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad54109420 0x7fad541a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.156+0000 7fad53e00640 1 -- 172.21.15.5:0/4022639635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad54104350 msgr2=0x7fad541a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:02.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.156+0000 7fad53e00640 1 --2- 172.21.15.5:0/4022639635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad54104350 0x7fad541a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.156+0000 7fad53e00640 1 -- 172.21.15.5:0/4022639635 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad5419b3e0 con 0x7fad5419a850 2024-04-24T06:48:02.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.156+0000 7fad53400640 1 --2- 172.21.15.5:0/4022639635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad54104350 0x7fad541a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:48:02.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.156+0000 7fad53e00640 1 --2- 172.21.15.5:0/4022639635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad5419a850 0x7fad5419ad00 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fad4c00e970 tx=0x7fad4c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:02.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.157+0000 7fad4be00640 1 -- 172.21.15.5:0/4022639635 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4c00ccc0 con 0x7fad5419a850 2024-04-24T06:48:02.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.157+0000 7fad4be00640 1 -- 172.21.15.5:0/4022639635 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad4c00ce60 con 0x7fad5419a850 2024-04-24T06:48:02.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.157+0000 7fad58a00640 1 -- 172.21.15.5:0/4022639635 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad5419b6d0 con 0x7fad5419a850 2024-04-24T06:48:02.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.157+0000 7fad4be00640 1 -- 172.21.15.5:0/4022639635 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4c010450 con 0x7fad5419a850 2024-04-24T06:48:02.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.157+0000 7fad58a00640 1 -- 172.21.15.5:0/4022639635 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad541a7880 con 0x7fad5419a850 2024-04-24T06:48:02.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.158+0000 7fad58a00640 1 -- 172.21.15.5:0/4022639635 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad54105820 con 0x7fad5419a850 2024-04-24T06:48:02.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.160+0000 7fad4be00640 1 -- 172.21.15.5:0/4022639635 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fad4c005020 con 0x7fad5419a850 2024-04-24T06:48:02.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.162+0000 7fad4be00640 1 --2- 172.21.15.5:0/4022639635 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fad380a3d90 0x7fad380a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:02.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.162+0000 7fad4be00640 1 -- 172.21.15.5:0/4022639635 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fad4c0d0b80 con 0x7fad5419a850 2024-04-24T06:48:02.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.162+0000 7fad53400640 1 --2- 172.21.15.5:0/4022639635 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fad380a3d90 0x7fad380a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:02.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.163+0000 7fad53400640 1 --2- 172.21.15.5:0/4022639635 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fad380a3d90 0x7fad380a6250 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fad44005fd0 tx=0x7fad440095c0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:02.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.167+0000 7fad4be00640 1 -- 172.21.15.5:0/4022639635 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fad5419b6d0 con 0x7fad5419a850 2024-04-24T06:48:02.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.426+0000 7fad58a00640 1 -- 172.21.15.5:0/4022639635 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad54104d00 con 0x7fad5419a850 2024-04-24T06:48:02.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.428+0000 7fad4be00640 1 -- 172.21.15.5:0/4022639635 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11859 (secure 0 0 0) 0x7fad4c09d1d0 con 0x7fad5419a850 2024-04-24T06:48:02.429 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:48:02.429 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1735462750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1735462750},{"type":"v1","addr":"172.21.15.5:6837","nonce":1735462750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1499469370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1499469370},{"type":"v1","addr":"172.21.15.5:6841","nonce":1499469370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3705638847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3705638847},{"type":"v1","addr":"172.21.15.37:6835","nonce":3705638847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T06:41:34.764490+0000","modified":"2024-04-24T06:46:02.765290+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24550,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6837/1390103296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1390103296},{"type":"v1","addr":"172.21.15.37:6837","nonce":1390103296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3217754215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3217754215},{"type":"v1","addr":"172.21.15.137:6835","nonce":3217754215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6837/3044222976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3044222976},{"type":"v1","addr":"172.21.15.137:6837","nonce":3044222976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.5:6839/3173201459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3173201459},{"type":"v1","addr":"172.21.15.5:6839","nonce":3173201459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6839/1889774232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1889774232},{"type":"v1","addr":"172.21.15.137:6839","nonce":1889774232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6839/3536721753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":3536721753},{"type":"v1","addr":"172.21.15.37:6839","nonce":3536721753}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24539,24550,24569]},"id":1}]} 2024-04-24T06:48:02.430 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-04-24T06:48:02.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.430+0000 7fad58a00640 1 -- 172.21.15.5:0/4022639635 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fad380a3d90 msgr2=0x7fad380a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:02.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.430+0000 7fad58a00640 1 --2- 172.21.15.5:0/4022639635 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fad380a3d90 0x7fad380a6250 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fad44005fd0 tx=0x7fad440095c0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.430+0000 7fad58a00640 1 -- 172.21.15.5:0/4022639635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad5419a850 msgr2=0x7fad5419ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:02.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.430+0000 7fad58a00640 1 --2- 172.21.15.5:0/4022639635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad5419a850 0x7fad5419ad00 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fad4c00e970 tx=0x7fad4c00ed70 comp rx=0 tx=0).stop 2024-04-24T06:48:02.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.431+0000 7fad58a00640 1 -- 172.21.15.5:0/4022639635 shutdown_connections 2024-04-24T06:48:02.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.431+0000 7fad58a00640 1 --2- 172.21.15.5:0/4022639635 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fad380a3d90 0x7fad380a6250 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.431+0000 7fad58a00640 1 --2- 172.21.15.5:0/4022639635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad5419a850 0x7fad5419ad00 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.431+0000 7fad58a00640 1 --2- 172.21.15.5:0/4022639635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad54109420 0x7fad541a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.431+0000 7fad58a00640 1 --2- 172.21.15.5:0/4022639635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad54104350 0x7fad541a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.431+0000 7fad58a00640 1 -- 172.21.15.5:0/4022639635 >> 172.21.15.5:0/4022639635 conn(0x7fad540ffb70 msgr2=0x7fad54101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:02.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.431+0000 7fad58a00640 1 -- 172.21.15.5:0/4022639635 shutdown_connections 2024-04-24T06:48:02.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.431+0000 7fad58a00640 1 -- 172.21.15.5:0/4022639635 wait complete. 2024-04-24T06:48:02.453 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T06:48:02.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.678+0000 7fc54de00640 1 -- 172.21.15.5:0/1948703290 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc548102fe0 msgr2=0x7fc5481033c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:02.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.678+0000 7fc54de00640 1 --2- 172.21.15.5:0/1948703290 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc548102fe0 0x7fc5481033c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc538009ca0 tx=0x7fc53802a950 comp rx=0 tx=0).stop 2024-04-24T06:48:02.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.679+0000 7fc54de00640 1 -- 172.21.15.5:0/1948703290 shutdown_connections 2024-04-24T06:48:02.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.679+0000 7fc54de00640 1 --2- 172.21.15.5:0/1948703290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc54810c400 0x7fc54810e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.679+0000 7fc54de00640 1 --2- 172.21.15.5:0/1948703290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc548103900 0x7fc54810bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.679+0000 7fc54de00640 1 --2- 172.21.15.5:0/1948703290 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc548102fe0 0x7fc5481033c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.679+0000 7fc54de00640 1 -- 172.21.15.5:0/1948703290 >> 172.21.15.5:0/1948703290 conn(0x7fc548077ab0 msgr2=0x7fc5480fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:02.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.679+0000 7fc54de00640 1 -- 172.21.15.5:0/1948703290 shutdown_connections 2024-04-24T06:48:02.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.679+0000 7fc54de00640 1 -- 172.21.15.5:0/1948703290 wait complete. 2024-04-24T06:48:02.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.680+0000 7fc54de00640 1 Processor -- start 2024-04-24T06:48:02.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.680+0000 7fc54de00640 1 -- start start 2024-04-24T06:48:02.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.681+0000 7fc54de00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc548102fe0 0x7fc5481a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:02.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.681+0000 7fc54de00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc548103900 0x7fc5481a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:02.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.681+0000 7fc54de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc54810c400 0x7fc54819a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:02.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.681+0000 7fc54de00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc5481121e0 con 0x7fc54810c400 2024-04-24T06:48:02.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.681+0000 7fc54de00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc548112060 con 0x7fc548103900 2024-04-24T06:48:02.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.681+0000 7fc54de00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc548112360 con 0x7fc548102fe0 2024-04-24T06:48:02.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.681+0000 7fc54ca00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc548102fe0 0x7fc5481a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:02.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.681+0000 7fc54d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc54810c400 0x7fc54819a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:02.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.681+0000 7fc54ca00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc548102fe0 0x7fc5481a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:54262/0 (socket says 172.21.15.5:54262) 2024-04-24T06:48:02.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.681+0000 7fc54ca00640 1 -- 172.21.15.5:0/1912471471 learned_addr learned my addr 172.21.15.5:0/1912471471 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:02.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.681+0000 7fc547e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc548103900 0x7fc5481a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:02.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.682+0000 7fc54ca00640 1 -- 172.21.15.5:0/1912471471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc548103900 msgr2=0x7fc5481a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:02.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.682+0000 7fc54ca00640 1 --2- 172.21.15.5:0/1912471471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc548103900 0x7fc5481a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.682+0000 7fc54ca00640 1 -- 172.21.15.5:0/1912471471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc54810c400 msgr2=0x7fc54819a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:02.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.682+0000 7fc54ca00640 1 --2- 172.21.15.5:0/1912471471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc54810c400 0x7fc54819a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.682+0000 7fc54ca00640 1 -- 172.21.15.5:0/1912471471 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc54819af90 con 0x7fc548102fe0 2024-04-24T06:48:02.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.682+0000 7fc54ca00640 1 --2- 172.21.15.5:0/1912471471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc548102fe0 0x7fc5481a0650 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc538005e30 tx=0x7fc538003c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:02.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.682+0000 7fc545600640 1 -- 172.21.15.5:0/1912471471 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc538034040 con 0x7fc548102fe0 2024-04-24T06:48:02.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.682+0000 7fc54de00640 1 -- 172.21.15.5:0/1912471471 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc54819b220 con 0x7fc548102fe0 2024-04-24T06:48:02.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.682+0000 7fc545600640 1 -- 172.21.15.5:0/1912471471 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc53802fe20 con 0x7fc548102fe0 2024-04-24T06:48:02.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.683+0000 7fc545600640 1 -- 172.21.15.5:0/1912471471 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5380047c0 con 0x7fc548102fe0 2024-04-24T06:48:02.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.683+0000 7fc54de00640 1 -- 172.21.15.5:0/1912471471 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5481a74a0 con 0x7fc548102fe0 2024-04-24T06:48:02.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.686+0000 7fc545600640 1 -- 172.21.15.5:0/1912471471 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fc538037020 con 0x7fc548102fe0 2024-04-24T06:48:02.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.686+0000 7fc55219c740 1 Processor -- start 2024-04-24T06:48:02.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.686+0000 7fc55219c740 1 -- start start 2024-04-24T06:48:02.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.686+0000 7fc55219c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56084abf8b40 0x56084abf8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:02.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.687+0000 7fc55219c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56084abf9460 0x56084ac035f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:02.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.687+0000 7fc55219c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56084ac03b30 0x56084ac05f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:02.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.687+0000 7fc55219c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56084ab5e1d0 con 0x56084abf8b40 2024-04-24T06:48:02.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.687+0000 7fc55219c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56084ab04450 con 0x56084abf9460 2024-04-24T06:48:02.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.687+0000 7fc55219c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56084ab5a190 con 0x56084ac03b30 2024-04-24T06:48:02.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.687+0000 7fc54d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56084abf8b40 0x56084abf8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:02.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.687+0000 7fc547e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56084ac03b30 0x56084ac05f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:02.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.687+0000 7fc54d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56084abf8b40 0x56084abf8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58986/0 (socket says 172.21.15.5:58986) 2024-04-24T06:48:02.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.687+0000 7fc54ca00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56084abf9460 0x56084ac035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:02.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.687+0000 7fc54d400640 1 -- 172.21.15.5:0/1009964349 learned_addr learned my addr 172.21.15.5:0/1009964349 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:02.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.687+0000 7fc547e00640 1 -- 172.21.15.5:0/1009964349 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56084abf9460 msgr2=0x56084ac035f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:02.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.687+0000 7fc547e00640 1 --2- 172.21.15.5:0/1009964349 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56084abf9460 0x56084ac035f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.687+0000 7fc547e00640 1 -- 172.21.15.5:0/1009964349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56084abf8b40 msgr2=0x56084abf8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:02.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.687+0000 7fc547e00640 1 --2- 172.21.15.5:0/1009964349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56084abf8b40 0x56084abf8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.687+0000 7fc547e00640 1 -- 172.21.15.5:0/1009964349 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56084abc1310 con 0x56084ac03b30 2024-04-24T06:48:02.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.688+0000 7fc545600640 1 --2- 172.21.15.5:0/1912471471 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc52c0a3d20 0x7fc52c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:02.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.688+0000 7fc545600640 1 -- 172.21.15.5:0/1912471471 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fc5380f1560 con 0x7fc548102fe0 2024-04-24T06:48:02.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.688+0000 7fc54d400640 1 --2- 172.21.15.5:0/1912471471 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc52c0a3d20 0x7fc52c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:02.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.688+0000 7fc547e00640 1 --2- 172.21.15.5:0/1009964349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56084ac03b30 0x56084ac05f20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc54819bb10 tx=0x7fc530011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1be95e667cf0ba4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:02.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.688+0000 7fc53dc00640 1 -- 172.21.15.5:0/1009964349 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc530004050 con 0x56084ac03b30 2024-04-24T06:48:02.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.688+0000 7fc53dc00640 1 -- 172.21.15.5:0/1009964349 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc530005500 con 0x56084ac03b30 2024-04-24T06:48:02.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.688+0000 7fc53dc00640 1 -- 172.21.15.5:0/1009964349 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc530004db0 con 0x56084ac03b30 2024-04-24T06:48:02.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.689+0000 7fc55219c740 1 -- 172.21.15.5:0/1009964349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56084ac03b30 msgr2=0x56084ac05f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:02.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.689+0000 7fc55219c740 1 --2- 172.21.15.5:0/1009964349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56084ac03b30 0x56084ac05f20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc54819bb10 tx=0x7fc530011810 comp rx=0 tx=0).stop 2024-04-24T06:48:02.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.689+0000 7fc54d400640 1 --2- 172.21.15.5:0/1912471471 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc52c0a3d20 0x7fc52c0a61e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc540009e60 tx=0x7fc540005f90 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:02.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.689+0000 7fc55219c740 1 -- 172.21.15.5:0/1009964349 shutdown_connections 2024-04-24T06:48:02.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.689+0000 7fc55219c740 1 --2- 172.21.15.5:0/1009964349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56084ac03b30 0x56084ac05f20 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.689+0000 7fc55219c740 1 --2- 172.21.15.5:0/1009964349 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56084abf9460 0x56084ac035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.689+0000 7fc55219c740 1 --2- 172.21.15.5:0/1009964349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56084abf8b40 0x56084abf8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.689+0000 7fc55219c740 1 -- 172.21.15.5:0/1009964349 >> 172.21.15.5:0/1009964349 conn(0x56084abf4e60 msgr2=0x56084abf5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:02.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.689+0000 7fc55219c740 1 -- 172.21.15.5:0/1009964349 shutdown_connections 2024-04-24T06:48:02.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.689+0000 7fc55219c740 1 -- 172.21.15.5:0/1009964349 wait complete. 2024-04-24T06:48:02.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.690+0000 7fc55219c740 1 -- start start 2024-04-24T06:48:02.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.690+0000 7fc55219c740 1 Processor -- start 2024-04-24T06:48:02.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.690+0000 7fc55219c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56084abf8b40 0x56084ac03590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:02.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.690+0000 7fc55219c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56084abf9460 0x56084ac00880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:02.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.690+0000 7fc55219c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56084ac03b30 0x56084ac00fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:02.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.690+0000 7fc55219c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56084ac069c0 con 0x56084ac03b30 2024-04-24T06:48:02.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.690+0000 7fc55219c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56084ab17810 con 0x56084abf8b40 2024-04-24T06:48:02.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.690+0000 7fc55219c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56084ac06b40 con 0x56084abf9460 2024-04-24T06:48:02.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.691+0000 7fc55219c740 10 client.0 _ll_register_callbacks cb 0x56084aae5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:48:02.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.691+0000 7fc54d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56084ac03b30 0x56084ac00fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:02.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.691+0000 7fc54ca00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56084abf8b40 0x56084ac03590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:02.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.691+0000 7fc547e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56084abf9460 0x56084ac00880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:02.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.691+0000 7fc54ca00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56084abf8b40 0x56084ac03590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:38232/0 (socket says 172.21.15.5:38232) 2024-04-24T06:48:02.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.691+0000 7fc54d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56084ac03b30 0x56084ac00fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58994/0 (socket says 172.21.15.5:58994) 2024-04-24T06:48:02.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.691+0000 7fc54ca00640 1 -- 172.21.15.5:0/2238547112 learned_addr learned my addr 172.21.15.5:0/2238547112 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:02.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.691+0000 7fc54ca00640 1 -- 172.21.15.5:0/2238547112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56084abf9460 msgr2=0x56084ac00880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:02.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.691+0000 7fc54ca00640 1 --2- 172.21.15.5:0/2238547112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56084abf9460 0x56084ac00880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.691+0000 7fc54ca00640 1 -- 172.21.15.5:0/2238547112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56084ac03b30 msgr2=0x56084ac00fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:02.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.691+0000 7fc54ca00640 1 --2- 172.21.15.5:0/2238547112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56084ac03b30 0x56084ac00fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.691+0000 7fc54ca00640 1 -- 172.21.15.5:0/2238547112 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56084ab283f0 con 0x56084abf8b40 2024-04-24T06:48:02.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.691+0000 7fc54ca00640 1 --2- 172.21.15.5:0/2238547112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56084abf8b40 0x56084ac03590 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc5380bd2c0 tx=0x7fc5380f4b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e836bfa723e99b9a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:02.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.692+0000 7fc536000640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:48:02.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.692+0000 7fc536000640 1 -- 172.21.15.5:0/2238547112 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53802c040 con 0x56084abf8b40 2024-04-24T06:48:02.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.692+0000 7fc536000640 1 -- 172.21.15.5:0/2238547112 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fc5380c5da0 con 0x56084abf8b40 2024-04-24T06:48:02.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.692+0000 7fc55219c740 1 -- 172.21.15.5:0/2238547112 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc530005500 con 0x56084abf8b40 2024-04-24T06:48:02.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.692+0000 7fc536000640 1 -- 172.21.15.5:0/2238547112 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc5380bd390 con 0x56084abf8b40 2024-04-24T06:48:02.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.692+0000 7fc55219c740 10 client.26144 fetch_fsmap learned FSMap version 25 2024-04-24T06:48:02.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.692+0000 7fc55219c740 1 -- 172.21.15.5:0/2238547112 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56084a8703f0 con 0x56084abf8b40 2024-04-24T06:48:02.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.692+0000 7fc536000640 1 -- 172.21.15.5:0/2238547112 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc530005500 con 0x56084abf8b40 2024-04-24T06:48:02.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.692+0000 7fc536000640 1 -- 172.21.15.5:0/2238547112 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc53804d030 con 0x56084abf8b40 2024-04-24T06:48:02.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.693+0000 7fc55219c740 10 client.26144 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:48:02.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.693+0000 7fc55219c740 10 client.26144 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-24T06:48:02.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.693+0000 7fc55219c740 20 client.26144 populate_metadata read hostname 'smithi005' 2024-04-24T06:48:02.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.693+0000 7fc55219c740 1 --2- 172.21.15.5:0/2238547112 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x56084ac13cc0 0x56084ac16140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:02.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.693+0000 7fc55219c740 4 client.26144 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:48:02.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.693+0000 7fc55219c740 1 -- 172.21.15.5:0/2238547112 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56084ab6bbf0 con 0x56084ac13cc0 2024-04-24T06:48:02.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.693+0000 7fc547e00640 1 --2- 172.21.15.5:0/2238547112 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x56084ac13cc0 0x56084ac16140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:02.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.694+0000 7fc547e00640 1 --2- 172.21.15.5:0/2238547112 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x56084ac13cc0 0x56084ac16140 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b8f905b765eb5878 server_cookie=7c67a54a7f8caee3 in_seq=0 out_seq=0 2024-04-24T06:48:02.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.694+0000 7fc536000640 10 client.26144 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:48:02.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.694+0000 7fc536000640 1 -- 172.21.15.5:0/2238547112 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56084ab6bbf0 con 0x56084ac13cc0 2024-04-24T06:48:02.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.694+0000 7fc536000640 10 client.26144 handle_command_reply: tid=0 2024-04-24T06:48:02.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.694+0000 7fc55219c740 1 client.26144 shutdown 2024-04-24T06:48:02.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.695+0000 7fc55219c740 1 -- 172.21.15.5:0/2238547112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56084abf8b40 msgr2=0x56084ac03590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:02.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.695+0000 7fc55219c740 1 --2- 172.21.15.5:0/2238547112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56084abf8b40 0x56084ac03590 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc5380bd2c0 tx=0x7fc5380f4b40 comp rx=0 tx=0).stop 2024-04-24T06:48:02.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.695+0000 7fc55219c740 1 -- 172.21.15.5:0/2238547112 shutdown_connections 2024-04-24T06:48:02.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.695+0000 7fc55219c740 1 --2- 172.21.15.5:0/2238547112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56084ac03b30 0x56084ac00fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.695+0000 7fc55219c740 1 --2- 172.21.15.5:0/2238547112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56084abf9460 0x56084ac00880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.695+0000 7fc55219c740 1 --2- 172.21.15.5:0/2238547112 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x56084ac13cc0 0x56084ac16140 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.695+0000 7fc55219c740 1 --2- 172.21.15.5:0/2238547112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56084abf8b40 0x56084ac03590 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.695+0000 7fc55219c740 1 -- 172.21.15.5:0/2238547112 >> 172.21.15.5:0/2238547112 conn(0x56084abf4e60 msgr2=0x56084abfab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:02.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.695+0000 7fc536000640 0 client.26144 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:48:02.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.695+0000 7fc55219c740 1 -- 172.21.15.5:0/2238547112 shutdown_connections 2024-04-24T06:48:02.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.695+0000 7fc55219c740 1 -- 172.21.15.5:0/2238547112 wait complete. 2024-04-24T06:48:02.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.695+0000 7fc55219c740 20 client.26144 trim_cache size 0 max 16384 2024-04-24T06:48:02.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.710+0000 7fc55219c740 1 Processor -- start 2024-04-24T06:48:02.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.711+0000 7fc55219c740 1 -- start start 2024-04-24T06:48:02.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.711+0000 7fc55219c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56084abf8b40 0x56084abfb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:02.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.711+0000 7fc55219c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56084abf9460 0x56084abff9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:02.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.711+0000 7fc55219c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56084ac03b30 0x56084ac030c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:02.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.711+0000 7fc55219c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc5481b2620 con 0x56084ac03b30 2024-04-24T06:48:02.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.711+0000 7fc55219c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc5481afef0 con 0x56084abf8b40 2024-04-24T06:48:02.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.711+0000 7fc55219c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc5481add40 con 0x56084abf9460 2024-04-24T06:48:02.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.711+0000 7fc54d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56084ac03b30 0x56084ac030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:02.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.711+0000 7fc54d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56084ac03b30 0x56084ac030c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59002/0 (socket says 172.21.15.5:59002) 2024-04-24T06:48:02.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.711+0000 7fc54d400640 1 -- 172.21.15.5:0/169496699 learned_addr learned my addr 172.21.15.5:0/169496699 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:02.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.711+0000 7fc54ca00640 1 --2- 172.21.15.5:0/169496699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56084abf8b40 0x56084abfb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:02.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.711+0000 7fc547e00640 1 --2- 172.21.15.5:0/169496699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56084abf9460 0x56084abff9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:02.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.712+0000 7fc54ca00640 1 -- 172.21.15.5:0/169496699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56084abf9460 msgr2=0x56084abff9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:02.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.712+0000 7fc54ca00640 1 --2- 172.21.15.5:0/169496699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56084abf9460 0x56084abff9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.712+0000 7fc54ca00640 1 -- 172.21.15.5:0/169496699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56084ac03b30 msgr2=0x56084ac030c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:02.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.712+0000 7fc54ca00640 1 --2- 172.21.15.5:0/169496699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56084ac03b30 0x56084ac030c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.712+0000 7fc54ca00640 1 -- 172.21.15.5:0/169496699 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56084a93a7d0 con 0x56084abf8b40 2024-04-24T06:48:02.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.712+0000 7fc54ca00640 1 --2- 172.21.15.5:0/169496699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56084abf8b40 0x56084abfb2a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc5380bca10 tx=0x7fc5380bbad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18e20d2a5e14e8f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:02.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.712+0000 7fc53d200640 1 -- 172.21.15.5:0/169496699 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5380bcf20 con 0x56084abf8b40 2024-04-24T06:48:02.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.712+0000 7fc53d200640 1 -- 172.21.15.5:0/169496699 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5380c5fa0 con 0x56084abf8b40 2024-04-24T06:48:02.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.712+0000 7fc53d200640 1 -- 172.21.15.5:0/169496699 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5380bd100 con 0x56084abf8b40 2024-04-24T06:48:02.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.713+0000 7fc55219c740 1 -- 172.21.15.5:0/169496699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56084abf8b40 msgr2=0x56084abfb2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:02.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.713+0000 7fc55219c740 1 --2- 172.21.15.5:0/169496699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56084abf8b40 0x56084abfb2a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc5380bca10 tx=0x7fc5380bbad0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.713+0000 7fc55219c740 1 -- 172.21.15.5:0/169496699 shutdown_connections 2024-04-24T06:48:02.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.713+0000 7fc55219c740 1 --2- 172.21.15.5:0/169496699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56084ac03b30 0x56084ac030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.713+0000 7fc55219c740 1 --2- 172.21.15.5:0/169496699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56084abf9460 0x56084abff9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.713+0000 7fc55219c740 1 --2- 172.21.15.5:0/169496699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56084abf8b40 0x56084abfb2a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.713+0000 7fc55219c740 1 -- 172.21.15.5:0/169496699 >> 172.21.15.5:0/169496699 conn(0x56084abf4e60 msgr2=0x56084abf36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:02.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.713+0000 7fc55219c740 1 -- 172.21.15.5:0/169496699 shutdown_connections 2024-04-24T06:48:02.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.713+0000 7fc55219c740 1 -- 172.21.15.5:0/169496699 wait complete. 2024-04-24T06:48:02.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.713+0000 7fc55219c740 1 -- start start 2024-04-24T06:48:02.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.714+0000 7fc55219c740 1 Processor -- start 2024-04-24T06:48:02.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.714+0000 7fc55219c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56084abf8b40 0x56084ac04f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:02.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.714+0000 7fc55219c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56084abf9460 0x56084ac054c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:02.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.714+0000 7fc55219c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56084ac03b30 0x56084ac0a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:02.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.714+0000 7fc55219c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc5481ae3b0 con 0x56084abf8b40 2024-04-24T06:48:02.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.714+0000 7fc55219c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc5481af820 con 0x56084abf9460 2024-04-24T06:48:02.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.714+0000 7fc55219c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc5481aea80 con 0x56084ac03b30 2024-04-24T06:48:02.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.714+0000 7fc55219c740 10 client.0 _ll_register_callbacks cb 0x7fc5380f4d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:48:02.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.715+0000 7fc54ca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56084abf8b40 0x56084ac04f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:02.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.715+0000 7fc547e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56084abf9460 0x56084ac054c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:02.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.715+0000 7fc54ca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56084abf8b40 0x56084ac04f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59014/0 (socket says 172.21.15.5:59014) 2024-04-24T06:48:02.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.715+0000 7fc54ca00640 1 -- 172.21.15.5:0/746634809 learned_addr learned my addr 172.21.15.5:0/746634809 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:02.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.715+0000 7fc54d400640 1 --2- 172.21.15.5:0/746634809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56084ac03b30 0x56084ac0a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:02.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.715+0000 7fc54ca00640 1 -- 172.21.15.5:0/746634809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56084ac03b30 msgr2=0x56084ac0a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:02.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.715+0000 7fc54ca00640 1 --2- 172.21.15.5:0/746634809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56084ac03b30 0x56084ac0a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.715+0000 7fc54ca00640 1 -- 172.21.15.5:0/746634809 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56084abf9460 msgr2=0x56084ac054c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:02.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.715+0000 7fc54ca00640 1 --2- 172.21.15.5:0/746634809 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56084abf9460 0x56084ac054c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.715+0000 7fc54ca00640 1 -- 172.21.15.5:0/746634809 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56084ac0a960 con 0x56084abf8b40 2024-04-24T06:48:02.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.715+0000 7fc547e00640 1 --2- 172.21.15.5:0/746634809 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56084abf9460 0x56084ac054c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:48:02.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.715+0000 7fc54ca00640 1 --2- 172.21.15.5:0/746634809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56084abf8b40 0x56084ac04f80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc5380bb870 tx=0x7fc5380f17e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed1afad8582d116e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:02.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.715+0000 7fc536000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:48:02.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.715+0000 7fc536000640 1 -- 172.21.15.5:0/746634809 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5380bd100 con 0x56084abf8b40 2024-04-24T06:48:02.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.716+0000 7fc536000640 1 -- 172.21.15.5:0/746634809 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fc5380dc300 con 0x56084abf8b40 2024-04-24T06:48:02.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.716+0000 7fc55219c740 1 -- 172.21.15.5:0/746634809 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56084abf4cb0 con 0x56084abf8b40 2024-04-24T06:48:02.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.716+0000 7fc536000640 1 -- 172.21.15.5:0/746634809 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc5380bcf20 con 0x56084abf8b40 2024-04-24T06:48:02.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.716+0000 7fc55219c740 10 client.16020 fetch_fsmap learned FSMap version 25 2024-04-24T06:48:02.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.716+0000 7fc55219c740 1 -- 172.21.15.5:0/746634809 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56084ac0a960 con 0x56084abf8b40 2024-04-24T06:48:02.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.716+0000 7fc536000640 1 -- 172.21.15.5:0/746634809 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56084abf4cb0 con 0x56084abf8b40 2024-04-24T06:48:02.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.716+0000 7fc536000640 1 -- 172.21.15.5:0/746634809 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc53804d070 con 0x56084abf8b40 2024-04-24T06:48:02.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.716+0000 7fc55219c740 10 client.16020 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:48:02.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.716+0000 7fc55219c740 10 client.16020 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-24T06:48:02.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.716+0000 7fc55219c740 20 client.16020 populate_metadata read hostname 'smithi005' 2024-04-24T06:48:02.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.716+0000 7fc55219c740 1 --2- 172.21.15.5:0/746634809 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x56084ac7ae00 0x56084ac7d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:02.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.716+0000 7fc55219c740 4 client.16020 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:48:02.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.716+0000 7fc55219c740 1 -- 172.21.15.5:0/746634809 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56084ab65a60 con 0x56084ac7ae00 2024-04-24T06:48:02.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.717+0000 7fc547e00640 1 --2- 172.21.15.5:0/746634809 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x56084ac7ae00 0x56084ac7d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:02.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.717+0000 7fc547e00640 1 --2- 172.21.15.5:0/746634809 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x56084ac7ae00 0x56084ac7d1f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3c688a0474d8f410 server_cookie=fef61a6422a8e637 in_seq=0 out_seq=0 2024-04-24T06:48:02.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.717+0000 7fc536000640 10 client.16020 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:48:02.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.717+0000 7fc536000640 1 -- 172.21.15.5:0/746634809 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56084ab65a60 con 0x56084ac7ae00 2024-04-24T06:48:02.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.718+0000 7fc536000640 10 client.16020 handle_command_reply: tid=0 2024-04-24T06:48:02.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.718+0000 7fc55219c740 1 client.16020 shutdown 2024-04-24T06:48:02.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.718+0000 7fc55219c740 1 -- 172.21.15.5:0/746634809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56084abf8b40 msgr2=0x56084ac04f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:02.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.718+0000 7fc55219c740 1 --2- 172.21.15.5:0/746634809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56084abf8b40 0x56084ac04f80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc5380bb870 tx=0x7fc5380f17e0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.718+0000 7fc55219c740 1 -- 172.21.15.5:0/746634809 shutdown_connections 2024-04-24T06:48:02.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.718+0000 7fc55219c740 1 --2- 172.21.15.5:0/746634809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56084ac03b30 0x56084ac0a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.718+0000 7fc55219c740 1 --2- 172.21.15.5:0/746634809 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x56084ac7ae00 0x56084ac7d1f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.718+0000 7fc55219c740 1 --2- 172.21.15.5:0/746634809 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56084abf9460 0x56084ac054c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.718+0000 7fc55219c740 1 --2- 172.21.15.5:0/746634809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56084abf8b40 0x56084ac04f80 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.718+0000 7fc55219c740 1 -- 172.21.15.5:0/746634809 >> 172.21.15.5:0/746634809 conn(0x56084abf4e60 msgr2=0x56084abf36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:02.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.718+0000 7fc536000640 0 client.16020 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:48:02.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.718+0000 7fc55219c740 1 -- 172.21.15.5:0/746634809 shutdown_connections 2024-04-24T06:48:02.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.718+0000 7fc55219c740 1 -- 172.21.15.5:0/746634809 wait complete. 2024-04-24T06:48:02.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.718+0000 7fc55219c740 20 client.16020 trim_cache size 0 max 16384 2024-04-24T06:48:02.727 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:48:02.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.719+0000 7fc54de00640 1 -- 172.21.15.5:0/1912471471 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc52c0a3d20 msgr2=0x7fc52c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:02.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.719+0000 7fc54de00640 1 --2- 172.21.15.5:0/1912471471 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc52c0a3d20 0x7fc52c0a61e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc540009e60 tx=0x7fc540005f90 comp rx=0 tx=0).stop 2024-04-24T06:48:02.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.719+0000 7fc54de00640 1 -- 172.21.15.5:0/1912471471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc548102fe0 msgr2=0x7fc5481a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:02.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.719+0000 7fc54de00640 1 --2- 172.21.15.5:0/1912471471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc548102fe0 0x7fc5481a0650 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc538005e30 tx=0x7fc538003c90 comp rx=0 tx=0).stop 2024-04-24T06:48:02.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.719+0000 7fc54de00640 1 -- 172.21.15.5:0/1912471471 shutdown_connections 2024-04-24T06:48:02.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.719+0000 7fc54de00640 1 --2- 172.21.15.5:0/1912471471 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc52c0a3d20 0x7fc52c0a61e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.719+0000 7fc54de00640 1 --2- 172.21.15.5:0/1912471471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc54810c400 0x7fc54819a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.719+0000 7fc54de00640 1 --2- 172.21.15.5:0/1912471471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc548103900 0x7fc5481a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.719+0000 7fc54de00640 1 --2- 172.21.15.5:0/1912471471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc548102fe0 0x7fc5481a0650 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.719+0000 7fc54de00640 1 -- 172.21.15.5:0/1912471471 >> 172.21.15.5:0/1912471471 conn(0x7fc548077ab0 msgr2=0x7fc5480fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:02.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.719+0000 7fc54de00640 1 -- 172.21.15.5:0/1912471471 shutdown_connections 2024-04-24T06:48:02.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.719+0000 7fc54de00640 1 -- 172.21.15.5:0/1912471471 wait complete. 2024-04-24T06:48:02.736 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:02 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/4022639635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:48:02.738 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T06:48:02.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:02 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/4022639635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:48:02.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.949+0000 7f3bee200640 1 -- 172.21.15.5:0/3446775193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3be8102070 msgr2=0x7f3be8102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:02.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.949+0000 7f3bee200640 1 --2- 172.21.15.5:0/3446775193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3be8102070 0x7f3be8102450 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8009900 tx=0x7f3bd802a3f0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.949+0000 7f3bee200640 1 -- 172.21.15.5:0/3446775193 shutdown_connections 2024-04-24T06:48:02.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.949+0000 7f3bee200640 1 --2- 172.21.15.5:0/3446775193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3be8107140 0x7f3be810ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.949+0000 7f3bee200640 1 --2- 172.21.15.5:0/3446775193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3be8102a20 0x7f3be8106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.949+0000 7f3bee200640 1 --2- 172.21.15.5:0/3446775193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3be8102070 0x7f3be8102450 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.949+0000 7f3bee200640 1 -- 172.21.15.5:0/3446775193 >> 172.21.15.5:0/3446775193 conn(0x7f3be80fd950 msgr2=0x7f3be80ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:02.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.949+0000 7f3bee200640 1 -- 172.21.15.5:0/3446775193 shutdown_connections 2024-04-24T06:48:02.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.949+0000 7f3bee200640 1 -- 172.21.15.5:0/3446775193 wait complete. 2024-04-24T06:48:02.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.950+0000 7f3bee200640 1 Processor -- start 2024-04-24T06:48:02.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.950+0000 7f3bee200640 1 -- start start 2024-04-24T06:48:02.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.950+0000 7f3bee200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3be8102070 0x7f3be8198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:02.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.950+0000 7f3bee200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3be8107140 0x7f3be8198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:02.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.951+0000 7f3bee200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3be819cfd0 0x7f3be819d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:02.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.951+0000 7f3bee200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3be8112250 con 0x7f3be819cfd0 2024-04-24T06:48:02.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.951+0000 7f3bee200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3be81120d0 con 0x7f3be8102070 2024-04-24T06:48:02.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.951+0000 7f3bee200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3be81123d0 con 0x7f3be8107140 2024-04-24T06:48:02.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.951+0000 7f3be7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3be8107140 0x7f3be8198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:02.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.951+0000 7f3bed800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3be819cfd0 0x7f3be819d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:02.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.951+0000 7f3bece00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3be8102070 0x7f3be8198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:02.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.951+0000 7f3be7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3be8107140 0x7f3be8198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:54326/0 (socket says 172.21.15.5:54326) 2024-04-24T06:48:02.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.951+0000 7f3be7e00640 1 -- 172.21.15.5:0/4190547155 learned_addr learned my addr 172.21.15.5:0/4190547155 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:02.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.951+0000 7f3be7e00640 1 -- 172.21.15.5:0/4190547155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3be8102070 msgr2=0x7f3be8198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:02.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.951+0000 7f3be7e00640 1 --2- 172.21.15.5:0/4190547155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3be8102070 0x7f3be8198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.951+0000 7f3be7e00640 1 -- 172.21.15.5:0/4190547155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3be819cfd0 msgr2=0x7f3be819d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:02.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.951+0000 7f3be7e00640 1 --2- 172.21.15.5:0/4190547155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3be819cfd0 0x7f3be819d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.951+0000 7f3be7e00640 1 -- 172.21.15.5:0/4190547155 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be819dac0 con 0x7f3be8107140 2024-04-24T06:48:02.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.952+0000 7f3be7e00640 1 --2- 172.21.15.5:0/4190547155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3be8107140 0x7f3be8198c50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3bd000ccb0 tx=0x7f3bd000a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:02.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.952+0000 7f3be5600640 1 -- 172.21.15.5:0/4190547155 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0004b20 con 0x7f3be8107140 2024-04-24T06:48:02.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.952+0000 7f3be5600640 1 -- 172.21.15.5:0/4190547155 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bd0004cc0 con 0x7f3be8107140 2024-04-24T06:48:02.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.952+0000 7f3be5600640 1 -- 172.21.15.5:0/4190547155 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd000f830 con 0x7f3be8107140 2024-04-24T06:48:02.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.952+0000 7f3bee200640 1 -- 172.21.15.5:0/4190547155 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3be819ddb0 con 0x7f3be8107140 2024-04-24T06:48:02.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.952+0000 7f3bee200640 1 -- 172.21.15.5:0/4190547155 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3be81a55f0 con 0x7f3be8107140 2024-04-24T06:48:02.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.955+0000 7f3bf2640740 1 Processor -- start 2024-04-24T06:48:02.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.955+0000 7f3be5600640 1 -- 172.21.15.5:0/4190547155 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f3bd000f9d0 con 0x7f3be8107140 2024-04-24T06:48:02.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.956+0000 7f3bf2640740 1 -- start start 2024-04-24T06:48:02.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.956+0000 7f3bf2640740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec76bd8f50 0x55ec76bd9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:02.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.956+0000 7f3bf2640740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec76bd9870 0x55ec76be3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:02.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.956+0000 7f3bf2640740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ec76be3f40 0x55ec76be6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:02.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.956+0000 7f3bf2640740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ec76b3a1d0 con 0x55ec76bd8f50 2024-04-24T06:48:02.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.956+0000 7f3bf2640740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ec76ae0450 con 0x55ec76be3f40 2024-04-24T06:48:02.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.956+0000 7f3bf2640740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ec76b36190 con 0x55ec76bd9870 2024-04-24T06:48:02.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.956+0000 7f3bed800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec76bd8f50 0x55ec76bd9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:02.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.956+0000 7f3bed800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec76bd8f50 0x55ec76bd9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59030/0 (socket says 172.21.15.5:59030) 2024-04-24T06:48:02.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.956+0000 7f3bed800640 1 -- 172.21.15.5:0/2074803588 learned_addr learned my addr 172.21.15.5:0/2074803588 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:02.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.956+0000 7f3bece00640 1 --2- 172.21.15.5:0/2074803588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec76bd9870 0x55ec76be3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:02.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.957+0000 7f3be7e00640 1 --2- 172.21.15.5:0/2074803588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ec76be3f40 0x55ec76be6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:02.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.957+0000 7f3bece00640 1 -- 172.21.15.5:0/2074803588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ec76be3f40 msgr2=0x55ec76be6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:02.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.957+0000 7f3bece00640 1 --2- 172.21.15.5:0/2074803588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ec76be3f40 0x55ec76be6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.957+0000 7f3bece00640 1 -- 172.21.15.5:0/2074803588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec76bd8f50 msgr2=0x55ec76bd9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:02.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.957+0000 7f3bece00640 1 --2- 172.21.15.5:0/2074803588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec76bd8f50 0x55ec76bd9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.957+0000 7f3bece00640 1 -- 172.21.15.5:0/2074803588 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec76b9d310 con 0x55ec76bd9870 2024-04-24T06:48:02.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.957+0000 7f3be7e00640 1 --2- 172.21.15.5:0/2074803588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ec76be3f40 0x55ec76be6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:48:02.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.957+0000 7f3bed800640 1 --2- 172.21.15.5:0/2074803588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec76bd8f50 0x55ec76bd9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:48:02.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.957+0000 7f3bece00640 1 --2- 172.21.15.5:0/2074803588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec76bd9870 0x55ec76be3a00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3bd802a9e0 tx=0x7f3bd80041a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dad8e5d540522130 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:02.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.957+0000 7f3be5600640 1 --2- 172.21.15.5:0/4190547155 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f3bc80a3da0 0x7f3bc80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:02.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.958+0000 7f3bde000640 1 -- 172.21.15.5:0/2074803588 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8035040 con 0x55ec76bd9870 2024-04-24T06:48:02.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.958+0000 7f3be5600640 1 -- 172.21.15.5:0/4190547155 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f3bd00d0ae0 con 0x7f3be8107140 2024-04-24T06:48:02.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.958+0000 7f3bde000640 1 -- 172.21.15.5:0/2074803588 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bd8005520 con 0x55ec76bd9870 2024-04-24T06:48:02.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.958+0000 7f3bde000640 1 -- 172.21.15.5:0/2074803588 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8003850 con 0x55ec76bd9870 2024-04-24T06:48:02.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.958+0000 7f3bed800640 1 --2- 172.21.15.5:0/4190547155 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f3bc80a3da0 0x7f3bc80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:02.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.958+0000 7f3bf2640740 1 -- 172.21.15.5:0/2074803588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec76bd9870 msgr2=0x55ec76be3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:02.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.958+0000 7f3bf2640740 1 --2- 172.21.15.5:0/2074803588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec76bd9870 0x55ec76be3a00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3bd802a9e0 tx=0x7f3bd80041a0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.959+0000 7f3bed800640 1 --2- 172.21.15.5:0/4190547155 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f3bc80a3da0 0x7f3bc80a6260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3be000b990 tx=0x7f3be000b2e0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:02.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.959+0000 7f3bf2640740 1 -- 172.21.15.5:0/2074803588 shutdown_connections 2024-04-24T06:48:02.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.959+0000 7f3bf2640740 1 --2- 172.21.15.5:0/2074803588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ec76be3f40 0x55ec76be6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.959+0000 7f3bf2640740 1 --2- 172.21.15.5:0/2074803588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec76bd9870 0x55ec76be3a00 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.959+0000 7f3bf2640740 1 --2- 172.21.15.5:0/2074803588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec76bd8f50 0x55ec76bd9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.959+0000 7f3bf2640740 1 -- 172.21.15.5:0/2074803588 >> 172.21.15.5:0/2074803588 conn(0x55ec76bd5270 msgr2=0x55ec76bd5650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:02.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.959+0000 7f3bf2640740 1 -- 172.21.15.5:0/2074803588 shutdown_connections 2024-04-24T06:48:02.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.959+0000 7f3bf2640740 1 -- 172.21.15.5:0/2074803588 wait complete. 2024-04-24T06:48:02.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.959+0000 7f3bf2640740 1 -- start start 2024-04-24T06:48:02.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.959+0000 7f3bf2640740 1 Processor -- start 2024-04-24T06:48:02.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.960+0000 7f3bf2640740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ec76bd8f50 0x55ec76be39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:02.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.960+0000 7f3bf2640740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec76bd9870 0x55ec76be0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:02.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.960+0000 7f3bf2640740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec76be3f40 0x55ec76be13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:02.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.960+0000 7f3bf2640740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ec76be6d70 con 0x55ec76be3f40 2024-04-24T06:48:02.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.960+0000 7f3bf2640740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ec76af3810 con 0x55ec76bd8f50 2024-04-24T06:48:02.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.960+0000 7f3bf2640740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ec76be6ef0 con 0x55ec76bd9870 2024-04-24T06:48:02.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.960+0000 7f3bf2640740 10 client.0 _ll_register_callbacks cb 0x55ec76ac1140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:48:02.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.960+0000 7f3be7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec76bd9870 0x55ec76be0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:02.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.960+0000 7f3bed800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec76be3f40 0x55ec76be13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:02.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.960+0000 7f3be7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec76bd9870 0x55ec76be0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:54336/0 (socket says 172.21.15.5:54336) 2024-04-24T06:48:02.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.960+0000 7f3be7e00640 1 -- 172.21.15.5:0/2342776532 learned_addr learned my addr 172.21.15.5:0/2342776532 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:02.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.960+0000 7f3bece00640 1 --2- 172.21.15.5:0/2342776532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ec76bd8f50 0x55ec76be39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:02.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.961+0000 7f3be7e00640 1 -- 172.21.15.5:0/2342776532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ec76bd8f50 msgr2=0x55ec76be39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:02.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.961+0000 7f3be7e00640 1 --2- 172.21.15.5:0/2342776532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ec76bd8f50 0x55ec76be39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.961+0000 7f3be7e00640 1 -- 172.21.15.5:0/2342776532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec76be3f40 msgr2=0x55ec76be13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:02.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.961+0000 7f3be7e00640 1 --2- 172.21.15.5:0/2342776532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec76be3f40 0x55ec76be13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.961+0000 7f3be7e00640 1 -- 172.21.15.5:0/2342776532 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec76b043f0 con 0x55ec76bd9870 2024-04-24T06:48:02.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.961+0000 7f3bed800640 1 --2- 172.21.15.5:0/2342776532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec76be3f40 0x55ec76be13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:48:02.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.961+0000 7f3be7e00640 1 --2- 172.21.15.5:0/2342776532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec76bd9870 0x55ec76be0ca0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3bd009cad0 tx=0x7f3bd0098de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba603f072015db09 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:02.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.961+0000 7f3bd6a00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:48:02.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.961+0000 7f3bd6a00640 1 -- 172.21.15.5:0/2342776532 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd001d040 con 0x55ec76bd9870 2024-04-24T06:48:02.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.961+0000 7f3bf2640740 1 -- 172.21.15.5:0/2342776532 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bd8005520 con 0x55ec76bd9870 2024-04-24T06:48:02.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.961+0000 7f3bd6a00640 1 -- 172.21.15.5:0/2342776532 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f3bd00991b0 con 0x55ec76bd9870 2024-04-24T06:48:02.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.962+0000 7f3bd6a00640 1 -- 172.21.15.5:0/2342776532 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3bd009c4a0 con 0x55ec76bd9870 2024-04-24T06:48:02.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.962+0000 7f3bf2640740 10 client.26179 fetch_fsmap learned FSMap version 25 2024-04-24T06:48:02.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.962+0000 7f3bf2640740 1 -- 172.21.15.5:0/2342776532 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ec7684c3f0 con 0x55ec76bd9870 2024-04-24T06:48:02.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.962+0000 7f3bd6a00640 1 -- 172.21.15.5:0/2342776532 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bd8005520 con 0x55ec76bd9870 2024-04-24T06:48:02.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.962+0000 7f3bd6a00640 1 -- 172.21.15.5:0/2342776532 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3bd000fcc0 con 0x55ec76bd9870 2024-04-24T06:48:02.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.962+0000 7f3bf2640740 10 client.26179 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:48:02.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.962+0000 7f3bf2640740 10 client.26179 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T06:48:02.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.962+0000 7f3bf2640740 20 client.26179 populate_metadata read hostname 'smithi005' 2024-04-24T06:48:02.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.962+0000 7f3bf2640740 1 --2- 172.21.15.5:0/2342776532 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55ec76bf4090 0x55ec76bf64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:02.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.962+0000 7f3bf2640740 4 client.26179 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:48:02.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.962+0000 7f3bf2640740 1 -- 172.21.15.5:0/2342776532 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec76b47bf0 con 0x55ec76bf4090 2024-04-24T06:48:02.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.963+0000 7f3bece00640 1 --2- 172.21.15.5:0/2342776532 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55ec76bf4090 0x55ec76bf64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:02.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.963+0000 7f3bece00640 1 --2- 172.21.15.5:0/2342776532 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55ec76bf4090 0x55ec76bf64c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e639ce2f213b64e0 server_cookie=4235c6fca8600500 in_seq=0 out_seq=0 2024-04-24T06:48:02.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.963+0000 7f3bd6a00640 10 client.26179 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:48:02.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.964+0000 7f3bd6a00640 1 -- 172.21.15.5:0/2342776532 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ec76b47bf0 con 0x55ec76bf4090 2024-04-24T06:48:02.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.964+0000 7f3bd6a00640 10 client.26179 handle_command_reply: tid=0 2024-04-24T06:48:02.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.964+0000 7f3bf2640740 1 client.26179 shutdown 2024-04-24T06:48:02.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.965+0000 7f3bf2640740 1 -- 172.21.15.5:0/2342776532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec76bd9870 msgr2=0x55ec76be0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:02.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.965+0000 7f3bf2640740 1 --2- 172.21.15.5:0/2342776532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec76bd9870 0x55ec76be0ca0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3bd009cad0 tx=0x7f3bd0098de0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.965+0000 7f3bf2640740 1 -- 172.21.15.5:0/2342776532 shutdown_connections 2024-04-24T06:48:02.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.965+0000 7f3bf2640740 1 --2- 172.21.15.5:0/2342776532 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55ec76bf4090 0x55ec76bf64c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.965+0000 7f3bf2640740 1 --2- 172.21.15.5:0/2342776532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec76be3f40 0x55ec76be13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.965+0000 7f3bf2640740 1 --2- 172.21.15.5:0/2342776532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec76bd9870 0x55ec76be0ca0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.965+0000 7f3bf2640740 1 --2- 172.21.15.5:0/2342776532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ec76bd8f50 0x55ec76be39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.965+0000 7f3bd6a00640 0 client.26179 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:48:02.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.965+0000 7f3bf2640740 1 -- 172.21.15.5:0/2342776532 >> 172.21.15.5:0/2342776532 conn(0x55ec76bd5270 msgr2=0x55ec76bdb070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:02.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.965+0000 7f3bf2640740 1 -- 172.21.15.5:0/2342776532 shutdown_connections 2024-04-24T06:48:02.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.965+0000 7f3bf2640740 1 -- 172.21.15.5:0/2342776532 wait complete. 2024-04-24T06:48:02.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.965+0000 7f3bf2640740 20 client.26179 trim_cache size 0 max 16384 2024-04-24T06:48:02.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.980+0000 7f3bf2640740 1 Processor -- start 2024-04-24T06:48:02.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.980+0000 7f3bf2640740 1 -- start start 2024-04-24T06:48:02.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.980+0000 7f3bf2640740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec76bd8f50 0x55ec76befc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:02.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.980+0000 7f3bf2640740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ec76bd9870 0x55ec76be34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:02.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.980+0000 7f3bf2640740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec76be3f40 0x55ec76bfbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:02.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.980+0000 7f3bf2640740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3be81ae2a0 con 0x55ec76bd8f50 2024-04-24T06:48:02.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.980+0000 7f3bf2640740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3be81abb70 con 0x55ec76bd9870 2024-04-24T06:48:02.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.980+0000 7f3bf2640740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3be81a99c0 con 0x55ec76be3f40 2024-04-24T06:48:02.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.980+0000 7f3bece00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec76bd8f50 0x55ec76befc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:02.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.981+0000 7f3bece00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec76bd8f50 0x55ec76befc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59050/0 (socket says 172.21.15.5:59050) 2024-04-24T06:48:02.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.981+0000 7f3bece00640 1 -- 172.21.15.5:0/2107033718 learned_addr learned my addr 172.21.15.5:0/2107033718 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:02.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.981+0000 7f3be7e00640 1 --2- 172.21.15.5:0/2107033718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ec76bd9870 0x55ec76be34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:02.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.981+0000 7f3bed800640 1 --2- 172.21.15.5:0/2107033718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec76be3f40 0x55ec76bfbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:02.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.981+0000 7f3bece00640 1 -- 172.21.15.5:0/2107033718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec76be3f40 msgr2=0x55ec76bfbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:02.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.981+0000 7f3bece00640 1 --2- 172.21.15.5:0/2107033718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec76be3f40 0x55ec76bfbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.981+0000 7f3bece00640 1 -- 172.21.15.5:0/2107033718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ec76bd9870 msgr2=0x55ec76be34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:02.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.981+0000 7f3bece00640 1 --2- 172.21.15.5:0/2107033718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ec76bd9870 0x55ec76be34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.981+0000 7f3bece00640 1 -- 172.21.15.5:0/2107033718 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec769167d0 con 0x55ec76bd8f50 2024-04-24T06:48:02.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.981+0000 7f3bece00640 1 --2- 172.21.15.5:0/2107033718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec76bd8f50 0x55ec76befc30 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8009900 tx=0x7f3bd8004df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c65c752ca6d82f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:02.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.982+0000 7f3bdd600640 1 -- 172.21.15.5:0/2107033718 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8057070 con 0x55ec76bd8f50 2024-04-24T06:48:02.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.982+0000 7f3bdd600640 1 -- 172.21.15.5:0/2107033718 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bd80046c0 con 0x55ec76bd8f50 2024-04-24T06:48:02.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.982+0000 7f3bdd600640 1 -- 172.21.15.5:0/2107033718 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8035040 con 0x55ec76bd8f50 2024-04-24T06:48:02.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.982+0000 7f3bf2640740 1 -- 172.21.15.5:0/2107033718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec76bd8f50 msgr2=0x55ec76befc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:02.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.982+0000 7f3bf2640740 1 --2- 172.21.15.5:0/2107033718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec76bd8f50 0x55ec76befc30 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8009900 tx=0x7f3bd8004df0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.982+0000 7f3bf2640740 1 -- 172.21.15.5:0/2107033718 shutdown_connections 2024-04-24T06:48:02.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.982+0000 7f3bf2640740 1 --2- 172.21.15.5:0/2107033718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec76be3f40 0x55ec76bfbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.982+0000 7f3bf2640740 1 --2- 172.21.15.5:0/2107033718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ec76bd9870 0x55ec76be34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.982+0000 7f3bf2640740 1 --2- 172.21.15.5:0/2107033718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec76bd8f50 0x55ec76befc30 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.982+0000 7f3bf2640740 1 -- 172.21.15.5:0/2107033718 >> 172.21.15.5:0/2107033718 conn(0x55ec76bd5270 msgr2=0x55ec76bdf5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:02.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.983+0000 7f3bf2640740 1 -- 172.21.15.5:0/2107033718 shutdown_connections 2024-04-24T06:48:02.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.983+0000 7f3bf2640740 1 -- 172.21.15.5:0/2107033718 wait complete. 2024-04-24T06:48:02.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.983+0000 7f3bf2640740 1 -- start start 2024-04-24T06:48:02.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.983+0000 7f3bf2640740 1 Processor -- start 2024-04-24T06:48:02.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.983+0000 7f3bf2640740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ec76bd8f50 0x55ec76bec330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:02.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.983+0000 7f3bf2640740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec76bd9870 0x55ec76bddf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:02.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.984+0000 7f3bf2640740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec76be3f40 0x55ec76bde4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:02.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.984+0000 7f3bf2640740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3be81aa030 con 0x55ec76be3f40 2024-04-24T06:48:02.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.984+0000 7f3bf2640740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3be81ab4a0 con 0x55ec76bd8f50 2024-04-24T06:48:02.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.984+0000 7f3bf2640740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3be81aa700 con 0x55ec76bd9870 2024-04-24T06:48:02.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.984+0000 7f3bf2640740 10 client.0 _ll_register_callbacks cb 0x7f3bd0005b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:48:02.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.984+0000 7f3bed800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec76be3f40 0x55ec76bde4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:02.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.984+0000 7f3be7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec76bd9870 0x55ec76bddf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:02.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.984+0000 7f3bece00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ec76bd8f50 0x55ec76bec330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:02.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.984+0000 7f3be7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec76bd9870 0x55ec76bddf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:54354/0 (socket says 172.21.15.5:54354) 2024-04-24T06:48:02.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.984+0000 7f3be7e00640 1 -- 172.21.15.5:0/3836939686 learned_addr learned my addr 172.21.15.5:0/3836939686 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:02.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.984+0000 7f3be7e00640 1 -- 172.21.15.5:0/3836939686 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ec76bd8f50 msgr2=0x55ec76bec330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:02.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.984+0000 7f3be7e00640 1 --2- 172.21.15.5:0/3836939686 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ec76bd8f50 0x55ec76bec330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.984+0000 7f3be7e00640 1 -- 172.21.15.5:0/3836939686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec76be3f40 msgr2=0x55ec76bde4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:02.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.984+0000 7f3be7e00640 1 --2- 172.21.15.5:0/3836939686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec76be3f40 0x55ec76bde4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.984+0000 7f3be7e00640 1 -- 172.21.15.5:0/3836939686 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec76bde9e0 con 0x55ec76bd9870 2024-04-24T06:48:02.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.984+0000 7f3bece00640 1 --2- 172.21.15.5:0/3836939686 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ec76bd8f50 0x55ec76bec330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:48:02.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.985+0000 7f3be7e00640 1 --2- 172.21.15.5:0/3836939686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec76bd9870 0x55ec76bddf60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3bd00ce840 tx=0x7f3bd009c290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=761e026c7febfa44 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:02.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.985+0000 7f3bd6a00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:48:02.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.985+0000 7f3bd6a00640 1 -- 172.21.15.5:0/3836939686 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd001d070 con 0x55ec76bd9870 2024-04-24T06:48:02.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.985+0000 7f3bf2640740 1 -- 172.21.15.5:0/3836939686 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bd80046c0 con 0x55ec76bd9870 2024-04-24T06:48:02.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.985+0000 7f3bd6a00640 1 -- 172.21.15.5:0/3836939686 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f3bd009ba00 con 0x55ec76bd9870 2024-04-24T06:48:02.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.985+0000 7f3bd6a00640 1 -- 172.21.15.5:0/3836939686 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3bd0082050 con 0x55ec76bd9870 2024-04-24T06:48:02.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.985+0000 7f3bf2640740 10 client.26185 fetch_fsmap learned FSMap version 25 2024-04-24T06:48:02.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.985+0000 7f3bf2640740 1 -- 172.21.15.5:0/3836939686 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ec76be5630 con 0x55ec76bd9870 2024-04-24T06:48:02.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.985+0000 7f3bd6a00640 1 -- 172.21.15.5:0/3836939686 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bd80046c0 con 0x55ec76bd9870 2024-04-24T06:48:02.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.986+0000 7f3bd6a00640 1 -- 172.21.15.5:0/3836939686 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3bd00870f0 con 0x55ec76bd9870 2024-04-24T06:48:02.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.986+0000 7f3bf2640740 10 client.26185 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:48:02.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.986+0000 7f3bf2640740 10 client.26185 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T06:48:02.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.986+0000 7f3bf2640740 20 client.26185 populate_metadata read hostname 'smithi005' 2024-04-24T06:48:02.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.986+0000 7f3bf2640740 1 --2- 172.21.15.5:0/3836939686 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55ec76c5aa20 0x55ec76c5ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:02.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.986+0000 7f3bf2640740 4 client.26185 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:48:02.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.986+0000 7f3bf2640740 1 -- 172.21.15.5:0/3836939686 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ec76b41a60 con 0x55ec76c5aa20 2024-04-24T06:48:02.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.986+0000 7f3bece00640 1 --2- 172.21.15.5:0/3836939686 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55ec76c5aa20 0x55ec76c5ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:02.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.987+0000 7f3bece00640 1 --2- 172.21.15.5:0/3836939686 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55ec76c5aa20 0x55ec76c5ce40 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9f8908413cd8569f server_cookie=633d8341a9f42379 in_seq=0 out_seq=0 2024-04-24T06:48:02.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.987+0000 7f3bd6a00640 10 client.26185 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:48:02.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.987+0000 7f3bd6a00640 1 -- 172.21.15.5:0/3836939686 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ec76b41a60 con 0x55ec76c5aa20 2024-04-24T06:48:02.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.987+0000 7f3bd6a00640 10 client.26185 handle_command_reply: tid=0 2024-04-24T06:48:02.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.987+0000 7f3bf2640740 1 client.26185 shutdown 2024-04-24T06:48:02.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.987+0000 7f3bf2640740 1 -- 172.21.15.5:0/3836939686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec76bd9870 msgr2=0x55ec76bddf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:02.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.987+0000 7f3bf2640740 1 --2- 172.21.15.5:0/3836939686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec76bd9870 0x55ec76bddf60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3bd00ce840 tx=0x7f3bd009c290 comp rx=0 tx=0).stop 2024-04-24T06:48:02.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.987+0000 7f3bf2640740 1 -- 172.21.15.5:0/3836939686 shutdown_connections 2024-04-24T06:48:02.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.987+0000 7f3bf2640740 1 --2- 172.21.15.5:0/3836939686 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55ec76c5aa20 0x55ec76c5ce40 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.987+0000 7f3bf2640740 1 --2- 172.21.15.5:0/3836939686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec76be3f40 0x55ec76bde4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.987+0000 7f3bf2640740 1 --2- 172.21.15.5:0/3836939686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec76bd9870 0x55ec76bddf60 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.987+0000 7f3bf2640740 1 --2- 172.21.15.5:0/3836939686 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ec76bd8f50 0x55ec76bec330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:02.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.987+0000 7f3bf2640740 1 -- 172.21.15.5:0/3836939686 >> 172.21.15.5:0/3836939686 conn(0x55ec76bd5270 msgr2=0x55ec76bcf6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:02.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.987+0000 7f3bd6a00640 0 client.26185 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:48:02.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.987+0000 7f3bf2640740 1 -- 172.21.15.5:0/3836939686 shutdown_connections 2024-04-24T06:48:02.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.987+0000 7f3bf2640740 1 -- 172.21.15.5:0/3836939686 wait complete. 2024-04-24T06:48:02.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.987+0000 7f3bf2640740 20 client.26185 trim_cache size 0 max 16384 2024-04-24T06:48:02.999 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:48:02.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.988+0000 7f3bee200640 1 -- 172.21.15.5:0/4190547155 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f3bc80a3da0 msgr2=0x7f3bc80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:03.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.988+0000 7f3bee200640 1 --2- 172.21.15.5:0/4190547155 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f3bc80a3da0 0x7f3bc80a6260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3be000b990 tx=0x7f3be000b2e0 comp rx=0 tx=0).stop 2024-04-24T06:48:03.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.988+0000 7f3bee200640 1 -- 172.21.15.5:0/4190547155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3be8107140 msgr2=0x7f3be8198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:03.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.988+0000 7f3bee200640 1 --2- 172.21.15.5:0/4190547155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3be8107140 0x7f3be8198c50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3bd000ccb0 tx=0x7f3bd000a620 comp rx=0 tx=0).stop 2024-04-24T06:48:03.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.988+0000 7f3bee200640 1 -- 172.21.15.5:0/4190547155 shutdown_connections 2024-04-24T06:48:03.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.988+0000 7f3bee200640 1 --2- 172.21.15.5:0/4190547155 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f3bc80a3da0 0x7f3bc80a6260 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:03.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.988+0000 7f3bee200640 1 --2- 172.21.15.5:0/4190547155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3be819cfd0 0x7f3be819d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:03.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.988+0000 7f3bee200640 1 --2- 172.21.15.5:0/4190547155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3be8107140 0x7f3be8198c50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:03.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.988+0000 7f3bee200640 1 --2- 172.21.15.5:0/4190547155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3be8102070 0x7f3be8198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:03.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.988+0000 7f3bee200640 1 -- 172.21.15.5:0/4190547155 >> 172.21.15.5:0/4190547155 conn(0x7f3be80fd950 msgr2=0x7f3be80ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:03.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.988+0000 7f3bee200640 1 -- 172.21.15.5:0/4190547155 shutdown_connections 2024-04-24T06:48:03.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:02.988+0000 7f3bee200640 1 -- 172.21.15.5:0/4190547155 wait complete. 2024-04-24T06:48:03.009 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T06:48:03.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:02 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/4022639635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:48:03.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.223+0000 7fbb5f600640 1 -- 172.21.15.5:0/84033304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb58109520 msgr2=0x7fbb581110f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:03.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.223+0000 7fbb5f600640 1 --2- 172.21.15.5:0/84033304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb58109520 0x7fbb581110f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbb50009ca0 tx=0x7fbb5002a8c0 comp rx=0 tx=0).stop 2024-04-24T06:48:03.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.224+0000 7fbb5f600640 1 -- 172.21.15.5:0/84033304 shutdown_connections 2024-04-24T06:48:03.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.224+0000 7fbb5f600640 1 --2- 172.21.15.5:0/84033304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb58109520 0x7fbb581110f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:03.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.224+0000 7fbb5f600640 1 --2- 172.21.15.5:0/84033304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb58104e00 0x7fbb58108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:03.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.224+0000 7fbb5f600640 1 --2- 172.21.15.5:0/84033304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb58104450 0x7fbb58104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:03.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.224+0000 7fbb5f600640 1 -- 172.21.15.5:0/84033304 >> 172.21.15.5:0/84033304 conn(0x7fbb580ffc90 msgr2=0x7fbb581020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:03.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.224+0000 7fbb5f600640 1 -- 172.21.15.5:0/84033304 shutdown_connections 2024-04-24T06:48:03.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.224+0000 7fbb5f600640 1 -- 172.21.15.5:0/84033304 wait complete. 2024-04-24T06:48:03.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.225+0000 7fbb5f600640 1 Processor -- start 2024-04-24T06:48:03.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.225+0000 7fbb5f600640 1 -- start start 2024-04-24T06:48:03.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.225+0000 7fbb5f600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb58104450 0x7fbb581a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:03.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.225+0000 7fbb5f600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb58109520 0x7fbb581a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:03.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.225+0000 7fbb5f600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb5819a8c0 0x7fbb5819ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:03.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.225+0000 7fbb5f600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbb58114600 con 0x7fbb58109520 2024-04-24T06:48:03.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.225+0000 7fbb5f600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbb58114480 con 0x7fbb5819a8c0 2024-04-24T06:48:03.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.225+0000 7fbb5f600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbb58114780 con 0x7fbb58104450 2024-04-24T06:48:03.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.226+0000 7fbb5cc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb58104450 0x7fbb581a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:03.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.226+0000 7fbb57e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb58109520 0x7fbb581a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:03.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.226+0000 7fbb5d600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb5819a8c0 0x7fbb5819ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:03.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.226+0000 7fbb5cc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb58104450 0x7fbb581a0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:54364/0 (socket says 172.21.15.5:54364) 2024-04-24T06:48:03.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.226+0000 7fbb5cc00640 1 -- 172.21.15.5:0/2026333958 learned_addr learned my addr 172.21.15.5:0/2026333958 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:03.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.226+0000 7fbb5cc00640 1 -- 172.21.15.5:0/2026333958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb5819a8c0 msgr2=0x7fbb5819ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:03.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.226+0000 7fbb5cc00640 1 --2- 172.21.15.5:0/2026333958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb5819a8c0 0x7fbb5819ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:03.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.226+0000 7fbb5cc00640 1 -- 172.21.15.5:0/2026333958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb58109520 msgr2=0x7fbb581a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:03.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.226+0000 7fbb5cc00640 1 --2- 172.21.15.5:0/2026333958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb58109520 0x7fbb581a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:03.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.226+0000 7fbb5cc00640 1 -- 172.21.15.5:0/2026333958 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb5819b3e0 con 0x7fbb58104450 2024-04-24T06:48:03.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.226+0000 7fbb57e00640 1 --2- 172.21.15.5:0/2026333958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb58109520 0x7fbb581a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:48:03.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.227+0000 7fbb5cc00640 1 --2- 172.21.15.5:0/2026333958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb58104450 0x7fbb581a0710 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800e9f0 tx=0x7fbb4800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:03.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.227+0000 7fbb55600640 1 -- 172.21.15.5:0/2026333958 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4800cd80 con 0x7fbb58104450 2024-04-24T06:48:03.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.227+0000 7fbb55600640 1 -- 172.21.15.5:0/2026333958 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb48010910 con 0x7fbb58104450 2024-04-24T06:48:03.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.227+0000 7fbb5f600640 1 -- 172.21.15.5:0/2026333958 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb5819b6d0 con 0x7fbb58104450 2024-04-24T06:48:03.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.227+0000 7fbb55600640 1 -- 172.21.15.5:0/2026333958 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb480188f0 con 0x7fbb58104450 2024-04-24T06:48:03.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.227+0000 7fbb5f600640 1 -- 172.21.15.5:0/2026333958 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb581a7920 con 0x7fbb58104450 2024-04-24T06:48:03.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.231+0000 7fbb623dc740 1 Processor -- start 2024-04-24T06:48:03.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.231+0000 7fbb623dc740 1 -- start start 2024-04-24T06:48:03.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.232+0000 7fbb623dc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99b814950 0x55d99b814d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:03.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.232+0000 7fbb623dc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d99b815270 0x55d99b81f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:03.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.232+0000 7fbb623dc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99b81f940 0x55d99b821d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:03.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.232+0000 7fbb623dc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d99b77a1d0 con 0x55d99b81f940 2024-04-24T06:48:03.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.232+0000 7fbb623dc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d99b720450 con 0x55d99b814950 2024-04-24T06:48:03.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.232+0000 7fbb623dc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d99b776190 con 0x55d99b815270 2024-04-24T06:48:03.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.232+0000 7fbb55600640 1 -- 172.21.15.5:0/2026333958 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fbb48010ac0 con 0x7fbb58104450 2024-04-24T06:48:03.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.232+0000 7fbb57e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99b81f940 0x55d99b821d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:03.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.232+0000 7fbb57e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99b81f940 0x55d99b821d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59086/0 (socket says 172.21.15.5:59086) 2024-04-24T06:48:03.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.232+0000 7fbb57e00640 1 -- 172.21.15.5:0/1202732163 learned_addr learned my addr 172.21.15.5:0/1202732163 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:03.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.232+0000 7fbb5d600640 1 --2- 172.21.15.5:0/1202732163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99b814950 0x55d99b814d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:03.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.233+0000 7fbb5cc00640 1 --2- 172.21.15.5:0/1202732163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d99b815270 0x55d99b81f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:03.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.233+0000 7fbb57e00640 1 -- 172.21.15.5:0/1202732163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d99b815270 msgr2=0x55d99b81f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:03.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.233+0000 7fbb57e00640 1 --2- 172.21.15.5:0/1202732163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d99b815270 0x55d99b81f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:03.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.233+0000 7fbb57e00640 1 -- 172.21.15.5:0/1202732163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99b814950 msgr2=0x55d99b814d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:03.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.233+0000 7fbb57e00640 1 --2- 172.21.15.5:0/1202732163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99b814950 0x55d99b814d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:03.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.233+0000 7fbb57e00640 1 -- 172.21.15.5:0/1202732163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d99b7dd310 con 0x55d99b81f940 2024-04-24T06:48:03.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.233+0000 7fbb5d600640 1 --2- 172.21.15.5:0/1202732163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99b814950 0x55d99b814d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:48:03.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.233+0000 7fbb57e00640 1 --2- 172.21.15.5:0/1202732163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99b81f940 0x55d99b821d30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbb40009fd0 tx=0x7fbb400118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b130791e86006142 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:03.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.233+0000 7fbb4dc00640 1 -- 172.21.15.5:0/1202732163 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4001e070 con 0x55d99b81f940 2024-04-24T06:48:03.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.234+0000 7fbb4dc00640 1 -- 172.21.15.5:0/1202732163 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb400035f0 con 0x55d99b81f940 2024-04-24T06:48:03.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.234+0000 7fbb4dc00640 1 -- 172.21.15.5:0/1202732163 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4000f7a0 con 0x55d99b81f940 2024-04-24T06:48:03.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.234+0000 7fbb55600640 1 --2- 172.21.15.5:0/2026333958 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fbb3c0a3d20 0x7fbb3c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:03.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.234+0000 7fbb623dc740 1 -- 172.21.15.5:0/1202732163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99b81f940 msgr2=0x55d99b821d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:03.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.234+0000 7fbb623dc740 1 --2- 172.21.15.5:0/1202732163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99b81f940 0x55d99b821d30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbb40009fd0 tx=0x7fbb400118d0 comp rx=0 tx=0).stop 2024-04-24T06:48:03.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.234+0000 7fbb623dc740 1 -- 172.21.15.5:0/1202732163 shutdown_connections 2024-04-24T06:48:03.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.234+0000 7fbb623dc740 1 --2- 172.21.15.5:0/1202732163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99b81f940 0x55d99b821d30 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:03.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.234+0000 7fbb623dc740 1 --2- 172.21.15.5:0/1202732163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d99b815270 0x55d99b81f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:03.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.234+0000 7fbb623dc740 1 --2- 172.21.15.5:0/1202732163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99b814950 0x55d99b814d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:03.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.234+0000 7fbb55600640 1 -- 172.21.15.5:0/2026333958 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fbb480d0bf0 con 0x7fbb58104450 2024-04-24T06:48:03.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.234+0000 7fbb623dc740 1 -- 172.21.15.5:0/1202732163 >> 172.21.15.5:0/1202732163 conn(0x55d99b810c70 msgr2=0x55d99b811050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:03.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.234+0000 7fbb5d600640 1 --2- 172.21.15.5:0/2026333958 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fbb3c0a3d20 0x7fbb3c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:03.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.234+0000 7fbb623dc740 1 -- 172.21.15.5:0/1202732163 shutdown_connections 2024-04-24T06:48:03.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.235+0000 7fbb623dc740 1 -- 172.21.15.5:0/1202732163 wait complete. 2024-04-24T06:48:03.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.235+0000 7fbb623dc740 1 -- start start 2024-04-24T06:48:03.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.235+0000 7fbb623dc740 1 Processor -- start 2024-04-24T06:48:03.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.235+0000 7fbb5d600640 1 --2- 172.21.15.5:0/2026333958 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fbb3c0a3d20 0x7fbb3c0a61e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbb50006010 tx=0x7fbb50005d80 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:03.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.236+0000 7fbb623dc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99b814950 0x55d99b81f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:03.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.236+0000 7fbb623dc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99b815270 0x55d99b81c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:03.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.236+0000 7fbb623dc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d99b81f940 0x55d99b81cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:03.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.236+0000 7fbb623dc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d99b8227d0 con 0x55d99b814950 2024-04-24T06:48:03.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.236+0000 7fbb623dc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d99b733810 con 0x55d99b815270 2024-04-24T06:48:03.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.236+0000 7fbb623dc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d99b822950 con 0x55d99b81f940 2024-04-24T06:48:03.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.236+0000 7fbb623dc740 10 client.0 _ll_register_callbacks cb 0x55d99b701140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:48:03.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.236+0000 7fbb5cc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99b814950 0x55d99b81f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:03.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.236+0000 7fbb5cc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99b814950 0x55d99b81f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59100/0 (socket says 172.21.15.5:59100) 2024-04-24T06:48:03.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.236+0000 7fbb5cc00640 1 -- 172.21.15.5:0/2681577792 learned_addr learned my addr 172.21.15.5:0/2681577792 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:03.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.236+0000 7fbb57e00640 1 --2- 172.21.15.5:0/2681577792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99b815270 0x55d99b81c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:03.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.236+0000 7fbb5d600640 1 --2- 172.21.15.5:0/2681577792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d99b81f940 0x55d99b81cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:03.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.237+0000 7fbb5cc00640 1 -- 172.21.15.5:0/2681577792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d99b81f940 msgr2=0x55d99b81cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:03.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.237+0000 7fbb5cc00640 1 --2- 172.21.15.5:0/2681577792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d99b81f940 0x55d99b81cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:03.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.237+0000 7fbb5cc00640 1 -- 172.21.15.5:0/2681577792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99b815270 msgr2=0x55d99b81c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:03.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.237+0000 7fbb5cc00640 1 --2- 172.21.15.5:0/2681577792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99b815270 0x55d99b81c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:03.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.237+0000 7fbb5cc00640 1 -- 172.21.15.5:0/2681577792 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d99b7443f0 con 0x55d99b814950 2024-04-24T06:48:03.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.237+0000 7fbb57e00640 1 --2- 172.21.15.5:0/2681577792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99b815270 0x55d99b81c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:48:03.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.237+0000 7fbb5cc00640 1 --2- 172.21.15.5:0/2681577792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99b814950 0x55d99b81f390 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbb480ce350 tx=0x7fbb4809cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7030b7865c009bb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:03.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.237+0000 7fbb46000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:48:03.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.237+0000 7fbb46000640 1 -- 172.21.15.5:0/2681577792 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4809cbf0 con 0x55d99b814950 2024-04-24T06:48:03.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.237+0000 7fbb46000640 1 -- 172.21.15.5:0/2681577792 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fbb480ce570 con 0x55d99b814950 2024-04-24T06:48:03.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.237+0000 7fbb623dc740 1 -- 172.21.15.5:0/2681577792 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb400035f0 con 0x55d99b814950 2024-04-24T06:48:03.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.237+0000 7fbb46000640 1 -- 172.21.15.5:0/2681577792 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbb480ce770 con 0x55d99b814950 2024-04-24T06:48:03.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.237+0000 7fbb623dc740 10 client.16047 fetch_fsmap learned FSMap version 25 2024-04-24T06:48:03.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.237+0000 7fbb623dc740 1 -- 172.21.15.5:0/2681577792 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d99b48c3f0 con 0x55d99b814950 2024-04-24T06:48:03.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.238+0000 7fbb46000640 1 -- 172.21.15.5:0/2681577792 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb400035f0 con 0x55d99b814950 2024-04-24T06:48:03.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.238+0000 7fbb46000640 1 -- 172.21.15.5:0/2681577792 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbb48098940 con 0x55d99b814950 2024-04-24T06:48:03.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.238+0000 7fbb623dc740 10 client.16047 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:48:03.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.238+0000 7fbb623dc740 10 client.16047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:48:03.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.238+0000 7fbb623dc740 20 client.16047 populate_metadata read hostname 'smithi005' 2024-04-24T06:48:03.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.238+0000 7fbb623dc740 1 --2- 172.21.15.5:0/2681577792 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55d99b82fb70 0x55d99b831fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:03.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.238+0000 7fbb623dc740 4 client.16047 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:48:03.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.238+0000 7fbb623dc740 1 -- 172.21.15.5:0/2681577792 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d99b787bf0 con 0x55d99b82fb70 2024-04-24T06:48:03.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.238+0000 7fbb57e00640 1 --2- 172.21.15.5:0/2681577792 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55d99b82fb70 0x55d99b831fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:03.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.239+0000 7fbb57e00640 1 --2- 172.21.15.5:0/2681577792 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55d99b82fb70 0x55d99b831fe0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78a11b914faf308a server_cookie=822f3821cb61af43 in_seq=0 out_seq=0 2024-04-24T06:48:03.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.239+0000 7fbb46000640 10 client.16047 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:48:03.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.240+0000 7fbb46000640 1 -- 172.21.15.5:0/2681577792 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d99b787bf0 con 0x55d99b82fb70 2024-04-24T06:48:03.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.240+0000 7fbb46000640 10 client.16047 handle_command_reply: tid=0 2024-04-24T06:48:03.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.240+0000 7fbb623dc740 1 client.16047 shutdown 2024-04-24T06:48:03.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.240+0000 7fbb623dc740 1 -- 172.21.15.5:0/2681577792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99b814950 msgr2=0x55d99b81f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:03.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.240+0000 7fbb623dc740 1 --2- 172.21.15.5:0/2681577792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99b814950 0x55d99b81f390 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbb480ce350 tx=0x7fbb4809cab0 comp rx=0 tx=0).stop 2024-04-24T06:48:03.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.240+0000 7fbb623dc740 1 -- 172.21.15.5:0/2681577792 shutdown_connections 2024-04-24T06:48:03.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.240+0000 7fbb623dc740 1 --2- 172.21.15.5:0/2681577792 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55d99b82fb70 0x55d99b831fe0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:03.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.240+0000 7fbb623dc740 1 --2- 172.21.15.5:0/2681577792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d99b81f940 0x55d99b81cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:03.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.240+0000 7fbb623dc740 1 --2- 172.21.15.5:0/2681577792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99b815270 0x55d99b81c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:03.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.240+0000 7fbb623dc740 1 --2- 172.21.15.5:0/2681577792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99b814950 0x55d99b81f390 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:03.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.240+0000 7fbb46000640 0 client.16047 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:48:03.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.240+0000 7fbb623dc740 1 -- 172.21.15.5:0/2681577792 >> 172.21.15.5:0/2681577792 conn(0x55d99b810c70 msgr2=0x55d99b816a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:03.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.240+0000 7fbb623dc740 1 -- 172.21.15.5:0/2681577792 shutdown_connections 2024-04-24T06:48:03.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.240+0000 7fbb623dc740 1 -- 172.21.15.5:0/2681577792 wait complete. 2024-04-24T06:48:03.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.241+0000 7fbb623dc740 20 client.16047 trim_cache size 0 max 16384 2024-04-24T06:48:03.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.256+0000 7fbb623dc740 1 Processor -- start 2024-04-24T06:48:03.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.256+0000 7fbb623dc740 1 -- start start 2024-04-24T06:48:03.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.256+0000 7fbb623dc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99b814950 0x55d99b810570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:03.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.256+0000 7fbb623dc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99b815270 0x55d99b81b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:03.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.256+0000 7fbb623dc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d99b81f940 0x55d99b81eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:03.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.256+0000 7fbb623dc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbb581b2830 con 0x55d99b815270 2024-04-24T06:48:03.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.256+0000 7fbb623dc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbb581b0100 con 0x55d99b814950 2024-04-24T06:48:03.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.256+0000 7fbb623dc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbb581adf50 con 0x55d99b81f940 2024-04-24T06:48:03.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.256+0000 7fbb5d600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d99b81f940 0x55d99b81eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:03.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.257+0000 7fbb57e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99b815270 0x55d99b81b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:03.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.257+0000 7fbb5d600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d99b81f940 0x55d99b81eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:54394/0 (socket says 172.21.15.5:54394) 2024-04-24T06:48:03.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.257+0000 7fbb5d600640 1 -- 172.21.15.5:0/2680896743 learned_addr learned my addr 172.21.15.5:0/2680896743 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:03.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.257+0000 7fbb5cc00640 1 --2- 172.21.15.5:0/2680896743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99b814950 0x55d99b810570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:03.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.257+0000 7fbb5d600640 1 -- 172.21.15.5:0/2680896743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99b814950 msgr2=0x55d99b810570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:03.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.257+0000 7fbb5d600640 1 --2- 172.21.15.5:0/2680896743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99b814950 0x55d99b810570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:03.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.257+0000 7fbb5d600640 1 -- 172.21.15.5:0/2680896743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99b815270 msgr2=0x55d99b81b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:03.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.257+0000 7fbb5d600640 1 --2- 172.21.15.5:0/2680896743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99b815270 0x55d99b81b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:03.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.257+0000 7fbb5d600640 1 -- 172.21.15.5:0/2680896743 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d99b5567d0 con 0x55d99b81f940 2024-04-24T06:48:03.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.257+0000 7fbb5cc00640 1 --2- 172.21.15.5:0/2680896743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99b814950 0x55d99b810570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:48:03.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.257+0000 7fbb57e00640 1 --2- 172.21.15.5:0/2680896743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99b815270 0x55d99b81b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:48:03.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.257+0000 7fbb5d600640 1 --2- 172.21.15.5:0/2680896743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d99b81f940 0x55d99b81eec0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbb50009ca0 tx=0x7fbb50041000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed4144befb0f61b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:03.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.258+0000 7fbb4d200640 1 -- 172.21.15.5:0/2680896743 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb50003bf0 con 0x55d99b81f940 2024-04-24T06:48:03.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.258+0000 7fbb4d200640 1 -- 172.21.15.5:0/2680896743 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb50036050 con 0x55d99b81f940 2024-04-24T06:48:03.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.258+0000 7fbb4d200640 1 -- 172.21.15.5:0/2680896743 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb50004520 con 0x55d99b81f940 2024-04-24T06:48:03.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.258+0000 7fbb623dc740 1 -- 172.21.15.5:0/2680896743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d99b81f940 msgr2=0x55d99b81eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:03.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.258+0000 7fbb623dc740 1 --2- 172.21.15.5:0/2680896743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d99b81f940 0x55d99b81eec0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbb50009ca0 tx=0x7fbb50041000 comp rx=0 tx=0).stop 2024-04-24T06:48:03.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.258+0000 7fbb623dc740 1 -- 172.21.15.5:0/2680896743 shutdown_connections 2024-04-24T06:48:03.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.258+0000 7fbb623dc740 1 --2- 172.21.15.5:0/2680896743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d99b81f940 0x55d99b81eec0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:03.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.258+0000 7fbb623dc740 1 --2- 172.21.15.5:0/2680896743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99b815270 0x55d99b81b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:03.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.258+0000 7fbb623dc740 1 --2- 172.21.15.5:0/2680896743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99b814950 0x55d99b810570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:03.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.259+0000 7fbb623dc740 1 -- 172.21.15.5:0/2680896743 >> 172.21.15.5:0/2680896743 conn(0x55d99b810c70 msgr2=0x55d99b81af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:03.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.259+0000 7fbb623dc740 1 -- 172.21.15.5:0/2680896743 shutdown_connections 2024-04-24T06:48:03.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.259+0000 7fbb623dc740 1 -- 172.21.15.5:0/2680896743 wait complete. 2024-04-24T06:48:03.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.259+0000 7fbb623dc740 1 -- start start 2024-04-24T06:48:03.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.259+0000 7fbb623dc740 1 Processor -- start 2024-04-24T06:48:03.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.260+0000 7fbb623dc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99b814950 0x55d99b820530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:03.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.260+0000 7fbb623dc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d99b815270 0x55d99b820a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:03.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.260+0000 7fbb623dc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99b81f940 0x55d99b826ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:03.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.260+0000 7fbb623dc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbb581ae5c0 con 0x55d99b814950 2024-04-24T06:48:03.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.260+0000 7fbb623dc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbb581afa30 con 0x55d99b81f940 2024-04-24T06:48:03.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.260+0000 7fbb623dc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbb581aec90 con 0x55d99b815270 2024-04-24T06:48:03.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.260+0000 7fbb623dc740 10 client.0 _ll_register_callbacks cb 0x7fbb4800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:48:03.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.260+0000 7fbb5cc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99b814950 0x55d99b820530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:03.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.260+0000 7fbb5cc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99b814950 0x55d99b820530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59118/0 (socket says 172.21.15.5:59118) 2024-04-24T06:48:03.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.260+0000 7fbb5cc00640 1 -- 172.21.15.5:0/3768661029 learned_addr learned my addr 172.21.15.5:0/3768661029 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:03.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.260+0000 7fbb57e00640 1 --2- 172.21.15.5:0/3768661029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d99b815270 0x55d99b820a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:03.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.260+0000 7fbb5d600640 1 --2- 172.21.15.5:0/3768661029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99b81f940 0x55d99b826ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:03.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.260+0000 7fbb5cc00640 1 -- 172.21.15.5:0/3768661029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d99b815270 msgr2=0x55d99b820a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:03.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.260+0000 7fbb5cc00640 1 --2- 172.21.15.5:0/3768661029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d99b815270 0x55d99b820a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:03.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.260+0000 7fbb5cc00640 1 -- 172.21.15.5:0/3768661029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99b81f940 msgr2=0x55d99b826ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:03.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.260+0000 7fbb5cc00640 1 --2- 172.21.15.5:0/3768661029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99b81f940 0x55d99b826ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:03.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.260+0000 7fbb5cc00640 1 -- 172.21.15.5:0/3768661029 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d99b827110 con 0x55d99b814950 2024-04-24T06:48:03.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.261+0000 7fbb5cc00640 1 --2- 172.21.15.5:0/3768661029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99b814950 0x55d99b820530 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbb4809c8f0 tx=0x7fbb480ce820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b31113fb8085be server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:03.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.261+0000 7fbb46000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:48:03.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.261+0000 7fbb46000640 1 -- 172.21.15.5:0/3768661029 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb48098e90 con 0x55d99b814950 2024-04-24T06:48:03.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.261+0000 7fbb46000640 1 -- 172.21.15.5:0/3768661029 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fbb4809c240 con 0x55d99b814950 2024-04-24T06:48:03.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.261+0000 7fbb623dc740 1 -- 172.21.15.5:0/3768661029 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb50036050 con 0x55d99b814950 2024-04-24T06:48:03.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.261+0000 7fbb46000640 1 -- 172.21.15.5:0/3768661029 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbb48098b50 con 0x55d99b814950 2024-04-24T06:48:03.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.261+0000 7fbb623dc740 10 client.16053 fetch_fsmap learned FSMap version 25 2024-04-24T06:48:03.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.261+0000 7fbb623dc740 1 -- 172.21.15.5:0/3768661029 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d99b828410 con 0x55d99b814950 2024-04-24T06:48:03.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.262+0000 7fbb46000640 1 -- 172.21.15.5:0/3768661029 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb50036050 con 0x55d99b814950 2024-04-24T06:48:03.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.262+0000 7fbb46000640 1 -- 172.21.15.5:0/3768661029 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbb480a2080 con 0x55d99b814950 2024-04-24T06:48:03.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.262+0000 7fbb623dc740 10 client.16053 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:48:03.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.262+0000 7fbb623dc740 10 client.16053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:48:03.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.262+0000 7fbb623dc740 20 client.16053 populate_metadata read hostname 'smithi005' 2024-04-24T06:48:03.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.262+0000 7fbb623dc740 1 --2- 172.21.15.5:0/3768661029 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55d99b896520 0x55d99b898940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:03.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.262+0000 7fbb623dc740 4 client.16053 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:48:03.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.262+0000 7fbb623dc740 1 -- 172.21.15.5:0/3768661029 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d99b781a60 con 0x55d99b896520 2024-04-24T06:48:03.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.262+0000 7fbb57e00640 1 --2- 172.21.15.5:0/3768661029 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55d99b896520 0x55d99b898940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:03.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.263+0000 7fbb57e00640 1 --2- 172.21.15.5:0/3768661029 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55d99b896520 0x55d99b898940 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5da7d0a5f248a8a6 server_cookie=7890acc4623c4237 in_seq=0 out_seq=0 2024-04-24T06:48:03.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.263+0000 7fbb46000640 10 client.16053 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:48:03.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.263+0000 7fbb46000640 1 -- 172.21.15.5:0/3768661029 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d99b781a60 con 0x55d99b896520 2024-04-24T06:48:03.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.263+0000 7fbb46000640 10 client.16053 handle_command_reply: tid=0 2024-04-24T06:48:03.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.263+0000 7fbb623dc740 1 client.16053 shutdown 2024-04-24T06:48:03.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.263+0000 7fbb623dc740 1 -- 172.21.15.5:0/3768661029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99b814950 msgr2=0x55d99b820530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:03.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.263+0000 7fbb623dc740 1 --2- 172.21.15.5:0/3768661029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99b814950 0x55d99b820530 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbb4809c8f0 tx=0x7fbb480ce820 comp rx=0 tx=0).stop 2024-04-24T06:48:03.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.263+0000 7fbb623dc740 1 -- 172.21.15.5:0/3768661029 shutdown_connections 2024-04-24T06:48:03.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.263+0000 7fbb623dc740 1 --2- 172.21.15.5:0/3768661029 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55d99b896520 0x55d99b898940 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:03.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.263+0000 7fbb623dc740 1 --2- 172.21.15.5:0/3768661029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99b81f940 0x55d99b826ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:03.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.263+0000 7fbb623dc740 1 --2- 172.21.15.5:0/3768661029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d99b815270 0x55d99b820a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:03.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.263+0000 7fbb623dc740 1 --2- 172.21.15.5:0/3768661029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99b814950 0x55d99b820530 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:03.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.263+0000 7fbb46000640 0 client.16053 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:48:03.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.263+0000 7fbb623dc740 1 -- 172.21.15.5:0/3768661029 >> 172.21.15.5:0/3768661029 conn(0x55d99b810c70 msgr2=0x55d99b81b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:03.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.264+0000 7fbb623dc740 1 -- 172.21.15.5:0/3768661029 shutdown_connections 2024-04-24T06:48:03.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.264+0000 7fbb623dc740 1 -- 172.21.15.5:0/3768661029 wait complete. 2024-04-24T06:48:03.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.264+0000 7fbb623dc740 20 client.16053 trim_cache size 0 max 16384 2024-04-24T06:48:03.273 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:48:03.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.264+0000 7fbb5f600640 1 -- 172.21.15.5:0/2026333958 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fbb3c0a3d20 msgr2=0x7fbb3c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:03.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.264+0000 7fbb5f600640 1 --2- 172.21.15.5:0/2026333958 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fbb3c0a3d20 0x7fbb3c0a61e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbb50006010 tx=0x7fbb50005d80 comp rx=0 tx=0).stop 2024-04-24T06:48:03.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.264+0000 7fbb5f600640 1 -- 172.21.15.5:0/2026333958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb58104450 msgr2=0x7fbb581a0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:03.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.264+0000 7fbb5f600640 1 --2- 172.21.15.5:0/2026333958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb58104450 0x7fbb581a0710 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800e9f0 tx=0x7fbb4800eeb0 comp rx=0 tx=0).stop 2024-04-24T06:48:03.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.264+0000 7fbb5f600640 1 -- 172.21.15.5:0/2026333958 shutdown_connections 2024-04-24T06:48:03.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.264+0000 7fbb5f600640 1 --2- 172.21.15.5:0/2026333958 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fbb3c0a3d20 0x7fbb3c0a61e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:03.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.264+0000 7fbb5f600640 1 --2- 172.21.15.5:0/2026333958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb5819a8c0 0x7fbb5819ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:03.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.264+0000 7fbb5f600640 1 --2- 172.21.15.5:0/2026333958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb58109520 0x7fbb581a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:03.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.264+0000 7fbb5f600640 1 --2- 172.21.15.5:0/2026333958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb58104450 0x7fbb581a0710 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:03.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.264+0000 7fbb5f600640 1 -- 172.21.15.5:0/2026333958 >> 172.21.15.5:0/2026333958 conn(0x7fbb580ffc90 msgr2=0x7fbb58100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:03.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.265+0000 7fbb5f600640 1 -- 172.21.15.5:0/2026333958 shutdown_connections 2024-04-24T06:48:03.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:03.265+0000 7fbb5f600640 1 -- 172.21.15.5:0/2026333958 wait complete. 2024-04-24T06:48:03.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:03 smithi005 ceph-mon[27798]: pgmap v123: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:48:03.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:03 smithi137 ceph-mon[33017]: pgmap v123: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:48:04.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:03 smithi037 ceph-mon[32864]: pgmap v123: 97 pgs: 97 active+clean; 88 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:48:04.285 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T06:48:04.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:48:04.285 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T06:48:04.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.514+0000 7f345dc00640 1 -- 172.21.15.5:0/2943305559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3458104df0 msgr2=0x7f3458108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:04.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.514+0000 7f345dc00640 1 --2- 172.21.15.5:0/2943305559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3458104df0 0x7f3458108e30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3440009860 tx=0x7f344002a340 comp rx=0 tx=0).stop 2024-04-24T06:48:04.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.514+0000 7f345dc00640 1 -- 172.21.15.5:0/2943305559 shutdown_connections 2024-04-24T06:48:04.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.514+0000 7f345dc00640 1 --2- 172.21.15.5:0/2943305559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3458109510 0x7f34581110e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.514+0000 7f345dc00640 1 --2- 172.21.15.5:0/2943305559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3458104df0 0x7f3458108e30 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.514+0000 7f345dc00640 1 --2- 172.21.15.5:0/2943305559 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3458104440 0x7f3458104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.514+0000 7f345dc00640 1 -- 172.21.15.5:0/2943305559 >> 172.21.15.5:0/2943305559 conn(0x7f34580ffc80 msgr2=0x7f34581020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:04.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.514+0000 7f345dc00640 1 -- 172.21.15.5:0/2943305559 shutdown_connections 2024-04-24T06:48:04.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.514+0000 7f345dc00640 1 -- 172.21.15.5:0/2943305559 wait complete. 2024-04-24T06:48:04.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.515+0000 7f345dc00640 1 Processor -- start 2024-04-24T06:48:04.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.515+0000 7f345dc00640 1 -- start start 2024-04-24T06:48:04.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.516+0000 7f345dc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3458104440 0x7f34581a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:04.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.516+0000 7f345dc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3458109510 0x7f34581a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:04.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.516+0000 7f345dc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f345819a940 0x7f345819adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:04.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.516+0000 7f345dc00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3458114580 con 0x7f345819a940 2024-04-24T06:48:04.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.516+0000 7f345dc00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3458114400 con 0x7f3458104440 2024-04-24T06:48:04.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.516+0000 7f345dc00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3458114700 con 0x7f3458109510 2024-04-24T06:48:04.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.516+0000 7f3457400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f345819a940 0x7f345819adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:04.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.516+0000 7f3457400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f345819a940 0x7f345819adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47390/0 (socket says 172.21.15.5:47390) 2024-04-24T06:48:04.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.516+0000 7f3457400640 1 -- 172.21.15.5:0/1391897689 learned_addr learned my addr 172.21.15.5:0/1391897689 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:04.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.516+0000 7f3456a00640 1 --2- 172.21.15.5:0/1391897689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3458104440 0x7f34581a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:04.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.517+0000 7f344fe00640 1 --2- 172.21.15.5:0/1391897689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3458109510 0x7f34581a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:04.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.517+0000 7f3456a00640 1 -- 172.21.15.5:0/1391897689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3458109510 msgr2=0x7f34581a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:04.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.517+0000 7f3456a00640 1 --2- 172.21.15.5:0/1391897689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3458109510 0x7f34581a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.517+0000 7f3456a00640 1 -- 172.21.15.5:0/1391897689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f345819a940 msgr2=0x7f345819adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:04.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.517+0000 7f3456a00640 1 --2- 172.21.15.5:0/1391897689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f345819a940 0x7f345819adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.517+0000 7f3456a00640 1 -- 172.21.15.5:0/1391897689 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f345819b460 con 0x7f3458104440 2024-04-24T06:48:04.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.517+0000 7f3457400640 1 --2- 172.21.15.5:0/1391897689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f345819a940 0x7f345819adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:48:04.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.517+0000 7f3456a00640 1 --2- 172.21.15.5:0/1391897689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3458104440 0x7f34581a0740 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f344800e9f0 tx=0x7f344800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:04.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.517+0000 7f344f400640 1 -- 172.21.15.5:0/1391897689 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344800ccc0 con 0x7f3458104440 2024-04-24T06:48:04.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.518+0000 7f344f400640 1 -- 172.21.15.5:0/1391897689 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3448010910 con 0x7f3458104440 2024-04-24T06:48:04.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.518+0000 7f345dc00640 1 -- 172.21.15.5:0/1391897689 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f345819b750 con 0x7f3458104440 2024-04-24T06:48:04.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.518+0000 7f344f400640 1 -- 172.21.15.5:0/1391897689 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3448018960 con 0x7f3458104440 2024-04-24T06:48:04.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.518+0000 7f345dc00640 1 -- 172.21.15.5:0/1391897689 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34581a79a0 con 0x7f3458104440 2024-04-24T06:48:04.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.521+0000 7f3460a38740 1 Processor -- start 2024-04-24T06:48:04.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.521+0000 7f344f400640 1 -- 172.21.15.5:0/1391897689 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f3448010ac0 con 0x7f3458104440 2024-04-24T06:48:04.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.521+0000 7f3460a38740 1 -- start start 2024-04-24T06:48:04.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.521+0000 7f3460a38740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7eba1090 0x559e7eba1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:04.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.522+0000 7f3460a38740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e7eba19b0 0x559e7ebabb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:04.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.522+0000 7f3460a38740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e7ebac080 0x559e7ebae470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:04.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.522+0000 7f3460a38740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559e7eb06e70 con 0x559e7eba1090 2024-04-24T06:48:04.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.522+0000 7f3460a38740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559e7eaad380 con 0x559e7eba19b0 2024-04-24T06:48:04.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.522+0000 7f3460a38740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559e7eb02e30 con 0x559e7ebac080 2024-04-24T06:48:04.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.522+0000 7f3456a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e7eba19b0 0x559e7ebabb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:04.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.522+0000 7f3457400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7eba1090 0x559e7eba1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:04.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.522+0000 7f3457400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7eba1090 0x559e7eba1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47400/0 (socket says 172.21.15.5:47400) 2024-04-24T06:48:04.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.522+0000 7f3457400640 1 -- 172.21.15.5:0/2077943680 learned_addr learned my addr 172.21.15.5:0/2077943680 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:04.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.522+0000 7f3456a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e7eba19b0 0x559e7ebabb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:52906/0 (socket says 172.21.15.5:52906) 2024-04-24T06:48:04.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.522+0000 7f344fe00640 1 --2- 172.21.15.5:0/2077943680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e7ebac080 0x559e7ebae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:04.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.522+0000 7f3457400640 1 -- 172.21.15.5:0/2077943680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e7ebac080 msgr2=0x559e7ebae470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:04.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.522+0000 7f3457400640 1 --2- 172.21.15.5:0/2077943680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e7ebac080 0x559e7ebae470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.522+0000 7f3457400640 1 -- 172.21.15.5:0/2077943680 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e7eba19b0 msgr2=0x559e7ebabb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:04.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.522+0000 7f3457400640 1 --2- 172.21.15.5:0/2077943680 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e7eba19b0 0x559e7ebabb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.522+0000 7f3457400640 1 -- 172.21.15.5:0/2077943680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e7e8e17e0 con 0x559e7eba1090 2024-04-24T06:48:04.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.523+0000 7f3457400640 1 --2- 172.21.15.5:0/2077943680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7eba1090 0x559e7eba1470 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f345000e930 tx=0x7f345000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc9df4dcd9f9b424 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:04.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.523+0000 7f3447e00640 1 -- 172.21.15.5:0/2077943680 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345000cda0 con 0x559e7eba1090 2024-04-24T06:48:04.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.523+0000 7f3447e00640 1 -- 172.21.15.5:0/2077943680 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f345001d070 con 0x559e7eba1090 2024-04-24T06:48:04.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.523+0000 7f3447e00640 1 -- 172.21.15.5:0/2077943680 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345000b5c0 con 0x559e7eba1090 2024-04-24T06:48:04.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.523+0000 7f344f400640 1 --2- 172.21.15.5:0/1391897689 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f343c0a3da0 0x7f343c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:04.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.523+0000 7f344f400640 1 -- 172.21.15.5:0/1391897689 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f34480d0b60 con 0x7f3458104440 2024-04-24T06:48:04.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.524+0000 7f3456a00640 1 --2- 172.21.15.5:0/1391897689 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f343c0a3da0 0x7f343c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:04.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.524+0000 7f3460a38740 1 -- 172.21.15.5:0/2077943680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7eba1090 msgr2=0x559e7eba1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:04.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.524+0000 7f3460a38740 1 --2- 172.21.15.5:0/2077943680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7eba1090 0x559e7eba1470 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f345000e930 tx=0x7f345000edf0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.524+0000 7f3460a38740 1 -- 172.21.15.5:0/2077943680 shutdown_connections 2024-04-24T06:48:04.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.524+0000 7f3460a38740 1 --2- 172.21.15.5:0/2077943680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e7ebac080 0x559e7ebae470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.524+0000 7f3460a38740 1 --2- 172.21.15.5:0/2077943680 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e7eba19b0 0x559e7ebabb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.524+0000 7f3460a38740 1 --2- 172.21.15.5:0/2077943680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7eba1090 0x559e7eba1470 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.524+0000 7f3460a38740 1 -- 172.21.15.5:0/2077943680 >> 172.21.15.5:0/2077943680 conn(0x559e7eb9d3b0 msgr2=0x559e7eb9d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:04.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.524+0000 7f3460a38740 1 -- 172.21.15.5:0/2077943680 shutdown_connections 2024-04-24T06:48:04.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.524+0000 7f3460a38740 1 -- 172.21.15.5:0/2077943680 wait complete. 2024-04-24T06:48:04.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.524+0000 7f3456a00640 1 --2- 172.21.15.5:0/1391897689 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f343c0a3da0 0x7f343c0a6260 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x559e7eba2650 tx=0x7f34480d4610 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:04.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.525+0000 7f3460a38740 1 -- start start 2024-04-24T06:48:04.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.525+0000 7f3460a38740 1 Processor -- start 2024-04-24T06:48:04.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.525+0000 7f3460a38740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7eba1090 0x559e7eba9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:04.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.526+0000 7f3460a38740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e7eba19b0 0x559e7eba9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:04.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.526+0000 7f3460a38740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e7ebac080 0x559e7eba9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:04.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.526+0000 7f3460a38740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559e7ebaee70 con 0x559e7eba1090 2024-04-24T06:48:04.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.526+0000 7f3460a38740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559e7eac0660 con 0x559e7eba19b0 2024-04-24T06:48:04.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.526+0000 7f3460a38740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559e7ebaeff0 con 0x559e7ebac080 2024-04-24T06:48:04.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.526+0000 7f3460a38740 10 client.0 _ll_register_callbacks cb 0x559e7eaaeaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:48:04.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.526+0000 7f3457400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7eba1090 0x559e7eba9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:04.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.526+0000 7f344fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e7eba19b0 0x559e7eba9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:04.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.526+0000 7f344fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e7eba19b0 0x559e7eba9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:52910/0 (socket says 172.21.15.5:52910) 2024-04-24T06:48:04.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.526+0000 7f3457400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7eba1090 0x559e7eba9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47410/0 (socket says 172.21.15.5:47410) 2024-04-24T06:48:04.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.526+0000 7f344fe00640 1 -- 172.21.15.5:0/587293403 learned_addr learned my addr 172.21.15.5:0/587293403 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:04.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.526+0000 7f3457400640 1 --2- 172.21.15.5:0/587293403 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e7ebac080 0x559e7eba9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:04.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.526+0000 7f344fe00640 1 -- 172.21.15.5:0/587293403 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e7ebac080 msgr2=0x559e7eba9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:04.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.526+0000 7f344fe00640 1 --2- 172.21.15.5:0/587293403 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e7ebac080 0x559e7eba9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.526+0000 7f344fe00640 1 -- 172.21.15.5:0/587293403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7eba1090 msgr2=0x559e7eba9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:04.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.526+0000 7f344fe00640 1 --2- 172.21.15.5:0/587293403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7eba1090 0x559e7eba9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.526+0000 7f344fe00640 1 -- 172.21.15.5:0/587293403 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e7e941f70 con 0x559e7eba19b0 2024-04-24T06:48:04.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.526+0000 7f3457400640 1 --2- 172.21.15.5:0/587293403 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e7ebac080 0x559e7eba9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:48:04.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.527+0000 7f344fe00640 1 --2- 172.21.15.5:0/587293403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e7eba19b0 0x559e7eba9570 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f344002a930 tx=0x7f3440030340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82631d4668a39985 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:04.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.527+0000 7f3444c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:48:04.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.527+0000 7f3444c00640 1 -- 172.21.15.5:0/587293403 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3440037040 con 0x559e7eba19b0 2024-04-24T06:48:04.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.527+0000 7f3444c00640 1 -- 172.21.15.5:0/587293403 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f3440030550 con 0x559e7eba19b0 2024-04-24T06:48:04.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.527+0000 7f3460a38740 1 -- 172.21.15.5:0/587293403 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f345001d070 con 0x559e7eba19b0 2024-04-24T06:48:04.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.527+0000 7f3444c00640 1 -- 172.21.15.5:0/587293403 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f344004c050 con 0x559e7eba19b0 2024-04-24T06:48:04.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.527+0000 7f3460a38740 10 client.26198 fetch_fsmap learned FSMap version 25 2024-04-24T06:48:04.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.527+0000 7f3460a38740 1 -- 172.21.15.5:0/587293403 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559e7e7d9170 con 0x559e7eba19b0 2024-04-24T06:48:04.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.528+0000 7f3444c00640 1 -- 172.21.15.5:0/587293403 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f345001d070 con 0x559e7eba19b0 2024-04-24T06:48:04.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.528+0000 7f3444c00640 1 -- 172.21.15.5:0/587293403 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f344003d450 con 0x559e7eba19b0 2024-04-24T06:48:04.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.528+0000 7f3460a38740 10 client.26198 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:48:04.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.528+0000 7f3460a38740 10 client.26198 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:48:04.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.528+0000 7f3460a38740 20 client.26198 populate_metadata read hostname 'smithi005' 2024-04-24T06:48:04.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.528+0000 7f3460a38740 1 --2- 172.21.15.5:0/587293403 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x559e7ebbc470 0x559e7ebbe8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:04.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.528+0000 7f3460a38740 4 client.26198 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:48:04.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.528+0000 7f3460a38740 1 -- 172.21.15.5:0/587293403 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e7eb14890 con 0x559e7ebbc470 2024-04-24T06:48:04.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.529+0000 7f3457400640 1 --2- 172.21.15.5:0/587293403 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x559e7ebbc470 0x559e7ebbe8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:04.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.529+0000 7f3457400640 1 --2- 172.21.15.5:0/587293403 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x559e7ebbc470 0x559e7ebbe8d0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=188f46b46a992885 server_cookie=134cae12d13e5ff1 in_seq=0 out_seq=0 2024-04-24T06:48:04.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.529+0000 7f3444c00640 10 client.26198 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:48:04.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.530+0000 7f3444c00640 1 -- 172.21.15.5:0/587293403 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559e7eb14890 con 0x559e7ebbc470 2024-04-24T06:48:04.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.530+0000 7f3444c00640 10 client.26198 handle_command_reply: tid=0 2024-04-24T06:48:04.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.530+0000 7f3460a38740 1 client.26198 shutdown 2024-04-24T06:48:04.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.530+0000 7f3460a38740 1 -- 172.21.15.5:0/587293403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e7eba19b0 msgr2=0x559e7eba9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:04.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.530+0000 7f3460a38740 1 --2- 172.21.15.5:0/587293403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e7eba19b0 0x559e7eba9570 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f344002a930 tx=0x7f3440030340 comp rx=0 tx=0).stop 2024-04-24T06:48:04.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.530+0000 7f3460a38740 1 -- 172.21.15.5:0/587293403 shutdown_connections 2024-04-24T06:48:04.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.530+0000 7f3460a38740 1 --2- 172.21.15.5:0/587293403 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x559e7ebbc470 0x559e7ebbe8d0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.530+0000 7f3460a38740 1 --2- 172.21.15.5:0/587293403 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e7ebac080 0x559e7eba9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.530+0000 7f3460a38740 1 --2- 172.21.15.5:0/587293403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e7eba19b0 0x559e7eba9570 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.530+0000 7f3460a38740 1 --2- 172.21.15.5:0/587293403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7eba1090 0x559e7eba9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.530+0000 7f3460a38740 1 -- 172.21.15.5:0/587293403 >> 172.21.15.5:0/587293403 conn(0x559e7eb9d3b0 msgr2=0x559e7eba31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:04.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.530+0000 7f3444c00640 0 client.26198 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:48:04.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.531+0000 7f3460a38740 1 -- 172.21.15.5:0/587293403 shutdown_connections 2024-04-24T06:48:04.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.531+0000 7f3460a38740 1 -- 172.21.15.5:0/587293403 wait complete. 2024-04-24T06:48:04.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.531+0000 7f3460a38740 20 client.26198 trim_cache size 0 max 16384 2024-04-24T06:48:04.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.551+0000 7f3460a38740 1 Processor -- start 2024-04-24T06:48:04.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.551+0000 7f3460a38740 1 -- start start 2024-04-24T06:48:04.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.551+0000 7f3460a38740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7eba1090 0x559e7ebb7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:04.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.551+0000 7f3460a38740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e7eba19b0 0x559e7ebaaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:04.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.551+0000 7f3460a38740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e7ebac080 0x559e7ebc41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:04.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.551+0000 7f3460a38740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f34581b2840 con 0x559e7eba1090 2024-04-24T06:48:04.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.551+0000 7f3460a38740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f34581b0110 con 0x559e7ebac080 2024-04-24T06:48:04.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.551+0000 7f3460a38740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f34581adf60 con 0x559e7eba19b0 2024-04-24T06:48:04.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.552+0000 7f3457400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7eba1090 0x559e7ebb7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:04.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.552+0000 7f344fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e7eba19b0 0x559e7ebaaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:04.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.552+0000 7f3457400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7eba1090 0x559e7ebb7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47416/0 (socket says 172.21.15.5:47416) 2024-04-24T06:48:04.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.552+0000 7f344fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e7eba19b0 0x559e7ebaaf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:52080/0 (socket says 172.21.15.5:52080) 2024-04-24T06:48:04.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.552+0000 7f3457400640 1 -- 172.21.15.5:0/435650538 learned_addr learned my addr 172.21.15.5:0/435650538 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:04.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.552+0000 7f3457400640 1 --2- 172.21.15.5:0/435650538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e7ebac080 0x559e7ebc41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:04.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.552+0000 7f344fe00640 1 -- 172.21.15.5:0/435650538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e7ebac080 msgr2=0x559e7ebc41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:04.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.552+0000 7f344fe00640 1 --2- 172.21.15.5:0/435650538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e7ebac080 0x559e7ebc41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.552+0000 7f344fe00640 1 -- 172.21.15.5:0/435650538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7eba1090 msgr2=0x559e7ebb7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:04.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.552+0000 7f344fe00640 1 --2- 172.21.15.5:0/435650538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7eba1090 0x559e7ebb7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.552+0000 7f344fe00640 1 -- 172.21.15.5:0/435650538 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e7eba7580 con 0x559e7eba19b0 2024-04-24T06:48:04.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.552+0000 7f3457400640 1 --2- 172.21.15.5:0/435650538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e7ebac080 0x559e7ebc41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:48:04.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.552+0000 7f344fe00640 1 --2- 172.21.15.5:0/435650538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e7eba19b0 0x559e7ebaaf70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f344003d8f0 tx=0x7f3440005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56a20b9876c7c30b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:04.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.553+0000 7f3447400640 1 -- 172.21.15.5:0/435650538 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3440055070 con 0x559e7eba19b0 2024-04-24T06:48:04.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.553+0000 7f3447400640 1 -- 172.21.15.5:0/435650538 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3440009450 con 0x559e7eba19b0 2024-04-24T06:48:04.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.553+0000 7f3447400640 1 -- 172.21.15.5:0/435650538 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3440037070 con 0x559e7eba19b0 2024-04-24T06:48:04.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.553+0000 7f3460a38740 1 -- 172.21.15.5:0/435650538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e7eba19b0 msgr2=0x559e7ebaaf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:04.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.553+0000 7f3460a38740 1 --2- 172.21.15.5:0/435650538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e7eba19b0 0x559e7ebaaf70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f344003d8f0 tx=0x7f3440005c90 comp rx=0 tx=0).stop 2024-04-24T06:48:04.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.554+0000 7f3460a38740 1 -- 172.21.15.5:0/435650538 shutdown_connections 2024-04-24T06:48:04.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.554+0000 7f3460a38740 1 --2- 172.21.15.5:0/435650538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e7ebac080 0x559e7ebc41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.554+0000 7f3460a38740 1 --2- 172.21.15.5:0/435650538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e7eba19b0 0x559e7ebaaf70 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.554+0000 7f3460a38740 1 --2- 172.21.15.5:0/435650538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7eba1090 0x559e7ebb7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.554+0000 7f3460a38740 1 -- 172.21.15.5:0/435650538 >> 172.21.15.5:0/435650538 conn(0x559e7eb9d3b0 msgr2=0x559e7ebaa860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:04.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.554+0000 7f3460a38740 1 -- 172.21.15.5:0/435650538 shutdown_connections 2024-04-24T06:48:04.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.554+0000 7f3460a38740 1 -- 172.21.15.5:0/435650538 wait complete. 2024-04-24T06:48:04.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.554+0000 7f3460a38740 1 -- start start 2024-04-24T06:48:04.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.554+0000 7f3460a38740 1 Processor -- start 2024-04-24T06:48:04.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.555+0000 7f3460a38740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e7eba1090 0x559e7ebb4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:04.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.555+0000 7f3460a38740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e7eba19b0 0x559e7ebb4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:04.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.555+0000 7f3460a38740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7ebac080 0x559e7ebb1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:04.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.555+0000 7f3460a38740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f34581ae5d0 con 0x559e7ebac080 2024-04-24T06:48:04.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.555+0000 7f3460a38740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f34581afa40 con 0x559e7eba1090 2024-04-24T06:48:04.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.555+0000 7f3460a38740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f34581aeca0 con 0x559e7eba19b0 2024-04-24T06:48:04.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.555+0000 7f3460a38740 10 client.0 _ll_register_callbacks cb 0x7f3448005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:48:04.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.555+0000 7f3457400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e7eba1090 0x559e7ebb4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:04.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.555+0000 7f344fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e7eba19b0 0x559e7ebb4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:04.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.555+0000 7f3457400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e7eba1090 0x559e7ebb4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:52920/0 (socket says 172.21.15.5:52920) 2024-04-24T06:48:04.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.555+0000 7f3457400640 1 -- 172.21.15.5:0/755705847 learned_addr learned my addr 172.21.15.5:0/755705847 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:04.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.555+0000 7f3457400640 1 --2- 172.21.15.5:0/755705847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7ebac080 0x559e7ebb1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:04.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.556+0000 7f344fe00640 1 -- 172.21.15.5:0/755705847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e7eba1090 msgr2=0x559e7ebb4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:04.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.556+0000 7f344fe00640 1 --2- 172.21.15.5:0/755705847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e7eba1090 0x559e7ebb4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.556+0000 7f344fe00640 1 -- 172.21.15.5:0/755705847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7ebac080 msgr2=0x559e7ebb1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:04.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.556+0000 7f344fe00640 1 --2- 172.21.15.5:0/755705847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7ebac080 0x559e7ebb1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.556+0000 7f344fe00640 1 -- 172.21.15.5:0/755705847 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e7ebb18d0 con 0x559e7eba19b0 2024-04-24T06:48:04.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.556+0000 7f344fe00640 1 --2- 172.21.15.5:0/755705847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e7eba19b0 0x559e7ebb4780 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f344003d870 tx=0x7f344003dae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b11b7e8843bd22f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:04.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.556+0000 7f3444c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:48:04.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.556+0000 7f3444c00640 1 -- 172.21.15.5:0/755705847 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3440037070 con 0x559e7eba19b0 2024-04-24T06:48:04.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.556+0000 7f3444c00640 1 -- 172.21.15.5:0/755705847 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f3440030750 con 0x559e7eba19b0 2024-04-24T06:48:04.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.556+0000 7f3460a38740 1 -- 172.21.15.5:0/755705847 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3440009450 con 0x559e7eba19b0 2024-04-24T06:48:04.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.557+0000 7f3444c00640 1 -- 172.21.15.5:0/755705847 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3440055070 con 0x559e7eba19b0 2024-04-24T06:48:04.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.557+0000 7f3460a38740 10 client.26218 fetch_fsmap learned FSMap version 25 2024-04-24T06:48:04.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.557+0000 7f3460a38740 1 -- 172.21.15.5:0/755705847 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559e7ec23460 con 0x559e7eba19b0 2024-04-24T06:48:04.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.557+0000 7f3444c00640 1 -- 172.21.15.5:0/755705847 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3440009450 con 0x559e7eba19b0 2024-04-24T06:48:04.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.557+0000 7f3444c00640 1 -- 172.21.15.5:0/755705847 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3440032440 con 0x559e7eba19b0 2024-04-24T06:48:04.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.557+0000 7f3460a38740 10 client.26218 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:48:04.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.557+0000 7f3460a38740 10 client.26218 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:48:04.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.557+0000 7f3460a38740 20 client.26218 populate_metadata read hostname 'smithi005' 2024-04-24T06:48:04.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.557+0000 7f3460a38740 1 --2- 172.21.15.5:0/755705847 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x559e7ec23af0 0x559e7ec25f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:04.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.557+0000 7f3460a38740 4 client.26218 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T06:48:04.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.557+0000 7f3460a38740 1 -- 172.21.15.5:0/755705847 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559e7eb0e700 con 0x559e7ec23af0 2024-04-24T06:48:04.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.558+0000 7f3457400640 1 --2- 172.21.15.5:0/755705847 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x559e7ec23af0 0x559e7ec25f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:04.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.558+0000 7f3457400640 1 --2- 172.21.15.5:0/755705847 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x559e7ec23af0 0x559e7ec25f10 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2929ce724af5aba server_cookie=2f7a5cc556d89597 in_seq=0 out_seq=0 2024-04-24T06:48:04.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.558+0000 7f3444c00640 10 client.26218 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:48:04.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.560+0000 7f3444c00640 1 -- 172.21.15.5:0/755705847 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559e7eb0e700 con 0x559e7ec23af0 2024-04-24T06:48:04.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.560+0000 7f3444c00640 10 client.26218 handle_command_reply: tid=0 2024-04-24T06:48:04.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.560+0000 7f3460a38740 1 client.26218 shutdown 2024-04-24T06:48:04.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.560+0000 7f3460a38740 1 -- 172.21.15.5:0/755705847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e7eba19b0 msgr2=0x559e7ebb4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:04.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.560+0000 7f3460a38740 1 --2- 172.21.15.5:0/755705847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e7eba19b0 0x559e7ebb4780 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f344003d870 tx=0x7f344003dae0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.560+0000 7f3460a38740 1 -- 172.21.15.5:0/755705847 shutdown_connections 2024-04-24T06:48:04.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.560+0000 7f3460a38740 1 --2- 172.21.15.5:0/755705847 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x559e7ec23af0 0x559e7ec25f10 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.560+0000 7f3460a38740 1 --2- 172.21.15.5:0/755705847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7ebac080 0x559e7ebb1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.560+0000 7f3460a38740 1 --2- 172.21.15.5:0/755705847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e7eba19b0 0x559e7ebb4780 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.560+0000 7f3444c00640 0 client.26218 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:48:04.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.560+0000 7f3460a38740 1 --2- 172.21.15.5:0/755705847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e7eba1090 0x559e7ebb4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.560+0000 7f3460a38740 1 -- 172.21.15.5:0/755705847 >> 172.21.15.5:0/755705847 conn(0x559e7eb9d3b0 msgr2=0x559e7ebab0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:04.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.560+0000 7f3460a38740 1 -- 172.21.15.5:0/755705847 shutdown_connections 2024-04-24T06:48:04.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.560+0000 7f3460a38740 1 -- 172.21.15.5:0/755705847 wait complete. 2024-04-24T06:48:04.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.560+0000 7f3460a38740 20 client.26218 trim_cache size 0 max 16384 2024-04-24T06:48:04.570 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:48:04.570 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-24T06:48:04.571 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "2fe0ad20-0ee9-40d8-8788-c73c6b2a576d", 2024-04-24T06:48:04.571 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-24T06:48:04.571 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:48:04.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.561+0000 7f345dc00640 1 -- 172.21.15.5:0/1391897689 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f343c0a3da0 msgr2=0x7f343c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:04.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.561+0000 7f345dc00640 1 --2- 172.21.15.5:0/1391897689 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f343c0a3da0 0x7f343c0a6260 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x559e7eba2650 tx=0x7f34480d4610 comp rx=0 tx=0).stop 2024-04-24T06:48:04.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.561+0000 7f345dc00640 1 -- 172.21.15.5:0/1391897689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3458104440 msgr2=0x7f34581a0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:04.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.561+0000 7f345dc00640 1 --2- 172.21.15.5:0/1391897689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3458104440 0x7f34581a0740 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f344800e9f0 tx=0x7f344800eeb0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.561+0000 7f345dc00640 1 -- 172.21.15.5:0/1391897689 shutdown_connections 2024-04-24T06:48:04.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.561+0000 7f345dc00640 1 --2- 172.21.15.5:0/1391897689 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f343c0a3da0 0x7f343c0a6260 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.561+0000 7f345dc00640 1 --2- 172.21.15.5:0/1391897689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f345819a940 0x7f345819adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.561+0000 7f345dc00640 1 --2- 172.21.15.5:0/1391897689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3458109510 0x7f34581a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.561+0000 7f345dc00640 1 --2- 172.21.15.5:0/1391897689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3458104440 0x7f34581a0740 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.561+0000 7f345dc00640 1 -- 172.21.15.5:0/1391897689 >> 172.21.15.5:0/1391897689 conn(0x7f34580ffc80 msgr2=0x7f3458101060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:04.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.561+0000 7f345dc00640 1 -- 172.21.15.5:0/1391897689 shutdown_connections 2024-04-24T06:48:04.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.561+0000 7f345dc00640 1 -- 172.21.15.5:0/1391897689 wait complete. 2024-04-24T06:48:04.584 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T06:48:04.584 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T06:48:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.807+0000 7f12a3200640 1 -- 172.21.15.5:0/2935765847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12a4104260 msgr2=0x7f12a4104640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.807+0000 7f12a3200640 1 --2- 172.21.15.5:0/2935765847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12a4104260 0x7f12a4104640 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f128c009900 tx=0x7f128c02a3f0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.807+0000 7f12a3200640 1 -- 172.21.15.5:0/2935765847 shutdown_connections 2024-04-24T06:48:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.807+0000 7f12a3200640 1 --2- 172.21.15.5:0/2935765847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12a4109330 0x7f12a4110f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.807+0000 7f12a3200640 1 --2- 172.21.15.5:0/2935765847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12a4104c10 0x7f12a4108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.807+0000 7f12a3200640 1 --2- 172.21.15.5:0/2935765847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12a4104260 0x7f12a4104640 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.807+0000 7f12a3200640 1 -- 172.21.15.5:0/2935765847 >> 172.21.15.5:0/2935765847 conn(0x7f12a40ffaa0 msgr2=0x7f12a4101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.807+0000 7f12a3200640 1 -- 172.21.15.5:0/2935765847 shutdown_connections 2024-04-24T06:48:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.807+0000 7f12a3200640 1 -- 172.21.15.5:0/2935765847 wait complete. 2024-04-24T06:48:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.808+0000 7f12a3200640 1 Processor -- start 2024-04-24T06:48:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.808+0000 7f12a3200640 1 -- start start 2024-04-24T06:48:04.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.809+0000 7f12a3200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12a4104260 0x7f12a41a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:04.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.809+0000 7f12a3200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12a4109330 0x7f12a41a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:04.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.809+0000 7f12a3200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12a419a810 0x7f12a419acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:04.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.809+0000 7f12a3200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f12a4114410 con 0x7f12a419a810 2024-04-24T06:48:04.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.809+0000 7f12a3200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f12a4114290 con 0x7f12a4109330 2024-04-24T06:48:04.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.809+0000 7f12a3200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f12a4114590 con 0x7f12a4104260 2024-04-24T06:48:04.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.809+0000 7f12a1e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12a4104260 0x7f12a41a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:04.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.809+0000 7f12a1e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12a4104260 0x7f12a41a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:52108/0 (socket says 172.21.15.5:52108) 2024-04-24T06:48:04.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.809+0000 7f12a1400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12a4109330 0x7f12a41a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:04.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.809+0000 7f12a2800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12a419a810 0x7f12a419acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:04.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.809+0000 7f12a1e00640 1 -- 172.21.15.5:0/3627504332 learned_addr learned my addr 172.21.15.5:0/3627504332 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:04.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.810+0000 7f12a1e00640 1 -- 172.21.15.5:0/3627504332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12a4109330 msgr2=0x7f12a41a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:04.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.810+0000 7f12a1e00640 1 --2- 172.21.15.5:0/3627504332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12a4109330 0x7f12a41a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.810+0000 7f12a1e00640 1 -- 172.21.15.5:0/3627504332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12a419a810 msgr2=0x7f12a419acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:04.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.810+0000 7f12a1e00640 1 --2- 172.21.15.5:0/3627504332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12a419a810 0x7f12a419acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.810+0000 7f12a1e00640 1 -- 172.21.15.5:0/3627504332 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12a419b3a0 con 0x7f12a4104260 2024-04-24T06:48:04.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.810+0000 7f12a2800640 1 --2- 172.21.15.5:0/3627504332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12a419a810 0x7f12a419acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:48:04.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.810+0000 7f12a1e00640 1 --2- 172.21.15.5:0/3627504332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12a4104260 0x7f12a41a0650 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f128c02a960 tx=0x7f128c003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:04.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.810+0000 7f129aa00640 1 -- 172.21.15.5:0/3627504332 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128c03f070 con 0x7f12a4104260 2024-04-24T06:48:04.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.810+0000 7f129aa00640 1 -- 172.21.15.5:0/3627504332 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f128c031c90 con 0x7f12a4104260 2024-04-24T06:48:04.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.810+0000 7f129aa00640 1 -- 172.21.15.5:0/3627504332 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128c035030 con 0x7f12a4104260 2024-04-24T06:48:04.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.810+0000 7f12a3200640 1 -- 172.21.15.5:0/3627504332 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12a419b630 con 0x7f12a4104260 2024-04-24T06:48:04.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.811+0000 7f12a3200640 1 -- 172.21.15.5:0/3627504332 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12a41a7740 con 0x7f12a4104260 2024-04-24T06:48:04.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.814+0000 7f129aa00640 1 -- 172.21.15.5:0/3627504332 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f128c042070 con 0x7f12a4104260 2024-04-24T06:48:04.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.814+0000 7f12abc5f740 1 Processor -- start 2024-04-24T06:48:04.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.814+0000 7f12abc5f740 1 -- start start 2024-04-24T06:48:04.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.815+0000 7f12abc5f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8117cda30 0x55f8117cde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:04.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.815+0000 7f12abc5f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8117ce350 0x55f8117d84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:04.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.815+0000 7f12abc5f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8117d8a20 0x55f8117dae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:04.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.815+0000 7f12abc5f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f8117331f0 con 0x55f8117cda30 2024-04-24T06:48:04.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.815+0000 7f12abc5f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f8116d9350 con 0x55f8117ce350 2024-04-24T06:48:04.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.815+0000 7f12abc5f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f81172f1b0 con 0x55f8117d8a20 2024-04-24T06:48:04.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.815+0000 7f12a1e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8117ce350 0x55f8117d84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:04.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.815+0000 7f12a1e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8117ce350 0x55f8117d84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:52948/0 (socket says 172.21.15.5:52948) 2024-04-24T06:48:04.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.815+0000 7f12a1400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8117d8a20 0x55f8117dae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:04.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.815+0000 7f12a1e00640 1 -- 172.21.15.5:0/249540945 learned_addr learned my addr 172.21.15.5:0/249540945 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:04.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.815+0000 7f12a2800640 1 --2- 172.21.15.5:0/249540945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8117cda30 0x55f8117cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:04.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.816+0000 7f12a1400640 1 -- 172.21.15.5:0/249540945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8117ce350 msgr2=0x55f8117d84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:04.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.816+0000 7f12a1400640 1 --2- 172.21.15.5:0/249540945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8117ce350 0x55f8117d84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.816+0000 7f12a1400640 1 -- 172.21.15.5:0/249540945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8117cda30 msgr2=0x55f8117cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:04.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.816+0000 7f12a1400640 1 --2- 172.21.15.5:0/249540945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8117cda30 0x55f8117cde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.816+0000 7f12a1400640 1 -- 172.21.15.5:0/249540945 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f81172df50 con 0x55f8117d8a20 2024-04-24T06:48:04.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.816+0000 7f12a1400640 1 --2- 172.21.15.5:0/249540945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8117d8a20 0x55f8117dae10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f12a419bb10 tx=0x7f12940107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d72308051d3e4d9c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:04.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.816+0000 7f129aa00640 1 --2- 172.21.15.5:0/3627504332 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f12880a3de0 0x7f12880a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:04.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.816+0000 7f1293400640 1 -- 172.21.15.5:0/249540945 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1294004030 con 0x55f8117d8a20 2024-04-24T06:48:04.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.817+0000 7f129aa00640 1 -- 172.21.15.5:0/3627504332 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f128c0f0f40 con 0x7f12a4104260 2024-04-24T06:48:04.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.817+0000 7f1293400640 1 -- 172.21.15.5:0/249540945 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12940041d0 con 0x55f8117d8a20 2024-04-24T06:48:04.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.817+0000 7f1293400640 1 -- 172.21.15.5:0/249540945 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12940116a0 con 0x55f8117d8a20 2024-04-24T06:48:04.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.817+0000 7f12a2800640 1 --2- 172.21.15.5:0/3627504332 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f12880a3de0 0x7f12880a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:04.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.818+0000 7f12abc5f740 1 -- 172.21.15.5:0/249540945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8117d8a20 msgr2=0x55f8117dae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:04.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.818+0000 7f12a2800640 1 --2- 172.21.15.5:0/3627504332 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f12880a3de0 0x7f12880a62a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f129c005fd0 tx=0x7f129c009500 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:04.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.818+0000 7f12abc5f740 1 --2- 172.21.15.5:0/249540945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8117d8a20 0x55f8117dae10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f12a419bb10 tx=0x7f12940107a0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.818+0000 7f12abc5f740 1 -- 172.21.15.5:0/249540945 shutdown_connections 2024-04-24T06:48:04.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.818+0000 7f12abc5f740 1 --2- 172.21.15.5:0/249540945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8117d8a20 0x55f8117dae10 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.818+0000 7f12abc5f740 1 --2- 172.21.15.5:0/249540945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8117ce350 0x55f8117d84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.818+0000 7f12abc5f740 1 --2- 172.21.15.5:0/249540945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8117cda30 0x55f8117cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.818+0000 7f12abc5f740 1 -- 172.21.15.5:0/249540945 >> 172.21.15.5:0/249540945 conn(0x55f8117c9d50 msgr2=0x55f8117ca130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:04.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.818+0000 7f12abc5f740 1 -- 172.21.15.5:0/249540945 shutdown_connections 2024-04-24T06:48:04.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.818+0000 7f12abc5f740 1 -- 172.21.15.5:0/249540945 wait complete. 2024-04-24T06:48:04.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.819+0000 7f12abc5f740 1 -- start start 2024-04-24T06:48:04.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.819+0000 7f12abc5f740 1 Processor -- start 2024-04-24T06:48:04.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.820+0000 7f12abc5f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8117cda30 0x55f8117d59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:04.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.820+0000 7f12abc5f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8117ce350 0x55f8117d5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:04.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.820+0000 7f12abc5f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8117d8a20 0x55f8117d6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:04.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.820+0000 7f12abc5f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f8117db810 con 0x55f8117d8a20 2024-04-24T06:48:04.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.820+0000 7f12abc5f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f8116ec690 con 0x55f8117cda30 2024-04-24T06:48:04.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.820+0000 7f12abc5f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f8117db990 con 0x55f8117ce350 2024-04-24T06:48:04.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.820+0000 7f12abc5f740 10 client.0 _ll_register_callbacks cb 0x55f811728650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:48:04.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.820+0000 7f12a1e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8117cda30 0x55f8117d59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:04.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.820+0000 7f12a1400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8117ce350 0x55f8117d5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:04.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.820+0000 7f12a1e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8117cda30 0x55f8117d59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:52964/0 (socket says 172.21.15.5:52964) 2024-04-24T06:48:04.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.820+0000 7f12a1e00640 1 -- 172.21.15.5:0/745456366 learned_addr learned my addr 172.21.15.5:0/745456366 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:04.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.821+0000 7f12a1400640 1 -- 172.21.15.5:0/745456366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8117cda30 msgr2=0x55f8117d59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:04.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.821+0000 7f12a1400640 1 --2- 172.21.15.5:0/745456366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8117cda30 0x55f8117d59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.821+0000 7f12a1400640 1 -- 172.21.15.5:0/745456366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8117d8a20 msgr2=0x55f8117d6430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T06:48:04.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.821+0000 7f12a2800640 1 --2- 172.21.15.5:0/745456366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8117d8a20 0x55f8117d6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:04.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.821+0000 7f12a1400640 1 --2- 172.21.15.5:0/745456366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8117d8a20 0x55f8117d6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.821+0000 7f12a1400640 1 -- 172.21.15.5:0/745456366 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8116eb920 con 0x55f8117ce350 2024-04-24T06:48:04.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.821+0000 7f12a1e00640 1 --2- 172.21.15.5:0/745456366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8117cda30 0x55f8117d59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:48:04.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.821+0000 7f12a2800640 1 --2- 172.21.15.5:0/745456366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8117d8a20 0x55f8117d6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:48:04.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.822+0000 7f12a1400640 1 --2- 172.21.15.5:0/745456366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8117ce350 0x55f8117d5ef0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1294000c00 tx=0x7f1294004380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df8dfae7c672b4dd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:04.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.822+0000 7f1287e00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:48:04.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.822+0000 7f1287e00640 1 -- 172.21.15.5:0/745456366 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129402a060 con 0x55f8117ce350 2024-04-24T06:48:04.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.822+0000 7f12abc5f740 1 -- 172.21.15.5:0/745456366 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12940041d0 con 0x55f8117ce350 2024-04-24T06:48:04.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.822+0000 7f1287e00640 1 -- 172.21.15.5:0/745456366 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f12940035f0 con 0x55f8117ce350 2024-04-24T06:48:04.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.822+0000 7f1287e00640 1 -- 172.21.15.5:0/745456366 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f12940037f0 con 0x55f8117ce350 2024-04-24T06:48:04.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.822+0000 7f12abc5f740 10 client.26230 fetch_fsmap learned FSMap version 25 2024-04-24T06:48:04.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.822+0000 7f12abc5f740 1 -- 172.21.15.5:0/745456366 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f81150f7d0 con 0x55f8117ce350 2024-04-24T06:48:04.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.822+0000 7f1287e00640 1 -- 172.21.15.5:0/745456366 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12940041d0 con 0x55f8117ce350 2024-04-24T06:48:04.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.823+0000 7f1287e00640 1 -- 172.21.15.5:0/745456366 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1294004990 con 0x55f8117ce350 2024-04-24T06:48:04.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.823+0000 7f12abc5f740 10 client.26230 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:48:04.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.823+0000 7f12abc5f740 10 client.26230 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:48:04.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.823+0000 7f12abc5f740 20 client.26230 populate_metadata read hostname 'smithi005' 2024-04-24T06:48:04.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.823+0000 7f12abc5f740 1 --2- 172.21.15.5:0/745456366 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55f8117f14d0 0x55f8117f38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:04.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.823+0000 7f12abc5f740 4 client.26230 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:48:04.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.823+0000 7f12abc5f740 1 -- 172.21.15.5:0/745456366 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f811740c10 con 0x55f8117f14d0 2024-04-24T06:48:04.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.824+0000 7f12a1e00640 1 --2- 172.21.15.5:0/745456366 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55f8117f14d0 0x55f8117f38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:04.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.824+0000 7f1287e00640 10 client.26230 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:48:04.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.824+0000 7f12a1e00640 1 --2- 172.21.15.5:0/745456366 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55f8117f14d0 0x55f8117f38c0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58f5a1386881e5ae server_cookie=dad368de6fc99cc4 in_seq=0 out_seq=0 2024-04-24T06:48:04.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.825+0000 7f1287e00640 1 -- 172.21.15.5:0/745456366 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f811740c10 con 0x55f8117f14d0 2024-04-24T06:48:04.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.825+0000 7f1287e00640 10 client.26230 handle_command_reply: tid=0 2024-04-24T06:48:04.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.825+0000 7f12abc5f740 1 client.26230 shutdown 2024-04-24T06:48:04.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.826+0000 7f12abc5f740 1 -- 172.21.15.5:0/745456366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8117ce350 msgr2=0x55f8117d5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:04.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.826+0000 7f12abc5f740 1 --2- 172.21.15.5:0/745456366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8117ce350 0x55f8117d5ef0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1294000c00 tx=0x7f1294004380 comp rx=0 tx=0).stop 2024-04-24T06:48:04.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.826+0000 7f12abc5f740 1 -- 172.21.15.5:0/745456366 shutdown_connections 2024-04-24T06:48:04.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.826+0000 7f12abc5f740 1 --2- 172.21.15.5:0/745456366 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55f8117f14d0 0x55f8117f38c0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.826+0000 7f12abc5f740 1 --2- 172.21.15.5:0/745456366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8117d8a20 0x55f8117d6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.826+0000 7f12abc5f740 1 --2- 172.21.15.5:0/745456366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8117ce350 0x55f8117d5ef0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.826+0000 7f12abc5f740 1 --2- 172.21.15.5:0/745456366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8117cda30 0x55f8117d59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.826+0000 7f1287e00640 0 client.26230 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:48:04.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.826+0000 7f12abc5f740 1 -- 172.21.15.5:0/745456366 >> 172.21.15.5:0/745456366 conn(0x55f8117c9d50 msgr2=0x55f8117c8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:04.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.826+0000 7f12abc5f740 1 -- 172.21.15.5:0/745456366 shutdown_connections 2024-04-24T06:48:04.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.826+0000 7f12abc5f740 1 -- 172.21.15.5:0/745456366 wait complete. 2024-04-24T06:48:04.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.826+0000 7f12abc5f740 20 client.26230 trim_cache size 0 max 16384 2024-04-24T06:48:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.850+0000 7f12abc5f740 1 Processor -- start 2024-04-24T06:48:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.850+0000 7f12abc5f740 1 -- start start 2024-04-24T06:48:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.850+0000 7f12abc5f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8117cda30 0x55f8117e4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.850+0000 7f12abc5f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8117ce350 0x55f8117d01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.850+0000 7f12abc5f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8117d8a20 0x55f8117f8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.850+0000 7f12abc5f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f12a41ae210 con 0x55f8117d8a20 2024-04-24T06:48:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.850+0000 7f12abc5f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f12a41abae0 con 0x55f8117cda30 2024-04-24T06:48:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.850+0000 7f12abc5f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f12a41a9930 con 0x55f8117ce350 2024-04-24T06:48:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.851+0000 7f12a1e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8117cda30 0x55f8117e4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.851+0000 7f12a1e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8117cda30 0x55f8117e4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:52970/0 (socket says 172.21.15.5:52970) 2024-04-24T06:48:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.851+0000 7f12a1e00640 1 -- 172.21.15.5:0/2796260086 learned_addr learned my addr 172.21.15.5:0/2796260086 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.851+0000 7f12a1400640 1 --2- 172.21.15.5:0/2796260086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8117ce350 0x55f8117d01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.851+0000 7f12a2800640 1 --2- 172.21.15.5:0/2796260086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8117d8a20 0x55f8117f8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.851+0000 7f12a1e00640 1 -- 172.21.15.5:0/2796260086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8117ce350 msgr2=0x55f8117d01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.851+0000 7f12a1e00640 1 --2- 172.21.15.5:0/2796260086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8117ce350 0x55f8117d01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.851+0000 7f12a1e00640 1 -- 172.21.15.5:0/2796260086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8117d8a20 msgr2=0x55f8117f8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.851+0000 7f12a1e00640 1 --2- 172.21.15.5:0/2796260086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8117d8a20 0x55f8117f8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.851+0000 7f12a1e00640 1 -- 172.21.15.5:0/2796260086 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8114453f0 con 0x55f8117cda30 2024-04-24T06:48:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.851+0000 7f12a1e00640 1 --2- 172.21.15.5:0/2796260086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8117cda30 0x55f8117e4700 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f128c0bce80 tx=0x7f128c0ee6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c0b45867fd7a373 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.852+0000 7f1292a00640 1 -- 172.21.15.5:0/2796260086 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128c049040 con 0x55f8117cda30 2024-04-24T06:48:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.852+0000 7f1292a00640 1 -- 172.21.15.5:0/2796260086 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f128c031e40 con 0x55f8117cda30 2024-04-24T06:48:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.852+0000 7f1292a00640 1 -- 172.21.15.5:0/2796260086 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128c0c5b60 con 0x55f8117cda30 2024-04-24T06:48:04.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.853+0000 7f12abc5f740 1 -- 172.21.15.5:0/2796260086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8117cda30 msgr2=0x55f8117e4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:04.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.853+0000 7f12abc5f740 1 --2- 172.21.15.5:0/2796260086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8117cda30 0x55f8117e4700 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f128c0bce80 tx=0x7f128c0ee6c0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.853+0000 7f12abc5f740 1 -- 172.21.15.5:0/2796260086 shutdown_connections 2024-04-24T06:48:04.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.853+0000 7f12abc5f740 1 --2- 172.21.15.5:0/2796260086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8117d8a20 0x55f8117f8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.853+0000 7f12abc5f740 1 --2- 172.21.15.5:0/2796260086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8117ce350 0x55f8117d01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.853+0000 7f12abc5f740 1 --2- 172.21.15.5:0/2796260086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8117cda30 0x55f8117e4700 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.853+0000 7f12abc5f740 1 -- 172.21.15.5:0/2796260086 >> 172.21.15.5:0/2796260086 conn(0x55f8117c9d50 msgr2=0x55f8117d6200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:04.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.853+0000 7f12abc5f740 1 -- 172.21.15.5:0/2796260086 shutdown_connections 2024-04-24T06:48:04.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.853+0000 7f12abc5f740 1 -- 172.21.15.5:0/2796260086 wait complete. 2024-04-24T06:48:04.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.853+0000 7f12abc5f740 1 -- start start 2024-04-24T06:48:04.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.854+0000 7f12abc5f740 1 Processor -- start 2024-04-24T06:48:04.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.854+0000 7f12abc5f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8117cda30 0x55f8117d27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:04.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.854+0000 7f12abc5f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8117ce350 0x55f8117d2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:04.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.854+0000 7f12abc5f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8117d8a20 0x55f8117d3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:04.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.854+0000 7f12abc5f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f12a41a9fa0 con 0x55f8117cda30 2024-04-24T06:48:04.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.854+0000 7f12abc5f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f12a41ab410 con 0x55f8117d8a20 2024-04-24T06:48:04.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.854+0000 7f12abc5f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f12a41aa670 con 0x55f8117ce350 2024-04-24T06:48:04.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.854+0000 7f12abc5f740 10 client.0 _ll_register_callbacks cb 0x7f128c0f4960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:48:04.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.854+0000 7f12a2800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8117d8a20 0x55f8117d3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:04.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.854+0000 7f12a1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8117cda30 0x55f8117d27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:04.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.854+0000 7f12a1400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8117ce350 0x55f8117d2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:04.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.854+0000 7f12a2800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8117d8a20 0x55f8117d3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:52976/0 (socket says 172.21.15.5:52976) 2024-04-24T06:48:04.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.854+0000 7f12a2800640 1 -- 172.21.15.5:0/2005543722 learned_addr learned my addr 172.21.15.5:0/2005543722 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:04.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.854+0000 7f12a1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8117cda30 0x55f8117d27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47472/0 (socket says 172.21.15.5:47472) 2024-04-24T06:48:04.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.855+0000 7f12a1400640 1 -- 172.21.15.5:0/2005543722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8117d8a20 msgr2=0x55f8117d3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:04.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.855+0000 7f12a1400640 1 --2- 172.21.15.5:0/2005543722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8117d8a20 0x55f8117d3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.855+0000 7f12a1400640 1 -- 172.21.15.5:0/2005543722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8117cda30 msgr2=0x55f8117d27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.855+0000 7f12a1400640 1 --2- 172.21.15.5:0/2005543722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8117cda30 0x55f8117d27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.855+0000 7f12a1400640 1 -- 172.21.15.5:0/2005543722 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8114050f0 con 0x55f8117ce350 2024-04-24T06:48:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.855+0000 7f12a1400640 1 --2- 172.21.15.5:0/2005543722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8117ce350 0x55f8117d2d10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f12940107e0 tx=0x7f1294010d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc6385aa022e0348 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.855+0000 7f1287e00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:48:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.856+0000 7f1287e00640 1 -- 172.21.15.5:0/2005543722 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129402a040 con 0x55f8117ce350 2024-04-24T06:48:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.856+0000 7f1287e00640 1 -- 172.21.15.5:0/2005543722 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f12940035f0 con 0x55f8117ce350 2024-04-24T06:48:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.856+0000 7f12abc5f740 1 -- 172.21.15.5:0/2005543722 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f81173aa80 con 0x55f8117ce350 2024-04-24T06:48:04.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.856+0000 7f1287e00640 1 -- 172.21.15.5:0/2005543722 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1294003cc0 con 0x55f8117ce350 2024-04-24T06:48:04.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.856+0000 7f12abc5f740 10 client.26236 fetch_fsmap learned FSMap version 25 2024-04-24T06:48:04.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.856+0000 7f12abc5f740 1 -- 172.21.15.5:0/2005543722 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f8116f8810 con 0x55f8117ce350 2024-04-24T06:48:04.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.857+0000 7f1287e00640 1 -- 172.21.15.5:0/2005543722 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f81173aa80 con 0x55f8117ce350 2024-04-24T06:48:04.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.857+0000 7f1287e00640 1 -- 172.21.15.5:0/2005543722 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1294005330 con 0x55f8117ce350 2024-04-24T06:48:04.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.857+0000 7f12abc5f740 10 client.26236 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:48:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.857+0000 7f12abc5f740 10 client.26236 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:48:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.857+0000 7f12abc5f740 20 client.26236 populate_metadata read hostname 'smithi005' 2024-04-24T06:48:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.857+0000 7f12abc5f740 1 --2- 172.21.15.5:0/2005543722 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55f811857fc0 0x55f81185a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.857+0000 7f12abc5f740 4 client.26236 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:48:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.857+0000 7f12abc5f740 1 -- 172.21.15.5:0/2005543722 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f8116b6360 con 0x55f811857fc0 2024-04-24T06:48:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.857+0000 7f12a1e00640 1 --2- 172.21.15.5:0/2005543722 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55f811857fc0 0x55f81185a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.858+0000 7f12a1e00640 1 --2- 172.21.15.5:0/2005543722 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55f811857fc0 0x55f81185a3e0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76e33d6da620cdfe server_cookie=f7a5d33af10a1037 in_seq=0 out_seq=0 2024-04-24T06:48:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.858+0000 7f1287e00640 10 client.26236 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:48:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.858+0000 7f1287e00640 1 -- 172.21.15.5:0/2005543722 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55f8116b6360 con 0x55f811857fc0 2024-04-24T06:48:04.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.858+0000 7f1287e00640 10 client.26236 handle_command_reply: tid=0 2024-04-24T06:48:04.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.858+0000 7f12abc5f740 1 client.26236 shutdown 2024-04-24T06:48:04.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.859+0000 7f12abc5f740 1 -- 172.21.15.5:0/2005543722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8117ce350 msgr2=0x55f8117d2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:04.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.859+0000 7f12abc5f740 1 --2- 172.21.15.5:0/2005543722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8117ce350 0x55f8117d2d10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f12940107e0 tx=0x7f1294010d20 comp rx=0 tx=0).stop 2024-04-24T06:48:04.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.859+0000 7f12abc5f740 1 -- 172.21.15.5:0/2005543722 shutdown_connections 2024-04-24T06:48:04.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.859+0000 7f12abc5f740 1 --2- 172.21.15.5:0/2005543722 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55f811857fc0 0x55f81185a3e0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.859+0000 7f12abc5f740 1 --2- 172.21.15.5:0/2005543722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8117d8a20 0x55f8117d3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.859+0000 7f12abc5f740 1 --2- 172.21.15.5:0/2005543722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8117ce350 0x55f8117d2d10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.859+0000 7f12abc5f740 1 --2- 172.21.15.5:0/2005543722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8117cda30 0x55f8117d27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.859+0000 7f12abc5f740 1 -- 172.21.15.5:0/2005543722 >> 172.21.15.5:0/2005543722 conn(0x55f8117c9d50 msgr2=0x55f8117c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:04.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.859+0000 7f1287e00640 0 client.26236 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:48:04.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.859+0000 7f12abc5f740 1 -- 172.21.15.5:0/2005543722 shutdown_connections 2024-04-24T06:48:04.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.859+0000 7f12abc5f740 1 -- 172.21.15.5:0/2005543722 wait complete. 2024-04-24T06:48:04.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.859+0000 7f12abc5f740 20 client.26236 trim_cache size 0 max 16384 2024-04-24T06:48:04.866 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:48:04.867 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1069 inodes in the stack)", 2024-04-24T06:48:04.867 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-24T06:48:04.867 INFO:teuthology.orchestra.run.smithi005.stdout: "2fe0ad20-0ee9-40d8-8788-c73c6b2a576d": { 2024-04-24T06:48:04.867 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-24T06:48:04.867 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "2fe0ad20-0ee9-40d8-8788-c73c6b2a576d", 2024-04-24T06:48:04.867 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-04-24T06:48:04.867 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:48:04.867 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:48:04.868 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:48:04.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.860+0000 7f12a3200640 1 -- 172.21.15.5:0/3627504332 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f12880a3de0 msgr2=0x7f12880a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:04.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.860+0000 7f12a3200640 1 --2- 172.21.15.5:0/3627504332 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f12880a3de0 0x7f12880a62a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f129c005fd0 tx=0x7f129c009500 comp rx=0 tx=0).stop 2024-04-24T06:48:04.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.860+0000 7f12a3200640 1 -- 172.21.15.5:0/3627504332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12a4104260 msgr2=0x7f12a41a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:04.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.860+0000 7f12a3200640 1 --2- 172.21.15.5:0/3627504332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12a4104260 0x7f12a41a0650 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f128c02a960 tx=0x7f128c003b90 comp rx=0 tx=0).stop 2024-04-24T06:48:04.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.860+0000 7f12a3200640 1 -- 172.21.15.5:0/3627504332 shutdown_connections 2024-04-24T06:48:04.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.860+0000 7f12a3200640 1 --2- 172.21.15.5:0/3627504332 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f12880a3de0 0x7f12880a62a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.860+0000 7f12a3200640 1 --2- 172.21.15.5:0/3627504332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12a419a810 0x7f12a419acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.860+0000 7f12a3200640 1 --2- 172.21.15.5:0/3627504332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12a4109330 0x7f12a41a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.860+0000 7f12a3200640 1 --2- 172.21.15.5:0/3627504332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12a4104260 0x7f12a41a0650 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:04.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.860+0000 7f12a3200640 1 -- 172.21.15.5:0/3627504332 >> 172.21.15.5:0/3627504332 conn(0x7f12a40ffaa0 msgr2=0x7f12a4100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:04.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.860+0000 7f12a3200640 1 -- 172.21.15.5:0/3627504332 shutdown_connections 2024-04-24T06:48:04.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:04.861+0000 7f12a3200640 1 -- 172.21.15.5:0/3627504332 wait complete. 2024-04-24T06:48:04.884 INFO:tasks.cephfs.filesystem:scrub status for tag:2fe0ad20-0ee9-40d8-8788-c73c6b2a576d - {'path': '/', 'tag': '2fe0ad20-0ee9-40d8-8788-c73c6b2a576d', 'options': 'recursive,force'} 2024-04-24T06:48:05.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:05 smithi137 ceph-mon[33017]: pgmap v124: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 38 KiB/s wr, 23 op/s 2024-04-24T06:48:05.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:05 smithi137 ceph-mon[33017]: scrub queued for path: / 2024-04-24T06:48:05.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:05 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:48:05.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:05 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:48:05.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:05 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:48:05.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:05 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:48:05.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:05 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:48:05.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:05 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:48:05.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:05 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:48:05.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:05 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:48:06.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:05 smithi005 ceph-mon[27798]: pgmap v124: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 38 KiB/s wr, 23 op/s 2024-04-24T06:48:06.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:05 smithi005 ceph-mon[27798]: scrub queued for path: / 2024-04-24T06:48:06.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:05 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:48:06.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:05 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:48:06.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:05 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:48:06.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:05 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:48:06.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:05 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:48:06.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:05 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:48:06.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:05 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:48:06.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:05 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:48:06.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:05 smithi037 ceph-mon[32864]: pgmap v124: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 38 KiB/s wr, 23 op/s 2024-04-24T06:48:06.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:05 smithi037 ceph-mon[32864]: scrub queued for path: / 2024-04-24T06:48:06.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:05 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:48:06.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:05 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:48:06.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:05 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:48:06.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:05 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:48:06.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:05 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:48:06.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:05 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:48:06.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:05 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:48:06.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:05 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:48:06.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:06 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:48:06.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:06 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:48:06.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:06 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:48:07.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:06 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:48:07.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:06 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:48:07.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:06 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:48:07.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:06 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:48:07.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:06 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:48:07.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:06 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:48:07.920 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:07 smithi037 ceph-mon[32864]: pgmap v125: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 38 KiB/s wr, 20 op/s 2024-04-24T06:48:07.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:07 smithi137 ceph-mon[33017]: pgmap v125: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 38 KiB/s wr, 20 op/s 2024-04-24T06:48:08.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:07 smithi005 ceph-mon[27798]: pgmap v125: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 38 KiB/s wr, 20 op/s 2024-04-24T06:48:09.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:09 smithi137 ceph-mon[33017]: pgmap v126: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 43 KiB/s wr, 22 op/s 2024-04-24T06:48:10.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:09 smithi005 ceph-mon[27798]: pgmap v126: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 43 KiB/s wr, 22 op/s 2024-04-24T06:48:10.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:09 smithi037 ceph-mon[32864]: pgmap v126: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 43 KiB/s wr, 22 op/s 2024-04-24T06:48:11.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:11 smithi137 ceph-mon[33017]: pgmap v127: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 43 KiB/s wr, 23 op/s 2024-04-24T06:48:12.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:11 smithi005 ceph-mon[27798]: pgmap v127: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 43 KiB/s wr, 23 op/s 2024-04-24T06:48:12.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:11 smithi037 ceph-mon[32864]: pgmap v127: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 43 KiB/s wr, 23 op/s 2024-04-24T06:48:13.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:13 smithi137 ceph-mon[33017]: pgmap v128: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 43 KiB/s wr, 21 op/s 2024-04-24T06:48:14.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:13 smithi005 ceph-mon[27798]: pgmap v128: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 43 KiB/s wr, 21 op/s 2024-04-24T06:48:14.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:13 smithi037 ceph-mon[32864]: pgmap v128: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 43 KiB/s wr, 21 op/s 2024-04-24T06:48:15.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:15 smithi137 ceph-mon[33017]: pgmap v129: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 43 KiB/s wr, 24 op/s 2024-04-24T06:48:16.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:15 smithi005 ceph-mon[27798]: pgmap v129: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 43 KiB/s wr, 24 op/s 2024-04-24T06:48:16.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:15 smithi037 ceph-mon[32864]: pgmap v129: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 43 KiB/s wr, 24 op/s 2024-04-24T06:48:17.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:17 smithi137 ceph-mon[33017]: pgmap v130: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.0 KiB/s wr, 5 op/s 2024-04-24T06:48:17.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:17 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:48:18.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:17 smithi005 ceph-mon[27798]: pgmap v130: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.0 KiB/s wr, 5 op/s 2024-04-24T06:48:18.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:17 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:48:18.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:17 smithi037 ceph-mon[32864]: pgmap v130: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.0 KiB/s wr, 5 op/s 2024-04-24T06:48:18.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:17 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:48:18.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:18 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:48:18.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:18 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:48:18.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:18 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:48:18.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:18 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:48:18.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:18 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:48:18.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:18 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:48:19.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:18 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:48:19.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:18 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:48:19.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:18 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:48:19.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:18 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:48:19.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:18 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:48:19.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:18 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:48:19.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:18 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:48:19.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:18 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:48:19.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:18 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:48:19.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:18 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:48:19.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:18 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:48:19.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:18 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:48:19.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:19 smithi137 ceph-mon[33017]: pgmap v131: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.8 KiB/s wr, 6 op/s 2024-04-24T06:48:20.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:19 smithi005 ceph-mon[27798]: pgmap v131: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.8 KiB/s wr, 6 op/s 2024-04-24T06:48:20.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:19 smithi037 ceph-mon[32864]: pgmap v131: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.8 KiB/s wr, 6 op/s 2024-04-24T06:48:21.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:21 smithi137 ceph-mon[33017]: pgmap v132: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-24T06:48:22.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:21 smithi005 ceph-mon[27798]: pgmap v132: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-24T06:48:22.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:21 smithi037 ceph-mon[32864]: pgmap v132: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-24T06:48:23.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:23 smithi137 ceph-mon[33017]: pgmap v133: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-04-24T06:48:24.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:23 smithi005 ceph-mon[27798]: pgmap v133: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-04-24T06:48:24.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:23 smithi037 ceph-mon[32864]: pgmap v133: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-04-24T06:48:25.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:25 smithi137 ceph-mon[33017]: pgmap v134: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-04-24T06:48:26.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:25 smithi005 ceph-mon[27798]: pgmap v134: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-04-24T06:48:26.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:25 smithi037 ceph-mon[32864]: pgmap v134: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-04-24T06:48:27.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:27 smithi137 ceph-mon[33017]: pgmap v135: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-04-24T06:48:28.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:27 smithi005 ceph-mon[27798]: pgmap v135: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-04-24T06:48:28.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:27 smithi037 ceph-mon[32864]: pgmap v135: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-04-24T06:48:29.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:29 smithi137 ceph-mon[33017]: pgmap v136: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-04-24T06:48:30.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:29 smithi005 ceph-mon[27798]: pgmap v136: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-04-24T06:48:30.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:29 smithi037 ceph-mon[32864]: pgmap v136: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-04-24T06:48:31.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:31 smithi137 ceph-mon[33017]: pgmap v137: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-24T06:48:32.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:31 smithi005 ceph-mon[27798]: pgmap v137: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-24T06:48:32.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:31 smithi037 ceph-mon[32864]: pgmap v137: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-24T06:48:33.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:33 smithi137 ceph-mon[33017]: pgmap v138: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 5 op/s 2024-04-24T06:48:34.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:33 smithi005 ceph-mon[27798]: pgmap v138: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 5 op/s 2024-04-24T06:48:34.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:33 smithi037 ceph-mon[32864]: pgmap v138: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 5 op/s 2024-04-24T06:48:34.885 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T06:48:34.886 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T06:48:35.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.111+0000 7f319ca00640 1 -- 172.21.15.5:0/3830749179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3198109320 msgr2=0x7f3198110ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:35.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.111+0000 7f319ca00640 1 --2- 172.21.15.5:0/3830749179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3198109320 0x7f3198110ef0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3190009ca0 tx=0x7f319002a8c0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.111+0000 7f319ca00640 1 -- 172.21.15.5:0/3830749179 shutdown_connections 2024-04-24T06:48:35.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.111+0000 7f319ca00640 1 --2- 172.21.15.5:0/3830749179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3198109320 0x7f3198110ef0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.111+0000 7f319ca00640 1 --2- 172.21.15.5:0/3830749179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3198104c00 0x7f3198108c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.111+0000 7f319ca00640 1 --2- 172.21.15.5:0/3830749179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3198104250 0x7f3198104630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.111+0000 7f319ca00640 1 -- 172.21.15.5:0/3830749179 >> 172.21.15.5:0/3830749179 conn(0x7f31980ffad0 msgr2=0x7f3198101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:35.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.111+0000 7f319ca00640 1 -- 172.21.15.5:0/3830749179 shutdown_connections 2024-04-24T06:48:35.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.112+0000 7f319ca00640 1 -- 172.21.15.5:0/3830749179 wait complete. 2024-04-24T06:48:35.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.112+0000 7f319ca00640 1 Processor -- start 2024-04-24T06:48:35.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.113+0000 7f319ca00640 1 -- start start 2024-04-24T06:48:35.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.113+0000 7f319ca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3198104250 0x7f31981a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:35.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.113+0000 7f319ca00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3198109320 0x7f31981a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:35.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.113+0000 7f319ca00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f319819a7b0 0x7f319819ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:35.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.113+0000 7f319ca00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3198114430 con 0x7f3198104250 2024-04-24T06:48:35.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.113+0000 7f319ca00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f31981142b0 con 0x7f319819a7b0 2024-04-24T06:48:35.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.113+0000 7f319ca00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f31981145b0 con 0x7f3198109320 2024-04-24T06:48:35.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.113+0000 7f3196800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3198104250 0x7f31981a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:35.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.113+0000 7f3196800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3198104250 0x7f31981a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55912/0 (socket says 172.21.15.5:55912) 2024-04-24T06:48:35.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.113+0000 7f3196800640 1 -- 172.21.15.5:0/3343565165 learned_addr learned my addr 172.21.15.5:0/3343565165 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:35.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.113+0000 7f3197200640 1 --2- 172.21.15.5:0/3343565165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f319819a7b0 0x7f319819ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:35.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.114+0000 7f3195e00640 1 --2- 172.21.15.5:0/3343565165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3198109320 0x7f31981a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:35.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.114+0000 7f3196800640 1 -- 172.21.15.5:0/3343565165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3198109320 msgr2=0x7f31981a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:35.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.114+0000 7f3196800640 1 --2- 172.21.15.5:0/3343565165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3198109320 0x7f31981a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.114+0000 7f3196800640 1 -- 172.21.15.5:0/3343565165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f319819a7b0 msgr2=0x7f319819ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:35.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.114+0000 7f3196800640 1 --2- 172.21.15.5:0/3343565165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f319819a7b0 0x7f319819ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.114+0000 7f3196800640 1 -- 172.21.15.5:0/3343565165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f319819b340 con 0x7f3198104250 2024-04-24T06:48:35.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.114+0000 7f3195e00640 1 --2- 172.21.15.5:0/3343565165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3198109320 0x7f31981a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:48:35.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.114+0000 7f3196800640 1 --2- 172.21.15.5:0/3343565165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3198104250 0x7f31981a0640 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f318800ea60 tx=0x7f318800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:35.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.114+0000 7f318f400640 1 -- 172.21.15.5:0/3343565165 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318800cde0 con 0x7f3198104250 2024-04-24T06:48:35.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.114+0000 7f318f400640 1 -- 172.21.15.5:0/3343565165 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31880035f0 con 0x7f3198104250 2024-04-24T06:48:35.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.114+0000 7f319ca00640 1 -- 172.21.15.5:0/3343565165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f319819b630 con 0x7f3198104250 2024-04-24T06:48:35.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.114+0000 7f318f400640 1 -- 172.21.15.5:0/3343565165 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3188010670 con 0x7f3198104250 2024-04-24T06:48:35.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.115+0000 7f319ca00640 1 -- 172.21.15.5:0/3343565165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31981a7810 con 0x7f3198104250 2024-04-24T06:48:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.118+0000 7f31a0237740 1 Processor -- start 2024-04-24T06:48:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.118+0000 7f31a0237740 1 -- start start 2024-04-24T06:48:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.118+0000 7f318f400640 1 -- 172.21.15.5:0/3343565165 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f31880037a0 con 0x7f3198104250 2024-04-24T06:48:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.118+0000 7f31a0237740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555703c32a30 0x555703c32e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.118+0000 7f31a0237740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555703c33350 0x555703c3d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.118+0000 7f31a0237740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555703c3da20 0x555703c3fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:35.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.118+0000 7f31a0237740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555703b981f0 con 0x555703c3da20 2024-04-24T06:48:35.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.118+0000 7f31a0237740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555703b3e350 con 0x555703c32a30 2024-04-24T06:48:35.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.118+0000 7f31a0237740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555703b941b0 con 0x555703c33350 2024-04-24T06:48:35.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.119+0000 7f3195e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555703c3da20 0x555703c3fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:35.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.119+0000 7f3195e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555703c3da20 0x555703c3fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55914/0 (socket says 172.21.15.5:55914) 2024-04-24T06:48:35.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.119+0000 7f3195e00640 1 -- 172.21.15.5:0/2319004236 learned_addr learned my addr 172.21.15.5:0/2319004236 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:35.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.119+0000 7f3196800640 1 --2- 172.21.15.5:0/2319004236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555703c33350 0x555703c3d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:35.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.119+0000 7f3197200640 1 --2- 172.21.15.5:0/2319004236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555703c32a30 0x555703c32e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:35.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.119+0000 7f3196800640 1 -- 172.21.15.5:0/2319004236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555703c32a30 msgr2=0x555703c32e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:35.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.119+0000 7f3196800640 1 --2- 172.21.15.5:0/2319004236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555703c32a30 0x555703c32e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.119+0000 7f3196800640 1 -- 172.21.15.5:0/2319004236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555703c3da20 msgr2=0x555703c3fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:35.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.119+0000 7f3196800640 1 --2- 172.21.15.5:0/2319004236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555703c3da20 0x555703c3fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.119+0000 7f3196800640 1 -- 172.21.15.5:0/2319004236 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555703b92f50 con 0x555703c33350 2024-04-24T06:48:35.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.119+0000 7f3196800640 1 --2- 172.21.15.5:0/2319004236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555703c33350 0x555703c3d4e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f31880ce2c0 tx=0x7f31880d9680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f2af7e81f4814be server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:35.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.120+0000 7f318f400640 1 --2- 172.21.15.5:0/3343565165 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f317c0a3d20 0x7f317c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.120+0000 7f3183400640 1 -- 172.21.15.5:0/2319004236 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31880e0020 con 0x555703c33350 2024-04-24T06:48:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.120+0000 7f3183400640 1 -- 172.21.15.5:0/2319004236 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f318809d210 con 0x555703c33350 2024-04-24T06:48:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.120+0000 7f3183400640 1 -- 172.21.15.5:0/2319004236 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3188098db0 con 0x555703c33350 2024-04-24T06:48:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.120+0000 7f318f400640 1 -- 172.21.15.5:0/3343565165 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f31880d0bc0 con 0x7f3198104250 2024-04-24T06:48:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.120+0000 7f3197200640 1 --2- 172.21.15.5:0/3343565165 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f317c0a3d20 0x7f317c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.121+0000 7f31a0237740 1 -- 172.21.15.5:0/2319004236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555703c33350 msgr2=0x555703c3d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.121+0000 7f31a0237740 1 --2- 172.21.15.5:0/2319004236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555703c33350 0x555703c3d4e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f31880ce2c0 tx=0x7f31880d9680 comp rx=0 tx=0).stop 2024-04-24T06:48:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.121+0000 7f3197200640 1 --2- 172.21.15.5:0/3343565165 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f317c0a3d20 0x7f317c0a61e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3190006010 tx=0x7f3190005d80 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.121+0000 7f31a0237740 1 -- 172.21.15.5:0/2319004236 shutdown_connections 2024-04-24T06:48:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.121+0000 7f31a0237740 1 --2- 172.21.15.5:0/2319004236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555703c3da20 0x555703c3fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.121+0000 7f31a0237740 1 --2- 172.21.15.5:0/2319004236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555703c33350 0x555703c3d4e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.121+0000 7f31a0237740 1 --2- 172.21.15.5:0/2319004236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555703c32a30 0x555703c32e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.121+0000 7f31a0237740 1 -- 172.21.15.5:0/2319004236 >> 172.21.15.5:0/2319004236 conn(0x555703c2ed50 msgr2=0x555703c2f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.121+0000 7f31a0237740 1 -- 172.21.15.5:0/2319004236 shutdown_connections 2024-04-24T06:48:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.121+0000 7f31a0237740 1 -- 172.21.15.5:0/2319004236 wait complete. 2024-04-24T06:48:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.121+0000 7f31a0237740 1 -- start start 2024-04-24T06:48:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.122+0000 7f31a0237740 1 Processor -- start 2024-04-24T06:48:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.122+0000 7f31a0237740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555703c32a30 0x555703c3a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.122+0000 7f31a0237740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555703c33350 0x555703c3ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.122+0000 7f31a0237740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555703c3da20 0x555703c3b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.122+0000 7f31a0237740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555703c40810 con 0x555703c3da20 2024-04-24T06:48:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.122+0000 7f31a0237740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555703b51690 con 0x555703c32a30 2024-04-24T06:48:35.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.122+0000 7f31a0237740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555703c40990 con 0x555703c33350 2024-04-24T06:48:35.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.122+0000 7f31a0237740 10 client.0 _ll_register_callbacks cb 0x555703b8d650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:48:35.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.123+0000 7f3197200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555703c3da20 0x555703c3b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:35.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.123+0000 7f3196800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555703c32a30 0x555703c3a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:35.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.123+0000 7f3197200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555703c3da20 0x555703c3b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55924/0 (socket says 172.21.15.5:55924) 2024-04-24T06:48:35.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.123+0000 7f3197200640 1 -- 172.21.15.5:0/1390733844 learned_addr learned my addr 172.21.15.5:0/1390733844 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:35.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.123+0000 7f3195e00640 1 --2- 172.21.15.5:0/1390733844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555703c33350 0x555703c3ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:35.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.123+0000 7f3195e00640 1 -- 172.21.15.5:0/1390733844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555703c32a30 msgr2=0x555703c3a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:35.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.123+0000 7f3195e00640 1 --2- 172.21.15.5:0/1390733844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555703c32a30 0x555703c3a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.123+0000 7f3195e00640 1 -- 172.21.15.5:0/1390733844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555703c3da20 msgr2=0x555703c3b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:35.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.123+0000 7f3195e00640 1 --2- 172.21.15.5:0/1390733844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555703c3da20 0x555703c3b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.123+0000 7f3195e00640 1 -- 172.21.15.5:0/1390733844 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555703b50920 con 0x555703c33350 2024-04-24T06:48:35.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.123+0000 7f3196800640 1 --2- 172.21.15.5:0/1390733844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555703c32a30 0x555703c3a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:48:35.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.123+0000 7f3197200640 1 --2- 172.21.15.5:0/1390733844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555703c3da20 0x555703c3b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:48:35.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.123+0000 7f3195e00640 1 --2- 172.21.15.5:0/1390733844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555703c33350 0x555703c3ae70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f31840136e0 tx=0x7f3184013ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad6d72692333166 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:35.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.123+0000 7f3177e00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:48:35.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.124+0000 7f3177e00640 1 -- 172.21.15.5:0/1390733844 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3184005020 con 0x555703c33350 2024-04-24T06:48:35.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.124+0000 7f3177e00640 1 -- 172.21.15.5:0/1390733844 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f31840051c0 con 0x555703c33350 2024-04-24T06:48:35.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.124+0000 7f31a0237740 1 -- 172.21.15.5:0/1390733844 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f318809d210 con 0x555703c33350 2024-04-24T06:48:35.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.124+0000 7f3177e00640 1 -- 172.21.15.5:0/1390733844 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f31840053c0 con 0x555703c33350 2024-04-24T06:48:35.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.124+0000 7f31a0237740 10 client.26248 fetch_fsmap learned FSMap version 25 2024-04-24T06:48:35.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.124+0000 7f31a0237740 1 -- 172.21.15.5:0/1390733844 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5557039747d0 con 0x555703c33350 2024-04-24T06:48:35.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.125+0000 7f3177e00640 1 -- 172.21.15.5:0/1390733844 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f318809d210 con 0x555703c33350 2024-04-24T06:48:35.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.125+0000 7f3177e00640 1 -- 172.21.15.5:0/1390733844 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f318401a030 con 0x555703c33350 2024-04-24T06:48:35.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.125+0000 7f31a0237740 10 client.26248 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:48:35.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.125+0000 7f31a0237740 10 client.26248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:48:35.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.125+0000 7f31a0237740 20 client.26248 populate_metadata read hostname 'smithi005' 2024-04-24T06:48:35.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.125+0000 7f31a0237740 1 --2- 172.21.15.5:0/1390733844 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x555703c4de70 0x555703c502f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:35.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.125+0000 7f31a0237740 4 client.26248 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:48:35.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.125+0000 7f31a0237740 1 -- 172.21.15.5:0/1390733844 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555703ba5c10 con 0x555703c4de70 2024-04-24T06:48:35.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.125+0000 7f3196800640 1 --2- 172.21.15.5:0/1390733844 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x555703c4de70 0x555703c502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:35.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.126+0000 7f3196800640 1 --2- 172.21.15.5:0/1390733844 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x555703c4de70 0x555703c502f0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8cb6f4584c28bed server_cookie=61812afec8585553 in_seq=0 out_seq=0 2024-04-24T06:48:35.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.126+0000 7f3177e00640 10 client.26248 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:48:35.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.127+0000 7f3177e00640 1 -- 172.21.15.5:0/1390733844 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555703ba5c10 con 0x555703c4de70 2024-04-24T06:48:35.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.127+0000 7f3177e00640 10 client.26248 handle_command_reply: tid=0 2024-04-24T06:48:35.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.127+0000 7f31a0237740 1 client.26248 shutdown 2024-04-24T06:48:35.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.127+0000 7f31a0237740 1 -- 172.21.15.5:0/1390733844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555703c33350 msgr2=0x555703c3ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:35.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.127+0000 7f31a0237740 1 --2- 172.21.15.5:0/1390733844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555703c33350 0x555703c3ae70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f31840136e0 tx=0x7f3184013ba0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.127+0000 7f31a0237740 1 -- 172.21.15.5:0/1390733844 shutdown_connections 2024-04-24T06:48:35.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.127+0000 7f31a0237740 1 --2- 172.21.15.5:0/1390733844 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x555703c4de70 0x555703c502f0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.127+0000 7f31a0237740 1 --2- 172.21.15.5:0/1390733844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555703c3da20 0x555703c3b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.127+0000 7f31a0237740 1 --2- 172.21.15.5:0/1390733844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555703c33350 0x555703c3ae70 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.127+0000 7f31a0237740 1 --2- 172.21.15.5:0/1390733844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555703c32a30 0x555703c3a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.127+0000 7f3177e00640 0 client.26248 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:48:35.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.127+0000 7f31a0237740 1 -- 172.21.15.5:0/1390733844 >> 172.21.15.5:0/1390733844 conn(0x555703c2ed50 msgr2=0x555703c34b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:35.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.127+0000 7f31a0237740 1 -- 172.21.15.5:0/1390733844 shutdown_connections 2024-04-24T06:48:35.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.127+0000 7f31a0237740 1 -- 172.21.15.5:0/1390733844 wait complete. 2024-04-24T06:48:35.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.127+0000 7f31a0237740 20 client.26248 trim_cache size 0 max 16384 2024-04-24T06:48:35.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.147+0000 7f31a0237740 1 Processor -- start 2024-04-24T06:48:35.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.147+0000 7f31a0237740 1 -- start start 2024-04-24T06:48:35.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.147+0000 7f31a0237740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555703c32a30 0x555703c496d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:35.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.147+0000 7f31a0237740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555703c33350 0x555703c35190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:35.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.147+0000 7f31a0237740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555703c3da20 0x555703c2de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:35.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.147+0000 7f31a0237740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f31981b2690 con 0x555703c3da20 2024-04-24T06:48:35.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.147+0000 7f31a0237740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f31981aff60 con 0x555703c33350 2024-04-24T06:48:35.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.147+0000 7f31a0237740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f31981addb0 con 0x555703c32a30 2024-04-24T06:48:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.147+0000 7f3197200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555703c3da20 0x555703c2de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.148+0000 7f3197200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555703c3da20 0x555703c2de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55940/0 (socket says 172.21.15.5:55940) 2024-04-24T06:48:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.148+0000 7f3197200640 1 -- 172.21.15.5:0/2038127877 learned_addr learned my addr 172.21.15.5:0/2038127877 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:35.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.148+0000 7f3195e00640 1 --2- 172.21.15.5:0/2038127877 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555703c33350 0x555703c35190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:35.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.148+0000 7f3196800640 1 --2- 172.21.15.5:0/2038127877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555703c32a30 0x555703c496d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:35.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.148+0000 7f3197200640 1 -- 172.21.15.5:0/2038127877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555703c32a30 msgr2=0x555703c496d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:35.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.148+0000 7f3197200640 1 --2- 172.21.15.5:0/2038127877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555703c32a30 0x555703c496d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.148+0000 7f3197200640 1 -- 172.21.15.5:0/2038127877 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555703c33350 msgr2=0x555703c35190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:35.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.148+0000 7f3197200640 1 --2- 172.21.15.5:0/2038127877 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555703c33350 0x555703c35190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.148+0000 7f3197200640 1 -- 172.21.15.5:0/2038127877 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557038aa3f0 con 0x555703c3da20 2024-04-24T06:48:35.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.148+0000 7f3195e00640 1 --2- 172.21.15.5:0/2038127877 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555703c33350 0x555703c35190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:48:35.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.148+0000 7f3197200640 1 --2- 172.21.15.5:0/2038127877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555703c3da20 0x555703c2de50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f319002ade0 tx=0x7f3190041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d86cdad443905a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:35.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.149+0000 7f3182a00640 1 -- 172.21.15.5:0/2038127877 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3190031450 con 0x555703c3da20 2024-04-24T06:48:35.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.149+0000 7f3182a00640 1 -- 172.21.15.5:0/2038127877 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3190036050 con 0x555703c3da20 2024-04-24T06:48:35.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.149+0000 7f3182a00640 1 -- 172.21.15.5:0/2038127877 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319002f610 con 0x555703c3da20 2024-04-24T06:48:35.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.150+0000 7f31a0237740 1 -- 172.21.15.5:0/2038127877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555703c3da20 msgr2=0x555703c2de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:35.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.150+0000 7f31a0237740 1 --2- 172.21.15.5:0/2038127877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555703c3da20 0x555703c2de50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f319002ade0 tx=0x7f3190041000 comp rx=0 tx=0).stop 2024-04-24T06:48:35.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.150+0000 7f31a0237740 1 -- 172.21.15.5:0/2038127877 shutdown_connections 2024-04-24T06:48:35.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.150+0000 7f31a0237740 1 --2- 172.21.15.5:0/2038127877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555703c3da20 0x555703c2de50 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.150+0000 7f31a0237740 1 --2- 172.21.15.5:0/2038127877 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555703c33350 0x555703c35190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.150+0000 7f31a0237740 1 --2- 172.21.15.5:0/2038127877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555703c32a30 0x555703c496d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.150+0000 7f31a0237740 1 -- 172.21.15.5:0/2038127877 >> 172.21.15.5:0/2038127877 conn(0x555703c2ed50 msgr2=0x555703c2d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:35.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.150+0000 7f31a0237740 1 -- 172.21.15.5:0/2038127877 shutdown_connections 2024-04-24T06:48:35.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.150+0000 7f31a0237740 1 -- 172.21.15.5:0/2038127877 wait complete. 2024-04-24T06:48:35.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.150+0000 7f31a0237740 1 -- start start 2024-04-24T06:48:35.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.151+0000 7f31a0237740 1 Processor -- start 2024-04-24T06:48:35.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.151+0000 7f31a0237740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555703c32a30 0x555703c45dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:35.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.151+0000 7f31a0237740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555703c33350 0x555703c371a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:35.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.151+0000 7f31a0237740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555703c3da20 0x555703c376e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:35.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.151+0000 7f31a0237740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f31981ae420 con 0x555703c3da20 2024-04-24T06:48:35.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.151+0000 7f31a0237740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f31981af890 con 0x555703c33350 2024-04-24T06:48:35.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.151+0000 7f31a0237740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f31981aeaf0 con 0x555703c32a30 2024-04-24T06:48:35.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.151+0000 7f31a0237740 10 client.0 _ll_register_callbacks cb 0x7f3184009820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:48:35.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.151+0000 7f3197200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555703c3da20 0x555703c376e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:35.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.151+0000 7f3196800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555703c32a30 0x555703c45dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:35.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.151+0000 7f3197200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555703c3da20 0x555703c376e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55942/0 (socket says 172.21.15.5:55942) 2024-04-24T06:48:35.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.151+0000 7f3197200640 1 -- 172.21.15.5:0/1281595788 learned_addr learned my addr 172.21.15.5:0/1281595788 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:35.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.151+0000 7f3195e00640 1 --2- 172.21.15.5:0/1281595788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555703c33350 0x555703c371a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:35.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.152+0000 7f3196800640 1 -- 172.21.15.5:0/1281595788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555703c33350 msgr2=0x555703c371a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:35.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.152+0000 7f3196800640 1 --2- 172.21.15.5:0/1281595788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555703c33350 0x555703c371a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.152+0000 7f3196800640 1 -- 172.21.15.5:0/1281595788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555703c3da20 msgr2=0x555703c376e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:35.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.152+0000 7f3196800640 1 --2- 172.21.15.5:0/1281595788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555703c3da20 0x555703c376e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.152+0000 7f3196800640 1 -- 172.21.15.5:0/1281595788 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55570386a0f0 con 0x555703c32a30 2024-04-24T06:48:35.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.152+0000 7f3197200640 1 --2- 172.21.15.5:0/1281595788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555703c3da20 0x555703c376e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:48:35.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.152+0000 7f3196800640 1 --2- 172.21.15.5:0/1281595788 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555703c32a30 0x555703c45dd0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f31880ce2c0 tx=0x7f3188003f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21f0a3b87357214c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:35.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.152+0000 7f3177e00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:48:35.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.152+0000 7f3177e00640 1 -- 172.21.15.5:0/1281595788 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3188047700 con 0x555703c32a30 2024-04-24T06:48:35.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.152+0000 7f3177e00640 1 -- 172.21.15.5:0/1281595788 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f31880bd0b0 con 0x555703c32a30 2024-04-24T06:48:35.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.152+0000 7f31a0237740 1 -- 172.21.15.5:0/1281595788 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3190036050 con 0x555703c32a30 2024-04-24T06:48:35.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.153+0000 7f3177e00640 1 -- 172.21.15.5:0/1281595788 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f31880478d0 con 0x555703c32a30 2024-04-24T06:48:35.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.153+0000 7f31a0237740 10 client.26254 fetch_fsmap learned FSMap version 25 2024-04-24T06:48:35.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.153+0000 7f31a0237740 1 -- 172.21.15.5:0/1281595788 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555703b5d810 con 0x555703c32a30 2024-04-24T06:48:35.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.153+0000 7f3177e00640 1 -- 172.21.15.5:0/1281595788 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3190036050 con 0x555703c32a30 2024-04-24T06:48:35.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.153+0000 7f3177e00640 1 -- 172.21.15.5:0/1281595788 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3188099180 con 0x555703c32a30 2024-04-24T06:48:35.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.153+0000 7f31a0237740 10 client.26254 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:48:35.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.153+0000 7f31a0237740 10 client.26254 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:48:35.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.153+0000 7f31a0237740 20 client.26254 populate_metadata read hostname 'smithi005' 2024-04-24T06:48:35.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.153+0000 7f31a0237740 1 --2- 172.21.15.5:0/1281595788 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x555703cb4150 0x555703cb6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:35.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.153+0000 7f31a0237740 4 client.26254 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:48:35.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.153+0000 7f31a0237740 1 -- 172.21.15.5:0/1281595788 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555703b9fa80 con 0x555703cb4150 2024-04-24T06:48:35.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.153+0000 7f3195e00640 1 --2- 172.21.15.5:0/1281595788 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x555703cb4150 0x555703cb6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:35.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.154+0000 7f3195e00640 1 --2- 172.21.15.5:0/1281595788 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x555703cb4150 0x555703cb6570 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc58e01bec61ea7b server_cookie=96c5ea6bdc2d6d6b in_seq=0 out_seq=0 2024-04-24T06:48:35.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.154+0000 7f3177e00640 10 client.26254 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:48:35.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.154+0000 7f3177e00640 1 -- 172.21.15.5:0/1281595788 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555703b9fa80 con 0x555703cb4150 2024-04-24T06:48:35.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.154+0000 7f3177e00640 10 client.26254 handle_command_reply: tid=0 2024-04-24T06:48:35.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.154+0000 7f31a0237740 1 client.26254 shutdown 2024-04-24T06:48:35.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.155+0000 7f31a0237740 1 -- 172.21.15.5:0/1281595788 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555703c32a30 msgr2=0x555703c45dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:35.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.155+0000 7f31a0237740 1 --2- 172.21.15.5:0/1281595788 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555703c32a30 0x555703c45dd0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f31880ce2c0 tx=0x7f3188003f30 comp rx=0 tx=0).stop 2024-04-24T06:48:35.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.155+0000 7f31a0237740 1 -- 172.21.15.5:0/1281595788 shutdown_connections 2024-04-24T06:48:35.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.155+0000 7f31a0237740 1 --2- 172.21.15.5:0/1281595788 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x555703cb4150 0x555703cb6570 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.155+0000 7f31a0237740 1 --2- 172.21.15.5:0/1281595788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555703c3da20 0x555703c376e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.155+0000 7f3177e00640 0 client.26254 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:48:35.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.155+0000 7f31a0237740 1 --2- 172.21.15.5:0/1281595788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555703c33350 0x555703c371a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.155+0000 7f31a0237740 1 --2- 172.21.15.5:0/1281595788 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555703c32a30 0x555703c45dd0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.155+0000 7f31a0237740 1 -- 172.21.15.5:0/1281595788 >> 172.21.15.5:0/1281595788 conn(0x555703c2ed50 msgr2=0x555703c3b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:35.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.155+0000 7f31a0237740 1 -- 172.21.15.5:0/1281595788 shutdown_connections 2024-04-24T06:48:35.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.155+0000 7f31a0237740 1 -- 172.21.15.5:0/1281595788 wait complete. 2024-04-24T06:48:35.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.155+0000 7f31a0237740 20 client.26254 trim_cache size 0 max 16384 2024-04-24T06:48:35.164 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:48:35.164 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-24T06:48:35.165 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-24T06:48:35.165 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:48:35.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.156+0000 7f319ca00640 1 -- 172.21.15.5:0/3343565165 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f317c0a3d20 msgr2=0x7f317c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:35.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.156+0000 7f319ca00640 1 --2- 172.21.15.5:0/3343565165 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f317c0a3d20 0x7f317c0a61e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3190006010 tx=0x7f3190005d80 comp rx=0 tx=0).stop 2024-04-24T06:48:35.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.156+0000 7f319ca00640 1 -- 172.21.15.5:0/3343565165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3198104250 msgr2=0x7f31981a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:35.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.156+0000 7f319ca00640 1 --2- 172.21.15.5:0/3343565165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3198104250 0x7f31981a0640 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f318800ea60 tx=0x7f318800ef20 comp rx=0 tx=0).stop 2024-04-24T06:48:35.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.156+0000 7f319ca00640 1 -- 172.21.15.5:0/3343565165 shutdown_connections 2024-04-24T06:48:35.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.156+0000 7f319ca00640 1 --2- 172.21.15.5:0/3343565165 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f317c0a3d20 0x7f317c0a61e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.156+0000 7f319ca00640 1 --2- 172.21.15.5:0/3343565165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f319819a7b0 0x7f319819ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.156+0000 7f319ca00640 1 --2- 172.21.15.5:0/3343565165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3198109320 0x7f31981a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.156+0000 7f319ca00640 1 --2- 172.21.15.5:0/3343565165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3198104250 0x7f31981a0640 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.156+0000 7f319ca00640 1 -- 172.21.15.5:0/3343565165 >> 172.21.15.5:0/3343565165 conn(0x7f31980ffad0 msgr2=0x7f3198101dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:35.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.156+0000 7f319ca00640 1 -- 172.21.15.5:0/3343565165 shutdown_connections 2024-04-24T06:48:35.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.156+0000 7f319ca00640 1 -- 172.21.15.5:0/3343565165 wait complete. 2024-04-24T06:48:35.175 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T06:48:35.175 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:48:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.388+0000 7f5db3800640 1 -- 172.21.15.5:0/1669941305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5dac103230 msgr2=0x7f5dac103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:35.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.388+0000 7f5db3800640 1 --2- 172.21.15.5:0/1669941305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5dac103230 0x7f5dac103610 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c009900 tx=0x7f5d9c02a480 comp rx=0 tx=0).stop 2024-04-24T06:48:35.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.388+0000 7f5db3800640 1 -- 172.21.15.5:0/1669941305 shutdown_connections 2024-04-24T06:48:35.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.388+0000 7f5db3800640 1 --2- 172.21.15.5:0/1669941305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5dac10c600 0x7f5dac10ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.388+0000 7f5db3800640 1 --2- 172.21.15.5:0/1669941305 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dac103b50 0x7f5dac10c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.388+0000 7f5db3800640 1 --2- 172.21.15.5:0/1669941305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5dac103230 0x7f5dac103610 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.388+0000 7f5db3800640 1 -- 172.21.15.5:0/1669941305 >> 172.21.15.5:0/1669941305 conn(0x7f5dac0780e0 msgr2=0x7f5dac0784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:35.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.388+0000 7f5db3800640 1 -- 172.21.15.5:0/1669941305 shutdown_connections 2024-04-24T06:48:35.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.388+0000 7f5db3800640 1 -- 172.21.15.5:0/1669941305 wait complete. 2024-04-24T06:48:35.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.389+0000 7f5db3800640 1 Processor -- start 2024-04-24T06:48:35.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.390+0000 7f5db3800640 1 -- start start 2024-04-24T06:48:35.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.390+0000 7f5db3800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5dac103230 0x7f5dac1a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:35.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.390+0000 7f5db3800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dac103b50 0x7f5dac1a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:35.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.390+0000 7f5db3800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5dac10c600 0x7f5dac19a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:35.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.390+0000 7f5db3800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac112390 con 0x7f5dac103230 2024-04-24T06:48:35.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.390+0000 7f5db3800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac112210 con 0x7f5dac103b50 2024-04-24T06:48:35.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.390+0000 7f5db3800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac112510 con 0x7f5dac10c600 2024-04-24T06:48:35.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.390+0000 7f5db0e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5dac103230 0x7f5dac1a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:35.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.390+0000 7f5db0e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5dac103230 0x7f5dac1a06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55962/0 (socket says 172.21.15.5:55962) 2024-04-24T06:48:35.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.390+0000 7f5db0e00640 1 -- 172.21.15.5:0/1024863854 learned_addr learned my addr 172.21.15.5:0/1024863854 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:35.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.390+0000 7f5da3e00640 1 --2- 172.21.15.5:0/1024863854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dac103b50 0x7f5dac1a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:35.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.390+0000 7f5db1800640 1 --2- 172.21.15.5:0/1024863854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5dac10c600 0x7f5dac19a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:35.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.391+0000 7f5da3e00640 1 -- 172.21.15.5:0/1024863854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5dac10c600 msgr2=0x7f5dac19a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:35.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.391+0000 7f5da3e00640 1 --2- 172.21.15.5:0/1024863854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5dac10c600 0x7f5dac19a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.391+0000 7f5da3e00640 1 -- 172.21.15.5:0/1024863854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5dac103230 msgr2=0x7f5dac1a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:35.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.391+0000 7f5da3e00640 1 --2- 172.21.15.5:0/1024863854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5dac103230 0x7f5dac1a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.391+0000 7f5da3e00640 1 -- 172.21.15.5:0/1024863854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dac19b010 con 0x7f5dac103b50 2024-04-24T06:48:35.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.391+0000 7f5db0e00640 1 --2- 172.21.15.5:0/1024863854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5dac103230 0x7f5dac1a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:48:35.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.391+0000 7f5da3e00640 1 --2- 172.21.15.5:0/1024863854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dac103b50 0x7f5dac1a0c20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400e9f0 tx=0x7f5d9400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:35.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.391+0000 7f5daa000640 1 -- 172.21.15.5:0/1024863854 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9400cd80 con 0x7f5dac103b50 2024-04-24T06:48:35.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.391+0000 7f5db3800640 1 -- 172.21.15.5:0/1024863854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dac19b300 con 0x7f5dac103b50 2024-04-24T06:48:35.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.391+0000 7f5daa000640 1 -- 172.21.15.5:0/1024863854 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d940035f0 con 0x7f5dac103b50 2024-04-24T06:48:35.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.391+0000 7f5daa000640 1 -- 172.21.15.5:0/1024863854 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d94010670 con 0x7f5dac103b50 2024-04-24T06:48:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.391+0000 7f5db3800640 1 -- 172.21.15.5:0/1024863854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dac1a75a0 con 0x7f5dac103b50 2024-04-24T06:48:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.393+0000 7f5db3800640 1 -- 172.21.15.5:0/1024863854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dac06af70 con 0x7f5dac103b50 2024-04-24T06:48:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.395+0000 7f5daa000640 1 -- 172.21.15.5:0/1024863854 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f5d940037a0 con 0x7f5dac103b50 2024-04-24T06:48:35.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.396+0000 7f5daa000640 1 --2- 172.21.15.5:0/1024863854 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f5d8c0a3da0 0x7f5d8c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:35.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.396+0000 7f5daa000640 1 -- 172.21.15.5:0/1024863854 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f5d940d0a80 con 0x7f5dac103b50 2024-04-24T06:48:35.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.396+0000 7f5db0e00640 1 --2- 172.21.15.5:0/1024863854 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f5d8c0a3da0 0x7f5d8c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:35.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.397+0000 7f5db0e00640 1 --2- 172.21.15.5:0/1024863854 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f5d8c0a3da0 0x7f5d8c0a6260 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c031550 tx=0x7f5d9c005a20 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:35.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.401+0000 7f5daa000640 1 -- 172.21.15.5:0/1024863854 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5d940d2e20 con 0x7f5dac103b50 2024-04-24T06:48:35.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.654+0000 7f5db3800640 1 -- 172.21.15.5:0/1024863854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5dac02d620 con 0x7f5dac103b50 2024-04-24T06:48:35.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.655+0000 7f5daa000640 1 -- 172.21.15.5:0/1024863854 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11859 (secure 0 0 0) 0x7f5d9409d0d0 con 0x7f5dac103b50 2024-04-24T06:48:35.656 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:48:35.657 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1735462750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1735462750},{"type":"v1","addr":"172.21.15.5:6837","nonce":1735462750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1499469370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1499469370},{"type":"v1","addr":"172.21.15.5:6841","nonce":1499469370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3705638847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3705638847},{"type":"v1","addr":"172.21.15.37:6835","nonce":3705638847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T06:41:34.764490+0000","modified":"2024-04-24T06:46:02.765290+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24550,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6837/1390103296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1390103296},{"type":"v1","addr":"172.21.15.37:6837","nonce":1390103296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3217754215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3217754215},{"type":"v1","addr":"172.21.15.137:6835","nonce":3217754215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6837/3044222976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3044222976},{"type":"v1","addr":"172.21.15.137:6837","nonce":3044222976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.5:6839/3173201459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3173201459},{"type":"v1","addr":"172.21.15.5:6839","nonce":3173201459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6839/1889774232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1889774232},{"type":"v1","addr":"172.21.15.137:6839","nonce":1889774232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6839/3536721753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":3536721753},{"type":"v1","addr":"172.21.15.37:6839","nonce":3536721753}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24539,24550,24569]},"id":1}]} 2024-04-24T06:48:35.657 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-04-24T06:48:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.658+0000 7f5db3800640 1 -- 172.21.15.5:0/1024863854 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f5d8c0a3da0 msgr2=0x7f5d8c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.658+0000 7f5db3800640 1 --2- 172.21.15.5:0/1024863854 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f5d8c0a3da0 0x7f5d8c0a6260 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c031550 tx=0x7f5d9c005a20 comp rx=0 tx=0).stop 2024-04-24T06:48:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.658+0000 7f5db3800640 1 -- 172.21.15.5:0/1024863854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dac103b50 msgr2=0x7f5dac1a0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.658+0000 7f5db3800640 1 --2- 172.21.15.5:0/1024863854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dac103b50 0x7f5dac1a0c20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400e9f0 tx=0x7f5d9400eeb0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.658+0000 7f5db3800640 1 -- 172.21.15.5:0/1024863854 shutdown_connections 2024-04-24T06:48:35.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.658+0000 7f5db3800640 1 --2- 172.21.15.5:0/1024863854 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f5d8c0a3da0 0x7f5d8c0a6260 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.658+0000 7f5db3800640 1 --2- 172.21.15.5:0/1024863854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5dac10c600 0x7f5dac19a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.658+0000 7f5db3800640 1 --2- 172.21.15.5:0/1024863854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dac103b50 0x7f5dac1a0c20 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.658+0000 7f5db3800640 1 --2- 172.21.15.5:0/1024863854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5dac103230 0x7f5dac1a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.658+0000 7f5db3800640 1 -- 172.21.15.5:0/1024863854 >> 172.21.15.5:0/1024863854 conn(0x7f5dac0780e0 msgr2=0x7f5dac102220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:35.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.658+0000 7f5db3800640 1 -- 172.21.15.5:0/1024863854 shutdown_connections 2024-04-24T06:48:35.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.658+0000 7f5db3800640 1 -- 172.21.15.5:0/1024863854 wait complete. 2024-04-24T06:48:35.679 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T06:48:35.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.888+0000 7f2bbbe00640 1 -- 172.21.15.5:0/3946788090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bbc109320 msgr2=0x7f2bbc110ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:35.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.888+0000 7f2bbbe00640 1 --2- 172.21.15.5:0/3946788090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bbc109320 0x7f2bbc110ef0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4009ca0 tx=0x7f2bb402a8b0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.889+0000 7f2bbbe00640 1 -- 172.21.15.5:0/3946788090 shutdown_connections 2024-04-24T06:48:35.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.889+0000 7f2bbbe00640 1 --2- 172.21.15.5:0/3946788090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bbc109320 0x7f2bbc110ef0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.889+0000 7f2bbbe00640 1 --2- 172.21.15.5:0/3946788090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2bbc104c00 0x7f2bbc108c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.889+0000 7f2bbbe00640 1 --2- 172.21.15.5:0/3946788090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bbc104250 0x7f2bbc104630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.889+0000 7f2bbbe00640 1 -- 172.21.15.5:0/3946788090 >> 172.21.15.5:0/3946788090 conn(0x7f2bbc0ffad0 msgr2=0x7f2bbc101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:35.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.889+0000 7f2bbbe00640 1 -- 172.21.15.5:0/3946788090 shutdown_connections 2024-04-24T06:48:35.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.889+0000 7f2bbbe00640 1 -- 172.21.15.5:0/3946788090 wait complete. 2024-04-24T06:48:35.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.890+0000 7f2bbbe00640 1 Processor -- start 2024-04-24T06:48:35.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.890+0000 7f2bbbe00640 1 -- start start 2024-04-24T06:48:35.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.890+0000 7f2bbbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bbc104250 0x7f2bbc1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:35.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.890+0000 7f2bbbe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2bbc109320 0x7f2bbc1a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:35.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.890+0000 7f2bbbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bbc19a7b0 0x7f2bbc19ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:35.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.890+0000 7f2bbbe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc114430 con 0x7f2bbc104250 2024-04-24T06:48:35.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.890+0000 7f2bbbe00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc1142b0 con 0x7f2bbc19a7b0 2024-04-24T06:48:35.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.890+0000 7f2bbbe00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc1145b0 con 0x7f2bbc109320 2024-04-24T06:48:35.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.891+0000 7f2bbaa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bbc104250 0x7f2bbc1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:35.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.891+0000 7f2bbaa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bbc104250 0x7f2bbc1a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55980/0 (socket says 172.21.15.5:55980) 2024-04-24T06:48:35.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.891+0000 7f2bbaa00640 1 -- 172.21.15.5:0/2867163884 learned_addr learned my addr 172.21.15.5:0/2867163884 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:35.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.891+0000 7f2bba000640 1 --2- 172.21.15.5:0/2867163884 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2bbc109320 0x7f2bbc1a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:35.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.891+0000 7f2bbb400640 1 --2- 172.21.15.5:0/2867163884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bbc19a7b0 0x7f2bbc19ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:35.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.891+0000 7f2bbaa00640 1 -- 172.21.15.5:0/2867163884 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2bbc109320 msgr2=0x7f2bbc1a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:35.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.891+0000 7f2bbaa00640 1 --2- 172.21.15.5:0/2867163884 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2bbc109320 0x7f2bbc1a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.891+0000 7f2bbaa00640 1 -- 172.21.15.5:0/2867163884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bbc19a7b0 msgr2=0x7f2bbc19ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:35.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.891+0000 7f2bbaa00640 1 --2- 172.21.15.5:0/2867163884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bbc19a7b0 0x7f2bbc19ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.891+0000 7f2bbaa00640 1 -- 172.21.15.5:0/2867163884 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bbc19b340 con 0x7f2bbc104250 2024-04-24T06:48:35.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.891+0000 7f2bbb400640 1 --2- 172.21.15.5:0/2867163884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bbc19a7b0 0x7f2bbc19ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:48:35.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.892+0000 7f2bbaa00640 1 --2- 172.21.15.5:0/2867163884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bbc104250 0x7f2bbc1a0640 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2bac00e9f0 tx=0x7f2bac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:35.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.892+0000 7f2bb3400640 1 -- 172.21.15.5:0/2867163884 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bac00cd80 con 0x7f2bbc104250 2024-04-24T06:48:35.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.892+0000 7f2bb3400640 1 -- 172.21.15.5:0/2867163884 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bac010910 con 0x7f2bbc104250 2024-04-24T06:48:35.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.892+0000 7f2bbbe00640 1 -- 172.21.15.5:0/2867163884 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bbc19b630 con 0x7f2bbc104250 2024-04-24T06:48:35.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.892+0000 7f2bb3400640 1 -- 172.21.15.5:0/2867163884 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bac018990 con 0x7f2bbc104250 2024-04-24T06:48:35.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.892+0000 7f2bbbe00640 1 -- 172.21.15.5:0/2867163884 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bbc1a7810 con 0x7f2bbc104250 2024-04-24T06:48:35.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.896+0000 7f2bc4596740 1 Processor -- start 2024-04-24T06:48:35.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.897+0000 7f2bc4596740 1 -- start start 2024-04-24T06:48:35.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.897+0000 7f2bb3400640 1 -- 172.21.15.5:0/2867163884 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f2bac010ac0 con 0x7f2bbc104250 2024-04-24T06:48:35.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.897+0000 7f2bc4596740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56438a409b40 0x56438a409f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:35.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.897+0000 7f2bc4596740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56438a40a460 0x56438a4145f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:35.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.897+0000 7f2bc4596740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a414b30 0x56438a416f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:35.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.897+0000 7f2bc4596740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56438a36f1d0 con 0x56438a409b40 2024-04-24T06:48:35.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.897+0000 7f2bc4596740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56438a315450 con 0x56438a414b30 2024-04-24T06:48:35.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.897+0000 7f2bc4596740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56438a36b190 con 0x56438a40a460 2024-04-24T06:48:35.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.897+0000 7f2bbb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56438a409b40 0x56438a409f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:35.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.897+0000 7f2bbb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56438a409b40 0x56438a409f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55988/0 (socket says 172.21.15.5:55988) 2024-04-24T06:48:35.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.898+0000 7f2bbb400640 1 -- 172.21.15.5:0/1083245996 learned_addr learned my addr 172.21.15.5:0/1083245996 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:35.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.897+0000 7f2bbaa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56438a40a460 0x56438a4145f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:35.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.898+0000 7f2bba000640 1 --2- 172.21.15.5:0/1083245996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a414b30 0x56438a416f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:35.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.898+0000 7f2bbb400640 1 -- 172.21.15.5:0/1083245996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56438a40a460 msgr2=0x56438a4145f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:35.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.898+0000 7f2bbb400640 1 --2- 172.21.15.5:0/1083245996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56438a40a460 0x56438a4145f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.898+0000 7f2bbb400640 1 -- 172.21.15.5:0/1083245996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a414b30 msgr2=0x56438a416f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:35.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.898+0000 7f2bbb400640 1 --2- 172.21.15.5:0/1083245996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a414b30 0x56438a416f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.898+0000 7f2bbb400640 1 -- 172.21.15.5:0/1083245996 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56438a3d2310 con 0x56438a409b40 2024-04-24T06:48:35.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.898+0000 7f2bba000640 1 --2- 172.21.15.5:0/1083245996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a414b30 0x56438a416f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:48:35.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.898+0000 7f2bbb400640 1 --2- 172.21.15.5:0/1083245996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56438a409b40 0x56438a409f20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4035000 tx=0x7f2bb4009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbc1bd2389cd3841 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:35.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.898+0000 7f2bb3400640 1 --2- 172.21.15.5:0/2867163884 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f2ba00a3d20 0x7f2ba00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:35.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.899+0000 7f2ba7e00640 1 -- 172.21.15.5:0/1083245996 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb4030750 con 0x56438a409b40 2024-04-24T06:48:35.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.899+0000 7f2ba7e00640 1 -- 172.21.15.5:0/1083245996 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bb4040070 con 0x56438a409b40 2024-04-24T06:48:35.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.899+0000 7f2bb3400640 1 -- 172.21.15.5:0/2867163884 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f2bac0d0c30 con 0x7f2bbc104250 2024-04-24T06:48:35.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.899+0000 7f2ba7e00640 1 -- 172.21.15.5:0/1083245996 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb4004460 con 0x56438a409b40 2024-04-24T06:48:35.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.899+0000 7f2bbaa00640 1 --2- 172.21.15.5:0/2867163884 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f2ba00a3d20 0x7f2ba00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:35.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.899+0000 7f2bc4596740 1 -- 172.21.15.5:0/1083245996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56438a409b40 msgr2=0x56438a409f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:35.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.899+0000 7f2bc4596740 1 --2- 172.21.15.5:0/1083245996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56438a409b40 0x56438a409f20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4035000 tx=0x7f2bb4009a20 comp rx=0 tx=0).stop 2024-04-24T06:48:35.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.899+0000 7f2bbaa00640 1 --2- 172.21.15.5:0/2867163884 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f2ba00a3d20 0x7f2ba00a61e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x56438a40b100 tx=0x7f2bac0d46e0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:35.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.900+0000 7f2bc4596740 1 -- 172.21.15.5:0/1083245996 shutdown_connections 2024-04-24T06:48:35.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.900+0000 7f2bc4596740 1 --2- 172.21.15.5:0/1083245996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a414b30 0x56438a416f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.900+0000 7f2bc4596740 1 --2- 172.21.15.5:0/1083245996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56438a40a460 0x56438a4145f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.900+0000 7f2bc4596740 1 --2- 172.21.15.5:0/1083245996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56438a409b40 0x56438a409f20 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.900+0000 7f2bc4596740 1 -- 172.21.15.5:0/1083245996 >> 172.21.15.5:0/1083245996 conn(0x56438a405e60 msgr2=0x56438a406240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:35.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.900+0000 7f2bc4596740 1 -- 172.21.15.5:0/1083245996 shutdown_connections 2024-04-24T06:48:35.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.900+0000 7f2bc4596740 1 -- 172.21.15.5:0/1083245996 wait complete. 2024-04-24T06:48:35.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.900+0000 7f2bc4596740 1 -- start start 2024-04-24T06:48:35.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.900+0000 7f2bc4596740 1 Processor -- start 2024-04-24T06:48:35.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.901+0000 7f2bc4596740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56438a409b40 0x56438a4145c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:35.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.901+0000 7f2bc4596740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56438a40a460 0x56438a411920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:35.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.901+0000 7f2bc4596740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a414b30 0x56438a412010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:35.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.901+0000 7f2bc4596740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56438a417960 con 0x56438a40a460 2024-04-24T06:48:35.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.901+0000 7f2bc4596740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56438a328810 con 0x56438a414b30 2024-04-24T06:48:35.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.901+0000 7f2bc4596740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56438a417ae0 con 0x56438a409b40 2024-04-24T06:48:35.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.901+0000 7f2bc4596740 10 client.0 _ll_register_callbacks cb 0x56438a2f6140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:48:35.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.901+0000 7f2bba000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56438a40a460 0x56438a411920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:35.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.901+0000 7f2bba000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56438a40a460 0x56438a411920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55996/0 (socket says 172.21.15.5:55996) 2024-04-24T06:48:35.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.901+0000 7f2bba000640 1 -- 172.21.15.5:0/2262609262 learned_addr learned my addr 172.21.15.5:0/2262609262 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:35.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.901+0000 7f2bbb400640 1 --2- 172.21.15.5:0/2262609262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a414b30 0x56438a412010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:35.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.902+0000 7f2bbb400640 1 --2- 172.21.15.5:0/2262609262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56438a409b40 0x56438a4145c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:35.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.902+0000 7f2bba000640 1 -- 172.21.15.5:0/2262609262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56438a409b40 msgr2=0x56438a4145c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:35.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.902+0000 7f2bba000640 1 --2- 172.21.15.5:0/2262609262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56438a409b40 0x56438a4145c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.902+0000 7f2bba000640 1 -- 172.21.15.5:0/2262609262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a414b30 msgr2=0x56438a412010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:35.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.902+0000 7f2bba000640 1 --2- 172.21.15.5:0/2262609262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a414b30 0x56438a412010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.902+0000 7f2bba000640 1 -- 172.21.15.5:0/2262609262 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56438a3393f0 con 0x56438a40a460 2024-04-24T06:48:35.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.902+0000 7f2bbb400640 1 --2- 172.21.15.5:0/2262609262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a414b30 0x56438a412010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:48:35.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.902+0000 7f2bba000640 1 --2- 172.21.15.5:0/2262609262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56438a40a460 0x56438a411920 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8012700 tx=0x7f2ba8012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b541ee2723895e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:35.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.902+0000 7f2ba4c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:48:35.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.902+0000 7f2ba4c00640 1 -- 172.21.15.5:0/2262609262 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba800f920 con 0x56438a40a460 2024-04-24T06:48:35.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.903+0000 7f2ba4c00640 1 -- 172.21.15.5:0/2262609262 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f2ba800fac0 con 0x56438a40a460 2024-04-24T06:48:35.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.903+0000 7f2bc4596740 1 -- 172.21.15.5:0/2262609262 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bb4040070 con 0x56438a40a460 2024-04-24T06:48:35.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.903+0000 7f2ba4c00640 1 -- 172.21.15.5:0/2262609262 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2ba800fcc0 con 0x56438a40a460 2024-04-24T06:48:35.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.903+0000 7f2bc4596740 10 client.16122 fetch_fsmap learned FSMap version 25 2024-04-24T06:48:35.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.903+0000 7f2bc4596740 1 -- 172.21.15.5:0/2262609262 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56438a0813f0 con 0x56438a40a460 2024-04-24T06:48:35.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.903+0000 7f2ba4c00640 1 -- 172.21.15.5:0/2262609262 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bb4040070 con 0x56438a40a460 2024-04-24T06:48:35.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.903+0000 7f2ba4c00640 1 -- 172.21.15.5:0/2262609262 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2ba80104c0 con 0x56438a40a460 2024-04-24T06:48:35.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.903+0000 7f2bc4596740 10 client.16122 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:48:35.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.903+0000 7f2bc4596740 10 client.16122 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-24T06:48:35.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.903+0000 7f2bc4596740 20 client.16122 populate_metadata read hostname 'smithi005' 2024-04-24T06:48:35.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.903+0000 7f2bc4596740 1 --2- 172.21.15.5:0/2262609262 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x56438a424e00 0x56438a427220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:35.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.904+0000 7f2bc4596740 4 client.16122 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:48:35.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.904+0000 7f2bc4596740 1 -- 172.21.15.5:0/2262609262 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56438a37cbf0 con 0x56438a424e00 2024-04-24T06:48:35.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.904+0000 7f2bbb400640 1 --2- 172.21.15.5:0/2262609262 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x56438a424e00 0x56438a427220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:35.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.904+0000 7f2bbb400640 1 --2- 172.21.15.5:0/2262609262 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x56438a424e00 0x56438a427220 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=19e892d7005ed360 server_cookie=9ac80bca9bdeb3fe in_seq=0 out_seq=0 2024-04-24T06:48:35.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.904+0000 7f2ba4c00640 10 client.16122 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:48:35.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.905+0000 7f2ba4c00640 1 -- 172.21.15.5:0/2262609262 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56438a37cbf0 con 0x56438a424e00 2024-04-24T06:48:35.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.905+0000 7f2ba4c00640 10 client.16122 handle_command_reply: tid=0 2024-04-24T06:48:35.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.905+0000 7f2bc4596740 1 client.16122 shutdown 2024-04-24T06:48:35.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.905+0000 7f2bc4596740 1 -- 172.21.15.5:0/2262609262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56438a40a460 msgr2=0x56438a411920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:35.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.905+0000 7f2bc4596740 1 --2- 172.21.15.5:0/2262609262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56438a40a460 0x56438a411920 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8012700 tx=0x7f2ba8012bc0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.905+0000 7f2bc4596740 1 -- 172.21.15.5:0/2262609262 shutdown_connections 2024-04-24T06:48:35.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.905+0000 7f2bc4596740 1 --2- 172.21.15.5:0/2262609262 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x56438a424e00 0x56438a427220 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.905+0000 7f2bc4596740 1 --2- 172.21.15.5:0/2262609262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a414b30 0x56438a412010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.905+0000 7f2bc4596740 1 --2- 172.21.15.5:0/2262609262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56438a40a460 0x56438a411920 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.905+0000 7f2bc4596740 1 --2- 172.21.15.5:0/2262609262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56438a409b40 0x56438a4145c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.905+0000 7f2ba4c00640 0 client.16122 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:48:35.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.905+0000 7f2bc4596740 1 -- 172.21.15.5:0/2262609262 >> 172.21.15.5:0/2262609262 conn(0x56438a405e60 msgr2=0x56438a40bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:35.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.906+0000 7f2bc4596740 1 -- 172.21.15.5:0/2262609262 shutdown_connections 2024-04-24T06:48:35.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.906+0000 7f2bc4596740 1 -- 172.21.15.5:0/2262609262 wait complete. 2024-04-24T06:48:35.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.906+0000 7f2bc4596740 20 client.16122 trim_cache size 0 max 16384 2024-04-24T06:48:35.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.921+0000 7f2bc4596740 1 Processor -- start 2024-04-24T06:48:35.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.921+0000 7f2bc4596740 1 -- start start 2024-04-24T06:48:35.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.921+0000 7f2bc4596740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56438a409b40 0x56438a420820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:35.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.921+0000 7f2bc4596740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56438a40a460 0x56438a413860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:35.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.921+0000 7f2bc4596740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a414b30 0x56438a42cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:35.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.921+0000 7f2bc4596740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc1b2750 con 0x56438a40a460 2024-04-24T06:48:35.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.921+0000 7f2bc4596740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc1b0020 con 0x56438a414b30 2024-04-24T06:48:35.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.921+0000 7f2bc4596740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc1ade70 con 0x56438a409b40 2024-04-24T06:48:35.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.921+0000 7f2bba000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56438a40a460 0x56438a413860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:35.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.921+0000 7f2bba000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56438a40a460 0x56438a413860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56008/0 (socket says 172.21.15.5:56008) 2024-04-24T06:48:35.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.921+0000 7f2bba000640 1 -- 172.21.15.5:0/2707172172 learned_addr learned my addr 172.21.15.5:0/2707172172 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:35.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.921+0000 7f2bbb400640 1 --2- 172.21.15.5:0/2707172172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56438a409b40 0x56438a420820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:35.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.922+0000 7f2bbb400640 1 --2- 172.21.15.5:0/2707172172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a414b30 0x56438a42cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:35.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.922+0000 7f2bba000640 1 -- 172.21.15.5:0/2707172172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56438a409b40 msgr2=0x56438a420820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:35.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.922+0000 7f2bba000640 1 --2- 172.21.15.5:0/2707172172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56438a409b40 0x56438a420820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.922+0000 7f2bba000640 1 -- 172.21.15.5:0/2707172172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a414b30 msgr2=0x56438a42cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:35.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.922+0000 7f2bba000640 1 --2- 172.21.15.5:0/2707172172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a414b30 0x56438a42cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.922+0000 7f2bba000640 1 -- 172.21.15.5:0/2707172172 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56438a14b7d0 con 0x56438a40a460 2024-04-24T06:48:35.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.922+0000 7f2bbb400640 1 --2- 172.21.15.5:0/2707172172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56438a409b40 0x56438a420820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:48:35.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.922+0000 7f2bba000640 1 --2- 172.21.15.5:0/2707172172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56438a40a460 0x56438a413860 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2ba800f810 tx=0x7f2ba800f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffaa699f9b5340a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:35.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.922+0000 7f2ba7400640 1 -- 172.21.15.5:0/2707172172 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba8029070 con 0x56438a40a460 2024-04-24T06:48:35.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.922+0000 7f2ba7400640 1 -- 172.21.15.5:0/2707172172 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ba80056f0 con 0x56438a40a460 2024-04-24T06:48:35.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.922+0000 7f2ba7400640 1 -- 172.21.15.5:0/2707172172 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba80035f0 con 0x56438a40a460 2024-04-24T06:48:35.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.923+0000 7f2bc4596740 1 -- 172.21.15.5:0/2707172172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56438a40a460 msgr2=0x56438a413860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:35.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.923+0000 7f2bc4596740 1 --2- 172.21.15.5:0/2707172172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56438a40a460 0x56438a413860 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2ba800f810 tx=0x7f2ba800f840 comp rx=0 tx=0).stop 2024-04-24T06:48:35.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.923+0000 7f2bc4596740 1 -- 172.21.15.5:0/2707172172 shutdown_connections 2024-04-24T06:48:35.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.923+0000 7f2bc4596740 1 --2- 172.21.15.5:0/2707172172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a414b30 0x56438a42cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.923+0000 7f2bc4596740 1 --2- 172.21.15.5:0/2707172172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56438a40a460 0x56438a413860 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.923+0000 7f2bc4596740 1 --2- 172.21.15.5:0/2707172172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56438a409b40 0x56438a420820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.923+0000 7f2bc4596740 1 -- 172.21.15.5:0/2707172172 >> 172.21.15.5:0/2707172172 conn(0x56438a405e60 msgr2=0x56438a413150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:35.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.923+0000 7f2bc4596740 1 -- 172.21.15.5:0/2707172172 shutdown_connections 2024-04-24T06:48:35.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.923+0000 7f2bc4596740 1 -- 172.21.15.5:0/2707172172 wait complete. 2024-04-24T06:48:35.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.924+0000 7f2bc4596740 1 -- start start 2024-04-24T06:48:35.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.924+0000 7f2bc4596740 1 Processor -- start 2024-04-24T06:48:35.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.924+0000 7f2bc4596740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56438a409b40 0x56438a415d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:35.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.924+0000 7f2bc4596740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a40a460 0x56438a416270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:35.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.924+0000 7f2bc4596740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56438a414b30 0x56438a41b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:35.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.924+0000 7f2bc4596740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc1ae4e0 con 0x56438a414b30 2024-04-24T06:48:35.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.924+0000 7f2bc4596740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc1af950 con 0x56438a40a460 2024-04-24T06:48:35.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.924+0000 7f2bc4596740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc1aebb0 con 0x56438a409b40 2024-04-24T06:48:35.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.924+0000 7f2bc4596740 10 client.0 _ll_register_callbacks cb 0x7f2ba8009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:48:35.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.925+0000 7f2bbb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56438a414b30 0x56438a41b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:35.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.925+0000 7f2bba000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a40a460 0x56438a416270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:35.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.925+0000 7f2bba000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a40a460 0x56438a416270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:33838/0 (socket says 172.21.15.5:33838) 2024-04-24T06:48:35.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.925+0000 7f2bbb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56438a414b30 0x56438a41b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56014/0 (socket says 172.21.15.5:56014) 2024-04-24T06:48:35.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.925+0000 7f2bba000640 1 -- 172.21.15.5:0/4283525389 learned_addr learned my addr 172.21.15.5:0/4283525389 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:35.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.925+0000 7f2bbb400640 1 --2- 172.21.15.5:0/4283525389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56438a409b40 0x56438a415d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:35.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.925+0000 7f2bba000640 1 -- 172.21.15.5:0/4283525389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56438a409b40 msgr2=0x56438a415d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:35.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.925+0000 7f2bba000640 1 --2- 172.21.15.5:0/4283525389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56438a409b40 0x56438a415d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.925+0000 7f2bba000640 1 -- 172.21.15.5:0/4283525389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56438a414b30 msgr2=0x56438a41b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:35.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.925+0000 7f2bba000640 1 --2- 172.21.15.5:0/4283525389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56438a414b30 0x56438a41b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.925+0000 7f2bba000640 1 -- 172.21.15.5:0/4283525389 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56438a41b850 con 0x56438a40a460 2024-04-24T06:48:35.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.925+0000 7f2bbb400640 1 --2- 172.21.15.5:0/4283525389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56438a414b30 0x56438a41b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:48:35.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.925+0000 7f2bba000640 1 --2- 172.21.15.5:0/4283525389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a40a460 0x56438a416270 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8012700 tx=0x7f2ba8010cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6f5705e5b754d9e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:35.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.925+0000 7f2ba4c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:48:35.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.926+0000 7f2ba4c00640 1 -- 172.21.15.5:0/4283525389 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba8029070 con 0x56438a40a460 2024-04-24T06:48:35.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.926+0000 7f2bc4596740 1 -- 172.21.15.5:0/4283525389 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ba80056f0 con 0x56438a40a460 2024-04-24T06:48:35.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.926+0000 7f2ba4c00640 1 -- 172.21.15.5:0/4283525389 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f2ba80050c0 con 0x56438a40a460 2024-04-24T06:48:35.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.926+0000 7f2ba4c00640 1 -- 172.21.15.5:0/4283525389 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2ba80052c0 con 0x56438a40a460 2024-04-24T06:48:35.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.926+0000 7f2bc4596740 10 client.26264 fetch_fsmap learned FSMap version 25 2024-04-24T06:48:35.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.926+0000 7f2bc4596740 1 -- 172.21.15.5:0/4283525389 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56438a48ba20 con 0x56438a40a460 2024-04-24T06:48:35.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.926+0000 7f2ba4c00640 1 -- 172.21.15.5:0/4283525389 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ba80056f0 con 0x56438a40a460 2024-04-24T06:48:35.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.927+0000 7f2ba4c00640 1 -- 172.21.15.5:0/4283525389 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2ba803f590 con 0x56438a40a460 2024-04-24T06:48:35.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.927+0000 7f2bc4596740 10 client.26264 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:48:35.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.927+0000 7f2bc4596740 10 client.26264 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-24T06:48:35.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.927+0000 7f2bc4596740 20 client.26264 populate_metadata read hostname 'smithi005' 2024-04-24T06:48:35.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.927+0000 7f2bc4596740 1 --2- 172.21.15.5:0/4283525389 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x56438a48c090 0x56438a48e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:35.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.927+0000 7f2bc4596740 4 client.26264 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:48:35.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.927+0000 7f2bc4596740 1 -- 172.21.15.5:0/4283525389 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56438a376a60 con 0x56438a48c090 2024-04-24T06:48:35.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.927+0000 7f2bbb400640 1 --2- 172.21.15.5:0/4283525389 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x56438a48c090 0x56438a48e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:35.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.927+0000 7f2bbb400640 1 --2- 172.21.15.5:0/4283525389 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x56438a48c090 0x56438a48e4b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cec69f7169048fe6 server_cookie=b11263e51871d204 in_seq=0 out_seq=0 2024-04-24T06:48:35.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.927+0000 7f2ba4c00640 10 client.26264 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:48:35.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.928+0000 7f2ba4c00640 1 -- 172.21.15.5:0/4283525389 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56438a376a60 con 0x56438a48c090 2024-04-24T06:48:35.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.928+0000 7f2ba4c00640 10 client.26264 handle_command_reply: tid=0 2024-04-24T06:48:35.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.928+0000 7f2bc4596740 1 client.26264 shutdown 2024-04-24T06:48:35.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.928+0000 7f2bc4596740 1 -- 172.21.15.5:0/4283525389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a40a460 msgr2=0x56438a416270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:35.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.928+0000 7f2bc4596740 1 --2- 172.21.15.5:0/4283525389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a40a460 0x56438a416270 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8012700 tx=0x7f2ba8010cd0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.928+0000 7f2bc4596740 1 -- 172.21.15.5:0/4283525389 shutdown_connections 2024-04-24T06:48:35.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.928+0000 7f2bc4596740 1 --2- 172.21.15.5:0/4283525389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56438a414b30 0x56438a41b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.928+0000 7f2bc4596740 1 --2- 172.21.15.5:0/4283525389 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x56438a48c090 0x56438a48e4b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.928+0000 7f2bc4596740 1 --2- 172.21.15.5:0/4283525389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56438a40a460 0x56438a416270 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.928+0000 7f2bc4596740 1 --2- 172.21.15.5:0/4283525389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56438a409b40 0x56438a415d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.928+0000 7f2bc4596740 1 -- 172.21.15.5:0/4283525389 >> 172.21.15.5:0/4283525389 conn(0x56438a405e60 msgr2=0x56438a4139e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:35.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.928+0000 7f2ba4c00640 0 client.26264 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:48:35.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.928+0000 7f2bc4596740 1 -- 172.21.15.5:0/4283525389 shutdown_connections 2024-04-24T06:48:35.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.928+0000 7f2bc4596740 1 -- 172.21.15.5:0/4283525389 wait complete. 2024-04-24T06:48:35.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.928+0000 7f2bc4596740 20 client.26264 trim_cache size 0 max 16384 2024-04-24T06:48:35.939 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:48:35.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.929+0000 7f2bbbe00640 1 -- 172.21.15.5:0/2867163884 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f2ba00a3d20 msgr2=0x7f2ba00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:35.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.929+0000 7f2bbbe00640 1 --2- 172.21.15.5:0/2867163884 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f2ba00a3d20 0x7f2ba00a61e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x56438a40b100 tx=0x7f2bac0d46e0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.929+0000 7f2bbbe00640 1 -- 172.21.15.5:0/2867163884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bbc104250 msgr2=0x7f2bbc1a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:35.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.929+0000 7f2bbbe00640 1 --2- 172.21.15.5:0/2867163884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bbc104250 0x7f2bbc1a0640 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2bac00e9f0 tx=0x7f2bac00eeb0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.929+0000 7f2bbbe00640 1 -- 172.21.15.5:0/2867163884 shutdown_connections 2024-04-24T06:48:35.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.929+0000 7f2bbbe00640 1 --2- 172.21.15.5:0/2867163884 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f2ba00a3d20 0x7f2ba00a61e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.929+0000 7f2bbbe00640 1 --2- 172.21.15.5:0/2867163884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bbc19a7b0 0x7f2bbc19ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.929+0000 7f2bbbe00640 1 --2- 172.21.15.5:0/2867163884 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2bbc109320 0x7f2bbc1a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.929+0000 7f2bbbe00640 1 --2- 172.21.15.5:0/2867163884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bbc104250 0x7f2bbc1a0640 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:35.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.929+0000 7f2bbbe00640 1 -- 172.21.15.5:0/2867163884 >> 172.21.15.5:0/2867163884 conn(0x7f2bbc0ffad0 msgr2=0x7f2bbc101dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:35.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.929+0000 7f2bbbe00640 1 -- 172.21.15.5:0/2867163884 shutdown_connections 2024-04-24T06:48:35.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:35.929+0000 7f2bbbe00640 1 -- 172.21.15.5:0/2867163884 wait complete. 2024-04-24T06:48:35.945 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:35 smithi137 ceph-mon[33017]: pgmap v139: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 7 op/s 2024-04-24T06:48:35.946 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:35 smithi005 ceph-mon[27798]: pgmap v139: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 7 op/s 2024-04-24T06:48:35.948 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T06:48:36.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:35 smithi037 ceph-mon[32864]: pgmap v139: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 7 op/s 2024-04-24T06:48:36.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.157+0000 7f37ecc00640 1 -- 172.21.15.5:0/4071449070 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37e8109330 msgr2=0x7f37e8110f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:36.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.157+0000 7f37ecc00640 1 --2- 172.21.15.5:0/4071449070 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37e8109330 0x7f37e8110f00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f37e0009be0 tx=0x7f37e002a6d0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.158+0000 7f37ecc00640 1 -- 172.21.15.5:0/4071449070 shutdown_connections 2024-04-24T06:48:36.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.158+0000 7f37ecc00640 1 --2- 172.21.15.5:0/4071449070 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37e8109330 0x7f37e8110f00 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.158+0000 7f37ecc00640 1 --2- 172.21.15.5:0/4071449070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e8104c10 0x7f37e8108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.158+0000 7f37ecc00640 1 --2- 172.21.15.5:0/4071449070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37e8104260 0x7f37e8104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.158+0000 7f37ecc00640 1 -- 172.21.15.5:0/4071449070 >> 172.21.15.5:0/4071449070 conn(0x7f37e80ffaa0 msgr2=0x7f37e8101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:36.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.158+0000 7f37ecc00640 1 -- 172.21.15.5:0/4071449070 shutdown_connections 2024-04-24T06:48:36.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.158+0000 7f37ecc00640 1 -- 172.21.15.5:0/4071449070 wait complete. 2024-04-24T06:48:36.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.159+0000 7f37ecc00640 1 Processor -- start 2024-04-24T06:48:36.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.159+0000 7f37ecc00640 1 -- start start 2024-04-24T06:48:36.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.160+0000 7f37ecc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e8104260 0x7f37e81a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:36.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.160+0000 7f37ecc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37e8109330 0x7f37e81a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:36.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.160+0000 7f37ecc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37e819a710 0x7f37e819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:36.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.160+0000 7f37ecc00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f37e81143f0 con 0x7f37e819a710 2024-04-24T06:48:36.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.160+0000 7f37ecc00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f37e8114270 con 0x7f37e8104260 2024-04-24T06:48:36.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.160+0000 7f37ecc00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f37e8114570 con 0x7f37e8109330 2024-04-24T06:48:36.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.160+0000 7f37df400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e8104260 0x7f37e81a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:36.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.160+0000 7f37e7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37e819a710 0x7f37e819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:36.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.160+0000 7f37df400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e8104260 0x7f37e81a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:33864/0 (socket says 172.21.15.5:33864) 2024-04-24T06:48:36.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.160+0000 7f37df400640 1 -- 172.21.15.5:0/3066189983 learned_addr learned my addr 172.21.15.5:0/3066189983 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:36.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.160+0000 7f37e7400640 1 --2- 172.21.15.5:0/3066189983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37e8109330 0x7f37e81a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:36.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.161+0000 7f37df400640 1 -- 172.21.15.5:0/3066189983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37e8109330 msgr2=0x7f37e81a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:36.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.161+0000 7f37df400640 1 --2- 172.21.15.5:0/3066189983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37e8109330 0x7f37e81a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.161+0000 7f37df400640 1 -- 172.21.15.5:0/3066189983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37e819a710 msgr2=0x7f37e819ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:36.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.161+0000 7f37df400640 1 --2- 172.21.15.5:0/3066189983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37e819a710 0x7f37e819ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.161+0000 7f37df400640 1 -- 172.21.15.5:0/3066189983 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e819b310 con 0x7f37e8104260 2024-04-24T06:48:36.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.161+0000 7f37df400640 1 --2- 172.21.15.5:0/3066189983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e8104260 0x7f37e81a0650 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f37d800e9c0 tx=0x7f37d800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:36.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.161+0000 7f37e4c00640 1 -- 172.21.15.5:0/3066189983 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d800ccc0 con 0x7f37e8104260 2024-04-24T06:48:36.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.162+0000 7f37ecc00640 1 -- 172.21.15.5:0/3066189983 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37e81a73f0 con 0x7f37e8104260 2024-04-24T06:48:36.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.162+0000 7f37e4c00640 1 -- 172.21.15.5:0/3066189983 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37d8005500 con 0x7f37e8104260 2024-04-24T06:48:36.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.162+0000 7f37e4c00640 1 -- 172.21.15.5:0/3066189983 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d8010430 con 0x7f37e8104260 2024-04-24T06:48:36.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.162+0000 7f37ecc00640 1 -- 172.21.15.5:0/3066189983 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37e81a7850 con 0x7f37e8104260 2024-04-24T06:48:36.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.165+0000 7f37f11c5740 1 Processor -- start 2024-04-24T06:48:36.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.165+0000 7f37f11c5740 1 -- start start 2024-04-24T06:48:36.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.165+0000 7f37f11c5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7574ceb40 0x55d7574cef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:36.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.165+0000 7f37f11c5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7574cf460 0x55d7574d95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:36.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.165+0000 7f37f11c5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7574d9b30 0x55d7574dbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:36.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.165+0000 7f37f11c5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d7574341d0 con 0x55d7574d9b30 2024-04-24T06:48:36.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.165+0000 7f37f11c5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d7573da450 con 0x55d7574ceb40 2024-04-24T06:48:36.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.165+0000 7f37f11c5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d757430190 con 0x55d7574cf460 2024-04-24T06:48:36.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.165+0000 7f37e7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7574d9b30 0x55d7574dbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:36.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.165+0000 7f37e4c00640 1 -- 172.21.15.5:0/3066189983 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f37d80046c0 con 0x7f37e8104260 2024-04-24T06:48:36.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.165+0000 7f37e7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7574d9b30 0x55d7574dbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56044/0 (socket says 172.21.15.5:56044) 2024-04-24T06:48:36.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.165+0000 7f37e7400640 1 -- 172.21.15.5:0/2932981930 learned_addr learned my addr 172.21.15.5:0/2932981930 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:36.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.165+0000 7f37e7e00640 1 --2- 172.21.15.5:0/2932981930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7574ceb40 0x55d7574cef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:36.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.166+0000 7f37e7400640 1 -- 172.21.15.5:0/2932981930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7574cf460 msgr2=0x55d7574d95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:36.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.166+0000 7f37df400640 1 --2- 172.21.15.5:0/2932981930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7574cf460 0x55d7574d95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:36.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.166+0000 7f37e7400640 1 --2- 172.21.15.5:0/2932981930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7574cf460 0x55d7574d95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.166+0000 7f37e7400640 1 -- 172.21.15.5:0/2932981930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7574ceb40 msgr2=0x55d7574cef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:36.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.166+0000 7f37e7400640 1 --2- 172.21.15.5:0/2932981930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7574ceb40 0x55d7574cef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.166+0000 7f37e7400640 1 -- 172.21.15.5:0/2932981930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d757497310 con 0x55d7574d9b30 2024-04-24T06:48:36.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.166+0000 7f37df400640 1 --2- 172.21.15.5:0/2932981930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7574cf460 0x55d7574d95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:48:36.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.166+0000 7f37e7e00640 1 --2- 172.21.15.5:0/2932981930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7574ceb40 0x55d7574cef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:48:36.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.166+0000 7f37e7400640 1 --2- 172.21.15.5:0/2932981930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7574d9b30 0x55d7574dbf20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f37d0009c00 tx=0x7f37d0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a559c480bde244 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:36.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.167+0000 7f37d7e00640 1 -- 172.21.15.5:0/2932981930 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d0004230 con 0x55d7574d9b30 2024-04-24T06:48:36.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.167+0000 7f37d7e00640 1 -- 172.21.15.5:0/2932981930 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37d00043d0 con 0x55d7574d9b30 2024-04-24T06:48:36.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.167+0000 7f37d7e00640 1 -- 172.21.15.5:0/2932981930 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d00196d0 con 0x55d7574d9b30 2024-04-24T06:48:36.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.167+0000 7f37e4c00640 1 --2- 172.21.15.5:0/3066189983 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f37cc0a3d20 0x7f37cc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:36.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.167+0000 7f37f11c5740 1 -- 172.21.15.5:0/2932981930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7574d9b30 msgr2=0x55d7574dbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:36.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.167+0000 7f37f11c5740 1 --2- 172.21.15.5:0/2932981930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7574d9b30 0x55d7574dbf20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f37d0009c00 tx=0x7f37d0011810 comp rx=0 tx=0).stop 2024-04-24T06:48:36.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.167+0000 7f37e4c00640 1 -- 172.21.15.5:0/3066189983 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f37d80d0b00 con 0x7f37e8104260 2024-04-24T06:48:36.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.168+0000 7f37f11c5740 1 -- 172.21.15.5:0/2932981930 shutdown_connections 2024-04-24T06:48:36.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.168+0000 7f37f11c5740 1 --2- 172.21.15.5:0/2932981930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7574d9b30 0x55d7574dbf20 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.168+0000 7f37f11c5740 1 --2- 172.21.15.5:0/2932981930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7574cf460 0x55d7574d95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.168+0000 7f37f11c5740 1 --2- 172.21.15.5:0/2932981930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7574ceb40 0x55d7574cef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.168+0000 7f37f11c5740 1 -- 172.21.15.5:0/2932981930 >> 172.21.15.5:0/2932981930 conn(0x55d7574cae60 msgr2=0x55d7574cb240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:36.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.168+0000 7f37e7e00640 1 --2- 172.21.15.5:0/3066189983 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f37cc0a3d20 0x7f37cc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:36.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.168+0000 7f37f11c5740 1 -- 172.21.15.5:0/2932981930 shutdown_connections 2024-04-24T06:48:36.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.168+0000 7f37f11c5740 1 -- 172.21.15.5:0/2932981930 wait complete. 2024-04-24T06:48:36.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.168+0000 7f37f11c5740 1 -- start start 2024-04-24T06:48:36.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.168+0000 7f37f11c5740 1 Processor -- start 2024-04-24T06:48:36.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.168+0000 7f37e7e00640 1 --2- 172.21.15.5:0/3066189983 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f37cc0a3d20 0x7f37cc0a61e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f37e0004af0 tx=0x7f37e0005fd0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:36.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.169+0000 7f37f11c5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7574ceb40 0x55d7574d9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:36.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.169+0000 7f37f11c5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7574cf460 0x55d7574d6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:36.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.169+0000 7f37f11c5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7574d9b30 0x55d7574d6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:36.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.169+0000 7f37f11c5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d7574dc9c0 con 0x55d7574cf460 2024-04-24T06:48:36.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.169+0000 7f37f11c5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d7573ed810 con 0x55d7574ceb40 2024-04-24T06:48:36.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.169+0000 7f37f11c5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d7574dcb40 con 0x55d7574d9b30 2024-04-24T06:48:36.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.169+0000 7f37f11c5740 10 client.0 _ll_register_callbacks cb 0x55d7573bb140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:48:36.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.169+0000 7f37df400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7574ceb40 0x55d7574d9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:36.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.169+0000 7f37e7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7574cf460 0x55d7574d6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:36.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.169+0000 7f37df400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7574ceb40 0x55d7574d9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:33890/0 (socket says 172.21.15.5:33890) 2024-04-24T06:48:36.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.169+0000 7f37df400640 1 -- 172.21.15.5:0/4028208081 learned_addr learned my addr 172.21.15.5:0/4028208081 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:36.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.169+0000 7f37e7e00640 1 --2- 172.21.15.5:0/4028208081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7574d9b30 0x55d7574d6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:36.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.170+0000 7f37df400640 1 -- 172.21.15.5:0/4028208081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7574d9b30 msgr2=0x55d7574d6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:36.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.170+0000 7f37df400640 1 --2- 172.21.15.5:0/4028208081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7574d9b30 0x55d7574d6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.170+0000 7f37df400640 1 -- 172.21.15.5:0/4028208081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7574cf460 msgr2=0x55d7574d6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:36.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.170+0000 7f37df400640 1 --2- 172.21.15.5:0/4028208081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7574cf460 0x55d7574d6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.170+0000 7f37df400640 1 -- 172.21.15.5:0/4028208081 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7573fe3f0 con 0x55d7574ceb40 2024-04-24T06:48:36.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.170+0000 7f37e7e00640 1 --2- 172.21.15.5:0/4028208081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7574d9b30 0x55d7574d6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:48:36.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.170+0000 7f37df400640 1 --2- 172.21.15.5:0/4028208081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7574ceb40 0x55d7574d9570 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f37d80058c0 tx=0x7f37d809c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22baf9a937ce4cc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:36.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.170+0000 7f37d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:48:36.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.170+0000 7f37d4c00640 1 -- 172.21.15.5:0/4028208081 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d80ce690 con 0x55d7574ceb40 2024-04-24T06:48:36.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.170+0000 7f37d4c00640 1 -- 172.21.15.5:0/4028208081 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f37d80a5430 con 0x55d7574ceb40 2024-04-24T06:48:36.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.170+0000 7f37f11c5740 1 -- 172.21.15.5:0/4028208081 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37d00043d0 con 0x55d7574ceb40 2024-04-24T06:48:36.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.171+0000 7f37d4c00640 1 -- 172.21.15.5:0/4028208081 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f37d809ca50 con 0x55d7574ceb40 2024-04-24T06:48:36.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.171+0000 7f37f11c5740 10 client.26276 fetch_fsmap learned FSMap version 25 2024-04-24T06:48:36.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.171+0000 7f37f11c5740 1 -- 172.21.15.5:0/4028208081 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d7571463f0 con 0x55d7574ceb40 2024-04-24T06:48:36.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.171+0000 7f37d4c00640 1 -- 172.21.15.5:0/4028208081 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37d00043d0 con 0x55d7574ceb40 2024-04-24T06:48:36.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.171+0000 7f37d4c00640 1 -- 172.21.15.5:0/4028208081 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f37d80870e0 con 0x55d7574ceb40 2024-04-24T06:48:36.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.171+0000 7f37f11c5740 10 client.26276 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:48:36.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.171+0000 7f37f11c5740 10 client.26276 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T06:48:36.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.171+0000 7f37f11c5740 20 client.26276 populate_metadata read hostname 'smithi005' 2024-04-24T06:48:36.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.171+0000 7f37f11c5740 1 --2- 172.21.15.5:0/4028208081 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55d7574e9bf0 0x55d7574ec060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:36.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.171+0000 7f37f11c5740 4 client.26276 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:48:36.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.171+0000 7f37f11c5740 1 -- 172.21.15.5:0/4028208081 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d757441bf0 con 0x55d7574e9bf0 2024-04-24T06:48:36.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.172+0000 7f37e7400640 1 --2- 172.21.15.5:0/4028208081 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55d7574e9bf0 0x55d7574ec060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:36.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.172+0000 7f37e7400640 1 --2- 172.21.15.5:0/4028208081 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55d7574e9bf0 0x55d7574ec060 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3a6e7a496c83b4fb server_cookie=93ee7f074df3f1f4 in_seq=0 out_seq=0 2024-04-24T06:48:36.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.172+0000 7f37d4c00640 10 client.26276 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:48:36.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.173+0000 7f37d4c00640 1 -- 172.21.15.5:0/4028208081 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d757441bf0 con 0x55d7574e9bf0 2024-04-24T06:48:36.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.173+0000 7f37d4c00640 10 client.26276 handle_command_reply: tid=0 2024-04-24T06:48:36.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.173+0000 7f37f11c5740 1 client.26276 shutdown 2024-04-24T06:48:36.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.173+0000 7f37f11c5740 1 -- 172.21.15.5:0/4028208081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7574ceb40 msgr2=0x55d7574d9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:36.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.173+0000 7f37f11c5740 1 --2- 172.21.15.5:0/4028208081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7574ceb40 0x55d7574d9570 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f37d80058c0 tx=0x7f37d809c650 comp rx=0 tx=0).stop 2024-04-24T06:48:36.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.173+0000 7f37f11c5740 1 -- 172.21.15.5:0/4028208081 shutdown_connections 2024-04-24T06:48:36.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.173+0000 7f37f11c5740 1 --2- 172.21.15.5:0/4028208081 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55d7574e9bf0 0x55d7574ec060 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.173+0000 7f37f11c5740 1 --2- 172.21.15.5:0/4028208081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7574d9b30 0x55d7574d6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.173+0000 7f37f11c5740 1 --2- 172.21.15.5:0/4028208081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7574cf460 0x55d7574d6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.173+0000 7f37f11c5740 1 --2- 172.21.15.5:0/4028208081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7574ceb40 0x55d7574d9570 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.173+0000 7f37f11c5740 1 -- 172.21.15.5:0/4028208081 >> 172.21.15.5:0/4028208081 conn(0x55d7574cae60 msgr2=0x55d7574d0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:36.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.173+0000 7f37d4c00640 0 client.26276 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:48:36.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.174+0000 7f37f11c5740 1 -- 172.21.15.5:0/4028208081 shutdown_connections 2024-04-24T06:48:36.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.174+0000 7f37f11c5740 1 -- 172.21.15.5:0/4028208081 wait complete. 2024-04-24T06:48:36.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.174+0000 7f37f11c5740 20 client.26276 trim_cache size 0 max 16384 2024-04-24T06:48:36.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.188+0000 7f37f11c5740 1 Processor -- start 2024-04-24T06:48:36.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.189+0000 7f37f11c5740 1 -- start start 2024-04-24T06:48:36.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.189+0000 7f37f11c5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7574ceb40 0x55d7574d59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:36.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.189+0000 7f37f11c5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7574cf460 0x55d7574c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:36.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.189+0000 7f37f11c5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7574d9b30 0x55d7574d90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:36.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.189+0000 7f37f11c5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f37e81b25c0 con 0x55d7574d9b30 2024-04-24T06:48:36.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.189+0000 7f37f11c5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f37e81afe90 con 0x55d7574cf460 2024-04-24T06:48:36.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.189+0000 7f37f11c5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f37e81adce0 con 0x55d7574ceb40 2024-04-24T06:48:36.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.189+0000 7f37e7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7574d9b30 0x55d7574d90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:36.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.189+0000 7f37e7400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7574cf460 0x55d7574c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:36.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.189+0000 7f37e7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7574d9b30 0x55d7574d90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56054/0 (socket says 172.21.15.5:56054) 2024-04-24T06:48:36.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.189+0000 7f37df400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7574ceb40 0x55d7574d59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:36.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.189+0000 7f37e7e00640 1 -- 172.21.15.5:0/2805418792 learned_addr learned my addr 172.21.15.5:0/2805418792 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:36.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.190+0000 7f37df400640 1 -- 172.21.15.5:0/2805418792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7574cf460 msgr2=0x55d7574c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:36.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.190+0000 7f37df400640 1 --2- 172.21.15.5:0/2805418792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7574cf460 0x55d7574c9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.190+0000 7f37df400640 1 -- 172.21.15.5:0/2805418792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7574d9b30 msgr2=0x55d7574d90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:36.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.190+0000 7f37df400640 1 --2- 172.21.15.5:0/2805418792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7574d9b30 0x55d7574d90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.190+0000 7f37df400640 1 -- 172.21.15.5:0/2805418792 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7572107d0 con 0x55d7574ceb40 2024-04-24T06:48:36.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.190+0000 7f37df400640 1 --2- 172.21.15.5:0/2805418792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7574ceb40 0x55d7574d59c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f37d8098eb0 tx=0x7f37d8009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5444938af3542437 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:36.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.190+0000 7f37d7400640 1 -- 172.21.15.5:0/2805418792 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d80ce690 con 0x55d7574ceb40 2024-04-24T06:48:36.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.190+0000 7f37d7400640 1 -- 172.21.15.5:0/2805418792 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37d80a0e00 con 0x55d7574ceb40 2024-04-24T06:48:36.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.190+0000 7f37d7400640 1 -- 172.21.15.5:0/2805418792 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d809c820 con 0x55d7574ceb40 2024-04-24T06:48:36.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.191+0000 7f37f11c5740 1 -- 172.21.15.5:0/2805418792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7574ceb40 msgr2=0x55d7574d59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:36.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.191+0000 7f37f11c5740 1 --2- 172.21.15.5:0/2805418792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7574ceb40 0x55d7574d59c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f37d8098eb0 tx=0x7f37d8009f90 comp rx=0 tx=0).stop 2024-04-24T06:48:36.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.191+0000 7f37f11c5740 1 -- 172.21.15.5:0/2805418792 shutdown_connections 2024-04-24T06:48:36.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.191+0000 7f37f11c5740 1 --2- 172.21.15.5:0/2805418792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7574d9b30 0x55d7574d90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.191+0000 7f37f11c5740 1 --2- 172.21.15.5:0/2805418792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7574cf460 0x55d7574c9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.191+0000 7f37f11c5740 1 --2- 172.21.15.5:0/2805418792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7574ceb40 0x55d7574d59c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.191+0000 7f37f11c5740 1 -- 172.21.15.5:0/2805418792 >> 172.21.15.5:0/2805418792 conn(0x55d7574cae60 msgr2=0x55d7574c96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:36.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.191+0000 7f37f11c5740 1 -- 172.21.15.5:0/2805418792 shutdown_connections 2024-04-24T06:48:36.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.191+0000 7f37f11c5740 1 -- 172.21.15.5:0/2805418792 wait complete. 2024-04-24T06:48:36.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.191+0000 7f37f11c5740 1 -- start start 2024-04-24T06:48:36.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.192+0000 7f37f11c5740 1 Processor -- start 2024-04-24T06:48:36.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.192+0000 7f37f11c5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7574ceb40 0x55d7574e2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:36.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.192+0000 7f37f11c5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7574cf460 0x55d7574d3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:36.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.192+0000 7f37f11c5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7574d9b30 0x55d7574d4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:36.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.192+0000 7f37f11c5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f37e81ae350 con 0x55d7574d9b30 2024-04-24T06:48:36.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.192+0000 7f37f11c5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f37e81af7c0 con 0x55d7574ceb40 2024-04-24T06:48:36.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.192+0000 7f37f11c5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f37e81aea20 con 0x55d7574cf460 2024-04-24T06:48:36.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.192+0000 7f37f11c5740 10 client.0 _ll_register_callbacks cb 0x7f37d8004a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:48:36.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.192+0000 7f37df400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7574ceb40 0x55d7574e2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:36.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.192+0000 7f37df400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7574ceb40 0x55d7574e2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:33916/0 (socket says 172.21.15.5:33916) 2024-04-24T06:48:36.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.192+0000 7f37df400640 1 -- 172.21.15.5:0/3527642172 learned_addr learned my addr 172.21.15.5:0/3527642172 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:36.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.192+0000 7f37e7e00640 1 --2- 172.21.15.5:0/3527642172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7574d9b30 0x55d7574d4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:36.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.192+0000 7f37e7400640 1 --2- 172.21.15.5:0/3527642172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7574cf460 0x55d7574d3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:36.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.193+0000 7f37df400640 1 -- 172.21.15.5:0/3527642172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7574cf460 msgr2=0x55d7574d3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:36.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.193+0000 7f37df400640 1 --2- 172.21.15.5:0/3527642172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7574cf460 0x55d7574d3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.193+0000 7f37df400640 1 -- 172.21.15.5:0/3527642172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7574d9b30 msgr2=0x55d7574d4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:36.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.193+0000 7f37df400640 1 --2- 172.21.15.5:0/3527642172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7574d9b30 0x55d7574d4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.193+0000 7f37df400640 1 -- 172.21.15.5:0/3527642172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7574dad20 con 0x55d7574ceb40 2024-04-24T06:48:36.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.193+0000 7f37e7e00640 1 --2- 172.21.15.5:0/3527642172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7574d9b30 0x55d7574d4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:48:36.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.193+0000 7f37df400640 1 --2- 172.21.15.5:0/3527642172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7574ceb40 0x55d7574e2260 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f37d80993d0 tx=0x7f37d80989f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eefadba6f2edba8b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:36.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.193+0000 7f37d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:48:36.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.193+0000 7f37d4c00640 1 -- 172.21.15.5:0/3527642172 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d809c820 con 0x55d7574ceb40 2024-04-24T06:48:36.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.193+0000 7f37d4c00640 1 -- 172.21.15.5:0/3527642172 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f37d8005000 con 0x55d7574ceb40 2024-04-24T06:48:36.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.193+0000 7f37f11c5740 1 -- 172.21.15.5:0/3527642172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37d80a0e00 con 0x55d7574ceb40 2024-04-24T06:48:36.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.194+0000 7f37d4c00640 1 -- 172.21.15.5:0/3527642172 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f37d80ce690 con 0x55d7574ceb40 2024-04-24T06:48:36.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.194+0000 7f37f11c5740 10 client.26282 fetch_fsmap learned FSMap version 25 2024-04-24T06:48:36.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.194+0000 7f37f11c5740 1 -- 172.21.15.5:0/3527642172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d7575506b0 con 0x55d7574ceb40 2024-04-24T06:48:36.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.194+0000 7f37d4c00640 1 -- 172.21.15.5:0/3527642172 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37d80a0e00 con 0x55d7574ceb40 2024-04-24T06:48:36.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.194+0000 7f37d4c00640 1 -- 172.21.15.5:0/3527642172 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f37d80a1f90 con 0x55d7574ceb40 2024-04-24T06:48:36.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.194+0000 7f37f11c5740 10 client.26282 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:48:36.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.194+0000 7f37f11c5740 10 client.26282 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T06:48:36.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.194+0000 7f37f11c5740 20 client.26282 populate_metadata read hostname 'smithi005' 2024-04-24T06:48:36.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.194+0000 7f37f11c5740 1 --2- 172.21.15.5:0/3527642172 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55d757550b80 0x55d757552f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:36.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.194+0000 7f37f11c5740 4 client.26282 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:48:36.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.194+0000 7f37f11c5740 1 -- 172.21.15.5:0/3527642172 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d75743ba60 con 0x55d757550b80 2024-04-24T06:48:36.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.194+0000 7f37e7400640 1 --2- 172.21.15.5:0/3527642172 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55d757550b80 0x55d757552f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:36.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.195+0000 7f37e7400640 1 --2- 172.21.15.5:0/3527642172 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55d757550b80 0x55d757552f70 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=afa297baf8a74e67 server_cookie=3293a5882c66bfa in_seq=0 out_seq=0 2024-04-24T06:48:36.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.195+0000 7f37d4c00640 10 client.26282 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:48:36.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.195+0000 7f37d4c00640 1 -- 172.21.15.5:0/3527642172 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d75743ba60 con 0x55d757550b80 2024-04-24T06:48:36.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.195+0000 7f37d4c00640 10 client.26282 handle_command_reply: tid=0 2024-04-24T06:48:36.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.195+0000 7f37f11c5740 1 client.26282 shutdown 2024-04-24T06:48:36.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.195+0000 7f37f11c5740 1 -- 172.21.15.5:0/3527642172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7574ceb40 msgr2=0x55d7574e2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:36.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.195+0000 7f37f11c5740 1 --2- 172.21.15.5:0/3527642172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7574ceb40 0x55d7574e2260 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f37d80993d0 tx=0x7f37d80989f0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.196+0000 7f37f11c5740 1 -- 172.21.15.5:0/3527642172 shutdown_connections 2024-04-24T06:48:36.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.196+0000 7f37f11c5740 1 --2- 172.21.15.5:0/3527642172 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55d757550b80 0x55d757552f70 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.196+0000 7f37f11c5740 1 --2- 172.21.15.5:0/3527642172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7574d9b30 0x55d7574d4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.196+0000 7f37f11c5740 1 --2- 172.21.15.5:0/3527642172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7574cf460 0x55d7574d3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.196+0000 7f37f11c5740 1 --2- 172.21.15.5:0/3527642172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7574ceb40 0x55d7574e2260 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.196+0000 7f37f11c5740 1 -- 172.21.15.5:0/3527642172 >> 172.21.15.5:0/3527642172 conn(0x55d7574cae60 msgr2=0x55d7574c96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:36.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.196+0000 7f37d4c00640 0 client.26282 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:48:36.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.196+0000 7f37f11c5740 1 -- 172.21.15.5:0/3527642172 shutdown_connections 2024-04-24T06:48:36.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.196+0000 7f37f11c5740 1 -- 172.21.15.5:0/3527642172 wait complete. 2024-04-24T06:48:36.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.196+0000 7f37f11c5740 20 client.26282 trim_cache size 0 max 16384 2024-04-24T06:48:36.206 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:48:36.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.196+0000 7f37ecc00640 1 -- 172.21.15.5:0/3066189983 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f37cc0a3d20 msgr2=0x7f37cc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:36.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.196+0000 7f37ecc00640 1 --2- 172.21.15.5:0/3066189983 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f37cc0a3d20 0x7f37cc0a61e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f37e0004af0 tx=0x7f37e0005fd0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.196+0000 7f37ecc00640 1 -- 172.21.15.5:0/3066189983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e8104260 msgr2=0x7f37e81a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:36.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.196+0000 7f37ecc00640 1 --2- 172.21.15.5:0/3066189983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e8104260 0x7f37e81a0650 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f37d800e9c0 tx=0x7f37d800ee80 comp rx=0 tx=0).stop 2024-04-24T06:48:36.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.196+0000 7f37ecc00640 1 -- 172.21.15.5:0/3066189983 shutdown_connections 2024-04-24T06:48:36.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.196+0000 7f37ecc00640 1 --2- 172.21.15.5:0/3066189983 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f37cc0a3d20 0x7f37cc0a61e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.196+0000 7f37ecc00640 1 --2- 172.21.15.5:0/3066189983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37e819a710 0x7f37e819ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.197+0000 7f37ecc00640 1 --2- 172.21.15.5:0/3066189983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37e8109330 0x7f37e81a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.197+0000 7f37ecc00640 1 --2- 172.21.15.5:0/3066189983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e8104260 0x7f37e81a0650 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.197+0000 7f37ecc00640 1 -- 172.21.15.5:0/3066189983 >> 172.21.15.5:0/3066189983 conn(0x7f37e80ffaa0 msgr2=0x7f37e8101d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:36.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.197+0000 7f37ecc00640 1 -- 172.21.15.5:0/3066189983 shutdown_connections 2024-04-24T06:48:36.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.197+0000 7f37ecc00640 1 -- 172.21.15.5:0/3066189983 wait complete. 2024-04-24T06:48:36.217 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T06:48:36.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.424+0000 7f693b400640 1 -- 172.21.15.5:0/3608025081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6934102230 msgr2=0x7f6934102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:36.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.424+0000 7f693b400640 1 --2- 172.21.15.5:0/3608025081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6934102230 0x7f6934102610 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6928009900 tx=0x7f692802a3f0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.424+0000 7f693b400640 1 -- 172.21.15.5:0/3608025081 shutdown_connections 2024-04-24T06:48:36.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.424+0000 7f693b400640 1 --2- 172.21.15.5:0/3608025081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6934107300 0x7f693410eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.424+0000 7f693b400640 1 --2- 172.21.15.5:0/3608025081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6934102be0 0x7f6934106c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.424+0000 7f693b400640 1 --2- 172.21.15.5:0/3608025081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6934102230 0x7f6934102610 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.424+0000 7f693b400640 1 -- 172.21.15.5:0/3608025081 >> 172.21.15.5:0/3608025081 conn(0x7f69340fdab0 msgr2=0x7f69340ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:36.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.425+0000 7f693b400640 1 -- 172.21.15.5:0/3608025081 shutdown_connections 2024-04-24T06:48:36.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.425+0000 7f693b400640 1 -- 172.21.15.5:0/3608025081 wait complete. 2024-04-24T06:48:36.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.426+0000 7f693b400640 1 Processor -- start 2024-04-24T06:48:36.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.426+0000 7f693b400640 1 -- start start 2024-04-24T06:48:36.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.426+0000 7f693b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6934102230 0x7f6934074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:36.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.426+0000 7f693b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6934107300 0x7f6934074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:36.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.426+0000 7f693b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6934074d80 0x7f69341a6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:36.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.426+0000 7f693b400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6934112400 con 0x7f6934074d80 2024-04-24T06:48:36.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.426+0000 7f693b400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6934112280 con 0x7f6934102230 2024-04-24T06:48:36.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.426+0000 7f693b400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6934112580 con 0x7f6934107300 2024-04-24T06:48:36.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.427+0000 7f6939400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6934074d80 0x7f69341a6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:36.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.427+0000 7f6939400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6934074d80 0x7f69341a6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56078/0 (socket says 172.21.15.5:56078) 2024-04-24T06:48:36.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.427+0000 7f6939400640 1 -- 172.21.15.5:0/656174833 learned_addr learned my addr 172.21.15.5:0/656174833 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:36.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.427+0000 7f6938a00640 1 --2- 172.21.15.5:0/656174833 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6934102230 0x7f6934074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:36.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.427+0000 7f6933e00640 1 --2- 172.21.15.5:0/656174833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6934107300 0x7f6934074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:36.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.427+0000 7f6939400640 1 -- 172.21.15.5:0/656174833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6934107300 msgr2=0x7f6934074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:36.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.427+0000 7f6939400640 1 --2- 172.21.15.5:0/656174833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6934107300 0x7f6934074840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.427+0000 7f6939400640 1 -- 172.21.15.5:0/656174833 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6934102230 msgr2=0x7f6934074300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:36.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.427+0000 7f6939400640 1 --2- 172.21.15.5:0/656174833 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6934102230 0x7f6934074300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.427+0000 7f6939400640 1 -- 172.21.15.5:0/656174833 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69341a7260 con 0x7f6934074d80 2024-04-24T06:48:36.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.427+0000 7f6939400640 1 --2- 172.21.15.5:0/656174833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6934074d80 0x7f69341a6bc0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f692c00eb10 tx=0x7f692c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:36.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.428+0000 7f6931600640 1 -- 172.21.15.5:0/656174833 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c004b20 con 0x7f6934074d80 2024-04-24T06:48:36.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.428+0000 7f693b400640 1 -- 172.21.15.5:0/656174833 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69341a7550 con 0x7f6934074d80 2024-04-24T06:48:36.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.428+0000 7f6931600640 1 -- 172.21.15.5:0/656174833 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f692c004cc0 con 0x7f6934074d80 2024-04-24T06:48:36.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.428+0000 7f6931600640 1 -- 172.21.15.5:0/656174833 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c010760 con 0x7f6934074d80 2024-04-24T06:48:36.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.428+0000 7f693b400640 1 -- 172.21.15.5:0/656174833 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69341a7aa0 con 0x7f6934074d80 2024-04-24T06:48:36.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.431+0000 7f693e206740 1 Processor -- start 2024-04-24T06:48:36.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.432+0000 7f6931600640 1 -- 172.21.15.5:0/656174833 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f692c005020 con 0x7f6934074d80 2024-04-24T06:48:36.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.432+0000 7f693e206740 1 -- start start 2024-04-24T06:48:36.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.432+0000 7f693e206740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590560fe950 0x5590560fed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:36.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.432+0000 7f693e206740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590560ff270 0x559056109400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:36.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.432+0000 7f693e206740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559056109940 0x55905610bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:36.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.432+0000 7f693e206740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5590560641d0 con 0x559056109940 2024-04-24T06:48:36.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.432+0000 7f693e206740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55905600a450 con 0x5590560ff270 2024-04-24T06:48:36.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.432+0000 7f693e206740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559056060190 con 0x5590560fe950 2024-04-24T06:48:36.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.433+0000 7f6933e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590560fe950 0x5590560fed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:36.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.433+0000 7f6938a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590560ff270 0x559056109400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:36.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.433+0000 7f6938a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590560ff270 0x559056109400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:33940/0 (socket says 172.21.15.5:33940) 2024-04-24T06:48:36.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.433+0000 7f6938a00640 1 -- 172.21.15.5:0/3143305783 learned_addr learned my addr 172.21.15.5:0/3143305783 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:36.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.433+0000 7f6933e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590560fe950 0x5590560fed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:55292/0 (socket says 172.21.15.5:55292) 2024-04-24T06:48:36.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.433+0000 7f6933e00640 1 --2- 172.21.15.5:0/3143305783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559056109940 0x55905610bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:36.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.433+0000 7f6938a00640 1 -- 172.21.15.5:0/3143305783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590560fe950 msgr2=0x5590560fed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:36.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.433+0000 7f6938a00640 1 --2- 172.21.15.5:0/3143305783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590560fe950 0x5590560fed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.433+0000 7f6938a00640 1 -- 172.21.15.5:0/3143305783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559056109940 msgr2=0x55905610bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:36.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.433+0000 7f6938a00640 1 --2- 172.21.15.5:0/3143305783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559056109940 0x55905610bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.433+0000 7f6938a00640 1 -- 172.21.15.5:0/3143305783 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590560c7310 con 0x5590560ff270 2024-04-24T06:48:36.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.433+0000 7f6933e00640 1 --2- 172.21.15.5:0/3143305783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590560fe950 0x5590560fed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:48:36.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.433+0000 7f6931600640 1 --2- 172.21.15.5:0/656174833 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f691c0a3d20 0x7f691c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:36.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.434+0000 7f6938a00640 1 --2- 172.21.15.5:0/3143305783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590560ff270 0x559056109400 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6928009900 tx=0x7f6928005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b763db0d9bf803b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:36.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.434+0000 7f6931600640 1 -- 172.21.15.5:0/656174833 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f692c0d0ce0 con 0x7f6934074d80 2024-04-24T06:48:36.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.434+0000 7f6933e00640 1 --2- 172.21.15.5:0/656174833 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f691c0a3d20 0x7f691c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:36.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.434+0000 7f6925c00640 1 -- 172.21.15.5:0/3143305783 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6928035040 con 0x5590560ff270 2024-04-24T06:48:36.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.434+0000 7f6925c00640 1 -- 172.21.15.5:0/3143305783 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6928004400 con 0x5590560ff270 2024-04-24T06:48:36.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.434+0000 7f6925c00640 1 -- 172.21.15.5:0/3143305783 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69280035f0 con 0x5590560ff270 2024-04-24T06:48:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.435+0000 7f693e206740 1 -- 172.21.15.5:0/3143305783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590560ff270 msgr2=0x559056109400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.435+0000 7f693e206740 1 --2- 172.21.15.5:0/3143305783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590560ff270 0x559056109400 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6928009900 tx=0x7f6928005de0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.435+0000 7f6933e00640 1 --2- 172.21.15.5:0/656174833 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f691c0a3d20 0x7f691c0a61e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x5590560fbe00 tx=0x7f6920004730 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.435+0000 7f693e206740 1 -- 172.21.15.5:0/3143305783 shutdown_connections 2024-04-24T06:48:36.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.435+0000 7f693e206740 1 --2- 172.21.15.5:0/3143305783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559056109940 0x55905610bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.435+0000 7f693e206740 1 --2- 172.21.15.5:0/3143305783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590560ff270 0x559056109400 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.435+0000 7f693e206740 1 --2- 172.21.15.5:0/3143305783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590560fe950 0x5590560fed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.435+0000 7f693e206740 1 -- 172.21.15.5:0/3143305783 >> 172.21.15.5:0/3143305783 conn(0x5590560fac70 msgr2=0x5590560fb050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:36.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.435+0000 7f693e206740 1 -- 172.21.15.5:0/3143305783 shutdown_connections 2024-04-24T06:48:36.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.435+0000 7f693e206740 1 -- 172.21.15.5:0/3143305783 wait complete. 2024-04-24T06:48:36.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.436+0000 7f693e206740 1 -- start start 2024-04-24T06:48:36.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.436+0000 7f693e206740 1 Processor -- start 2024-04-24T06:48:36.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.436+0000 7f693e206740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590560fe950 0x5590561093d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:36.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.436+0000 7f693e206740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590560ff270 0x5590561066c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:36.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.436+0000 7f693e206740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559056109940 0x559056106db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:36.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.436+0000 7f693e206740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55905610c770 con 0x5590560fe950 2024-04-24T06:48:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.436+0000 7f693e206740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55905601d810 con 0x5590560ff270 2024-04-24T06:48:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.436+0000 7f693e206740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55905610c8f0 con 0x559056109940 2024-04-24T06:48:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.437+0000 7f693e206740 10 client.0 _ll_register_callbacks cb 0x559055feb140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:48:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.437+0000 7f6938a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590560fe950 0x5590561093d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.437+0000 7f6938a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590560fe950 0x5590561093d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56092/0 (socket says 172.21.15.5:56092) 2024-04-24T06:48:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.437+0000 7f6938a00640 1 -- 172.21.15.5:0/832352345 learned_addr learned my addr 172.21.15.5:0/832352345 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.437+0000 7f6933e00640 1 --2- 172.21.15.5:0/832352345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590560ff270 0x5590561066c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.437+0000 7f6939400640 1 --2- 172.21.15.5:0/832352345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559056109940 0x559056106db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.437+0000 7f6938a00640 1 -- 172.21.15.5:0/832352345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559056109940 msgr2=0x559056106db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.437+0000 7f6938a00640 1 --2- 172.21.15.5:0/832352345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559056109940 0x559056106db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.437+0000 7f6938a00640 1 -- 172.21.15.5:0/832352345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590560ff270 msgr2=0x5590561066c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.437+0000 7f6938a00640 1 --2- 172.21.15.5:0/832352345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590560ff270 0x5590561066c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.437+0000 7f6938a00640 1 -- 172.21.15.5:0/832352345 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55905602e3f0 con 0x5590560fe950 2024-04-24T06:48:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.438+0000 7f6938a00640 1 --2- 172.21.15.5:0/832352345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590560fe950 0x5590561093d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6928031550 tx=0x7f6928004a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bedb702175e410f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.438+0000 7f691a000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:48:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.438+0000 7f691a000640 1 -- 172.21.15.5:0/832352345 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6928052060 con 0x5590560fe950 2024-04-24T06:48:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.438+0000 7f691a000640 1 -- 172.21.15.5:0/832352345 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f6928003800 con 0x5590560fe950 2024-04-24T06:48:36.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.438+0000 7f693e206740 1 -- 172.21.15.5:0/832352345 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6928004400 con 0x5590560fe950 2024-04-24T06:48:36.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.438+0000 7f691a000640 1 -- 172.21.15.5:0/832352345 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6928035040 con 0x5590560fe950 2024-04-24T06:48:36.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.438+0000 7f693e206740 10 client.16158 fetch_fsmap learned FSMap version 25 2024-04-24T06:48:36.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.438+0000 7f693e206740 1 -- 172.21.15.5:0/832352345 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559055d763f0 con 0x5590560fe950 2024-04-24T06:48:36.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.438+0000 7f691a000640 1 -- 172.21.15.5:0/832352345 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6928004400 con 0x5590560fe950 2024-04-24T06:48:36.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.439+0000 7f691a000640 1 -- 172.21.15.5:0/832352345 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f692804c070 con 0x5590560fe950 2024-04-24T06:48:36.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.439+0000 7f693e206740 10 client.16158 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:48:36.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.439+0000 7f693e206740 10 client.16158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:48:36.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.439+0000 7f693e206740 20 client.16158 populate_metadata read hostname 'smithi005' 2024-04-24T06:48:36.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.439+0000 7f693e206740 1 --2- 172.21.15.5:0/832352345 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x559056119980 0x55905611bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:36.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.439+0000 7f693e206740 4 client.16158 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:48:36.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.439+0000 7f693e206740 1 -- 172.21.15.5:0/832352345 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559056071bf0 con 0x559056119980 2024-04-24T06:48:36.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.439+0000 7f6933e00640 1 --2- 172.21.15.5:0/832352345 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x559056119980 0x55905611bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:36.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.440+0000 7f6933e00640 1 --2- 172.21.15.5:0/832352345 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x559056119980 0x55905611bd70 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5689a79d74b8a7a server_cookie=1ee7ece59d448dd7 in_seq=0 out_seq=0 2024-04-24T06:48:36.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.440+0000 7f691a000640 10 client.16158 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:48:36.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.441+0000 7f691a000640 1 -- 172.21.15.5:0/832352345 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559056071bf0 con 0x559056119980 2024-04-24T06:48:36.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.441+0000 7f691a000640 10 client.16158 handle_command_reply: tid=0 2024-04-24T06:48:36.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.441+0000 7f693e206740 1 client.16158 shutdown 2024-04-24T06:48:36.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.441+0000 7f693e206740 1 -- 172.21.15.5:0/832352345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590560fe950 msgr2=0x5590561093d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:36.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.441+0000 7f693e206740 1 --2- 172.21.15.5:0/832352345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590560fe950 0x5590561093d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6928031550 tx=0x7f6928004a80 comp rx=0 tx=0).stop 2024-04-24T06:48:36.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.441+0000 7f693e206740 1 -- 172.21.15.5:0/832352345 shutdown_connections 2024-04-24T06:48:36.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.441+0000 7f693e206740 1 --2- 172.21.15.5:0/832352345 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x559056119980 0x55905611bd70 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.441+0000 7f693e206740 1 --2- 172.21.15.5:0/832352345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559056109940 0x559056106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.441+0000 7f693e206740 1 --2- 172.21.15.5:0/832352345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590560ff270 0x5590561066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.441+0000 7f693e206740 1 --2- 172.21.15.5:0/832352345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590560fe950 0x5590561093d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.441+0000 7f691a000640 0 client.16158 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:48:36.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.441+0000 7f693e206740 1 -- 172.21.15.5:0/832352345 >> 172.21.15.5:0/832352345 conn(0x5590560fac70 msgr2=0x559056100a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:36.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.441+0000 7f693e206740 1 -- 172.21.15.5:0/832352345 shutdown_connections 2024-04-24T06:48:36.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.441+0000 7f693e206740 1 -- 172.21.15.5:0/832352345 wait complete. 2024-04-24T06:48:36.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.441+0000 7f693e206740 20 client.16158 trim_cache size 0 max 16384 2024-04-24T06:48:36.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.456+0000 7f693e206740 1 Processor -- start 2024-04-24T06:48:36.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.456+0000 7f693e206740 1 -- start start 2024-04-24T06:48:36.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.457+0000 7f693e206740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590560fe950 0x559056115630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:36.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.457+0000 7f693e206740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590560ff270 0x559056108f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:36.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.457+0000 7f693e206740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559056109940 0x5590561213a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:36.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.457+0000 7f693e206740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f69341b2850 con 0x5590560fe950 2024-04-24T06:48:36.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.457+0000 7f693e206740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f69341b0120 con 0x559056109940 2024-04-24T06:48:36.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.457+0000 7f693e206740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f69341adf70 con 0x5590560ff270 2024-04-24T06:48:36.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.457+0000 7f6933e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590560ff270 0x559056108f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:36.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.457+0000 7f6933e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590560ff270 0x559056108f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:55308/0 (socket says 172.21.15.5:55308) 2024-04-24T06:48:36.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.457+0000 7f6939400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559056109940 0x5590561213a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:36.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.457+0000 7f6933e00640 1 -- 172.21.15.5:0/670671620 learned_addr learned my addr 172.21.15.5:0/670671620 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:36.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.457+0000 7f6938a00640 1 --2- 172.21.15.5:0/670671620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590560fe950 0x559056115630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:36.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.458+0000 7f6933e00640 1 -- 172.21.15.5:0/670671620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559056109940 msgr2=0x5590561213a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:36.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.458+0000 7f6933e00640 1 --2- 172.21.15.5:0/670671620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559056109940 0x5590561213a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.458+0000 7f6933e00640 1 -- 172.21.15.5:0/670671620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590560fe950 msgr2=0x559056115630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:36.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.458+0000 7f6933e00640 1 --2- 172.21.15.5:0/670671620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590560fe950 0x559056115630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.458+0000 7f6933e00640 1 -- 172.21.15.5:0/670671620 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559055e407d0 con 0x5590560ff270 2024-04-24T06:48:36.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.458+0000 7f6939400640 1 --2- 172.21.15.5:0/670671620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559056109940 0x5590561213a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:48:36.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.458+0000 7f6933e00640 1 --2- 172.21.15.5:0/670671620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590560ff270 0x559056108f00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f693406b790 tx=0x7f6920016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56c573dd9f5f3621 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:36.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.458+0000 7f6925200640 1 -- 172.21.15.5:0/670671620 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692002e3b0 con 0x5590560ff270 2024-04-24T06:48:36.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.458+0000 7f6925200640 1 -- 172.21.15.5:0/670671620 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6920005020 con 0x5590560ff270 2024-04-24T06:48:36.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.458+0000 7f6925200640 1 -- 172.21.15.5:0/670671620 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6920004a60 con 0x5590560ff270 2024-04-24T06:48:36.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.459+0000 7f693e206740 1 -- 172.21.15.5:0/670671620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590560ff270 msgr2=0x559056108f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:36.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.459+0000 7f693e206740 1 --2- 172.21.15.5:0/670671620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590560ff270 0x559056108f00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f693406b790 tx=0x7f6920016000 comp rx=0 tx=0).stop 2024-04-24T06:48:36.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.459+0000 7f693e206740 1 -- 172.21.15.5:0/670671620 shutdown_connections 2024-04-24T06:48:36.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.459+0000 7f693e206740 1 --2- 172.21.15.5:0/670671620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559056109940 0x5590561213a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.459+0000 7f693e206740 1 --2- 172.21.15.5:0/670671620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590560ff270 0x559056108f00 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.459+0000 7f693e206740 1 --2- 172.21.15.5:0/670671620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590560fe950 0x559056115630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.459+0000 7f693e206740 1 -- 172.21.15.5:0/670671620 >> 172.21.15.5:0/670671620 conn(0x5590560fac70 msgr2=0x559056101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:36.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.459+0000 7f693e206740 1 -- 172.21.15.5:0/670671620 shutdown_connections 2024-04-24T06:48:36.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.459+0000 7f693e206740 1 -- 172.21.15.5:0/670671620 wait complete. 2024-04-24T06:48:36.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.459+0000 7f693e206740 1 -- start start 2024-04-24T06:48:36.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.460+0000 7f693e206740 1 Processor -- start 2024-04-24T06:48:36.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.460+0000 7f693e206740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590560fe950 0x559056111f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:36.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.460+0000 7f693e206740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590560ff270 0x55905610a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:36.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.460+0000 7f693e206740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559056109940 0x55905610ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:36.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.460+0000 7f693e206740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f69341ae5e0 con 0x5590560fe950 2024-04-24T06:48:36.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.460+0000 7f693e206740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f69341afa50 con 0x559056109940 2024-04-24T06:48:36.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.460+0000 7f693e206740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f69341aecb0 con 0x5590560ff270 2024-04-24T06:48:36.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.460+0000 7f693e206740 10 client.0 _ll_register_callbacks cb 0x7f6920009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:48:36.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.460+0000 7f6938a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590560fe950 0x559056111f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:36.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.460+0000 7f6938a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590560fe950 0x559056111f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56104/0 (socket says 172.21.15.5:56104) 2024-04-24T06:48:36.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.460+0000 7f6938a00640 1 -- 172.21.15.5:0/1635436734 learned_addr learned my addr 172.21.15.5:0/1635436734 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:36.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.461+0000 7f6939400640 1 --2- 172.21.15.5:0/1635436734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559056109940 0x55905610ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:36.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.461+0000 7f6933e00640 1 --2- 172.21.15.5:0/1635436734 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590560ff270 0x55905610a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:36.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.461+0000 7f6938a00640 1 -- 172.21.15.5:0/1635436734 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590560ff270 msgr2=0x55905610a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:36.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.461+0000 7f6938a00640 1 --2- 172.21.15.5:0/1635436734 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590560ff270 0x55905610a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.461+0000 7f6938a00640 1 -- 172.21.15.5:0/1635436734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559056109940 msgr2=0x55905610ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:36.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.461+0000 7f6938a00640 1 --2- 172.21.15.5:0/1635436734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559056109940 0x55905610ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.461+0000 7f6938a00640 1 -- 172.21.15.5:0/1635436734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55905610b5e0 con 0x5590560fe950 2024-04-24T06:48:36.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.461+0000 7f6938a00640 1 --2- 172.21.15.5:0/1635436734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590560fe950 0x559056111f60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f69280092a0 tx=0x7f6928032fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d54571c8af502a5b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:36.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.461+0000 7f691a000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:48:36.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.461+0000 7f691a000640 1 -- 172.21.15.5:0/1635436734 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6928063060 con 0x5590560fe950 2024-04-24T06:48:36.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.462+0000 7f691a000640 1 -- 172.21.15.5:0/1635436734 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f69280315b0 con 0x5590560fe950 2024-04-24T06:48:36.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.462+0000 7f693e206740 1 -- 172.21.15.5:0/1635436734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6920005020 con 0x5590560fe950 2024-04-24T06:48:36.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.462+0000 7f691a000640 1 -- 172.21.15.5:0/1635436734 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6928052050 con 0x5590560fe950 2024-04-24T06:48:36.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.462+0000 7f693e206740 10 client.16164 fetch_fsmap learned FSMap version 25 2024-04-24T06:48:36.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.462+0000 7f693e206740 1 -- 172.21.15.5:0/1635436734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55905617fcd0 con 0x5590560fe950 2024-04-24T06:48:36.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.462+0000 7f691a000640 1 -- 172.21.15.5:0/1635436734 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6920005020 con 0x5590560fe950 2024-04-24T06:48:36.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.462+0000 7f691a000640 1 -- 172.21.15.5:0/1635436734 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f692804c040 con 0x5590560fe950 2024-04-24T06:48:36.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.462+0000 7f693e206740 10 client.16164 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:48:36.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.462+0000 7f693e206740 10 client.16164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:48:36.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.462+0000 7f693e206740 20 client.16164 populate_metadata read hostname 'smithi005' 2024-04-24T06:48:36.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.463+0000 7f693e206740 1 --2- 172.21.15.5:0/1635436734 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x559056180450 0x559056182870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:36.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.463+0000 7f693e206740 4 client.16164 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:48:36.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.463+0000 7f693e206740 1 -- 172.21.15.5:0/1635436734 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55905606ba60 con 0x559056180450 2024-04-24T06:48:36.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.463+0000 7f6933e00640 1 --2- 172.21.15.5:0/1635436734 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x559056180450 0x559056182870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:36.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.463+0000 7f6933e00640 1 --2- 172.21.15.5:0/1635436734 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x559056180450 0x559056182870 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb2bb07ed5f5c513 server_cookie=65e2785d571012a2 in_seq=0 out_seq=0 2024-04-24T06:48:36.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.464+0000 7f691a000640 10 client.16164 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:48:36.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.464+0000 7f691a000640 1 -- 172.21.15.5:0/1635436734 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55905606ba60 con 0x559056180450 2024-04-24T06:48:36.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.464+0000 7f691a000640 10 client.16164 handle_command_reply: tid=0 2024-04-24T06:48:36.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.464+0000 7f693e206740 1 client.16164 shutdown 2024-04-24T06:48:36.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.464+0000 7f693e206740 1 -- 172.21.15.5:0/1635436734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590560fe950 msgr2=0x559056111f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:36.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.464+0000 7f693e206740 1 --2- 172.21.15.5:0/1635436734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590560fe950 0x559056111f60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f69280092a0 tx=0x7f6928032fb0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.464+0000 7f693e206740 1 -- 172.21.15.5:0/1635436734 shutdown_connections 2024-04-24T06:48:36.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.464+0000 7f693e206740 1 --2- 172.21.15.5:0/1635436734 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x559056180450 0x559056182870 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.464+0000 7f693e206740 1 --2- 172.21.15.5:0/1635436734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559056109940 0x55905610ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.464+0000 7f693e206740 1 --2- 172.21.15.5:0/1635436734 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590560ff270 0x55905610a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.464+0000 7f691a000640 0 client.16164 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:48:36.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.464+0000 7f693e206740 1 --2- 172.21.15.5:0/1635436734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590560fe950 0x559056111f60 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.464+0000 7f693e206740 1 -- 172.21.15.5:0/1635436734 >> 172.21.15.5:0/1635436734 conn(0x5590560fac70 msgr2=0x5590560f9560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:36.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.464+0000 7f693e206740 1 -- 172.21.15.5:0/1635436734 shutdown_connections 2024-04-24T06:48:36.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.464+0000 7f693e206740 1 -- 172.21.15.5:0/1635436734 wait complete. 2024-04-24T06:48:36.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.464+0000 7f693e206740 20 client.16164 trim_cache size 0 max 16384 2024-04-24T06:48:36.476 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:48:36.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.465+0000 7f693b400640 1 -- 172.21.15.5:0/656174833 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f691c0a3d20 msgr2=0x7f691c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:36.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.465+0000 7f693b400640 1 --2- 172.21.15.5:0/656174833 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f691c0a3d20 0x7f691c0a61e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x5590560fbe00 tx=0x7f6920004730 comp rx=0 tx=0).stop 2024-04-24T06:48:36.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.465+0000 7f693b400640 1 -- 172.21.15.5:0/656174833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6934074d80 msgr2=0x7f69341a6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:36.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.465+0000 7f693b400640 1 --2- 172.21.15.5:0/656174833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6934074d80 0x7f69341a6bc0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f692c00eb10 tx=0x7f692c00c2d0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.465+0000 7f693b400640 1 -- 172.21.15.5:0/656174833 shutdown_connections 2024-04-24T06:48:36.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.465+0000 7f693b400640 1 --2- 172.21.15.5:0/656174833 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f691c0a3d20 0x7f691c0a61e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.465+0000 7f693b400640 1 --2- 172.21.15.5:0/656174833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6934074d80 0x7f69341a6bc0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.465+0000 7f693b400640 1 --2- 172.21.15.5:0/656174833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6934107300 0x7f6934074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.465+0000 7f693b400640 1 --2- 172.21.15.5:0/656174833 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6934102230 0x7f6934074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:36.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.465+0000 7f693b400640 1 -- 172.21.15.5:0/656174833 >> 172.21.15.5:0/656174833 conn(0x7f69340fdab0 msgr2=0x7f69340ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:36.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.465+0000 7f693b400640 1 -- 172.21.15.5:0/656174833 shutdown_connections 2024-04-24T06:48:36.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:36.465+0000 7f693b400640 1 -- 172.21.15.5:0/656174833 wait complete. 2024-04-24T06:48:36.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:36 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1024863854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:48:36.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:36 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/1024863854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:48:37.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:36 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/1024863854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:48:37.484 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T06:48:37.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:48:37.485 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T06:48:37.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.716+0000 7f543b600640 1 -- 172.21.15.5:0/1398310882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5434104450 msgr2=0x7f5434104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:37.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.716+0000 7f543b600640 1 --2- 172.21.15.5:0/1398310882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5434104450 0x7f5434104830 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5424009ca0 tx=0x7f542402a8c0 comp rx=0 tx=0).stop 2024-04-24T06:48:37.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.717+0000 7f543b600640 1 -- 172.21.15.5:0/1398310882 shutdown_connections 2024-04-24T06:48:37.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.717+0000 7f543b600640 1 --2- 172.21.15.5:0/1398310882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5434109520 0x7f54341110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:37.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.717+0000 7f543b600640 1 --2- 172.21.15.5:0/1398310882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5434104e00 0x7f5434108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:37.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.717+0000 7f543b600640 1 --2- 172.21.15.5:0/1398310882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5434104450 0x7f5434104830 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:37.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.717+0000 7f543b600640 1 -- 172.21.15.5:0/1398310882 >> 172.21.15.5:0/1398310882 conn(0x7f54340ffc90 msgr2=0x7f54341020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:37.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.717+0000 7f543b600640 1 -- 172.21.15.5:0/1398310882 shutdown_connections 2024-04-24T06:48:37.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.717+0000 7f543b600640 1 -- 172.21.15.5:0/1398310882 wait complete. 2024-04-24T06:48:37.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.718+0000 7f543b600640 1 Processor -- start 2024-04-24T06:48:37.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.718+0000 7f543b600640 1 -- start start 2024-04-24T06:48:37.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.718+0000 7f543b600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5434109520 0x7f543419ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:37.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.718+0000 7f543b600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f543419b3a0 0x7f543419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:37.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.718+0000 7f543b600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f543419fc60 0x7f54341a0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:37.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.718+0000 7f543b600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5434114600 con 0x7f543419fc60 2024-04-24T06:48:37.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.718+0000 7f543b600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5434114480 con 0x7f543419b3a0 2024-04-24T06:48:37.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.718+0000 7f543b600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5434114780 con 0x7f5434109520 2024-04-24T06:48:37.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.719+0000 7f5439600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f543419fc60 0x7f54341a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:37.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.719+0000 7f5439600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f543419fc60 0x7f54341a0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56124/0 (socket says 172.21.15.5:56124) 2024-04-24T06:48:37.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.719+0000 7f5439600640 1 -- 172.21.15.5:0/2075280671 learned_addr learned my addr 172.21.15.5:0/2075280671 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:37.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.719+0000 7f5438c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5434109520 0x7f543419ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:37.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.719+0000 7f5433e00640 1 --2- 172.21.15.5:0/2075280671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f543419b3a0 0x7f543419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:37.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.719+0000 7f5439600640 1 -- 172.21.15.5:0/2075280671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5434109520 msgr2=0x7f543419ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:37.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.719+0000 7f5439600640 1 --2- 172.21.15.5:0/2075280671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5434109520 0x7f543419ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:37.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.719+0000 7f5439600640 1 -- 172.21.15.5:0/2075280671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f543419b3a0 msgr2=0x7f543419f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:37.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.719+0000 7f5439600640 1 --2- 172.21.15.5:0/2075280671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f543419b3a0 0x7f543419f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:37.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.719+0000 7f5439600640 1 -- 172.21.15.5:0/2075280671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54341a79d0 con 0x7f543419fc60 2024-04-24T06:48:37.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.719+0000 7f5438c00640 1 --2- 172.21.15.5:0/2075280671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5434109520 0x7f543419ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:48:37.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.720+0000 7f5439600640 1 --2- 172.21.15.5:0/2075280671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f543419fc60 0x7f54341a0110 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f542c00e970 tx=0x7f542c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:37.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.720+0000 7f5431600640 1 -- 172.21.15.5:0/2075280671 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f542c00ccc0 con 0x7f543419fc60 2024-04-24T06:48:37.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.720+0000 7f5431600640 1 -- 172.21.15.5:0/2075280671 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f542c00ce60 con 0x7f543419fc60 2024-04-24T06:48:37.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.720+0000 7f543b600640 1 -- 172.21.15.5:0/2075280671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54341a7c00 con 0x7f543419fc60 2024-04-24T06:48:37.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.720+0000 7f5431600640 1 -- 172.21.15.5:0/2075280671 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f542c010490 con 0x7f543419fc60 2024-04-24T06:48:37.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.720+0000 7f543b600640 1 -- 172.21.15.5:0/2075280671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54341a80d0 con 0x7f543419fc60 2024-04-24T06:48:37.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.723+0000 7f543e43e740 1 Processor -- start 2024-04-24T06:48:37.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.724+0000 7f5431600640 1 -- 172.21.15.5:0/2075280671 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f542c005020 con 0x7f543419fc60 2024-04-24T06:48:37.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.724+0000 7f543e43e740 1 -- start start 2024-04-24T06:48:37.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.724+0000 7f543e43e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565f5026090 0x5565f5026470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:37.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.724+0000 7f543e43e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565f50269b0 0x5565f5030b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:37.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.724+0000 7f543e43e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565f5031080 0x5565f5033470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:37.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.724+0000 7f543e43e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5565f4f8be70 con 0x5565f50269b0 2024-04-24T06:48:37.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.724+0000 7f543e43e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5565f4f32380 con 0x5565f5031080 2024-04-24T06:48:37.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.724+0000 7f543e43e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5565f4f87e30 con 0x5565f5026090 2024-04-24T06:48:37.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.725+0000 7f5433e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565f5026090 0x5565f5026470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:37.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.725+0000 7f5438c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565f50269b0 0x5565f5030b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:37.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.725+0000 7f5433e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565f5026090 0x5565f5026470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:55322/0 (socket says 172.21.15.5:55322) 2024-04-24T06:48:37.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.725+0000 7f5433e00640 1 -- 172.21.15.5:0/4179783352 learned_addr learned my addr 172.21.15.5:0/4179783352 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:37.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.725+0000 7f5433e00640 1 --2- 172.21.15.5:0/4179783352 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565f5031080 0x5565f5033470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:37.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.725+0000 7f5433e00640 1 -- 172.21.15.5:0/4179783352 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565f5031080 msgr2=0x5565f5033470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:37.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.725+0000 7f5433e00640 1 --2- 172.21.15.5:0/4179783352 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565f5031080 0x5565f5033470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:37.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.725+0000 7f5433e00640 1 -- 172.21.15.5:0/4179783352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565f50269b0 msgr2=0x5565f5030b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:37.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.725+0000 7f5433e00640 1 --2- 172.21.15.5:0/4179783352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565f50269b0 0x5565f5030b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:37.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.725+0000 7f5433e00640 1 -- 172.21.15.5:0/4179783352 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565f4d667e0 con 0x5565f5026090 2024-04-24T06:48:37.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.725+0000 7f5438c00640 1 --2- 172.21.15.5:0/4179783352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565f50269b0 0x5565f5030b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:48:37.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.725+0000 7f5433e00640 1 --2- 172.21.15.5:0/4179783352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565f5026090 0x5565f5026470 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f541c0058c0 tx=0x7f541c005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=253c91c242191192 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:37.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.725+0000 7f5431600640 1 --2- 172.21.15.5:0/2075280671 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f54180a3d20 0x7f54180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:37.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.726+0000 7f5429c00640 1 -- 172.21.15.5:0/4179783352 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541c004030 con 0x5565f5026090 2024-04-24T06:48:37.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.726+0000 7f5429c00640 1 -- 172.21.15.5:0/4179783352 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f541c005500 con 0x5565f5026090 2024-04-24T06:48:37.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.726+0000 7f5429c00640 1 -- 172.21.15.5:0/4179783352 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541c004d10 con 0x5565f5026090 2024-04-24T06:48:37.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.726+0000 7f5431600640 1 -- 172.21.15.5:0/2075280671 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f542c0d0bd0 con 0x7f543419fc60 2024-04-24T06:48:37.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.726+0000 7f5438c00640 1 --2- 172.21.15.5:0/2075280671 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f54180a3d20 0x7f54180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:37.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.726+0000 7f543e43e740 1 -- 172.21.15.5:0/4179783352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565f5026090 msgr2=0x5565f5026470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:37.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.726+0000 7f543e43e740 1 --2- 172.21.15.5:0/4179783352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565f5026090 0x5565f5026470 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f541c0058c0 tx=0x7f541c005850 comp rx=0 tx=0).stop 2024-04-24T06:48:37.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.727+0000 7f543e43e740 1 -- 172.21.15.5:0/4179783352 shutdown_connections 2024-04-24T06:48:37.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.727+0000 7f543e43e740 1 --2- 172.21.15.5:0/4179783352 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565f5031080 0x5565f5033470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:37.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.727+0000 7f543e43e740 1 --2- 172.21.15.5:0/4179783352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565f50269b0 0x5565f5030b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:37.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.727+0000 7f543e43e740 1 --2- 172.21.15.5:0/4179783352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565f5026090 0x5565f5026470 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:37.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.727+0000 7f543e43e740 1 -- 172.21.15.5:0/4179783352 >> 172.21.15.5:0/4179783352 conn(0x5565f50223b0 msgr2=0x5565f5022790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:37.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.727+0000 7f543e43e740 1 -- 172.21.15.5:0/4179783352 shutdown_connections 2024-04-24T06:48:37.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.727+0000 7f5438c00640 1 --2- 172.21.15.5:0/2075280671 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f54180a3d20 0x7f54180a61e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5424004af0 tx=0x7f5424000f40 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:37.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.727+0000 7f543e43e740 1 -- 172.21.15.5:0/4179783352 wait complete. 2024-04-24T06:48:37.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.727+0000 7f543e43e740 1 -- start start 2024-04-24T06:48:37.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.727+0000 7f543e43e740 1 Processor -- start 2024-04-24T06:48:37.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.728+0000 7f543e43e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565f5026090 0x5565f502dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:37.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.728+0000 7f543e43e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565f50269b0 0x5565f502e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:37.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.728+0000 7f543e43e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565f5031080 0x5565f502ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:37.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.728+0000 7f543e43e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5565f5033e70 con 0x5565f5026090 2024-04-24T06:48:37.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.728+0000 7f543e43e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5565f4f45660 con 0x5565f5031080 2024-04-24T06:48:37.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.728+0000 7f543e43e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5565f5033ff0 con 0x5565f50269b0 2024-04-24T06:48:37.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.728+0000 7f543e43e740 10 client.0 _ll_register_callbacks cb 0x5565f4f33aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:48:37.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.728+0000 7f5438c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565f5026090 0x5565f502dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:37.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.728+0000 7f5433e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565f50269b0 0x5565f502e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:37.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.728+0000 7f5439600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565f5031080 0x5565f502ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:37.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.728+0000 7f5433e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565f50269b0 0x5565f502e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:55330/0 (socket says 172.21.15.5:55330) 2024-04-24T06:48:37.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.728+0000 7f5439600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565f5031080 0x5565f502ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:33992/0 (socket says 172.21.15.5:33992) 2024-04-24T06:48:37.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.728+0000 7f5438c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565f5026090 0x5565f502dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56140/0 (socket says 172.21.15.5:56140) 2024-04-24T06:48:37.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.728+0000 7f5433e00640 1 -- 172.21.15.5:0/1539846158 learned_addr learned my addr 172.21.15.5:0/1539846158 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:37.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.729+0000 7f5439600640 1 -- 172.21.15.5:0/1539846158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565f50269b0 msgr2=0x5565f502e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:37.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.729+0000 7f5439600640 1 --2- 172.21.15.5:0/1539846158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565f50269b0 0x5565f502e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:37.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.729+0000 7f5439600640 1 -- 172.21.15.5:0/1539846158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565f5026090 msgr2=0x5565f502dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:37.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.729+0000 7f5439600640 1 --2- 172.21.15.5:0/1539846158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565f5026090 0x5565f502dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:37.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.729+0000 7f5439600640 1 -- 172.21.15.5:0/1539846158 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565f4dc6f70 con 0x5565f5031080 2024-04-24T06:48:37.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.729+0000 7f5438c00640 1 --2- 172.21.15.5:0/1539846158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565f5026090 0x5565f502dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:48:37.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.729+0000 7f5439600640 1 --2- 172.21.15.5:0/1539846158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565f5031080 0x5565f502ea70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f542c0ce300 tx=0x7f542c09d340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8876146ddece21e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:37.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.729+0000 7f5422000640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:48:37.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.729+0000 7f5422000640 1 -- 172.21.15.5:0/1539846158 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f542c0a57f0 con 0x5565f5031080 2024-04-24T06:48:37.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.729+0000 7f5422000640 1 -- 172.21.15.5:0/1539846158 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f542c0991f0 con 0x5565f5031080 2024-04-24T06:48:37.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.729+0000 7f543e43e740 1 -- 172.21.15.5:0/1539846158 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f541c005500 con 0x5565f5031080 2024-04-24T06:48:37.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.730+0000 7f5422000640 1 -- 172.21.15.5:0/1539846158 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f542c005310 con 0x5565f5031080 2024-04-24T06:48:37.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.730+0000 7f543e43e740 10 client.26312 fetch_fsmap learned FSMap version 25 2024-04-24T06:48:37.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.730+0000 7f543e43e740 1 -- 172.21.15.5:0/1539846158 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5565f4c5e170 con 0x5565f5031080 2024-04-24T06:48:37.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.730+0000 7f5422000640 1 -- 172.21.15.5:0/1539846158 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f541c005500 con 0x5565f5031080 2024-04-24T06:48:37.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.730+0000 7f5422000640 1 -- 172.21.15.5:0/1539846158 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f542c09bfd0 con 0x5565f5031080 2024-04-24T06:48:37.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.730+0000 7f543e43e740 10 client.26312 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:48:37.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.730+0000 7f543e43e740 10 client.26312 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:48:37.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.730+0000 7f543e43e740 20 client.26312 populate_metadata read hostname 'smithi005' 2024-04-24T06:48:37.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.730+0000 7f543e43e740 1 --2- 172.21.15.5:0/1539846158 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5565f50413f0 0x5565f5043870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:37.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.730+0000 7f543e43e740 4 client.26312 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:48:37.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.730+0000 7f543e43e740 1 -- 172.21.15.5:0/1539846158 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565f4f99890 con 0x5565f50413f0 2024-04-24T06:48:37.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.731+0000 7f5438c00640 1 --2- 172.21.15.5:0/1539846158 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5565f50413f0 0x5565f5043870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:37.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.731+0000 7f5438c00640 1 --2- 172.21.15.5:0/1539846158 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5565f50413f0 0x5565f5043870 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce29041d94f9acfc server_cookie=af18c09347b5e5ef in_seq=0 out_seq=0 2024-04-24T06:48:37.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.731+0000 7f5422000640 10 client.26312 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:48:37.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.732+0000 7f5422000640 1 -- 172.21.15.5:0/1539846158 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5565f4f99890 con 0x5565f50413f0 2024-04-24T06:48:37.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.732+0000 7f5422000640 10 client.26312 handle_command_reply: tid=0 2024-04-24T06:48:37.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.732+0000 7f543e43e740 1 client.26312 shutdown 2024-04-24T06:48:37.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.732+0000 7f543e43e740 1 -- 172.21.15.5:0/1539846158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565f5031080 msgr2=0x5565f502ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:37.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.732+0000 7f543e43e740 1 --2- 172.21.15.5:0/1539846158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565f5031080 0x5565f502ea70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f542c0ce300 tx=0x7f542c09d340 comp rx=0 tx=0).stop 2024-04-24T06:48:37.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.732+0000 7f543e43e740 1 -- 172.21.15.5:0/1539846158 shutdown_connections 2024-04-24T06:48:37.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.732+0000 7f543e43e740 1 --2- 172.21.15.5:0/1539846158 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5565f50413f0 0x5565f5043870 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:37.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.732+0000 7f543e43e740 1 --2- 172.21.15.5:0/1539846158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565f5031080 0x5565f502ea70 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:37.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.732+0000 7f543e43e740 1 --2- 172.21.15.5:0/1539846158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565f50269b0 0x5565f502e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:37.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.732+0000 7f5422000640 0 client.26312 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:48:37.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.732+0000 7f543e43e740 1 --2- 172.21.15.5:0/1539846158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565f5026090 0x5565f502dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:37.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.732+0000 7f543e43e740 1 -- 172.21.15.5:0/1539846158 >> 172.21.15.5:0/1539846158 conn(0x5565f50223b0 msgr2=0x5565f50281b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:37.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.733+0000 7f543e43e740 1 -- 172.21.15.5:0/1539846158 shutdown_connections 2024-04-24T06:48:37.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.733+0000 7f543e43e740 1 -- 172.21.15.5:0/1539846158 wait complete. 2024-04-24T06:48:37.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.733+0000 7f543e43e740 20 client.26312 trim_cache size 0 max 16384 2024-04-24T06:48:37.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.752+0000 7f543e43e740 1 Processor -- start 2024-04-24T06:48:37.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.753+0000 7f543e43e740 1 -- start start 2024-04-24T06:48:37.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.753+0000 7f543e43e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565f5026090 0x5565f503cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:37.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.753+0000 7f543e43e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565f50269b0 0x5565f502e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:37.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.753+0000 7f543e43e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565f5031080 0x5565f5049140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:37.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.753+0000 7f543e43e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f54341b2dc0 con 0x5565f5031080 2024-04-24T06:48:37.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.753+0000 7f543e43e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f54341b0690 con 0x5565f5026090 2024-04-24T06:48:37.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.753+0000 7f543e43e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f54341ae500 con 0x5565f50269b0 2024-04-24T06:48:37.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.753+0000 7f5439600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565f5031080 0x5565f5049140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:37.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.753+0000 7f5439600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565f5031080 0x5565f5049140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56156/0 (socket says 172.21.15.5:56156) 2024-04-24T06:48:37.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.753+0000 7f5439600640 1 -- 172.21.15.5:0/2173885270 learned_addr learned my addr 172.21.15.5:0/2173885270 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:37.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.754+0000 7f5433e00640 1 --2- 172.21.15.5:0/2173885270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565f50269b0 0x5565f502e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:37.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.754+0000 7f5438c00640 1 --2- 172.21.15.5:0/2173885270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565f5026090 0x5565f503cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:37.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.754+0000 7f5439600640 1 -- 172.21.15.5:0/2173885270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565f50269b0 msgr2=0x5565f502e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:37.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.754+0000 7f5439600640 1 --2- 172.21.15.5:0/2173885270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565f50269b0 0x5565f502e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:37.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.754+0000 7f5439600640 1 -- 172.21.15.5:0/2173885270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565f5026090 msgr2=0x5565f503cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:37.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.754+0000 7f5439600640 1 --2- 172.21.15.5:0/2173885270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565f5026090 0x5565f503cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:37.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.754+0000 7f5439600640 1 -- 172.21.15.5:0/2173885270 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565f5021e40 con 0x5565f5031080 2024-04-24T06:48:37.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.754+0000 7f5439600640 1 --2- 172.21.15.5:0/2173885270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565f5031080 0x5565f5049140 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f542c00e940 tx=0x7f542c0b6200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cb89de206290ae9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:37.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.754+0000 7f5429200640 1 -- 172.21.15.5:0/2173885270 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f542c0a57f0 con 0x5565f5031080 2024-04-24T06:48:37.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.754+0000 7f5429200640 1 -- 172.21.15.5:0/2173885270 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f542c0054b0 con 0x5565f5031080 2024-04-24T06:48:37.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.754+0000 7f5429200640 1 -- 172.21.15.5:0/2173885270 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f542c09c4e0 con 0x5565f5031080 2024-04-24T06:48:37.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.755+0000 7f543e43e740 1 -- 172.21.15.5:0/2173885270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565f5031080 msgr2=0x5565f5049140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:37.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.755+0000 7f543e43e740 1 --2- 172.21.15.5:0/2173885270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565f5031080 0x5565f5049140 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f542c00e940 tx=0x7f542c0b6200 comp rx=0 tx=0).stop 2024-04-24T06:48:37.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.755+0000 7f543e43e740 1 -- 172.21.15.5:0/2173885270 shutdown_connections 2024-04-24T06:48:37.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.755+0000 7f543e43e740 1 --2- 172.21.15.5:0/2173885270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565f5031080 0x5565f5049140 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:37.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.755+0000 7f543e43e740 1 --2- 172.21.15.5:0/2173885270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565f50269b0 0x5565f502e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:37.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.755+0000 7f543e43e740 1 --2- 172.21.15.5:0/2173885270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565f5026090 0x5565f503cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:37.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.755+0000 7f543e43e740 1 -- 172.21.15.5:0/2173885270 >> 172.21.15.5:0/2173885270 conn(0x5565f50223b0 msgr2=0x5565f502dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:37.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.755+0000 7f543e43e740 1 -- 172.21.15.5:0/2173885270 shutdown_connections 2024-04-24T06:48:37.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.755+0000 7f543e43e740 1 -- 172.21.15.5:0/2173885270 wait complete. 2024-04-24T06:48:37.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.756+0000 7f543e43e740 1 -- start start 2024-04-24T06:48:37.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.756+0000 7f543e43e740 1 Processor -- start 2024-04-24T06:48:37.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.756+0000 7f543e43e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565f5026090 0x5565f502aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:37.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.756+0000 7f543e43e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565f50269b0 0x5565f502b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:37.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.756+0000 7f543e43e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565f5031080 0x5565f502b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:37.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.757+0000 7f543e43e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f54341aeb50 con 0x5565f50269b0 2024-04-24T06:48:37.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.757+0000 7f543e43e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f54341affc0 con 0x5565f5026090 2024-04-24T06:48:37.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.757+0000 7f543e43e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f54341af220 con 0x5565f5031080 2024-04-24T06:48:37.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.757+0000 7f543e43e740 10 client.0 _ll_register_callbacks cb 0x7f542c005690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:48:37.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.757+0000 7f5433e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565f50269b0 0x5565f502b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:37.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.757+0000 7f5438c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565f5026090 0x5565f502aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:37.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.757+0000 7f5438c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565f5026090 0x5565f502aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:34000/0 (socket says 172.21.15.5:34000) 2024-04-24T06:48:37.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.757+0000 7f5438c00640 1 -- 172.21.15.5:0/3310836220 learned_addr learned my addr 172.21.15.5:0/3310836220 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:37.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.757+0000 7f5439600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565f5031080 0x5565f502b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:37.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.757+0000 7f5438c00640 1 -- 172.21.15.5:0/3310836220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565f5031080 msgr2=0x5565f502b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:37.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.757+0000 7f5438c00640 1 --2- 172.21.15.5:0/3310836220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565f5031080 0x5565f502b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:37.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.757+0000 7f5438c00640 1 -- 172.21.15.5:0/3310836220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565f50269b0 msgr2=0x5565f502b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:37.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.757+0000 7f5438c00640 1 --2- 172.21.15.5:0/3310836220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565f50269b0 0x5565f502b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:37.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.757+0000 7f5438c00640 1 -- 172.21.15.5:0/3310836220 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565f50a6bb0 con 0x5565f5026090 2024-04-24T06:48:37.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.757+0000 7f5438c00640 1 --2- 172.21.15.5:0/3310836220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565f5026090 0x5565f502aeb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5424009ca0 tx=0x7f54240095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c492749fc418e3d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:37.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.757+0000 7f5422000640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:48:37.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.758+0000 7f5422000640 1 -- 172.21.15.5:0/3310836220 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f542405a070 con 0x5565f5026090 2024-04-24T06:48:37.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.758+0000 7f5422000640 1 -- 172.21.15.5:0/3310836220 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f5424056410 con 0x5565f5026090 2024-04-24T06:48:37.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.758+0000 7f543e43e740 1 -- 172.21.15.5:0/3310836220 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5565f4f93700 con 0x5565f5026090 2024-04-24T06:48:37.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.758+0000 7f5422000640 1 -- 172.21.15.5:0/3310836220 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5424066050 con 0x5565f5026090 2024-04-24T06:48:37.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.758+0000 7f543e43e740 10 client.26318 fetch_fsmap learned FSMap version 25 2024-04-24T06:48:37.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.758+0000 7f543e43e740 1 -- 172.21.15.5:0/3310836220 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5565f50a7f30 con 0x5565f5026090 2024-04-24T06:48:37.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.758+0000 7f5422000640 1 -- 172.21.15.5:0/3310836220 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5565f4f93700 con 0x5565f5026090 2024-04-24T06:48:37.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.759+0000 7f5422000640 1 -- 172.21.15.5:0/3310836220 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f54240305c0 con 0x5565f5026090 2024-04-24T06:48:37.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.759+0000 7f543e43e740 10 client.26318 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:48:37.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.759+0000 7f543e43e740 10 client.26318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:48:37.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.759+0000 7f543e43e740 20 client.26318 populate_metadata read hostname 'smithi005' 2024-04-24T06:48:37.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.759+0000 7f543e43e740 1 --2- 172.21.15.5:0/3310836220 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5565f50a86c0 0x5565f50aaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:37.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.759+0000 7f543e43e740 4 client.26318 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T06:48:37.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.759+0000 7f543e43e740 1 -- 172.21.15.5:0/3310836220 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5565f4f0f980 con 0x5565f50a86c0 2024-04-24T06:48:37.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.759+0000 7f5433e00640 1 --2- 172.21.15.5:0/3310836220 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5565f50a86c0 0x5565f50aaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:37.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.759+0000 7f5433e00640 1 --2- 172.21.15.5:0/3310836220 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5565f50a86c0 0x5565f50aaae0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbb80ebef2c3d48c server_cookie=d486c23a645b7ee6 in_seq=0 out_seq=0 2024-04-24T06:48:37.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.759+0000 7f5422000640 10 client.26318 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:48:37.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.761+0000 7f5422000640 1 -- 172.21.15.5:0/3310836220 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5565f4f0f980 con 0x5565f50a86c0 2024-04-24T06:48:37.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.761+0000 7f5422000640 10 client.26318 handle_command_reply: tid=0 2024-04-24T06:48:37.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.761+0000 7f543e43e740 1 client.26318 shutdown 2024-04-24T06:48:37.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.761+0000 7f543e43e740 1 -- 172.21.15.5:0/3310836220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565f5026090 msgr2=0x5565f502aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:37.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.761+0000 7f543e43e740 1 --2- 172.21.15.5:0/3310836220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565f5026090 0x5565f502aeb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5424009ca0 tx=0x7f54240095c0 comp rx=0 tx=0).stop 2024-04-24T06:48:37.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.761+0000 7f543e43e740 1 -- 172.21.15.5:0/3310836220 shutdown_connections 2024-04-24T06:48:37.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.761+0000 7f543e43e740 1 --2- 172.21.15.5:0/3310836220 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5565f50a86c0 0x5565f50aaae0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:37.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.761+0000 7f543e43e740 1 --2- 172.21.15.5:0/3310836220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565f5031080 0x5565f502b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:37.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.761+0000 7f543e43e740 1 --2- 172.21.15.5:0/3310836220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565f50269b0 0x5565f502b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:37.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.761+0000 7f543e43e740 1 --2- 172.21.15.5:0/3310836220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565f5026090 0x5565f502aeb0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:37.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.761+0000 7f5422000640 0 client.26318 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:48:37.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.761+0000 7f543e43e740 1 -- 172.21.15.5:0/3310836220 >> 172.21.15.5:0/3310836220 conn(0x5565f50223b0 msgr2=0x5565f5028110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:37.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.761+0000 7f543e43e740 1 -- 172.21.15.5:0/3310836220 shutdown_connections 2024-04-24T06:48:37.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.761+0000 7f543e43e740 1 -- 172.21.15.5:0/3310836220 wait complete. 2024-04-24T06:48:37.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.761+0000 7f543e43e740 20 client.26318 trim_cache size 0 max 16384 2024-04-24T06:48:37.770 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:48:37.770 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-24T06:48:37.770 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "c1c7c1f0-155a-4cb5-a018-7a30f9b07028", 2024-04-24T06:48:37.770 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-24T06:48:37.770 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:48:37.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.762+0000 7f543b600640 1 -- 172.21.15.5:0/2075280671 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f54180a3d20 msgr2=0x7f54180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:37.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.762+0000 7f543b600640 1 --2- 172.21.15.5:0/2075280671 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f54180a3d20 0x7f54180a61e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5424004af0 tx=0x7f5424000f40 comp rx=0 tx=0).stop 2024-04-24T06:48:37.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.762+0000 7f543b600640 1 -- 172.21.15.5:0/2075280671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f543419fc60 msgr2=0x7f54341a0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:37.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.762+0000 7f543b600640 1 --2- 172.21.15.5:0/2075280671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f543419fc60 0x7f54341a0110 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f542c00e970 tx=0x7f542c00ed70 comp rx=0 tx=0).stop 2024-04-24T06:48:37.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.762+0000 7f543b600640 1 -- 172.21.15.5:0/2075280671 shutdown_connections 2024-04-24T06:48:37.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.762+0000 7f543b600640 1 --2- 172.21.15.5:0/2075280671 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f54180a3d20 0x7f54180a61e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:37.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.762+0000 7f543b600640 1 --2- 172.21.15.5:0/2075280671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f543419fc60 0x7f54341a0110 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:37.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.762+0000 7f543b600640 1 --2- 172.21.15.5:0/2075280671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f543419b3a0 0x7f543419f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:37.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.762+0000 7f543b600640 1 --2- 172.21.15.5:0/2075280671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5434109520 0x7f543419ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:37.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.762+0000 7f543b600640 1 -- 172.21.15.5:0/2075280671 >> 172.21.15.5:0/2075280671 conn(0x7f54340ffc90 msgr2=0x7f5434101030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:37.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.762+0000 7f543b600640 1 -- 172.21.15.5:0/2075280671 shutdown_connections 2024-04-24T06:48:37.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:37.763+0000 7f543b600640 1 -- 172.21.15.5:0/2075280671 wait complete. 2024-04-24T06:48:37.783 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T06:48:37.783 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T06:48:37.852 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:37 smithi005 ceph-mon[27798]: pgmap v140: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:48:37.919 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:37 smithi037 ceph-mon[32864]: pgmap v140: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:48:37.945 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:37 smithi137 ceph-mon[33017]: pgmap v140: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:48:38.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.011+0000 7fa3a6400640 1 -- 172.21.15.5:0/3422185775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3a0109510 msgr2=0x7fa3a01110e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:38.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.011+0000 7fa3a6400640 1 --2- 172.21.15.5:0/3422185775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3a0109510 0x7fa3a01110e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa398009ca0 tx=0x7fa39802a8c0 comp rx=0 tx=0).stop 2024-04-24T06:48:38.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.011+0000 7fa3a6400640 1 -- 172.21.15.5:0/3422185775 shutdown_connections 2024-04-24T06:48:38.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.011+0000 7fa3a6400640 1 --2- 172.21.15.5:0/3422185775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3a0109510 0x7fa3a01110e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:38.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.011+0000 7fa3a6400640 1 --2- 172.21.15.5:0/3422185775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3a0104df0 0x7fa3a0108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:38.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.011+0000 7fa3a6400640 1 --2- 172.21.15.5:0/3422185775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3a0104440 0x7fa3a0104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:38.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.011+0000 7fa3a6400640 1 -- 172.21.15.5:0/3422185775 >> 172.21.15.5:0/3422185775 conn(0x7fa3a00ffc80 msgr2=0x7fa3a01020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:38.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.012+0000 7fa3a6400640 1 -- 172.21.15.5:0/3422185775 shutdown_connections 2024-04-24T06:48:38.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.012+0000 7fa3a6400640 1 -- 172.21.15.5:0/3422185775 wait complete. 2024-04-24T06:48:38.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.013+0000 7fa3a6400640 1 Processor -- start 2024-04-24T06:48:38.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.013+0000 7fa3a6400640 1 -- start start 2024-04-24T06:48:38.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.013+0000 7fa3a6400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3a0104440 0x7fa3a01a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:38.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.013+0000 7fa3a6400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3a0109510 0x7fa3a01a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:38.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.013+0000 7fa3a6400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3a019a980 0x7fa3a019ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:38.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.013+0000 7fa3a6400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a0114600 con 0x7fa3a0109510 2024-04-24T06:48:38.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.013+0000 7fa3a6400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a0114480 con 0x7fa3a0104440 2024-04-24T06:48:38.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.013+0000 7fa3a6400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a0114780 con 0x7fa3a019a980 2024-04-24T06:48:38.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.014+0000 7fa39f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3a0104440 0x7fa3a01a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:38.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.014+0000 7fa39f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3a0104440 0x7fa3a01a0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:34022/0 (socket says 172.21.15.5:34022) 2024-04-24T06:48:38.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.014+0000 7fa39f400640 1 -- 172.21.15.5:0/231312338 learned_addr learned my addr 172.21.15.5:0/231312338 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:38.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.014+0000 7fa39ea00640 1 --2- 172.21.15.5:0/231312338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3a0109510 0x7fa3a01a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:38.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.014+0000 7fa39fe00640 1 --2- 172.21.15.5:0/231312338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3a019a980 0x7fa3a019ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:38.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.014+0000 7fa39f400640 1 -- 172.21.15.5:0/231312338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3a019a980 msgr2=0x7fa3a019ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:38.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.014+0000 7fa39f400640 1 --2- 172.21.15.5:0/231312338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3a019a980 0x7fa3a019ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:38.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.014+0000 7fa39f400640 1 -- 172.21.15.5:0/231312338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3a0109510 msgr2=0x7fa3a01a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:38.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.014+0000 7fa39f400640 1 --2- 172.21.15.5:0/231312338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3a0109510 0x7fa3a01a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:38.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.014+0000 7fa39f400640 1 -- 172.21.15.5:0/231312338 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a019b4a0 con 0x7fa3a0104440 2024-04-24T06:48:38.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.014+0000 7fa39ea00640 1 --2- 172.21.15.5:0/231312338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3a0109510 0x7fa3a01a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:48:38.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.014+0000 7fa39f400640 1 --2- 172.21.15.5:0/231312338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3a0104440 0x7fa3a01a0780 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa39000e9f0 tx=0x7fa39000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:38.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.015+0000 7fa397e00640 1 -- 172.21.15.5:0/231312338 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39000cd80 con 0x7fa3a0104440 2024-04-24T06:48:38.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.015+0000 7fa397e00640 1 -- 172.21.15.5:0/231312338 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa390010910 con 0x7fa3a0104440 2024-04-24T06:48:38.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.015+0000 7fa3a6400640 1 -- 172.21.15.5:0/231312338 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3a019b790 con 0x7fa3a0104440 2024-04-24T06:48:38.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.015+0000 7fa397e00640 1 -- 172.21.15.5:0/231312338 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa390018990 con 0x7fa3a0104440 2024-04-24T06:48:38.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.015+0000 7fa3a6400640 1 -- 172.21.15.5:0/231312338 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3a01a7990 con 0x7fa3a0104440 2024-04-24T06:48:38.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.019+0000 7fa3a93ae740 1 Processor -- start 2024-04-24T06:48:38.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.019+0000 7fa3a93ae740 1 -- start start 2024-04-24T06:48:38.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.019+0000 7fa397e00640 1 -- 172.21.15.5:0/231312338 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fa390010ac0 con 0x7fa3a0104440 2024-04-24T06:48:38.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.020+0000 7fa3a93ae740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620904f1860 0x5620904f1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:38.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.020+0000 7fa3a93ae740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5620904f2180 0x5620904fc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:38.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.020+0000 7fa3a93ae740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620904fc850 0x5620904fec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:38.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.020+0000 7fa3a93ae740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5620904571f0 con 0x5620904fc850 2024-04-24T06:48:38.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.020+0000 7fa3a93ae740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5620903fd350 con 0x5620904f2180 2024-04-24T06:48:38.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.020+0000 7fa3a93ae740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5620904531b0 con 0x5620904f1860 2024-04-24T06:48:38.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.020+0000 7fa39fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620904f1860 0x5620904f1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:38.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.020+0000 7fa39fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620904f1860 0x5620904f1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:55372/0 (socket says 172.21.15.5:55372) 2024-04-24T06:48:38.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.020+0000 7fa39fe00640 1 -- 172.21.15.5:0/498972074 learned_addr learned my addr 172.21.15.5:0/498972074 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:38.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.020+0000 7fa39ea00640 1 --2- 172.21.15.5:0/498972074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620904fc850 0x5620904fec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:38.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.020+0000 7fa39fe00640 1 -- 172.21.15.5:0/498972074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5620904f2180 msgr2=0x5620904fc310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T06:48:38.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.020+0000 7fa39fe00640 1 --2- 172.21.15.5:0/498972074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5620904f2180 0x5620904fc310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:38.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.020+0000 7fa39fe00640 1 -- 172.21.15.5:0/498972074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620904fc850 msgr2=0x5620904fec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:38.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.020+0000 7fa39fe00640 1 --2- 172.21.15.5:0/498972074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620904fc850 0x5620904fec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:38.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.020+0000 7fa39fe00640 1 -- 172.21.15.5:0/498972074 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562090451f50 con 0x5620904f1860 2024-04-24T06:48:38.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.021+0000 7fa39fe00640 1 --2- 172.21.15.5:0/498972074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620904f1860 0x5620904f1c40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa398009870 tx=0x7fa3980098a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c0bc0435ecaaa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:38.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.021+0000 7fa394c00640 1 -- 172.21.15.5:0/498972074 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa398031db0 con 0x5620904f1860 2024-04-24T06:48:38.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.021+0000 7fa394c00640 1 -- 172.21.15.5:0/498972074 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa398036040 con 0x5620904f1860 2024-04-24T06:48:38.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.021+0000 7fa394c00640 1 -- 172.21.15.5:0/498972074 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa398004380 con 0x5620904f1860 2024-04-24T06:48:38.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.021+0000 7fa397e00640 1 --2- 172.21.15.5:0/231312338 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fa3840a3d20 0x7fa3840a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:38.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.022+0000 7fa397e00640 1 -- 172.21.15.5:0/231312338 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fa3900d0ce0 con 0x7fa3a0104440 2024-04-24T06:48:38.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.022+0000 7fa39f400640 1 --2- 172.21.15.5:0/231312338 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fa3840a3d20 0x7fa3840a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:38.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.022+0000 7fa3a93ae740 1 -- 172.21.15.5:0/498972074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620904f1860 msgr2=0x5620904f1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:38.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.022+0000 7fa3a93ae740 1 --2- 172.21.15.5:0/498972074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620904f1860 0x5620904f1c40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa398009870 tx=0x7fa3980098a0 comp rx=0 tx=0).stop 2024-04-24T06:48:38.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.022+0000 7fa3a93ae740 1 -- 172.21.15.5:0/498972074 shutdown_connections 2024-04-24T06:48:38.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.022+0000 7fa3a93ae740 1 --2- 172.21.15.5:0/498972074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620904fc850 0x5620904fec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:38.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.022+0000 7fa3a93ae740 1 --2- 172.21.15.5:0/498972074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5620904f2180 0x5620904fc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:38.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.022+0000 7fa3a93ae740 1 --2- 172.21.15.5:0/498972074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620904f1860 0x5620904f1c40 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:38.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.022+0000 7fa3a93ae740 1 -- 172.21.15.5:0/498972074 >> 172.21.15.5:0/498972074 conn(0x5620904edb80 msgr2=0x5620904edf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:38.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.022+0000 7fa3a93ae740 1 -- 172.21.15.5:0/498972074 shutdown_connections 2024-04-24T06:48:38.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.023+0000 7fa3a93ae740 1 -- 172.21.15.5:0/498972074 wait complete. 2024-04-24T06:48:38.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.023+0000 7fa39f400640 1 --2- 172.21.15.5:0/231312338 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fa3840a3d20 0x7fa3840a61e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x5620904f2e20 tx=0x7fa390009310 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:38.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.023+0000 7fa3a93ae740 1 -- start start 2024-04-24T06:48:38.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.023+0000 7fa3a93ae740 1 Processor -- start 2024-04-24T06:48:38.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.024+0000 7fa3a93ae740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5620904f1860 0x5620904f9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:38.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.024+0000 7fa3a93ae740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620904f2180 0x5620904f9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:38.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.024+0000 7fa3a93ae740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620904fc850 0x5620904fa1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:38.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.024+0000 7fa3a93ae740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5620904ff5e0 con 0x5620904fc850 2024-04-24T06:48:38.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.024+0000 7fa3a93ae740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562090410690 con 0x5620904f1860 2024-04-24T06:48:38.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.024+0000 7fa3a93ae740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5620904ff760 con 0x5620904f2180 2024-04-24T06:48:38.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.024+0000 7fa3a93ae740 10 client.0 _ll_register_callbacks cb 0x56209044c650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:48:38.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.024+0000 7fa39ea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620904f2180 0x5620904f9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:38.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.024+0000 7fa39fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5620904f1860 0x5620904f9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:38.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.024+0000 7fa39ea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620904f2180 0x5620904f9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:55386/0 (socket says 172.21.15.5:55386) 2024-04-24T06:48:38.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.024+0000 7fa39ea00640 1 -- 172.21.15.5:0/4215655037 learned_addr learned my addr 172.21.15.5:0/4215655037 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:38.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.025+0000 7fa39fe00640 1 --2- 172.21.15.5:0/4215655037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620904fc850 0x5620904fa1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:38.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.025+0000 7fa39ea00640 1 -- 172.21.15.5:0/4215655037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5620904f1860 msgr2=0x5620904f9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:38.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.025+0000 7fa39ea00640 1 --2- 172.21.15.5:0/4215655037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5620904f1860 0x5620904f9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:38.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.025+0000 7fa39ea00640 1 -- 172.21.15.5:0/4215655037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620904fc850 msgr2=0x5620904fa1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:38.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.025+0000 7fa39ea00640 1 --2- 172.21.15.5:0/4215655037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620904fc850 0x5620904fa1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:38.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.025+0000 7fa39ea00640 1 -- 172.21.15.5:0/4215655037 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56209040f920 con 0x5620904f2180 2024-04-24T06:48:38.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.025+0000 7fa39fe00640 1 --2- 172.21.15.5:0/4215655037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5620904f1860 0x5620904f9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:48:38.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.025+0000 7fa39ea00640 1 --2- 172.21.15.5:0/4215655037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620904f2180 0x5620904f9c70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa38c013790 tx=0x7fa38c013c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7956db1e856848b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:38.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.025+0000 7fa389600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:48:38.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.025+0000 7fa389600640 1 -- 172.21.15.5:0/4215655037 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38c011dd0 con 0x5620904f2180 2024-04-24T06:48:38.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.026+0000 7fa389600640 1 -- 172.21.15.5:0/4215655037 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fa38c004020 con 0x5620904f2180 2024-04-24T06:48:38.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.026+0000 7fa3a93ae740 1 -- 172.21.15.5:0/4215655037 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa398036040 con 0x5620904f2180 2024-04-24T06:48:38.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.026+0000 7fa389600640 1 -- 172.21.15.5:0/4215655037 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa38c004220 con 0x5620904f2180 2024-04-24T06:48:38.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.026+0000 7fa3a93ae740 10 client.26341 fetch_fsmap learned FSMap version 25 2024-04-24T06:48:38.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.026+0000 7fa3a93ae740 1 -- 172.21.15.5:0/4215655037 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5620902337d0 con 0x5620904f2180 2024-04-24T06:48:38.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.026+0000 7fa389600640 1 -- 172.21.15.5:0/4215655037 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa398036040 con 0x5620904f2180 2024-04-24T06:48:38.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.026+0000 7fa389600640 1 -- 172.21.15.5:0/4215655037 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa38c01a030 con 0x5620904f2180 2024-04-24T06:48:38.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.027+0000 7fa3a93ae740 10 client.26341 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:48:38.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.027+0000 7fa3a93ae740 10 client.26341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:48:38.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.027+0000 7fa3a93ae740 20 client.26341 populate_metadata read hostname 'smithi005' 2024-04-24T06:48:38.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.027+0000 7fa3a93ae740 1 --2- 172.21.15.5:0/4215655037 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x56209050cc00 0x56209050f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:38.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.027+0000 7fa3a93ae740 4 client.26341 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:48:38.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.027+0000 7fa3a93ae740 1 -- 172.21.15.5:0/4215655037 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562090464c10 con 0x56209050cc00 2024-04-24T06:48:38.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.027+0000 7fa39fe00640 1 --2- 172.21.15.5:0/4215655037 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x56209050cc00 0x56209050f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:38.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.028+0000 7fa39fe00640 1 --2- 172.21.15.5:0/4215655037 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x56209050cc00 0x56209050f020 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6bfd56b9dad0fee server_cookie=7cb7db27a24c99bd in_seq=0 out_seq=0 2024-04-24T06:48:38.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.028+0000 7fa389600640 10 client.26341 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:48:38.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.028+0000 7fa389600640 1 -- 172.21.15.5:0/4215655037 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562090464c10 con 0x56209050cc00 2024-04-24T06:48:38.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.028+0000 7fa389600640 10 client.26341 handle_command_reply: tid=0 2024-04-24T06:48:38.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.029+0000 7fa3a93ae740 1 client.26341 shutdown 2024-04-24T06:48:38.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.029+0000 7fa3a93ae740 1 -- 172.21.15.5:0/4215655037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620904f2180 msgr2=0x5620904f9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:38.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.029+0000 7fa3a93ae740 1 --2- 172.21.15.5:0/4215655037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620904f2180 0x5620904f9c70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa38c013790 tx=0x7fa38c013c50 comp rx=0 tx=0).stop 2024-04-24T06:48:38.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.029+0000 7fa3a93ae740 1 -- 172.21.15.5:0/4215655037 shutdown_connections 2024-04-24T06:48:38.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.029+0000 7fa3a93ae740 1 --2- 172.21.15.5:0/4215655037 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x56209050cc00 0x56209050f020 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:38.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.029+0000 7fa3a93ae740 1 --2- 172.21.15.5:0/4215655037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620904fc850 0x5620904fa1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:38.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.029+0000 7fa3a93ae740 1 --2- 172.21.15.5:0/4215655037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620904f2180 0x5620904f9c70 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:38.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.029+0000 7fa3a93ae740 1 --2- 172.21.15.5:0/4215655037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5620904f1860 0x5620904f9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:38.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.029+0000 7fa389600640 0 client.26341 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:48:38.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.029+0000 7fa3a93ae740 1 -- 172.21.15.5:0/4215655037 >> 172.21.15.5:0/4215655037 conn(0x5620904edb80 msgr2=0x5620904f3980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:38.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.029+0000 7fa3a93ae740 1 -- 172.21.15.5:0/4215655037 shutdown_connections 2024-04-24T06:48:38.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.029+0000 7fa3a93ae740 1 -- 172.21.15.5:0/4215655037 wait complete. 2024-04-24T06:48:38.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.030+0000 7fa3a93ae740 20 client.26341 trim_cache size 0 max 16384 2024-04-24T06:48:38.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.055+0000 7fa3a93ae740 1 Processor -- start 2024-04-24T06:48:38.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.055+0000 7fa3a93ae740 1 -- start start 2024-04-24T06:48:38.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.056+0000 7fa3a93ae740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620904f1860 0x5620905084d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:38.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.056+0000 7fa3a93ae740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5620904f2180 0x5620904f8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:38.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.056+0000 7fa3a93ae740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620904fc850 0x5620904ecaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:38.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.056+0000 7fa3a93ae740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a01b28a0 con 0x5620904f1860 2024-04-24T06:48:38.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.056+0000 7fa3a93ae740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a01b0170 con 0x5620904f2180 2024-04-24T06:48:38.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.056+0000 7fa3a93ae740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a01adfe0 con 0x5620904fc850 2024-04-24T06:48:38.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.056+0000 7fa39fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620904f1860 0x5620905084d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:38.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.056+0000 7fa39ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5620904f2180 0x5620904f8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:38.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.056+0000 7fa39ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5620904f2180 0x5620904f8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:34026/0 (socket says 172.21.15.5:34026) 2024-04-24T06:48:38.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.056+0000 7fa39fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620904f1860 0x5620905084d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56208/0 (socket says 172.21.15.5:56208) 2024-04-24T06:48:38.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.056+0000 7fa39ea00640 1 -- 172.21.15.5:0/2638578273 learned_addr learned my addr 172.21.15.5:0/2638578273 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:38.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.056+0000 7fa39fe00640 1 --2- 172.21.15.5:0/2638578273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620904fc850 0x5620904ecaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:38.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.057+0000 7fa39fe00640 1 -- 172.21.15.5:0/2638578273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620904fc850 msgr2=0x5620904ecaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:38.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.057+0000 7fa39fe00640 1 --2- 172.21.15.5:0/2638578273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620904fc850 0x5620904ecaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:38.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.057+0000 7fa39fe00640 1 -- 172.21.15.5:0/2638578273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5620904f2180 msgr2=0x5620904f8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:38.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.057+0000 7fa39fe00640 1 --2- 172.21.15.5:0/2638578273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5620904f2180 0x5620904f8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:38.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.057+0000 7fa39fe00640 1 -- 172.21.15.5:0/2638578273 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620901693f0 con 0x5620904f1860 2024-04-24T06:48:38.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.057+0000 7fa39fe00640 1 --2- 172.21.15.5:0/2638578273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620904f1860 0x5620905084d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa398009bf0 tx=0x7fa398009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15050c69a8cfea35 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:38.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.057+0000 7fa38be00640 1 -- 172.21.15.5:0/2638578273 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa398031590 con 0x5620904f1860 2024-04-24T06:48:38.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.057+0000 7fa38be00640 1 -- 172.21.15.5:0/2638578273 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa398036040 con 0x5620904f1860 2024-04-24T06:48:38.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.057+0000 7fa38be00640 1 -- 172.21.15.5:0/2638578273 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa398004760 con 0x5620904f1860 2024-04-24T06:48:38.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.058+0000 7fa3a93ae740 1 -- 172.21.15.5:0/2638578273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620904f1860 msgr2=0x5620905084d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:38.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.058+0000 7fa3a93ae740 1 --2- 172.21.15.5:0/2638578273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620904f1860 0x5620905084d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa398009bf0 tx=0x7fa398009800 comp rx=0 tx=0).stop 2024-04-24T06:48:38.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.058+0000 7fa3a93ae740 1 -- 172.21.15.5:0/2638578273 shutdown_connections 2024-04-24T06:48:38.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.058+0000 7fa3a93ae740 1 --2- 172.21.15.5:0/2638578273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620904fc850 0x5620904ecaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:38.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.058+0000 7fa3a93ae740 1 --2- 172.21.15.5:0/2638578273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5620904f2180 0x5620904f8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:38.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.058+0000 7fa3a93ae740 1 --2- 172.21.15.5:0/2638578273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620904f1860 0x5620905084d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:38.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.058+0000 7fa3a93ae740 1 -- 172.21.15.5:0/2638578273 >> 172.21.15.5:0/2638578273 conn(0x5620904edb80 msgr2=0x5620904ec390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:38.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.059+0000 7fa3a93ae740 1 -- 172.21.15.5:0/2638578273 shutdown_connections 2024-04-24T06:48:38.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.059+0000 7fa3a93ae740 1 -- 172.21.15.5:0/2638578273 wait complete. 2024-04-24T06:48:38.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.059+0000 7fa3a93ae740 1 -- start start 2024-04-24T06:48:38.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.059+0000 7fa3a93ae740 1 Processor -- start 2024-04-24T06:48:38.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.060+0000 7fa3a93ae740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5620904f1860 0x56209056b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:38.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.060+0000 7fa3a93ae740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620904f2180 0x56209056ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:38.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.060+0000 7fa3a93ae740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620904fc850 0x56209056bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:38.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.060+0000 7fa3a93ae740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a01ae630 con 0x5620904f2180 2024-04-24T06:48:38.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.060+0000 7fa3a93ae740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a01afaa0 con 0x5620904f1860 2024-04-24T06:48:38.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.060+0000 7fa3a93ae740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a01aed00 con 0x5620904fc850 2024-04-24T06:48:38.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.060+0000 7fa3a93ae740 10 client.0 _ll_register_callbacks cb 0x7fa38c009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:48:38.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.060+0000 7fa39fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5620904f1860 0x56209056b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:38.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.060+0000 7fa39fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5620904f1860 0x56209056b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:34036/0 (socket says 172.21.15.5:34036) 2024-04-24T06:48:38.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.060+0000 7fa39fe00640 1 -- 172.21.15.5:0/2329070698 learned_addr learned my addr 172.21.15.5:0/2329070698 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:48:38.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.061+0000 7fa39fe00640 1 --2- 172.21.15.5:0/2329070698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620904fc850 0x56209056bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:38.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.061+0000 7fa39ea00640 1 --2- 172.21.15.5:0/2329070698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620904f2180 0x56209056ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:38.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.061+0000 7fa39fe00640 1 -- 172.21.15.5:0/2329070698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620904fc850 msgr2=0x56209056bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:38.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.061+0000 7fa39fe00640 1 --2- 172.21.15.5:0/2329070698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620904fc850 0x56209056bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:38.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.061+0000 7fa39fe00640 1 -- 172.21.15.5:0/2329070698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620904f2180 msgr2=0x56209056ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:48:38.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.061+0000 7fa39fe00640 1 --2- 172.21.15.5:0/2329070698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620904f2180 0x56209056ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:38.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.061+0000 7fa39fe00640 1 -- 172.21.15.5:0/2329070698 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620901290f0 con 0x5620904f1860 2024-04-24T06:48:38.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.061+0000 7fa39fe00640 1 --2- 172.21.15.5:0/2329070698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5620904f1860 0x56209056b4d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa398009a60 tx=0x7fa39804a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=607f096ae68be382 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:48:38.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.061+0000 7fa389600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:48:38.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.061+0000 7fa389600640 1 -- 172.21.15.5:0/2329070698 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39805a070 con 0x5620904f1860 2024-04-24T06:48:38.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.062+0000 7fa3a93ae740 1 -- 172.21.15.5:0/2329070698 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa398036040 con 0x5620904f1860 2024-04-24T06:48:38.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.062+0000 7fa389600640 1 -- 172.21.15.5:0/2329070698 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fa398004b40 con 0x5620904f1860 2024-04-24T06:48:38.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.062+0000 7fa389600640 1 -- 172.21.15.5:0/2329070698 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa3980042b0 con 0x5620904f1860 2024-04-24T06:48:38.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.062+0000 7fa3a93ae740 10 client.26333 fetch_fsmap learned FSMap version 25 2024-04-24T06:48:38.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.062+0000 7fa3a93ae740 1 -- 172.21.15.5:0/2329070698 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56209041c810 con 0x5620904f1860 2024-04-24T06:48:38.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.062+0000 7fa389600640 1 -- 172.21.15.5:0/2329070698 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa398036040 con 0x5620904f1860 2024-04-24T06:48:38.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.062+0000 7fa389600640 1 -- 172.21.15.5:0/2329070698 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa39802f710 con 0x5620904f1860 2024-04-24T06:48:38.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.063+0000 7fa3a93ae740 10 client.26333 fetch_fsmap finished waiting for FSMap version 25 2024-04-24T06:48:38.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.063+0000 7fa3a93ae740 10 client.26333 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:48:38.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.063+0000 7fa3a93ae740 20 client.26333 populate_metadata read hostname 'smithi005' 2024-04-24T06:48:38.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.063+0000 7fa3a93ae740 1 --2- 172.21.15.5:0/2329070698 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x562090504e00 0x562090586180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:48:38.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.063+0000 7fa3a93ae740 4 client.26333 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:48:38.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.063+0000 7fa3a93ae740 1 -- 172.21.15.5:0/2329070698 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56209045ea80 con 0x562090504e00 2024-04-24T06:48:38.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.063+0000 7fa39ea00640 1 --2- 172.21.15.5:0/2329070698 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x562090504e00 0x562090586180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:48:38.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.064+0000 7fa39ea00640 1 --2- 172.21.15.5:0/2329070698 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x562090504e00 0x562090586180 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d8e9e7e6655a0a server_cookie=829d4aaf3eb1751b in_seq=0 out_seq=0 2024-04-24T06:48:38.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.064+0000 7fa389600640 10 client.26333 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:48:38.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.064+0000 7fa389600640 1 -- 172.21.15.5:0/2329070698 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x56209045ea80 con 0x562090504e00 2024-04-24T06:48:38.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.064+0000 7fa389600640 10 client.26333 handle_command_reply: tid=0 2024-04-24T06:48:38.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.064+0000 7fa3a93ae740 1 client.26333 shutdown 2024-04-24T06:48:38.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.065+0000 7fa3a93ae740 1 -- 172.21.15.5:0/2329070698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5620904f1860 msgr2=0x56209056b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:38.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.065+0000 7fa3a93ae740 1 --2- 172.21.15.5:0/2329070698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5620904f1860 0x56209056b4d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa398009a60 tx=0x7fa39804a040 comp rx=0 tx=0).stop 2024-04-24T06:48:38.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.065+0000 7fa3a93ae740 1 -- 172.21.15.5:0/2329070698 shutdown_connections 2024-04-24T06:48:38.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.065+0000 7fa3a93ae740 1 --2- 172.21.15.5:0/2329070698 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x562090504e00 0x562090586180 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:38.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.065+0000 7fa3a93ae740 1 --2- 172.21.15.5:0/2329070698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620904fc850 0x56209056bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:38.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.065+0000 7fa3a93ae740 1 --2- 172.21.15.5:0/2329070698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620904f2180 0x56209056ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:38.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.065+0000 7fa3a93ae740 1 --2- 172.21.15.5:0/2329070698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5620904f1860 0x56209056b4d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:38.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.065+0000 7fa389600640 0 client.26333 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:48:38.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.065+0000 7fa3a93ae740 1 -- 172.21.15.5:0/2329070698 >> 172.21.15.5:0/2329070698 conn(0x5620904edb80 msgr2=0x5620904f8710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:38.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.065+0000 7fa3a93ae740 1 -- 172.21.15.5:0/2329070698 shutdown_connections 2024-04-24T06:48:38.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.065+0000 7fa3a93ae740 1 -- 172.21.15.5:0/2329070698 wait complete. 2024-04-24T06:48:38.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.065+0000 7fa3a93ae740 20 client.26333 trim_cache size 0 max 16384 2024-04-24T06:48:38.073 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:48:38.073 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (923 inodes in the stack)", 2024-04-24T06:48:38.073 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-24T06:48:38.073 INFO:teuthology.orchestra.run.smithi005.stdout: "c1c7c1f0-155a-4cb5-a018-7a30f9b07028": { 2024-04-24T06:48:38.073 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-24T06:48:38.073 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "c1c7c1f0-155a-4cb5-a018-7a30f9b07028", 2024-04-24T06:48:38.073 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-04-24T06:48:38.073 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:48:38.074 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:48:38.074 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:48:38.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.066+0000 7fa3a6400640 1 -- 172.21.15.5:0/231312338 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fa3840a3d20 msgr2=0x7fa3840a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:38.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.066+0000 7fa3a6400640 1 --2- 172.21.15.5:0/231312338 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fa3840a3d20 0x7fa3840a61e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x5620904f2e20 tx=0x7fa390009310 comp rx=0 tx=0).stop 2024-04-24T06:48:38.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.066+0000 7fa3a6400640 1 -- 172.21.15.5:0/231312338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3a0104440 msgr2=0x7fa3a01a0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:48:38.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.066+0000 7fa3a6400640 1 --2- 172.21.15.5:0/231312338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3a0104440 0x7fa3a01a0780 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa39000e9f0 tx=0x7fa39000eeb0 comp rx=0 tx=0).stop 2024-04-24T06:48:38.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.067+0000 7fa3a6400640 1 -- 172.21.15.5:0/231312338 shutdown_connections 2024-04-24T06:48:38.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.067+0000 7fa3a6400640 1 --2- 172.21.15.5:0/231312338 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fa3840a3d20 0x7fa3840a61e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:38.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.067+0000 7fa3a6400640 1 --2- 172.21.15.5:0/231312338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3a019a980 0x7fa3a019ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:38.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.067+0000 7fa3a6400640 1 --2- 172.21.15.5:0/231312338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3a0109510 0x7fa3a01a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:38.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.067+0000 7fa3a6400640 1 --2- 172.21.15.5:0/231312338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3a0104440 0x7fa3a01a0780 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:48:38.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.067+0000 7fa3a6400640 1 -- 172.21.15.5:0/231312338 >> 172.21.15.5:0/231312338 conn(0x7fa3a00ffc80 msgr2=0x7fa3a0101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:48:38.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.067+0000 7fa3a6400640 1 -- 172.21.15.5:0/231312338 shutdown_connections 2024-04-24T06:48:38.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:48:38.067+0000 7fa3a6400640 1 -- 172.21.15.5:0/231312338 wait complete. 2024-04-24T06:48:38.088 INFO:tasks.cephfs.filesystem:scrub status for tag:c1c7c1f0-155a-4cb5-a018-7a30f9b07028 - {'path': '/', 'tag': 'c1c7c1f0-155a-4cb5-a018-7a30f9b07028', 'options': 'recursive,force'} 2024-04-24T06:48:38.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:38 smithi137 ceph-mon[33017]: scrub queued for path: / 2024-04-24T06:48:38.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:38 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:48:38.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:38 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:48:38.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:38 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:48:38.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:38 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:48:38.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:38 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:48:38.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:38 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:48:38.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:38 smithi137 ceph-mon[33017]: pgmap v141: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:48:39.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:38 smithi005 ceph-mon[27798]: scrub queued for path: / 2024-04-24T06:48:39.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:38 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:48:39.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:38 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:48:39.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:38 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:48:39.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:38 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:48:39.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:38 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:48:39.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:38 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:48:39.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:38 smithi005 ceph-mon[27798]: pgmap v141: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:48:39.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:38 smithi037 ceph-mon[32864]: scrub queued for path: / 2024-04-24T06:48:39.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:38 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:48:39.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:38 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:48:39.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:38 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:48:39.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:38 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:48:39.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:38 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:48:39.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:38 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:48:39.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:38 smithi037 ceph-mon[32864]: pgmap v141: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:48:39.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:39 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:48:39.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:39 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:48:39.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:39 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:48:39.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:39 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:48:39.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:39 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:48:40.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:39 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:48:40.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:39 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:48:40.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:39 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:48:40.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:39 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:48:40.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:39 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:48:40.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:39 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:48:40.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:39 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:48:40.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:39 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:48:40.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:39 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:48:40.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:39 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:48:40.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:40 smithi137 ceph-mon[33017]: pgmap v142: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-04-24T06:48:41.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:40 smithi005 ceph-mon[27798]: pgmap v142: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-04-24T06:48:41.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:40 smithi037 ceph-mon[32864]: pgmap v142: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-04-24T06:48:43.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:43 smithi137 ceph-mon[33017]: pgmap v143: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:48:43.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:43 smithi005 ceph-mon[27798]: pgmap v143: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:48:43.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:43 smithi037 ceph-mon[32864]: pgmap v143: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:48:45.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:45 smithi137 ceph-mon[33017]: pgmap v144: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:48:45.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:45 smithi005 ceph-mon[27798]: pgmap v144: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:48:45.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:45 smithi037 ceph-mon[32864]: pgmap v144: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:48:47.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:47 smithi137 ceph-mon[33017]: pgmap v145: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:48:47.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:47 smithi005 ceph-mon[27798]: pgmap v145: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:48:47.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:47 smithi037 ceph-mon[32864]: pgmap v145: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:48:49.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:49 smithi137 ceph-mon[33017]: pgmap v146: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:48:49.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:49 smithi005 ceph-mon[27798]: pgmap v146: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:48:49.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:49 smithi037 ceph-mon[32864]: pgmap v146: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:48:51.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:51 smithi005 ceph-mon[27798]: pgmap v147: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-04-24T06:48:51.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:51 smithi037 ceph-mon[32864]: pgmap v147: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-04-24T06:48:51.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:51 smithi137 ceph-mon[33017]: pgmap v147: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-04-24T06:48:53.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:53 smithi005 ceph-mon[27798]: pgmap v148: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:48:53.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:53 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:48:53.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:53 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:48:53.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:53 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:48:53.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:53 smithi037 ceph-mon[32864]: pgmap v148: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:48:53.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:53 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:48:53.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:53 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:48:53.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:53 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:48:53.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:53 smithi137 ceph-mon[33017]: pgmap v148: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:48:53.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:53 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:48:53.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:53 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:48:53.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:53 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:48:54.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:54 smithi005 ceph-mon[27798]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T06:48:54.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:54 smithi037 ceph-mon[32864]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T06:48:54.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:54 smithi137 ceph-mon[33017]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T06:48:55.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:55 smithi005 ceph-mon[27798]: pgmap v149: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:48:55.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:55 smithi037 ceph-mon[32864]: pgmap v149: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:48:55.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:55 smithi137 ceph-mon[33017]: pgmap v149: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:48:57.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:57 smithi005 ceph-mon[27798]: pgmap v150: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:48:57.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:57 smithi037 ceph-mon[32864]: pgmap v150: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:48:57.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:57 smithi137 ceph-mon[33017]: pgmap v150: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:48:59.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:59 smithi005 ceph-mon[27798]: pgmap v151: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:48:59.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:48:59 smithi005 ceph-mon[27798]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T06:48:59.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:59 smithi037 ceph-mon[32864]: pgmap v151: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:48:59.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:48:59 smithi037 ceph-mon[32864]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T06:48:59.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:59 smithi137 ceph-mon[33017]: pgmap v151: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:48:59.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:48:59 smithi137 ceph-mon[33017]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T06:49:01.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:01 smithi005 ceph-mon[27798]: pgmap v152: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T06:49:01.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:01 smithi037 ceph-mon[32864]: pgmap v152: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T06:49:01.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:01 smithi137 ceph-mon[33017]: pgmap v152: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T06:49:03.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:03 smithi037 ceph-mon[32864]: pgmap v153: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:49:03.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:03 smithi137 ceph-mon[33017]: pgmap v153: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:49:03.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:03 smithi005 ceph-mon[27798]: pgmap v153: 97 pgs: 97 active+clean; 88 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:49:05.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:05 smithi037 ceph-mon[32864]: pgmap v154: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-24T06:49:05.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:05 smithi137 ceph-mon[33017]: pgmap v154: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-24T06:49:05.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:05 smithi005 ceph-mon[27798]: pgmap v154: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-24T06:49:07.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:07 smithi037 ceph-mon[32864]: pgmap v155: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-24T06:49:07.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:07 smithi137 ceph-mon[33017]: pgmap v155: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-24T06:49:07.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:07 smithi005 ceph-mon[27798]: pgmap v155: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-24T06:49:08.090 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T06:49:08.090 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T06:49:08.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.323+0000 7f1ea6600640 1 -- 172.21.15.5:0/3999175770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ea010e4f0 msgr2=0x7f1ea010e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:08.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.323+0000 7f1ea6600640 1 --2- 172.21.15.5:0/3999175770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ea010e4f0 0x7f1ea010e8d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1e90009ca0 tx=0x7f1e9002a950 comp rx=0 tx=0).stop 2024-04-24T06:49:08.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.323+0000 7f1ea6600640 1 -- 172.21.15.5:0/3999175770 shutdown_connections 2024-04-24T06:49:08.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.324+0000 7f1ea6600640 1 --2- 172.21.15.5:0/3999175770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ea0108ee0 0x7f1ea01152b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.324+0000 7f1ea6600640 1 --2- 172.21.15.5:0/3999175770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ea0108540 0x7f1ea01089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.324+0000 7f1ea6600640 1 --2- 172.21.15.5:0/3999175770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ea010e4f0 0x7f1ea010e8d0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.324+0000 7f1ea6600640 1 -- 172.21.15.5:0/3999175770 >> 172.21.15.5:0/3999175770 conn(0x7f1ea0077e10 msgr2=0x7f1ea0106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:08.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.324+0000 7f1ea6600640 1 -- 172.21.15.5:0/3999175770 shutdown_connections 2024-04-24T06:49:08.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.324+0000 7f1ea6600640 1 -- 172.21.15.5:0/3999175770 wait complete. 2024-04-24T06:49:08.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.325+0000 7f1ea6600640 1 Processor -- start 2024-04-24T06:49:08.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.325+0000 7f1ea6600640 1 -- start start 2024-04-24T06:49:08.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.325+0000 7f1ea6600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ea0108540 0x7f1ea019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:08.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.325+0000 7f1ea6600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ea0108ee0 0x7f1ea019f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:08.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.325+0000 7f1ea6600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ea010e4f0 0x7f1ea01a3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:08.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.325+0000 7f1ea6600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea01187c0 con 0x7f1ea010e4f0 2024-04-24T06:49:08.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.325+0000 7f1ea6600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea0118640 con 0x7f1ea0108ee0 2024-04-24T06:49:08.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.325+0000 7f1ea6600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea0118940 con 0x7f1ea0108540 2024-04-24T06:49:08.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.325+0000 7f1ea5c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ea010e4f0 0x7f1ea01a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:08.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.326+0000 7f1ea5c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ea010e4f0 0x7f1ea01a3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32780/0 (socket says 172.21.15.5:32780) 2024-04-24T06:49:08.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.326+0000 7f1ea5c00640 1 -- 172.21.15.5:0/3113514849 learned_addr learned my addr 172.21.15.5:0/3113514849 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:08.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.326+0000 7f1ea5200640 1 --2- 172.21.15.5:0/3113514849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ea0108540 0x7f1ea019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:08.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.326+0000 7f1e9fe00640 1 --2- 172.21.15.5:0/3113514849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ea0108ee0 0x7f1ea019f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:08.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.326+0000 7f1ea5c00640 1 -- 172.21.15.5:0/3113514849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ea0108540 msgr2=0x7f1ea019ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:08.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.326+0000 7f1ea5c00640 1 --2- 172.21.15.5:0/3113514849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ea0108540 0x7f1ea019ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.326+0000 7f1ea5c00640 1 -- 172.21.15.5:0/3113514849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ea0108ee0 msgr2=0x7f1ea019f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:08.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.326+0000 7f1ea5c00640 1 --2- 172.21.15.5:0/3113514849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ea0108ee0 0x7f1ea019f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.326+0000 7f1ea5c00640 1 -- 172.21.15.5:0/3113514849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea01a3d70 con 0x7f1ea010e4f0 2024-04-24T06:49:08.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.326+0000 7f1ea5c00640 1 --2- 172.21.15.5:0/3113514849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ea010e4f0 0x7f1ea01a3640 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1e9800e970 tx=0x7f1e9800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:08.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.327+0000 7f1e9d600640 1 -- 172.21.15.5:0/3113514849 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9800ccc0 con 0x7f1ea010e4f0 2024-04-24T06:49:08.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.327+0000 7f1e9d600640 1 -- 172.21.15.5:0/3113514849 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e9800ce60 con 0x7f1ea010e4f0 2024-04-24T06:49:08.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.327+0000 7f1ea6600640 1 -- 172.21.15.5:0/3113514849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ea01a4060 con 0x7f1ea010e4f0 2024-04-24T06:49:08.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.327+0000 7f1e9d600640 1 -- 172.21.15.5:0/3113514849 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e98010490 con 0x7f1ea010e4f0 2024-04-24T06:49:08.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.327+0000 7f1ea6600640 1 -- 172.21.15.5:0/3113514849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ea01ab8f0 con 0x7f1ea010e4f0 2024-04-24T06:49:08.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.331+0000 7f1eaa9ef740 1 Processor -- start 2024-04-24T06:49:08.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.331+0000 7f1e9d600640 1 -- 172.21.15.5:0/3113514849 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f1e98005020 con 0x7f1ea010e4f0 2024-04-24T06:49:08.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.332+0000 7f1eaa9ef740 1 -- start start 2024-04-24T06:49:08.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.332+0000 7f1eaa9ef740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a009b5ea30 0x55a009b5ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:08.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.332+0000 7f1eaa9ef740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a009b5f350 0x55a009b694e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:08.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.332+0000 7f1eaa9ef740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a009b69a20 0x55a009b6be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:08.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.332+0000 7f1eaa9ef740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a009ac41f0 con 0x55a009b5ea30 2024-04-24T06:49:08.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.332+0000 7f1eaa9ef740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a009a6a350 con 0x55a009b69a20 2024-04-24T06:49:08.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.332+0000 7f1eaa9ef740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a009ac01b0 con 0x55a009b5f350 2024-04-24T06:49:08.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.332+0000 7f1e9fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a009b5ea30 0x55a009b5ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.333+0000 7f1e9fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a009b5ea30 0x55a009b5ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32794/0 (socket says 172.21.15.5:32794) 2024-04-24T06:49:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.333+0000 7f1e9fe00640 1 -- 172.21.15.5:0/1491732963 learned_addr learned my addr 172.21.15.5:0/1491732963 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.333+0000 7f1ea5200640 1 --2- 172.21.15.5:0/1491732963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a009b5f350 0x55a009b694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.333+0000 7f1e9fe00640 1 --2- 172.21.15.5:0/1491732963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a009b69a20 0x55a009b6be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.333+0000 7f1e9d600640 1 --2- 172.21.15.5:0/3113514849 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1e840a3d20 0x7f1e840a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.333+0000 7f1ea5200640 1 -- 172.21.15.5:0/1491732963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a009b69a20 msgr2=0x55a009b6be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.333+0000 7f1ea5200640 1 --2- 172.21.15.5:0/1491732963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a009b69a20 0x55a009b6be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.333+0000 7f1ea5200640 1 -- 172.21.15.5:0/1491732963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a009b5ea30 msgr2=0x55a009b5ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.333+0000 7f1ea5200640 1 --2- 172.21.15.5:0/1491732963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a009b5ea30 0x55a009b5ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.333+0000 7f1ea5200640 1 -- 172.21.15.5:0/1491732963 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a009abef50 con 0x55a009b5f350 2024-04-24T06:49:08.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.333+0000 7f1e9d600640 1 -- 172.21.15.5:0/3113514849 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f1e980d0b80 con 0x7f1ea010e4f0 2024-04-24T06:49:08.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.333+0000 7f1ea5c00640 1 --2- 172.21.15.5:0/3113514849 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1e840a3d20 0x7f1e840a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:08.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.333+0000 7f1ea5200640 1 --2- 172.21.15.5:0/1491732963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a009b5f350 0x55a009b694e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1e9002ae70 tx=0x7f1e90009820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0cb35bc773f6e77 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:08.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.334+0000 7f1e96000640 1 -- 172.21.15.5:0/1491732963 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e90048070 con 0x55a009b5f350 2024-04-24T06:49:08.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.334+0000 7f1e96000640 1 -- 172.21.15.5:0/1491732963 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e9002f930 con 0x55a009b5f350 2024-04-24T06:49:08.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.334+0000 7f1e96000640 1 -- 172.21.15.5:0/1491732963 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e90034040 con 0x55a009b5f350 2024-04-24T06:49:08.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.335+0000 7f1eaa9ef740 1 -- 172.21.15.5:0/1491732963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a009b5f350 msgr2=0x55a009b694e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:08.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.335+0000 7f1ea5c00640 1 --2- 172.21.15.5:0/3113514849 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1e840a3d20 0x7f1e840a61e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1e9809cac0 tx=0x7f1e98009720 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:08.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.335+0000 7f1eaa9ef740 1 --2- 172.21.15.5:0/1491732963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a009b5f350 0x55a009b694e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1e9002ae70 tx=0x7f1e90009820 comp rx=0 tx=0).stop 2024-04-24T06:49:08.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.335+0000 7f1eaa9ef740 1 -- 172.21.15.5:0/1491732963 shutdown_connections 2024-04-24T06:49:08.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.335+0000 7f1eaa9ef740 1 --2- 172.21.15.5:0/1491732963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a009b69a20 0x55a009b6be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.335+0000 7f1eaa9ef740 1 --2- 172.21.15.5:0/1491732963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a009b5f350 0x55a009b694e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.335+0000 7f1eaa9ef740 1 --2- 172.21.15.5:0/1491732963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a009b5ea30 0x55a009b5ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.335+0000 7f1eaa9ef740 1 -- 172.21.15.5:0/1491732963 >> 172.21.15.5:0/1491732963 conn(0x55a009b5ad50 msgr2=0x55a009b5b130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:08.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.335+0000 7f1eaa9ef740 1 -- 172.21.15.5:0/1491732963 shutdown_connections 2024-04-24T06:49:08.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.335+0000 7f1eaa9ef740 1 -- 172.21.15.5:0/1491732963 wait complete. 2024-04-24T06:49:08.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.335+0000 7f1eaa9ef740 1 -- start start 2024-04-24T06:49:08.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.336+0000 7f1eaa9ef740 1 Processor -- start 2024-04-24T06:49:08.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.336+0000 7f1eaa9ef740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a009b5ea30 0x55a009b66a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:08.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.336+0000 7f1eaa9ef740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a009b5f350 0x55a009b66f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:08.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.336+0000 7f1eaa9ef740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a009b69a20 0x55a009b67480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:08.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.336+0000 7f1eaa9ef740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a009b6c7b0 con 0x55a009b5f350 2024-04-24T06:49:08.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.336+0000 7f1eaa9ef740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a009a7d690 con 0x55a009b69a20 2024-04-24T06:49:08.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.336+0000 7f1eaa9ef740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a009b6c930 con 0x55a009b5ea30 2024-04-24T06:49:08.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.336+0000 7f1eaa9ef740 10 client.0 _ll_register_callbacks cb 0x55a009ab9650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:49:08.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.336+0000 7f1ea5200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a009b5f350 0x55a009b66f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:08.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.337+0000 7f1ea5200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a009b5f350 0x55a009b66f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32810/0 (socket says 172.21.15.5:32810) 2024-04-24T06:49:08.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.337+0000 7f1ea5200640 1 -- 172.21.15.5:0/1246642945 learned_addr learned my addr 172.21.15.5:0/1246642945 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:08.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.337+0000 7f1e9fe00640 1 --2- 172.21.15.5:0/1246642945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a009b5ea30 0x55a009b66a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:08.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.337+0000 7f1e9fe00640 1 --2- 172.21.15.5:0/1246642945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a009b69a20 0x55a009b67480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:08.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.337+0000 7f1ea5200640 1 -- 172.21.15.5:0/1246642945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a009b5ea30 msgr2=0x55a009b66a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:08.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.337+0000 7f1ea5200640 1 --2- 172.21.15.5:0/1246642945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a009b5ea30 0x55a009b66a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.337+0000 7f1ea5200640 1 -- 172.21.15.5:0/1246642945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a009b69a20 msgr2=0x55a009b67480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:08.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.337+0000 7f1ea5200640 1 --2- 172.21.15.5:0/1246642945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a009b69a20 0x55a009b67480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.337+0000 7f1ea5200640 1 -- 172.21.15.5:0/1246642945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a009a7c920 con 0x55a009b5f350 2024-04-24T06:49:08.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.337+0000 7f1e9fe00640 1 --2- 172.21.15.5:0/1246642945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a009b69a20 0x55a009b67480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:49:08.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.337+0000 7f1ea5200640 1 --2- 172.21.15.5:0/1246642945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a009b5f350 0x55a009b66f40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1e90006010 tx=0x7f1e9002f7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=103c8e1dae3a7f55 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:08.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.337+0000 7f1e8ea00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:49:08.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.338+0000 7f1e8ea00640 1 -- 172.21.15.5:0/1246642945 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e90048030 con 0x55a009b5f350 2024-04-24T06:49:08.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.338+0000 7f1e8ea00640 1 -- 172.21.15.5:0/1246642945 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f1e90040050 con 0x55a009b5f350 2024-04-24T06:49:08.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.338+0000 7f1eaa9ef740 1 -- 172.21.15.5:0/1246642945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e9002f930 con 0x55a009b5f350 2024-04-24T06:49:08.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.338+0000 7f1e8ea00640 1 -- 172.21.15.5:0/1246642945 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1e90034040 con 0x55a009b5f350 2024-04-24T06:49:08.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.338+0000 7f1eaa9ef740 10 client.16212 fetch_fsmap learned FSMap version 27 2024-04-24T06:49:08.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.338+0000 7f1eaa9ef740 1 -- 172.21.15.5:0/1246642945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a0098a07d0 con 0x55a009b5f350 2024-04-24T06:49:08.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.338+0000 7f1e8ea00640 1 -- 172.21.15.5:0/1246642945 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e9002f930 con 0x55a009b5f350 2024-04-24T06:49:08.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.339+0000 7f1e8ea00640 1 -- 172.21.15.5:0/1246642945 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f1e9003a430 con 0x55a009b5f350 2024-04-24T06:49:08.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.339+0000 7f1eaa9ef740 10 client.16212 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:49:08.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.339+0000 7f1eaa9ef740 10 client.16212 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:49:08.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.339+0000 7f1eaa9ef740 20 client.16212 populate_metadata read hostname 'smithi005' 2024-04-24T06:49:08.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.339+0000 7f1eaa9ef740 1 --2- 172.21.15.5:0/1246642945 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a009b79c50 0x55a009b7c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:08.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.339+0000 7f1eaa9ef740 4 client.16212 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:49:08.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.339+0000 7f1eaa9ef740 1 -- 172.21.15.5:0/1246642945 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a009ad1c10 con 0x55a009b79c50 2024-04-24T06:49:08.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.339+0000 7f1e9fe00640 1 --2- 172.21.15.5:0/1246642945 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a009b79c50 0x55a009b7c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:08.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.340+0000 7f1e9fe00640 1 --2- 172.21.15.5:0/1246642945 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a009b79c50 0x55a009b7c040 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc2384ca8f3bd6cd server_cookie=1dcf24959d99b597 in_seq=0 out_seq=0 2024-04-24T06:49:08.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.340+0000 7f1e8ea00640 10 client.16212 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:49:08.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.341+0000 7f1e8ea00640 1 -- 172.21.15.5:0/1246642945 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a009ad1c10 con 0x55a009b79c50 2024-04-24T06:49:08.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.341+0000 7f1e8ea00640 10 client.16212 handle_command_reply: tid=0 2024-04-24T06:49:08.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.341+0000 7f1eaa9ef740 1 client.16212 shutdown 2024-04-24T06:49:08.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.341+0000 7f1eaa9ef740 1 -- 172.21.15.5:0/1246642945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a009b5f350 msgr2=0x55a009b66f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:08.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.341+0000 7f1eaa9ef740 1 --2- 172.21.15.5:0/1246642945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a009b5f350 0x55a009b66f40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1e90006010 tx=0x7f1e9002f7d0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.341+0000 7f1eaa9ef740 1 -- 172.21.15.5:0/1246642945 shutdown_connections 2024-04-24T06:49:08.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.341+0000 7f1eaa9ef740 1 --2- 172.21.15.5:0/1246642945 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a009b79c50 0x55a009b7c040 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.341+0000 7f1eaa9ef740 1 --2- 172.21.15.5:0/1246642945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a009b69a20 0x55a009b67480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.341+0000 7f1eaa9ef740 1 --2- 172.21.15.5:0/1246642945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a009b5f350 0x55a009b66f40 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.341+0000 7f1eaa9ef740 1 --2- 172.21.15.5:0/1246642945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a009b5ea30 0x55a009b66a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.341+0000 7f1e8ea00640 0 client.16212 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:49:08.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.341+0000 7f1eaa9ef740 1 -- 172.21.15.5:0/1246642945 >> 172.21.15.5:0/1246642945 conn(0x55a009b5ad50 msgr2=0x55a009b59fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:08.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.341+0000 7f1eaa9ef740 1 -- 172.21.15.5:0/1246642945 shutdown_connections 2024-04-24T06:49:08.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.341+0000 7f1eaa9ef740 1 -- 172.21.15.5:0/1246642945 wait complete. 2024-04-24T06:49:08.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.341+0000 7f1eaa9ef740 20 client.16212 trim_cache size 0 max 16384 2024-04-24T06:49:08.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.361+0000 7f1eaa9ef740 1 Processor -- start 2024-04-24T06:49:08.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.361+0000 7f1eaa9ef740 1 -- start start 2024-04-24T06:49:08.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.361+0000 7f1eaa9ef740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a009b5ea30 0x55a009b69130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:08.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.361+0000 7f1eaa9ef740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a009b5f350 0x55a009b5a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:08.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.361+0000 7f1eaa9ef740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a009b69a20 0x55a009b652d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:08.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.361+0000 7f1eaa9ef740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea01b6a00 con 0x55a009b5f350 2024-04-24T06:49:08.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.361+0000 7f1eaa9ef740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea01b42d0 con 0x55a009b5ea30 2024-04-24T06:49:08.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.361+0000 7f1eaa9ef740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea01b2120 con 0x55a009b69a20 2024-04-24T06:49:08.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.361+0000 7f1ea5200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a009b5f350 0x55a009b5a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:08.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.361+0000 7f1e9fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a009b69a20 0x55a009b652d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:08.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.362+0000 7f1e9fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a009b69a20 0x55a009b652d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:52338/0 (socket says 172.21.15.5:52338) 2024-04-24T06:49:08.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.362+0000 7f1e9fe00640 1 -- 172.21.15.5:0/1304981674 learned_addr learned my addr 172.21.15.5:0/1304981674 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:08.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.362+0000 7f1ea5200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a009b5f350 0x55a009b5a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32814/0 (socket says 172.21.15.5:32814) 2024-04-24T06:49:08.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.362+0000 7f1e9fe00640 1 --2- 172.21.15.5:0/1304981674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a009b5ea30 0x55a009b69130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:08.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.362+0000 7f1e9fe00640 1 -- 172.21.15.5:0/1304981674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a009b5ea30 msgr2=0x55a009b69130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:08.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.362+0000 7f1e9fe00640 1 --2- 172.21.15.5:0/1304981674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a009b5ea30 0x55a009b69130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.362+0000 7f1e9fe00640 1 -- 172.21.15.5:0/1304981674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a009b5f350 msgr2=0x55a009b5a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:08.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.362+0000 7f1e9fe00640 1 --2- 172.21.15.5:0/1304981674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a009b5f350 0x55a009b5a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.362+0000 7f1e9fe00640 1 -- 172.21.15.5:0/1304981674 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0097d63f0 con 0x55a009b69a20 2024-04-24T06:49:08.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.362+0000 7f1ea5200640 1 --2- 172.21.15.5:0/1304981674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a009b5f350 0x55a009b5a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:49:08.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.362+0000 7f1e9fe00640 1 --2- 172.21.15.5:0/1304981674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a009b69a20 0x55a009b652d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1e88004700 tx=0x7f1e8800b650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f64d8332dc93b3e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:08.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.362+0000 7f1e95600640 1 -- 172.21.15.5:0/1304981674 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e88013e40 con 0x55a009b69a20 2024-04-24T06:49:08.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.363+0000 7f1e95600640 1 -- 172.21.15.5:0/1304981674 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e88005020 con 0x55a009b69a20 2024-04-24T06:49:08.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.363+0000 7f1e95600640 1 -- 172.21.15.5:0/1304981674 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e88004a00 con 0x55a009b69a20 2024-04-24T06:49:08.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.363+0000 7f1eaa9ef740 1 -- 172.21.15.5:0/1304981674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a009b69a20 msgr2=0x55a009b652d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:08.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.363+0000 7f1eaa9ef740 1 --2- 172.21.15.5:0/1304981674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a009b69a20 0x55a009b652d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1e88004700 tx=0x7f1e8800b650 comp rx=0 tx=0).stop 2024-04-24T06:49:08.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.363+0000 7f1eaa9ef740 1 -- 172.21.15.5:0/1304981674 shutdown_connections 2024-04-24T06:49:08.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.363+0000 7f1eaa9ef740 1 --2- 172.21.15.5:0/1304981674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a009b69a20 0x55a009b652d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.363+0000 7f1eaa9ef740 1 --2- 172.21.15.5:0/1304981674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a009b5f350 0x55a009b5a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.363+0000 7f1eaa9ef740 1 --2- 172.21.15.5:0/1304981674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a009b5ea30 0x55a009b69130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.363+0000 7f1eaa9ef740 1 -- 172.21.15.5:0/1304981674 >> 172.21.15.5:0/1304981674 conn(0x55a009b5ad50 msgr2=0x55a009b5ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:08.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.363+0000 7f1eaa9ef740 1 -- 172.21.15.5:0/1304981674 shutdown_connections 2024-04-24T06:49:08.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.363+0000 7f1eaa9ef740 1 -- 172.21.15.5:0/1304981674 wait complete. 2024-04-24T06:49:08.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.364+0000 7f1eaa9ef740 1 -- start start 2024-04-24T06:49:08.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.364+0000 7f1eaa9ef740 1 Processor -- start 2024-04-24T06:49:08.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.364+0000 7f1eaa9ef740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a009b5ea30 0x55a009b71e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:08.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.364+0000 7f1eaa9ef740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a009b5f350 0x55a009b63230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:08.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.364+0000 7f1eaa9ef740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a009b69a20 0x55a009b63770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:08.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.364+0000 7f1eaa9ef740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea01b2790 con 0x55a009b5f350 2024-04-24T06:49:08.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.364+0000 7f1eaa9ef740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea01b3c00 con 0x55a009b69a20 2024-04-24T06:49:08.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.364+0000 7f1eaa9ef740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea01b2e60 con 0x55a009b5ea30 2024-04-24T06:49:08.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.364+0000 7f1eaa9ef740 10 client.0 _ll_register_callbacks cb 0x7f1e90046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:49:08.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.365+0000 7f1ea5200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a009b5f350 0x55a009b63230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:08.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.365+0000 7f1e9fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a009b5ea30 0x55a009b71e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:08.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.365+0000 7f1ea5200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a009b5f350 0x55a009b63230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32828/0 (socket says 172.21.15.5:32828) 2024-04-24T06:49:08.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.365+0000 7f1ea5200640 1 -- 172.21.15.5:0/946902669 learned_addr learned my addr 172.21.15.5:0/946902669 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:08.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.365+0000 7f1e9fe00640 1 --2- 172.21.15.5:0/946902669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a009b69a20 0x55a009b63770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:08.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.365+0000 7f1ea5200640 1 -- 172.21.15.5:0/946902669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a009b5ea30 msgr2=0x55a009b71e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:08.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.365+0000 7f1ea5200640 1 --2- 172.21.15.5:0/946902669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a009b5ea30 0x55a009b71e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.365+0000 7f1ea5200640 1 -- 172.21.15.5:0/946902669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a009b69a20 msgr2=0x55a009b63770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:08.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.365+0000 7f1ea5200640 1 --2- 172.21.15.5:0/946902669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a009b69a20 0x55a009b63770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.365+0000 7f1ea5200640 1 -- 172.21.15.5:0/946902669 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0097960f0 con 0x55a009b5f350 2024-04-24T06:49:08.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.365+0000 7f1ea5200640 1 --2- 172.21.15.5:0/946902669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a009b5f350 0x55a009b63230 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1e90009ca0 tx=0x7f1e9002ff20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93243df694316f3a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:08.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.365+0000 7f1e8ea00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:49:08.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.366+0000 7f1e8ea00640 1 -- 172.21.15.5:0/946902669 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e90048070 con 0x55a009b5f350 2024-04-24T06:49:08.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.366+0000 7f1e8ea00640 1 -- 172.21.15.5:0/946902669 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f1e9003ac00 con 0x55a009b5f350 2024-04-24T06:49:08.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.366+0000 7f1eaa9ef740 1 -- 172.21.15.5:0/946902669 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e88005020 con 0x55a009b5f350 2024-04-24T06:49:08.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.366+0000 7f1e8ea00640 1 -- 172.21.15.5:0/946902669 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1e9002f930 con 0x55a009b5f350 2024-04-24T06:49:08.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.366+0000 7f1eaa9ef740 10 client.16218 fetch_fsmap learned FSMap version 27 2024-04-24T06:49:08.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.366+0000 7f1eaa9ef740 1 -- 172.21.15.5:0/946902669 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a009a89810 con 0x55a009b5f350 2024-04-24T06:49:08.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.366+0000 7f1e8ea00640 1 -- 172.21.15.5:0/946902669 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e88005020 con 0x55a009b5f350 2024-04-24T06:49:08.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.367+0000 7f1e8ea00640 1 -- 172.21.15.5:0/946902669 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f1e90030440 con 0x55a009b5f350 2024-04-24T06:49:08.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.367+0000 7f1eaa9ef740 10 client.16218 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:49:08.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.367+0000 7f1eaa9ef740 10 client.16218 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:49:08.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.367+0000 7f1eaa9ef740 20 client.16218 populate_metadata read hostname 'smithi005' 2024-04-24T06:49:08.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.367+0000 7f1eaa9ef740 1 --2- 172.21.15.5:0/946902669 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a009be00c0 0x55a009be24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:08.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.367+0000 7f1eaa9ef740 4 client.16218 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:49:08.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.367+0000 7f1eaa9ef740 1 -- 172.21.15.5:0/946902669 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a009acba80 con 0x55a009be00c0 2024-04-24T06:49:08.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.367+0000 7f1e9fe00640 1 --2- 172.21.15.5:0/946902669 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a009be00c0 0x55a009be24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:08.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.367+0000 7f1e9fe00640 1 --2- 172.21.15.5:0/946902669 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a009be00c0 0x55a009be24e0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81c587e497a58635 server_cookie=1f46c369c0741c60 in_seq=0 out_seq=0 2024-04-24T06:49:08.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.367+0000 7f1e8ea00640 10 client.16218 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:49:08.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.368+0000 7f1e8ea00640 1 -- 172.21.15.5:0/946902669 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a009acba80 con 0x55a009be00c0 2024-04-24T06:49:08.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.368+0000 7f1e8ea00640 10 client.16218 handle_command_reply: tid=0 2024-04-24T06:49:08.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.368+0000 7f1eaa9ef740 1 client.16218 shutdown 2024-04-24T06:49:08.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.368+0000 7f1eaa9ef740 1 -- 172.21.15.5:0/946902669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a009b5f350 msgr2=0x55a009b63230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:08.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.368+0000 7f1eaa9ef740 1 --2- 172.21.15.5:0/946902669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a009b5f350 0x55a009b63230 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1e90009ca0 tx=0x7f1e9002ff20 comp rx=0 tx=0).stop 2024-04-24T06:49:08.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.368+0000 7f1eaa9ef740 1 -- 172.21.15.5:0/946902669 shutdown_connections 2024-04-24T06:49:08.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.368+0000 7f1eaa9ef740 1 --2- 172.21.15.5:0/946902669 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a009be00c0 0x55a009be24e0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.368+0000 7f1eaa9ef740 1 --2- 172.21.15.5:0/946902669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a009b69a20 0x55a009b63770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.368+0000 7f1eaa9ef740 1 --2- 172.21.15.5:0/946902669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a009b5f350 0x55a009b63230 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.368+0000 7f1eaa9ef740 1 --2- 172.21.15.5:0/946902669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a009b5ea30 0x55a009b71e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.368+0000 7f1eaa9ef740 1 -- 172.21.15.5:0/946902669 >> 172.21.15.5:0/946902669 conn(0x55a009b5ad50 msgr2=0x55a009b5ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:08.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.368+0000 7f1e8ea00640 0 client.16218 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:49:08.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.368+0000 7f1eaa9ef740 1 -- 172.21.15.5:0/946902669 shutdown_connections 2024-04-24T06:49:08.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.368+0000 7f1eaa9ef740 1 -- 172.21.15.5:0/946902669 wait complete. 2024-04-24T06:49:08.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.368+0000 7f1eaa9ef740 20 client.16218 trim_cache size 0 max 16384 2024-04-24T06:49:08.379 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:49:08.379 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-24T06:49:08.379 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-24T06:49:08.379 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:49:08.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.369+0000 7f1ea6600640 1 -- 172.21.15.5:0/3113514849 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1e840a3d20 msgr2=0x7f1e840a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:08.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.369+0000 7f1ea6600640 1 --2- 172.21.15.5:0/3113514849 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1e840a3d20 0x7f1e840a61e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1e9809cac0 tx=0x7f1e98009720 comp rx=0 tx=0).stop 2024-04-24T06:49:08.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.369+0000 7f1ea6600640 1 -- 172.21.15.5:0/3113514849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ea010e4f0 msgr2=0x7f1ea01a3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:08.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.369+0000 7f1ea6600640 1 --2- 172.21.15.5:0/3113514849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ea010e4f0 0x7f1ea01a3640 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1e9800e970 tx=0x7f1e9800ed70 comp rx=0 tx=0).stop 2024-04-24T06:49:08.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.369+0000 7f1ea6600640 1 -- 172.21.15.5:0/3113514849 shutdown_connections 2024-04-24T06:49:08.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.369+0000 7f1ea6600640 1 --2- 172.21.15.5:0/3113514849 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1e840a3d20 0x7f1e840a61e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.369+0000 7f1ea6600640 1 --2- 172.21.15.5:0/3113514849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ea010e4f0 0x7f1ea01a3640 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.369+0000 7f1ea6600640 1 --2- 172.21.15.5:0/3113514849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ea0108ee0 0x7f1ea019f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.369+0000 7f1ea6600640 1 --2- 172.21.15.5:0/3113514849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ea0108540 0x7f1ea019ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.369+0000 7f1ea6600640 1 -- 172.21.15.5:0/3113514849 >> 172.21.15.5:0/3113514849 conn(0x7f1ea0077e10 msgr2=0x7f1ea0078530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:08.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.369+0000 7f1ea6600640 1 -- 172.21.15.5:0/3113514849 shutdown_connections 2024-04-24T06:49:08.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.369+0000 7f1ea6600640 1 -- 172.21.15.5:0/3113514849 wait complete. 2024-04-24T06:49:08.388 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T06:49:08.388 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:49:08.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.598+0000 7f2fe7e00640 1 -- 172.21.15.5:0/4001680094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fe8104cf0 msgr2=0x7f2fe8108d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:08.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.598+0000 7f2fe7e00640 1 --2- 172.21.15.5:0/4001680094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fe8104cf0 0x7f2fe8108d30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0009860 tx=0x7f2fd002a340 comp rx=0 tx=0).stop 2024-04-24T06:49:08.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.598+0000 7f2fe7e00640 1 -- 172.21.15.5:0/4001680094 shutdown_connections 2024-04-24T06:49:08.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.598+0000 7f2fe7e00640 1 --2- 172.21.15.5:0/4001680094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2fe8109410 0x7f2fe8110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.598+0000 7f2fe7e00640 1 --2- 172.21.15.5:0/4001680094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fe8104cf0 0x7f2fe8108d30 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.598+0000 7f2fe7e00640 1 --2- 172.21.15.5:0/4001680094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fe8104340 0x7f2fe8104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.599+0000 7f2fe7e00640 1 -- 172.21.15.5:0/4001680094 >> 172.21.15.5:0/4001680094 conn(0x7f2fe80ffb80 msgr2=0x7f2fe8101fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:08.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.599+0000 7f2fe7e00640 1 -- 172.21.15.5:0/4001680094 shutdown_connections 2024-04-24T06:49:08.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.599+0000 7f2fe7e00640 1 -- 172.21.15.5:0/4001680094 wait complete. 2024-04-24T06:49:08.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.600+0000 7f2fe7e00640 1 Processor -- start 2024-04-24T06:49:08.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.600+0000 7f2fe7e00640 1 -- start start 2024-04-24T06:49:08.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.601+0000 7f2fe7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2fe8104340 0x7f2fe81a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:08.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.601+0000 7f2fe7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fe8109410 0x7f2fe81a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:08.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.601+0000 7f2fe7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fe819a800 0x7f2fe819acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:08.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.601+0000 7f2fe7e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe81144a0 con 0x7f2fe8109410 2024-04-24T06:49:08.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.601+0000 7f2fe7e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe8114320 con 0x7f2fe8104340 2024-04-24T06:49:08.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.601+0000 7f2fe7e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe8114620 con 0x7f2fe819a800 2024-04-24T06:49:08.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.601+0000 7f2fdfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fe8109410 0x7f2fe81a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:08.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.601+0000 7f2fe6a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2fe8104340 0x7f2fe81a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:08.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.601+0000 7f2fdfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fe8109410 0x7f2fe81a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32848/0 (socket says 172.21.15.5:32848) 2024-04-24T06:49:08.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.601+0000 7f2fdfe00640 1 -- 172.21.15.5:0/937725176 learned_addr learned my addr 172.21.15.5:0/937725176 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:08.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.601+0000 7f2fe7400640 1 --2- 172.21.15.5:0/937725176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fe819a800 0x7f2fe819acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:08.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.602+0000 7f2fe6a00640 1 -- 172.21.15.5:0/937725176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fe819a800 msgr2=0x7f2fe819acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:08.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.602+0000 7f2fe6a00640 1 --2- 172.21.15.5:0/937725176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fe819a800 0x7f2fe819acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.602+0000 7f2fe6a00640 1 -- 172.21.15.5:0/937725176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fe8109410 msgr2=0x7f2fe81a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:08.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.602+0000 7f2fe6a00640 1 --2- 172.21.15.5:0/937725176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fe8109410 0x7f2fe81a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.602+0000 7f2fe6a00640 1 -- 172.21.15.5:0/937725176 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe819b390 con 0x7f2fe8104340 2024-04-24T06:49:08.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.602+0000 7f2fdfe00640 1 --2- 172.21.15.5:0/937725176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fe8109410 0x7f2fe81a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:49:08.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.602+0000 7f2fe6a00640 1 --2- 172.21.15.5:0/937725176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2fe8104340 0x7f2fe81a0690 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2fd800e790 tx=0x7f2fd800ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:08.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.602+0000 7f2fdf400640 1 -- 172.21.15.5:0/937725176 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd800eea0 con 0x7f2fe8104340 2024-04-24T06:49:08.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.602+0000 7f2fe7e00640 1 -- 172.21.15.5:0/937725176 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fe819b680 con 0x7f2fe8104340 2024-04-24T06:49:08.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.603+0000 7f2fe7e00640 1 -- 172.21.15.5:0/937725176 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fe81a7880 con 0x7f2fe8104340 2024-04-24T06:49:08.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.603+0000 7f2fdf400640 1 -- 172.21.15.5:0/937725176 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fd800ccc0 con 0x7f2fe8104340 2024-04-24T06:49:08.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.603+0000 7f2fdf400640 1 -- 172.21.15.5:0/937725176 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8018660 con 0x7f2fe8104340 2024-04-24T06:49:08.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.604+0000 7f2fe7e00640 1 -- 172.21.15.5:0/937725176 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fe8105810 con 0x7f2fe8104340 2024-04-24T06:49:08.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.606+0000 7f2fdf400640 1 -- 172.21.15.5:0/937725176 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f2fd8018860 con 0x7f2fe8104340 2024-04-24T06:49:08.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.607+0000 7f2fdf400640 1 --2- 172.21.15.5:0/937725176 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f2fcc0a3d10 0x7f2fcc0a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:08.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.607+0000 7f2fdf400640 1 -- 172.21.15.5:0/937725176 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f2fd80d0a50 con 0x7f2fe8104340 2024-04-24T06:49:08.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.608+0000 7f2fdfe00640 1 --2- 172.21.15.5:0/937725176 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f2fcc0a3d10 0x7f2fcc0a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:08.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.608+0000 7f2fdfe00640 1 --2- 172.21.15.5:0/937725176 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f2fcc0a3d10 0x7f2fcc0a61d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2fe819bb50 tx=0x7f2fd00056d0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:08.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.610+0000 7f2fdf400640 1 -- 172.21.15.5:0/937725176 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2fe819b680 con 0x7f2fe8104340 2024-04-24T06:49:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.860+0000 7f2fe7e00640 1 -- 172.21.15.5:0/937725176 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2fe810c640 con 0x7f2fe8104340 2024-04-24T06:49:08.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.862+0000 7f2fdf400640 1 -- 172.21.15.5:0/937725176 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11856 (secure 0 0 0) 0x7f2fd809ca50 con 0x7f2fe8104340 2024-04-24T06:49:08.863 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:49:08.863 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1735462750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1735462750},{"type":"v1","addr":"172.21.15.5:6837","nonce":1735462750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1499469370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1499469370},{"type":"v1","addr":"172.21.15.5:6841","nonce":1499469370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3705638847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3705638847},{"type":"v1","addr":"172.21.15.37:6835","nonce":3705638847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T06:41:34.764490+0000","modified":"2024-04-24T06:48:57.768410+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24550,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6837/1390103296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1390103296},{"type":"v1","addr":"172.21.15.37:6837","nonce":1390103296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3217754215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3217754215},{"type":"v1","addr":"172.21.15.137:6835","nonce":3217754215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6837/3044222976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3044222976},{"type":"v1","addr":"172.21.15.137:6837","nonce":3044222976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.5:6839/3173201459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3173201459},{"type":"v1","addr":"172.21.15.5:6839","nonce":3173201459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6839/1889774232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1889774232},{"type":"v1","addr":"172.21.15.137:6839","nonce":1889774232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6839/3536721753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":3536721753},{"type":"v1","addr":"172.21.15.37:6839","nonce":3536721753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24550,24569,24539]},"id":1}]} 2024-04-24T06:49:08.863 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-04-24T06:49:08.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.864+0000 7f2fe7e00640 1 -- 172.21.15.5:0/937725176 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f2fcc0a3d10 msgr2=0x7f2fcc0a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:08.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.865+0000 7f2fe7e00640 1 --2- 172.21.15.5:0/937725176 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f2fcc0a3d10 0x7f2fcc0a61d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2fe819bb50 tx=0x7f2fd00056d0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.865+0000 7f2fe7e00640 1 -- 172.21.15.5:0/937725176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2fe8104340 msgr2=0x7f2fe81a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:08.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.865+0000 7f2fe7e00640 1 --2- 172.21.15.5:0/937725176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2fe8104340 0x7f2fe81a0690 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2fd800e790 tx=0x7f2fd800ec50 comp rx=0 tx=0).stop 2024-04-24T06:49:08.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.865+0000 7f2fe7e00640 1 -- 172.21.15.5:0/937725176 shutdown_connections 2024-04-24T06:49:08.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.865+0000 7f2fe7e00640 1 --2- 172.21.15.5:0/937725176 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f2fcc0a3d10 0x7f2fcc0a61d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.865+0000 7f2fe7e00640 1 --2- 172.21.15.5:0/937725176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fe819a800 0x7f2fe819acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.865+0000 7f2fe7e00640 1 --2- 172.21.15.5:0/937725176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fe8109410 0x7f2fe81a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.865+0000 7f2fe7e00640 1 --2- 172.21.15.5:0/937725176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2fe8104340 0x7f2fe81a0690 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:08.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.865+0000 7f2fe7e00640 1 -- 172.21.15.5:0/937725176 >> 172.21.15.5:0/937725176 conn(0x7f2fe80ffb80 msgr2=0x7f2fe8101f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:08.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.865+0000 7f2fe7e00640 1 -- 172.21.15.5:0/937725176 shutdown_connections 2024-04-24T06:49:08.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:08.865+0000 7f2fe7e00640 1 -- 172.21.15.5:0/937725176 wait complete. 2024-04-24T06:49:08.887 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T06:49:09.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.089+0000 7f702b000640 1 -- 172.21.15.5:0/858945663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7024102a40 msgr2=0x7f7024106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:09.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.089+0000 7f702b000640 1 --2- 172.21.15.5:0/858945663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7024102a40 0x7f7024106a80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7010009860 tx=0x7f701002a340 comp rx=0 tx=0).stop 2024-04-24T06:49:09.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.089+0000 7f702b000640 1 -- 172.21.15.5:0/858945663 shutdown_connections 2024-04-24T06:49:09.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.089+0000 7f702b000640 1 --2- 172.21.15.5:0/858945663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7024107160 0x7f702410ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.089+0000 7f702b000640 1 --2- 172.21.15.5:0/858945663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7024102a40 0x7f7024106a80 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.089+0000 7f702b000640 1 --2- 172.21.15.5:0/858945663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7024102090 0x7f7024102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.089+0000 7f702b000640 1 -- 172.21.15.5:0/858945663 >> 172.21.15.5:0/858945663 conn(0x7f70240fd910 msgr2=0x7f70240ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:09.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.090+0000 7f702b000640 1 -- 172.21.15.5:0/858945663 shutdown_connections 2024-04-24T06:49:09.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.090+0000 7f702b000640 1 -- 172.21.15.5:0/858945663 wait complete. 2024-04-24T06:49:09.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.091+0000 7f702b000640 1 Processor -- start 2024-04-24T06:49:09.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.091+0000 7f702b000640 1 -- start start 2024-04-24T06:49:09.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.091+0000 7f702b000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7024102090 0x7f70241a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.091+0000 7f702b000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7024107160 0x7f70241a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.091+0000 7f702b000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f702419a750 0x7f702419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.091+0000 7f702b000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f70241121d0 con 0x7f702419a750 2024-04-24T06:49:09.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.091+0000 7f702b000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7024112050 con 0x7f7024102090 2024-04-24T06:49:09.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.091+0000 7f702b000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7024112350 con 0x7f7024107160 2024-04-24T06:49:09.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.092+0000 7f7029c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7024102090 0x7f70241a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.092+0000 7f7029c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7024102090 0x7f70241a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:41598/0 (socket says 172.21.15.5:41598) 2024-04-24T06:49:09.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.092+0000 7f7029200640 1 --2- 172.21.15.5:0/2469400415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7024107160 0x7f70241a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.092+0000 7f7029c00640 1 -- 172.21.15.5:0/2469400415 learned_addr learned my addr 172.21.15.5:0/2469400415 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:09.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.092+0000 7f702a600640 1 --2- 172.21.15.5:0/2469400415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f702419a750 0x7f702419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.092+0000 7f702a600640 1 -- 172.21.15.5:0/2469400415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7024107160 msgr2=0x7f70241a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:09.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.092+0000 7f702a600640 1 --2- 172.21.15.5:0/2469400415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7024107160 0x7f70241a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.092+0000 7f702a600640 1 -- 172.21.15.5:0/2469400415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7024102090 msgr2=0x7f70241a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:09.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.092+0000 7f702a600640 1 --2- 172.21.15.5:0/2469400415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7024102090 0x7f70241a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.092+0000 7f702a600640 1 -- 172.21.15.5:0/2469400415 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f702419b350 con 0x7f702419a750 2024-04-24T06:49:09.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.092+0000 7f7029200640 1 --2- 172.21.15.5:0/2469400415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7024107160 0x7f70241a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:49:09.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.092+0000 7f7029c00640 1 --2- 172.21.15.5:0/2469400415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7024102090 0x7f70241a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:49:09.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.093+0000 7f702a600640 1 --2- 172.21.15.5:0/2469400415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f702419a750 0x7f702419abd0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f701c009fd0 tx=0x7f701c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:09.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.093+0000 7f7022000640 1 -- 172.21.15.5:0/2469400415 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c00ce20 con 0x7f702419a750 2024-04-24T06:49:09.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.093+0000 7f7022000640 1 -- 172.21.15.5:0/2469400415 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f701c005020 con 0x7f702419a750 2024-04-24T06:49:09.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.093+0000 7f702b000640 1 -- 172.21.15.5:0/2469400415 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70241a7430 con 0x7f702419a750 2024-04-24T06:49:09.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.093+0000 7f7022000640 1 -- 172.21.15.5:0/2469400415 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c010720 con 0x7f702419a750 2024-04-24T06:49:09.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.093+0000 7f702b000640 1 -- 172.21.15.5:0/2469400415 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70241a77e0 con 0x7f702419a750 2024-04-24T06:49:09.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.097+0000 7f702f5d0740 1 Processor -- start 2024-04-24T06:49:09.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.098+0000 7f7022000640 1 -- 172.21.15.5:0/2469400415 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f701c005540 con 0x7f702419a750 2024-04-24T06:49:09.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.098+0000 7f702f5d0740 1 -- start start 2024-04-24T06:49:09.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.098+0000 7f702f5d0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1221e6b40 0x55b1221e6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.098+0000 7f702f5d0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1221e7460 0x55b1221f15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.098+0000 7f702f5d0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1221f1b30 0x55b1221f3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.098+0000 7f702f5d0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b12214c1d0 con 0x55b1221e6b40 2024-04-24T06:49:09.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.098+0000 7f702f5d0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b1220f2450 con 0x55b1221e7460 2024-04-24T06:49:09.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.098+0000 7f702f5d0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b122148190 con 0x55b1221f1b30 2024-04-24T06:49:09.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.099+0000 7f7029200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1221e6b40 0x55b1221e6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.099+0000 7f7029200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1221e6b40 0x55b1221e6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32868/0 (socket says 172.21.15.5:32868) 2024-04-24T06:49:09.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.099+0000 7f7029200640 1 -- 172.21.15.5:0/3782935974 learned_addr learned my addr 172.21.15.5:0/3782935974 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.099+0000 7f7029c00640 1 --2- 172.21.15.5:0/3782935974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1221e7460 0x55b1221f15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.099+0000 7f7029200640 1 --2- 172.21.15.5:0/3782935974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1221f1b30 0x55b1221f3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.099+0000 7f7029c00640 1 -- 172.21.15.5:0/3782935974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1221f1b30 msgr2=0x55b1221f3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.099+0000 7f7029c00640 1 --2- 172.21.15.5:0/3782935974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1221f1b30 0x55b1221f3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.099+0000 7f7029c00640 1 -- 172.21.15.5:0/3782935974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1221e6b40 msgr2=0x55b1221e6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.099+0000 7f7029c00640 1 --2- 172.21.15.5:0/3782935974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1221e6b40 0x55b1221e6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.099+0000 7f7029c00640 1 -- 172.21.15.5:0/3782935974 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1221af310 con 0x55b1221e7460 2024-04-24T06:49:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.100+0000 7f7029c00640 1 --2- 172.21.15.5:0/3782935974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1221e7460 0x55b1221f15f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7018011510 tx=0x7f70180119d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2affad17437b7d0d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.100+0000 7f7022000640 1 --2- 172.21.15.5:0/2469400415 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f700c0a3d20 0x7f700c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.100+0000 7f7022000640 1 -- 172.21.15.5:0/2469400415 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f701c0d0ec0 con 0x7f702419a750 2024-04-24T06:49:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.100+0000 7f7016a00640 1 -- 172.21.15.5:0/3782935974 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701801e070 con 0x55b1221e7460 2024-04-24T06:49:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.100+0000 7f7029200640 1 --2- 172.21.15.5:0/2469400415 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f700c0a3d20 0x7f700c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.100+0000 7f7016a00640 1 -- 172.21.15.5:0/3782935974 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f701800faa0 con 0x55b1221e7460 2024-04-24T06:49:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.100+0000 7f7016a00640 1 -- 172.21.15.5:0/3782935974 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7018019b50 con 0x55b1221e7460 2024-04-24T06:49:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.101+0000 7f702f5d0740 1 -- 172.21.15.5:0/3782935974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1221e7460 msgr2=0x55b1221f15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.101+0000 7f702f5d0740 1 --2- 172.21.15.5:0/3782935974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1221e7460 0x55b1221f15f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7018011510 tx=0x7f70180119d0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.101+0000 7f7029200640 1 --2- 172.21.15.5:0/2469400415 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f700c0a3d20 0x7f700c0a61e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55b1221e3ff0 tx=0x7f70100058d0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:09.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.101+0000 7f702f5d0740 1 -- 172.21.15.5:0/3782935974 shutdown_connections 2024-04-24T06:49:09.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.101+0000 7f702f5d0740 1 --2- 172.21.15.5:0/3782935974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1221f1b30 0x55b1221f3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.101+0000 7f702f5d0740 1 --2- 172.21.15.5:0/3782935974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1221e7460 0x55b1221f15f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.101+0000 7f702f5d0740 1 --2- 172.21.15.5:0/3782935974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1221e6b40 0x55b1221e6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.101+0000 7f702f5d0740 1 -- 172.21.15.5:0/3782935974 >> 172.21.15.5:0/3782935974 conn(0x55b1221e2e60 msgr2=0x55b1221e3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:09.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.101+0000 7f702f5d0740 1 -- 172.21.15.5:0/3782935974 shutdown_connections 2024-04-24T06:49:09.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.101+0000 7f702f5d0740 1 -- 172.21.15.5:0/3782935974 wait complete. 2024-04-24T06:49:09.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.101+0000 7f702f5d0740 1 -- start start 2024-04-24T06:49:09.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.102+0000 7f702f5d0740 1 Processor -- start 2024-04-24T06:49:09.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.102+0000 7f702f5d0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1221e6b40 0x55b1221f1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.102+0000 7f702f5d0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1221e7460 0x55b1221ee820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.102+0000 7f702f5d0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1221f1b30 0x55b1221eee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.102+0000 7f702f5d0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b1221f49c0 con 0x55b1221e6b40 2024-04-24T06:49:09.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.102+0000 7f702f5d0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b122105810 con 0x55b1221f1b30 2024-04-24T06:49:09.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.102+0000 7f702f5d0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b1221f4b40 con 0x55b1221e7460 2024-04-24T06:49:09.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.102+0000 7f702f5d0740 10 client.0 _ll_register_callbacks cb 0x55b1220d3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:49:09.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.102+0000 7f7029c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1221e6b40 0x55b1221f1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.102+0000 7f7029c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1221e6b40 0x55b1221f1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32880/0 (socket says 172.21.15.5:32880) 2024-04-24T06:49:09.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.102+0000 7f7029c00640 1 -- 172.21.15.5:0/452337099 learned_addr learned my addr 172.21.15.5:0/452337099 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:09.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.102+0000 7f702a600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1221f1b30 0x55b1221eee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.103+0000 7f7029200640 1 --2- 172.21.15.5:0/452337099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1221e7460 0x55b1221ee820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.103+0000 7f7029c00640 1 -- 172.21.15.5:0/452337099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1221e7460 msgr2=0x55b1221ee820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:09.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.103+0000 7f7029c00640 1 --2- 172.21.15.5:0/452337099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1221e7460 0x55b1221ee820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.103+0000 7f7029c00640 1 -- 172.21.15.5:0/452337099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1221f1b30 msgr2=0x55b1221eee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:09.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.103+0000 7f7029c00640 1 --2- 172.21.15.5:0/452337099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1221f1b30 0x55b1221eee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.103+0000 7f7029c00640 1 -- 172.21.15.5:0/452337099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1221163f0 con 0x55b1221e6b40 2024-04-24T06:49:09.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.103+0000 7f702a600640 1 --2- 172.21.15.5:0/452337099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1221f1b30 0x55b1221eee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:49:09.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.103+0000 7f7029c00640 1 --2- 172.21.15.5:0/452337099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1221e6b40 0x55b1221f1530 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7018009880 tx=0x7f7018009d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c91fe61175f6d45a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:09.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.103+0000 7f700b400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:49:09.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.103+0000 7f700b400640 1 -- 172.21.15.5:0/452337099 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701801e070 con 0x55b1221e6b40 2024-04-24T06:49:09.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.104+0000 7f700b400640 1 -- 172.21.15.5:0/452337099 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f7018004040 con 0x55b1221e6b40 2024-04-24T06:49:09.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.104+0000 7f702f5d0740 1 -- 172.21.15.5:0/452337099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f701800faa0 con 0x55b1221e6b40 2024-04-24T06:49:09.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.104+0000 7f700b400640 1 -- 172.21.15.5:0/452337099 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7018031050 con 0x55b1221e6b40 2024-04-24T06:49:09.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.104+0000 7f702f5d0740 10 client.16236 fetch_fsmap learned FSMap version 27 2024-04-24T06:49:09.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.104+0000 7f702f5d0740 1 -- 172.21.15.5:0/452337099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b121e5e3f0 con 0x55b1221e6b40 2024-04-24T06:49:09.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.104+0000 7f700b400640 1 -- 172.21.15.5:0/452337099 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f701800faa0 con 0x55b1221e6b40 2024-04-24T06:49:09.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.104+0000 7f700b400640 1 -- 172.21.15.5:0/452337099 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f7018003710 con 0x55b1221e6b40 2024-04-24T06:49:09.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.104+0000 7f702f5d0740 10 client.16236 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:49:09.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.104+0000 7f702f5d0740 10 client.16236 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-24T06:49:09.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.104+0000 7f702f5d0740 20 client.16236 populate_metadata read hostname 'smithi005' 2024-04-24T06:49:09.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.105+0000 7f702f5d0740 1 --2- 172.21.15.5:0/452337099 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55b122201c70 0x55b122204060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.105+0000 7f702f5d0740 4 client.16236 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:49:09.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.105+0000 7f702f5d0740 1 -- 172.21.15.5:0/452337099 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b122159bf0 con 0x55b122201c70 2024-04-24T06:49:09.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.105+0000 7f7029200640 1 --2- 172.21.15.5:0/452337099 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55b122201c70 0x55b122204060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.105+0000 7f7029200640 1 --2- 172.21.15.5:0/452337099 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55b122201c70 0x55b122204060 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4f2545fd73984b8f server_cookie=5121ed83d05b94a in_seq=0 out_seq=0 2024-04-24T06:49:09.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.105+0000 7f700b400640 10 client.16236 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:49:09.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.106+0000 7f700b400640 1 -- 172.21.15.5:0/452337099 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b122159bf0 con 0x55b122201c70 2024-04-24T06:49:09.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.106+0000 7f700b400640 10 client.16236 handle_command_reply: tid=0 2024-04-24T06:49:09.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.106+0000 7f702f5d0740 1 client.16236 shutdown 2024-04-24T06:49:09.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.107+0000 7f702f5d0740 1 -- 172.21.15.5:0/452337099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1221e6b40 msgr2=0x55b1221f1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:09.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.107+0000 7f702f5d0740 1 --2- 172.21.15.5:0/452337099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1221e6b40 0x55b1221f1530 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7018009880 tx=0x7f7018009d60 comp rx=0 tx=0).stop 2024-04-24T06:49:09.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.107+0000 7f702f5d0740 1 -- 172.21.15.5:0/452337099 shutdown_connections 2024-04-24T06:49:09.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.107+0000 7f702f5d0740 1 --2- 172.21.15.5:0/452337099 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55b122201c70 0x55b122204060 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.107+0000 7f702f5d0740 1 --2- 172.21.15.5:0/452337099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1221f1b30 0x55b1221eee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.107+0000 7f702f5d0740 1 --2- 172.21.15.5:0/452337099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1221e7460 0x55b1221ee820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.107+0000 7f700b400640 0 client.16236 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:49:09.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.107+0000 7f702f5d0740 1 --2- 172.21.15.5:0/452337099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1221e6b40 0x55b1221f1530 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.107+0000 7f702f5d0740 1 -- 172.21.15.5:0/452337099 >> 172.21.15.5:0/452337099 conn(0x55b1221e2e60 msgr2=0x55b1221e8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:09.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.107+0000 7f702f5d0740 1 -- 172.21.15.5:0/452337099 shutdown_connections 2024-04-24T06:49:09.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.107+0000 7f702f5d0740 1 -- 172.21.15.5:0/452337099 wait complete. 2024-04-24T06:49:09.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.107+0000 7f702f5d0740 20 client.16236 trim_cache size 0 max 16384 2024-04-24T06:49:09.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.122+0000 7f702f5d0740 1 Processor -- start 2024-04-24T06:49:09.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.122+0000 7f702f5d0740 1 -- start start 2024-04-24T06:49:09.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.122+0000 7f702f5d0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1221e6b40 0x55b1221ed6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.122+0000 7f702f5d0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1221e7460 0x55b1221e8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.122+0000 7f702f5d0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1221f1b30 0x55b1221f1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.122+0000 7f702f5d0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f70241b2610 con 0x55b1221e7460 2024-04-24T06:49:09.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.122+0000 7f702f5d0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f70241afee0 con 0x55b1221f1b30 2024-04-24T06:49:09.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.123+0000 7f702f5d0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f70241add30 con 0x55b1221e6b40 2024-04-24T06:49:09.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.123+0000 7f7029200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1221e7460 0x55b1221e8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.123+0000 7f7029200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1221e7460 0x55b1221e8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32892/0 (socket says 172.21.15.5:32892) 2024-04-24T06:49:09.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.123+0000 7f7029200640 1 -- 172.21.15.5:0/303706563 learned_addr learned my addr 172.21.15.5:0/303706563 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:09.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.123+0000 7f7029c00640 1 --2- 172.21.15.5:0/303706563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1221e6b40 0x55b1221ed6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.123+0000 7f702a600640 1 --2- 172.21.15.5:0/303706563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1221f1b30 0x55b1221f1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.123+0000 7f7029200640 1 -- 172.21.15.5:0/303706563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1221e6b40 msgr2=0x55b1221ed6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:09.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.123+0000 7f7029200640 1 --2- 172.21.15.5:0/303706563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1221e6b40 0x55b1221ed6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.123+0000 7f7029200640 1 -- 172.21.15.5:0/303706563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1221f1b30 msgr2=0x55b1221f1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:09.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.123+0000 7f7029200640 1 --2- 172.21.15.5:0/303706563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1221f1b30 0x55b1221f1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.123+0000 7f7029200640 1 -- 172.21.15.5:0/303706563 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b121f287d0 con 0x55b1221e7460 2024-04-24T06:49:09.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.123+0000 7f7029c00640 1 --2- 172.21.15.5:0/303706563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1221e6b40 0x55b1221ed6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:49:09.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.124+0000 7f7029200640 1 --2- 172.21.15.5:0/303706563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1221e7460 0x55b1221e8de0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7010009830 tx=0x7f7010039000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb998e21ae3e335 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:09.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.124+0000 7f7016000640 1 -- 172.21.15.5:0/303706563 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7010059070 con 0x55b1221e7460 2024-04-24T06:49:09.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.124+0000 7f7016000640 1 -- 172.21.15.5:0/303706563 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70100309d0 con 0x55b1221e7460 2024-04-24T06:49:09.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.124+0000 7f7016000640 1 -- 172.21.15.5:0/303706563 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7010037040 con 0x55b1221e7460 2024-04-24T06:49:09.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.125+0000 7f702f5d0740 1 -- 172.21.15.5:0/303706563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1221e7460 msgr2=0x55b1221e8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:09.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.125+0000 7f702f5d0740 1 --2- 172.21.15.5:0/303706563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1221e7460 0x55b1221e8de0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7010009830 tx=0x7f7010039000 comp rx=0 tx=0).stop 2024-04-24T06:49:09.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.125+0000 7f702f5d0740 1 -- 172.21.15.5:0/303706563 shutdown_connections 2024-04-24T06:49:09.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.125+0000 7f702f5d0740 1 --2- 172.21.15.5:0/303706563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1221f1b30 0x55b1221f1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.125+0000 7f702f5d0740 1 --2- 172.21.15.5:0/303706563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1221e7460 0x55b1221e8de0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.125+0000 7f702f5d0740 1 --2- 172.21.15.5:0/303706563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1221e6b40 0x55b1221ed6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.125+0000 7f702f5d0740 1 -- 172.21.15.5:0/303706563 >> 172.21.15.5:0/303706563 conn(0x55b1221e2e60 msgr2=0x55b1221ecfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:09.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.125+0000 7f702f5d0740 1 -- 172.21.15.5:0/303706563 shutdown_connections 2024-04-24T06:49:09.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.125+0000 7f702f5d0740 1 -- 172.21.15.5:0/303706563 wait complete. 2024-04-24T06:49:09.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.125+0000 7f702f5d0740 1 -- start start 2024-04-24T06:49:09.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.125+0000 7f702f5d0740 1 Processor -- start 2024-04-24T06:49:09.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.126+0000 7f702f5d0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1221e6b40 0x55b1221f1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.126+0000 7f702f5d0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1221e7460 0x55b1221f2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.126+0000 7f702f5d0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1221f1b30 0x55b1221f2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.126+0000 7f702f5d0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f70241ae3a0 con 0x55b1221f1b30 2024-04-24T06:49:09.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.126+0000 7f702f5d0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f70241af810 con 0x55b1221e6b40 2024-04-24T06:49:09.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.126+0000 7f702f5d0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f70241aea70 con 0x55b1221e7460 2024-04-24T06:49:09.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.126+0000 7f702f5d0740 10 client.0 _ll_register_callbacks cb 0x7f701801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:49:09.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.126+0000 7f702a600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1221f1b30 0x55b1221f2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.126+0000 7f702a600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1221f1b30 0x55b1221f2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32896/0 (socket says 172.21.15.5:32896) 2024-04-24T06:49:09.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.126+0000 7f702a600640 1 -- 172.21.15.5:0/3458167107 learned_addr learned my addr 172.21.15.5:0/3458167107 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:09.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.126+0000 7f7029200640 1 --2- 172.21.15.5:0/3458167107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1221e7460 0x55b1221f2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.126+0000 7f7029c00640 1 --2- 172.21.15.5:0/3458167107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1221e6b40 0x55b1221f1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.127+0000 7f702a600640 1 -- 172.21.15.5:0/3458167107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1221e7460 msgr2=0x55b1221f2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:09.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.127+0000 7f702a600640 1 --2- 172.21.15.5:0/3458167107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1221e7460 0x55b1221f2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.127+0000 7f702a600640 1 -- 172.21.15.5:0/3458167107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1221e6b40 msgr2=0x55b1221f1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:09.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.127+0000 7f702a600640 1 --2- 172.21.15.5:0/3458167107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1221e6b40 0x55b1221f1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.127+0000 7f702a600640 1 -- 172.21.15.5:0/3458167107 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1221f8e40 con 0x55b1221f1b30 2024-04-24T06:49:09.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.127+0000 7f7029200640 1 --2- 172.21.15.5:0/3458167107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1221e7460 0x55b1221f2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:49:09.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.127+0000 7f702a600640 1 --2- 172.21.15.5:0/3458167107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1221f1b30 0x55b1221f2990 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f701c0bbfd0 tx=0x7f701c005940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fdbc48dcb1d3608 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:09.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.127+0000 7f700b400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:49:09.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.127+0000 7f700b400640 1 -- 172.21.15.5:0/3458167107 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c0a5a80 con 0x55b1221f1b30 2024-04-24T06:49:09.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.127+0000 7f700b400640 1 -- 172.21.15.5:0/3458167107 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f701c0b6170 con 0x55b1221f1b30 2024-04-24T06:49:09.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.127+0000 7f702f5d0740 1 -- 172.21.15.5:0/3458167107 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70100309d0 con 0x55b1221f1b30 2024-04-24T06:49:09.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.127+0000 7f700b400640 1 -- 172.21.15.5:0/3458167107 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f701c082050 con 0x55b1221f1b30 2024-04-24T06:49:09.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.127+0000 7f702f5d0740 10 client.16242 fetch_fsmap learned FSMap version 27 2024-04-24T06:49:09.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.127+0000 7f702f5d0740 1 -- 172.21.15.5:0/3458167107 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b1221fa140 con 0x55b1221f1b30 2024-04-24T06:49:09.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.128+0000 7f700b400640 1 -- 172.21.15.5:0/3458167107 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70100309d0 con 0x55b1221f1b30 2024-04-24T06:49:09.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.128+0000 7f700b400640 1 -- 172.21.15.5:0/3458167107 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f701c09bcd0 con 0x55b1221f1b30 2024-04-24T06:49:09.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.128+0000 7f702f5d0740 10 client.16242 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:49:09.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.128+0000 7f702f5d0740 10 client.16242 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-24T06:49:09.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.128+0000 7f702f5d0740 20 client.16242 populate_metadata read hostname 'smithi005' 2024-04-24T06:49:09.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.128+0000 7f702f5d0740 1 --2- 172.21.15.5:0/3458167107 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55b1222680b0 0x55b12226a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.128+0000 7f702f5d0740 4 client.16242 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:49:09.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.128+0000 7f702f5d0740 1 -- 172.21.15.5:0/3458167107 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b122153a60 con 0x55b1222680b0 2024-04-24T06:49:09.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.128+0000 7f7029c00640 1 --2- 172.21.15.5:0/3458167107 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55b1222680b0 0x55b12226a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.129+0000 7f7029c00640 1 --2- 172.21.15.5:0/3458167107 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55b1222680b0 0x55b12226a4a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=916baa2821340845 server_cookie=f9adf96032ea49a4 in_seq=0 out_seq=0 2024-04-24T06:49:09.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.129+0000 7f700b400640 10 client.16242 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:49:09.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.129+0000 7f700b400640 1 -- 172.21.15.5:0/3458167107 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b122153a60 con 0x55b1222680b0 2024-04-24T06:49:09.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.129+0000 7f700b400640 10 client.16242 handle_command_reply: tid=0 2024-04-24T06:49:09.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.129+0000 7f702f5d0740 1 client.16242 shutdown 2024-04-24T06:49:09.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.129+0000 7f702f5d0740 1 -- 172.21.15.5:0/3458167107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1221f1b30 msgr2=0x55b1221f2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:09.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.129+0000 7f702f5d0740 1 --2- 172.21.15.5:0/3458167107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1221f1b30 0x55b1221f2990 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f701c0bbfd0 tx=0x7f701c005940 comp rx=0 tx=0).stop 2024-04-24T06:49:09.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.129+0000 7f702f5d0740 1 -- 172.21.15.5:0/3458167107 shutdown_connections 2024-04-24T06:49:09.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.129+0000 7f702f5d0740 1 --2- 172.21.15.5:0/3458167107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1221f1b30 0x55b1221f2990 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.129+0000 7f702f5d0740 1 --2- 172.21.15.5:0/3458167107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b1221e7460 0x55b1221f2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.129+0000 7f702f5d0740 1 --2- 172.21.15.5:0/3458167107 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55b1222680b0 0x55b12226a4a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.129+0000 7f702f5d0740 1 --2- 172.21.15.5:0/3458167107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1221e6b40 0x55b1221f1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.129+0000 7f702f5d0740 1 -- 172.21.15.5:0/3458167107 >> 172.21.15.5:0/3458167107 conn(0x55b1221e2e60 msgr2=0x55b1221e1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:09.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.129+0000 7f700b400640 0 client.16242 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:49:09.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.129+0000 7f702f5d0740 1 -- 172.21.15.5:0/3458167107 shutdown_connections 2024-04-24T06:49:09.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.130+0000 7f702f5d0740 1 -- 172.21.15.5:0/3458167107 wait complete. 2024-04-24T06:49:09.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.130+0000 7f702f5d0740 20 client.16242 trim_cache size 0 max 16384 2024-04-24T06:49:09.140 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:49:09.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.130+0000 7f702b000640 1 -- 172.21.15.5:0/2469400415 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f700c0a3d20 msgr2=0x7f700c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:09.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.130+0000 7f702b000640 1 --2- 172.21.15.5:0/2469400415 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f700c0a3d20 0x7f700c0a61e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55b1221e3ff0 tx=0x7f70100058d0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.130+0000 7f702b000640 1 -- 172.21.15.5:0/2469400415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f702419a750 msgr2=0x7f702419abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:09.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.130+0000 7f702b000640 1 --2- 172.21.15.5:0/2469400415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f702419a750 0x7f702419abd0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f701c009fd0 tx=0x7f701c00ef10 comp rx=0 tx=0).stop 2024-04-24T06:49:09.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.130+0000 7f702b000640 1 -- 172.21.15.5:0/2469400415 shutdown_connections 2024-04-24T06:49:09.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.130+0000 7f702b000640 1 --2- 172.21.15.5:0/2469400415 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f700c0a3d20 0x7f700c0a61e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.130+0000 7f702b000640 1 --2- 172.21.15.5:0/2469400415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f702419a750 0x7f702419abd0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.130+0000 7f702b000640 1 --2- 172.21.15.5:0/2469400415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7024107160 0x7f70241a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.130+0000 7f702b000640 1 --2- 172.21.15.5:0/2469400415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7024102090 0x7f70241a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.130+0000 7f702b000640 1 -- 172.21.15.5:0/2469400415 >> 172.21.15.5:0/2469400415 conn(0x7f70240fd910 msgr2=0x7f70240fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:09.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.131+0000 7f702b000640 1 -- 172.21.15.5:0/2469400415 shutdown_connections 2024-04-24T06:49:09.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.131+0000 7f702b000640 1 -- 172.21.15.5:0/2469400415 wait complete. 2024-04-24T06:49:09.150 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T06:49:09.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.372+0000 7f4383200640 1 -- 172.21.15.5:0/2298808904 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43841016f0 msgr2=0x7f4384105bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:09.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.372+0000 7f4383200640 1 --2- 172.21.15.5:0/2298808904 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43841016f0 0x7f4384105bb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4370009860 tx=0x7f437002a3d0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.372+0000 7f4383200640 1 -- 172.21.15.5:0/2298808904 shutdown_connections 2024-04-24T06:49:09.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.372+0000 7f4383200640 1 --2- 172.21.15.5:0/2298808904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43841061d0 0x7f4384106630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.372+0000 7f4383200640 1 --2- 172.21.15.5:0/2298808904 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43841016f0 0x7f4384105bb0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.372+0000 7f4383200640 1 --2- 172.21.15.5:0/2298808904 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4384100dd0 0x7f43841011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.372+0000 7f4383200640 1 -- 172.21.15.5:0/2298808904 >> 172.21.15.5:0/2298808904 conn(0x7f4384077ab0 msgr2=0x7f43840fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:09.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.372+0000 7f4383200640 1 -- 172.21.15.5:0/2298808904 shutdown_connections 2024-04-24T06:49:09.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.372+0000 7f4383200640 1 -- 172.21.15.5:0/2298808904 wait complete. 2024-04-24T06:49:09.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.373+0000 7f4383200640 1 Processor -- start 2024-04-24T06:49:09.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.374+0000 7f4383200640 1 -- start start 2024-04-24T06:49:09.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.374+0000 7f4383200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4384100dd0 0x7f438419edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.374+0000 7f4383200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43841016f0 0x7f438419f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.374+0000 7f4383200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43841061d0 0x7f43841a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.374+0000 7f4383200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f43841187f0 con 0x7f43841016f0 2024-04-24T06:49:09.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.374+0000 7f4383200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4384118670 con 0x7f4384100dd0 2024-04-24T06:49:09.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.374+0000 7f4383200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4384118970 con 0x7f43841061d0 2024-04-24T06:49:09.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.374+0000 7f4381400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43841016f0 0x7f438419f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.375+0000 7f4381400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43841016f0 0x7f438419f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32906/0 (socket says 172.21.15.5:32906) 2024-04-24T06:49:09.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.375+0000 7f4381400640 1 -- 172.21.15.5:0/3273355234 learned_addr learned my addr 172.21.15.5:0/3273355234 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:09.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.375+0000 7f4382800640 1 --2- 172.21.15.5:0/3273355234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43841061d0 0x7f43841a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.375+0000 7f4381e00640 1 --2- 172.21.15.5:0/3273355234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4384100dd0 0x7f438419edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.375+0000 7f4381400640 1 -- 172.21.15.5:0/3273355234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43841061d0 msgr2=0x7f43841a36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:09.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.375+0000 7f4381400640 1 --2- 172.21.15.5:0/3273355234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43841061d0 0x7f43841a36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.375+0000 7f4381400640 1 -- 172.21.15.5:0/3273355234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4384100dd0 msgr2=0x7f438419edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:09.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.375+0000 7f4381400640 1 --2- 172.21.15.5:0/3273355234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4384100dd0 0x7f438419edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.375+0000 7f4381400640 1 -- 172.21.15.5:0/3273355234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43841a3de0 con 0x7f43841016f0 2024-04-24T06:49:09.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.375+0000 7f4381e00640 1 --2- 172.21.15.5:0/3273355234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4384100dd0 0x7f438419edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:49:09.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.376+0000 7f4381400640 1 --2- 172.21.15.5:0/3273355234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43841016f0 0x7f438419f330 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4370009980 tx=0x7f4370030950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:09.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.376+0000 7f437aa00640 1 -- 172.21.15.5:0/3273355234 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4370040070 con 0x7f43841016f0 2024-04-24T06:49:09.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.376+0000 7f4383200640 1 -- 172.21.15.5:0/3273355234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43841a4070 con 0x7f43841016f0 2024-04-24T06:49:09.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.376+0000 7f437aa00640 1 -- 172.21.15.5:0/3273355234 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4370005620 con 0x7f43841016f0 2024-04-24T06:49:09.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.376+0000 7f437aa00640 1 -- 172.21.15.5:0/3273355234 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43700049b0 con 0x7f43841016f0 2024-04-24T06:49:09.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.376+0000 7f4383200640 1 -- 172.21.15.5:0/3273355234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43841ab960 con 0x7f43841016f0 2024-04-24T06:49:09.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.379+0000 7f438ba10740 1 Processor -- start 2024-04-24T06:49:09.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.380+0000 7f437aa00640 1 -- 172.21.15.5:0/3273355234 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f4370043070 con 0x7f43841016f0 2024-04-24T06:49:09.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.380+0000 7f438ba10740 1 -- start start 2024-04-24T06:49:09.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.380+0000 7f438ba10740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639fb1cab40 0x5639fb1caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.380+0000 7f438ba10740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639fb1cb460 0x5639fb1d55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.380+0000 7f438ba10740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639fb1d5b30 0x5639fb1d7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.380+0000 7f438ba10740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5639fb1301d0 con 0x5639fb1d5b30 2024-04-24T06:49:09.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.380+0000 7f438ba10740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5639fb0d6450 con 0x5639fb1cb460 2024-04-24T06:49:09.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.380+0000 7f438ba10740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5639fb12c190 con 0x5639fb1cab40 2024-04-24T06:49:09.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.381+0000 7f4381e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639fb1cb460 0x5639fb1d55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.381+0000 7f4382800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639fb1cab40 0x5639fb1caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.381+0000 7f4381e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639fb1cb460 0x5639fb1d55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:41662/0 (socket says 172.21.15.5:41662) 2024-04-24T06:49:09.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.381+0000 7f4381e00640 1 -- 172.21.15.5:0/1673893233 learned_addr learned my addr 172.21.15.5:0/1673893233 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:09.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.381+0000 7f4381400640 1 --2- 172.21.15.5:0/1673893233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639fb1d5b30 0x5639fb1d7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.381+0000 7f4382800640 1 -- 172.21.15.5:0/1673893233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639fb1cb460 msgr2=0x5639fb1d55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:09.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.381+0000 7f4382800640 1 --2- 172.21.15.5:0/1673893233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639fb1cb460 0x5639fb1d55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.381+0000 7f4382800640 1 -- 172.21.15.5:0/1673893233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639fb1d5b30 msgr2=0x5639fb1d7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:09.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.381+0000 7f4382800640 1 --2- 172.21.15.5:0/1673893233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639fb1d5b30 0x5639fb1d7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.381+0000 7f4382800640 1 -- 172.21.15.5:0/1673893233 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639fb193310 con 0x5639fb1cab40 2024-04-24T06:49:09.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.381+0000 7f4381e00640 1 --2- 172.21.15.5:0/1673893233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639fb1cb460 0x5639fb1d55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:49:09.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.381+0000 7f4382800640 1 --2- 172.21.15.5:0/1673893233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639fb1cab40 0x5639fb1caf20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f437c009800 tx=0x7f437c0095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48059626ad746b1a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:09.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.381+0000 7f437aa00640 1 --2- 172.21.15.5:0/3273355234 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f43680a3d20 0x7f43680a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.382+0000 7f436f000640 1 -- 172.21.15.5:0/1673893233 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437c01d070 con 0x5639fb1cab40 2024-04-24T06:49:09.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.382+0000 7f437aa00640 1 -- 172.21.15.5:0/3273355234 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f43700f1520 con 0x7f43841016f0 2024-04-24T06:49:09.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.382+0000 7f436f000640 1 -- 172.21.15.5:0/1673893233 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f437c003e40 con 0x5639fb1cab40 2024-04-24T06:49:09.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.382+0000 7f436f000640 1 -- 172.21.15.5:0/1673893233 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437c018b50 con 0x5639fb1cab40 2024-04-24T06:49:09.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.382+0000 7f4381e00640 1 --2- 172.21.15.5:0/3273355234 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f43680a3d20 0x7f43680a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.382+0000 7f438ba10740 1 -- 172.21.15.5:0/1673893233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639fb1cab40 msgr2=0x5639fb1caf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:09.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.382+0000 7f438ba10740 1 --2- 172.21.15.5:0/1673893233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639fb1cab40 0x5639fb1caf20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f437c009800 tx=0x7f437c0095c0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.383+0000 7f438ba10740 1 -- 172.21.15.5:0/1673893233 shutdown_connections 2024-04-24T06:49:09.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.383+0000 7f438ba10740 1 --2- 172.21.15.5:0/1673893233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639fb1d5b30 0x5639fb1d7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.383+0000 7f438ba10740 1 --2- 172.21.15.5:0/1673893233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639fb1cb460 0x5639fb1d55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.383+0000 7f438ba10740 1 --2- 172.21.15.5:0/1673893233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639fb1cab40 0x5639fb1caf20 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.383+0000 7f438ba10740 1 -- 172.21.15.5:0/1673893233 >> 172.21.15.5:0/1673893233 conn(0x5639fb1c6e60 msgr2=0x5639fb1c7240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:09.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.383+0000 7f4381e00640 1 --2- 172.21.15.5:0/3273355234 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f43680a3d20 0x7f43680a61e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x5639fb1cc100 tx=0x7f4374005f90 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:09.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.383+0000 7f438ba10740 1 -- 172.21.15.5:0/1673893233 shutdown_connections 2024-04-24T06:49:09.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.383+0000 7f438ba10740 1 -- 172.21.15.5:0/1673893233 wait complete. 2024-04-24T06:49:09.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.383+0000 7f438ba10740 1 -- start start 2024-04-24T06:49:09.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.383+0000 7f438ba10740 1 Processor -- start 2024-04-24T06:49:09.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.384+0000 7f438ba10740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639fb1cab40 0x5639fb1d55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.384+0000 7f438ba10740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639fb1cb460 0x5639fb1d28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.384+0000 7f438ba10740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639fb1d5b30 0x5639fb1d2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.384+0000 7f438ba10740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5639fb1d8a10 con 0x5639fb1cb460 2024-04-24T06:49:09.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.384+0000 7f438ba10740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5639fb0e9810 con 0x5639fb1cab40 2024-04-24T06:49:09.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.384+0000 7f438ba10740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5639fb1d8b90 con 0x5639fb1d5b30 2024-04-24T06:49:09.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.384+0000 7f438ba10740 10 client.0 _ll_register_callbacks cb 0x5639fb0b7140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:49:09.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.384+0000 7f4381400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639fb1cb460 0x5639fb1d28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.384+0000 7f4381400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639fb1cb460 0x5639fb1d28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32930/0 (socket says 172.21.15.5:32930) 2024-04-24T06:49:09.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.384+0000 7f4381400640 1 -- 172.21.15.5:0/2951146896 learned_addr learned my addr 172.21.15.5:0/2951146896 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:09.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.384+0000 7f4382800640 1 --2- 172.21.15.5:0/2951146896 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639fb1d5b30 0x5639fb1d2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.384+0000 7f4381e00640 1 --2- 172.21.15.5:0/2951146896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639fb1cab40 0x5639fb1d55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.385+0000 7f4381400640 1 -- 172.21.15.5:0/2951146896 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639fb1d5b30 msgr2=0x5639fb1d2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:09.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.385+0000 7f4381400640 1 --2- 172.21.15.5:0/2951146896 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639fb1d5b30 0x5639fb1d2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.385+0000 7f4381400640 1 -- 172.21.15.5:0/2951146896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639fb1cab40 msgr2=0x5639fb1d55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:09.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.385+0000 7f4381400640 1 --2- 172.21.15.5:0/2951146896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639fb1cab40 0x5639fb1d55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.385+0000 7f4381400640 1 -- 172.21.15.5:0/2951146896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639fb0fa3f0 con 0x5639fb1cb460 2024-04-24T06:49:09.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.385+0000 7f4381400640 1 --2- 172.21.15.5:0/2951146896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639fb1cb460 0x5639fb1d28b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f43700bd220 tx=0x7f4370009490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb9541103985536 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:09.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.385+0000 7f4363400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:49:09.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.385+0000 7f4363400640 1 -- 172.21.15.5:0/2951146896 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437004a040 con 0x5639fb1cb460 2024-04-24T06:49:09.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.385+0000 7f4363400640 1 -- 172.21.15.5:0/2951146896 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f4370031400 con 0x5639fb1cb460 2024-04-24T06:49:09.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.385+0000 7f438ba10740 1 -- 172.21.15.5:0/2951146896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5639fb13dbf0 con 0x5639fb1cb460 2024-04-24T06:49:09.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.386+0000 7f4363400640 1 -- 172.21.15.5:0/2951146896 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f43700bd060 con 0x5639fb1cb460 2024-04-24T06:49:09.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.386+0000 7f438ba10740 10 client.16254 fetch_fsmap learned FSMap version 27 2024-04-24T06:49:09.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.386+0000 7f438ba10740 1 -- 172.21.15.5:0/2951146896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5639fae423f0 con 0x5639fb1cb460 2024-04-24T06:49:09.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.386+0000 7f4363400640 1 -- 172.21.15.5:0/2951146896 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5639fb13dbf0 con 0x5639fb1cb460 2024-04-24T06:49:09.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.386+0000 7f4363400640 1 -- 172.21.15.5:0/2951146896 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f4370004b50 con 0x5639fb1cb460 2024-04-24T06:49:09.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.386+0000 7f438ba10740 10 client.16254 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:49:09.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.386+0000 7f438ba10740 10 client.16254 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T06:49:09.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.386+0000 7f438ba10740 20 client.16254 populate_metadata read hostname 'smithi005' 2024-04-24T06:49:09.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.386+0000 7f438ba10740 1 --2- 172.21.15.5:0/2951146896 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x5639fb1e5c80 0x5639fb1e8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.386+0000 7f438ba10740 4 client.16254 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:49:09.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.386+0000 7f438ba10740 1 -- 172.21.15.5:0/2951146896 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639fb137a60 con 0x5639fb1e5c80 2024-04-24T06:49:09.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.387+0000 7f4381e00640 1 --2- 172.21.15.5:0/2951146896 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x5639fb1e5c80 0x5639fb1e8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.387+0000 7f4381e00640 1 --2- 172.21.15.5:0/2951146896 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x5639fb1e5c80 0x5639fb1e8070 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f25c5c85d20f1b79 server_cookie=f928b6625b1cf1af in_seq=0 out_seq=0 2024-04-24T06:49:09.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.387+0000 7f4363400640 10 client.16254 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:49:09.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.388+0000 7f4363400640 1 -- 172.21.15.5:0/2951146896 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5639fb137a60 con 0x5639fb1e5c80 2024-04-24T06:49:09.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.388+0000 7f4363400640 10 client.16254 handle_command_reply: tid=0 2024-04-24T06:49:09.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.388+0000 7f438ba10740 1 client.16254 shutdown 2024-04-24T06:49:09.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.388+0000 7f438ba10740 1 -- 172.21.15.5:0/2951146896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639fb1cb460 msgr2=0x5639fb1d28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:09.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.388+0000 7f438ba10740 1 --2- 172.21.15.5:0/2951146896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639fb1cb460 0x5639fb1d28b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f43700bd220 tx=0x7f4370009490 comp rx=0 tx=0).stop 2024-04-24T06:49:09.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.388+0000 7f438ba10740 1 -- 172.21.15.5:0/2951146896 shutdown_connections 2024-04-24T06:49:09.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.388+0000 7f438ba10740 1 --2- 172.21.15.5:0/2951146896 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x5639fb1e5c80 0x5639fb1e8070 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.388+0000 7f438ba10740 1 --2- 172.21.15.5:0/2951146896 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639fb1d5b30 0x5639fb1d2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.388+0000 7f438ba10740 1 --2- 172.21.15.5:0/2951146896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639fb1cb460 0x5639fb1d28b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.388+0000 7f438ba10740 1 --2- 172.21.15.5:0/2951146896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639fb1cab40 0x5639fb1d55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.388+0000 7f4363400640 0 client.16254 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:49:09.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.388+0000 7f438ba10740 1 -- 172.21.15.5:0/2951146896 >> 172.21.15.5:0/2951146896 conn(0x5639fb1c6e60 msgr2=0x5639fb1ccc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:09.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.389+0000 7f438ba10740 1 -- 172.21.15.5:0/2951146896 shutdown_connections 2024-04-24T06:49:09.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.389+0000 7f438ba10740 1 -- 172.21.15.5:0/2951146896 wait complete. 2024-04-24T06:49:09.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.389+0000 7f438ba10740 20 client.16254 trim_cache size 0 max 16384 2024-04-24T06:49:09.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.404+0000 7f438ba10740 1 Processor -- start 2024-04-24T06:49:09.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.404+0000 7f438ba10740 1 -- start start 2024-04-24T06:49:09.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.404+0000 7f438ba10740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639fb1cab40 0x5639fb1c5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.404+0000 7f438ba10740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639fb1cb460 0x5639fb1d50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.404+0000 7f438ba10740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639fb1d5b30 0x5639fb1ed900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.404+0000 7f438ba10740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f43841b6a90 con 0x5639fb1d5b30 2024-04-24T06:49:09.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.404+0000 7f438ba10740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f43841b4360 con 0x5639fb1cab40 2024-04-24T06:49:09.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.404+0000 7f438ba10740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f43841b21b0 con 0x5639fb1cb460 2024-04-24T06:49:09.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.404+0000 7f4382800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639fb1d5b30 0x5639fb1ed900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.404+0000 7f4382800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639fb1d5b30 0x5639fb1ed900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32944/0 (socket says 172.21.15.5:32944) 2024-04-24T06:49:09.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.405+0000 7f4382800640 1 -- 172.21.15.5:0/3523803766 learned_addr learned my addr 172.21.15.5:0/3523803766 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:09.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.405+0000 7f4381e00640 1 --2- 172.21.15.5:0/3523803766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639fb1cab40 0x5639fb1c5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.405+0000 7f4381400640 1 --2- 172.21.15.5:0/3523803766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639fb1cb460 0x5639fb1d50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.405+0000 7f4382800640 1 -- 172.21.15.5:0/3523803766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639fb1cb460 msgr2=0x5639fb1d50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:09.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.405+0000 7f4382800640 1 --2- 172.21.15.5:0/3523803766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639fb1cb460 0x5639fb1d50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.405+0000 7f4382800640 1 -- 172.21.15.5:0/3523803766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639fb1cab40 msgr2=0x5639fb1c5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:09.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.405+0000 7f4382800640 1 --2- 172.21.15.5:0/3523803766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639fb1cab40 0x5639fb1c5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.405+0000 7f4382800640 1 -- 172.21.15.5:0/3523803766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639fb1d3310 con 0x5639fb1d5b30 2024-04-24T06:49:09.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.405+0000 7f4382800640 1 --2- 172.21.15.5:0/3523803766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639fb1d5b30 0x5639fb1ed900 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f437c01b040 tx=0x7f437c018730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18130e58b5da0b13 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:09.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.406+0000 7f436e600640 1 -- 172.21.15.5:0/3523803766 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437c029070 con 0x5639fb1d5b30 2024-04-24T06:49:09.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.406+0000 7f436e600640 1 -- 172.21.15.5:0/3523803766 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f437c01e040 con 0x5639fb1d5b30 2024-04-24T06:49:09.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.406+0000 7f436e600640 1 -- 172.21.15.5:0/3523803766 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437c01d030 con 0x5639fb1d5b30 2024-04-24T06:49:09.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.406+0000 7f438ba10740 1 -- 172.21.15.5:0/3523803766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639fb1d5b30 msgr2=0x5639fb1ed900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:09.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.406+0000 7f438ba10740 1 --2- 172.21.15.5:0/3523803766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639fb1d5b30 0x5639fb1ed900 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f437c01b040 tx=0x7f437c018730 comp rx=0 tx=0).stop 2024-04-24T06:49:09.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.406+0000 7f438ba10740 1 -- 172.21.15.5:0/3523803766 shutdown_connections 2024-04-24T06:49:09.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.406+0000 7f438ba10740 1 --2- 172.21.15.5:0/3523803766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639fb1d5b30 0x5639fb1ed900 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.406+0000 7f438ba10740 1 --2- 172.21.15.5:0/3523803766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639fb1cb460 0x5639fb1d50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.406+0000 7f438ba10740 1 --2- 172.21.15.5:0/3523803766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639fb1cab40 0x5639fb1c5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.406+0000 7f438ba10740 1 -- 172.21.15.5:0/3523803766 >> 172.21.15.5:0/3523803766 conn(0x5639fb1c6e60 msgr2=0x5639fb1c56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:09.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.407+0000 7f438ba10740 1 -- 172.21.15.5:0/3523803766 shutdown_connections 2024-04-24T06:49:09.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.407+0000 7f438ba10740 1 -- 172.21.15.5:0/3523803766 wait complete. 2024-04-24T06:49:09.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.407+0000 7f438ba10740 1 -- start start 2024-04-24T06:49:09.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.407+0000 7f438ba10740 1 Processor -- start 2024-04-24T06:49:09.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.408+0000 7f438ba10740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639fb1cab40 0x5639fb1d6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.408+0000 7f438ba10740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639fb1cb460 0x5639fb1d7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.408+0000 7f438ba10740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639fb1d5b30 0x5639fb1dc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.408+0000 7f438ba10740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f43841b2820 con 0x5639fb1cab40 2024-04-24T06:49:09.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.408+0000 7f438ba10740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f43841b3c90 con 0x5639fb1cb460 2024-04-24T06:49:09.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.408+0000 7f438ba10740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f43841b2ef0 con 0x5639fb1d5b30 2024-04-24T06:49:09.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.408+0000 7f438ba10740 10 client.0 _ll_register_callbacks cb 0x7f43700097b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:49:09.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.408+0000 7f4381e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639fb1cab40 0x5639fb1d6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.408+0000 7f4381e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639fb1cab40 0x5639fb1d6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32954/0 (socket says 172.21.15.5:32954) 2024-04-24T06:49:09.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.408+0000 7f4381e00640 1 -- 172.21.15.5:0/1918213158 learned_addr learned my addr 172.21.15.5:0/1918213158 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:09.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.408+0000 7f4381400640 1 --2- 172.21.15.5:0/1918213158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639fb1cb460 0x5639fb1d7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.408+0000 7f4382800640 1 --2- 172.21.15.5:0/1918213158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639fb1d5b30 0x5639fb1dc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.408+0000 7f4381e00640 1 -- 172.21.15.5:0/1918213158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639fb1d5b30 msgr2=0x5639fb1dc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:09.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.408+0000 7f4381e00640 1 --2- 172.21.15.5:0/1918213158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639fb1d5b30 0x5639fb1dc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.408+0000 7f4381e00640 1 -- 172.21.15.5:0/1918213158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639fb1cb460 msgr2=0x5639fb1d7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:09.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.408+0000 7f4381e00640 1 --2- 172.21.15.5:0/1918213158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639fb1cb460 0x5639fb1d7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.408+0000 7f4381e00640 1 -- 172.21.15.5:0/1918213158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639faf0c7d0 con 0x5639fb1cab40 2024-04-24T06:49:09.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.408+0000 7f4381e00640 1 --2- 172.21.15.5:0/1918213158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639fb1cab40 0x5639fb1d6f20 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4374013010 tx=0x7f43740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d36d804426580e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:09.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.409+0000 7f4363400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:49:09.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.409+0000 7f4363400640 1 -- 172.21.15.5:0/1918213158 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437400fe10 con 0x5639fb1cab40 2024-04-24T06:49:09.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.409+0000 7f4363400640 1 -- 172.21.15.5:0/1918213158 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f437402b380 con 0x5639fb1cab40 2024-04-24T06:49:09.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.409+0000 7f438ba10740 1 -- 172.21.15.5:0/1918213158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f437c01e040 con 0x5639fb1cab40 2024-04-24T06:49:09.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.409+0000 7f4363400640 1 -- 172.21.15.5:0/1918213158 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f437402b580 con 0x5639fb1cab40 2024-04-24T06:49:09.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.409+0000 7f438ba10740 10 client.16260 fetch_fsmap learned FSMap version 27 2024-04-24T06:49:09.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.409+0000 7f438ba10740 1 -- 172.21.15.5:0/1918213158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5639fb24c7e0 con 0x5639fb1cab40 2024-04-24T06:49:09.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.409+0000 7f4363400640 1 -- 172.21.15.5:0/1918213158 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f437c01e040 con 0x5639fb1cab40 2024-04-24T06:49:09.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.409+0000 7f4363400640 1 -- 172.21.15.5:0/1918213158 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f4374004a00 con 0x5639fb1cab40 2024-04-24T06:49:09.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.410+0000 7f438ba10740 10 client.16260 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:49:09.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.410+0000 7f438ba10740 10 client.16260 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T06:49:09.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.410+0000 7f438ba10740 20 client.16260 populate_metadata read hostname 'smithi005' 2024-04-24T06:49:09.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.410+0000 7f438ba10740 1 --2- 172.21.15.5:0/1918213158 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x5639fb24ceb0 0x5639fb24f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.410+0000 7f438ba10740 4 client.16260 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:49:09.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.410+0000 7f438ba10740 1 -- 172.21.15.5:0/1918213158 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5639fb0b34d0 con 0x5639fb24ceb0 2024-04-24T06:49:09.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.410+0000 7f4381400640 1 --2- 172.21.15.5:0/1918213158 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x5639fb24ceb0 0x5639fb24f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.411+0000 7f4381400640 1 --2- 172.21.15.5:0/1918213158 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x5639fb24ceb0 0x5639fb24f2f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d203b0c2988807a2 server_cookie=aeac8f9bcb611775 in_seq=0 out_seq=0 2024-04-24T06:49:09.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.411+0000 7f4363400640 10 client.16260 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:49:09.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.411+0000 7f4363400640 1 -- 172.21.15.5:0/1918213158 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5639fb0b34d0 con 0x5639fb24ceb0 2024-04-24T06:49:09.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.411+0000 7f4363400640 10 client.16260 handle_command_reply: tid=0 2024-04-24T06:49:09.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.411+0000 7f438ba10740 1 client.16260 shutdown 2024-04-24T06:49:09.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.411+0000 7f438ba10740 1 -- 172.21.15.5:0/1918213158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639fb1cab40 msgr2=0x5639fb1d6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:09.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.411+0000 7f438ba10740 1 --2- 172.21.15.5:0/1918213158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639fb1cab40 0x5639fb1d6f20 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4374013010 tx=0x7f43740046c0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.411+0000 7f438ba10740 1 -- 172.21.15.5:0/1918213158 shutdown_connections 2024-04-24T06:49:09.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.411+0000 7f438ba10740 1 --2- 172.21.15.5:0/1918213158 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x5639fb24ceb0 0x5639fb24f2f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.411+0000 7f438ba10740 1 --2- 172.21.15.5:0/1918213158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639fb1d5b30 0x5639fb1dc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.411+0000 7f438ba10740 1 --2- 172.21.15.5:0/1918213158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639fb1cb460 0x5639fb1d7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.411+0000 7f438ba10740 1 --2- 172.21.15.5:0/1918213158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5639fb1cab40 0x5639fb1d6f20 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.411+0000 7f438ba10740 1 -- 172.21.15.5:0/1918213158 >> 172.21.15.5:0/1918213158 conn(0x5639fb1c6e60 msgr2=0x5639fb1c5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:09.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.411+0000 7f4363400640 0 client.16260 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:49:09.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.411+0000 7f438ba10740 1 -- 172.21.15.5:0/1918213158 shutdown_connections 2024-04-24T06:49:09.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.411+0000 7f438ba10740 1 -- 172.21.15.5:0/1918213158 wait complete. 2024-04-24T06:49:09.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.411+0000 7f438ba10740 20 client.16260 trim_cache size 0 max 16384 2024-04-24T06:49:09.424 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:49:09.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.412+0000 7f4383200640 1 -- 172.21.15.5:0/3273355234 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f43680a3d20 msgr2=0x7f43680a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:09.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.412+0000 7f4383200640 1 --2- 172.21.15.5:0/3273355234 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f43680a3d20 0x7f43680a61e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x5639fb1cc100 tx=0x7f4374005f90 comp rx=0 tx=0).stop 2024-04-24T06:49:09.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.412+0000 7f4383200640 1 -- 172.21.15.5:0/3273355234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43841016f0 msgr2=0x7f438419f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:09.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.412+0000 7f4383200640 1 --2- 172.21.15.5:0/3273355234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43841016f0 0x7f438419f330 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4370009980 tx=0x7f4370030950 comp rx=0 tx=0).stop 2024-04-24T06:49:09.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.412+0000 7f4383200640 1 -- 172.21.15.5:0/3273355234 shutdown_connections 2024-04-24T06:49:09.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.412+0000 7f4383200640 1 --2- 172.21.15.5:0/3273355234 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f43680a3d20 0x7f43680a61e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.412+0000 7f4383200640 1 --2- 172.21.15.5:0/3273355234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43841061d0 0x7f43841a36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.412+0000 7f4383200640 1 --2- 172.21.15.5:0/3273355234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43841016f0 0x7f438419f330 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.412+0000 7f4383200640 1 --2- 172.21.15.5:0/3273355234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4384100dd0 0x7f438419edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.412+0000 7f4383200640 1 -- 172.21.15.5:0/3273355234 >> 172.21.15.5:0/3273355234 conn(0x7f4384077ab0 msgr2=0x7f4384103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:09.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.412+0000 7f4383200640 1 -- 172.21.15.5:0/3273355234 shutdown_connections 2024-04-24T06:49:09.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.412+0000 7f4383200640 1 -- 172.21.15.5:0/3273355234 wait complete. 2024-04-24T06:49:09.430 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:09 smithi005 ceph-mon[27798]: pgmap v156: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-24T06:49:09.430 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:09 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/937725176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:49:09.434 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T06:49:09.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.648+0000 7f6712e00640 1 -- 172.21.15.5:0/1446089029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f670c112de0 msgr2=0x7f670c1151d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:09.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.648+0000 7f6712e00640 1 --2- 172.21.15.5:0/1446089029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f670c112de0 0x7f670c1151d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6704009ca0 tx=0x7f670402a950 comp rx=0 tx=0).stop 2024-04-24T06:49:09.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.648+0000 7f6712e00640 1 -- 172.21.15.5:0/1446089029 shutdown_connections 2024-04-24T06:49:09.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.648+0000 7f6712e00640 1 --2- 172.21.15.5:0/1446089029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f670c112de0 0x7f670c1151d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.648+0000 7f6712e00640 1 --2- 172.21.15.5:0/1446089029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f670c074cc0 0x7f670c1128a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.648+0000 7f6712e00640 1 --2- 172.21.15.5:0/1446089029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f670c0743a0 0x7f670c074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.648+0000 7f6712e00640 1 -- 172.21.15.5:0/1446089029 >> 172.21.15.5:0/1446089029 conn(0x7f670c0fd8e0 msgr2=0x7f670c0ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:09.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.648+0000 7f6712e00640 1 -- 172.21.15.5:0/1446089029 shutdown_connections 2024-04-24T06:49:09.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.648+0000 7f6712e00640 1 -- 172.21.15.5:0/1446089029 wait complete. 2024-04-24T06:49:09.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.650+0000 7f6712e00640 1 Processor -- start 2024-04-24T06:49:09.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.650+0000 7f6712e00640 1 -- start start 2024-04-24T06:49:09.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.650+0000 7f6712e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f670c0743a0 0x7f670c19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.650+0000 7f6712e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f670c074cc0 0x7f670c19f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.650+0000 7f6712e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f670c112de0 0x7f670c1a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.650+0000 7f6712e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f670c1187f0 con 0x7f670c074cc0 2024-04-24T06:49:09.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.650+0000 7f6712e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f670c118670 con 0x7f670c0743a0 2024-04-24T06:49:09.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.650+0000 7f6712e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f670c118970 con 0x7f670c112de0 2024-04-24T06:49:09.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.651+0000 7f6711a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f670c0743a0 0x7f670c19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.651+0000 7f6712400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f670c112de0 0x7f670c1a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.651+0000 7f6711000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f670c074cc0 0x7f670c19f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.651+0000 7f6711a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f670c0743a0 0x7f670c19ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:41712/0 (socket says 172.21.15.5:41712) 2024-04-24T06:49:09.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.651+0000 7f6711a00640 1 -- 172.21.15.5:0/3988690087 learned_addr learned my addr 172.21.15.5:0/3988690087 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:09.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.651+0000 7f6711a00640 1 -- 172.21.15.5:0/3988690087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f670c112de0 msgr2=0x7f670c1a3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:09.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.651+0000 7f6711a00640 1 --2- 172.21.15.5:0/3988690087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f670c112de0 0x7f670c1a3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.651+0000 7f6711a00640 1 -- 172.21.15.5:0/3988690087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f670c074cc0 msgr2=0x7f670c19f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:09.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.651+0000 7f6711a00640 1 --2- 172.21.15.5:0/3988690087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f670c074cc0 0x7f670c19f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.651+0000 7f6711a00640 1 -- 172.21.15.5:0/3988690087 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f670c1a3d60 con 0x7f670c0743a0 2024-04-24T06:49:09.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.651+0000 7f6712400640 1 --2- 172.21.15.5:0/3988690087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f670c112de0 0x7f670c1a3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:49:09.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.652+0000 7f6711a00640 1 --2- 172.21.15.5:0/3988690087 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f670c0743a0 0x7f670c19ed70 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f670000e9f0 tx=0x7f670000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:09.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.652+0000 7f670a000640 1 -- 172.21.15.5:0/3988690087 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670000cd80 con 0x7f670c0743a0 2024-04-24T06:49:09.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.652+0000 7f670a000640 1 -- 172.21.15.5:0/3988690087 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6700005500 con 0x7f670c0743a0 2024-04-24T06:49:09.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.652+0000 7f6712e00640 1 -- 172.21.15.5:0/3988690087 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f670c1a4050 con 0x7f670c0743a0 2024-04-24T06:49:09.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.652+0000 7f670a000640 1 -- 172.21.15.5:0/3988690087 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6700010690 con 0x7f670c0743a0 2024-04-24T06:49:09.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.652+0000 7f6712e00640 1 -- 172.21.15.5:0/3988690087 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f670c1ab8e0 con 0x7f670c0743a0 2024-04-24T06:49:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.656+0000 7f670a000640 1 -- 172.21.15.5:0/3988690087 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f67000046c0 con 0x7f670c0743a0 2024-04-24T06:49:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.656+0000 7f67173ca740 1 Processor -- start 2024-04-24T06:49:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.656+0000 7f67173ca740 1 -- start start 2024-04-24T06:49:09.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.656+0000 7f67173ca740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a263f41b40 0x55a263f41f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.656+0000 7f67173ca740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a263f42460 0x55a263f4c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.656+0000 7f67173ca740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a263f4cb30 0x55a263f4ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.657+0000 7f67173ca740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a263ea71d0 con 0x55a263f41b40 2024-04-24T06:49:09.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.657+0000 7f67173ca740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a263e4d450 con 0x55a263f42460 2024-04-24T06:49:09.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.657+0000 7f67173ca740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a263ea3190 con 0x55a263f4cb30 2024-04-24T06:49:09.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.657+0000 7f6711a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a263f42460 0x55a263f4c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.657+0000 7f6712400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a263f41b40 0x55a263f41f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.657+0000 7f6711000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a263f4cb30 0x55a263f4ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.657+0000 7f6711a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a263f42460 0x55a263f4c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:41728/0 (socket says 172.21.15.5:41728) 2024-04-24T06:49:09.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.657+0000 7f6711a00640 1 -- 172.21.15.5:0/3217700247 learned_addr learned my addr 172.21.15.5:0/3217700247 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:09.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.657+0000 7f6711a00640 1 -- 172.21.15.5:0/3217700247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a263f4cb30 msgr2=0x55a263f4ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:09.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.657+0000 7f6711a00640 1 --2- 172.21.15.5:0/3217700247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a263f4cb30 0x55a263f4ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.657+0000 7f6711a00640 1 -- 172.21.15.5:0/3217700247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a263f41b40 msgr2=0x55a263f41f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:09.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.657+0000 7f6711a00640 1 --2- 172.21.15.5:0/3217700247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a263f41b40 0x55a263f41f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.657+0000 7f6711a00640 1 -- 172.21.15.5:0/3217700247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a263f0a310 con 0x55a263f42460 2024-04-24T06:49:09.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.658+0000 7f6711a00640 1 --2- 172.21.15.5:0/3217700247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a263f42460 0x55a263f4c5f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f67000ce280 tx=0x7f67000d4ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12824f32883210bc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:09.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.658+0000 7f670a000640 1 --2- 172.21.15.5:0/3988690087 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f66f40a3d20 0x7f66f40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.658+0000 7f6712400640 1 --2- 172.21.15.5:0/3988690087 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f66f40a3d20 0x7f66f40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.658+0000 7f66faa00640 1 -- 172.21.15.5:0/3217700247 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6700098e30 con 0x55a263f42460 2024-04-24T06:49:09.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.658+0000 7f670a000640 1 -- 172.21.15.5:0/3988690087 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f67000d0c40 con 0x7f670c0743a0 2024-04-24T06:49:09.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.658+0000 7f66faa00640 1 -- 172.21.15.5:0/3217700247 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f670009d290 con 0x55a263f42460 2024-04-24T06:49:09.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.658+0000 7f66faa00640 1 -- 172.21.15.5:0/3217700247 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67000a5160 con 0x55a263f42460 2024-04-24T06:49:09.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.659+0000 7f67173ca740 1 -- 172.21.15.5:0/3217700247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a263f42460 msgr2=0x55a263f4c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:09.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.659+0000 7f67173ca740 1 --2- 172.21.15.5:0/3217700247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a263f42460 0x55a263f4c5f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f67000ce280 tx=0x7f67000d4ce0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.659+0000 7f6712400640 1 --2- 172.21.15.5:0/3988690087 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f66f40a3d20 0x7f66f40a61e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6704004af0 tx=0x7f6704005d80 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:09.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.659+0000 7f67173ca740 1 -- 172.21.15.5:0/3217700247 shutdown_connections 2024-04-24T06:49:09.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.659+0000 7f67173ca740 1 --2- 172.21.15.5:0/3217700247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a263f4cb30 0x55a263f4ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.659+0000 7f67173ca740 1 --2- 172.21.15.5:0/3217700247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a263f42460 0x55a263f4c5f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.659+0000 7f67173ca740 1 --2- 172.21.15.5:0/3217700247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a263f41b40 0x55a263f41f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.659+0000 7f67173ca740 1 -- 172.21.15.5:0/3217700247 >> 172.21.15.5:0/3217700247 conn(0x55a263f3de60 msgr2=0x55a263f3e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:09.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.659+0000 7f67173ca740 1 -- 172.21.15.5:0/3217700247 shutdown_connections 2024-04-24T06:49:09.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.659+0000 7f67173ca740 1 -- 172.21.15.5:0/3217700247 wait complete. 2024-04-24T06:49:09.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.660+0000 7f67173ca740 1 -- start start 2024-04-24T06:49:09.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.660+0000 7f67173ca740 1 Processor -- start 2024-04-24T06:49:09.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.660+0000 7f67173ca740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a263f41b40 0x55a263f4c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.660+0000 7f67173ca740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a263f42460 0x55a263f498b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.660+0000 7f67173ca740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a263f4cb30 0x55a263f49fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.661+0000 7f67173ca740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a263f4f9c0 con 0x55a263f42460 2024-04-24T06:49:09.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.661+0000 7f67173ca740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a263e60810 con 0x55a263f41b40 2024-04-24T06:49:09.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.661+0000 7f6711a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a263f41b40 0x55a263f4c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.661+0000 7f67173ca740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a263f4fb40 con 0x55a263f4cb30 2024-04-24T06:49:09.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.661+0000 7f6711000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a263f42460 0x55a263f498b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.661+0000 7f6712400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a263f4cb30 0x55a263f49fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.661+0000 7f6711a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a263f41b40 0x55a263f4c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:41732/0 (socket says 172.21.15.5:41732) 2024-04-24T06:49:09.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.661+0000 7f6711a00640 1 -- 172.21.15.5:0/1180717553 learned_addr learned my addr 172.21.15.5:0/1180717553 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:09.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.661+0000 7f67173ca740 10 client.0 _ll_register_callbacks cb 0x55a263e2e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:49:09.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.661+0000 7f6711a00640 1 -- 172.21.15.5:0/1180717553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a263f4cb30 msgr2=0x55a263f49fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:09.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.661+0000 7f6711a00640 1 --2- 172.21.15.5:0/1180717553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a263f4cb30 0x55a263f49fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.661+0000 7f6711a00640 1 -- 172.21.15.5:0/1180717553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a263f42460 msgr2=0x55a263f498b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:09.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.661+0000 7f6711a00640 1 --2- 172.21.15.5:0/1180717553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a263f42460 0x55a263f498b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.661+0000 7f6711a00640 1 -- 172.21.15.5:0/1180717553 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a263e713f0 con 0x55a263f41b40 2024-04-24T06:49:09.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.662+0000 7f6711a00640 1 --2- 172.21.15.5:0/1180717553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a263f41b40 0x55a263f4c5c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f67000d9700 tx=0x7f6700009720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=207ecce8c782d956 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:09.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.662+0000 7f66ef400640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:49:09.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.662+0000 7f66ef400640 1 -- 172.21.15.5:0/1180717553 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67000a5370 con 0x55a263f41b40 2024-04-24T06:49:09.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.662+0000 7f66ef400640 1 -- 172.21.15.5:0/1180717553 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f67000ce5c0 con 0x55a263f41b40 2024-04-24T06:49:09.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.662+0000 7f67173ca740 1 -- 172.21.15.5:0/1180717553 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f670009d290 con 0x55a263f41b40 2024-04-24T06:49:09.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.662+0000 7f66ef400640 1 -- 172.21.15.5:0/1180717553 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f67000a55b0 con 0x55a263f41b40 2024-04-24T06:49:09.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.662+0000 7f67173ca740 10 client.26405 fetch_fsmap learned FSMap version 27 2024-04-24T06:49:09.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.662+0000 7f67173ca740 1 -- 172.21.15.5:0/1180717553 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a263bb93f0 con 0x55a263f41b40 2024-04-24T06:49:09.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.663+0000 7f66ef400640 1 -- 172.21.15.5:0/1180717553 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f670009d290 con 0x55a263f41b40 2024-04-24T06:49:09.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.663+0000 7f66ef400640 1 -- 172.21.15.5:0/1180717553 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f670009cdf0 con 0x55a263f41b40 2024-04-24T06:49:09.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.663+0000 7f67173ca740 10 client.26405 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:49:09.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.663+0000 7f67173ca740 10 client.26405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:49:09.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.663+0000 7f67173ca740 20 client.26405 populate_metadata read hostname 'smithi005' 2024-04-24T06:49:09.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.663+0000 7f67173ca740 1 --2- 172.21.15.5:0/1180717553 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a263f5cbe0 0x55a263f5efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.663+0000 7f67173ca740 4 client.26405 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:49:09.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.663+0000 7f67173ca740 1 -- 172.21.15.5:0/1180717553 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a263eb4bf0 con 0x55a263f5cbe0 2024-04-24T06:49:09.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.663+0000 7f6711000640 1 --2- 172.21.15.5:0/1180717553 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a263f5cbe0 0x55a263f5efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.664+0000 7f6711000640 1 --2- 172.21.15.5:0/1180717553 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a263f5cbe0 0x55a263f5efd0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=489a0ed5522099a3 server_cookie=70aa73d7ca91927 in_seq=0 out_seq=0 2024-04-24T06:49:09.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.664+0000 7f66ef400640 10 client.26405 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:49:09.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.665+0000 7f66ef400640 1 -- 172.21.15.5:0/1180717553 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a263eb4bf0 con 0x55a263f5cbe0 2024-04-24T06:49:09.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.665+0000 7f66ef400640 10 client.26405 handle_command_reply: tid=0 2024-04-24T06:49:09.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.665+0000 7f67173ca740 1 client.26405 shutdown 2024-04-24T06:49:09.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.665+0000 7f67173ca740 1 -- 172.21.15.5:0/1180717553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a263f41b40 msgr2=0x55a263f4c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:09.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.665+0000 7f67173ca740 1 --2- 172.21.15.5:0/1180717553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a263f41b40 0x55a263f4c5c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f67000d9700 tx=0x7f6700009720 comp rx=0 tx=0).stop 2024-04-24T06:49:09.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.665+0000 7f67173ca740 1 -- 172.21.15.5:0/1180717553 shutdown_connections 2024-04-24T06:49:09.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.665+0000 7f67173ca740 1 --2- 172.21.15.5:0/1180717553 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a263f5cbe0 0x55a263f5efd0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.665+0000 7f67173ca740 1 --2- 172.21.15.5:0/1180717553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a263f4cb30 0x55a263f49fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.665+0000 7f66ef400640 0 client.26405 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:49:09.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.665+0000 7f67173ca740 1 --2- 172.21.15.5:0/1180717553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a263f42460 0x55a263f498b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.665+0000 7f67173ca740 1 --2- 172.21.15.5:0/1180717553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a263f41b40 0x55a263f4c5c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.665+0000 7f67173ca740 1 -- 172.21.15.5:0/1180717553 >> 172.21.15.5:0/1180717553 conn(0x55a263f3de60 msgr2=0x55a263f43b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:09.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.665+0000 7f67173ca740 1 -- 172.21.15.5:0/1180717553 shutdown_connections 2024-04-24T06:49:09.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.665+0000 7f67173ca740 1 -- 172.21.15.5:0/1180717553 wait complete. 2024-04-24T06:49:09.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.665+0000 7f67173ca740 20 client.26405 trim_cache size 0 max 16384 2024-04-24T06:49:09.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.680+0000 7f67173ca740 1 Processor -- start 2024-04-24T06:49:09.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.680+0000 7f67173ca740 1 -- start start 2024-04-24T06:49:09.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.681+0000 7f67173ca740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a263f41b40 0x55a263f44080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.681+0000 7f67173ca740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a263f42460 0x55a263f4c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.681+0000 7f67173ca740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a263f4cb30 0x55a263f64570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.681+0000 7f67173ca740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f670c1b6a40 con 0x55a263f4cb30 2024-04-24T06:49:09.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.681+0000 7f67173ca740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f670c1b4310 con 0x55a263f41b40 2024-04-24T06:49:09.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.681+0000 7f67173ca740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f670c1b2160 con 0x55a263f42460 2024-04-24T06:49:09.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.681+0000 7f6712400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a263f4cb30 0x55a263f64570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.681+0000 7f6711a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a263f41b40 0x55a263f44080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.681+0000 7f6712400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a263f4cb30 0x55a263f64570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33004/0 (socket says 172.21.15.5:33004) 2024-04-24T06:49:09.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.681+0000 7f6711a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a263f41b40 0x55a263f44080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:41748/0 (socket says 172.21.15.5:41748) 2024-04-24T06:49:09.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.681+0000 7f6712400640 1 -- 172.21.15.5:0/1121554023 learned_addr learned my addr 172.21.15.5:0/1121554023 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:09.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.681+0000 7f6711000640 1 --2- 172.21.15.5:0/1121554023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a263f42460 0x55a263f4c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.682+0000 7f6711000640 1 -- 172.21.15.5:0/1121554023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a263f41b40 msgr2=0x55a263f44080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:09.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.682+0000 7f6711000640 1 --2- 172.21.15.5:0/1121554023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a263f41b40 0x55a263f44080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.682+0000 7f6711000640 1 -- 172.21.15.5:0/1121554023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a263f4cb30 msgr2=0x55a263f64570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:09.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.682+0000 7f6711000640 1 --2- 172.21.15.5:0/1121554023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a263f4cb30 0x55a263f64570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.682+0000 7f6711000640 1 -- 172.21.15.5:0/1121554023 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a263f42290 con 0x55a263f42460 2024-04-24T06:49:09.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.682+0000 7f6711000640 1 --2- 172.21.15.5:0/1121554023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a263f42460 0x55a263f4c0f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x55a263f5f510 tx=0x7f66fc005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55ee4484c3f13c30 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:09.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.682+0000 7f66fa000640 1 -- 172.21.15.5:0/1121554023 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66fc0203d0 con 0x55a263f42460 2024-04-24T06:49:09.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.683+0000 7f66fa000640 1 -- 172.21.15.5:0/1121554023 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66fc026070 con 0x55a263f42460 2024-04-24T06:49:09.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.683+0000 7f66fa000640 1 -- 172.21.15.5:0/1121554023 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66fc01ee20 con 0x55a263f42460 2024-04-24T06:49:09.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.683+0000 7f67173ca740 1 -- 172.21.15.5:0/1121554023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a263f42460 msgr2=0x55a263f4c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:09.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.683+0000 7f67173ca740 1 --2- 172.21.15.5:0/1121554023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a263f42460 0x55a263f4c0f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x55a263f5f510 tx=0x7f66fc005f90 comp rx=0 tx=0).stop 2024-04-24T06:49:09.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.683+0000 7f67173ca740 1 -- 172.21.15.5:0/1121554023 shutdown_connections 2024-04-24T06:49:09.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.683+0000 7f67173ca740 1 --2- 172.21.15.5:0/1121554023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a263f4cb30 0x55a263f64570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.683+0000 7f67173ca740 1 --2- 172.21.15.5:0/1121554023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a263f42460 0x55a263f4c0f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.683+0000 7f67173ca740 1 --2- 172.21.15.5:0/1121554023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a263f41b40 0x55a263f44080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.683+0000 7f67173ca740 1 -- 172.21.15.5:0/1121554023 >> 172.21.15.5:0/1121554023 conn(0x55a263f3de60 msgr2=0x55a263f3e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:09.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.683+0000 7f67173ca740 1 -- 172.21.15.5:0/1121554023 shutdown_connections 2024-04-24T06:49:09.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.683+0000 7f67173ca740 1 -- 172.21.15.5:0/1121554023 wait complete. 2024-04-24T06:49:09.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.684+0000 7f67173ca740 1 -- start start 2024-04-24T06:49:09.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.684+0000 7f67173ca740 1 Processor -- start 2024-04-24T06:49:09.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.684+0000 7f67173ca740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a263f41b40 0x55a263f4d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.684+0000 7f67173ca740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a263f42460 0x55a263f4ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.684+0000 7f67173ca740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a263f4cb30 0x55a263f4e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.684+0000 7f67173ca740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f670c1b27d0 con 0x55a263f4cb30 2024-04-24T06:49:09.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.684+0000 7f67173ca740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f670c1b3c40 con 0x55a263f41b40 2024-04-24T06:49:09.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.684+0000 7f67173ca740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f670c1b2ea0 con 0x55a263f42460 2024-04-24T06:49:09.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.684+0000 7f67173ca740 10 client.0 _ll_register_callbacks cb 0x7f6700005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:49:09.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.685+0000 7f6712400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a263f4cb30 0x55a263f4e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.685+0000 7f6712400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a263f4cb30 0x55a263f4e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33006/0 (socket says 172.21.15.5:33006) 2024-04-24T06:49:09.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.685+0000 7f6712400640 1 -- 172.21.15.5:0/804058206 learned_addr learned my addr 172.21.15.5:0/804058206 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:09.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.685+0000 7f6711000640 1 --2- 172.21.15.5:0/804058206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a263f42460 0x55a263f4ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.685+0000 7f6711a00640 1 --2- 172.21.15.5:0/804058206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a263f41b40 0x55a263f4d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.685+0000 7f6712400640 1 -- 172.21.15.5:0/804058206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a263f42460 msgr2=0x55a263f4ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:09.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.685+0000 7f6712400640 1 --2- 172.21.15.5:0/804058206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a263f42460 0x55a263f4ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.685+0000 7f6712400640 1 -- 172.21.15.5:0/804058206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a263f41b40 msgr2=0x55a263f4d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:09.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.685+0000 7f6712400640 1 --2- 172.21.15.5:0/804058206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a263f41b40 0x55a263f4d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.685+0000 7f6712400640 1 -- 172.21.15.5:0/804058206 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a263c837d0 con 0x55a263f4cb30 2024-04-24T06:49:09.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.685+0000 7f6711a00640 1 --2- 172.21.15.5:0/804058206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a263f41b40 0x55a263f4d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:49:09.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.685+0000 7f6712400640 1 --2- 172.21.15.5:0/804058206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a263f4cb30 0x55a263f4e310 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f670402af40 tx=0x7f67040095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=913be395a0c7aa3b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:09.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.685+0000 7f66ef400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:49:09.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.686+0000 7f66ef400640 1 -- 172.21.15.5:0/804058206 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6704059060 con 0x55a263f4cb30 2024-04-24T06:49:09.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.686+0000 7f66ef400640 1 -- 172.21.15.5:0/804058206 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f6704054640 con 0x55a263f4cb30 2024-04-24T06:49:09.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.686+0000 7f67173ca740 1 -- 172.21.15.5:0/804058206 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66fc026070 con 0x55a263f4cb30 2024-04-24T06:49:09.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.686+0000 7f66ef400640 1 -- 172.21.15.5:0/804058206 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6704034040 con 0x55a263f4cb30 2024-04-24T06:49:09.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.686+0000 7f67173ca740 10 client.16278 fetch_fsmap learned FSMap version 27 2024-04-24T06:49:09.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.686+0000 7f67173ca740 1 -- 172.21.15.5:0/804058206 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a263fc2ed0 con 0x55a263f4cb30 2024-04-24T06:49:09.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.686+0000 7f66ef400640 1 -- 172.21.15.5:0/804058206 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66fc026070 con 0x55a263f4cb30 2024-04-24T06:49:09.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.687+0000 7f66ef400640 1 -- 172.21.15.5:0/804058206 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f670403a990 con 0x55a263f4cb30 2024-04-24T06:49:09.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.687+0000 7f67173ca740 10 client.16278 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:49:09.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.687+0000 7f67173ca740 10 client.16278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:49:09.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.687+0000 7f67173ca740 20 client.16278 populate_metadata read hostname 'smithi005' 2024-04-24T06:49:09.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.687+0000 7f67173ca740 1 --2- 172.21.15.5:0/804058206 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a263fc3650 0x55a263fc5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:09.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.687+0000 7f67173ca740 4 client.16278 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:49:09.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.687+0000 7f67173ca740 1 -- 172.21.15.5:0/804058206 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a263eaea60 con 0x55a263fc3650 2024-04-24T06:49:09.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.687+0000 7f6711a00640 1 --2- 172.21.15.5:0/804058206 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a263fc3650 0x55a263fc5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:09.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.687+0000 7f6711a00640 1 --2- 172.21.15.5:0/804058206 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a263fc3650 0x55a263fc5a70 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fccce479ec3f217 server_cookie=124c4e6c56308549 in_seq=0 out_seq=0 2024-04-24T06:49:09.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.687+0000 7f66ef400640 10 client.16278 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:49:09.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.688+0000 7f66ef400640 1 -- 172.21.15.5:0/804058206 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a263eaea60 con 0x55a263fc3650 2024-04-24T06:49:09.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.688+0000 7f66ef400640 10 client.16278 handle_command_reply: tid=0 2024-04-24T06:49:09.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.688+0000 7f67173ca740 1 client.16278 shutdown 2024-04-24T06:49:09.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.688+0000 7f67173ca740 1 -- 172.21.15.5:0/804058206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a263f4cb30 msgr2=0x55a263f4e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:09.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.688+0000 7f67173ca740 1 --2- 172.21.15.5:0/804058206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a263f4cb30 0x55a263f4e310 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f670402af40 tx=0x7f67040095c0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.688+0000 7f67173ca740 1 -- 172.21.15.5:0/804058206 shutdown_connections 2024-04-24T06:49:09.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.688+0000 7f67173ca740 1 --2- 172.21.15.5:0/804058206 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a263fc3650 0x55a263fc5a70 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.688+0000 7f67173ca740 1 --2- 172.21.15.5:0/804058206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a263f4cb30 0x55a263f4e310 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.688+0000 7f67173ca740 1 --2- 172.21.15.5:0/804058206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a263f42460 0x55a263f4ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.688+0000 7f67173ca740 1 --2- 172.21.15.5:0/804058206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a263f41b40 0x55a263f4d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.688+0000 7f67173ca740 1 -- 172.21.15.5:0/804058206 >> 172.21.15.5:0/804058206 conn(0x55a263f3de60 msgr2=0x55a263f3c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:09.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.688+0000 7f66ef400640 0 client.16278 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:49:09.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.688+0000 7f67173ca740 1 -- 172.21.15.5:0/804058206 shutdown_connections 2024-04-24T06:49:09.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.688+0000 7f67173ca740 1 -- 172.21.15.5:0/804058206 wait complete. 2024-04-24T06:49:09.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.688+0000 7f67173ca740 20 client.16278 trim_cache size 0 max 16384 2024-04-24T06:49:09.698 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:49:09.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.689+0000 7f6712e00640 1 -- 172.21.15.5:0/3988690087 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f66f40a3d20 msgr2=0x7f66f40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:09.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.689+0000 7f6712e00640 1 --2- 172.21.15.5:0/3988690087 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f66f40a3d20 0x7f66f40a61e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6704004af0 tx=0x7f6704005d80 comp rx=0 tx=0).stop 2024-04-24T06:49:09.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.689+0000 7f6712e00640 1 -- 172.21.15.5:0/3988690087 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f670c0743a0 msgr2=0x7f670c19ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:09.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.689+0000 7f6712e00640 1 --2- 172.21.15.5:0/3988690087 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f670c0743a0 0x7f670c19ed70 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f670000e9f0 tx=0x7f670000edf0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.689+0000 7f6712e00640 1 -- 172.21.15.5:0/3988690087 shutdown_connections 2024-04-24T06:49:09.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.689+0000 7f6712e00640 1 --2- 172.21.15.5:0/3988690087 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f66f40a3d20 0x7f66f40a61e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.689+0000 7f6712e00640 1 --2- 172.21.15.5:0/3988690087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f670c112de0 0x7f670c1a3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.689+0000 7f6712e00640 1 --2- 172.21.15.5:0/3988690087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f670c074cc0 0x7f670c19f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.689+0000 7f6712e00640 1 --2- 172.21.15.5:0/3988690087 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f670c0743a0 0x7f670c19ed70 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:09.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.689+0000 7f6712e00640 1 -- 172.21.15.5:0/3988690087 >> 172.21.15.5:0/3988690087 conn(0x7f670c0fd8e0 msgr2=0x7f670c0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:09.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.689+0000 7f6712e00640 1 -- 172.21.15.5:0/3988690087 shutdown_connections 2024-04-24T06:49:09.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:09.689+0000 7f6712e00640 1 -- 172.21.15.5:0/3988690087 wait complete. 2024-04-24T06:49:09.701 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:09 smithi137 ceph-mon[33017]: pgmap v156: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-24T06:49:09.701 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:09 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/937725176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:49:09.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:09 smithi037 ceph-mon[32864]: pgmap v156: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-24T06:49:09.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:09 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/937725176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:49:10.709 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T06:49:10.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:49:10.709 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T06:49:10.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.935+0000 7f8ff5e00640 1 -- 172.21.15.5:0/1631503319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ff0102fe0 msgr2=0x7f8ff01033c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:10.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.935+0000 7f8ff5e00640 1 --2- 172.21.15.5:0/1631503319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ff0102fe0 0x7f8ff01033c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0009900 tx=0x7f8fe002a480 comp rx=0 tx=0).stop 2024-04-24T06:49:10.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.935+0000 7f8ff5e00640 1 -- 172.21.15.5:0/1631503319 shutdown_connections 2024-04-24T06:49:10.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.935+0000 7f8ff5e00640 1 --2- 172.21.15.5:0/1631503319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ff010c400 0x7f8ff010e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:10.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.935+0000 7f8ff5e00640 1 --2- 172.21.15.5:0/1631503319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ff0103900 0x7f8ff010bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:10.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.935+0000 7f8ff5e00640 1 --2- 172.21.15.5:0/1631503319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ff0102fe0 0x7f8ff01033c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:10.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.935+0000 7f8ff5e00640 1 -- 172.21.15.5:0/1631503319 >> 172.21.15.5:0/1631503319 conn(0x7f8ff0077ab0 msgr2=0x7f8ff00fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:10.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.935+0000 7f8ff5e00640 1 -- 172.21.15.5:0/1631503319 shutdown_connections 2024-04-24T06:49:10.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.935+0000 7f8ff5e00640 1 -- 172.21.15.5:0/1631503319 wait complete. 2024-04-24T06:49:10.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.936+0000 7f8ff5e00640 1 Processor -- start 2024-04-24T06:49:10.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.936+0000 7f8ff5e00640 1 -- start start 2024-04-24T06:49:10.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.936+0000 7f8ff5e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ff0102fe0 0x7f8ff019e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:10.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.936+0000 7f8ff5e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ff0103900 0x7f8ff019e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:10.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.936+0000 7f8ff5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ff010c400 0x7f8ff01985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:10.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.937+0000 7f8ff5e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff01121e0 con 0x7f8ff010c400 2024-04-24T06:49:10.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.937+0000 7f8ff5e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff0112060 con 0x7f8ff0103900 2024-04-24T06:49:10.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.937+0000 7f8ff5e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff0112360 con 0x7f8ff0102fe0 2024-04-24T06:49:10.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.937+0000 7f8ff5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ff010c400 0x7f8ff01985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:10.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.937+0000 7f8ff5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ff010c400 0x7f8ff01985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33026/0 (socket says 172.21.15.5:33026) 2024-04-24T06:49:10.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.937+0000 7f8ff5400640 1 -- 172.21.15.5:0/509457147 learned_addr learned my addr 172.21.15.5:0/509457147 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:10.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.937+0000 7f8ff4a00640 1 --2- 172.21.15.5:0/509457147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ff0102fe0 0x7f8ff019e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:10.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.937+0000 7f8fefe00640 1 --2- 172.21.15.5:0/509457147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ff0103900 0x7f8ff019e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:10.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.937+0000 7f8ff5400640 1 -- 172.21.15.5:0/509457147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ff0102fe0 msgr2=0x7f8ff019e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:10.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.937+0000 7f8ff5400640 1 --2- 172.21.15.5:0/509457147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ff0102fe0 0x7f8ff019e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:10.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.937+0000 7f8ff5400640 1 -- 172.21.15.5:0/509457147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ff0103900 msgr2=0x7f8ff019e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:10.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.937+0000 7f8ff5400640 1 --2- 172.21.15.5:0/509457147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ff0103900 0x7f8ff019e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:10.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.937+0000 7f8ff5400640 1 -- 172.21.15.5:0/509457147 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff0198d20 con 0x7f8ff010c400 2024-04-24T06:49:10.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.937+0000 7f8ff4a00640 1 --2- 172.21.15.5:0/509457147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ff0102fe0 0x7f8ff019e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:49:10.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.937+0000 7f8fefe00640 1 --2- 172.21.15.5:0/509457147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ff0103900 0x7f8ff019e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:49:10.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.938+0000 7f8ff5400640 1 --2- 172.21.15.5:0/509457147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ff010c400 0x7f8ff01985a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800eb10 tx=0x7f8fe800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:10.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.938+0000 7f8fed600640 1 -- 172.21.15.5:0/509457147 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8004b20 con 0x7f8ff010c400 2024-04-24T06:49:10.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.938+0000 7f8ff5e00640 1 -- 172.21.15.5:0/509457147 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ff0199010 con 0x7f8ff010c400 2024-04-24T06:49:10.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.938+0000 7f8fed600640 1 -- 172.21.15.5:0/509457147 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fe8004cc0 con 0x7f8ff010c400 2024-04-24T06:49:10.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.938+0000 7f8ff5e00640 1 -- 172.21.15.5:0/509457147 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ff01a52b0 con 0x7f8ff010c400 2024-04-24T06:49:10.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.938+0000 7f8fed600640 1 -- 172.21.15.5:0/509457147 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8010700 con 0x7f8ff010c400 2024-04-24T06:49:10.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.942+0000 7f8ffa1e0740 1 Processor -- start 2024-04-24T06:49:10.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.942+0000 7f8fed600640 1 -- 172.21.15.5:0/509457147 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f8fe8005000 con 0x7f8ff010c400 2024-04-24T06:49:10.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.942+0000 7f8ffa1e0740 1 -- start start 2024-04-24T06:49:10.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.942+0000 7f8ffa1e0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d388d0bfa0 0x55d388d0c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:10.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.942+0000 7f8ffa1e0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d388d0c950 0x55d388d1b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:10.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.943+0000 7f8ffa1e0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d388d1b670 0x55d388d1da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:10.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.943+0000 7f8ffa1e0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d388c71e70 con 0x55d388d0bfa0 2024-04-24T06:49:10.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.943+0000 7f8ffa1e0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d388c18380 con 0x55d388d1b670 2024-04-24T06:49:10.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.943+0000 7f8ffa1e0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d388c6de30 con 0x55d388d0c950 2024-04-24T06:49:10.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.943+0000 7f8fefe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d388d0bfa0 0x55d388d0c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:10.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.943+0000 7f8ff4a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d388d0c950 0x55d388d1b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:10.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.943+0000 7f8ff4a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d388d0c950 0x55d388d1b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:52588/0 (socket says 172.21.15.5:52588) 2024-04-24T06:49:10.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.943+0000 7f8ff4a00640 1 -- 172.21.15.5:0/1780141089 learned_addr learned my addr 172.21.15.5:0/1780141089 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:10.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.943+0000 7f8fefe00640 1 --2- 172.21.15.5:0/1780141089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d388d1b670 0x55d388d1da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:10.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.943+0000 7f8fefe00640 1 -- 172.21.15.5:0/1780141089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d388d0c950 msgr2=0x55d388d1b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:10.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.943+0000 7f8fefe00640 1 --2- 172.21.15.5:0/1780141089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d388d0c950 0x55d388d1b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:10.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.943+0000 7f8fefe00640 1 -- 172.21.15.5:0/1780141089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d388d1b670 msgr2=0x55d388d1da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:10.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.943+0000 7f8fefe00640 1 --2- 172.21.15.5:0/1780141089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d388d1b670 0x55d388d1da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:10.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.943+0000 7f8fefe00640 1 -- 172.21.15.5:0/1780141089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d388a4c7e0 con 0x55d388d0bfa0 2024-04-24T06:49:10.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.943+0000 7f8ff4a00640 1 --2- 172.21.15.5:0/1780141089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d388d0c950 0x55d388d1b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:49:10.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.944+0000 7f8fed600640 1 --2- 172.21.15.5:0/509457147 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f8fd40a3d20 0x7f8fd40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:10.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.944+0000 7f8fefe00640 1 --2- 172.21.15.5:0/1780141089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d388d0bfa0 0x55d388d0c380 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc009fd0 tx=0x7f8fdc005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2844257f82e3c1f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:10.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.944+0000 7f8fed600640 1 -- 172.21.15.5:0/509457147 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f8fe80d0e80 con 0x7f8ff010c400 2024-04-24T06:49:10.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.944+0000 7f8fe5c00640 1 -- 172.21.15.5:0/1780141089 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc004230 con 0x55d388d0bfa0 2024-04-24T06:49:10.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.944+0000 7f8ff4a00640 1 --2- 172.21.15.5:0/509457147 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f8fd40a3d20 0x7f8fd40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:10.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.944+0000 7f8fe5c00640 1 -- 172.21.15.5:0/1780141089 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fdc0043d0 con 0x55d388d0bfa0 2024-04-24T06:49:10.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.944+0000 7f8fe5c00640 1 -- 172.21.15.5:0/1780141089 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc0047a0 con 0x55d388d0bfa0 2024-04-24T06:49:10.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.945+0000 7f8ffa1e0740 1 -- 172.21.15.5:0/1780141089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d388d0bfa0 msgr2=0x55d388d0c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:10.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.945+0000 7f8ffa1e0740 1 --2- 172.21.15.5:0/1780141089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d388d0bfa0 0x55d388d0c380 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc009fd0 tx=0x7f8fdc005850 comp rx=0 tx=0).stop 2024-04-24T06:49:10.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.945+0000 7f8ff4a00640 1 --2- 172.21.15.5:0/509457147 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f8fd40a3d20 0x7f8fd40a61e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x55d388d0eba0 tx=0x7f8fe0031510 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:10.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.945+0000 7f8ffa1e0740 1 -- 172.21.15.5:0/1780141089 shutdown_connections 2024-04-24T06:49:10.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.945+0000 7f8ffa1e0740 1 --2- 172.21.15.5:0/1780141089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d388d1b670 0x55d388d1da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:10.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.945+0000 7f8ffa1e0740 1 --2- 172.21.15.5:0/1780141089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d388d0c950 0x55d388d1b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:10.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.945+0000 7f8ffa1e0740 1 --2- 172.21.15.5:0/1780141089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d388d0bfa0 0x55d388d0c380 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:10.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.945+0000 7f8ffa1e0740 1 -- 172.21.15.5:0/1780141089 >> 172.21.15.5:0/1780141089 conn(0x55d388d08590 msgr2=0x55d388d08970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:10.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.945+0000 7f8ffa1e0740 1 -- 172.21.15.5:0/1780141089 shutdown_connections 2024-04-24T06:49:10.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.945+0000 7f8ffa1e0740 1 -- 172.21.15.5:0/1780141089 wait complete. 2024-04-24T06:49:10.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.946+0000 7f8ffa1e0740 1 -- start start 2024-04-24T06:49:10.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.946+0000 7f8ffa1e0740 1 Processor -- start 2024-04-24T06:49:10.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.946+0000 7f8ffa1e0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d388d0bfa0 0x55d388d185f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:10.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.946+0000 7f8ffa1e0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d388d0c950 0x55d388d18b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:10.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.946+0000 7f8ffa1e0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d388d1b670 0x55d388d19070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:10.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.946+0000 7f8ffa1e0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d388d1e430 con 0x55d388d0bfa0 2024-04-24T06:49:10.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.946+0000 7f8ffa1e0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d388c2b660 con 0x55d388d1b670 2024-04-24T06:49:10.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.946+0000 7f8ffa1e0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d388d1e5b0 con 0x55d388d0c950 2024-04-24T06:49:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.946+0000 7f8ffa1e0740 10 client.0 _ll_register_callbacks cb 0x55d388c19aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:49:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.947+0000 7f8ff4a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d388d0bfa0 0x55d388d185f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.947+0000 7f8ff4a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d388d0bfa0 0x55d388d185f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33040/0 (socket says 172.21.15.5:33040) 2024-04-24T06:49:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.947+0000 7f8ff4a00640 1 -- 172.21.15.5:0/1230143890 learned_addr learned my addr 172.21.15.5:0/1230143890 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.947+0000 7f8ff5400640 1 --2- 172.21.15.5:0/1230143890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d388d1b670 0x55d388d19070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.947+0000 7f8fefe00640 1 --2- 172.21.15.5:0/1230143890 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d388d0c950 0x55d388d18b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.947+0000 7f8ff4a00640 1 -- 172.21.15.5:0/1230143890 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d388d0c950 msgr2=0x55d388d18b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.947+0000 7f8ff4a00640 1 --2- 172.21.15.5:0/1230143890 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d388d0c950 0x55d388d18b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.947+0000 7f8ff4a00640 1 -- 172.21.15.5:0/1230143890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d388d1b670 msgr2=0x55d388d19070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.947+0000 7f8ff4a00640 1 --2- 172.21.15.5:0/1230143890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d388d1b670 0x55d388d19070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.947+0000 7f8ff4a00640 1 -- 172.21.15.5:0/1230143890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d388aacf70 con 0x55d388d0bfa0 2024-04-24T06:49:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.947+0000 7f8ff5400640 1 --2- 172.21.15.5:0/1230143890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d388d1b670 0x55d388d19070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:49:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.948+0000 7f8ff4a00640 1 --2- 172.21.15.5:0/1230143890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d388d0bfa0 0x55d388d185f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8fe00054a0 tx=0x7f8fe0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac09b3d4b1db55f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.948+0000 7f8fda000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:49:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.948+0000 7f8fda000640 1 -- 172.21.15.5:0/1230143890 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0035040 con 0x55d388d0bfa0 2024-04-24T06:49:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.948+0000 7f8fda000640 1 -- 172.21.15.5:0/1230143890 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f8fe00315b0 con 0x55d388d0bfa0 2024-04-24T06:49:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.948+0000 7f8ffa1e0740 1 -- 172.21.15.5:0/1230143890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fdc0043d0 con 0x55d388d0bfa0 2024-04-24T06:49:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.948+0000 7f8fda000640 1 -- 172.21.15.5:0/1230143890 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8fe0005650 con 0x55d388d0bfa0 2024-04-24T06:49:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.948+0000 7f8ffa1e0740 10 client.16290 fetch_fsmap learned FSMap version 27 2024-04-24T06:49:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.948+0000 7f8ffa1e0740 1 -- 172.21.15.5:0/1230143890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d388944170 con 0x55d388d0bfa0 2024-04-24T06:49:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.948+0000 7f8fda000640 1 -- 172.21.15.5:0/1230143890 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fdc0043d0 con 0x55d388d0bfa0 2024-04-24T06:49:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.949+0000 7f8fda000640 1 -- 172.21.15.5:0/1230143890 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f8fe0044030 con 0x55d388d0bfa0 2024-04-24T06:49:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.949+0000 7f8ffa1e0740 10 client.16290 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:49:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.949+0000 7f8ffa1e0740 10 client.16290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:49:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.949+0000 7f8ffa1e0740 20 client.16290 populate_metadata read hostname 'smithi005' 2024-04-24T06:49:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.949+0000 7f8ffa1e0740 1 --2- 172.21.15.5:0/1230143890 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55d388d2b930 0x55d388d2dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.949+0000 7f8ffa1e0740 4 client.16290 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:49:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.949+0000 7f8ffa1e0740 1 -- 172.21.15.5:0/1230143890 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d388c7f890 con 0x55d388d2b930 2024-04-24T06:49:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.949+0000 7f8fefe00640 1 --2- 172.21.15.5:0/1230143890 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55d388d2b930 0x55d388d2dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.950+0000 7f8fefe00640 1 --2- 172.21.15.5:0/1230143890 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55d388d2b930 0x55d388d2dd60 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b3710402b3c2220 server_cookie=851e32ffeed61217 in_seq=0 out_seq=0 2024-04-24T06:49:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.950+0000 7f8fda000640 10 client.16290 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:49:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.951+0000 7f8fda000640 1 -- 172.21.15.5:0/1230143890 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d388c7f890 con 0x55d388d2b930 2024-04-24T06:49:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.951+0000 7f8fda000640 10 client.16290 handle_command_reply: tid=0 2024-04-24T06:49:10.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.951+0000 7f8ffa1e0740 1 client.16290 shutdown 2024-04-24T06:49:10.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.951+0000 7f8ffa1e0740 1 -- 172.21.15.5:0/1230143890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d388d0bfa0 msgr2=0x55d388d185f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:10.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.951+0000 7f8ffa1e0740 1 --2- 172.21.15.5:0/1230143890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d388d0bfa0 0x55d388d185f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8fe00054a0 tx=0x7f8fe0009260 comp rx=0 tx=0).stop 2024-04-24T06:49:10.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.951+0000 7f8ffa1e0740 1 -- 172.21.15.5:0/1230143890 shutdown_connections 2024-04-24T06:49:10.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.951+0000 7f8ffa1e0740 1 --2- 172.21.15.5:0/1230143890 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55d388d2b930 0x55d388d2dd60 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:10.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.951+0000 7f8ffa1e0740 1 --2- 172.21.15.5:0/1230143890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d388d1b670 0x55d388d19070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:10.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.951+0000 7f8ffa1e0740 1 --2- 172.21.15.5:0/1230143890 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d388d0c950 0x55d388d18b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:10.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.951+0000 7f8fda000640 0 client.16290 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:49:10.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.951+0000 7f8ffa1e0740 1 --2- 172.21.15.5:0/1230143890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d388d0bfa0 0x55d388d185f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:10.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.951+0000 7f8ffa1e0740 1 -- 172.21.15.5:0/1230143890 >> 172.21.15.5:0/1230143890 conn(0x55d388d08590 msgr2=0x55d388d0e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:10.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.951+0000 7f8ffa1e0740 1 -- 172.21.15.5:0/1230143890 shutdown_connections 2024-04-24T06:49:10.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.951+0000 7f8ffa1e0740 1 -- 172.21.15.5:0/1230143890 wait complete. 2024-04-24T06:49:10.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.951+0000 7f8ffa1e0740 20 client.16290 trim_cache size 0 max 16384 2024-04-24T06:49:10.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.971+0000 7f8ffa1e0740 1 Processor -- start 2024-04-24T06:49:10.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.971+0000 7f8ffa1e0740 1 -- start start 2024-04-24T06:49:10.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.971+0000 7f8ffa1e0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d388d0bfa0 0x55d388d27330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:10.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.971+0000 7f8ffa1e0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d388d0c950 0x55d388d17750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:10.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.971+0000 7f8ffa1e0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d388d1b670 0x55d388d0e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:10.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.972+0000 7f8ffa1e0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff01ae240 con 0x55d388d0bfa0 2024-04-24T06:49:10.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.972+0000 7f8ffa1e0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff01abb10 con 0x55d388d0c950 2024-04-24T06:49:10.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.972+0000 7f8ffa1e0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff01a9960 con 0x55d388d1b670 2024-04-24T06:49:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.972+0000 7f8ff4a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d388d0bfa0 0x55d388d27330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.972+0000 7f8fefe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d388d0c950 0x55d388d17750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.972+0000 7f8ff5400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d388d1b670 0x55d388d0e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.972+0000 7f8ff4a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d388d0bfa0 0x55d388d27330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33050/0 (socket says 172.21.15.5:33050) 2024-04-24T06:49:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.972+0000 7f8ff4a00640 1 -- 172.21.15.5:0/2325479012 learned_addr learned my addr 172.21.15.5:0/2325479012 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.972+0000 7f8ff4a00640 1 -- 172.21.15.5:0/2325479012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d388d1b670 msgr2=0x55d388d0e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.972+0000 7f8ff4a00640 1 --2- 172.21.15.5:0/2325479012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d388d1b670 0x55d388d0e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.972+0000 7f8ff4a00640 1 -- 172.21.15.5:0/2325479012 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d388d0c950 msgr2=0x55d388d17750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.972+0000 7f8ff4a00640 1 --2- 172.21.15.5:0/2325479012 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d388d0c950 0x55d388d17750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.972+0000 7f8ff4a00640 1 -- 172.21.15.5:0/2325479012 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d388d03e80 con 0x55d388d0bfa0 2024-04-24T06:49:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.973+0000 7f8ff4a00640 1 --2- 172.21.15.5:0/2325479012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d388d0bfa0 0x55d388d27330 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0042040 tx=0x7f8fe0005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=952e7c7969248c30 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.973+0000 7f8fe5200640 1 -- 172.21.15.5:0/2325479012 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0004df0 con 0x55d388d0bfa0 2024-04-24T06:49:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.973+0000 7f8fe5200640 1 -- 172.21.15.5:0/2325479012 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fe005f020 con 0x55d388d0bfa0 2024-04-24T06:49:10.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.973+0000 7f8fe5200640 1 -- 172.21.15.5:0/2325479012 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe005ac30 con 0x55d388d0bfa0 2024-04-24T06:49:10.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.974+0000 7f8ffa1e0740 1 -- 172.21.15.5:0/2325479012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d388d0bfa0 msgr2=0x55d388d27330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:10.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.974+0000 7f8ffa1e0740 1 --2- 172.21.15.5:0/2325479012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d388d0bfa0 0x55d388d27330 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0042040 tx=0x7f8fe0005b50 comp rx=0 tx=0).stop 2024-04-24T06:49:10.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.974+0000 7f8ffa1e0740 1 -- 172.21.15.5:0/2325479012 shutdown_connections 2024-04-24T06:49:10.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.974+0000 7f8ffa1e0740 1 --2- 172.21.15.5:0/2325479012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d388d1b670 0x55d388d0e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:10.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.974+0000 7f8ffa1e0740 1 --2- 172.21.15.5:0/2325479012 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d388d0c950 0x55d388d17750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:10.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.974+0000 7f8ffa1e0740 1 --2- 172.21.15.5:0/2325479012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d388d0bfa0 0x55d388d27330 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:10.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.974+0000 7f8ffa1e0740 1 -- 172.21.15.5:0/2325479012 >> 172.21.15.5:0/2325479012 conn(0x55d388d08590 msgr2=0x55d388d0db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:10.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.974+0000 7f8ffa1e0740 1 -- 172.21.15.5:0/2325479012 shutdown_connections 2024-04-24T06:49:10.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.974+0000 7f8ffa1e0740 1 -- 172.21.15.5:0/2325479012 wait complete. 2024-04-24T06:49:10.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.975+0000 7f8ffa1e0740 1 -- start start 2024-04-24T06:49:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.975+0000 7f8ffa1e0740 1 Processor -- start 2024-04-24T06:49:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.975+0000 7f8ffa1e0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d388d0bfa0 0x55d388d238b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.975+0000 7f8ffa1e0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d388d0c950 0x55d388d0f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.975+0000 7f8ffa1e0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d388d1b670 0x55d388d0fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.975+0000 7f8ffa1e0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff01a9fd0 con 0x55d388d1b670 2024-04-24T06:49:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.975+0000 7f8ffa1e0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff01ab440 con 0x55d388d0bfa0 2024-04-24T06:49:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.975+0000 7f8ffa1e0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff01aa6a0 con 0x55d388d0c950 2024-04-24T06:49:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.975+0000 7f8ffa1e0740 10 client.0 _ll_register_callbacks cb 0x7f8fe0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:49:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.976+0000 7f8ff5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d388d1b670 0x55d388d0fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.976+0000 7f8ff4a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d388d0bfa0 0x55d388d238b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.976+0000 7f8ff5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d388d1b670 0x55d388d0fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33054/0 (socket says 172.21.15.5:33054) 2024-04-24T06:49:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.976+0000 7f8fefe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d388d0c950 0x55d388d0f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.976+0000 7f8ff5400640 1 -- 172.21.15.5:0/3358652238 learned_addr learned my addr 172.21.15.5:0/3358652238 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.976+0000 7f8ff5400640 1 -- 172.21.15.5:0/3358652238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d388d0c950 msgr2=0x55d388d0f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.976+0000 7f8ff5400640 1 --2- 172.21.15.5:0/3358652238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d388d0c950 0x55d388d0f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.976+0000 7f8ff5400640 1 -- 172.21.15.5:0/3358652238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d388d0bfa0 msgr2=0x55d388d238b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.976+0000 7f8ff5400640 1 --2- 172.21.15.5:0/3358652238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d388d0bfa0 0x55d388d238b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.976+0000 7f8ff5400640 1 -- 172.21.15.5:0/3358652238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d388d10320 con 0x55d388d1b670 2024-04-24T06:49:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.976+0000 7f8ff4a00640 1 --2- 172.21.15.5:0/3358652238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d388d0bfa0 0x55d388d238b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:49:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.976+0000 7f8ff5400640 1 --2- 172.21.15.5:0/3358652238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d388d1b670 0x55d388d0fde0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8fe809cd60 tx=0x7f8fe800cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0df274c51a0443b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.976+0000 7f8fda000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:49:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.977+0000 7f8fda000640 1 -- 172.21.15.5:0/3358652238 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe80ba070 con 0x55d388d1b670 2024-04-24T06:49:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.977+0000 7f8fda000640 1 -- 172.21.15.5:0/3358652238 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f8fe80054f0 con 0x55d388d1b670 2024-04-24T06:49:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.977+0000 7f8ffa1e0740 1 -- 172.21.15.5:0/3358652238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fe005f020 con 0x55d388d1b670 2024-04-24T06:49:10.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.977+0000 7f8fda000640 1 -- 172.21.15.5:0/3358652238 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8fe80a5a40 con 0x55d388d1b670 2024-04-24T06:49:10.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.977+0000 7f8ffa1e0740 10 client.16296 fetch_fsmap learned FSMap version 27 2024-04-24T06:49:10.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.977+0000 7f8ffa1e0740 1 -- 172.21.15.5:0/3358652238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d388d96690 con 0x55d388d1b670 2024-04-24T06:49:10.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.977+0000 7f8fda000640 1 -- 172.21.15.5:0/3358652238 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fe005f020 con 0x55d388d1b670 2024-04-24T06:49:10.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.977+0000 7f8fda000640 1 -- 172.21.15.5:0/3358652238 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f8fe80bd0b0 con 0x55d388d1b670 2024-04-24T06:49:10.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.977+0000 7f8ffa1e0740 10 client.16296 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:49:10.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.977+0000 7f8ffa1e0740 10 client.16296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:49:10.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.977+0000 7f8ffa1e0740 20 client.16296 populate_metadata read hostname 'smithi005' 2024-04-24T06:49:10.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.977+0000 7f8ffa1e0740 1 --2- 172.21.15.5:0/3358652238 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55d388d96ce0 0x55d388d99100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:10.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.977+0000 7f8ffa1e0740 4 client.16296 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T06:49:10.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.977+0000 7f8ffa1e0740 1 -- 172.21.15.5:0/3358652238 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d388c79700 con 0x55d388d96ce0 2024-04-24T06:49:10.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.978+0000 7f8ff4a00640 1 --2- 172.21.15.5:0/3358652238 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55d388d96ce0 0x55d388d99100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:10.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.978+0000 7f8ff4a00640 1 --2- 172.21.15.5:0/3358652238 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55d388d96ce0 0x55d388d99100 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59ebdc0aafc89f04 server_cookie=4cf5c32781e8f575 in_seq=0 out_seq=0 2024-04-24T06:49:10.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.978+0000 7f8fda000640 10 client.16296 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:49:10.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.979+0000 7f8fda000640 1 -- 172.21.15.5:0/3358652238 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d388c79700 con 0x55d388d96ce0 2024-04-24T06:49:10.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.979+0000 7f8fda000640 10 client.16296 handle_command_reply: tid=0 2024-04-24T06:49:10.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.980+0000 7f8ffa1e0740 1 client.16296 shutdown 2024-04-24T06:49:10.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.980+0000 7f8ffa1e0740 1 -- 172.21.15.5:0/3358652238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d388d1b670 msgr2=0x55d388d0fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:10.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.980+0000 7f8ffa1e0740 1 --2- 172.21.15.5:0/3358652238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d388d1b670 0x55d388d0fde0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8fe809cd60 tx=0x7f8fe800cee0 comp rx=0 tx=0).stop 2024-04-24T06:49:10.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.980+0000 7f8ffa1e0740 1 -- 172.21.15.5:0/3358652238 shutdown_connections 2024-04-24T06:49:10.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.980+0000 7f8ffa1e0740 1 --2- 172.21.15.5:0/3358652238 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55d388d96ce0 0x55d388d99100 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:10.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.980+0000 7f8ffa1e0740 1 --2- 172.21.15.5:0/3358652238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d388d1b670 0x55d388d0fde0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:10.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.980+0000 7f8ffa1e0740 1 --2- 172.21.15.5:0/3358652238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d388d0c950 0x55d388d0f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:10.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.980+0000 7f8ffa1e0740 1 --2- 172.21.15.5:0/3358652238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d388d0bfa0 0x55d388d238b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:10.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.980+0000 7f8ffa1e0740 1 -- 172.21.15.5:0/3358652238 >> 172.21.15.5:0/3358652238 conn(0x55d388d08590 msgr2=0x55d388d18590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:10.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.980+0000 7f8fda000640 0 client.16296 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:49:10.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.980+0000 7f8ffa1e0740 1 -- 172.21.15.5:0/3358652238 shutdown_connections 2024-04-24T06:49:10.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.980+0000 7f8ffa1e0740 1 -- 172.21.15.5:0/3358652238 wait complete. 2024-04-24T06:49:10.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.980+0000 7f8ffa1e0740 20 client.16296 trim_cache size 0 max 16384 2024-04-24T06:49:10.988 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:49:10.988 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-24T06:49:10.988 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "6360ce3d-970b-4317-9d39-197ca4571947", 2024-04-24T06:49:10.988 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-24T06:49:10.989 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:49:10.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.981+0000 7f8ff5e00640 1 -- 172.21.15.5:0/509457147 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f8fd40a3d20 msgr2=0x7f8fd40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:10.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.981+0000 7f8ff5e00640 1 --2- 172.21.15.5:0/509457147 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f8fd40a3d20 0x7f8fd40a61e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x55d388d0eba0 tx=0x7f8fe0031510 comp rx=0 tx=0).stop 2024-04-24T06:49:10.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.981+0000 7f8ff5e00640 1 -- 172.21.15.5:0/509457147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ff010c400 msgr2=0x7f8ff01985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:10.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.981+0000 7f8ff5e00640 1 --2- 172.21.15.5:0/509457147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ff010c400 0x7f8ff01985a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800eb10 tx=0x7f8fe800c2d0 comp rx=0 tx=0).stop 2024-04-24T06:49:10.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.981+0000 7f8ff5e00640 1 -- 172.21.15.5:0/509457147 shutdown_connections 2024-04-24T06:49:10.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.981+0000 7f8ff5e00640 1 --2- 172.21.15.5:0/509457147 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f8fd40a3d20 0x7f8fd40a61e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:10.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.981+0000 7f8ff5e00640 1 --2- 172.21.15.5:0/509457147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ff010c400 0x7f8ff01985a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:10.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.981+0000 7f8ff5e00640 1 --2- 172.21.15.5:0/509457147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ff0103900 0x7f8ff019e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:10.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.981+0000 7f8ff5e00640 1 --2- 172.21.15.5:0/509457147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ff0102fe0 0x7f8ff019e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:10.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.981+0000 7f8ff5e00640 1 -- 172.21.15.5:0/509457147 >> 172.21.15.5:0/509457147 conn(0x7f8ff0077ab0 msgr2=0x7f8ff00fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:10.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.981+0000 7f8ff5e00640 1 -- 172.21.15.5:0/509457147 shutdown_connections 2024-04-24T06:49:10.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:10.981+0000 7f8ff5e00640 1 -- 172.21.15.5:0/509457147 wait complete. 2024-04-24T06:49:11.004 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T06:49:11.004 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T06:49:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.228+0000 7f6f11e00640 1 -- 172.21.15.5:0/396886368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f0c102070 msgr2=0x7f6f0c102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.228+0000 7f6f11e00640 1 --2- 172.21.15.5:0/396886368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f0c102070 0x7f6f0c102450 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6f00009900 tx=0x7f6f0002a3f0 comp rx=0 tx=0).stop 2024-04-24T06:49:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.228+0000 7f6f11e00640 1 -- 172.21.15.5:0/396886368 shutdown_connections 2024-04-24T06:49:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.228+0000 7f6f11e00640 1 --2- 172.21.15.5:0/396886368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f0c107140 0x7f6f0c10ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.228+0000 7f6f11e00640 1 --2- 172.21.15.5:0/396886368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f0c102a20 0x7f6f0c106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.228+0000 7f6f11e00640 1 --2- 172.21.15.5:0/396886368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f0c102070 0x7f6f0c102450 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.228+0000 7f6f11e00640 1 -- 172.21.15.5:0/396886368 >> 172.21.15.5:0/396886368 conn(0x7f6f0c0fd950 msgr2=0x7f6f0c0ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.228+0000 7f6f11e00640 1 -- 172.21.15.5:0/396886368 shutdown_connections 2024-04-24T06:49:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.228+0000 7f6f11e00640 1 -- 172.21.15.5:0/396886368 wait complete. 2024-04-24T06:49:11.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.229+0000 7f6f11e00640 1 Processor -- start 2024-04-24T06:49:11.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.229+0000 7f6f11e00640 1 -- start start 2024-04-24T06:49:11.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.230+0000 7f6f11e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f0c102070 0x7f6f0c074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:11.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.230+0000 7f6f11e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f0c107140 0x7f6f0c074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:11.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.230+0000 7f6f11e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f0c074d80 0x7f6f0c1a6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:11.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.230+0000 7f6f11e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c112220 con 0x7f6f0c102070 2024-04-24T06:49:11.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.230+0000 7f6f11e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c1120a0 con 0x7f6f0c074d80 2024-04-24T06:49:11.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.230+0000 7f6f11e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c1123a0 con 0x7f6f0c107140 2024-04-24T06:49:11.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.230+0000 7f6f10a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f0c102070 0x7f6f0c074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:11.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.230+0000 7f6f10a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f0c102070 0x7f6f0c074300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33074/0 (socket says 172.21.15.5:33074) 2024-04-24T06:49:11.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.230+0000 7f6f10a00640 1 -- 172.21.15.5:0/2506131953 learned_addr learned my addr 172.21.15.5:0/2506131953 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:11.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.230+0000 7f6f11400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f0c074d80 0x7f6f0c1a6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:11.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.231+0000 7f6f0be00640 1 --2- 172.21.15.5:0/2506131953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f0c107140 0x7f6f0c074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:11.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.231+0000 7f6f11400640 1 -- 172.21.15.5:0/2506131953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f0c107140 msgr2=0x7f6f0c074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:11.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.231+0000 7f6f11400640 1 --2- 172.21.15.5:0/2506131953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f0c107140 0x7f6f0c074840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:11.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.231+0000 7f6f11400640 1 -- 172.21.15.5:0/2506131953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f0c102070 msgr2=0x7f6f0c074300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:11.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.231+0000 7f6f11400640 1 --2- 172.21.15.5:0/2506131953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f0c102070 0x7f6f0c074300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:11.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.231+0000 7f6f11400640 1 -- 172.21.15.5:0/2506131953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f0c1a70b0 con 0x7f6f0c074d80 2024-04-24T06:49:11.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.231+0000 7f6f10a00640 1 --2- 172.21.15.5:0/2506131953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f0c102070 0x7f6f0c074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:49:11.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.231+0000 7f6f11400640 1 --2- 172.21.15.5:0/2506131953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f0c074d80 0x7f6f0c1a6a10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6f04009fd0 tx=0x7f6f0400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:11.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.232+0000 7f6f09600640 1 -- 172.21.15.5:0/2506131953 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0400ce20 con 0x7f6f0c074d80 2024-04-24T06:49:11.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.232+0000 7f6f09600640 1 -- 172.21.15.5:0/2506131953 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f04005020 con 0x7f6f0c074d80 2024-04-24T06:49:11.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.232+0000 7f6f11e00640 1 -- 172.21.15.5:0/2506131953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f0c1a73a0 con 0x7f6f0c074d80 2024-04-24T06:49:11.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.232+0000 7f6f09600640 1 -- 172.21.15.5:0/2506131953 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f04010720 con 0x7f6f0c074d80 2024-04-24T06:49:11.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.232+0000 7f6f11e00640 1 -- 172.21.15.5:0/2506131953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f0c1a78f0 con 0x7f6f0c074d80 2024-04-24T06:49:11.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.236+0000 7f6f1625d740 1 Processor -- start 2024-04-24T06:49:11.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.236+0000 7f6f09600640 1 -- 172.21.15.5:0/2506131953 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f6f04005540 con 0x7f6f0c074d80 2024-04-24T06:49:11.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.236+0000 7f6f1625d740 1 -- start start 2024-04-24T06:49:11.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.237+0000 7f6f1625d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608b9bc9a30 0x5608b9bc9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:11.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.237+0000 7f6f1625d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608b9bca350 0x5608b9bd44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:11.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.237+0000 7f6f1625d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b9bd4a20 0x5608b9bd6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:11.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.237+0000 7f6f1625d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5608b9b2f1f0 con 0x5608b9bd4a20 2024-04-24T06:49:11.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.237+0000 7f6f1625d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5608b9ad5350 con 0x5608b9bca350 2024-04-24T06:49:11.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.237+0000 7f6f1625d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5608b9b2b1b0 con 0x5608b9bc9a30 2024-04-24T06:49:11.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.237+0000 7f6f10a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608b9bca350 0x5608b9bd44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:11.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.237+0000 7f6f10a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608b9bca350 0x5608b9bd44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:41844/0 (socket says 172.21.15.5:41844) 2024-04-24T06:49:11.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.237+0000 7f6f10a00640 1 -- 172.21.15.5:0/84807033 learned_addr learned my addr 172.21.15.5:0/84807033 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:11.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.237+0000 7f6f0be00640 1 --2- 172.21.15.5:0/84807033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b9bd4a20 0x5608b9bd6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:11.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.237+0000 7f6f10a00640 1 -- 172.21.15.5:0/84807033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608b9bc9a30 msgr2=0x5608b9bc9e10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T06:49:11.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.237+0000 7f6f10a00640 1 --2- 172.21.15.5:0/84807033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608b9bc9a30 0x5608b9bc9e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:11.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.237+0000 7f6f10a00640 1 -- 172.21.15.5:0/84807033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b9bd4a20 msgr2=0x5608b9bd6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:11.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.237+0000 7f6f10a00640 1 --2- 172.21.15.5:0/84807033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b9bd4a20 0x5608b9bd6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:11.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.237+0000 7f6f10a00640 1 -- 172.21.15.5:0/84807033 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608b9b29f50 con 0x5608b9bca350 2024-04-24T06:49:11.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.237+0000 7f6f0be00640 1 --2- 172.21.15.5:0/84807033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b9bd4a20 0x5608b9bd6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:49:11.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.238+0000 7f6f10a00640 1 --2- 172.21.15.5:0/84807033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608b9bca350 0x5608b9bd44e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6f00005c40 tx=0x7f6f00005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb61da9ddd2b6cc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:11.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.238+0000 7f6f09600640 1 --2- 172.21.15.5:0/2506131953 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f6ef40a3da0 0x7f6ef40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:11.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.238+0000 7f6efdc00640 1 -- 172.21.15.5:0/84807033 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f00040070 con 0x5608b9bca350 2024-04-24T06:49:11.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.238+0000 7f6efdc00640 1 -- 172.21.15.5:0/84807033 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f000047a0 con 0x5608b9bca350 2024-04-24T06:49:11.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.238+0000 7f6efdc00640 1 -- 172.21.15.5:0/84807033 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f00035040 con 0x5608b9bca350 2024-04-24T06:49:11.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.238+0000 7f6f09600640 1 -- 172.21.15.5:0/2506131953 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f6f040d0dc0 con 0x7f6f0c074d80 2024-04-24T06:49:11.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.239+0000 7f6f0be00640 1 --2- 172.21.15.5:0/2506131953 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f6ef40a3da0 0x7f6ef40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:11.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.239+0000 7f6f1625d740 1 -- 172.21.15.5:0/84807033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608b9bca350 msgr2=0x5608b9bd44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:11.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.239+0000 7f6f1625d740 1 --2- 172.21.15.5:0/84807033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608b9bca350 0x5608b9bd44e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6f00005c40 tx=0x7f6f00005dd0 comp rx=0 tx=0).stop 2024-04-24T06:49:11.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.240+0000 7f6f0be00640 1 --2- 172.21.15.5:0/2506131953 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f6ef40a3da0 0x7f6ef40a6260 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x5608b9bc6ee0 tx=0x7f6ef8005f90 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:11.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.240+0000 7f6f1625d740 1 -- 172.21.15.5:0/84807033 shutdown_connections 2024-04-24T06:49:11.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.240+0000 7f6f1625d740 1 --2- 172.21.15.5:0/84807033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b9bd4a20 0x5608b9bd6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:11.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.240+0000 7f6f1625d740 1 --2- 172.21.15.5:0/84807033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608b9bca350 0x5608b9bd44e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:11.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.240+0000 7f6f1625d740 1 --2- 172.21.15.5:0/84807033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608b9bc9a30 0x5608b9bc9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:11.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.240+0000 7f6f1625d740 1 -- 172.21.15.5:0/84807033 >> 172.21.15.5:0/84807033 conn(0x5608b9bc5d50 msgr2=0x5608b9bc6130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:11.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.240+0000 7f6f1625d740 1 -- 172.21.15.5:0/84807033 shutdown_connections 2024-04-24T06:49:11.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.240+0000 7f6f1625d740 1 -- 172.21.15.5:0/84807033 wait complete. 2024-04-24T06:49:11.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.240+0000 7f6f1625d740 1 -- start start 2024-04-24T06:49:11.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.240+0000 7f6f1625d740 1 Processor -- start 2024-04-24T06:49:11.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.241+0000 7f6f1625d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608b9bc9a30 0x5608b9bd1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:11.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.241+0000 7f6f1625d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b9bca350 0x5608b9bd1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:11.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.241+0000 7f6f1625d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608b9bd4a20 0x5608b9bd2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:11.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.241+0000 7f6f1625d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5608b9bd77b0 con 0x5608b9bca350 2024-04-24T06:49:11.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.241+0000 7f6f1625d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5608b9ae8690 con 0x5608b9bc9a30 2024-04-24T06:49:11.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.241+0000 7f6f1625d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5608b9bd7930 con 0x5608b9bd4a20 2024-04-24T06:49:11.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.241+0000 7f6f1625d740 10 client.0 _ll_register_callbacks cb 0x5608b9b24650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:49:11.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.241+0000 7f6f10a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608b9bc9a30 0x5608b9bd1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:11.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.241+0000 7f6f11400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608b9bd4a20 0x5608b9bd2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:11.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.241+0000 7f6f10a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608b9bc9a30 0x5608b9bd1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:41848/0 (socket says 172.21.15.5:41848) 2024-04-24T06:49:11.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.241+0000 7f6f0be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b9bca350 0x5608b9bd1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:11.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.241+0000 7f6f10a00640 1 -- 172.21.15.5:0/1490185968 learned_addr learned my addr 172.21.15.5:0/1490185968 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:11.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.241+0000 7f6f11400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608b9bd4a20 0x5608b9bd2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:52654/0 (socket says 172.21.15.5:52654) 2024-04-24T06:49:11.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.242+0000 7f6f11400640 1 -- 172.21.15.5:0/1490185968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608b9bc9a30 msgr2=0x5608b9bd1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:11.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.242+0000 7f6f11400640 1 --2- 172.21.15.5:0/1490185968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608b9bc9a30 0x5608b9bd1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:11.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.242+0000 7f6f11400640 1 -- 172.21.15.5:0/1490185968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b9bca350 msgr2=0x5608b9bd1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:11.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.242+0000 7f6f11400640 1 --2- 172.21.15.5:0/1490185968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b9bca350 0x5608b9bd1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:11.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.242+0000 7f6f11400640 1 -- 172.21.15.5:0/1490185968 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608b9ae7920 con 0x5608b9bd4a20 2024-04-24T06:49:11.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.242+0000 7f6f10a00640 1 --2- 172.21.15.5:0/1490185968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608b9bc9a30 0x5608b9bd1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:49:11.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.242+0000 7f6f11400640 1 --2- 172.21.15.5:0/1490185968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608b9bd4a20 0x5608b9bd2400 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6f040ce520 tx=0x7f6f0409d5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21353c97d765afa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:11.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.242+0000 7f6ef2000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:49:11.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.242+0000 7f6ef2000640 1 -- 172.21.15.5:0/1490185968 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f040b6260 con 0x5608b9bd4a20 2024-04-24T06:49:11.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.242+0000 7f6ef2000640 1 -- 172.21.15.5:0/1490185968 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f6f040993e0 con 0x5608b9bd4a20 2024-04-24T06:49:11.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.243+0000 7f6f1625d740 1 -- 172.21.15.5:0/1490185968 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f000047a0 con 0x5608b9bd4a20 2024-04-24T06:49:11.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.243+0000 7f6ef2000640 1 -- 172.21.15.5:0/1490185968 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6f0409cfd0 con 0x5608b9bd4a20 2024-04-24T06:49:11.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.243+0000 7f6f1625d740 10 client.26452 fetch_fsmap learned FSMap version 27 2024-04-24T06:49:11.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.243+0000 7f6f1625d740 1 -- 172.21.15.5:0/1490185968 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5608b990b7d0 con 0x5608b9bd4a20 2024-04-24T06:49:11.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.243+0000 7f6ef2000640 1 -- 172.21.15.5:0/1490185968 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f000047a0 con 0x5608b9bd4a20 2024-04-24T06:49:11.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.243+0000 7f6ef2000640 1 -- 172.21.15.5:0/1490185968 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f6f040bd0d0 con 0x5608b9bd4a20 2024-04-24T06:49:11.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.243+0000 7f6f1625d740 10 client.26452 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:49:11.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.243+0000 7f6f1625d740 10 client.26452 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:49:11.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.244+0000 7f6f1625d740 20 client.26452 populate_metadata read hostname 'smithi005' 2024-04-24T06:49:11.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.244+0000 7f6f1625d740 1 --2- 172.21.15.5:0/1490185968 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5608b9be4d30 0x5608b9be71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:11.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.244+0000 7f6f1625d740 4 client.26452 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:49:11.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.244+0000 7f6f1625d740 1 -- 172.21.15.5:0/1490185968 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608b9b3cc10 con 0x5608b9be4d30 2024-04-24T06:49:11.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.244+0000 7f6f10a00640 1 --2- 172.21.15.5:0/1490185968 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5608b9be4d30 0x5608b9be71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:11.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.245+0000 7f6f10a00640 1 --2- 172.21.15.5:0/1490185968 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5608b9be4d30 0x5608b9be71d0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18fbe961a13ef79c server_cookie=e84391cfb72914dc in_seq=0 out_seq=0 2024-04-24T06:49:11.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.245+0000 7f6ef2000640 10 client.26452 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:49:11.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.246+0000 7f6ef2000640 1 -- 172.21.15.5:0/1490185968 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5608b9b3cc10 con 0x5608b9be4d30 2024-04-24T06:49:11.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.246+0000 7f6ef2000640 10 client.26452 handle_command_reply: tid=0 2024-04-24T06:49:11.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.246+0000 7f6f1625d740 1 client.26452 shutdown 2024-04-24T06:49:11.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.246+0000 7f6f1625d740 1 -- 172.21.15.5:0/1490185968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608b9bd4a20 msgr2=0x5608b9bd2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:11.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.246+0000 7f6f1625d740 1 --2- 172.21.15.5:0/1490185968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608b9bd4a20 0x5608b9bd2400 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6f040ce520 tx=0x7f6f0409d5b0 comp rx=0 tx=0).stop 2024-04-24T06:49:11.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.246+0000 7f6f1625d740 1 -- 172.21.15.5:0/1490185968 shutdown_connections 2024-04-24T06:49:11.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.246+0000 7f6f1625d740 1 --2- 172.21.15.5:0/1490185968 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5608b9be4d30 0x5608b9be71d0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:11.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.246+0000 7f6f1625d740 1 --2- 172.21.15.5:0/1490185968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608b9bd4a20 0x5608b9bd2400 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:11.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.246+0000 7f6f1625d740 1 --2- 172.21.15.5:0/1490185968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b9bca350 0x5608b9bd1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:11.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.246+0000 7f6f1625d740 1 --2- 172.21.15.5:0/1490185968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608b9bc9a30 0x5608b9bd1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:11.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.246+0000 7f6f1625d740 1 -- 172.21.15.5:0/1490185968 >> 172.21.15.5:0/1490185968 conn(0x5608b9bc5d50 msgr2=0x5608b9bc4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:11.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.246+0000 7f6ef2000640 0 client.26452 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:49:11.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.247+0000 7f6f1625d740 1 -- 172.21.15.5:0/1490185968 shutdown_connections 2024-04-24T06:49:11.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.247+0000 7f6f1625d740 1 -- 172.21.15.5:0/1490185968 wait complete. 2024-04-24T06:49:11.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.247+0000 7f6f1625d740 20 client.26452 trim_cache size 0 max 16384 2024-04-24T06:49:11.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.271+0000 7f6f1625d740 1 Processor -- start 2024-04-24T06:49:11.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.271+0000 7f6f1625d740 1 -- start start 2024-04-24T06:49:11.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.271+0000 7f6f1625d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b9bc9a30 0x5608b9be06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:11.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.271+0000 7f6f1625d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608b9bca350 0x5608b9bd0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:11.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.271+0000 7f6f1625d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608b9bd4a20 0x5608b9bcb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:11.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.271+0000 7f6f1625d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c1b2600 con 0x5608b9bc9a30 2024-04-24T06:49:11.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.271+0000 7f6f1625d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c1afed0 con 0x5608b9bca350 2024-04-24T06:49:11.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.271+0000 7f6f1625d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c1add20 con 0x5608b9bd4a20 2024-04-24T06:49:11.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.271+0000 7f6f0be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608b9bca350 0x5608b9bd0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:11.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.272+0000 7f6f0be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608b9bca350 0x5608b9bd0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:41856/0 (socket says 172.21.15.5:41856) 2024-04-24T06:49:11.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.272+0000 7f6f0be00640 1 -- 172.21.15.5:0/721994173 learned_addr learned my addr 172.21.15.5:0/721994173 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:11.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.272+0000 7f6f10a00640 1 --2- 172.21.15.5:0/721994173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b9bc9a30 0x5608b9be06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:11.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.272+0000 7f6f11400640 1 --2- 172.21.15.5:0/721994173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608b9bd4a20 0x5608b9bcb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:11.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.272+0000 7f6f0be00640 1 -- 172.21.15.5:0/721994173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608b9bd4a20 msgr2=0x5608b9bcb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:11.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.272+0000 7f6f0be00640 1 --2- 172.21.15.5:0/721994173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608b9bd4a20 0x5608b9bcb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:11.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.272+0000 7f6f0be00640 1 -- 172.21.15.5:0/721994173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b9bc9a30 msgr2=0x5608b9be06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:11.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.272+0000 7f6f0be00640 1 --2- 172.21.15.5:0/721994173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b9bc9a30 0x5608b9be06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:11.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.272+0000 7f6f0be00640 1 -- 172.21.15.5:0/721994173 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608b98413f0 con 0x5608b9bca350 2024-04-24T06:49:11.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.272+0000 7f6f11400640 1 --2- 172.21.15.5:0/721994173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608b9bd4a20 0x5608b9bcb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:49:11.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.272+0000 7f6f10a00640 1 --2- 172.21.15.5:0/721994173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b9bc9a30 0x5608b9be06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:49:11.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.272+0000 7f6f0be00640 1 --2- 172.21.15.5:0/721994173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608b9bca350 0x5608b9bd0b00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6ef80118b0 tx=0x7f6ef8011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9961d0e7646a3d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:11.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.273+0000 7f6efd200640 1 -- 172.21.15.5:0/721994173 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef8004230 con 0x5608b9bca350 2024-04-24T06:49:11.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.273+0000 7f6efd200640 1 -- 172.21.15.5:0/721994173 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ef801de50 con 0x5608b9bca350 2024-04-24T06:49:11.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.273+0000 7f6efd200640 1 -- 172.21.15.5:0/721994173 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef8003d70 con 0x5608b9bca350 2024-04-24T06:49:11.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.274+0000 7f6f1625d740 1 -- 172.21.15.5:0/721994173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608b9bca350 msgr2=0x5608b9bd0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:11.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.274+0000 7f6f1625d740 1 --2- 172.21.15.5:0/721994173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608b9bca350 0x5608b9bd0b00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6ef80118b0 tx=0x7f6ef8011000 comp rx=0 tx=0).stop 2024-04-24T06:49:11.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.274+0000 7f6f1625d740 1 -- 172.21.15.5:0/721994173 shutdown_connections 2024-04-24T06:49:11.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.274+0000 7f6f1625d740 1 --2- 172.21.15.5:0/721994173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608b9bd4a20 0x5608b9bcb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:11.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.274+0000 7f6f1625d740 1 --2- 172.21.15.5:0/721994173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608b9bca350 0x5608b9bd0b00 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:11.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.274+0000 7f6f1625d740 1 --2- 172.21.15.5:0/721994173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b9bc9a30 0x5608b9be06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:11.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.274+0000 7f6f1625d740 1 -- 172.21.15.5:0/721994173 >> 172.21.15.5:0/721994173 conn(0x5608b9bc5d50 msgr2=0x5608b9bcb290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:11.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.274+0000 7f6f1625d740 1 -- 172.21.15.5:0/721994173 shutdown_connections 2024-04-24T06:49:11.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.274+0000 7f6f1625d740 1 -- 172.21.15.5:0/721994173 wait complete. 2024-04-24T06:49:11.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.275+0000 7f6f1625d740 1 -- start start 2024-04-24T06:49:11.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.275+0000 7f6f1625d740 1 Processor -- start 2024-04-24T06:49:11.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.275+0000 7f6f1625d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608b9bc9a30 0x5608b9bdd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:11.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.276+0000 7f6f1625d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b9bca350 0x5608b9bce480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:11.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.276+0000 7f6f1625d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608b9bd4a20 0x5608b9bcea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:11.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.276+0000 7f6f1625d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c1ae390 con 0x5608b9bca350 2024-04-24T06:49:11.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.276+0000 7f6f1625d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c1af800 con 0x5608b9bd4a20 2024-04-24T06:49:11.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.276+0000 7f6f1625d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c1aea60 con 0x5608b9bc9a30 2024-04-24T06:49:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.276+0000 7f6f1625d740 10 client.0 _ll_register_callbacks cb 0x7f6f04005830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:49:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.276+0000 7f6f10a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608b9bc9a30 0x5608b9bdd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.276+0000 7f6f10a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608b9bc9a30 0x5608b9bdd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:52668/0 (socket says 172.21.15.5:52668) 2024-04-24T06:49:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.276+0000 7f6f10a00640 1 -- 172.21.15.5:0/1910685731 learned_addr learned my addr 172.21.15.5:0/1910685731 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.276+0000 7f6f11400640 1 --2- 172.21.15.5:0/1910685731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608b9bd4a20 0x5608b9bcea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.276+0000 7f6f0be00640 1 --2- 172.21.15.5:0/1910685731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b9bca350 0x5608b9bce480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.276+0000 7f6f10a00640 1 -- 172.21.15.5:0/1910685731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608b9bd4a20 msgr2=0x5608b9bcea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.276+0000 7f6f10a00640 1 --2- 172.21.15.5:0/1910685731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608b9bd4a20 0x5608b9bcea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.276+0000 7f6f10a00640 1 -- 172.21.15.5:0/1910685731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b9bca350 msgr2=0x5608b9bce480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:11.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.276+0000 7f6f10a00640 1 --2- 172.21.15.5:0/1910685731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b9bca350 0x5608b9bce480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:11.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.276+0000 7f6f10a00640 1 -- 172.21.15.5:0/1910685731 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608b98010f0 con 0x5608b9bc9a30 2024-04-24T06:49:11.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.277+0000 7f6f11400640 1 --2- 172.21.15.5:0/1910685731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608b9bd4a20 0x5608b9bcea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:49:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.277+0000 7f6f10a00640 1 --2- 172.21.15.5:0/1910685731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608b9bc9a30 0x5608b9bdd0b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6f00004c20 tx=0x7f6f0002ab70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a3f6288c5c05c3e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.277+0000 7f6ef2000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:49:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.279+0000 7f6ef2000640 1 -- 172.21.15.5:0/1910685731 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f00035040 con 0x5608b9bc9a30 2024-04-24T06:49:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.279+0000 7f6ef2000640 1 -- 172.21.15.5:0/1910685731 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f6f000050a0 con 0x5608b9bc9a30 2024-04-24T06:49:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.279+0000 7f6f1625d740 1 -- 172.21.15.5:0/1910685731 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ef801de50 con 0x5608b9bc9a30 2024-04-24T06:49:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.280+0000 7f6ef2000640 1 -- 172.21.15.5:0/1910685731 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6f00060050 con 0x5608b9bc9a30 2024-04-24T06:49:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.280+0000 7f6f1625d740 10 client.26458 fetch_fsmap learned FSMap version 27 2024-04-24T06:49:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.280+0000 7f6f1625d740 1 -- 172.21.15.5:0/1910685731 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5608b9af4810 con 0x5608b9bc9a30 2024-04-24T06:49:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.281+0000 7f6ef2000640 1 -- 172.21.15.5:0/1910685731 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ef801de50 con 0x5608b9bc9a30 2024-04-24T06:49:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.281+0000 7f6ef2000640 1 -- 172.21.15.5:0/1910685731 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f6f00003680 con 0x5608b9bc9a30 2024-04-24T06:49:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.281+0000 7f6f1625d740 10 client.26458 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:49:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.281+0000 7f6f1625d740 10 client.26458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:49:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.281+0000 7f6f1625d740 20 client.26458 populate_metadata read hostname 'smithi005' 2024-04-24T06:49:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.281+0000 7f6f1625d740 1 --2- 172.21.15.5:0/1910685731 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5608b9c4b3c0 0x5608b9c4d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.281+0000 7f6f1625d740 4 client.26458 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:49:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.281+0000 7f6f1625d740 1 -- 172.21.15.5:0/1910685731 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5608b9b36a80 con 0x5608b9c4b3c0 2024-04-24T06:49:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.281+0000 7f6f0be00640 1 --2- 172.21.15.5:0/1910685731 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5608b9c4b3c0 0x5608b9c4d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:11.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.282+0000 7f6f0be00640 1 --2- 172.21.15.5:0/1910685731 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5608b9c4b3c0 0x5608b9c4d7b0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1aad163448a3b4a server_cookie=5604b12a4405549e in_seq=0 out_seq=0 2024-04-24T06:49:11.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.282+0000 7f6ef2000640 10 client.26458 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:49:11.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.282+0000 7f6ef2000640 1 -- 172.21.15.5:0/1910685731 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x5608b9b36a80 con 0x5608b9c4b3c0 2024-04-24T06:49:11.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.282+0000 7f6ef2000640 10 client.26458 handle_command_reply: tid=0 2024-04-24T06:49:11.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.282+0000 7f6f1625d740 1 client.26458 shutdown 2024-04-24T06:49:11.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.283+0000 7f6f1625d740 1 -- 172.21.15.5:0/1910685731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608b9bc9a30 msgr2=0x5608b9bdd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:11.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.283+0000 7f6f1625d740 1 --2- 172.21.15.5:0/1910685731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608b9bc9a30 0x5608b9bdd0b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6f00004c20 tx=0x7f6f0002ab70 comp rx=0 tx=0).stop 2024-04-24T06:49:11.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.283+0000 7f6f1625d740 1 -- 172.21.15.5:0/1910685731 shutdown_connections 2024-04-24T06:49:11.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.283+0000 7f6f1625d740 1 --2- 172.21.15.5:0/1910685731 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5608b9c4b3c0 0x5608b9c4d7b0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:11.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.283+0000 7f6f1625d740 1 --2- 172.21.15.5:0/1910685731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608b9bd4a20 0x5608b9bcea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:11.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.283+0000 7f6f1625d740 1 --2- 172.21.15.5:0/1910685731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b9bca350 0x5608b9bce480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:11.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.283+0000 7f6ef2000640 0 client.26458 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:49:11.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.283+0000 7f6f1625d740 1 --2- 172.21.15.5:0/1910685731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608b9bc9a30 0x5608b9bdd0b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:11.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.283+0000 7f6f1625d740 1 -- 172.21.15.5:0/1910685731 >> 172.21.15.5:0/1910685731 conn(0x5608b9bc5d50 msgr2=0x5608b9bd21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:11.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.283+0000 7f6f1625d740 1 -- 172.21.15.5:0/1910685731 shutdown_connections 2024-04-24T06:49:11.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.283+0000 7f6f1625d740 1 -- 172.21.15.5:0/1910685731 wait complete. 2024-04-24T06:49:11.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.283+0000 7f6f1625d740 20 client.26458 trim_cache size 0 max 16384 2024-04-24T06:49:11.288 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:49:11.288 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (982 inodes in the stack)", 2024-04-24T06:49:11.288 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-24T06:49:11.288 INFO:teuthology.orchestra.run.smithi005.stdout: "6360ce3d-970b-4317-9d39-197ca4571947": { 2024-04-24T06:49:11.289 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-24T06:49:11.289 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "6360ce3d-970b-4317-9d39-197ca4571947", 2024-04-24T06:49:11.289 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-04-24T06:49:11.289 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:49:11.289 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:49:11.289 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:49:11.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.284+0000 7f6f11e00640 1 -- 172.21.15.5:0/2506131953 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f6ef40a3da0 msgr2=0x7f6ef40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:11.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.284+0000 7f6f11e00640 1 --2- 172.21.15.5:0/2506131953 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f6ef40a3da0 0x7f6ef40a6260 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x5608b9bc6ee0 tx=0x7f6ef8005f90 comp rx=0 tx=0).stop 2024-04-24T06:49:11.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.284+0000 7f6f11e00640 1 -- 172.21.15.5:0/2506131953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f0c074d80 msgr2=0x7f6f0c1a6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:11.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.284+0000 7f6f11e00640 1 --2- 172.21.15.5:0/2506131953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f0c074d80 0x7f6f0c1a6a10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6f04009fd0 tx=0x7f6f0400ef10 comp rx=0 tx=0).stop 2024-04-24T06:49:11.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.284+0000 7f6f11e00640 1 -- 172.21.15.5:0/2506131953 shutdown_connections 2024-04-24T06:49:11.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.284+0000 7f6f11e00640 1 --2- 172.21.15.5:0/2506131953 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f6ef40a3da0 0x7f6ef40a6260 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:11.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.284+0000 7f6f11e00640 1 --2- 172.21.15.5:0/2506131953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f0c074d80 0x7f6f0c1a6a10 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:11.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.284+0000 7f6f11e00640 1 --2- 172.21.15.5:0/2506131953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f0c107140 0x7f6f0c074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:11.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.284+0000 7f6f11e00640 1 --2- 172.21.15.5:0/2506131953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f0c102070 0x7f6f0c074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:11.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.284+0000 7f6f11e00640 1 -- 172.21.15.5:0/2506131953 >> 172.21.15.5:0/2506131953 conn(0x7f6f0c0fd950 msgr2=0x7f6f0c0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:11.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.284+0000 7f6f11e00640 1 -- 172.21.15.5:0/2506131953 shutdown_connections 2024-04-24T06:49:11.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:11.284+0000 7f6f11e00640 1 -- 172.21.15.5:0/2506131953 wait complete. 2024-04-24T06:49:11.307 INFO:tasks.cephfs.filesystem:scrub status for tag:6360ce3d-970b-4317-9d39-197ca4571947 - {'path': '/', 'tag': '6360ce3d-970b-4317-9d39-197ca4571947', 'options': 'recursive,force'} 2024-04-24T06:49:11.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:11 smithi005 ceph-mon[27798]: pgmap v157: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-24T06:49:11.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:11 smithi137 ceph-mon[33017]: pgmap v157: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-24T06:49:11.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:11 smithi037 ceph-mon[32864]: pgmap v157: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-24T06:49:12.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:12 smithi137 ceph-mon[33017]: scrub queued for path: / 2024-04-24T06:49:12.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:12 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:49:12.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:12 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:49:12.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:12 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:49:12.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:12 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:49:12.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:12 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:49:12.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:12 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:49:12.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:12 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:49:12.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:12 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:49:12.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:12 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:49:12.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:12 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:49:12.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:12 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:49:12.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:12 smithi005 ceph-mon[27798]: scrub queued for path: / 2024-04-24T06:49:12.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:12 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:49:12.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:12 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:49:12.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:12 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:49:12.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:12 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:49:12.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:12 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:49:12.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:12 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:49:12.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:12 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:49:12.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:12 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:49:12.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:12 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:49:12.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:12 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:49:12.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:12 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:49:12.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:12 smithi037 ceph-mon[32864]: scrub queued for path: / 2024-04-24T06:49:12.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:12 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:49:12.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:12 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:49:12.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:12 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:49:12.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:12 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:49:12.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:12 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:49:12.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:12 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:49:12.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:12 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:49:12.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:12 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:49:12.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:12 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:49:12.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:12 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:49:12.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:12 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:49:13.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:13 smithi137 ceph-mon[33017]: pgmap v158: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-24T06:49:13.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:13 smithi005 ceph-mon[27798]: pgmap v158: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-24T06:49:13.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:13 smithi037 ceph-mon[32864]: pgmap v158: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-24T06:49:15.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:15 smithi137 ceph-mon[33017]: pgmap v159: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-24T06:49:15.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:15 smithi005 ceph-mon[27798]: pgmap v159: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-24T06:49:15.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:15 smithi037 ceph-mon[32864]: pgmap v159: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-24T06:49:17.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:17 smithi005 ceph-mon[27798]: pgmap v160: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-24T06:49:17.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:17 smithi137 ceph-mon[33017]: pgmap v160: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-24T06:49:17.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:17 smithi037 ceph-mon[32864]: pgmap v160: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-24T06:49:19.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:19 smithi137 ceph-mon[33017]: pgmap v161: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-24T06:49:19.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:19 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:49:19.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:19 smithi005 ceph-mon[27798]: pgmap v161: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-24T06:49:19.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:19 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:49:19.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:19 smithi037 ceph-mon[32864]: pgmap v161: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-24T06:49:19.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:19 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:49:20.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:20 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:49:20.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:20 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:49:20.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:20 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:49:20.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:20 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:49:20.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:20 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:49:20.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:20 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:49:20.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:20 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:49:20.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:20 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:49:20.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:20 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:49:21.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:21 smithi137 ceph-mon[33017]: pgmap v162: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6 op/s 2024-04-24T06:49:21.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:21 smithi005 ceph-mon[27798]: pgmap v162: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6 op/s 2024-04-24T06:49:21.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:21 smithi037 ceph-mon[32864]: pgmap v162: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6 op/s 2024-04-24T06:49:23.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:23 smithi137 ceph-mon[33017]: pgmap v163: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:49:23.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:23 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:49:23.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:23 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:49:23.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:23 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:49:23.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:23 smithi005 ceph-mon[27798]: pgmap v163: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:49:23.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:23 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:49:23.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:23 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:49:23.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:23 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:49:23.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:23 smithi037 ceph-mon[32864]: pgmap v163: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:49:23.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:23 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:49:23.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:23 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:49:23.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:23 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:49:25.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:25 smithi137 ceph-mon[33017]: pgmap v164: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-04-24T06:49:25.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:25 smithi005 ceph-mon[27798]: pgmap v164: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-04-24T06:49:25.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:25 smithi037 ceph-mon[32864]: pgmap v164: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-04-24T06:49:27.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:27 smithi137 ceph-mon[33017]: pgmap v165: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:49:27.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:27 smithi005 ceph-mon[27798]: pgmap v165: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:49:27.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:27 smithi037 ceph-mon[32864]: pgmap v165: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:49:29.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:29 smithi137 ceph-mon[33017]: pgmap v166: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:49:29.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:29 smithi005 ceph-mon[27798]: pgmap v166: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:49:29.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:29 smithi037 ceph-mon[32864]: pgmap v166: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:49:31.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:31 smithi137 ceph-mon[33017]: pgmap v167: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T06:49:31.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:31 smithi005 ceph-mon[27798]: pgmap v167: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T06:49:31.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:31 smithi037 ceph-mon[32864]: pgmap v167: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T06:49:33.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:33 smithi137 ceph-mon[33017]: pgmap v168: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:49:33.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:33 smithi005 ceph-mon[27798]: pgmap v168: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:49:33.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:33 smithi037 ceph-mon[32864]: pgmap v168: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:49:35.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:35 smithi137 ceph-mon[33017]: pgmap v169: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:49:35.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:35 smithi005 ceph-mon[27798]: pgmap v169: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:49:35.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:35 smithi037 ceph-mon[32864]: pgmap v169: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:49:37.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:37 smithi137 ceph-mon[33017]: pgmap v170: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:49:37.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:37 smithi005 ceph-mon[27798]: pgmap v170: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:49:37.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:37 smithi037 ceph-mon[32864]: pgmap v170: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:49:39.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:39 smithi137 ceph-mon[33017]: pgmap v171: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:49:39.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:39 smithi005 ceph-mon[27798]: pgmap v171: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:49:39.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:39 smithi037 ceph-mon[32864]: pgmap v171: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:49:41.309 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T06:49:41.309 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T06:49:41.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.527+0000 7f0559800640 1 -- 172.21.15.5:0/2929067553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05541029f0 msgr2=0x7f0554106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:41.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.527+0000 7f0559800640 1 --2- 172.21.15.5:0/2929067553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05541029f0 0x7f0554106a30 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f053c009900 tx=0x7f053c02a410 comp rx=0 tx=0).stop 2024-04-24T06:49:41.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.528+0000 7f0559800640 1 -- 172.21.15.5:0/2929067553 shutdown_connections 2024-04-24T06:49:41.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.528+0000 7f0559800640 1 --2- 172.21.15.5:0/2929067553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0554107110 0x7f055410ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:41.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.528+0000 7f0559800640 1 --2- 172.21.15.5:0/2929067553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05541029f0 0x7f0554106a30 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:41.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.528+0000 7f0559800640 1 --2- 172.21.15.5:0/2929067553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0554102040 0x7f0554102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:41.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.528+0000 7f0559800640 1 -- 172.21.15.5:0/2929067553 >> 172.21.15.5:0/2929067553 conn(0x7f05540fd8e0 msgr2=0x7f05540ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:41.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.528+0000 7f0559800640 1 -- 172.21.15.5:0/2929067553 shutdown_connections 2024-04-24T06:49:41.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.528+0000 7f0559800640 1 -- 172.21.15.5:0/2929067553 wait complete. 2024-04-24T06:49:41.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.529+0000 7f0559800640 1 Processor -- start 2024-04-24T06:49:41.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.529+0000 7f0559800640 1 -- start start 2024-04-24T06:49:41.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.529+0000 7f0559800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0554102040 0x7f05541a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:41.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.529+0000 7f0559800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0554107110 0x7f05541a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:41.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.529+0000 7f0559800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f055419a810 0x7f055419acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:41.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.529+0000 7f0559800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f05541121f0 con 0x7f0554102040 2024-04-24T06:49:41.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.529+0000 7f0559800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0554112070 con 0x7f055419a810 2024-04-24T06:49:41.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.529+0000 7f0559800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0554112370 con 0x7f0554107110 2024-04-24T06:49:41.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.530+0000 7f0553e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0554102040 0x7f05541a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:41.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.530+0000 7f054be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0554107110 0x7f05541a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:41.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.530+0000 7f0553e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0554102040 0x7f05541a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37848/0 (socket says 172.21.15.5:37848) 2024-04-24T06:49:41.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.530+0000 7f0553e00640 1 -- 172.21.15.5:0/3445362472 learned_addr learned my addr 172.21.15.5:0/3445362472 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:41.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.530+0000 7f0558e00640 1 --2- 172.21.15.5:0/3445362472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f055419a810 0x7f055419acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:41.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.530+0000 7f0553e00640 1 -- 172.21.15.5:0/3445362472 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0554107110 msgr2=0x7f05541a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:41.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.530+0000 7f0553e00640 1 --2- 172.21.15.5:0/3445362472 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0554107110 0x7f05541a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:41.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.530+0000 7f0553e00640 1 -- 172.21.15.5:0/3445362472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f055419a810 msgr2=0x7f055419acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:41.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.530+0000 7f0553e00640 1 --2- 172.21.15.5:0/3445362472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f055419a810 0x7f055419acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:41.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.530+0000 7f0553e00640 1 -- 172.21.15.5:0/3445362472 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f055419b3a0 con 0x7f0554102040 2024-04-24T06:49:41.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.531+0000 7f0553e00640 1 --2- 172.21.15.5:0/3445362472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0554102040 0x7f05541a0600 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f054400e9f0 tx=0x7f054400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:41.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.531+0000 7f0551600640 1 -- 172.21.15.5:0/3445362472 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054400cd80 con 0x7f0554102040 2024-04-24T06:49:41.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.531+0000 7f0551600640 1 -- 172.21.15.5:0/3445362472 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0544005500 con 0x7f0554102040 2024-04-24T06:49:41.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.531+0000 7f0559800640 1 -- 172.21.15.5:0/3445362472 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f055419b690 con 0x7f0554102040 2024-04-24T06:49:41.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.531+0000 7f0551600640 1 -- 172.21.15.5:0/3445362472 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0544010690 con 0x7f0554102040 2024-04-24T06:49:41.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.531+0000 7f0559800640 1 -- 172.21.15.5:0/3445362472 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05541a7750 con 0x7f0554102040 2024-04-24T06:49:41.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.534+0000 7f0551600640 1 -- 172.21.15.5:0/3445362472 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f05440046c0 con 0x7f0554102040 2024-04-24T06:49:41.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.534+0000 7f055dde0740 1 Processor -- start 2024-04-24T06:49:41.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.535+0000 7f055dde0740 1 -- start start 2024-04-24T06:49:41.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.535+0000 7f055dde0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f550300a30 0x55f550300e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:41.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.535+0000 7f055dde0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f550301350 0x55f55030b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:41.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.535+0000 7f055dde0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f55030ba20 0x55f55030de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:41.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.535+0000 7f055dde0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f5502661f0 con 0x55f550301350 2024-04-24T06:49:41.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.535+0000 7f055dde0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f55020c350 con 0x55f550300a30 2024-04-24T06:49:41.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.535+0000 7f055dde0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f5502621b0 con 0x55f55030ba20 2024-04-24T06:49:41.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.535+0000 7f0553e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f550301350 0x55f55030b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:41.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.535+0000 7f054be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f55030ba20 0x55f55030de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:41.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.535+0000 7f054be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f55030ba20 0x55f55030de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:60718/0 (socket says 172.21.15.5:60718) 2024-04-24T06:49:41.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.535+0000 7f054be00640 1 -- 172.21.15.5:0/3507247313 learned_addr learned my addr 172.21.15.5:0/3507247313 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:41.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.535+0000 7f0558e00640 1 --2- 172.21.15.5:0/3507247313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f550300a30 0x55f550300e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:41.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.536+0000 7f054be00640 1 -- 172.21.15.5:0/3507247313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f550300a30 msgr2=0x55f550300e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:41.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.536+0000 7f054be00640 1 --2- 172.21.15.5:0/3507247313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f550300a30 0x55f550300e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:41.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.536+0000 7f054be00640 1 -- 172.21.15.5:0/3507247313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f550301350 msgr2=0x55f55030b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:41.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.536+0000 7f054be00640 1 --2- 172.21.15.5:0/3507247313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f550301350 0x55f55030b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:41.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.536+0000 7f054be00640 1 -- 172.21.15.5:0/3507247313 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f550260f50 con 0x55f55030ba20 2024-04-24T06:49:41.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.536+0000 7f0558e00640 1 --2- 172.21.15.5:0/3507247313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f550300a30 0x55f550300e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:49:41.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.536+0000 7f0551600640 1 --2- 172.21.15.5:0/3445362472 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f05380a3d20 0x7f05380a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:41.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.536+0000 7f054be00640 1 --2- 172.21.15.5:0/3507247313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f55030ba20 0x55f55030de10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f053c009900 tx=0x7f053c005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=795190a7ce9b8d40 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:41.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.536+0000 7f0551600640 1 -- 172.21.15.5:0/3445362472 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f05440d0c60 con 0x7f0554102040 2024-04-24T06:49:41.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.537+0000 7f0549600640 1 -- 172.21.15.5:0/3507247313 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053c035040 con 0x55f55030ba20 2024-04-24T06:49:41.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.537+0000 7f0549600640 1 -- 172.21.15.5:0/3507247313 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f053c005520 con 0x55f55030ba20 2024-04-24T06:49:41.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.537+0000 7f0549600640 1 -- 172.21.15.5:0/3507247313 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053c0035f0 con 0x55f55030ba20 2024-04-24T06:49:41.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.537+0000 7f0558e00640 1 --2- 172.21.15.5:0/3445362472 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f05380a3d20 0x7f05380a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:41.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.537+0000 7f0558e00640 1 --2- 172.21.15.5:0/3445362472 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f05380a3d20 0x7f05380a61e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f054c009fd0 tx=0x7f054c009500 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:41.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.537+0000 7f055dde0740 1 -- 172.21.15.5:0/3507247313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f55030ba20 msgr2=0x55f55030de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:41.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.537+0000 7f055dde0740 1 --2- 172.21.15.5:0/3507247313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f55030ba20 0x55f55030de10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f053c009900 tx=0x7f053c005de0 comp rx=0 tx=0).stop 2024-04-24T06:49:41.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.537+0000 7f055dde0740 1 -- 172.21.15.5:0/3507247313 shutdown_connections 2024-04-24T06:49:41.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.537+0000 7f055dde0740 1 --2- 172.21.15.5:0/3507247313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f55030ba20 0x55f55030de10 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:41.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.537+0000 7f055dde0740 1 --2- 172.21.15.5:0/3507247313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f550301350 0x55f55030b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:41.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.537+0000 7f055dde0740 1 --2- 172.21.15.5:0/3507247313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f550300a30 0x55f550300e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:41.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.538+0000 7f055dde0740 1 -- 172.21.15.5:0/3507247313 >> 172.21.15.5:0/3507247313 conn(0x55f5502fcd50 msgr2=0x55f5502fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:41.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.538+0000 7f055dde0740 1 -- 172.21.15.5:0/3507247313 shutdown_connections 2024-04-24T06:49:41.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.538+0000 7f055dde0740 1 -- 172.21.15.5:0/3507247313 wait complete. 2024-04-24T06:49:41.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.538+0000 7f055dde0740 1 -- start start 2024-04-24T06:49:41.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.538+0000 7f055dde0740 1 Processor -- start 2024-04-24T06:49:41.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.539+0000 7f055dde0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f550300a30 0x55f550308990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:41.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.539+0000 7f055dde0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f550301350 0x55f550308ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:41.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.539+0000 7f055dde0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f55030ba20 0x55f550309410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:41.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.539+0000 7f055dde0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f55030e7b0 con 0x55f550301350 2024-04-24T06:49:41.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.539+0000 7f055dde0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f55021f690 con 0x55f55030ba20 2024-04-24T06:49:41.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.539+0000 7f055dde0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f55030e930 con 0x55f550300a30 2024-04-24T06:49:41.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.539+0000 7f055dde0740 10 client.0 _ll_register_callbacks cb 0x55f55025b650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:49:41.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.539+0000 7f054be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f550301350 0x55f550308ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:41.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.539+0000 7f0553e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f550300a30 0x55f550308990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:41.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.539+0000 7f0553e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f550300a30 0x55f550308990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:60730/0 (socket says 172.21.15.5:60730) 2024-04-24T06:49:41.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.539+0000 7f0553e00640 1 -- 172.21.15.5:0/1434073042 learned_addr learned my addr 172.21.15.5:0/1434073042 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:41.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.539+0000 7f0558e00640 1 --2- 172.21.15.5:0/1434073042 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f55030ba20 0x55f550309410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:41.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.540+0000 7f0553e00640 1 -- 172.21.15.5:0/1434073042 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f55030ba20 msgr2=0x55f550309410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:41.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.540+0000 7f0553e00640 1 --2- 172.21.15.5:0/1434073042 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f55030ba20 0x55f550309410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:41.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.540+0000 7f0553e00640 1 -- 172.21.15.5:0/1434073042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f550301350 msgr2=0x55f550308ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:41.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.540+0000 7f0553e00640 1 --2- 172.21.15.5:0/1434073042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f550301350 0x55f550308ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:41.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.540+0000 7f0553e00640 1 -- 172.21.15.5:0/1434073042 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f55021e920 con 0x55f550300a30 2024-04-24T06:49:41.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.540+0000 7f054be00640 1 --2- 172.21.15.5:0/1434073042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f550301350 0x55f550308ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:49:41.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.540+0000 7f0553e00640 1 --2- 172.21.15.5:0/1434073042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f550300a30 0x55f550308990 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f05440ce3c0 tx=0x7f0544004ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2942992101d1248 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:41.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.540+0000 7f0542000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:49:41.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.540+0000 7f0542000640 1 -- 172.21.15.5:0/1434073042 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054409c510 con 0x55f550300a30 2024-04-24T06:49:41.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.540+0000 7f0542000640 1 -- 172.21.15.5:0/1434073042 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f05440a5390 con 0x55f550300a30 2024-04-24T06:49:41.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.540+0000 7f055dde0740 1 -- 172.21.15.5:0/1434073042 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f053c005520 con 0x55f550300a30 2024-04-24T06:49:41.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.541+0000 7f0542000640 1 -- 172.21.15.5:0/1434073042 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f05440ce7e0 con 0x55f550300a30 2024-04-24T06:49:41.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.541+0000 7f055dde0740 10 client.26470 fetch_fsmap learned FSMap version 27 2024-04-24T06:49:41.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.541+0000 7f055dde0740 1 -- 172.21.15.5:0/1434073042 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f5500427d0 con 0x55f550300a30 2024-04-24T06:49:41.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.541+0000 7f0542000640 1 -- 172.21.15.5:0/1434073042 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f053c005520 con 0x55f550300a30 2024-04-24T06:49:41.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.541+0000 7f0542000640 1 -- 172.21.15.5:0/1434073042 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f054409c000 con 0x55f550300a30 2024-04-24T06:49:41.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.541+0000 7f055dde0740 10 client.26470 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:49:41.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.541+0000 7f055dde0740 10 client.26470 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:49:41.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.541+0000 7f055dde0740 20 client.26470 populate_metadata read hostname 'smithi005' 2024-04-24T06:49:41.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.541+0000 7f055dde0740 1 --2- 172.21.15.5:0/1434073042 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55f55031bd80 0x55f55031e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:41.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.541+0000 7f055dde0740 4 client.26470 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:49:41.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.541+0000 7f055dde0740 1 -- 172.21.15.5:0/1434073042 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f550273c10 con 0x55f55031bd80 2024-04-24T06:49:41.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.542+0000 7f054be00640 1 --2- 172.21.15.5:0/1434073042 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55f55031bd80 0x55f55031e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:41.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.542+0000 7f054be00640 1 --2- 172.21.15.5:0/1434073042 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55f55031bd80 0x55f55031e210 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1798d03e04e4055 server_cookie=72612f534b512635 in_seq=0 out_seq=0 2024-04-24T06:49:41.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.542+0000 7f0542000640 10 client.26470 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:49:41.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.543+0000 7f0542000640 1 -- 172.21.15.5:0/1434073042 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f550273c10 con 0x55f55031bd80 2024-04-24T06:49:41.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.543+0000 7f0542000640 10 client.26470 handle_command_reply: tid=0 2024-04-24T06:49:41.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.543+0000 7f055dde0740 1 client.26470 shutdown 2024-04-24T06:49:41.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.543+0000 7f055dde0740 1 -- 172.21.15.5:0/1434073042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f550300a30 msgr2=0x55f550308990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:41.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.543+0000 7f055dde0740 1 --2- 172.21.15.5:0/1434073042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f550300a30 0x55f550308990 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f05440ce3c0 tx=0x7f0544004ba0 comp rx=0 tx=0).stop 2024-04-24T06:49:41.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.543+0000 7f055dde0740 1 -- 172.21.15.5:0/1434073042 shutdown_connections 2024-04-24T06:49:41.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.543+0000 7f055dde0740 1 --2- 172.21.15.5:0/1434073042 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55f55031bd80 0x55f55031e210 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:41.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.543+0000 7f055dde0740 1 --2- 172.21.15.5:0/1434073042 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f55030ba20 0x55f550309410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:41.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.543+0000 7f055dde0740 1 --2- 172.21.15.5:0/1434073042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f550301350 0x55f550308ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:41.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.543+0000 7f0542000640 0 client.26470 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:49:41.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.543+0000 7f055dde0740 1 --2- 172.21.15.5:0/1434073042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f550300a30 0x55f550308990 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:41.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.543+0000 7f055dde0740 1 -- 172.21.15.5:0/1434073042 >> 172.21.15.5:0/1434073042 conn(0x55f5502fcd50 msgr2=0x55f5502fbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:41.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.544+0000 7f055dde0740 1 -- 172.21.15.5:0/1434073042 shutdown_connections 2024-04-24T06:49:41.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.544+0000 7f055dde0740 1 -- 172.21.15.5:0/1434073042 wait complete. 2024-04-24T06:49:41.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.544+0000 7f055dde0740 20 client.26470 trim_cache size 0 max 16384 2024-04-24T06:49:41.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.564+0000 7f055dde0740 1 Processor -- start 2024-04-24T06:49:41.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.564+0000 7f055dde0740 1 -- start start 2024-04-24T06:49:41.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.564+0000 7f055dde0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f550300a30 0x55f550317670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:41.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.564+0000 7f055dde0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f550301350 0x55f550302890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:41.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.564+0000 7f055dde0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f55030ba20 0x55f550323b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:41.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.564+0000 7f055dde0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f05541b2580 con 0x55f55030ba20 2024-04-24T06:49:41.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.564+0000 7f055dde0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f05541afe50 con 0x55f550300a30 2024-04-24T06:49:41.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.564+0000 7f055dde0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f05541adca0 con 0x55f550301350 2024-04-24T06:49:41.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.564+0000 7f0558e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f55030ba20 0x55f550323b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:41.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.564+0000 7f0553e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f550300a30 0x55f550317670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:41.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.564+0000 7f0558e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f55030ba20 0x55f550323b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37884/0 (socket says 172.21.15.5:37884) 2024-04-24T06:49:41.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.564+0000 7f0558e00640 1 -- 172.21.15.5:0/2948810539 learned_addr learned my addr 172.21.15.5:0/2948810539 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:41.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.564+0000 7f054be00640 1 --2- 172.21.15.5:0/2948810539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f550301350 0x55f550302890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:41.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.565+0000 7f0553e00640 1 -- 172.21.15.5:0/2948810539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f550301350 msgr2=0x55f550302890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:41.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.565+0000 7f0553e00640 1 --2- 172.21.15.5:0/2948810539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f550301350 0x55f550302890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:41.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.565+0000 7f0553e00640 1 -- 172.21.15.5:0/2948810539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f55030ba20 msgr2=0x55f550323b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:41.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.565+0000 7f0553e00640 1 --2- 172.21.15.5:0/2948810539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f55030ba20 0x55f550323b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:41.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.565+0000 7f0553e00640 1 -- 172.21.15.5:0/2948810539 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f54ff783f0 con 0x55f550300a30 2024-04-24T06:49:41.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.565+0000 7f0553e00640 1 --2- 172.21.15.5:0/2948810539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f550300a30 0x55f550317670 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0544099010 tx=0x7f0544009720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d74fe891211eda42 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:41.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.565+0000 7f0548c00640 1 -- 172.21.15.5:0/2948810539 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054409c980 con 0x55f550300a30 2024-04-24T06:49:41.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.565+0000 7f0548c00640 1 -- 172.21.15.5:0/2948810539 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05440a55e0 con 0x55f550300a30 2024-04-24T06:49:41.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.565+0000 7f0548c00640 1 -- 172.21.15.5:0/2948810539 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05440ce5e0 con 0x55f550300a30 2024-04-24T06:49:41.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.566+0000 7f055dde0740 1 -- 172.21.15.5:0/2948810539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f550300a30 msgr2=0x55f550317670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:41.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.566+0000 7f055dde0740 1 --2- 172.21.15.5:0/2948810539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f550300a30 0x55f550317670 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0544099010 tx=0x7f0544009720 comp rx=0 tx=0).stop 2024-04-24T06:49:41.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.566+0000 7f055dde0740 1 -- 172.21.15.5:0/2948810539 shutdown_connections 2024-04-24T06:49:41.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.566+0000 7f055dde0740 1 --2- 172.21.15.5:0/2948810539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f55030ba20 0x55f550323b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:41.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.566+0000 7f055dde0740 1 --2- 172.21.15.5:0/2948810539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f550301350 0x55f550302890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:41.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.566+0000 7f055dde0740 1 --2- 172.21.15.5:0/2948810539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f550300a30 0x55f550317670 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:41.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.566+0000 7f055dde0740 1 -- 172.21.15.5:0/2948810539 >> 172.21.15.5:0/2948810539 conn(0x55f5502fcd50 msgr2=0x55f550302180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:41.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.566+0000 7f055dde0740 1 -- 172.21.15.5:0/2948810539 shutdown_connections 2024-04-24T06:49:41.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.566+0000 7f055dde0740 1 -- 172.21.15.5:0/2948810539 wait complete. 2024-04-24T06:49:41.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.567+0000 7f055dde0740 1 -- start start 2024-04-24T06:49:41.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.567+0000 7f055dde0740 1 Processor -- start 2024-04-24T06:49:41.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.567+0000 7f055dde0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f550300a30 0x55f550313b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:41.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.567+0000 7f055dde0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f550301350 0x55f5503140c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:41.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.567+0000 7f055dde0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f55030ba20 0x55f550304d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:41.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.567+0000 7f055dde0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f05541ae310 con 0x55f55030ba20 2024-04-24T06:49:41.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.567+0000 7f055dde0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f05541af780 con 0x55f550300a30 2024-04-24T06:49:41.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.567+0000 7f055dde0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f05541ae9e0 con 0x55f550301350 2024-04-24T06:49:41.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.567+0000 7f055dde0740 10 client.0 _ll_register_callbacks cb 0x7f05440d4400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:49:41.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.568+0000 7f0553e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f550300a30 0x55f550313b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:41.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.568+0000 7f0558e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f55030ba20 0x55f550304d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:41.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.568+0000 7f0553e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f550300a30 0x55f550313b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:46008/0 (socket says 172.21.15.5:46008) 2024-04-24T06:49:41.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.568+0000 7f0553e00640 1 -- 172.21.15.5:0/2890385705 learned_addr learned my addr 172.21.15.5:0/2890385705 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:41.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.568+0000 7f054be00640 1 --2- 172.21.15.5:0/2890385705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f550301350 0x55f5503140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:41.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.568+0000 7f0553e00640 1 -- 172.21.15.5:0/2890385705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f550301350 msgr2=0x55f5503140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:41.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.568+0000 7f0553e00640 1 --2- 172.21.15.5:0/2890385705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f550301350 0x55f5503140c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:41.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.568+0000 7f0553e00640 1 -- 172.21.15.5:0/2890385705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f55030ba20 msgr2=0x55f550304d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:41.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.568+0000 7f0553e00640 1 --2- 172.21.15.5:0/2890385705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f55030ba20 0x55f550304d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:41.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.568+0000 7f0553e00640 1 -- 172.21.15.5:0/2890385705 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f54ff380f0 con 0x55f550300a30 2024-04-24T06:49:41.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.568+0000 7f0558e00640 1 --2- 172.21.15.5:0/2890385705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f55030ba20 0x55f550304d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:49:41.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.568+0000 7f0553e00640 1 --2- 172.21.15.5:0/2890385705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f550300a30 0x55f550313b80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f054409ae60 tx=0x7f05440cecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74cf00db31485203 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:41.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.568+0000 7f0542000640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:49:41.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.569+0000 7f0542000640 1 -- 172.21.15.5:0/2890385705 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054409c980 con 0x55f550300a30 2024-04-24T06:49:41.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.569+0000 7f0542000640 1 -- 172.21.15.5:0/2890385705 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f0544098900 con 0x55f550300a30 2024-04-24T06:49:41.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.569+0000 7f055dde0740 1 -- 172.21.15.5:0/2890385705 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05440a55e0 con 0x55f550300a30 2024-04-24T06:49:41.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.569+0000 7f0542000640 1 -- 172.21.15.5:0/2890385705 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f05440ce7f0 con 0x55f550300a30 2024-04-24T06:49:41.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.569+0000 7f055dde0740 10 client.26465 fetch_fsmap learned FSMap version 27 2024-04-24T06:49:41.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.569+0000 7f055dde0740 1 -- 172.21.15.5:0/2890385705 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f55022b810 con 0x55f550300a30 2024-04-24T06:49:41.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.569+0000 7f0542000640 1 -- 172.21.15.5:0/2890385705 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05440a55e0 con 0x55f550300a30 2024-04-24T06:49:41.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.569+0000 7f0542000640 1 -- 172.21.15.5:0/2890385705 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f05440a20f0 con 0x55f550300a30 2024-04-24T06:49:41.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.569+0000 7f055dde0740 10 client.26465 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:49:41.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.569+0000 7f055dde0740 10 client.26465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:49:41.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.569+0000 7f055dde0740 20 client.26465 populate_metadata read hostname 'smithi005' 2024-04-24T06:49:41.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.569+0000 7f055dde0740 1 --2- 172.21.15.5:0/2890385705 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55f5503820b0 0x55f5503844d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:41.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.570+0000 7f055dde0740 4 client.26465 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:49:41.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.570+0000 7f055dde0740 1 -- 172.21.15.5:0/2890385705 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f55026da80 con 0x55f5503820b0 2024-04-24T06:49:41.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.570+0000 7f054be00640 1 --2- 172.21.15.5:0/2890385705 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55f5503820b0 0x55f5503844d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:41.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.570+0000 7f054be00640 1 --2- 172.21.15.5:0/2890385705 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55f5503820b0 0x55f5503844d0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97fa39d0f9ed10d9 server_cookie=ed15ecc64083888f in_seq=0 out_seq=0 2024-04-24T06:49:41.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.570+0000 7f0542000640 10 client.26465 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:49:41.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.571+0000 7f0542000640 1 -- 172.21.15.5:0/2890385705 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f55026da80 con 0x55f5503820b0 2024-04-24T06:49:41.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.571+0000 7f0542000640 10 client.26465 handle_command_reply: tid=0 2024-04-24T06:49:41.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.571+0000 7f055dde0740 1 client.26465 shutdown 2024-04-24T06:49:41.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.571+0000 7f055dde0740 1 -- 172.21.15.5:0/2890385705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f550300a30 msgr2=0x55f550313b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:41.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.571+0000 7f055dde0740 1 --2- 172.21.15.5:0/2890385705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f550300a30 0x55f550313b80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f054409ae60 tx=0x7f05440cecc0 comp rx=0 tx=0).stop 2024-04-24T06:49:41.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.571+0000 7f055dde0740 1 -- 172.21.15.5:0/2890385705 shutdown_connections 2024-04-24T06:49:41.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.571+0000 7f055dde0740 1 --2- 172.21.15.5:0/2890385705 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55f5503820b0 0x55f5503844d0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:41.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.571+0000 7f055dde0740 1 --2- 172.21.15.5:0/2890385705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f55030ba20 0x55f550304d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:41.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.571+0000 7f055dde0740 1 --2- 172.21.15.5:0/2890385705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f550301350 0x55f5503140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:41.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.571+0000 7f0542000640 0 client.26465 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:49:41.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.571+0000 7f055dde0740 1 --2- 172.21.15.5:0/2890385705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f550300a30 0x55f550313b80 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:41.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.571+0000 7f055dde0740 1 -- 172.21.15.5:0/2890385705 >> 172.21.15.5:0/2890385705 conn(0x55f5502fcd50 msgr2=0x55f5503091e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:41.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.571+0000 7f055dde0740 1 -- 172.21.15.5:0/2890385705 shutdown_connections 2024-04-24T06:49:41.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.571+0000 7f055dde0740 1 -- 172.21.15.5:0/2890385705 wait complete. 2024-04-24T06:49:41.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.572+0000 7f055dde0740 20 client.26465 trim_cache size 0 max 16384 2024-04-24T06:49:41.584 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:49:41.584 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-24T06:49:41.584 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-24T06:49:41.585 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:49:41.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.572+0000 7f0559800640 1 -- 172.21.15.5:0/3445362472 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f05380a3d20 msgr2=0x7f05380a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:41.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.572+0000 7f0559800640 1 --2- 172.21.15.5:0/3445362472 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f05380a3d20 0x7f05380a61e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f054c009fd0 tx=0x7f054c009500 comp rx=0 tx=0).stop 2024-04-24T06:49:41.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.572+0000 7f0559800640 1 -- 172.21.15.5:0/3445362472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0554102040 msgr2=0x7f05541a0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:41.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.572+0000 7f0559800640 1 --2- 172.21.15.5:0/3445362472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0554102040 0x7f05541a0600 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f054400e9f0 tx=0x7f054400edf0 comp rx=0 tx=0).stop 2024-04-24T06:49:41.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.572+0000 7f0559800640 1 -- 172.21.15.5:0/3445362472 shutdown_connections 2024-04-24T06:49:41.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.572+0000 7f0559800640 1 --2- 172.21.15.5:0/3445362472 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f05380a3d20 0x7f05380a61e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:41.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.572+0000 7f0559800640 1 --2- 172.21.15.5:0/3445362472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f055419a810 0x7f055419acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:41.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.572+0000 7f0559800640 1 --2- 172.21.15.5:0/3445362472 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0554107110 0x7f05541a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:41.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.572+0000 7f0559800640 1 --2- 172.21.15.5:0/3445362472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0554102040 0x7f05541a0600 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:41.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.572+0000 7f0559800640 1 -- 172.21.15.5:0/3445362472 >> 172.21.15.5:0/3445362472 conn(0x7f05540fd8e0 msgr2=0x7f05540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:41.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.572+0000 7f0559800640 1 -- 172.21.15.5:0/3445362472 shutdown_connections 2024-04-24T06:49:41.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.573+0000 7f0559800640 1 -- 172.21.15.5:0/3445362472 wait complete. 2024-04-24T06:49:41.591 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T06:49:41.592 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:49:41.651 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:41 smithi005 ceph-mon[27798]: pgmap v172: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T06:49:41.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:41 smithi137 ceph-mon[33017]: pgmap v172: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T06:49:41.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:41 smithi037 ceph-mon[32864]: pgmap v172: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T06:49:41.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.787+0000 7f0882c00640 1 -- 172.21.15.5:0/1398771974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f087c112d80 msgr2=0x7f087c115170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:41.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.787+0000 7f0882c00640 1 --2- 172.21.15.5:0/1398771974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f087c112d80 0x7f087c115170 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0874009c10 tx=0x7f087402a6e0 comp rx=0 tx=0).stop 2024-04-24T06:49:41.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.787+0000 7f0882c00640 1 -- 172.21.15.5:0/1398771974 shutdown_connections 2024-04-24T06:49:41.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.787+0000 7f0882c00640 1 --2- 172.21.15.5:0/1398771974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f087c112d80 0x7f087c115170 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:41.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.787+0000 7f0882c00640 1 --2- 172.21.15.5:0/1398771974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f087c076e50 0x7f087c112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:41.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.787+0000 7f0882c00640 1 --2- 172.21.15.5:0/1398771974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f087c076530 0x7f087c076910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:41.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.787+0000 7f0882c00640 1 -- 172.21.15.5:0/1398771974 >> 172.21.15.5:0/1398771974 conn(0x7f087c0ffb70 msgr2=0x7f087c101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:41.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.787+0000 7f0882c00640 1 -- 172.21.15.5:0/1398771974 shutdown_connections 2024-04-24T06:49:41.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.787+0000 7f0882c00640 1 -- 172.21.15.5:0/1398771974 wait complete. 2024-04-24T06:49:41.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.788+0000 7f0882c00640 1 Processor -- start 2024-04-24T06:49:41.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.788+0000 7f0882c00640 1 -- start start 2024-04-24T06:49:41.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.789+0000 7f0882c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f087c076530 0x7f087c19ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:41.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.789+0000 7f0882c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f087c076e50 0x7f087c19f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:41.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.789+0000 7f0882c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f087c112d80 0x7f087c1a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:41.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.789+0000 7f0882c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f087c118850 con 0x7f087c112d80 2024-04-24T06:49:41.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.789+0000 7f0882c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f087c1186d0 con 0x7f087c076e50 2024-04-24T06:49:41.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.789+0000 7f0882c00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f087c1189d0 con 0x7f087c076530 2024-04-24T06:49:41.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.789+0000 7f0882200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f087c112d80 0x7f087c1a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:41.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.789+0000 7f0881800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f087c076530 0x7f087c19ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:41.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.789+0000 7f0880e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f087c076e50 0x7f087c19f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:41.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.789+0000 7f0881800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f087c076530 0x7f087c19ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:60758/0 (socket says 172.21.15.5:60758) 2024-04-24T06:49:41.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.789+0000 7f0881800640 1 -- 172.21.15.5:0/1150190994 learned_addr learned my addr 172.21.15.5:0/1150190994 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:41.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.789+0000 7f0880e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f087c076e50 0x7f087c19f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:46018/0 (socket says 172.21.15.5:46018) 2024-04-24T06:49:41.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.790+0000 7f0881800640 1 -- 172.21.15.5:0/1150190994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f087c076e50 msgr2=0x7f087c19f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:41.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.790+0000 7f0881800640 1 --2- 172.21.15.5:0/1150190994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f087c076e50 0x7f087c19f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:41.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.790+0000 7f0881800640 1 -- 172.21.15.5:0/1150190994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f087c112d80 msgr2=0x7f087c1a3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:41.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.790+0000 7f0881800640 1 --2- 172.21.15.5:0/1150190994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f087c112d80 0x7f087c1a3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:41.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.790+0000 7f0881800640 1 -- 172.21.15.5:0/1150190994 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f087c1a3db0 con 0x7f087c076530 2024-04-24T06:49:41.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.790+0000 7f0881800640 1 --2- 172.21.15.5:0/1150190994 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f087c076530 0x7f087c19ee10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f087000e9c0 tx=0x7f087000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:41.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.790+0000 7f087a000640 1 -- 172.21.15.5:0/1150190994 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087000cde0 con 0x7f087c076530 2024-04-24T06:49:41.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.790+0000 7f087a000640 1 -- 172.21.15.5:0/1150190994 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08700035f0 con 0x7f087c076530 2024-04-24T06:49:41.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.790+0000 7f0882c00640 1 -- 172.21.15.5:0/1150190994 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f087c1a40a0 con 0x7f087c076530 2024-04-24T06:49:41.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.790+0000 7f087a000640 1 -- 172.21.15.5:0/1150190994 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0870010690 con 0x7f087c076530 2024-04-24T06:49:41.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.790+0000 7f0882c00640 1 -- 172.21.15.5:0/1150190994 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f087c1ab930 con 0x7f087c076530 2024-04-24T06:49:41.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.791+0000 7f0882c00640 1 -- 172.21.15.5:0/1150190994 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f087c04fe00 con 0x7f087c076530 2024-04-24T06:49:41.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.794+0000 7f087a000640 1 -- 172.21.15.5:0/1150190994 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f08700037a0 con 0x7f087c076530 2024-04-24T06:49:41.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.795+0000 7f087a000640 1 --2- 172.21.15.5:0/1150190994 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f08640a3d10 0x7f08640a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:41.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.795+0000 7f087a000640 1 -- 172.21.15.5:0/1150190994 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f08700d0c20 con 0x7f087c076530 2024-04-24T06:49:41.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.795+0000 7f0880e00640 1 --2- 172.21.15.5:0/1150190994 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f08640a3d10 0x7f08640a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:41.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.796+0000 7f0880e00640 1 --2- 172.21.15.5:0/1150190994 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f08640a3d10 0x7f08640a61d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f087c102040 tx=0x7f086c005f90 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:41.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:41.797+0000 7f087a000640 1 -- 172.21.15.5:0/1150190994 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f087009d300 con 0x7f087c076530 2024-04-24T06:49:42.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.049+0000 7f0882c00640 1 -- 172.21.15.5:0/1150190994 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f087c1a0110 con 0x7f087c076530 2024-04-24T06:49:42.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.050+0000 7f087a000640 1 -- 172.21.15.5:0/1150190994 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11856 (secure 0 0 0) 0x7f087009cc00 con 0x7f087c076530 2024-04-24T06:49:42.051 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:49:42.052 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1735462750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1735462750},{"type":"v1","addr":"172.21.15.5:6837","nonce":1735462750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1499469370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1499469370},{"type":"v1","addr":"172.21.15.5:6841","nonce":1499469370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3705638847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3705638847},{"type":"v1","addr":"172.21.15.37:6835","nonce":3705638847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T06:41:34.764490+0000","modified":"2024-04-24T06:48:57.768410+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24550,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6837/1390103296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1390103296},{"type":"v1","addr":"172.21.15.37:6837","nonce":1390103296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3217754215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3217754215},{"type":"v1","addr":"172.21.15.137:6835","nonce":3217754215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6837/3044222976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3044222976},{"type":"v1","addr":"172.21.15.137:6837","nonce":3044222976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.5:6839/3173201459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3173201459},{"type":"v1","addr":"172.21.15.5:6839","nonce":3173201459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6839/1889774232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1889774232},{"type":"v1","addr":"172.21.15.137:6839","nonce":1889774232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6839/3536721753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":3536721753},{"type":"v1","addr":"172.21.15.37:6839","nonce":3536721753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24550,24569,24539]},"id":1}]} 2024-04-24T06:49:42.052 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-04-24T06:49:42.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.053+0000 7f0882c00640 1 -- 172.21.15.5:0/1150190994 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f08640a3d10 msgr2=0x7f08640a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:42.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.053+0000 7f0882c00640 1 --2- 172.21.15.5:0/1150190994 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f08640a3d10 0x7f08640a61d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f087c102040 tx=0x7f086c005f90 comp rx=0 tx=0).stop 2024-04-24T06:49:42.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.053+0000 7f0882c00640 1 -- 172.21.15.5:0/1150190994 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f087c076530 msgr2=0x7f087c19ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:42.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.053+0000 7f0882c00640 1 --2- 172.21.15.5:0/1150190994 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f087c076530 0x7f087c19ee10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f087000e9c0 tx=0x7f087000ee80 comp rx=0 tx=0).stop 2024-04-24T06:49:42.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.053+0000 7f0882c00640 1 -- 172.21.15.5:0/1150190994 shutdown_connections 2024-04-24T06:49:42.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.053+0000 7f0882c00640 1 --2- 172.21.15.5:0/1150190994 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f08640a3d10 0x7f08640a61d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.053+0000 7f0882c00640 1 --2- 172.21.15.5:0/1150190994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f087c112d80 0x7f087c1a3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.053+0000 7f0882c00640 1 --2- 172.21.15.5:0/1150190994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f087c076e50 0x7f087c19f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.053+0000 7f0882c00640 1 --2- 172.21.15.5:0/1150190994 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f087c076530 0x7f087c19ee10 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.053+0000 7f0882c00640 1 -- 172.21.15.5:0/1150190994 >> 172.21.15.5:0/1150190994 conn(0x7f087c0ffb70 msgr2=0x7f087c114510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:42.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.053+0000 7f0882c00640 1 -- 172.21.15.5:0/1150190994 shutdown_connections 2024-04-24T06:49:42.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.053+0000 7f0882c00640 1 -- 172.21.15.5:0/1150190994 wait complete. 2024-04-24T06:49:42.076 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T06:49:42.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.291+0000 7fcc5f200640 1 -- 172.21.15.5:0/3380116542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc58102be0 msgr2=0x7fcc58106c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:42.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.291+0000 7fcc5f200640 1 --2- 172.21.15.5:0/3380116542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc58102be0 0x7fcc58106c20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcc44009860 tx=0x7fcc4402a340 comp rx=0 tx=0).stop 2024-04-24T06:49:42.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.291+0000 7fcc5f200640 1 -- 172.21.15.5:0/3380116542 shutdown_connections 2024-04-24T06:49:42.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.291+0000 7fcc5f200640 1 --2- 172.21.15.5:0/3380116542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc58107300 0x7fcc5810eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.291+0000 7fcc5f200640 1 --2- 172.21.15.5:0/3380116542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc58102be0 0x7fcc58106c20 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.291+0000 7fcc5f200640 1 --2- 172.21.15.5:0/3380116542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc58102230 0x7fcc58102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.291+0000 7fcc5f200640 1 -- 172.21.15.5:0/3380116542 >> 172.21.15.5:0/3380116542 conn(0x7fcc580fdab0 msgr2=0x7fcc580ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:42.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.291+0000 7fcc5f200640 1 -- 172.21.15.5:0/3380116542 shutdown_connections 2024-04-24T06:49:42.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.291+0000 7fcc5f200640 1 -- 172.21.15.5:0/3380116542 wait complete. 2024-04-24T06:49:42.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.293+0000 7fcc5f200640 1 Processor -- start 2024-04-24T06:49:42.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.293+0000 7fcc5f200640 1 -- start start 2024-04-24T06:49:42.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.293+0000 7fcc5f200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc58102230 0x7fcc581a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.293+0000 7fcc5f200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc58107300 0x7fcc581a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.294+0000 7fcc5f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc5819a990 0x7fcc5819ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.294+0000 7fcc5f200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcc58112370 con 0x7fcc5819a990 2024-04-24T06:49:42.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.294+0000 7fcc5f200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcc581121f0 con 0x7fcc58102230 2024-04-24T06:49:42.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.294+0000 7fcc5f200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcc581124f0 con 0x7fcc58107300 2024-04-24T06:49:42.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.294+0000 7fcc5d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc5819a990 0x7fcc5819ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.294+0000 7fcc57400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc58107300 0x7fcc581a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.294+0000 7fcc5d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc5819a990 0x7fcc5819ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37928/0 (socket says 172.21.15.5:37928) 2024-04-24T06:49:42.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.294+0000 7fcc5d200640 1 -- 172.21.15.5:0/3618480769 learned_addr learned my addr 172.21.15.5:0/3618480769 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:42.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.294+0000 7fcc57e00640 1 --2- 172.21.15.5:0/3618480769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc58102230 0x7fcc581a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.294+0000 7fcc57400640 1 -- 172.21.15.5:0/3618480769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc58102230 msgr2=0x7fcc581a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:42.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.294+0000 7fcc57400640 1 --2- 172.21.15.5:0/3618480769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc58102230 0x7fcc581a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.294+0000 7fcc57400640 1 -- 172.21.15.5:0/3618480769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc5819a990 msgr2=0x7fcc5819ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:42.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.294+0000 7fcc57400640 1 --2- 172.21.15.5:0/3618480769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc5819a990 0x7fcc5819ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.294+0000 7fcc57400640 1 -- 172.21.15.5:0/3618480769 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc5819b4b0 con 0x7fcc58107300 2024-04-24T06:49:42.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.294+0000 7fcc5d200640 1 --2- 172.21.15.5:0/3618480769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc5819a990 0x7fcc5819ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:49:42.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.295+0000 7fcc57400640 1 --2- 172.21.15.5:0/3618480769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc58107300 0x7fcc581a0cd0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcc4402a860 tx=0x7fcc44033d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:42.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.295+0000 7fcc54c00640 1 -- 172.21.15.5:0/3618480769 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc44037040 con 0x7fcc58107300 2024-04-24T06:49:42.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.295+0000 7fcc54c00640 1 -- 172.21.15.5:0/3618480769 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc44030550 con 0x7fcc58107300 2024-04-24T06:49:42.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.295+0000 7fcc54c00640 1 -- 172.21.15.5:0/3618480769 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc440039f0 con 0x7fcc58107300 2024-04-24T06:49:42.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.295+0000 7fcc5f200640 1 -- 172.21.15.5:0/3618480769 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc5819b740 con 0x7fcc58107300 2024-04-24T06:49:42.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.295+0000 7fcc5f200640 1 -- 172.21.15.5:0/3618480769 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc581a7940 con 0x7fcc58107300 2024-04-24T06:49:42.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.299+0000 7fcc62013740 1 Processor -- start 2024-04-24T06:49:42.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.299+0000 7fcc62013740 1 -- start start 2024-04-24T06:49:42.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.300+0000 7fcc54c00640 1 -- 172.21.15.5:0/3618480769 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fcc44042020 con 0x7fcc58107300 2024-04-24T06:49:42.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.300+0000 7fcc62013740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8e93c3950 0x55d8e93c3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.300+0000 7fcc62013740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8e93c4270 0x55d8e93ce400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.300+0000 7fcc62013740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8e93ce940 0x55d8e93d0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.300+0000 7fcc62013740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d8e93291d0 con 0x55d8e93c3950 2024-04-24T06:49:42.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.300+0000 7fcc62013740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d8e92cf450 con 0x55d8e93c4270 2024-04-24T06:49:42.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.300+0000 7fcc62013740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d8e9325190 con 0x55d8e93ce940 2024-04-24T06:49:42.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.300+0000 7fcc57e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8e93c4270 0x55d8e93ce400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.300+0000 7fcc57e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8e93c4270 0x55d8e93ce400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:46062/0 (socket says 172.21.15.5:46062) 2024-04-24T06:49:42.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.300+0000 7fcc57e00640 1 -- 172.21.15.5:0/1442435550 learned_addr learned my addr 172.21.15.5:0/1442435550 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:42.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.300+0000 7fcc5d200640 1 --2- 172.21.15.5:0/1442435550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8e93c3950 0x55d8e93c3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.300+0000 7fcc57400640 1 --2- 172.21.15.5:0/1442435550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8e93ce940 0x55d8e93d0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.301+0000 7fcc57e00640 1 -- 172.21.15.5:0/1442435550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8e93ce940 msgr2=0x55d8e93d0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:42.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.301+0000 7fcc57e00640 1 --2- 172.21.15.5:0/1442435550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8e93ce940 0x55d8e93d0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.301+0000 7fcc57e00640 1 -- 172.21.15.5:0/1442435550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8e93c3950 msgr2=0x55d8e93c3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:42.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.301+0000 7fcc57e00640 1 --2- 172.21.15.5:0/1442435550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8e93c3950 0x55d8e93c3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.301+0000 7fcc57e00640 1 -- 172.21.15.5:0/1442435550 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8e938c310 con 0x55d8e93c4270 2024-04-24T06:49:42.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.301+0000 7fcc57e00640 1 --2- 172.21.15.5:0/1442435550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8e93c4270 0x55d8e93ce400 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800bc90 tx=0x7fcc480107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2c9943ab522832c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:42.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.301+0000 7fcc4d600640 1 -- 172.21.15.5:0/1442435550 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc48005220 con 0x55d8e93c4270 2024-04-24T06:49:42.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.301+0000 7fcc4d600640 1 -- 172.21.15.5:0/1442435550 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc4800be40 con 0x55d8e93c4270 2024-04-24T06:49:42.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.301+0000 7fcc4d600640 1 -- 172.21.15.5:0/1442435550 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc48011770 con 0x55d8e93c4270 2024-04-24T06:49:42.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.302+0000 7fcc54c00640 1 --2- 172.21.15.5:0/3618480769 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fcc3c0a3da0 0x7fcc3c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.302+0000 7fcc62013740 1 -- 172.21.15.5:0/1442435550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8e93c4270 msgr2=0x55d8e93ce400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:42.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.302+0000 7fcc62013740 1 --2- 172.21.15.5:0/1442435550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8e93c4270 0x55d8e93ce400 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800bc90 tx=0x7fcc480107a0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.303+0000 7fcc62013740 1 -- 172.21.15.5:0/1442435550 shutdown_connections 2024-04-24T06:49:42.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.303+0000 7fcc54c00640 1 -- 172.21.15.5:0/3618480769 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fcc440efc20 con 0x7fcc58107300 2024-04-24T06:49:42.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.303+0000 7fcc62013740 1 --2- 172.21.15.5:0/1442435550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8e93ce940 0x55d8e93d0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.303+0000 7fcc62013740 1 --2- 172.21.15.5:0/1442435550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8e93c4270 0x55d8e93ce400 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.303+0000 7fcc62013740 1 --2- 172.21.15.5:0/1442435550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8e93c3950 0x55d8e93c3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.303+0000 7fcc62013740 1 -- 172.21.15.5:0/1442435550 >> 172.21.15.5:0/1442435550 conn(0x55d8e93bfc70 msgr2=0x55d8e93c0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:42.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.303+0000 7fcc5d200640 1 --2- 172.21.15.5:0/3618480769 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fcc3c0a3da0 0x7fcc3c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.303+0000 7fcc62013740 1 -- 172.21.15.5:0/1442435550 shutdown_connections 2024-04-24T06:49:42.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.303+0000 7fcc62013740 1 -- 172.21.15.5:0/1442435550 wait complete. 2024-04-24T06:49:42.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.303+0000 7fcc62013740 1 -- start start 2024-04-24T06:49:42.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.304+0000 7fcc5d200640 1 --2- 172.21.15.5:0/3618480769 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fcc3c0a3da0 0x7fcc3c0a6260 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcc50005c40 tx=0x7fcc500095c0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:42.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.304+0000 7fcc62013740 1 Processor -- start 2024-04-24T06:49:42.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.304+0000 7fcc62013740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8e93c3950 0x55d8e93ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.304+0000 7fcc62013740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8e93c4270 0x55d8e93cb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.304+0000 7fcc62013740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8e93ce940 0x55d8e93cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.304+0000 7fcc62013740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d8e93d17d0 con 0x55d8e93c3950 2024-04-24T06:49:42.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.304+0000 7fcc62013740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d8e92e2810 con 0x55d8e93c4270 2024-04-24T06:49:42.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.304+0000 7fcc62013740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d8e93d1950 con 0x55d8e93ce940 2024-04-24T06:49:42.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.305+0000 7fcc62013740 10 client.0 _ll_register_callbacks cb 0x55d8e92b0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:49:42.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.305+0000 7fcc5d200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8e93ce940 0x55d8e93cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.305+0000 7fcc5d200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8e93ce940 0x55d8e93cbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:60802/0 (socket says 172.21.15.5:60802) 2024-04-24T06:49:42.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.305+0000 7fcc5d200640 1 -- 172.21.15.5:0/2510924060 learned_addr learned my addr 172.21.15.5:0/2510924060 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:42.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.305+0000 7fcc57400640 1 --2- 172.21.15.5:0/2510924060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8e93c4270 0x55d8e93cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.305+0000 7fcc57e00640 1 --2- 172.21.15.5:0/2510924060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8e93c3950 0x55d8e93ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.305+0000 7fcc5d200640 1 -- 172.21.15.5:0/2510924060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8e93c4270 msgr2=0x55d8e93cb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:42.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.305+0000 7fcc5d200640 1 --2- 172.21.15.5:0/2510924060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8e93c4270 0x55d8e93cb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.305+0000 7fcc5d200640 1 -- 172.21.15.5:0/2510924060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8e93c3950 msgr2=0x55d8e93ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:42.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.305+0000 7fcc5d200640 1 --2- 172.21.15.5:0/2510924060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8e93c3950 0x55d8e93ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.305+0000 7fcc5d200640 1 -- 172.21.15.5:0/2510924060 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8e92f33f0 con 0x55d8e93ce940 2024-04-24T06:49:42.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.306+0000 7fcc57400640 1 --2- 172.21.15.5:0/2510924060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8e93c4270 0x55d8e93cb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:49:42.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.306+0000 7fcc5d200640 1 --2- 172.21.15.5:0/2510924060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8e93ce940 0x55d8e93cbda0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcc5000bfd0 tx=0x7fcc50004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9b0f6955b592717 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:42.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.306+0000 7fcc42000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:49:42.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.306+0000 7fcc42000640 1 -- 172.21.15.5:0/2510924060 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc500197e0 con 0x55d8e93ce940 2024-04-24T06:49:42.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.306+0000 7fcc62013740 1 -- 172.21.15.5:0/2510924060 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d8e9336bf0 con 0x55d8e93ce940 2024-04-24T06:49:42.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.306+0000 7fcc42000640 1 -- 172.21.15.5:0/2510924060 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fcc50019980 con 0x55d8e93ce940 2024-04-24T06:49:42.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.306+0000 7fcc42000640 1 -- 172.21.15.5:0/2510924060 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcc50009e40 con 0x55d8e93ce940 2024-04-24T06:49:42.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.306+0000 7fcc62013740 10 client.26494 fetch_fsmap learned FSMap version 27 2024-04-24T06:49:42.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.306+0000 7fcc62013740 1 -- 172.21.15.5:0/2510924060 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d8e903b3f0 con 0x55d8e93ce940 2024-04-24T06:49:42.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.307+0000 7fcc42000640 1 -- 172.21.15.5:0/2510924060 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d8e9336bf0 con 0x55d8e93ce940 2024-04-24T06:49:42.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.307+0000 7fcc42000640 1 -- 172.21.15.5:0/2510924060 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7fcc5000b040 con 0x55d8e93ce940 2024-04-24T06:49:42.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.307+0000 7fcc62013740 10 client.26494 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:49:42.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.307+0000 7fcc62013740 10 client.26494 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-24T06:49:42.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.307+0000 7fcc62013740 20 client.26494 populate_metadata read hostname 'smithi005' 2024-04-24T06:49:42.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.307+0000 7fcc62013740 1 --2- 172.21.15.5:0/2510924060 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55d8e93deb30 0x55d8e93e0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.307+0000 7fcc62013740 4 client.26494 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:49:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.307+0000 7fcc62013740 1 -- 172.21.15.5:0/2510924060 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8e9330a60 con 0x55d8e93deb30 2024-04-24T06:49:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.308+0000 7fcc57e00640 1 --2- 172.21.15.5:0/2510924060 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55d8e93deb30 0x55d8e93e0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.308+0000 7fcc57e00640 1 --2- 172.21.15.5:0/2510924060 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55d8e93deb30 0x55d8e93e0f80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=916c956f19dd5fcc server_cookie=12bf6d8d60666ef in_seq=0 out_seq=0 2024-04-24T06:49:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.308+0000 7fcc42000640 10 client.26494 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:49:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.309+0000 7fcc42000640 1 -- 172.21.15.5:0/2510924060 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d8e9330a60 con 0x55d8e93deb30 2024-04-24T06:49:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.309+0000 7fcc42000640 10 client.26494 handle_command_reply: tid=0 2024-04-24T06:49:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.309+0000 7fcc62013740 1 client.26494 shutdown 2024-04-24T06:49:42.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.309+0000 7fcc62013740 1 -- 172.21.15.5:0/2510924060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8e93ce940 msgr2=0x55d8e93cbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:42.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.309+0000 7fcc62013740 1 --2- 172.21.15.5:0/2510924060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8e93ce940 0x55d8e93cbda0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcc5000bfd0 tx=0x7fcc50004ab0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.309+0000 7fcc62013740 1 -- 172.21.15.5:0/2510924060 shutdown_connections 2024-04-24T06:49:42.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.309+0000 7fcc62013740 1 --2- 172.21.15.5:0/2510924060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8e93ce940 0x55d8e93cbda0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.309+0000 7fcc62013740 1 --2- 172.21.15.5:0/2510924060 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55d8e93deb30 0x55d8e93e0f80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.309+0000 7fcc62013740 1 --2- 172.21.15.5:0/2510924060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8e93c4270 0x55d8e93cb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.309+0000 7fcc62013740 1 --2- 172.21.15.5:0/2510924060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8e93c3950 0x55d8e93ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.309+0000 7fcc42000640 0 client.26494 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:49:42.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.309+0000 7fcc62013740 1 -- 172.21.15.5:0/2510924060 >> 172.21.15.5:0/2510924060 conn(0x55d8e93bfc70 msgr2=0x55d8e93c5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:42.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.310+0000 7fcc62013740 1 -- 172.21.15.5:0/2510924060 shutdown_connections 2024-04-24T06:49:42.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.310+0000 7fcc62013740 1 -- 172.21.15.5:0/2510924060 wait complete. 2024-04-24T06:49:42.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.310+0000 7fcc62013740 20 client.26494 trim_cache size 0 max 16384 2024-04-24T06:49:42.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.325+0000 7fcc62013740 1 Processor -- start 2024-04-24T06:49:42.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.325+0000 7fcc62013740 1 -- start start 2024-04-24T06:49:42.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.325+0000 7fcc62013740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8e93c3950 0x55d8e93da690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.325+0000 7fcc62013740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8e93c4270 0x55d8e93cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.325+0000 7fcc62013740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8e93ce940 0x55d8e93ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.326+0000 7fcc62013740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcc581b2820 con 0x55d8e93c3950 2024-04-24T06:49:42.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.326+0000 7fcc62013740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcc581b00f0 con 0x55d8e93ce940 2024-04-24T06:49:42.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.326+0000 7fcc62013740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcc581adf40 con 0x55d8e93c4270 2024-04-24T06:49:42.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.326+0000 7fcc57e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8e93c3950 0x55d8e93da690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.326+0000 7fcc57e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8e93c3950 0x55d8e93da690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37970/0 (socket says 172.21.15.5:37970) 2024-04-24T06:49:42.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.326+0000 7fcc57e00640 1 -- 172.21.15.5:0/159608917 learned_addr learned my addr 172.21.15.5:0/159608917 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:42.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.326+0000 7fcc57400640 1 --2- 172.21.15.5:0/159608917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8e93c4270 0x55d8e93cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.326+0000 7fcc5d200640 1 --2- 172.21.15.5:0/159608917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8e93ce940 0x55d8e93ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.326+0000 7fcc57e00640 1 -- 172.21.15.5:0/159608917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8e93c4270 msgr2=0x55d8e93cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:42.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.326+0000 7fcc57e00640 1 --2- 172.21.15.5:0/159608917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8e93c4270 0x55d8e93cdea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.326+0000 7fcc57e00640 1 -- 172.21.15.5:0/159608917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8e93ce940 msgr2=0x55d8e93ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:42.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.326+0000 7fcc57e00640 1 --2- 172.21.15.5:0/159608917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8e93ce940 0x55d8e93ce3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.326+0000 7fcc57e00640 1 -- 172.21.15.5:0/159608917 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8e91057d0 con 0x55d8e93c3950 2024-04-24T06:49:42.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.326+0000 7fcc5d200640 1 --2- 172.21.15.5:0/159608917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8e93ce940 0x55d8e93ce3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:49:42.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.327+0000 7fcc57e00640 1 --2- 172.21.15.5:0/159608917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8e93c3950 0x55d8e93da690 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fcc48000c00 tx=0x7fcc48005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a9ecccab4b3d31c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:42.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.327+0000 7fcc4cc00640 1 -- 172.21.15.5:0/159608917 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc48005340 con 0x55d8e93c3950 2024-04-24T06:49:42.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.327+0000 7fcc4cc00640 1 -- 172.21.15.5:0/159608917 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc4802b1d0 con 0x55d8e93c3950 2024-04-24T06:49:42.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.327+0000 7fcc4cc00640 1 -- 172.21.15.5:0/159608917 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc480048c0 con 0x55d8e93c3950 2024-04-24T06:49:42.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.328+0000 7fcc62013740 1 -- 172.21.15.5:0/159608917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8e93c3950 msgr2=0x55d8e93da690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:42.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.328+0000 7fcc62013740 1 --2- 172.21.15.5:0/159608917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8e93c3950 0x55d8e93da690 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fcc48000c00 tx=0x7fcc48005640 comp rx=0 tx=0).stop 2024-04-24T06:49:42.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.328+0000 7fcc62013740 1 -- 172.21.15.5:0/159608917 shutdown_connections 2024-04-24T06:49:42.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.328+0000 7fcc62013740 1 --2- 172.21.15.5:0/159608917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8e93ce940 0x55d8e93ce3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.328+0000 7fcc62013740 1 --2- 172.21.15.5:0/159608917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8e93c4270 0x55d8e93cdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.328+0000 7fcc62013740 1 --2- 172.21.15.5:0/159608917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8e93c3950 0x55d8e93da690 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.328+0000 7fcc62013740 1 -- 172.21.15.5:0/159608917 >> 172.21.15.5:0/159608917 conn(0x55d8e93bfc70 msgr2=0x55d8e93c6940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.328+0000 7fcc62013740 1 -- 172.21.15.5:0/159608917 shutdown_connections 2024-04-24T06:49:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.328+0000 7fcc62013740 1 -- 172.21.15.5:0/159608917 wait complete. 2024-04-24T06:49:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.328+0000 7fcc62013740 1 -- start start 2024-04-24T06:49:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.329+0000 7fcc62013740 1 Processor -- start 2024-04-24T06:49:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.329+0000 7fcc62013740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8e93c3950 0x55d8e93cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.329+0000 7fcc62013740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8e93c4270 0x55d8e93d01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.329+0000 7fcc62013740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8e93ce940 0x55d8e93d5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.329+0000 7fcc62013740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcc581ae5b0 con 0x55d8e93ce940 2024-04-24T06:49:42.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.329+0000 7fcc62013740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcc581afa20 con 0x55d8e93c3950 2024-04-24T06:49:42.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.329+0000 7fcc62013740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcc581aec80 con 0x55d8e93c4270 2024-04-24T06:49:42.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.329+0000 7fcc62013740 10 client.0 _ll_register_callbacks cb 0x7fcc5000ff50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:49:42.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.329+0000 7fcc5d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8e93ce940 0x55d8e93d5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.329+0000 7fcc5d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8e93ce940 0x55d8e93d5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37982/0 (socket says 172.21.15.5:37982) 2024-04-24T06:49:42.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.329+0000 7fcc5d200640 1 -- 172.21.15.5:0/3385531177 learned_addr learned my addr 172.21.15.5:0/3385531177 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:42.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.329+0000 7fcc57e00640 1 --2- 172.21.15.5:0/3385531177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8e93c3950 0x55d8e93cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.330+0000 7fcc5d200640 1 -- 172.21.15.5:0/3385531177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8e93c4270 msgr2=0x55d8e93d01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:42.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.330+0000 7fcc5d200640 1 --2- 172.21.15.5:0/3385531177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8e93c4270 0x55d8e93d01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.330+0000 7fcc5d200640 1 -- 172.21.15.5:0/3385531177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8e93c3950 msgr2=0x55d8e93cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:42.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.330+0000 7fcc5d200640 1 --2- 172.21.15.5:0/3385531177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8e93c3950 0x55d8e93cfc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.330+0000 7fcc5d200640 1 -- 172.21.15.5:0/3385531177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8e93d5650 con 0x55d8e93ce940 2024-04-24T06:49:42.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.330+0000 7fcc5d200640 1 --2- 172.21.15.5:0/3385531177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8e93ce940 0x55d8e93d5110 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fcc50009750 tx=0x7fcc5002fbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e08bcab2713b321 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:42.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.330+0000 7fcc42000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:49:42.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.330+0000 7fcc42000640 1 -- 172.21.15.5:0/3385531177 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5000f220 con 0x55d8e93ce940 2024-04-24T06:49:42.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.330+0000 7fcc42000640 1 -- 172.21.15.5:0/3385531177 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fcc5000f3c0 con 0x55d8e93ce940 2024-04-24T06:49:42.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.330+0000 7fcc62013740 1 -- 172.21.15.5:0/3385531177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc4802b1d0 con 0x55d8e93ce940 2024-04-24T06:49:42.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.331+0000 7fcc42000640 1 -- 172.21.15.5:0/3385531177 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcc5000f5c0 con 0x55d8e93ce940 2024-04-24T06:49:42.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.331+0000 7fcc62013740 10 client.16353 fetch_fsmap learned FSMap version 27 2024-04-24T06:49:42.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.331+0000 7fcc62013740 1 -- 172.21.15.5:0/3385531177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d8e9445740 con 0x55d8e93ce940 2024-04-24T06:49:42.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.331+0000 7fcc42000640 1 -- 172.21.15.5:0/3385531177 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc4802b1d0 con 0x55d8e93ce940 2024-04-24T06:49:42.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.331+0000 7fcc42000640 1 -- 172.21.15.5:0/3385531177 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7fcc5003e070 con 0x55d8e93ce940 2024-04-24T06:49:42.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.331+0000 7fcc62013740 10 client.16353 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:49:42.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.331+0000 7fcc62013740 10 client.16353 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-24T06:49:42.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.331+0000 7fcc62013740 20 client.16353 populate_metadata read hostname 'smithi005' 2024-04-24T06:49:42.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.331+0000 7fcc62013740 1 --2- 172.21.15.5:0/3385531177 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55d8e9445eb0 0x55d8e94482f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.331+0000 7fcc62013740 4 client.16353 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:49:42.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.331+0000 7fcc62013740 1 -- 172.21.15.5:0/3385531177 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d8e92ac4d0 con 0x55d8e9445eb0 2024-04-24T06:49:42.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.332+0000 7fcc57e00640 1 --2- 172.21.15.5:0/3385531177 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55d8e9445eb0 0x55d8e94482f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.332+0000 7fcc57e00640 1 --2- 172.21.15.5:0/3385531177 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55d8e9445eb0 0x55d8e94482f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=301397ceec598373 server_cookie=e9f148d098057fdf in_seq=0 out_seq=0 2024-04-24T06:49:42.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.332+0000 7fcc42000640 10 client.16353 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:49:42.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.332+0000 7fcc42000640 1 -- 172.21.15.5:0/3385531177 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d8e92ac4d0 con 0x55d8e9445eb0 2024-04-24T06:49:42.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.332+0000 7fcc42000640 10 client.16353 handle_command_reply: tid=0 2024-04-24T06:49:42.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.332+0000 7fcc62013740 1 client.16353 shutdown 2024-04-24T06:49:42.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.333+0000 7fcc62013740 1 -- 172.21.15.5:0/3385531177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8e93ce940 msgr2=0x55d8e93d5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:42.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.333+0000 7fcc62013740 1 --2- 172.21.15.5:0/3385531177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8e93ce940 0x55d8e93d5110 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fcc50009750 tx=0x7fcc5002fbf0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.333+0000 7fcc62013740 1 -- 172.21.15.5:0/3385531177 shutdown_connections 2024-04-24T06:49:42.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.333+0000 7fcc62013740 1 --2- 172.21.15.5:0/3385531177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8e93ce940 0x55d8e93d5110 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.333+0000 7fcc62013740 1 --2- 172.21.15.5:0/3385531177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8e93c4270 0x55d8e93d01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.333+0000 7fcc62013740 1 --2- 172.21.15.5:0/3385531177 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55d8e9445eb0 0x55d8e94482f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.333+0000 7fcc62013740 1 --2- 172.21.15.5:0/3385531177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d8e93c3950 0x55d8e93cfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.333+0000 7fcc62013740 1 -- 172.21.15.5:0/3385531177 >> 172.21.15.5:0/3385531177 conn(0x55d8e93bfc70 msgr2=0x55d8e93be560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:42.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.333+0000 7fcc42000640 0 client.16353 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:49:42.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.333+0000 7fcc62013740 1 -- 172.21.15.5:0/3385531177 shutdown_connections 2024-04-24T06:49:42.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.333+0000 7fcc62013740 1 -- 172.21.15.5:0/3385531177 wait complete. 2024-04-24T06:49:42.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.333+0000 7fcc62013740 20 client.16353 trim_cache size 0 max 16384 2024-04-24T06:49:42.341 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:49:42.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.334+0000 7fcc5f200640 1 -- 172.21.15.5:0/3618480769 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fcc3c0a3da0 msgr2=0x7fcc3c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:42.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.334+0000 7fcc5f200640 1 --2- 172.21.15.5:0/3618480769 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fcc3c0a3da0 0x7fcc3c0a6260 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcc50005c40 tx=0x7fcc500095c0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.334+0000 7fcc5f200640 1 -- 172.21.15.5:0/3618480769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc58107300 msgr2=0x7fcc581a0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:42.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.334+0000 7fcc5f200640 1 --2- 172.21.15.5:0/3618480769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc58107300 0x7fcc581a0cd0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcc4402a860 tx=0x7fcc44033d90 comp rx=0 tx=0).stop 2024-04-24T06:49:42.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.334+0000 7fcc5f200640 1 -- 172.21.15.5:0/3618480769 shutdown_connections 2024-04-24T06:49:42.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.334+0000 7fcc5f200640 1 --2- 172.21.15.5:0/3618480769 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fcc3c0a3da0 0x7fcc3c0a6260 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.334+0000 7fcc5f200640 1 --2- 172.21.15.5:0/3618480769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc5819a990 0x7fcc5819ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.334+0000 7fcc5f200640 1 --2- 172.21.15.5:0/3618480769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc58107300 0x7fcc581a0cd0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.334+0000 7fcc5f200640 1 --2- 172.21.15.5:0/3618480769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc58102230 0x7fcc581a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.334+0000 7fcc5f200640 1 -- 172.21.15.5:0/3618480769 >> 172.21.15.5:0/3618480769 conn(0x7fcc580fdab0 msgr2=0x7fcc580fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:42.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.334+0000 7fcc5f200640 1 -- 172.21.15.5:0/3618480769 shutdown_connections 2024-04-24T06:49:42.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.334+0000 7fcc5f200640 1 -- 172.21.15.5:0/3618480769 wait complete. 2024-04-24T06:49:42.353 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T06:49:42.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.554+0000 7f4389a00640 1 -- 172.21.15.5:0/816692646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4384104c40 msgr2=0x7f4384108c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:42.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.554+0000 7f4389a00640 1 --2- 172.21.15.5:0/816692646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4384104c40 0x7f4384108c80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f436c009ca0 tx=0x7f436c02a8c0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.554+0000 7f4389a00640 1 -- 172.21.15.5:0/816692646 shutdown_connections 2024-04-24T06:49:42.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.554+0000 7f4389a00640 1 --2- 172.21.15.5:0/816692646 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4384109360 0x7f4384110f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.554+0000 7f4389a00640 1 --2- 172.21.15.5:0/816692646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4384104c40 0x7f4384108c80 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.554+0000 7f4389a00640 1 --2- 172.21.15.5:0/816692646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4384104290 0x7f4384104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.554+0000 7f4389a00640 1 -- 172.21.15.5:0/816692646 >> 172.21.15.5:0/816692646 conn(0x7f43840ffaf0 msgr2=0x7f4384101f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:42.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.554+0000 7f4389a00640 1 -- 172.21.15.5:0/816692646 shutdown_connections 2024-04-24T06:49:42.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.554+0000 7f4389a00640 1 -- 172.21.15.5:0/816692646 wait complete. 2024-04-24T06:49:42.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.555+0000 7f4389a00640 1 Processor -- start 2024-04-24T06:49:42.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.555+0000 7f4389a00640 1 -- start start 2024-04-24T06:49:42.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.556+0000 7f4389a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4384104290 0x7f43841a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.556+0000 7f4389a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4384109360 0x7f43841a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.556+0000 7f4389a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f438419a7e0 0x7f438419ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.556+0000 7f4389a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4384114440 con 0x7f4384109360 2024-04-24T06:49:42.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.556+0000 7f4389a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f43841142c0 con 0x7f438419a7e0 2024-04-24T06:49:42.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.556+0000 7f4389a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f43841145c0 con 0x7f4384104290 2024-04-24T06:49:42.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.556+0000 7f4383400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4384109360 0x7f43841a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.556+0000 7f4383400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4384109360 0x7f43841a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38000/0 (socket says 172.21.15.5:38000) 2024-04-24T06:49:42.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.556+0000 7f4383400640 1 -- 172.21.15.5:0/2345642762 learned_addr learned my addr 172.21.15.5:0/2345642762 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:42.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.556+0000 7f4383e00640 1 --2- 172.21.15.5:0/2345642762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4384104290 0x7f43841a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.556+0000 7f4389000640 1 --2- 172.21.15.5:0/2345642762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f438419a7e0 0x7f438419ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.556+0000 7f4383400640 1 -- 172.21.15.5:0/2345642762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4384104290 msgr2=0x7f43841a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:42.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.557+0000 7f4383400640 1 --2- 172.21.15.5:0/2345642762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4384104290 0x7f43841a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.557+0000 7f4383400640 1 -- 172.21.15.5:0/2345642762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f438419a7e0 msgr2=0x7f438419ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:42.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.557+0000 7f4383400640 1 --2- 172.21.15.5:0/2345642762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f438419a7e0 0x7f438419ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.557+0000 7f4383400640 1 -- 172.21.15.5:0/2345642762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f438419b370 con 0x7f4384109360 2024-04-24T06:49:42.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.557+0000 7f4383e00640 1 --2- 172.21.15.5:0/2345642762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4384104290 0x7f43841a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:49:42.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.557+0000 7f4383400640 1 --2- 172.21.15.5:0/2345642762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4384109360 0x7f43841a0bb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f436c000f80 tx=0x7f436c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:42.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.557+0000 7f4380c00640 1 -- 172.21.15.5:0/2345642762 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c031650 con 0x7f4384109360 2024-04-24T06:49:42.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.557+0000 7f4380c00640 1 -- 172.21.15.5:0/2345642762 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f436c0317f0 con 0x7f4384109360 2024-04-24T06:49:42.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.557+0000 7f4389a00640 1 -- 172.21.15.5:0/2345642762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f438419b600 con 0x7f4384109360 2024-04-24T06:49:42.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.557+0000 7f4380c00640 1 -- 172.21.15.5:0/2345642762 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c02f980 con 0x7f4384109360 2024-04-24T06:49:42.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.558+0000 7f4389a00640 1 -- 172.21.15.5:0/2345642762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43841a77e0 con 0x7f4384109360 2024-04-24T06:49:42.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.561+0000 7f438de2c740 1 Processor -- start 2024-04-24T06:49:42.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.561+0000 7f4380c00640 1 -- 172.21.15.5:0/2345642762 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f436c02fb40 con 0x7f4384109360 2024-04-24T06:49:42.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.561+0000 7f438de2c740 1 -- start start 2024-04-24T06:49:42.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.562+0000 7f438de2c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56387cdc7b40 0x56387cdc7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.562+0000 7f438de2c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56387cdc8460 0x56387cdd25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.562+0000 7f438de2c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56387cdd2b30 0x56387cdd4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.562+0000 7f438de2c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56387cd2d1d0 con 0x56387cdd2b30 2024-04-24T06:49:42.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.562+0000 7f438de2c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56387ccd3450 con 0x56387cdc7b40 2024-04-24T06:49:42.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.562+0000 7f438de2c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56387cd29190 con 0x56387cdc8460 2024-04-24T06:49:42.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.562+0000 7f4383e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56387cdc8460 0x56387cdd25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.562+0000 7f4383e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56387cdc8460 0x56387cdd25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:60838/0 (socket says 172.21.15.5:60838) 2024-04-24T06:49:42.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.562+0000 7f4389000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56387cdc7b40 0x56387cdc7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.562+0000 7f4383e00640 1 -- 172.21.15.5:0/2475760421 learned_addr learned my addr 172.21.15.5:0/2475760421 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:42.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.562+0000 7f4383400640 1 --2- 172.21.15.5:0/2475760421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56387cdd2b30 0x56387cdd4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.562+0000 7f4383e00640 1 -- 172.21.15.5:0/2475760421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56387cdc7b40 msgr2=0x56387cdc7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:42.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.562+0000 7f4383e00640 1 --2- 172.21.15.5:0/2475760421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56387cdc7b40 0x56387cdc7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.562+0000 7f4383e00640 1 -- 172.21.15.5:0/2475760421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56387cdd2b30 msgr2=0x56387cdd4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:42.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.562+0000 7f4383e00640 1 --2- 172.21.15.5:0/2475760421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56387cdd2b30 0x56387cdd4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.562+0000 7f4383e00640 1 -- 172.21.15.5:0/2475760421 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56387cd90310 con 0x56387cdc8460 2024-04-24T06:49:42.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.563+0000 7f4383e00640 1 --2- 172.21.15.5:0/2475760421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56387cdc8460 0x56387cdd25f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f43740115c0 tx=0x7f4374011a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bf20480cc4569bc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:42.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.563+0000 7f4379600640 1 -- 172.21.15.5:0/2475760421 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437400f640 con 0x56387cdc8460 2024-04-24T06:49:42.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.563+0000 7f4379600640 1 -- 172.21.15.5:0/2475760421 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f437400fcc0 con 0x56387cdc8460 2024-04-24T06:49:42.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.563+0000 7f4379600640 1 -- 172.21.15.5:0/2475760421 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4374019dc0 con 0x56387cdc8460 2024-04-24T06:49:42.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.563+0000 7f4380c00640 1 --2- 172.21.15.5:0/2345642762 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f43680a3d20 0x7f43680a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.563+0000 7f4380c00640 1 -- 172.21.15.5:0/2345642762 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f436c0f1a80 con 0x7f4384109360 2024-04-24T06:49:42.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.563+0000 7f4389000640 1 --2- 172.21.15.5:0/2345642762 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f43680a3d20 0x7f43680a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.564+0000 7f438de2c740 1 -- 172.21.15.5:0/2475760421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56387cdc8460 msgr2=0x56387cdd25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:42.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.564+0000 7f438de2c740 1 --2- 172.21.15.5:0/2475760421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56387cdc8460 0x56387cdd25f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f43740115c0 tx=0x7f4374011a80 comp rx=0 tx=0).stop 2024-04-24T06:49:42.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.564+0000 7f438de2c740 1 -- 172.21.15.5:0/2475760421 shutdown_connections 2024-04-24T06:49:42.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.564+0000 7f438de2c740 1 --2- 172.21.15.5:0/2475760421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56387cdd2b30 0x56387cdd4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.564+0000 7f438de2c740 1 --2- 172.21.15.5:0/2475760421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56387cdc8460 0x56387cdd25f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.564+0000 7f438de2c740 1 --2- 172.21.15.5:0/2475760421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56387cdc7b40 0x56387cdc7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.564+0000 7f438de2c740 1 -- 172.21.15.5:0/2475760421 >> 172.21.15.5:0/2475760421 conn(0x56387cdc3e60 msgr2=0x56387cdc4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:42.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.564+0000 7f438de2c740 1 -- 172.21.15.5:0/2475760421 shutdown_connections 2024-04-24T06:49:42.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.564+0000 7f4389000640 1 --2- 172.21.15.5:0/2345642762 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f43680a3d20 0x7f43680a61e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f437c009d30 tx=0x7f437c005f90 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:42.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.564+0000 7f438de2c740 1 -- 172.21.15.5:0/2475760421 wait complete. 2024-04-24T06:49:42.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.565+0000 7f438de2c740 1 -- start start 2024-04-24T06:49:42.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.565+0000 7f438de2c740 1 Processor -- start 2024-04-24T06:49:42.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.565+0000 7f438de2c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56387cdc7b40 0x56387cdd25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.565+0000 7f438de2c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56387cdc8460 0x56387cdcf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.565+0000 7f438de2c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56387cdd2b30 0x56387cdd0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.565+0000 7f438de2c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56387cdd59c0 con 0x56387cdc7b40 2024-04-24T06:49:42.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.565+0000 7f438de2c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56387cce6810 con 0x56387cdc8460 2024-04-24T06:49:42.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.565+0000 7f438de2c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56387cdd5b40 con 0x56387cdd2b30 2024-04-24T06:49:42.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.565+0000 7f438de2c740 10 client.0 _ll_register_callbacks cb 0x56387ccb4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:49:42.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.566+0000 7f4383e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56387cdc7b40 0x56387cdd25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.566+0000 7f4383e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56387cdc7b40 0x56387cdd25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38022/0 (socket says 172.21.15.5:38022) 2024-04-24T06:49:42.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.566+0000 7f4383e00640 1 -- 172.21.15.5:0/592184862 learned_addr learned my addr 172.21.15.5:0/592184862 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:42.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.566+0000 7f4389000640 1 --2- 172.21.15.5:0/592184862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56387cdd2b30 0x56387cdd0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.566+0000 7f4383400640 1 --2- 172.21.15.5:0/592184862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56387cdc8460 0x56387cdcf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.566+0000 7f4389000640 1 -- 172.21.15.5:0/592184862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56387cdc8460 msgr2=0x56387cdcf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:42.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.566+0000 7f4389000640 1 --2- 172.21.15.5:0/592184862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56387cdc8460 0x56387cdcf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.566+0000 7f4389000640 1 -- 172.21.15.5:0/592184862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56387cdc7b40 msgr2=0x56387cdd25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:42.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.566+0000 7f4389000640 1 --2- 172.21.15.5:0/592184862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56387cdc7b40 0x56387cdd25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.566+0000 7f4389000640 1 -- 172.21.15.5:0/592184862 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56387ccf73f0 con 0x56387cdd2b30 2024-04-24T06:49:42.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.566+0000 7f4383e00640 1 --2- 172.21.15.5:0/592184862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56387cdc7b40 0x56387cdd25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:49:42.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.566+0000 7f4389000640 1 --2- 172.21.15.5:0/592184862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56387cdd2b30 0x56387cdd0010 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f437c01e9d0 tx=0x7f437c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8694fe1ba51bdc0a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:42.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.566+0000 7f4372000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:49:42.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.567+0000 7f4372000640 1 -- 172.21.15.5:0/592184862 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437c02e060 con 0x56387cdd2b30 2024-04-24T06:49:42.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.567+0000 7f438de2c740 1 -- 172.21.15.5:0/592184862 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f437400fcc0 con 0x56387cdd2b30 2024-04-24T06:49:42.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.567+0000 7f4372000640 1 -- 172.21.15.5:0/592184862 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f437c018030 con 0x56387cdd2b30 2024-04-24T06:49:42.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.567+0000 7f4372000640 1 -- 172.21.15.5:0/592184862 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f437c009e20 con 0x56387cdd2b30 2024-04-24T06:49:42.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.567+0000 7f438de2c740 10 client.26509 fetch_fsmap learned FSMap version 27 2024-04-24T06:49:42.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.567+0000 7f438de2c740 1 -- 172.21.15.5:0/592184862 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56387ca3f3f0 con 0x56387cdd2b30 2024-04-24T06:49:42.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.567+0000 7f4372000640 1 -- 172.21.15.5:0/592184862 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f437400fcc0 con 0x56387cdd2b30 2024-04-24T06:49:42.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.568+0000 7f4372000640 1 -- 172.21.15.5:0/592184862 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f437c00f550 con 0x56387cdd2b30 2024-04-24T06:49:42.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.568+0000 7f438de2c740 10 client.26509 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:49:42.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.568+0000 7f438de2c740 10 client.26509 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T06:49:42.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.568+0000 7f438de2c740 20 client.26509 populate_metadata read hostname 'smithi005' 2024-04-24T06:49:42.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.568+0000 7f438de2c740 1 --2- 172.21.15.5:0/592184862 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x56387cde2d60 0x56387cde51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.568+0000 7f438de2c740 4 client.26509 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:49:42.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.568+0000 7f438de2c740 1 -- 172.21.15.5:0/592184862 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56387cd3abf0 con 0x56387cde2d60 2024-04-24T06:49:42.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.568+0000 7f4383e00640 1 --2- 172.21.15.5:0/592184862 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x56387cde2d60 0x56387cde51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.569+0000 7f4383e00640 1 --2- 172.21.15.5:0/592184862 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x56387cde2d60 0x56387cde51b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=16a85ea7351b6bd0 server_cookie=f02150e6f60b6039 in_seq=0 out_seq=0 2024-04-24T06:49:42.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.569+0000 7f4372000640 10 client.26509 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:49:42.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.570+0000 7f4372000640 1 -- 172.21.15.5:0/592184862 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56387cd3abf0 con 0x56387cde2d60 2024-04-24T06:49:42.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.570+0000 7f4372000640 10 client.26509 handle_command_reply: tid=0 2024-04-24T06:49:42.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.570+0000 7f438de2c740 1 client.26509 shutdown 2024-04-24T06:49:42.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.570+0000 7f438de2c740 1 -- 172.21.15.5:0/592184862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56387cdd2b30 msgr2=0x56387cdd0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:42.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.570+0000 7f438de2c740 1 --2- 172.21.15.5:0/592184862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56387cdd2b30 0x56387cdd0010 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f437c01e9d0 tx=0x7f437c004730 comp rx=0 tx=0).stop 2024-04-24T06:49:42.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.570+0000 7f438de2c740 1 -- 172.21.15.5:0/592184862 shutdown_connections 2024-04-24T06:49:42.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.570+0000 7f438de2c740 1 --2- 172.21.15.5:0/592184862 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x56387cde2d60 0x56387cde51b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.570+0000 7f438de2c740 1 --2- 172.21.15.5:0/592184862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56387cdd2b30 0x56387cdd0010 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.570+0000 7f438de2c740 1 --2- 172.21.15.5:0/592184862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56387cdc8460 0x56387cdcf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.570+0000 7f438de2c740 1 --2- 172.21.15.5:0/592184862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56387cdc7b40 0x56387cdd25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.570+0000 7f4372000640 0 client.26509 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:49:42.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.570+0000 7f438de2c740 1 -- 172.21.15.5:0/592184862 >> 172.21.15.5:0/592184862 conn(0x56387cdc3e60 msgr2=0x56387cdc9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:42.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.571+0000 7f438de2c740 1 -- 172.21.15.5:0/592184862 shutdown_connections 2024-04-24T06:49:42.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.571+0000 7f438de2c740 1 -- 172.21.15.5:0/592184862 wait complete. 2024-04-24T06:49:42.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.571+0000 7f438de2c740 20 client.26509 trim_cache size 0 max 16384 2024-04-24T06:49:42.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.586+0000 7f438de2c740 1 Processor -- start 2024-04-24T06:49:42.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.586+0000 7f438de2c740 1 -- start start 2024-04-24T06:49:42.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.586+0000 7f438de2c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56387cdc7b40 0x56387cdde880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.586+0000 7f438de2c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56387cdc8460 0x56387cdd2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.586+0000 7f438de2c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56387cdd2b30 0x56387cdeaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.586+0000 7f438de2c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f43841b2690 con 0x56387cdc8460 2024-04-24T06:49:42.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.586+0000 7f438de2c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f43841aff60 con 0x56387cdd2b30 2024-04-24T06:49:42.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.586+0000 7f438de2c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f43841addb0 con 0x56387cdc7b40 2024-04-24T06:49:42.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.586+0000 7f4383400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56387cdc8460 0x56387cdd2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.586+0000 7f4383400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56387cdc8460 0x56387cdd2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38024/0 (socket says 172.21.15.5:38024) 2024-04-24T06:49:42.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.586+0000 7f4383400640 1 -- 172.21.15.5:0/4116510034 learned_addr learned my addr 172.21.15.5:0/4116510034 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:42.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.587+0000 7f4383e00640 1 --2- 172.21.15.5:0/4116510034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56387cdc7b40 0x56387cdde880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.587+0000 7f4389000640 1 --2- 172.21.15.5:0/4116510034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56387cdd2b30 0x56387cdeaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.587+0000 7f4383400640 1 -- 172.21.15.5:0/4116510034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56387cdc7b40 msgr2=0x56387cdde880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:42.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.587+0000 7f4383400640 1 --2- 172.21.15.5:0/4116510034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56387cdc7b40 0x56387cdde880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.587+0000 7f4383400640 1 -- 172.21.15.5:0/4116510034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56387cdd2b30 msgr2=0x56387cdeaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:42.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.587+0000 7f4383400640 1 --2- 172.21.15.5:0/4116510034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56387cdd2b30 0x56387cdeaa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.587+0000 7f4383400640 1 -- 172.21.15.5:0/4116510034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56387cb097d0 con 0x56387cdc8460 2024-04-24T06:49:42.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.587+0000 7f4383400640 1 --2- 172.21.15.5:0/4116510034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56387cdc8460 0x56387cdd2110 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f436c0ef2f0 tx=0x7f436c02f480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20777277821ae1a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:42.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.587+0000 7f4378c00640 1 -- 172.21.15.5:0/4116510034 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c0f8070 con 0x56387cdc8460 2024-04-24T06:49:42.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.587+0000 7f4378c00640 1 -- 172.21.15.5:0/4116510034 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f436c036070 con 0x56387cdc8460 2024-04-24T06:49:42.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.587+0000 7f4378c00640 1 -- 172.21.15.5:0/4116510034 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c0c1e90 con 0x56387cdc8460 2024-04-24T06:49:42.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.588+0000 7f438de2c740 1 -- 172.21.15.5:0/4116510034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56387cdc8460 msgr2=0x56387cdd2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:42.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.588+0000 7f438de2c740 1 --2- 172.21.15.5:0/4116510034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56387cdc8460 0x56387cdd2110 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f436c0ef2f0 tx=0x7f436c02f480 comp rx=0 tx=0).stop 2024-04-24T06:49:42.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.588+0000 7f438de2c740 1 -- 172.21.15.5:0/4116510034 shutdown_connections 2024-04-24T06:49:42.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.588+0000 7f438de2c740 1 --2- 172.21.15.5:0/4116510034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56387cdd2b30 0x56387cdeaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.588+0000 7f438de2c740 1 --2- 172.21.15.5:0/4116510034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56387cdc8460 0x56387cdd2110 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.588+0000 7f438de2c740 1 --2- 172.21.15.5:0/4116510034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56387cdc7b40 0x56387cdde880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.588+0000 7f438de2c740 1 -- 172.21.15.5:0/4116510034 >> 172.21.15.5:0/4116510034 conn(0x56387cdc3e60 msgr2=0x56387cdcab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:42.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.588+0000 7f438de2c740 1 -- 172.21.15.5:0/4116510034 shutdown_connections 2024-04-24T06:49:42.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.588+0000 7f438de2c740 1 -- 172.21.15.5:0/4116510034 wait complete. 2024-04-24T06:49:42.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.589+0000 7f438de2c740 1 -- start start 2024-04-24T06:49:42.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.589+0000 7f438de2c740 1 Processor -- start 2024-04-24T06:49:42.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.589+0000 7f438de2c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56387cdc7b40 0x56387cdd36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.589+0000 7f438de2c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56387cdd2b30 0x56387cdd3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.589+0000 7f438de2c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56387cddb480 0x56387cdd4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.589+0000 7f438de2c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f43841ae420 con 0x56387cdc7b40 2024-04-24T06:49:42.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.589+0000 7f438de2c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f43841af890 con 0x56387cddb480 2024-04-24T06:49:42.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.589+0000 7f438de2c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f43841aeaf0 con 0x56387cdd2b30 2024-04-24T06:49:42.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.589+0000 7f438de2c740 10 client.0 _ll_register_callbacks cb 0x7f437c009980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:49:42.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.590+0000 7f4383e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56387cdc7b40 0x56387cdd36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.590+0000 7f4383e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56387cdc7b40 0x56387cdd36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38036/0 (socket says 172.21.15.5:38036) 2024-04-24T06:49:42.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.590+0000 7f4383e00640 1 -- 172.21.15.5:0/484423687 learned_addr learned my addr 172.21.15.5:0/484423687 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:42.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.590+0000 7f4389000640 1 --2- 172.21.15.5:0/484423687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56387cddb480 0x56387cdd4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.590+0000 7f4383400640 1 --2- 172.21.15.5:0/484423687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56387cdd2b30 0x56387cdd3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.590+0000 7f4383e00640 1 -- 172.21.15.5:0/484423687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56387cdd2b30 msgr2=0x56387cdd3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:42.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.590+0000 7f4383e00640 1 --2- 172.21.15.5:0/484423687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56387cdd2b30 0x56387cdd3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.590+0000 7f4383e00640 1 -- 172.21.15.5:0/484423687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56387cddb480 msgr2=0x56387cdd4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:42.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.590+0000 7f4383e00640 1 --2- 172.21.15.5:0/484423687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56387cddb480 0x56387cdd4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.590+0000 7f4383e00640 1 -- 172.21.15.5:0/484423687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56387cdd90c0 con 0x56387cdc7b40 2024-04-24T06:49:42.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.590+0000 7f4389000640 1 --2- 172.21.15.5:0/484423687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56387cddb480 0x56387cdd4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:49:42.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.590+0000 7f4383e00640 1 --2- 172.21.15.5:0/484423687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56387cdc7b40 0x56387cdd36a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f43740198f0 tx=0x7f4374005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=704dc98ec4a22bc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:42.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.590+0000 7f4372000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:49:42.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.591+0000 7f4372000640 1 -- 172.21.15.5:0/484423687 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4374005040 con 0x56387cdc7b40 2024-04-24T06:49:42.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.591+0000 7f438de2c740 1 -- 172.21.15.5:0/484423687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f436c036070 con 0x56387cdc7b40 2024-04-24T06:49:42.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.591+0000 7f4372000640 1 -- 172.21.15.5:0/484423687 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f4374016040 con 0x56387cdc7b40 2024-04-24T06:49:42.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.591+0000 7f4372000640 1 -- 172.21.15.5:0/484423687 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f43740051e0 con 0x56387cdc7b40 2024-04-24T06:49:42.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.591+0000 7f438de2c740 10 client.16371 fetch_fsmap learned FSMap version 27 2024-04-24T06:49:42.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.591+0000 7f438de2c740 1 -- 172.21.15.5:0/484423687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56387ce52050 con 0x56387cdc7b40 2024-04-24T06:49:42.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.591+0000 7f4372000640 1 -- 172.21.15.5:0/484423687 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f436c036070 con 0x56387cdc7b40 2024-04-24T06:49:42.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.591+0000 7f4372000640 1 -- 172.21.15.5:0/484423687 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f43740035f0 con 0x56387cdc7b40 2024-04-24T06:49:42.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.591+0000 7f438de2c740 10 client.16371 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:49:42.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.591+0000 7f438de2c740 10 client.16371 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T06:49:42.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.591+0000 7f438de2c740 20 client.16371 populate_metadata read hostname 'smithi005' 2024-04-24T06:49:42.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.591+0000 7f438de2c740 1 --2- 172.21.15.5:0/484423687 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x56387ce52590 0x56387ce54980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.591+0000 7f438de2c740 4 client.16371 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:49:42.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.591+0000 7f438de2c740 1 -- 172.21.15.5:0/484423687 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56387cd34a60 con 0x56387ce52590 2024-04-24T06:49:42.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.592+0000 7f4383400640 1 --2- 172.21.15.5:0/484423687 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x56387ce52590 0x56387ce54980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.592+0000 7f4383400640 1 --2- 172.21.15.5:0/484423687 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x56387ce52590 0x56387ce54980 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b8bae9408b6638d2 server_cookie=ed1d8f09aac0fa9a in_seq=0 out_seq=0 2024-04-24T06:49:42.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.592+0000 7f4372000640 10 client.16371 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:49:42.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.593+0000 7f4372000640 1 -- 172.21.15.5:0/484423687 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56387cd34a60 con 0x56387ce52590 2024-04-24T06:49:42.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.593+0000 7f4372000640 10 client.16371 handle_command_reply: tid=0 2024-04-24T06:49:42.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.593+0000 7f438de2c740 1 client.16371 shutdown 2024-04-24T06:49:42.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.593+0000 7f438de2c740 1 -- 172.21.15.5:0/484423687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56387cdc7b40 msgr2=0x56387cdd36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:42.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.593+0000 7f438de2c740 1 --2- 172.21.15.5:0/484423687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56387cdc7b40 0x56387cdd36a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f43740198f0 tx=0x7f4374005870 comp rx=0 tx=0).stop 2024-04-24T06:49:42.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.593+0000 7f438de2c740 1 -- 172.21.15.5:0/484423687 shutdown_connections 2024-04-24T06:49:42.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.593+0000 7f438de2c740 1 --2- 172.21.15.5:0/484423687 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x56387ce52590 0x56387ce54980 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.593+0000 7f438de2c740 1 --2- 172.21.15.5:0/484423687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56387cddb480 0x56387cdd4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.593+0000 7f438de2c740 1 --2- 172.21.15.5:0/484423687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56387cdd2b30 0x56387cdd3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.593+0000 7f438de2c740 1 --2- 172.21.15.5:0/484423687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56387cdc7b40 0x56387cdd36a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.593+0000 7f438de2c740 1 -- 172.21.15.5:0/484423687 >> 172.21.15.5:0/484423687 conn(0x56387cdc3e60 msgr2=0x56387cdc26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:42.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.593+0000 7f4372000640 0 client.16371 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:49:42.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.593+0000 7f438de2c740 1 -- 172.21.15.5:0/484423687 shutdown_connections 2024-04-24T06:49:42.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.593+0000 7f438de2c740 1 -- 172.21.15.5:0/484423687 wait complete. 2024-04-24T06:49:42.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.593+0000 7f438de2c740 20 client.16371 trim_cache size 0 max 16384 2024-04-24T06:49:42.602 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:49:42.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.594+0000 7f4389a00640 1 -- 172.21.15.5:0/2345642762 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f43680a3d20 msgr2=0x7f43680a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:42.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.594+0000 7f4389a00640 1 --2- 172.21.15.5:0/2345642762 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f43680a3d20 0x7f43680a61e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f437c009d30 tx=0x7f437c005f90 comp rx=0 tx=0).stop 2024-04-24T06:49:42.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.594+0000 7f4389a00640 1 -- 172.21.15.5:0/2345642762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4384109360 msgr2=0x7f43841a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:42.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.594+0000 7f4389a00640 1 --2- 172.21.15.5:0/2345642762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4384109360 0x7f43841a0bb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f436c000f80 tx=0x7f436c003f10 comp rx=0 tx=0).stop 2024-04-24T06:49:42.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.594+0000 7f4389a00640 1 -- 172.21.15.5:0/2345642762 shutdown_connections 2024-04-24T06:49:42.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.594+0000 7f4389a00640 1 --2- 172.21.15.5:0/2345642762 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f43680a3d20 0x7f43680a61e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.594+0000 7f4389a00640 1 --2- 172.21.15.5:0/2345642762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f438419a7e0 0x7f438419ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.594+0000 7f4389a00640 1 --2- 172.21.15.5:0/2345642762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4384109360 0x7f43841a0bb0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.594+0000 7f4389a00640 1 --2- 172.21.15.5:0/2345642762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4384104290 0x7f43841a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.594+0000 7f4389a00640 1 -- 172.21.15.5:0/2345642762 >> 172.21.15.5:0/2345642762 conn(0x7f43840ffaf0 msgr2=0x7f4384100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:42.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.594+0000 7f4389a00640 1 -- 172.21.15.5:0/2345642762 shutdown_connections 2024-04-24T06:49:42.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.594+0000 7f4389a00640 1 -- 172.21.15.5:0/2345642762 wait complete. 2024-04-24T06:49:42.611 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:42 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1150190994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:49:42.614 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T06:49:42.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:42 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/1150190994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:49:42.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:42 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/1150190994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:49:42.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.823+0000 7f78ffe00640 1 -- 172.21.15.5:0/1382622011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7900100dd0 msgr2=0x7f79001011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:42.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.823+0000 7f78ffe00640 1 --2- 172.21.15.5:0/1382622011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7900100dd0 0x7f79001011b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f78f0009c10 tx=0x7f78f002a770 comp rx=0 tx=0).stop 2024-04-24T06:49:42.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.823+0000 7f78ffe00640 1 -- 172.21.15.5:0/1382622011 shutdown_connections 2024-04-24T06:49:42.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.823+0000 7f78ffe00640 1 --2- 172.21.15.5:0/1382622011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79001061d0 0x7f7900106630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.823+0000 7f78ffe00640 1 --2- 172.21.15.5:0/1382622011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79001016f0 0x7f7900105bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.823+0000 7f78ffe00640 1 --2- 172.21.15.5:0/1382622011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7900100dd0 0x7f79001011b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.823+0000 7f78ffe00640 1 -- 172.21.15.5:0/1382622011 >> 172.21.15.5:0/1382622011 conn(0x7f7900077ab0 msgr2=0x7f79000fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:42.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.824+0000 7f78ffe00640 1 -- 172.21.15.5:0/1382622011 shutdown_connections 2024-04-24T06:49:42.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.824+0000 7f78ffe00640 1 -- 172.21.15.5:0/1382622011 wait complete. 2024-04-24T06:49:42.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.825+0000 7f78ffe00640 1 Processor -- start 2024-04-24T06:49:42.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.825+0000 7f78ffe00640 1 -- start start 2024-04-24T06:49:42.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.825+0000 7f78ffe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7900100dd0 0x7f790019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.826+0000 7f78ffe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79001016f0 0x7f790019f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.826+0000 7f78ffe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79001061d0 0x7f79001a3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.826+0000 7f78ffe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f79001187a0 con 0x7f79001061d0 2024-04-24T06:49:42.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.826+0000 7f78ffe00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7900118620 con 0x7f7900100dd0 2024-04-24T06:49:42.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.826+0000 7f78ffe00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7900118920 con 0x7f79001016f0 2024-04-24T06:49:42.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.826+0000 7f78ff400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79001061d0 0x7f79001a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.826+0000 7f78f6000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79001016f0 0x7f790019f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.826+0000 7f78fea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7900100dd0 0x7f790019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.826+0000 7f78f6000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79001016f0 0x7f790019f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:60880/0 (socket says 172.21.15.5:60880) 2024-04-24T06:49:42.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.826+0000 7f78ff400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79001061d0 0x7f79001a3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38056/0 (socket says 172.21.15.5:38056) 2024-04-24T06:49:42.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.826+0000 7f78f6000640 1 -- 172.21.15.5:0/1683500976 learned_addr learned my addr 172.21.15.5:0/1683500976 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:42.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.826+0000 7f78ff400640 1 -- 172.21.15.5:0/1683500976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79001016f0 msgr2=0x7f790019f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:42.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.826+0000 7f78ff400640 1 --2- 172.21.15.5:0/1683500976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79001016f0 0x7f790019f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.826+0000 7f78ff400640 1 -- 172.21.15.5:0/1683500976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7900100dd0 msgr2=0x7f790019ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:42.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.826+0000 7f78ff400640 1 --2- 172.21.15.5:0/1683500976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7900100dd0 0x7f790019ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.826+0000 7f78ff400640 1 -- 172.21.15.5:0/1683500976 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79001a3d70 con 0x7f79001061d0 2024-04-24T06:49:42.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.827+0000 7f78ff400640 1 --2- 172.21.15.5:0/1683500976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79001061d0 0x7f79001a3640 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f78f800e970 tx=0x7f78f800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:42.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.827+0000 7f78f7e00640 1 -- 172.21.15.5:0/1683500976 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f800ccc0 con 0x7f79001061d0 2024-04-24T06:49:42.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.827+0000 7f78ffe00640 1 -- 172.21.15.5:0/1683500976 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79001a4060 con 0x7f79001061d0 2024-04-24T06:49:42.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.827+0000 7f78f7e00640 1 -- 172.21.15.5:0/1683500976 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78f8005020 con 0x7f79001061d0 2024-04-24T06:49:42.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.827+0000 7f78f7e00640 1 -- 172.21.15.5:0/1683500976 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f8004a00 con 0x7f79001061d0 2024-04-24T06:49:42.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.827+0000 7f78ffe00640 1 -- 172.21.15.5:0/1683500976 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79001ab8f0 con 0x7f79001061d0 2024-04-24T06:49:42.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.831+0000 7f79089e3740 1 Processor -- start 2024-04-24T06:49:42.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.832+0000 7f78f7e00640 1 -- 172.21.15.5:0/1683500976 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f78f80051d0 con 0x7f79001061d0 2024-04-24T06:49:42.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.832+0000 7f79089e3740 1 -- start start 2024-04-24T06:49:42.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.832+0000 7f79089e3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d101c6b40 0x564d101c6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.832+0000 7f79089e3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d101c7460 0x564d101d15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.832+0000 7f79089e3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d101d1b30 0x564d101d3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.832+0000 7f79089e3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564d1012c1d0 con 0x564d101d1b30 2024-04-24T06:49:42.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.832+0000 7f79089e3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564d100d2450 con 0x564d101c6b40 2024-04-24T06:49:42.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.832+0000 7f79089e3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564d10128190 con 0x564d101c7460 2024-04-24T06:49:42.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.833+0000 7f78fea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d101c7460 0x564d101d15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.833+0000 7f78f6000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d101c6b40 0x564d101c6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.833+0000 7f78fea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d101c7460 0x564d101d15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:60886/0 (socket says 172.21.15.5:60886) 2024-04-24T06:49:42.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.833+0000 7f78fea00640 1 -- 172.21.15.5:0/2853940975 learned_addr learned my addr 172.21.15.5:0/2853940975 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:42.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.833+0000 7f78f6000640 1 --2- 172.21.15.5:0/2853940975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d101d1b30 0x564d101d3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.833+0000 7f78f6000640 1 -- 172.21.15.5:0/2853940975 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d101c7460 msgr2=0x564d101d15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:42.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.833+0000 7f78f6000640 1 --2- 172.21.15.5:0/2853940975 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d101c7460 0x564d101d15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.833+0000 7f78f6000640 1 -- 172.21.15.5:0/2853940975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d101d1b30 msgr2=0x564d101d3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:42.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.833+0000 7f78f7e00640 1 --2- 172.21.15.5:0/1683500976 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f78e40a3d20 0x7f78e40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.833+0000 7f78f6000640 1 --2- 172.21.15.5:0/2853940975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d101d1b30 0x564d101d3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.834+0000 7f78f6000640 1 -- 172.21.15.5:0/2853940975 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d1018f310 con 0x564d101c6b40 2024-04-24T06:49:42.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.834+0000 7f78fea00640 1 --2- 172.21.15.5:0/2853940975 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d101c7460 0x564d101d15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:49:42.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.834+0000 7f78f7e00640 1 -- 172.21.15.5:0/1683500976 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f78f80d0760 con 0x7f79001061d0 2024-04-24T06:49:42.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.834+0000 7f78ff400640 1 --2- 172.21.15.5:0/1683500976 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f78e40a3d20 0x7f78e40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.834+0000 7f78f6000640 1 --2- 172.21.15.5:0/2853940975 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d101c6b40 0x564d101c6f20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f78e8005c80 tx=0x7f78e8009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f205a24ad7b6fb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:42.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.834+0000 7f78efe00640 1 -- 172.21.15.5:0/2853940975 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e8005020 con 0x564d101c6b40 2024-04-24T06:49:42.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.834+0000 7f78efe00640 1 -- 172.21.15.5:0/2853940975 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78e80047a0 con 0x564d101c6b40 2024-04-24T06:49:42.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.834+0000 7f78efe00640 1 -- 172.21.15.5:0/2853940975 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e800fa20 con 0x564d101c6b40 2024-04-24T06:49:42.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.835+0000 7f79089e3740 1 -- 172.21.15.5:0/2853940975 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d101c6b40 msgr2=0x564d101c6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:42.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.835+0000 7f78ff400640 1 --2- 172.21.15.5:0/1683500976 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f78e40a3d20 0x7f78e40a61e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f78f801a040 tx=0x7f78f80046c0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:42.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.835+0000 7f79089e3740 1 --2- 172.21.15.5:0/2853940975 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d101c6b40 0x564d101c6f20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f78e8005c80 tx=0x7f78e8009680 comp rx=0 tx=0).stop 2024-04-24T06:49:42.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.835+0000 7f79089e3740 1 -- 172.21.15.5:0/2853940975 shutdown_connections 2024-04-24T06:49:42.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.835+0000 7f79089e3740 1 --2- 172.21.15.5:0/2853940975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d101d1b30 0x564d101d3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.835+0000 7f79089e3740 1 --2- 172.21.15.5:0/2853940975 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d101c7460 0x564d101d15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.835+0000 7f79089e3740 1 --2- 172.21.15.5:0/2853940975 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d101c6b40 0x564d101c6f20 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.835+0000 7f79089e3740 1 -- 172.21.15.5:0/2853940975 >> 172.21.15.5:0/2853940975 conn(0x564d101c2e60 msgr2=0x564d101c3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:42.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.835+0000 7f79089e3740 1 -- 172.21.15.5:0/2853940975 shutdown_connections 2024-04-24T06:49:42.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.835+0000 7f79089e3740 1 -- 172.21.15.5:0/2853940975 wait complete. 2024-04-24T06:49:42.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.836+0000 7f79089e3740 1 -- start start 2024-04-24T06:49:42.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.836+0000 7f79089e3740 1 Processor -- start 2024-04-24T06:49:42.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.836+0000 7f79089e3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d101c6b40 0x564d101d15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.837+0000 7f79089e3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d101c7460 0x564d101ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.837+0000 7f79089e3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d101d1b30 0x564d101cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.837+0000 7f79089e3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564d101d49c0 con 0x564d101c7460 2024-04-24T06:49:42.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.837+0000 7f79089e3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564d100e5810 con 0x564d101c6b40 2024-04-24T06:49:42.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.837+0000 7f79089e3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564d101d4b40 con 0x564d101d1b30 2024-04-24T06:49:42.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.837+0000 7f79089e3740 10 client.0 _ll_register_callbacks cb 0x564d100b3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:49:42.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.837+0000 7f78fea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d101c7460 0x564d101ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.837+0000 7f78fea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d101c7460 0x564d101ce8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38070/0 (socket says 172.21.15.5:38070) 2024-04-24T06:49:42.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.837+0000 7f78fea00640 1 -- 172.21.15.5:0/1875222250 learned_addr learned my addr 172.21.15.5:0/1875222250 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:42.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.837+0000 7f78f6000640 1 --2- 172.21.15.5:0/1875222250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d101d1b30 0x564d101cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.837+0000 7f78f6000640 1 --2- 172.21.15.5:0/1875222250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d101c6b40 0x564d101d15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.837+0000 7f78fea00640 1 -- 172.21.15.5:0/1875222250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d101d1b30 msgr2=0x564d101cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:42.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.837+0000 7f78fea00640 1 --2- 172.21.15.5:0/1875222250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d101d1b30 0x564d101cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.837+0000 7f78fea00640 1 -- 172.21.15.5:0/1875222250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d101c6b40 msgr2=0x564d101d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:42.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.837+0000 7f78fea00640 1 --2- 172.21.15.5:0/1875222250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d101c6b40 0x564d101d15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.837+0000 7f78fea00640 1 -- 172.21.15.5:0/1875222250 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d100f63f0 con 0x564d101c7460 2024-04-24T06:49:42.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.837+0000 7f78f6000640 1 --2- 172.21.15.5:0/1875222250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d101c6b40 0x564d101d15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:49:42.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.838+0000 7f78fea00640 1 --2- 172.21.15.5:0/1875222250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d101c7460 0x564d101ce8d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f78f0009b30 tx=0x7f78f0030c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c694962a6fce83f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:42.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.838+0000 7f78ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:49:42.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.838+0000 7f78ecc00640 1 -- 172.21.15.5:0/1875222250 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f0036040 con 0x564d101c7460 2024-04-24T06:49:42.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.838+0000 7f78ecc00640 1 -- 172.21.15.5:0/1875222250 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f78f00305a0 con 0x564d101c7460 2024-04-24T06:49:42.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.838+0000 7f79089e3740 1 -- 172.21.15.5:0/1875222250 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78e80047a0 con 0x564d101c7460 2024-04-24T06:49:42.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.838+0000 7f78ecc00640 1 -- 172.21.15.5:0/1875222250 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f78f0030e90 con 0x564d101c7460 2024-04-24T06:49:42.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.838+0000 7f79089e3740 10 client.16383 fetch_fsmap learned FSMap version 27 2024-04-24T06:49:42.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.838+0000 7f79089e3740 1 -- 172.21.15.5:0/1875222250 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564d0fe3e3f0 con 0x564d101c7460 2024-04-24T06:49:42.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.839+0000 7f78ecc00640 1 -- 172.21.15.5:0/1875222250 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78e80047a0 con 0x564d101c7460 2024-04-24T06:49:42.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.839+0000 7f78ecc00640 1 -- 172.21.15.5:0/1875222250 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f78f0004640 con 0x564d101c7460 2024-04-24T06:49:42.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.839+0000 7f79089e3740 10 client.16383 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:49:42.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.839+0000 7f79089e3740 10 client.16383 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:49:42.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.839+0000 7f79089e3740 20 client.16383 populate_metadata read hostname 'smithi005' 2024-04-24T06:49:42.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.839+0000 7f79089e3740 1 --2- 172.21.15.5:0/1875222250 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x564d101e1d60 0x564d101e4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.839+0000 7f79089e3740 4 client.16383 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:49:42.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.839+0000 7f79089e3740 1 -- 172.21.15.5:0/1875222250 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d10139bf0 con 0x564d101e1d60 2024-04-24T06:49:42.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.840+0000 7f78f6000640 1 --2- 172.21.15.5:0/1875222250 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x564d101e1d60 0x564d101e4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.840+0000 7f78f6000640 1 --2- 172.21.15.5:0/1875222250 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x564d101e1d60 0x564d101e4150 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f3cf610398c0127 server_cookie=fb4b9991d373ee1b in_seq=0 out_seq=0 2024-04-24T06:49:42.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.840+0000 7f78ecc00640 10 client.16383 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:49:42.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.841+0000 7f78ecc00640 1 -- 172.21.15.5:0/1875222250 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564d10139bf0 con 0x564d101e1d60 2024-04-24T06:49:42.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.841+0000 7f78ecc00640 10 client.16383 handle_command_reply: tid=0 2024-04-24T06:49:42.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.841+0000 7f79089e3740 1 client.16383 shutdown 2024-04-24T06:49:42.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.841+0000 7f79089e3740 1 -- 172.21.15.5:0/1875222250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d101c7460 msgr2=0x564d101ce8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:42.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.841+0000 7f79089e3740 1 --2- 172.21.15.5:0/1875222250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d101c7460 0x564d101ce8d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f78f0009b30 tx=0x7f78f0030c60 comp rx=0 tx=0).stop 2024-04-24T06:49:42.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.841+0000 7f79089e3740 1 -- 172.21.15.5:0/1875222250 shutdown_connections 2024-04-24T06:49:42.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.841+0000 7f79089e3740 1 --2- 172.21.15.5:0/1875222250 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x564d101e1d60 0x564d101e4150 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.841+0000 7f79089e3740 1 --2- 172.21.15.5:0/1875222250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d101d1b30 0x564d101cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.841+0000 7f79089e3740 1 --2- 172.21.15.5:0/1875222250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d101c7460 0x564d101ce8d0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.841+0000 7f78ecc00640 0 client.16383 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:49:42.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.841+0000 7f79089e3740 1 --2- 172.21.15.5:0/1875222250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d101c6b40 0x564d101d15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.841+0000 7f79089e3740 1 -- 172.21.15.5:0/1875222250 >> 172.21.15.5:0/1875222250 conn(0x564d101c2e60 msgr2=0x564d101c8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:42.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.841+0000 7f79089e3740 1 -- 172.21.15.5:0/1875222250 shutdown_connections 2024-04-24T06:49:42.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.841+0000 7f79089e3740 1 -- 172.21.15.5:0/1875222250 wait complete. 2024-04-24T06:49:42.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.842+0000 7f79089e3740 20 client.16383 trim_cache size 0 max 16384 2024-04-24T06:49:42.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.857+0000 7f79089e3740 1 Processor -- start 2024-04-24T06:49:42.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.857+0000 7f79089e3740 1 -- start start 2024-04-24T06:49:42.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.857+0000 7f79089e3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d101c6b40 0x564d101dd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.857+0000 7f79089e3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d101c7460 0x564d101c1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.857+0000 7f79089e3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d101d1b30 0x564d101d1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.857+0000 7f79089e3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f79001b6a60 con 0x564d101c7460 2024-04-24T06:49:42.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.857+0000 7f79089e3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f79001b4330 con 0x564d101d1b30 2024-04-24T06:49:42.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.857+0000 7f79089e3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f79001b2180 con 0x564d101c6b40 2024-04-24T06:49:42.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.858+0000 7f78fea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d101c7460 0x564d101c1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.858+0000 7f78fea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d101c7460 0x564d101c1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38084/0 (socket says 172.21.15.5:38084) 2024-04-24T06:49:42.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.858+0000 7f78fea00640 1 -- 172.21.15.5:0/186587564 learned_addr learned my addr 172.21.15.5:0/186587564 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.858+0000 7f78f6000640 1 --2- 172.21.15.5:0/186587564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d101c6b40 0x564d101dd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.858+0000 7f78f6000640 1 --2- 172.21.15.5:0/186587564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d101d1b30 0x564d101d1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.858+0000 7f78fea00640 1 -- 172.21.15.5:0/186587564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d101c6b40 msgr2=0x564d101dd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.858+0000 7f78fea00640 1 --2- 172.21.15.5:0/186587564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d101c6b40 0x564d101dd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.858+0000 7f78fea00640 1 -- 172.21.15.5:0/186587564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d101d1b30 msgr2=0x564d101d1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.858+0000 7f78fea00640 1 --2- 172.21.15.5:0/186587564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d101d1b30 0x564d101d1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.858+0000 7f78fea00640 1 -- 172.21.15.5:0/186587564 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d0ff087d0 con 0x564d101c7460 2024-04-24T06:49:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.858+0000 7f78fea00640 1 --2- 172.21.15.5:0/186587564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d101c7460 0x564d101c1dd0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f78f002ac90 tx=0x7f78f0047cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=550158dc42dde308 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:42.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.859+0000 7f78ef400640 1 -- 172.21.15.5:0/186587564 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f0054030 con 0x564d101c7460 2024-04-24T06:49:42.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.859+0000 7f78ef400640 1 -- 172.21.15.5:0/186587564 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78f0031440 con 0x564d101c7460 2024-04-24T06:49:42.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.859+0000 7f78ef400640 1 -- 172.21.15.5:0/186587564 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f0036040 con 0x564d101c7460 2024-04-24T06:49:42.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.859+0000 7f79089e3740 1 -- 172.21.15.5:0/186587564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d101c7460 msgr2=0x564d101c1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:42.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.859+0000 7f79089e3740 1 --2- 172.21.15.5:0/186587564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d101c7460 0x564d101c1dd0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f78f002ac90 tx=0x7f78f0047cf0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.859+0000 7f79089e3740 1 -- 172.21.15.5:0/186587564 shutdown_connections 2024-04-24T06:49:42.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.859+0000 7f79089e3740 1 --2- 172.21.15.5:0/186587564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d101d1b30 0x564d101d1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.859+0000 7f79089e3740 1 --2- 172.21.15.5:0/186587564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d101c7460 0x564d101c1dd0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.859+0000 7f79089e3740 1 --2- 172.21.15.5:0/186587564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d101c6b40 0x564d101dd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.859+0000 7f79089e3740 1 -- 172.21.15.5:0/186587564 >> 172.21.15.5:0/186587564 conn(0x564d101c2e60 msgr2=0x564d101c16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:42.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.859+0000 7f79089e3740 1 -- 172.21.15.5:0/186587564 shutdown_connections 2024-04-24T06:49:42.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.860+0000 7f79089e3740 1 -- 172.21.15.5:0/186587564 wait complete. 2024-04-24T06:49:42.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.860+0000 7f79089e3740 1 -- start start 2024-04-24T06:49:42.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.860+0000 7f79089e3740 1 Processor -- start 2024-04-24T06:49:42.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.861+0000 7f79089e3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d101c6b40 0x564d101cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.861+0000 7f79089e3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d101c7460 0x564d101d9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.861+0000 7f79089e3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d101d1b30 0x564d101d2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.861+0000 7f79089e3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f79001b27f0 con 0x564d101c7460 2024-04-24T06:49:42.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.861+0000 7f79089e3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f79001b3c60 con 0x564d101c6b40 2024-04-24T06:49:42.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.861+0000 7f79089e3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f79001b2ec0 con 0x564d101d1b30 2024-04-24T06:49:42.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.861+0000 7f79089e3740 10 client.0 _ll_register_callbacks cb 0x7f78f8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:49:42.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.861+0000 7f78f6000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d101c6b40 0x564d101cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.861+0000 7f78f6000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d101c6b40 0x564d101cc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:46184/0 (socket says 172.21.15.5:46184) 2024-04-24T06:49:42.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.861+0000 7f78f6000640 1 -- 172.21.15.5:0/1365326669 learned_addr learned my addr 172.21.15.5:0/1365326669 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:42.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.861+0000 7f78f6000640 1 --2- 172.21.15.5:0/1365326669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d101d1b30 0x564d101d2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.861+0000 7f78fea00640 1 --2- 172.21.15.5:0/1365326669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d101c7460 0x564d101d9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.861+0000 7f78f6000640 1 -- 172.21.15.5:0/1365326669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d101d1b30 msgr2=0x564d101d2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:42.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.861+0000 7f78f6000640 1 --2- 172.21.15.5:0/1365326669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d101d1b30 0x564d101d2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.861+0000 7f78f6000640 1 -- 172.21.15.5:0/1365326669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d101c7460 msgr2=0x564d101d9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:42.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.861+0000 7f78f6000640 1 --2- 172.21.15.5:0/1365326669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d101c7460 0x564d101d9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.861+0000 7f78f6000640 1 -- 172.21.15.5:0/1365326669 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d101d3290 con 0x564d101c6b40 2024-04-24T06:49:42.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.862+0000 7f78f6000640 1 --2- 172.21.15.5:0/1365326669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d101c6b40 0x564d101cc5d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f78e8009340 tx=0x7f78f0000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b243f0b44bd2b5b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:42.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.862+0000 7f78ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:49:42.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.862+0000 7f78ecc00640 1 -- 172.21.15.5:0/1365326669 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e802f810 con 0x564d101c6b40 2024-04-24T06:49:42.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.862+0000 7f78ecc00640 1 -- 172.21.15.5:0/1365326669 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f78e8005000 con 0x564d101c6b40 2024-04-24T06:49:42.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.862+0000 7f79089e3740 1 -- 172.21.15.5:0/1365326669 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78f0031440 con 0x564d101c6b40 2024-04-24T06:49:42.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.862+0000 7f78ecc00640 1 -- 172.21.15.5:0/1365326669 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f78e8005200 con 0x564d101c6b40 2024-04-24T06:49:42.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.862+0000 7f79089e3740 10 client.26525 fetch_fsmap learned FSMap version 27 2024-04-24T06:49:42.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.862+0000 7f79089e3740 1 -- 172.21.15.5:0/1365326669 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564d102489d0 con 0x564d101c6b40 2024-04-24T06:49:42.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.863+0000 7f78ecc00640 1 -- 172.21.15.5:0/1365326669 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78f0031440 con 0x564d101c6b40 2024-04-24T06:49:42.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.863+0000 7f78ecc00640 1 -- 172.21.15.5:0/1365326669 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f78e8039020 con 0x564d101c6b40 2024-04-24T06:49:42.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.863+0000 7f79089e3740 10 client.26525 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:49:42.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.863+0000 7f79089e3740 10 client.26525 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:49:42.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.863+0000 7f79089e3740 20 client.26525 populate_metadata read hostname 'smithi005' 2024-04-24T06:49:42.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.863+0000 7f79089e3740 1 --2- 172.21.15.5:0/1365326669 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x564d10248f40 0x564d1024b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:42.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.863+0000 7f79089e3740 4 client.26525 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:49:42.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.863+0000 7f79089e3740 1 -- 172.21.15.5:0/1365326669 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564d10133a60 con 0x564d10248f40 2024-04-24T06:49:42.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.863+0000 7f78fea00640 1 --2- 172.21.15.5:0/1365326669 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x564d10248f40 0x564d1024b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:42.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.864+0000 7f78fea00640 1 --2- 172.21.15.5:0/1365326669 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x564d10248f40 0x564d1024b360 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc8ebf99ad52512f server_cookie=e774c804aa1fde5a in_seq=0 out_seq=0 2024-04-24T06:49:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.864+0000 7f78ecc00640 10 client.26525 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:49:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.864+0000 7f78ecc00640 1 -- 172.21.15.5:0/1365326669 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564d10133a60 con 0x564d10248f40 2024-04-24T06:49:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.864+0000 7f78ecc00640 10 client.26525 handle_command_reply: tid=0 2024-04-24T06:49:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.864+0000 7f79089e3740 1 client.26525 shutdown 2024-04-24T06:49:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.864+0000 7f79089e3740 1 -- 172.21.15.5:0/1365326669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d101c6b40 msgr2=0x564d101cc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.864+0000 7f79089e3740 1 --2- 172.21.15.5:0/1365326669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d101c6b40 0x564d101cc5d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f78e8009340 tx=0x7f78f0000f40 comp rx=0 tx=0).stop 2024-04-24T06:49:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.864+0000 7f79089e3740 1 -- 172.21.15.5:0/1365326669 shutdown_connections 2024-04-24T06:49:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.864+0000 7f79089e3740 1 --2- 172.21.15.5:0/1365326669 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x564d10248f40 0x564d1024b360 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.864+0000 7f79089e3740 1 --2- 172.21.15.5:0/1365326669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564d101d1b30 0x564d101d2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.864+0000 7f79089e3740 1 --2- 172.21.15.5:0/1365326669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d101c7460 0x564d101d9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.864+0000 7f79089e3740 1 --2- 172.21.15.5:0/1365326669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d101c6b40 0x564d101cc5d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.864+0000 7f79089e3740 1 -- 172.21.15.5:0/1365326669 >> 172.21.15.5:0/1365326669 conn(0x564d101c2e60 msgr2=0x564d101c1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.864+0000 7f78ecc00640 0 client.26525 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:49:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.865+0000 7f79089e3740 1 -- 172.21.15.5:0/1365326669 shutdown_connections 2024-04-24T06:49:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.865+0000 7f79089e3740 1 -- 172.21.15.5:0/1365326669 wait complete. 2024-04-24T06:49:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.865+0000 7f79089e3740 20 client.26525 trim_cache size 0 max 16384 2024-04-24T06:49:42.874 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:49:42.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.865+0000 7f78ffe00640 1 -- 172.21.15.5:0/1683500976 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f78e40a3d20 msgr2=0x7f78e40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:42.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.865+0000 7f78ffe00640 1 --2- 172.21.15.5:0/1683500976 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f78e40a3d20 0x7f78e40a61e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f78f801a040 tx=0x7f78f80046c0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.865+0000 7f78ffe00640 1 -- 172.21.15.5:0/1683500976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79001061d0 msgr2=0x7f79001a3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:42.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.865+0000 7f78ffe00640 1 --2- 172.21.15.5:0/1683500976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79001061d0 0x7f79001a3640 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f78f800e970 tx=0x7f78f800ee30 comp rx=0 tx=0).stop 2024-04-24T06:49:42.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.865+0000 7f78ffe00640 1 -- 172.21.15.5:0/1683500976 shutdown_connections 2024-04-24T06:49:42.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.865+0000 7f78ffe00640 1 --2- 172.21.15.5:0/1683500976 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f78e40a3d20 0x7f78e40a61e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.865+0000 7f78ffe00640 1 --2- 172.21.15.5:0/1683500976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79001061d0 0x7f79001a3640 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.865+0000 7f78ffe00640 1 --2- 172.21.15.5:0/1683500976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79001016f0 0x7f790019f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.865+0000 7f78ffe00640 1 --2- 172.21.15.5:0/1683500976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7900100dd0 0x7f790019ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:42.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.865+0000 7f78ffe00640 1 -- 172.21.15.5:0/1683500976 >> 172.21.15.5:0/1683500976 conn(0x7f7900077ab0 msgr2=0x7f7900103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:42.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.866+0000 7f78ffe00640 1 -- 172.21.15.5:0/1683500976 shutdown_connections 2024-04-24T06:49:42.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:42.866+0000 7f78ffe00640 1 -- 172.21.15.5:0/1683500976 wait complete. 2024-04-24T06:49:43.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:43 smithi137 ceph-mon[33017]: pgmap v173: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:49:43.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:43 smithi005 ceph-mon[27798]: pgmap v173: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:49:43.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:43 smithi037 ceph-mon[32864]: pgmap v173: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:49:43.885 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T06:49:43.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:49:43.886 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T06:49:44.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.116+0000 7fcf9b600640 1 -- 172.21.15.5:0/3245913103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf94104450 msgr2=0x7fcf94104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:44.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.116+0000 7fcf9b600640 1 --2- 172.21.15.5:0/3245913103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf94104450 0x7fcf94104830 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c009860 tx=0x7fcf7c02a340 comp rx=0 tx=0).stop 2024-04-24T06:49:44.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.116+0000 7fcf9b600640 1 -- 172.21.15.5:0/3245913103 shutdown_connections 2024-04-24T06:49:44.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.117+0000 7fcf9b600640 1 --2- 172.21.15.5:0/3245913103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf94109520 0x7fcf941110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.117+0000 7fcf9b600640 1 --2- 172.21.15.5:0/3245913103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf94104e00 0x7fcf94108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.117+0000 7fcf9b600640 1 --2- 172.21.15.5:0/3245913103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf94104450 0x7fcf94104830 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.117+0000 7fcf9b600640 1 -- 172.21.15.5:0/3245913103 >> 172.21.15.5:0/3245913103 conn(0x7fcf940ffc90 msgr2=0x7fcf941020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:44.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.117+0000 7fcf9b600640 1 -- 172.21.15.5:0/3245913103 shutdown_connections 2024-04-24T06:49:44.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.117+0000 7fcf9b600640 1 -- 172.21.15.5:0/3245913103 wait complete. 2024-04-24T06:49:44.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.118+0000 7fcf9b600640 1 Processor -- start 2024-04-24T06:49:44.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.118+0000 7fcf9b600640 1 -- start start 2024-04-24T06:49:44.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.118+0000 7fcf9b600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf94104450 0x7fcf941a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:44.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.118+0000 7fcf9b600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf94109520 0x7fcf941a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:44.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.118+0000 7fcf9b600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf9419a9c0 0x7fcf9419ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:44.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.118+0000 7fcf9b600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcf941145e0 con 0x7fcf94109520 2024-04-24T06:49:44.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.118+0000 7fcf9b600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcf94114460 con 0x7fcf9419a9c0 2024-04-24T06:49:44.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.118+0000 7fcf9b600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcf94114760 con 0x7fcf94104450 2024-04-24T06:49:44.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.119+0000 7fcf93e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf94109520 0x7fcf941a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:44.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.119+0000 7fcf93e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf94109520 0x7fcf941a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47268/0 (socket says 172.21.15.5:47268) 2024-04-24T06:49:44.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.119+0000 7fcf93e00640 1 -- 172.21.15.5:0/1450228442 learned_addr learned my addr 172.21.15.5:0/1450228442 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:44.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.119+0000 7fcf99600640 1 --2- 172.21.15.5:0/1450228442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf9419a9c0 0x7fcf9419ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:44.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.119+0000 7fcf98c00640 1 --2- 172.21.15.5:0/1450228442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf94104450 0x7fcf941a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:44.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.119+0000 7fcf93e00640 1 -- 172.21.15.5:0/1450228442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf94104450 msgr2=0x7fcf941a0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:44.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.119+0000 7fcf93e00640 1 --2- 172.21.15.5:0/1450228442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf94104450 0x7fcf941a0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.119+0000 7fcf93e00640 1 -- 172.21.15.5:0/1450228442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf9419a9c0 msgr2=0x7fcf9419ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:44.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.119+0000 7fcf93e00640 1 --2- 172.21.15.5:0/1450228442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf9419a9c0 0x7fcf9419ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.119+0000 7fcf93e00640 1 -- 172.21.15.5:0/1450228442 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf9419b4e0 con 0x7fcf94109520 2024-04-24T06:49:44.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.119+0000 7fcf93e00640 1 --2- 172.21.15.5:0/1450228442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf94109520 0x7fcf941a0d50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fcf8400c3a0 tx=0x7fcf8400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:44.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.120+0000 7fcf91600640 1 -- 172.21.15.5:0/1450228442 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8401b070 con 0x7fcf94109520 2024-04-24T06:49:44.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.120+0000 7fcf9b600640 1 -- 172.21.15.5:0/1450228442 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf9419b7d0 con 0x7fcf94109520 2024-04-24T06:49:44.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.120+0000 7fcf91600640 1 -- 172.21.15.5:0/1450228442 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf8400bb10 con 0x7fcf94109520 2024-04-24T06:49:44.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.120+0000 7fcf91600640 1 -- 172.21.15.5:0/1450228442 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf84016c20 con 0x7fcf94109520 2024-04-24T06:49:44.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.120+0000 7fcf9b600640 1 -- 172.21.15.5:0/1450228442 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf941a7a20 con 0x7fcf94109520 2024-04-24T06:49:44.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.123+0000 7fcf9e454740 1 Processor -- start 2024-04-24T06:49:44.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.123+0000 7fcf91600640 1 -- 172.21.15.5:0/1450228442 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fcf8401e070 con 0x7fcf94109520 2024-04-24T06:49:44.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.123+0000 7fcf9e454740 1 -- start start 2024-04-24T06:49:44.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.123+0000 7fcf9e454740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558090dcf090 0x558090dcf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:44.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.123+0000 7fcf9e454740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558090dcf9b0 0x558090dd9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:44.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.124+0000 7fcf9e454740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558090dda080 0x558090ddc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:44.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.124+0000 7fcf9e454740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558090d34e70 con 0x558090dcf9b0 2024-04-24T06:49:44.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.124+0000 7fcf9e454740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558090cdb380 con 0x558090dda080 2024-04-24T06:49:44.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.124+0000 7fcf9e454740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558090d30e30 con 0x558090dcf090 2024-04-24T06:49:44.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.124+0000 7fcf98c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558090dcf9b0 0x558090dd9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:44.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.124+0000 7fcf99600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558090dcf090 0x558090dcf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:44.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.124+0000 7fcf99600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558090dcf090 0x558090dcf470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:52590/0 (socket says 172.21.15.5:52590) 2024-04-24T06:49:44.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.124+0000 7fcf99600640 1 -- 172.21.15.5:0/2246871476 learned_addr learned my addr 172.21.15.5:0/2246871476 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:44.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.124+0000 7fcf93e00640 1 --2- 172.21.15.5:0/2246871476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558090dda080 0x558090ddc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:44.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.124+0000 7fcf99600640 1 -- 172.21.15.5:0/2246871476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558090dda080 msgr2=0x558090ddc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:44.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.124+0000 7fcf99600640 1 --2- 172.21.15.5:0/2246871476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558090dda080 0x558090ddc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.124+0000 7fcf99600640 1 -- 172.21.15.5:0/2246871476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558090dcf9b0 msgr2=0x558090dd9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:44.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.124+0000 7fcf99600640 1 --2- 172.21.15.5:0/2246871476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558090dcf9b0 0x558090dd9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.124+0000 7fcf99600640 1 -- 172.21.15.5:0/2246871476 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558090b0f7e0 con 0x558090dcf090 2024-04-24T06:49:44.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.124+0000 7fcf98c00640 1 --2- 172.21.15.5:0/2246871476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558090dcf9b0 0x558090dd9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:49:44.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.125+0000 7fcf99600640 1 --2- 172.21.15.5:0/2246871476 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558090dcf090 0x558090dcf470 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c009c20 tx=0x7fcf8c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=566efbe3c6fbdb2a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:44.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.125+0000 7fcf89c00640 1 -- 172.21.15.5:0/2246871476 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c01e070 con 0x558090dcf090 2024-04-24T06:49:44.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.125+0000 7fcf89c00640 1 -- 172.21.15.5:0/2246871476 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf8c019b10 con 0x558090dcf090 2024-04-24T06:49:44.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.125+0000 7fcf89c00640 1 -- 172.21.15.5:0/2246871476 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c00e9e0 con 0x558090dcf090 2024-04-24T06:49:44.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.125+0000 7fcf91600640 1 --2- 172.21.15.5:0/1450228442 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fcf780a3d20 0x7fcf780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:44.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.125+0000 7fcf91600640 1 -- 172.21.15.5:0/1450228442 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fcf840d0840 con 0x7fcf94109520 2024-04-24T06:49:44.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.125+0000 7fcf98c00640 1 --2- 172.21.15.5:0/1450228442 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fcf780a3d20 0x7fcf780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:44.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.126+0000 7fcf9e454740 1 -- 172.21.15.5:0/2246871476 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558090dcf090 msgr2=0x558090dcf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:44.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.126+0000 7fcf9e454740 1 --2- 172.21.15.5:0/2246871476 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558090dcf090 0x558090dcf470 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c009c20 tx=0x7fcf8c011810 comp rx=0 tx=0).stop 2024-04-24T06:49:44.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.126+0000 7fcf9e454740 1 -- 172.21.15.5:0/2246871476 shutdown_connections 2024-04-24T06:49:44.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.126+0000 7fcf9e454740 1 --2- 172.21.15.5:0/2246871476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558090dda080 0x558090ddc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.126+0000 7fcf9e454740 1 --2- 172.21.15.5:0/2246871476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558090dcf9b0 0x558090dd9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.126+0000 7fcf9e454740 1 --2- 172.21.15.5:0/2246871476 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558090dcf090 0x558090dcf470 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.126+0000 7fcf9e454740 1 -- 172.21.15.5:0/2246871476 >> 172.21.15.5:0/2246871476 conn(0x558090dcb3b0 msgr2=0x558090dcb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:44.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.126+0000 7fcf9e454740 1 -- 172.21.15.5:0/2246871476 shutdown_connections 2024-04-24T06:49:44.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.126+0000 7fcf9e454740 1 -- 172.21.15.5:0/2246871476 wait complete. 2024-04-24T06:49:44.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.126+0000 7fcf98c00640 1 --2- 172.21.15.5:0/1450228442 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fcf780a3d20 0x7fcf780a61e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c009830 tx=0x7fcf7c0056d0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:44.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.127+0000 7fcf9e454740 1 -- start start 2024-04-24T06:49:44.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.127+0000 7fcf9e454740 1 Processor -- start 2024-04-24T06:49:44.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.127+0000 7fcf9e454740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558090dcf090 0x558090dd7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:44.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.127+0000 7fcf9e454740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558090dcf9b0 0x558090dd7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:44.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.127+0000 7fcf9e454740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558090dda080 0x558090dd7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:44.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.128+0000 7fcf9e454740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558090ddce70 con 0x558090dcf9b0 2024-04-24T06:49:44.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.128+0000 7fcf9e454740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558090cee660 con 0x558090dda080 2024-04-24T06:49:44.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.128+0000 7fcf9e454740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558090ddcff0 con 0x558090dcf090 2024-04-24T06:49:44.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.128+0000 7fcf9e454740 10 client.0 _ll_register_callbacks cb 0x558090cdcaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:49:44.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.128+0000 7fcf93e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558090dcf9b0 0x558090dd7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:44.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.128+0000 7fcf98c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558090dcf090 0x558090dd7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:44.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.128+0000 7fcf98c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558090dcf090 0x558090dd7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:52604/0 (socket says 172.21.15.5:52604) 2024-04-24T06:49:44.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.128+0000 7fcf98c00640 1 -- 172.21.15.5:0/4242000386 learned_addr learned my addr 172.21.15.5:0/4242000386 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:44.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.128+0000 7fcf93e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558090dcf9b0 0x558090dd7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47284/0 (socket says 172.21.15.5:47284) 2024-04-24T06:49:44.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.128+0000 7fcf99600640 1 --2- 172.21.15.5:0/4242000386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558090dda080 0x558090dd7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:44.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.128+0000 7fcf93e00640 1 -- 172.21.15.5:0/4242000386 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558090dcf090 msgr2=0x558090dd7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:44.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.128+0000 7fcf93e00640 1 --2- 172.21.15.5:0/4242000386 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558090dcf090 0x558090dd7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.128+0000 7fcf93e00640 1 -- 172.21.15.5:0/4242000386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558090dda080 msgr2=0x558090dd7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:44.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.128+0000 7fcf93e00640 1 --2- 172.21.15.5:0/4242000386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558090dda080 0x558090dd7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.128+0000 7fcf93e00640 1 -- 172.21.15.5:0/4242000386 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558090b6ff70 con 0x558090dcf9b0 2024-04-24T06:49:44.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.128+0000 7fcf99600640 1 --2- 172.21.15.5:0/4242000386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558090dda080 0x558090dd7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:49:44.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.129+0000 7fcf93e00640 1 --2- 172.21.15.5:0/4242000386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558090dcf9b0 0x558090dd7550 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fcf840cdf10 tx=0x7fcf840984e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c98e736e29fb9e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:44.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.129+0000 7fcf82000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:49:44.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.129+0000 7fcf82000640 1 -- 172.21.15.5:0/4242000386 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8409c380 con 0x558090dcf9b0 2024-04-24T06:49:44.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.129+0000 7fcf82000640 1 -- 172.21.15.5:0/4242000386 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fcf840870f0 con 0x558090dcf9b0 2024-04-24T06:49:44.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.129+0000 7fcf9e454740 1 -- 172.21.15.5:0/4242000386 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf8c019b10 con 0x558090dcf9b0 2024-04-24T06:49:44.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.129+0000 7fcf82000640 1 -- 172.21.15.5:0/4242000386 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcf8409c1a0 con 0x558090dcf9b0 2024-04-24T06:49:44.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.129+0000 7fcf9e454740 10 client.16401 fetch_fsmap learned FSMap version 27 2024-04-24T06:49:44.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.129+0000 7fcf9e454740 1 -- 172.21.15.5:0/4242000386 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558090a07170 con 0x558090dcf9b0 2024-04-24T06:49:44.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.129+0000 7fcf82000640 1 -- 172.21.15.5:0/4242000386 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf8c019b10 con 0x558090dcf9b0 2024-04-24T06:49:44.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.130+0000 7fcf82000640 1 -- 172.21.15.5:0/4242000386 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7fcf8400b040 con 0x558090dcf9b0 2024-04-24T06:49:44.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.130+0000 7fcf9e454740 10 client.16401 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:49:44.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.130+0000 7fcf9e454740 10 client.16401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:49:44.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.130+0000 7fcf9e454740 20 client.16401 populate_metadata read hostname 'smithi005' 2024-04-24T06:49:44.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.130+0000 7fcf9e454740 1 --2- 172.21.15.5:0/4242000386 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x558090dea480 0x558090dec890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:44.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.130+0000 7fcf9e454740 4 client.16401 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:49:44.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.130+0000 7fcf9e454740 1 -- 172.21.15.5:0/4242000386 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558090d42890 con 0x558090dea480 2024-04-24T06:49:44.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.130+0000 7fcf98c00640 1 --2- 172.21.15.5:0/4242000386 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x558090dea480 0x558090dec890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:44.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.131+0000 7fcf98c00640 1 --2- 172.21.15.5:0/4242000386 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x558090dea480 0x558090dec890 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbcfa8494d27b3b5 server_cookie=566e000345cf2ac4 in_seq=0 out_seq=0 2024-04-24T06:49:44.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.131+0000 7fcf82000640 10 client.16401 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:49:44.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.132+0000 7fcf82000640 1 -- 172.21.15.5:0/4242000386 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558090d42890 con 0x558090dea480 2024-04-24T06:49:44.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.132+0000 7fcf82000640 10 client.16401 handle_command_reply: tid=0 2024-04-24T06:49:44.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.132+0000 7fcf9e454740 1 client.16401 shutdown 2024-04-24T06:49:44.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.132+0000 7fcf9e454740 1 -- 172.21.15.5:0/4242000386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558090dcf9b0 msgr2=0x558090dd7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:44.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.132+0000 7fcf9e454740 1 --2- 172.21.15.5:0/4242000386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558090dcf9b0 0x558090dd7550 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fcf840cdf10 tx=0x7fcf840984e0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.132+0000 7fcf9e454740 1 -- 172.21.15.5:0/4242000386 shutdown_connections 2024-04-24T06:49:44.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.132+0000 7fcf9e454740 1 --2- 172.21.15.5:0/4242000386 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x558090dea480 0x558090dec890 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.132+0000 7fcf9e454740 1 --2- 172.21.15.5:0/4242000386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558090dda080 0x558090dd7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.132+0000 7fcf9e454740 1 --2- 172.21.15.5:0/4242000386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558090dcf9b0 0x558090dd7550 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.132+0000 7fcf9e454740 1 --2- 172.21.15.5:0/4242000386 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558090dcf090 0x558090dd7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.132+0000 7fcf82000640 0 client.16401 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:49:44.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.132+0000 7fcf9e454740 1 -- 172.21.15.5:0/4242000386 >> 172.21.15.5:0/4242000386 conn(0x558090dcb3b0 msgr2=0x558090dd10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:44.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.133+0000 7fcf9e454740 1 -- 172.21.15.5:0/4242000386 shutdown_connections 2024-04-24T06:49:44.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.133+0000 7fcf9e454740 1 -- 172.21.15.5:0/4242000386 wait complete. 2024-04-24T06:49:44.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.133+0000 7fcf9e454740 20 client.16401 trim_cache size 0 max 16384 2024-04-24T06:49:44.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.153+0000 7fcf9e454740 1 Processor -- start 2024-04-24T06:49:44.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.153+0000 7fcf9e454740 1 -- start start 2024-04-24T06:49:44.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.153+0000 7fcf9e454740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558090dcf090 0x558090de5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:44.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.153+0000 7fcf9e454740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558090dcf9b0 0x558090dd5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:44.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.153+0000 7fcf9e454740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558090dda080 0x558090df2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:44.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.153+0000 7fcf9e454740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcf941b27e0 con 0x558090dcf090 2024-04-24T06:49:44.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.153+0000 7fcf9e454740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcf941b00b0 con 0x558090dda080 2024-04-24T06:49:44.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.153+0000 7fcf9e454740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcf941adf00 con 0x558090dcf9b0 2024-04-24T06:49:44.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.154+0000 7fcf98c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558090dcf090 0x558090de5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:44.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.154+0000 7fcf98c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558090dcf090 0x558090de5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47298/0 (socket says 172.21.15.5:47298) 2024-04-24T06:49:44.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.154+0000 7fcf98c00640 1 -- 172.21.15.5:0/2369376218 learned_addr learned my addr 172.21.15.5:0/2369376218 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:44.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.154+0000 7fcf99600640 1 --2- 172.21.15.5:0/2369376218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558090dda080 0x558090df2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:44.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.154+0000 7fcf93e00640 1 --2- 172.21.15.5:0/2369376218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558090dcf9b0 0x558090dd5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:44.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.154+0000 7fcf98c00640 1 -- 172.21.15.5:0/2369376218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558090dcf9b0 msgr2=0x558090dd5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:44.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.154+0000 7fcf98c00640 1 --2- 172.21.15.5:0/2369376218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558090dcf9b0 0x558090dd5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.154+0000 7fcf98c00640 1 -- 172.21.15.5:0/2369376218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558090dda080 msgr2=0x558090df2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:44.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.154+0000 7fcf98c00640 1 --2- 172.21.15.5:0/2369376218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558090dda080 0x558090df2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.154+0000 7fcf98c00640 1 -- 172.21.15.5:0/2369376218 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558090dcf7e0 con 0x558090dcf090 2024-04-24T06:49:44.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.154+0000 7fcf99600640 1 --2- 172.21.15.5:0/2369376218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558090dda080 0x558090df2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:49:44.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.154+0000 7fcf98c00640 1 --2- 172.21.15.5:0/2369376218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558090dcf090 0x558090de5d60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c02a860 tx=0x7fcf7c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4786889404c886bf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:44.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.155+0000 7fcf89200640 1 -- 172.21.15.5:0/2369376218 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c037040 con 0x558090dcf090 2024-04-24T06:49:44.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.155+0000 7fcf89200640 1 -- 172.21.15.5:0/2369376218 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf7c053070 con 0x558090dcf090 2024-04-24T06:49:44.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.155+0000 7fcf89200640 1 -- 172.21.15.5:0/2369376218 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c003a70 con 0x558090dcf090 2024-04-24T06:49:44.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.156+0000 7fcf9e454740 1 -- 172.21.15.5:0/2369376218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558090dcf090 msgr2=0x558090de5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:44.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.156+0000 7fcf9e454740 1 --2- 172.21.15.5:0/2369376218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558090dcf090 0x558090de5d60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c02a860 tx=0x7fcf7c005f90 comp rx=0 tx=0).stop 2024-04-24T06:49:44.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.156+0000 7fcf9e454740 1 -- 172.21.15.5:0/2369376218 shutdown_connections 2024-04-24T06:49:44.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.156+0000 7fcf9e454740 1 --2- 172.21.15.5:0/2369376218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558090dda080 0x558090df2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.156+0000 7fcf9e454740 1 --2- 172.21.15.5:0/2369376218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558090dcf9b0 0x558090dd5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.156+0000 7fcf9e454740 1 --2- 172.21.15.5:0/2369376218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558090dcf090 0x558090de5d60 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.156+0000 7fcf9e454740 1 -- 172.21.15.5:0/2369376218 >> 172.21.15.5:0/2369376218 conn(0x558090dcb3b0 msgr2=0x558090dd90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:44.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.156+0000 7fcf9e454740 1 -- 172.21.15.5:0/2369376218 shutdown_connections 2024-04-24T06:49:44.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.156+0000 7fcf9e454740 1 -- 172.21.15.5:0/2369376218 wait complete. 2024-04-24T06:49:44.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.156+0000 7fcf9e454740 1 -- start start 2024-04-24T06:49:44.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.157+0000 7fcf9e454740 1 Processor -- start 2024-04-24T06:49:44.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.157+0000 7fcf9e454740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558090dcf9b0 0x558090de2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:44.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.157+0000 7fcf9e454740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558090dda080 0x558090ddf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:44.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.157+0000 7fcf9e454740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558090ddf780 0x558090ddfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:44.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.157+0000 7fcf9e454740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcf941ae570 con 0x558090ddf780 2024-04-24T06:49:44.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.157+0000 7fcf9e454740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcf941af9e0 con 0x558090dcf9b0 2024-04-24T06:49:44.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.157+0000 7fcf9e454740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcf941aec40 con 0x558090dda080 2024-04-24T06:49:44.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.157+0000 7fcf9e454740 10 client.0 _ll_register_callbacks cb 0x7fcf840d6220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:49:44.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.157+0000 7fcf98c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558090dcf9b0 0x558090de2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:44.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.157+0000 7fcf99600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558090ddf780 0x558090ddfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:44.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.157+0000 7fcf93e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558090dda080 0x558090ddf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:44.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.157+0000 7fcf98c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558090dcf9b0 0x558090de2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:45494/0 (socket says 172.21.15.5:45494) 2024-04-24T06:49:44.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.157+0000 7fcf98c00640 1 -- 172.21.15.5:0/3250118011 learned_addr learned my addr 172.21.15.5:0/3250118011 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:44.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.158+0000 7fcf98c00640 1 -- 172.21.15.5:0/3250118011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558090dda080 msgr2=0x558090ddf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:44.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.158+0000 7fcf98c00640 1 --2- 172.21.15.5:0/3250118011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558090dda080 0x558090ddf240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.158+0000 7fcf98c00640 1 -- 172.21.15.5:0/3250118011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558090ddf780 msgr2=0x558090ddfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:44.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.158+0000 7fcf98c00640 1 --2- 172.21.15.5:0/3250118011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558090ddf780 0x558090ddfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.158+0000 7fcf98c00640 1 -- 172.21.15.5:0/3250118011 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558090e591f0 con 0x558090dcf9b0 2024-04-24T06:49:44.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.158+0000 7fcf93e00640 1 --2- 172.21.15.5:0/3250118011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558090dda080 0x558090ddf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:49:44.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.158+0000 7fcf98c00640 1 --2- 172.21.15.5:0/3250118011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558090dcf9b0 0x558090de2540 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c009860 tx=0x7fcf7c0035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c97b39b64c4e99e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:44.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.158+0000 7fcf82000640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:49:44.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.158+0000 7fcf82000640 1 -- 172.21.15.5:0/3250118011 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c037040 con 0x558090dcf9b0 2024-04-24T06:49:44.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.158+0000 7fcf9e454740 1 -- 172.21.15.5:0/3250118011 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf7c053070 con 0x558090dcf9b0 2024-04-24T06:49:44.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.158+0000 7fcf82000640 1 -- 172.21.15.5:0/3250118011 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fcf7c003810 con 0x558090dcf9b0 2024-04-24T06:49:44.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.159+0000 7fcf82000640 1 -- 172.21.15.5:0/3250118011 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcf7c067050 con 0x558090dcf9b0 2024-04-24T06:49:44.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.159+0000 7fcf9e454740 10 client.26543 fetch_fsmap learned FSMap version 27 2024-04-24T06:49:44.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.159+0000 7fcf9e454740 1 -- 172.21.15.5:0/3250118011 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558090e5a500 con 0x558090dcf9b0 2024-04-24T06:49:44.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.159+0000 7fcf82000640 1 -- 172.21.15.5:0/3250118011 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf7c053070 con 0x558090dcf9b0 2024-04-24T06:49:44.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.159+0000 7fcf82000640 1 -- 172.21.15.5:0/3250118011 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7fcf7c04e760 con 0x558090dcf9b0 2024-04-24T06:49:44.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.159+0000 7fcf9e454740 10 client.26543 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:49:44.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.159+0000 7fcf9e454740 10 client.26543 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:49:44.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.159+0000 7fcf9e454740 20 client.26543 populate_metadata read hostname 'smithi005' 2024-04-24T06:49:44.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.159+0000 7fcf9e454740 1 --2- 172.21.15.5:0/3250118011 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x558090e5ac20 0x558090e5d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:44.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.159+0000 7fcf9e454740 4 client.26543 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T06:49:44.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.159+0000 7fcf9e454740 1 -- 172.21.15.5:0/3250118011 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558090d3c700 con 0x558090e5ac20 2024-04-24T06:49:44.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.160+0000 7fcf93e00640 1 --2- 172.21.15.5:0/3250118011 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x558090e5ac20 0x558090e5d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:44.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.160+0000 7fcf93e00640 1 --2- 172.21.15.5:0/3250118011 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x558090e5ac20 0x558090e5d040 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d31873e88864e3ba server_cookie=c07ebca1ff2cc2c5 in_seq=0 out_seq=0 2024-04-24T06:49:44.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.160+0000 7fcf82000640 10 client.26543 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:49:44.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.161+0000 7fcf82000640 1 -- 172.21.15.5:0/3250118011 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558090d3c700 con 0x558090e5ac20 2024-04-24T06:49:44.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.161+0000 7fcf82000640 10 client.26543 handle_command_reply: tid=0 2024-04-24T06:49:44.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.161+0000 7fcf9e454740 1 client.26543 shutdown 2024-04-24T06:49:44.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.162+0000 7fcf9e454740 1 -- 172.21.15.5:0/3250118011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558090dcf9b0 msgr2=0x558090de2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:44.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.162+0000 7fcf9e454740 1 --2- 172.21.15.5:0/3250118011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558090dcf9b0 0x558090de2540 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c009860 tx=0x7fcf7c0035f0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.162+0000 7fcf9e454740 1 -- 172.21.15.5:0/3250118011 shutdown_connections 2024-04-24T06:49:44.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.162+0000 7fcf9e454740 1 --2- 172.21.15.5:0/3250118011 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x558090e5ac20 0x558090e5d040 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.162+0000 7fcf9e454740 1 --2- 172.21.15.5:0/3250118011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558090ddf780 0x558090ddfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.162+0000 7fcf9e454740 1 --2- 172.21.15.5:0/3250118011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558090dda080 0x558090ddf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.162+0000 7fcf9e454740 1 --2- 172.21.15.5:0/3250118011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558090dcf9b0 0x558090de2540 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.162+0000 7fcf82000640 0 client.26543 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:49:44.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.162+0000 7fcf9e454740 1 -- 172.21.15.5:0/3250118011 >> 172.21.15.5:0/3250118011 conn(0x558090dcb3b0 msgr2=0x558090dc9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:44.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.162+0000 7fcf9e454740 1 -- 172.21.15.5:0/3250118011 shutdown_connections 2024-04-24T06:49:44.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.162+0000 7fcf9e454740 1 -- 172.21.15.5:0/3250118011 wait complete. 2024-04-24T06:49:44.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.162+0000 7fcf9e454740 20 client.26543 trim_cache size 0 max 16384 2024-04-24T06:49:44.170 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:49:44.170 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-24T06:49:44.170 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "4309dfcc-4eb2-4d1a-8b2d-27fa3c60e9f8", 2024-04-24T06:49:44.171 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-24T06:49:44.171 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:49:44.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.163+0000 7fcf9b600640 1 -- 172.21.15.5:0/1450228442 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fcf780a3d20 msgr2=0x7fcf780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:44.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.163+0000 7fcf9b600640 1 --2- 172.21.15.5:0/1450228442 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fcf780a3d20 0x7fcf780a61e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c009830 tx=0x7fcf7c0056d0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.163+0000 7fcf9b600640 1 -- 172.21.15.5:0/1450228442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf94109520 msgr2=0x7fcf941a0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:44.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.163+0000 7fcf9b600640 1 --2- 172.21.15.5:0/1450228442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf94109520 0x7fcf941a0d50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fcf8400c3a0 tx=0x7fcf8400c860 comp rx=0 tx=0).stop 2024-04-24T06:49:44.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.163+0000 7fcf9b600640 1 -- 172.21.15.5:0/1450228442 shutdown_connections 2024-04-24T06:49:44.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.163+0000 7fcf9b600640 1 --2- 172.21.15.5:0/1450228442 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fcf780a3d20 0x7fcf780a61e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.163+0000 7fcf9b600640 1 --2- 172.21.15.5:0/1450228442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf9419a9c0 0x7fcf9419ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.163+0000 7fcf9b600640 1 --2- 172.21.15.5:0/1450228442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf94109520 0x7fcf941a0d50 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.163+0000 7fcf9b600640 1 --2- 172.21.15.5:0/1450228442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf94104450 0x7fcf941a0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.163+0000 7fcf9b600640 1 -- 172.21.15.5:0/1450228442 >> 172.21.15.5:0/1450228442 conn(0x7fcf940ffc90 msgr2=0x7fcf94101fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:44.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.163+0000 7fcf9b600640 1 -- 172.21.15.5:0/1450228442 shutdown_connections 2024-04-24T06:49:44.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.163+0000 7fcf9b600640 1 -- 172.21.15.5:0/1450228442 wait complete. 2024-04-24T06:49:44.186 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T06:49:44.186 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T06:49:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.415+0000 7f7fe9200640 1 -- 172.21.15.5:0/3873837405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fe4104c30 msgr2=0x7f7fe4108c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.415+0000 7f7fe9200640 1 --2- 172.21.15.5:0/3873837405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fe4104c30 0x7f7fe4108c70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0009900 tx=0x7f7fd002a3f0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.415+0000 7f7fe9200640 1 -- 172.21.15.5:0/3873837405 shutdown_connections 2024-04-24T06:49:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.415+0000 7f7fe9200640 1 --2- 172.21.15.5:0/3873837405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fe4109350 0x7f7fe4110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.415+0000 7f7fe9200640 1 --2- 172.21.15.5:0/3873837405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fe4104c30 0x7f7fe4108c70 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.415+0000 7f7fe9200640 1 --2- 172.21.15.5:0/3873837405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fe4104280 0x7f7fe4104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.415+0000 7f7fe9200640 1 -- 172.21.15.5:0/3873837405 >> 172.21.15.5:0/3873837405 conn(0x7f7fe40ffae0 msgr2=0x7f7fe4101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.415+0000 7f7fe9200640 1 -- 172.21.15.5:0/3873837405 shutdown_connections 2024-04-24T06:49:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.415+0000 7f7fe9200640 1 -- 172.21.15.5:0/3873837405 wait complete. 2024-04-24T06:49:44.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.416+0000 7f7fe9200640 1 Processor -- start 2024-04-24T06:49:44.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.416+0000 7f7fe9200640 1 -- start start 2024-04-24T06:49:44.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.417+0000 7f7fe9200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fe4104280 0x7f7fe41a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:44.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.417+0000 7f7fe9200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fe4109350 0x7f7fe41a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:44.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.417+0000 7f7fe9200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fe419a820 0x7f7fe419acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:44.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.417+0000 7f7fe9200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe4114410 con 0x7f7fe419a820 2024-04-24T06:49:44.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.417+0000 7f7fe9200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe4114290 con 0x7f7fe4109350 2024-04-24T06:49:44.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.417+0000 7f7fe9200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe4114590 con 0x7f7fe4104280 2024-04-24T06:49:44.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.417+0000 7f7fe3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fe4104280 0x7f7fe41a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:44.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.417+0000 7f7fe2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fe4109350 0x7f7fe41a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:44.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.417+0000 7f7fe3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fe4104280 0x7f7fe41a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:52638/0 (socket says 172.21.15.5:52638) 2024-04-24T06:49:44.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.417+0000 7f7fe3400640 1 -- 172.21.15.5:0/1257001464 learned_addr learned my addr 172.21.15.5:0/1257001464 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:44.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.417+0000 7f7fe3e00640 1 --2- 172.21.15.5:0/1257001464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fe419a820 0x7f7fe419acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:44.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.418+0000 7f7fe3400640 1 -- 172.21.15.5:0/1257001464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fe4109350 msgr2=0x7f7fe41a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:44.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.418+0000 7f7fe3400640 1 --2- 172.21.15.5:0/1257001464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fe4109350 0x7f7fe41a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.418+0000 7f7fe3400640 1 -- 172.21.15.5:0/1257001464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fe419a820 msgr2=0x7f7fe419acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:44.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.418+0000 7f7fe3400640 1 --2- 172.21.15.5:0/1257001464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fe419a820 0x7f7fe419acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.418+0000 7f7fe3400640 1 -- 172.21.15.5:0/1257001464 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fe419b3b0 con 0x7f7fe4104280 2024-04-24T06:49:44.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.418+0000 7f7fe3400640 1 --2- 172.21.15.5:0/1257001464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fe4104280 0x7f7fe41a06b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7fd400e9f0 tx=0x7f7fd400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:44.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.418+0000 7f7fdbe00640 1 -- 172.21.15.5:0/1257001464 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd400cd80 con 0x7f7fe4104280 2024-04-24T06:49:44.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.418+0000 7f7fe9200640 1 -- 172.21.15.5:0/1257001464 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fe419b6a0 con 0x7f7fe4104280 2024-04-24T06:49:44.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.418+0000 7f7fdbe00640 1 -- 172.21.15.5:0/1257001464 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fd4010910 con 0x7f7fe4104280 2024-04-24T06:49:44.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.418+0000 7f7fdbe00640 1 -- 172.21.15.5:0/1257001464 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd40188f0 con 0x7f7fe4104280 2024-04-24T06:49:44.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.418+0000 7f7fe9200640 1 -- 172.21.15.5:0/1257001464 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fe41a78a0 con 0x7f7fe4104280 2024-04-24T06:49:44.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.422+0000 7f7fed660740 1 Processor -- start 2024-04-24T06:49:44.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.422+0000 7f7fed660740 1 -- start start 2024-04-24T06:49:44.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.422+0000 7f7fdbe00640 1 -- 172.21.15.5:0/1257001464 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f7fd4010ac0 con 0x7f7fe4104280 2024-04-24T06:49:44.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.422+0000 7f7fed660740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebce18b780 0x55ebce18bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:44.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.422+0000 7f7fed660740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebce18c130 0x55ebce19ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:44.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.422+0000 7f7fed660740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ebce19f230 0x55ebce1a1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:44.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.422+0000 7f7fed660740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ebce0f11f0 con 0x55ebce18b780 2024-04-24T06:49:44.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.422+0000 7f7fed660740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ebce097350 con 0x55ebce19f230 2024-04-24T06:49:44.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.422+0000 7f7fed660740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ebce0ed1b0 con 0x55ebce18c130 2024-04-24T06:49:44.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.423+0000 7f7fe3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebce18c130 0x55ebce19ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:44.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.423+0000 7f7fe2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ebce19f230 0x55ebce1a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:44.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.423+0000 7f7fe3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebce18c130 0x55ebce19ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:52642/0 (socket says 172.21.15.5:52642) 2024-04-24T06:49:44.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.423+0000 7f7fe2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ebce19f230 0x55ebce1a1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:45518/0 (socket says 172.21.15.5:45518) 2024-04-24T06:49:44.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.423+0000 7f7fe3400640 1 -- 172.21.15.5:0/1698052278 learned_addr learned my addr 172.21.15.5:0/1698052278 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:44.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.423+0000 7f7fe3e00640 1 --2- 172.21.15.5:0/1698052278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebce18b780 0x55ebce18bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:44.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.423+0000 7f7fe3400640 1 -- 172.21.15.5:0/1698052278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ebce19f230 msgr2=0x55ebce1a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:44.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.423+0000 7f7fe3400640 1 --2- 172.21.15.5:0/1698052278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ebce19f230 0x55ebce1a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.423+0000 7f7fe3400640 1 -- 172.21.15.5:0/1698052278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebce18b780 msgr2=0x55ebce18bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:44.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.423+0000 7f7fe3400640 1 --2- 172.21.15.5:0/1698052278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebce18b780 0x55ebce18bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.423+0000 7f7fe3400640 1 -- 172.21.15.5:0/1698052278 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebce0ebf50 con 0x55ebce18c130 2024-04-24T06:49:44.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.423+0000 7f7fe3e00640 1 --2- 172.21.15.5:0/1698052278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebce18b780 0x55ebce18bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:49:44.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.424+0000 7f7fe3400640 1 --2- 172.21.15.5:0/1698052278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebce18c130 0x55ebce19ecf0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7fd40ce250 tx=0x7f7fd40d96d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=275ead5bd4149aa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:44.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.424+0000 7f7fdbe00640 1 --2- 172.21.15.5:0/1257001464 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f7fc80a3da0 0x7f7fc80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:44.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.424+0000 7f7fd8c00640 1 -- 172.21.15.5:0/1698052278 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd40e0020 con 0x55ebce18c130 2024-04-24T06:49:44.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.424+0000 7f7fd8c00640 1 -- 172.21.15.5:0/1698052278 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fd409d260 con 0x55ebce18c130 2024-04-24T06:49:44.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.424+0000 7f7fd8c00640 1 -- 172.21.15.5:0/1698052278 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd409c4c0 con 0x55ebce18c130 2024-04-24T06:49:44.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.424+0000 7f7fdbe00640 1 -- 172.21.15.5:0/1257001464 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f7fd40d0c10 con 0x7f7fe4104280 2024-04-24T06:49:44.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.424+0000 7f7fe3e00640 1 --2- 172.21.15.5:0/1257001464 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f7fc80a3da0 0x7f7fc80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:44.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.425+0000 7f7fed660740 1 -- 172.21.15.5:0/1698052278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebce18c130 msgr2=0x55ebce19ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:44.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.425+0000 7f7fed660740 1 --2- 172.21.15.5:0/1698052278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebce18c130 0x55ebce19ecf0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7fd40ce250 tx=0x7f7fd40d96d0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.425+0000 7f7fe3e00640 1 --2- 172.21.15.5:0/1257001464 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f7fc80a3da0 0x7f7fc80a6260 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc005fd0 tx=0x7f7fdc009500 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:44.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.426+0000 7f7fed660740 1 -- 172.21.15.5:0/1698052278 shutdown_connections 2024-04-24T06:49:44.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.426+0000 7f7fed660740 1 --2- 172.21.15.5:0/1698052278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ebce19f230 0x55ebce1a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.426+0000 7f7fed660740 1 --2- 172.21.15.5:0/1698052278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebce18c130 0x55ebce19ecf0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.426+0000 7f7fed660740 1 --2- 172.21.15.5:0/1698052278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebce18b780 0x55ebce18bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.426+0000 7f7fed660740 1 -- 172.21.15.5:0/1698052278 >> 172.21.15.5:0/1698052278 conn(0x55ebce187d50 msgr2=0x55ebce188130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:44.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.426+0000 7f7fed660740 1 -- 172.21.15.5:0/1698052278 shutdown_connections 2024-04-24T06:49:44.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.426+0000 7f7fed660740 1 -- 172.21.15.5:0/1698052278 wait complete. 2024-04-24T06:49:44.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.426+0000 7f7fed660740 1 -- start start 2024-04-24T06:49:44.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.427+0000 7f7fed660740 1 Processor -- start 2024-04-24T06:49:44.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.427+0000 7f7fed660740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebce18b780 0x55ebce19c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:44.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.427+0000 7f7fed660740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ebce18c130 0x55ebce19c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:44.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.427+0000 7f7fed660740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebce19f230 0x55ebce19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:44.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.427+0000 7f7fed660740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ebce1a1ff0 con 0x55ebce18b780 2024-04-24T06:49:44.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.427+0000 7f7fed660740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ebce0aa690 con 0x55ebce18c130 2024-04-24T06:49:44.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.427+0000 7f7fed660740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ebce1a2170 con 0x55ebce19f230 2024-04-24T06:49:44.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.427+0000 7f7fed660740 10 client.0 _ll_register_callbacks cb 0x55ebce0e6650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:49:44.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.427+0000 7f7fe2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ebce18c130 0x55ebce19c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:44.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.428+0000 7f7fe2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ebce18c130 0x55ebce19c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:45530/0 (socket says 172.21.15.5:45530) 2024-04-24T06:49:44.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.428+0000 7f7fe2a00640 1 -- 172.21.15.5:0/3989857382 learned_addr learned my addr 172.21.15.5:0/3989857382 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:44.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.428+0000 7f7fe3e00640 1 --2- 172.21.15.5:0/3989857382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebce19f230 0x55ebce19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:44.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.428+0000 7f7fe3400640 1 --2- 172.21.15.5:0/3989857382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebce18b780 0x55ebce19c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:44.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.428+0000 7f7fe2a00640 1 -- 172.21.15.5:0/3989857382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebce19f230 msgr2=0x55ebce19cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:44.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.428+0000 7f7fe2a00640 1 --2- 172.21.15.5:0/3989857382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebce19f230 0x55ebce19cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.428+0000 7f7fe2a00640 1 -- 172.21.15.5:0/3989857382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebce18b780 msgr2=0x55ebce19c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:44.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.428+0000 7f7fe2a00640 1 --2- 172.21.15.5:0/3989857382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebce18b780 0x55ebce19c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.428+0000 7f7fe2a00640 1 -- 172.21.15.5:0/3989857382 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebce0a9920 con 0x55ebce18c130 2024-04-24T06:49:44.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.428+0000 7f7fe2a00640 1 --2- 172.21.15.5:0/3989857382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ebce18c130 0x55ebce19c650 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0009900 tx=0x7f7fd00043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a31df32d7a1b4829 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:44.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.428+0000 7f7fcd600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:49:44.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.429+0000 7f7fcd600640 1 -- 172.21.15.5:0/3989857382 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd0035040 con 0x55ebce18c130 2024-04-24T06:49:44.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.429+0000 7f7fed660740 1 -- 172.21.15.5:0/3989857382 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fd409d260 con 0x55ebce18c130 2024-04-24T06:49:44.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.429+0000 7f7fcd600640 1 -- 172.21.15.5:0/3989857382 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f7fd0005450 con 0x55ebce18c130 2024-04-24T06:49:44.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.429+0000 7f7fcd600640 1 -- 172.21.15.5:0/3989857382 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7fd0005650 con 0x55ebce18c130 2024-04-24T06:49:44.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.429+0000 7f7fed660740 10 client.26555 fetch_fsmap learned FSMap version 27 2024-04-24T06:49:44.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.429+0000 7f7fed660740 1 -- 172.21.15.5:0/3989857382 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ebcdecd7d0 con 0x55ebce18c130 2024-04-24T06:49:44.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.430+0000 7f7fcd600640 1 -- 172.21.15.5:0/3989857382 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fd409d260 con 0x55ebce18c130 2024-04-24T06:49:44.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.430+0000 7f7fcd600640 1 -- 172.21.15.5:0/3989857382 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f7fd0003a30 con 0x55ebce18c130 2024-04-24T06:49:44.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.430+0000 7f7fed660740 10 client.26555 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:49:44.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.430+0000 7f7fed660740 10 client.26555 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:49:44.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.430+0000 7f7fed660740 20 client.26555 populate_metadata read hostname 'smithi005' 2024-04-24T06:49:44.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.430+0000 7f7fed660740 1 --2- 172.21.15.5:0/3989857382 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55ebce1af4a0 0x55ebce1b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:44.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.430+0000 7f7fed660740 4 client.26555 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:49:44.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.430+0000 7f7fed660740 1 -- 172.21.15.5:0/3989857382 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ebce0fec10 con 0x55ebce1af4a0 2024-04-24T06:49:44.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.430+0000 7f7fe3400640 1 --2- 172.21.15.5:0/3989857382 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55ebce1af4a0 0x55ebce1b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:44.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.431+0000 7f7fcd600640 10 client.26555 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:49:44.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.431+0000 7f7fe3400640 1 --2- 172.21.15.5:0/3989857382 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55ebce1af4a0 0x55ebce1b18d0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5be0e98ea6bde8d server_cookie=df026428aff56418 in_seq=0 out_seq=0 2024-04-24T06:49:44.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.432+0000 7f7fcd600640 1 -- 172.21.15.5:0/3989857382 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ebce0fec10 con 0x55ebce1af4a0 2024-04-24T06:49:44.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.432+0000 7f7fcd600640 10 client.26555 handle_command_reply: tid=0 2024-04-24T06:49:44.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.432+0000 7f7fed660740 1 client.26555 shutdown 2024-04-24T06:49:44.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.432+0000 7f7fed660740 1 -- 172.21.15.5:0/3989857382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ebce18c130 msgr2=0x55ebce19c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:44.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.432+0000 7f7fed660740 1 --2- 172.21.15.5:0/3989857382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ebce18c130 0x55ebce19c650 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0009900 tx=0x7f7fd00043b0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.432+0000 7f7fed660740 1 -- 172.21.15.5:0/3989857382 shutdown_connections 2024-04-24T06:49:44.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.432+0000 7f7fed660740 1 --2- 172.21.15.5:0/3989857382 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55ebce1af4a0 0x55ebce1b18d0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.432+0000 7f7fed660740 1 --2- 172.21.15.5:0/3989857382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebce19f230 0x55ebce19cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.432+0000 7f7fed660740 1 --2- 172.21.15.5:0/3989857382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ebce18c130 0x55ebce19c650 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.432+0000 7f7fed660740 1 --2- 172.21.15.5:0/3989857382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebce18b780 0x55ebce19c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.432+0000 7f7fcd600640 0 client.26555 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:49:44.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.432+0000 7f7fed660740 1 -- 172.21.15.5:0/3989857382 >> 172.21.15.5:0/3989857382 conn(0x55ebce187d50 msgr2=0x55ebce18db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:44.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.433+0000 7f7fed660740 1 -- 172.21.15.5:0/3989857382 shutdown_connections 2024-04-24T06:49:44.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.433+0000 7f7fed660740 1 -- 172.21.15.5:0/3989857382 wait complete. 2024-04-24T06:49:44.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.433+0000 7f7fed660740 20 client.26555 trim_cache size 0 max 16384 2024-04-24T06:49:44.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.458+0000 7f7fed660740 1 Processor -- start 2024-04-24T06:49:44.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.458+0000 7f7fed660740 1 -- start start 2024-04-24T06:49:44.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.459+0000 7f7fed660740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebce18b780 0x55ebce1aaef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:44.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.459+0000 7f7fed660740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ebce18c130 0x55ebce182e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:44.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.459+0000 7f7fed660740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebce19f230 0x55ebce1b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:44.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.459+0000 7f7fed660740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe41ae2e0 con 0x55ebce19f230 2024-04-24T06:49:44.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.459+0000 7f7fed660740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe41abbb0 con 0x55ebce18c130 2024-04-24T06:49:44.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.459+0000 7f7fed660740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe41a9a00 con 0x55ebce18b780 2024-04-24T06:49:44.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.459+0000 7f7fe2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ebce18c130 0x55ebce182e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:44.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.459+0000 7f7fe3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebce18b780 0x55ebce1aaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:44.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.459+0000 7f7fe2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ebce18c130 0x55ebce182e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:45538/0 (socket says 172.21.15.5:45538) 2024-04-24T06:49:44.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.459+0000 7f7fe2a00640 1 -- 172.21.15.5:0/1178231112 learned_addr learned my addr 172.21.15.5:0/1178231112 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:44.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.459+0000 7f7fe3e00640 1 --2- 172.21.15.5:0/1178231112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebce19f230 0x55ebce1b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:44.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.460+0000 7f7fe2a00640 1 -- 172.21.15.5:0/1178231112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebce18b780 msgr2=0x55ebce1aaef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:44.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.460+0000 7f7fe2a00640 1 --2- 172.21.15.5:0/1178231112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebce18b780 0x55ebce1aaef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.460+0000 7f7fe2a00640 1 -- 172.21.15.5:0/1178231112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebce19f230 msgr2=0x55ebce1b71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:44.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.460+0000 7f7fe2a00640 1 --2- 172.21.15.5:0/1178231112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebce19f230 0x55ebce1b71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.460+0000 7f7fe2a00640 1 -- 172.21.15.5:0/1178231112 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebce19b0c0 con 0x55ebce18c130 2024-04-24T06:49:44.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.460+0000 7f7fe2a00640 1 --2- 172.21.15.5:0/1178231112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ebce18c130 0x55ebce182e60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0006010 tx=0x7f7fd0005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=315e6f1318238a6e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:44.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.460+0000 7f7fcfe00640 1 -- 172.21.15.5:0/1178231112 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd0035030 con 0x55ebce18c130 2024-04-24T06:49:44.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.460+0000 7f7fcfe00640 1 -- 172.21.15.5:0/1178231112 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fd0041070 con 0x55ebce18c130 2024-04-24T06:49:44.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.460+0000 7f7fcfe00640 1 -- 172.21.15.5:0/1178231112 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd00038e0 con 0x55ebce18c130 2024-04-24T06:49:44.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.461+0000 7f7fed660740 1 -- 172.21.15.5:0/1178231112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ebce18c130 msgr2=0x55ebce182e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:44.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.461+0000 7f7fed660740 1 --2- 172.21.15.5:0/1178231112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ebce18c130 0x55ebce182e60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0006010 tx=0x7f7fd0005d50 comp rx=0 tx=0).stop 2024-04-24T06:49:44.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.461+0000 7f7fed660740 1 -- 172.21.15.5:0/1178231112 shutdown_connections 2024-04-24T06:49:44.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.461+0000 7f7fed660740 1 --2- 172.21.15.5:0/1178231112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebce19f230 0x55ebce1b71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.461+0000 7f7fed660740 1 --2- 172.21.15.5:0/1178231112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ebce18c130 0x55ebce182e60 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.461+0000 7f7fed660740 1 --2- 172.21.15.5:0/1178231112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebce18b780 0x55ebce1aaef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.461+0000 7f7fed660740 1 -- 172.21.15.5:0/1178231112 >> 172.21.15.5:0/1178231112 conn(0x55ebce187d50 msgr2=0x55ebce18db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:44.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.461+0000 7f7fed660740 1 -- 172.21.15.5:0/1178231112 shutdown_connections 2024-04-24T06:49:44.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.461+0000 7f7fed660740 1 -- 172.21.15.5:0/1178231112 wait complete. 2024-04-24T06:49:44.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.462+0000 7f7fed660740 1 -- start start 2024-04-24T06:49:44.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.462+0000 7f7fed660740 1 Processor -- start 2024-04-24T06:49:44.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.463+0000 7f7fed660740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebce18b780 0x55ebce198d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:44.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.463+0000 7f7fed660740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ebce18c130 0x55ebce199280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:44.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.463+0000 7f7fed660740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebce19f230 0x55ebce1997c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:44.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.463+0000 7f7fed660740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe41aa070 con 0x55ebce18b780 2024-04-24T06:49:44.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.463+0000 7f7fed660740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe41ab4e0 con 0x55ebce18c130 2024-04-24T06:49:44.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.463+0000 7f7fed660740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe41aa740 con 0x55ebce19f230 2024-04-24T06:49:44.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.463+0000 7f7fed660740 10 client.0 _ll_register_callbacks cb 0x7f7fd0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:49:44.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.463+0000 7f7fe2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ebce18c130 0x55ebce199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:44.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.463+0000 7f7fe2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ebce18c130 0x55ebce199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:45552/0 (socket says 172.21.15.5:45552) 2024-04-24T06:49:44.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.463+0000 7f7fe2a00640 1 -- 172.21.15.5:0/3059085672 learned_addr learned my addr 172.21.15.5:0/3059085672 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:49:44.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.463+0000 7f7fe3400640 1 --2- 172.21.15.5:0/3059085672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebce18b780 0x55ebce198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:44.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.463+0000 7f7fe3e00640 1 --2- 172.21.15.5:0/3059085672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebce19f230 0x55ebce1997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:44.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.463+0000 7f7fe2a00640 1 -- 172.21.15.5:0/3059085672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebce19f230 msgr2=0x55ebce1997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:44.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.463+0000 7f7fe2a00640 1 --2- 172.21.15.5:0/3059085672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebce19f230 0x55ebce1997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.463+0000 7f7fe2a00640 1 -- 172.21.15.5:0/3059085672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebce18b780 msgr2=0x55ebce198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:49:44.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.463+0000 7f7fe2a00640 1 --2- 172.21.15.5:0/3059085672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebce18b780 0x55ebce198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.464+0000 7f7fe2a00640 1 -- 172.21.15.5:0/3059085672 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebcde033f0 con 0x55ebce18c130 2024-04-24T06:49:44.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.464+0000 7f7fe2a00640 1 --2- 172.21.15.5:0/3059085672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ebce18c130 0x55ebce199280 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7fd002a910 tx=0x7f7fd0000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11a51cbb176e52ce server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:49:44.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.464+0000 7f7fcd600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:49:44.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.464+0000 7f7fcd600640 1 -- 172.21.15.5:0/3059085672 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd0035070 con 0x55ebce18c130 2024-04-24T06:49:44.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.464+0000 7f7fed660740 1 -- 172.21.15.5:0/3059085672 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fd0041070 con 0x55ebce18c130 2024-04-24T06:49:44.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.464+0000 7f7fcd600640 1 -- 172.21.15.5:0/3059085672 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f7fd0004390 con 0x55ebce18c130 2024-04-24T06:49:44.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.465+0000 7f7fcd600640 1 -- 172.21.15.5:0/3059085672 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7fd0031700 con 0x55ebce18c130 2024-04-24T06:49:44.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.465+0000 7f7fed660740 10 client.26561 fetch_fsmap learned FSMap version 27 2024-04-24T06:49:44.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.465+0000 7f7fed660740 1 -- 172.21.15.5:0/3059085672 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ebcddc30f0 con 0x55ebce18c130 2024-04-24T06:49:44.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.465+0000 7f7fcd600640 1 -- 172.21.15.5:0/3059085672 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fd0041070 con 0x55ebce18c130 2024-04-24T06:49:44.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.465+0000 7f7fcd600640 1 -- 172.21.15.5:0/3059085672 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f7fd0058020 con 0x55ebce18c130 2024-04-24T06:49:44.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.465+0000 7f7fed660740 10 client.26561 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:49:44.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.465+0000 7f7fed660740 10 client.26561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:49:44.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.465+0000 7f7fed660740 20 client.26561 populate_metadata read hostname 'smithi005' 2024-04-24T06:49:44.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.466+0000 7f7fed660740 1 --2- 172.21.15.5:0/3059085672 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55ebce215df0 0x55ebce218210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:49:44.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.466+0000 7f7fed660740 4 client.26561 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:49:44.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.466+0000 7f7fed660740 1 -- 172.21.15.5:0/3059085672 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ebce0f8a80 con 0x55ebce215df0 2024-04-24T06:49:44.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.466+0000 7f7fe3400640 1 --2- 172.21.15.5:0/3059085672 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55ebce215df0 0x55ebce218210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:49:44.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.466+0000 7f7fe3400640 1 --2- 172.21.15.5:0/3059085672 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55ebce215df0 0x55ebce218210 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0ae473aa0d73f03 server_cookie=9071a532e8653973 in_seq=0 out_seq=0 2024-04-24T06:49:44.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.466+0000 7f7fcd600640 10 client.26561 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:49:44.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.467+0000 7f7fcd600640 1 -- 172.21.15.5:0/3059085672 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55ebce0f8a80 con 0x55ebce215df0 2024-04-24T06:49:44.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.467+0000 7f7fcd600640 10 client.26561 handle_command_reply: tid=0 2024-04-24T06:49:44.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.467+0000 7f7fed660740 1 client.26561 shutdown 2024-04-24T06:49:44.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.467+0000 7f7fed660740 1 -- 172.21.15.5:0/3059085672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ebce18c130 msgr2=0x55ebce199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:44.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.467+0000 7f7fed660740 1 --2- 172.21.15.5:0/3059085672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ebce18c130 0x55ebce199280 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7fd002a910 tx=0x7f7fd0000bc0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.467+0000 7f7fed660740 1 -- 172.21.15.5:0/3059085672 shutdown_connections 2024-04-24T06:49:44.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.467+0000 7f7fed660740 1 --2- 172.21.15.5:0/3059085672 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55ebce215df0 0x55ebce218210 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.467+0000 7f7fed660740 1 --2- 172.21.15.5:0/3059085672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebce19f230 0x55ebce1997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.467+0000 7f7fed660740 1 --2- 172.21.15.5:0/3059085672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ebce18c130 0x55ebce199280 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.467+0000 7f7fed660740 1 --2- 172.21.15.5:0/3059085672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebce18b780 0x55ebce198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.467+0000 7f7fed660740 1 -- 172.21.15.5:0/3059085672 >> 172.21.15.5:0/3059085672 conn(0x55ebce187d50 msgr2=0x55ebce19bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:44.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.467+0000 7f7fcd600640 0 client.26561 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:49:44.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.468+0000 7f7fed660740 1 -- 172.21.15.5:0/3059085672 shutdown_connections 2024-04-24T06:49:44.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.468+0000 7f7fed660740 1 -- 172.21.15.5:0/3059085672 wait complete. 2024-04-24T06:49:44.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.468+0000 7f7fed660740 20 client.26561 trim_cache size 0 max 16384 2024-04-24T06:49:44.475 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:49:44.475 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1151 inodes in the stack)", 2024-04-24T06:49:44.475 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-24T06:49:44.475 INFO:teuthology.orchestra.run.smithi005.stdout: "4309dfcc-4eb2-4d1a-8b2d-27fa3c60e9f8": { 2024-04-24T06:49:44.476 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-24T06:49:44.476 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "4309dfcc-4eb2-4d1a-8b2d-27fa3c60e9f8", 2024-04-24T06:49:44.476 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-04-24T06:49:44.476 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:49:44.476 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:49:44.476 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:49:44.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.469+0000 7f7fe9200640 1 -- 172.21.15.5:0/1257001464 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f7fc80a3da0 msgr2=0x7f7fc80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:44.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.469+0000 7f7fe9200640 1 --2- 172.21.15.5:0/1257001464 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f7fc80a3da0 0x7f7fc80a6260 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc005fd0 tx=0x7f7fdc009500 comp rx=0 tx=0).stop 2024-04-24T06:49:44.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.469+0000 7f7fe9200640 1 -- 172.21.15.5:0/1257001464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fe4104280 msgr2=0x7f7fe41a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:49:44.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.469+0000 7f7fe9200640 1 --2- 172.21.15.5:0/1257001464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fe4104280 0x7f7fe41a06b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7fd400e9f0 tx=0x7f7fd400eeb0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.469+0000 7f7fe9200640 1 -- 172.21.15.5:0/1257001464 shutdown_connections 2024-04-24T06:49:44.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.469+0000 7f7fe9200640 1 --2- 172.21.15.5:0/1257001464 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f7fc80a3da0 0x7f7fc80a6260 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.469+0000 7f7fe9200640 1 --2- 172.21.15.5:0/1257001464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fe419a820 0x7f7fe419acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.469+0000 7f7fe9200640 1 --2- 172.21.15.5:0/1257001464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fe4109350 0x7f7fe41a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.469+0000 7f7fe9200640 1 --2- 172.21.15.5:0/1257001464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fe4104280 0x7f7fe41a06b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:49:44.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.469+0000 7f7fe9200640 1 -- 172.21.15.5:0/1257001464 >> 172.21.15.5:0/1257001464 conn(0x7f7fe40ffae0 msgr2=0x7f7fe4101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:49:44.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.469+0000 7f7fe9200640 1 -- 172.21.15.5:0/1257001464 shutdown_connections 2024-04-24T06:49:44.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:49:44.469+0000 7f7fe9200640 1 -- 172.21.15.5:0/1257001464 wait complete. 2024-04-24T06:49:44.490 INFO:tasks.cephfs.filesystem:scrub status for tag:4309dfcc-4eb2-4d1a-8b2d-27fa3c60e9f8 - {'path': '/', 'tag': '4309dfcc-4eb2-4d1a-8b2d-27fa3c60e9f8', 'options': 'recursive,force'} 2024-04-24T06:49:45.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:45 smithi137 ceph-mon[33017]: pgmap v174: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:49:45.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:45 smithi137 ceph-mon[33017]: scrub queued for path: / 2024-04-24T06:49:45.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:45 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:49:45.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:45 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:49:45.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:45 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:49:45.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:45 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:49:45.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:45 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:49:45.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:45 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:49:45.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:45 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:49:45.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:45 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:49:45.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:45 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:49:45.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:45 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:49:45.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:45 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:49:45.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:45 smithi005 ceph-mon[27798]: pgmap v174: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:49:45.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:45 smithi005 ceph-mon[27798]: scrub queued for path: / 2024-04-24T06:49:45.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:45 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:49:45.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:45 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:49:45.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:45 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:49:45.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:45 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:49:45.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:45 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:49:45.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:45 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:49:45.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:45 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:49:45.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:45 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:49:45.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:45 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:49:45.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:45 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:49:45.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:45 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:49:45.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:45 smithi037 ceph-mon[32864]: pgmap v174: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:49:45.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:45 smithi037 ceph-mon[32864]: scrub queued for path: / 2024-04-24T06:49:45.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:45 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:49:45.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:45 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:49:45.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:45 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:49:45.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:45 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:49:45.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:45 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:49:45.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:45 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:49:45.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:45 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:49:45.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:45 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:49:45.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:45 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:49:45.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:45 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:49:45.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:45 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:49:47.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:47 smithi137 ceph-mon[33017]: pgmap v175: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:49:47.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:47 smithi005 ceph-mon[27798]: pgmap v175: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:49:47.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:47 smithi037 ceph-mon[32864]: pgmap v175: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:49:49.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:49 smithi137 ceph-mon[33017]: pgmap v176: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:49:49.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:49 smithi005 ceph-mon[27798]: pgmap v176: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:49:49.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:49 smithi037 ceph-mon[32864]: pgmap v176: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:49:51.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:51 smithi137 ceph-mon[33017]: pgmap v177: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-04-24T06:49:51.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:51 smithi005 ceph-mon[27798]: pgmap v177: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-04-24T06:49:51.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:51 smithi037 ceph-mon[32864]: pgmap v177: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-04-24T06:49:53.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:53 smithi137 ceph-mon[33017]: pgmap v178: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:49:53.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:53 smithi005 ceph-mon[27798]: pgmap v178: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:49:53.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:53 smithi037 ceph-mon[32864]: pgmap v178: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:49:55.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:55 smithi137 ceph-mon[33017]: pgmap v179: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:49:55.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:55 smithi005 ceph-mon[27798]: pgmap v179: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:49:55.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:55 smithi037 ceph-mon[32864]: pgmap v179: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:49:57.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:57 smithi137 ceph-mon[33017]: pgmap v180: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:49:57.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:57 smithi005 ceph-mon[27798]: pgmap v180: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:49:57.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:57 smithi037 ceph-mon[32864]: pgmap v180: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:49:58.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:58 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:49:58.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:58 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:49:58.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:58 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:49:58.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:58 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:49:58.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:58 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:49:58.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:58 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:49:58.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:58 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:49:58.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:58 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:49:58.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:58 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:49:59.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:49:59 smithi137 ceph-mon[33017]: pgmap v181: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:49:59.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:49:59 smithi005 ceph-mon[27798]: pgmap v181: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:49:59.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:49:59 smithi037 ceph-mon[32864]: pgmap v181: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:50:00.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:00 smithi137 ceph-mon[33017]: overall HEALTH_OK 2024-04-24T06:50:00.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:00 smithi005 ceph-mon[27798]: overall HEALTH_OK 2024-04-24T06:50:00.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:00 smithi037 ceph-mon[32864]: overall HEALTH_OK 2024-04-24T06:50:01.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:01 smithi137 ceph-mon[33017]: pgmap v182: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-04-24T06:50:01.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:01 smithi005 ceph-mon[27798]: pgmap v182: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-04-24T06:50:01.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:01 smithi037 ceph-mon[32864]: pgmap v182: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-04-24T06:50:03.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:03 smithi137 ceph-mon[33017]: pgmap v183: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:50:03.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:03 smithi005 ceph-mon[27798]: pgmap v183: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:50:03.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:03 smithi037 ceph-mon[32864]: pgmap v183: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:50:05.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:05 smithi137 ceph-mon[33017]: pgmap v184: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-24T06:50:05.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:05 smithi005 ceph-mon[27798]: pgmap v184: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-24T06:50:05.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:05 smithi037 ceph-mon[32864]: pgmap v184: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-24T06:50:07.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:07 smithi005 ceph-mon[27798]: pgmap v185: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-24T06:50:07.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:07 smithi037 ceph-mon[32864]: pgmap v185: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-24T06:50:07.945 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:07 smithi137 ceph-mon[33017]: pgmap v185: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-24T06:50:09.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:09 smithi005 ceph-mon[27798]: pgmap v186: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-24T06:50:09.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:09 smithi037 ceph-mon[32864]: pgmap v186: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-24T06:50:09.945 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:09 smithi137 ceph-mon[33017]: pgmap v186: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-24T06:50:11.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:11 smithi005 ceph-mon[27798]: pgmap v187: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-24T06:50:11.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:11 smithi037 ceph-mon[32864]: pgmap v187: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-24T06:50:11.945 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:11 smithi137 ceph-mon[33017]: pgmap v187: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-24T06:50:13.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:13 smithi005 ceph-mon[27798]: pgmap v188: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-24T06:50:13.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:13 smithi037 ceph-mon[32864]: pgmap v188: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-24T06:50:13.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:13 smithi137 ceph-mon[33017]: pgmap v188: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-24T06:50:14.490 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T06:50:14.491 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T06:50:14.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.720+0000 7f7620200640 1 -- 172.21.15.5:0/1540598926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76181090c0 msgr2=0x7f7618115570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:14.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.720+0000 7f7620200640 1 --2- 172.21.15.5:0/1540598926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76181090c0 0x7f7618115570 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7610009ca0 tx=0x7f761002a950 comp rx=0 tx=0).stop 2024-04-24T06:50:14.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.720+0000 7f7620200640 1 -- 172.21.15.5:0/1540598926 shutdown_connections 2024-04-24T06:50:14.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.720+0000 7f7620200640 1 --2- 172.21.15.5:0/1540598926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76181090c0 0x7f7618115570 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:14.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.720+0000 7f7620200640 1 --2- 172.21.15.5:0/1540598926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7618108720 0x7f7618108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:14.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.720+0000 7f7620200640 1 --2- 172.21.15.5:0/1540598926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f761810a590 0x7f761810a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:14.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.720+0000 7f7620200640 1 -- 172.21.15.5:0/1540598926 >> 172.21.15.5:0/1540598926 conn(0x7f7618077e70 msgr2=0x7f7618106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:14.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.720+0000 7f7620200640 1 -- 172.21.15.5:0/1540598926 shutdown_connections 2024-04-24T06:50:14.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.720+0000 7f7620200640 1 -- 172.21.15.5:0/1540598926 wait complete. 2024-04-24T06:50:14.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.721+0000 7f7620200640 1 Processor -- start 2024-04-24T06:50:14.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.721+0000 7f7620200640 1 -- start start 2024-04-24T06:50:14.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.721+0000 7f7620200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7618108720 0x7f761819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:14.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.722+0000 7f7620200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76181090c0 0x7f761819f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:14.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.722+0000 7f7620200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f761810a590 0x7f76181a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:14.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.722+0000 7f7620200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7618118b40 con 0x7f761810a590 2024-04-24T06:50:14.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.722+0000 7f7620200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f76181189c0 con 0x7f76181090c0 2024-04-24T06:50:14.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.722+0000 7f7620200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7618118cc0 con 0x7f7618108720 2024-04-24T06:50:14.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.722+0000 7f761d800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7618108720 0x7f761819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:14.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.722+0000 7f761d800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7618108720 0x7f761819ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:59732/0 (socket says 172.21.15.5:59732) 2024-04-24T06:50:14.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.722+0000 7f761d800640 1 -- 172.21.15.5:0/3311191320 learned_addr learned my addr 172.21.15.5:0/3311191320 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:14.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.722+0000 7f761ce00640 1 --2- 172.21.15.5:0/3311191320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76181090c0 0x7f761819f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:14.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.722+0000 7f761e200640 1 --2- 172.21.15.5:0/3311191320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f761810a590 0x7f76181a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:14.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.722+0000 7f761d800640 1 -- 172.21.15.5:0/3311191320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76181090c0 msgr2=0x7f761819f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:14.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.722+0000 7f761d800640 1 --2- 172.21.15.5:0/3311191320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76181090c0 0x7f761819f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:14.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.722+0000 7f761d800640 1 -- 172.21.15.5:0/3311191320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f761810a590 msgr2=0x7f76181a37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:14.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.722+0000 7f761d800640 1 --2- 172.21.15.5:0/3311191320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f761810a590 0x7f76181a37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:14.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.722+0000 7f761d800640 1 -- 172.21.15.5:0/3311191320 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76181a3ef0 con 0x7f7618108720 2024-04-24T06:50:14.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.723+0000 7f761d800640 1 --2- 172.21.15.5:0/3311191320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7618108720 0x7f761819ef00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f760c00e9f0 tx=0x7f760c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:14.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.723+0000 7f7616000640 1 -- 172.21.15.5:0/3311191320 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c00cd80 con 0x7f7618108720 2024-04-24T06:50:14.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.723+0000 7f7620200640 1 -- 172.21.15.5:0/3311191320 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76181a41e0 con 0x7f7618108720 2024-04-24T06:50:14.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.723+0000 7f7616000640 1 -- 172.21.15.5:0/3311191320 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f760c010910 con 0x7f7618108720 2024-04-24T06:50:14.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.723+0000 7f7616000640 1 -- 172.21.15.5:0/3311191320 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c0188f0 con 0x7f7618108720 2024-04-24T06:50:14.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.723+0000 7f7620200640 1 -- 172.21.15.5:0/3311191320 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76181aba70 con 0x7f7618108720 2024-04-24T06:50:14.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.726+0000 7f7622f8b740 1 Processor -- start 2024-04-24T06:50:14.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.727+0000 7f7616000640 1 -- 172.21.15.5:0/3311191320 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f760c010ac0 con 0x7f7618108720 2024-04-24T06:50:14.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.727+0000 7f7622f8b740 1 -- start start 2024-04-24T06:50:14.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.727+0000 7f7622f8b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556743c07860 0x556743c07c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:14.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.727+0000 7f7622f8b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556743c08180 0x556743c12310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:14.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.727+0000 7f7622f8b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556743c12850 0x556743c14c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:14.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.727+0000 7f7622f8b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556743b6d1f0 con 0x556743c07860 2024-04-24T06:50:14.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.727+0000 7f7622f8b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556743b13350 con 0x556743c08180 2024-04-24T06:50:14.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.727+0000 7f7622f8b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556743b691b0 con 0x556743c12850 2024-04-24T06:50:14.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.727+0000 7f761e200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556743c07860 0x556743c07c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:14.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.727+0000 7f761e200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556743c07860 0x556743c07c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36062/0 (socket says 172.21.15.5:36062) 2024-04-24T06:50:14.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.727+0000 7f761e200640 1 -- 172.21.15.5:0/495591671 learned_addr learned my addr 172.21.15.5:0/495591671 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:14.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.728+0000 7f761d800640 1 --2- 172.21.15.5:0/495591671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556743c08180 0x556743c12310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:14.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.728+0000 7f761ce00640 1 --2- 172.21.15.5:0/495591671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556743c12850 0x556743c14c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:14.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.728+0000 7f761e200640 1 -- 172.21.15.5:0/495591671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556743c12850 msgr2=0x556743c14c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:14.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.728+0000 7f761e200640 1 --2- 172.21.15.5:0/495591671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556743c12850 0x556743c14c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:14.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.728+0000 7f761e200640 1 -- 172.21.15.5:0/495591671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556743c08180 msgr2=0x556743c12310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:14.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.728+0000 7f761e200640 1 --2- 172.21.15.5:0/495591671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556743c08180 0x556743c12310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:14.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.728+0000 7f761e200640 1 -- 172.21.15.5:0/495591671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556743b67f50 con 0x556743c07860 2024-04-24T06:50:14.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.728+0000 7f761d800640 1 --2- 172.21.15.5:0/495591671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556743c08180 0x556743c12310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:50:14.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.728+0000 7f7616000640 1 --2- 172.21.15.5:0/3311191320 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f76000a3d20 0x7f76000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:14.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.728+0000 7f7616000640 1 -- 172.21.15.5:0/3311191320 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f760c0d0ba0 con 0x7f7618108720 2024-04-24T06:50:14.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.728+0000 7f761e200640 1 --2- 172.21.15.5:0/495591671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556743c07860 0x556743c07c40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7610009ca0 tx=0x7f7610005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6bc4a79ec8d6205 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:14.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.728+0000 7f761d800640 1 --2- 172.21.15.5:0/3311191320 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f76000a3d20 0x7f76000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:14.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.729+0000 7f760aa00640 1 -- 172.21.15.5:0/495591671 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7610048070 con 0x556743c07860 2024-04-24T06:50:14.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.729+0000 7f760aa00640 1 -- 172.21.15.5:0/495591671 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7610003e50 con 0x556743c07860 2024-04-24T06:50:14.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.729+0000 7f760aa00640 1 -- 172.21.15.5:0/495591671 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7610034040 con 0x556743c07860 2024-04-24T06:50:14.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.730+0000 7f7622f8b740 1 -- 172.21.15.5:0/495591671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556743c07860 msgr2=0x556743c07c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:14.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.730+0000 7f761d800640 1 --2- 172.21.15.5:0/3311191320 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f76000a3d20 0x7f76000a61e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f760c0ce3b0 tx=0x7f760c0d46f0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:14.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.730+0000 7f7622f8b740 1 --2- 172.21.15.5:0/495591671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556743c07860 0x556743c07c40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7610009ca0 tx=0x7f7610005f80 comp rx=0 tx=0).stop 2024-04-24T06:50:14.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.730+0000 7f7622f8b740 1 -- 172.21.15.5:0/495591671 shutdown_connections 2024-04-24T06:50:14.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.730+0000 7f7622f8b740 1 --2- 172.21.15.5:0/495591671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556743c12850 0x556743c14c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:14.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.730+0000 7f7622f8b740 1 --2- 172.21.15.5:0/495591671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556743c08180 0x556743c12310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:14.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.730+0000 7f7622f8b740 1 --2- 172.21.15.5:0/495591671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556743c07860 0x556743c07c40 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:14.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.730+0000 7f7622f8b740 1 -- 172.21.15.5:0/495591671 >> 172.21.15.5:0/495591671 conn(0x556743c03b80 msgr2=0x556743c03f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:14.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.730+0000 7f7622f8b740 1 -- 172.21.15.5:0/495591671 shutdown_connections 2024-04-24T06:50:14.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.730+0000 7f7622f8b740 1 -- 172.21.15.5:0/495591671 wait complete. 2024-04-24T06:50:14.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.730+0000 7f7622f8b740 1 -- start start 2024-04-24T06:50:14.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.731+0000 7f7622f8b740 1 Processor -- start 2024-04-24T06:50:14.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.731+0000 7f7622f8b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556743c07860 0x556743c0f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:14.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.731+0000 7f7622f8b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556743c08180 0x556743c0fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:14.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.731+0000 7f7622f8b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556743c12850 0x556743c102d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:14.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.731+0000 7f7622f8b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556743c155e0 con 0x556743c08180 2024-04-24T06:50:14.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.731+0000 7f7622f8b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556743b26690 con 0x556743c12850 2024-04-24T06:50:14.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.731+0000 7f7622f8b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556743c15760 con 0x556743c07860 2024-04-24T06:50:14.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.731+0000 7f7622f8b740 10 client.0 _ll_register_callbacks cb 0x556743b62650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:50:14.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.731+0000 7f761ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556743c08180 0x556743c0fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:14.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.731+0000 7f761ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556743c08180 0x556743c0fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36074/0 (socket says 172.21.15.5:36074) 2024-04-24T06:50:14.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.731+0000 7f761ce00640 1 -- 172.21.15.5:0/1104382063 learned_addr learned my addr 172.21.15.5:0/1104382063 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:14.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.732+0000 7f761e200640 1 --2- 172.21.15.5:0/1104382063 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556743c07860 0x556743c0f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:14.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.732+0000 7f761e200640 1 --2- 172.21.15.5:0/1104382063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556743c12850 0x556743c102d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:14.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.732+0000 7f761ce00640 1 -- 172.21.15.5:0/1104382063 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556743c07860 msgr2=0x556743c0f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:14.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.732+0000 7f761ce00640 1 --2- 172.21.15.5:0/1104382063 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556743c07860 0x556743c0f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:14.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.732+0000 7f761ce00640 1 -- 172.21.15.5:0/1104382063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556743c12850 msgr2=0x556743c102d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:14.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.732+0000 7f761ce00640 1 --2- 172.21.15.5:0/1104382063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556743c12850 0x556743c102d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:14.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.732+0000 7f761ce00640 1 -- 172.21.15.5:0/1104382063 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556743b25920 con 0x556743c08180 2024-04-24T06:50:14.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.732+0000 7f761ce00640 1 --2- 172.21.15.5:0/1104382063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556743c08180 0x556743c0fd90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7604012720 tx=0x7f7604012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e101fcf28f06ee89 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:14.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.732+0000 7f75ff400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:50:14.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.732+0000 7f75ff400640 1 -- 172.21.15.5:0/1104382063 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760400f920 con 0x556743c08180 2024-04-24T06:50:14.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.732+0000 7f75ff400640 1 -- 172.21.15.5:0/1104382063 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f760400fac0 con 0x556743c08180 2024-04-24T06:50:14.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.733+0000 7f7622f8b740 1 -- 172.21.15.5:0/1104382063 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7610003e50 con 0x556743c08180 2024-04-24T06:50:14.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.733+0000 7f75ff400640 1 -- 172.21.15.5:0/1104382063 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f760400fd00 con 0x556743c08180 2024-04-24T06:50:14.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.733+0000 7f7622f8b740 10 client.16434 fetch_fsmap learned FSMap version 27 2024-04-24T06:50:14.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.733+0000 7f7622f8b740 1 -- 172.21.15.5:0/1104382063 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5567439497d0 con 0x556743c08180 2024-04-24T06:50:14.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.733+0000 7f75ff400640 1 -- 172.21.15.5:0/1104382063 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7610003e50 con 0x556743c08180 2024-04-24T06:50:14.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.733+0000 7f75ff400640 1 -- 172.21.15.5:0/1104382063 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f760401c590 con 0x556743c08180 2024-04-24T06:50:14.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.733+0000 7f7622f8b740 10 client.16434 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:50:14.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.733+0000 7f7622f8b740 10 client.16434 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:50:14.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.734+0000 7f7622f8b740 20 client.16434 populate_metadata read hostname 'smithi005' 2024-04-24T06:50:14.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.734+0000 7f7622f8b740 1 --2- 172.21.15.5:0/1104382063 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x556743c22bd0 0x556743c24ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:14.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.734+0000 7f7622f8b740 4 client.16434 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:50:14.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.734+0000 7f7622f8b740 1 -- 172.21.15.5:0/1104382063 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556743b7ac10 con 0x556743c22bd0 2024-04-24T06:50:14.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.734+0000 7f761e200640 1 --2- 172.21.15.5:0/1104382063 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x556743c22bd0 0x556743c24ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:14.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.735+0000 7f761e200640 1 --2- 172.21.15.5:0/1104382063 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x556743c22bd0 0x556743c24ff0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77604b06c83c07b8 server_cookie=426faaa1d89941ed in_seq=0 out_seq=0 2024-04-24T06:50:14.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.735+0000 7f75ff400640 10 client.16434 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:50:14.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.735+0000 7f75ff400640 1 -- 172.21.15.5:0/1104382063 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556743b7ac10 con 0x556743c22bd0 2024-04-24T06:50:14.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.735+0000 7f75ff400640 10 client.16434 handle_command_reply: tid=0 2024-04-24T06:50:14.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.735+0000 7f7622f8b740 1 client.16434 shutdown 2024-04-24T06:50:14.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.736+0000 7f7622f8b740 1 -- 172.21.15.5:0/1104382063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556743c08180 msgr2=0x556743c0fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:14.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.736+0000 7f7622f8b740 1 --2- 172.21.15.5:0/1104382063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556743c08180 0x556743c0fd90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7604012720 tx=0x7f7604012be0 comp rx=0 tx=0).stop 2024-04-24T06:50:14.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.736+0000 7f7622f8b740 1 -- 172.21.15.5:0/1104382063 shutdown_connections 2024-04-24T06:50:14.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.736+0000 7f7622f8b740 1 --2- 172.21.15.5:0/1104382063 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x556743c22bd0 0x556743c24ff0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:14.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.736+0000 7f7622f8b740 1 --2- 172.21.15.5:0/1104382063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556743c12850 0x556743c102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:14.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.736+0000 7f7622f8b740 1 --2- 172.21.15.5:0/1104382063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556743c08180 0x556743c0fd90 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:14.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.736+0000 7f7622f8b740 1 --2- 172.21.15.5:0/1104382063 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556743c07860 0x556743c0f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:14.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.736+0000 7f75ff400640 0 client.16434 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:50:14.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.736+0000 7f7622f8b740 1 -- 172.21.15.5:0/1104382063 >> 172.21.15.5:0/1104382063 conn(0x556743c03b80 msgr2=0x556743c09980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:14.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.736+0000 7f7622f8b740 1 -- 172.21.15.5:0/1104382063 shutdown_connections 2024-04-24T06:50:14.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.736+0000 7f7622f8b740 1 -- 172.21.15.5:0/1104382063 wait complete. 2024-04-24T06:50:14.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.736+0000 7f7622f8b740 20 client.16434 trim_cache size 0 max 16384 2024-04-24T06:50:14.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.755+0000 7f7622f8b740 1 Processor -- start 2024-04-24T06:50:14.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.756+0000 7f7622f8b740 1 -- start start 2024-04-24T06:50:14.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.756+0000 7f7622f8b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556743c07860 0x556743c0e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:14.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.756+0000 7f7622f8b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556743c08180 0x556743c1e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:14.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.756+0000 7f7622f8b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556743c12850 0x556743c100f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:14.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.756+0000 7f7622f8b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f76181b6b90 con 0x556743c12850 2024-04-24T06:50:14.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.756+0000 7f7622f8b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f76181b4460 con 0x556743c07860 2024-04-24T06:50:14.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.756+0000 7f7622f8b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f76181b22b0 con 0x556743c08180 2024-04-24T06:50:14.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.756+0000 7f761e200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556743c12850 0x556743c100f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:14.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.756+0000 7f761e200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556743c12850 0x556743c100f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36086/0 (socket says 172.21.15.5:36086) 2024-04-24T06:50:14.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.756+0000 7f761e200640 1 -- 172.21.15.5:0/1587167768 learned_addr learned my addr 172.21.15.5:0/1587167768 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:14.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.756+0000 7f761e200640 1 --2- 172.21.15.5:0/1587167768 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556743c07860 0x556743c0e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:14.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.756+0000 7f761ce00640 1 --2- 172.21.15.5:0/1587167768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556743c08180 0x556743c1e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:14.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.757+0000 7f761e200640 1 -- 172.21.15.5:0/1587167768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556743c08180 msgr2=0x556743c1e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:14.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.757+0000 7f761e200640 1 --2- 172.21.15.5:0/1587167768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556743c08180 0x556743c1e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:14.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.757+0000 7f761e200640 1 -- 172.21.15.5:0/1587167768 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556743c07860 msgr2=0x556743c0e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:14.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.757+0000 7f761e200640 1 --2- 172.21.15.5:0/1587167768 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556743c07860 0x556743c0e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:14.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.757+0000 7f761e200640 1 -- 172.21.15.5:0/1587167768 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55674387f3f0 con 0x556743c12850 2024-04-24T06:50:14.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.757+0000 7f761e200640 1 --2- 172.21.15.5:0/1587167768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556743c12850 0x556743c100f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7610004af0 tx=0x7f7610005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd321377d08cbe7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:14.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.757+0000 7f760a000640 1 -- 172.21.15.5:0/1587167768 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7610048040 con 0x556743c12850 2024-04-24T06:50:14.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.757+0000 7f760a000640 1 -- 172.21.15.5:0/1587167768 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7610049040 con 0x556743c12850 2024-04-24T06:50:14.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.757+0000 7f760a000640 1 -- 172.21.15.5:0/1587167768 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7610004660 con 0x556743c12850 2024-04-24T06:50:14.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.758+0000 7f7622f8b740 1 -- 172.21.15.5:0/1587167768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556743c12850 msgr2=0x556743c100f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:14.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.758+0000 7f7622f8b740 1 --2- 172.21.15.5:0/1587167768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556743c12850 0x556743c100f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7610004af0 tx=0x7f7610005f80 comp rx=0 tx=0).stop 2024-04-24T06:50:14.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.758+0000 7f7622f8b740 1 -- 172.21.15.5:0/1587167768 shutdown_connections 2024-04-24T06:50:14.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.758+0000 7f7622f8b740 1 --2- 172.21.15.5:0/1587167768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556743c12850 0x556743c100f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:14.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.758+0000 7f7622f8b740 1 --2- 172.21.15.5:0/1587167768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556743c08180 0x556743c1e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:14.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.758+0000 7f7622f8b740 1 --2- 172.21.15.5:0/1587167768 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556743c07860 0x556743c0e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:14.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.758+0000 7f7622f8b740 1 -- 172.21.15.5:0/1587167768 >> 172.21.15.5:0/1587167768 conn(0x556743c03b80 msgr2=0x556743c02c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:14.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.758+0000 7f7622f8b740 1 -- 172.21.15.5:0/1587167768 shutdown_connections 2024-04-24T06:50:14.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.758+0000 7f7622f8b740 1 -- 172.21.15.5:0/1587167768 wait complete. 2024-04-24T06:50:14.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.758+0000 7f7622f8b740 1 -- start start 2024-04-24T06:50:14.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.759+0000 7f7622f8b740 1 Processor -- start 2024-04-24T06:50:14.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.759+0000 7f7622f8b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556743c07860 0x556743c165c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:14.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.759+0000 7f7622f8b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556743c08180 0x556743c16b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:14.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.759+0000 7f7622f8b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556743c12850 0x556743c17040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:14.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.759+0000 7f7622f8b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f76181b2920 con 0x556743c08180 2024-04-24T06:50:14.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.759+0000 7f7622f8b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f76181b3d90 con 0x556743c12850 2024-04-24T06:50:14.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.759+0000 7f7622f8b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f76181b2ff0 con 0x556743c07860 2024-04-24T06:50:14.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.759+0000 7f7622f8b740 10 client.0 _ll_register_callbacks cb 0x7f7604009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:50:14.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.759+0000 7f761ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556743c08180 0x556743c16b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:14.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.759+0000 7f761ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556743c08180 0x556743c16b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36096/0 (socket says 172.21.15.5:36096) 2024-04-24T06:50:14.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.759+0000 7f761ce00640 1 -- 172.21.15.5:0/4164368385 learned_addr learned my addr 172.21.15.5:0/4164368385 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:14.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.759+0000 7f761e200640 1 --2- 172.21.15.5:0/4164368385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556743c07860 0x556743c165c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:14.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.759+0000 7f761e200640 1 --2- 172.21.15.5:0/4164368385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556743c12850 0x556743c17040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:14.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.760+0000 7f761ce00640 1 -- 172.21.15.5:0/4164368385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556743c07860 msgr2=0x556743c165c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:14.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.760+0000 7f761ce00640 1 --2- 172.21.15.5:0/4164368385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556743c07860 0x556743c165c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:14.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.760+0000 7f761ce00640 1 -- 172.21.15.5:0/4164368385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556743c12850 msgr2=0x556743c17040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:14.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.760+0000 7f761ce00640 1 --2- 172.21.15.5:0/4164368385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556743c12850 0x556743c17040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:14.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.760+0000 7f761ce00640 1 -- 172.21.15.5:0/4164368385 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55674383f0f0 con 0x556743c08180 2024-04-24T06:50:14.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.760+0000 7f761ce00640 1 --2- 172.21.15.5:0/4164368385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556743c08180 0x556743c16b00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x556743c0f0b0 tx=0x7f760400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa0d4dbe16a4858c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:14.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.760+0000 7f75ff400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:50:14.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.760+0000 7f75ff400640 1 -- 172.21.15.5:0/4164368385 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760402a030 con 0x556743c08180 2024-04-24T06:50:14.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.760+0000 7f75ff400640 1 -- 172.21.15.5:0/4164368385 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f7604018040 con 0x556743c08180 2024-04-24T06:50:14.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.760+0000 7f7622f8b740 1 -- 172.21.15.5:0/4164368385 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7610049040 con 0x556743c08180 2024-04-24T06:50:14.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.761+0000 7f75ff400640 1 -- 172.21.15.5:0/4164368385 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f760400fb30 con 0x556743c08180 2024-04-24T06:50:14.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.761+0000 7f7622f8b740 10 client.16440 fetch_fsmap learned FSMap version 27 2024-04-24T06:50:14.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.761+0000 7f7622f8b740 1 -- 172.21.15.5:0/4164368385 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556743b32810 con 0x556743c08180 2024-04-24T06:50:14.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.761+0000 7f75ff400640 1 -- 172.21.15.5:0/4164368385 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7610049040 con 0x556743c08180 2024-04-24T06:50:14.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.761+0000 7f75ff400640 1 -- 172.21.15.5:0/4164368385 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f76040036f0 con 0x556743c08180 2024-04-24T06:50:14.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.761+0000 7f7622f8b740 10 client.16440 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:50:14.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.761+0000 7f7622f8b740 10 client.16440 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:50:14.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.761+0000 7f7622f8b740 20 client.16440 populate_metadata read hostname 'smithi005' 2024-04-24T06:50:14.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.761+0000 7f7622f8b740 1 --2- 172.21.15.5:0/4164368385 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x556743c899a0 0x556743c8bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:14.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.761+0000 7f7622f8b740 4 client.16440 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:50:14.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.761+0000 7f7622f8b740 1 -- 172.21.15.5:0/4164368385 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556743b74a80 con 0x556743c899a0 2024-04-24T06:50:14.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.762+0000 7f761e200640 1 --2- 172.21.15.5:0/4164368385 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x556743c899a0 0x556743c8bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:14.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.762+0000 7f761e200640 1 --2- 172.21.15.5:0/4164368385 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x556743c899a0 0x556743c8bdc0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=143e0c849a57c237 server_cookie=ca602826345a6d0f in_seq=0 out_seq=0 2024-04-24T06:50:14.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.762+0000 7f75ff400640 10 client.16440 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:50:14.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.763+0000 7f75ff400640 1 -- 172.21.15.5:0/4164368385 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556743b74a80 con 0x556743c899a0 2024-04-24T06:50:14.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.763+0000 7f75ff400640 10 client.16440 handle_command_reply: tid=0 2024-04-24T06:50:14.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.763+0000 7f7622f8b740 1 client.16440 shutdown 2024-04-24T06:50:14.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.763+0000 7f7622f8b740 1 -- 172.21.15.5:0/4164368385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556743c08180 msgr2=0x556743c16b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:14.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.763+0000 7f7622f8b740 1 --2- 172.21.15.5:0/4164368385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556743c08180 0x556743c16b00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x556743c0f0b0 tx=0x7f760400f790 comp rx=0 tx=0).stop 2024-04-24T06:50:14.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.763+0000 7f7622f8b740 1 -- 172.21.15.5:0/4164368385 shutdown_connections 2024-04-24T06:50:14.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.763+0000 7f7622f8b740 1 --2- 172.21.15.5:0/4164368385 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x556743c899a0 0x556743c8bdc0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:14.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.763+0000 7f7622f8b740 1 --2- 172.21.15.5:0/4164368385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556743c12850 0x556743c17040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:14.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.763+0000 7f7622f8b740 1 --2- 172.21.15.5:0/4164368385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556743c08180 0x556743c16b00 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:14.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.763+0000 7f7622f8b740 1 --2- 172.21.15.5:0/4164368385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556743c07860 0x556743c165c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:14.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.763+0000 7f75ff400640 0 client.16440 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:50:14.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.763+0000 7f7622f8b740 1 -- 172.21.15.5:0/4164368385 >> 172.21.15.5:0/4164368385 conn(0x556743c03b80 msgr2=0x556743c08fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:14.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.763+0000 7f7622f8b740 1 -- 172.21.15.5:0/4164368385 shutdown_connections 2024-04-24T06:50:14.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.763+0000 7f7622f8b740 1 -- 172.21.15.5:0/4164368385 wait complete. 2024-04-24T06:50:14.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.764+0000 7f7622f8b740 20 client.16440 trim_cache size 0 max 16384 2024-04-24T06:50:14.773 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:50:14.773 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-24T06:50:14.773 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-24T06:50:14.773 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:50:14.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.764+0000 7f7620200640 1 -- 172.21.15.5:0/3311191320 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f76000a3d20 msgr2=0x7f76000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:14.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.764+0000 7f7620200640 1 --2- 172.21.15.5:0/3311191320 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f76000a3d20 0x7f76000a61e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f760c0ce3b0 tx=0x7f760c0d46f0 comp rx=0 tx=0).stop 2024-04-24T06:50:14.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.764+0000 7f7620200640 1 -- 172.21.15.5:0/3311191320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7618108720 msgr2=0x7f761819ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:14.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.764+0000 7f7620200640 1 --2- 172.21.15.5:0/3311191320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7618108720 0x7f761819ef00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f760c00e9f0 tx=0x7f760c00eeb0 comp rx=0 tx=0).stop 2024-04-24T06:50:14.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.764+0000 7f7620200640 1 -- 172.21.15.5:0/3311191320 shutdown_connections 2024-04-24T06:50:14.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.764+0000 7f7620200640 1 --2- 172.21.15.5:0/3311191320 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f76000a3d20 0x7f76000a61e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:14.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.764+0000 7f7620200640 1 --2- 172.21.15.5:0/3311191320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f761810a590 0x7f76181a37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:14.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.764+0000 7f7620200640 1 --2- 172.21.15.5:0/3311191320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76181090c0 0x7f761819f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:14.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.764+0000 7f7620200640 1 --2- 172.21.15.5:0/3311191320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7618108720 0x7f761819ef00 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:14.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.764+0000 7f7620200640 1 -- 172.21.15.5:0/3311191320 >> 172.21.15.5:0/3311191320 conn(0x7f7618077e70 msgr2=0x7f76180783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:14.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.764+0000 7f7620200640 1 -- 172.21.15.5:0/3311191320 shutdown_connections 2024-04-24T06:50:14.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.764+0000 7f7620200640 1 -- 172.21.15.5:0/3311191320 wait complete. 2024-04-24T06:50:14.786 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T06:50:14.786 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:50:14.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.991+0000 7fddfb200640 1 -- 172.21.15.5:0/2586917586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddfc107200 msgr2=0x7fddfc10edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:14.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.991+0000 7fddfb200640 1 --2- 172.21.15.5:0/2586917586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddfc107200 0x7fddfc10edd0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fddf4009ca0 tx=0x7fddf402a8c0 comp rx=0 tx=0).stop 2024-04-24T06:50:14.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.992+0000 7fddfb200640 1 -- 172.21.15.5:0/2586917586 shutdown_connections 2024-04-24T06:50:14.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.992+0000 7fddfb200640 1 --2- 172.21.15.5:0/2586917586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddfc107200 0x7fddfc10edd0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:14.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.992+0000 7fddfb200640 1 --2- 172.21.15.5:0/2586917586 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddfc102ae0 0x7fddfc106b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:14.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.992+0000 7fddfb200640 1 --2- 172.21.15.5:0/2586917586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddfc102130 0x7fddfc102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:14.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.992+0000 7fddfb200640 1 -- 172.21.15.5:0/2586917586 >> 172.21.15.5:0/2586917586 conn(0x7fddfc0fd9f0 msgr2=0x7fddfc0ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:14.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.992+0000 7fddfb200640 1 -- 172.21.15.5:0/2586917586 shutdown_connections 2024-04-24T06:50:14.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.992+0000 7fddfb200640 1 -- 172.21.15.5:0/2586917586 wait complete. 2024-04-24T06:50:14.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.993+0000 7fddfb200640 1 Processor -- start 2024-04-24T06:50:14.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.993+0000 7fddfb200640 1 -- start start 2024-04-24T06:50:14.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.993+0000 7fddfb200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddfc102130 0x7fddfc1a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:14.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.993+0000 7fddfb200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddfc107200 0x7fddfc1a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:14.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.993+0000 7fddfb200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddfc19a7a0 0x7fddfc19ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:14.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.993+0000 7fddfb200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc1122f0 con 0x7fddfc102130 2024-04-24T06:50:14.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.993+0000 7fddfb200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc112170 con 0x7fddfc107200 2024-04-24T06:50:14.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.993+0000 7fddfb200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc112470 con 0x7fddfc19a7a0 2024-04-24T06:50:14.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.994+0000 7fddf9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddfc102130 0x7fddfc1a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:14.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.994+0000 7fddf9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddfc102130 0x7fddfc1a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36106/0 (socket says 172.21.15.5:36106) 2024-04-24T06:50:14.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.994+0000 7fddf9e00640 1 -- 172.21.15.5:0/3232981574 learned_addr learned my addr 172.21.15.5:0/3232981574 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:14.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.994+0000 7fddf3e00640 1 --2- 172.21.15.5:0/3232981574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddfc107200 0x7fddfc1a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:14.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.994+0000 7fddfa800640 1 --2- 172.21.15.5:0/3232981574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddfc19a7a0 0x7fddfc19ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:14.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.994+0000 7fddf9e00640 1 -- 172.21.15.5:0/3232981574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddfc19a7a0 msgr2=0x7fddfc19ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:14.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.994+0000 7fddf9e00640 1 --2- 172.21.15.5:0/3232981574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddfc19a7a0 0x7fddfc19ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:14.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.994+0000 7fddf9e00640 1 -- 172.21.15.5:0/3232981574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddfc107200 msgr2=0x7fddfc1a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:14.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.994+0000 7fddf9e00640 1 --2- 172.21.15.5:0/3232981574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddfc107200 0x7fddfc1a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:14.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.994+0000 7fddf9e00640 1 -- 172.21.15.5:0/3232981574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddfc19b330 con 0x7fddfc102130 2024-04-24T06:50:14.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.994+0000 7fddf9e00640 1 --2- 172.21.15.5:0/3232981574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddfc102130 0x7fddfc1a0630 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fddec00e9f0 tx=0x7fddec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:14.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.995+0000 7fddf2a00640 1 -- 172.21.15.5:0/3232981574 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec00cd80 con 0x7fddfc102130 2024-04-24T06:50:14.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.995+0000 7fddf2a00640 1 -- 172.21.15.5:0/3232981574 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddec010910 con 0x7fddfc102130 2024-04-24T06:50:15.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.995+0000 7fddfb200640 1 -- 172.21.15.5:0/3232981574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddfc19b620 con 0x7fddfc102130 2024-04-24T06:50:15.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.995+0000 7fddf2a00640 1 -- 172.21.15.5:0/3232981574 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec018990 con 0x7fddfc102130 2024-04-24T06:50:15.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.995+0000 7fddfb200640 1 -- 172.21.15.5:0/3232981574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddfc1a7800 con 0x7fddfc102130 2024-04-24T06:50:15.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.996+0000 7fddfb200640 1 -- 172.21.15.5:0/3232981574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddfc103600 con 0x7fddfc102130 2024-04-24T06:50:15.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:14.998+0000 7fddf2a00640 1 -- 172.21.15.5:0/3232981574 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fddec010ac0 con 0x7fddfc102130 2024-04-24T06:50:15.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.000+0000 7fddf2a00640 1 --2- 172.21.15.5:0/3232981574 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fdde00a3d10 0x7fdde00a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:15.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.000+0000 7fddf2a00640 1 -- 172.21.15.5:0/3232981574 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fddec0d0b80 con 0x7fddfc102130 2024-04-24T06:50:15.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.000+0000 7fddf3e00640 1 --2- 172.21.15.5:0/3232981574 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fdde00a3d10 0x7fdde00a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:15.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.001+0000 7fddf3e00640 1 --2- 172.21.15.5:0/3232981574 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fdde00a3d10 0x7fdde00a61d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fddfc10f280 tx=0x7fdde4005f90 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:15.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.004+0000 7fddf2a00640 1 -- 172.21.15.5:0/3232981574 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fddfc19b620 con 0x7fddfc102130 2024-04-24T06:50:15.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.253+0000 7fddfb200640 1 -- 172.21.15.5:0/3232981574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fddfc04fe00 con 0x7fddfc102130 2024-04-24T06:50:15.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.254+0000 7fddf2a00640 1 -- 172.21.15.5:0/3232981574 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11856 (secure 0 0 0) 0x7fddec09cb80 con 0x7fddfc102130 2024-04-24T06:50:15.257 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:50:15.257 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1735462750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1735462750},{"type":"v1","addr":"172.21.15.5:6837","nonce":1735462750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1499469370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1499469370},{"type":"v1","addr":"172.21.15.5:6841","nonce":1499469370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3705638847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3705638847},{"type":"v1","addr":"172.21.15.37:6835","nonce":3705638847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T06:41:34.764490+0000","modified":"2024-04-24T06:48:57.768410+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24550,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6837/1390103296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1390103296},{"type":"v1","addr":"172.21.15.37:6837","nonce":1390103296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3217754215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3217754215},{"type":"v1","addr":"172.21.15.137:6835","nonce":3217754215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6837/3044222976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3044222976},{"type":"v1","addr":"172.21.15.137:6837","nonce":3044222976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.5:6839/3173201459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3173201459},{"type":"v1","addr":"172.21.15.5:6839","nonce":3173201459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6839/1889774232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1889774232},{"type":"v1","addr":"172.21.15.137:6839","nonce":1889774232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6839/3536721753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":3536721753},{"type":"v1","addr":"172.21.15.37:6839","nonce":3536721753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24550,24569,24539]},"id":1}]} 2024-04-24T06:50:15.257 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-04-24T06:50:15.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.258+0000 7fddfb200640 1 -- 172.21.15.5:0/3232981574 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fdde00a3d10 msgr2=0x7fdde00a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:15.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.258+0000 7fddfb200640 1 --2- 172.21.15.5:0/3232981574 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fdde00a3d10 0x7fdde00a61d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fddfc10f280 tx=0x7fdde4005f90 comp rx=0 tx=0).stop 2024-04-24T06:50:15.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.258+0000 7fddfb200640 1 -- 172.21.15.5:0/3232981574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddfc102130 msgr2=0x7fddfc1a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:15.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.258+0000 7fddfb200640 1 --2- 172.21.15.5:0/3232981574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddfc102130 0x7fddfc1a0630 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fddec00e9f0 tx=0x7fddec00eeb0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.258+0000 7fddfb200640 1 -- 172.21.15.5:0/3232981574 shutdown_connections 2024-04-24T06:50:15.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.258+0000 7fddfb200640 1 --2- 172.21.15.5:0/3232981574 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fdde00a3d10 0x7fdde00a61d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.258+0000 7fddfb200640 1 --2- 172.21.15.5:0/3232981574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddfc19a7a0 0x7fddfc19ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.258+0000 7fddfb200640 1 --2- 172.21.15.5:0/3232981574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddfc107200 0x7fddfc1a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.258+0000 7fddfb200640 1 --2- 172.21.15.5:0/3232981574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddfc102130 0x7fddfc1a0630 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.258+0000 7fddfb200640 1 -- 172.21.15.5:0/3232981574 >> 172.21.15.5:0/3232981574 conn(0x7fddfc0fd9f0 msgr2=0x7fddfc0fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:15.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.258+0000 7fddfb200640 1 -- 172.21.15.5:0/3232981574 shutdown_connections 2024-04-24T06:50:15.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.258+0000 7fddfb200640 1 -- 172.21.15.5:0/3232981574 wait complete. 2024-04-24T06:50:15.280 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T06:50:15.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.503+0000 7fd153200640 1 -- 172.21.15.5:0/2492157483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd14c103900 msgr2=0x7fd14c10bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:15.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.503+0000 7fd153200640 1 --2- 172.21.15.5:0/2492157483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd14c103900 0x7fd14c10bde0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd13c009900 tx=0x7fd13c02a480 comp rx=0 tx=0).stop 2024-04-24T06:50:15.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.503+0000 7fd153200640 1 -- 172.21.15.5:0/2492157483 shutdown_connections 2024-04-24T06:50:15.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.503+0000 7fd153200640 1 --2- 172.21.15.5:0/2492157483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd14c10c400 0x7fd14c10e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.503+0000 7fd153200640 1 --2- 172.21.15.5:0/2492157483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd14c103900 0x7fd14c10bde0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.503+0000 7fd153200640 1 --2- 172.21.15.5:0/2492157483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd14c102fe0 0x7fd14c1033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.503+0000 7fd153200640 1 -- 172.21.15.5:0/2492157483 >> 172.21.15.5:0/2492157483 conn(0x7fd14c077ab0 msgr2=0x7fd14c0fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:15.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.503+0000 7fd153200640 1 -- 172.21.15.5:0/2492157483 shutdown_connections 2024-04-24T06:50:15.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.503+0000 7fd153200640 1 -- 172.21.15.5:0/2492157483 wait complete. 2024-04-24T06:50:15.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.504+0000 7fd153200640 1 Processor -- start 2024-04-24T06:50:15.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.504+0000 7fd153200640 1 -- start start 2024-04-24T06:50:15.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.505+0000 7fd153200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd14c102fe0 0x7fd14c1a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:15.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.505+0000 7fd153200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd14c103900 0x7fd14c1a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:15.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.505+0000 7fd153200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd14c10c400 0x7fd14c19a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:15.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.505+0000 7fd153200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd14c112200 con 0x7fd14c103900 2024-04-24T06:50:15.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.505+0000 7fd153200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd14c112080 con 0x7fd14c102fe0 2024-04-24T06:50:15.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.505+0000 7fd153200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd14c112380 con 0x7fd14c10c400 2024-04-24T06:50:15.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.505+0000 7fd151400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd14c103900 0x7fd14c1a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:15.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.505+0000 7fd151e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd14c102fe0 0x7fd14c1a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:15.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.505+0000 7fd151e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd14c102fe0 0x7fd14c1a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:33300/0 (socket says 172.21.15.5:33300) 2024-04-24T06:50:15.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.505+0000 7fd151400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd14c103900 0x7fd14c1a0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36128/0 (socket says 172.21.15.5:36128) 2024-04-24T06:50:15.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.505+0000 7fd151e00640 1 -- 172.21.15.5:0/3513631313 learned_addr learned my addr 172.21.15.5:0/3513631313 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:15.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.505+0000 7fd152800640 1 --2- 172.21.15.5:0/3513631313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd14c10c400 0x7fd14c19a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:15.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.506+0000 7fd151400640 1 -- 172.21.15.5:0/3513631313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd14c10c400 msgr2=0x7fd14c19a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:15.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.506+0000 7fd151400640 1 --2- 172.21.15.5:0/3513631313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd14c10c400 0x7fd14c19a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.506+0000 7fd151400640 1 -- 172.21.15.5:0/3513631313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd14c102fe0 msgr2=0x7fd14c1a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:15.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.506+0000 7fd151400640 1 --2- 172.21.15.5:0/3513631313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd14c102fe0 0x7fd14c1a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.506+0000 7fd151400640 1 -- 172.21.15.5:0/3513631313 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd14c19af70 con 0x7fd14c103900 2024-04-24T06:50:15.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.506+0000 7fd151400640 1 --2- 172.21.15.5:0/3513631313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd14c103900 0x7fd14c1a0b70 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd13c02a9a0 tx=0x7fd13c003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:15.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.506+0000 7fd146a00640 1 -- 172.21.15.5:0/3513631313 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13c03f070 con 0x7fd14c103900 2024-04-24T06:50:15.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.506+0000 7fd146a00640 1 -- 172.21.15.5:0/3513631313 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd13c031c90 con 0x7fd14c103900 2024-04-24T06:50:15.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.506+0000 7fd146a00640 1 -- 172.21.15.5:0/3513631313 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13c035030 con 0x7fd14c103900 2024-04-24T06:50:15.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.507+0000 7fd153200640 1 -- 172.21.15.5:0/3513631313 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd14c19b200 con 0x7fd14c103900 2024-04-24T06:50:15.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.507+0000 7fd153200640 1 -- 172.21.15.5:0/3513631313 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd14c1a7480 con 0x7fd14c103900 2024-04-24T06:50:15.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.510+0000 7fd157602740 1 Processor -- start 2024-04-24T06:50:15.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.510+0000 7fd146a00640 1 -- 172.21.15.5:0/3513631313 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fd13c042070 con 0x7fd14c103900 2024-04-24T06:50:15.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.510+0000 7fd157602740 1 -- start start 2024-04-24T06:50:15.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.510+0000 7fd157602740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39feacb40 0x55b39feacf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:15.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.510+0000 7fd157602740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39fead460 0x55b39feb75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:15.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.510+0000 7fd157602740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39feb7b30 0x55b39feb9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:15.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.510+0000 7fd157602740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b39fe121d0 con 0x55b39fead460 2024-04-24T06:50:15.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.510+0000 7fd157602740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b39fdb8450 con 0x55b39feacb40 2024-04-24T06:50:15.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.510+0000 7fd157602740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b39fe0e190 con 0x55b39feb7b30 2024-04-24T06:50:15.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.511+0000 7fd151e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39fead460 0x55b39feb75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:15.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.511+0000 7fd151e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39fead460 0x55b39feb75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36144/0 (socket says 172.21.15.5:36144) 2024-04-24T06:50:15.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.511+0000 7fd151e00640 1 -- 172.21.15.5:0/2460971642 learned_addr learned my addr 172.21.15.5:0/2460971642 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:15.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.511+0000 7fd152800640 1 --2- 172.21.15.5:0/2460971642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39feacb40 0x55b39feacf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:15.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.511+0000 7fd151400640 1 --2- 172.21.15.5:0/2460971642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39feb7b30 0x55b39feb9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:15.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.511+0000 7fd152800640 1 -- 172.21.15.5:0/2460971642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39feb7b30 msgr2=0x55b39feb9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:15.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.511+0000 7fd152800640 1 --2- 172.21.15.5:0/2460971642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39feb7b30 0x55b39feb9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.511+0000 7fd152800640 1 -- 172.21.15.5:0/2460971642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39fead460 msgr2=0x55b39feb75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:15.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.511+0000 7fd152800640 1 --2- 172.21.15.5:0/2460971642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39fead460 0x55b39feb75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.511+0000 7fd152800640 1 -- 172.21.15.5:0/2460971642 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b39fe75310 con 0x55b39feacb40 2024-04-24T06:50:15.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.512+0000 7fd152800640 1 --2- 172.21.15.5:0/2460971642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39feacb40 0x55b39feacf20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd148009f00 tx=0x7fd148011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4d1f6ffb61a3343 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:15.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.512+0000 7fd146a00640 1 --2- 172.21.15.5:0/3513631313 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fd1340a3da0 0x7fd1340a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:15.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.512+0000 7fd146a00640 1 -- 172.21.15.5:0/3513631313 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fd13c0f10a0 con 0x7fd14c103900 2024-04-24T06:50:15.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.512+0000 7fd13b000640 1 -- 172.21.15.5:0/2460971642 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd148004d20 con 0x55b39feacb40 2024-04-24T06:50:15.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.512+0000 7fd13b000640 1 -- 172.21.15.5:0/2460971642 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd148004050 con 0x55b39feacb40 2024-04-24T06:50:15.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.512+0000 7fd13b000640 1 -- 172.21.15.5:0/2460971642 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd148005220 con 0x55b39feacb40 2024-04-24T06:50:15.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.512+0000 7fd151e00640 1 --2- 172.21.15.5:0/3513631313 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fd1340a3da0 0x7fd1340a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:15.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.513+0000 7fd151e00640 1 --2- 172.21.15.5:0/3513631313 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fd1340a3da0 0x7fd1340a6260 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x55b39feae100 tx=0x7fd140005f90 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:15.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.513+0000 7fd157602740 1 -- 172.21.15.5:0/2460971642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39feacb40 msgr2=0x55b39feacf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:15.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.513+0000 7fd157602740 1 --2- 172.21.15.5:0/2460971642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39feacb40 0x55b39feacf20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd148009f00 tx=0x7fd148011810 comp rx=0 tx=0).stop 2024-04-24T06:50:15.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.513+0000 7fd157602740 1 -- 172.21.15.5:0/2460971642 shutdown_connections 2024-04-24T06:50:15.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.513+0000 7fd157602740 1 --2- 172.21.15.5:0/2460971642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39feb7b30 0x55b39feb9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.513+0000 7fd157602740 1 --2- 172.21.15.5:0/2460971642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39fead460 0x55b39feb75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.513+0000 7fd157602740 1 --2- 172.21.15.5:0/2460971642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39feacb40 0x55b39feacf20 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.513+0000 7fd157602740 1 -- 172.21.15.5:0/2460971642 >> 172.21.15.5:0/2460971642 conn(0x55b39fea8e60 msgr2=0x55b39fea9240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:15.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.513+0000 7fd157602740 1 -- 172.21.15.5:0/2460971642 shutdown_connections 2024-04-24T06:50:15.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.513+0000 7fd157602740 1 -- 172.21.15.5:0/2460971642 wait complete. 2024-04-24T06:50:15.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.513+0000 7fd157602740 1 -- start start 2024-04-24T06:50:15.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.514+0000 7fd157602740 1 Processor -- start 2024-04-24T06:50:15.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.514+0000 7fd157602740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39feacb40 0x55b39feb75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:15.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.514+0000 7fd157602740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39fead460 0x55b39feb48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:15.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.514+0000 7fd157602740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39feb7b30 0x55b39feb5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:15.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.514+0000 7fd157602740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b39feba9c0 con 0x55b39feacb40 2024-04-24T06:50:15.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.514+0000 7fd157602740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b39fdcb810 con 0x55b39fead460 2024-04-24T06:50:15.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.514+0000 7fd157602740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b39febab40 con 0x55b39feb7b30 2024-04-24T06:50:15.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.514+0000 7fd157602740 10 client.0 _ll_register_callbacks cb 0x55b39fd99140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:50:15.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.514+0000 7fd151e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39feacb40 0x55b39feb75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:15.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.514+0000 7fd151400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39fead460 0x55b39feb48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:15.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.514+0000 7fd151e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39feacb40 0x55b39feb75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36146/0 (socket says 172.21.15.5:36146) 2024-04-24T06:50:15.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.514+0000 7fd151e00640 1 -- 172.21.15.5:0/4198414979 learned_addr learned my addr 172.21.15.5:0/4198414979 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:15.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.515+0000 7fd152800640 1 --2- 172.21.15.5:0/4198414979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39feb7b30 0x55b39feb5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:15.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.515+0000 7fd151400640 1 -- 172.21.15.5:0/4198414979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39feb7b30 msgr2=0x55b39feb5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:15.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.515+0000 7fd151400640 1 --2- 172.21.15.5:0/4198414979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39feb7b30 0x55b39feb5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.515+0000 7fd151400640 1 -- 172.21.15.5:0/4198414979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39feacb40 msgr2=0x55b39feb75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:15.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.515+0000 7fd151400640 1 --2- 172.21.15.5:0/4198414979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39feacb40 0x55b39feb75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.515+0000 7fd151400640 1 -- 172.21.15.5:0/4198414979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b39fddc3f0 con 0x55b39fead460 2024-04-24T06:50:15.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.515+0000 7fd151e00640 1 --2- 172.21.15.5:0/4198414979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39feacb40 0x55b39feb75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:50:15.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.515+0000 7fd151400640 1 --2- 172.21.15.5:0/4198414979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39fead460 0x55b39feb48d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd13c0ee7a0 tx=0x7fd13c005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41fc67cc3c14af40 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:15.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.515+0000 7fd12f400640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:50:15.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.515+0000 7fd12f400640 1 -- 172.21.15.5:0/4198414979 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13c049070 con 0x55b39fead460 2024-04-24T06:50:15.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.515+0000 7fd157602740 1 -- 172.21.15.5:0/4198414979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd148004050 con 0x55b39fead460 2024-04-24T06:50:15.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.516+0000 7fd12f400640 1 -- 172.21.15.5:0/4198414979 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fd13c0f4cc0 con 0x55b39fead460 2024-04-24T06:50:15.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.516+0000 7fd12f400640 1 -- 172.21.15.5:0/4198414979 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd13c0b9340 con 0x55b39fead460 2024-04-24T06:50:15.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.516+0000 7fd157602740 10 client.26597 fetch_fsmap learned FSMap version 27 2024-04-24T06:50:15.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.516+0000 7fd157602740 1 -- 172.21.15.5:0/4198414979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b39fb243f0 con 0x55b39fead460 2024-04-24T06:50:15.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.516+0000 7fd12f400640 1 -- 172.21.15.5:0/4198414979 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd148004050 con 0x55b39fead460 2024-04-24T06:50:15.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.516+0000 7fd12f400640 1 -- 172.21.15.5:0/4198414979 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7fd13c0f3110 con 0x55b39fead460 2024-04-24T06:50:15.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.516+0000 7fd157602740 10 client.26597 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:50:15.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.516+0000 7fd157602740 10 client.26597 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-24T06:50:15.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.516+0000 7fd157602740 20 client.26597 populate_metadata read hostname 'smithi005' 2024-04-24T06:50:15.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.516+0000 7fd157602740 1 --2- 172.21.15.5:0/4198414979 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55b39fec7c40 0x55b39feca090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:15.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.516+0000 7fd157602740 4 client.26597 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:50:15.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.516+0000 7fd157602740 1 -- 172.21.15.5:0/4198414979 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b39fe1fbf0 con 0x55b39fec7c40 2024-04-24T06:50:15.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.517+0000 7fd151e00640 1 --2- 172.21.15.5:0/4198414979 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55b39fec7c40 0x55b39feca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:15.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.517+0000 7fd151e00640 1 --2- 172.21.15.5:0/4198414979 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55b39fec7c40 0x55b39feca090 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6dc792c3dbcc7182 server_cookie=2c352ee57e5faf7c in_seq=0 out_seq=0 2024-04-24T06:50:15.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.517+0000 7fd12f400640 10 client.26597 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:50:15.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.518+0000 7fd12f400640 1 -- 172.21.15.5:0/4198414979 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b39fe1fbf0 con 0x55b39fec7c40 2024-04-24T06:50:15.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.518+0000 7fd12f400640 10 client.26597 handle_command_reply: tid=0 2024-04-24T06:50:15.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.518+0000 7fd157602740 1 client.26597 shutdown 2024-04-24T06:50:15.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.518+0000 7fd157602740 1 -- 172.21.15.5:0/4198414979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39fead460 msgr2=0x55b39feb48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:15.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.518+0000 7fd157602740 1 --2- 172.21.15.5:0/4198414979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39fead460 0x55b39feb48d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd13c0ee7a0 tx=0x7fd13c005b50 comp rx=0 tx=0).stop 2024-04-24T06:50:15.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.518+0000 7fd157602740 1 -- 172.21.15.5:0/4198414979 shutdown_connections 2024-04-24T06:50:15.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.518+0000 7fd157602740 1 --2- 172.21.15.5:0/4198414979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39feb7b30 0x55b39feb5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.518+0000 7fd157602740 1 --2- 172.21.15.5:0/4198414979 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55b39fec7c40 0x55b39feca090 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.518+0000 7fd157602740 1 --2- 172.21.15.5:0/4198414979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39fead460 0x55b39feb48d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.518+0000 7fd157602740 1 --2- 172.21.15.5:0/4198414979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39feacb40 0x55b39feb75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.518+0000 7fd12f400640 0 client.26597 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:50:15.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.518+0000 7fd157602740 1 -- 172.21.15.5:0/4198414979 >> 172.21.15.5:0/4198414979 conn(0x55b39fea8e60 msgr2=0x55b39feaeb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:15.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.519+0000 7fd157602740 1 -- 172.21.15.5:0/4198414979 shutdown_connections 2024-04-24T06:50:15.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.519+0000 7fd157602740 1 -- 172.21.15.5:0/4198414979 wait complete. 2024-04-24T06:50:15.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.519+0000 7fd157602740 20 client.26597 trim_cache size 0 max 16384 2024-04-24T06:50:15.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.534+0000 7fd157602740 1 Processor -- start 2024-04-24T06:50:15.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.534+0000 7fd157602740 1 -- start start 2024-04-24T06:50:15.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.534+0000 7fd157602740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39feacb40 0x55b39fec3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:15.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.534+0000 7fd157602740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39fead460 0x55b39feb9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:15.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.534+0000 7fd157602740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39feb7b30 0x55b39feb7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:15.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.534+0000 7fd157602740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd14c1b2660 con 0x55b39fead460 2024-04-24T06:50:15.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.534+0000 7fd157602740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd14c1aff30 con 0x55b39feacb40 2024-04-24T06:50:15.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.534+0000 7fd157602740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd14c1add80 con 0x55b39feb7b30 2024-04-24T06:50:15.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.535+0000 7fd151400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39fead460 0x55b39feb9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:15.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.535+0000 7fd151400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39fead460 0x55b39feb9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36152/0 (socket says 172.21.15.5:36152) 2024-04-24T06:50:15.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.535+0000 7fd151400640 1 -- 172.21.15.5:0/4017740230 learned_addr learned my addr 172.21.15.5:0/4017740230 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:15.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.535+0000 7fd152800640 1 --2- 172.21.15.5:0/4017740230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39feb7b30 0x55b39feb7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:15.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.535+0000 7fd151e00640 1 --2- 172.21.15.5:0/4017740230 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39feacb40 0x55b39fec3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:15.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.535+0000 7fd151400640 1 -- 172.21.15.5:0/4017740230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39feb7b30 msgr2=0x55b39feb7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:15.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.535+0000 7fd151400640 1 --2- 172.21.15.5:0/4017740230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39feb7b30 0x55b39feb7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.535+0000 7fd151400640 1 -- 172.21.15.5:0/4017740230 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39feacb40 msgr2=0x55b39fec3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:15.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.535+0000 7fd151400640 1 --2- 172.21.15.5:0/4017740230 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39feacb40 0x55b39fec3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.535+0000 7fd151400640 1 -- 172.21.15.5:0/4017740230 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b39fbee7d0 con 0x55b39fead460 2024-04-24T06:50:15.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.535+0000 7fd151400640 1 --2- 172.21.15.5:0/4017740230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39fead460 0x55b39feb9e70 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd13c0bcda0 tx=0x7fd13c03b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1426e4da3a5a4cfc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:15.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.535+0000 7fd13a600640 1 -- 172.21.15.5:0/4017740230 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13c0db070 con 0x55b39fead460 2024-04-24T06:50:15.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.536+0000 7fd13a600640 1 -- 172.21.15.5:0/4017740230 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd13c0c5c60 con 0x55b39fead460 2024-04-24T06:50:15.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.536+0000 7fd13a600640 1 -- 172.21.15.5:0/4017740230 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13c049050 con 0x55b39fead460 2024-04-24T06:50:15.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.536+0000 7fd157602740 1 -- 172.21.15.5:0/4017740230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39fead460 msgr2=0x55b39feb9e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:15.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.536+0000 7fd157602740 1 --2- 172.21.15.5:0/4017740230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39fead460 0x55b39feb9e70 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd13c0bcda0 tx=0x7fd13c03b720 comp rx=0 tx=0).stop 2024-04-24T06:50:15.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.536+0000 7fd157602740 1 -- 172.21.15.5:0/4017740230 shutdown_connections 2024-04-24T06:50:15.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.536+0000 7fd157602740 1 --2- 172.21.15.5:0/4017740230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39feb7b30 0x55b39feb7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.536+0000 7fd157602740 1 --2- 172.21.15.5:0/4017740230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39fead460 0x55b39feb9e70 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.536+0000 7fd157602740 1 --2- 172.21.15.5:0/4017740230 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39feacb40 0x55b39fec3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.536+0000 7fd157602740 1 -- 172.21.15.5:0/4017740230 >> 172.21.15.5:0/4017740230 conn(0x55b39fea8e60 msgr2=0x55b39feb31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:15.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.536+0000 7fd157602740 1 -- 172.21.15.5:0/4017740230 shutdown_connections 2024-04-24T06:50:15.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.536+0000 7fd157602740 1 -- 172.21.15.5:0/4017740230 wait complete. 2024-04-24T06:50:15.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.537+0000 7fd157602740 1 -- start start 2024-04-24T06:50:15.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.537+0000 7fd157602740 1 Processor -- start 2024-04-24T06:50:15.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.537+0000 7fd157602740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39feacb40 0x55b39feb0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:15.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.537+0000 7fd157602740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39fead460 0x55b39feb10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:15.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.537+0000 7fd157602740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39feb7b30 0x55b39feb1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:15.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.537+0000 7fd157602740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd14c1ae3f0 con 0x55b39fead460 2024-04-24T06:50:15.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.537+0000 7fd157602740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd14c1af860 con 0x55b39feb7b30 2024-04-24T06:50:15.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.537+0000 7fd157602740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd14c1aeac0 con 0x55b39feacb40 2024-04-24T06:50:15.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.538+0000 7fd157602740 10 client.0 _ll_register_callbacks cb 0x7fd13c009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:50:15.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.538+0000 7fd151400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39fead460 0x55b39feb10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:15.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.538+0000 7fd151400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39fead460 0x55b39feb10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36156/0 (socket says 172.21.15.5:36156) 2024-04-24T06:50:15.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.538+0000 7fd151400640 1 -- 172.21.15.5:0/4118515096 learned_addr learned my addr 172.21.15.5:0/4118515096 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:15.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.538+0000 7fd151e00640 1 --2- 172.21.15.5:0/4118515096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39feacb40 0x55b39feb0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:15.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.538+0000 7fd152800640 1 --2- 172.21.15.5:0/4118515096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39feb7b30 0x55b39feb1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:15.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.538+0000 7fd151e00640 1 -- 172.21.15.5:0/4118515096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39feb7b30 msgr2=0x55b39feb1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:15.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.538+0000 7fd151e00640 1 --2- 172.21.15.5:0/4118515096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39feb7b30 0x55b39feb1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.538+0000 7fd151e00640 1 -- 172.21.15.5:0/4118515096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39fead460 msgr2=0x55b39feb10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:15.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.538+0000 7fd151e00640 1 --2- 172.21.15.5:0/4118515096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39fead460 0x55b39feb10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.538+0000 7fd151e00640 1 -- 172.21.15.5:0/4118515096 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b39febee90 con 0x55b39feacb40 2024-04-24T06:50:15.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.538+0000 7fd152800640 1 --2- 172.21.15.5:0/4118515096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39feb7b30 0x55b39feb1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:50:15.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.538+0000 7fd151400640 1 --2- 172.21.15.5:0/4118515096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39fead460 0x55b39feb10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:50:15.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.539+0000 7fd151e00640 1 --2- 172.21.15.5:0/4118515096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39feacb40 0x55b39feb0ba0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd13c0bb350 tx=0x7fd1400046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=734dc935a274487b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:15.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.539+0000 7fd12f400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:50:15.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.539+0000 7fd12f400640 1 -- 172.21.15.5:0/4118515096 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14002e040 con 0x55b39feacb40 2024-04-24T06:50:15.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.539+0000 7fd157602740 1 -- 172.21.15.5:0/4118515096 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd13c0c5c60 con 0x55b39feacb40 2024-04-24T06:50:15.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.539+0000 7fd12f400640 1 -- 172.21.15.5:0/4118515096 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fd14002e1e0 con 0x55b39feacb40 2024-04-24T06:50:15.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.539+0000 7fd12f400640 1 -- 172.21.15.5:0/4118515096 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd14002e3e0 con 0x55b39feacb40 2024-04-24T06:50:15.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.539+0000 7fd157602740 10 client.26611 fetch_fsmap learned FSMap version 27 2024-04-24T06:50:15.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.539+0000 7fd157602740 1 -- 172.21.15.5:0/4118515096 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b39fec0170 con 0x55b39feacb40 2024-04-24T06:50:15.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.539+0000 7fd12f400640 1 -- 172.21.15.5:0/4118515096 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd13c0c5c60 con 0x55b39feacb40 2024-04-24T06:50:15.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.540+0000 7fd12f400640 1 -- 172.21.15.5:0/4118515096 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7fd140039cb0 con 0x55b39feacb40 2024-04-24T06:50:15.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.540+0000 7fd157602740 10 client.26611 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:50:15.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.540+0000 7fd157602740 10 client.26611 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-24T06:50:15.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.540+0000 7fd157602740 20 client.26611 populate_metadata read hostname 'smithi005' 2024-04-24T06:50:15.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.540+0000 7fd157602740 1 --2- 172.21.15.5:0/4118515096 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55b39ff2e1d0 0x55b39ff305f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:15.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.540+0000 7fd157602740 4 client.26611 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:50:15.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.540+0000 7fd157602740 1 -- 172.21.15.5:0/4118515096 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b39fe19a60 con 0x55b39ff2e1d0 2024-04-24T06:50:15.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.540+0000 7fd151400640 1 --2- 172.21.15.5:0/4118515096 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55b39ff2e1d0 0x55b39ff305f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:15.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.540+0000 7fd151400640 1 --2- 172.21.15.5:0/4118515096 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55b39ff2e1d0 0x55b39ff305f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=72d0df6912b50d5e server_cookie=4cc47248b74577cc in_seq=0 out_seq=0 2024-04-24T06:50:15.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.540+0000 7fd12f400640 10 client.26611 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:50:15.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.541+0000 7fd12f400640 1 -- 172.21.15.5:0/4118515096 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b39fe19a60 con 0x55b39ff2e1d0 2024-04-24T06:50:15.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.541+0000 7fd12f400640 10 client.26611 handle_command_reply: tid=0 2024-04-24T06:50:15.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.541+0000 7fd157602740 1 client.26611 shutdown 2024-04-24T06:50:15.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.541+0000 7fd157602740 1 -- 172.21.15.5:0/4118515096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39feacb40 msgr2=0x55b39feb0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:15.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.541+0000 7fd157602740 1 --2- 172.21.15.5:0/4118515096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39feacb40 0x55b39feb0ba0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd13c0bb350 tx=0x7fd1400046c0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.541+0000 7fd157602740 1 -- 172.21.15.5:0/4118515096 shutdown_connections 2024-04-24T06:50:15.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.541+0000 7fd157602740 1 --2- 172.21.15.5:0/4118515096 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55b39ff2e1d0 0x55b39ff305f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.541+0000 7fd157602740 1 --2- 172.21.15.5:0/4118515096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b39feb7b30 0x55b39feb1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.541+0000 7fd157602740 1 --2- 172.21.15.5:0/4118515096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b39fead460 0x55b39feb10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.541+0000 7fd12f400640 0 client.26611 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:50:15.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.541+0000 7fd157602740 1 --2- 172.21.15.5:0/4118515096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39feacb40 0x55b39feb0ba0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.541+0000 7fd157602740 1 -- 172.21.15.5:0/4118515096 >> 172.21.15.5:0/4118515096 conn(0x55b39fea8e60 msgr2=0x55b39fea76c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:15.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.541+0000 7fd157602740 1 -- 172.21.15.5:0/4118515096 shutdown_connections 2024-04-24T06:50:15.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.541+0000 7fd157602740 1 -- 172.21.15.5:0/4118515096 wait complete. 2024-04-24T06:50:15.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.541+0000 7fd157602740 20 client.26611 trim_cache size 0 max 16384 2024-04-24T06:50:15.552 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:50:15.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.542+0000 7fd153200640 1 -- 172.21.15.5:0/3513631313 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fd1340a3da0 msgr2=0x7fd1340a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:15.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.542+0000 7fd153200640 1 --2- 172.21.15.5:0/3513631313 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fd1340a3da0 0x7fd1340a6260 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x55b39feae100 tx=0x7fd140005f90 comp rx=0 tx=0).stop 2024-04-24T06:50:15.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.542+0000 7fd153200640 1 -- 172.21.15.5:0/3513631313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd14c103900 msgr2=0x7fd14c1a0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:15.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.542+0000 7fd153200640 1 --2- 172.21.15.5:0/3513631313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd14c103900 0x7fd14c1a0b70 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd13c02a9a0 tx=0x7fd13c003b20 comp rx=0 tx=0).stop 2024-04-24T06:50:15.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.542+0000 7fd153200640 1 -- 172.21.15.5:0/3513631313 shutdown_connections 2024-04-24T06:50:15.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.542+0000 7fd153200640 1 --2- 172.21.15.5:0/3513631313 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fd1340a3da0 0x7fd1340a6260 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.542+0000 7fd153200640 1 --2- 172.21.15.5:0/3513631313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd14c10c400 0x7fd14c19a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.542+0000 7fd153200640 1 --2- 172.21.15.5:0/3513631313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd14c103900 0x7fd14c1a0b70 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.542+0000 7fd153200640 1 --2- 172.21.15.5:0/3513631313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd14c102fe0 0x7fd14c1a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.542+0000 7fd153200640 1 -- 172.21.15.5:0/3513631313 >> 172.21.15.5:0/3513631313 conn(0x7fd14c077ab0 msgr2=0x7fd14c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:15.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.542+0000 7fd153200640 1 -- 172.21.15.5:0/3513631313 shutdown_connections 2024-04-24T06:50:15.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.542+0000 7fd153200640 1 -- 172.21.15.5:0/3513631313 wait complete. 2024-04-24T06:50:15.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:15 smithi005 ceph-mon[27798]: pgmap v189: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-24T06:50:15.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:15 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/3232981574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:50:15.561 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T06:50:15.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:15 smithi137 ceph-mon[33017]: pgmap v189: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-24T06:50:15.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:15 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/3232981574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:50:15.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.765+0000 7fb13d800640 1 -- 172.21.15.5:0/1369218747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb138103900 msgr2=0x7fb13810bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:15.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.765+0000 7fb13d800640 1 --2- 172.21.15.5:0/1369218747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb138103900 0x7fb13810bde0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb120009be0 tx=0x7fb12002a750 comp rx=0 tx=0).stop 2024-04-24T06:50:15.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.765+0000 7fb13d800640 1 -- 172.21.15.5:0/1369218747 shutdown_connections 2024-04-24T06:50:15.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.765+0000 7fb13d800640 1 --2- 172.21.15.5:0/1369218747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb13810c400 0x7fb13810e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.765+0000 7fb13d800640 1 --2- 172.21.15.5:0/1369218747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb138103900 0x7fb13810bde0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.765+0000 7fb13d800640 1 --2- 172.21.15.5:0/1369218747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb138102fe0 0x7fb1381033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.765+0000 7fb13d800640 1 -- 172.21.15.5:0/1369218747 >> 172.21.15.5:0/1369218747 conn(0x7fb138077ab0 msgr2=0x7fb1380fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:15.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.765+0000 7fb13d800640 1 -- 172.21.15.5:0/1369218747 shutdown_connections 2024-04-24T06:50:15.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.765+0000 7fb13d800640 1 -- 172.21.15.5:0/1369218747 wait complete. 2024-04-24T06:50:15.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:15 smithi037 ceph-mon[32864]: pgmap v189: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-24T06:50:15.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:15 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/3232981574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:50:15.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.766+0000 7fb13d800640 1 Processor -- start 2024-04-24T06:50:15.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.766+0000 7fb13d800640 1 -- start start 2024-04-24T06:50:15.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.767+0000 7fb13d800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb138102fe0 0x7fb138100c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:15.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.767+0000 7fb13d800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb138103900 0x7fb138101170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:15.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.767+0000 7fb13d800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb13810c400 0x7fb1381016b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:15.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.767+0000 7fb13d800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb138112160 con 0x7fb13810c400 2024-04-24T06:50:15.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.767+0000 7fb13d800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb138111fe0 con 0x7fb138102fe0 2024-04-24T06:50:15.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.767+0000 7fb13d800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb1381122e0 con 0x7fb138103900 2024-04-24T06:50:15.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.767+0000 7fb13ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb13810c400 0x7fb1381016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:15.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.767+0000 7fb13ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb13810c400 0x7fb1381016b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36180/0 (socket says 172.21.15.5:36180) 2024-04-24T06:50:15.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.767+0000 7fb13ce00640 1 -- 172.21.15.5:0/2085644797 learned_addr learned my addr 172.21.15.5:0/2085644797 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:15.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.767+0000 7fb137400640 1 --2- 172.21.15.5:0/2085644797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb138103900 0x7fb138101170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:15.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.768+0000 7fb137e00640 1 --2- 172.21.15.5:0/2085644797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb138102fe0 0x7fb138100c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:15.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.768+0000 7fb137400640 1 -- 172.21.15.5:0/2085644797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb138102fe0 msgr2=0x7fb138100c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:15.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.768+0000 7fb137400640 1 --2- 172.21.15.5:0/2085644797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb138102fe0 0x7fb138100c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.768+0000 7fb137400640 1 -- 172.21.15.5:0/2085644797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb13810c400 msgr2=0x7fb1381016b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:15.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.768+0000 7fb137400640 1 --2- 172.21.15.5:0/2085644797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb13810c400 0x7fb1381016b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.768+0000 7fb137400640 1 -- 172.21.15.5:0/2085644797 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1381a6ca0 con 0x7fb138103900 2024-04-24T06:50:15.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.768+0000 7fb13ce00640 1 --2- 172.21.15.5:0/2085644797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb13810c400 0x7fb1381016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:50:15.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.768+0000 7fb137400640 1 --2- 172.21.15.5:0/2085644797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb138103900 0x7fb138101170 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb120034040 tx=0x7fb1200305d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:15.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.769+0000 7fb134c00640 1 -- 172.21.15.5:0/2085644797 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb120041020 con 0x7fb138103900 2024-04-24T06:50:15.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.769+0000 7fb134c00640 1 -- 172.21.15.5:0/2085644797 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb120037040 con 0x7fb138103900 2024-04-24T06:50:15.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.769+0000 7fb13d800640 1 -- 172.21.15.5:0/2085644797 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1381a6f30 con 0x7fb138103900 2024-04-24T06:50:15.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.769+0000 7fb134c00640 1 -- 172.21.15.5:0/2085644797 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb120036070 con 0x7fb138103900 2024-04-24T06:50:15.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.769+0000 7fb13d800640 1 -- 172.21.15.5:0/2085644797 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1381a73a0 con 0x7fb138103900 2024-04-24T06:50:15.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.772+0000 7fb141c06740 1 Processor -- start 2024-04-24T06:50:15.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.772+0000 7fb134c00640 1 -- 172.21.15.5:0/2085644797 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fb120044020 con 0x7fb138103900 2024-04-24T06:50:15.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.772+0000 7fb141c06740 1 -- start start 2024-04-24T06:50:15.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.773+0000 7fb141c06740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1f16b9b40 0x55a1f16b9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:15.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.773+0000 7fb141c06740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1f16ba460 0x55a1f16c45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:15.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.773+0000 7fb141c06740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1f16c4b30 0x55a1f16c6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:15.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.773+0000 7fb141c06740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a1f161f1d0 con 0x55a1f16b9b40 2024-04-24T06:50:15.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.773+0000 7fb141c06740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a1f15c5450 con 0x55a1f16ba460 2024-04-24T06:50:15.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.773+0000 7fb141c06740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a1f161b190 con 0x55a1f16c4b30 2024-04-24T06:50:15.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.773+0000 7fb13ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1f16b9b40 0x55a1f16b9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:15.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.773+0000 7fb137e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1f16ba460 0x55a1f16c45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:15.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.773+0000 7fb137e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1f16ba460 0x55a1f16c45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:33354/0 (socket says 172.21.15.5:33354) 2024-04-24T06:50:15.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.773+0000 7fb137e00640 1 -- 172.21.15.5:0/2193164261 learned_addr learned my addr 172.21.15.5:0/2193164261 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:15.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.773+0000 7fb137400640 1 --2- 172.21.15.5:0/2193164261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1f16c4b30 0x55a1f16c6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:15.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.774+0000 7fb137e00640 1 -- 172.21.15.5:0/2193164261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1f16c4b30 msgr2=0x55a1f16c6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:15.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.774+0000 7fb137e00640 1 --2- 172.21.15.5:0/2193164261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1f16c4b30 0x55a1f16c6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.774+0000 7fb137e00640 1 -- 172.21.15.5:0/2193164261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1f16b9b40 msgr2=0x55a1f16b9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:15.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.774+0000 7fb137e00640 1 --2- 172.21.15.5:0/2193164261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1f16b9b40 0x55a1f16b9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.774+0000 7fb137e00640 1 -- 172.21.15.5:0/2193164261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1f1682310 con 0x55a1f16ba460 2024-04-24T06:50:15.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.774+0000 7fb137e00640 1 --2- 172.21.15.5:0/2193164261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1f16ba460 0x55a1f16c45f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb128009cc0 tx=0x7fb128011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce0bd4871d2fe7dd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:15.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.774+0000 7fb134c00640 1 --2- 172.21.15.5:0/2085644797 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb1180a3da0 0x7fb1180a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:15.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.774+0000 7fb12d600640 1 -- 172.21.15.5:0/2193164261 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb128005220 con 0x55a1f16ba460 2024-04-24T06:50:15.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.774+0000 7fb12d600640 1 -- 172.21.15.5:0/2193164261 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb128004c80 con 0x55a1f16ba460 2024-04-24T06:50:15.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.774+0000 7fb12d600640 1 -- 172.21.15.5:0/2193164261 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1280197d0 con 0x55a1f16ba460 2024-04-24T06:50:15.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.774+0000 7fb134c00640 1 -- 172.21.15.5:0/2085644797 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fb1200f1150 con 0x7fb138103900 2024-04-24T06:50:15.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.774+0000 7fb13ce00640 1 --2- 172.21.15.5:0/2085644797 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb1180a3da0 0x7fb1180a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:15.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.775+0000 7fb141c06740 1 -- 172.21.15.5:0/2193164261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1f16ba460 msgr2=0x55a1f16c45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:15.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.775+0000 7fb141c06740 1 --2- 172.21.15.5:0/2193164261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1f16ba460 0x55a1f16c45f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb128009cc0 tx=0x7fb128011810 comp rx=0 tx=0).stop 2024-04-24T06:50:15.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.775+0000 7fb141c06740 1 -- 172.21.15.5:0/2193164261 shutdown_connections 2024-04-24T06:50:15.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.775+0000 7fb141c06740 1 --2- 172.21.15.5:0/2193164261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1f16c4b30 0x55a1f16c6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.775+0000 7fb13ce00640 1 --2- 172.21.15.5:0/2085644797 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb1180a3da0 0x7fb1180a6260 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb130011630 tx=0x7fb13000b390 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:15.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.775+0000 7fb141c06740 1 --2- 172.21.15.5:0/2193164261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1f16ba460 0x55a1f16c45f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.775+0000 7fb141c06740 1 --2- 172.21.15.5:0/2193164261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1f16b9b40 0x55a1f16b9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.775+0000 7fb141c06740 1 -- 172.21.15.5:0/2193164261 >> 172.21.15.5:0/2193164261 conn(0x55a1f16b5e60 msgr2=0x55a1f16b6240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:15.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.775+0000 7fb141c06740 1 -- 172.21.15.5:0/2193164261 shutdown_connections 2024-04-24T06:50:15.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.775+0000 7fb141c06740 1 -- 172.21.15.5:0/2193164261 wait complete. 2024-04-24T06:50:15.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.776+0000 7fb141c06740 1 -- start start 2024-04-24T06:50:15.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.776+0000 7fb141c06740 1 Processor -- start 2024-04-24T06:50:15.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.777+0000 7fb141c06740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1f16b9b40 0x55a1f16c45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:15.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.777+0000 7fb141c06740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1f16ba460 0x55a1f16c18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:15.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.777+0000 7fb141c06740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1f16c4b30 0x55a1f16c2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:15.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.777+0000 7fb141c06740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a1f16c79c0 con 0x55a1f16b9b40 2024-04-24T06:50:15.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.777+0000 7fb141c06740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a1f15d8810 con 0x55a1f16c4b30 2024-04-24T06:50:15.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.777+0000 7fb141c06740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a1f16c7b40 con 0x55a1f16ba460 2024-04-24T06:50:15.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.777+0000 7fb141c06740 10 client.0 _ll_register_callbacks cb 0x55a1f15a6140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:50:15.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.777+0000 7fb137e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1f16b9b40 0x55a1f16c45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:15.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.777+0000 7fb137e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1f16b9b40 0x55a1f16c45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36202/0 (socket says 172.21.15.5:36202) 2024-04-24T06:50:15.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.777+0000 7fb137e00640 1 -- 172.21.15.5:0/1081223076 learned_addr learned my addr 172.21.15.5:0/1081223076 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:15.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.777+0000 7fb137400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1f16ba460 0x55a1f16c18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:15.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.777+0000 7fb13ce00640 1 --2- 172.21.15.5:0/1081223076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1f16c4b30 0x55a1f16c2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:15.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.777+0000 7fb137e00640 1 -- 172.21.15.5:0/1081223076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1f16ba460 msgr2=0x55a1f16c18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:15.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.777+0000 7fb137e00640 1 --2- 172.21.15.5:0/1081223076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1f16ba460 0x55a1f16c18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.777+0000 7fb137e00640 1 -- 172.21.15.5:0/1081223076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1f16c4b30 msgr2=0x55a1f16c2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:15.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.777+0000 7fb137e00640 1 --2- 172.21.15.5:0/1081223076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1f16c4b30 0x55a1f16c2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.777+0000 7fb137e00640 1 -- 172.21.15.5:0/1081223076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1f15e93f0 con 0x55a1f16b9b40 2024-04-24T06:50:15.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.778+0000 7fb137e00640 1 --2- 172.21.15.5:0/1081223076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1f16b9b40 0x55a1f16c45e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb128009c90 tx=0x7fb1280048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a2e496b2116a99 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:15.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.778+0000 7fb126000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:50:15.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.778+0000 7fb126000640 1 -- 172.21.15.5:0/1081223076 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb128029070 con 0x55a1f16b9b40 2024-04-24T06:50:15.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.778+0000 7fb126000640 1 -- 172.21.15.5:0/1081223076 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fb128019920 con 0x55a1f16b9b40 2024-04-24T06:50:15.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.778+0000 7fb141c06740 1 -- 172.21.15.5:0/1081223076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb128004c80 con 0x55a1f16b9b40 2024-04-24T06:50:15.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.778+0000 7fb126000640 1 -- 172.21.15.5:0/1081223076 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb128004ae0 con 0x55a1f16b9b40 2024-04-24T06:50:15.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.778+0000 7fb141c06740 10 client.16476 fetch_fsmap learned FSMap version 27 2024-04-24T06:50:15.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.778+0000 7fb141c06740 1 -- 172.21.15.5:0/1081223076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a1f13313f0 con 0x55a1f16b9b40 2024-04-24T06:50:15.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.779+0000 7fb126000640 1 -- 172.21.15.5:0/1081223076 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb128004c80 con 0x55a1f16b9b40 2024-04-24T06:50:15.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.779+0000 7fb126000640 1 -- 172.21.15.5:0/1081223076 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7fb128022900 con 0x55a1f16b9b40 2024-04-24T06:50:15.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.779+0000 7fb141c06740 10 client.16476 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:50:15.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.779+0000 7fb141c06740 10 client.16476 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T06:50:15.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.779+0000 7fb141c06740 20 client.16476 populate_metadata read hostname 'smithi005' 2024-04-24T06:50:15.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.779+0000 7fb141c06740 1 --2- 172.21.15.5:0/1081223076 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55a1f16d4c50 0x55a1f16d7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:15.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.779+0000 7fb141c06740 4 client.16476 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:50:15.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.779+0000 7fb141c06740 1 -- 172.21.15.5:0/1081223076 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1f162cbf0 con 0x55a1f16d4c50 2024-04-24T06:50:15.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.780+0000 7fb137400640 1 --2- 172.21.15.5:0/1081223076 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55a1f16d4c50 0x55a1f16d7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:15.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.780+0000 7fb137400640 1 --2- 172.21.15.5:0/1081223076 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55a1f16d4c50 0x55a1f16d7040 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8e6ea8274c0b2f90 server_cookie=710fc3743f94d965 in_seq=0 out_seq=0 2024-04-24T06:50:15.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.780+0000 7fb126000640 10 client.16476 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:50:15.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.781+0000 7fb126000640 1 -- 172.21.15.5:0/1081223076 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a1f162cbf0 con 0x55a1f16d4c50 2024-04-24T06:50:15.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.781+0000 7fb126000640 10 client.16476 handle_command_reply: tid=0 2024-04-24T06:50:15.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.781+0000 7fb141c06740 1 client.16476 shutdown 2024-04-24T06:50:15.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.781+0000 7fb141c06740 1 -- 172.21.15.5:0/1081223076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1f16b9b40 msgr2=0x55a1f16c45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:15.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.781+0000 7fb141c06740 1 --2- 172.21.15.5:0/1081223076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1f16b9b40 0x55a1f16c45e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb128009c90 tx=0x7fb1280048f0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.781+0000 7fb141c06740 1 -- 172.21.15.5:0/1081223076 shutdown_connections 2024-04-24T06:50:15.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.781+0000 7fb141c06740 1 --2- 172.21.15.5:0/1081223076 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55a1f16d4c50 0x55a1f16d7040 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.781+0000 7fb141c06740 1 --2- 172.21.15.5:0/1081223076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1f16c4b30 0x55a1f16c2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.781+0000 7fb141c06740 1 --2- 172.21.15.5:0/1081223076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1f16ba460 0x55a1f16c18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.781+0000 7fb141c06740 1 --2- 172.21.15.5:0/1081223076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1f16b9b40 0x55a1f16c45e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.781+0000 7fb126000640 0 client.16476 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:50:15.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.781+0000 7fb141c06740 1 -- 172.21.15.5:0/1081223076 >> 172.21.15.5:0/1081223076 conn(0x55a1f16b5e60 msgr2=0x55a1f16bbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:15.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.782+0000 7fb141c06740 1 -- 172.21.15.5:0/1081223076 shutdown_connections 2024-04-24T06:50:15.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.782+0000 7fb141c06740 1 -- 172.21.15.5:0/1081223076 wait complete. 2024-04-24T06:50:15.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.782+0000 7fb141c06740 20 client.16476 trim_cache size 0 max 16384 2024-04-24T06:50:15.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.797+0000 7fb141c06740 1 Processor -- start 2024-04-24T06:50:15.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.797+0000 7fb141c06740 1 -- start start 2024-04-24T06:50:15.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.797+0000 7fb141c06740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1f16b9b40 0x55a1f16d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:15.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.797+0000 7fb141c06740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1f16ba460 0x55a1f16c4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:15.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.797+0000 7fb141c06740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1f16c4b30 0x55a1f16dc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:15.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.797+0000 7fb141c06740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb1381b2640 con 0x55a1f16ba460 2024-04-24T06:50:15.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.797+0000 7fb141c06740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb1381aff10 con 0x55a1f16b9b40 2024-04-24T06:50:15.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.797+0000 7fb141c06740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb1381add60 con 0x55a1f16c4b30 2024-04-24T06:50:15.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.797+0000 7fb137400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1f16ba460 0x55a1f16c4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:15.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.798+0000 7fb137e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1f16b9b40 0x55a1f16d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:15.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.798+0000 7fb137e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1f16b9b40 0x55a1f16d0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:33374/0 (socket says 172.21.15.5:33374) 2024-04-24T06:50:15.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.798+0000 7fb137400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1f16ba460 0x55a1f16c4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36216/0 (socket says 172.21.15.5:36216) 2024-04-24T06:50:15.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.798+0000 7fb137e00640 1 -- 172.21.15.5:0/4043315949 learned_addr learned my addr 172.21.15.5:0/4043315949 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:15.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.798+0000 7fb13ce00640 1 --2- 172.21.15.5:0/4043315949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1f16c4b30 0x55a1f16dc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:15.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.798+0000 7fb137400640 1 -- 172.21.15.5:0/4043315949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1f16c4b30 msgr2=0x55a1f16dc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:15.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.798+0000 7fb137400640 1 --2- 172.21.15.5:0/4043315949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1f16c4b30 0x55a1f16dc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.798+0000 7fb137400640 1 -- 172.21.15.5:0/4043315949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1f16b9b40 msgr2=0x55a1f16d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:15.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.798+0000 7fb137400640 1 --2- 172.21.15.5:0/4043315949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1f16b9b40 0x55a1f16d0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.798+0000 7fb137400640 1 -- 172.21.15.5:0/4043315949 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1f16c2380 con 0x55a1f16ba460 2024-04-24T06:50:15.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.798+0000 7fb137400640 1 --2- 172.21.15.5:0/4043315949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1f16ba460 0x55a1f16c4110 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb1200ee8b0 tx=0x7fb1200f3660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f96a3d817633614 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:15.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.799+0000 7fb12cc00640 1 -- 172.21.15.5:0/4043315949 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1200bd800 con 0x55a1f16ba460 2024-04-24T06:50:15.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.799+0000 7fb12cc00640 1 -- 172.21.15.5:0/4043315949 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1200bd100 con 0x55a1f16ba460 2024-04-24T06:50:15.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.799+0000 7fb12cc00640 1 -- 172.21.15.5:0/4043315949 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1200b9770 con 0x55a1f16ba460 2024-04-24T06:50:15.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.799+0000 7fb141c06740 1 -- 172.21.15.5:0/4043315949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1f16ba460 msgr2=0x55a1f16c4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:15.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.799+0000 7fb141c06740 1 --2- 172.21.15.5:0/4043315949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1f16ba460 0x55a1f16c4110 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb1200ee8b0 tx=0x7fb1200f3660 comp rx=0 tx=0).stop 2024-04-24T06:50:15.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.799+0000 7fb141c06740 1 -- 172.21.15.5:0/4043315949 shutdown_connections 2024-04-24T06:50:15.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.799+0000 7fb141c06740 1 --2- 172.21.15.5:0/4043315949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1f16c4b30 0x55a1f16dc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.799+0000 7fb141c06740 1 --2- 172.21.15.5:0/4043315949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1f16ba460 0x55a1f16c4110 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.799+0000 7fb141c06740 1 --2- 172.21.15.5:0/4043315949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1f16b9b40 0x55a1f16d0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.799+0000 7fb141c06740 1 -- 172.21.15.5:0/4043315949 >> 172.21.15.5:0/4043315949 conn(0x55a1f16b5e60 msgr2=0x55a1f16bcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:15.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.799+0000 7fb141c06740 1 -- 172.21.15.5:0/4043315949 shutdown_connections 2024-04-24T06:50:15.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.799+0000 7fb141c06740 1 -- 172.21.15.5:0/4043315949 wait complete. 2024-04-24T06:50:15.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.800+0000 7fb141c06740 1 -- start start 2024-04-24T06:50:15.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.800+0000 7fb141c06740 1 Processor -- start 2024-04-24T06:50:15.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.801+0000 7fb141c06740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1f16b9b40 0x55a1f16bf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:15.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.801+0000 7fb141c06740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1f16ba460 0x55a1f16ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:15.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.801+0000 7fb141c06740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1f16c4b30 0x55a1f16c5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:15.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.801+0000 7fb141c06740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb1381ae3d0 con 0x55a1f16ba460 2024-04-24T06:50:15.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.801+0000 7fb141c06740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb1381af840 con 0x55a1f16c4b30 2024-04-24T06:50:15.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.801+0000 7fb141c06740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb1381aeaa0 con 0x55a1f16b9b40 2024-04-24T06:50:15.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.801+0000 7fb141c06740 10 client.0 _ll_register_callbacks cb 0x7fb13000bb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:50:15.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.801+0000 7fb137400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1f16ba460 0x55a1f16ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:15.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.801+0000 7fb137400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1f16ba460 0x55a1f16ccd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36226/0 (socket says 172.21.15.5:36226) 2024-04-24T06:50:15.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.801+0000 7fb137400640 1 -- 172.21.15.5:0/2391535657 learned_addr learned my addr 172.21.15.5:0/2391535657 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:15.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.801+0000 7fb137e00640 1 --2- 172.21.15.5:0/2391535657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1f16b9b40 0x55a1f16bf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:15.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.801+0000 7fb13ce00640 1 --2- 172.21.15.5:0/2391535657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1f16c4b30 0x55a1f16c5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:15.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.801+0000 7fb137400640 1 -- 172.21.15.5:0/2391535657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1f16b9b40 msgr2=0x55a1f16bf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:15.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.801+0000 7fb137400640 1 --2- 172.21.15.5:0/2391535657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1f16b9b40 0x55a1f16bf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.801+0000 7fb137400640 1 -- 172.21.15.5:0/2391535657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1f16c4b30 msgr2=0x55a1f16c5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:15.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.801+0000 7fb137400640 1 --2- 172.21.15.5:0/2391535657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1f16c4b30 0x55a1f16c5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.801+0000 7fb137400640 1 -- 172.21.15.5:0/2391535657 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1f13fb7d0 con 0x55a1f16ba460 2024-04-24T06:50:15.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.802+0000 7fb137400640 1 --2- 172.21.15.5:0/2391535657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1f16ba460 0x55a1f16ccd90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb1200bce00 tx=0x7fb1200c5d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3db327ae83188687 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:15.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.802+0000 7fb126000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:50:15.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.802+0000 7fb126000640 1 -- 172.21.15.5:0/2391535657 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1200c14b0 con 0x55a1f16ba460 2024-04-24T06:50:15.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.802+0000 7fb126000640 1 -- 172.21.15.5:0/2391535657 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fb1200eeb30 con 0x55a1f16ba460 2024-04-24T06:50:15.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.802+0000 7fb141c06740 1 -- 172.21.15.5:0/2391535657 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1200bd100 con 0x55a1f16ba460 2024-04-24T06:50:15.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.802+0000 7fb126000640 1 -- 172.21.15.5:0/2391535657 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb1200c1120 con 0x55a1f16ba460 2024-04-24T06:50:15.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.802+0000 7fb141c06740 10 client.16482 fetch_fsmap learned FSMap version 27 2024-04-24T06:50:15.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.802+0000 7fb141c06740 1 -- 172.21.15.5:0/2391535657 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a1f173b560 con 0x55a1f16ba460 2024-04-24T06:50:15.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.802+0000 7fb126000640 1 -- 172.21.15.5:0/2391535657 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1200bd100 con 0x55a1f16ba460 2024-04-24T06:50:15.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.802+0000 7fb126000640 1 -- 172.21.15.5:0/2391535657 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7fb1200d1ba0 con 0x55a1f16ba460 2024-04-24T06:50:15.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.803+0000 7fb141c06740 10 client.16482 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:50:15.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.803+0000 7fb141c06740 10 client.16482 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T06:50:15.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.803+0000 7fb141c06740 20 client.16482 populate_metadata read hostname 'smithi005' 2024-04-24T06:50:15.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.803+0000 7fb141c06740 1 --2- 172.21.15.5:0/2391535657 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55a1f173bba0 0x55a1f173dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:15.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.803+0000 7fb141c06740 4 client.16482 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:50:15.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.803+0000 7fb141c06740 1 -- 172.21.15.5:0/2391535657 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a1f1626a60 con 0x55a1f173bba0 2024-04-24T06:50:15.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.803+0000 7fb137e00640 1 --2- 172.21.15.5:0/2391535657 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55a1f173bba0 0x55a1f173dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:15.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.804+0000 7fb137e00640 1 --2- 172.21.15.5:0/2391535657 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55a1f173bba0 0x55a1f173dfc0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fb512f12b092286b server_cookie=dd2fb3a858bc9b94 in_seq=0 out_seq=0 2024-04-24T06:50:15.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.804+0000 7fb126000640 10 client.16482 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:50:15.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.804+0000 7fb126000640 1 -- 172.21.15.5:0/2391535657 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a1f1626a60 con 0x55a1f173bba0 2024-04-24T06:50:15.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.804+0000 7fb126000640 10 client.16482 handle_command_reply: tid=0 2024-04-24T06:50:15.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.804+0000 7fb141c06740 1 client.16482 shutdown 2024-04-24T06:50:15.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.804+0000 7fb141c06740 1 -- 172.21.15.5:0/2391535657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1f16ba460 msgr2=0x55a1f16ccd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:15.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.804+0000 7fb141c06740 1 --2- 172.21.15.5:0/2391535657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1f16ba460 0x55a1f16ccd90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb1200bce00 tx=0x7fb1200c5d70 comp rx=0 tx=0).stop 2024-04-24T06:50:15.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.804+0000 7fb141c06740 1 -- 172.21.15.5:0/2391535657 shutdown_connections 2024-04-24T06:50:15.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.804+0000 7fb141c06740 1 --2- 172.21.15.5:0/2391535657 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55a1f173bba0 0x55a1f173dfc0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.804+0000 7fb141c06740 1 --2- 172.21.15.5:0/2391535657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1f16c4b30 0x55a1f16c5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.804+0000 7fb141c06740 1 --2- 172.21.15.5:0/2391535657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1f16ba460 0x55a1f16ccd90 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.804+0000 7fb141c06740 1 --2- 172.21.15.5:0/2391535657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a1f16b9b40 0x55a1f16bf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.804+0000 7fb126000640 0 client.16482 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:50:15.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.804+0000 7fb141c06740 1 -- 172.21.15.5:0/2391535657 >> 172.21.15.5:0/2391535657 conn(0x55a1f16b5e60 msgr2=0x55a1f16b46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:15.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.804+0000 7fb141c06740 1 -- 172.21.15.5:0/2391535657 shutdown_connections 2024-04-24T06:50:15.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.804+0000 7fb141c06740 1 -- 172.21.15.5:0/2391535657 wait complete. 2024-04-24T06:50:15.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.805+0000 7fb141c06740 20 client.16482 trim_cache size 0 max 16384 2024-04-24T06:50:15.817 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:50:15.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.805+0000 7fb13d800640 1 -- 172.21.15.5:0/2085644797 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb1180a3da0 msgr2=0x7fb1180a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:15.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.805+0000 7fb13d800640 1 --2- 172.21.15.5:0/2085644797 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb1180a3da0 0x7fb1180a6260 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb130011630 tx=0x7fb13000b390 comp rx=0 tx=0).stop 2024-04-24T06:50:15.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.805+0000 7fb13d800640 1 -- 172.21.15.5:0/2085644797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb138103900 msgr2=0x7fb138101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:15.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.805+0000 7fb13d800640 1 --2- 172.21.15.5:0/2085644797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb138103900 0x7fb138101170 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb120034040 tx=0x7fb1200305d0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.805+0000 7fb13d800640 1 -- 172.21.15.5:0/2085644797 shutdown_connections 2024-04-24T06:50:15.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.805+0000 7fb13d800640 1 --2- 172.21.15.5:0/2085644797 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb1180a3da0 0x7fb1180a6260 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.805+0000 7fb13d800640 1 --2- 172.21.15.5:0/2085644797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb13810c400 0x7fb1381016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.805+0000 7fb13d800640 1 --2- 172.21.15.5:0/2085644797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb138103900 0x7fb138101170 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.805+0000 7fb13d800640 1 --2- 172.21.15.5:0/2085644797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb138102fe0 0x7fb138100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:15.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.805+0000 7fb13d800640 1 -- 172.21.15.5:0/2085644797 >> 172.21.15.5:0/2085644797 conn(0x7fb138077ab0 msgr2=0x7fb1380fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:15.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.806+0000 7fb13d800640 1 -- 172.21.15.5:0/2085644797 shutdown_connections 2024-04-24T06:50:15.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:15.806+0000 7fb13d800640 1 -- 172.21.15.5:0/2085644797 wait complete. 2024-04-24T06:50:15.825 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T06:50:16.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.039+0000 7fa1e9600640 1 -- 172.21.15.5:0/3482233829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1e4104c00 msgr2=0x7fa1e4108c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:16.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.039+0000 7fa1e9600640 1 --2- 172.21.15.5:0/3482233829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1e4104c00 0x7fa1e4108c40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc009900 tx=0x7fa1cc02a3f0 comp rx=0 tx=0).stop 2024-04-24T06:50:16.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.039+0000 7fa1e9600640 1 -- 172.21.15.5:0/3482233829 shutdown_connections 2024-04-24T06:50:16.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.039+0000 7fa1e9600640 1 --2- 172.21.15.5:0/3482233829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1e4109320 0x7fa1e4110ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:16.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.039+0000 7fa1e9600640 1 --2- 172.21.15.5:0/3482233829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1e4104c00 0x7fa1e4108c40 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:16.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.039+0000 7fa1e9600640 1 --2- 172.21.15.5:0/3482233829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1e4104250 0x7fa1e4104630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:16.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.039+0000 7fa1e9600640 1 -- 172.21.15.5:0/3482233829 >> 172.21.15.5:0/3482233829 conn(0x7fa1e40ffad0 msgr2=0x7fa1e4101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:16.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.039+0000 7fa1e9600640 1 -- 172.21.15.5:0/3482233829 shutdown_connections 2024-04-24T06:50:16.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.039+0000 7fa1e9600640 1 -- 172.21.15.5:0/3482233829 wait complete. 2024-04-24T06:50:16.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.041+0000 7fa1e9600640 1 Processor -- start 2024-04-24T06:50:16.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.041+0000 7fa1e9600640 1 -- start start 2024-04-24T06:50:16.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.041+0000 7fa1e9600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1e4104250 0x7fa1e41a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:16.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.041+0000 7fa1e9600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1e4109320 0x7fa1e41a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:16.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.041+0000 7fa1e9600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1e419a780 0x7fa1e419ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:16.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.041+0000 7fa1e9600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e4114400 con 0x7fa1e419a780 2024-04-24T06:50:16.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.042+0000 7fa1e9600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e4114280 con 0x7fa1e4104250 2024-04-24T06:50:16.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.042+0000 7fa1e9600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e4114580 con 0x7fa1e4109320 2024-04-24T06:50:16.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.042+0000 7fa1e3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1e4104250 0x7fa1e41a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:16.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.042+0000 7fa1dbe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1e4109320 0x7fa1e41a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:16.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.042+0000 7fa1e3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1e4104250 0x7fa1e41a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:33402/0 (socket says 172.21.15.5:33402) 2024-04-24T06:50:16.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.042+0000 7fa1e3e00640 1 -- 172.21.15.5:0/1688689459 learned_addr learned my addr 172.21.15.5:0/1688689459 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:16.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.042+0000 7fa1e8c00640 1 --2- 172.21.15.5:0/1688689459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1e419a780 0x7fa1e419ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:16.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.042+0000 7fa1e3e00640 1 -- 172.21.15.5:0/1688689459 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1e4109320 msgr2=0x7fa1e41a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:16.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.042+0000 7fa1e3e00640 1 --2- 172.21.15.5:0/1688689459 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1e4109320 0x7fa1e41a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:16.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.042+0000 7fa1e3e00640 1 -- 172.21.15.5:0/1688689459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1e419a780 msgr2=0x7fa1e419ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:16.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.042+0000 7fa1e3e00640 1 --2- 172.21.15.5:0/1688689459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1e419a780 0x7fa1e419ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:16.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.042+0000 7fa1e3e00640 1 -- 172.21.15.5:0/1688689459 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e419b310 con 0x7fa1e4104250 2024-04-24T06:50:16.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.043+0000 7fa1e3e00640 1 --2- 172.21.15.5:0/1688689459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1e4104250 0x7fa1e41a0610 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa1d400e9f0 tx=0x7fa1d400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:16.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.043+0000 7fa1e1600640 1 -- 172.21.15.5:0/1688689459 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d400cd80 con 0x7fa1e4104250 2024-04-24T06:50:16.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.043+0000 7fa1e9600640 1 -- 172.21.15.5:0/1688689459 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1e419b600 con 0x7fa1e4104250 2024-04-24T06:50:16.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.043+0000 7fa1e1600640 1 -- 172.21.15.5:0/1688689459 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1d4010910 con 0x7fa1e4104250 2024-04-24T06:50:16.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.043+0000 7fa1e1600640 1 -- 172.21.15.5:0/1688689459 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d40188f0 con 0x7fa1e4104250 2024-04-24T06:50:16.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.043+0000 7fa1e9600640 1 -- 172.21.15.5:0/1688689459 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1e41a77b0 con 0x7fa1e4104250 2024-04-24T06:50:16.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.047+0000 7fa1eda5a740 1 Processor -- start 2024-04-24T06:50:16.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.048+0000 7fa1eda5a740 1 -- start start 2024-04-24T06:50:16.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.048+0000 7fa1eda5a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8eadb40 0x5573b8eadf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:16.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.048+0000 7fa1eda5a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573b8eae460 0x5573b8eb85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:16.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.048+0000 7fa1eda5a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573b8eb8b30 0x5573b8ebaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:16.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.048+0000 7fa1eda5a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5573b8e131d0 con 0x5573b8eb8b30 2024-04-24T06:50:16.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.048+0000 7fa1eda5a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5573b8db9450 con 0x5573b8eae460 2024-04-24T06:50:16.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.048+0000 7fa1eda5a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5573b8e0f190 con 0x5573b8eadb40 2024-04-24T06:50:16.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.048+0000 7fa1e1600640 1 -- 172.21.15.5:0/1688689459 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fa1d4010ac0 con 0x7fa1e4104250 2024-04-24T06:50:16.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.048+0000 7fa1dbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573b8eb8b30 0x5573b8ebaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:16.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.048+0000 7fa1dbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573b8eb8b30 0x5573b8ebaf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36258/0 (socket says 172.21.15.5:36258) 2024-04-24T06:50:16.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.048+0000 7fa1dbe00640 1 -- 172.21.15.5:0/1447167069 learned_addr learned my addr 172.21.15.5:0/1447167069 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:16.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.049+0000 7fa1e8c00640 1 --2- 172.21.15.5:0/1447167069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8eadb40 0x5573b8eadf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:16.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.049+0000 7fa1e3e00640 1 --2- 172.21.15.5:0/1447167069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573b8eae460 0x5573b8eb85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:16.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.049+0000 7fa1dbe00640 1 -- 172.21.15.5:0/1447167069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8eadb40 msgr2=0x5573b8eadf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:16.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.049+0000 7fa1dbe00640 1 --2- 172.21.15.5:0/1447167069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8eadb40 0x5573b8eadf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:16.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.049+0000 7fa1dbe00640 1 -- 172.21.15.5:0/1447167069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573b8eae460 msgr2=0x5573b8eb85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:16.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.049+0000 7fa1dbe00640 1 --2- 172.21.15.5:0/1447167069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573b8eae460 0x5573b8eb85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:16.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.049+0000 7fa1dbe00640 1 -- 172.21.15.5:0/1447167069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573b8e76310 con 0x5573b8eb8b30 2024-04-24T06:50:16.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.049+0000 7fa1e3e00640 1 --2- 172.21.15.5:0/1447167069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573b8eae460 0x5573b8eb85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:50:16.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.049+0000 7fa1dbe00640 1 --2- 172.21.15.5:0/1447167069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573b8eb8b30 0x5573b8ebaf20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc009900 tx=0x7fa1cc005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d57e360ccc27b23d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:16.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.050+0000 7fa1d9200640 1 -- 172.21.15.5:0/1447167069 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc035040 con 0x5573b8eb8b30 2024-04-24T06:50:16.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.050+0000 7fa1d9200640 1 -- 172.21.15.5:0/1447167069 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1cc004480 con 0x5573b8eb8b30 2024-04-24T06:50:16.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.050+0000 7fa1d9200640 1 -- 172.21.15.5:0/1447167069 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc0035f0 con 0x5573b8eb8b30 2024-04-24T06:50:16.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.050+0000 7fa1e1600640 1 --2- 172.21.15.5:0/1688689459 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fa1c80a3d20 0x7fa1c80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:16.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.050+0000 7fa1e1600640 1 -- 172.21.15.5:0/1688689459 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fa1d40d0bf0 con 0x7fa1e4104250 2024-04-24T06:50:16.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.050+0000 7fa1e8c00640 1 --2- 172.21.15.5:0/1688689459 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fa1c80a3d20 0x7fa1c80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:16.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.050+0000 7fa1eda5a740 1 -- 172.21.15.5:0/1447167069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573b8eb8b30 msgr2=0x5573b8ebaf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:16.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.050+0000 7fa1eda5a740 1 --2- 172.21.15.5:0/1447167069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573b8eb8b30 0x5573b8ebaf20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc009900 tx=0x7fa1cc005bf0 comp rx=0 tx=0).stop 2024-04-24T06:50:16.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.051+0000 7fa1eda5a740 1 -- 172.21.15.5:0/1447167069 shutdown_connections 2024-04-24T06:50:16.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.051+0000 7fa1eda5a740 1 --2- 172.21.15.5:0/1447167069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573b8eb8b30 0x5573b8ebaf20 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:16.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.051+0000 7fa1eda5a740 1 --2- 172.21.15.5:0/1447167069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573b8eae460 0x5573b8eb85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:16.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.051+0000 7fa1eda5a740 1 --2- 172.21.15.5:0/1447167069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8eadb40 0x5573b8eadf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:16.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.051+0000 7fa1eda5a740 1 -- 172.21.15.5:0/1447167069 >> 172.21.15.5:0/1447167069 conn(0x5573b8ea9e60 msgr2=0x5573b8eaa240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:16.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.051+0000 7fa1eda5a740 1 -- 172.21.15.5:0/1447167069 shutdown_connections 2024-04-24T06:50:16.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.051+0000 7fa1eda5a740 1 -- 172.21.15.5:0/1447167069 wait complete. 2024-04-24T06:50:16.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.051+0000 7fa1e8c00640 1 --2- 172.21.15.5:0/1688689459 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fa1c80a3d20 0x7fa1c80a61e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc005fd0 tx=0x7fa1dc009600 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:16.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.052+0000 7fa1eda5a740 1 -- start start 2024-04-24T06:50:16.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.052+0000 7fa1eda5a740 1 Processor -- start 2024-04-24T06:50:16.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.052+0000 7fa1eda5a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8eadb40 0x5573b8eb85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:16.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.052+0000 7fa1eda5a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573b8eae460 0x5573b8eb58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:16.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.052+0000 7fa1eda5a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573b8eb8b30 0x5573b8eb6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:16.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.052+0000 7fa1eda5a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5573b8ebb960 con 0x5573b8eb8b30 2024-04-24T06:50:16.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.052+0000 7fa1eda5a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5573b8dcc810 con 0x5573b8eae460 2024-04-24T06:50:16.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.052+0000 7fa1eda5a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5573b8ebbae0 con 0x5573b8eadb40 2024-04-24T06:50:16.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.052+0000 7fa1eda5a740 10 client.0 _ll_register_callbacks cb 0x5573b8d9a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:50:16.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.053+0000 7fa1e8c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573b8eb8b30 0x5573b8eb6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:16.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.053+0000 7fa1e8c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573b8eb8b30 0x5573b8eb6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36272/0 (socket says 172.21.15.5:36272) 2024-04-24T06:50:16.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.053+0000 7fa1e8c00640 1 -- 172.21.15.5:0/490469489 learned_addr learned my addr 172.21.15.5:0/490469489 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:16.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.053+0000 7fa1dbe00640 1 --2- 172.21.15.5:0/490469489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573b8eae460 0x5573b8eb58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:16.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.053+0000 7fa1e3e00640 1 --2- 172.21.15.5:0/490469489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8eadb40 0x5573b8eb85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:16.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.053+0000 7fa1e3e00640 1 -- 172.21.15.5:0/490469489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573b8eae460 msgr2=0x5573b8eb58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:16.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.053+0000 7fa1e3e00640 1 --2- 172.21.15.5:0/490469489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573b8eae460 0x5573b8eb58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:16.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.053+0000 7fa1e3e00640 1 -- 172.21.15.5:0/490469489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573b8eb8b30 msgr2=0x5573b8eb6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:16.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.053+0000 7fa1e3e00640 1 --2- 172.21.15.5:0/490469489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573b8eb8b30 0x5573b8eb6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:16.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.053+0000 7fa1e3e00640 1 -- 172.21.15.5:0/490469489 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573b8ddd3f0 con 0x5573b8eadb40 2024-04-24T06:50:16.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.053+0000 7fa1e8c00640 1 --2- 172.21.15.5:0/490469489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573b8eb8b30 0x5573b8eb6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:50:16.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.053+0000 7fa1e3e00640 1 --2- 172.21.15.5:0/490469489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8eadb40 0x5573b8eb85e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa1d40ce2f0 tx=0x7fa1d4011f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6246542e65694ac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:16.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.053+0000 7fa1d1600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:50:16.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.054+0000 7fa1d1600640 1 -- 172.21.15.5:0/490469489 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d40ce570 con 0x5573b8eadb40 2024-04-24T06:50:16.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.054+0000 7fa1d1600640 1 -- 172.21.15.5:0/490469489 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fa1d40ce710 con 0x5573b8eadb40 2024-04-24T06:50:16.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.054+0000 7fa1eda5a740 1 -- 172.21.15.5:0/490469489 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1cc004480 con 0x5573b8eadb40 2024-04-24T06:50:16.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.054+0000 7fa1d1600640 1 -- 172.21.15.5:0/490469489 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa1d409c040 con 0x5573b8eadb40 2024-04-24T06:50:16.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.054+0000 7fa1eda5a740 10 client.26641 fetch_fsmap learned FSMap version 27 2024-04-24T06:50:16.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.054+0000 7fa1eda5a740 1 -- 172.21.15.5:0/490469489 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5573b8b253f0 con 0x5573b8eadb40 2024-04-24T06:50:16.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.054+0000 7fa1d1600640 1 -- 172.21.15.5:0/490469489 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1cc004480 con 0x5573b8eadb40 2024-04-24T06:50:16.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.054+0000 7fa1d1600640 1 -- 172.21.15.5:0/490469489 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7fa1d409bab0 con 0x5573b8eadb40 2024-04-24T06:50:16.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.055+0000 7fa1eda5a740 10 client.26641 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:50:16.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.055+0000 7fa1eda5a740 10 client.26641 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:50:16.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.055+0000 7fa1eda5a740 20 client.26641 populate_metadata read hostname 'smithi005' 2024-04-24T06:50:16.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.055+0000 7fa1eda5a740 1 --2- 172.21.15.5:0/490469489 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5573b8ec8bc0 0x5573b8ecafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:16.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.055+0000 7fa1eda5a740 4 client.26641 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:50:16.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.055+0000 7fa1eda5a740 1 -- 172.21.15.5:0/490469489 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573b8e20bf0 con 0x5573b8ec8bc0 2024-04-24T06:50:16.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.055+0000 7fa1dbe00640 1 --2- 172.21.15.5:0/490469489 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5573b8ec8bc0 0x5573b8ecafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:16.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.056+0000 7fa1dbe00640 1 --2- 172.21.15.5:0/490469489 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5573b8ec8bc0 0x5573b8ecafd0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe2918a0ff7f5861 server_cookie=d626754f952903a2 in_seq=0 out_seq=0 2024-04-24T06:50:16.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.056+0000 7fa1d1600640 10 client.26641 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:50:16.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.058+0000 7fa1d1600640 1 -- 172.21.15.5:0/490469489 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5573b8e20bf0 con 0x5573b8ec8bc0 2024-04-24T06:50:16.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.058+0000 7fa1d1600640 10 client.26641 handle_command_reply: tid=0 2024-04-24T06:50:16.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.058+0000 7fa1eda5a740 1 client.26641 shutdown 2024-04-24T06:50:16.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.058+0000 7fa1eda5a740 1 -- 172.21.15.5:0/490469489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8eadb40 msgr2=0x5573b8eb85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:16.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.058+0000 7fa1eda5a740 1 --2- 172.21.15.5:0/490469489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8eadb40 0x5573b8eb85e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa1d40ce2f0 tx=0x7fa1d4011f60 comp rx=0 tx=0).stop 2024-04-24T06:50:16.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.058+0000 7fa1eda5a740 1 -- 172.21.15.5:0/490469489 shutdown_connections 2024-04-24T06:50:16.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.058+0000 7fa1eda5a740 1 --2- 172.21.15.5:0/490469489 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5573b8ec8bc0 0x5573b8ecafd0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:16.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.058+0000 7fa1eda5a740 1 --2- 172.21.15.5:0/490469489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573b8eb8b30 0x5573b8eb6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:16.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.058+0000 7fa1eda5a740 1 --2- 172.21.15.5:0/490469489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573b8eae460 0x5573b8eb58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:16.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.058+0000 7fa1d1600640 0 client.26641 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:50:16.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.058+0000 7fa1eda5a740 1 --2- 172.21.15.5:0/490469489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8eadb40 0x5573b8eb85e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:16.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.058+0000 7fa1eda5a740 1 -- 172.21.15.5:0/490469489 >> 172.21.15.5:0/490469489 conn(0x5573b8ea9e60 msgr2=0x5573b8eafc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:16.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.058+0000 7fa1eda5a740 1 -- 172.21.15.5:0/490469489 shutdown_connections 2024-04-24T06:50:16.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.058+0000 7fa1eda5a740 1 -- 172.21.15.5:0/490469489 wait complete. 2024-04-24T06:50:16.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.058+0000 7fa1eda5a740 20 client.26641 trim_cache size 0 max 16384 2024-04-24T06:50:16.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.072+0000 7fa1eda5a740 1 Processor -- start 2024-04-24T06:50:16.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.073+0000 7fa1eda5a740 1 -- start start 2024-04-24T06:50:16.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.073+0000 7fa1eda5a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573b8eadb40 0x5573b8ec4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:16.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.073+0000 7fa1eda5a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573b8eae460 0x5573b8eb8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:16.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.073+0000 7fa1eda5a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8eb8b30 0x5573b8ed08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:16.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.073+0000 7fa1eda5a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e41b26e0 con 0x5573b8eadb40 2024-04-24T06:50:16.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.073+0000 7fa1eda5a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e41affb0 con 0x5573b8eae460 2024-04-24T06:50:16.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.073+0000 7fa1eda5a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e41ade00 con 0x5573b8eb8b30 2024-04-24T06:50:16.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.073+0000 7fa1e3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573b8eadb40 0x5573b8ec4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:16.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.073+0000 7fa1e3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573b8eadb40 0x5573b8ec4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36286/0 (socket says 172.21.15.5:36286) 2024-04-24T06:50:16.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.073+0000 7fa1e3e00640 1 -- 172.21.15.5:0/3188265242 learned_addr learned my addr 172.21.15.5:0/3188265242 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:16.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.073+0000 7fa1dbe00640 1 --2- 172.21.15.5:0/3188265242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573b8eae460 0x5573b8eb8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:16.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.073+0000 7fa1e8c00640 1 --2- 172.21.15.5:0/3188265242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8eb8b30 0x5573b8ed08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:16.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.074+0000 7fa1e3e00640 1 -- 172.21.15.5:0/3188265242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8eb8b30 msgr2=0x5573b8ed08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:16.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.074+0000 7fa1e3e00640 1 --2- 172.21.15.5:0/3188265242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8eb8b30 0x5573b8ed08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:16.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.074+0000 7fa1e3e00640 1 -- 172.21.15.5:0/3188265242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573b8eae460 msgr2=0x5573b8eb8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:16.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.074+0000 7fa1e3e00640 1 --2- 172.21.15.5:0/3188265242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573b8eae460 0x5573b8eb8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:16.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.074+0000 7fa1e3e00640 1 -- 172.21.15.5:0/3188265242 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573b8bef7d0 con 0x5573b8eadb40 2024-04-24T06:50:16.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.074+0000 7fa1e3e00640 1 --2- 172.21.15.5:0/3188265242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573b8eadb40 0x5573b8ec4820 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa1d400e9c0 tx=0x7fa1d40bd2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9474af58a15457 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:16.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.074+0000 7fa1d3e00640 1 -- 172.21.15.5:0/3188265242 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d409c1e0 con 0x5573b8eadb40 2024-04-24T06:50:16.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.074+0000 7fa1d3e00640 1 -- 172.21.15.5:0/3188265242 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1d40ce570 con 0x5573b8eadb40 2024-04-24T06:50:16.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.074+0000 7fa1d3e00640 1 -- 172.21.15.5:0/3188265242 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4098c30 con 0x5573b8eadb40 2024-04-24T06:50:16.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.075+0000 7fa1eda5a740 1 -- 172.21.15.5:0/3188265242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573b8eadb40 msgr2=0x5573b8ec4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:16.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.075+0000 7fa1eda5a740 1 --2- 172.21.15.5:0/3188265242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573b8eadb40 0x5573b8ec4820 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa1d400e9c0 tx=0x7fa1d40bd2c0 comp rx=0 tx=0).stop 2024-04-24T06:50:16.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.075+0000 7fa1eda5a740 1 -- 172.21.15.5:0/3188265242 shutdown_connections 2024-04-24T06:50:16.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.075+0000 7fa1eda5a740 1 --2- 172.21.15.5:0/3188265242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8eb8b30 0x5573b8ed08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:16.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.075+0000 7fa1eda5a740 1 --2- 172.21.15.5:0/3188265242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573b8eae460 0x5573b8eb8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:16.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.075+0000 7fa1eda5a740 1 --2- 172.21.15.5:0/3188265242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573b8eadb40 0x5573b8ec4820 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:16.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.075+0000 7fa1eda5a740 1 -- 172.21.15.5:0/3188265242 >> 172.21.15.5:0/3188265242 conn(0x5573b8ea9e60 msgr2=0x5573b8eb0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:16.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.075+0000 7fa1eda5a740 1 -- 172.21.15.5:0/3188265242 shutdown_connections 2024-04-24T06:50:16.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.075+0000 7fa1eda5a740 1 -- 172.21.15.5:0/3188265242 wait complete. 2024-04-24T06:50:16.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.076+0000 7fa1eda5a740 1 -- start start 2024-04-24T06:50:16.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.076+0000 7fa1eda5a740 1 Processor -- start 2024-04-24T06:50:16.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.076+0000 7fa1eda5a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8eadb40 0x5573b8ec1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:16.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.076+0000 7fa1eda5a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573b8eae460 0x5573b8eb2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:16.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.076+0000 7fa1eda5a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573b8eb8b30 0x5573b8eb3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:16.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.076+0000 7fa1eda5a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e41ae470 con 0x5573b8eb8b30 2024-04-24T06:50:16.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.076+0000 7fa1eda5a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e41af8e0 con 0x5573b8eae460 2024-04-24T06:50:16.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.076+0000 7fa1eda5a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e41aeb40 con 0x5573b8eadb40 2024-04-24T06:50:16.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.076+0000 7fa1eda5a740 10 client.0 _ll_register_callbacks cb 0x7fa1d4098e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:50:16.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.077+0000 7fa1e8c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573b8eb8b30 0x5573b8eb3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:16.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.077+0000 7fa1e8c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573b8eb8b30 0x5573b8eb3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36288/0 (socket says 172.21.15.5:36288) 2024-04-24T06:50:16.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.077+0000 7fa1e8c00640 1 -- 172.21.15.5:0/2438249354 learned_addr learned my addr 172.21.15.5:0/2438249354 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:16.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.077+0000 7fa1dbe00640 1 --2- 172.21.15.5:0/2438249354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573b8eae460 0x5573b8eb2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:16.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.077+0000 7fa1e3e00640 1 --2- 172.21.15.5:0/2438249354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8eadb40 0x5573b8ec1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:16.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.077+0000 7fa1e8c00640 1 -- 172.21.15.5:0/2438249354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8eadb40 msgr2=0x5573b8ec1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:16.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.077+0000 7fa1e8c00640 1 --2- 172.21.15.5:0/2438249354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8eadb40 0x5573b8ec1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:16.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.077+0000 7fa1e8c00640 1 -- 172.21.15.5:0/2438249354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573b8eae460 msgr2=0x5573b8eb2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:16.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.077+0000 7fa1e8c00640 1 --2- 172.21.15.5:0/2438249354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573b8eae460 0x5573b8eb2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:16.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.077+0000 7fa1e8c00640 1 -- 172.21.15.5:0/2438249354 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573b8eb37d0 con 0x5573b8eb8b30 2024-04-24T06:50:16.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.077+0000 7fa1dbe00640 1 --2- 172.21.15.5:0/2438249354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573b8eae460 0x5573b8eb2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:50:16.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.077+0000 7fa1e3e00640 1 --2- 172.21.15.5:0/2438249354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8eadb40 0x5573b8ec1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:50:16.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.078+0000 7fa1e8c00640 1 --2- 172.21.15.5:0/2438249354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573b8eb8b30 0x5573b8eb3090 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc020770 tx=0x7fa1dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c5a028e4156149d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:16.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.078+0000 7fa1d1600640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:50:16.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.078+0000 7fa1d1600640 1 -- 172.21.15.5:0/2438249354 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc02f060 con 0x5573b8eb8b30 2024-04-24T06:50:16.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.078+0000 7fa1d1600640 1 -- 172.21.15.5:0/2438249354 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fa1dc01d600 con 0x5573b8eb8b30 2024-04-24T06:50:16.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.078+0000 7fa1eda5a740 1 -- 172.21.15.5:0/2438249354 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1d40ce570 con 0x5573b8eb8b30 2024-04-24T06:50:16.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.078+0000 7fa1d1600640 1 -- 172.21.15.5:0/2438249354 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa1dc01d800 con 0x5573b8eb8b30 2024-04-24T06:50:16.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.078+0000 7fa1eda5a740 10 client.16500 fetch_fsmap learned FSMap version 27 2024-04-24T06:50:16.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.078+0000 7fa1eda5a740 1 -- 172.21.15.5:0/2438249354 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5573b8f2f110 con 0x5573b8eb8b30 2024-04-24T06:50:16.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.078+0000 7fa1d1600640 1 -- 172.21.15.5:0/2438249354 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1d40ce570 con 0x5573b8eb8b30 2024-04-24T06:50:16.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.079+0000 7fa1d1600640 1 -- 172.21.15.5:0/2438249354 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7fa1dc029070 con 0x5573b8eb8b30 2024-04-24T06:50:16.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.079+0000 7fa1eda5a740 10 client.16500 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:50:16.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.079+0000 7fa1eda5a740 10 client.16500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:50:16.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.079+0000 7fa1eda5a740 20 client.16500 populate_metadata read hostname 'smithi005' 2024-04-24T06:50:16.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.079+0000 7fa1eda5a740 1 --2- 172.21.15.5:0/2438249354 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5573b8f2f8c0 0x5573b8f31ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:16.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.079+0000 7fa1eda5a740 4 client.16500 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:50:16.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.079+0000 7fa1eda5a740 1 -- 172.21.15.5:0/2438249354 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5573b8e1aa60 con 0x5573b8f2f8c0 2024-04-24T06:50:16.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.079+0000 7fa1e3e00640 1 --2- 172.21.15.5:0/2438249354 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5573b8f2f8c0 0x5573b8f31ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:16.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.080+0000 7fa1e3e00640 1 --2- 172.21.15.5:0/2438249354 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5573b8f2f8c0 0x5573b8f31ce0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68c160843c4b25da server_cookie=ea5decb1cf4985b in_seq=0 out_seq=0 2024-04-24T06:50:16.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.080+0000 7fa1d1600640 10 client.16500 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:50:16.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.080+0000 7fa1d1600640 1 -- 172.21.15.5:0/2438249354 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5573b8e1aa60 con 0x5573b8f2f8c0 2024-04-24T06:50:16.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.080+0000 7fa1d1600640 10 client.16500 handle_command_reply: tid=0 2024-04-24T06:50:16.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.080+0000 7fa1eda5a740 1 client.16500 shutdown 2024-04-24T06:50:16.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.080+0000 7fa1eda5a740 1 -- 172.21.15.5:0/2438249354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573b8eb8b30 msgr2=0x5573b8eb3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:16.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.080+0000 7fa1eda5a740 1 --2- 172.21.15.5:0/2438249354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573b8eb8b30 0x5573b8eb3090 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc020770 tx=0x7fa1dc004a40 comp rx=0 tx=0).stop 2024-04-24T06:50:16.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.080+0000 7fa1eda5a740 1 -- 172.21.15.5:0/2438249354 shutdown_connections 2024-04-24T06:50:16.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.080+0000 7fa1eda5a740 1 --2- 172.21.15.5:0/2438249354 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5573b8f2f8c0 0x5573b8f31ce0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:16.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.080+0000 7fa1eda5a740 1 --2- 172.21.15.5:0/2438249354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573b8eb8b30 0x5573b8eb3090 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:16.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.080+0000 7fa1eda5a740 1 --2- 172.21.15.5:0/2438249354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573b8eae460 0x5573b8eb2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:16.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.080+0000 7fa1eda5a740 1 --2- 172.21.15.5:0/2438249354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8eadb40 0x5573b8ec1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:16.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.080+0000 7fa1eda5a740 1 -- 172.21.15.5:0/2438249354 >> 172.21.15.5:0/2438249354 conn(0x5573b8ea9e60 msgr2=0x5573b8ea86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:16.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.080+0000 7fa1d1600640 0 client.16500 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:50:16.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.081+0000 7fa1eda5a740 1 -- 172.21.15.5:0/2438249354 shutdown_connections 2024-04-24T06:50:16.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.081+0000 7fa1eda5a740 1 -- 172.21.15.5:0/2438249354 wait complete. 2024-04-24T06:50:16.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.081+0000 7fa1eda5a740 20 client.16500 trim_cache size 0 max 16384 2024-04-24T06:50:16.090 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:50:16.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.081+0000 7fa1e9600640 1 -- 172.21.15.5:0/1688689459 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fa1c80a3d20 msgr2=0x7fa1c80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:16.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.081+0000 7fa1e9600640 1 --2- 172.21.15.5:0/1688689459 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fa1c80a3d20 0x7fa1c80a61e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc005fd0 tx=0x7fa1dc009600 comp rx=0 tx=0).stop 2024-04-24T06:50:16.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.081+0000 7fa1e9600640 1 -- 172.21.15.5:0/1688689459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1e4104250 msgr2=0x7fa1e41a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:16.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.081+0000 7fa1e9600640 1 --2- 172.21.15.5:0/1688689459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1e4104250 0x7fa1e41a0610 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa1d400e9f0 tx=0x7fa1d400eeb0 comp rx=0 tx=0).stop 2024-04-24T06:50:16.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.082+0000 7fa1e9600640 1 -- 172.21.15.5:0/1688689459 shutdown_connections 2024-04-24T06:50:16.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.082+0000 7fa1e9600640 1 --2- 172.21.15.5:0/1688689459 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fa1c80a3d20 0x7fa1c80a61e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:16.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.082+0000 7fa1e9600640 1 --2- 172.21.15.5:0/1688689459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1e419a780 0x7fa1e419ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:16.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.082+0000 7fa1e9600640 1 --2- 172.21.15.5:0/1688689459 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1e4109320 0x7fa1e41a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:16.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.082+0000 7fa1e9600640 1 --2- 172.21.15.5:0/1688689459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1e4104250 0x7fa1e41a0610 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:16.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.082+0000 7fa1e9600640 1 -- 172.21.15.5:0/1688689459 >> 172.21.15.5:0/1688689459 conn(0x7fa1e40ffad0 msgr2=0x7fa1e4101e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:16.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.082+0000 7fa1e9600640 1 -- 172.21.15.5:0/1688689459 shutdown_connections 2024-04-24T06:50:16.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:16.082+0000 7fa1e9600640 1 -- 172.21.15.5:0/1688689459 wait complete. 2024-04-24T06:50:17.102 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T06:50:17.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:50:17.102 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T06:50:17.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.332+0000 7f9e11e00640 1 -- 172.21.15.5:0/3769152074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e0c112c90 msgr2=0x7f9e0c115080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:17.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.332+0000 7f9e11e00640 1 --2- 172.21.15.5:0/3769152074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e0c112c90 0x7f9e0c115080 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9e04009900 tx=0x7f9e0402a3f0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.332+0000 7f9e11e00640 1 -- 172.21.15.5:0/3769152074 shutdown_connections 2024-04-24T06:50:17.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.332+0000 7f9e11e00640 1 --2- 172.21.15.5:0/3769152074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e0c112c90 0x7f9e0c115080 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.332+0000 7f9e11e00640 1 --2- 172.21.15.5:0/3769152074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e0c076e50 0x7f9e0c112750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.332+0000 7f9e11e00640 1 --2- 172.21.15.5:0/3769152074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e0c076530 0x7f9e0c076910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.332+0000 7f9e11e00640 1 -- 172.21.15.5:0/3769152074 >> 172.21.15.5:0/3769152074 conn(0x7f9e0c0ffaa0 msgr2=0x7f9e0c101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:17.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.332+0000 7f9e11e00640 1 -- 172.21.15.5:0/3769152074 shutdown_connections 2024-04-24T06:50:17.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.332+0000 7f9e11e00640 1 -- 172.21.15.5:0/3769152074 wait complete. 2024-04-24T06:50:17.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.333+0000 7f9e11e00640 1 Processor -- start 2024-04-24T06:50:17.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.333+0000 7f9e11e00640 1 -- start start 2024-04-24T06:50:17.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.334+0000 7f9e11e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e0c076530 0x7f9e0c1a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:17.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.334+0000 7f9e11e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e0c076e50 0x7f9e0c1a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:17.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.334+0000 7f9e11e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e0c112c90 0x7f9e0c19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:17.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.334+0000 7f9e11e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0c1187d0 con 0x7f9e0c076530 2024-04-24T06:50:17.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.334+0000 7f9e11e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0c118650 con 0x7f9e0c076e50 2024-04-24T06:50:17.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.334+0000 7f9e11e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0c118950 con 0x7f9e0c112c90 2024-04-24T06:50:17.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.334+0000 7f9e10a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e0c076530 0x7f9e0c1a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:17.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.334+0000 7f9e03e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e0c076e50 0x7f9e0c1a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:17.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.334+0000 7f9e10a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e0c076530 0x7f9e0c1a4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36310/0 (socket says 172.21.15.5:36310) 2024-04-24T06:50:17.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.334+0000 7f9e03e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e0c076e50 0x7f9e0c1a4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:33472/0 (socket says 172.21.15.5:33472) 2024-04-24T06:50:17.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.334+0000 7f9e10a00640 1 -- 172.21.15.5:0/450077910 learned_addr learned my addr 172.21.15.5:0/450077910 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:17.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.334+0000 7f9e11400640 1 --2- 172.21.15.5:0/450077910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e0c112c90 0x7f9e0c19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:17.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.335+0000 7f9e10a00640 1 -- 172.21.15.5:0/450077910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e0c112c90 msgr2=0x7f9e0c19eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:17.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.335+0000 7f9e10a00640 1 --2- 172.21.15.5:0/450077910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e0c112c90 0x7f9e0c19eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.335+0000 7f9e10a00640 1 -- 172.21.15.5:0/450077910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e0c076e50 msgr2=0x7f9e0c1a4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:17.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.335+0000 7f9e10a00640 1 --2- 172.21.15.5:0/450077910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e0c076e50 0x7f9e0c1a4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.335+0000 7f9e10a00640 1 -- 172.21.15.5:0/450077910 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e0c19f270 con 0x7f9e0c076530 2024-04-24T06:50:17.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.335+0000 7f9e10a00640 1 --2- 172.21.15.5:0/450077910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e0c076530 0x7f9e0c1a4940 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc00ebc0 tx=0x7f9dfc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:17.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.335+0000 7f9e0a000640 1 -- 172.21.15.5:0/450077910 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc010630 con 0x7f9e0c076530 2024-04-24T06:50:17.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.335+0000 7f9e0a000640 1 -- 172.21.15.5:0/450077910 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dfc010cb0 con 0x7f9e0c076530 2024-04-24T06:50:17.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.335+0000 7f9e0a000640 1 -- 172.21.15.5:0/450077910 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc018bd0 con 0x7f9e0c076530 2024-04-24T06:50:17.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.335+0000 7f9e11e00640 1 -- 172.21.15.5:0/450077910 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e0c19f560 con 0x7f9e0c076530 2024-04-24T06:50:17.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.335+0000 7f9e11e00640 1 -- 172.21.15.5:0/450077910 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e0c1ab800 con 0x7f9e0c076530 2024-04-24T06:50:17.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.339+0000 7f9e161f4740 1 Processor -- start 2024-04-24T06:50:17.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.339+0000 7f9e0a000640 1 -- 172.21.15.5:0/450077910 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f9dfc0107d0 con 0x7f9e0c076530 2024-04-24T06:50:17.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.339+0000 7f9e161f4740 1 -- start start 2024-04-24T06:50:17.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.339+0000 7f9e161f4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55978c162270 0x55978c162650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:17.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.340+0000 7f9e161f4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55978c162b90 0x55978c16cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:17.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.340+0000 7f9e161f4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55978c16d260 0x55978c16f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:17.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.340+0000 7f9e161f4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55978c0c7e70 con 0x55978c162b90 2024-04-24T06:50:17.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.340+0000 7f9e161f4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55978c06e380 con 0x55978c16d260 2024-04-24T06:50:17.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.340+0000 7f9e161f4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55978c0c3e30 con 0x55978c162270 2024-04-24T06:50:17.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.340+0000 7f9e10a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55978c162b90 0x55978c16cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:17.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.340+0000 7f9e03e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55978c16d260 0x55978c16f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:17.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.340+0000 7f9e03e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55978c16d260 0x55978c16f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:33476/0 (socket says 172.21.15.5:33476) 2024-04-24T06:50:17.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.340+0000 7f9e03e00640 1 -- 172.21.15.5:0/2397096014 learned_addr learned my addr 172.21.15.5:0/2397096014 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:17.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.340+0000 7f9e11400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55978c162270 0x55978c162650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:17.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.340+0000 7f9e03e00640 1 -- 172.21.15.5:0/2397096014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55978c162270 msgr2=0x55978c162650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:17.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.340+0000 7f9e03e00640 1 --2- 172.21.15.5:0/2397096014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55978c162270 0x55978c162650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.340+0000 7f9e03e00640 1 -- 172.21.15.5:0/2397096014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55978c162b90 msgr2=0x55978c16cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:17.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.340+0000 7f9e03e00640 1 --2- 172.21.15.5:0/2397096014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55978c162b90 0x55978c16cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.340+0000 7f9e03e00640 1 -- 172.21.15.5:0/2397096014 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55978bea27e0 con 0x55978c16d260 2024-04-24T06:50:17.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.341+0000 7f9e03e00640 1 --2- 172.21.15.5:0/2397096014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55978c16d260 0x55978c16f650 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9df4009c00 tx=0x7f9df4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3dfb3e1d7b918f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:17.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.341+0000 7f9e0a000640 1 --2- 172.21.15.5:0/450077910 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f9df00a3da0 0x7f9df00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:17.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.341+0000 7f9e0a000640 1 -- 172.21.15.5:0/450077910 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f9dfc0d0da0 con 0x7f9e0c076530 2024-04-24T06:50:17.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.341+0000 7f9e01c00640 1 -- 172.21.15.5:0/2397096014 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df4004230 con 0x55978c16d260 2024-04-24T06:50:17.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.341+0000 7f9e01c00640 1 -- 172.21.15.5:0/2397096014 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9df40043d0 con 0x55978c16d260 2024-04-24T06:50:17.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.341+0000 7f9e11400640 1 --2- 172.21.15.5:0/450077910 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f9df00a3da0 0x7f9df00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:17.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.341+0000 7f9e01c00640 1 -- 172.21.15.5:0/2397096014 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df40196f0 con 0x55978c16d260 2024-04-24T06:50:17.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.342+0000 7f9e161f4740 1 -- 172.21.15.5:0/2397096014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55978c16d260 msgr2=0x55978c16f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:17.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.342+0000 7f9e161f4740 1 --2- 172.21.15.5:0/2397096014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55978c16d260 0x55978c16f650 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9df4009c00 tx=0x7f9df4011810 comp rx=0 tx=0).stop 2024-04-24T06:50:17.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.342+0000 7f9e11400640 1 --2- 172.21.15.5:0/450077910 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f9df00a3da0 0x7f9df00a6260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9e04006010 tx=0x7f9e04009600 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:17.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.342+0000 7f9e161f4740 1 -- 172.21.15.5:0/2397096014 shutdown_connections 2024-04-24T06:50:17.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.342+0000 7f9e161f4740 1 --2- 172.21.15.5:0/2397096014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55978c16d260 0x55978c16f650 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.342+0000 7f9e161f4740 1 --2- 172.21.15.5:0/2397096014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55978c162b90 0x55978c16cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.342+0000 7f9e161f4740 1 --2- 172.21.15.5:0/2397096014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55978c162270 0x55978c162650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.342+0000 7f9e161f4740 1 -- 172.21.15.5:0/2397096014 >> 172.21.15.5:0/2397096014 conn(0x55978c15e590 msgr2=0x55978c15e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:17.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.342+0000 7f9e161f4740 1 -- 172.21.15.5:0/2397096014 shutdown_connections 2024-04-24T06:50:17.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.342+0000 7f9e161f4740 1 -- 172.21.15.5:0/2397096014 wait complete. 2024-04-24T06:50:17.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.343+0000 7f9e161f4740 1 -- start start 2024-04-24T06:50:17.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.343+0000 7f9e161f4740 1 Processor -- start 2024-04-24T06:50:17.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.343+0000 7f9e161f4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55978c162270 0x55978c16a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:17.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.343+0000 7f9e161f4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55978c162b90 0x55978c16a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:17.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.343+0000 7f9e161f4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55978c16d260 0x55978c16abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:17.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.343+0000 7f9e161f4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55978c170050 con 0x55978c162270 2024-04-24T06:50:17.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.343+0000 7f9e161f4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55978c081660 con 0x55978c162b90 2024-04-24T06:50:17.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.343+0000 7f9e161f4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55978c1701d0 con 0x55978c16d260 2024-04-24T06:50:17.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.344+0000 7f9e161f4740 10 client.0 _ll_register_callbacks cb 0x55978c06faa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:50:17.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.344+0000 7f9e10a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55978c162270 0x55978c16a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:17.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.344+0000 7f9e10a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55978c162270 0x55978c16a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36332/0 (socket says 172.21.15.5:36332) 2024-04-24T06:50:17.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.344+0000 7f9e10a00640 1 -- 172.21.15.5:0/981975448 learned_addr learned my addr 172.21.15.5:0/981975448 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:17.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.344+0000 7f9e03e00640 1 --2- 172.21.15.5:0/981975448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55978c162b90 0x55978c16a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:17.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.344+0000 7f9e11400640 1 --2- 172.21.15.5:0/981975448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55978c16d260 0x55978c16abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:17.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.344+0000 7f9e10a00640 1 -- 172.21.15.5:0/981975448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55978c16d260 msgr2=0x55978c16abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:17.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.344+0000 7f9e10a00640 1 --2- 172.21.15.5:0/981975448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55978c16d260 0x55978c16abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.344+0000 7f9e10a00640 1 -- 172.21.15.5:0/981975448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55978c162b90 msgr2=0x55978c16a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:17.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.344+0000 7f9e10a00640 1 --2- 172.21.15.5:0/981975448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55978c162b90 0x55978c16a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.344+0000 7f9e10a00640 1 -- 172.21.15.5:0/981975448 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55978bf02f70 con 0x55978c162270 2024-04-24T06:50:17.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.344+0000 7f9e03e00640 1 --2- 172.21.15.5:0/981975448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55978c162b90 0x55978c16a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:50:17.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.345+0000 7f9e10a00640 1 --2- 172.21.15.5:0/981975448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55978c162270 0x55978c16a150 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc09cb00 tx=0x7f9dfc00ef80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=352c81b7a53f1f62 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:17.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.345+0000 7f9dfa000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:50:17.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.345+0000 7f9dfa000640 1 -- 172.21.15.5:0/981975448 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc010e60 con 0x55978c162270 2024-04-24T06:50:17.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.345+0000 7f9dfa000640 1 -- 172.21.15.5:0/981975448 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f9dfc0bd0b0 con 0x55978c162270 2024-04-24T06:50:17.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.345+0000 7f9e161f4740 1 -- 172.21.15.5:0/981975448 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9df40043d0 con 0x55978c162270 2024-04-24T06:50:17.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.345+0000 7f9dfa000640 1 -- 172.21.15.5:0/981975448 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9dfc09cbd0 con 0x55978c162270 2024-04-24T06:50:17.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.345+0000 7f9e161f4740 10 client.16512 fetch_fsmap learned FSMap version 27 2024-04-24T06:50:17.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.345+0000 7f9e161f4740 1 -- 172.21.15.5:0/981975448 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55978bd9a170 con 0x55978c162270 2024-04-24T06:50:17.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.345+0000 7f9dfa000640 1 -- 172.21.15.5:0/981975448 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9df40043d0 con 0x55978c162270 2024-04-24T06:50:17.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.346+0000 7f9dfa000640 1 -- 172.21.15.5:0/981975448 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f9dfc0b6140 con 0x55978c162270 2024-04-24T06:50:17.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.346+0000 7f9e161f4740 10 client.16512 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:50:17.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.346+0000 7f9e161f4740 10 client.16512 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:50:17.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.346+0000 7f9e161f4740 20 client.16512 populate_metadata read hostname 'smithi005' 2024-04-24T06:50:17.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.346+0000 7f9e161f4740 1 --2- 172.21.15.5:0/981975448 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55978c17d5d0 0x55978c17fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:17.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.346+0000 7f9e161f4740 4 client.16512 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:50:17.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.346+0000 7f9e161f4740 1 -- 172.21.15.5:0/981975448 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55978c0d5890 con 0x55978c17d5d0 2024-04-24T06:50:17.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.346+0000 7f9e03e00640 1 --2- 172.21.15.5:0/981975448 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55978c17d5d0 0x55978c17fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:17.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.347+0000 7f9e03e00640 1 --2- 172.21.15.5:0/981975448 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55978c17d5d0 0x55978c17fa00 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb38b0362091a07d server_cookie=7ad48080bf30ee87 in_seq=0 out_seq=0 2024-04-24T06:50:17.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.347+0000 7f9dfa000640 10 client.16512 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:50:17.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.348+0000 7f9dfa000640 1 -- 172.21.15.5:0/981975448 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55978c0d5890 con 0x55978c17d5d0 2024-04-24T06:50:17.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.348+0000 7f9dfa000640 10 client.16512 handle_command_reply: tid=0 2024-04-24T06:50:17.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.348+0000 7f9e161f4740 1 client.16512 shutdown 2024-04-24T06:50:17.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.348+0000 7f9e161f4740 1 -- 172.21.15.5:0/981975448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55978c162270 msgr2=0x55978c16a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:17.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.348+0000 7f9e161f4740 1 --2- 172.21.15.5:0/981975448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55978c162270 0x55978c16a150 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc09cb00 tx=0x7f9dfc00ef80 comp rx=0 tx=0).stop 2024-04-24T06:50:17.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.348+0000 7f9e161f4740 1 -- 172.21.15.5:0/981975448 shutdown_connections 2024-04-24T06:50:17.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.348+0000 7f9e161f4740 1 --2- 172.21.15.5:0/981975448 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55978c17d5d0 0x55978c17fa00 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.348+0000 7f9e161f4740 1 --2- 172.21.15.5:0/981975448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55978c16d260 0x55978c16abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.348+0000 7f9e161f4740 1 --2- 172.21.15.5:0/981975448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55978c162b90 0x55978c16a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.348+0000 7f9dfa000640 0 client.16512 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:50:17.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.348+0000 7f9e161f4740 1 --2- 172.21.15.5:0/981975448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55978c162270 0x55978c16a150 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.348+0000 7f9e161f4740 1 -- 172.21.15.5:0/981975448 >> 172.21.15.5:0/981975448 conn(0x55978c15e590 msgr2=0x55978c164290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:17.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.349+0000 7f9e161f4740 1 -- 172.21.15.5:0/981975448 shutdown_connections 2024-04-24T06:50:17.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.349+0000 7f9e161f4740 1 -- 172.21.15.5:0/981975448 wait complete. 2024-04-24T06:50:17.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.349+0000 7f9e161f4740 20 client.16512 trim_cache size 0 max 16384 2024-04-24T06:50:17.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.368+0000 7f9e161f4740 1 Processor -- start 2024-04-24T06:50:17.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.368+0000 7f9e161f4740 1 -- start start 2024-04-24T06:50:17.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.369+0000 7f9e161f4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55978c162270 0x55978c178f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:17.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.369+0000 7f9e161f4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55978c162b90 0x55978c1852a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:17.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.369+0000 7f9e161f4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55978c16d260 0x55978c1857e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:17.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.369+0000 7f9e161f4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0c1b6a40 con 0x55978c16d260 2024-04-24T06:50:17.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.369+0000 7f9e161f4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0c1b4310 con 0x55978c162270 2024-04-24T06:50:17.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.369+0000 7f9e161f4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0c1b2160 con 0x55978c162b90 2024-04-24T06:50:17.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.369+0000 7f9e11400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55978c16d260 0x55978c1857e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:17.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.369+0000 7f9e10a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55978c162270 0x55978c178f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:17.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.369+0000 7f9e11400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55978c16d260 0x55978c1857e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36334/0 (socket says 172.21.15.5:36334) 2024-04-24T06:50:17.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.369+0000 7f9e11400640 1 -- 172.21.15.5:0/3139607261 learned_addr learned my addr 172.21.15.5:0/3139607261 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:17.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.369+0000 7f9e10a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55978c162270 0x55978c178f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:33500/0 (socket says 172.21.15.5:33500) 2024-04-24T06:50:17.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.369+0000 7f9e03e00640 1 --2- 172.21.15.5:0/3139607261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55978c162b90 0x55978c1852a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:17.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.369+0000 7f9e10a00640 1 -- 172.21.15.5:0/3139607261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55978c162b90 msgr2=0x55978c1852a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:17.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.369+0000 7f9e10a00640 1 --2- 172.21.15.5:0/3139607261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55978c162b90 0x55978c1852a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.369+0000 7f9e10a00640 1 -- 172.21.15.5:0/3139607261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55978c16d260 msgr2=0x55978c1857e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:17.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.369+0000 7f9e10a00640 1 --2- 172.21.15.5:0/3139607261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55978c16d260 0x55978c1857e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.370+0000 7f9e10a00640 1 -- 172.21.15.5:0/3139607261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55978c159e80 con 0x55978c162270 2024-04-24T06:50:17.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.370+0000 7f9e10a00640 1 --2- 172.21.15.5:0/3139607261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55978c162270 0x55978c178f40 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc09cb00 tx=0x7f9dfc00cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35340a7c026935b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:17.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.370+0000 7f9e01200640 1 -- 172.21.15.5:0/3139607261 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc010e60 con 0x55978c162270 2024-04-24T06:50:17.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.370+0000 7f9e01200640 1 -- 172.21.15.5:0/3139607261 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dfc0d45e0 con 0x55978c162270 2024-04-24T06:50:17.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.370+0000 7f9e01200640 1 -- 172.21.15.5:0/3139607261 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc09c760 con 0x55978c162270 2024-04-24T06:50:17.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.371+0000 7f9e161f4740 1 -- 172.21.15.5:0/3139607261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55978c162270 msgr2=0x55978c178f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:17.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.371+0000 7f9e161f4740 1 --2- 172.21.15.5:0/3139607261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55978c162270 0x55978c178f40 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc09cb00 tx=0x7f9dfc00cf90 comp rx=0 tx=0).stop 2024-04-24T06:50:17.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.371+0000 7f9e161f4740 1 -- 172.21.15.5:0/3139607261 shutdown_connections 2024-04-24T06:50:17.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.371+0000 7f9e161f4740 1 --2- 172.21.15.5:0/3139607261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55978c16d260 0x55978c1857e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.371+0000 7f9e161f4740 1 --2- 172.21.15.5:0/3139607261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55978c162b90 0x55978c1852a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.371+0000 7f9e161f4740 1 --2- 172.21.15.5:0/3139607261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55978c162270 0x55978c178f40 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.371+0000 7f9e161f4740 1 -- 172.21.15.5:0/3139607261 >> 172.21.15.5:0/3139607261 conn(0x55978c15e590 msgr2=0x55978c15d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:17.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.371+0000 7f9e161f4740 1 -- 172.21.15.5:0/3139607261 shutdown_connections 2024-04-24T06:50:17.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.371+0000 7f9e161f4740 1 -- 172.21.15.5:0/3139607261 wait complete. 2024-04-24T06:50:17.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.371+0000 7f9e161f4740 1 -- start start 2024-04-24T06:50:17.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.372+0000 7f9e161f4740 1 Processor -- start 2024-04-24T06:50:17.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.372+0000 7f9e161f4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55978c162270 0x55978c171540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:17.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.372+0000 7f9e161f4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55978c162b90 0x55978c171a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:17.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.372+0000 7f9e161f4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55978c16d260 0x55978c1e2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:17.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.372+0000 7f9e161f4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0c1b27d0 con 0x55978c162b90 2024-04-24T06:50:17.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.372+0000 7f9e161f4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0c1b3c40 con 0x55978c16d260 2024-04-24T06:50:17.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.372+0000 7f9e161f4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0c1b2ea0 con 0x55978c162270 2024-04-24T06:50:17.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.372+0000 7f9e161f4740 10 client.0 _ll_register_callbacks cb 0x7f9dfc0099b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:50:17.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.372+0000 7f9e03e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55978c162b90 0x55978c171a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:17.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.372+0000 7f9e11400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55978c16d260 0x55978c1e2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:17.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.372+0000 7f9e03e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55978c162b90 0x55978c171a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36348/0 (socket says 172.21.15.5:36348) 2024-04-24T06:50:17.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.372+0000 7f9e03e00640 1 -- 172.21.15.5:0/3607108844 learned_addr learned my addr 172.21.15.5:0/3607108844 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:17.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.372+0000 7f9e10a00640 1 --2- 172.21.15.5:0/3607108844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55978c162270 0x55978c171540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:17.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.373+0000 7f9e03e00640 1 -- 172.21.15.5:0/3607108844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55978c162270 msgr2=0x55978c171540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:17.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.373+0000 7f9e03e00640 1 --2- 172.21.15.5:0/3607108844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55978c162270 0x55978c171540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.373+0000 7f9e03e00640 1 -- 172.21.15.5:0/3607108844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55978c16d260 msgr2=0x55978c1e2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:17.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.373+0000 7f9e03e00640 1 --2- 172.21.15.5:0/3607108844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55978c16d260 0x55978c1e2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.373+0000 7f9e03e00640 1 -- 172.21.15.5:0/3607108844 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55978c1e3050 con 0x55978c162b90 2024-04-24T06:50:17.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.373+0000 7f9e11400640 1 --2- 172.21.15.5:0/3607108844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55978c16d260 0x55978c1e2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:50:17.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.373+0000 7f9e03e00640 1 --2- 172.21.15.5:0/3607108844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55978c162b90 0x55978c171a80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9df4009710 tx=0x7f9df4009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab8ebe89a2d2be3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:17.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.373+0000 7f9dfa000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:50:17.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.373+0000 7f9dfa000640 1 -- 172.21.15.5:0/3607108844 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df401f040 con 0x55978c162b90 2024-04-24T06:50:17.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.373+0000 7f9dfa000640 1 -- 172.21.15.5:0/3607108844 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f9df40052c0 con 0x55978c162b90 2024-04-24T06:50:17.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.373+0000 7f9e161f4740 1 -- 172.21.15.5:0/3607108844 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9dfc0d45e0 con 0x55978c162b90 2024-04-24T06:50:17.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.374+0000 7f9dfa000640 1 -- 172.21.15.5:0/3607108844 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9df40054c0 con 0x55978c162b90 2024-04-24T06:50:17.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.374+0000 7f9e161f4740 10 client.16518 fetch_fsmap learned FSMap version 27 2024-04-24T06:50:17.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.374+0000 7f9e161f4740 1 -- 172.21.15.5:0/3607108844 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55978c1e4330 con 0x55978c162b90 2024-04-24T06:50:17.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.374+0000 7f9dfa000640 1 -- 172.21.15.5:0/3607108844 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dfc0d45e0 con 0x55978c162b90 2024-04-24T06:50:17.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.374+0000 7f9dfa000640 1 -- 172.21.15.5:0/3607108844 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f9df4033050 con 0x55978c162b90 2024-04-24T06:50:17.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.374+0000 7f9e161f4740 10 client.16518 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:50:17.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.374+0000 7f9e161f4740 10 client.16518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:50:17.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.374+0000 7f9e161f4740 20 client.16518 populate_metadata read hostname 'smithi005' 2024-04-24T06:50:17.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.374+0000 7f9e161f4740 1 --2- 172.21.15.5:0/3607108844 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55978c1e4970 0x55978c1e6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:17.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.374+0000 7f9e161f4740 4 client.16518 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T06:50:17.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.374+0000 7f9e161f4740 1 -- 172.21.15.5:0/3607108844 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55978c0cf700 con 0x55978c1e4970 2024-04-24T06:50:17.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.375+0000 7f9e10a00640 1 --2- 172.21.15.5:0/3607108844 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55978c1e4970 0x55978c1e6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:17.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.375+0000 7f9e10a00640 1 --2- 172.21.15.5:0/3607108844 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55978c1e4970 0x55978c1e6d90 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=559fa9cdf40cddf9 server_cookie=dfee2b531b07b6d3 in_seq=0 out_seq=0 2024-04-24T06:50:17.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.375+0000 7f9dfa000640 10 client.16518 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:50:17.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.376+0000 7f9dfa000640 1 -- 172.21.15.5:0/3607108844 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55978c0cf700 con 0x55978c1e4970 2024-04-24T06:50:17.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.376+0000 7f9dfa000640 10 client.16518 handle_command_reply: tid=0 2024-04-24T06:50:17.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.377+0000 7f9e161f4740 1 client.16518 shutdown 2024-04-24T06:50:17.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.377+0000 7f9e161f4740 1 -- 172.21.15.5:0/3607108844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55978c162b90 msgr2=0x55978c171a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:17.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.377+0000 7f9e161f4740 1 --2- 172.21.15.5:0/3607108844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55978c162b90 0x55978c171a80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9df4009710 tx=0x7f9df4009f00 comp rx=0 tx=0).stop 2024-04-24T06:50:17.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.377+0000 7f9e161f4740 1 -- 172.21.15.5:0/3607108844 shutdown_connections 2024-04-24T06:50:17.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.377+0000 7f9e161f4740 1 --2- 172.21.15.5:0/3607108844 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55978c1e4970 0x55978c1e6d90 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.377+0000 7f9e161f4740 1 --2- 172.21.15.5:0/3607108844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55978c16d260 0x55978c1e2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.377+0000 7f9e161f4740 1 --2- 172.21.15.5:0/3607108844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55978c162b90 0x55978c171a80 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.377+0000 7f9e161f4740 1 --2- 172.21.15.5:0/3607108844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55978c162270 0x55978c171540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.377+0000 7f9dfa000640 0 client.16518 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:50:17.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.377+0000 7f9e161f4740 1 -- 172.21.15.5:0/3607108844 >> 172.21.15.5:0/3607108844 conn(0x55978c15e590 msgr2=0x55978c16a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:17.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.377+0000 7f9e161f4740 1 -- 172.21.15.5:0/3607108844 shutdown_connections 2024-04-24T06:50:17.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.377+0000 7f9e161f4740 1 -- 172.21.15.5:0/3607108844 wait complete. 2024-04-24T06:50:17.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.377+0000 7f9e161f4740 20 client.16518 trim_cache size 0 max 16384 2024-04-24T06:50:17.389 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:50:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-24T06:50:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "4ba79d26-e007-41a2-b0f3-41e7963806d1", 2024-04-24T06:50:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-24T06:50:17.389 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:50:17.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.378+0000 7f9e11e00640 1 -- 172.21.15.5:0/450077910 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f9df00a3da0 msgr2=0x7f9df00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:17.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.378+0000 7f9e11e00640 1 --2- 172.21.15.5:0/450077910 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f9df00a3da0 0x7f9df00a6260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9e04006010 tx=0x7f9e04009600 comp rx=0 tx=0).stop 2024-04-24T06:50:17.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.378+0000 7f9e11e00640 1 -- 172.21.15.5:0/450077910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e0c076530 msgr2=0x7f9e0c1a4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:17.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.378+0000 7f9e11e00640 1 --2- 172.21.15.5:0/450077910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e0c076530 0x7f9e0c1a4940 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc00ebc0 tx=0x7f9dfc00c6a0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.378+0000 7f9e11e00640 1 -- 172.21.15.5:0/450077910 shutdown_connections 2024-04-24T06:50:17.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.378+0000 7f9e11e00640 1 --2- 172.21.15.5:0/450077910 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f9df00a3da0 0x7f9df00a6260 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.378+0000 7f9e11e00640 1 --2- 172.21.15.5:0/450077910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e0c112c90 0x7f9e0c19eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.378+0000 7f9e11e00640 1 --2- 172.21.15.5:0/450077910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e0c076e50 0x7f9e0c1a4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.378+0000 7f9e11e00640 1 --2- 172.21.15.5:0/450077910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e0c076530 0x7f9e0c1a4940 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.378+0000 7f9e11e00640 1 -- 172.21.15.5:0/450077910 >> 172.21.15.5:0/450077910 conn(0x7f9e0c0ffaa0 msgr2=0x7f9e0c1142b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:17.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.378+0000 7f9e11e00640 1 -- 172.21.15.5:0/450077910 shutdown_connections 2024-04-24T06:50:17.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.378+0000 7f9e11e00640 1 -- 172.21.15.5:0/450077910 wait complete. 2024-04-24T06:50:17.405 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T06:50:17.405 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T06:50:17.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.635+0000 7f506d600640 1 -- 172.21.15.5:0/2711781812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5068102070 msgr2=0x7f5068102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:17.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.635+0000 7f506d600640 1 --2- 172.21.15.5:0/2711781812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5068102070 0x7f5068102450 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5058009900 tx=0x7f505802a3f0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.635+0000 7f506d600640 1 -- 172.21.15.5:0/2711781812 shutdown_connections 2024-04-24T06:50:17.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.635+0000 7f506d600640 1 --2- 172.21.15.5:0/2711781812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5068107140 0x7f506810ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.635+0000 7f506d600640 1 --2- 172.21.15.5:0/2711781812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5068102a20 0x7f5068106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.635+0000 7f506d600640 1 --2- 172.21.15.5:0/2711781812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5068102070 0x7f5068102450 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.635+0000 7f506d600640 1 -- 172.21.15.5:0/2711781812 >> 172.21.15.5:0/2711781812 conn(0x7f50680fd950 msgr2=0x7f50680ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:17.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.635+0000 7f506d600640 1 -- 172.21.15.5:0/2711781812 shutdown_connections 2024-04-24T06:50:17.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.635+0000 7f506d600640 1 -- 172.21.15.5:0/2711781812 wait complete. 2024-04-24T06:50:17.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.636+0000 7f506d600640 1 Processor -- start 2024-04-24T06:50:17.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.637+0000 7f506d600640 1 -- start start 2024-04-24T06:50:17.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.637+0000 7f506d600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5068102070 0x7f50681a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:17.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.637+0000 7f506d600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5068107140 0x7f50681a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:17.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.637+0000 7f506d600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f506819a730 0x7f506819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:17.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.637+0000 7f506d600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f50681121d0 con 0x7f5068107140 2024-04-24T06:50:17.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.637+0000 7f506d600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5068112050 con 0x7f5068102070 2024-04-24T06:50:17.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.637+0000 7f506d600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5068112350 con 0x7f506819a730 2024-04-24T06:50:17.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.637+0000 7f5067e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5068102070 0x7f50681a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:17.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.637+0000 7f5067e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5068102070 0x7f50681a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:33532/0 (socket says 172.21.15.5:33532) 2024-04-24T06:50:17.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.637+0000 7f5067e00640 1 -- 172.21.15.5:0/1815707578 learned_addr learned my addr 172.21.15.5:0/1815707578 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:17.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.637+0000 7f506cc00640 1 --2- 172.21.15.5:0/1815707578 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f506819a730 0x7f506819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:17.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.638+0000 7f5067400640 1 --2- 172.21.15.5:0/1815707578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5068107140 0x7f50681a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:17.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.638+0000 7f5067e00640 1 -- 172.21.15.5:0/1815707578 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f506819a730 msgr2=0x7f506819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:17.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.638+0000 7f5067e00640 1 --2- 172.21.15.5:0/1815707578 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f506819a730 0x7f506819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.638+0000 7f5067e00640 1 -- 172.21.15.5:0/1815707578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5068107140 msgr2=0x7f50681a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:17.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.638+0000 7f5067e00640 1 --2- 172.21.15.5:0/1815707578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5068107140 0x7f50681a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.638+0000 7f5067e00640 1 -- 172.21.15.5:0/1815707578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f506819b330 con 0x7f5068102070 2024-04-24T06:50:17.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.638+0000 7f5067400640 1 --2- 172.21.15.5:0/1815707578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5068107140 0x7f50681a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:50:17.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.638+0000 7f506cc00640 1 --2- 172.21.15.5:0/1815707578 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f506819a730 0x7f506819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:50:17.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.638+0000 7f5067e00640 1 --2- 172.21.15.5:0/1815707578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5068102070 0x7f50681a0670 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5058006010 tx=0x7f50580042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:17.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.639+0000 7f5064c00640 1 -- 172.21.15.5:0/1815707578 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505803f030 con 0x7f5068102070 2024-04-24T06:50:17.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.639+0000 7f5064c00640 1 -- 172.21.15.5:0/1815707578 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5058003ce0 con 0x7f5068102070 2024-04-24T06:50:17.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.639+0000 7f5064c00640 1 -- 172.21.15.5:0/1815707578 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5058035070 con 0x7f5068102070 2024-04-24T06:50:17.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.639+0000 7f506d600640 1 -- 172.21.15.5:0/1815707578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f506819b5c0 con 0x7f5068102070 2024-04-24T06:50:17.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.639+0000 7f506d600640 1 -- 172.21.15.5:0/1815707578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50681a76b0 con 0x7f5068102070 2024-04-24T06:50:17.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.644+0000 7f5071a35740 1 Processor -- start 2024-04-24T06:50:17.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.644+0000 7f5064c00640 1 -- 172.21.15.5:0/1815707578 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f5058042030 con 0x7f5068102070 2024-04-24T06:50:17.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.644+0000 7f5071a35740 1 -- start start 2024-04-24T06:50:17.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.645+0000 7f5071a35740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee8f1cc780 0x55ee8f1ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:17.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.645+0000 7f5071a35740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee8f1cd130 0x55ee8f1dfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:17.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.645+0000 7f5071a35740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8f1e0230 0x55ee8f1e2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:17.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.645+0000 7f5071a35740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ee8f1321f0 con 0x55ee8f1e0230 2024-04-24T06:50:17.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.645+0000 7f5071a35740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ee8f0d8350 con 0x55ee8f1cd130 2024-04-24T06:50:17.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.645+0000 7f5071a35740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ee8f12e1b0 con 0x55ee8f1cc780 2024-04-24T06:50:17.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.645+0000 7f5067e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee8f1cd130 0x55ee8f1dfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:17.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.645+0000 7f506cc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee8f1cc780 0x55ee8f1ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:17.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.645+0000 7f5067e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee8f1cd130 0x55ee8f1dfcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:33544/0 (socket says 172.21.15.5:33544) 2024-04-24T06:50:17.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.645+0000 7f5067e00640 1 -- 172.21.15.5:0/2196135532 learned_addr learned my addr 172.21.15.5:0/2196135532 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:17.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.645+0000 7f5067400640 1 --2- 172.21.15.5:0/2196135532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8f1e0230 0x55ee8f1e2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:17.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.646+0000 7f5067e00640 1 -- 172.21.15.5:0/2196135532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee8f1cc780 msgr2=0x55ee8f1ccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:17.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.646+0000 7f5067e00640 1 --2- 172.21.15.5:0/2196135532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee8f1cc780 0x55ee8f1ccb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.646+0000 7f5067e00640 1 -- 172.21.15.5:0/2196135532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8f1e0230 msgr2=0x55ee8f1e2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:17.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.646+0000 7f5067e00640 1 --2- 172.21.15.5:0/2196135532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8f1e0230 0x55ee8f1e2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.646+0000 7f5067e00640 1 -- 172.21.15.5:0/2196135532 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee8f12cf50 con 0x55ee8f1cd130 2024-04-24T06:50:17.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.646+0000 7f5064c00640 1 --2- 172.21.15.5:0/1815707578 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f504c0a3da0 0x7f504c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:17.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.646+0000 7f506cc00640 1 --2- 172.21.15.5:0/2196135532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee8f1cc780 0x55ee8f1ccb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:50:17.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.646+0000 7f5064c00640 1 -- 172.21.15.5:0/1815707578 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f50580f0d10 con 0x7f5068102070 2024-04-24T06:50:17.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.646+0000 7f5067e00640 1 --2- 172.21.15.5:0/2196135532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee8f1cd130 0x55ee8f1dfcf0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f50580ee350 tx=0x7f50580f4f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df1513574500d31c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:17.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.646+0000 7f506cc00640 1 --2- 172.21.15.5:0/1815707578 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f504c0a3da0 0x7f504c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:17.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.647+0000 7f505d200640 1 -- 172.21.15.5:0/2196135532 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50580bca30 con 0x55ee8f1cd130 2024-04-24T06:50:17.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.647+0000 7f505d200640 1 -- 172.21.15.5:0/2196135532 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f505803b580 con 0x55ee8f1cd130 2024-04-24T06:50:17.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.647+0000 7f505d200640 1 -- 172.21.15.5:0/2196135532 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50580044e0 con 0x55ee8f1cd130 2024-04-24T06:50:17.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.648+0000 7f5071a35740 1 -- 172.21.15.5:0/2196135532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee8f1cd130 msgr2=0x55ee8f1dfcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:17.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.648+0000 7f506cc00640 1 --2- 172.21.15.5:0/1815707578 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f504c0a3da0 0x7f504c0a6260 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5060005fd0 tx=0x7f5060009600 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:17.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.648+0000 7f5071a35740 1 --2- 172.21.15.5:0/2196135532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee8f1cd130 0x55ee8f1dfcf0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f50580ee350 tx=0x7f50580f4f40 comp rx=0 tx=0).stop 2024-04-24T06:50:17.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.648+0000 7f5071a35740 1 -- 172.21.15.5:0/2196135532 shutdown_connections 2024-04-24T06:50:17.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.648+0000 7f5071a35740 1 --2- 172.21.15.5:0/2196135532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8f1e0230 0x55ee8f1e2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.648+0000 7f5071a35740 1 --2- 172.21.15.5:0/2196135532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee8f1cd130 0x55ee8f1dfcf0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.648+0000 7f5071a35740 1 --2- 172.21.15.5:0/2196135532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee8f1cc780 0x55ee8f1ccb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.648+0000 7f5071a35740 1 -- 172.21.15.5:0/2196135532 >> 172.21.15.5:0/2196135532 conn(0x55ee8f1c8d50 msgr2=0x55ee8f1c9130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:17.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.649+0000 7f5071a35740 1 -- 172.21.15.5:0/2196135532 shutdown_connections 2024-04-24T06:50:17.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.649+0000 7f5071a35740 1 -- 172.21.15.5:0/2196135532 wait complete. 2024-04-24T06:50:17.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.649+0000 7f5071a35740 1 -- start start 2024-04-24T06:50:17.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.650+0000 7f5071a35740 1 Processor -- start 2024-04-24T06:50:17.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.650+0000 7f5071a35740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee8f1cc780 0x55ee8f1dd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:17.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.650+0000 7f5071a35740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8f1cd130 0x55ee8f1dd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:17.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.650+0000 7f5071a35740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee8f1e0230 0x55ee8f1ddca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:17.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.650+0000 7f5071a35740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ee8f1e2db0 con 0x55ee8f1cd130 2024-04-24T06:50:17.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.650+0000 7f5071a35740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ee8f0eb690 con 0x55ee8f1e0230 2024-04-24T06:50:17.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.650+0000 7f5071a35740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ee8f1e2f30 con 0x55ee8f1cc780 2024-04-24T06:50:17.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.650+0000 7f5071a35740 10 client.0 _ll_register_callbacks cb 0x55ee8f127650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:50:17.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.651+0000 7f5067e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee8f1cc780 0x55ee8f1dd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:17.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.651+0000 7f5067e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee8f1cc780 0x55ee8f1dd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:60052/0 (socket says 172.21.15.5:60052) 2024-04-24T06:50:17.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.651+0000 7f5067e00640 1 -- 172.21.15.5:0/804954181 learned_addr learned my addr 172.21.15.5:0/804954181 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:17.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.651+0000 7f506cc00640 1 --2- 172.21.15.5:0/804954181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee8f1e0230 0x55ee8f1ddca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:17.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.651+0000 7f5067e00640 1 -- 172.21.15.5:0/804954181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee8f1e0230 msgr2=0x55ee8f1ddca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:17.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.651+0000 7f5067e00640 1 --2- 172.21.15.5:0/804954181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee8f1e0230 0x55ee8f1ddca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.651+0000 7f5067e00640 1 -- 172.21.15.5:0/804954181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8f1cd130 msgr2=0x55ee8f1dd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:17.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.651+0000 7f5067400640 1 --2- 172.21.15.5:0/804954181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8f1cd130 0x55ee8f1dd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:17.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.651+0000 7f5067e00640 1 --2- 172.21.15.5:0/804954181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8f1cd130 0x55ee8f1dd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.651+0000 7f5067e00640 1 -- 172.21.15.5:0/804954181 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee8f0ea920 con 0x55ee8f1cc780 2024-04-24T06:50:17.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.651+0000 7f5067400640 1 --2- 172.21.15.5:0/804954181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8f1cd130 0x55ee8f1dd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:50:17.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.652+0000 7f5067e00640 1 --2- 172.21.15.5:0/804954181 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee8f1cc780 0x55ee8f1dd220 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f50580f5010 tx=0x7f50580f4b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7df773c609c7d456 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:17.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.652+0000 7f5051600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:50:17.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.652+0000 7f5051600640 1 -- 172.21.15.5:0/804954181 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5058049040 con 0x55ee8f1cc780 2024-04-24T06:50:17.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.652+0000 7f5051600640 1 -- 172.21.15.5:0/804954181 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f50580ee690 con 0x55ee8f1cc780 2024-04-24T06:50:17.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.652+0000 7f5071a35740 1 -- 172.21.15.5:0/804954181 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ee8f13fc10 con 0x55ee8f1cc780 2024-04-24T06:50:17.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.652+0000 7f5051600640 1 -- 172.21.15.5:0/804954181 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f505803bcb0 con 0x55ee8f1cc780 2024-04-24T06:50:17.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.652+0000 7f5071a35740 10 client.26677 fetch_fsmap learned FSMap version 27 2024-04-24T06:50:17.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.652+0000 7f5071a35740 1 -- 172.21.15.5:0/804954181 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ee8ef0e7d0 con 0x55ee8f1cc780 2024-04-24T06:50:17.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.653+0000 7f5051600640 1 -- 172.21.15.5:0/804954181 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ee8f13fc10 con 0x55ee8f1cc780 2024-04-24T06:50:17.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.653+0000 7f5051600640 1 -- 172.21.15.5:0/804954181 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f50580f5040 con 0x55ee8f1cc780 2024-04-24T06:50:17.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.653+0000 7f5071a35740 10 client.26677 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:50:17.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.653+0000 7f5071a35740 10 client.26677 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:50:17.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.653+0000 7f5071a35740 20 client.26677 populate_metadata read hostname 'smithi005' 2024-04-24T06:50:17.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.653+0000 7f5071a35740 1 --2- 172.21.15.5:0/804954181 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55ee8f1f01b0 0x55ee8f1f25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:17.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.653+0000 7f5071a35740 4 client.26677 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:50:17.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.653+0000 7f5071a35740 1 -- 172.21.15.5:0/804954181 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee8f139a80 con 0x55ee8f1f01b0 2024-04-24T06:50:17.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.653+0000 7f5067400640 1 --2- 172.21.15.5:0/804954181 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55ee8f1f01b0 0x55ee8f1f25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:17.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.654+0000 7f5051600640 10 client.26677 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:50:17.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.654+0000 7f5067400640 1 --2- 172.21.15.5:0/804954181 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55ee8f1f01b0 0x55ee8f1f25a0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9412d473f02b28f6 server_cookie=eb79a96652de2ece in_seq=0 out_seq=0 2024-04-24T06:50:17.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.655+0000 7f5051600640 1 -- 172.21.15.5:0/804954181 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ee8f139a80 con 0x55ee8f1f01b0 2024-04-24T06:50:17.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.655+0000 7f5051600640 10 client.26677 handle_command_reply: tid=0 2024-04-24T06:50:17.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.655+0000 7f5071a35740 1 client.26677 shutdown 2024-04-24T06:50:17.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.656+0000 7f5071a35740 1 -- 172.21.15.5:0/804954181 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee8f1cc780 msgr2=0x55ee8f1dd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:17.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.656+0000 7f5071a35740 1 --2- 172.21.15.5:0/804954181 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee8f1cc780 0x55ee8f1dd220 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f50580f5010 tx=0x7f50580f4b90 comp rx=0 tx=0).stop 2024-04-24T06:50:17.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.656+0000 7f5071a35740 1 -- 172.21.15.5:0/804954181 shutdown_connections 2024-04-24T06:50:17.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.656+0000 7f5071a35740 1 --2- 172.21.15.5:0/804954181 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55ee8f1f01b0 0x55ee8f1f25a0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.656+0000 7f5071a35740 1 --2- 172.21.15.5:0/804954181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee8f1e0230 0x55ee8f1ddca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.656+0000 7f5051600640 0 client.26677 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:50:17.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.656+0000 7f5071a35740 1 --2- 172.21.15.5:0/804954181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8f1cd130 0x55ee8f1dd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.656+0000 7f5071a35740 1 --2- 172.21.15.5:0/804954181 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee8f1cc780 0x55ee8f1dd220 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.656+0000 7f5071a35740 1 -- 172.21.15.5:0/804954181 >> 172.21.15.5:0/804954181 conn(0x55ee8f1c8d50 msgr2=0x55ee8f1cebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:17.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.656+0000 7f5071a35740 1 -- 172.21.15.5:0/804954181 shutdown_connections 2024-04-24T06:50:17.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.656+0000 7f5071a35740 1 -- 172.21.15.5:0/804954181 wait complete. 2024-04-24T06:50:17.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.656+0000 7f5071a35740 20 client.26677 trim_cache size 0 max 16384 2024-04-24T06:50:17.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.681+0000 7f5071a35740 1 Processor -- start 2024-04-24T06:50:17.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.681+0000 7f5071a35740 1 -- start start 2024-04-24T06:50:17.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.681+0000 7f5071a35740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8f1c8d50 0x55ee8f1f55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:17.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.682+0000 7f5071a35740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee8f1cc780 0x55ee8f1f5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:17.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.682+0000 7f5071a35740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee8f1cd130 0x55ee8f1da4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:17.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.682+0000 7f5071a35740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f50681ae270 con 0x55ee8f1c8d50 2024-04-24T06:50:17.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.682+0000 7f5071a35740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f50681abb40 con 0x55ee8f1cc780 2024-04-24T06:50:17.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.682+0000 7f5071a35740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f50681a99b0 con 0x55ee8f1cd130 2024-04-24T06:50:17.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.682+0000 7f5067400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee8f1cc780 0x55ee8f1f5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:17.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.682+0000 7f506cc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee8f1cd130 0x55ee8f1da4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:17.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.682+0000 7f5067400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee8f1cc780 0x55ee8f1f5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:33574/0 (socket says 172.21.15.5:33574) 2024-04-24T06:50:17.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.682+0000 7f5067400640 1 -- 172.21.15.5:0/2712780113 learned_addr learned my addr 172.21.15.5:0/2712780113 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:17.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.682+0000 7f5067e00640 1 --2- 172.21.15.5:0/2712780113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8f1c8d50 0x55ee8f1f55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:17.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.682+0000 7f5067400640 1 -- 172.21.15.5:0/2712780113 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee8f1cd130 msgr2=0x55ee8f1da4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:17.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.682+0000 7f5067400640 1 --2- 172.21.15.5:0/2712780113 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee8f1cd130 0x55ee8f1da4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.682+0000 7f5067400640 1 -- 172.21.15.5:0/2712780113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8f1c8d50 msgr2=0x55ee8f1f55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:17.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.682+0000 7f5067400640 1 --2- 172.21.15.5:0/2712780113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8f1c8d50 0x55ee8f1f55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.682+0000 7f5067400640 1 -- 172.21.15.5:0/2712780113 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee8ee443f0 con 0x55ee8f1cc780 2024-04-24T06:50:17.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.683+0000 7f5067400640 1 --2- 172.21.15.5:0/2712780113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee8f1cc780 0x55ee8f1f5b00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5054003860 tx=0x7f5054005d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4dccccaa4a18855 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:17.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.683+0000 7f5053e00640 1 -- 172.21.15.5:0/2712780113 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5054013b30 con 0x55ee8f1cc780 2024-04-24T06:50:17.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.683+0000 7f5053e00640 1 -- 172.21.15.5:0/2712780113 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f505401e030 con 0x55ee8f1cc780 2024-04-24T06:50:17.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.683+0000 7f5053e00640 1 -- 172.21.15.5:0/2712780113 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5054011640 con 0x55ee8f1cc780 2024-04-24T06:50:17.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.684+0000 7f5071a35740 1 -- 172.21.15.5:0/2712780113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee8f1cc780 msgr2=0x55ee8f1f5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:17.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.684+0000 7f5071a35740 1 --2- 172.21.15.5:0/2712780113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee8f1cc780 0x55ee8f1f5b00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5054003860 tx=0x7f5054005d00 comp rx=0 tx=0).stop 2024-04-24T06:50:17.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.684+0000 7f5071a35740 1 -- 172.21.15.5:0/2712780113 shutdown_connections 2024-04-24T06:50:17.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.684+0000 7f5071a35740 1 --2- 172.21.15.5:0/2712780113 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee8f1cd130 0x55ee8f1da4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.684+0000 7f5071a35740 1 --2- 172.21.15.5:0/2712780113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee8f1cc780 0x55ee8f1f5b00 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.684+0000 7f5071a35740 1 --2- 172.21.15.5:0/2712780113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8f1c8d50 0x55ee8f1f55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.684+0000 7f5071a35740 1 -- 172.21.15.5:0/2712780113 >> 172.21.15.5:0/2712780113 conn(0x55ee8f1dbe60 msgr2=0x55ee8f1eba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:17.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.684+0000 7f5071a35740 1 -- 172.21.15.5:0/2712780113 shutdown_connections 2024-04-24T06:50:17.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.684+0000 7f5071a35740 1 -- 172.21.15.5:0/2712780113 wait complete. 2024-04-24T06:50:17.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.685+0000 7f5071a35740 1 -- start start 2024-04-24T06:50:17.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.685+0000 7f5071a35740 1 Processor -- start 2024-04-24T06:50:17.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.686+0000 7f5071a35740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee8f1c8d50 0x55ee8f1f32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:17.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.686+0000 7f5071a35740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8f1cc780 0x55ee8f1f37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:17.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.686+0000 7f5071a35740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee8f1cd130 0x55ee8f1f3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:17.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.686+0000 7f5071a35740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f50681aa000 con 0x55ee8f1cc780 2024-04-24T06:50:17.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.686+0000 7f5071a35740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f50681ab470 con 0x55ee8f1c8d50 2024-04-24T06:50:17.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.686+0000 7f5071a35740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f50681aa6d0 con 0x55ee8f1cd130 2024-04-24T06:50:17.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.686+0000 7f5071a35740 10 client.0 _ll_register_callbacks cb 0x7f5060009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:50:17.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.686+0000 7f5067e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee8f1c8d50 0x55ee8f1f32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:17.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.686+0000 7f5067e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee8f1c8d50 0x55ee8f1f32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:33578/0 (socket says 172.21.15.5:33578) 2024-04-24T06:50:17.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.686+0000 7f5067e00640 1 -- 172.21.15.5:0/4281145700 learned_addr learned my addr 172.21.15.5:0/4281145700 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:17.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.686+0000 7f5067400640 1 --2- 172.21.15.5:0/4281145700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8f1cc780 0x55ee8f1f37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:17.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.686+0000 7f506cc00640 1 --2- 172.21.15.5:0/4281145700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee8f1cd130 0x55ee8f1f3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:17.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.686+0000 7f5067e00640 1 -- 172.21.15.5:0/4281145700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee8f1cd130 msgr2=0x55ee8f1f3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:17.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.686+0000 7f5067e00640 1 --2- 172.21.15.5:0/4281145700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee8f1cd130 0x55ee8f1f3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.686+0000 7f5067e00640 1 -- 172.21.15.5:0/4281145700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8f1cc780 msgr2=0x55ee8f1f37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:17.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.686+0000 7f5067e00640 1 --2- 172.21.15.5:0/4281145700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8f1cc780 0x55ee8f1f37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.686+0000 7f5067e00640 1 -- 172.21.15.5:0/4281145700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee8ee040f0 con 0x55ee8f1c8d50 2024-04-24T06:50:17.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.687+0000 7f5067e00640 1 --2- 172.21.15.5:0/4281145700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee8f1c8d50 0x55ee8f1f32a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5058005ab0 tx=0x7f5058009620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0367f3c52e08586 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:17.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.687+0000 7f5051600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:50:17.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.687+0000 7f5051600640 1 -- 172.21.15.5:0/4281145700 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5058049070 con 0x55ee8f1c8d50 2024-04-24T06:50:17.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.687+0000 7f5051600640 1 -- 172.21.15.5:0/4281145700 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f50580baf60 con 0x55ee8f1c8d50 2024-04-24T06:50:17.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.687+0000 7f5071a35740 1 -- 172.21.15.5:0/4281145700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f505401e030 con 0x55ee8f1c8d50 2024-04-24T06:50:17.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.688+0000 7f5051600640 1 -- 172.21.15.5:0/4281145700 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f50580bc5c0 con 0x55ee8f1c8d50 2024-04-24T06:50:17.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.688+0000 7f5071a35740 10 client.26672 fetch_fsmap learned FSMap version 27 2024-04-24T06:50:17.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.688+0000 7f5071a35740 1 -- 172.21.15.5:0/4281145700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ee8f0f7810 con 0x55ee8f1c8d50 2024-04-24T06:50:17.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.688+0000 7f5051600640 1 -- 172.21.15.5:0/4281145700 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f505401e030 con 0x55ee8f1c8d50 2024-04-24T06:50:17.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.688+0000 7f5051600640 1 -- 172.21.15.5:0/4281145700 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f50580ee690 con 0x55ee8f1c8d50 2024-04-24T06:50:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.688+0000 7f5071a35740 10 client.26672 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:50:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.688+0000 7f5071a35740 10 client.26672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:50:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.688+0000 7f5071a35740 20 client.26672 populate_metadata read hostname 'smithi005' 2024-04-24T06:50:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.688+0000 7f5071a35740 1 --2- 172.21.15.5:0/4281145700 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55ee8f1e0230 0x55ee8f2588e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.688+0000 7f5071a35740 4 client.26672 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:50:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.688+0000 7f5071a35740 1 -- 172.21.15.5:0/4281145700 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ee8f0b5360 con 0x55ee8f1e0230 2024-04-24T06:50:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.688+0000 7f5067400640 1 --2- 172.21.15.5:0/4281145700 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55ee8f1e0230 0x55ee8f2588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.689+0000 7f5067400640 1 --2- 172.21.15.5:0/4281145700 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55ee8f1e0230 0x55ee8f2588e0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9ea50dce51fe6e6 server_cookie=27045502fec1c730 in_seq=0 out_seq=0 2024-04-24T06:50:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.689+0000 7f5051600640 10 client.26672 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:50:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.690+0000 7f5051600640 1 -- 172.21.15.5:0/4281145700 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55ee8f0b5360 con 0x55ee8f1e0230 2024-04-24T06:50:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.690+0000 7f5051600640 10 client.26672 handle_command_reply: tid=0 2024-04-24T06:50:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.690+0000 7f5071a35740 1 client.26672 shutdown 2024-04-24T06:50:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.690+0000 7f5071a35740 1 -- 172.21.15.5:0/4281145700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee8f1c8d50 msgr2=0x55ee8f1f32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.690+0000 7f5071a35740 1 --2- 172.21.15.5:0/4281145700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee8f1c8d50 0x55ee8f1f32a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5058005ab0 tx=0x7f5058009620 comp rx=0 tx=0).stop 2024-04-24T06:50:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.690+0000 7f5071a35740 1 -- 172.21.15.5:0/4281145700 shutdown_connections 2024-04-24T06:50:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.690+0000 7f5071a35740 1 --2- 172.21.15.5:0/4281145700 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55ee8f1e0230 0x55ee8f2588e0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.690+0000 7f5071a35740 1 --2- 172.21.15.5:0/4281145700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee8f1cd130 0x55ee8f1f3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.690+0000 7f5071a35740 1 --2- 172.21.15.5:0/4281145700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee8f1cc780 0x55ee8f1f37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.690+0000 7f5071a35740 1 --2- 172.21.15.5:0/4281145700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee8f1c8d50 0x55ee8f1f32a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.690+0000 7f5071a35740 1 -- 172.21.15.5:0/4281145700 >> 172.21.15.5:0/4281145700 conn(0x55ee8f1dbe60 msgr2=0x55ee8f1ce2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:17.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.690+0000 7f5051600640 0 client.26672 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:50:17.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.690+0000 7f5071a35740 1 -- 172.21.15.5:0/4281145700 shutdown_connections 2024-04-24T06:50:17.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.690+0000 7f5071a35740 1 -- 172.21.15.5:0/4281145700 wait complete. 2024-04-24T06:50:17.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.691+0000 7f5071a35740 20 client.26672 trim_cache size 0 max 16384 2024-04-24T06:50:17.699 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:50:17.700 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (701 inodes in the stack)", 2024-04-24T06:50:17.700 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-24T06:50:17.700 INFO:teuthology.orchestra.run.smithi005.stdout: "4ba79d26-e007-41a2-b0f3-41e7963806d1": { 2024-04-24T06:50:17.700 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-24T06:50:17.700 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "4ba79d26-e007-41a2-b0f3-41e7963806d1", 2024-04-24T06:50:17.701 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-04-24T06:50:17.701 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:50:17.701 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:50:17.701 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:50:17.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.691+0000 7f506d600640 1 -- 172.21.15.5:0/1815707578 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f504c0a3da0 msgr2=0x7f504c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:17.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.691+0000 7f506d600640 1 --2- 172.21.15.5:0/1815707578 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f504c0a3da0 0x7f504c0a6260 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5060005fd0 tx=0x7f5060009600 comp rx=0 tx=0).stop 2024-04-24T06:50:17.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.691+0000 7f506d600640 1 -- 172.21.15.5:0/1815707578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5068102070 msgr2=0x7f50681a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:17.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.691+0000 7f506d600640 1 --2- 172.21.15.5:0/1815707578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5068102070 0x7f50681a0670 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5058006010 tx=0x7f50580042f0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.692+0000 7f506d600640 1 -- 172.21.15.5:0/1815707578 shutdown_connections 2024-04-24T06:50:17.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.692+0000 7f506d600640 1 --2- 172.21.15.5:0/1815707578 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f504c0a3da0 0x7f504c0a6260 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.692+0000 7f506d600640 1 --2- 172.21.15.5:0/1815707578 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f506819a730 0x7f506819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.692+0000 7f506d600640 1 --2- 172.21.15.5:0/1815707578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5068107140 0x7f50681a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.692+0000 7f506d600640 1 --2- 172.21.15.5:0/1815707578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5068102070 0x7f50681a0670 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:17.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.692+0000 7f506d600640 1 -- 172.21.15.5:0/1815707578 >> 172.21.15.5:0/1815707578 conn(0x7f50680fd950 msgr2=0x7f50680ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:17.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.692+0000 7f506d600640 1 -- 172.21.15.5:0/1815707578 shutdown_connections 2024-04-24T06:50:17.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:17.692+0000 7f506d600640 1 -- 172.21.15.5:0/1815707578 wait complete. 2024-04-24T06:50:17.713 INFO:tasks.cephfs.filesystem:scrub status for tag:4ba79d26-e007-41a2-b0f3-41e7963806d1 - {'path': '/', 'tag': '4ba79d26-e007-41a2-b0f3-41e7963806d1', 'options': 'recursive,force'} 2024-04-24T06:50:17.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:17 smithi005 ceph-mon[27798]: pgmap v190: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-24T06:50:17.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:17 smithi037 ceph-mon[32864]: pgmap v190: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-24T06:50:17.945 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:17 smithi137 ceph-mon[33017]: pgmap v190: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-24T06:50:18.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:18 smithi005 ceph-mon[27798]: scrub queued for path: / 2024-04-24T06:50:18.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:18 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:50:18.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:18 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:50:18.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:18 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:50:18.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:18 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:50:18.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:18 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:50:18.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:18 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:50:18.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:18 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:50:18.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:18 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:50:18.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:18 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:50:18.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:18 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:50:18.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:18 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:50:18.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:18 smithi037 ceph-mon[32864]: scrub queued for path: / 2024-04-24T06:50:18.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:18 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:50:18.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:18 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:50:18.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:18 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:50:18.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:18 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:50:18.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:18 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:50:18.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:18 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:50:18.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:18 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:50:18.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:18 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:50:18.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:18 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:50:18.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:18 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:50:18.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:18 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:50:18.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:18 smithi137 ceph-mon[33017]: scrub queued for path: / 2024-04-24T06:50:18.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:18 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:50:18.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:18 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:50:18.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:18 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:50:18.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:18 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:50:18.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:18 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:50:18.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:18 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:50:18.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:18 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:50:18.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:18 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:50:18.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:18 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:50:18.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:18 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:50:18.948 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:18 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:50:19.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:19 smithi005 ceph-mon[27798]: pgmap v191: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-24T06:50:19.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:19 smithi037 ceph-mon[32864]: pgmap v191: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-24T06:50:19.945 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:19 smithi137 ceph-mon[33017]: pgmap v191: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-24T06:50:20.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:20 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:50:20.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:20 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:50:20.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:20 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:50:21.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:21 smithi005 ceph-mon[27798]: pgmap v192: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6 op/s 2024-04-24T06:50:21.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:21 smithi037 ceph-mon[32864]: pgmap v192: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6 op/s 2024-04-24T06:50:21.945 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:21 smithi137 ceph-mon[33017]: pgmap v192: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6 op/s 2024-04-24T06:50:22.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:22 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:50:22.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:22 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:50:22.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:22 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:50:22.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:22 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:50:22.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:22 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:50:22.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:22 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:50:22.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:22 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:50:22.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:22 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:50:22.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:22 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:50:23.945 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:23 smithi137 ceph-mon[33017]: pgmap v193: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:50:24.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:23 smithi005 ceph-mon[27798]: pgmap v193: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:50:24.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:23 smithi037 ceph-mon[32864]: pgmap v193: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:50:25.945 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:25 smithi137 ceph-mon[33017]: pgmap v194: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-04-24T06:50:26.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:25 smithi005 ceph-mon[27798]: pgmap v194: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-04-24T06:50:26.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:25 smithi037 ceph-mon[32864]: pgmap v194: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-04-24T06:50:27.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:27 smithi137 ceph-mon[33017]: pgmap v195: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:50:28.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:27 smithi005 ceph-mon[27798]: pgmap v195: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:50:28.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:27 smithi037 ceph-mon[32864]: pgmap v195: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:50:28.945 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:28 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:50:28.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:28 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:50:28.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:28 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:50:29.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:28 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:50:29.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:28 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:50:29.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:28 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:50:29.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:28 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:50:29.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:28 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:50:29.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:28 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:50:29.945 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:29 smithi137 ceph-mon[33017]: pgmap v196: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-24T06:50:30.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:29 smithi005 ceph-mon[27798]: pgmap v196: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-24T06:50:30.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:29 smithi037 ceph-mon[32864]: pgmap v196: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-24T06:50:30.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:30 smithi137 ceph-mon[33017]: pgmap v197: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:50:31.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:30 smithi005 ceph-mon[27798]: pgmap v197: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:50:31.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:30 smithi037 ceph-mon[32864]: pgmap v197: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:50:33.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:33 smithi005 ceph-mon[27798]: pgmap v198: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-24T06:50:33.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:33 smithi037 ceph-mon[32864]: pgmap v198: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-24T06:50:33.649 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:33 smithi137 ceph-mon[33017]: pgmap v198: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-24T06:50:35.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:35 smithi005 ceph-mon[27798]: pgmap v199: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:50:35.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:35 smithi037 ceph-mon[32864]: pgmap v199: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:50:35.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:35 smithi137 ceph-mon[33017]: pgmap v199: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:50:37.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:37 smithi005 ceph-mon[27798]: pgmap v200: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-24T06:50:37.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:37 smithi037 ceph-mon[32864]: pgmap v200: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-24T06:50:37.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:37 smithi137 ceph-mon[33017]: pgmap v200: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-24T06:50:39.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:39 smithi005 ceph-mon[27798]: pgmap v201: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-24T06:50:39.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:39 smithi037 ceph-mon[32864]: pgmap v201: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-24T06:50:39.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:39 smithi137 ceph-mon[33017]: pgmap v201: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-24T06:50:41.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:41 smithi005 ceph-mon[27798]: pgmap v202: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-24T06:50:41.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:41 smithi037 ceph-mon[32864]: pgmap v202: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-24T06:50:41.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:41 smithi137 ceph-mon[33017]: pgmap v202: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-24T06:50:43.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:43 smithi005 ceph-mon[27798]: pgmap v203: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:50:43.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:43 smithi037 ceph-mon[32864]: pgmap v203: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:50:43.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:43 smithi137 ceph-mon[33017]: pgmap v203: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:50:45.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:45 smithi037 ceph-mon[32864]: pgmap v204: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:50:45.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:45 smithi137 ceph-mon[33017]: pgmap v204: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:50:45.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:45 smithi005 ceph-mon[27798]: pgmap v204: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:50:47.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:47 smithi037 ceph-mon[32864]: pgmap v205: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:50:47.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:47 smithi005 ceph-mon[27798]: pgmap v205: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:50:47.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:47 smithi137 ceph-mon[33017]: pgmap v205: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:50:47.714 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T06:50:47.714 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T06:50:47.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.943+0000 7f110c200640 1 -- 172.21.15.5:0/2894397522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11041067b0 msgr2=0x7f11041111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:47.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.943+0000 7f110c200640 1 --2- 172.21.15.5:0/2894397522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11041067b0 0x7f11041111b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f10fc009ca0 tx=0x7f10fc02a8c0 comp rx=0 tx=0).stop 2024-04-24T06:50:47.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.944+0000 7f110c200640 1 -- 172.21.15.5:0/2894397522 shutdown_connections 2024-04-24T06:50:47.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.944+0000 7f110c200640 1 --2- 172.21.15.5:0/2894397522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11041067b0 0x7f11041111b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:47.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.944+0000 7f110c200640 1 --2- 172.21.15.5:0/2894397522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1104105e10 0x7f1104106270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:47.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.944+0000 7f110c200640 1 --2- 172.21.15.5:0/2894397522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1104104530 0x7f1104104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:47.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.944+0000 7f110c200640 1 -- 172.21.15.5:0/2894397522 >> 172.21.15.5:0/2894397522 conn(0x7f11040ffc90 msgr2=0x7f11041020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:47.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.944+0000 7f110c200640 1 -- 172.21.15.5:0/2894397522 shutdown_connections 2024-04-24T06:50:47.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.944+0000 7f110c200640 1 -- 172.21.15.5:0/2894397522 wait complete. 2024-04-24T06:50:47.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.945+0000 7f110c200640 1 Processor -- start 2024-04-24T06:50:47.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.945+0000 7f110c200640 1 -- start start 2024-04-24T06:50:47.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.946+0000 7f110c200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1104104530 0x7f11041a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:47.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.946+0000 7f110c200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1104105e10 0x7f11041a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:47.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.946+0000 7f110c200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11041067b0 0x7f110419a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:47.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.946+0000 7f110c200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1104114680 con 0x7f1104104530 2024-04-24T06:50:47.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.946+0000 7f110c200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1104114500 con 0x7f1104105e10 2024-04-24T06:50:47.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.946+0000 7f110c200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1104114800 con 0x7f11041067b0 2024-04-24T06:50:47.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.946+0000 7f1109800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1104104530 0x7f11041a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:47.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.946+0000 7f1109800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1104104530 0x7f11041a0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53500/0 (socket says 172.21.15.5:53500) 2024-04-24T06:50:47.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.946+0000 7f1109800640 1 -- 172.21.15.5:0/3420600572 learned_addr learned my addr 172.21.15.5:0/3420600572 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:47.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.946+0000 7f110a200640 1 --2- 172.21.15.5:0/3420600572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11041067b0 0x7f110419a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:47.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.946+0000 7f1108e00640 1 --2- 172.21.15.5:0/3420600572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1104105e10 0x7f11041a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:47.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.947+0000 7f1109800640 1 -- 172.21.15.5:0/3420600572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11041067b0 msgr2=0x7f110419a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:47.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.947+0000 7f1109800640 1 --2- 172.21.15.5:0/3420600572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11041067b0 0x7f110419a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:47.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.947+0000 7f1109800640 1 -- 172.21.15.5:0/3420600572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1104105e10 msgr2=0x7f11041a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:47.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.947+0000 7f1109800640 1 --2- 172.21.15.5:0/3420600572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1104105e10 0x7f11041a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:47.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.947+0000 7f1109800640 1 -- 172.21.15.5:0/3420600572 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f110419b0e0 con 0x7f1104104530 2024-04-24T06:50:47.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.947+0000 7f1109800640 1 --2- 172.21.15.5:0/3420600572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1104104530 0x7f11041a0760 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f10f800e9f0 tx=0x7f10f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:47.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.947+0000 7f1102000640 1 -- 172.21.15.5:0/3420600572 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f800cd80 con 0x7f1104104530 2024-04-24T06:50:47.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.947+0000 7f1102000640 1 -- 172.21.15.5:0/3420600572 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10f8010910 con 0x7f1104104530 2024-04-24T06:50:47.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.947+0000 7f110c200640 1 -- 172.21.15.5:0/3420600572 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f110419b3d0 con 0x7f1104104530 2024-04-24T06:50:47.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.947+0000 7f1102000640 1 -- 172.21.15.5:0/3420600572 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f8018990 con 0x7f1104104530 2024-04-24T06:50:47.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.948+0000 7f110c200640 1 -- 172.21.15.5:0/3420600572 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11041a7620 con 0x7f1104104530 2024-04-24T06:50:47.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.951+0000 7f1102000640 1 -- 172.21.15.5:0/3420600572 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f10f8010ac0 con 0x7f1104104530 2024-04-24T06:50:47.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.951+0000 7f110ef63740 1 Processor -- start 2024-04-24T06:50:47.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.951+0000 7f110ef63740 1 -- start start 2024-04-24T06:50:47.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.952+0000 7f110ef63740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf5898860 0x55daf5898c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:47.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.952+0000 7f110ef63740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf5899180 0x55daf58a3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:47.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.952+0000 7f110ef63740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf58a3850 0x55daf58a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:47.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.952+0000 7f110ef63740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55daf57fe1f0 con 0x55daf58a3850 2024-04-24T06:50:47.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.952+0000 7f110ef63740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55daf57a4350 con 0x55daf5899180 2024-04-24T06:50:47.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.952+0000 7f110ef63740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55daf57fa1b0 con 0x55daf5898860 2024-04-24T06:50:47.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.952+0000 7f1108e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf58a3850 0x55daf58a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:47.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.952+0000 7f1109800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf5899180 0x55daf58a3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:47.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.952+0000 7f1108e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf58a3850 0x55daf58a5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53506/0 (socket says 172.21.15.5:53506) 2024-04-24T06:50:47.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.952+0000 7f1108e00640 1 -- 172.21.15.5:0/1874060650 learned_addr learned my addr 172.21.15.5:0/1874060650 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:47.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.952+0000 7f110a200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf5898860 0x55daf5898c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:47.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.953+0000 7f1108e00640 1 -- 172.21.15.5:0/1874060650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf5898860 msgr2=0x55daf5898c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:47.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.953+0000 7f1108e00640 1 --2- 172.21.15.5:0/1874060650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf5898860 0x55daf5898c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:47.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.953+0000 7f1108e00640 1 -- 172.21.15.5:0/1874060650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf5899180 msgr2=0x55daf58a3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:47.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.953+0000 7f1108e00640 1 --2- 172.21.15.5:0/1874060650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf5899180 0x55daf58a3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:47.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.953+0000 7f1108e00640 1 -- 172.21.15.5:0/1874060650 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55daf57f8f50 con 0x55daf58a3850 2024-04-24T06:50:47.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.953+0000 7f1109800640 1 --2- 172.21.15.5:0/1874060650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf5899180 0x55daf58a3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:50:47.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.953+0000 7f1102000640 1 --2- 172.21.15.5:0/3420600572 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f10ec0a3d20 0x7f10ec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:47.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.953+0000 7f1108e00640 1 --2- 172.21.15.5:0/1874060650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf58a3850 0x55daf58a5c40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f10f0009c60 tx=0x7f10f0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd1b5468a61af085 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:47.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.953+0000 7f1102000640 1 -- 172.21.15.5:0/3420600572 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f10f80d0b70 con 0x7f1104104530 2024-04-24T06:50:47.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.953+0000 7f10f6a00640 1 -- 172.21.15.5:0/1874060650 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f0004230 con 0x55daf58a3850 2024-04-24T06:50:47.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.953+0000 7f110a200640 1 --2- 172.21.15.5:0/3420600572 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f10ec0a3d20 0x7f10ec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:47.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.953+0000 7f10f6a00640 1 -- 172.21.15.5:0/1874060650 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10f00043d0 con 0x55daf58a3850 2024-04-24T06:50:47.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.953+0000 7f10f6a00640 1 -- 172.21.15.5:0/1874060650 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f0004dc0 con 0x55daf58a3850 2024-04-24T06:50:47.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.954+0000 7f110ef63740 1 -- 172.21.15.5:0/1874060650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf58a3850 msgr2=0x55daf58a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:47.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.954+0000 7f110ef63740 1 --2- 172.21.15.5:0/1874060650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf58a3850 0x55daf58a5c40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f10f0009c60 tx=0x7f10f0011810 comp rx=0 tx=0).stop 2024-04-24T06:50:47.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.954+0000 7f110a200640 1 --2- 172.21.15.5:0/3420600572 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f10ec0a3d20 0x7f10ec0a61e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f10fc009430 tx=0x7f10fc009360 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:47.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.954+0000 7f110ef63740 1 -- 172.21.15.5:0/1874060650 shutdown_connections 2024-04-24T06:50:47.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.954+0000 7f110ef63740 1 --2- 172.21.15.5:0/1874060650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf58a3850 0x55daf58a5c40 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:47.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.954+0000 7f110ef63740 1 --2- 172.21.15.5:0/1874060650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf5899180 0x55daf58a3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:47.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.954+0000 7f110ef63740 1 --2- 172.21.15.5:0/1874060650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf5898860 0x55daf5898c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:47.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.954+0000 7f110ef63740 1 -- 172.21.15.5:0/1874060650 >> 172.21.15.5:0/1874060650 conn(0x55daf5894b80 msgr2=0x55daf5894f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:47.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.954+0000 7f110ef63740 1 -- 172.21.15.5:0/1874060650 shutdown_connections 2024-04-24T06:50:47.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.954+0000 7f110ef63740 1 -- 172.21.15.5:0/1874060650 wait complete. 2024-04-24T06:50:47.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.955+0000 7f110ef63740 1 -- start start 2024-04-24T06:50:47.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.955+0000 7f110ef63740 1 Processor -- start 2024-04-24T06:50:47.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.955+0000 7f110ef63740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf5898860 0x55daf58a0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:47.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.955+0000 7f110ef63740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf5899180 0x55daf58a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:47.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.956+0000 7f110ef63740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf58a3850 0x55daf58a1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:47.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.956+0000 7f110ef63740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55daf58a6640 con 0x55daf58a3850 2024-04-24T06:50:47.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.956+0000 7f110ef63740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55daf57b7690 con 0x55daf5899180 2024-04-24T06:50:47.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.956+0000 7f110ef63740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55daf58a67c0 con 0x55daf5898860 2024-04-24T06:50:47.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.956+0000 7f110ef63740 10 client.0 _ll_register_callbacks cb 0x55daf57f3650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:50:47.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.956+0000 7f110a200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf58a3850 0x55daf58a1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:47.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.956+0000 7f1108e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf5899180 0x55daf58a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:47.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.956+0000 7f1109800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf5898860 0x55daf58a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:47.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.956+0000 7f1108e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf5899180 0x55daf58a0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:51618/0 (socket says 172.21.15.5:51618) 2024-04-24T06:50:47.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.956+0000 7f1108e00640 1 -- 172.21.15.5:0/4127900721 learned_addr learned my addr 172.21.15.5:0/4127900721 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:47.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.956+0000 7f110a200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf58a3850 0x55daf58a1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53508/0 (socket says 172.21.15.5:53508) 2024-04-24T06:50:47.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.956+0000 7f1108e00640 1 -- 172.21.15.5:0/4127900721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf5898860 msgr2=0x55daf58a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:47.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.956+0000 7f1108e00640 1 --2- 172.21.15.5:0/4127900721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf5898860 0x55daf58a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:47.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.956+0000 7f1108e00640 1 -- 172.21.15.5:0/4127900721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf58a3850 msgr2=0x55daf58a1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:47.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.956+0000 7f1108e00640 1 --2- 172.21.15.5:0/4127900721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf58a3850 0x55daf58a1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:47.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.956+0000 7f1108e00640 1 -- 172.21.15.5:0/4127900721 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55daf57b6920 con 0x55daf5899180 2024-04-24T06:50:47.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.956+0000 7f110a200640 1 --2- 172.21.15.5:0/4127900721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf58a3850 0x55daf58a1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:50:47.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.956+0000 7f1109800640 1 --2- 172.21.15.5:0/4127900721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf5898860 0x55daf58a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:50:47.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.957+0000 7f1108e00640 1 --2- 172.21.15.5:0/4127900721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf5899180 0x55daf58a0cc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f10f0005110 tx=0x7f10f0005140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fd2a651fe17c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:47.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.957+0000 7f10eb400640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:50:47.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.957+0000 7f10eb400640 1 -- 172.21.15.5:0/4127900721 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f0024420 con 0x55daf5899180 2024-04-24T06:50:47.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.957+0000 7f110ef63740 1 -- 172.21.15.5:0/4127900721 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10f00043d0 con 0x55daf5899180 2024-04-24T06:50:47.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.957+0000 7f10eb400640 1 -- 172.21.15.5:0/4127900721 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f10f00245c0 con 0x55daf5899180 2024-04-24T06:50:47.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.957+0000 7f10eb400640 1 -- 172.21.15.5:0/4127900721 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f10f0032050 con 0x55daf5899180 2024-04-24T06:50:47.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.957+0000 7f110ef63740 10 client.26684 fetch_fsmap learned FSMap version 27 2024-04-24T06:50:47.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.957+0000 7f110ef63740 1 -- 172.21.15.5:0/4127900721 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55daf55da7d0 con 0x55daf5899180 2024-04-24T06:50:47.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.958+0000 7f10eb400640 1 -- 172.21.15.5:0/4127900721 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10f00043d0 con 0x55daf5899180 2024-04-24T06:50:47.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.958+0000 7f10eb400640 1 -- 172.21.15.5:0/4127900721 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f10f0018070 con 0x55daf5899180 2024-04-24T06:50:47.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.958+0000 7f110ef63740 10 client.26684 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:50:47.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.958+0000 7f110ef63740 10 client.26684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:50:47.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.958+0000 7f110ef63740 20 client.26684 populate_metadata read hostname 'smithi005' 2024-04-24T06:50:47.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.958+0000 7f110ef63740 1 --2- 172.21.15.5:0/4127900721 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55daf58b3bc0 0x55daf58b5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:47.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.958+0000 7f110ef63740 4 client.26684 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:50:47.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.958+0000 7f110ef63740 1 -- 172.21.15.5:0/4127900721 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55daf580bc10 con 0x55daf58b3bc0 2024-04-24T06:50:47.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.958+0000 7f1109800640 1 --2- 172.21.15.5:0/4127900721 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55daf58b3bc0 0x55daf58b5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:47.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.959+0000 7f1109800640 1 --2- 172.21.15.5:0/4127900721 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55daf58b3bc0 0x55daf58b5fb0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca735eda8ee69ad0 server_cookie=5f21fd05e44e9dea in_seq=0 out_seq=0 2024-04-24T06:50:47.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.959+0000 7f10eb400640 10 client.26684 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:50:47.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.960+0000 7f10eb400640 1 -- 172.21.15.5:0/4127900721 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55daf580bc10 con 0x55daf58b3bc0 2024-04-24T06:50:47.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.960+0000 7f10eb400640 10 client.26684 handle_command_reply: tid=0 2024-04-24T06:50:47.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.960+0000 7f110ef63740 1 client.26684 shutdown 2024-04-24T06:50:47.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.960+0000 7f110ef63740 1 -- 172.21.15.5:0/4127900721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf5899180 msgr2=0x55daf58a0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:47.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.960+0000 7f110ef63740 1 --2- 172.21.15.5:0/4127900721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf5899180 0x55daf58a0cc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f10f0005110 tx=0x7f10f0005140 comp rx=0 tx=0).stop 2024-04-24T06:50:47.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.960+0000 7f110ef63740 1 -- 172.21.15.5:0/4127900721 shutdown_connections 2024-04-24T06:50:47.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.960+0000 7f110ef63740 1 --2- 172.21.15.5:0/4127900721 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55daf58b3bc0 0x55daf58b5fb0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:47.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.960+0000 7f110ef63740 1 --2- 172.21.15.5:0/4127900721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf58a3850 0x55daf58a1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:47.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.960+0000 7f110ef63740 1 --2- 172.21.15.5:0/4127900721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf5899180 0x55daf58a0cc0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:47.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.960+0000 7f110ef63740 1 --2- 172.21.15.5:0/4127900721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf5898860 0x55daf58a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:47.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.960+0000 7f10eb400640 0 client.26684 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:50:47.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.960+0000 7f110ef63740 1 -- 172.21.15.5:0/4127900721 >> 172.21.15.5:0/4127900721 conn(0x55daf5894b80 msgr2=0x55daf589a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:47.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.960+0000 7f110ef63740 1 -- 172.21.15.5:0/4127900721 shutdown_connections 2024-04-24T06:50:47.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.960+0000 7f110ef63740 1 -- 172.21.15.5:0/4127900721 wait complete. 2024-04-24T06:50:47.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.960+0000 7f110ef63740 20 client.26684 trim_cache size 0 max 16384 2024-04-24T06:50:47.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.980+0000 7f110ef63740 1 Processor -- start 2024-04-24T06:50:47.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.980+0000 7f110ef63740 1 -- start start 2024-04-24T06:50:47.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.980+0000 7f110ef63740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf5898860 0x55daf58af560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:47.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.980+0000 7f110ef63740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf5899180 0x55daf589b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:47.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.980+0000 7f110ef63740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf58a3850 0x55daf58bb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:47.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.980+0000 7f110ef63740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f11041b2760 con 0x55daf5899180 2024-04-24T06:50:47.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.980+0000 7f110ef63740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f11041b0030 con 0x55daf5898860 2024-04-24T06:50:47.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.980+0000 7f110ef63740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f11041ade80 con 0x55daf58a3850 2024-04-24T06:50:47.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.981+0000 7f110a200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf58a3850 0x55daf58bb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:47.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.981+0000 7f1108e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf5899180 0x55daf589b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:47.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.981+0000 7f110a200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf58a3850 0x55daf58bb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:57158/0 (socket says 172.21.15.5:57158) 2024-04-24T06:50:47.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.981+0000 7f110a200640 1 -- 172.21.15.5:0/3799122600 learned_addr learned my addr 172.21.15.5:0/3799122600 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:47.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.981+0000 7f1109800640 1 --2- 172.21.15.5:0/3799122600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf5898860 0x55daf58af560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:47.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.981+0000 7f1108e00640 1 -- 172.21.15.5:0/3799122600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf58a3850 msgr2=0x55daf58bb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:47.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.981+0000 7f1108e00640 1 --2- 172.21.15.5:0/3799122600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf58a3850 0x55daf58bb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:47.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.981+0000 7f1108e00640 1 -- 172.21.15.5:0/3799122600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf5898860 msgr2=0x55daf58af560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:47.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.981+0000 7f1108e00640 1 --2- 172.21.15.5:0/3799122600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf5898860 0x55daf58af560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:47.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.981+0000 7f1108e00640 1 -- 172.21.15.5:0/3799122600 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55daf5898fb0 con 0x55daf5899180 2024-04-24T06:50:47.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.981+0000 7f110a200640 1 --2- 172.21.15.5:0/3799122600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf58a3850 0x55daf58bb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:50:47.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.981+0000 7f1108e00640 1 --2- 172.21.15.5:0/3799122600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf5899180 0x55daf589b0d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f10f0011d10 tx=0x7f10f0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=982c52d7ab36e213 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:47.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.982+0000 7f10f6000640 1 -- 172.21.15.5:0/3799122600 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f0019770 con 0x55daf5899180 2024-04-24T06:50:47.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.982+0000 7f10f6000640 1 -- 172.21.15.5:0/3799122600 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10f0029070 con 0x55daf5899180 2024-04-24T06:50:47.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.982+0000 7f10f6000640 1 -- 172.21.15.5:0/3799122600 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f00247c0 con 0x55daf5899180 2024-04-24T06:50:47.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.982+0000 7f110ef63740 1 -- 172.21.15.5:0/3799122600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf5899180 msgr2=0x55daf589b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:47.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.982+0000 7f110ef63740 1 --2- 172.21.15.5:0/3799122600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf5899180 0x55daf589b0d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f10f0011d10 tx=0x7f10f0000bc0 comp rx=0 tx=0).stop 2024-04-24T06:50:47.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.982+0000 7f110ef63740 1 -- 172.21.15.5:0/3799122600 shutdown_connections 2024-04-24T06:50:47.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.982+0000 7f110ef63740 1 --2- 172.21.15.5:0/3799122600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf58a3850 0x55daf58bb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:47.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.982+0000 7f110ef63740 1 --2- 172.21.15.5:0/3799122600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf5899180 0x55daf589b0d0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:47.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.982+0000 7f110ef63740 1 --2- 172.21.15.5:0/3799122600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf5898860 0x55daf58af560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:47.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.982+0000 7f110ef63740 1 -- 172.21.15.5:0/3799122600 >> 172.21.15.5:0/3799122600 conn(0x55daf5894b80 msgr2=0x55daf589f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:47.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.982+0000 7f110ef63740 1 -- 172.21.15.5:0/3799122600 shutdown_connections 2024-04-24T06:50:47.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.983+0000 7f110ef63740 1 -- 172.21.15.5:0/3799122600 wait complete. 2024-04-24T06:50:47.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.983+0000 7f110ef63740 1 -- start start 2024-04-24T06:50:47.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.983+0000 7f110ef63740 1 Processor -- start 2024-04-24T06:50:47.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.984+0000 7f110ef63740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf5898860 0x55daf58a6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:47.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.984+0000 7f110ef63740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf5899180 0x55daf58a6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:47.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.984+0000 7f110ef63740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf58a3850 0x55daf58a74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:47.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.984+0000 7f110ef63740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f11041ae4f0 con 0x55daf58a3850 2024-04-24T06:50:47.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.984+0000 7f110ef63740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f11041af960 con 0x55daf5898860 2024-04-24T06:50:47.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.984+0000 7f110ef63740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f11041aebc0 con 0x55daf5899180 2024-04-24T06:50:47.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.984+0000 7f110ef63740 10 client.0 _ll_register_callbacks cb 0x7f10f80d44f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:50:47.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.984+0000 7f110a200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf58a3850 0x55daf58a74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:47.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.984+0000 7f110a200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf58a3850 0x55daf58a74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53524/0 (socket says 172.21.15.5:53524) 2024-04-24T06:50:47.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.984+0000 7f110a200640 1 -- 172.21.15.5:0/1450365616 learned_addr learned my addr 172.21.15.5:0/1450365616 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:47.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.984+0000 7f1109800640 1 --2- 172.21.15.5:0/1450365616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf5898860 0x55daf58a6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:47.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.984+0000 7f1108e00640 1 --2- 172.21.15.5:0/1450365616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf5899180 0x55daf58a6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:47.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.984+0000 7f110a200640 1 -- 172.21.15.5:0/1450365616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf5899180 msgr2=0x55daf58a6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:47.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.984+0000 7f110a200640 1 --2- 172.21.15.5:0/1450365616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf5899180 0x55daf58a6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:47.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.984+0000 7f110a200640 1 -- 172.21.15.5:0/1450365616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf5898860 msgr2=0x55daf58a6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:47.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.984+0000 7f110a200640 1 --2- 172.21.15.5:0/1450365616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf5898860 0x55daf58a6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:47.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.984+0000 7f110a200640 1 -- 172.21.15.5:0/1450365616 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55daf55103f0 con 0x55daf58a3850 2024-04-24T06:50:47.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.984+0000 7f1109800640 1 --2- 172.21.15.5:0/1450365616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf5898860 0x55daf58a6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:50:47.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.985+0000 7f110a200640 1 --2- 172.21.15.5:0/1450365616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf58a3850 0x55daf58a74b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f10fc005e30 tx=0x7f10fc02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec46ae1c77ae951 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:47.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.985+0000 7f10eb400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:50:47.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.985+0000 7f10eb400640 1 -- 172.21.15.5:0/1450365616 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10fc031720 con 0x55daf58a3850 2024-04-24T06:50:47.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.985+0000 7f10eb400640 1 -- 172.21.15.5:0/1450365616 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f10fc042030 con 0x55daf58a3850 2024-04-24T06:50:47.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.985+0000 7f110ef63740 1 -- 172.21.15.5:0/1450365616 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10f0029070 con 0x55daf58a3850 2024-04-24T06:50:47.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.985+0000 7f10eb400640 1 -- 172.21.15.5:0/1450365616 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f10fc0318c0 con 0x55daf58a3850 2024-04-24T06:50:47.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.985+0000 7f110ef63740 10 client.16551 fetch_fsmap learned FSMap version 27 2024-04-24T06:50:47.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.985+0000 7f110ef63740 1 -- 172.21.15.5:0/1450365616 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55daf54d00f0 con 0x55daf58a3850 2024-04-24T06:50:47.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.985+0000 7f10eb400640 1 -- 172.21.15.5:0/1450365616 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10f0029070 con 0x55daf58a3850 2024-04-24T06:50:47.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.986+0000 7f10eb400640 1 -- 172.21.15.5:0/1450365616 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f10fc004360 con 0x55daf58a3850 2024-04-24T06:50:47.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.986+0000 7f110ef63740 10 client.16551 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:50:47.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.986+0000 7f110ef63740 10 client.16551 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:50:47.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.986+0000 7f110ef63740 20 client.16551 populate_metadata read hostname 'smithi005' 2024-04-24T06:50:47.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.986+0000 7f110ef63740 1 --2- 172.21.15.5:0/1450365616 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55daf59198d0 0x55daf591bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:47.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.986+0000 7f110ef63740 4 client.16551 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:50:47.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.986+0000 7f110ef63740 1 -- 172.21.15.5:0/1450365616 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55daf5805a80 con 0x55daf59198d0 2024-04-24T06:50:47.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.986+0000 7f1109800640 1 --2- 172.21.15.5:0/1450365616 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55daf59198d0 0x55daf591bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:47.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.986+0000 7f1109800640 1 --2- 172.21.15.5:0/1450365616 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55daf59198d0 0x55daf591bcf0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b12df02eb931fbdf server_cookie=626adc9a21c83fb6 in_seq=0 out_seq=0 2024-04-24T06:50:47.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.986+0000 7f10eb400640 10 client.16551 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:50:47.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.987+0000 7f10eb400640 1 -- 172.21.15.5:0/1450365616 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55daf5805a80 con 0x55daf59198d0 2024-04-24T06:50:47.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.987+0000 7f10eb400640 10 client.16551 handle_command_reply: tid=0 2024-04-24T06:50:47.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.987+0000 7f110ef63740 1 client.16551 shutdown 2024-04-24T06:50:47.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.987+0000 7f110ef63740 1 -- 172.21.15.5:0/1450365616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf58a3850 msgr2=0x55daf58a74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:47.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.987+0000 7f110ef63740 1 --2- 172.21.15.5:0/1450365616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf58a3850 0x55daf58a74b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f10fc005e30 tx=0x7f10fc02a8c0 comp rx=0 tx=0).stop 2024-04-24T06:50:47.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.987+0000 7f110ef63740 1 -- 172.21.15.5:0/1450365616 shutdown_connections 2024-04-24T06:50:47.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.987+0000 7f110ef63740 1 --2- 172.21.15.5:0/1450365616 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55daf59198d0 0x55daf591bcf0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:47.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.987+0000 7f110ef63740 1 --2- 172.21.15.5:0/1450365616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf58a3850 0x55daf58a74b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:47.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.987+0000 7f110ef63740 1 --2- 172.21.15.5:0/1450365616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf5899180 0x55daf58a6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:47.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.987+0000 7f110ef63740 1 --2- 172.21.15.5:0/1450365616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf5898860 0x55daf58a6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:47.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.987+0000 7f10eb400640 0 client.16551 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:50:47.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.987+0000 7f110ef63740 1 -- 172.21.15.5:0/1450365616 >> 172.21.15.5:0/1450365616 conn(0x55daf5894b80 msgr2=0x55daf58a0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:47.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.987+0000 7f110ef63740 1 -- 172.21.15.5:0/1450365616 shutdown_connections 2024-04-24T06:50:47.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.987+0000 7f110ef63740 1 -- 172.21.15.5:0/1450365616 wait complete. 2024-04-24T06:50:47.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.987+0000 7f110ef63740 20 client.16551 trim_cache size 0 max 16384 2024-04-24T06:50:47.996 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:50:47.996 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-24T06:50:47.996 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-24T06:50:47.997 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:50:47.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.988+0000 7f110c200640 1 -- 172.21.15.5:0/3420600572 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f10ec0a3d20 msgr2=0x7f10ec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:47.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.988+0000 7f110c200640 1 --2- 172.21.15.5:0/3420600572 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f10ec0a3d20 0x7f10ec0a61e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f10fc009430 tx=0x7f10fc009360 comp rx=0 tx=0).stop 2024-04-24T06:50:47.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.988+0000 7f110c200640 1 -- 172.21.15.5:0/3420600572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1104104530 msgr2=0x7f11041a0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:47.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.988+0000 7f110c200640 1 --2- 172.21.15.5:0/3420600572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1104104530 0x7f11041a0760 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f10f800e9f0 tx=0x7f10f800eeb0 comp rx=0 tx=0).stop 2024-04-24T06:50:47.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.988+0000 7f110c200640 1 -- 172.21.15.5:0/3420600572 shutdown_connections 2024-04-24T06:50:47.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.988+0000 7f110c200640 1 --2- 172.21.15.5:0/3420600572 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f10ec0a3d20 0x7f10ec0a61e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:47.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.988+0000 7f110c200640 1 --2- 172.21.15.5:0/3420600572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11041067b0 0x7f110419a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:47.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.988+0000 7f110c200640 1 --2- 172.21.15.5:0/3420600572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1104105e10 0x7f11041a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:47.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.988+0000 7f110c200640 1 --2- 172.21.15.5:0/3420600572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1104104530 0x7f11041a0760 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:47.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.988+0000 7f110c200640 1 -- 172.21.15.5:0/3420600572 >> 172.21.15.5:0/3420600572 conn(0x7f11040ffc90 msgr2=0x7f11041001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:47.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.988+0000 7f110c200640 1 -- 172.21.15.5:0/3420600572 shutdown_connections 2024-04-24T06:50:47.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:47.988+0000 7f110c200640 1 -- 172.21.15.5:0/3420600572 wait complete. 2024-04-24T06:50:48.007 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T06:50:48.008 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:50:48.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.219+0000 7f76f9200640 1 -- 172.21.15.5:0/2739520332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76f4104460 msgr2=0x7f76f4104840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:48.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.219+0000 7f76f9200640 1 --2- 172.21.15.5:0/2739520332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76f4104460 0x7f76f4104840 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f76e4009860 tx=0x7f76e402a340 comp rx=0 tx=0).stop 2024-04-24T06:50:48.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.219+0000 7f76f9200640 1 -- 172.21.15.5:0/2739520332 shutdown_connections 2024-04-24T06:50:48.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.219+0000 7f76f9200640 1 --2- 172.21.15.5:0/2739520332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f4109530 0x7f76f4111100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.219+0000 7f76f9200640 1 --2- 172.21.15.5:0/2739520332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76f4104e10 0x7f76f4108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.219+0000 7f76f9200640 1 --2- 172.21.15.5:0/2739520332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76f4104460 0x7f76f4104840 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.219+0000 7f76f9200640 1 -- 172.21.15.5:0/2739520332 >> 172.21.15.5:0/2739520332 conn(0x7f76f40ffcc0 msgr2=0x7f76f41020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:48.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.219+0000 7f76f9200640 1 -- 172.21.15.5:0/2739520332 shutdown_connections 2024-04-24T06:50:48.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.220+0000 7f76f9200640 1 -- 172.21.15.5:0/2739520332 wait complete. 2024-04-24T06:50:48.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.221+0000 7f76f9200640 1 Processor -- start 2024-04-24T06:50:48.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.221+0000 7f76f9200640 1 -- start start 2024-04-24T06:50:48.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.221+0000 7f76f9200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76f4104460 0x7f76f419ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:48.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.222+0000 7f76f9200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f4109530 0x7f76f419b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:48.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.222+0000 7f76f9200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76f419f4f0 0x7f76f419f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:48.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.222+0000 7f76f9200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f76f41145c0 con 0x7f76f4109530 2024-04-24T06:50:48.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.222+0000 7f76f9200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f76f4114440 con 0x7f76f4104460 2024-04-24T06:50:48.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.222+0000 7f76f9200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f76f4114740 con 0x7f76f419f4f0 2024-04-24T06:50:48.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.222+0000 7f76f1e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76f4104460 0x7f76f419ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:48.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.222+0000 7f76f1e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76f4104460 0x7f76f419ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:51658/0 (socket says 172.21.15.5:51658) 2024-04-24T06:50:48.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.222+0000 7f76f1e00640 1 -- 172.21.15.5:0/3836747043 learned_addr learned my addr 172.21.15.5:0/3836747043 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:48.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.222+0000 7f76f2800640 1 --2- 172.21.15.5:0/3836747043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76f419f4f0 0x7f76f419f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:48.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.222+0000 7f76f1400640 1 --2- 172.21.15.5:0/3836747043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f4109530 0x7f76f419b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:48.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.222+0000 7f76f1e00640 1 -- 172.21.15.5:0/3836747043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76f419f4f0 msgr2=0x7f76f419f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:48.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.222+0000 7f76f1e00640 1 --2- 172.21.15.5:0/3836747043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76f419f4f0 0x7f76f419f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.222+0000 7f76f1e00640 1 -- 172.21.15.5:0/3836747043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f4109530 msgr2=0x7f76f419b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:48.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.222+0000 7f76f1e00640 1 --2- 172.21.15.5:0/3836747043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f4109530 0x7f76f419b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.222+0000 7f76f1e00640 1 -- 172.21.15.5:0/3836747043 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f419ffe0 con 0x7f76f4104460 2024-04-24T06:50:48.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.222+0000 7f76f1400640 1 --2- 172.21.15.5:0/3836747043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f4109530 0x7f76f419b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:50:48.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.223+0000 7f76f1e00640 1 --2- 172.21.15.5:0/3836747043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76f4104460 0x7f76f419ac00 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f76e4035040 tx=0x7f76e4033b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:48.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.223+0000 7f76eaa00640 1 -- 172.21.15.5:0/3836747043 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e4037040 con 0x7f76f4104460 2024-04-24T06:50:48.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.223+0000 7f76eaa00640 1 -- 172.21.15.5:0/3836747043 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76e4040070 con 0x7f76f4104460 2024-04-24T06:50:48.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.223+0000 7f76f9200640 1 -- 172.21.15.5:0/3836747043 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76f41a0270 con 0x7f76f4104460 2024-04-24T06:50:48.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.223+0000 7f76eaa00640 1 -- 172.21.15.5:0/3836747043 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e40039b0 con 0x7f76f4104460 2024-04-24T06:50:48.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.223+0000 7f76f9200640 1 -- 172.21.15.5:0/3836747043 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76f41a7a30 con 0x7f76f4104460 2024-04-24T06:50:48.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.224+0000 7f76f9200640 1 -- 172.21.15.5:0/3836747043 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76f4069300 con 0x7f76f4104460 2024-04-24T06:50:48.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.226+0000 7f76eaa00640 1 -- 172.21.15.5:0/3836747043 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f76e40305e0 con 0x7f76f4104460 2024-04-24T06:50:48.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.228+0000 7f76eaa00640 1 --2- 172.21.15.5:0/3836747043 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f76d40a3d20 0x7f76d40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.228+0000 7f76eaa00640 1 -- 172.21.15.5:0/3836747043 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f76e40f07a0 con 0x7f76f4104460 2024-04-24T06:50:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.228+0000 7f76f1400640 1 --2- 172.21.15.5:0/3836747043 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f76d40a3d20 0x7f76d40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.229+0000 7f76f1400640 1 --2- 172.21.15.5:0/3836747043 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f76d40a3d20 0x7f76d40a61e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f76f4100220 tx=0x7f76dc009500 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:48.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.233+0000 7f76eaa00640 1 -- 172.21.15.5:0/3836747043 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f76e40bcf00 con 0x7f76f4104460 2024-04-24T06:50:48.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.482+0000 7f76f9200640 1 -- 172.21.15.5:0/3836747043 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76f419bf50 con 0x7f76f4104460 2024-04-24T06:50:48.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.484+0000 7f76eaa00640 1 -- 172.21.15.5:0/3836747043 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11856 (secure 0 0 0) 0x7f76e40bc800 con 0x7f76f4104460 2024-04-24T06:50:48.487 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:50:48.487 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1735462750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1735462750},{"type":"v1","addr":"172.21.15.5:6837","nonce":1735462750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1499469370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1499469370},{"type":"v1","addr":"172.21.15.5:6841","nonce":1499469370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3705638847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3705638847},{"type":"v1","addr":"172.21.15.37:6835","nonce":3705638847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T06:41:34.764490+0000","modified":"2024-04-24T06:48:57.768410+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24550,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6837/1390103296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1390103296},{"type":"v1","addr":"172.21.15.37:6837","nonce":1390103296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3217754215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3217754215},{"type":"v1","addr":"172.21.15.137:6835","nonce":3217754215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6837/3044222976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3044222976},{"type":"v1","addr":"172.21.15.137:6837","nonce":3044222976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.5:6839/3173201459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3173201459},{"type":"v1","addr":"172.21.15.5:6839","nonce":3173201459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6839/1889774232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1889774232},{"type":"v1","addr":"172.21.15.137:6839","nonce":1889774232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6839/3536721753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":3536721753},{"type":"v1","addr":"172.21.15.37:6839","nonce":3536721753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24550,24569,24539]},"id":1}]} 2024-04-24T06:50:48.489 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-04-24T06:50:48.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.486+0000 7f76f9200640 1 -- 172.21.15.5:0/3836747043 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f76d40a3d20 msgr2=0x7f76d40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:48.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.486+0000 7f76f9200640 1 --2- 172.21.15.5:0/3836747043 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f76d40a3d20 0x7f76d40a61e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f76f4100220 tx=0x7f76dc009500 comp rx=0 tx=0).stop 2024-04-24T06:50:48.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.486+0000 7f76f9200640 1 -- 172.21.15.5:0/3836747043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76f4104460 msgr2=0x7f76f419ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:48.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.486+0000 7f76f9200640 1 --2- 172.21.15.5:0/3836747043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76f4104460 0x7f76f419ac00 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f76e4035040 tx=0x7f76e4033b60 comp rx=0 tx=0).stop 2024-04-24T06:50:48.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.487+0000 7f76f9200640 1 -- 172.21.15.5:0/3836747043 shutdown_connections 2024-04-24T06:50:48.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.487+0000 7f76f9200640 1 --2- 172.21.15.5:0/3836747043 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f76d40a3d20 0x7f76d40a61e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.487+0000 7f76f9200640 1 --2- 172.21.15.5:0/3836747043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76f419f4f0 0x7f76f419f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.487+0000 7f76f9200640 1 --2- 172.21.15.5:0/3836747043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f4109530 0x7f76f419b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.487+0000 7f76f9200640 1 --2- 172.21.15.5:0/3836747043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76f4104460 0x7f76f419ac00 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.487+0000 7f76f9200640 1 -- 172.21.15.5:0/3836747043 >> 172.21.15.5:0/3836747043 conn(0x7f76f40ffcc0 msgr2=0x7f76f4101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:48.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.487+0000 7f76f9200640 1 -- 172.21.15.5:0/3836747043 shutdown_connections 2024-04-24T06:50:48.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.487+0000 7f76f9200640 1 -- 172.21.15.5:0/3836747043 wait complete. 2024-04-24T06:50:48.508 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T06:50:48.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.719+0000 7f08f6200640 1 -- 172.21.15.5:0/2411840429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08f0108540 msgr2=0x7f08f01089a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:48.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.719+0000 7f08f6200640 1 --2- 172.21.15.5:0/2411840429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08f0108540 0x7f08f01089a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f08e0009900 tx=0x7f08e002a480 comp rx=0 tx=0).stop 2024-04-24T06:50:48.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.720+0000 7f08f6200640 1 -- 172.21.15.5:0/2411840429 shutdown_connections 2024-04-24T06:50:48.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.720+0000 7f08f6200640 1 --2- 172.21.15.5:0/2411840429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08f0108ee0 0x7f08f01152b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.720+0000 7f08f6200640 1 --2- 172.21.15.5:0/2411840429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08f0108540 0x7f08f01089a0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.720+0000 7f08f6200640 1 --2- 172.21.15.5:0/2411840429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08f010e4f0 0x7f08f010e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.720+0000 7f08f6200640 1 -- 172.21.15.5:0/2411840429 >> 172.21.15.5:0/2411840429 conn(0x7f08f0077e10 msgr2=0x7f08f0106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:48.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.720+0000 7f08f6200640 1 -- 172.21.15.5:0/2411840429 shutdown_connections 2024-04-24T06:50:48.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.720+0000 7f08f6200640 1 -- 172.21.15.5:0/2411840429 wait complete. 2024-04-24T06:50:48.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.721+0000 7f08f6200640 1 Processor -- start 2024-04-24T06:50:48.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.721+0000 7f08f6200640 1 -- start start 2024-04-24T06:50:48.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.721+0000 7f08f6200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08f0108540 0x7f08f019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:48.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.721+0000 7f08f6200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08f0108ee0 0x7f08f019f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:48.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.721+0000 7f08f6200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08f010e4f0 0x7f08f01a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:48.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.721+0000 7f08f6200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f08f0118820 con 0x7f08f0108ee0 2024-04-24T06:50:48.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.721+0000 7f08f6200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f08f01186a0 con 0x7f08f0108540 2024-04-24T06:50:48.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.721+0000 7f08f6200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f08f01189a0 con 0x7f08f010e4f0 2024-04-24T06:50:48.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.722+0000 7f08efe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08f0108ee0 0x7f08f019f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:48.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.722+0000 7f08f5800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08f010e4f0 0x7f08f01a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:48.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.722+0000 7f08f5800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08f010e4f0 0x7f08f01a3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:57182/0 (socket says 172.21.15.5:57182) 2024-04-24T06:50:48.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.722+0000 7f08f5800640 1 -- 172.21.15.5:0/2945867805 learned_addr learned my addr 172.21.15.5:0/2945867805 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:48.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.722+0000 7f08f4e00640 1 --2- 172.21.15.5:0/2945867805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08f0108540 0x7f08f019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:48.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.722+0000 7f08f5800640 1 -- 172.21.15.5:0/2945867805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08f0108540 msgr2=0x7f08f019eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:48.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.722+0000 7f08f5800640 1 --2- 172.21.15.5:0/2945867805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08f0108540 0x7f08f019eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.722+0000 7f08f5800640 1 -- 172.21.15.5:0/2945867805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08f0108ee0 msgr2=0x7f08f019f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:48.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.722+0000 7f08f5800640 1 --2- 172.21.15.5:0/2945867805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08f0108ee0 0x7f08f019f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.722+0000 7f08f5800640 1 -- 172.21.15.5:0/2945867805 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08f01a3d90 con 0x7f08f010e4f0 2024-04-24T06:50:48.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.722+0000 7f08f4e00640 1 --2- 172.21.15.5:0/2945867805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08f0108540 0x7f08f019eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:50:48.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.722+0000 7f08efe00640 1 --2- 172.21.15.5:0/2945867805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08f0108ee0 0x7f08f019f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:50:48.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.723+0000 7f08f5800640 1 --2- 172.21.15.5:0/2945867805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08f010e4f0 0x7f08f01a3660 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f08e800b970 tx=0x7f08e800be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:48.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.723+0000 7f08ed600640 1 -- 172.21.15.5:0/2945867805 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e8019070 con 0x7f08f010e4f0 2024-04-24T06:50:48.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.723+0000 7f08ed600640 1 -- 172.21.15.5:0/2945867805 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08e800cb40 con 0x7f08f010e4f0 2024-04-24T06:50:48.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.723+0000 7f08ed600640 1 -- 172.21.15.5:0/2945867805 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e8007650 con 0x7f08f010e4f0 2024-04-24T06:50:48.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.723+0000 7f08f6200640 1 -- 172.21.15.5:0/2945867805 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08f01a4080 con 0x7f08f010e4f0 2024-04-24T06:50:48.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.723+0000 7f08f6200640 1 -- 172.21.15.5:0/2945867805 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08f01ab910 con 0x7f08f010e4f0 2024-04-24T06:50:48.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.726+0000 7f08fa5d9740 1 Processor -- start 2024-04-24T06:50:48.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.727+0000 7f08ed600640 1 -- 172.21.15.5:0/2945867805 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f08e801c070 con 0x7f08f010e4f0 2024-04-24T06:50:48.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.727+0000 7f08fa5d9740 1 -- start start 2024-04-24T06:50:48.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.727+0000 7f08fa5d9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c14f57fb40 0x55c14f57ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:48.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.727+0000 7f08fa5d9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c14f580460 0x55c14f58a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:48.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.727+0000 7f08fa5d9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c14f58ab30 0x55c14f58cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:48.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.727+0000 7f08fa5d9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c14f4e51d0 con 0x55c14f580460 2024-04-24T06:50:48.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.727+0000 7f08fa5d9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c14f48b450 con 0x55c14f58ab30 2024-04-24T06:50:48.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.727+0000 7f08fa5d9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c14f4e1190 con 0x55c14f57fb40 2024-04-24T06:50:48.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.727+0000 7f08efe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c14f57fb40 0x55c14f57ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:48.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.727+0000 7f08f4e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c14f580460 0x55c14f58a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:48.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.727+0000 7f08efe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c14f57fb40 0x55c14f57ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:57184/0 (socket says 172.21.15.5:57184) 2024-04-24T06:50:48.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.727+0000 7f08efe00640 1 -- 172.21.15.5:0/2892750500 learned_addr learned my addr 172.21.15.5:0/2892750500 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:48.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.728+0000 7f08efe00640 1 --2- 172.21.15.5:0/2892750500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c14f58ab30 0x55c14f58cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:48.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.728+0000 7f08efe00640 1 -- 172.21.15.5:0/2892750500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c14f58ab30 msgr2=0x55c14f58cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:48.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.728+0000 7f08efe00640 1 --2- 172.21.15.5:0/2892750500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c14f58ab30 0x55c14f58cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.728+0000 7f08efe00640 1 -- 172.21.15.5:0/2892750500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c14f580460 msgr2=0x55c14f58a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:48.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.728+0000 7f08efe00640 1 --2- 172.21.15.5:0/2892750500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c14f580460 0x55c14f58a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.728+0000 7f08efe00640 1 -- 172.21.15.5:0/2892750500 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c14f548310 con 0x55c14f57fb40 2024-04-24T06:50:48.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.728+0000 7f08ed600640 1 --2- 172.21.15.5:0/2945867805 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f08d80a3d20 0x7f08d80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:48.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.728+0000 7f08efe00640 1 --2- 172.21.15.5:0/2892750500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c14f57fb40 0x55c14f57ff20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f08e0031490 tx=0x7f08e0031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b89c28e6461490af server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:48.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.728+0000 7f08ed600640 1 -- 172.21.15.5:0/2945867805 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f08e80cfca0 con 0x7f08f010e4f0 2024-04-24T06:50:48.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.728+0000 7f08de000640 1 -- 172.21.15.5:0/2892750500 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e0035040 con 0x55c14f57fb40 2024-04-24T06:50:48.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.729+0000 7f08f4e00640 1 --2- 172.21.15.5:0/2945867805 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f08d80a3d20 0x7f08d80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:48.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.729+0000 7f08de000640 1 -- 172.21.15.5:0/2892750500 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08e0031540 con 0x55c14f57fb40 2024-04-24T06:50:48.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.729+0000 7f08de000640 1 -- 172.21.15.5:0/2892750500 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e003d6e0 con 0x55c14f57fb40 2024-04-24T06:50:48.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.729+0000 7f08fa5d9740 1 -- 172.21.15.5:0/2892750500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c14f57fb40 msgr2=0x55c14f57ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:48.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.729+0000 7f08fa5d9740 1 --2- 172.21.15.5:0/2892750500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c14f57fb40 0x55c14f57ff20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f08e0031490 tx=0x7f08e0031400 comp rx=0 tx=0).stop 2024-04-24T06:50:48.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.729+0000 7f08f4e00640 1 --2- 172.21.15.5:0/2945867805 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f08d80a3d20 0x7f08d80a61e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x55c14f581100 tx=0x7f08e4005f90 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:48.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.729+0000 7f08fa5d9740 1 -- 172.21.15.5:0/2892750500 shutdown_connections 2024-04-24T06:50:48.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.729+0000 7f08fa5d9740 1 --2- 172.21.15.5:0/2892750500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c14f58ab30 0x55c14f58cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.729+0000 7f08fa5d9740 1 --2- 172.21.15.5:0/2892750500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c14f580460 0x55c14f58a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.729+0000 7f08fa5d9740 1 --2- 172.21.15.5:0/2892750500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c14f57fb40 0x55c14f57ff20 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.729+0000 7f08fa5d9740 1 -- 172.21.15.5:0/2892750500 >> 172.21.15.5:0/2892750500 conn(0x55c14f57be60 msgr2=0x55c14f57c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:48.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.729+0000 7f08fa5d9740 1 -- 172.21.15.5:0/2892750500 shutdown_connections 2024-04-24T06:50:48.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.730+0000 7f08fa5d9740 1 -- 172.21.15.5:0/2892750500 wait complete. 2024-04-24T06:50:48.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.730+0000 7f08fa5d9740 1 -- start start 2024-04-24T06:50:48.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.730+0000 7f08fa5d9740 1 Processor -- start 2024-04-24T06:50:48.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.731+0000 7f08fa5d9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c14f57fb40 0x55c14f58a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:48.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.731+0000 7f08fa5d9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c14f580460 0x55c14f587850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:48.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.731+0000 7f08fa5d9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c14f58ab30 0x55c14f587f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:48.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.731+0000 7f08fa5d9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c14f58d960 con 0x55c14f58ab30 2024-04-24T06:50:48.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.731+0000 7f08fa5d9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c14f49e810 con 0x55c14f580460 2024-04-24T06:50:48.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.731+0000 7f08fa5d9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c14f58dae0 con 0x55c14f57fb40 2024-04-24T06:50:48.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.731+0000 7f08fa5d9740 10 client.0 _ll_register_callbacks cb 0x55c14f46c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:50:48.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.731+0000 7f08f5800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c14f58ab30 0x55c14f587f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:48.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.731+0000 7f08efe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c14f580460 0x55c14f587850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:48.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.731+0000 7f08f4e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c14f57fb40 0x55c14f58a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:48.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.731+0000 7f08efe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c14f580460 0x55c14f587850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:51702/0 (socket says 172.21.15.5:51702) 2024-04-24T06:50:48.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.731+0000 7f08f5800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c14f58ab30 0x55c14f587f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53578/0 (socket says 172.21.15.5:53578) 2024-04-24T06:50:48.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.731+0000 7f08efe00640 1 -- 172.21.15.5:0/8308519 learned_addr learned my addr 172.21.15.5:0/8308519 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:48.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.732+0000 7f08f5800640 1 -- 172.21.15.5:0/8308519 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c14f57fb40 msgr2=0x55c14f58a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:48.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.732+0000 7f08f5800640 1 --2- 172.21.15.5:0/8308519 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c14f57fb40 0x55c14f58a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.732+0000 7f08f5800640 1 -- 172.21.15.5:0/8308519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c14f580460 msgr2=0x55c14f587850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:48.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.732+0000 7f08f5800640 1 --2- 172.21.15.5:0/8308519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c14f580460 0x55c14f587850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.732+0000 7f08f5800640 1 -- 172.21.15.5:0/8308519 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c14f4af3f0 con 0x55c14f58ab30 2024-04-24T06:50:48.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.732+0000 7f08efe00640 1 --2- 172.21.15.5:0/8308519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c14f580460 0x55c14f587850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:50:48.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.732+0000 7f08f4e00640 1 --2- 172.21.15.5:0/8308519 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c14f57fb40 0x55c14f58a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:50:48.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.732+0000 7f08f5800640 1 --2- 172.21.15.5:0/8308519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c14f58ab30 0x55c14f587f90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f08e80cd4b0 tx=0x7f08e809be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb4ca1e8db160ac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:48.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.732+0000 7f08d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:50:48.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.732+0000 7f08d2a00640 1 -- 172.21.15.5:0/8308519 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e80b5060 con 0x55c14f58ab30 2024-04-24T06:50:48.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.732+0000 7f08d2a00640 1 -- 172.21.15.5:0/8308519 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f08e8098420 con 0x55c14f58ab30 2024-04-24T06:50:48.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.732+0000 7f08fa5d9740 1 -- 172.21.15.5:0/8308519 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08e0031540 con 0x55c14f58ab30 2024-04-24T06:50:48.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.732+0000 7f08d2a00640 1 -- 172.21.15.5:0/8308519 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f08e807abd0 con 0x55c14f58ab30 2024-04-24T06:50:48.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.732+0000 7f08fa5d9740 10 client.16569 fetch_fsmap learned FSMap version 27 2024-04-24T06:50:48.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.732+0000 7f08fa5d9740 1 -- 172.21.15.5:0/8308519 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c14f1f73f0 con 0x55c14f58ab30 2024-04-24T06:50:48.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.733+0000 7f08d2a00640 1 -- 172.21.15.5:0/8308519 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08e0031540 con 0x55c14f58ab30 2024-04-24T06:50:48.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.733+0000 7f08d2a00640 1 -- 172.21.15.5:0/8308519 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f08e8086040 con 0x55c14f58ab30 2024-04-24T06:50:48.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.733+0000 7f08fa5d9740 10 client.16569 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:50:48.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.733+0000 7f08fa5d9740 10 client.16569 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-24T06:50:48.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.733+0000 7f08fa5d9740 20 client.16569 populate_metadata read hostname 'smithi005' 2024-04-24T06:50:48.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.733+0000 7f08fa5d9740 1 --2- 172.21.15.5:0/8308519 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55c14f59abf0 0x55c14f59d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:48.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.733+0000 7f08fa5d9740 4 client.16569 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:50:48.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.733+0000 7f08fa5d9740 1 -- 172.21.15.5:0/8308519 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c14f4f2bf0 con 0x55c14f59abf0 2024-04-24T06:50:48.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.733+0000 7f08f4e00640 1 --2- 172.21.15.5:0/8308519 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55c14f59abf0 0x55c14f59d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:48.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.734+0000 7f08f4e00640 1 --2- 172.21.15.5:0/8308519 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55c14f59abf0 0x55c14f59d020 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1c1f0a52834191d8 server_cookie=2c91ed630ea8f8a2 in_seq=0 out_seq=0 2024-04-24T06:50:48.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.734+0000 7f08d2a00640 10 client.16569 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:50:48.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.735+0000 7f08d2a00640 1 -- 172.21.15.5:0/8308519 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c14f4f2bf0 con 0x55c14f59abf0 2024-04-24T06:50:48.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.735+0000 7f08d2a00640 10 client.16569 handle_command_reply: tid=0 2024-04-24T06:50:48.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.735+0000 7f08fa5d9740 1 client.16569 shutdown 2024-04-24T06:50:48.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.735+0000 7f08fa5d9740 1 -- 172.21.15.5:0/8308519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c14f58ab30 msgr2=0x55c14f587f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:48.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.735+0000 7f08fa5d9740 1 --2- 172.21.15.5:0/8308519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c14f58ab30 0x55c14f587f90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f08e80cd4b0 tx=0x7f08e809be70 comp rx=0 tx=0).stop 2024-04-24T06:50:48.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.735+0000 7f08fa5d9740 1 -- 172.21.15.5:0/8308519 shutdown_connections 2024-04-24T06:50:48.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.735+0000 7f08fa5d9740 1 --2- 172.21.15.5:0/8308519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c14f58ab30 0x55c14f587f90 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.735+0000 7f08fa5d9740 1 --2- 172.21.15.5:0/8308519 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55c14f59abf0 0x55c14f59d020 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.735+0000 7f08fa5d9740 1 --2- 172.21.15.5:0/8308519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c14f580460 0x55c14f587850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.735+0000 7f08fa5d9740 1 --2- 172.21.15.5:0/8308519 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c14f57fb40 0x55c14f58a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.735+0000 7f08d2a00640 0 client.16569 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:50:48.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.735+0000 7f08fa5d9740 1 -- 172.21.15.5:0/8308519 >> 172.21.15.5:0/8308519 conn(0x55c14f57be60 msgr2=0x55c14f581b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:48.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.735+0000 7f08fa5d9740 1 -- 172.21.15.5:0/8308519 shutdown_connections 2024-04-24T06:50:48.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.736+0000 7f08fa5d9740 1 -- 172.21.15.5:0/8308519 wait complete. 2024-04-24T06:50:48.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.736+0000 7f08fa5d9740 20 client.16569 trim_cache size 0 max 16384 2024-04-24T06:50:48.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.751+0000 7f08fa5d9740 1 Processor -- start 2024-04-24T06:50:48.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.751+0000 7f08fa5d9740 1 -- start start 2024-04-24T06:50:48.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.751+0000 7f08fa5d9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c14f57fb40 0x55c14f596850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:48.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.751+0000 7f08fa5d9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c14f580460 0x55c14f586830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:48.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.751+0000 7f08fa5d9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c14f58ab30 0x55c14f58a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:48.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.751+0000 7f08fa5d9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f08f01b6a90 con 0x55c14f57fb40 2024-04-24T06:50:48.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.751+0000 7f08fa5d9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f08f01b4360 con 0x55c14f58ab30 2024-04-24T06:50:48.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.751+0000 7f08fa5d9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f08f01b21b0 con 0x55c14f580460 2024-04-24T06:50:48.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.751+0000 7f08f4e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c14f57fb40 0x55c14f596850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:48.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.751+0000 7f08f4e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c14f57fb40 0x55c14f596850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53582/0 (socket says 172.21.15.5:53582) 2024-04-24T06:50:48.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.751+0000 7f08f4e00640 1 -- 172.21.15.5:0/1427535282 learned_addr learned my addr 172.21.15.5:0/1427535282 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:48.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.751+0000 7f08efe00640 1 --2- 172.21.15.5:0/1427535282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c14f580460 0x55c14f586830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:48.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.751+0000 7f08f5800640 1 --2- 172.21.15.5:0/1427535282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c14f58ab30 0x55c14f58a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:48.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.752+0000 7f08f4e00640 1 -- 172.21.15.5:0/1427535282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c14f580460 msgr2=0x55c14f586830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:48.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.752+0000 7f08f4e00640 1 --2- 172.21.15.5:0/1427535282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c14f580460 0x55c14f586830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.752+0000 7f08f4e00640 1 -- 172.21.15.5:0/1427535282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c14f58ab30 msgr2=0x55c14f58a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:48.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.752+0000 7f08f4e00640 1 --2- 172.21.15.5:0/1427535282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c14f58ab30 0x55c14f58a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.752+0000 7f08f4e00640 1 -- 172.21.15.5:0/1427535282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c14f2c17d0 con 0x55c14f57fb40 2024-04-24T06:50:48.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.752+0000 7f08efe00640 1 --2- 172.21.15.5:0/1427535282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c14f580460 0x55c14f586830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:50:48.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.752+0000 7f08f5800640 1 --2- 172.21.15.5:0/1427535282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c14f58ab30 0x55c14f58a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:50:48.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.752+0000 7f08f4e00640 1 --2- 172.21.15.5:0/1427535282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c14f57fb40 0x55c14f596850 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f08e400fca0 tx=0x7f08e4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f250a9bb524afc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:48.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.752+0000 7f08dd600640 1 -- 172.21.15.5:0/1427535282 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e4031070 con 0x55c14f57fb40 2024-04-24T06:50:48.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.752+0000 7f08dd600640 1 -- 172.21.15.5:0/1427535282 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08e401d860 con 0x55c14f57fb40 2024-04-24T06:50:48.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.752+0000 7f08dd600640 1 -- 172.21.15.5:0/1427535282 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e4029810 con 0x55c14f57fb40 2024-04-24T06:50:48.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.753+0000 7f08fa5d9740 1 -- 172.21.15.5:0/1427535282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c14f57fb40 msgr2=0x55c14f596850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:48.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.753+0000 7f08fa5d9740 1 --2- 172.21.15.5:0/1427535282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c14f57fb40 0x55c14f596850 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f08e400fca0 tx=0x7f08e4019000 comp rx=0 tx=0).stop 2024-04-24T06:50:48.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.753+0000 7f08fa5d9740 1 -- 172.21.15.5:0/1427535282 shutdown_connections 2024-04-24T06:50:48.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.753+0000 7f08fa5d9740 1 --2- 172.21.15.5:0/1427535282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c14f58ab30 0x55c14f58a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.753+0000 7f08fa5d9740 1 --2- 172.21.15.5:0/1427535282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c14f580460 0x55c14f586830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.753+0000 7f08fa5d9740 1 --2- 172.21.15.5:0/1427535282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c14f57fb40 0x55c14f596850 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.753+0000 7f08fa5d9740 1 -- 172.21.15.5:0/1427535282 >> 172.21.15.5:0/1427535282 conn(0x55c14f57be60 msgr2=0x55c14f586120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:48.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.753+0000 7f08fa5d9740 1 -- 172.21.15.5:0/1427535282 shutdown_connections 2024-04-24T06:50:48.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.753+0000 7f08fa5d9740 1 -- 172.21.15.5:0/1427535282 wait complete. 2024-04-24T06:50:48.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.754+0000 7f08fa5d9740 1 -- start start 2024-04-24T06:50:48.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.754+0000 7f08fa5d9740 1 Processor -- start 2024-04-24T06:50:48.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.754+0000 7f08fa5d9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c14f57fb40 0x55c14f58b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:48.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.754+0000 7f08fa5d9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c14f580460 0x55c14f58b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:48.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.754+0000 7f08fa5d9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c14f58ab30 0x55c14f591d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:48.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.754+0000 7f08fa5d9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f08f01b2820 con 0x55c14f57fb40 2024-04-24T06:50:48.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.754+0000 7f08fa5d9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f08f01b3c90 con 0x55c14f580460 2024-04-24T06:50:48.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.754+0000 7f08fa5d9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f08f01b2ef0 con 0x55c14f58ab30 2024-04-24T06:50:48.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.754+0000 7f08fa5d9740 10 client.0 _ll_register_callbacks cb 0x7f08e40096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:50:48.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.754+0000 7f08f4e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c14f57fb40 0x55c14f58b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:48.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.754+0000 7f08f4e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c14f57fb40 0x55c14f58b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53588/0 (socket says 172.21.15.5:53588) 2024-04-24T06:50:48.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.754+0000 7f08f4e00640 1 -- 172.21.15.5:0/2409290055 learned_addr learned my addr 172.21.15.5:0/2409290055 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:48.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.755+0000 7f08efe00640 1 --2- 172.21.15.5:0/2409290055 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c14f580460 0x55c14f58b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:48.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.755+0000 7f08f5800640 1 --2- 172.21.15.5:0/2409290055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c14f58ab30 0x55c14f591d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:48.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.755+0000 7f08f4e00640 1 -- 172.21.15.5:0/2409290055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c14f58ab30 msgr2=0x55c14f591d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:48.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.755+0000 7f08f4e00640 1 --2- 172.21.15.5:0/2409290055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c14f58ab30 0x55c14f591d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.755+0000 7f08f4e00640 1 -- 172.21.15.5:0/2409290055 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c14f580460 msgr2=0x55c14f58b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:48.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.755+0000 7f08f4e00640 1 --2- 172.21.15.5:0/2409290055 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c14f580460 0x55c14f58b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.755+0000 7f08f4e00640 1 -- 172.21.15.5:0/2409290055 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c14f58be80 con 0x55c14f57fb40 2024-04-24T06:50:48.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.755+0000 7f08f5800640 1 --2- 172.21.15.5:0/2409290055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c14f58ab30 0x55c14f591d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:50:48.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.755+0000 7f08f4e00640 1 --2- 172.21.15.5:0/2409290055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c14f57fb40 0x55c14f58b050 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f08e400ff80 tx=0x7f08e402b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1892e2e444c8c5d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:48.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.755+0000 7f08d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:50:48.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.756+0000 7f08d2a00640 1 -- 172.21.15.5:0/2409290055 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e4031040 con 0x55c14f57fb40 2024-04-24T06:50:48.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.756+0000 7f08d2a00640 1 -- 172.21.15.5:0/2409290055 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f08e401da10 con 0x55c14f57fb40 2024-04-24T06:50:48.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.756+0000 7f08fa5d9740 1 -- 172.21.15.5:0/2409290055 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08e401d860 con 0x55c14f57fb40 2024-04-24T06:50:48.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.756+0000 7f08d2a00640 1 -- 172.21.15.5:0/2409290055 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f08e401dc10 con 0x55c14f57fb40 2024-04-24T06:50:48.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.756+0000 7f08fa5d9740 10 client.16575 fetch_fsmap learned FSMap version 27 2024-04-24T06:50:48.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.756+0000 7f08fa5d9740 1 -- 172.21.15.5:0/2409290055 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c14f5933d0 con 0x55c14f57fb40 2024-04-24T06:50:48.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.756+0000 7f08d2a00640 1 -- 172.21.15.5:0/2409290055 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08e401d860 con 0x55c14f57fb40 2024-04-24T06:50:48.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.757+0000 7f08d2a00640 1 -- 172.21.15.5:0/2409290055 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f08e4004b50 con 0x55c14f57fb40 2024-04-24T06:50:48.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.757+0000 7f08fa5d9740 10 client.16575 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:50:48.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.757+0000 7f08fa5d9740 10 client.16575 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-24T06:50:48.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.757+0000 7f08fa5d9740 20 client.16575 populate_metadata read hostname 'smithi005' 2024-04-24T06:50:48.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.757+0000 7f08fa5d9740 1 --2- 172.21.15.5:0/2409290055 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55c14f6013c0 0x55c14f6037e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:48.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.757+0000 7f08fa5d9740 4 client.16575 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:50:48.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.757+0000 7f08fa5d9740 1 -- 172.21.15.5:0/2409290055 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c14f4eca60 con 0x55c14f6013c0 2024-04-24T06:50:48.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.757+0000 7f08efe00640 1 --2- 172.21.15.5:0/2409290055 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55c14f6013c0 0x55c14f6037e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:48.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.758+0000 7f08efe00640 1 --2- 172.21.15.5:0/2409290055 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55c14f6013c0 0x55c14f6037e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1760eafcda43bdc4 server_cookie=89da8aecae4586cf in_seq=0 out_seq=0 2024-04-24T06:50:48.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.758+0000 7f08d2a00640 10 client.16575 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:50:48.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.758+0000 7f08d2a00640 1 -- 172.21.15.5:0/2409290055 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c14f4eca60 con 0x55c14f6013c0 2024-04-24T06:50:48.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.758+0000 7f08d2a00640 10 client.16575 handle_command_reply: tid=0 2024-04-24T06:50:48.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.758+0000 7f08fa5d9740 1 client.16575 shutdown 2024-04-24T06:50:48.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.758+0000 7f08fa5d9740 1 -- 172.21.15.5:0/2409290055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c14f57fb40 msgr2=0x55c14f58b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:48.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.758+0000 7f08fa5d9740 1 --2- 172.21.15.5:0/2409290055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c14f57fb40 0x55c14f58b050 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f08e400ff80 tx=0x7f08e402b4f0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.758+0000 7f08fa5d9740 1 -- 172.21.15.5:0/2409290055 shutdown_connections 2024-04-24T06:50:48.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.758+0000 7f08fa5d9740 1 --2- 172.21.15.5:0/2409290055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c14f58ab30 0x55c14f591d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.758+0000 7f08fa5d9740 1 --2- 172.21.15.5:0/2409290055 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55c14f6013c0 0x55c14f6037e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.758+0000 7f08fa5d9740 1 --2- 172.21.15.5:0/2409290055 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c14f580460 0x55c14f58b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.758+0000 7f08fa5d9740 1 --2- 172.21.15.5:0/2409290055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c14f57fb40 0x55c14f58b050 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.758+0000 7f08d2a00640 0 client.16575 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:50:48.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.758+0000 7f08fa5d9740 1 -- 172.21.15.5:0/2409290055 >> 172.21.15.5:0/2409290055 conn(0x55c14f57be60 msgr2=0x55c14f581290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:48.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.758+0000 7f08fa5d9740 1 -- 172.21.15.5:0/2409290055 shutdown_connections 2024-04-24T06:50:48.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.758+0000 7f08fa5d9740 1 -- 172.21.15.5:0/2409290055 wait complete. 2024-04-24T06:50:48.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.759+0000 7f08fa5d9740 20 client.16575 trim_cache size 0 max 16384 2024-04-24T06:50:48.770 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:50:48.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.759+0000 7f08f6200640 1 -- 172.21.15.5:0/2945867805 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f08d80a3d20 msgr2=0x7f08d80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:48.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.759+0000 7f08f6200640 1 --2- 172.21.15.5:0/2945867805 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f08d80a3d20 0x7f08d80a61e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x55c14f581100 tx=0x7f08e4005f90 comp rx=0 tx=0).stop 2024-04-24T06:50:48.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.759+0000 7f08f6200640 1 -- 172.21.15.5:0/2945867805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08f010e4f0 msgr2=0x7f08f01a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:48.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.759+0000 7f08f6200640 1 --2- 172.21.15.5:0/2945867805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08f010e4f0 0x7f08f01a3660 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f08e800b970 tx=0x7f08e800be30 comp rx=0 tx=0).stop 2024-04-24T06:50:48.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.759+0000 7f08f6200640 1 -- 172.21.15.5:0/2945867805 shutdown_connections 2024-04-24T06:50:48.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.759+0000 7f08f6200640 1 --2- 172.21.15.5:0/2945867805 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f08d80a3d20 0x7f08d80a61e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.759+0000 7f08f6200640 1 --2- 172.21.15.5:0/2945867805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08f010e4f0 0x7f08f01a3660 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.759+0000 7f08f6200640 1 --2- 172.21.15.5:0/2945867805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08f0108ee0 0x7f08f019f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.759+0000 7f08f6200640 1 --2- 172.21.15.5:0/2945867805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08f0108540 0x7f08f019eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.759+0000 7f08f6200640 1 -- 172.21.15.5:0/2945867805 >> 172.21.15.5:0/2945867805 conn(0x7f08f0077e10 msgr2=0x7f08f0078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:48.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.759+0000 7f08f6200640 1 -- 172.21.15.5:0/2945867805 shutdown_connections 2024-04-24T06:50:48.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.759+0000 7f08f6200640 1 -- 172.21.15.5:0/2945867805 wait complete. 2024-04-24T06:50:48.778 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T06:50:48.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.982+0000 7ffa5be00640 1 -- 172.21.15.5:0/4072555244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa5c100dd0 msgr2=0x7ffa5c1011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:48.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.982+0000 7ffa5be00640 1 --2- 172.21.15.5:0/4072555244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa5c100dd0 0x7ffa5c1011b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c009900 tx=0x7ffa4c02a480 comp rx=0 tx=0).stop 2024-04-24T06:50:48.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.982+0000 7ffa5be00640 1 -- 172.21.15.5:0/4072555244 shutdown_connections 2024-04-24T06:50:48.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.982+0000 7ffa5be00640 1 --2- 172.21.15.5:0/4072555244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa5c1061d0 0x7ffa5c106630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.982+0000 7ffa5be00640 1 --2- 172.21.15.5:0/4072555244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa5c1016f0 0x7ffa5c105bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.982+0000 7ffa5be00640 1 --2- 172.21.15.5:0/4072555244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa5c100dd0 0x7ffa5c1011b0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.982+0000 7ffa5be00640 1 -- 172.21.15.5:0/4072555244 >> 172.21.15.5:0/4072555244 conn(0x7ffa5c077ab0 msgr2=0x7ffa5c0fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:48.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.982+0000 7ffa5be00640 1 -- 172.21.15.5:0/4072555244 shutdown_connections 2024-04-24T06:50:48.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.982+0000 7ffa5be00640 1 -- 172.21.15.5:0/4072555244 wait complete. 2024-04-24T06:50:48.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.983+0000 7ffa5be00640 1 Processor -- start 2024-04-24T06:50:48.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.984+0000 7ffa5be00640 1 -- start start 2024-04-24T06:50:48.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.984+0000 7ffa5be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa5c100dd0 0x7ffa5c1a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:48.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.984+0000 7ffa5be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa5c1016f0 0x7ffa5c1a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:48.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.984+0000 7ffa5be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa5c1061d0 0x7ffa5c19eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:48.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.984+0000 7ffa5be00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c1187f0 con 0x7ffa5c1061d0 2024-04-24T06:50:48.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.984+0000 7ffa5be00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c118670 con 0x7ffa5c100dd0 2024-04-24T06:50:48.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.984+0000 7ffa5be00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c118970 con 0x7ffa5c1016f0 2024-04-24T06:50:48.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.985+0000 7ffa5b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa5c1061d0 0x7ffa5c19eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:48.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.985+0000 7ffa5aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa5c100dd0 0x7ffa5c1a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:48.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.985+0000 7ffa5aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa5c100dd0 0x7ffa5c1a4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:51750/0 (socket says 172.21.15.5:51750) 2024-04-24T06:50:48.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.985+0000 7ffa5aa00640 1 -- 172.21.15.5:0/595434667 learned_addr learned my addr 172.21.15.5:0/595434667 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:48.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.985+0000 7ffa52000640 1 --2- 172.21.15.5:0/595434667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa5c1016f0 0x7ffa5c1a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:48.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.985+0000 7ffa5b400640 1 -- 172.21.15.5:0/595434667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa5c1016f0 msgr2=0x7ffa5c1a4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:48.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.985+0000 7ffa5b400640 1 --2- 172.21.15.5:0/595434667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa5c1016f0 0x7ffa5c1a4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.985+0000 7ffa5b400640 1 -- 172.21.15.5:0/595434667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa5c100dd0 msgr2=0x7ffa5c1a4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:48.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.985+0000 7ffa5b400640 1 --2- 172.21.15.5:0/595434667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa5c100dd0 0x7ffa5c1a4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.985+0000 7ffa5b400640 1 -- 172.21.15.5:0/595434667 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa5c19f430 con 0x7ffa5c1061d0 2024-04-24T06:50:48.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.985+0000 7ffa5aa00640 1 --2- 172.21.15.5:0/595434667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa5c100dd0 0x7ffa5c1a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:50:48.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.985+0000 7ffa5b400640 1 --2- 172.21.15.5:0/595434667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa5c1061d0 0x7ffa5c19eb70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ffa5400e940 tx=0x7ffa5400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:48.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.986+0000 7ffa60a00640 1 -- 172.21.15.5:0/595434667 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa54004d20 con 0x7ffa5c1061d0 2024-04-24T06:50:48.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.986+0000 7ffa60a00640 1 -- 172.21.15.5:0/595434667 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa54005020 con 0x7ffa5c1061d0 2024-04-24T06:50:48.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.986+0000 7ffa60a00640 1 -- 172.21.15.5:0/595434667 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa540116b0 con 0x7ffa5c1061d0 2024-04-24T06:50:48.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.986+0000 7ffa5be00640 1 -- 172.21.15.5:0/595434667 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa5c19f720 con 0x7ffa5c1061d0 2024-04-24T06:50:48.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.986+0000 7ffa5be00640 1 -- 172.21.15.5:0/595434667 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa5c1ab9e0 con 0x7ffa5c1061d0 2024-04-24T06:50:48.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.989+0000 7ffa64c11740 1 Processor -- start 2024-04-24T06:50:48.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.989+0000 7ffa60a00640 1 -- 172.21.15.5:0/595434667 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7ffa540054d0 con 0x7ffa5c1061d0 2024-04-24T06:50:48.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.989+0000 7ffa64c11740 1 -- start start 2024-04-24T06:50:48.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.990+0000 7ffa64c11740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f21e93b40 0x557f21e93f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:48.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.990+0000 7ffa64c11740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f21e94460 0x557f21e9e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:48.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.990+0000 7ffa64c11740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f21e9eb30 0x557f21ea0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:48.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.990+0000 7ffa64c11740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557f21df91d0 con 0x557f21e94460 2024-04-24T06:50:48.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.990+0000 7ffa64c11740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557f21d9f450 con 0x557f21e93b40 2024-04-24T06:50:48.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.990+0000 7ffa64c11740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557f21df5190 con 0x557f21e9eb30 2024-04-24T06:50:48.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.990+0000 7ffa5aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f21e94460 0x557f21e9e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:48.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.990+0000 7ffa5aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f21e94460 0x557f21e9e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53602/0 (socket says 172.21.15.5:53602) 2024-04-24T06:50:48.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.990+0000 7ffa52000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f21e93b40 0x557f21e93f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:48.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.990+0000 7ffa5aa00640 1 -- 172.21.15.5:0/3220642829 learned_addr learned my addr 172.21.15.5:0/3220642829 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:48.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.990+0000 7ffa52000640 1 --2- 172.21.15.5:0/3220642829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f21e9eb30 0x557f21ea0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:48.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.991+0000 7ffa5aa00640 1 -- 172.21.15.5:0/3220642829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f21e9eb30 msgr2=0x557f21ea0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:48.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.991+0000 7ffa5aa00640 1 --2- 172.21.15.5:0/3220642829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f21e9eb30 0x557f21ea0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.991+0000 7ffa5aa00640 1 -- 172.21.15.5:0/3220642829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f21e93b40 msgr2=0x557f21e93f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:48.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.991+0000 7ffa5aa00640 1 --2- 172.21.15.5:0/3220642829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f21e93b40 0x557f21e93f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:48.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.991+0000 7ffa5aa00640 1 -- 172.21.15.5:0/3220642829 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f21e5c310 con 0x557f21e94460 2024-04-24T06:50:48.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.991+0000 7ffa60a00640 1 --2- 172.21.15.5:0/595434667 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ffa400a3da0 0x7ffa400a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:48.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.991+0000 7ffa60a00640 1 -- 172.21.15.5:0/595434667 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7ffa540d07c0 con 0x7ffa5c1061d0 2024-04-24T06:50:48.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.991+0000 7ffa5aa00640 1 --2- 172.21.15.5:0/3220642829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f21e94460 0x557f21e9e5f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c005ec0 tx=0x7ffa4c0043f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a328830ce7813e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:48.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.991+0000 7ffa52000640 1 --2- 172.21.15.5:0/595434667 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ffa400a3da0 0x7ffa400a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:48.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.991+0000 7ffa4be00640 1 -- 172.21.15.5:0/3220642829 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c040070 con 0x557f21e94460 2024-04-24T06:50:48.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.991+0000 7ffa4be00640 1 -- 172.21.15.5:0/3220642829 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa4c0047a0 con 0x557f21e94460 2024-04-24T06:50:48.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.991+0000 7ffa4be00640 1 -- 172.21.15.5:0/3220642829 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c035040 con 0x557f21e94460 2024-04-24T06:50:49.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.992+0000 7ffa64c11740 1 -- 172.21.15.5:0/3220642829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f21e94460 msgr2=0x557f21e9e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:49.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.992+0000 7ffa64c11740 1 --2- 172.21.15.5:0/3220642829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f21e94460 0x557f21e9e5f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c005ec0 tx=0x7ffa4c0043f0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.992+0000 7ffa52000640 1 --2- 172.21.15.5:0/595434667 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ffa400a3da0 0x7ffa400a6260 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x557f21e90ff0 tx=0x7ffa44009770 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:49.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.992+0000 7ffa64c11740 1 -- 172.21.15.5:0/3220642829 shutdown_connections 2024-04-24T06:50:49.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.992+0000 7ffa64c11740 1 --2- 172.21.15.5:0/3220642829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f21e9eb30 0x557f21ea0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.992+0000 7ffa64c11740 1 --2- 172.21.15.5:0/3220642829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f21e94460 0x557f21e9e5f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.992+0000 7ffa64c11740 1 --2- 172.21.15.5:0/3220642829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f21e93b40 0x557f21e93f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.992+0000 7ffa64c11740 1 -- 172.21.15.5:0/3220642829 >> 172.21.15.5:0/3220642829 conn(0x557f21e8fe60 msgr2=0x557f21e90240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:49.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.992+0000 7ffa64c11740 1 -- 172.21.15.5:0/3220642829 shutdown_connections 2024-04-24T06:50:49.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.992+0000 7ffa64c11740 1 -- 172.21.15.5:0/3220642829 wait complete. 2024-04-24T06:50:49.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.993+0000 7ffa64c11740 1 -- start start 2024-04-24T06:50:49.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.993+0000 7ffa64c11740 1 Processor -- start 2024-04-24T06:50:49.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.993+0000 7ffa64c11740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f21e93b40 0x557f21e9e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:49.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.994+0000 7ffa64c11740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f21e94460 0x557f21e9b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:49.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.994+0000 7ffa64c11740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f21e9eb30 0x557f21e9bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:49.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.994+0000 7ffa64c11740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557f21ea1960 con 0x557f21e93b40 2024-04-24T06:50:49.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.994+0000 7ffa64c11740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557f21db2810 con 0x557f21e9eb30 2024-04-24T06:50:49.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.994+0000 7ffa64c11740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557f21ea1ae0 con 0x557f21e94460 2024-04-24T06:50:49.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.994+0000 7ffa64c11740 10 client.0 _ll_register_callbacks cb 0x557f21d80140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:50:49.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.994+0000 7ffa5aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f21e93b40 0x557f21e9e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:49.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.994+0000 7ffa52000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f21e94460 0x557f21e9b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:49.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.994+0000 7ffa52000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f21e94460 0x557f21e9b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:57236/0 (socket says 172.21.15.5:57236) 2024-04-24T06:50:49.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.994+0000 7ffa5aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f21e93b40 0x557f21e9e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53604/0 (socket says 172.21.15.5:53604) 2024-04-24T06:50:49.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.994+0000 7ffa52000640 1 -- 172.21.15.5:0/2660521774 learned_addr learned my addr 172.21.15.5:0/2660521774 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:49.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.994+0000 7ffa5b400640 1 --2- 172.21.15.5:0/2660521774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f21e9eb30 0x557f21e9bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:49.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.994+0000 7ffa52000640 1 -- 172.21.15.5:0/2660521774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f21e9eb30 msgr2=0x557f21e9bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:49.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.994+0000 7ffa52000640 1 --2- 172.21.15.5:0/2660521774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f21e9eb30 0x557f21e9bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.994+0000 7ffa52000640 1 -- 172.21.15.5:0/2660521774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f21e93b40 msgr2=0x557f21e9e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:49.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.994+0000 7ffa52000640 1 --2- 172.21.15.5:0/2660521774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f21e93b40 0x557f21e9e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.994+0000 7ffa52000640 1 -- 172.21.15.5:0/2660521774 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f21dc33f0 con 0x557f21e94460 2024-04-24T06:50:49.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.994+0000 7ffa5aa00640 1 --2- 172.21.15.5:0/2660521774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f21e93b40 0x557f21e9e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:50:49.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.995+0000 7ffa52000640 1 --2- 172.21.15.5:0/2660521774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f21e94460 0x557f21e9b8a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ffa44011a80 tx=0x7ffa44004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0651139950b1f6e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:49.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.995+0000 7ffa48c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:50:49.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.995+0000 7ffa48c00640 1 -- 172.21.15.5:0/2660521774 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44004480 con 0x557f21e94460 2024-04-24T06:50:49.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.995+0000 7ffa64c11740 1 -- 172.21.15.5:0/2660521774 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa4c0047a0 con 0x557f21e94460 2024-04-24T06:50:49.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.995+0000 7ffa48c00640 1 -- 172.21.15.5:0/2660521774 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7ffa44009040 con 0x557f21e94460 2024-04-24T06:50:49.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.995+0000 7ffa48c00640 1 -- 172.21.15.5:0/2660521774 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ffa44021bc0 con 0x557f21e94460 2024-04-24T06:50:49.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.996+0000 7ffa64c11740 10 client.26734 fetch_fsmap learned FSMap version 27 2024-04-24T06:50:49.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.996+0000 7ffa64c11740 1 -- 172.21.15.5:0/2660521774 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557f21b0b3f0 con 0x557f21e94460 2024-04-24T06:50:49.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.996+0000 7ffa48c00640 1 -- 172.21.15.5:0/2660521774 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa4c0047a0 con 0x557f21e94460 2024-04-24T06:50:49.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.996+0000 7ffa48c00640 1 -- 172.21.15.5:0/2660521774 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7ffa4401d070 con 0x557f21e94460 2024-04-24T06:50:49.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.996+0000 7ffa64c11740 10 client.26734 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:50:49.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.996+0000 7ffa64c11740 10 client.26734 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T06:50:49.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.996+0000 7ffa64c11740 20 client.26734 populate_metadata read hostname 'smithi005' 2024-04-24T06:50:49.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.996+0000 7ffa64c11740 1 --2- 172.21.15.5:0/2660521774 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x557f21eaed10 0x557f21eb11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:49.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.996+0000 7ffa64c11740 4 client.26734 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:50:49.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.996+0000 7ffa64c11740 1 -- 172.21.15.5:0/2660521774 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f21e06bf0 con 0x557f21eaed10 2024-04-24T06:50:49.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.996+0000 7ffa5aa00640 1 --2- 172.21.15.5:0/2660521774 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x557f21eaed10 0x557f21eb11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:49.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.997+0000 7ffa5aa00640 1 --2- 172.21.15.5:0/2660521774 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x557f21eaed10 0x557f21eb11b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4dd1c5e2d6e7b30a server_cookie=f24f8baf3f7cf9eb in_seq=0 out_seq=0 2024-04-24T06:50:49.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.997+0000 7ffa48c00640 10 client.26734 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:50:49.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.998+0000 7ffa48c00640 1 -- 172.21.15.5:0/2660521774 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557f21e06bf0 con 0x557f21eaed10 2024-04-24T06:50:49.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.998+0000 7ffa48c00640 10 client.26734 handle_command_reply: tid=0 2024-04-24T06:50:49.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.998+0000 7ffa64c11740 1 client.26734 shutdown 2024-04-24T06:50:49.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.998+0000 7ffa64c11740 1 -- 172.21.15.5:0/2660521774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f21e94460 msgr2=0x557f21e9b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:49.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.998+0000 7ffa64c11740 1 --2- 172.21.15.5:0/2660521774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f21e94460 0x557f21e9b8a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ffa44011a80 tx=0x7ffa44004730 comp rx=0 tx=0).stop 2024-04-24T06:50:49.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.998+0000 7ffa64c11740 1 -- 172.21.15.5:0/2660521774 shutdown_connections 2024-04-24T06:50:49.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.998+0000 7ffa64c11740 1 --2- 172.21.15.5:0/2660521774 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x557f21eaed10 0x557f21eb11b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.998+0000 7ffa64c11740 1 --2- 172.21.15.5:0/2660521774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f21e9eb30 0x557f21e9bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.998+0000 7ffa64c11740 1 --2- 172.21.15.5:0/2660521774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f21e94460 0x557f21e9b8a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.998+0000 7ffa64c11740 1 --2- 172.21.15.5:0/2660521774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f21e93b40 0x557f21e9e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.998+0000 7ffa48c00640 0 client.26734 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:50:49.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.998+0000 7ffa64c11740 1 -- 172.21.15.5:0/2660521774 >> 172.21.15.5:0/2660521774 conn(0x557f21e8fe60 msgr2=0x557f21e95b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:49.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.998+0000 7ffa64c11740 1 -- 172.21.15.5:0/2660521774 shutdown_connections 2024-04-24T06:50:49.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.998+0000 7ffa64c11740 1 -- 172.21.15.5:0/2660521774 wait complete. 2024-04-24T06:50:49.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:48.998+0000 7ffa64c11740 20 client.26734 trim_cache size 0 max 16384 2024-04-24T06:50:49.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.013+0000 7ffa64c11740 1 Processor -- start 2024-04-24T06:50:49.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.013+0000 7ffa64c11740 1 -- start start 2024-04-24T06:50:49.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.014+0000 7ffa64c11740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f21e93b40 0x557f21e8f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:49.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.014+0000 7ffa64c11740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f21e94460 0x557f21e9a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:49.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.014+0000 7ffa64c11740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f21e9eb30 0x557f21e9e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:49.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.014+0000 7ffa64c11740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c1b6a10 con 0x557f21e94460 2024-04-24T06:50:49.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.014+0000 7ffa64c11740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c1b42e0 con 0x557f21e9eb30 2024-04-24T06:50:49.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.014+0000 7ffa64c11740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c1b2130 con 0x557f21e93b40 2024-04-24T06:50:49.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.014+0000 7ffa52000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f21e94460 0x557f21e9a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:49.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.014+0000 7ffa52000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f21e94460 0x557f21e9a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53606/0 (socket says 172.21.15.5:53606) 2024-04-24T06:50:49.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.014+0000 7ffa52000640 1 -- 172.21.15.5:0/1586906093 learned_addr learned my addr 172.21.15.5:0/1586906093 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:49.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.014+0000 7ffa5aa00640 1 --2- 172.21.15.5:0/1586906093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f21e93b40 0x557f21e8f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:49.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.014+0000 7ffa5b400640 1 --2- 172.21.15.5:0/1586906093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f21e9eb30 0x557f21e9e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:49.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.014+0000 7ffa52000640 1 -- 172.21.15.5:0/1586906093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f21e93b40 msgr2=0x557f21e8f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:49.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.014+0000 7ffa52000640 1 --2- 172.21.15.5:0/1586906093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f21e93b40 0x557f21e8f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.014+0000 7ffa52000640 1 -- 172.21.15.5:0/1586906093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f21e9eb30 msgr2=0x557f21e9e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:49.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.014+0000 7ffa52000640 1 --2- 172.21.15.5:0/1586906093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f21e9eb30 0x557f21e9e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.014+0000 7ffa52000640 1 -- 172.21.15.5:0/1586906093 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f21bd57d0 con 0x557f21e94460 2024-04-24T06:50:49.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.014+0000 7ffa5aa00640 1 --2- 172.21.15.5:0/1586906093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f21e93b40 0x557f21e8f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:50:49.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.015+0000 7ffa52000640 1 --2- 172.21.15.5:0/1586906093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f21e94460 0x557f21e9a880 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ffa440097e0 tx=0x7ffa44004220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c4aafdd65d39c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:49.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.015+0000 7ffa4b400640 1 -- 172.21.15.5:0/1586906093 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44021800 con 0x557f21e94460 2024-04-24T06:50:49.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.015+0000 7ffa4b400640 1 -- 172.21.15.5:0/1586906093 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa44011840 con 0x557f21e94460 2024-04-24T06:50:49.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.015+0000 7ffa4b400640 1 -- 172.21.15.5:0/1586906093 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44004d60 con 0x557f21e94460 2024-04-24T06:50:49.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.016+0000 7ffa64c11740 1 -- 172.21.15.5:0/1586906093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f21e94460 msgr2=0x557f21e9a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:49.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.016+0000 7ffa64c11740 1 --2- 172.21.15.5:0/1586906093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f21e94460 0x557f21e9a880 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ffa440097e0 tx=0x7ffa44004220 comp rx=0 tx=0).stop 2024-04-24T06:50:49.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.016+0000 7ffa64c11740 1 -- 172.21.15.5:0/1586906093 shutdown_connections 2024-04-24T06:50:49.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.016+0000 7ffa64c11740 1 --2- 172.21.15.5:0/1586906093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f21e9eb30 0x557f21e9e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.016+0000 7ffa64c11740 1 --2- 172.21.15.5:0/1586906093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f21e94460 0x557f21e9a880 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.016+0000 7ffa64c11740 1 --2- 172.21.15.5:0/1586906093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f21e93b40 0x557f21e8f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.016+0000 7ffa64c11740 1 -- 172.21.15.5:0/1586906093 >> 172.21.15.5:0/1586906093 conn(0x557f21e8fe60 msgr2=0x557f21e9a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:49.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.016+0000 7ffa64c11740 1 -- 172.21.15.5:0/1586906093 shutdown_connections 2024-04-24T06:50:49.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.016+0000 7ffa64c11740 1 -- 172.21.15.5:0/1586906093 wait complete. 2024-04-24T06:50:49.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.016+0000 7ffa64c11740 1 -- start start 2024-04-24T06:50:49.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.017+0000 7ffa64c11740 1 Processor -- start 2024-04-24T06:50:49.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.017+0000 7ffa64c11740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f21e93b40 0x557f21e9ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:49.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.017+0000 7ffa64c11740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f21e94460 0x557f21e9f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:49.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.017+0000 7ffa64c11740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f21e9eb30 0x557f21e9f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:49.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.017+0000 7ffa64c11740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c1b27a0 con 0x557f21e9eb30 2024-04-24T06:50:49.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.017+0000 7ffa64c11740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c1b3c10 con 0x557f21e94460 2024-04-24T06:50:49.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.017+0000 7ffa64c11740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c1b2e70 con 0x557f21e93b40 2024-04-24T06:50:49.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.017+0000 7ffa64c11740 10 client.0 _ll_register_callbacks cb 0x7ffa44003f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:50:49.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.017+0000 7ffa5b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f21e9eb30 0x557f21e9f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:49.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.017+0000 7ffa5aa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f21e93b40 0x557f21e9ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:49.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.017+0000 7ffa52000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f21e94460 0x557f21e9f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:49.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.018+0000 7ffa5aa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f21e93b40 0x557f21e9ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:57246/0 (socket says 172.21.15.5:57246) 2024-04-24T06:50:49.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.018+0000 7ffa52000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f21e94460 0x557f21e9f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:51788/0 (socket says 172.21.15.5:51788) 2024-04-24T06:50:49.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.018+0000 7ffa5b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f21e9eb30 0x557f21e9f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53608/0 (socket says 172.21.15.5:53608) 2024-04-24T06:50:49.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.018+0000 7ffa5aa00640 1 -- 172.21.15.5:0/3017835381 learned_addr learned my addr 172.21.15.5:0/3017835381 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:49.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.018+0000 7ffa5aa00640 1 -- 172.21.15.5:0/3017835381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f21e94460 msgr2=0x557f21e9f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:49.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.018+0000 7ffa5aa00640 1 --2- 172.21.15.5:0/3017835381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f21e94460 0x557f21e9f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.018+0000 7ffa5aa00640 1 -- 172.21.15.5:0/3017835381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f21e9eb30 msgr2=0x557f21e9f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:49.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.018+0000 7ffa5aa00640 1 --2- 172.21.15.5:0/3017835381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f21e9eb30 0x557f21e9f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.018+0000 7ffa5aa00640 1 -- 172.21.15.5:0/3017835381 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f21ea5f10 con 0x557f21e93b40 2024-04-24T06:50:49.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.018+0000 7ffa52000640 1 --2- 172.21.15.5:0/3017835381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f21e94460 0x557f21e9f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:50:49.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.018+0000 7ffa5b400640 1 --2- 172.21.15.5:0/3017835381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f21e9eb30 0x557f21e9f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:50:49.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.018+0000 7ffa5aa00640 1 --2- 172.21.15.5:0/3017835381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f21e93b40 0x557f21e9ef50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c004460 tx=0x7ffa4c02a480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6938285c2721bcc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:49.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.018+0000 7ffa48c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:50:49.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.018+0000 7ffa48c00640 1 -- 172.21.15.5:0/3017835381 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c040040 con 0x557f21e93b40 2024-04-24T06:50:49.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.019+0000 7ffa64c11740 1 -- 172.21.15.5:0/3017835381 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557f21e00a60 con 0x557f21e93b40 2024-04-24T06:50:49.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.019+0000 7ffa48c00640 1 -- 172.21.15.5:0/3017835381 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7ffa4c050790 con 0x557f21e93b40 2024-04-24T06:50:49.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.019+0000 7ffa48c00640 1 -- 172.21.15.5:0/3017835381 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ffa4c035040 con 0x557f21e93b40 2024-04-24T06:50:49.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.019+0000 7ffa64c11740 10 client.26740 fetch_fsmap learned FSMap version 27 2024-04-24T06:50:49.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.019+0000 7ffa64c11740 1 -- 172.21.15.5:0/3017835381 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557f21ea7270 con 0x557f21e93b40 2024-04-24T06:50:49.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.019+0000 7ffa48c00640 1 -- 172.21.15.5:0/3017835381 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x557f21e00a60 con 0x557f21e93b40 2024-04-24T06:50:49.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.019+0000 7ffa48c00640 1 -- 172.21.15.5:0/3017835381 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7ffa4c031660 con 0x557f21e93b40 2024-04-24T06:50:49.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.019+0000 7ffa64c11740 10 client.26740 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:50:49.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.019+0000 7ffa64c11740 10 client.26740 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T06:50:49.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.019+0000 7ffa64c11740 20 client.26740 populate_metadata read hostname 'smithi005' 2024-04-24T06:50:49.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.019+0000 7ffa64c11740 1 --2- 172.21.15.5:0/3017835381 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x557f21f152f0 0x557f21f176e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:49.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.019+0000 7ffa64c11740 4 client.26740 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:50:49.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.019+0000 7ffa64c11740 1 -- 172.21.15.5:0/3017835381 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557f21d7c4d0 con 0x557f21f152f0 2024-04-24T06:50:49.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.020+0000 7ffa52000640 1 --2- 172.21.15.5:0/3017835381 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x557f21f152f0 0x557f21f176e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:49.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.020+0000 7ffa52000640 1 --2- 172.21.15.5:0/3017835381 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x557f21f152f0 0x557f21f176e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bd55d399a61ace9 server_cookie=8cc566213454c47a in_seq=0 out_seq=0 2024-04-24T06:50:49.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.020+0000 7ffa48c00640 10 client.26740 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:50:49.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.021+0000 7ffa48c00640 1 -- 172.21.15.5:0/3017835381 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557f21d7c4d0 con 0x557f21f152f0 2024-04-24T06:50:49.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.021+0000 7ffa48c00640 10 client.26740 handle_command_reply: tid=0 2024-04-24T06:50:49.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.021+0000 7ffa64c11740 1 client.26740 shutdown 2024-04-24T06:50:49.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.021+0000 7ffa64c11740 1 -- 172.21.15.5:0/3017835381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f21e93b40 msgr2=0x557f21e9ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:49.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.021+0000 7ffa64c11740 1 --2- 172.21.15.5:0/3017835381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f21e93b40 0x557f21e9ef50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c004460 tx=0x7ffa4c02a480 comp rx=0 tx=0).stop 2024-04-24T06:50:49.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.021+0000 7ffa64c11740 1 -- 172.21.15.5:0/3017835381 shutdown_connections 2024-04-24T06:50:49.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.021+0000 7ffa64c11740 1 --2- 172.21.15.5:0/3017835381 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x557f21f152f0 0x557f21f176e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.021+0000 7ffa64c11740 1 --2- 172.21.15.5:0/3017835381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f21e9eb30 0x557f21e9f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.021+0000 7ffa64c11740 1 --2- 172.21.15.5:0/3017835381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f21e94460 0x557f21e9f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.021+0000 7ffa64c11740 1 --2- 172.21.15.5:0/3017835381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f21e93b40 0x557f21e9ef50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.021+0000 7ffa48c00640 0 client.26740 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:50:49.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.021+0000 7ffa64c11740 1 -- 172.21.15.5:0/3017835381 >> 172.21.15.5:0/3017835381 conn(0x557f21e8fe60 msgr2=0x557f21e9aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:49.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.021+0000 7ffa64c11740 1 -- 172.21.15.5:0/3017835381 shutdown_connections 2024-04-24T06:50:49.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.021+0000 7ffa64c11740 1 -- 172.21.15.5:0/3017835381 wait complete. 2024-04-24T06:50:49.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.021+0000 7ffa64c11740 20 client.26740 trim_cache size 0 max 16384 2024-04-24T06:50:49.031 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:50:49.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.022+0000 7ffa5be00640 1 -- 172.21.15.5:0/595434667 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ffa400a3da0 msgr2=0x7ffa400a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:49.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.022+0000 7ffa5be00640 1 --2- 172.21.15.5:0/595434667 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ffa400a3da0 0x7ffa400a6260 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x557f21e90ff0 tx=0x7ffa44009770 comp rx=0 tx=0).stop 2024-04-24T06:50:49.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.022+0000 7ffa5be00640 1 -- 172.21.15.5:0/595434667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa5c1061d0 msgr2=0x7ffa5c19eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:49.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.022+0000 7ffa5be00640 1 --2- 172.21.15.5:0/595434667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa5c1061d0 0x7ffa5c19eb70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ffa5400e940 tx=0x7ffa5400ee00 comp rx=0 tx=0).stop 2024-04-24T06:50:49.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.022+0000 7ffa5be00640 1 -- 172.21.15.5:0/595434667 shutdown_connections 2024-04-24T06:50:49.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.022+0000 7ffa5be00640 1 --2- 172.21.15.5:0/595434667 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ffa400a3da0 0x7ffa400a6260 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.022+0000 7ffa5be00640 1 --2- 172.21.15.5:0/595434667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa5c1061d0 0x7ffa5c19eb70 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.022+0000 7ffa5be00640 1 --2- 172.21.15.5:0/595434667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa5c1016f0 0x7ffa5c1a4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.022+0000 7ffa5be00640 1 --2- 172.21.15.5:0/595434667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa5c100dd0 0x7ffa5c1a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.022+0000 7ffa5be00640 1 -- 172.21.15.5:0/595434667 >> 172.21.15.5:0/595434667 conn(0x7ffa5c077ab0 msgr2=0x7ffa5c103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:49.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.022+0000 7ffa5be00640 1 -- 172.21.15.5:0/595434667 shutdown_connections 2024-04-24T06:50:49.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.022+0000 7ffa5be00640 1 -- 172.21.15.5:0/595434667 wait complete. 2024-04-24T06:50:49.041 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T06:50:49.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.235+0000 7f2ec1a00640 1 -- 172.21.15.5:0/666141550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ebc109330 msgr2=0x7f2ebc110f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:49.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.235+0000 7f2ec1a00640 1 --2- 172.21.15.5:0/666141550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ebc109330 0x7f2ebc110f00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4009ca0 tx=0x7f2eb402a8c0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.235+0000 7f2ec1a00640 1 -- 172.21.15.5:0/666141550 shutdown_connections 2024-04-24T06:50:49.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.235+0000 7f2ec1a00640 1 --2- 172.21.15.5:0/666141550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ebc109330 0x7f2ebc110f00 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.235+0000 7f2ec1a00640 1 --2- 172.21.15.5:0/666141550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ebc104c10 0x7f2ebc108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.235+0000 7f2ec1a00640 1 --2- 172.21.15.5:0/666141550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ebc104260 0x7f2ebc104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.235+0000 7f2ec1a00640 1 -- 172.21.15.5:0/666141550 >> 172.21.15.5:0/666141550 conn(0x7f2ebc0ffaa0 msgr2=0x7f2ebc101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:49.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.236+0000 7f2ec1a00640 1 -- 172.21.15.5:0/666141550 shutdown_connections 2024-04-24T06:50:49.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.236+0000 7f2ec1a00640 1 -- 172.21.15.5:0/666141550 wait complete. 2024-04-24T06:50:49.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.237+0000 7f2ec1a00640 1 Processor -- start 2024-04-24T06:50:49.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.237+0000 7f2ec1a00640 1 -- start start 2024-04-24T06:50:49.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.238+0000 7f2ec1a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ebc104260 0x7f2ebc1a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:49.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.238+0000 7f2ec1a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ebc109330 0x7f2ebc1a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:49.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.238+0000 7f2ec1a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ebc19a760 0x7f2ebc19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:49.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.238+0000 7f2ec1a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2ebc114420 con 0x7f2ebc104260 2024-04-24T06:50:49.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.238+0000 7f2ec1a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2ebc1142a0 con 0x7f2ebc19a760 2024-04-24T06:50:49.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.238+0000 7f2ec1a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2ebc1145a0 con 0x7f2ebc109330 2024-04-24T06:50:49.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.238+0000 7f2ebbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ebc104260 0x7f2ebc1a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:49.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.238+0000 7f2ebb400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ebc109330 0x7f2ebc1a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:49.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.238+0000 7f2ebbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ebc104260 0x7f2ebc1a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53620/0 (socket says 172.21.15.5:53620) 2024-04-24T06:50:49.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.238+0000 7f2ebbe00640 1 -- 172.21.15.5:0/2156155604 learned_addr learned my addr 172.21.15.5:0/2156155604 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:49.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.238+0000 7f2ec1000640 1 --2- 172.21.15.5:0/2156155604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ebc19a760 0x7f2ebc19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:49.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.239+0000 7f2ebbe00640 1 -- 172.21.15.5:0/2156155604 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ebc109330 msgr2=0x7f2ebc1a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:49.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.239+0000 7f2ebbe00640 1 --2- 172.21.15.5:0/2156155604 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ebc109330 0x7f2ebc1a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.239+0000 7f2ebbe00640 1 -- 172.21.15.5:0/2156155604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ebc19a760 msgr2=0x7f2ebc19abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:49.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.239+0000 7f2ebbe00640 1 --2- 172.21.15.5:0/2156155604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ebc19a760 0x7f2ebc19abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.239+0000 7f2ebbe00640 1 -- 172.21.15.5:0/2156155604 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ebc19b360 con 0x7f2ebc104260 2024-04-24T06:50:49.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.239+0000 7f2ebbe00640 1 --2- 172.21.15.5:0/2156155604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ebc104260 0x7f2ebc1a06a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2eac00e9f0 tx=0x7f2eac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:49.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.239+0000 7f2eb8c00640 1 -- 172.21.15.5:0/2156155604 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eac00cd80 con 0x7f2ebc104260 2024-04-24T06:50:49.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.239+0000 7f2eb8c00640 1 -- 172.21.15.5:0/2156155604 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2eac010910 con 0x7f2ebc104260 2024-04-24T06:50:49.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.239+0000 7f2eb8c00640 1 -- 172.21.15.5:0/2156155604 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eac018990 con 0x7f2ebc104260 2024-04-24T06:50:49.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.239+0000 7f2ec1a00640 1 -- 172.21.15.5:0/2156155604 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ebc1a7440 con 0x7f2ebc104260 2024-04-24T06:50:49.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.240+0000 7f2ec1a00640 1 -- 172.21.15.5:0/2156155604 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ebc1a7840 con 0x7f2ebc104260 2024-04-24T06:50:49.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.243+0000 7f2eb8c00640 1 -- 172.21.15.5:0/2156155604 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f2eac010ac0 con 0x7f2ebc104260 2024-04-24T06:50:49.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.243+0000 7f2ec5fbc740 1 Processor -- start 2024-04-24T06:50:49.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.243+0000 7f2ec5fbc740 1 -- start start 2024-04-24T06:50:49.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.244+0000 7f2ec5fbc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df8603fb40 0x55df8603ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:49.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.244+0000 7f2ec5fbc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df86040460 0x55df8604a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:49.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.244+0000 7f2ec5fbc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df8604ab30 0x55df8604cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:49.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.244+0000 7f2ec5fbc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55df85fa51d0 con 0x55df86040460 2024-04-24T06:50:49.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.244+0000 7f2ec5fbc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55df85f4b450 con 0x55df8603fb40 2024-04-24T06:50:49.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.244+0000 7f2ec5fbc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55df85fa1190 con 0x55df8604ab30 2024-04-24T06:50:49.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.244+0000 7f2ebbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df86040460 0x55df8604a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:49.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.244+0000 7f2ec1000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df8603fb40 0x55df8603ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:49.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.244+0000 7f2ebbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df86040460 0x55df8604a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53630/0 (socket says 172.21.15.5:53630) 2024-04-24T06:50:49.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.244+0000 7f2ec1000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df8603fb40 0x55df8603ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:51816/0 (socket says 172.21.15.5:51816) 2024-04-24T06:50:49.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.244+0000 7f2ebbe00640 1 -- 172.21.15.5:0/3644513455 learned_addr learned my addr 172.21.15.5:0/3644513455 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:49.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.244+0000 7f2ebb400640 1 --2- 172.21.15.5:0/3644513455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df8604ab30 0x55df8604cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:49.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.245+0000 7f2ec1000640 1 -- 172.21.15.5:0/3644513455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df8604ab30 msgr2=0x55df8604cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:49.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.245+0000 7f2ec1000640 1 --2- 172.21.15.5:0/3644513455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df8604ab30 0x55df8604cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.245+0000 7f2ec1000640 1 -- 172.21.15.5:0/3644513455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df86040460 msgr2=0x55df8604a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:49.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.245+0000 7f2ec1000640 1 --2- 172.21.15.5:0/3644513455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df86040460 0x55df8604a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.245+0000 7f2ec1000640 1 -- 172.21.15.5:0/3644513455 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df86008310 con 0x55df8603fb40 2024-04-24T06:50:49.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.245+0000 7f2eb8c00640 1 --2- 172.21.15.5:0/2156155604 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f2ea00a3d20 0x7f2ea00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:49.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.245+0000 7f2ec1000640 1 --2- 172.21.15.5:0/3644513455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df8603fb40 0x55df8603ff20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4005e30 tx=0x7f2eb40093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44f968e010c43e19 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:49.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.245+0000 7f2eb8c00640 1 -- 172.21.15.5:0/2156155604 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f2eac0d0b70 con 0x7f2ebc104260 2024-04-24T06:50:49.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.245+0000 7f2ebbe00640 1 --2- 172.21.15.5:0/2156155604 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f2ea00a3d20 0x7f2ea00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:49.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.245+0000 7f2eb1600640 1 -- 172.21.15.5:0/3644513455 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4031db0 con 0x55df8603fb40 2024-04-24T06:50:49.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.245+0000 7f2eb1600640 1 -- 172.21.15.5:0/3644513455 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2eb4036030 con 0x55df8603fb40 2024-04-24T06:50:49.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.245+0000 7f2eb1600640 1 -- 172.21.15.5:0/3644513455 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb40042a0 con 0x55df8603fb40 2024-04-24T06:50:49.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.246+0000 7f2ec5fbc740 1 -- 172.21.15.5:0/3644513455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df8603fb40 msgr2=0x55df8603ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:49.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.246+0000 7f2ec5fbc740 1 --2- 172.21.15.5:0/3644513455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df8603fb40 0x55df8603ff20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4005e30 tx=0x7f2eb40093e0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.246+0000 7f2ebbe00640 1 --2- 172.21.15.5:0/2156155604 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f2ea00a3d20 0x7f2ea00a61e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2eac09cb00 tx=0x7f2eac0d4620 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:49.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.246+0000 7f2ec5fbc740 1 -- 172.21.15.5:0/3644513455 shutdown_connections 2024-04-24T06:50:49.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.246+0000 7f2ec5fbc740 1 --2- 172.21.15.5:0/3644513455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df8604ab30 0x55df8604cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.246+0000 7f2ec5fbc740 1 --2- 172.21.15.5:0/3644513455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df86040460 0x55df8604a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.246+0000 7f2ec5fbc740 1 --2- 172.21.15.5:0/3644513455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df8603fb40 0x55df8603ff20 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.246+0000 7f2ec5fbc740 1 -- 172.21.15.5:0/3644513455 >> 172.21.15.5:0/3644513455 conn(0x55df8603be60 msgr2=0x55df8603c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:49.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.246+0000 7f2ec5fbc740 1 -- 172.21.15.5:0/3644513455 shutdown_connections 2024-04-24T06:50:49.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.247+0000 7f2ec5fbc740 1 -- 172.21.15.5:0/3644513455 wait complete. 2024-04-24T06:50:49.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.247+0000 7f2ec5fbc740 1 -- start start 2024-04-24T06:50:49.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.247+0000 7f2ec5fbc740 1 Processor -- start 2024-04-24T06:50:49.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.248+0000 7f2ec5fbc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df8603fb40 0x55df8604a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:49.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.248+0000 7f2ec5fbc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df86040460 0x55df86047840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:49.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.248+0000 7f2ec5fbc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df8604ab30 0x55df86047e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:49.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.248+0000 7f2ec5fbc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55df8604d7e0 con 0x55df86040460 2024-04-24T06:50:49.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.248+0000 7f2ec5fbc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55df85f5e810 con 0x55df8604ab30 2024-04-24T06:50:49.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.248+0000 7f2ec5fbc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55df8604d960 con 0x55df8603fb40 2024-04-24T06:50:49.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.248+0000 7f2ec5fbc740 10 client.0 _ll_register_callbacks cb 0x55df85f2c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:50:49.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.248+0000 7f2ec1000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df8603fb40 0x55df8604a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:49.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.248+0000 7f2ebb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df86040460 0x55df86047840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:49.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.248+0000 7f2ec1000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df8603fb40 0x55df8604a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:57274/0 (socket says 172.21.15.5:57274) 2024-04-24T06:50:49.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.248+0000 7f2ec1000640 1 -- 172.21.15.5:0/4155867665 learned_addr learned my addr 172.21.15.5:0/4155867665 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:49.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.248+0000 7f2ec1000640 1 --2- 172.21.15.5:0/4155867665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df8604ab30 0x55df86047e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:49.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.249+0000 7f2ebb400640 1 -- 172.21.15.5:0/4155867665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df8603fb40 msgr2=0x55df8604a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:49.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.249+0000 7f2ebb400640 1 --2- 172.21.15.5:0/4155867665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df8603fb40 0x55df8604a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.249+0000 7f2ebb400640 1 -- 172.21.15.5:0/4155867665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df8604ab30 msgr2=0x55df86047e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:49.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.249+0000 7f2ebb400640 1 --2- 172.21.15.5:0/4155867665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df8604ab30 0x55df86047e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.249+0000 7f2ebb400640 1 -- 172.21.15.5:0/4155867665 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df85f6f3f0 con 0x55df86040460 2024-04-24T06:50:49.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.249+0000 7f2ec1000640 1 --2- 172.21.15.5:0/4155867665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df8603fb40 0x55df8604a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:50:49.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.249+0000 7f2ebb400640 1 --2- 172.21.15.5:0/4155867665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df86040460 0x55df86047840 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc19bfb0 tx=0x7f2ea8012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8993b7f7dbd96e5a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:49.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.249+0000 7f2ea6000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:50:49.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.249+0000 7f2ea6000640 1 -- 172.21.15.5:0/4155867665 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea800f8f0 con 0x55df86040460 2024-04-24T06:50:49.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.249+0000 7f2ea6000640 1 -- 172.21.15.5:0/4155867665 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f2ea800fa90 con 0x55df86040460 2024-04-24T06:50:49.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.249+0000 7f2ec5fbc740 1 -- 172.21.15.5:0/4155867665 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2eb4036030 con 0x55df86040460 2024-04-24T06:50:49.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.249+0000 7f2ea6000640 1 -- 172.21.15.5:0/4155867665 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2ea800fc90 con 0x55df86040460 2024-04-24T06:50:49.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.249+0000 7f2ec5fbc740 10 client.16605 fetch_fsmap learned FSMap version 27 2024-04-24T06:50:49.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.250+0000 7f2ec5fbc740 1 -- 172.21.15.5:0/4155867665 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55df85cb73f0 con 0x55df86040460 2024-04-24T06:50:49.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.250+0000 7f2ea6000640 1 -- 172.21.15.5:0/4155867665 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2eb4036030 con 0x55df86040460 2024-04-24T06:50:49.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.250+0000 7f2ea6000640 1 -- 172.21.15.5:0/4155867665 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f2ea8010430 con 0x55df86040460 2024-04-24T06:50:49.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.250+0000 7f2ec5fbc740 10 client.16605 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:50:49.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.250+0000 7f2ec5fbc740 10 client.16605 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:50:49.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.250+0000 7f2ec5fbc740 20 client.16605 populate_metadata read hostname 'smithi005' 2024-04-24T06:50:49.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.250+0000 7f2ec5fbc740 1 --2- 172.21.15.5:0/4155867665 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55df8605ac30 0x55df8605d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:49.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.250+0000 7f2ec5fbc740 4 client.16605 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:50:49.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.250+0000 7f2ec5fbc740 1 -- 172.21.15.5:0/4155867665 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df85fb2bf0 con 0x55df8605ac30 2024-04-24T06:50:49.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.251+0000 7f2ec1000640 1 --2- 172.21.15.5:0/4155867665 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55df8605ac30 0x55df8605d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:49.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.251+0000 7f2ec1000640 1 --2- 172.21.15.5:0/4155867665 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55df8605ac30 0x55df8605d050 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80e88e8ec42e7076 server_cookie=a70b05c6ed27390f in_seq=0 out_seq=0 2024-04-24T06:50:49.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.251+0000 7f2ea6000640 10 client.16605 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:50:49.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.252+0000 7f2ea6000640 1 -- 172.21.15.5:0/4155867665 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55df85fb2bf0 con 0x55df8605ac30 2024-04-24T06:50:49.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.252+0000 7f2ea6000640 10 client.16605 handle_command_reply: tid=0 2024-04-24T06:50:49.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.252+0000 7f2ec5fbc740 1 client.16605 shutdown 2024-04-24T06:50:49.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.252+0000 7f2ec5fbc740 1 -- 172.21.15.5:0/4155867665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df86040460 msgr2=0x55df86047840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:49.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.253+0000 7f2ec5fbc740 1 --2- 172.21.15.5:0/4155867665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df86040460 0x55df86047840 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc19bfb0 tx=0x7f2ea8012b60 comp rx=0 tx=0).stop 2024-04-24T06:50:49.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.253+0000 7f2ec5fbc740 1 -- 172.21.15.5:0/4155867665 shutdown_connections 2024-04-24T06:50:49.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.253+0000 7f2ec5fbc740 1 --2- 172.21.15.5:0/4155867665 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55df8605ac30 0x55df8605d050 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.253+0000 7f2ec5fbc740 1 --2- 172.21.15.5:0/4155867665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df8604ab30 0x55df86047e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.253+0000 7f2ec5fbc740 1 --2- 172.21.15.5:0/4155867665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df86040460 0x55df86047840 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.253+0000 7f2ec5fbc740 1 --2- 172.21.15.5:0/4155867665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df8603fb40 0x55df8604a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.253+0000 7f2ea6000640 0 client.16605 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:50:49.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.253+0000 7f2ec5fbc740 1 -- 172.21.15.5:0/4155867665 >> 172.21.15.5:0/4155867665 conn(0x55df8603be60 msgr2=0x55df86041b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:49.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.253+0000 7f2ec5fbc740 1 -- 172.21.15.5:0/4155867665 shutdown_connections 2024-04-24T06:50:49.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.253+0000 7f2ec5fbc740 1 -- 172.21.15.5:0/4155867665 wait complete. 2024-04-24T06:50:49.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.253+0000 7f2ec5fbc740 20 client.16605 trim_cache size 0 max 16384 2024-04-24T06:50:49.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.268+0000 7f2ec5fbc740 1 Processor -- start 2024-04-24T06:50:49.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.268+0000 7f2ec5fbc740 1 -- start start 2024-04-24T06:50:49.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.268+0000 7f2ec5fbc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df8603fb40 0x55df860566d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:49.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.268+0000 7f2ec5fbc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df86040460 0x55df860477e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:49.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.268+0000 7f2ec5fbc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df8604ab30 0x55df8604a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:49.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.268+0000 7f2ec5fbc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2ebc1b2610 con 0x55df8604ab30 2024-04-24T06:50:49.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.268+0000 7f2ec5fbc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2ebc1afee0 con 0x55df8603fb40 2024-04-24T06:50:49.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.268+0000 7f2ec5fbc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2ebc1add30 con 0x55df86040460 2024-04-24T06:50:49.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.268+0000 7f2ec1000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df8603fb40 0x55df860566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:49.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.268+0000 7f2ec1000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df8604ab30 0x55df8604a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:49.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.268+0000 7f2ec1000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df8604ab30 0x55df8604a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53650/0 (socket says 172.21.15.5:53650) 2024-04-24T06:50:49.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.268+0000 7f2ec1000640 1 -- 172.21.15.5:0/3603833143 learned_addr learned my addr 172.21.15.5:0/3603833143 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:49.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.269+0000 7f2ebb400640 1 --2- 172.21.15.5:0/3603833143 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df86040460 0x55df860477e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:49.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.269+0000 7f2ec1000640 1 -- 172.21.15.5:0/3603833143 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df86040460 msgr2=0x55df860477e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:49.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.269+0000 7f2ec1000640 1 --2- 172.21.15.5:0/3603833143 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df86040460 0x55df860477e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.269+0000 7f2ec1000640 1 -- 172.21.15.5:0/3603833143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df8603fb40 msgr2=0x55df860566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:49.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.269+0000 7f2ec1000640 1 --2- 172.21.15.5:0/3603833143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df8603fb40 0x55df860566d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.269+0000 7f2ec1000640 1 -- 172.21.15.5:0/3603833143 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df85d817d0 con 0x55df8604ab30 2024-04-24T06:50:49.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.269+0000 7f2ebb400640 1 --2- 172.21.15.5:0/3603833143 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df86040460 0x55df860477e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:50:49.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.269+0000 7f2ec1000640 1 --2- 172.21.15.5:0/3603833143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df8604ab30 0x55df8604a080 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4000f80 tx=0x7f2eb40319c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b71ee77dc0b073 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:49.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.271+0000 7f2eb0c00640 1 -- 172.21.15.5:0/3603833143 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4003970 con 0x55df8604ab30 2024-04-24T06:50:49.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.272+0000 7f2eb0c00640 1 -- 172.21.15.5:0/3603833143 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2eb4036040 con 0x55df8604ab30 2024-04-24T06:50:49.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.272+0000 7f2eb0c00640 1 -- 172.21.15.5:0/3603833143 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb40047d0 con 0x55df8604ab30 2024-04-24T06:50:49.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.272+0000 7f2ec5fbc740 1 -- 172.21.15.5:0/3603833143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df8604ab30 msgr2=0x55df8604a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:49.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.272+0000 7f2ec5fbc740 1 --2- 172.21.15.5:0/3603833143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df8604ab30 0x55df8604a080 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4000f80 tx=0x7f2eb40319c0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.272+0000 7f2ec5fbc740 1 -- 172.21.15.5:0/3603833143 shutdown_connections 2024-04-24T06:50:49.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.272+0000 7f2ec5fbc740 1 --2- 172.21.15.5:0/3603833143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df8604ab30 0x55df8604a080 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.272+0000 7f2ec5fbc740 1 --2- 172.21.15.5:0/3603833143 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df86040460 0x55df860477e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.272+0000 7f2ec5fbc740 1 --2- 172.21.15.5:0/3603833143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df8603fb40 0x55df860566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.272+0000 7f2ec5fbc740 1 -- 172.21.15.5:0/3603833143 >> 172.21.15.5:0/3603833143 conn(0x55df8603be60 msgr2=0x55df860470d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:49.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.273+0000 7f2ec5fbc740 1 -- 172.21.15.5:0/3603833143 shutdown_connections 2024-04-24T06:50:49.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.273+0000 7f2ec5fbc740 1 -- 172.21.15.5:0/3603833143 wait complete. 2024-04-24T06:50:49.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.273+0000 7f2ec5fbc740 1 -- start start 2024-04-24T06:50:49.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.273+0000 7f2ec5fbc740 1 Processor -- start 2024-04-24T06:50:49.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.274+0000 7f2ec5fbc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df8603fb40 0x55df8604ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:49.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.274+0000 7f2ec5fbc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df86040460 0x55df8604bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:49.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.274+0000 7f2ec5fbc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df8604ab30 0x55df8604c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:49.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.274+0000 7f2ec5fbc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2ebc1ae3a0 con 0x55df8604ab30 2024-04-24T06:50:49.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.274+0000 7f2ec5fbc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2ebc1af810 con 0x55df8603fb40 2024-04-24T06:50:49.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.274+0000 7f2ec5fbc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2ebc1aea70 con 0x55df86040460 2024-04-24T06:50:49.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.274+0000 7f2ec5fbc740 10 client.0 _ll_register_callbacks cb 0x7f2ea8009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:50:49.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.274+0000 7f2ec1000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df8603fb40 0x55df8604ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:49.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.274+0000 7f2ebb400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df86040460 0x55df8604bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:49.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.274+0000 7f2ebb400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df86040460 0x55df8604bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:57278/0 (socket says 172.21.15.5:57278) 2024-04-24T06:50:49.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.274+0000 7f2ebb400640 1 -- 172.21.15.5:0/281820648 learned_addr learned my addr 172.21.15.5:0/281820648 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.274+0000 7f2ec1000640 1 --2- 172.21.15.5:0/281820648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df8604ab30 0x55df8604c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.274+0000 7f2ebb400640 1 -- 172.21.15.5:0/281820648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df8603fb40 msgr2=0x55df8604ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.275+0000 7f2ebb400640 1 --2- 172.21.15.5:0/281820648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df8603fb40 0x55df8604ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.275+0000 7f2ebb400640 1 -- 172.21.15.5:0/281820648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df8604ab30 msgr2=0x55df8604c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.275+0000 7f2ebb400640 1 --2- 172.21.15.5:0/281820648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df8604ab30 0x55df8604c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.275+0000 7f2ebb400640 1 -- 172.21.15.5:0/281820648 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df86051020 con 0x55df86040460 2024-04-24T06:50:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.275+0000 7f2ec1000640 1 --2- 172.21.15.5:0/281820648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df8603fb40 0x55df8604ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:50:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.275+0000 7f2ebb400640 1 --2- 172.21.15.5:0/281820648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df86040460 0x55df8604bf50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8001eb0 tx=0x7f2ea8012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=361895ccb7c3ef0c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:49.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.275+0000 7f2ea6000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:50:49.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.275+0000 7f2ea6000640 1 -- 172.21.15.5:0/281820648 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea8029070 con 0x55df86040460 2024-04-24T06:50:49.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.275+0000 7f2ea6000640 1 -- 172.21.15.5:0/281820648 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f2ea8010430 con 0x55df86040460 2024-04-24T06:50:49.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.275+0000 7f2ec5fbc740 1 -- 172.21.15.5:0/281820648 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2eb4036040 con 0x55df86040460 2024-04-24T06:50:49.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.276+0000 7f2ea6000640 1 -- 172.21.15.5:0/281820648 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2ea8034050 con 0x55df86040460 2024-04-24T06:50:49.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.276+0000 7f2ec5fbc740 10 client.26755 fetch_fsmap learned FSMap version 27 2024-04-24T06:50:49.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.276+0000 7f2ec5fbc740 1 -- 172.21.15.5:0/281820648 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55df860c1560 con 0x55df86040460 2024-04-24T06:50:49.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.276+0000 7f2ea6000640 1 -- 172.21.15.5:0/281820648 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2eb4036040 con 0x55df86040460 2024-04-24T06:50:49.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.276+0000 7f2ea6000640 1 -- 172.21.15.5:0/281820648 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f2ea80035f0 con 0x55df86040460 2024-04-24T06:50:49.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.276+0000 7f2ec5fbc740 10 client.26755 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:50:49.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.276+0000 7f2ec5fbc740 10 client.26755 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:50:49.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.276+0000 7f2ec5fbc740 20 client.26755 populate_metadata read hostname 'smithi005' 2024-04-24T06:50:49.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.276+0000 7f2ec5fbc740 1 --2- 172.21.15.5:0/281820648 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55df860c1d10 0x55df860c4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:49.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.276+0000 7f2ec5fbc740 4 client.26755 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:50:49.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.276+0000 7f2ec5fbc740 1 -- 172.21.15.5:0/281820648 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55df86043440 con 0x55df860c1d10 2024-04-24T06:50:49.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.277+0000 7f2ec1000640 1 --2- 172.21.15.5:0/281820648 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55df860c1d10 0x55df860c4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:49.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.277+0000 7f2ec1000640 1 --2- 172.21.15.5:0/281820648 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55df860c1d10 0x55df860c4130 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1779ec7e2d6aefb server_cookie=829144bc142c63d1 in_seq=0 out_seq=0 2024-04-24T06:50:49.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.277+0000 7f2ea6000640 10 client.26755 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:50:49.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.278+0000 7f2ea6000640 1 -- 172.21.15.5:0/281820648 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55df86043440 con 0x55df860c1d10 2024-04-24T06:50:49.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.278+0000 7f2ea6000640 10 client.26755 handle_command_reply: tid=0 2024-04-24T06:50:49.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.278+0000 7f2ec5fbc740 1 client.26755 shutdown 2024-04-24T06:50:49.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.278+0000 7f2ec5fbc740 1 -- 172.21.15.5:0/281820648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df86040460 msgr2=0x55df8604bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:49.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.278+0000 7f2ec5fbc740 1 --2- 172.21.15.5:0/281820648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df86040460 0x55df8604bf50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8001eb0 tx=0x7f2ea8012000 comp rx=0 tx=0).stop 2024-04-24T06:50:49.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.278+0000 7f2ec5fbc740 1 -- 172.21.15.5:0/281820648 shutdown_connections 2024-04-24T06:50:49.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.278+0000 7f2ec5fbc740 1 --2- 172.21.15.5:0/281820648 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55df860c1d10 0x55df860c4130 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.278+0000 7f2ec5fbc740 1 --2- 172.21.15.5:0/281820648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55df8604ab30 0x55df8604c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.278+0000 7f2ec5fbc740 1 --2- 172.21.15.5:0/281820648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df86040460 0x55df8604bf50 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.278+0000 7f2ec5fbc740 1 --2- 172.21.15.5:0/281820648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df8603fb40 0x55df8604ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.278+0000 7f2ec5fbc740 1 -- 172.21.15.5:0/281820648 >> 172.21.15.5:0/281820648 conn(0x55df8603be60 msgr2=0x55df8603a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:49.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.278+0000 7f2ea6000640 0 client.26755 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:50:49.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.278+0000 7f2ec5fbc740 1 -- 172.21.15.5:0/281820648 shutdown_connections 2024-04-24T06:50:49.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.278+0000 7f2ec5fbc740 1 -- 172.21.15.5:0/281820648 wait complete. 2024-04-24T06:50:49.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.278+0000 7f2ec5fbc740 20 client.26755 trim_cache size 0 max 16384 2024-04-24T06:50:49.297 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:50:49.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.279+0000 7f2ec1a00640 1 -- 172.21.15.5:0/2156155604 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f2ea00a3d20 msgr2=0x7f2ea00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:49.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.279+0000 7f2ec1a00640 1 --2- 172.21.15.5:0/2156155604 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f2ea00a3d20 0x7f2ea00a61e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2eac09cb00 tx=0x7f2eac0d4620 comp rx=0 tx=0).stop 2024-04-24T06:50:49.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.279+0000 7f2ec1a00640 1 -- 172.21.15.5:0/2156155604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ebc104260 msgr2=0x7f2ebc1a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:49.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.279+0000 7f2ec1a00640 1 --2- 172.21.15.5:0/2156155604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ebc104260 0x7f2ebc1a06a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2eac00e9f0 tx=0x7f2eac00eeb0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.279+0000 7f2ec1a00640 1 -- 172.21.15.5:0/2156155604 shutdown_connections 2024-04-24T06:50:49.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.279+0000 7f2ec1a00640 1 --2- 172.21.15.5:0/2156155604 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f2ea00a3d20 0x7f2ea00a61e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.279+0000 7f2ec1a00640 1 --2- 172.21.15.5:0/2156155604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ebc19a760 0x7f2ebc19abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.279+0000 7f2ec1a00640 1 --2- 172.21.15.5:0/2156155604 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ebc109330 0x7f2ebc1a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.279+0000 7f2ec1a00640 1 --2- 172.21.15.5:0/2156155604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ebc104260 0x7f2ebc1a06a0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:49.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.279+0000 7f2ec1a00640 1 -- 172.21.15.5:0/2156155604 >> 172.21.15.5:0/2156155604 conn(0x7f2ebc0ffaa0 msgr2=0x7f2ebc100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:49.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.279+0000 7f2ec1a00640 1 -- 172.21.15.5:0/2156155604 shutdown_connections 2024-04-24T06:50:49.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:49.279+0000 7f2ec1a00640 1 -- 172.21.15.5:0/2156155604 wait complete. 2024-04-24T06:50:49.300 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:49 smithi005 ceph-mon[27798]: pgmap v206: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T06:50:49.300 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:49 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/3836747043' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:50:49.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:49 smithi037 ceph-mon[32864]: pgmap v206: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T06:50:49.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:49 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/3836747043' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:50:49.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:49 smithi137 ceph-mon[33017]: pgmap v206: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-24T06:50:49.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:49 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/3836747043' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:50:50.302 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T06:50:50.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:50:50.302 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T06:50:50.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.524+0000 7f3649c00640 1 -- 172.21.15.5:0/3180147566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3644104df0 msgr2=0x7f3644108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:50.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.524+0000 7f3649c00640 1 --2- 172.21.15.5:0/3180147566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3644104df0 0x7f3644108e30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3630009900 tx=0x7f363002a410 comp rx=0 tx=0).stop 2024-04-24T06:50:50.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.524+0000 7f3649c00640 1 -- 172.21.15.5:0/3180147566 shutdown_connections 2024-04-24T06:50:50.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.524+0000 7f3649c00640 1 --2- 172.21.15.5:0/3180147566 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3644109510 0x7f36441110e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.524+0000 7f3649c00640 1 --2- 172.21.15.5:0/3180147566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3644104df0 0x7f3644108e30 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.524+0000 7f3649c00640 1 --2- 172.21.15.5:0/3180147566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3644104440 0x7f3644104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.524+0000 7f3649c00640 1 -- 172.21.15.5:0/3180147566 >> 172.21.15.5:0/3180147566 conn(0x7f36440ffc80 msgr2=0x7f36441020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:50.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.524+0000 7f3649c00640 1 -- 172.21.15.5:0/3180147566 shutdown_connections 2024-04-24T06:50:50.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.524+0000 7f3649c00640 1 -- 172.21.15.5:0/3180147566 wait complete. 2024-04-24T06:50:50.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.525+0000 7f3649c00640 1 Processor -- start 2024-04-24T06:50:50.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.525+0000 7f3649c00640 1 -- start start 2024-04-24T06:50:50.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.526+0000 7f3649c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3644104440 0x7f36441a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:50.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.526+0000 7f3649c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3644109510 0x7f36441a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:50.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.526+0000 7f3649c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f364419a8d0 0x7f364419ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:50.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.526+0000 7f3649c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f36441145d0 con 0x7f364419a8d0 2024-04-24T06:50:50.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.526+0000 7f3649c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3644114450 con 0x7f3644104440 2024-04-24T06:50:50.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.526+0000 7f3649c00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3644114750 con 0x7f3644109510 2024-04-24T06:50:50.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.526+0000 7f3642a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3644104440 0x7f36441a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:50.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.526+0000 7f3643400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f364419a8d0 0x7f364419ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:50.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.526+0000 7f3642a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3644104440 0x7f36441a0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:51872/0 (socket says 172.21.15.5:51872) 2024-04-24T06:50:50.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.526+0000 7f3642a00640 1 -- 172.21.15.5:0/1506564499 learned_addr learned my addr 172.21.15.5:0/1506564499 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:50.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.526+0000 7f3642000640 1 --2- 172.21.15.5:0/1506564499 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3644109510 0x7f36441a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:50.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.527+0000 7f3642a00640 1 -- 172.21.15.5:0/1506564499 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3644109510 msgr2=0x7f36441a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:50.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.527+0000 7f3642a00640 1 --2- 172.21.15.5:0/1506564499 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3644109510 0x7f36441a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.527+0000 7f3642a00640 1 -- 172.21.15.5:0/1506564499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f364419a8d0 msgr2=0x7f364419ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.527+0000 7f3642a00640 1 --2- 172.21.15.5:0/1506564499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f364419a8d0 0x7f364419ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.527+0000 7f3642a00640 1 -- 172.21.15.5:0/1506564499 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f364419b3f0 con 0x7f3644104440 2024-04-24T06:50:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.527+0000 7f3643400640 1 --2- 172.21.15.5:0/1506564499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f364419a8d0 0x7f364419ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:50:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.527+0000 7f3642a00640 1 --2- 172.21.15.5:0/1506564499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3644104440 0x7f36441a0720 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f363400ea60 tx=0x7f363400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.527+0000 7f363b400640 1 -- 172.21.15.5:0/1506564499 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363400cde0 con 0x7f3644104440 2024-04-24T06:50:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.527+0000 7f3649c00640 1 -- 172.21.15.5:0/1506564499 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f364419b6e0 con 0x7f3644104440 2024-04-24T06:50:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.527+0000 7f363b400640 1 -- 172.21.15.5:0/1506564499 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36340035f0 con 0x7f3644104440 2024-04-24T06:50:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.527+0000 7f363b400640 1 -- 172.21.15.5:0/1506564499 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3634010670 con 0x7f3644104440 2024-04-24T06:50:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.528+0000 7f3649c00640 1 -- 172.21.15.5:0/1506564499 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36441a7980 con 0x7f3644104440 2024-04-24T06:50:50.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.531+0000 7f364cb81740 1 Processor -- start 2024-04-24T06:50:50.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.531+0000 7f363b400640 1 -- 172.21.15.5:0/1506564499 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f36340037a0 con 0x7f3644104440 2024-04-24T06:50:50.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.531+0000 7f364cb81740 1 -- start start 2024-04-24T06:50:50.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.531+0000 7f364cb81740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b4e6f090 0x5575b4e6f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:50.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.531+0000 7f364cb81740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575b4e6f9b0 0x5575b4e79b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:50.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.531+0000 7f364cb81740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575b4e7a080 0x5575b4e7c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:50.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.531+0000 7f364cb81740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5575b4dd4e70 con 0x5575b4e7a080 2024-04-24T06:50:50.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.531+0000 7f364cb81740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5575b4d7b380 con 0x5575b4e6f9b0 2024-04-24T06:50:50.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.531+0000 7f364cb81740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5575b4dd0e30 con 0x5575b4e6f090 2024-04-24T06:50:50.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.532+0000 7f3643400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b4e6f090 0x5575b4e6f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:50.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.532+0000 7f3642a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575b4e6f9b0 0x5575b4e79b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:50.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.532+0000 7f3643400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b4e6f090 0x5575b4e6f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:57308/0 (socket says 172.21.15.5:57308) 2024-04-24T06:50:50.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.532+0000 7f3643400640 1 -- 172.21.15.5:0/825305558 learned_addr learned my addr 172.21.15.5:0/825305558 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:50.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.532+0000 7f3642000640 1 --2- 172.21.15.5:0/825305558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575b4e7a080 0x5575b4e7c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:50.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.532+0000 7f3643400640 1 -- 172.21.15.5:0/825305558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575b4e6f9b0 msgr2=0x5575b4e79b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:50.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.532+0000 7f3643400640 1 --2- 172.21.15.5:0/825305558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575b4e6f9b0 0x5575b4e79b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.532+0000 7f3643400640 1 -- 172.21.15.5:0/825305558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575b4e7a080 msgr2=0x5575b4e7c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:50.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.532+0000 7f3643400640 1 --2- 172.21.15.5:0/825305558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575b4e7a080 0x5575b4e7c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.532+0000 7f3643400640 1 -- 172.21.15.5:0/825305558 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575b4baf7e0 con 0x5575b4e6f090 2024-04-24T06:50:50.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.532+0000 7f3643400640 1 --2- 172.21.15.5:0/825305558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b4e6f090 0x5575b4e6f470 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f363c011790 tx=0x7f363c011c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a0260dafc40cf1b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:50.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.533+0000 7f363b400640 1 --2- 172.21.15.5:0/1506564499 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f36280a3d20 0x7f36280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:50.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.533+0000 7f362fe00640 1 -- 172.21.15.5:0/825305558 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363c003c30 con 0x5575b4e6f090 2024-04-24T06:50:50.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.533+0000 7f362fe00640 1 -- 172.21.15.5:0/825305558 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f363c00fe80 con 0x5575b4e6f090 2024-04-24T06:50:50.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.533+0000 7f362fe00640 1 -- 172.21.15.5:0/825305558 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363c0193e0 con 0x5575b4e6f090 2024-04-24T06:50:50.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.533+0000 7f363b400640 1 -- 172.21.15.5:0/1506564499 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f36340d0b50 con 0x7f3644104440 2024-04-24T06:50:50.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.533+0000 7f3642a00640 1 --2- 172.21.15.5:0/1506564499 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f36280a3d20 0x7f36280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:50.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.533+0000 7f364cb81740 1 -- 172.21.15.5:0/825305558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b4e6f090 msgr2=0x5575b4e6f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:50.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.533+0000 7f364cb81740 1 --2- 172.21.15.5:0/825305558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b4e6f090 0x5575b4e6f470 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f363c011790 tx=0x7f363c011c50 comp rx=0 tx=0).stop 2024-04-24T06:50:50.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.533+0000 7f3642a00640 1 --2- 172.21.15.5:0/1506564499 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f36280a3d20 0x7f36280a61e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x5575b4e70650 tx=0x7f36340d4600 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:50.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.534+0000 7f364cb81740 1 -- 172.21.15.5:0/825305558 shutdown_connections 2024-04-24T06:50:50.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.534+0000 7f364cb81740 1 --2- 172.21.15.5:0/825305558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575b4e7a080 0x5575b4e7c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.534+0000 7f364cb81740 1 --2- 172.21.15.5:0/825305558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575b4e6f9b0 0x5575b4e79b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.534+0000 7f364cb81740 1 --2- 172.21.15.5:0/825305558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b4e6f090 0x5575b4e6f470 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.534+0000 7f364cb81740 1 -- 172.21.15.5:0/825305558 >> 172.21.15.5:0/825305558 conn(0x5575b4e6b3b0 msgr2=0x5575b4e6b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:50.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.534+0000 7f364cb81740 1 -- 172.21.15.5:0/825305558 shutdown_connections 2024-04-24T06:50:50.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.534+0000 7f364cb81740 1 -- 172.21.15.5:0/825305558 wait complete. 2024-04-24T06:50:50.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.534+0000 7f364cb81740 1 -- start start 2024-04-24T06:50:50.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.534+0000 7f364cb81740 1 Processor -- start 2024-04-24T06:50:50.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.535+0000 7f364cb81740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575b4e6f090 0x5575b4e77050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:50.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.535+0000 7f364cb81740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575b4e6f9b0 0x5575b4e77590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:50.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.535+0000 7f364cb81740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b4e7a080 0x5575b4e77ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:50.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.535+0000 7f364cb81740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5575b4e7ce70 con 0x5575b4e6f9b0 2024-04-24T06:50:50.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.535+0000 7f364cb81740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5575b4d8e660 con 0x5575b4e6f090 2024-04-24T06:50:50.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.535+0000 7f364cb81740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5575b4e7cff0 con 0x5575b4e7a080 2024-04-24T06:50:50.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.535+0000 7f364cb81740 10 client.0 _ll_register_callbacks cb 0x5575b4d7caa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:50:50.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.535+0000 7f3642000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575b4e6f9b0 0x5575b4e77590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:50.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.535+0000 7f3643400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575b4e6f090 0x5575b4e77050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:50.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.535+0000 7f3642000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575b4e6f9b0 0x5575b4e77590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53700/0 (socket says 172.21.15.5:53700) 2024-04-24T06:50:50.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.535+0000 7f3643400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575b4e6f090 0x5575b4e77050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:51900/0 (socket says 172.21.15.5:51900) 2024-04-24T06:50:50.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.535+0000 7f3642000640 1 -- 172.21.15.5:0/3282020208 learned_addr learned my addr 172.21.15.5:0/3282020208 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:50.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.535+0000 7f3643400640 1 --2- 172.21.15.5:0/3282020208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b4e7a080 0x5575b4e77ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:50.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.536+0000 7f3642000640 1 -- 172.21.15.5:0/3282020208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b4e7a080 msgr2=0x5575b4e77ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:50.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.536+0000 7f3642000640 1 --2- 172.21.15.5:0/3282020208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b4e7a080 0x5575b4e77ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.536+0000 7f3642000640 1 -- 172.21.15.5:0/3282020208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575b4e6f090 msgr2=0x5575b4e77050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:50.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.536+0000 7f3642000640 1 --2- 172.21.15.5:0/3282020208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575b4e6f090 0x5575b4e77050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.536+0000 7f3642000640 1 -- 172.21.15.5:0/3282020208 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575b4c0ff70 con 0x5575b4e6f9b0 2024-04-24T06:50:50.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.536+0000 7f3643400640 1 --2- 172.21.15.5:0/3282020208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b4e7a080 0x5575b4e77ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:50:50.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.536+0000 7f3642000640 1 --2- 172.21.15.5:0/3282020208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575b4e6f9b0 0x5575b4e77590 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3630009900 tx=0x7f36300043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38c3463455926b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:50.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.536+0000 7f362cc00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:50:50.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.536+0000 7f362cc00640 1 -- 172.21.15.5:0/3282020208 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3630035040 con 0x5575b4e6f9b0 2024-04-24T06:50:50.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.536+0000 7f362cc00640 1 -- 172.21.15.5:0/3282020208 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f3630005450 con 0x5575b4e6f9b0 2024-04-24T06:50:50.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.536+0000 7f364cb81740 1 -- 172.21.15.5:0/3282020208 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5575b4de2890 con 0x5575b4e6f9b0 2024-04-24T06:50:50.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.537+0000 7f362cc00640 1 -- 172.21.15.5:0/3282020208 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3630005650 con 0x5575b4e6f9b0 2024-04-24T06:50:50.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.537+0000 7f364cb81740 10 client.16623 fetch_fsmap learned FSMap version 27 2024-04-24T06:50:50.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.537+0000 7f364cb81740 1 -- 172.21.15.5:0/3282020208 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5575b4aa7170 con 0x5575b4e6f9b0 2024-04-24T06:50:50.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.537+0000 7f362cc00640 1 -- 172.21.15.5:0/3282020208 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5575b4de2890 con 0x5575b4e6f9b0 2024-04-24T06:50:50.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.537+0000 7f362cc00640 1 -- 172.21.15.5:0/3282020208 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f3630003990 con 0x5575b4e6f9b0 2024-04-24T06:50:50.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.537+0000 7f364cb81740 10 client.16623 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:50:50.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.537+0000 7f364cb81740 10 client.16623 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:50:50.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.537+0000 7f364cb81740 20 client.16623 populate_metadata read hostname 'smithi005' 2024-04-24T06:50:50.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.537+0000 7f364cb81740 1 --2- 172.21.15.5:0/3282020208 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5575b4e8a430 0x5575b4e8c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:50.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.537+0000 7f364cb81740 4 client.16623 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:50:50.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.537+0000 7f364cb81740 1 -- 172.21.15.5:0/3282020208 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575b4ddc700 con 0x5575b4e8a430 2024-04-24T06:50:50.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.538+0000 7f3643400640 1 --2- 172.21.15.5:0/3282020208 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5575b4e8a430 0x5575b4e8c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:50.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.538+0000 7f3643400640 1 --2- 172.21.15.5:0/3282020208 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5575b4e8a430 0x5575b4e8c820 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a32a451bcba34b3 server_cookie=5ee0bfe1ef29853f in_seq=0 out_seq=0 2024-04-24T06:50:50.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.538+0000 7f362cc00640 10 client.16623 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:50:50.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.539+0000 7f362cc00640 1 -- 172.21.15.5:0/3282020208 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5575b4ddc700 con 0x5575b4e8a430 2024-04-24T06:50:50.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.539+0000 7f362cc00640 10 client.16623 handle_command_reply: tid=0 2024-04-24T06:50:50.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.539+0000 7f364cb81740 1 client.16623 shutdown 2024-04-24T06:50:50.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.539+0000 7f364cb81740 1 -- 172.21.15.5:0/3282020208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575b4e6f9b0 msgr2=0x5575b4e77590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:50.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.539+0000 7f364cb81740 1 --2- 172.21.15.5:0/3282020208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575b4e6f9b0 0x5575b4e77590 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3630009900 tx=0x7f36300043b0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.539+0000 7f364cb81740 1 -- 172.21.15.5:0/3282020208 shutdown_connections 2024-04-24T06:50:50.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.539+0000 7f364cb81740 1 --2- 172.21.15.5:0/3282020208 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5575b4e8a430 0x5575b4e8c820 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.539+0000 7f364cb81740 1 --2- 172.21.15.5:0/3282020208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b4e7a080 0x5575b4e77ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.539+0000 7f364cb81740 1 --2- 172.21.15.5:0/3282020208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575b4e6f9b0 0x5575b4e77590 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.539+0000 7f362cc00640 0 client.16623 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:50:50.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.539+0000 7f364cb81740 1 --2- 172.21.15.5:0/3282020208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575b4e6f090 0x5575b4e77050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.539+0000 7f364cb81740 1 -- 172.21.15.5:0/3282020208 >> 172.21.15.5:0/3282020208 conn(0x5575b4e6b3b0 msgr2=0x5575b4e711b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:50.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.540+0000 7f364cb81740 1 -- 172.21.15.5:0/3282020208 shutdown_connections 2024-04-24T06:50:50.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.540+0000 7f364cb81740 1 -- 172.21.15.5:0/3282020208 wait complete. 2024-04-24T06:50:50.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.540+0000 7f364cb81740 20 client.16623 trim_cache size 0 max 16384 2024-04-24T06:50:50.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.559+0000 7f364cb81740 1 Processor -- start 2024-04-24T06:50:50.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.559+0000 7f364cb81740 1 -- start start 2024-04-24T06:50:50.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.560+0000 7f364cb81740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575b4e6f090 0x5575b4e85d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:50.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.560+0000 7f364cb81740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575b4e6f9b0 0x5575b4e717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:50.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.560+0000 7f364cb81740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b4e7a080 0x5575b4e920f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:50.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.560+0000 7f364cb81740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f36441b2870 con 0x5575b4e6f090 2024-04-24T06:50:50.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.560+0000 7f364cb81740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f36441b0140 con 0x5575b4e6f9b0 2024-04-24T06:50:50.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.560+0000 7f364cb81740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f36441adf90 con 0x5575b4e7a080 2024-04-24T06:50:50.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.560+0000 7f3643400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575b4e6f090 0x5575b4e85d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:50.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.560+0000 7f3643400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575b4e6f090 0x5575b4e85d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53714/0 (socket says 172.21.15.5:53714) 2024-04-24T06:50:50.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.560+0000 7f3643400640 1 -- 172.21.15.5:0/296300041 learned_addr learned my addr 172.21.15.5:0/296300041 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:50.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.560+0000 7f3642000640 1 --2- 172.21.15.5:0/296300041 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575b4e6f9b0 0x5575b4e717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:50.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.560+0000 7f3643400640 1 --2- 172.21.15.5:0/296300041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b4e7a080 0x5575b4e920f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:50.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.560+0000 7f3643400640 1 -- 172.21.15.5:0/296300041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b4e7a080 msgr2=0x5575b4e920f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:50.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.560+0000 7f3643400640 1 --2- 172.21.15.5:0/296300041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b4e7a080 0x5575b4e920f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.560+0000 7f3643400640 1 -- 172.21.15.5:0/296300041 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575b4e6f9b0 msgr2=0x5575b4e717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:50.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.560+0000 7f3643400640 1 --2- 172.21.15.5:0/296300041 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575b4e6f9b0 0x5575b4e717f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.560+0000 7f3643400640 1 -- 172.21.15.5:0/296300041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575b4e6bcf0 con 0x5575b4e6f090 2024-04-24T06:50:50.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.561+0000 7f3643400640 1 --2- 172.21.15.5:0/296300041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575b4e6f090 0x5575b4e85d30 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f363c004590 tx=0x7f363c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922f409e6fbc2acf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:50.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.561+0000 7f362f400640 1 -- 172.21.15.5:0/296300041 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363c02e020 con 0x5575b4e6f090 2024-04-24T06:50:50.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.561+0000 7f362f400640 1 -- 172.21.15.5:0/296300041 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f363c0046f0 con 0x5575b4e6f090 2024-04-24T06:50:50.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.561+0000 7f362f400640 1 -- 172.21.15.5:0/296300041 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363c01a6a0 con 0x5575b4e6f090 2024-04-24T06:50:50.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.562+0000 7f364cb81740 1 -- 172.21.15.5:0/296300041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575b4e6f090 msgr2=0x5575b4e85d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:50.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.562+0000 7f364cb81740 1 --2- 172.21.15.5:0/296300041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575b4e6f090 0x5575b4e85d30 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f363c004590 tx=0x7f363c009500 comp rx=0 tx=0).stop 2024-04-24T06:50:50.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.562+0000 7f364cb81740 1 -- 172.21.15.5:0/296300041 shutdown_connections 2024-04-24T06:50:50.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.562+0000 7f364cb81740 1 --2- 172.21.15.5:0/296300041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b4e7a080 0x5575b4e920f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.562+0000 7f364cb81740 1 --2- 172.21.15.5:0/296300041 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575b4e6f9b0 0x5575b4e717f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.562+0000 7f364cb81740 1 --2- 172.21.15.5:0/296300041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575b4e6f090 0x5575b4e85d30 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.562+0000 7f364cb81740 1 -- 172.21.15.5:0/296300041 >> 172.21.15.5:0/296300041 conn(0x5575b4e6b3b0 msgr2=0x5575b4e79110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:50.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.562+0000 7f364cb81740 1 -- 172.21.15.5:0/296300041 shutdown_connections 2024-04-24T06:50:50.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.562+0000 7f364cb81740 1 -- 172.21.15.5:0/296300041 wait complete. 2024-04-24T06:50:50.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.562+0000 7f364cb81740 1 -- start start 2024-04-24T06:50:50.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.563+0000 7f364cb81740 1 Processor -- start 2024-04-24T06:50:50.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.563+0000 7f364cb81740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575b4e6f090 0x5575b4e7e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:50.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.563+0000 7f364cb81740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b4e6f9b0 0x5575b4e7e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:50.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.563+0000 7f364cb81740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575b4e7a080 0x5575b4eef960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.563+0000 7f364cb81740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f36441ae600 con 0x5575b4e7a080 2024-04-24T06:50:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.563+0000 7f364cb81740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f36441afa70 con 0x5575b4e6f090 2024-04-24T06:50:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.563+0000 7f364cb81740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f36441aecd0 con 0x5575b4e6f9b0 2024-04-24T06:50:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.563+0000 7f364cb81740 10 client.0 _ll_register_callbacks cb 0x7f3630009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:50:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.563+0000 7f3643400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575b4e6f090 0x5575b4e7e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.563+0000 7f3643400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575b4e7a080 0x5575b4eef960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:50.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.564+0000 7f3643400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575b4e7a080 0x5575b4eef960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53728/0 (socket says 172.21.15.5:53728) 2024-04-24T06:50:50.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.564+0000 7f3643400640 1 -- 172.21.15.5:0/409313037 learned_addr learned my addr 172.21.15.5:0/409313037 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:50.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.564+0000 7f3642000640 1 --2- 172.21.15.5:0/409313037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b4e6f9b0 0x5575b4e7e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:50.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.564+0000 7f3643400640 1 -- 172.21.15.5:0/409313037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b4e6f9b0 msgr2=0x5575b4e7e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:50.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.564+0000 7f3643400640 1 --2- 172.21.15.5:0/409313037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b4e6f9b0 0x5575b4e7e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.564+0000 7f3643400640 1 -- 172.21.15.5:0/409313037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575b4e6f090 msgr2=0x5575b4e7e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:50.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.564+0000 7f3643400640 1 --2- 172.21.15.5:0/409313037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575b4e6f090 0x5575b4e7e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.564+0000 7f3643400640 1 -- 172.21.15.5:0/409313037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575b4eefea0 con 0x5575b4e7a080 2024-04-24T06:50:50.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.564+0000 7f3643400640 1 --2- 172.21.15.5:0/409313037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575b4e7a080 0x5575b4eef960 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f363c0046c0 tx=0x7f363c009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=346f0e957c9170cc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:50.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.564+0000 7f362cc00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:50:50.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.565+0000 7f362cc00640 1 -- 172.21.15.5:0/409313037 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363c02e070 con 0x5575b4e7a080 2024-04-24T06:50:50.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.565+0000 7f362cc00640 1 -- 172.21.15.5:0/409313037 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f363c003a80 con 0x5575b4e7a080 2024-04-24T06:50:50.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.565+0000 7f364cb81740 1 -- 172.21.15.5:0/409313037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f363c0046f0 con 0x5575b4e7a080 2024-04-24T06:50:50.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.565+0000 7f362cc00640 1 -- 172.21.15.5:0/409313037 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f363c046050 con 0x5575b4e7a080 2024-04-24T06:50:50.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.565+0000 7f364cb81740 10 client.16629 fetch_fsmap learned FSMap version 27 2024-04-24T06:50:50.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.565+0000 7f364cb81740 1 -- 172.21.15.5:0/409313037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5575b4ef10a0 con 0x5575b4e7a080 2024-04-24T06:50:50.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.565+0000 7f362cc00640 1 -- 172.21.15.5:0/409313037 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f363c0046f0 con 0x5575b4e7a080 2024-04-24T06:50:50.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.565+0000 7f362cc00640 1 -- 172.21.15.5:0/409313037 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f363c018070 con 0x5575b4e7a080 2024-04-24T06:50:50.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.565+0000 7f364cb81740 10 client.16629 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:50:50.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.565+0000 7f364cb81740 10 client.16629 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:50:50.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.566+0000 7f364cb81740 20 client.16629 populate_metadata read hostname 'smithi005' 2024-04-24T06:50:50.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.566+0000 7f364cb81740 1 --2- 172.21.15.5:0/409313037 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5575b4ef15c0 0x5575b4ef39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:50.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.566+0000 7f364cb81740 4 client.16629 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T06:50:50.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.566+0000 7f364cb81740 1 -- 172.21.15.5:0/409313037 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5575b4d58980 con 0x5575b4ef15c0 2024-04-24T06:50:50.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.566+0000 7f3642000640 1 --2- 172.21.15.5:0/409313037 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5575b4ef15c0 0x5575b4ef39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:50.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.566+0000 7f3642000640 1 --2- 172.21.15.5:0/409313037 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5575b4ef15c0 0x5575b4ef39b0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ede86cd4b4e8138 server_cookie=e11c8d99aeba33ea in_seq=0 out_seq=0 2024-04-24T06:50:50.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.566+0000 7f362cc00640 10 client.16629 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:50:50.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.568+0000 7f362cc00640 1 -- 172.21.15.5:0/409313037 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5575b4d58980 con 0x5575b4ef15c0 2024-04-24T06:50:50.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.568+0000 7f362cc00640 10 client.16629 handle_command_reply: tid=0 2024-04-24T06:50:50.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.568+0000 7f364cb81740 1 client.16629 shutdown 2024-04-24T06:50:50.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.568+0000 7f364cb81740 1 -- 172.21.15.5:0/409313037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575b4e7a080 msgr2=0x5575b4eef960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:50.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.568+0000 7f364cb81740 1 --2- 172.21.15.5:0/409313037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575b4e7a080 0x5575b4eef960 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f363c0046c0 tx=0x7f363c009fd0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.568+0000 7f364cb81740 1 -- 172.21.15.5:0/409313037 shutdown_connections 2024-04-24T06:50:50.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.568+0000 7f364cb81740 1 --2- 172.21.15.5:0/409313037 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5575b4ef15c0 0x5575b4ef39b0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.568+0000 7f364cb81740 1 --2- 172.21.15.5:0/409313037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575b4e7a080 0x5575b4eef960 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.568+0000 7f364cb81740 1 --2- 172.21.15.5:0/409313037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5575b4e6f9b0 0x5575b4e7e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.568+0000 7f364cb81740 1 --2- 172.21.15.5:0/409313037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575b4e6f090 0x5575b4e7e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.568+0000 7f362cc00640 0 client.16629 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:50:50.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.568+0000 7f364cb81740 1 -- 172.21.15.5:0/409313037 >> 172.21.15.5:0/409313037 conn(0x5575b4e6b3b0 msgr2=0x5575b4e69c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:50.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.568+0000 7f364cb81740 1 -- 172.21.15.5:0/409313037 shutdown_connections 2024-04-24T06:50:50.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.568+0000 7f364cb81740 1 -- 172.21.15.5:0/409313037 wait complete. 2024-04-24T06:50:50.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.568+0000 7f364cb81740 20 client.16629 trim_cache size 0 max 16384 2024-04-24T06:50:50.577 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:50:50.577 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-24T06:50:50.577 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "83a47208-daee-4ff2-864d-55a5d112d1dc", 2024-04-24T06:50:50.577 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-24T06:50:50.578 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:50:50.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.569+0000 7f3649c00640 1 -- 172.21.15.5:0/1506564499 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f36280a3d20 msgr2=0x7f36280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:50.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.569+0000 7f3649c00640 1 --2- 172.21.15.5:0/1506564499 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f36280a3d20 0x7f36280a61e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x5575b4e70650 tx=0x7f36340d4600 comp rx=0 tx=0).stop 2024-04-24T06:50:50.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.569+0000 7f3649c00640 1 -- 172.21.15.5:0/1506564499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3644104440 msgr2=0x7f36441a0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:50.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.569+0000 7f3649c00640 1 --2- 172.21.15.5:0/1506564499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3644104440 0x7f36441a0720 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f363400ea60 tx=0x7f363400ef20 comp rx=0 tx=0).stop 2024-04-24T06:50:50.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.569+0000 7f3649c00640 1 -- 172.21.15.5:0/1506564499 shutdown_connections 2024-04-24T06:50:50.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.569+0000 7f3649c00640 1 --2- 172.21.15.5:0/1506564499 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f36280a3d20 0x7f36280a61e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.569+0000 7f3649c00640 1 --2- 172.21.15.5:0/1506564499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f364419a8d0 0x7f364419ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.569+0000 7f3649c00640 1 --2- 172.21.15.5:0/1506564499 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3644109510 0x7f36441a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.569+0000 7f3649c00640 1 --2- 172.21.15.5:0/1506564499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3644104440 0x7f36441a0720 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.569+0000 7f3649c00640 1 -- 172.21.15.5:0/1506564499 >> 172.21.15.5:0/1506564499 conn(0x7f36440ffc80 msgr2=0x7f3644101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:50.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.569+0000 7f3649c00640 1 -- 172.21.15.5:0/1506564499 shutdown_connections 2024-04-24T06:50:50.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.569+0000 7f3649c00640 1 -- 172.21.15.5:0/1506564499 wait complete. 2024-04-24T06:50:50.591 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T06:50:50.592 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T06:50:50.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.827+0000 7f08f3e00640 1 -- 172.21.15.5:0/924304164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08f4103960 msgr2=0x7f08f410be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:50.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.827+0000 7f08f3e00640 1 --2- 172.21.15.5:0/924304164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08f4103960 0x7f08f410be40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f08e0009900 tx=0x7f08e002a480 comp rx=0 tx=0).stop 2024-04-24T06:50:50.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.827+0000 7f08f3e00640 1 -- 172.21.15.5:0/924304164 shutdown_connections 2024-04-24T06:50:50.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.827+0000 7f08f3e00640 1 --2- 172.21.15.5:0/924304164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08f410c410 0x7f08f410e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.827+0000 7f08f3e00640 1 --2- 172.21.15.5:0/924304164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08f4103960 0x7f08f410be40 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.827+0000 7f08f3e00640 1 --2- 172.21.15.5:0/924304164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08f4103040 0x7f08f4103420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.827+0000 7f08f3e00640 1 -- 172.21.15.5:0/924304164 >> 172.21.15.5:0/924304164 conn(0x7f08f4077ab0 msgr2=0x7f08f40fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:50.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.827+0000 7f08f3e00640 1 -- 172.21.15.5:0/924304164 shutdown_connections 2024-04-24T06:50:50.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.827+0000 7f08f3e00640 1 -- 172.21.15.5:0/924304164 wait complete. 2024-04-24T06:50:50.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.828+0000 7f08f3e00640 1 Processor -- start 2024-04-24T06:50:50.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.828+0000 7f08f3e00640 1 -- start start 2024-04-24T06:50:50.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.829+0000 7f08f3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08f4103040 0x7f08f4100c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:50.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.829+0000 7f08f3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08f4103960 0x7f08f41011d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:50.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.829+0000 7f08f3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08f410c410 0x7f08f4101710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:50.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.829+0000 7f08f3e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f08f41121f0 con 0x7f08f4103960 2024-04-24T06:50:50.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.829+0000 7f08f3e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f08f4112070 con 0x7f08f4103040 2024-04-24T06:50:50.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.829+0000 7f08f3e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f08f4112370 con 0x7f08f410c410 2024-04-24T06:50:50.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.829+0000 7f08f2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08f4103040 0x7f08f4100c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:50.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.829+0000 7f08f2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08f4103040 0x7f08f4100c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:51942/0 (socket says 172.21.15.5:51942) 2024-04-24T06:50:50.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.829+0000 7f08f2a00640 1 -- 172.21.15.5:0/1862736390 learned_addr learned my addr 172.21.15.5:0/1862736390 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:50.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.829+0000 7f08f3400640 1 --2- 172.21.15.5:0/1862736390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08f410c410 0x7f08f4101710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:50.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.829+0000 7f08f2000640 1 --2- 172.21.15.5:0/1862736390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08f4103960 0x7f08f41011d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:50.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.830+0000 7f08f3400640 1 -- 172.21.15.5:0/1862736390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08f4103040 msgr2=0x7f08f4100c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:50.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.830+0000 7f08f3400640 1 --2- 172.21.15.5:0/1862736390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08f4103040 0x7f08f4100c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.830+0000 7f08f3400640 1 -- 172.21.15.5:0/1862736390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08f4103960 msgr2=0x7f08f41011d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:50.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.830+0000 7f08f3400640 1 --2- 172.21.15.5:0/1862736390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08f4103960 0x7f08f41011d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.830+0000 7f08f3400640 1 -- 172.21.15.5:0/1862736390 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08f41a6cb0 con 0x7f08f410c410 2024-04-24T06:50:50.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.830+0000 7f08f2a00640 1 --2- 172.21.15.5:0/1862736390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08f4103040 0x7f08f4100c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:50:50.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.830+0000 7f08f3400640 1 --2- 172.21.15.5:0/1862736390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08f410c410 0x7f08f4101710 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f08ec009fd0 tx=0x7f08ec00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:50.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.830+0000 7f08eb400640 1 -- 172.21.15.5:0/1862736390 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec00ce20 con 0x7f08f410c410 2024-04-24T06:50:50.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.830+0000 7f08eb400640 1 -- 172.21.15.5:0/1862736390 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08ec005020 con 0x7f08f410c410 2024-04-24T06:50:50.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.830+0000 7f08f3e00640 1 -- 172.21.15.5:0/1862736390 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08f41a6fa0 con 0x7f08f410c410 2024-04-24T06:50:50.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.830+0000 7f08eb400640 1 -- 172.21.15.5:0/1862736390 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec010720 con 0x7f08f410c410 2024-04-24T06:50:50.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.830+0000 7f08f3e00640 1 -- 172.21.15.5:0/1862736390 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08f41a7470 con 0x7f08f410c410 2024-04-24T06:50:50.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.834+0000 7f08eb400640 1 -- 172.21.15.5:0/1862736390 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f08ec005540 con 0x7f08f410c410 2024-04-24T06:50:50.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.835+0000 7f08fc629740 1 Processor -- start 2024-04-24T06:50:50.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.835+0000 7f08fc629740 1 -- start start 2024-04-24T06:50:50.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.835+0000 7f08fc629740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626b4173a30 0x5626b4173e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:50.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.835+0000 7f08fc629740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626b4174350 0x5626b417e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:50.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.835+0000 7f08fc629740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626b417ea20 0x5626b4180e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:50.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.835+0000 7f08fc629740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5626b40d91f0 con 0x5626b4174350 2024-04-24T06:50:50.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.835+0000 7f08fc629740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5626b407f350 con 0x5626b4173a30 2024-04-24T06:50:50.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.835+0000 7f08fc629740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5626b40d51b0 con 0x5626b417ea20 2024-04-24T06:50:50.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.836+0000 7f08f2000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626b4173a30 0x5626b4173e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:50.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.836+0000 7f08f2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626b4174350 0x5626b417e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:50.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.836+0000 7f08f2000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626b4173a30 0x5626b4173e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:51950/0 (socket says 172.21.15.5:51950) 2024-04-24T06:50:50.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.836+0000 7f08f2000640 1 -- 172.21.15.5:0/4005106635 learned_addr learned my addr 172.21.15.5:0/4005106635 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:50.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.836+0000 7f08f2000640 1 --2- 172.21.15.5:0/4005106635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626b417ea20 0x5626b4180e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:50.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.836+0000 7f08f2000640 1 -- 172.21.15.5:0/4005106635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626b417ea20 msgr2=0x5626b4180e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:50.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.836+0000 7f08f2000640 1 --2- 172.21.15.5:0/4005106635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626b417ea20 0x5626b4180e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.836+0000 7f08f2000640 1 -- 172.21.15.5:0/4005106635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626b4174350 msgr2=0x5626b417e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:50.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.836+0000 7f08f2000640 1 --2- 172.21.15.5:0/4005106635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626b4174350 0x5626b417e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.836+0000 7f08f2000640 1 -- 172.21.15.5:0/4005106635 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626b40d3f50 con 0x5626b4173a30 2024-04-24T06:50:50.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.837+0000 7f08f2000640 1 --2- 172.21.15.5:0/4005106635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626b4173a30 0x5626b4173e10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f08e0005ec0 tx=0x7f08e0005bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=823a05025557c9b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:50.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.837+0000 7f08eb400640 1 --2- 172.21.15.5:0/1862736390 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f08d80a3da0 0x7f08d80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:50.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.837+0000 7f08eb400640 1 -- 172.21.15.5:0/1862736390 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f08ec0d0da0 con 0x7f08f410c410 2024-04-24T06:50:50.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.837+0000 7f08f2a00640 1 --2- 172.21.15.5:0/1862736390 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f08d80a3da0 0x7f08d80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:50.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.837+0000 7f08dfe00640 1 -- 172.21.15.5:0/4005106635 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e0035040 con 0x5626b4173a30 2024-04-24T06:50:50.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.837+0000 7f08dfe00640 1 -- 172.21.15.5:0/4005106635 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08e00046c0 con 0x5626b4173a30 2024-04-24T06:50:50.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.837+0000 7f08dfe00640 1 -- 172.21.15.5:0/4005106635 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e0003850 con 0x5626b4173a30 2024-04-24T06:50:50.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.839+0000 7f08fc629740 1 -- 172.21.15.5:0/4005106635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626b4173a30 msgr2=0x5626b4173e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:50.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.839+0000 7f08fc629740 1 --2- 172.21.15.5:0/4005106635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626b4173a30 0x5626b4173e10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f08e0005ec0 tx=0x7f08e0005bc0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.839+0000 7f08f2a00640 1 --2- 172.21.15.5:0/1862736390 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f08d80a3da0 0x7f08d80a6260 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f08e4009620 tx=0x7f08e4005f90 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:50.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.839+0000 7f08fc629740 1 -- 172.21.15.5:0/4005106635 shutdown_connections 2024-04-24T06:50:50.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.839+0000 7f08fc629740 1 --2- 172.21.15.5:0/4005106635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626b417ea20 0x5626b4180e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.839+0000 7f08fc629740 1 --2- 172.21.15.5:0/4005106635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626b4174350 0x5626b417e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.839+0000 7f08fc629740 1 --2- 172.21.15.5:0/4005106635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626b4173a30 0x5626b4173e10 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.839+0000 7f08fc629740 1 -- 172.21.15.5:0/4005106635 >> 172.21.15.5:0/4005106635 conn(0x5626b416fd50 msgr2=0x5626b4170130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:50.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.839+0000 7f08fc629740 1 -- 172.21.15.5:0/4005106635 shutdown_connections 2024-04-24T06:50:50.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.839+0000 7f08fc629740 1 -- 172.21.15.5:0/4005106635 wait complete. 2024-04-24T06:50:50.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.840+0000 7f08fc629740 1 -- start start 2024-04-24T06:50:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.840+0000 7f08fc629740 1 Processor -- start 2024-04-24T06:50:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.841+0000 7f08fc629740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626b4173a30 0x5626b417ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.841+0000 7f08fc629740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626b4174350 0x5626b417bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.841+0000 7f08fc629740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626b417ea20 0x5626b417c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.841+0000 7f08fc629740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5626b41817b0 con 0x5626b4173a30 2024-04-24T06:50:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.841+0000 7f08fc629740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5626b4092690 con 0x5626b417ea20 2024-04-24T06:50:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.841+0000 7f08fc629740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5626b4181930 con 0x5626b4174350 2024-04-24T06:50:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.841+0000 7f08fc629740 10 client.0 _ll_register_callbacks cb 0x5626b40ce650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:50:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.841+0000 7f08f2000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626b4174350 0x5626b417bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.841+0000 7f08f2000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626b4174350 0x5626b417bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:57354/0 (socket says 172.21.15.5:57354) 2024-04-24T06:50:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.841+0000 7f08f2000640 1 -- 172.21.15.5:0/2213111843 learned_addr learned my addr 172.21.15.5:0/2213111843 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.841+0000 7f08f2a00640 1 --2- 172.21.15.5:0/2213111843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626b4173a30 0x5626b417ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.841+0000 7f08f3400640 1 --2- 172.21.15.5:0/2213111843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626b417ea20 0x5626b417c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.842+0000 7f08f2000640 1 -- 172.21.15.5:0/2213111843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626b417ea20 msgr2=0x5626b417c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.842+0000 7f08f2000640 1 --2- 172.21.15.5:0/2213111843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626b417ea20 0x5626b417c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.842+0000 7f08f2000640 1 -- 172.21.15.5:0/2213111843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626b4173a30 msgr2=0x5626b417ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.842+0000 7f08f2000640 1 --2- 172.21.15.5:0/2213111843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626b4173a30 0x5626b417ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.842+0000 7f08f2000640 1 -- 172.21.15.5:0/2213111843 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626b4091920 con 0x5626b4174350 2024-04-24T06:50:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.842+0000 7f08f2000640 1 --2- 172.21.15.5:0/2213111843 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626b4174350 0x5626b417bf60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f08e002a4c0 tx=0x7f08e0004aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a3f45b236f53c25 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.842+0000 7f08dcc00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:50:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.842+0000 7f08dcc00640 1 -- 172.21.15.5:0/2213111843 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e0035040 con 0x5626b4174350 2024-04-24T06:50:50.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.842+0000 7f08dcc00640 1 -- 172.21.15.5:0/2213111843 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f08e00051e0 con 0x5626b4174350 2024-04-24T06:50:50.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.842+0000 7f08fc629740 1 -- 172.21.15.5:0/2213111843 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08e00046c0 con 0x5626b4174350 2024-04-24T06:50:50.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.843+0000 7f08dcc00640 1 -- 172.21.15.5:0/2213111843 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f08e0057050 con 0x5626b4174350 2024-04-24T06:50:50.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.843+0000 7f08fc629740 10 client.26785 fetch_fsmap learned FSMap version 27 2024-04-24T06:50:50.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.843+0000 7f08fc629740 1 -- 172.21.15.5:0/2213111843 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5626b3eb57d0 con 0x5626b4174350 2024-04-24T06:50:50.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.843+0000 7f08dcc00640 1 -- 172.21.15.5:0/2213111843 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08e00046c0 con 0x5626b4174350 2024-04-24T06:50:50.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.843+0000 7f08dcc00640 1 -- 172.21.15.5:0/2213111843 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f08e0052030 con 0x5626b4174350 2024-04-24T06:50:50.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.843+0000 7f08fc629740 10 client.26785 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:50:50.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.843+0000 7f08fc629740 10 client.26785 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:50:50.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.843+0000 7f08fc629740 20 client.26785 populate_metadata read hostname 'smithi005' 2024-04-24T06:50:50.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.843+0000 7f08fc629740 1 --2- 172.21.15.5:0/2213111843 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5626b418ec60 0x5626b4191050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:50.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.843+0000 7f08fc629740 4 client.26785 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:50:50.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.843+0000 7f08fc629740 1 -- 172.21.15.5:0/2213111843 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626b40e6c10 con 0x5626b418ec60 2024-04-24T06:50:50.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.844+0000 7f08f2a00640 1 --2- 172.21.15.5:0/2213111843 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5626b418ec60 0x5626b4191050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:50.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.844+0000 7f08f2a00640 1 --2- 172.21.15.5:0/2213111843 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5626b418ec60 0x5626b4191050 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ace4e4799b16f628 server_cookie=471c50cb8b88ccdd in_seq=0 out_seq=0 2024-04-24T06:50:50.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.844+0000 7f08dcc00640 10 client.26785 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:50:50.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.846+0000 7f08dcc00640 1 -- 172.21.15.5:0/2213111843 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5626b40e6c10 con 0x5626b418ec60 2024-04-24T06:50:50.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.846+0000 7f08dcc00640 10 client.26785 handle_command_reply: tid=0 2024-04-24T06:50:50.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.846+0000 7f08fc629740 1 client.26785 shutdown 2024-04-24T06:50:50.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.846+0000 7f08fc629740 1 -- 172.21.15.5:0/2213111843 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626b4174350 msgr2=0x5626b417bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:50.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.846+0000 7f08fc629740 1 --2- 172.21.15.5:0/2213111843 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626b4174350 0x5626b417bf60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f08e002a4c0 tx=0x7f08e0004aa0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.846+0000 7f08fc629740 1 -- 172.21.15.5:0/2213111843 shutdown_connections 2024-04-24T06:50:50.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.846+0000 7f08fc629740 1 --2- 172.21.15.5:0/2213111843 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5626b418ec60 0x5626b4191050 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.846+0000 7f08fc629740 1 --2- 172.21.15.5:0/2213111843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626b417ea20 0x5626b417c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.846+0000 7f08fc629740 1 --2- 172.21.15.5:0/2213111843 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626b4174350 0x5626b417bf60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.846+0000 7f08fc629740 1 --2- 172.21.15.5:0/2213111843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626b4173a30 0x5626b417ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.846+0000 7f08dcc00640 0 client.26785 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:50:50.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.846+0000 7f08fc629740 1 -- 172.21.15.5:0/2213111843 >> 172.21.15.5:0/2213111843 conn(0x5626b416fd50 msgr2=0x5626b416efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:50.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.847+0000 7f08fc629740 1 -- 172.21.15.5:0/2213111843 shutdown_connections 2024-04-24T06:50:50.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.847+0000 7f08fc629740 1 -- 172.21.15.5:0/2213111843 wait complete. 2024-04-24T06:50:50.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.847+0000 7f08fc629740 20 client.26785 trim_cache size 0 max 16384 2024-04-24T06:50:50.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.872+0000 7f08fc629740 1 Processor -- start 2024-04-24T06:50:50.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.872+0000 7f08fc629740 1 -- start start 2024-04-24T06:50:50.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.872+0000 7f08fc629740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626b4173a30 0x5626b4175890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:50.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.872+0000 7f08fc629740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626b4174350 0x5626b4194070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:50.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.872+0000 7f08fc629740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626b417ea20 0x5626b41945b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:50.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.872+0000 7f08fc629740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f08f41b25b0 con 0x5626b4174350 2024-04-24T06:50:50.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.872+0000 7f08fc629740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f08f41afe80 con 0x5626b4173a30 2024-04-24T06:50:50.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.872+0000 7f08fc629740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f08f41adcd0 con 0x5626b417ea20 2024-04-24T06:50:50.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.873+0000 7f08f3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626b417ea20 0x5626b41945b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:50.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.873+0000 7f08f2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626b4173a30 0x5626b4175890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:50.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.873+0000 7f08f2000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626b4174350 0x5626b4194070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:50.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.873+0000 7f08f2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626b4173a30 0x5626b4175890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:51962/0 (socket says 172.21.15.5:51962) 2024-04-24T06:50:50.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.873+0000 7f08f3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626b417ea20 0x5626b41945b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:57364/0 (socket says 172.21.15.5:57364) 2024-04-24T06:50:50.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.873+0000 7f08f2a00640 1 -- 172.21.15.5:0/1444071405 learned_addr learned my addr 172.21.15.5:0/1444071405 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:50.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.873+0000 7f08f2a00640 1 -- 172.21.15.5:0/1444071405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626b417ea20 msgr2=0x5626b41945b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:50.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.873+0000 7f08f2a00640 1 --2- 172.21.15.5:0/1444071405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626b417ea20 0x5626b41945b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.873+0000 7f08f2a00640 1 -- 172.21.15.5:0/1444071405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626b4174350 msgr2=0x5626b4194070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:50.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.873+0000 7f08f2a00640 1 --2- 172.21.15.5:0/1444071405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626b4174350 0x5626b4194070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.873+0000 7f08f2a00640 1 -- 172.21.15.5:0/1444071405 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626b3deb3f0 con 0x5626b4173a30 2024-04-24T06:50:50.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.873+0000 7f08f3400640 1 --2- 172.21.15.5:0/1444071405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626b417ea20 0x5626b41945b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:50:50.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.873+0000 7f08f2a00640 1 --2- 172.21.15.5:0/1444071405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626b4173a30 0x5626b4175890 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x5626b4181990 tx=0x7f08e401a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b831b540ae2171eb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:50.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.874+0000 7f08df400640 1 -- 172.21.15.5:0/1444071405 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e4004020 con 0x5626b4173a30 2024-04-24T06:50:50.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.874+0000 7f08df400640 1 -- 172.21.15.5:0/1444071405 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08e402b900 con 0x5626b4173a30 2024-04-24T06:50:50.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.874+0000 7f08df400640 1 -- 172.21.15.5:0/1444071405 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e4005480 con 0x5626b4173a30 2024-04-24T06:50:50.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.875+0000 7f08fc629740 1 -- 172.21.15.5:0/1444071405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626b4173a30 msgr2=0x5626b4175890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:50.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.875+0000 7f08fc629740 1 --2- 172.21.15.5:0/1444071405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626b4173a30 0x5626b4175890 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x5626b4181990 tx=0x7f08e401a000 comp rx=0 tx=0).stop 2024-04-24T06:50:50.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.875+0000 7f08fc629740 1 -- 172.21.15.5:0/1444071405 shutdown_connections 2024-04-24T06:50:50.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.875+0000 7f08fc629740 1 --2- 172.21.15.5:0/1444071405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626b417ea20 0x5626b41945b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.875+0000 7f08fc629740 1 --2- 172.21.15.5:0/1444071405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626b4174350 0x5626b4194070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.875+0000 7f08fc629740 1 --2- 172.21.15.5:0/1444071405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626b4173a30 0x5626b4175890 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.875+0000 7f08fc629740 1 -- 172.21.15.5:0/1444071405 >> 172.21.15.5:0/1444071405 conn(0x5626b416fd50 msgr2=0x5626b4175180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:50.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.875+0000 7f08fc629740 1 -- 172.21.15.5:0/1444071405 shutdown_connections 2024-04-24T06:50:50.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.875+0000 7f08fc629740 1 -- 172.21.15.5:0/1444071405 wait complete. 2024-04-24T06:50:50.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.875+0000 7f08fc629740 1 -- start start 2024-04-24T06:50:50.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.876+0000 7f08fc629740 1 Processor -- start 2024-04-24T06:50:50.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.876+0000 7f08fc629740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626b4173a30 0x5626b4191c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:50.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.876+0000 7f08fc629740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626b4174350 0x5626b4192180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:50.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.876+0000 7f08fc629740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626b417ea20 0x5626b41926c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:50.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.876+0000 7f08fc629740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f08f41ae340 con 0x5626b4173a30 2024-04-24T06:50:50.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.876+0000 7f08fc629740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f08f41af7b0 con 0x5626b4174350 2024-04-24T06:50:50.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.876+0000 7f08fc629740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f08f41aea10 con 0x5626b417ea20 2024-04-24T06:50:50.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.876+0000 7f08fc629740 10 client.0 _ll_register_callbacks cb 0x7f08e4009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:50:50.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.877+0000 7f08f3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626b417ea20 0x5626b41926c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:50.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.877+0000 7f08f2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626b4173a30 0x5626b4191c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:50.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.877+0000 7f08f3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626b417ea20 0x5626b41926c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:57372/0 (socket says 172.21.15.5:57372) 2024-04-24T06:50:50.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.877+0000 7f08f3400640 1 -- 172.21.15.5:0/2058195706 learned_addr learned my addr 172.21.15.5:0/2058195706 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:50:50.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.877+0000 7f08f2000640 1 --2- 172.21.15.5:0/2058195706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626b4174350 0x5626b4192180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:50.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.877+0000 7f08f3400640 1 -- 172.21.15.5:0/2058195706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626b4174350 msgr2=0x5626b4192180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:50.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.877+0000 7f08f3400640 1 --2- 172.21.15.5:0/2058195706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626b4174350 0x5626b4192180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.877+0000 7f08f3400640 1 -- 172.21.15.5:0/2058195706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626b4173a30 msgr2=0x5626b4191c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:50:50.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.877+0000 7f08f3400640 1 --2- 172.21.15.5:0/2058195706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626b4173a30 0x5626b4191c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.877+0000 7f08f3400640 1 -- 172.21.15.5:0/2058195706 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626b3dab0f0 con 0x5626b417ea20 2024-04-24T06:50:50.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.877+0000 7f08f3400640 1 --2- 172.21.15.5:0/2058195706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626b417ea20 0x5626b41926c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0991b0 tx=0x7f08ec09cec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=613d37f86f57c1ac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:50:50.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.877+0000 7f08dcc00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:50:50.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.878+0000 7f08dcc00640 1 -- 172.21.15.5:0/2058195706 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec005290 con 0x5626b417ea20 2024-04-24T06:50:50.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.878+0000 7f08fc629740 1 -- 172.21.15.5:0/2058195706 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08e402b900 con 0x5626b417ea20 2024-04-24T06:50:50.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.878+0000 7f08dcc00640 1 -- 172.21.15.5:0/2058195706 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f08ec0ce780 con 0x5626b417ea20 2024-04-24T06:50:50.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.878+0000 7f08dcc00640 1 -- 172.21.15.5:0/2058195706 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f08ec082050 con 0x5626b417ea20 2024-04-24T06:50:50.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.878+0000 7f08fc629740 10 client.26791 fetch_fsmap learned FSMap version 27 2024-04-24T06:50:50.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.878+0000 7f08fc629740 1 -- 172.21.15.5:0/2058195706 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5626b409e810 con 0x5626b417ea20 2024-04-24T06:50:50.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.878+0000 7f08dcc00640 1 -- 172.21.15.5:0/2058195706 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08e402b900 con 0x5626b417ea20 2024-04-24T06:50:50.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.878+0000 7f08dcc00640 1 -- 172.21.15.5:0/2058195706 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f08ec0bd0e0 con 0x5626b417ea20 2024-04-24T06:50:50.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.879+0000 7f08fc629740 10 client.26791 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:50:50.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.879+0000 7f08fc629740 10 client.26791 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:50:50.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.879+0000 7f08fc629740 20 client.26791 populate_metadata read hostname 'smithi005' 2024-04-24T06:50:50.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.879+0000 7f08fc629740 1 --2- 172.21.15.5:0/2058195706 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5626b41f5650 0x5626b41f7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:50:50.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.879+0000 7f08fc629740 4 client.26791 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:50:50.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.879+0000 7f08fc629740 1 -- 172.21.15.5:0/2058195706 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5626b40e0a80 con 0x5626b41f5650 2024-04-24T06:50:50.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.879+0000 7f08f2a00640 1 --2- 172.21.15.5:0/2058195706 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5626b41f5650 0x5626b41f7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:50:50.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.880+0000 7f08f2a00640 1 --2- 172.21.15.5:0/2058195706 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5626b41f5650 0x5626b41f7a70 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=876d9b39bc1fa194 server_cookie=15ec574d46fe3d0b in_seq=0 out_seq=0 2024-04-24T06:50:50.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.880+0000 7f08dcc00640 10 client.26791 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:50:50.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.880+0000 7f08dcc00640 1 -- 172.21.15.5:0/2058195706 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x5626b40e0a80 con 0x5626b41f5650 2024-04-24T06:50:50.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.880+0000 7f08dcc00640 10 client.26791 handle_command_reply: tid=0 2024-04-24T06:50:50.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.880+0000 7f08fc629740 1 client.26791 shutdown 2024-04-24T06:50:50.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.880+0000 7f08fc629740 1 -- 172.21.15.5:0/2058195706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626b417ea20 msgr2=0x5626b41926c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:50.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.880+0000 7f08fc629740 1 --2- 172.21.15.5:0/2058195706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626b417ea20 0x5626b41926c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0991b0 tx=0x7f08ec09cec0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.880+0000 7f08fc629740 1 -- 172.21.15.5:0/2058195706 shutdown_connections 2024-04-24T06:50:50.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.880+0000 7f08fc629740 1 --2- 172.21.15.5:0/2058195706 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5626b41f5650 0x5626b41f7a70 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.880+0000 7f08fc629740 1 --2- 172.21.15.5:0/2058195706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626b417ea20 0x5626b41926c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.880+0000 7f08dcc00640 0 client.26791 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:50:50.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.880+0000 7f08fc629740 1 --2- 172.21.15.5:0/2058195706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626b4174350 0x5626b4192180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.880+0000 7f08fc629740 1 --2- 172.21.15.5:0/2058195706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626b4173a30 0x5626b4191c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.880+0000 7f08fc629740 1 -- 172.21.15.5:0/2058195706 >> 172.21.15.5:0/2058195706 conn(0x5626b416fd50 msgr2=0x5626b417c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:50.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.881+0000 7f08fc629740 1 -- 172.21.15.5:0/2058195706 shutdown_connections 2024-04-24T06:50:50.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.881+0000 7f08fc629740 1 -- 172.21.15.5:0/2058195706 wait complete. 2024-04-24T06:50:50.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.881+0000 7f08fc629740 20 client.26791 trim_cache size 0 max 16384 2024-04-24T06:50:50.890 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:50:50.890 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (847 inodes in the stack)", 2024-04-24T06:50:50.890 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-24T06:50:50.891 INFO:teuthology.orchestra.run.smithi005.stdout: "83a47208-daee-4ff2-864d-55a5d112d1dc": { 2024-04-24T06:50:50.891 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-24T06:50:50.891 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "83a47208-daee-4ff2-864d-55a5d112d1dc", 2024-04-24T06:50:50.891 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-04-24T06:50:50.891 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:50:50.891 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:50:50.891 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:50:50.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.881+0000 7f08f3e00640 1 -- 172.21.15.5:0/1862736390 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f08d80a3da0 msgr2=0x7f08d80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:50.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.881+0000 7f08f3e00640 1 --2- 172.21.15.5:0/1862736390 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f08d80a3da0 0x7f08d80a6260 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f08e4009620 tx=0x7f08e4005f90 comp rx=0 tx=0).stop 2024-04-24T06:50:50.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.881+0000 7f08f3e00640 1 -- 172.21.15.5:0/1862736390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08f410c410 msgr2=0x7f08f4101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:50:50.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.881+0000 7f08f3e00640 1 --2- 172.21.15.5:0/1862736390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08f410c410 0x7f08f4101710 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f08ec009fd0 tx=0x7f08ec00ef10 comp rx=0 tx=0).stop 2024-04-24T06:50:50.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.882+0000 7f08f3e00640 1 -- 172.21.15.5:0/1862736390 shutdown_connections 2024-04-24T06:50:50.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.882+0000 7f08f3e00640 1 --2- 172.21.15.5:0/1862736390 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f08d80a3da0 0x7f08d80a6260 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.882+0000 7f08f3e00640 1 --2- 172.21.15.5:0/1862736390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08f410c410 0x7f08f4101710 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.882+0000 7f08f3e00640 1 --2- 172.21.15.5:0/1862736390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08f4103960 0x7f08f41011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.882+0000 7f08f3e00640 1 --2- 172.21.15.5:0/1862736390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08f4103040 0x7f08f4100c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:50:50.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.882+0000 7f08f3e00640 1 -- 172.21.15.5:0/1862736390 >> 172.21.15.5:0/1862736390 conn(0x7f08f4077ab0 msgr2=0x7f08f40fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:50:50.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.882+0000 7f08f3e00640 1 -- 172.21.15.5:0/1862736390 shutdown_connections 2024-04-24T06:50:50.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:50:50.882+0000 7f08f3e00640 1 -- 172.21.15.5:0/1862736390 wait complete. 2024-04-24T06:50:50.904 INFO:tasks.cephfs.filesystem:scrub status for tag:83a47208-daee-4ff2-864d-55a5d112d1dc - {'path': '/', 'tag': '83a47208-daee-4ff2-864d-55a5d112d1dc', 'options': 'recursive,force'} 2024-04-24T06:50:51.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:51 smithi137 ceph-mon[33017]: pgmap v207: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:50:51.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:51 smithi137 ceph-mon[33017]: scrub queued for path: / 2024-04-24T06:50:51.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:51 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:50:51.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:51 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:50:51.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:51 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:50:51.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:51 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:50:51.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:51 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:50:51.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:51 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:50:51.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:51 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:50:51.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:51 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:50:51.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:51 smithi005 ceph-mon[27798]: pgmap v207: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:50:51.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:51 smithi005 ceph-mon[27798]: scrub queued for path: / 2024-04-24T06:50:51.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:51 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:50:51.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:51 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:50:51.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:51 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:50:51.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:51 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:50:51.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:51 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:50:51.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:51 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:50:51.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:51 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:50:51.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:51 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:50:51.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:51 smithi037 ceph-mon[32864]: pgmap v207: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:50:51.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:51 smithi037 ceph-mon[32864]: scrub queued for path: / 2024-04-24T06:50:51.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:51 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:50:51.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:51 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:50:51.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:51 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:50:51.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:51 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:50:51.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:51 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:50:51.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:51 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:50:51.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:51 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:50:51.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:51 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:50:52.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:52 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:50:52.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:52 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:50:52.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:52 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:50:52.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:52 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:50:52.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:52 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:50:52.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:52 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:50:52.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:52 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:50:52.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:52 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:50:52.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:52 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:50:53.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:53 smithi137 ceph-mon[33017]: pgmap v208: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:50:53.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:53 smithi005 ceph-mon[27798]: pgmap v208: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:50:53.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:53 smithi037 ceph-mon[32864]: pgmap v208: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:50:55.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:55 smithi137 ceph-mon[33017]: pgmap v209: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:50:55.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:55 smithi005 ceph-mon[27798]: pgmap v209: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:50:55.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:55 smithi037 ceph-mon[32864]: pgmap v209: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:50:57.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:57 smithi137 ceph-mon[33017]: pgmap v210: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:50:57.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:57 smithi005 ceph-mon[27798]: pgmap v210: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:50:57.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:57 smithi037 ceph-mon[32864]: pgmap v210: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:50:59.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:50:59 smithi137 ceph-mon[33017]: pgmap v211: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:50:59.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:50:59 smithi005 ceph-mon[27798]: pgmap v211: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:50:59.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:50:59 smithi037 ceph-mon[32864]: pgmap v211: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:51:01.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:01 smithi137 ceph-mon[33017]: pgmap v212: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:51:01.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:01 smithi005 ceph-mon[27798]: pgmap v212: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:51:01.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:01 smithi037 ceph-mon[32864]: pgmap v212: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:51:03.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:03 smithi137 ceph-mon[33017]: pgmap v213: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:51:03.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:03 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:51:03.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:03 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:51:03.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:03 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:51:03.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:03 smithi005 ceph-mon[27798]: pgmap v213: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:51:03.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:03 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:51:03.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:03 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:51:03.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:03 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:51:03.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:03 smithi037 ceph-mon[32864]: pgmap v213: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:51:03.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:03 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:51:03.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:03 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:51:03.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:03 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:51:05.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:05 smithi137 ceph-mon[33017]: pgmap v214: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-24T06:51:05.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:05 smithi005 ceph-mon[27798]: pgmap v214: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-24T06:51:05.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:05 smithi037 ceph-mon[32864]: pgmap v214: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-24T06:51:07.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:07 smithi137 ceph-mon[33017]: pgmap v215: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-24T06:51:07.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:07 smithi005 ceph-mon[27798]: pgmap v215: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-24T06:51:07.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:07 smithi037 ceph-mon[32864]: pgmap v215: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-24T06:51:09.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:09 smithi137 ceph-mon[33017]: pgmap v216: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-24T06:51:09.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:09 smithi137 ceph-mon[33017]: osdmap e90: 12 total, 12 up, 12 in 2024-04-24T06:51:09.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:09 smithi005 ceph-mon[27798]: pgmap v216: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-24T06:51:09.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:09 smithi005 ceph-mon[27798]: osdmap e90: 12 total, 12 up, 12 in 2024-04-24T06:51:09.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:09 smithi037 ceph-mon[32864]: pgmap v216: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-24T06:51:09.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:09 smithi037 ceph-mon[32864]: osdmap e90: 12 total, 12 up, 12 in 2024-04-24T06:51:11.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:11 smithi137 ceph-mon[33017]: pgmap v218: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-24T06:51:11.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:11 smithi005 ceph-mon[27798]: pgmap v218: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-24T06:51:11.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:11 smithi037 ceph-mon[32864]: pgmap v218: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-24T06:51:13.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:13 smithi137 ceph-mon[33017]: pgmap v219: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-24T06:51:13.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:13 smithi005 ceph-mon[27798]: pgmap v219: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-24T06:51:13.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:13 smithi037 ceph-mon[32864]: pgmap v219: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-24T06:51:15.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:15 smithi137 ceph-mon[33017]: pgmap v220: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-24T06:51:15.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:15 smithi005 ceph-mon[27798]: pgmap v220: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-24T06:51:15.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:15 smithi037 ceph-mon[32864]: pgmap v220: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-24T06:51:17.445 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:17 smithi137 ceph-mon[33017]: osdmap e91: 12 total, 12 up, 12 in 2024-04-24T06:51:17.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:17 smithi137 ceph-mon[33017]: pgmap v222: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 4 op/s 2024-04-24T06:51:17.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:17 smithi005 ceph-mon[27798]: osdmap e91: 12 total, 12 up, 12 in 2024-04-24T06:51:17.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:17 smithi005 ceph-mon[27798]: pgmap v222: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 4 op/s 2024-04-24T06:51:17.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:17 smithi037 ceph-mon[32864]: osdmap e91: 12 total, 12 up, 12 in 2024-04-24T06:51:17.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:17 smithi037 ceph-mon[32864]: pgmap v222: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 4 op/s 2024-04-24T06:51:19.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:19 smithi005 ceph-mon[27798]: pgmap v223: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 103 B/s wr, 5 op/s 2024-04-24T06:51:19.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:19 smithi037 ceph-mon[32864]: pgmap v223: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 103 B/s wr, 5 op/s 2024-04-24T06:51:19.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:19 smithi137 ceph-mon[33017]: pgmap v223: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 103 B/s wr, 5 op/s 2024-04-24T06:51:20.905 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T06:51:20.906 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T06:51:21.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.130+0000 7f2665400640 1 -- 172.21.15.5:0/2100413581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2660105e10 msgr2=0x7f2660106270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:21.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.130+0000 7f2665400640 1 --2- 172.21.15.5:0/2100413581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2660105e10 0x7f2660106270 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2648009930 tx=0x7f264802a440 comp rx=0 tx=0).stop 2024-04-24T06:51:21.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.130+0000 7f2665400640 1 -- 172.21.15.5:0/2100413581 shutdown_connections 2024-04-24T06:51:21.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.130+0000 7f2665400640 1 --2- 172.21.15.5:0/2100413581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26601067b0 0x7f26601111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.130+0000 7f2665400640 1 --2- 172.21.15.5:0/2100413581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2660105e10 0x7f2660106270 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.130+0000 7f2665400640 1 --2- 172.21.15.5:0/2100413581 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2660104530 0x7f2660104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.130+0000 7f2665400640 1 -- 172.21.15.5:0/2100413581 >> 172.21.15.5:0/2100413581 conn(0x7f26600ffc90 msgr2=0x7f26601020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:21.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.130+0000 7f2665400640 1 -- 172.21.15.5:0/2100413581 shutdown_connections 2024-04-24T06:51:21.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.130+0000 7f2665400640 1 -- 172.21.15.5:0/2100413581 wait complete. 2024-04-24T06:51:21.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.131+0000 7f2665400640 1 Processor -- start 2024-04-24T06:51:21.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.132+0000 7f2665400640 1 -- start start 2024-04-24T06:51:21.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.132+0000 7f2665400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2660104530 0x7f26601a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:21.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.132+0000 7f2665400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2660105e10 0x7f26601a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:21.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.132+0000 7f2665400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26601067b0 0x7f266019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:21.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.132+0000 7f2665400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f26601146f0 con 0x7f26601067b0 2024-04-24T06:51:21.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.132+0000 7f2665400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2660114570 con 0x7f2660105e10 2024-04-24T06:51:21.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.132+0000 7f2665400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2660114870 con 0x7f2660104530 2024-04-24T06:51:21.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.132+0000 7f265f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26601067b0 0x7f266019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:21.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.132+0000 7f265e800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2660104530 0x7f26601a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:21.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.132+0000 7f265e800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2660104530 0x7f26601a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:51998/0 (socket says 172.21.15.5:51998) 2024-04-24T06:51:21.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.132+0000 7f265e800640 1 -- 172.21.15.5:0/2320778257 learned_addr learned my addr 172.21.15.5:0/2320778257 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:21.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.133+0000 7f2657e00640 1 --2- 172.21.15.5:0/2320778257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2660105e10 0x7f26601a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:21.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.133+0000 7f265e800640 1 -- 172.21.15.5:0/2320778257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2660105e10 msgr2=0x7f26601a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:21.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.133+0000 7f265e800640 1 --2- 172.21.15.5:0/2320778257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2660105e10 0x7f26601a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.133+0000 7f265e800640 1 -- 172.21.15.5:0/2320778257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26601067b0 msgr2=0x7f266019a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:21.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.133+0000 7f265e800640 1 --2- 172.21.15.5:0/2320778257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26601067b0 0x7f266019a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.133+0000 7f265e800640 1 -- 172.21.15.5:0/2320778257 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f266019b150 con 0x7f2660104530 2024-04-24T06:51:21.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.133+0000 7f265f200640 1 --2- 172.21.15.5:0/2320778257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26601067b0 0x7f266019a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:51:21.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.133+0000 7f265e800640 1 --2- 172.21.15.5:0/2320778257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2660104530 0x7f26601a0820 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f265000e9f0 tx=0x7f265000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:21.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.133+0000 7f2657400640 1 -- 172.21.15.5:0/2320778257 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265000ccc0 con 0x7f2660104530 2024-04-24T06:51:21.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.134+0000 7f2657400640 1 -- 172.21.15.5:0/2320778257 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f265000ce60 con 0x7f2660104530 2024-04-24T06:51:21.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.134+0000 7f2665400640 1 -- 172.21.15.5:0/2320778257 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f266019b440 con 0x7f2660104530 2024-04-24T06:51:21.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.134+0000 7f2657400640 1 -- 172.21.15.5:0/2320778257 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2650010690 con 0x7f2660104530 2024-04-24T06:51:21.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.134+0000 7f2665400640 1 -- 172.21.15.5:0/2320778257 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26601a76e0 con 0x7f2660104530 2024-04-24T06:51:21.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.137+0000 7f26683da740 1 Processor -- start 2024-04-24T06:51:21.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.137+0000 7f26683da740 1 -- start start 2024-04-24T06:51:21.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.137+0000 7f2657400640 1 -- 172.21.15.5:0/2320778257 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f26500046c0 con 0x7f2660104530 2024-04-24T06:51:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.137+0000 7f26683da740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f35a49860 0x558f35a49c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.137+0000 7f26683da740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f35a4a180 0x558f35a54310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.137+0000 7f26683da740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558f35a54850 0x558f35a56c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.137+0000 7f26683da740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558f359af1f0 con 0x558f35a49860 2024-04-24T06:51:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.137+0000 7f26683da740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558f35955350 con 0x558f35a54850 2024-04-24T06:51:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.137+0000 7f26683da740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558f359ab1b0 con 0x558f35a4a180 2024-04-24T06:51:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.138+0000 7f265f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f35a49860 0x558f35a49c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.138+0000 7f265f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f35a49860 0x558f35a49c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51534/0 (socket says 172.21.15.5:51534) 2024-04-24T06:51:21.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.138+0000 7f265f200640 1 -- 172.21.15.5:0/1937730300 learned_addr learned my addr 172.21.15.5:0/1937730300 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:21.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.138+0000 7f265e800640 1 --2- 172.21.15.5:0/1937730300 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f35a4a180 0x558f35a54310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:21.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.138+0000 7f2657e00640 1 --2- 172.21.15.5:0/1937730300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558f35a54850 0x558f35a56c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:21.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.138+0000 7f265f200640 1 -- 172.21.15.5:0/1937730300 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f35a4a180 msgr2=0x558f35a54310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:21.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.138+0000 7f265f200640 1 --2- 172.21.15.5:0/1937730300 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f35a4a180 0x558f35a54310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.138+0000 7f265f200640 1 -- 172.21.15.5:0/1937730300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558f35a54850 msgr2=0x558f35a56c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:21.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.138+0000 7f265f200640 1 --2- 172.21.15.5:0/1937730300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558f35a54850 0x558f35a56c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.138+0000 7f265f200640 1 -- 172.21.15.5:0/1937730300 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f359a9f50 con 0x558f35a49860 2024-04-24T06:51:21.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.138+0000 7f265f200640 1 --2- 172.21.15.5:0/1937730300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f35a49860 0x558f35a49c40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2658011810 tx=0x7f2658011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f94ab91d4c576ce0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:21.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.139+0000 7f264f400640 1 -- 172.21.15.5:0/1937730300 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2658011310 con 0x558f35a49860 2024-04-24T06:51:21.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.139+0000 7f264f400640 1 -- 172.21.15.5:0/1937730300 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f265800f9a0 con 0x558f35a49860 2024-04-24T06:51:21.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.139+0000 7f264f400640 1 -- 172.21.15.5:0/1937730300 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2658005690 con 0x558f35a49860 2024-04-24T06:51:21.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.139+0000 7f2657400640 1 --2- 172.21.15.5:0/2320778257 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f26440a3da0 0x7f26440a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:21.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.139+0000 7f2657400640 1 -- 172.21.15.5:0/2320778257 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f26500d0bc0 con 0x7f2660104530 2024-04-24T06:51:21.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.139+0000 7f265e800640 1 --2- 172.21.15.5:0/2320778257 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f26440a3da0 0x7f26440a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.140+0000 7f26683da740 1 -- 172.21.15.5:0/1937730300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f35a49860 msgr2=0x558f35a49c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.140+0000 7f26683da740 1 --2- 172.21.15.5:0/1937730300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f35a49860 0x558f35a49c40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2658011810 tx=0x7f2658011cd0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.140+0000 7f26683da740 1 -- 172.21.15.5:0/1937730300 shutdown_connections 2024-04-24T06:51:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.140+0000 7f26683da740 1 --2- 172.21.15.5:0/1937730300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558f35a54850 0x558f35a56c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.140+0000 7f26683da740 1 --2- 172.21.15.5:0/1937730300 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f35a4a180 0x558f35a54310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.140+0000 7f26683da740 1 --2- 172.21.15.5:0/1937730300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f35a49860 0x558f35a49c40 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.140+0000 7f26683da740 1 -- 172.21.15.5:0/1937730300 >> 172.21.15.5:0/1937730300 conn(0x558f35a45b80 msgr2=0x558f35a45f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.140+0000 7f26683da740 1 -- 172.21.15.5:0/1937730300 shutdown_connections 2024-04-24T06:51:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.140+0000 7f26683da740 1 -- 172.21.15.5:0/1937730300 wait complete. 2024-04-24T06:51:21.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.140+0000 7f265e800640 1 --2- 172.21.15.5:0/2320778257 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f26440a3da0 0x7f26440a6260 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x558f35a4ae20 tx=0x7f26500d4670 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:21.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.140+0000 7f26683da740 1 -- start start 2024-04-24T06:51:21.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.141+0000 7f26683da740 1 Processor -- start 2024-04-24T06:51:21.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.141+0000 7f26683da740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f35a49860 0x558f35a517c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:21.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.141+0000 7f26683da740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f35a4a180 0x558f35a51d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:21.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.141+0000 7f26683da740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558f35a54850 0x558f35a52240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:21.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.141+0000 7f26683da740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558f35a57640 con 0x558f35a49860 2024-04-24T06:51:21.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.141+0000 7f26683da740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558f35968690 con 0x558f35a54850 2024-04-24T06:51:21.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.141+0000 7f26683da740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558f35a577c0 con 0x558f35a4a180 2024-04-24T06:51:21.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.141+0000 7f26683da740 10 client.0 _ll_register_callbacks cb 0x558f359a4650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:51:21.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.141+0000 7f265f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f35a49860 0x558f35a517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:21.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.141+0000 7f265f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f35a49860 0x558f35a517c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51540/0 (socket says 172.21.15.5:51540) 2024-04-24T06:51:21.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.141+0000 7f265f200640 1 -- 172.21.15.5:0/268320492 learned_addr learned my addr 172.21.15.5:0/268320492 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:21.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.142+0000 7f2657e00640 1 --2- 172.21.15.5:0/268320492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f35a4a180 0x558f35a51d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:21.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.142+0000 7f265f200640 1 --2- 172.21.15.5:0/268320492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558f35a54850 0x558f35a52240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:21.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.142+0000 7f265f200640 1 -- 172.21.15.5:0/268320492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f35a4a180 msgr2=0x558f35a51d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:21.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.142+0000 7f265f200640 1 --2- 172.21.15.5:0/268320492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f35a4a180 0x558f35a51d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.142+0000 7f265f200640 1 -- 172.21.15.5:0/268320492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558f35a54850 msgr2=0x558f35a52240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:21.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.142+0000 7f265f200640 1 --2- 172.21.15.5:0/268320492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558f35a54850 0x558f35a52240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.142+0000 7f265f200640 1 -- 172.21.15.5:0/268320492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f35967920 con 0x558f35a49860 2024-04-24T06:51:21.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.142+0000 7f265f200640 1 --2- 172.21.15.5:0/268320492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f35a49860 0x558f35a517c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2658011550 tx=0x7f26580052b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5702370f48d58e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:21.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.142+0000 7f2643e00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:51:21.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.142+0000 7f2643e00640 1 -- 172.21.15.5:0/268320492 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2658003b70 con 0x558f35a49860 2024-04-24T06:51:21.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.143+0000 7f26683da740 1 -- 172.21.15.5:0/268320492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f265800f9a0 con 0x558f35a49860 2024-04-24T06:51:21.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.143+0000 7f2643e00640 1 -- 172.21.15.5:0/268320492 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f2658003d10 con 0x558f35a49860 2024-04-24T06:51:21.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.143+0000 7f2643e00640 1 -- 172.21.15.5:0/268320492 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2658004b40 con 0x558f35a49860 2024-04-24T06:51:21.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.143+0000 7f26683da740 10 client.16659 fetch_fsmap learned FSMap version 27 2024-04-24T06:51:21.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.143+0000 7f26683da740 1 -- 172.21.15.5:0/268320492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558f3578b7d0 con 0x558f35a49860 2024-04-24T06:51:21.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.143+0000 7f2643e00640 1 -- 172.21.15.5:0/268320492 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f265800f9a0 con 0x558f35a49860 2024-04-24T06:51:21.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.143+0000 7f2643e00640 1 -- 172.21.15.5:0/268320492 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f2658022070 con 0x558f35a49860 2024-04-24T06:51:21.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.143+0000 7f26683da740 10 client.16659 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:51:21.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.143+0000 7f26683da740 10 client.16659 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:51:21.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.143+0000 7f26683da740 20 client.16659 populate_metadata read hostname 'smithi005' 2024-04-24T06:51:21.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.143+0000 7f26683da740 1 --2- 172.21.15.5:0/268320492 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x558f35a64ae0 0x558f35a66ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:21.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.143+0000 7f26683da740 4 client.16659 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:51:21.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.143+0000 7f26683da740 1 -- 172.21.15.5:0/268320492 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f359bcc10 con 0x558f35a64ae0 2024-04-24T06:51:21.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.144+0000 7f2657e00640 1 --2- 172.21.15.5:0/268320492 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x558f35a64ae0 0x558f35a66ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:21.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.144+0000 7f2657e00640 1 --2- 172.21.15.5:0/268320492 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x558f35a64ae0 0x558f35a66ed0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a547167b27317da server_cookie=5951201554dfa8ee in_seq=0 out_seq=0 2024-04-24T06:51:21.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.144+0000 7f2643e00640 10 client.16659 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:51:21.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.145+0000 7f2643e00640 1 -- 172.21.15.5:0/268320492 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558f359bcc10 con 0x558f35a64ae0 2024-04-24T06:51:21.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.145+0000 7f2643e00640 10 client.16659 handle_command_reply: tid=0 2024-04-24T06:51:21.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.145+0000 7f26683da740 1 client.16659 shutdown 2024-04-24T06:51:21.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.146+0000 7f26683da740 1 -- 172.21.15.5:0/268320492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f35a49860 msgr2=0x558f35a517c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:21.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.146+0000 7f26683da740 1 --2- 172.21.15.5:0/268320492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f35a49860 0x558f35a517c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2658011550 tx=0x7f26580052b0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.146+0000 7f26683da740 1 -- 172.21.15.5:0/268320492 shutdown_connections 2024-04-24T06:51:21.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.146+0000 7f26683da740 1 --2- 172.21.15.5:0/268320492 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x558f35a64ae0 0x558f35a66ed0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.146+0000 7f26683da740 1 --2- 172.21.15.5:0/268320492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558f35a54850 0x558f35a52240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.146+0000 7f26683da740 1 --2- 172.21.15.5:0/268320492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f35a4a180 0x558f35a51d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.146+0000 7f26683da740 1 --2- 172.21.15.5:0/268320492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f35a49860 0x558f35a517c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.146+0000 7f2643e00640 0 client.16659 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:51:21.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.146+0000 7f26683da740 1 -- 172.21.15.5:0/268320492 >> 172.21.15.5:0/268320492 conn(0x558f35a45b80 msgr2=0x558f35a4b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:21.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.146+0000 7f26683da740 1 -- 172.21.15.5:0/268320492 shutdown_connections 2024-04-24T06:51:21.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.146+0000 7f26683da740 1 -- 172.21.15.5:0/268320492 wait complete. 2024-04-24T06:51:21.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.146+0000 7f26683da740 20 client.16659 trim_cache size 0 max 16384 2024-04-24T06:51:21.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.166+0000 7f26683da740 1 Processor -- start 2024-04-24T06:51:21.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.166+0000 7f26683da740 1 -- start start 2024-04-24T06:51:21.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.166+0000 7f26683da740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f35a49860 0x558f35a60500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:21.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.166+0000 7f26683da740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f35a4a180 0x558f35a4b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:21.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.166+0000 7f26683da740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558f35a54850 0x558f35a6c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:21.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.166+0000 7f26683da740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f26601b2840 con 0x558f35a49860 2024-04-24T06:51:21.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.166+0000 7f26683da740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f26601b0110 con 0x558f35a54850 2024-04-24T06:51:21.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.166+0000 7f26683da740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f26601adf80 con 0x558f35a4a180 2024-04-24T06:51:21.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.166+0000 7f265f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f35a49860 0x558f35a60500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:21.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.166+0000 7f265f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f35a49860 0x558f35a60500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51554/0 (socket says 172.21.15.5:51554) 2024-04-24T06:51:21.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.166+0000 7f265f200640 1 -- 172.21.15.5:0/1068440315 learned_addr learned my addr 172.21.15.5:0/1068440315 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:21.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.167+0000 7f2657e00640 1 --2- 172.21.15.5:0/1068440315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f35a4a180 0x558f35a4b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:21.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.167+0000 7f265f200640 1 --2- 172.21.15.5:0/1068440315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558f35a54850 0x558f35a6c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:21.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.167+0000 7f265f200640 1 -- 172.21.15.5:0/1068440315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f35a4a180 msgr2=0x558f35a4b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:21.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.167+0000 7f265f200640 1 --2- 172.21.15.5:0/1068440315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f35a4a180 0x558f35a4b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.167+0000 7f265f200640 1 -- 172.21.15.5:0/1068440315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558f35a54850 msgr2=0x558f35a6c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:21.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.167+0000 7f265f200640 1 --2- 172.21.15.5:0/1068440315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558f35a54850 0x558f35a6c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.167+0000 7f265f200640 1 -- 172.21.15.5:0/1068440315 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f356c13f0 con 0x558f35a49860 2024-04-24T06:51:21.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.167+0000 7f2657e00640 1 --2- 172.21.15.5:0/1068440315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f35a4a180 0x558f35a4b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:51:21.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.167+0000 7f265f200640 1 --2- 172.21.15.5:0/1068440315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f35a49860 0x558f35a60500 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2658011220 tx=0x7f2658009b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda429f452ef0de4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:21.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.168+0000 7f264ea00640 1 -- 172.21.15.5:0/1068440315 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2658004ee0 con 0x558f35a49860 2024-04-24T06:51:21.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.168+0000 7f264ea00640 1 -- 172.21.15.5:0/1068440315 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f265801f040 con 0x558f35a49860 2024-04-24T06:51:21.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.168+0000 7f264ea00640 1 -- 172.21.15.5:0/1068440315 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26580046a0 con 0x558f35a49860 2024-04-24T06:51:21.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.168+0000 7f26683da740 1 -- 172.21.15.5:0/1068440315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f35a49860 msgr2=0x558f35a60500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:21.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.168+0000 7f26683da740 1 --2- 172.21.15.5:0/1068440315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f35a49860 0x558f35a60500 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2658011220 tx=0x7f2658009b10 comp rx=0 tx=0).stop 2024-04-24T06:51:21.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.168+0000 7f26683da740 1 -- 172.21.15.5:0/1068440315 shutdown_connections 2024-04-24T06:51:21.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.168+0000 7f26683da740 1 --2- 172.21.15.5:0/1068440315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558f35a54850 0x558f35a6c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.168+0000 7f26683da740 1 --2- 172.21.15.5:0/1068440315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f35a4a180 0x558f35a4b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.168+0000 7f26683da740 1 --2- 172.21.15.5:0/1068440315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f35a49860 0x558f35a60500 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.168+0000 7f26683da740 1 -- 172.21.15.5:0/1068440315 >> 172.21.15.5:0/1068440315 conn(0x558f35a45b80 msgr2=0x558f35a4b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:21.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.168+0000 7f26683da740 1 -- 172.21.15.5:0/1068440315 shutdown_connections 2024-04-24T06:51:21.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.168+0000 7f26683da740 1 -- 172.21.15.5:0/1068440315 wait complete. 2024-04-24T06:51:21.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.169+0000 7f26683da740 1 -- start start 2024-04-24T06:51:21.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.169+0000 7f26683da740 1 Processor -- start 2024-04-24T06:51:21.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.169+0000 7f26683da740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f35a49860 0x558f35a5cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:21.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.169+0000 7f26683da740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f35a4a180 0x558f35a4e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:21.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.169+0000 7f26683da740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558f35a54850 0x558f35a4e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:21.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.170+0000 7f26683da740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f26601ae5d0 con 0x558f35a49860 2024-04-24T06:51:21.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.170+0000 7f26683da740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f26601afa40 con 0x558f35a54850 2024-04-24T06:51:21.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.170+0000 7f26683da740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f26601aeca0 con 0x558f35a4a180 2024-04-24T06:51:21.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.170+0000 7f26683da740 10 client.0 _ll_register_callbacks cb 0x7f2650005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:51:21.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.170+0000 7f265f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f35a49860 0x558f35a5cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:21.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.170+0000 7f2657e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f35a4a180 0x558f35a4e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:21.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.170+0000 7f2657e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f35a4a180 0x558f35a4e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:52026/0 (socket says 172.21.15.5:52026) 2024-04-24T06:51:21.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.170+0000 7f265f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f35a49860 0x558f35a5cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51556/0 (socket says 172.21.15.5:51556) 2024-04-24T06:51:21.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.170+0000 7f2657e00640 1 -- 172.21.15.5:0/418591448 learned_addr learned my addr 172.21.15.5:0/418591448 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:21.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.170+0000 7f265f200640 1 --2- 172.21.15.5:0/418591448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558f35a54850 0x558f35a4e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:21.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.170+0000 7f2657e00640 1 -- 172.21.15.5:0/418591448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558f35a54850 msgr2=0x558f35a4e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:21.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.170+0000 7f2657e00640 1 --2- 172.21.15.5:0/418591448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558f35a54850 0x558f35a4e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.170+0000 7f2657e00640 1 -- 172.21.15.5:0/418591448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f35a49860 msgr2=0x558f35a5cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:21.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.170+0000 7f2657e00640 1 --2- 172.21.15.5:0/418591448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f35a49860 0x558f35a5cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.170+0000 7f2657e00640 1 -- 172.21.15.5:0/418591448 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f356810f0 con 0x558f35a4a180 2024-04-24T06:51:21.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.170+0000 7f265f200640 1 --2- 172.21.15.5:0/418591448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f35a49860 0x558f35a5cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:51:21.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.171+0000 7f2657e00640 1 --2- 172.21.15.5:0/418591448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f35a4a180 0x558f35a4e0b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2648005c20 tx=0x7f2648004560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74b99e5bb9b29a92 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:21.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.171+0000 7f2643e00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:51:21.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.171+0000 7f2643e00640 1 -- 172.21.15.5:0/418591448 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2648035040 con 0x558f35a4a180 2024-04-24T06:51:21.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.171+0000 7f2643e00640 1 -- 172.21.15.5:0/418591448 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f2648005240 con 0x558f35a4a180 2024-04-24T06:51:21.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.171+0000 7f26683da740 1 -- 172.21.15.5:0/418591448 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f265801f040 con 0x558f35a4a180 2024-04-24T06:51:21.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.171+0000 7f2643e00640 1 -- 172.21.15.5:0/418591448 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2648005440 con 0x558f35a4a180 2024-04-24T06:51:21.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.171+0000 7f26683da740 10 client.26809 fetch_fsmap learned FSMap version 27 2024-04-24T06:51:21.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.171+0000 7f26683da740 1 -- 172.21.15.5:0/418591448 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558f35974810 con 0x558f35a4a180 2024-04-24T06:51:21.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.172+0000 7f2643e00640 1 -- 172.21.15.5:0/418591448 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f265801f040 con 0x558f35a4a180 2024-04-24T06:51:21.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.172+0000 7f2643e00640 1 -- 172.21.15.5:0/418591448 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f2648004920 con 0x558f35a4a180 2024-04-24T06:51:21.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.172+0000 7f26683da740 10 client.26809 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:51:21.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.172+0000 7f26683da740 10 client.26809 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:51:21.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.172+0000 7f26683da740 20 client.26809 populate_metadata read hostname 'smithi005' 2024-04-24T06:51:21.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.172+0000 7f26683da740 1 --2- 172.21.15.5:0/418591448 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x558f35ad39a0 0x558f35ad5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:21.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.172+0000 7f26683da740 4 client.26809 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:51:21.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.172+0000 7f26683da740 1 -- 172.21.15.5:0/418591448 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558f359b6a80 con 0x558f35ad39a0 2024-04-24T06:51:21.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.172+0000 7f265f200640 1 --2- 172.21.15.5:0/418591448 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x558f35ad39a0 0x558f35ad5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:21.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.173+0000 7f265f200640 1 --2- 172.21.15.5:0/418591448 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x558f35ad39a0 0x558f35ad5dc0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4757782661c57cc2 server_cookie=7b9d80925453e8e4 in_seq=0 out_seq=0 2024-04-24T06:51:21.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.173+0000 7f2643e00640 10 client.26809 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:51:21.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.173+0000 7f2643e00640 1 -- 172.21.15.5:0/418591448 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558f359b6a80 con 0x558f35ad39a0 2024-04-24T06:51:21.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.173+0000 7f2643e00640 10 client.26809 handle_command_reply: tid=0 2024-04-24T06:51:21.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.173+0000 7f26683da740 1 client.26809 shutdown 2024-04-24T06:51:21.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.173+0000 7f26683da740 1 -- 172.21.15.5:0/418591448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f35a4a180 msgr2=0x558f35a4e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:21.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.173+0000 7f26683da740 1 --2- 172.21.15.5:0/418591448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f35a4a180 0x558f35a4e0b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2648005c20 tx=0x7f2648004560 comp rx=0 tx=0).stop 2024-04-24T06:51:21.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.173+0000 7f26683da740 1 -- 172.21.15.5:0/418591448 shutdown_connections 2024-04-24T06:51:21.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.173+0000 7f26683da740 1 --2- 172.21.15.5:0/418591448 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x558f35ad39a0 0x558f35ad5dc0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.173+0000 7f26683da740 1 --2- 172.21.15.5:0/418591448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558f35a54850 0x558f35a4e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.173+0000 7f26683da740 1 --2- 172.21.15.5:0/418591448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f35a4a180 0x558f35a4e0b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.173+0000 7f26683da740 1 --2- 172.21.15.5:0/418591448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f35a49860 0x558f35a5cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.173+0000 7f2643e00640 0 client.26809 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:51:21.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.173+0000 7f26683da740 1 -- 172.21.15.5:0/418591448 >> 172.21.15.5:0/418591448 conn(0x558f35a45b80 msgr2=0x558f35a52010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:21.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.173+0000 7f26683da740 1 -- 172.21.15.5:0/418591448 shutdown_connections 2024-04-24T06:51:21.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.173+0000 7f26683da740 1 -- 172.21.15.5:0/418591448 wait complete. 2024-04-24T06:51:21.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.173+0000 7f26683da740 20 client.26809 trim_cache size 0 max 16384 2024-04-24T06:51:21.181 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:51:21.181 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-24T06:51:21.181 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-24T06:51:21.181 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:51:21.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.174+0000 7f2665400640 1 -- 172.21.15.5:0/2320778257 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f26440a3da0 msgr2=0x7f26440a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:21.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.174+0000 7f2665400640 1 --2- 172.21.15.5:0/2320778257 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f26440a3da0 0x7f26440a6260 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x558f35a4ae20 tx=0x7f26500d4670 comp rx=0 tx=0).stop 2024-04-24T06:51:21.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.174+0000 7f2665400640 1 -- 172.21.15.5:0/2320778257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2660104530 msgr2=0x7f26601a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:21.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.174+0000 7f2665400640 1 --2- 172.21.15.5:0/2320778257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2660104530 0x7f26601a0820 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f265000e9f0 tx=0x7f265000edf0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.174+0000 7f2665400640 1 -- 172.21.15.5:0/2320778257 shutdown_connections 2024-04-24T06:51:21.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.174+0000 7f2665400640 1 --2- 172.21.15.5:0/2320778257 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f26440a3da0 0x7f26440a6260 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.174+0000 7f2665400640 1 --2- 172.21.15.5:0/2320778257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26601067b0 0x7f266019a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.174+0000 7f2665400640 1 --2- 172.21.15.5:0/2320778257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2660105e10 0x7f26601a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.174+0000 7f2665400640 1 --2- 172.21.15.5:0/2320778257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2660104530 0x7f26601a0820 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.174+0000 7f2665400640 1 -- 172.21.15.5:0/2320778257 >> 172.21.15.5:0/2320778257 conn(0x7f26600ffc90 msgr2=0x7f26601002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:21.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.174+0000 7f2665400640 1 -- 172.21.15.5:0/2320778257 shutdown_connections 2024-04-24T06:51:21.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.174+0000 7f2665400640 1 -- 172.21.15.5:0/2320778257 wait complete. 2024-04-24T06:51:21.192 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T06:51:21.193 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:51:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.405+0000 7f031ea00640 1 -- 172.21.15.5:0/1643547764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0318104350 msgr2=0x7f0318104730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.405+0000 7f031ea00640 1 --2- 172.21.15.5:0/1643547764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0318104350 0x7f0318104730 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0310009900 tx=0x7f031002a3f0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.405+0000 7f031ea00640 1 -- 172.21.15.5:0/1643547764 shutdown_connections 2024-04-24T06:51:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.405+0000 7f031ea00640 1 --2- 172.21.15.5:0/1643547764 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0318109420 0x7f0318110ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.405+0000 7f031ea00640 1 --2- 172.21.15.5:0/1643547764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0318104d00 0x7f0318108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.405+0000 7f031ea00640 1 --2- 172.21.15.5:0/1643547764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0318104350 0x7f0318104730 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.405+0000 7f031ea00640 1 -- 172.21.15.5:0/1643547764 >> 172.21.15.5:0/1643547764 conn(0x7f03180ffb70 msgr2=0x7f0318101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:21.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.405+0000 7f031ea00640 1 -- 172.21.15.5:0/1643547764 shutdown_connections 2024-04-24T06:51:21.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.405+0000 7f031ea00640 1 -- 172.21.15.5:0/1643547764 wait complete. 2024-04-24T06:51:21.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.406+0000 7f031ea00640 1 Processor -- start 2024-04-24T06:51:21.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.406+0000 7f031ea00640 1 -- start start 2024-04-24T06:51:21.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.407+0000 7f031ea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0318104350 0x7f03181a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:21.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.407+0000 7f031ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0318109420 0x7f03181a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:21.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.407+0000 7f031ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f031819a830 0x7f031819ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:21.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.407+0000 7f031ea00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0318114530 con 0x7f031819a830 2024-04-24T06:51:21.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.407+0000 7f031ea00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f03181143b0 con 0x7f0318109420 2024-04-24T06:51:21.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.407+0000 7f031ea00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f03181146b0 con 0x7f0318104350 2024-04-24T06:51:21.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.407+0000 7f031e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f031819a830 0x7f031819ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:21.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.407+0000 7f031e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f031819a830 0x7f031819ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51584/0 (socket says 172.21.15.5:51584) 2024-04-24T06:51:21.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.407+0000 7f031e000640 1 -- 172.21.15.5:0/2633919844 learned_addr learned my addr 172.21.15.5:0/2633919844 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:21.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.407+0000 7f031cc00640 1 --2- 172.21.15.5:0/2633919844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0318109420 0x7f03181a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:21.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.407+0000 7f031d600640 1 --2- 172.21.15.5:0/2633919844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0318104350 0x7f03181a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.408+0000 7f031e000640 1 -- 172.21.15.5:0/2633919844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0318104350 msgr2=0x7f03181a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.408+0000 7f031e000640 1 --2- 172.21.15.5:0/2633919844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0318104350 0x7f03181a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.408+0000 7f031e000640 1 -- 172.21.15.5:0/2633919844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0318109420 msgr2=0x7f03181a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.408+0000 7f031e000640 1 --2- 172.21.15.5:0/2633919844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0318109420 0x7f03181a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.408+0000 7f031e000640 1 -- 172.21.15.5:0/2633919844 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f031819b3c0 con 0x7f031819a830 2024-04-24T06:51:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.408+0000 7f031e000640 1 --2- 172.21.15.5:0/2633919844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f031819a830 0x7f031819ace0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f030800eb10 tx=0x7f030800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.408+0000 7f0316000640 1 -- 172.21.15.5:0/2633919844 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0308004b20 con 0x7f031819a830 2024-04-24T06:51:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.408+0000 7f031ea00640 1 -- 172.21.15.5:0/2633919844 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f031819b6b0 con 0x7f031819a830 2024-04-24T06:51:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.408+0000 7f0316000640 1 -- 172.21.15.5:0/2633919844 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0308004cc0 con 0x7f031819a830 2024-04-24T06:51:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.408+0000 7f0316000640 1 -- 172.21.15.5:0/2633919844 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0308010760 con 0x7f031819a830 2024-04-24T06:51:21.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.408+0000 7f031ea00640 1 -- 172.21.15.5:0/2633919844 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03181a7890 con 0x7f031819a830 2024-04-24T06:51:21.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.410+0000 7f031ea00640 1 -- 172.21.15.5:0/2633919844 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0318105820 con 0x7f031819a830 2024-04-24T06:51:21.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.412+0000 7f0316000640 1 -- 172.21.15.5:0/2633919844 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f0308005000 con 0x7f031819a830 2024-04-24T06:51:21.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.413+0000 7f0316000640 1 --2- 172.21.15.5:0/2633919844 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f02fc0a3d10 0x7f02fc0a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:21.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.413+0000 7f0316000640 1 -- 172.21.15.5:0/2633919844 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f03080d1240 con 0x7f031819a830 2024-04-24T06:51:21.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.414+0000 7f031d600640 1 --2- 172.21.15.5:0/2633919844 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f02fc0a3d10 0x7f02fc0a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:21.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.414+0000 7f031d600640 1 --2- 172.21.15.5:0/2633919844 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f02fc0a3d10 0x7f02fc0a61d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0310000c00 tx=0x7f0310005b50 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:21.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.416+0000 7f0316000640 1 -- 172.21.15.5:0/2633919844 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f031819b6b0 con 0x7f031819a830 2024-04-24T06:51:21.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:21 smithi005 ceph-mon[27798]: pgmap v224: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-24T06:51:21.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:21 smithi037 ceph-mon[32864]: pgmap v224: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-24T06:51:21.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.663+0000 7f031ea00640 1 -- 172.21.15.5:0/2633919844 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f031819bd60 con 0x7f031819a830 2024-04-24T06:51:21.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.664+0000 7f0316000640 1 -- 172.21.15.5:0/2633919844 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11856 (secure 0 0 0) 0x7f031819bd60 con 0x7f031819a830 2024-04-24T06:51:21.665 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:51:21.665 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1735462750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1735462750},{"type":"v1","addr":"172.21.15.5:6837","nonce":1735462750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1499469370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1499469370},{"type":"v1","addr":"172.21.15.5:6841","nonce":1499469370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3705638847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3705638847},{"type":"v1","addr":"172.21.15.37:6835","nonce":3705638847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T06:41:34.764490+0000","modified":"2024-04-24T06:48:57.768410+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24550,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6837/1390103296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1390103296},{"type":"v1","addr":"172.21.15.37:6837","nonce":1390103296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3217754215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3217754215},{"type":"v1","addr":"172.21.15.137:6835","nonce":3217754215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6837/3044222976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3044222976},{"type":"v1","addr":"172.21.15.137:6837","nonce":3044222976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.5:6839/3173201459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3173201459},{"type":"v1","addr":"172.21.15.5:6839","nonce":3173201459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6839/1889774232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1889774232},{"type":"v1","addr":"172.21.15.137:6839","nonce":1889774232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6839/3536721753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":3536721753},{"type":"v1","addr":"172.21.15.37:6839","nonce":3536721753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24550,24569,24539]},"id":1}]} 2024-04-24T06:51:21.666 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-04-24T06:51:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.667+0000 7f031ea00640 1 -- 172.21.15.5:0/2633919844 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f02fc0a3d10 msgr2=0x7f02fc0a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.667+0000 7f031ea00640 1 --2- 172.21.15.5:0/2633919844 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f02fc0a3d10 0x7f02fc0a61d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0310000c00 tx=0x7f0310005b50 comp rx=0 tx=0).stop 2024-04-24T06:51:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.667+0000 7f031ea00640 1 -- 172.21.15.5:0/2633919844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f031819a830 msgr2=0x7f031819ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.667+0000 7f031ea00640 1 --2- 172.21.15.5:0/2633919844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f031819a830 0x7f031819ace0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f030800eb10 tx=0x7f030800c2d0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.667+0000 7f031ea00640 1 -- 172.21.15.5:0/2633919844 shutdown_connections 2024-04-24T06:51:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.667+0000 7f031ea00640 1 --2- 172.21.15.5:0/2633919844 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f02fc0a3d10 0x7f02fc0a61d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.667+0000 7f031ea00640 1 --2- 172.21.15.5:0/2633919844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f031819a830 0x7f031819ace0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.667+0000 7f031ea00640 1 --2- 172.21.15.5:0/2633919844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0318109420 0x7f03181a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.667+0000 7f031ea00640 1 --2- 172.21.15.5:0/2633919844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0318104350 0x7f03181a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.667+0000 7f031ea00640 1 -- 172.21.15.5:0/2633919844 >> 172.21.15.5:0/2633919844 conn(0x7f03180ffb70 msgr2=0x7f0318101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.667+0000 7f031ea00640 1 -- 172.21.15.5:0/2633919844 shutdown_connections 2024-04-24T06:51:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.667+0000 7f031ea00640 1 -- 172.21.15.5:0/2633919844 wait complete. 2024-04-24T06:51:21.688 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T06:51:21.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:21 smithi137 ceph-mon[33017]: pgmap v224: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-24T06:51:21.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.896+0000 7f92a4a00640 1 -- 172.21.15.5:0/2718438358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92a0108540 msgr2=0x7f92a01089a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:21.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.896+0000 7f92a4a00640 1 --2- 172.21.15.5:0/2718438358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92a0108540 0x7f92a01089a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9288009890 tx=0x7f928802a3f0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.896+0000 7f92a4a00640 1 -- 172.21.15.5:0/2718438358 shutdown_connections 2024-04-24T06:51:21.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.896+0000 7f92a4a00640 1 --2- 172.21.15.5:0/2718438358 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92a0108ee0 0x7f92a01152b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.896+0000 7f92a4a00640 1 --2- 172.21.15.5:0/2718438358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92a0108540 0x7f92a01089a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.896+0000 7f92a4a00640 1 --2- 172.21.15.5:0/2718438358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92a010e4f0 0x7f92a010e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.896+0000 7f92a4a00640 1 -- 172.21.15.5:0/2718438358 >> 172.21.15.5:0/2718438358 conn(0x7f92a0077e10 msgr2=0x7f92a0106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:21.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.896+0000 7f92a4a00640 1 -- 172.21.15.5:0/2718438358 shutdown_connections 2024-04-24T06:51:21.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.896+0000 7f92a4a00640 1 -- 172.21.15.5:0/2718438358 wait complete. 2024-04-24T06:51:21.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.897+0000 7f92a4a00640 1 Processor -- start 2024-04-24T06:51:21.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.897+0000 7f92a4a00640 1 -- start start 2024-04-24T06:51:21.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.898+0000 7f92a4a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92a0108540 0x7f92a019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:21.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.898+0000 7f92a4a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92a0108ee0 0x7f92a019f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:21.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.898+0000 7f92a4a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92a010e4f0 0x7f92a01a3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:21.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.898+0000 7f92a4a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f92a01187a0 con 0x7f92a0108540 2024-04-24T06:51:21.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.898+0000 7f92a4a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f92a0118620 con 0x7f92a010e4f0 2024-04-24T06:51:21.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.898+0000 7f92a4a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f92a0118920 con 0x7f92a0108ee0 2024-04-24T06:51:21.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.898+0000 7f929e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92a0108540 0x7f92a019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:21.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.898+0000 7f929e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92a0108540 0x7f92a019ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51596/0 (socket says 172.21.15.5:51596) 2024-04-24T06:51:21.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.898+0000 7f929e800640 1 -- 172.21.15.5:0/64096492 learned_addr learned my addr 172.21.15.5:0/64096492 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:21.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.898+0000 7f929f200640 1 --2- 172.21.15.5:0/64096492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92a010e4f0 0x7f92a01a3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:21.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.898+0000 7f9297e00640 1 --2- 172.21.15.5:0/64096492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92a0108ee0 0x7f92a019f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:21.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.899+0000 7f929e800640 1 -- 172.21.15.5:0/64096492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92a0108ee0 msgr2=0x7f92a019f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:21.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.899+0000 7f929e800640 1 --2- 172.21.15.5:0/64096492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92a0108ee0 0x7f92a019f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.899+0000 7f929e800640 1 -- 172.21.15.5:0/64096492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92a010e4f0 msgr2=0x7f92a01a3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:21.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.899+0000 7f929e800640 1 --2- 172.21.15.5:0/64096492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92a010e4f0 0x7f92a01a3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.899+0000 7f929e800640 1 -- 172.21.15.5:0/64096492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a01a3d50 con 0x7f92a0108540 2024-04-24T06:51:21.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.899+0000 7f929e800640 1 --2- 172.21.15.5:0/64096492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92a0108540 0x7f92a019ed60 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f929000e9c0 tx=0x7f929000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:21.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.899+0000 7f9297400640 1 -- 172.21.15.5:0/64096492 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929000ccc0 con 0x7f92a0108540 2024-04-24T06:51:21.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.899+0000 7f9297400640 1 -- 172.21.15.5:0/64096492 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9290005500 con 0x7f92a0108540 2024-04-24T06:51:21.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.899+0000 7f92a4a00640 1 -- 172.21.15.5:0/64096492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92a01a4040 con 0x7f92a0108540 2024-04-24T06:51:21.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.899+0000 7f9297400640 1 -- 172.21.15.5:0/64096492 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9290010430 con 0x7f92a0108540 2024-04-24T06:51:21.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.899+0000 7f92a4a00640 1 -- 172.21.15.5:0/64096492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92a01ab8d0 con 0x7f92a0108540 2024-04-24T06:51:21.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.903+0000 7f92a836c740 1 Processor -- start 2024-04-24T06:51:21.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.903+0000 7f92a836c740 1 -- start start 2024-04-24T06:51:21.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.903+0000 7f9297400640 1 -- 172.21.15.5:0/64096492 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f92900046c0 con 0x7f92a0108540 2024-04-24T06:51:21.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.903+0000 7f92a836c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557273beb40 0x5557273bef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:21.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.903+0000 7f92a836c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557273bf460 0x5557273c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:21.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.903+0000 7f92a836c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557273c9b30 0x5557273cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:21.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.903+0000 7f92a836c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5557273241d0 con 0x5557273c9b30 2024-04-24T06:51:21.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.903+0000 7f92a836c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5557272ca450 con 0x5557273bf460 2024-04-24T06:51:21.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.903+0000 7f92a836c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555727320190 con 0x5557273beb40 2024-04-24T06:51:21.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.903+0000 7f9297e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557273c9b30 0x5557273cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:21.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.903+0000 7f9297e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557273c9b30 0x5557273cbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51612/0 (socket says 172.21.15.5:51612) 2024-04-24T06:51:21.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.903+0000 7f9297e00640 1 -- 172.21.15.5:0/509572847 learned_addr learned my addr 172.21.15.5:0/509572847 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:21.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.903+0000 7f929f200640 1 --2- 172.21.15.5:0/509572847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557273beb40 0x5557273bef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:21.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.904+0000 7f929e800640 1 --2- 172.21.15.5:0/509572847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557273bf460 0x5557273c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:21.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.904+0000 7f929f200640 1 -- 172.21.15.5:0/509572847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557273bf460 msgr2=0x5557273c95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:21.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.904+0000 7f929f200640 1 --2- 172.21.15.5:0/509572847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557273bf460 0x5557273c95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.904+0000 7f929f200640 1 -- 172.21.15.5:0/509572847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557273c9b30 msgr2=0x5557273cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:21.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.904+0000 7f929f200640 1 --2- 172.21.15.5:0/509572847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557273c9b30 0x5557273cbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.904+0000 7f929f200640 1 -- 172.21.15.5:0/509572847 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555727387310 con 0x5557273beb40 2024-04-24T06:51:21.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.904+0000 7f9297e00640 1 --2- 172.21.15.5:0/509572847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557273c9b30 0x5557273cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:51:21.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.904+0000 7f929f200640 1 --2- 172.21.15.5:0/509572847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557273beb40 0x5557273bef20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9298009f30 tx=0x7f9298010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6522b77b7feeec2f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:21.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.905+0000 7f928f400640 1 -- 172.21.15.5:0/509572847 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9298016070 con 0x5557273beb40 2024-04-24T06:51:21.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.905+0000 7f9297400640 1 --2- 172.21.15.5:0/64096492 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f92840a3d20 0x7f92840a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:21.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.905+0000 7f928f400640 1 -- 172.21.15.5:0/509572847 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9298004d20 con 0x5557273beb40 2024-04-24T06:51:21.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.905+0000 7f928f400640 1 -- 172.21.15.5:0/509572847 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9298012830 con 0x5557273beb40 2024-04-24T06:51:21.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.905+0000 7f9297400640 1 -- 172.21.15.5:0/64096492 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f92900d0ad0 con 0x7f92a0108540 2024-04-24T06:51:21.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.905+0000 7f929e800640 1 --2- 172.21.15.5:0/64096492 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f92840a3d20 0x7f92840a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:21.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.905+0000 7f92a836c740 1 -- 172.21.15.5:0/509572847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557273beb40 msgr2=0x5557273bef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:21.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.905+0000 7f92a836c740 1 --2- 172.21.15.5:0/509572847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557273beb40 0x5557273bef20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9298009f30 tx=0x7f9298010800 comp rx=0 tx=0).stop 2024-04-24T06:51:21.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.905+0000 7f92a836c740 1 -- 172.21.15.5:0/509572847 shutdown_connections 2024-04-24T06:51:21.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.906+0000 7f92a836c740 1 --2- 172.21.15.5:0/509572847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557273c9b30 0x5557273cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.906+0000 7f92a836c740 1 --2- 172.21.15.5:0/509572847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557273bf460 0x5557273c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.906+0000 7f92a836c740 1 --2- 172.21.15.5:0/509572847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557273beb40 0x5557273bef20 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.906+0000 7f92a836c740 1 -- 172.21.15.5:0/509572847 >> 172.21.15.5:0/509572847 conn(0x5557273bae60 msgr2=0x5557273bb240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:21.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.906+0000 7f929e800640 1 --2- 172.21.15.5:0/64096492 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f92840a3d20 0x7f92840a61e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x5557273c0100 tx=0x7f92900bd040 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:21.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.906+0000 7f92a836c740 1 -- 172.21.15.5:0/509572847 shutdown_connections 2024-04-24T06:51:21.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.906+0000 7f92a836c740 1 -- 172.21.15.5:0/509572847 wait complete. 2024-04-24T06:51:21.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.906+0000 7f92a836c740 1 -- start start 2024-04-24T06:51:21.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.906+0000 7f92a836c740 1 Processor -- start 2024-04-24T06:51:21.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.907+0000 7f92a836c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557273beb40 0x5557273c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:21.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.907+0000 7f92a836c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557273bf460 0x5557273c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:21.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.907+0000 7f92a836c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557273c9b30 0x5557273c6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:21.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.907+0000 7f92a836c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5557273cc9c0 con 0x5557273beb40 2024-04-24T06:51:21.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.907+0000 7f92a836c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5557272dd810 con 0x5557273c9b30 2024-04-24T06:51:21.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.907+0000 7f92a836c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5557273ccb40 con 0x5557273bf460 2024-04-24T06:51:21.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.907+0000 7f92a836c740 10 client.0 _ll_register_callbacks cb 0x5557272ab140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:51:21.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.907+0000 7f929f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557273beb40 0x5557273c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:21.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.907+0000 7f9297e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557273bf460 0x5557273c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:21.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.907+0000 7f9297e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557273bf460 0x5557273c68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:52076/0 (socket says 172.21.15.5:52076) 2024-04-24T06:51:21.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.907+0000 7f9297e00640 1 -- 172.21.15.5:0/1038673521 learned_addr learned my addr 172.21.15.5:0/1038673521 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:21.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.907+0000 7f929f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557273beb40 0x5557273c95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51628/0 (socket says 172.21.15.5:51628) 2024-04-24T06:51:21.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.907+0000 7f929f200640 1 --2- 172.21.15.5:0/1038673521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557273c9b30 0x5557273c6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:21.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.907+0000 7f9297e00640 1 -- 172.21.15.5:0/1038673521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557273c9b30 msgr2=0x5557273c6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:21.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.907+0000 7f9297e00640 1 --2- 172.21.15.5:0/1038673521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557273c9b30 0x5557273c6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.907+0000 7f9297e00640 1 -- 172.21.15.5:0/1038673521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557273beb40 msgr2=0x5557273c95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:21.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.907+0000 7f9297e00640 1 --2- 172.21.15.5:0/1038673521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557273beb40 0x5557273c95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.907+0000 7f9297e00640 1 -- 172.21.15.5:0/1038673521 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557272ee3f0 con 0x5557273bf460 2024-04-24T06:51:21.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.908+0000 7f9297e00640 1 --2- 172.21.15.5:0/1038673521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557273bf460 0x5557273c68b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9288009780 tx=0x7f92880304a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e78cf66790d9f011 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:21.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.908+0000 7f9283e00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:51:21.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.908+0000 7f9283e00640 1 -- 172.21.15.5:0/1038673521 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9288037040 con 0x5557273bf460 2024-04-24T06:51:21.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.908+0000 7f92a836c740 1 -- 172.21.15.5:0/1038673521 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9298004d20 con 0x5557273bf460 2024-04-24T06:51:21.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.908+0000 7f9283e00640 1 -- 172.21.15.5:0/1038673521 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f92880035f0 con 0x5557273bf460 2024-04-24T06:51:21.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.908+0000 7f9283e00640 1 -- 172.21.15.5:0/1038673521 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f928804c050 con 0x5557273bf460 2024-04-24T06:51:21.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.908+0000 7f92a836c740 10 client.26827 fetch_fsmap learned FSMap version 27 2024-04-24T06:51:21.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.908+0000 7f92a836c740 1 -- 172.21.15.5:0/1038673521 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5557270363f0 con 0x5557273bf460 2024-04-24T06:51:21.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.909+0000 7f9283e00640 1 -- 172.21.15.5:0/1038673521 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9298004d20 con 0x5557273bf460 2024-04-24T06:51:21.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.909+0000 7f9283e00640 1 -- 172.21.15.5:0/1038673521 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f9288030c40 con 0x5557273bf460 2024-04-24T06:51:21.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.909+0000 7f92a836c740 10 client.26827 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:51:21.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.909+0000 7f92a836c740 10 client.26827 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-24T06:51:21.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.909+0000 7f92a836c740 20 client.26827 populate_metadata read hostname 'smithi005' 2024-04-24T06:51:21.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.909+0000 7f92a836c740 1 --2- 172.21.15.5:0/1038673521 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x5557273d9d40 0x5557273dc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:21.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.909+0000 7f92a836c740 4 client.26827 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:51:21.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.909+0000 7f92a836c740 1 -- 172.21.15.5:0/1038673521 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555727331bf0 con 0x5557273d9d40 2024-04-24T06:51:21.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.909+0000 7f929f200640 1 --2- 172.21.15.5:0/1038673521 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x5557273d9d40 0x5557273dc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:21.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.910+0000 7f929f200640 1 --2- 172.21.15.5:0/1038673521 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x5557273d9d40 0x5557273dc150 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8d43c84a05181fd6 server_cookie=2952be3dc94523f6 in_seq=0 out_seq=0 2024-04-24T06:51:21.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.910+0000 7f9283e00640 10 client.26827 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:51:21.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.910+0000 7f9283e00640 1 -- 172.21.15.5:0/1038673521 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555727331bf0 con 0x5557273d9d40 2024-04-24T06:51:21.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.910+0000 7f9283e00640 10 client.26827 handle_command_reply: tid=0 2024-04-24T06:51:21.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.911+0000 7f92a836c740 1 client.26827 shutdown 2024-04-24T06:51:21.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.911+0000 7f92a836c740 1 -- 172.21.15.5:0/1038673521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557273bf460 msgr2=0x5557273c68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:21.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.911+0000 7f92a836c740 1 --2- 172.21.15.5:0/1038673521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557273bf460 0x5557273c68b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9288009780 tx=0x7f92880304a0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.911+0000 7f92a836c740 1 -- 172.21.15.5:0/1038673521 shutdown_connections 2024-04-24T06:51:21.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.911+0000 7f92a836c740 1 --2- 172.21.15.5:0/1038673521 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x5557273d9d40 0x5557273dc150 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.911+0000 7f92a836c740 1 --2- 172.21.15.5:0/1038673521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557273c9b30 0x5557273c6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.911+0000 7f92a836c740 1 --2- 172.21.15.5:0/1038673521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557273bf460 0x5557273c68b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.911+0000 7f92a836c740 1 --2- 172.21.15.5:0/1038673521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557273beb40 0x5557273c95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.911+0000 7f92a836c740 1 -- 172.21.15.5:0/1038673521 >> 172.21.15.5:0/1038673521 conn(0x5557273bae60 msgr2=0x5557273c0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:21.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.911+0000 7f9283e00640 0 client.26827 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:51:21.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.911+0000 7f92a836c740 1 -- 172.21.15.5:0/1038673521 shutdown_connections 2024-04-24T06:51:21.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.911+0000 7f92a836c740 1 -- 172.21.15.5:0/1038673521 wait complete. 2024-04-24T06:51:21.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.911+0000 7f92a836c740 20 client.26827 trim_cache size 0 max 16384 2024-04-24T06:51:21.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.926+0000 7f92a836c740 1 Processor -- start 2024-04-24T06:51:21.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.926+0000 7f92a836c740 1 -- start start 2024-04-24T06:51:21.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.926+0000 7f92a836c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557273beb40 0x5557273d58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:21.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.926+0000 7f92a836c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557273bf460 0x5557273c90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:21.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.926+0000 7f92a836c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557273c9b30 0x5557273e1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:21.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.926+0000 7f92a836c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f92a01b6b70 con 0x5557273c9b30 2024-04-24T06:51:21.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.926+0000 7f92a836c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f92a01b4440 con 0x5557273bf460 2024-04-24T06:51:21.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.926+0000 7f92a836c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f92a01b22b0 con 0x5557273beb40 2024-04-24T06:51:21.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.927+0000 7f929f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557273c9b30 0x5557273e1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:21.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.927+0000 7f929f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557273c9b30 0x5557273e1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51640/0 (socket says 172.21.15.5:51640) 2024-04-24T06:51:21.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.927+0000 7f929f200640 1 -- 172.21.15.5:0/1768610624 learned_addr learned my addr 172.21.15.5:0/1768610624 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:21.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.927+0000 7f929f200640 1 --2- 172.21.15.5:0/1768610624 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557273beb40 0x5557273d58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:21.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.927+0000 7f9297e00640 1 --2- 172.21.15.5:0/1768610624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557273bf460 0x5557273c90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:21.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.927+0000 7f929f200640 1 -- 172.21.15.5:0/1768610624 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557273beb40 msgr2=0x5557273d58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:21.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.927+0000 7f929f200640 1 --2- 172.21.15.5:0/1768610624 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557273beb40 0x5557273d58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.927+0000 7f929f200640 1 -- 172.21.15.5:0/1768610624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557273bf460 msgr2=0x5557273c90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:21.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.927+0000 7f929f200640 1 --2- 172.21.15.5:0/1768610624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557273bf460 0x5557273c90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.927+0000 7f929f200640 1 -- 172.21.15.5:0/1768610624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557273c7310 con 0x5557273c9b30 2024-04-24T06:51:21.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.927+0000 7f929f200640 1 --2- 172.21.15.5:0/1768610624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557273c9b30 0x5557273e1a20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9298011010 tx=0x7f9298014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e182d4b9a2b02393 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:21.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.928+0000 7f928ea00640 1 -- 172.21.15.5:0/1768610624 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9298024040 con 0x5557273c9b30 2024-04-24T06:51:21.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.928+0000 7f928ea00640 1 -- 172.21.15.5:0/1768610624 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9298025040 con 0x5557273c9b30 2024-04-24T06:51:21.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.928+0000 7f928ea00640 1 -- 172.21.15.5:0/1768610624 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9298016040 con 0x5557273c9b30 2024-04-24T06:51:21.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.928+0000 7f92a836c740 1 -- 172.21.15.5:0/1768610624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557273c9b30 msgr2=0x5557273e1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:21.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.928+0000 7f92a836c740 1 --2- 172.21.15.5:0/1768610624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557273c9b30 0x5557273e1a20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9298011010 tx=0x7f9298014040 comp rx=0 tx=0).stop 2024-04-24T06:51:21.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.928+0000 7f92a836c740 1 -- 172.21.15.5:0/1768610624 shutdown_connections 2024-04-24T06:51:21.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.928+0000 7f92a836c740 1 --2- 172.21.15.5:0/1768610624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557273c9b30 0x5557273e1a20 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.928+0000 7f92a836c740 1 --2- 172.21.15.5:0/1768610624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557273bf460 0x5557273c90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.928+0000 7f92a836c740 1 --2- 172.21.15.5:0/1768610624 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557273beb40 0x5557273d58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.928+0000 7f92a836c740 1 -- 172.21.15.5:0/1768610624 >> 172.21.15.5:0/1768610624 conn(0x5557273bae60 msgr2=0x5557273c1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:21.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.928+0000 7f92a836c740 1 -- 172.21.15.5:0/1768610624 shutdown_connections 2024-04-24T06:51:21.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.928+0000 7f92a836c740 1 -- 172.21.15.5:0/1768610624 wait complete. 2024-04-24T06:51:21.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.929+0000 7f92a836c740 1 -- start start 2024-04-24T06:51:21.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.929+0000 7f92a836c740 1 Processor -- start 2024-04-24T06:51:21.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.929+0000 7f92a836c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557273beb40 0x5557273cac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:21.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.929+0000 7f92a836c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557273bf460 0x5557273cb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:21.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.929+0000 7f92a836c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557273c9b30 0x5557273d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:21.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.929+0000 7f92a836c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f92a01b2900 con 0x5557273c9b30 2024-04-24T06:51:21.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.929+0000 7f92a836c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f92a01b3d70 con 0x5557273beb40 2024-04-24T06:51:21.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.929+0000 7f92a836c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f92a01b2fd0 con 0x5557273bf460 2024-04-24T06:51:21.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.930+0000 7f92a836c740 10 client.0 _ll_register_callbacks cb 0x7f92880096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:51:21.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.930+0000 7f9297e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557273bf460 0x5557273cb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:21.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.930+0000 7f9297e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557273bf460 0x5557273cb190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:52088/0 (socket says 172.21.15.5:52088) 2024-04-24T06:51:21.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.930+0000 7f9297e00640 1 -- 172.21.15.5:0/2305152386 learned_addr learned my addr 172.21.15.5:0/2305152386 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:21.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.930+0000 7f929f200640 1 --2- 172.21.15.5:0/2305152386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557273c9b30 0x5557273d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:21.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.930+0000 7f929f200640 1 --2- 172.21.15.5:0/2305152386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557273beb40 0x5557273cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:21.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.930+0000 7f9297e00640 1 -- 172.21.15.5:0/2305152386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557273beb40 msgr2=0x5557273cac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:21.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.930+0000 7f9297e00640 1 --2- 172.21.15.5:0/2305152386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557273beb40 0x5557273cac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.930+0000 7f9297e00640 1 -- 172.21.15.5:0/2305152386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557273c9b30 msgr2=0x5557273d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:21.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.930+0000 7f9297e00640 1 --2- 172.21.15.5:0/2305152386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557273c9b30 0x5557273d0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.930+0000 7f9297e00640 1 -- 172.21.15.5:0/2305152386 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557271007d0 con 0x5557273bf460 2024-04-24T06:51:21.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.930+0000 7f9297e00640 1 --2- 172.21.15.5:0/2305152386 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557273bf460 0x5557273cb190 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9288009890 tx=0x7f92880058c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b26c87a29cca20ae server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:21.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.930+0000 7f9283e00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:51:21.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.931+0000 7f9283e00640 1 -- 172.21.15.5:0/2305152386 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9288037040 con 0x5557273bf460 2024-04-24T06:51:21.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.931+0000 7f9283e00640 1 -- 172.21.15.5:0/2305152386 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f928803dc80 con 0x5557273bf460 2024-04-24T06:51:21.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.931+0000 7f92a836c740 1 -- 172.21.15.5:0/2305152386 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9298025040 con 0x5557273bf460 2024-04-24T06:51:21.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.931+0000 7f9283e00640 1 -- 172.21.15.5:0/2305152386 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f928805e050 con 0x5557273bf460 2024-04-24T06:51:21.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.931+0000 7f92a836c740 10 client.26833 fetch_fsmap learned FSMap version 27 2024-04-24T06:51:21.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.931+0000 7f92a836c740 1 -- 172.21.15.5:0/2305152386 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5557274407a0 con 0x5557273bf460 2024-04-24T06:51:21.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.931+0000 7f9283e00640 1 -- 172.21.15.5:0/2305152386 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9298025040 con 0x5557273bf460 2024-04-24T06:51:21.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.931+0000 7f9283e00640 1 -- 172.21.15.5:0/2305152386 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f92880035f0 con 0x5557273bf460 2024-04-24T06:51:21.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.931+0000 7f92a836c740 10 client.26833 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:51:21.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.931+0000 7f92a836c740 10 client.26833 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-24T06:51:21.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.931+0000 7f92a836c740 20 client.26833 populate_metadata read hostname 'smithi005' 2024-04-24T06:51:21.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.932+0000 7f92a836c740 1 --2- 172.21.15.5:0/2305152386 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x555727440fa0 0x5557274433c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:21.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.932+0000 7f92a836c740 4 client.26833 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:51:21.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.932+0000 7f92a836c740 1 -- 172.21.15.5:0/2305152386 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55572732ba60 con 0x555727440fa0 2024-04-24T06:51:21.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.932+0000 7f929f200640 1 --2- 172.21.15.5:0/2305152386 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x555727440fa0 0x5557274433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:21.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.932+0000 7f929f200640 1 --2- 172.21.15.5:0/2305152386 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x555727440fa0 0x5557274433c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1e59e2d0c51039b server_cookie=acdc1bbd99d4bbc7 in_seq=0 out_seq=0 2024-04-24T06:51:21.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.932+0000 7f9283e00640 10 client.26833 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:51:21.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.933+0000 7f9283e00640 1 -- 172.21.15.5:0/2305152386 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55572732ba60 con 0x555727440fa0 2024-04-24T06:51:21.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.933+0000 7f9283e00640 10 client.26833 handle_command_reply: tid=0 2024-04-24T06:51:21.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.933+0000 7f92a836c740 1 client.26833 shutdown 2024-04-24T06:51:21.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.933+0000 7f92a836c740 1 -- 172.21.15.5:0/2305152386 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557273bf460 msgr2=0x5557273cb190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:21.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.933+0000 7f92a836c740 1 --2- 172.21.15.5:0/2305152386 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557273bf460 0x5557273cb190 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9288009890 tx=0x7f92880058c0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.933+0000 7f92a836c740 1 -- 172.21.15.5:0/2305152386 shutdown_connections 2024-04-24T06:51:21.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.933+0000 7f92a836c740 1 --2- 172.21.15.5:0/2305152386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557273c9b30 0x5557273d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.933+0000 7f92a836c740 1 --2- 172.21.15.5:0/2305152386 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557273bf460 0x5557273cb190 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.933+0000 7f92a836c740 1 --2- 172.21.15.5:0/2305152386 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x555727440fa0 0x5557274433c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.933+0000 7f92a836c740 1 --2- 172.21.15.5:0/2305152386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557273beb40 0x5557273cac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.933+0000 7f9283e00640 0 client.26833 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:51:21.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.933+0000 7f92a836c740 1 -- 172.21.15.5:0/2305152386 >> 172.21.15.5:0/2305152386 conn(0x5557273bae60 msgr2=0x5557273b96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:21.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.933+0000 7f92a836c740 1 -- 172.21.15.5:0/2305152386 shutdown_connections 2024-04-24T06:51:21.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.933+0000 7f92a836c740 1 -- 172.21.15.5:0/2305152386 wait complete. 2024-04-24T06:51:21.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.933+0000 7f92a836c740 20 client.26833 trim_cache size 0 max 16384 2024-04-24T06:51:21.943 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:51:21.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.934+0000 7f92a4a00640 1 -- 172.21.15.5:0/64096492 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f92840a3d20 msgr2=0x7f92840a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:21.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.934+0000 7f92a4a00640 1 --2- 172.21.15.5:0/64096492 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f92840a3d20 0x7f92840a61e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x5557273c0100 tx=0x7f92900bd040 comp rx=0 tx=0).stop 2024-04-24T06:51:21.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.934+0000 7f92a4a00640 1 -- 172.21.15.5:0/64096492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92a0108540 msgr2=0x7f92a019ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:21.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.934+0000 7f92a4a00640 1 --2- 172.21.15.5:0/64096492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92a0108540 0x7f92a019ed60 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f929000e9c0 tx=0x7f929000ee80 comp rx=0 tx=0).stop 2024-04-24T06:51:21.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.934+0000 7f92a4a00640 1 -- 172.21.15.5:0/64096492 shutdown_connections 2024-04-24T06:51:21.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.934+0000 7f92a4a00640 1 --2- 172.21.15.5:0/64096492 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f92840a3d20 0x7f92840a61e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.934+0000 7f92a4a00640 1 --2- 172.21.15.5:0/64096492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92a010e4f0 0x7f92a01a3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.934+0000 7f92a4a00640 1 --2- 172.21.15.5:0/64096492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92a0108ee0 0x7f92a019f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.934+0000 7f92a4a00640 1 --2- 172.21.15.5:0/64096492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92a0108540 0x7f92a019ed60 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:21.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.934+0000 7f92a4a00640 1 -- 172.21.15.5:0/64096492 >> 172.21.15.5:0/64096492 conn(0x7f92a0077e10 msgr2=0x7f92a0078530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:21.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.934+0000 7f92a4a00640 1 -- 172.21.15.5:0/64096492 shutdown_connections 2024-04-24T06:51:21.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:21.934+0000 7f92a4a00640 1 -- 172.21.15.5:0/64096492 wait complete. 2024-04-24T06:51:21.955 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T06:51:22.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.135+0000 7ff288600640 1 -- 172.21.15.5:0/434878015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff280109520 msgr2=0x7ff2801110f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:22.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.135+0000 7ff288600640 1 --2- 172.21.15.5:0/434878015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff280109520 0x7ff2801110f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff278009be0 tx=0x7ff27802a6c0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.135+0000 7ff288600640 1 -- 172.21.15.5:0/434878015 shutdown_connections 2024-04-24T06:51:22.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.135+0000 7ff288600640 1 --2- 172.21.15.5:0/434878015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff280109520 0x7ff2801110f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.135+0000 7ff288600640 1 --2- 172.21.15.5:0/434878015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff280104e00 0x7ff280108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.135+0000 7ff288600640 1 --2- 172.21.15.5:0/434878015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff280104450 0x7ff280104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.135+0000 7ff288600640 1 -- 172.21.15.5:0/434878015 >> 172.21.15.5:0/434878015 conn(0x7ff2800ffc90 msgr2=0x7ff2801020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:22.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.135+0000 7ff288600640 1 -- 172.21.15.5:0/434878015 shutdown_connections 2024-04-24T06:51:22.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.136+0000 7ff288600640 1 -- 172.21.15.5:0/434878015 wait complete. 2024-04-24T06:51:22.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.136+0000 7ff288600640 1 Processor -- start 2024-04-24T06:51:22.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.137+0000 7ff288600640 1 -- start start 2024-04-24T06:51:22.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.137+0000 7ff288600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff280104450 0x7ff2801a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:22.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.137+0000 7ff288600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff280109520 0x7ff2801a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:22.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.137+0000 7ff288600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff28019a9b0 0x7ff28019ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:22.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.137+0000 7ff288600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff280114560 con 0x7ff280104450 2024-04-24T06:51:22.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.137+0000 7ff288600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff2801143e0 con 0x7ff280109520 2024-04-24T06:51:22.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.137+0000 7ff288600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff2801146e0 con 0x7ff28019a9b0 2024-04-24T06:51:22.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.137+0000 7ff285200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff280109520 0x7ff2801a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:22.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.137+0000 7ff285200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff280109520 0x7ff2801a0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:39526/0 (socket says 172.21.15.5:39526) 2024-04-24T06:51:22.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.137+0000 7ff285200640 1 -- 172.21.15.5:0/131290683 learned_addr learned my addr 172.21.15.5:0/131290683 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:22.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.137+0000 7ff285c00640 1 --2- 172.21.15.5:0/131290683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff280104450 0x7ff2801a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:22.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.138+0000 7ff286600640 1 --2- 172.21.15.5:0/131290683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff28019a9b0 0x7ff28019ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:22.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.138+0000 7ff285200640 1 -- 172.21.15.5:0/131290683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff28019a9b0 msgr2=0x7ff28019ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:22.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.138+0000 7ff285200640 1 --2- 172.21.15.5:0/131290683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff28019a9b0 0x7ff28019ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.138+0000 7ff285200640 1 -- 172.21.15.5:0/131290683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff280104450 msgr2=0x7ff2801a07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:22.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.138+0000 7ff285200640 1 --2- 172.21.15.5:0/131290683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff280104450 0x7ff2801a07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.138+0000 7ff285200640 1 -- 172.21.15.5:0/131290683 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff28019b4d0 con 0x7ff280109520 2024-04-24T06:51:22.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.138+0000 7ff286600640 1 --2- 172.21.15.5:0/131290683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff28019a9b0 0x7ff28019ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:51:22.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.138+0000 7ff285200640 1 --2- 172.21.15.5:0/131290683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff280109520 0x7ff2801a0cf0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff26c00e9a0 tx=0x7ff26c00ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:22.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.138+0000 7ff27e000640 1 -- 172.21.15.5:0/131290683 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26c00cd80 con 0x7ff280109520 2024-04-24T06:51:22.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.138+0000 7ff27e000640 1 -- 172.21.15.5:0/131290683 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff26c0035f0 con 0x7ff280109520 2024-04-24T06:51:22.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.138+0000 7ff27e000640 1 -- 172.21.15.5:0/131290683 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26c010670 con 0x7ff280109520 2024-04-24T06:51:22.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.138+0000 7ff288600640 1 -- 172.21.15.5:0/131290683 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff28019b7c0 con 0x7ff280109520 2024-04-24T06:51:22.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.139+0000 7ff288600640 1 -- 172.21.15.5:0/131290683 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2801a79c0 con 0x7ff280109520 2024-04-24T06:51:22.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.141+0000 7ff28b58e740 1 Processor -- start 2024-04-24T06:51:22.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.142+0000 7ff28b58e740 1 -- start start 2024-04-24T06:51:22.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.142+0000 7ff28b58e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55952e8e2950 0x55952e8e2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:22.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.142+0000 7ff28b58e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55952e8e3270 0x55952e8ed400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:22.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.142+0000 7ff28b58e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55952e8ed940 0x55952e8efd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:22.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.142+0000 7ff28b58e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55952e8481d0 con 0x55952e8ed940 2024-04-24T06:51:22.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.142+0000 7ff28b58e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55952e7ee450 con 0x55952e8e3270 2024-04-24T06:51:22.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.142+0000 7ff28b58e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55952e844190 con 0x55952e8e2950 2024-04-24T06:51:22.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.142+0000 7ff286600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55952e8e2950 0x55952e8e2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:22.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.142+0000 7ff285c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55952e8e3270 0x55952e8ed400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:22.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.142+0000 7ff286600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55952e8e2950 0x55952e8e2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:52116/0 (socket says 172.21.15.5:52116) 2024-04-24T06:51:22.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.142+0000 7ff286600640 1 -- 172.21.15.5:0/2618046963 learned_addr learned my addr 172.21.15.5:0/2618046963 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:22.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.143+0000 7ff286600640 1 -- 172.21.15.5:0/2618046963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55952e8e3270 msgr2=0x55952e8ed400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:22.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.143+0000 7ff286600640 1 --2- 172.21.15.5:0/2618046963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55952e8e3270 0x55952e8ed400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.143+0000 7ff286600640 1 -- 172.21.15.5:0/2618046963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55952e8ed940 msgr2=0x55952e8efd30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T06:51:22.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.143+0000 7ff286600640 1 --2- 172.21.15.5:0/2618046963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55952e8ed940 0x55952e8efd30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.143+0000 7ff286600640 1 -- 172.21.15.5:0/2618046963 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55952e8ab310 con 0x55952e8e2950 2024-04-24T06:51:22.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.143+0000 7ff286600640 1 --2- 172.21.15.5:0/2618046963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55952e8e2950 0x55952e8e2d30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff278033000 tx=0x7ff278030a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e33614905a4e7eb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:22.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.143+0000 7ff272a00640 1 -- 172.21.15.5:0/2618046963 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278041030 con 0x55952e8e2950 2024-04-24T06:51:22.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.143+0000 7ff272a00640 1 -- 172.21.15.5:0/2618046963 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff278037040 con 0x55952e8e2950 2024-04-24T06:51:22.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.143+0000 7ff272a00640 1 -- 172.21.15.5:0/2618046963 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278036040 con 0x55952e8e2950 2024-04-24T06:51:22.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.144+0000 7ff27e000640 1 -- 172.21.15.5:0/131290683 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7ff26c0037a0 con 0x7ff280109520 2024-04-24T06:51:22.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.144+0000 7ff28b58e740 1 -- 172.21.15.5:0/2618046963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55952e8e2950 msgr2=0x55952e8e2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:22.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.144+0000 7ff28b58e740 1 --2- 172.21.15.5:0/2618046963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55952e8e2950 0x55952e8e2d30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff278033000 tx=0x7ff278030a30 comp rx=0 tx=0).stop 2024-04-24T06:51:22.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.144+0000 7ff28b58e740 1 -- 172.21.15.5:0/2618046963 shutdown_connections 2024-04-24T06:51:22.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.144+0000 7ff28b58e740 1 --2- 172.21.15.5:0/2618046963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55952e8ed940 0x55952e8efd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.144+0000 7ff28b58e740 1 --2- 172.21.15.5:0/2618046963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55952e8e3270 0x55952e8ed400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.144+0000 7ff28b58e740 1 --2- 172.21.15.5:0/2618046963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55952e8e2950 0x55952e8e2d30 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.144+0000 7ff28b58e740 1 -- 172.21.15.5:0/2618046963 >> 172.21.15.5:0/2618046963 conn(0x55952e8dec70 msgr2=0x55952e8df050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:22.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.144+0000 7ff28b58e740 1 -- 172.21.15.5:0/2618046963 shutdown_connections 2024-04-24T06:51:22.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.144+0000 7ff28b58e740 1 -- 172.21.15.5:0/2618046963 wait complete. 2024-04-24T06:51:22.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.145+0000 7ff28b58e740 1 -- start start 2024-04-24T06:51:22.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.145+0000 7ff28b58e740 1 Processor -- start 2024-04-24T06:51:22.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.145+0000 7ff27e000640 1 --2- 172.21.15.5:0/131290683 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ff2680a3d80 0x7ff2680a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:22.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.145+0000 7ff27e000640 1 -- 172.21.15.5:0/131290683 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7ff26c0d09b0 con 0x7ff280109520 2024-04-24T06:51:22.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.145+0000 7ff286600640 1 --2- 172.21.15.5:0/131290683 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ff2680a3d80 0x7ff2680a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:22.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.145+0000 7ff28b58e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55952e8e2950 0x55952e8ed400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:22.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.145+0000 7ff28b58e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55952e8e3270 0x55952e8ea6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:22.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.146+0000 7ff28b58e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55952e8ed940 0x55952e8eae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:22.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.146+0000 7ff28b58e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55952e8f0770 con 0x55952e8ed940 2024-04-24T06:51:22.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.146+0000 7ff28b58e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55952e801810 con 0x55952e8e2950 2024-04-24T06:51:22.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.146+0000 7ff28b58e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55952e8f08f0 con 0x55952e8e3270 2024-04-24T06:51:22.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.146+0000 7ff28b58e740 10 client.0 _ll_register_callbacks cb 0x55952e7cf140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:51:22.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.146+0000 7ff285c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55952e8e2950 0x55952e8ed400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:22.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.146+0000 7ff285c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55952e8e2950 0x55952e8ed400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:39546/0 (socket says 172.21.15.5:39546) 2024-04-24T06:51:22.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.146+0000 7ff285c00640 1 -- 172.21.15.5:0/1524940983 learned_addr learned my addr 172.21.15.5:0/1524940983 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:22.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.146+0000 7ff285200640 1 --2- 172.21.15.5:0/1524940983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55952e8e3270 0x55952e8ea6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:22.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.146+0000 7ff285200640 1 -- 172.21.15.5:0/1524940983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55952e8e2950 msgr2=0x55952e8ed400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:22.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.146+0000 7ff285200640 1 --2- 172.21.15.5:0/1524940983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55952e8e2950 0x55952e8ed400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.146+0000 7ff285200640 1 -- 172.21.15.5:0/1524940983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55952e8ed940 msgr2=0x55952e8eae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:22.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.146+0000 7ff285200640 1 --2- 172.21.15.5:0/1524940983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55952e8ed940 0x55952e8eae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.146+0000 7ff285200640 1 -- 172.21.15.5:0/1524940983 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55952e8123f0 con 0x55952e8e3270 2024-04-24T06:51:22.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.146+0000 7ff285c00640 1 --2- 172.21.15.5:0/1524940983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55952e8e2950 0x55952e8ed400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:51:22.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.146+0000 7ff285200640 1 --2- 172.21.15.5:0/1524940983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55952e8e3270 0x55952e8ea6f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff26c0ce0f0 tx=0x7ff26c09c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a7aa7622b1c8942 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:22.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.146+0000 7ff267400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:51:22.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.147+0000 7ff286600640 1 --2- 172.21.15.5:0/131290683 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ff2680a3d80 0x7ff2680a6240 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff278005f00 tx=0x7ff278003cc0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:22.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.147+0000 7ff267400640 1 -- 172.21.15.5:0/1524940983 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26c09c940 con 0x55952e8e3270 2024-04-24T06:51:22.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.147+0000 7ff267400640 1 -- 172.21.15.5:0/1524940983 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7ff26c09bd90 con 0x55952e8e3270 2024-04-24T06:51:22.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.147+0000 7ff28b58e740 1 -- 172.21.15.5:0/1524940983 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff278037040 con 0x55952e8e3270 2024-04-24T06:51:22.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.147+0000 7ff267400640 1 -- 172.21.15.5:0/1524940983 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff26c0aa050 con 0x55952e8e3270 2024-04-24T06:51:22.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.147+0000 7ff28b58e740 10 client.26845 fetch_fsmap learned FSMap version 27 2024-04-24T06:51:22.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.147+0000 7ff28b58e740 1 -- 172.21.15.5:0/1524940983 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55952e55a3f0 con 0x55952e8e3270 2024-04-24T06:51:22.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.147+0000 7ff267400640 1 -- 172.21.15.5:0/1524940983 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff278037040 con 0x55952e8e3270 2024-04-24T06:51:22.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.147+0000 7ff267400640 1 -- 172.21.15.5:0/1524940983 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7ff26c098720 con 0x55952e8e3270 2024-04-24T06:51:22.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.147+0000 7ff28b58e740 10 client.26845 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:51:22.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.147+0000 7ff28b58e740 10 client.26845 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T06:51:22.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.147+0000 7ff28b58e740 20 client.26845 populate_metadata read hostname 'smithi005' 2024-04-24T06:51:22.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.147+0000 7ff28b58e740 1 --2- 172.21.15.5:0/1524940983 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55952e8fdba0 0x55952e8fff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:22.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.147+0000 7ff28b58e740 4 client.26845 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:51:22.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.147+0000 7ff28b58e740 1 -- 172.21.15.5:0/1524940983 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55952e855bf0 con 0x55952e8fdba0 2024-04-24T06:51:22.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.148+0000 7ff285c00640 1 --2- 172.21.15.5:0/1524940983 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55952e8fdba0 0x55952e8fff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:22.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.148+0000 7ff285c00640 1 --2- 172.21.15.5:0/1524940983 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55952e8fdba0 0x55952e8fff90 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6b3e74e075de60a5 server_cookie=623e01ac20c92cbc in_seq=0 out_seq=0 2024-04-24T06:51:22.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.148+0000 7ff267400640 10 client.26845 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:51:22.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.149+0000 7ff267400640 1 -- 172.21.15.5:0/1524940983 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55952e855bf0 con 0x55952e8fdba0 2024-04-24T06:51:22.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.149+0000 7ff267400640 10 client.26845 handle_command_reply: tid=0 2024-04-24T06:51:22.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.149+0000 7ff28b58e740 1 client.26845 shutdown 2024-04-24T06:51:22.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.149+0000 7ff28b58e740 1 -- 172.21.15.5:0/1524940983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55952e8e3270 msgr2=0x55952e8ea6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:22.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.149+0000 7ff28b58e740 1 --2- 172.21.15.5:0/1524940983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55952e8e3270 0x55952e8ea6f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff26c0ce0f0 tx=0x7ff26c09c480 comp rx=0 tx=0).stop 2024-04-24T06:51:22.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.149+0000 7ff28b58e740 1 -- 172.21.15.5:0/1524940983 shutdown_connections 2024-04-24T06:51:22.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.149+0000 7ff28b58e740 1 --2- 172.21.15.5:0/1524940983 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55952e8fdba0 0x55952e8fff90 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.149+0000 7ff28b58e740 1 --2- 172.21.15.5:0/1524940983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55952e8ed940 0x55952e8eae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.149+0000 7ff28b58e740 1 --2- 172.21.15.5:0/1524940983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55952e8e3270 0x55952e8ea6f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.149+0000 7ff267400640 0 client.26845 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:51:22.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.149+0000 7ff28b58e740 1 --2- 172.21.15.5:0/1524940983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55952e8e2950 0x55952e8ed400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.149+0000 7ff28b58e740 1 -- 172.21.15.5:0/1524940983 >> 172.21.15.5:0/1524940983 conn(0x55952e8dec70 msgr2=0x55952e8e4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:22.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.149+0000 7ff28b58e740 1 -- 172.21.15.5:0/1524940983 shutdown_connections 2024-04-24T06:51:22.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.150+0000 7ff28b58e740 1 -- 172.21.15.5:0/1524940983 wait complete. 2024-04-24T06:51:22.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.150+0000 7ff28b58e740 20 client.26845 trim_cache size 0 max 16384 2024-04-24T06:51:22.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.164+0000 7ff28b58e740 1 Processor -- start 2024-04-24T06:51:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.164+0000 7ff28b58e740 1 -- start start 2024-04-24T06:51:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.164+0000 7ff28b58e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55952e8e2950 0x55952e8ea6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.164+0000 7ff28b58e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55952e8e3270 0x55952e8e5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.164+0000 7ff28b58e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55952e8ed940 0x55952e8de570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.164+0000 7ff28b58e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff2801b2880 con 0x55952e8e2950 2024-04-24T06:51:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.164+0000 7ff28b58e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff2801b0150 con 0x55952e8ed940 2024-04-24T06:51:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.164+0000 7ff28b58e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff2801adfc0 con 0x55952e8e3270 2024-04-24T06:51:22.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.165+0000 7ff285200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55952e8e3270 0x55952e8e5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:22.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.165+0000 7ff285c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55952e8e2950 0x55952e8ea6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:22.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.165+0000 7ff286600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55952e8ed940 0x55952e8de570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:22.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.165+0000 7ff285200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55952e8e3270 0x55952e8e5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:52140/0 (socket says 172.21.15.5:52140) 2024-04-24T06:51:22.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.165+0000 7ff285200640 1 -- 172.21.15.5:0/2694004837 learned_addr learned my addr 172.21.15.5:0/2694004837 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:22.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.165+0000 7ff285200640 1 -- 172.21.15.5:0/2694004837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55952e8ed940 msgr2=0x55952e8de570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:22.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.165+0000 7ff285200640 1 --2- 172.21.15.5:0/2694004837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55952e8ed940 0x55952e8de570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.165+0000 7ff285200640 1 -- 172.21.15.5:0/2694004837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55952e8e2950 msgr2=0x55952e8ea6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:22.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.165+0000 7ff285200640 1 --2- 172.21.15.5:0/2694004837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55952e8e2950 0x55952e8ea6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.165+0000 7ff285200640 1 -- 172.21.15.5:0/2694004837 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55952e6247d0 con 0x55952e8e3270 2024-04-24T06:51:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.165+0000 7ff285200640 1 --2- 172.21.15.5:0/2694004837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55952e8e3270 0x55952e8e5190 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff26c09c8d0 tx=0x7ff26c003d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3468dda84fbe0544 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.166+0000 7ff272000640 1 -- 172.21.15.5:0/2694004837 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26c09c2a0 con 0x55952e8e3270 2024-04-24T06:51:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.166+0000 7ff272000640 1 -- 172.21.15.5:0/2694004837 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff26c09bb80 con 0x55952e8e3270 2024-04-24T06:51:22.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.166+0000 7ff272000640 1 -- 172.21.15.5:0/2694004837 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26c09bd90 con 0x55952e8e3270 2024-04-24T06:51:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.166+0000 7ff28b58e740 1 -- 172.21.15.5:0/2694004837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55952e8e3270 msgr2=0x55952e8e5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.166+0000 7ff28b58e740 1 --2- 172.21.15.5:0/2694004837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55952e8e3270 0x55952e8e5190 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff26c09c8d0 tx=0x7ff26c003d20 comp rx=0 tx=0).stop 2024-04-24T06:51:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.166+0000 7ff28b58e740 1 -- 172.21.15.5:0/2694004837 shutdown_connections 2024-04-24T06:51:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.166+0000 7ff28b58e740 1 --2- 172.21.15.5:0/2694004837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55952e8ed940 0x55952e8de570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.166+0000 7ff28b58e740 1 --2- 172.21.15.5:0/2694004837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55952e8e3270 0x55952e8e5190 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.166+0000 7ff28b58e740 1 --2- 172.21.15.5:0/2694004837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55952e8e2950 0x55952e8ea6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.166+0000 7ff28b58e740 1 -- 172.21.15.5:0/2694004837 >> 172.21.15.5:0/2694004837 conn(0x55952e8dec70 msgr2=0x55952e8e9fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:22.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.167+0000 7ff28b58e740 1 -- 172.21.15.5:0/2694004837 shutdown_connections 2024-04-24T06:51:22.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.167+0000 7ff28b58e740 1 -- 172.21.15.5:0/2694004837 wait complete. 2024-04-24T06:51:22.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.167+0000 7ff28b58e740 1 -- start start 2024-04-24T06:51:22.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.167+0000 7ff28b58e740 1 Processor -- start 2024-04-24T06:51:22.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.168+0000 7ff28b58e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55952e8e2950 0x55952e8e69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:22.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.168+0000 7ff28b58e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55952e8e3270 0x55952e8e6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:22.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.168+0000 7ff28b58e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55952e8ed940 0x55952e8e7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:22.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.168+0000 7ff28b58e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff2801ae610 con 0x55952e8e3270 2024-04-24T06:51:22.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.168+0000 7ff28b58e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff2801afa80 con 0x55952e8ed940 2024-04-24T06:51:22.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.168+0000 7ff28b58e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff2801aece0 con 0x55952e8e2950 2024-04-24T06:51:22.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.168+0000 7ff28b58e740 10 client.0 _ll_register_callbacks cb 0x7ff26c0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:51:22.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.168+0000 7ff285c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55952e8e2950 0x55952e8e69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:22.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.168+0000 7ff285200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55952e8e3270 0x55952e8e6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:22.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.168+0000 7ff285c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55952e8e2950 0x55952e8e69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:52142/0 (socket says 172.21.15.5:52142) 2024-04-24T06:51:22.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.168+0000 7ff285c00640 1 -- 172.21.15.5:0/1999232140 learned_addr learned my addr 172.21.15.5:0/1999232140 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:22.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.168+0000 7ff286600640 1 --2- 172.21.15.5:0/1999232140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55952e8ed940 0x55952e8e7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:22.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.168+0000 7ff285c00640 1 -- 172.21.15.5:0/1999232140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55952e8ed940 msgr2=0x55952e8e7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:22.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.168+0000 7ff285c00640 1 --2- 172.21.15.5:0/1999232140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55952e8ed940 0x55952e8e7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.168+0000 7ff285c00640 1 -- 172.21.15.5:0/1999232140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55952e8e3270 msgr2=0x55952e8e6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:22.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.168+0000 7ff285c00640 1 --2- 172.21.15.5:0/1999232140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55952e8e3270 0x55952e8e6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.168+0000 7ff285c00640 1 -- 172.21.15.5:0/1999232140 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55952e8f4ca0 con 0x55952e8e2950 2024-04-24T06:51:22.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.168+0000 7ff285200640 1 --2- 172.21.15.5:0/1999232140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55952e8e3270 0x55952e8e6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:51:22.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.169+0000 7ff285c00640 1 --2- 172.21.15.5:0/1999232140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55952e8e2950 0x55952e8e69c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x55952e900790 tx=0x7ff2740122a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a1f061b8d1884f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:22.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.169+0000 7ff267400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:51:22.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.169+0000 7ff267400640 1 -- 172.21.15.5:0/1999232140 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff274020820 con 0x55952e8e2950 2024-04-24T06:51:22.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.169+0000 7ff28b58e740 1 -- 172.21.15.5:0/1999232140 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff26c09bb80 con 0x55952e8e2950 2024-04-24T06:51:22.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.169+0000 7ff267400640 1 -- 172.21.15.5:0/1999232140 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7ff274005020 con 0x55952e8e2950 2024-04-24T06:51:22.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.169+0000 7ff267400640 1 -- 172.21.15.5:0/1999232140 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff274005220 con 0x55952e8e2950 2024-04-24T06:51:22.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.169+0000 7ff28b58e740 10 client.26851 fetch_fsmap learned FSMap version 27 2024-04-24T06:51:22.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.169+0000 7ff28b58e740 1 -- 172.21.15.5:0/1999232140 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55952e8f5fa0 con 0x55952e8e2950 2024-04-24T06:51:22.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.169+0000 7ff267400640 1 -- 172.21.15.5:0/1999232140 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff26c09bb80 con 0x55952e8e2950 2024-04-24T06:51:22.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.170+0000 7ff267400640 1 -- 172.21.15.5:0/1999232140 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7ff274012860 con 0x55952e8e2950 2024-04-24T06:51:22.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.170+0000 7ff28b58e740 10 client.26851 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:51:22.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.170+0000 7ff28b58e740 10 client.26851 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T06:51:22.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.170+0000 7ff28b58e740 20 client.26851 populate_metadata read hostname 'smithi005' 2024-04-24T06:51:22.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.170+0000 7ff28b58e740 1 --2- 172.21.15.5:0/1999232140 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55952e8fdba0 0x55952e966110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:22.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.170+0000 7ff28b58e740 4 client.26851 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:51:22.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.170+0000 7ff28b58e740 1 -- 172.21.15.5:0/1999232140 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55952e84fa60 con 0x55952e8fdba0 2024-04-24T06:51:22.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.170+0000 7ff285200640 1 --2- 172.21.15.5:0/1999232140 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55952e8fdba0 0x55952e966110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:22.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.170+0000 7ff285200640 1 --2- 172.21.15.5:0/1999232140 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55952e8fdba0 0x55952e966110 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=86224263cde27064 server_cookie=9da9a57962b96ee3 in_seq=0 out_seq=0 2024-04-24T06:51:22.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.170+0000 7ff267400640 10 client.26851 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:51:22.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.171+0000 7ff267400640 1 -- 172.21.15.5:0/1999232140 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55952e84fa60 con 0x55952e8fdba0 2024-04-24T06:51:22.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.171+0000 7ff267400640 10 client.26851 handle_command_reply: tid=0 2024-04-24T06:51:22.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.171+0000 7ff28b58e740 1 client.26851 shutdown 2024-04-24T06:51:22.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.171+0000 7ff28b58e740 1 -- 172.21.15.5:0/1999232140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55952e8e2950 msgr2=0x55952e8e69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:22.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.171+0000 7ff28b58e740 1 --2- 172.21.15.5:0/1999232140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55952e8e2950 0x55952e8e69c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x55952e900790 tx=0x7ff2740122a0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.171+0000 7ff28b58e740 1 -- 172.21.15.5:0/1999232140 shutdown_connections 2024-04-24T06:51:22.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.171+0000 7ff28b58e740 1 --2- 172.21.15.5:0/1999232140 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55952e8fdba0 0x55952e966110 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.171+0000 7ff28b58e740 1 --2- 172.21.15.5:0/1999232140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55952e8ed940 0x55952e8e7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.171+0000 7ff28b58e740 1 --2- 172.21.15.5:0/1999232140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55952e8e3270 0x55952e8e6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.171+0000 7ff267400640 0 client.26851 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:51:22.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.171+0000 7ff28b58e740 1 --2- 172.21.15.5:0/1999232140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55952e8e2950 0x55952e8e69c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.171+0000 7ff28b58e740 1 -- 172.21.15.5:0/1999232140 >> 172.21.15.5:0/1999232140 conn(0x55952e8dec70 msgr2=0x55952e8f8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:22.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.171+0000 7ff28b58e740 1 -- 172.21.15.5:0/1999232140 shutdown_connections 2024-04-24T06:51:22.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.171+0000 7ff28b58e740 1 -- 172.21.15.5:0/1999232140 wait complete. 2024-04-24T06:51:22.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.171+0000 7ff28b58e740 20 client.26851 trim_cache size 0 max 16384 2024-04-24T06:51:22.181 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:51:22.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.172+0000 7ff288600640 1 -- 172.21.15.5:0/131290683 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ff2680a3d80 msgr2=0x7ff2680a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:22.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.172+0000 7ff288600640 1 --2- 172.21.15.5:0/131290683 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ff2680a3d80 0x7ff2680a6240 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff278005f00 tx=0x7ff278003cc0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.172+0000 7ff288600640 1 -- 172.21.15.5:0/131290683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff280109520 msgr2=0x7ff2801a0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:22.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.172+0000 7ff288600640 1 --2- 172.21.15.5:0/131290683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff280109520 0x7ff2801a0cf0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff26c00e9a0 tx=0x7ff26c00ee60 comp rx=0 tx=0).stop 2024-04-24T06:51:22.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.172+0000 7ff288600640 1 -- 172.21.15.5:0/131290683 shutdown_connections 2024-04-24T06:51:22.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.172+0000 7ff288600640 1 --2- 172.21.15.5:0/131290683 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ff2680a3d80 0x7ff2680a6240 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.172+0000 7ff288600640 1 --2- 172.21.15.5:0/131290683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff28019a9b0 0x7ff28019ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.172+0000 7ff288600640 1 --2- 172.21.15.5:0/131290683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff280109520 0x7ff2801a0cf0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.172+0000 7ff288600640 1 --2- 172.21.15.5:0/131290683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff280104450 0x7ff2801a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.172+0000 7ff288600640 1 -- 172.21.15.5:0/131290683 >> 172.21.15.5:0/131290683 conn(0x7ff2800ffc90 msgr2=0x7ff280101fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:22.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.172+0000 7ff288600640 1 -- 172.21.15.5:0/131290683 shutdown_connections 2024-04-24T06:51:22.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.172+0000 7ff288600640 1 -- 172.21.15.5:0/131290683 wait complete. 2024-04-24T06:51:22.189 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T06:51:22.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.376+0000 7f7123200640 1 -- 172.21.15.5:0/2195874513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7124102070 msgr2=0x7f7124102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:22.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.376+0000 7f7123200640 1 --2- 172.21.15.5:0/2195874513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7124102070 0x7f7124102450 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7114009900 tx=0x7f711402a3f0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.376+0000 7f7123200640 1 -- 172.21.15.5:0/2195874513 shutdown_connections 2024-04-24T06:51:22.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.376+0000 7f7123200640 1 --2- 172.21.15.5:0/2195874513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7124107140 0x7f712410ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.376+0000 7f7123200640 1 --2- 172.21.15.5:0/2195874513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7124102a20 0x7f7124106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.376+0000 7f7123200640 1 --2- 172.21.15.5:0/2195874513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7124102070 0x7f7124102450 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.376+0000 7f7123200640 1 -- 172.21.15.5:0/2195874513 >> 172.21.15.5:0/2195874513 conn(0x7f71240fd950 msgr2=0x7f71240ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:22.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.376+0000 7f7123200640 1 -- 172.21.15.5:0/2195874513 shutdown_connections 2024-04-24T06:51:22.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.376+0000 7f7123200640 1 -- 172.21.15.5:0/2195874513 wait complete. 2024-04-24T06:51:22.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.377+0000 7f7123200640 1 Processor -- start 2024-04-24T06:51:22.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.377+0000 7f7123200640 1 -- start start 2024-04-24T06:51:22.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.377+0000 7f7123200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7124102070 0x7f7124074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:22.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.377+0000 7f7123200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7124107140 0x7f7124074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:22.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.377+0000 7f7123200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7124074d80 0x7f71241a6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:22.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.377+0000 7f7123200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7124112240 con 0x7f7124107140 2024-04-24T06:51:22.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.377+0000 7f7123200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f71241120c0 con 0x7f7124102070 2024-04-24T06:51:22.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.377+0000 7f7123200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f71241123c0 con 0x7f7124074d80 2024-04-24T06:51:22.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.378+0000 7f7121e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7124102070 0x7f7124074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:22.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.378+0000 7f7122800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7124074d80 0x7f71241a6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:22.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.378+0000 7f7121e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7124102070 0x7f7124074300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:39584/0 (socket says 172.21.15.5:39584) 2024-04-24T06:51:22.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.378+0000 7f7121400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7124107140 0x7f7124074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:22.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.378+0000 7f7121e00640 1 -- 172.21.15.5:0/728369454 learned_addr learned my addr 172.21.15.5:0/728369454 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:22.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.378+0000 7f7121e00640 1 -- 172.21.15.5:0/728369454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7124074d80 msgr2=0x7f71241a6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:22.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.378+0000 7f7121e00640 1 --2- 172.21.15.5:0/728369454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7124074d80 0x7f71241a6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.378+0000 7f7121e00640 1 -- 172.21.15.5:0/728369454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7124107140 msgr2=0x7f7124074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:22.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.378+0000 7f7121e00640 1 --2- 172.21.15.5:0/728369454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7124107140 0x7f7124074840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.378+0000 7f7121e00640 1 -- 172.21.15.5:0/728369454 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71241a7030 con 0x7f7124102070 2024-04-24T06:51:22.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.378+0000 7f7121e00640 1 --2- 172.21.15.5:0/728369454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7124102070 0x7f7124074300 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7114004770 tx=0x7f7114003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:22.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.378+0000 7f711aa00640 1 -- 172.21.15.5:0/728369454 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711403f070 con 0x7f7124102070 2024-04-24T06:51:22.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.379+0000 7f711aa00640 1 -- 172.21.15.5:0/728369454 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7114031ca0 con 0x7f7124102070 2024-04-24T06:51:22.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.379+0000 7f711aa00640 1 -- 172.21.15.5:0/728369454 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7114035030 con 0x7f7124102070 2024-04-24T06:51:22.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.379+0000 7f7123200640 1 -- 172.21.15.5:0/728369454 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71241a72c0 con 0x7f7124102070 2024-04-24T06:51:22.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.379+0000 7f7123200640 1 -- 172.21.15.5:0/728369454 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71241a7760 con 0x7f7124102070 2024-04-24T06:51:22.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.382+0000 7f712bb6d740 1 Processor -- start 2024-04-24T06:51:22.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.382+0000 7f711aa00640 1 -- 172.21.15.5:0/728369454 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f7114042070 con 0x7f7124102070 2024-04-24T06:51:22.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.382+0000 7f712bb6d740 1 -- start start 2024-04-24T06:51:22.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.382+0000 7f712bb6d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf1211ab40 0x55bf1211af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:22.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.382+0000 7f712bb6d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf1211b460 0x55bf121255f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:22.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.382+0000 7f712bb6d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bf12125b30 0x55bf12127f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:22.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.382+0000 7f712bb6d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bf120801d0 con 0x55bf1211ab40 2024-04-24T06:51:22.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.382+0000 7f712bb6d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bf12026450 con 0x55bf1211b460 2024-04-24T06:51:22.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.382+0000 7f712bb6d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55bf1207c190 con 0x55bf12125b30 2024-04-24T06:51:22.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.383+0000 7f7121e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf1211b460 0x55bf121255f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:22.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.383+0000 7f7122800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf1211ab40 0x55bf1211af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:22.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.383+0000 7f7121e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf1211b460 0x55bf121255f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:39598/0 (socket says 172.21.15.5:39598) 2024-04-24T06:51:22.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.383+0000 7f7121e00640 1 -- 172.21.15.5:0/1683559237 learned_addr learned my addr 172.21.15.5:0/1683559237 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:22.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.383+0000 7f7121400640 1 --2- 172.21.15.5:0/1683559237 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bf12125b30 0x55bf12127f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:22.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.383+0000 7f7121e00640 1 -- 172.21.15.5:0/1683559237 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bf12125b30 msgr2=0x55bf12127f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:22.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.383+0000 7f7121e00640 1 --2- 172.21.15.5:0/1683559237 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bf12125b30 0x55bf12127f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.383+0000 7f7121e00640 1 -- 172.21.15.5:0/1683559237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf1211ab40 msgr2=0x55bf1211af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:22.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.383+0000 7f7121e00640 1 --2- 172.21.15.5:0/1683559237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf1211ab40 0x55bf1211af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.383+0000 7f7121e00640 1 -- 172.21.15.5:0/1683559237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf120e3310 con 0x55bf1211b460 2024-04-24T06:51:22.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.383+0000 7f7121400640 1 --2- 172.21.15.5:0/1683559237 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bf12125b30 0x55bf12127f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:51:22.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.383+0000 7f7122800640 1 --2- 172.21.15.5:0/1683559237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf1211ab40 0x55bf1211af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:51:22.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.384+0000 7f7121e00640 1 --2- 172.21.15.5:0/1683559237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf1211b460 0x55bf121255f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f71140ee510 tx=0x7f71140f91a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3b794b0442ac697 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:22.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.384+0000 7f711aa00640 1 --2- 172.21.15.5:0/728369454 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f71080a3d50 0x7f71080a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:22.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.384+0000 7f710f400640 1 -- 172.21.15.5:0/1683559237 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71140b9170 con 0x55bf1211b460 2024-04-24T06:51:22.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.384+0000 7f710f400640 1 -- 172.21.15.5:0/1683559237 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71140fe070 con 0x55bf1211b460 2024-04-24T06:51:22.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.384+0000 7f710f400640 1 -- 172.21.15.5:0/1683559237 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71140bc650 con 0x55bf1211b460 2024-04-24T06:51:22.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.384+0000 7f711aa00640 1 -- 172.21.15.5:0/728369454 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f71140f0ef0 con 0x7f7124102070 2024-04-24T06:51:22.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.384+0000 7f7122800640 1 --2- 172.21.15.5:0/728369454 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f71080a3d50 0x7f71080a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:22.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.385+0000 7f712bb6d740 1 -- 172.21.15.5:0/1683559237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf1211b460 msgr2=0x55bf121255f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:22.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.385+0000 7f712bb6d740 1 --2- 172.21.15.5:0/1683559237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf1211b460 0x55bf121255f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f71140ee510 tx=0x7f71140f91a0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.385+0000 7f712bb6d740 1 -- 172.21.15.5:0/1683559237 shutdown_connections 2024-04-24T06:51:22.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.385+0000 7f712bb6d740 1 --2- 172.21.15.5:0/1683559237 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bf12125b30 0x55bf12127f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.385+0000 7f712bb6d740 1 --2- 172.21.15.5:0/1683559237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf1211b460 0x55bf121255f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.385+0000 7f712bb6d740 1 --2- 172.21.15.5:0/1683559237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf1211ab40 0x55bf1211af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.385+0000 7f712bb6d740 1 -- 172.21.15.5:0/1683559237 >> 172.21.15.5:0/1683559237 conn(0x55bf12116e60 msgr2=0x55bf12117240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:22.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.385+0000 7f712bb6d740 1 -- 172.21.15.5:0/1683559237 shutdown_connections 2024-04-24T06:51:22.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.385+0000 7f7122800640 1 --2- 172.21.15.5:0/728369454 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f71080a3d50 0x7f71080a6210 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f711c005fd0 tx=0x7f711c009600 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:22.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.385+0000 7f712bb6d740 1 -- 172.21.15.5:0/1683559237 wait complete. 2024-04-24T06:51:22.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.385+0000 7f712bb6d740 1 -- start start 2024-04-24T06:51:22.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.386+0000 7f712bb6d740 1 Processor -- start 2024-04-24T06:51:22.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.386+0000 7f712bb6d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf1211ab40 0x55bf121255e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:22.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.386+0000 7f712bb6d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bf1211b460 0x55bf121228d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:22.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.386+0000 7f712bb6d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf12125b30 0x55bf12123010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:22.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.386+0000 7f712bb6d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bf121286e0 con 0x55bf1211ab40 2024-04-24T06:51:22.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.386+0000 7f712bb6d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bf12039810 con 0x55bf12125b30 2024-04-24T06:51:22.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.386+0000 7f712bb6d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55bf12128860 con 0x55bf1211b460 2024-04-24T06:51:22.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.386+0000 7f712bb6d740 10 client.0 _ll_register_callbacks cb 0x55bf12007140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:51:22.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.386+0000 7f7121400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bf1211b460 0x55bf121228d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:22.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.386+0000 7f7121e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf1211ab40 0x55bf121255e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:22.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.386+0000 7f7122800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf12125b30 0x55bf12123010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:22.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.386+0000 7f7121400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bf1211b460 0x55bf121228d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:52170/0 (socket says 172.21.15.5:52170) 2024-04-24T06:51:22.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.386+0000 7f7121400640 1 -- 172.21.15.5:0/325893716 learned_addr learned my addr 172.21.15.5:0/325893716 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:22.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.387+0000 7f7121400640 1 -- 172.21.15.5:0/325893716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf12125b30 msgr2=0x55bf12123010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:22.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.387+0000 7f7121400640 1 --2- 172.21.15.5:0/325893716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf12125b30 0x55bf12123010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.387+0000 7f7121400640 1 -- 172.21.15.5:0/325893716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf1211ab40 msgr2=0x55bf121255e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:22.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.387+0000 7f7121400640 1 --2- 172.21.15.5:0/325893716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf1211ab40 0x55bf121255e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.387+0000 7f7121400640 1 -- 172.21.15.5:0/325893716 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf1204a3f0 con 0x55bf1211b460 2024-04-24T06:51:22.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.387+0000 7f7121400640 1 --2- 172.21.15.5:0/325893716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bf1211b460 0x55bf121228d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f71100126e0 tx=0x7f7110012ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78a1a69596fa6bb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:22.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.387+0000 7f7103e00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:51:22.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.387+0000 7f7103e00640 1 -- 172.21.15.5:0/325893716 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110004050 con 0x55bf1211b460 2024-04-24T06:51:22.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.387+0000 7f712bb6d740 1 -- 172.21.15.5:0/325893716 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71140fe070 con 0x55bf1211b460 2024-04-24T06:51:22.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.387+0000 7f7103e00640 1 -- 172.21.15.5:0/325893716 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f71100041f0 con 0x55bf1211b460 2024-04-24T06:51:22.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.388+0000 7f7103e00640 1 -- 172.21.15.5:0/325893716 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f71100043f0 con 0x55bf1211b460 2024-04-24T06:51:22.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.388+0000 7f712bb6d740 10 client.26863 fetch_fsmap learned FSMap version 27 2024-04-24T06:51:22.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.388+0000 7f712bb6d740 1 -- 172.21.15.5:0/325893716 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bf11d923f0 con 0x55bf1211b460 2024-04-24T06:51:22.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.388+0000 7f7103e00640 1 -- 172.21.15.5:0/325893716 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71140fe070 con 0x55bf1211b460 2024-04-24T06:51:22.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.388+0000 7f7103e00640 1 -- 172.21.15.5:0/325893716 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f71100039f0 con 0x55bf1211b460 2024-04-24T06:51:22.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.388+0000 7f712bb6d740 10 client.26863 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:51:22.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.388+0000 7f712bb6d740 10 client.26863 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:51:22.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.388+0000 7f712bb6d740 20 client.26863 populate_metadata read hostname 'smithi005' 2024-04-24T06:51:22.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.388+0000 7f712bb6d740 1 --2- 172.21.15.5:0/325893716 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55bf12135a80 0x55bf12137e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:22.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.388+0000 7f712bb6d740 4 client.26863 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:51:22.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.388+0000 7f712bb6d740 1 -- 172.21.15.5:0/325893716 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf1208dbf0 con 0x55bf12135a80 2024-04-24T06:51:22.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.388+0000 7f7121e00640 1 --2- 172.21.15.5:0/325893716 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55bf12135a80 0x55bf12137e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:22.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.389+0000 7f7121e00640 1 --2- 172.21.15.5:0/325893716 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55bf12135a80 0x55bf12137e90 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe523daf195a7a3d server_cookie=c2d09cd210902306 in_seq=0 out_seq=0 2024-04-24T06:51:22.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.389+0000 7f7103e00640 10 client.26863 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:51:22.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.390+0000 7f7103e00640 1 -- 172.21.15.5:0/325893716 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bf1208dbf0 con 0x55bf12135a80 2024-04-24T06:51:22.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.390+0000 7f7103e00640 10 client.26863 handle_command_reply: tid=0 2024-04-24T06:51:22.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.390+0000 7f712bb6d740 1 client.26863 shutdown 2024-04-24T06:51:22.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.390+0000 7f712bb6d740 1 -- 172.21.15.5:0/325893716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bf1211b460 msgr2=0x55bf121228d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:22.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.390+0000 7f712bb6d740 1 --2- 172.21.15.5:0/325893716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bf1211b460 0x55bf121228d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f71100126e0 tx=0x7f7110012ae0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.390+0000 7f712bb6d740 1 -- 172.21.15.5:0/325893716 shutdown_connections 2024-04-24T06:51:22.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.390+0000 7f712bb6d740 1 --2- 172.21.15.5:0/325893716 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55bf12135a80 0x55bf12137e90 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.390+0000 7f712bb6d740 1 --2- 172.21.15.5:0/325893716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf12125b30 0x55bf12123010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.390+0000 7f712bb6d740 1 --2- 172.21.15.5:0/325893716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bf1211b460 0x55bf121228d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.390+0000 7f712bb6d740 1 --2- 172.21.15.5:0/325893716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf1211ab40 0x55bf121255e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.390+0000 7f7103e00640 0 client.26863 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:51:22.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.390+0000 7f712bb6d740 1 -- 172.21.15.5:0/325893716 >> 172.21.15.5:0/325893716 conn(0x55bf12116e60 msgr2=0x55bf1211cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:22.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.390+0000 7f712bb6d740 1 -- 172.21.15.5:0/325893716 shutdown_connections 2024-04-24T06:51:22.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.390+0000 7f712bb6d740 1 -- 172.21.15.5:0/325893716 wait complete. 2024-04-24T06:51:22.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.390+0000 7f712bb6d740 20 client.26863 trim_cache size 0 max 16384 2024-04-24T06:51:22.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.405+0000 7f712bb6d740 1 Processor -- start 2024-04-24T06:51:22.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.405+0000 7f712bb6d740 1 -- start start 2024-04-24T06:51:22.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.405+0000 7f712bb6d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf1211ab40 0x55bf121315a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:22.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.405+0000 7f712bb6d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf1211b460 0x55bf12125110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:22.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.405+0000 7f712bb6d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bf12125b30 0x55bf1213d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:22.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.405+0000 7f712bb6d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f71241b2720 con 0x55bf1211ab40 2024-04-24T06:51:22.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.405+0000 7f712bb6d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f71241afff0 con 0x55bf1211b460 2024-04-24T06:51:22.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.405+0000 7f712bb6d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f71241ade60 con 0x55bf12125b30 2024-04-24T06:51:22.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.406+0000 7f7121400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf1211b460 0x55bf12125110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:22.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.406+0000 7f7121400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf1211b460 0x55bf12125110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:39618/0 (socket says 172.21.15.5:39618) 2024-04-24T06:51:22.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.406+0000 7f7121400640 1 -- 172.21.15.5:0/1028792320 learned_addr learned my addr 172.21.15.5:0/1028792320 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:22.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.406+0000 7f7121e00640 1 --2- 172.21.15.5:0/1028792320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf1211ab40 0x55bf121315a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:22.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.406+0000 7f7122800640 1 --2- 172.21.15.5:0/1028792320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bf12125b30 0x55bf1213d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:22.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.406+0000 7f7121400640 1 -- 172.21.15.5:0/1028792320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bf12125b30 msgr2=0x55bf1213d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:22.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.406+0000 7f7121400640 1 --2- 172.21.15.5:0/1028792320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bf12125b30 0x55bf1213d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.406+0000 7f7121400640 1 -- 172.21.15.5:0/1028792320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf1211ab40 msgr2=0x55bf121315a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:22.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.406+0000 7f7121400640 1 --2- 172.21.15.5:0/1028792320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf1211ab40 0x55bf121315a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.406+0000 7f7121400640 1 -- 172.21.15.5:0/1028792320 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf12123380 con 0x55bf1211b460 2024-04-24T06:51:22.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.407+0000 7f7121400640 1 --2- 172.21.15.5:0/1028792320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf1211b460 0x55bf12125110 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7110005bc0 tx=0x7f711000f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=482168f2725dbe2f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:22.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.407+0000 7f710ea00640 1 -- 172.21.15.5:0/1028792320 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110028070 con 0x55bf1211b460 2024-04-24T06:51:22.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.407+0000 7f710ea00640 1 -- 172.21.15.5:0/1028792320 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7110005520 con 0x55bf1211b460 2024-04-24T06:51:22.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.407+0000 7f710ea00640 1 -- 172.21.15.5:0/1028792320 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711000f670 con 0x55bf1211b460 2024-04-24T06:51:22.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.407+0000 7f712bb6d740 1 -- 172.21.15.5:0/1028792320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf1211b460 msgr2=0x55bf12125110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:22.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.407+0000 7f712bb6d740 1 --2- 172.21.15.5:0/1028792320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf1211b460 0x55bf12125110 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7110005bc0 tx=0x7f711000f3c0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.408+0000 7f712bb6d740 1 -- 172.21.15.5:0/1028792320 shutdown_connections 2024-04-24T06:51:22.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.408+0000 7f712bb6d740 1 --2- 172.21.15.5:0/1028792320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bf12125b30 0x55bf1213d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.408+0000 7f712bb6d740 1 --2- 172.21.15.5:0/1028792320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf1211b460 0x55bf12125110 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.408+0000 7f712bb6d740 1 --2- 172.21.15.5:0/1028792320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf1211ab40 0x55bf121315a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.408+0000 7f712bb6d740 1 -- 172.21.15.5:0/1028792320 >> 172.21.15.5:0/1028792320 conn(0x55bf12116e60 msgr2=0x55bf12117240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:22.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.408+0000 7f712bb6d740 1 -- 172.21.15.5:0/1028792320 shutdown_connections 2024-04-24T06:51:22.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.408+0000 7f712bb6d740 1 -- 172.21.15.5:0/1028792320 wait complete. 2024-04-24T06:51:22.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.408+0000 7f712bb6d740 1 -- start start 2024-04-24T06:51:22.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.408+0000 7f712bb6d740 1 Processor -- start 2024-04-24T06:51:22.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.409+0000 7f712bb6d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf1211ab40 0x55bf1212dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:22.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.409+0000 7f712bb6d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bf1211b460 0x55bf1212e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:22.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.409+0000 7f712bb6d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf12125b30 0x55bf12128c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:22.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.409+0000 7f712bb6d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f71241ae4b0 con 0x55bf1211ab40 2024-04-24T06:51:22.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.409+0000 7f712bb6d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f71241af920 con 0x55bf12125b30 2024-04-24T06:51:22.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.409+0000 7f712bb6d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f71241aeb80 con 0x55bf1211b460 2024-04-24T06:51:22.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.409+0000 7f712bb6d740 10 client.0 _ll_register_callbacks cb 0x7f7110009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:51:22.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.409+0000 7f7121e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf1211ab40 0x55bf1212dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:22.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.409+0000 7f7121400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bf1211b460 0x55bf1212e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:22.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.409+0000 7f7122800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf12125b30 0x55bf12128c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:22.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.409+0000 7f7121400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bf1211b460 0x55bf1212e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:52182/0 (socket says 172.21.15.5:52182) 2024-04-24T06:51:22.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.409+0000 7f7122800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf12125b30 0x55bf12128c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:39632/0 (socket says 172.21.15.5:39632) 2024-04-24T06:51:22.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.409+0000 7f7121400640 1 -- 172.21.15.5:0/1963154570 learned_addr learned my addr 172.21.15.5:0/1963154570 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:22.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.409+0000 7f7121400640 1 -- 172.21.15.5:0/1963154570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf12125b30 msgr2=0x55bf12128c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:22.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.409+0000 7f7121400640 1 --2- 172.21.15.5:0/1963154570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf12125b30 0x55bf12128c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.409+0000 7f7121400640 1 -- 172.21.15.5:0/1963154570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf1211ab40 msgr2=0x55bf1212dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:22.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.409+0000 7f7121400640 1 --2- 172.21.15.5:0/1963154570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf1211ab40 0x55bf1212dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.409+0000 7f7121400640 1 -- 172.21.15.5:0/1963154570 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf11e5c7d0 con 0x55bf1211b460 2024-04-24T06:51:22.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.409+0000 7f7121e00640 1 --2- 172.21.15.5:0/1963154570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf1211ab40 0x55bf1212dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:51:22.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.410+0000 7f7121400640 1 --2- 172.21.15.5:0/1963154570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bf1211b460 0x55bf1212e000 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7110012420 tx=0x7f7110003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1005fcd9b2540c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:22.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.410+0000 7f7103e00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:51:22.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.410+0000 7f7103e00640 1 -- 172.21.15.5:0/1963154570 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110028070 con 0x55bf1211b460 2024-04-24T06:51:22.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.410+0000 7f712bb6d740 1 -- 172.21.15.5:0/1963154570 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7110005520 con 0x55bf1211b460 2024-04-24T06:51:22.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.410+0000 7f7103e00640 1 -- 172.21.15.5:0/1963154570 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f7110003ba0 con 0x55bf1211b460 2024-04-24T06:51:22.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.410+0000 7f7103e00640 1 -- 172.21.15.5:0/1963154570 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f711000f670 con 0x55bf1211b460 2024-04-24T06:51:22.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.410+0000 7f712bb6d740 10 client.26869 fetch_fsmap learned FSMap version 27 2024-04-24T06:51:22.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.410+0000 7f712bb6d740 1 -- 172.21.15.5:0/1963154570 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bf1219ca50 con 0x55bf1211b460 2024-04-24T06:51:22.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.410+0000 7f7103e00640 1 -- 172.21.15.5:0/1963154570 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7110005520 con 0x55bf1211b460 2024-04-24T06:51:22.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.411+0000 7f7103e00640 1 -- 172.21.15.5:0/1963154570 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f7110010af0 con 0x55bf1211b460 2024-04-24T06:51:22.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.411+0000 7f712bb6d740 10 client.26869 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:51:22.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.411+0000 7f712bb6d740 10 client.26869 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:51:22.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.411+0000 7f712bb6d740 20 client.26869 populate_metadata read hostname 'smithi005' 2024-04-24T06:51:22.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.411+0000 7f712bb6d740 1 --2- 172.21.15.5:0/1963154570 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55bf1219d090 0x55bf1219f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:22.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.411+0000 7f712bb6d740 4 client.26869 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:51:22.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.411+0000 7f712bb6d740 1 -- 172.21.15.5:0/1963154570 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bf12087a60 con 0x55bf1219d090 2024-04-24T06:51:22.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.411+0000 7f7121e00640 1 --2- 172.21.15.5:0/1963154570 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55bf1219d090 0x55bf1219f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:22.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.411+0000 7f7121e00640 1 --2- 172.21.15.5:0/1963154570 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55bf1219d090 0x55bf1219f4b0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f66bd0d6e70c320d server_cookie=5d1a69316d7822e8 in_seq=0 out_seq=0 2024-04-24T06:51:22.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.411+0000 7f7103e00640 10 client.26869 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:51:22.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.412+0000 7f7103e00640 1 -- 172.21.15.5:0/1963154570 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bf12087a60 con 0x55bf1219d090 2024-04-24T06:51:22.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.412+0000 7f7103e00640 10 client.26869 handle_command_reply: tid=0 2024-04-24T06:51:22.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.412+0000 7f712bb6d740 1 client.26869 shutdown 2024-04-24T06:51:22.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.412+0000 7f712bb6d740 1 -- 172.21.15.5:0/1963154570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bf1211b460 msgr2=0x55bf1212e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:22.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.412+0000 7f712bb6d740 1 --2- 172.21.15.5:0/1963154570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bf1211b460 0x55bf1212e000 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7110012420 tx=0x7f7110003820 comp rx=0 tx=0).stop 2024-04-24T06:51:22.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.412+0000 7f712bb6d740 1 -- 172.21.15.5:0/1963154570 shutdown_connections 2024-04-24T06:51:22.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.412+0000 7f712bb6d740 1 --2- 172.21.15.5:0/1963154570 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55bf1219d090 0x55bf1219f4b0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.412+0000 7f712bb6d740 1 --2- 172.21.15.5:0/1963154570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf12125b30 0x55bf12128c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.412+0000 7f7103e00640 0 client.26869 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:51:22.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.412+0000 7f712bb6d740 1 --2- 172.21.15.5:0/1963154570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bf1211b460 0x55bf1212e000 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.412+0000 7f712bb6d740 1 --2- 172.21.15.5:0/1963154570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf1211ab40 0x55bf1212dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.412+0000 7f712bb6d740 1 -- 172.21.15.5:0/1963154570 >> 172.21.15.5:0/1963154570 conn(0x55bf12116e60 msgr2=0x55bf121156c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:22.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.412+0000 7f712bb6d740 1 -- 172.21.15.5:0/1963154570 shutdown_connections 2024-04-24T06:51:22.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.412+0000 7f712bb6d740 1 -- 172.21.15.5:0/1963154570 wait complete. 2024-04-24T06:51:22.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.412+0000 7f712bb6d740 20 client.26869 trim_cache size 0 max 16384 2024-04-24T06:51:22.422 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:51:22.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.413+0000 7f7123200640 1 -- 172.21.15.5:0/728369454 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f71080a3d50 msgr2=0x7f71080a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:22.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.413+0000 7f7123200640 1 --2- 172.21.15.5:0/728369454 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f71080a3d50 0x7f71080a6210 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f711c005fd0 tx=0x7f711c009600 comp rx=0 tx=0).stop 2024-04-24T06:51:22.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.413+0000 7f7123200640 1 -- 172.21.15.5:0/728369454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7124102070 msgr2=0x7f7124074300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:22.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.413+0000 7f7123200640 1 --2- 172.21.15.5:0/728369454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7124102070 0x7f7124074300 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7114004770 tx=0x7f7114003c10 comp rx=0 tx=0).stop 2024-04-24T06:51:22.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.413+0000 7f7123200640 1 -- 172.21.15.5:0/728369454 shutdown_connections 2024-04-24T06:51:22.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.413+0000 7f7123200640 1 --2- 172.21.15.5:0/728369454 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f71080a3d50 0x7f71080a6210 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.413+0000 7f7123200640 1 --2- 172.21.15.5:0/728369454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7124074d80 0x7f71241a6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.413+0000 7f7123200640 1 --2- 172.21.15.5:0/728369454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7124107140 0x7f7124074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.413+0000 7f7123200640 1 --2- 172.21.15.5:0/728369454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7124102070 0x7f7124074300 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:22.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.413+0000 7f7123200640 1 -- 172.21.15.5:0/728369454 >> 172.21.15.5:0/728369454 conn(0x7f71240fd950 msgr2=0x7f71240ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:22.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.413+0000 7f7123200640 1 -- 172.21.15.5:0/728369454 shutdown_connections 2024-04-24T06:51:22.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:22.413+0000 7f7123200640 1 -- 172.21.15.5:0/728369454 wait complete. 2024-04-24T06:51:22.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:22 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/2633919844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:51:22.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:22 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:51:22.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:22 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/2633919844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:51:22.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:22 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:51:22.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:22 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/2633919844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:51:22.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:22 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:51:23.430 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T06:51:23.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:51:23.431 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T06:51:23.515 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:23 smithi005 ceph-mon[27798]: pgmap v225: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-24T06:51:23.515 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:23 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:51:23.515 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:23 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:51:23.515 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:23 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:51:23.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:23 smithi037 ceph-mon[32864]: pgmap v225: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-24T06:51:23.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:23 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:51:23.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:23 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:51:23.519 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:23 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:51:23.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.662+0000 7f4bc2800640 1 -- 172.21.15.5:0/941281910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bbc101510 msgr2=0x7f4bbc10e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:23.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.662+0000 7f4bc2800640 1 --2- 172.21.15.5:0/941281910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bbc101510 0x7f4bbc10e1d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8009900 tx=0x7f4ba802a3f0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.662+0000 7f4bc2800640 1 -- 172.21.15.5:0/941281910 shutdown_connections 2024-04-24T06:51:23.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.662+0000 7f4bc2800640 1 --2- 172.21.15.5:0/941281910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4bbc10e710 0x7f4bbc110b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.662+0000 7f4bc2800640 1 --2- 172.21.15.5:0/941281910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bbc101510 0x7f4bbc10e1d0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.662+0000 7f4bc2800640 1 --2- 172.21.15.5:0/941281910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4bbc100b60 0x7f4bbc100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.662+0000 7f4bc2800640 1 -- 172.21.15.5:0/941281910 >> 172.21.15.5:0/941281910 conn(0x7f4bbc077ab0 msgr2=0x7f4bbc0fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:23.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.663+0000 7f4bc2800640 1 -- 172.21.15.5:0/941281910 shutdown_connections 2024-04-24T06:51:23.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.663+0000 7f4bc2800640 1 -- 172.21.15.5:0/941281910 wait complete. 2024-04-24T06:51:23.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.664+0000 7f4bc2800640 1 Processor -- start 2024-04-24T06:51:23.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.664+0000 7f4bc2800640 1 -- start start 2024-04-24T06:51:23.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.664+0000 7f4bc2800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bbc100b60 0x7f4bbc1a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:23.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.664+0000 7f4bc2800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4bbc101510 0x7f4bbc1a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:23.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.664+0000 7f4bc2800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4bbc10e710 0x7f4bbc19a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:23.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.664+0000 7f4bc2800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc114470 con 0x7f4bbc10e710 2024-04-24T06:51:23.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.664+0000 7f4bc2800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc1142f0 con 0x7f4bbc100b60 2024-04-24T06:51:23.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.664+0000 7f4bc2800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc1145f0 con 0x7f4bbc101510 2024-04-24T06:51:23.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.665+0000 7f4bc1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4bbc10e710 0x7f4bbc19a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:23.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.665+0000 7f4bc1400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bbc100b60 0x7f4bbc1a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:23.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.665+0000 7f4bc1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4bbc10e710 0x7f4bbc19a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51784/0 (socket says 172.21.15.5:51784) 2024-04-24T06:51:23.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.665+0000 7f4bc1e00640 1 -- 172.21.15.5:0/3323931962 learned_addr learned my addr 172.21.15.5:0/3323931962 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:23.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.665+0000 7f4bc0a00640 1 --2- 172.21.15.5:0/3323931962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4bbc101510 0x7f4bbc1a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.665+0000 7f4bc1e00640 1 -- 172.21.15.5:0/3323931962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4bbc101510 msgr2=0x7f4bbc1a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.665+0000 7f4bc1e00640 1 --2- 172.21.15.5:0/3323931962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4bbc101510 0x7f4bbc1a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.665+0000 7f4bc1e00640 1 -- 172.21.15.5:0/3323931962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bbc100b60 msgr2=0x7f4bbc1a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.665+0000 7f4bc1e00640 1 --2- 172.21.15.5:0/3323931962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bbc100b60 0x7f4bbc1a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.665+0000 7f4bc1e00640 1 -- 172.21.15.5:0/3323931962 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bbc19b030 con 0x7f4bbc10e710 2024-04-24T06:51:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.666+0000 7f4bc1e00640 1 --2- 172.21.15.5:0/3323931962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4bbc10e710 0x7f4bbc19a770 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f4bb400eb10 tx=0x7f4bb400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:23.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.666+0000 7f4bba000640 1 -- 172.21.15.5:0/3323931962 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4004b20 con 0x7f4bbc10e710 2024-04-24T06:51:23.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.666+0000 7f4bc2800640 1 -- 172.21.15.5:0/3323931962 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bbc19b320 con 0x7f4bbc10e710 2024-04-24T06:51:23.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.666+0000 7f4bba000640 1 -- 172.21.15.5:0/3323931962 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bb4004cc0 con 0x7f4bbc10e710 2024-04-24T06:51:23.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.666+0000 7f4bba000640 1 -- 172.21.15.5:0/3323931962 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4010790 con 0x7f4bbc10e710 2024-04-24T06:51:23.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.666+0000 7f4bc2800640 1 -- 172.21.15.5:0/3323931962 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bbc1a75e0 con 0x7f4bbc10e710 2024-04-24T06:51:23.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.669+0000 7f4bc6d97740 1 Processor -- start 2024-04-24T06:51:23.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.670+0000 7f4bba000640 1 -- 172.21.15.5:0/3323931962 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f4bb4005020 con 0x7f4bbc10e710 2024-04-24T06:51:23.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.670+0000 7f4bc6d97740 1 -- start start 2024-04-24T06:51:23.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.670+0000 7f4bc6d97740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a62ec46fc0 0x55a62ec473a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:23.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.670+0000 7f4bc6d97740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a62ec47970 0x55a62ec5a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:23.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.670+0000 7f4bc6d97740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a62ec5aa70 0x55a62ec5ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:23.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.670+0000 7f4bc6d97740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a62ebace70 con 0x55a62ec5aa70 2024-04-24T06:51:23.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.670+0000 7f4bc6d97740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a62eb53380 con 0x55a62ec47970 2024-04-24T06:51:23.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.670+0000 7f4bc6d97740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a62eba8e30 con 0x55a62ec46fc0 2024-04-24T06:51:23.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.671+0000 7f4bc1400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a62ec47970 0x55a62ec5a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:23.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.671+0000 7f4bc0a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a62ec46fc0 0x55a62ec473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:23.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.671+0000 7f4bc1400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a62ec47970 0x55a62ec5a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:39668/0 (socket says 172.21.15.5:39668) 2024-04-24T06:51:23.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.671+0000 7f4bc1400640 1 -- 172.21.15.5:0/487825968 learned_addr learned my addr 172.21.15.5:0/487825968 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:23.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.671+0000 7f4bc0a00640 1 --2- 172.21.15.5:0/487825968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a62ec5aa70 0x55a62ec5ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:23.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.671+0000 7f4bc1400640 1 -- 172.21.15.5:0/487825968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a62ec46fc0 msgr2=0x55a62ec473a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:23.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.671+0000 7f4bc1400640 1 --2- 172.21.15.5:0/487825968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a62ec46fc0 0x55a62ec473a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.671+0000 7f4bc1400640 1 -- 172.21.15.5:0/487825968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a62ec5aa70 msgr2=0x55a62ec5ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:23.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.671+0000 7f4bc1400640 1 --2- 172.21.15.5:0/487825968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a62ec5aa70 0x55a62ec5ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.671+0000 7f4bc1400640 1 -- 172.21.15.5:0/487825968 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a62e9877e0 con 0x55a62ec47970 2024-04-24T06:51:23.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.672+0000 7f4bba000640 1 --2- 172.21.15.5:0/3323931962 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f4ba40a3d20 0x7f4ba40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:23.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.672+0000 7f4bc1400640 1 --2- 172.21.15.5:0/487825968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a62ec47970 0x55a62ec5a530 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0009cf0 tx=0x7f4bb0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8524916fb3006cd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:23.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.672+0000 7f4bba000640 1 -- 172.21.15.5:0/3323931962 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f4bb40d0f30 con 0x7f4bbc10e710 2024-04-24T06:51:23.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.672+0000 7f4bc0a00640 1 --2- 172.21.15.5:0/3323931962 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f4ba40a3d20 0x7f4ba40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:23.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.672+0000 7f4bae600640 1 -- 172.21.15.5:0/487825968 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb001e070 con 0x55a62ec47970 2024-04-24T06:51:23.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.672+0000 7f4bae600640 1 -- 172.21.15.5:0/487825968 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bb0019b10 con 0x55a62ec47970 2024-04-24T06:51:23.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.672+0000 7f4bae600640 1 -- 172.21.15.5:0/487825968 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb000e9f0 con 0x55a62ec47970 2024-04-24T06:51:23.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.673+0000 7f4bc6d97740 1 -- 172.21.15.5:0/487825968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a62ec47970 msgr2=0x55a62ec5a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:23.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.673+0000 7f4bc0a00640 1 --2- 172.21.15.5:0/3323931962 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f4ba40a3d20 0x7f4ba40a61e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8000c00 tx=0x7f4ba8041040 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:23.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.673+0000 7f4bc6d97740 1 --2- 172.21.15.5:0/487825968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a62ec47970 0x55a62ec5a530 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0009cf0 tx=0x7f4bb0011810 comp rx=0 tx=0).stop 2024-04-24T06:51:23.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.673+0000 7f4bc6d97740 1 -- 172.21.15.5:0/487825968 shutdown_connections 2024-04-24T06:51:23.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.673+0000 7f4bc6d97740 1 --2- 172.21.15.5:0/487825968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a62ec5aa70 0x55a62ec5ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.673+0000 7f4bc6d97740 1 --2- 172.21.15.5:0/487825968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a62ec47970 0x55a62ec5a530 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.673+0000 7f4bc6d97740 1 --2- 172.21.15.5:0/487825968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a62ec46fc0 0x55a62ec473a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.673+0000 7f4bc6d97740 1 -- 172.21.15.5:0/487825968 >> 172.21.15.5:0/487825968 conn(0x55a62ec43590 msgr2=0x55a62ec43970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:23.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.673+0000 7f4bc6d97740 1 -- 172.21.15.5:0/487825968 shutdown_connections 2024-04-24T06:51:23.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.673+0000 7f4bc6d97740 1 -- 172.21.15.5:0/487825968 wait complete. 2024-04-24T06:51:23.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.674+0000 7f4bc6d97740 1 -- start start 2024-04-24T06:51:23.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.674+0000 7f4bc6d97740 1 Processor -- start 2024-04-24T06:51:23.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.674+0000 7f4bc6d97740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a62ec46fc0 0x55a62ec579e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:23.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.674+0000 7f4bc6d97740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a62ec47970 0x55a62ec57f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:23.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.674+0000 7f4bc6d97740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a62ec5aa70 0x55a62ec58460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:23.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.674+0000 7f4bc6d97740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a62ec5d830 con 0x55a62ec47970 2024-04-24T06:51:23.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.674+0000 7f4bc6d97740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a62eb66660 con 0x55a62ec46fc0 2024-04-24T06:51:23.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.674+0000 7f4bc6d97740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a62ec5d9b0 con 0x55a62ec5aa70 2024-04-24T06:51:23.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.674+0000 7f4bc6d97740 10 client.0 _ll_register_callbacks cb 0x55a62eb54aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:51:23.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.675+0000 7f4bc0a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a62ec47970 0x55a62ec57f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:23.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.675+0000 7f4bc0a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a62ec47970 0x55a62ec57f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51808/0 (socket says 172.21.15.5:51808) 2024-04-24T06:51:23.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.675+0000 7f4bc1e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a62ec5aa70 0x55a62ec58460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:23.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.675+0000 7f4bc0a00640 1 -- 172.21.15.5:0/523433666 learned_addr learned my addr 172.21.15.5:0/523433666 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:23.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.675+0000 7f4bc1400640 1 --2- 172.21.15.5:0/523433666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a62ec46fc0 0x55a62ec579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:23.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.675+0000 7f4bc1e00640 1 -- 172.21.15.5:0/523433666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a62ec46fc0 msgr2=0x55a62ec579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:23.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.675+0000 7f4bc1e00640 1 --2- 172.21.15.5:0/523433666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a62ec46fc0 0x55a62ec579e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.675+0000 7f4bc1e00640 1 -- 172.21.15.5:0/523433666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a62ec47970 msgr2=0x55a62ec57f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:23.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.675+0000 7f4bc1e00640 1 --2- 172.21.15.5:0/523433666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a62ec47970 0x55a62ec57f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.675+0000 7f4bc1e00640 1 -- 172.21.15.5:0/523433666 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a62e9e7f70 con 0x55a62ec5aa70 2024-04-24T06:51:23.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.675+0000 7f4bc0a00640 1 --2- 172.21.15.5:0/523433666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a62ec47970 0x55a62ec57f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:51:23.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.675+0000 7f4bc1e00640 1 --2- 172.21.15.5:0/523433666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a62ec5aa70 0x55a62ec58460 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4bb40ce5e0 tx=0x7f4bb409c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=742707498c047722 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:23.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.675+0000 7f4ba2a00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:51:23.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.676+0000 7f4ba2a00640 1 -- 172.21.15.5:0/523433666 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb40b5070 con 0x55a62ec5aa70 2024-04-24T06:51:23.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.676+0000 7f4ba2a00640 1 -- 172.21.15.5:0/523433666 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f4bb40a5640 con 0x55a62ec5aa70 2024-04-24T06:51:23.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.676+0000 7f4bc6d97740 1 -- 172.21.15.5:0/523433666 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bb0019b10 con 0x55a62ec5aa70 2024-04-24T06:51:23.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.676+0000 7f4ba2a00640 1 -- 172.21.15.5:0/523433666 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4bb40a1210 con 0x55a62ec5aa70 2024-04-24T06:51:23.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.676+0000 7f4bc6d97740 10 client.26881 fetch_fsmap learned FSMap version 27 2024-04-24T06:51:23.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.676+0000 7f4bc6d97740 1 -- 172.21.15.5:0/523433666 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a62e87f170 con 0x55a62ec5aa70 2024-04-24T06:51:23.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.676+0000 7f4ba2a00640 1 -- 172.21.15.5:0/523433666 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bb0019b10 con 0x55a62ec5aa70 2024-04-24T06:51:23.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.676+0000 7f4ba2a00640 1 -- 172.21.15.5:0/523433666 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f4bb4005310 con 0x55a62ec5aa70 2024-04-24T06:51:23.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.676+0000 7f4bc6d97740 10 client.26881 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:51:23.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.676+0000 7f4bc6d97740 10 client.26881 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:51:23.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.677+0000 7f4bc6d97740 20 client.26881 populate_metadata read hostname 'smithi005' 2024-04-24T06:51:23.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.677+0000 7f4bc6d97740 1 --2- 172.21.15.5:0/523433666 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a62ec6ae40 0x55a62ec6d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:23.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.677+0000 7f4bc6d97740 4 client.26881 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:51:23.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.677+0000 7f4bc6d97740 1 -- 172.21.15.5:0/523433666 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a62ebba890 con 0x55a62ec6ae40 2024-04-24T06:51:23.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.677+0000 7f4bc1400640 1 --2- 172.21.15.5:0/523433666 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a62ec6ae40 0x55a62ec6d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:23.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.678+0000 7f4bc1400640 1 --2- 172.21.15.5:0/523433666 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a62ec6ae40 0x55a62ec6d270 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fba72a9d4924be4c server_cookie=743baf0723d2ec7e in_seq=0 out_seq=0 2024-04-24T06:51:23.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.678+0000 7f4ba2a00640 10 client.26881 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:51:23.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.678+0000 7f4ba2a00640 1 -- 172.21.15.5:0/523433666 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a62ebba890 con 0x55a62ec6ae40 2024-04-24T06:51:23.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.678+0000 7f4ba2a00640 10 client.26881 handle_command_reply: tid=0 2024-04-24T06:51:23.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.678+0000 7f4bc6d97740 1 client.26881 shutdown 2024-04-24T06:51:23.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.679+0000 7f4bc6d97740 1 -- 172.21.15.5:0/523433666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a62ec5aa70 msgr2=0x55a62ec58460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:23.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.679+0000 7f4bc6d97740 1 --2- 172.21.15.5:0/523433666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a62ec5aa70 0x55a62ec58460 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4bb40ce5e0 tx=0x7f4bb409c880 comp rx=0 tx=0).stop 2024-04-24T06:51:23.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.679+0000 7f4bc6d97740 1 -- 172.21.15.5:0/523433666 shutdown_connections 2024-04-24T06:51:23.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.679+0000 7f4bc6d97740 1 --2- 172.21.15.5:0/523433666 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a62ec6ae40 0x55a62ec6d270 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.679+0000 7f4bc6d97740 1 --2- 172.21.15.5:0/523433666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a62ec5aa70 0x55a62ec58460 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.679+0000 7f4bc6d97740 1 --2- 172.21.15.5:0/523433666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a62ec47970 0x55a62ec57f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.679+0000 7f4bc6d97740 1 --2- 172.21.15.5:0/523433666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a62ec46fc0 0x55a62ec579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.679+0000 7f4ba2a00640 0 client.26881 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:51:23.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.679+0000 7f4bc6d97740 1 -- 172.21.15.5:0/523433666 >> 172.21.15.5:0/523433666 conn(0x55a62ec43590 msgr2=0x55a62ec49400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:23.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.679+0000 7f4bc6d97740 1 -- 172.21.15.5:0/523433666 shutdown_connections 2024-04-24T06:51:23.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.679+0000 7f4bc6d97740 1 -- 172.21.15.5:0/523433666 wait complete. 2024-04-24T06:51:23.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.679+0000 7f4bc6d97740 20 client.26881 trim_cache size 0 max 16384 2024-04-24T06:51:23.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:23 smithi137 ceph-mon[33017]: pgmap v225: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-24T06:51:23.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:23 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:51:23.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:23 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:51:23.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:23 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:51:23.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.699+0000 7f4bc6d97740 1 Processor -- start 2024-04-24T06:51:23.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.699+0000 7f4bc6d97740 1 -- start start 2024-04-24T06:51:23.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.699+0000 7f4bc6d97740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a62ec46fc0 0x55a62ec66730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:23.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.699+0000 7f4bc6d97740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a62ec47970 0x55a62ec72b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:23.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.699+0000 7f4bc6d97740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a62ec5aa70 0x55a62ec73080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:23.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.699+0000 7f4bc6d97740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc1ae300 con 0x55a62ec47970 2024-04-24T06:51:23.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.699+0000 7f4bc6d97740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc1abbd0 con 0x55a62ec46fc0 2024-04-24T06:51:23.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.699+0000 7f4bc6d97740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc1a9a40 con 0x55a62ec5aa70 2024-04-24T06:51:23.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.700+0000 7f4bc1e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a62ec5aa70 0x55a62ec73080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:23.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.700+0000 7f4bc0a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a62ec47970 0x55a62ec72b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:23.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.700+0000 7f4bc1400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a62ec46fc0 0x55a62ec66730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:23.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.700+0000 7f4bc1e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a62ec5aa70 0x55a62ec73080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:52218/0 (socket says 172.21.15.5:52218) 2024-04-24T06:51:23.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.700+0000 7f4bc1e00640 1 -- 172.21.15.5:0/2769403906 learned_addr learned my addr 172.21.15.5:0/2769403906 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:23.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.700+0000 7f4bc1e00640 1 -- 172.21.15.5:0/2769403906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a62ec46fc0 msgr2=0x55a62ec66730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:23.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.700+0000 7f4bc1e00640 1 --2- 172.21.15.5:0/2769403906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a62ec46fc0 0x55a62ec66730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.700+0000 7f4bc1e00640 1 -- 172.21.15.5:0/2769403906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a62ec47970 msgr2=0x55a62ec72b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:23.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.700+0000 7f4bc1e00640 1 --2- 172.21.15.5:0/2769403906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a62ec47970 0x55a62ec72b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.700+0000 7f4bc1e00640 1 -- 172.21.15.5:0/2769403906 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a62ec531b0 con 0x55a62ec5aa70 2024-04-24T06:51:23.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.700+0000 7f4bc1400640 1 --2- 172.21.15.5:0/2769403906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a62ec46fc0 0x55a62ec66730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:51:23.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.700+0000 7f4bc1e00640 1 --2- 172.21.15.5:0/2769403906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a62ec5aa70 0x55a62ec73080 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4099830 tx=0x7f4bb409cfe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bd0f93039eda0c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:23.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.701+0000 7f4badc00640 1 -- 172.21.15.5:0/2769403906 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb40b5070 con 0x55a62ec5aa70 2024-04-24T06:51:23.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.701+0000 7f4badc00640 1 -- 172.21.15.5:0/2769403906 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bb4005310 con 0x55a62ec5aa70 2024-04-24T06:51:23.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.701+0000 7f4badc00640 1 -- 172.21.15.5:0/2769403906 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb409c2b0 con 0x55a62ec5aa70 2024-04-24T06:51:23.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.701+0000 7f4bc6d97740 1 -- 172.21.15.5:0/2769403906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a62ec5aa70 msgr2=0x55a62ec73080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:23.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.701+0000 7f4bc6d97740 1 --2- 172.21.15.5:0/2769403906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a62ec5aa70 0x55a62ec73080 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4099830 tx=0x7f4bb409cfe0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.701+0000 7f4bc6d97740 1 -- 172.21.15.5:0/2769403906 shutdown_connections 2024-04-24T06:51:23.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.701+0000 7f4bc6d97740 1 --2- 172.21.15.5:0/2769403906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a62ec5aa70 0x55a62ec73080 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.701+0000 7f4bc6d97740 1 --2- 172.21.15.5:0/2769403906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a62ec47970 0x55a62ec72b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.701+0000 7f4bc6d97740 1 --2- 172.21.15.5:0/2769403906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a62ec46fc0 0x55a62ec66730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.701+0000 7f4bc6d97740 1 -- 172.21.15.5:0/2769403906 >> 172.21.15.5:0/2769403906 conn(0x55a62ec43590 msgr2=0x55a62ec58230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:23.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.702+0000 7f4bc6d97740 1 -- 172.21.15.5:0/2769403906 shutdown_connections 2024-04-24T06:51:23.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.702+0000 7f4bc6d97740 1 -- 172.21.15.5:0/2769403906 wait complete. 2024-04-24T06:51:23.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.702+0000 7f4bc6d97740 1 -- start start 2024-04-24T06:51:23.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.702+0000 7f4bc6d97740 1 Processor -- start 2024-04-24T06:51:23.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.703+0000 7f4bc6d97740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a62ec46fc0 0x55a62ec54a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:23.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.703+0000 7f4bc6d97740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a62ec47970 0x55a62ec54fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:23.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.703+0000 7f4bc6d97740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a62ec5aa70 0x55a62ecd03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:23.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.703+0000 7f4bc6d97740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc1aa090 con 0x55a62ec46fc0 2024-04-24T06:51:23.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.703+0000 7f4bc6d97740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc1ab500 con 0x55a62ec47970 2024-04-24T06:51:23.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.703+0000 7f4bc6d97740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc1aa760 con 0x55a62ec5aa70 2024-04-24T06:51:23.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.703+0000 7f4bc6d97740 10 client.0 _ll_register_callbacks cb 0x7f4bb4005870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:51:23.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.703+0000 7f4bc1400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a62ec46fc0 0x55a62ec54a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:23.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.703+0000 7f4bc1400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a62ec46fc0 0x55a62ec54a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51832/0 (socket says 172.21.15.5:51832) 2024-04-24T06:51:23.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.703+0000 7f4bc0a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a62ec47970 0x55a62ec54fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:23.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.703+0000 7f4bc1400640 1 -- 172.21.15.5:0/1631088918 learned_addr learned my addr 172.21.15.5:0/1631088918 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:23.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.703+0000 7f4bc1e00640 1 --2- 172.21.15.5:0/1631088918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a62ec5aa70 0x55a62ecd03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:23.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.703+0000 7f4bc0a00640 1 -- 172.21.15.5:0/1631088918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a62ec5aa70 msgr2=0x55a62ecd03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:23.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.703+0000 7f4bc0a00640 1 --2- 172.21.15.5:0/1631088918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a62ec5aa70 0x55a62ecd03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.703+0000 7f4bc0a00640 1 -- 172.21.15.5:0/1631088918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a62ec46fc0 msgr2=0x55a62ec54a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:23.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.703+0000 7f4bc0a00640 1 --2- 172.21.15.5:0/1631088918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a62ec46fc0 0x55a62ec54a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.703+0000 7f4bc0a00640 1 -- 172.21.15.5:0/1631088918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a62ec55890 con 0x55a62ec47970 2024-04-24T06:51:23.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.703+0000 7f4bc1e00640 1 --2- 172.21.15.5:0/1631088918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a62ec5aa70 0x55a62ecd03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:51:23.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.703+0000 7f4bc1400640 1 --2- 172.21.15.5:0/1631088918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a62ec46fc0 0x55a62ec54a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:51:23.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.704+0000 7f4bc0a00640 1 --2- 172.21.15.5:0/1631088918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a62ec47970 0x55a62ec54fc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x55a62ec58fb0 tx=0x7f4ba802a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=721e7e2e1763c099 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:23.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.704+0000 7f4ba2a00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:51:23.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.704+0000 7f4ba2a00640 1 -- 172.21.15.5:0/1631088918 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba8035070 con 0x55a62ec47970 2024-04-24T06:51:23.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.704+0000 7f4bc6d97740 1 -- 172.21.15.5:0/1631088918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bb4005310 con 0x55a62ec47970 2024-04-24T06:51:23.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.704+0000 7f4ba2a00640 1 -- 172.21.15.5:0/1631088918 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f4ba80035f0 con 0x55a62ec47970 2024-04-24T06:51:23.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.704+0000 7f4ba2a00640 1 -- 172.21.15.5:0/1631088918 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4ba80037f0 con 0x55a62ec47970 2024-04-24T06:51:23.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.704+0000 7f4bc6d97740 10 client.26876 fetch_fsmap learned FSMap version 27 2024-04-24T06:51:23.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.704+0000 7f4bc6d97740 1 -- 172.21.15.5:0/1631088918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a62ecd1a80 con 0x55a62ec47970 2024-04-24T06:51:23.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.705+0000 7f4ba2a00640 1 -- 172.21.15.5:0/1631088918 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bb4005310 con 0x55a62ec47970 2024-04-24T06:51:23.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.705+0000 7f4ba2a00640 1 -- 172.21.15.5:0/1631088918 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f4ba8054020 con 0x55a62ec47970 2024-04-24T06:51:23.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.705+0000 7f4bc6d97740 10 client.26876 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:51:23.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.705+0000 7f4bc6d97740 10 client.26876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:51:23.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.705+0000 7f4bc6d97740 20 client.26876 populate_metadata read hostname 'smithi005' 2024-04-24T06:51:23.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.705+0000 7f4bc6d97740 1 --2- 172.21.15.5:0/1631088918 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a62ecd21e0 0x55a62ecd4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:23.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.705+0000 7f4bc6d97740 4 client.26876 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T06:51:23.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.705+0000 7f4bc6d97740 1 -- 172.21.15.5:0/1631088918 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a62ebb4700 con 0x55a62ecd21e0 2024-04-24T06:51:23.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.705+0000 7f4bc1400640 1 --2- 172.21.15.5:0/1631088918 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a62ecd21e0 0x55a62ecd4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:23.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.706+0000 7f4bc1400640 1 --2- 172.21.15.5:0/1631088918 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a62ecd21e0 0x55a62ecd4600 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a1683d4aff98dfd server_cookie=e7cfce97d8ccfe35 in_seq=0 out_seq=0 2024-04-24T06:51:23.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.706+0000 7f4ba2a00640 10 client.26876 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:51:23.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.707+0000 7f4ba2a00640 1 -- 172.21.15.5:0/1631088918 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a62ebb4700 con 0x55a62ecd21e0 2024-04-24T06:51:23.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.707+0000 7f4ba2a00640 10 client.26876 handle_command_reply: tid=0 2024-04-24T06:51:23.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.707+0000 7f4bc6d97740 1 client.26876 shutdown 2024-04-24T06:51:23.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.707+0000 7f4bc6d97740 1 -- 172.21.15.5:0/1631088918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a62ec47970 msgr2=0x55a62ec54fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:23.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.707+0000 7f4bc6d97740 1 --2- 172.21.15.5:0/1631088918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a62ec47970 0x55a62ec54fc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x55a62ec58fb0 tx=0x7f4ba802a3f0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.708+0000 7f4bc6d97740 1 -- 172.21.15.5:0/1631088918 shutdown_connections 2024-04-24T06:51:23.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.708+0000 7f4bc6d97740 1 --2- 172.21.15.5:0/1631088918 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a62ecd21e0 0x55a62ecd4600 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.708+0000 7f4bc6d97740 1 --2- 172.21.15.5:0/1631088918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a62ec5aa70 0x55a62ecd03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.708+0000 7f4bc6d97740 1 --2- 172.21.15.5:0/1631088918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a62ec47970 0x55a62ec54fc0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.708+0000 7f4bc6d97740 1 --2- 172.21.15.5:0/1631088918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a62ec46fc0 0x55a62ec54a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.708+0000 7f4bc6d97740 1 -- 172.21.15.5:0/1631088918 >> 172.21.15.5:0/1631088918 conn(0x55a62ec43590 msgr2=0x55a62ec57980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:23.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.708+0000 7f4ba2a00640 0 client.26876 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:51:23.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.708+0000 7f4bc6d97740 1 -- 172.21.15.5:0/1631088918 shutdown_connections 2024-04-24T06:51:23.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.708+0000 7f4bc6d97740 1 -- 172.21.15.5:0/1631088918 wait complete. 2024-04-24T06:51:23.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.708+0000 7f4bc6d97740 20 client.26876 trim_cache size 0 max 16384 2024-04-24T06:51:23.716 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:51:23.717 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-24T06:51:23.717 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "cb698d49-14e7-4d2b-8fdb-1e4931817a5c", 2024-04-24T06:51:23.717 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-24T06:51:23.717 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:51:23.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.709+0000 7f4bc2800640 1 -- 172.21.15.5:0/3323931962 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f4ba40a3d20 msgr2=0x7f4ba40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:23.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.709+0000 7f4bc2800640 1 --2- 172.21.15.5:0/3323931962 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f4ba40a3d20 0x7f4ba40a61e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8000c00 tx=0x7f4ba8041040 comp rx=0 tx=0).stop 2024-04-24T06:51:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.709+0000 7f4bc2800640 1 -- 172.21.15.5:0/3323931962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4bbc10e710 msgr2=0x7f4bbc19a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.709+0000 7f4bc2800640 1 --2- 172.21.15.5:0/3323931962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4bbc10e710 0x7f4bbc19a770 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f4bb400eb10 tx=0x7f4bb400c2d0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.709+0000 7f4bc2800640 1 -- 172.21.15.5:0/3323931962 shutdown_connections 2024-04-24T06:51:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.709+0000 7f4bc2800640 1 --2- 172.21.15.5:0/3323931962 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f4ba40a3d20 0x7f4ba40a61e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.709+0000 7f4bc2800640 1 --2- 172.21.15.5:0/3323931962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4bbc10e710 0x7f4bbc19a770 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.709+0000 7f4bc2800640 1 --2- 172.21.15.5:0/3323931962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4bbc101510 0x7f4bbc1a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.709+0000 7f4bc2800640 1 --2- 172.21.15.5:0/3323931962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bbc100b60 0x7f4bbc1a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.709+0000 7f4bc2800640 1 -- 172.21.15.5:0/3323931962 >> 172.21.15.5:0/3323931962 conn(0x7f4bbc077ab0 msgr2=0x7f4bbc0ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.709+0000 7f4bc2800640 1 -- 172.21.15.5:0/3323931962 shutdown_connections 2024-04-24T06:51:23.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.709+0000 7f4bc2800640 1 -- 172.21.15.5:0/3323931962 wait complete. 2024-04-24T06:51:23.731 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T06:51:23.731 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T06:51:23.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.957+0000 7f3094a00640 1 -- 172.21.15.5:0/2259810262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3090102a20 msgr2=0x7f3090106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:23.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.957+0000 7f3094a00640 1 --2- 172.21.15.5:0/2259810262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3090102a20 0x7f3090106a60 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f307c009900 tx=0x7f307c02a3f0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.957+0000 7f3094a00640 1 -- 172.21.15.5:0/2259810262 shutdown_connections 2024-04-24T06:51:23.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.957+0000 7f3094a00640 1 --2- 172.21.15.5:0/2259810262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3090107140 0x7f309010ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.957+0000 7f3094a00640 1 --2- 172.21.15.5:0/2259810262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3090102a20 0x7f3090106a60 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.957+0000 7f3094a00640 1 --2- 172.21.15.5:0/2259810262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3090102070 0x7f3090102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.957+0000 7f3094a00640 1 -- 172.21.15.5:0/2259810262 >> 172.21.15.5:0/2259810262 conn(0x7f30900fd950 msgr2=0x7f30900ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:23.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.957+0000 7f3094a00640 1 -- 172.21.15.5:0/2259810262 shutdown_connections 2024-04-24T06:51:23.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.957+0000 7f3094a00640 1 -- 172.21.15.5:0/2259810262 wait complete. 2024-04-24T06:51:23.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.958+0000 7f3094a00640 1 Processor -- start 2024-04-24T06:51:23.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.959+0000 7f3094a00640 1 -- start start 2024-04-24T06:51:23.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.959+0000 7f3094a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3090102070 0x7f30901a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:23.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.959+0000 7f3094a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3090107140 0x7f30901a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:23.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.959+0000 7f3094a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f309019a730 0x7f309019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:23.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.959+0000 7f3094a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3090112200 con 0x7f309019a730 2024-04-24T06:51:23.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.959+0000 7f3094a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3090112080 con 0x7f3090102070 2024-04-24T06:51:23.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.959+0000 7f3094a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3090112380 con 0x7f3090107140 2024-04-24T06:51:23.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.960+0000 7f308f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3090102070 0x7f30901a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:23.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.960+0000 7f308f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3090102070 0x7f30901a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:50844/0 (socket says 172.21.15.5:50844) 2024-04-24T06:51:23.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.960+0000 7f308f400640 1 -- 172.21.15.5:0/3450881261 learned_addr learned my addr 172.21.15.5:0/3450881261 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:23.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.960+0000 7f308ea00640 1 --2- 172.21.15.5:0/3450881261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3090107140 0x7f30901a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:23.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.960+0000 7f308fe00640 1 --2- 172.21.15.5:0/3450881261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f309019a730 0x7f309019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:23.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.960+0000 7f308ea00640 1 -- 172.21.15.5:0/3450881261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3090102070 msgr2=0x7f30901a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:23.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.960+0000 7f308ea00640 1 --2- 172.21.15.5:0/3450881261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3090102070 0x7f30901a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.960+0000 7f308ea00640 1 -- 172.21.15.5:0/3450881261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f309019a730 msgr2=0x7f309019abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:23.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.960+0000 7f308ea00640 1 --2- 172.21.15.5:0/3450881261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f309019a730 0x7f309019abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.960+0000 7f308ea00640 1 -- 172.21.15.5:0/3450881261 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f309019b330 con 0x7f3090107140 2024-04-24T06:51:23.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.960+0000 7f308f400640 1 --2- 172.21.15.5:0/3450881261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3090102070 0x7f30901a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:51:23.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.961+0000 7f308fe00640 1 --2- 172.21.15.5:0/3450881261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f309019a730 0x7f309019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:51:23.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.961+0000 7f308ea00640 1 --2- 172.21.15.5:0/3450881261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3090107140 0x7f30901a0bb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f307c02a910 tx=0x7f307c003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:23.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.961+0000 7f3087e00640 1 -- 172.21.15.5:0/3450881261 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c03f070 con 0x7f3090107140 2024-04-24T06:51:23.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.961+0000 7f3094a00640 1 -- 172.21.15.5:0/3450881261 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f309019b5c0 con 0x7f3090107140 2024-04-24T06:51:23.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.961+0000 7f3094a00640 1 -- 172.21.15.5:0/3450881261 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30901a7750 con 0x7f3090107140 2024-04-24T06:51:23.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.961+0000 7f3087e00640 1 -- 172.21.15.5:0/3450881261 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f307c031c90 con 0x7f3090107140 2024-04-24T06:51:23.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.961+0000 7f3087e00640 1 -- 172.21.15.5:0/3450881261 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c035030 con 0x7f3090107140 2024-04-24T06:51:23.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.965+0000 7f3098e22740 1 Processor -- start 2024-04-24T06:51:23.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.966+0000 7f3098e22740 1 -- start start 2024-04-24T06:51:23.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.966+0000 7f3087e00640 1 -- 172.21.15.5:0/3450881261 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f307c042070 con 0x7f3090107140 2024-04-24T06:51:23.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.966+0000 7f3098e22740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c19c7cea30 0x55c19c7cee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:23.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.966+0000 7f3098e22740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c19c7cf350 0x55c19c7d94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:23.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.966+0000 7f3098e22740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c7d9a20 0x55c19c7dbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:23.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.966+0000 7f3098e22740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c19c7341f0 con 0x55c19c7d9a20 2024-04-24T06:51:23.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.966+0000 7f3098e22740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c19c6da350 con 0x55c19c7cea30 2024-04-24T06:51:23.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.966+0000 7f3098e22740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c19c7301b0 con 0x55c19c7cf350 2024-04-24T06:51:23.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.966+0000 7f308fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c19c7cea30 0x55c19c7cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:23.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.966+0000 7f308f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c19c7cf350 0x55c19c7d94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:23.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.966+0000 7f308f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c19c7cf350 0x55c19c7d94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:44918/0 (socket says 172.21.15.5:44918) 2024-04-24T06:51:23.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.966+0000 7f308fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c19c7cea30 0x55c19c7cee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:50850/0 (socket says 172.21.15.5:50850) 2024-04-24T06:51:23.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.966+0000 7f308f400640 1 -- 172.21.15.5:0/2781281521 learned_addr learned my addr 172.21.15.5:0/2781281521 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:23.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.967+0000 7f308ea00640 1 --2- 172.21.15.5:0/2781281521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c7d9a20 0x55c19c7dbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:23.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.967+0000 7f308f400640 1 -- 172.21.15.5:0/2781281521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c19c7cea30 msgr2=0x55c19c7cee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:23.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.967+0000 7f308f400640 1 --2- 172.21.15.5:0/2781281521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c19c7cea30 0x55c19c7cee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.967+0000 7f308f400640 1 -- 172.21.15.5:0/2781281521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c7d9a20 msgr2=0x55c19c7dbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:23.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.967+0000 7f308f400640 1 --2- 172.21.15.5:0/2781281521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c7d9a20 0x55c19c7dbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.967+0000 7f308f400640 1 -- 172.21.15.5:0/2781281521 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c19c72ef50 con 0x55c19c7cf350 2024-04-24T06:51:23.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.967+0000 7f308fe00640 1 --2- 172.21.15.5:0/2781281521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c19c7cea30 0x55c19c7cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:51:23.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.968+0000 7f308f400640 1 --2- 172.21.15.5:0/2781281521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c19c7cf350 0x55c19c7d94e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3080011650 tx=0x7f3080009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=637f667b74468081 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:23.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.968+0000 7f307be00640 1 -- 172.21.15.5:0/2781281521 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308001e070 con 0x55c19c7cf350 2024-04-24T06:51:23.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.968+0000 7f307be00640 1 -- 172.21.15.5:0/2781281521 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f308000fba0 con 0x55c19c7cf350 2024-04-24T06:51:23.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.968+0000 7f307be00640 1 -- 172.21.15.5:0/2781281521 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3080019cd0 con 0x55c19c7cf350 2024-04-24T06:51:23.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.968+0000 7f3087e00640 1 --2- 172.21.15.5:0/3450881261 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f30740a3da0 0x7f30740a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:23.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.968+0000 7f3087e00640 1 -- 172.21.15.5:0/3450881261 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f307c0f0cf0 con 0x7f3090107140 2024-04-24T06:51:23.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.969+0000 7f308fe00640 1 --2- 172.21.15.5:0/3450881261 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f30740a3da0 0x7f30740a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:23.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.969+0000 7f3098e22740 1 -- 172.21.15.5:0/2781281521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c19c7cf350 msgr2=0x55c19c7d94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:23.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.969+0000 7f3098e22740 1 --2- 172.21.15.5:0/2781281521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c19c7cf350 0x55c19c7d94e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3080011650 tx=0x7f3080009f90 comp rx=0 tx=0).stop 2024-04-24T06:51:23.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.969+0000 7f3098e22740 1 -- 172.21.15.5:0/2781281521 shutdown_connections 2024-04-24T06:51:23.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.969+0000 7f3098e22740 1 --2- 172.21.15.5:0/2781281521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c7d9a20 0x55c19c7dbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.969+0000 7f3098e22740 1 --2- 172.21.15.5:0/2781281521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c19c7cf350 0x55c19c7d94e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.969+0000 7f3098e22740 1 --2- 172.21.15.5:0/2781281521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c19c7cea30 0x55c19c7cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.969+0000 7f3098e22740 1 -- 172.21.15.5:0/2781281521 >> 172.21.15.5:0/2781281521 conn(0x55c19c7cad50 msgr2=0x55c19c7cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:23.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.969+0000 7f3098e22740 1 -- 172.21.15.5:0/2781281521 shutdown_connections 2024-04-24T06:51:23.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.969+0000 7f3098e22740 1 -- 172.21.15.5:0/2781281521 wait complete. 2024-04-24T06:51:23.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.970+0000 7f308fe00640 1 --2- 172.21.15.5:0/3450881261 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f30740a3da0 0x7f30740a6260 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3088005fd0 tx=0x7f3088009500 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:23.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.970+0000 7f3098e22740 1 -- start start 2024-04-24T06:51:23.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.970+0000 7f3098e22740 1 Processor -- start 2024-04-24T06:51:23.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.971+0000 7f3098e22740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c19c7cea30 0x55c19c7d6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:23.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.971+0000 7f3098e22740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c7cf350 0x55c19c7d6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:23.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.971+0000 7f3098e22740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c19c7d9a20 0x55c19c7d7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:23.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.971+0000 7f3098e22740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c19c7dc810 con 0x55c19c7cf350 2024-04-24T06:51:23.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.971+0000 7f3098e22740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c19c6ed690 con 0x55c19c7cea30 2024-04-24T06:51:23.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.971+0000 7f3098e22740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c19c7dc990 con 0x55c19c7d9a20 2024-04-24T06:51:23.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.971+0000 7f3098e22740 10 client.0 _ll_register_callbacks cb 0x55c19c729650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:51:23.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.971+0000 7f308f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c19c7cea30 0x55c19c7d6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:23.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.971+0000 7f308f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c19c7cea30 0x55c19c7d6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:50860/0 (socket says 172.21.15.5:50860) 2024-04-24T06:51:23.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.971+0000 7f308f400640 1 -- 172.21.15.5:0/1027868422 learned_addr learned my addr 172.21.15.5:0/1027868422 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:23.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.971+0000 7f308ea00640 1 --2- 172.21.15.5:0/1027868422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c7cf350 0x55c19c7d6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:23.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.971+0000 7f308fe00640 1 --2- 172.21.15.5:0/1027868422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c19c7d9a20 0x55c19c7d7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:23.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.972+0000 7f308f400640 1 -- 172.21.15.5:0/1027868422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c19c7d9a20 msgr2=0x55c19c7d7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:23.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.972+0000 7f308f400640 1 --2- 172.21.15.5:0/1027868422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c19c7d9a20 0x55c19c7d7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.972+0000 7f308f400640 1 -- 172.21.15.5:0/1027868422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c7cf350 msgr2=0x55c19c7d6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:23.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.972+0000 7f308f400640 1 --2- 172.21.15.5:0/1027868422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c7cf350 0x55c19c7d6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.972+0000 7f308f400640 1 -- 172.21.15.5:0/1027868422 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c19c6ec920 con 0x55c19c7cea30 2024-04-24T06:51:23.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.972+0000 7f308fe00640 1 --2- 172.21.15.5:0/1027868422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c19c7d9a20 0x55c19c7d7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:51:23.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.972+0000 7f308f400640 1 --2- 172.21.15.5:0/1027868422 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c19c7cea30 0x55c19c7d6990 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3080019990 tx=0x7f308000fb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd8c90df38c70eba server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:23.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.972+0000 7f3078c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:51:23.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.972+0000 7f3078c00640 1 -- 172.21.15.5:0/1027868422 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3080027060 con 0x55c19c7cea30 2024-04-24T06:51:23.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.972+0000 7f3078c00640 1 -- 172.21.15.5:0/1027868422 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f308000fdd0 con 0x55c19c7cea30 2024-04-24T06:51:23.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.972+0000 7f3098e22740 1 -- 172.21.15.5:0/1027868422 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f308000fba0 con 0x55c19c7cea30 2024-04-24T06:51:23.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.973+0000 7f3078c00640 1 -- 172.21.15.5:0/1027868422 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f308001e040 con 0x55c19c7cea30 2024-04-24T06:51:23.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.973+0000 7f3098e22740 10 client.26888 fetch_fsmap learned FSMap version 27 2024-04-24T06:51:23.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.973+0000 7f3098e22740 1 -- 172.21.15.5:0/1027868422 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c19c5107d0 con 0x55c19c7cea30 2024-04-24T06:51:23.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.973+0000 7f3078c00640 1 -- 172.21.15.5:0/1027868422 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f308000fba0 con 0x55c19c7cea30 2024-04-24T06:51:23.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.973+0000 7f3078c00640 1 -- 172.21.15.5:0/1027868422 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f3080021030 con 0x55c19c7cea30 2024-04-24T06:51:23.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.974+0000 7f3098e22740 10 client.26888 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:51:23.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.974+0000 7f3098e22740 10 client.26888 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:51:23.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.974+0000 7f3098e22740 20 client.26888 populate_metadata read hostname 'smithi005' 2024-04-24T06:51:23.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.974+0000 7f3098e22740 1 --2- 172.21.15.5:0/1027868422 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55c19c7e9e10 0x55c19c7ec200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:23.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.974+0000 7f3098e22740 4 client.26888 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:51:23.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.974+0000 7f3098e22740 1 -- 172.21.15.5:0/1027868422 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c19c741c10 con 0x55c19c7e9e10 2024-04-24T06:51:23.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.974+0000 7f308ea00640 1 --2- 172.21.15.5:0/1027868422 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55c19c7e9e10 0x55c19c7ec200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:23.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.975+0000 7f308ea00640 1 --2- 172.21.15.5:0/1027868422 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55c19c7e9e10 0x55c19c7ec200 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93ace30ea82c70b6 server_cookie=7fc6965e0e5b9e21 in_seq=0 out_seq=0 2024-04-24T06:51:23.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.975+0000 7f3078c00640 10 client.26888 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:51:23.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.976+0000 7f3078c00640 1 -- 172.21.15.5:0/1027868422 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c19c741c10 con 0x55c19c7e9e10 2024-04-24T06:51:23.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.976+0000 7f3078c00640 10 client.26888 handle_command_reply: tid=0 2024-04-24T06:51:23.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.976+0000 7f3098e22740 1 client.26888 shutdown 2024-04-24T06:51:23.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.976+0000 7f3098e22740 1 -- 172.21.15.5:0/1027868422 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c19c7cea30 msgr2=0x55c19c7d6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:23.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.976+0000 7f3098e22740 1 --2- 172.21.15.5:0/1027868422 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c19c7cea30 0x55c19c7d6990 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3080019990 tx=0x7f308000fb10 comp rx=0 tx=0).stop 2024-04-24T06:51:23.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.976+0000 7f3098e22740 1 -- 172.21.15.5:0/1027868422 shutdown_connections 2024-04-24T06:51:23.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.976+0000 7f3098e22740 1 --2- 172.21.15.5:0/1027868422 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55c19c7e9e10 0x55c19c7ec200 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.976+0000 7f3078c00640 0 client.26888 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:51:23.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.976+0000 7f3098e22740 1 --2- 172.21.15.5:0/1027868422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c19c7d9a20 0x55c19c7d7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.976+0000 7f3098e22740 1 --2- 172.21.15.5:0/1027868422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c7cf350 0x55c19c7d6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.976+0000 7f3098e22740 1 --2- 172.21.15.5:0/1027868422 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c19c7cea30 0x55c19c7d6990 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:23.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.976+0000 7f3098e22740 1 -- 172.21.15.5:0/1027868422 >> 172.21.15.5:0/1027868422 conn(0x55c19c7cad50 msgr2=0x55c19c7c9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:23.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.976+0000 7f3098e22740 1 -- 172.21.15.5:0/1027868422 shutdown_connections 2024-04-24T06:51:23.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.976+0000 7f3098e22740 1 -- 172.21.15.5:0/1027868422 wait complete. 2024-04-24T06:51:23.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:23.977+0000 7f3098e22740 20 client.26888 trim_cache size 0 max 16384 2024-04-24T06:51:24.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.002+0000 7f3098e22740 1 Processor -- start 2024-04-24T06:51:24.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.002+0000 7f3098e22740 1 -- start start 2024-04-24T06:51:24.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.002+0000 7f3098e22740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c7cea30 0x55c19c7e5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:24.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.003+0000 7f3098e22740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c19c7cf350 0x55c19c7c5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:24.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.003+0000 7f3098e22740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c19c7d9a20 0x55c19c7d1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:24.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.003+0000 7f3098e22740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f30700110a0 con 0x55c19c7cea30 2024-04-24T06:51:24.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.003+0000 7f3098e22740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f307000e970 con 0x55c19c7d9a20 2024-04-24T06:51:24.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.003+0000 7f3098e22740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f307000c7e0 con 0x55c19c7cf350 2024-04-24T06:51:24.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.003+0000 7f308ea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c19c7cf350 0x55c19c7c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:24.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.003+0000 7f308ea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c19c7cf350 0x55c19c7c5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:44930/0 (socket says 172.21.15.5:44930) 2024-04-24T06:51:24.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.003+0000 7f308ea00640 1 -- 172.21.15.5:0/813815733 learned_addr learned my addr 172.21.15.5:0/813815733 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:24.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.003+0000 7f308f400640 1 --2- 172.21.15.5:0/813815733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c7cea30 0x55c19c7e5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:24.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.003+0000 7f308fe00640 1 --2- 172.21.15.5:0/813815733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c19c7d9a20 0x55c19c7d1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:24.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.003+0000 7f308ea00640 1 -- 172.21.15.5:0/813815733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c19c7d9a20 msgr2=0x55c19c7d1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:24.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.003+0000 7f308ea00640 1 --2- 172.21.15.5:0/813815733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c19c7d9a20 0x55c19c7d1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:24.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.003+0000 7f308ea00640 1 -- 172.21.15.5:0/813815733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c7cea30 msgr2=0x55c19c7e5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:24.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.003+0000 7f308ea00640 1 --2- 172.21.15.5:0/813815733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c7cea30 0x55c19c7e5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:24.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.003+0000 7f308ea00640 1 -- 172.21.15.5:0/813815733 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c19c7d5940 con 0x55c19c7cf350 2024-04-24T06:51:24.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.003+0000 7f308fe00640 1 --2- 172.21.15.5:0/813815733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c19c7d9a20 0x55c19c7d1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:51:24.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.004+0000 7f308ea00640 1 --2- 172.21.15.5:0/813815733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c19c7cf350 0x55c19c7c5e60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f307c0ee430 tx=0x7f307c034000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be1c89f90edd5ed2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:24.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.004+0000 7f307b400640 1 -- 172.21.15.5:0/813815733 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c049040 con 0x55c19c7cf350 2024-04-24T06:51:24.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.004+0000 7f307b400640 1 -- 172.21.15.5:0/813815733 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f307c0bd320 con 0x55c19c7cf350 2024-04-24T06:51:24.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.004+0000 7f307b400640 1 -- 172.21.15.5:0/813815733 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c0c5890 con 0x55c19c7cf350 2024-04-24T06:51:24.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.005+0000 7f3098e22740 1 -- 172.21.15.5:0/813815733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c19c7cf350 msgr2=0x55c19c7c5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:24.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.005+0000 7f3098e22740 1 --2- 172.21.15.5:0/813815733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c19c7cf350 0x55c19c7c5e60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f307c0ee430 tx=0x7f307c034000 comp rx=0 tx=0).stop 2024-04-24T06:51:24.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.005+0000 7f3098e22740 1 -- 172.21.15.5:0/813815733 shutdown_connections 2024-04-24T06:51:24.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.005+0000 7f3098e22740 1 --2- 172.21.15.5:0/813815733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c19c7d9a20 0x55c19c7d1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:24.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.005+0000 7f3098e22740 1 --2- 172.21.15.5:0/813815733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c19c7cf350 0x55c19c7c5e60 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:24.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.005+0000 7f3098e22740 1 --2- 172.21.15.5:0/813815733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c7cea30 0x55c19c7e5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:24.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.005+0000 7f3098e22740 1 -- 172.21.15.5:0/813815733 >> 172.21.15.5:0/813815733 conn(0x55c19c7cad50 msgr2=0x55c19c7c9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:24.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.005+0000 7f3098e22740 1 -- 172.21.15.5:0/813815733 shutdown_connections 2024-04-24T06:51:24.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.005+0000 7f3098e22740 1 -- 172.21.15.5:0/813815733 wait complete. 2024-04-24T06:51:24.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.005+0000 7f3098e22740 1 -- start start 2024-04-24T06:51:24.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.006+0000 7f3098e22740 1 Processor -- start 2024-04-24T06:51:24.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.006+0000 7f3098e22740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c19c7cea30 0x55c19c7d3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:24.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.006+0000 7f3098e22740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c7cf350 0x55c19c7d4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:24.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.006+0000 7f3098e22740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c19c7d9a20 0x55c19c857ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:24.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.006+0000 7f3098e22740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f307000ce30 con 0x55c19c7cf350 2024-04-24T06:51:24.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.006+0000 7f3098e22740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f307000e2a0 con 0x55c19c7d9a20 2024-04-24T06:51:24.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.006+0000 7f3098e22740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f307000d500 con 0x55c19c7cea30 2024-04-24T06:51:24.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.006+0000 7f3098e22740 10 client.0 _ll_register_callbacks cb 0x7f308001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:51:24.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.006+0000 7f308f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c19c7cea30 0x55c19c7d3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:24.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.006+0000 7f308fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c19c7d9a20 0x55c19c857ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:24.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.006+0000 7f308f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c19c7cea30 0x55c19c7d3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:44940/0 (socket says 172.21.15.5:44940) 2024-04-24T06:51:24.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.006+0000 7f308f400640 1 -- 172.21.15.5:0/2926645533 learned_addr learned my addr 172.21.15.5:0/2926645533 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:24.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.006+0000 7f308fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c19c7d9a20 0x55c19c857ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:50876/0 (socket says 172.21.15.5:50876) 2024-04-24T06:51:24.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.007+0000 7f308ea00640 1 --2- 172.21.15.5:0/2926645533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c7cf350 0x55c19c7d4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:24.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.007+0000 7f308fe00640 1 -- 172.21.15.5:0/2926645533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c19c7cea30 msgr2=0x55c19c7d3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:24.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.007+0000 7f308fe00640 1 --2- 172.21.15.5:0/2926645533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c19c7cea30 0x55c19c7d3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:24.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.007+0000 7f308fe00640 1 -- 172.21.15.5:0/2926645533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c7cf350 msgr2=0x55c19c7d4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:24.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.007+0000 7f308fe00640 1 --2- 172.21.15.5:0/2926645533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c7cf350 0x55c19c7d4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:24.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.007+0000 7f308fe00640 1 -- 172.21.15.5:0/2926645533 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c19c4463f0 con 0x55c19c7d9a20 2024-04-24T06:51:24.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.007+0000 7f308f400640 1 --2- 172.21.15.5:0/2926645533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c19c7cea30 0x55c19c7d3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:51:24.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.007+0000 7f308fe00640 1 --2- 172.21.15.5:0/2926645533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c19c7d9a20 0x55c19c857ab0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3088011eb0 tx=0x7f3088004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d36e13ca71f5c166 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:24.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.007+0000 7f3078c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:51:24.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.008+0000 7f3078c00640 1 -- 172.21.15.5:0/2926645533 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308802e060 con 0x55c19c7d9a20 2024-04-24T06:51:24.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.008+0000 7f3098e22740 1 -- 172.21.15.5:0/2926645533 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f307c0bd320 con 0x55c19c7d9a20 2024-04-24T06:51:24.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.008+0000 7f3078c00640 1 -- 172.21.15.5:0/2926645533 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f308801ec70 con 0x55c19c7d9a20 2024-04-24T06:51:24.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.008+0000 7f3078c00640 1 -- 172.21.15.5:0/2926645533 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3088032420 con 0x55c19c7d9a20 2024-04-24T06:51:24.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.008+0000 7f3098e22740 10 client.26894 fetch_fsmap learned FSMap version 27 2024-04-24T06:51:24.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.008+0000 7f3098e22740 1 -- 172.21.15.5:0/2926645533 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c19c4060f0 con 0x55c19c7d9a20 2024-04-24T06:51:24.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.009+0000 7f3078c00640 1 -- 172.21.15.5:0/2926645533 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f307c0bd320 con 0x55c19c7d9a20 2024-04-24T06:51:24.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.009+0000 7f3078c00640 1 -- 172.21.15.5:0/2926645533 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f3088028070 con 0x55c19c7d9a20 2024-04-24T06:51:24.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.009+0000 7f3098e22740 10 client.26894 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:51:24.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.009+0000 7f3098e22740 10 client.26894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:51:24.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.009+0000 7f3098e22740 20 client.26894 populate_metadata read hostname 'smithi005' 2024-04-24T06:51:24.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.010+0000 7f3098e22740 1 --2- 172.21.15.5:0/2926645533 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55c19c8596b0 0x55c19c85bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:24.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.010+0000 7f3098e22740 4 client.26894 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:51:24.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.010+0000 7f3098e22740 1 -- 172.21.15.5:0/2926645533 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c19c73ba80 con 0x55c19c8596b0 2024-04-24T06:51:24.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.010+0000 7f308f400640 1 --2- 172.21.15.5:0/2926645533 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55c19c8596b0 0x55c19c85bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:24.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.010+0000 7f308f400640 1 --2- 172.21.15.5:0/2926645533 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55c19c8596b0 0x55c19c85bad0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e36f6fc1c486433 server_cookie=bc41ef73289a202f in_seq=0 out_seq=0 2024-04-24T06:51:24.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.010+0000 7f3078c00640 10 client.26894 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:51:24.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.011+0000 7f3078c00640 1 -- 172.21.15.5:0/2926645533 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55c19c73ba80 con 0x55c19c8596b0 2024-04-24T06:51:24.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.011+0000 7f3078c00640 10 client.26894 handle_command_reply: tid=0 2024-04-24T06:51:24.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.011+0000 7f3098e22740 1 client.26894 shutdown 2024-04-24T06:51:24.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.011+0000 7f3098e22740 1 -- 172.21.15.5:0/2926645533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c19c7d9a20 msgr2=0x55c19c857ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:24.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.011+0000 7f3098e22740 1 --2- 172.21.15.5:0/2926645533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c19c7d9a20 0x55c19c857ab0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3088011eb0 tx=0x7f3088004ab0 comp rx=0 tx=0).stop 2024-04-24T06:51:24.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.011+0000 7f3098e22740 1 -- 172.21.15.5:0/2926645533 shutdown_connections 2024-04-24T06:51:24.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.011+0000 7f3098e22740 1 --2- 172.21.15.5:0/2926645533 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55c19c8596b0 0x55c19c85bad0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:24.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.011+0000 7f3098e22740 1 --2- 172.21.15.5:0/2926645533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c19c7d9a20 0x55c19c857ab0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:24.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.011+0000 7f3098e22740 1 --2- 172.21.15.5:0/2926645533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19c7cf350 0x55c19c7d4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:24.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.011+0000 7f3078c00640 0 client.26894 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:51:24.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.011+0000 7f3098e22740 1 --2- 172.21.15.5:0/2926645533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c19c7cea30 0x55c19c7d3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:24.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.011+0000 7f3098e22740 1 -- 172.21.15.5:0/2926645533 >> 172.21.15.5:0/2926645533 conn(0x55c19c7cad50 msgr2=0x55c19c7d6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:24.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.011+0000 7f3098e22740 1 -- 172.21.15.5:0/2926645533 shutdown_connections 2024-04-24T06:51:24.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.011+0000 7f3098e22740 1 -- 172.21.15.5:0/2926645533 wait complete. 2024-04-24T06:51:24.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.012+0000 7f3098e22740 20 client.26894 trim_cache size 0 max 16384 2024-04-24T06:51:24.021 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:51:24.022 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1133 inodes in the stack)", 2024-04-24T06:51:24.022 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-24T06:51:24.022 INFO:teuthology.orchestra.run.smithi005.stdout: "cb698d49-14e7-4d2b-8fdb-1e4931817a5c": { 2024-04-24T06:51:24.022 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-24T06:51:24.022 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "cb698d49-14e7-4d2b-8fdb-1e4931817a5c", 2024-04-24T06:51:24.022 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-04-24T06:51:24.022 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:51:24.023 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:51:24.023 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:51:24.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.012+0000 7f3094a00640 1 -- 172.21.15.5:0/3450881261 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f30740a3da0 msgr2=0x7f30740a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:24.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.012+0000 7f3094a00640 1 --2- 172.21.15.5:0/3450881261 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f30740a3da0 0x7f30740a6260 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3088005fd0 tx=0x7f3088009500 comp rx=0 tx=0).stop 2024-04-24T06:51:24.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.012+0000 7f3094a00640 1 -- 172.21.15.5:0/3450881261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3090107140 msgr2=0x7f30901a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:24.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.012+0000 7f3094a00640 1 --2- 172.21.15.5:0/3450881261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3090107140 0x7f30901a0bb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f307c02a910 tx=0x7f307c003b20 comp rx=0 tx=0).stop 2024-04-24T06:51:24.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.012+0000 7f3094a00640 1 -- 172.21.15.5:0/3450881261 shutdown_connections 2024-04-24T06:51:24.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.013+0000 7f3094a00640 1 --2- 172.21.15.5:0/3450881261 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f30740a3da0 0x7f30740a6260 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:24.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.013+0000 7f3094a00640 1 --2- 172.21.15.5:0/3450881261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f309019a730 0x7f309019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:24.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.013+0000 7f3094a00640 1 --2- 172.21.15.5:0/3450881261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3090107140 0x7f30901a0bb0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:24.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.013+0000 7f3094a00640 1 --2- 172.21.15.5:0/3450881261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3090102070 0x7f30901a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:24.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.013+0000 7f3094a00640 1 -- 172.21.15.5:0/3450881261 >> 172.21.15.5:0/3450881261 conn(0x7f30900fd950 msgr2=0x7f30900ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:24.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.013+0000 7f3094a00640 1 -- 172.21.15.5:0/3450881261 shutdown_connections 2024-04-24T06:51:24.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:24.013+0000 7f3094a00640 1 -- 172.21.15.5:0/3450881261 wait complete. 2024-04-24T06:51:24.035 INFO:tasks.cephfs.filesystem:scrub status for tag:cb698d49-14e7-4d2b-8fdb-1e4931817a5c - {'path': '/', 'tag': 'cb698d49-14e7-4d2b-8fdb-1e4931817a5c', 'options': 'recursive,force'} 2024-04-24T06:51:24.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:24 smithi137 ceph-mon[33017]: scrub queued for path: / 2024-04-24T06:51:24.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:24 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:51:24.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:24 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:51:24.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:24 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:51:24.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:24 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:51:24.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:24 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:51:24.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:24 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:51:24.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:24 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:51:24.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:24 smithi005 ceph-mon[27798]: scrub queued for path: / 2024-04-24T06:51:24.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:24 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:51:24.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:24 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:51:24.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:24 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:51:24.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:24 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:51:24.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:24 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:51:24.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:24 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:51:24.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:24 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:51:24.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:24 smithi037 ceph-mon[32864]: scrub queued for path: / 2024-04-24T06:51:24.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:24 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:51:24.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:24 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:51:24.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:24 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:51:24.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:24 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:51:24.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:24 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:51:24.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:24 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:51:24.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:24 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:51:25.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:25 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:51:25.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:25 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:51:25.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:25 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:51:25.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:25 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:51:25.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:25 smithi137 ceph-mon[33017]: pgmap v226: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-24T06:51:25.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:25 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:51:25.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:25 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:51:25.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:25 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:51:25.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:25 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:51:25.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:25 smithi005 ceph-mon[27798]: pgmap v226: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-24T06:51:25.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:25 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:51:25.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:25 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:51:25.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:25 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:51:25.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:25 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:51:25.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:25 smithi037 ceph-mon[32864]: pgmap v226: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-24T06:51:27.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:27 smithi137 ceph-mon[33017]: pgmap v227: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-24T06:51:27.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:27 smithi005 ceph-mon[27798]: pgmap v227: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-24T06:51:27.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:27 smithi037 ceph-mon[32864]: pgmap v227: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-24T06:51:29.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:29 smithi137 ceph-mon[33017]: pgmap v228: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:51:29.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:29 smithi005 ceph-mon[27798]: pgmap v228: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:51:29.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:29 smithi037 ceph-mon[32864]: pgmap v228: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:51:31.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:31 smithi137 ceph-mon[33017]: pgmap v229: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:51:31.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:31 smithi005 ceph-mon[27798]: pgmap v229: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:51:31.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:31 smithi037 ceph-mon[32864]: pgmap v229: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:51:33.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:33 smithi137 ceph-mon[33017]: pgmap v230: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:51:33.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:33 smithi005 ceph-mon[27798]: pgmap v230: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:51:33.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:33 smithi037 ceph-mon[32864]: pgmap v230: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:51:35.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:35 smithi137 ceph-mon[33017]: pgmap v231: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:51:35.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:35 smithi005 ceph-mon[27798]: pgmap v231: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:51:35.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:35 smithi037 ceph-mon[32864]: pgmap v231: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:51:37.570 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:37 smithi037 ceph-mon[32864]: pgmap v232: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:51:37.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:37 smithi137 ceph-mon[33017]: pgmap v232: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:51:37.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:37 smithi005 ceph-mon[27798]: pgmap v232: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:51:38.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:38 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:51:38.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:38 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:51:38.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:38 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:51:38.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:38 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:51:38.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:38 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:51:38.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:38 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:51:38.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:38 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:51:38.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:38 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:51:38.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:38 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:51:39.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:39 smithi137 ceph-mon[33017]: pgmap v233: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:51:39.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:39 smithi005 ceph-mon[27798]: pgmap v233: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:51:39.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:39 smithi037 ceph-mon[32864]: pgmap v233: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:51:41.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:41 smithi137 ceph-mon[33017]: pgmap v234: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-24T06:51:41.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:41 smithi005 ceph-mon[27798]: pgmap v234: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-24T06:51:41.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:41 smithi037 ceph-mon[32864]: pgmap v234: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-24T06:51:43.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:43 smithi137 ceph-mon[33017]: pgmap v235: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:51:43.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:43 smithi005 ceph-mon[27798]: pgmap v235: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:51:43.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:43 smithi037 ceph-mon[32864]: pgmap v235: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:51:45.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:45 smithi137 ceph-mon[33017]: pgmap v236: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:51:45.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:45 smithi005 ceph-mon[27798]: pgmap v236: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:51:45.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:45 smithi037 ceph-mon[32864]: pgmap v236: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:51:47.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:47 smithi137 ceph-mon[33017]: pgmap v237: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:51:47.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:47 smithi005 ceph-mon[27798]: pgmap v237: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:51:47.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:47 smithi037 ceph-mon[32864]: pgmap v237: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:51:49.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:49 smithi137 ceph-mon[33017]: pgmap v238: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:51:49.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:49 smithi005 ceph-mon[27798]: pgmap v238: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:51:49.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:49 smithi037 ceph-mon[32864]: pgmap v238: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:51:51.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:51 smithi137 ceph-mon[33017]: pgmap v239: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:51:51.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:51 smithi005 ceph-mon[27798]: pgmap v239: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:51:51.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:51 smithi037 ceph-mon[32864]: pgmap v239: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-24T06:51:53.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:53 smithi137 ceph-mon[33017]: pgmap v240: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:51:53.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:53 smithi005 ceph-mon[27798]: pgmap v240: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:51:53.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:53 smithi037 ceph-mon[32864]: pgmap v240: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:51:54.036 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T06:51:54.037 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T06:51:54.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.261+0000 7f2d67200640 1 -- 172.21.15.5:0/936439705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d6810c400 msgr2=0x7f2d6810e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:54.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.261+0000 7f2d67200640 1 --2- 172.21.15.5:0/936439705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d6810c400 0x7f2d6810e870 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2d60009900 tx=0x7f2d6002a480 comp rx=0 tx=0).stop 2024-04-24T06:51:54.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.261+0000 7f2d67200640 1 -- 172.21.15.5:0/936439705 shutdown_connections 2024-04-24T06:51:54.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.261+0000 7f2d67200640 1 --2- 172.21.15.5:0/936439705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d6810c400 0x7f2d6810e870 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.261+0000 7f2d67200640 1 --2- 172.21.15.5:0/936439705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d68103900 0x7f2d6810bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.261+0000 7f2d67200640 1 --2- 172.21.15.5:0/936439705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d68102fe0 0x7f2d681033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.261+0000 7f2d67200640 1 -- 172.21.15.5:0/936439705 >> 172.21.15.5:0/936439705 conn(0x7f2d68077ab0 msgr2=0x7f2d680fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:54.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.261+0000 7f2d67200640 1 -- 172.21.15.5:0/936439705 shutdown_connections 2024-04-24T06:51:54.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.261+0000 7f2d67200640 1 -- 172.21.15.5:0/936439705 wait complete. 2024-04-24T06:51:54.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.262+0000 7f2d67200640 1 Processor -- start 2024-04-24T06:51:54.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.263+0000 7f2d67200640 1 -- start start 2024-04-24T06:51:54.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.263+0000 7f2d67200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d68102fe0 0x7f2d68198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:54.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.263+0000 7f2d67200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d68103900 0x7f2d68198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:54.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.263+0000 7f2d67200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d6810c400 0x7f2d6819cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:54.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.263+0000 7f2d67200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2d68112200 con 0x7f2d6810c400 2024-04-24T06:51:54.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.263+0000 7f2d67200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2d68112080 con 0x7f2d68102fe0 2024-04-24T06:51:54.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.263+0000 7f2d67200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2d68112380 con 0x7f2d68103900 2024-04-24T06:51:54.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.263+0000 7f2d65400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d68103900 0x7f2d68198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:54.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.263+0000 7f2d65400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d68103900 0x7f2d68198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:40926/0 (socket says 172.21.15.5:40926) 2024-04-24T06:51:54.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.263+0000 7f2d65400640 1 -- 172.21.15.5:0/3224524496 learned_addr learned my addr 172.21.15.5:0/3224524496 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:54.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.263+0000 7f2d65e00640 1 --2- 172.21.15.5:0/3224524496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d68102fe0 0x7f2d68198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:54.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.263+0000 7f2d66800640 1 --2- 172.21.15.5:0/3224524496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d6810c400 0x7f2d6819cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:54.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.264+0000 7f2d65e00640 1 -- 172.21.15.5:0/3224524496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d68103900 msgr2=0x7f2d68198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:54.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.264+0000 7f2d65e00640 1 --2- 172.21.15.5:0/3224524496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d68103900 0x7f2d68198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.264+0000 7f2d65e00640 1 -- 172.21.15.5:0/3224524496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d6810c400 msgr2=0x7f2d6819cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:54.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.264+0000 7f2d65e00640 1 --2- 172.21.15.5:0/3224524496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d6810c400 0x7f2d6819cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.264+0000 7f2d65e00640 1 -- 172.21.15.5:0/3224524496 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d6819d6f0 con 0x7f2d68102fe0 2024-04-24T06:51:54.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.264+0000 7f2d65e00640 1 --2- 172.21.15.5:0/3224524496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d68102fe0 0x7f2d68198700 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2d58009fd0 tx=0x7f2d5800ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:54.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.264+0000 7f2d5ea00640 1 -- 172.21.15.5:0/3224524496 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d58004b20 con 0x7f2d68102fe0 2024-04-24T06:51:54.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.265+0000 7f2d5ea00640 1 -- 172.21.15.5:0/3224524496 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d58004cc0 con 0x7f2d68102fe0 2024-04-24T06:51:54.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.265+0000 7f2d5ea00640 1 -- 172.21.15.5:0/3224524496 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d580107b0 con 0x7f2d68102fe0 2024-04-24T06:51:54.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.265+0000 7f2d67200640 1 -- 172.21.15.5:0/3224524496 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d6819d9e0 con 0x7f2d68102fe0 2024-04-24T06:51:54.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.265+0000 7f2d67200640 1 -- 172.21.15.5:0/3224524496 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d681a5270 con 0x7f2d68102fe0 2024-04-24T06:51:54.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.268+0000 7f2d6fe19740 1 Processor -- start 2024-04-24T06:51:54.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.268+0000 7f2d6fe19740 1 -- start start 2024-04-24T06:51:54.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.268+0000 7f2d5ea00640 1 -- 172.21.15.5:0/3224524496 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f2d58005020 con 0x7f2d68102fe0 2024-04-24T06:51:54.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.268+0000 7f2d6fe19740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d418f98e40 0x55d418f99220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:54.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.268+0000 7f2d6fe19740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d418f99760 0x55d418fa38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:54.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.268+0000 7f2d6fe19740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d418fa3e30 0x55d418fa6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:54.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.268+0000 7f2d6fe19740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d418efa1f0 con 0x55d418f99760 2024-04-24T06:51:54.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.268+0000 7f2d6fe19740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d418ea0350 con 0x55d418fa3e30 2024-04-24T06:51:54.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.268+0000 7f2d6fe19740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d418ef61b0 con 0x55d418f98e40 2024-04-24T06:51:54.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.269+0000 7f2d65e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d418f99760 0x55d418fa38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:54.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.269+0000 7f2d66800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d418f98e40 0x55d418f99220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:54.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.269+0000 7f2d66800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d418f98e40 0x55d418f99220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:40938/0 (socket says 172.21.15.5:40938) 2024-04-24T06:51:54.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.269+0000 7f2d66800640 1 -- 172.21.15.5:0/3150569715 learned_addr learned my addr 172.21.15.5:0/3150569715 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:54.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.269+0000 7f2d65400640 1 --2- 172.21.15.5:0/3150569715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d418fa3e30 0x55d418fa6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:54.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.269+0000 7f2d66800640 1 -- 172.21.15.5:0/3150569715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d418fa3e30 msgr2=0x55d418fa6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:54.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.269+0000 7f2d66800640 1 --2- 172.21.15.5:0/3150569715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d418fa3e30 0x55d418fa6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.269+0000 7f2d66800640 1 -- 172.21.15.5:0/3150569715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d418f99760 msgr2=0x55d418fa38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:54.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.269+0000 7f2d66800640 1 --2- 172.21.15.5:0/3150569715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d418f99760 0x55d418fa38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.269+0000 7f2d66800640 1 -- 172.21.15.5:0/3150569715 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d418ef4f50 con 0x55d418f98e40 2024-04-24T06:51:54.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.269+0000 7f2d65e00640 1 --2- 172.21.15.5:0/3150569715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d418f99760 0x55d418fa38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:51:54.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.270+0000 7f2d66800640 1 --2- 172.21.15.5:0/3150569715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d418f98e40 0x55d418f99220 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2d6002aa70 tx=0x7f2d60005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0cdc19b45900088 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:54.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.270+0000 7f2d57400640 1 -- 172.21.15.5:0/3150569715 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d60040070 con 0x55d418f98e40 2024-04-24T06:51:54.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.270+0000 7f2d57400640 1 -- 172.21.15.5:0/3150569715 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d60004300 con 0x55d418f98e40 2024-04-24T06:51:54.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.270+0000 7f2d57400640 1 -- 172.21.15.5:0/3150569715 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d60035040 con 0x55d418f98e40 2024-04-24T06:51:54.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.270+0000 7f2d5ea00640 1 --2- 172.21.15.5:0/3224524496 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f2d4c0a3da0 0x7f2d4c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:54.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.270+0000 7f2d5ea00640 1 -- 172.21.15.5:0/3224524496 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f2d580d0ef0 con 0x7f2d68102fe0 2024-04-24T06:51:54.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.271+0000 7f2d65e00640 1 --2- 172.21.15.5:0/3224524496 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f2d4c0a3da0 0x7f2d4c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:54.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.271+0000 7f2d6fe19740 1 -- 172.21.15.5:0/3150569715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d418f98e40 msgr2=0x55d418f99220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:54.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.271+0000 7f2d6fe19740 1 --2- 172.21.15.5:0/3150569715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d418f98e40 0x55d418f99220 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2d6002aa70 tx=0x7f2d60005c00 comp rx=0 tx=0).stop 2024-04-24T06:51:54.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.271+0000 7f2d6fe19740 1 -- 172.21.15.5:0/3150569715 shutdown_connections 2024-04-24T06:51:54.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.271+0000 7f2d6fe19740 1 --2- 172.21.15.5:0/3150569715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d418fa3e30 0x55d418fa6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.271+0000 7f2d6fe19740 1 --2- 172.21.15.5:0/3150569715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d418f99760 0x55d418fa38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.271+0000 7f2d6fe19740 1 --2- 172.21.15.5:0/3150569715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d418f98e40 0x55d418f99220 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.271+0000 7f2d6fe19740 1 -- 172.21.15.5:0/3150569715 >> 172.21.15.5:0/3150569715 conn(0x55d418f95160 msgr2=0x55d418f95540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:54.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.271+0000 7f2d6fe19740 1 -- 172.21.15.5:0/3150569715 shutdown_connections 2024-04-24T06:51:54.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.271+0000 7f2d6fe19740 1 -- 172.21.15.5:0/3150569715 wait complete. 2024-04-24T06:51:54.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.271+0000 7f2d65e00640 1 --2- 172.21.15.5:0/3224524496 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f2d4c0a3da0 0x7f2d4c0a6260 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2d580ce6e0 tx=0x7f2d580bd040 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:54.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.272+0000 7f2d6fe19740 1 -- start start 2024-04-24T06:51:54.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.272+0000 7f2d6fe19740 1 Processor -- start 2024-04-24T06:51:54.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.272+0000 7f2d6fe19740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d418f98e40 0x55d418fa0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:54.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.272+0000 7f2d6fe19740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d418f99760 0x55d418fa1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:54.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.272+0000 7f2d6fe19740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d418fa3e30 0x55d418fa1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:54.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.272+0000 7f2d6fe19740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d418fa6bc0 con 0x55d418f98e40 2024-04-24T06:51:54.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.272+0000 7f2d6fe19740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d418eb3690 con 0x55d418fa3e30 2024-04-24T06:51:54.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.272+0000 7f2d6fe19740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d418fa6d40 con 0x55d418f99760 2024-04-24T06:51:54.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.272+0000 7f2d6fe19740 10 client.0 _ll_register_callbacks cb 0x55d418eef650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:51:54.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.273+0000 7f2d66800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d418f98e40 0x55d418fa0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:54.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.273+0000 7f2d66800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d418f98e40 0x55d418fa0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54300/0 (socket says 172.21.15.5:54300) 2024-04-24T06:51:54.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.273+0000 7f2d66800640 1 -- 172.21.15.5:0/2194759433 learned_addr learned my addr 172.21.15.5:0/2194759433 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:54.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.273+0000 7f2d65400640 1 --2- 172.21.15.5:0/2194759433 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d418f99760 0x55d418fa1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:54.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.273+0000 7f2d66800640 1 --2- 172.21.15.5:0/2194759433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d418fa3e30 0x55d418fa1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:54.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.273+0000 7f2d65400640 1 -- 172.21.15.5:0/2194759433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d418fa3e30 msgr2=0x55d418fa1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:54.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.273+0000 7f2d65400640 1 --2- 172.21.15.5:0/2194759433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d418fa3e30 0x55d418fa1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.273+0000 7f2d65400640 1 -- 172.21.15.5:0/2194759433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d418f98e40 msgr2=0x55d418fa0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:54.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.273+0000 7f2d65400640 1 --2- 172.21.15.5:0/2194759433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d418f98e40 0x55d418fa0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.273+0000 7f2d65400640 1 -- 172.21.15.5:0/2194759433 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d418eb2920 con 0x55d418f99760 2024-04-24T06:51:54.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.273+0000 7f2d66800640 1 --2- 172.21.15.5:0/2194759433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d418fa3e30 0x55d418fa1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:51:54.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.273+0000 7f2d65400640 1 --2- 172.21.15.5:0/2194759433 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d418f99760 0x55d418fa1330 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2d680ff7c0 tx=0x7f2d50012bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a59a7fcfee26968 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:54.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.273+0000 7f2d4be00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:51:54.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.274+0000 7f2d4be00640 1 -- 172.21.15.5:0/2194759433 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5000f8c0 con 0x55d418f99760 2024-04-24T06:51:54.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.274+0000 7f2d6fe19740 1 -- 172.21.15.5:0/2194759433 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d60004300 con 0x55d418f99760 2024-04-24T06:51:54.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.274+0000 7f2d4be00640 1 -- 172.21.15.5:0/2194759433 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f2d5000fa60 con 0x55d418f99760 2024-04-24T06:51:54.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.274+0000 7f2d4be00640 1 -- 172.21.15.5:0/2194759433 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2d5000fc60 con 0x55d418f99760 2024-04-24T06:51:54.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.274+0000 7f2d6fe19740 10 client.26917 fetch_fsmap learned FSMap version 27 2024-04-24T06:51:54.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.274+0000 7f2d6fe19740 1 -- 172.21.15.5:0/2194759433 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d418cd67d0 con 0x55d418f99760 2024-04-24T06:51:54.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.274+0000 7f2d4be00640 1 -- 172.21.15.5:0/2194759433 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d60004300 con 0x55d418f99760 2024-04-24T06:51:54.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.275+0000 7f2d4be00640 1 -- 172.21.15.5:0/2194759433 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f2d50004a70 con 0x55d418f99760 2024-04-24T06:51:54.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.275+0000 7f2d6fe19740 10 client.26917 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:51:54.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.275+0000 7f2d6fe19740 10 client.26917 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:51:54.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.275+0000 7f2d6fe19740 20 client.26917 populate_metadata read hostname 'smithi005' 2024-04-24T06:51:54.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.275+0000 7f2d6fe19740 1 --2- 172.21.15.5:0/2194759433 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55d418fb41b0 0x55d418fb65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:54.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.275+0000 7f2d6fe19740 4 client.26917 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:51:54.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.275+0000 7f2d6fe19740 1 -- 172.21.15.5:0/2194759433 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d418f07c10 con 0x55d418fb41b0 2024-04-24T06:51:54.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.275+0000 7f2d66800640 1 --2- 172.21.15.5:0/2194759433 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55d418fb41b0 0x55d418fb65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:54.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.276+0000 7f2d66800640 1 --2- 172.21.15.5:0/2194759433 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55d418fb41b0 0x55d418fb65c0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e09dfe9b46f8a257 server_cookie=ed745e5b2fef9d02 in_seq=0 out_seq=0 2024-04-24T06:51:54.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.276+0000 7f2d4be00640 10 client.26917 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:51:54.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.276+0000 7f2d4be00640 1 -- 172.21.15.5:0/2194759433 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d418f07c10 con 0x55d418fb41b0 2024-04-24T06:51:54.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.276+0000 7f2d4be00640 10 client.26917 handle_command_reply: tid=0 2024-04-24T06:51:54.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.276+0000 7f2d6fe19740 1 client.26917 shutdown 2024-04-24T06:51:54.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.277+0000 7f2d6fe19740 1 -- 172.21.15.5:0/2194759433 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d418f99760 msgr2=0x55d418fa1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:54.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.277+0000 7f2d6fe19740 1 --2- 172.21.15.5:0/2194759433 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d418f99760 0x55d418fa1330 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2d680ff7c0 tx=0x7f2d50012bd0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.277+0000 7f2d6fe19740 1 -- 172.21.15.5:0/2194759433 shutdown_connections 2024-04-24T06:51:54.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.277+0000 7f2d6fe19740 1 --2- 172.21.15.5:0/2194759433 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55d418fb41b0 0x55d418fb65c0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.277+0000 7f2d6fe19740 1 --2- 172.21.15.5:0/2194759433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d418fa3e30 0x55d418fa1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.277+0000 7f2d6fe19740 1 --2- 172.21.15.5:0/2194759433 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d418f99760 0x55d418fa1330 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.277+0000 7f2d4be00640 0 client.26917 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:51:54.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.277+0000 7f2d6fe19740 1 --2- 172.21.15.5:0/2194759433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d418f98e40 0x55d418fa0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.277+0000 7f2d6fe19740 1 -- 172.21.15.5:0/2194759433 >> 172.21.15.5:0/2194759433 conn(0x55d418f95160 msgr2=0x55d418f8ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:54.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.277+0000 7f2d6fe19740 1 -- 172.21.15.5:0/2194759433 shutdown_connections 2024-04-24T06:51:54.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.277+0000 7f2d6fe19740 1 -- 172.21.15.5:0/2194759433 wait complete. 2024-04-24T06:51:54.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.277+0000 7f2d6fe19740 20 client.26917 trim_cache size 0 max 16384 2024-04-24T06:51:54.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.297+0000 7f2d6fe19740 1 Processor -- start 2024-04-24T06:51:54.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.297+0000 7f2d6fe19740 1 -- start start 2024-04-24T06:51:54.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.297+0000 7f2d6fe19740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d418f98e40 0x55d418f8fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:54.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.297+0000 7f2d6fe19740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d418f99760 0x55d418f9f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:54.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.297+0000 7f2d6fe19740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d418fa3e30 0x55d418fafab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:54.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.297+0000 7f2d6fe19740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2d681ae290 con 0x55d418f98e40 2024-04-24T06:51:54.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.297+0000 7f2d6fe19740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2d681abb60 con 0x55d418fa3e30 2024-04-24T06:51:54.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.297+0000 7f2d6fe19740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2d681a99d0 con 0x55d418f99760 2024-04-24T06:51:54.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.297+0000 7f2d65400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d418f99760 0x55d418f9f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:54.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.297+0000 7f2d65400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d418f99760 0x55d418f9f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:40946/0 (socket says 172.21.15.5:40946) 2024-04-24T06:51:54.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.297+0000 7f2d65400640 1 -- 172.21.15.5:0/2578902159 learned_addr learned my addr 172.21.15.5:0/2578902159 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:54.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.298+0000 7f2d66800640 1 --2- 172.21.15.5:0/2578902159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d418f98e40 0x55d418f8fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:54.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.298+0000 7f2d65400640 1 -- 172.21.15.5:0/2578902159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d418fa3e30 msgr2=0x55d418fafab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:54.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.298+0000 7f2d66800640 1 --2- 172.21.15.5:0/2578902159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d418fa3e30 0x55d418fafab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:54.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.298+0000 7f2d65400640 1 --2- 172.21.15.5:0/2578902159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d418fa3e30 0x55d418fafab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.298+0000 7f2d65400640 1 -- 172.21.15.5:0/2578902159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d418f98e40 msgr2=0x55d418f8fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:54.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.298+0000 7f2d65400640 1 --2- 172.21.15.5:0/2578902159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d418f98e40 0x55d418f8fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.298+0000 7f2d65400640 1 -- 172.21.15.5:0/2578902159 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d418fa6590 con 0x55d418f99760 2024-04-24T06:51:54.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.298+0000 7f2d66800640 1 --2- 172.21.15.5:0/2578902159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d418f98e40 0x55d418f8fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:51:54.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.298+0000 7f2d65400640 1 --2- 172.21.15.5:0/2578902159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d418f99760 0x55d418f9f6e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2d500059f0 tx=0x7f2d5000fe80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16dcd8991ead4054 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:54.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.299+0000 7f2d56a00640 1 -- 172.21.15.5:0/2578902159 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5001c610 con 0x55d418f99760 2024-04-24T06:51:54.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.299+0000 7f2d56a00640 1 -- 172.21.15.5:0/2578902159 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d50029020 con 0x55d418f99760 2024-04-24T06:51:54.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.299+0000 7f2d56a00640 1 -- 172.21.15.5:0/2578902159 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d50005120 con 0x55d418f99760 2024-04-24T06:51:54.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.299+0000 7f2d6fe19740 1 -- 172.21.15.5:0/2578902159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d418f99760 msgr2=0x55d418f9f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:54.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.299+0000 7f2d6fe19740 1 --2- 172.21.15.5:0/2578902159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d418f99760 0x55d418f9f6e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2d500059f0 tx=0x7f2d5000fe80 comp rx=0 tx=0).stop 2024-04-24T06:51:54.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.299+0000 7f2d6fe19740 1 -- 172.21.15.5:0/2578902159 shutdown_connections 2024-04-24T06:51:54.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.299+0000 7f2d6fe19740 1 --2- 172.21.15.5:0/2578902159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d418fa3e30 0x55d418fafab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.299+0000 7f2d6fe19740 1 --2- 172.21.15.5:0/2578902159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d418f99760 0x55d418f9f6e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.299+0000 7f2d6fe19740 1 --2- 172.21.15.5:0/2578902159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d418f98e40 0x55d418f8fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.299+0000 7f2d6fe19740 1 -- 172.21.15.5:0/2578902159 >> 172.21.15.5:0/2578902159 conn(0x55d418f95160 msgr2=0x55d418fa1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:54.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.299+0000 7f2d6fe19740 1 -- 172.21.15.5:0/2578902159 shutdown_connections 2024-04-24T06:51:54.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.299+0000 7f2d6fe19740 1 -- 172.21.15.5:0/2578902159 wait complete. 2024-04-24T06:51:54.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.300+0000 7f2d6fe19740 1 -- start start 2024-04-24T06:51:54.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.300+0000 7f2d6fe19740 1 Processor -- start 2024-04-24T06:51:54.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.300+0000 7f2d6fe19740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d418f98e40 0x55d418fac350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:54.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.300+0000 7f2d6fe19740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d418f99760 0x55d418f93b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:54.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.300+0000 7f2d6fe19740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d418fa3e30 0x55d418f940a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:54.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.300+0000 7f2d6fe19740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2d681aa020 con 0x55d418fa3e30 2024-04-24T06:51:54.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.300+0000 7f2d6fe19740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2d681ab490 con 0x55d418f98e40 2024-04-24T06:51:54.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.300+0000 7f2d6fe19740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2d681aa6f0 con 0x55d418f99760 2024-04-24T06:51:54.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.301+0000 7f2d6fe19740 10 client.0 _ll_register_callbacks cb 0x7f2d50009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:51:54.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.301+0000 7f2d66800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d418f98e40 0x55d418fac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:54.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.301+0000 7f2d65400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d418f99760 0x55d418f93b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:54.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.301+0000 7f2d66800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d418f98e40 0x55d418fac350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:44412/0 (socket says 172.21.15.5:44412) 2024-04-24T06:51:54.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.301+0000 7f2d66800640 1 -- 172.21.15.5:0/3457674919 learned_addr learned my addr 172.21.15.5:0/3457674919 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:54.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.301+0000 7f2d66800640 1 --2- 172.21.15.5:0/3457674919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d418fa3e30 0x55d418f940a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:54.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.301+0000 7f2d65400640 1 -- 172.21.15.5:0/3457674919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d418f98e40 msgr2=0x55d418fac350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:54.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.301+0000 7f2d65400640 1 --2- 172.21.15.5:0/3457674919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d418f98e40 0x55d418fac350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.301+0000 7f2d65400640 1 -- 172.21.15.5:0/3457674919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d418fa3e30 msgr2=0x55d418f940a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:54.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.301+0000 7f2d65400640 1 --2- 172.21.15.5:0/3457674919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d418fa3e30 0x55d418f940a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.301+0000 7f2d65400640 1 -- 172.21.15.5:0/3457674919 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d418c0c3f0 con 0x55d418f99760 2024-04-24T06:51:54.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.302+0000 7f2d65400640 1 --2- 172.21.15.5:0/3457674919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d418f99760 0x55d418f93b60 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55d418fb11a0 tx=0x7f2d500048a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94911833b155fef8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:54.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.302+0000 7f2d4be00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:51:54.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.302+0000 7f2d4be00640 1 -- 172.21.15.5:0/3457674919 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d50028040 con 0x55d418f99760 2024-04-24T06:51:54.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.302+0000 7f2d6fe19740 1 -- 172.21.15.5:0/3457674919 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d50029020 con 0x55d418f99760 2024-04-24T06:51:54.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.302+0000 7f2d4be00640 1 -- 172.21.15.5:0/3457674919 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f2d50018070 con 0x55d418f99760 2024-04-24T06:51:54.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.302+0000 7f2d4be00640 1 -- 172.21.15.5:0/3457674919 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2d50004f90 con 0x55d418f99760 2024-04-24T06:51:54.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.302+0000 7f2d6fe19740 10 client.26923 fetch_fsmap learned FSMap version 27 2024-04-24T06:51:54.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.302+0000 7f2d6fe19740 1 -- 172.21.15.5:0/3457674919 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d418bcc0f0 con 0x55d418f99760 2024-04-24T06:51:54.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.303+0000 7f2d4be00640 1 -- 172.21.15.5:0/3457674919 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d50029020 con 0x55d418f99760 2024-04-24T06:51:54.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.303+0000 7f2d4be00640 1 -- 172.21.15.5:0/3457674919 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f2d5001cb50 con 0x55d418f99760 2024-04-24T06:51:54.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.303+0000 7f2d6fe19740 10 client.26923 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:51:54.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.303+0000 7f2d6fe19740 10 client.26923 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:51:54.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.303+0000 7f2d6fe19740 20 client.26923 populate_metadata read hostname 'smithi005' 2024-04-24T06:51:54.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.303+0000 7f2d6fe19740 1 --2- 172.21.15.5:0/3457674919 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55d418fa91f0 0x55d418fa9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:54.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.303+0000 7f2d6fe19740 4 client.26923 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:51:54.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.303+0000 7f2d6fe19740 1 -- 172.21.15.5:0/3457674919 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d418f01a80 con 0x55d418fa91f0 2024-04-24T06:51:54.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.303+0000 7f2d66800640 1 --2- 172.21.15.5:0/3457674919 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55d418fa91f0 0x55d418fa9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:54.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.304+0000 7f2d66800640 1 --2- 172.21.15.5:0/3457674919 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55d418fa91f0 0x55d418fa9600 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d6d9db11962d03e server_cookie=f589759750ec524a in_seq=0 out_seq=0 2024-04-24T06:51:54.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.304+0000 7f2d4be00640 10 client.26923 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:51:54.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.304+0000 7f2d4be00640 1 -- 172.21.15.5:0/3457674919 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d418f01a80 con 0x55d418fa91f0 2024-04-24T06:51:54.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.304+0000 7f2d4be00640 10 client.26923 handle_command_reply: tid=0 2024-04-24T06:51:54.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.304+0000 7f2d6fe19740 1 client.26923 shutdown 2024-04-24T06:51:54.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.304+0000 7f2d6fe19740 1 -- 172.21.15.5:0/3457674919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d418f99760 msgr2=0x55d418f93b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:54.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.304+0000 7f2d6fe19740 1 --2- 172.21.15.5:0/3457674919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d418f99760 0x55d418f93b60 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55d418fb11a0 tx=0x7f2d500048a0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.304+0000 7f2d6fe19740 1 -- 172.21.15.5:0/3457674919 shutdown_connections 2024-04-24T06:51:54.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.304+0000 7f2d6fe19740 1 --2- 172.21.15.5:0/3457674919 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55d418fa91f0 0x55d418fa9600 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.304+0000 7f2d6fe19740 1 --2- 172.21.15.5:0/3457674919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d418fa3e30 0x55d418f940a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.304+0000 7f2d6fe19740 1 --2- 172.21.15.5:0/3457674919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d418f99760 0x55d418f93b60 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.304+0000 7f2d6fe19740 1 --2- 172.21.15.5:0/3457674919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d418f98e40 0x55d418fac350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.304+0000 7f2d4be00640 0 client.26923 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:51:54.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.304+0000 7f2d6fe19740 1 -- 172.21.15.5:0/3457674919 >> 172.21.15.5:0/3457674919 conn(0x55d418f95160 msgr2=0x55d418f9a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:54.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.304+0000 7f2d6fe19740 1 -- 172.21.15.5:0/3457674919 shutdown_connections 2024-04-24T06:51:54.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.304+0000 7f2d6fe19740 1 -- 172.21.15.5:0/3457674919 wait complete. 2024-04-24T06:51:54.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.305+0000 7f2d6fe19740 20 client.26923 trim_cache size 0 max 16384 2024-04-24T06:51:54.314 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:51:54.314 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-24T06:51:54.314 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-24T06:51:54.314 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:51:54.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.305+0000 7f2d67200640 1 -- 172.21.15.5:0/3224524496 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f2d4c0a3da0 msgr2=0x7f2d4c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:54.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.305+0000 7f2d67200640 1 --2- 172.21.15.5:0/3224524496 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f2d4c0a3da0 0x7f2d4c0a6260 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2d580ce6e0 tx=0x7f2d580bd040 comp rx=0 tx=0).stop 2024-04-24T06:51:54.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.305+0000 7f2d67200640 1 -- 172.21.15.5:0/3224524496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d68102fe0 msgr2=0x7f2d68198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:54.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.305+0000 7f2d67200640 1 --2- 172.21.15.5:0/3224524496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d68102fe0 0x7f2d68198700 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2d58009fd0 tx=0x7f2d5800ef90 comp rx=0 tx=0).stop 2024-04-24T06:51:54.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.305+0000 7f2d67200640 1 -- 172.21.15.5:0/3224524496 shutdown_connections 2024-04-24T06:51:54.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.305+0000 7f2d67200640 1 --2- 172.21.15.5:0/3224524496 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f2d4c0a3da0 0x7f2d4c0a6260 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.305+0000 7f2d67200640 1 --2- 172.21.15.5:0/3224524496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d6810c400 0x7f2d6819cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.305+0000 7f2d67200640 1 --2- 172.21.15.5:0/3224524496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d68103900 0x7f2d68198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.305+0000 7f2d67200640 1 --2- 172.21.15.5:0/3224524496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d68102fe0 0x7f2d68198700 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.305+0000 7f2d67200640 1 -- 172.21.15.5:0/3224524496 >> 172.21.15.5:0/3224524496 conn(0x7f2d68077ab0 msgr2=0x7f2d680fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:54.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.305+0000 7f2d67200640 1 -- 172.21.15.5:0/3224524496 shutdown_connections 2024-04-24T06:51:54.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.306+0000 7f2d67200640 1 -- 172.21.15.5:0/3224524496 wait complete. 2024-04-24T06:51:54.324 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T06:51:54.325 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:51:54.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.522+0000 7f4108e00640 1 -- 172.21.15.5:0/2435485897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41041087b0 msgr2=0x7f4104108b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:54.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.522+0000 7f4108e00640 1 --2- 172.21.15.5:0/2435485897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41041087b0 0x7f4104108b90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f40f4009930 tx=0x7f40f402a420 comp rx=0 tx=0).stop 2024-04-24T06:51:54.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.522+0000 7f4108e00640 1 -- 172.21.15.5:0/2435485897 shutdown_connections 2024-04-24T06:51:54.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.522+0000 7f4108e00640 1 --2- 172.21.15.5:0/2435485897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f410410d8a0 0x7f4104115420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.523+0000 7f4108e00640 1 --2- 172.21.15.5:0/2435485897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4104109160 0x7f410410d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.523+0000 7f4108e00640 1 --2- 172.21.15.5:0/2435485897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41041087b0 0x7f4104108b90 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.523+0000 7f4108e00640 1 -- 172.21.15.5:0/2435485897 >> 172.21.15.5:0/2435485897 conn(0x7f4104077dd0 msgr2=0x7f4104106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:54.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.523+0000 7f4108e00640 1 -- 172.21.15.5:0/2435485897 shutdown_connections 2024-04-24T06:51:54.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.523+0000 7f4108e00640 1 -- 172.21.15.5:0/2435485897 wait complete. 2024-04-24T06:51:54.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.524+0000 7f4108e00640 1 Processor -- start 2024-04-24T06:51:54.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.524+0000 7f4108e00640 1 -- start start 2024-04-24T06:51:54.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.524+0000 7f4108e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41041087b0 0x7f410419edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:54.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.524+0000 7f4108e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f410410d8a0 0x7f410419f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:54.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.524+0000 7f4108e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41041a3690 0x7f41041a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:54.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.524+0000 7f4108e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f41041188b0 con 0x7f41041a3690 2024-04-24T06:51:54.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.524+0000 7f4108e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4104118730 con 0x7f41041087b0 2024-04-24T06:51:54.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.524+0000 7f4108e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4104118a30 con 0x7f410410d8a0 2024-04-24T06:51:54.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.524+0000 7f4103e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41041a3690 0x7f41041a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:54.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.524+0000 7f4102a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f410410d8a0 0x7f410419f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:54.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.524+0000 7f4103e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41041a3690 0x7f41041a3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54324/0 (socket says 172.21.15.5:54324) 2024-04-24T06:51:54.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.524+0000 7f4103e00640 1 -- 172.21.15.5:0/516575692 learned_addr learned my addr 172.21.15.5:0/516575692 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:54.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.525+0000 7f4103400640 1 --2- 172.21.15.5:0/516575692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41041087b0 0x7f410419edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:54.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.525+0000 7f4102a00640 1 -- 172.21.15.5:0/516575692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41041087b0 msgr2=0x7f410419edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:54.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.525+0000 7f4102a00640 1 --2- 172.21.15.5:0/516575692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41041087b0 0x7f410419edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.525+0000 7f4102a00640 1 -- 172.21.15.5:0/516575692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41041a3690 msgr2=0x7f41041a3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:54.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.525+0000 7f4102a00640 1 --2- 172.21.15.5:0/516575692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41041a3690 0x7f41041a3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.525+0000 7f4102a00640 1 -- 172.21.15.5:0/516575692 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41041a4180 con 0x7f410410d8a0 2024-04-24T06:51:54.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.525+0000 7f4103e00640 1 --2- 172.21.15.5:0/516575692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41041a3690 0x7f41041a3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:51:54.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.525+0000 7f4102a00640 1 --2- 172.21.15.5:0/516575692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f410410d8a0 0x7f410419f310 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f40f000e9c0 tx=0x7f40f000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:54.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.525+0000 7f40fbe00640 1 -- 172.21.15.5:0/516575692 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f000cd80 con 0x7f410410d8a0 2024-04-24T06:51:54.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.525+0000 7f40fbe00640 1 -- 172.21.15.5:0/516575692 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40f00035f0 con 0x7f410410d8a0 2024-04-24T06:51:54.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.525+0000 7f40fbe00640 1 -- 172.21.15.5:0/516575692 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f0010690 con 0x7f410410d8a0 2024-04-24T06:51:54.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.525+0000 7f4108e00640 1 -- 172.21.15.5:0/516575692 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41041a4470 con 0x7f410410d8a0 2024-04-24T06:51:54.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.526+0000 7f4108e00640 1 -- 172.21.15.5:0/516575692 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41041abc60 con 0x7f410410d8a0 2024-04-24T06:51:54.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.527+0000 7f4108e00640 1 -- 172.21.15.5:0/516575692 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f410404fe00 con 0x7f410410d8a0 2024-04-24T06:51:54.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.530+0000 7f40fbe00640 1 -- 172.21.15.5:0/516575692 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f40f00037a0 con 0x7f410410d8a0 2024-04-24T06:51:54.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.531+0000 7f40fbe00640 1 --2- 172.21.15.5:0/516575692 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f40e80a3d10 0x7f40e80a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:54.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.531+0000 7f40fbe00640 1 -- 172.21.15.5:0/516575692 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f40f00d0c70 con 0x7f410410d8a0 2024-04-24T06:51:54.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.531+0000 7f4103400640 1 --2- 172.21.15.5:0/516575692 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f40e80a3d10 0x7f40e80a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:54.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.532+0000 7f4103400640 1 --2- 172.21.15.5:0/516575692 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f40e80a3d10 0x7f40e80a61d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f40f4009900 tx=0x7f40f40058d0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:54.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.533+0000 7f40fbe00640 1 -- 172.21.15.5:0/516575692 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f40f009d2a0 con 0x7f410410d8a0 2024-04-24T06:51:54.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.785+0000 7f4108e00640 1 -- 172.21.15.5:0/516575692 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41041a0120 con 0x7f410410d8a0 2024-04-24T06:51:54.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.786+0000 7f40fbe00640 1 -- 172.21.15.5:0/516575692 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11856 (secure 0 0 0) 0x7f41041a0120 con 0x7f410410d8a0 2024-04-24T06:51:54.788 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:51:54.789 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1735462750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1735462750},{"type":"v1","addr":"172.21.15.5:6837","nonce":1735462750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1499469370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1499469370},{"type":"v1","addr":"172.21.15.5:6841","nonce":1499469370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3705638847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3705638847},{"type":"v1","addr":"172.21.15.37:6835","nonce":3705638847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T06:41:34.764490+0000","modified":"2024-04-24T06:48:57.768410+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24550,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6837/1390103296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1390103296},{"type":"v1","addr":"172.21.15.37:6837","nonce":1390103296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3217754215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3217754215},{"type":"v1","addr":"172.21.15.137:6835","nonce":3217754215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6837/3044222976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3044222976},{"type":"v1","addr":"172.21.15.137:6837","nonce":3044222976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.5:6839/3173201459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3173201459},{"type":"v1","addr":"172.21.15.5:6839","nonce":3173201459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6839/1889774232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1889774232},{"type":"v1","addr":"172.21.15.137:6839","nonce":1889774232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6839/3536721753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":3536721753},{"type":"v1","addr":"172.21.15.37:6839","nonce":3536721753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24550,24569,24539]},"id":1}]} 2024-04-24T06:51:54.789 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-04-24T06:51:54.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.789+0000 7f4108e00640 1 -- 172.21.15.5:0/516575692 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f40e80a3d10 msgr2=0x7f40e80a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:54.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.789+0000 7f4108e00640 1 --2- 172.21.15.5:0/516575692 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f40e80a3d10 0x7f40e80a61d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f40f4009900 tx=0x7f40f40058d0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.789+0000 7f4108e00640 1 -- 172.21.15.5:0/516575692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f410410d8a0 msgr2=0x7f410419f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:54.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.789+0000 7f4108e00640 1 --2- 172.21.15.5:0/516575692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f410410d8a0 0x7f410419f310 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f40f000e9c0 tx=0x7f40f000ee80 comp rx=0 tx=0).stop 2024-04-24T06:51:54.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.789+0000 7f4108e00640 1 -- 172.21.15.5:0/516575692 shutdown_connections 2024-04-24T06:51:54.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.789+0000 7f4108e00640 1 --2- 172.21.15.5:0/516575692 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f40e80a3d10 0x7f40e80a61d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.789+0000 7f4108e00640 1 --2- 172.21.15.5:0/516575692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41041a3690 0x7f41041a3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.789+0000 7f4108e00640 1 --2- 172.21.15.5:0/516575692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f410410d8a0 0x7f410419f310 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.789+0000 7f4108e00640 1 --2- 172.21.15.5:0/516575692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41041087b0 0x7f410419edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:54.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.789+0000 7f4108e00640 1 -- 172.21.15.5:0/516575692 >> 172.21.15.5:0/516575692 conn(0x7f4104077dd0 msgr2=0x7f4104079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:54.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.790+0000 7f4108e00640 1 -- 172.21.15.5:0/516575692 shutdown_connections 2024-04-24T06:51:54.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:54.790+0000 7f4108e00640 1 -- 172.21.15.5:0/516575692 wait complete. 2024-04-24T06:51:54.813 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T06:51:55.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.034+0000 7f8488c00640 1 -- 172.21.15.5:0/1218951261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8484102be0 msgr2=0x7f8484106c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:55.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.034+0000 7f8488c00640 1 --2- 172.21.15.5:0/1218951261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8484102be0 0x7f8484106c20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f846c009900 tx=0x7f846c02a400 comp rx=0 tx=0).stop 2024-04-24T06:51:55.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.034+0000 7f8488c00640 1 -- 172.21.15.5:0/1218951261 shutdown_connections 2024-04-24T06:51:55.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.034+0000 7f8488c00640 1 --2- 172.21.15.5:0/1218951261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8484107300 0x7f848410eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.034+0000 7f8488c00640 1 --2- 172.21.15.5:0/1218951261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8484102be0 0x7f8484106c20 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.034+0000 7f8488c00640 1 --2- 172.21.15.5:0/1218951261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8484102230 0x7f8484102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.034+0000 7f8488c00640 1 -- 172.21.15.5:0/1218951261 >> 172.21.15.5:0/1218951261 conn(0x7f84840fdab0 msgr2=0x7f84840ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:55.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.034+0000 7f8488c00640 1 -- 172.21.15.5:0/1218951261 shutdown_connections 2024-04-24T06:51:55.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.035+0000 7f8488c00640 1 -- 172.21.15.5:0/1218951261 wait complete. 2024-04-24T06:51:55.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.035+0000 7f8488c00640 1 Processor -- start 2024-04-24T06:51:55.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.036+0000 7f8488c00640 1 -- start start 2024-04-24T06:51:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.036+0000 7f8488c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8484102230 0x7f84841a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.036+0000 7f8488c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8484107300 0x7f84841a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.036+0000 7f8488c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f848419a920 0x7f848419add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.036+0000 7f8488c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f84841123c0 con 0x7f8484102230 2024-04-24T06:51:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.036+0000 7f8488c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8484112240 con 0x7f8484107300 2024-04-24T06:51:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.036+0000 7f8488c00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8484112540 con 0x7f848419a920 2024-04-24T06:51:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.036+0000 7f8481e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8484102230 0x7f84841a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.036+0000 7f8479400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8484107300 0x7f84841a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.036+0000 7f8479400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8484107300 0x7f84841a0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:44438/0 (socket says 172.21.15.5:44438) 2024-04-24T06:51:55.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.036+0000 7f8479400640 1 -- 172.21.15.5:0/3333628468 learned_addr learned my addr 172.21.15.5:0/3333628468 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:55.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.037+0000 7f8482800640 1 --2- 172.21.15.5:0/3333628468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f848419a920 0x7f848419add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.037+0000 7f8479400640 1 -- 172.21.15.5:0/3333628468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f848419a920 msgr2=0x7f848419add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:55.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.037+0000 7f8479400640 1 --2- 172.21.15.5:0/3333628468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f848419a920 0x7f848419add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.037+0000 7f8479400640 1 -- 172.21.15.5:0/3333628468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8484102230 msgr2=0x7f84841a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:55.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.037+0000 7f8479400640 1 --2- 172.21.15.5:0/3333628468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8484102230 0x7f84841a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.037+0000 7f8479400640 1 -- 172.21.15.5:0/3333628468 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f848419b440 con 0x7f8484107300 2024-04-24T06:51:55.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.037+0000 7f8481e00640 1 --2- 172.21.15.5:0/3333628468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8484102230 0x7f84841a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:51:55.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.037+0000 7f8479400640 1 --2- 172.21.15.5:0/3333628468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8484107300 0x7f84841a0cb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f846c02a920 tx=0x7f846c003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:55.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.038+0000 7f847b400640 1 -- 172.21.15.5:0/3333628468 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846c03f070 con 0x7f8484107300 2024-04-24T06:51:55.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.038+0000 7f8488c00640 1 -- 172.21.15.5:0/3333628468 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f848419b6d0 con 0x7f8484107300 2024-04-24T06:51:55.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.038+0000 7f847b400640 1 -- 172.21.15.5:0/3333628468 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f846c031c90 con 0x7f8484107300 2024-04-24T06:51:55.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.038+0000 7f847b400640 1 -- 172.21.15.5:0/3333628468 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846c035030 con 0x7f8484107300 2024-04-24T06:51:55.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.038+0000 7f8488c00640 1 -- 172.21.15.5:0/3333628468 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84841a78c0 con 0x7f8484107300 2024-04-24T06:51:55.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.041+0000 7f848bba1740 1 Processor -- start 2024-04-24T06:51:55.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.041+0000 7f847b400640 1 -- 172.21.15.5:0/3333628468 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f846c042070 con 0x7f8484107300 2024-04-24T06:51:55.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.041+0000 7f848bba1740 1 -- start start 2024-04-24T06:51:55.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.042+0000 7f848bba1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd96fea950 0x55dd96fead30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.042+0000 7f848bba1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd96feb270 0x55dd96ff5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.042+0000 7f848bba1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd96ff5940 0x55dd96ff7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.042+0000 7f848bba1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dd96f501d0 con 0x55dd96feb270 2024-04-24T06:51:55.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.042+0000 7f848bba1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55dd96ef6450 con 0x55dd96ff5940 2024-04-24T06:51:55.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.042+0000 7f848bba1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55dd96f4c190 con 0x55dd96fea950 2024-04-24T06:51:55.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.042+0000 7f8482800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd96fea950 0x55dd96fead30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.042+0000 7f8479400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd96ff5940 0x55dd96ff7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.042+0000 7f8482800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd96fea950 0x55dd96fead30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:41004/0 (socket says 172.21.15.5:41004) 2024-04-24T06:51:55.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.042+0000 7f8482800640 1 -- 172.21.15.5:0/3308791572 learned_addr learned my addr 172.21.15.5:0/3308791572 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:55.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.042+0000 7f8481e00640 1 --2- 172.21.15.5:0/3308791572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd96feb270 0x55dd96ff5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.042+0000 7f8482800640 1 -- 172.21.15.5:0/3308791572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd96ff5940 msgr2=0x55dd96ff7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:55.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.042+0000 7f8482800640 1 --2- 172.21.15.5:0/3308791572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd96ff5940 0x55dd96ff7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.042+0000 7f8482800640 1 -- 172.21.15.5:0/3308791572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd96feb270 msgr2=0x55dd96ff5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:55.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.042+0000 7f8482800640 1 --2- 172.21.15.5:0/3308791572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd96feb270 0x55dd96ff5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.042+0000 7f8482800640 1 -- 172.21.15.5:0/3308791572 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd96fb3310 con 0x55dd96fea950 2024-04-24T06:51:55.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.043+0000 7f8482800640 1 --2- 172.21.15.5:0/3308791572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd96fea950 0x55dd96fead30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f847c009f50 tx=0x7f847c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1debb378c2f92c52 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:55.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.043+0000 7f8473000640 1 -- 172.21.15.5:0/3308791572 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847c004d20 con 0x55dd96fea950 2024-04-24T06:51:55.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.043+0000 7f8473000640 1 -- 172.21.15.5:0/3308791572 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f847c004050 con 0x55dd96fea950 2024-04-24T06:51:55.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.043+0000 7f8473000640 1 -- 172.21.15.5:0/3308791572 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847c005280 con 0x55dd96fea950 2024-04-24T06:51:55.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.043+0000 7f847b400640 1 --2- 172.21.15.5:0/3333628468 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f84680a3d20 0x7f84680a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.043+0000 7f847b400640 1 -- 172.21.15.5:0/3333628468 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f846c0f0e90 con 0x7f8484107300 2024-04-24T06:51:55.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.044+0000 7f8481e00640 1 --2- 172.21.15.5:0/3333628468 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f84680a3d20 0x7f84680a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.044+0000 7f848bba1740 1 -- 172.21.15.5:0/3308791572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd96fea950 msgr2=0x55dd96fead30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:55.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.044+0000 7f848bba1740 1 --2- 172.21.15.5:0/3308791572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd96fea950 0x55dd96fead30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f847c009f50 tx=0x7f847c0118d0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.044+0000 7f848bba1740 1 -- 172.21.15.5:0/3308791572 shutdown_connections 2024-04-24T06:51:55.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.044+0000 7f848bba1740 1 --2- 172.21.15.5:0/3308791572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd96ff5940 0x55dd96ff7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.044+0000 7f848bba1740 1 --2- 172.21.15.5:0/3308791572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd96feb270 0x55dd96ff5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.044+0000 7f848bba1740 1 --2- 172.21.15.5:0/3308791572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd96fea950 0x55dd96fead30 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.044+0000 7f848bba1740 1 -- 172.21.15.5:0/3308791572 >> 172.21.15.5:0/3308791572 conn(0x55dd96fe6c70 msgr2=0x55dd96fe7050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:55.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.044+0000 7f848bba1740 1 -- 172.21.15.5:0/3308791572 shutdown_connections 2024-04-24T06:51:55.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.044+0000 7f848bba1740 1 -- 172.21.15.5:0/3308791572 wait complete. 2024-04-24T06:51:55.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.044+0000 7f8481e00640 1 --2- 172.21.15.5:0/3333628468 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f84680a3d20 0x7f84680a61e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x55dd96febf10 tx=0x7f8474005f90 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:55.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.045+0000 7f848bba1740 1 -- start start 2024-04-24T06:51:55.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.045+0000 7f848bba1740 1 Processor -- start 2024-04-24T06:51:55.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.045+0000 7f848bba1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd96fea950 0x55dd96ff5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.045+0000 7f848bba1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd96feb270 0x55dd96ff26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.045+0000 7f848bba1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd96ff5940 0x55dd96ff2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.045+0000 7f848bba1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dd96ff87d0 con 0x55dd96fea950 2024-04-24T06:51:55.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.045+0000 7f848bba1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55dd96f09810 con 0x55dd96feb270 2024-04-24T06:51:55.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.045+0000 7f848bba1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55dd96ff8950 con 0x55dd96ff5940 2024-04-24T06:51:55.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.045+0000 7f848bba1740 10 client.0 _ll_register_callbacks cb 0x55dd96ed7140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:51:55.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.046+0000 7f8481e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd96fea950 0x55dd96ff5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.046+0000 7f8481e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd96fea950 0x55dd96ff5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54348/0 (socket says 172.21.15.5:54348) 2024-04-24T06:51:55.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.046+0000 7f8481e00640 1 -- 172.21.15.5:0/652058825 learned_addr learned my addr 172.21.15.5:0/652058825 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:55.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.046+0000 7f8482800640 1 --2- 172.21.15.5:0/652058825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd96ff5940 0x55dd96ff2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.046+0000 7f8479400640 1 --2- 172.21.15.5:0/652058825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd96feb270 0x55dd96ff26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.046+0000 7f8481e00640 1 -- 172.21.15.5:0/652058825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd96ff5940 msgr2=0x55dd96ff2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:55.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.046+0000 7f8481e00640 1 --2- 172.21.15.5:0/652058825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd96ff5940 0x55dd96ff2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.046+0000 7f8481e00640 1 -- 172.21.15.5:0/652058825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd96feb270 msgr2=0x55dd96ff26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:55.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.046+0000 7f8481e00640 1 --2- 172.21.15.5:0/652058825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd96feb270 0x55dd96ff26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.046+0000 7f8481e00640 1 -- 172.21.15.5:0/652058825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd96f1a3f0 con 0x55dd96fea950 2024-04-24T06:51:55.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.047+0000 7f8481e00640 1 --2- 172.21.15.5:0/652058825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd96fea950 0x55dd96ff5370 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f847400f9b0 tx=0x7f8474004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e256d0cb59f3f5fb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:55.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.047+0000 7f8467400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:51:55.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.047+0000 7f8467400640 1 -- 172.21.15.5:0/652058825 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847402e060 con 0x55dd96fea950 2024-04-24T06:51:55.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.047+0000 7f8467400640 1 -- 172.21.15.5:0/652058825 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f8474018030 con 0x55dd96fea950 2024-04-24T06:51:55.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.047+0000 7f848bba1740 1 -- 172.21.15.5:0/652058825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f847c004050 con 0x55dd96fea950 2024-04-24T06:51:55.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.047+0000 7f8467400640 1 -- 172.21.15.5:0/652058825 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f847401e5a0 con 0x55dd96fea950 2024-04-24T06:51:55.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.047+0000 7f848bba1740 10 client.16791 fetch_fsmap learned FSMap version 27 2024-04-24T06:51:55.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.047+0000 7f848bba1740 1 -- 172.21.15.5:0/652058825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dd96c623f0 con 0x55dd96fea950 2024-04-24T06:51:55.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.048+0000 7f8467400640 1 -- 172.21.15.5:0/652058825 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f847c004050 con 0x55dd96fea950 2024-04-24T06:51:55.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.048+0000 7f8467400640 1 -- 172.21.15.5:0/652058825 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f8474029980 con 0x55dd96fea950 2024-04-24T06:51:55.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.048+0000 7f848bba1740 10 client.16791 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:51:55.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.048+0000 7f848bba1740 10 client.16791 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-24T06:51:55.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.048+0000 7f848bba1740 20 client.16791 populate_metadata read hostname 'smithi005' 2024-04-24T06:51:55.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.048+0000 7f848bba1740 1 --2- 172.21.15.5:0/652058825 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55dd97005bb0 0x55dd97007fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.048+0000 7f848bba1740 4 client.16791 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:51:55.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.048+0000 7f848bba1740 1 -- 172.21.15.5:0/652058825 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd96f5dbf0 con 0x55dd97005bb0 2024-04-24T06:51:55.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.048+0000 7f8479400640 1 --2- 172.21.15.5:0/652058825 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55dd97005bb0 0x55dd97007fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.049+0000 7f8467400640 10 client.16791 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:51:55.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.049+0000 7f8479400640 1 --2- 172.21.15.5:0/652058825 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55dd97005bb0 0x55dd97007fc0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7a412214ab465eb8 server_cookie=cd06130b5f83c69e in_seq=0 out_seq=0 2024-04-24T06:51:55.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.050+0000 7f8467400640 1 -- 172.21.15.5:0/652058825 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dd96f5dbf0 con 0x55dd97005bb0 2024-04-24T06:51:55.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.050+0000 7f8467400640 10 client.16791 handle_command_reply: tid=0 2024-04-24T06:51:55.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.050+0000 7f848bba1740 1 client.16791 shutdown 2024-04-24T06:51:55.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.050+0000 7f848bba1740 1 -- 172.21.15.5:0/652058825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd96fea950 msgr2=0x55dd96ff5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:55.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.050+0000 7f848bba1740 1 --2- 172.21.15.5:0/652058825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd96fea950 0x55dd96ff5370 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f847400f9b0 tx=0x7f8474004730 comp rx=0 tx=0).stop 2024-04-24T06:51:55.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.050+0000 7f848bba1740 1 -- 172.21.15.5:0/652058825 shutdown_connections 2024-04-24T06:51:55.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.050+0000 7f848bba1740 1 --2- 172.21.15.5:0/652058825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd96ff5940 0x55dd96ff2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.050+0000 7f848bba1740 1 --2- 172.21.15.5:0/652058825 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55dd97005bb0 0x55dd97007fc0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.050+0000 7f848bba1740 1 --2- 172.21.15.5:0/652058825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd96feb270 0x55dd96ff26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.050+0000 7f848bba1740 1 --2- 172.21.15.5:0/652058825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd96fea950 0x55dd96ff5370 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.050+0000 7f8467400640 0 client.16791 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:51:55.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.050+0000 7f848bba1740 1 -- 172.21.15.5:0/652058825 >> 172.21.15.5:0/652058825 conn(0x55dd96fe6c70 msgr2=0x55dd96feca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:55.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.050+0000 7f848bba1740 1 -- 172.21.15.5:0/652058825 shutdown_connections 2024-04-24T06:51:55.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.050+0000 7f848bba1740 1 -- 172.21.15.5:0/652058825 wait complete. 2024-04-24T06:51:55.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.050+0000 7f848bba1740 20 client.16791 trim_cache size 0 max 16384 2024-04-24T06:51:55.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.065+0000 7f848bba1740 1 Processor -- start 2024-04-24T06:51:55.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.065+0000 7f848bba1740 1 -- start start 2024-04-24T06:51:55.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.066+0000 7f848bba1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd96fea950 0x55dd96fed0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.066+0000 7f848bba1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd96feb270 0x55dd96fe5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.066+0000 7f848bba1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd96ff5940 0x55dd96ff4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.066+0000 7f848bba1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f84841b28b0 con 0x55dd96ff5940 2024-04-24T06:51:55.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.066+0000 7f848bba1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f84841b0180 con 0x55dd96fea950 2024-04-24T06:51:55.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.066+0000 7f848bba1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f84841adff0 con 0x55dd96feb270 2024-04-24T06:51:55.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.066+0000 7f8479400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd96feb270 0x55dd96fe5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.066+0000 7f8479400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd96feb270 0x55dd96fe5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:41032/0 (socket says 172.21.15.5:41032) 2024-04-24T06:51:55.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.066+0000 7f8479400640 1 -- 172.21.15.5:0/2191738678 learned_addr learned my addr 172.21.15.5:0/2191738678 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:55.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.066+0000 7f8482800640 1 --2- 172.21.15.5:0/2191738678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd96ff5940 0x55dd96ff4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.066+0000 7f8481e00640 1 --2- 172.21.15.5:0/2191738678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd96fea950 0x55dd96fed0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.067+0000 7f8479400640 1 -- 172.21.15.5:0/2191738678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd96fea950 msgr2=0x55dd96fed0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:55.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.067+0000 7f8479400640 1 --2- 172.21.15.5:0/2191738678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd96fea950 0x55dd96fed0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.067+0000 7f8479400640 1 -- 172.21.15.5:0/2191738678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd96ff5940 msgr2=0x55dd96ff4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:55.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.067+0000 7f8479400640 1 --2- 172.21.15.5:0/2191738678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd96ff5940 0x55dd96ff4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.067+0000 7f8479400640 1 -- 172.21.15.5:0/2191738678 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd96d2c7d0 con 0x55dd96feb270 2024-04-24T06:51:55.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.067+0000 7f8482800640 1 --2- 172.21.15.5:0/2191738678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd96ff5940 0x55dd96ff4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:51:55.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.067+0000 7f8479400640 1 --2- 172.21.15.5:0/2191738678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd96feb270 0x55dd96fe5c70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f846c0bcb70 tx=0x7f846c0dd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f33c70cd3d358f8a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:55.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.067+0000 7f8472600640 1 -- 172.21.15.5:0/2191738678 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846c049040 con 0x55dd96feb270 2024-04-24T06:51:55.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.067+0000 7f8472600640 1 -- 172.21.15.5:0/2191738678 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f846c0bce70 con 0x55dd96feb270 2024-04-24T06:51:55.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.067+0000 7f8472600640 1 -- 172.21.15.5:0/2191738678 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846c03b460 con 0x55dd96feb270 2024-04-24T06:51:55.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.068+0000 7f848bba1740 1 -- 172.21.15.5:0/2191738678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd96feb270 msgr2=0x55dd96fe5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:55.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.068+0000 7f848bba1740 1 --2- 172.21.15.5:0/2191738678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd96feb270 0x55dd96fe5c70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f846c0bcb70 tx=0x7f846c0dd040 comp rx=0 tx=0).stop 2024-04-24T06:51:55.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.068+0000 7f848bba1740 1 -- 172.21.15.5:0/2191738678 shutdown_connections 2024-04-24T06:51:55.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.068+0000 7f848bba1740 1 --2- 172.21.15.5:0/2191738678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd96ff5940 0x55dd96ff4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.068+0000 7f848bba1740 1 --2- 172.21.15.5:0/2191738678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd96feb270 0x55dd96fe5c70 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.068+0000 7f848bba1740 1 --2- 172.21.15.5:0/2191738678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd96fea950 0x55dd96fed0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.068+0000 7f848bba1740 1 -- 172.21.15.5:0/2191738678 >> 172.21.15.5:0/2191738678 conn(0x55dd96fe6c70 msgr2=0x55dd96fe5560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:55.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.068+0000 7f848bba1740 1 -- 172.21.15.5:0/2191738678 shutdown_connections 2024-04-24T06:51:55.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.068+0000 7f848bba1740 1 -- 172.21.15.5:0/2191738678 wait complete. 2024-04-24T06:51:55.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.069+0000 7f848bba1740 1 -- start start 2024-04-24T06:51:55.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.069+0000 7f848bba1740 1 Processor -- start 2024-04-24T06:51:55.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.069+0000 7f848bba1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd96fea950 0x55dd96ffdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.069+0000 7f848bba1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd96feb270 0x55dd96ffe0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.069+0000 7f848bba1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd96ff5940 0x55dd96ff8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.069+0000 7f848bba1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f84841ae640 con 0x55dd96ff5940 2024-04-24T06:51:55.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.069+0000 7f848bba1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f84841afab0 con 0x55dd96fea950 2024-04-24T06:51:55.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.069+0000 7f848bba1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f84841aed10 con 0x55dd96feb270 2024-04-24T06:51:55.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.069+0000 7f848bba1740 10 client.0 _ll_register_callbacks cb 0x7f8474009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:51:55.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.070+0000 7f8482800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd96ff5940 0x55dd96ff8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.070+0000 7f8479400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd96feb270 0x55dd96ffe0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.070+0000 7f8479400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd96feb270 0x55dd96ffe0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:41044/0 (socket says 172.21.15.5:41044) 2024-04-24T06:51:55.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.070+0000 7f8479400640 1 -- 172.21.15.5:0/958412382 learned_addr learned my addr 172.21.15.5:0/958412382 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:55.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.070+0000 7f8481e00640 1 --2- 172.21.15.5:0/958412382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd96fea950 0x55dd96ffdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.070+0000 7f8479400640 1 -- 172.21.15.5:0/958412382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd96fea950 msgr2=0x55dd96ffdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:55.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.070+0000 7f8479400640 1 --2- 172.21.15.5:0/958412382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd96fea950 0x55dd96ffdbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.070+0000 7f8479400640 1 -- 172.21.15.5:0/958412382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd96ff5940 msgr2=0x55dd96ff8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:55.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.070+0000 7f8479400640 1 --2- 172.21.15.5:0/958412382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd96ff5940 0x55dd96ff8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.070+0000 7f8479400640 1 -- 172.21.15.5:0/958412382 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd96ff9570 con 0x55dd96feb270 2024-04-24T06:51:55.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.070+0000 7f8479400640 1 --2- 172.21.15.5:0/958412382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd96feb270 0x55dd96ffe0f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f846c0b9840 tx=0x7f846c005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad801215b048f9df server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:55.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.070+0000 7f8467400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:51:55.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.071+0000 7f8467400640 1 -- 172.21.15.5:0/958412382 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846c049040 con 0x55dd96feb270 2024-04-24T06:51:55.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.071+0000 7f8467400640 1 -- 172.21.15.5:0/958412382 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f846c0b94e0 con 0x55dd96feb270 2024-04-24T06:51:55.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.071+0000 7f848bba1740 1 -- 172.21.15.5:0/958412382 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f846c0bce70 con 0x55dd96feb270 2024-04-24T06:51:55.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.071+0000 7f8467400640 1 -- 172.21.15.5:0/958412382 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f846c0bc9b0 con 0x55dd96feb270 2024-04-24T06:51:55.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.071+0000 7f848bba1740 10 client.26947 fetch_fsmap learned FSMap version 27 2024-04-24T06:51:55.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.071+0000 7f848bba1740 1 -- 172.21.15.5:0/958412382 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dd9706cd60 con 0x55dd96feb270 2024-04-24T06:51:55.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.071+0000 7f8467400640 1 -- 172.21.15.5:0/958412382 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f846c0bce70 con 0x55dd96feb270 2024-04-24T06:51:55.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.071+0000 7f8467400640 1 -- 172.21.15.5:0/958412382 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f846c0c5630 con 0x55dd96feb270 2024-04-24T06:51:55.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.071+0000 7f848bba1740 10 client.26947 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:51:55.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.071+0000 7f848bba1740 10 client.26947 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-24T06:51:55.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.071+0000 7f848bba1740 20 client.26947 populate_metadata read hostname 'smithi005' 2024-04-24T06:51:55.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.071+0000 7f848bba1740 1 --2- 172.21.15.5:0/958412382 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55dd9706d3f0 0x55dd9706f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.071+0000 7f848bba1740 4 client.26947 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:51:55.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.071+0000 7f848bba1740 1 -- 172.21.15.5:0/958412382 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dd96f57a60 con 0x55dd9706d3f0 2024-04-24T06:51:55.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.072+0000 7f8481e00640 1 --2- 172.21.15.5:0/958412382 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55dd9706d3f0 0x55dd9706f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.072+0000 7f8481e00640 1 --2- 172.21.15.5:0/958412382 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55dd9706d3f0 0x55dd9706f810 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=47b8c6bf491e8060 server_cookie=2a19d643641d287f in_seq=0 out_seq=0 2024-04-24T06:51:55.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.072+0000 7f8467400640 10 client.26947 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:51:55.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.072+0000 7f8467400640 1 -- 172.21.15.5:0/958412382 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dd96f57a60 con 0x55dd9706d3f0 2024-04-24T06:51:55.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.072+0000 7f8467400640 10 client.26947 handle_command_reply: tid=0 2024-04-24T06:51:55.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.072+0000 7f848bba1740 1 client.26947 shutdown 2024-04-24T06:51:55.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.073+0000 7f848bba1740 1 -- 172.21.15.5:0/958412382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd96feb270 msgr2=0x55dd96ffe0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:55.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.073+0000 7f848bba1740 1 --2- 172.21.15.5:0/958412382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd96feb270 0x55dd96ffe0f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f846c0b9840 tx=0x7f846c005d50 comp rx=0 tx=0).stop 2024-04-24T06:51:55.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.073+0000 7f848bba1740 1 -- 172.21.15.5:0/958412382 shutdown_connections 2024-04-24T06:51:55.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.073+0000 7f848bba1740 1 --2- 172.21.15.5:0/958412382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd96ff5940 0x55dd96ff8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.073+0000 7f848bba1740 1 --2- 172.21.15.5:0/958412382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd96feb270 0x55dd96ffe0f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.073+0000 7f848bba1740 1 --2- 172.21.15.5:0/958412382 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55dd9706d3f0 0x55dd9706f810 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.073+0000 7f848bba1740 1 --2- 172.21.15.5:0/958412382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd96fea950 0x55dd96ffdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.073+0000 7f8467400640 0 client.26947 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:51:55.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.073+0000 7f848bba1740 1 -- 172.21.15.5:0/958412382 >> 172.21.15.5:0/958412382 conn(0x55dd96fe6c70 msgr2=0x55dd96fe5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:55.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.073+0000 7f848bba1740 1 -- 172.21.15.5:0/958412382 shutdown_connections 2024-04-24T06:51:55.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.073+0000 7f848bba1740 1 -- 172.21.15.5:0/958412382 wait complete. 2024-04-24T06:51:55.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.073+0000 7f848bba1740 20 client.26947 trim_cache size 0 max 16384 2024-04-24T06:51:55.082 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:51:55.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.074+0000 7f8488c00640 1 -- 172.21.15.5:0/3333628468 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f84680a3d20 msgr2=0x7f84680a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:55.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.074+0000 7f8488c00640 1 --2- 172.21.15.5:0/3333628468 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f84680a3d20 0x7f84680a61e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x55dd96febf10 tx=0x7f8474005f90 comp rx=0 tx=0).stop 2024-04-24T06:51:55.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.074+0000 7f8488c00640 1 -- 172.21.15.5:0/3333628468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8484107300 msgr2=0x7f84841a0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:55.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.074+0000 7f8488c00640 1 --2- 172.21.15.5:0/3333628468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8484107300 0x7f84841a0cb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f846c02a920 tx=0x7f846c003b20 comp rx=0 tx=0).stop 2024-04-24T06:51:55.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.074+0000 7f8488c00640 1 -- 172.21.15.5:0/3333628468 shutdown_connections 2024-04-24T06:51:55.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.074+0000 7f8488c00640 1 --2- 172.21.15.5:0/3333628468 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f84680a3d20 0x7f84680a61e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.074+0000 7f8488c00640 1 --2- 172.21.15.5:0/3333628468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f848419a920 0x7f848419add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.074+0000 7f8488c00640 1 --2- 172.21.15.5:0/3333628468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8484107300 0x7f84841a0cb0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.074+0000 7f8488c00640 1 --2- 172.21.15.5:0/3333628468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8484102230 0x7f84841a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.074+0000 7f8488c00640 1 -- 172.21.15.5:0/3333628468 >> 172.21.15.5:0/3333628468 conn(0x7f84840fdab0 msgr2=0x7f84840ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:55.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.074+0000 7f8488c00640 1 -- 172.21.15.5:0/3333628468 shutdown_connections 2024-04-24T06:51:55.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.074+0000 7f8488c00640 1 -- 172.21.15.5:0/3333628468 wait complete. 2024-04-24T06:51:55.093 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T06:51:55.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.291+0000 7f3c75600640 1 -- 172.21.15.5:0/2344785021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c70108ee0 msgr2=0x7f3c701152b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:55.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.291+0000 7f3c75600640 1 --2- 172.21.15.5:0/2344785021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c70108ee0 0x7f3c701152b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3c68009c10 tx=0x7f3c6802a780 comp rx=0 tx=0).stop 2024-04-24T06:51:55.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.291+0000 7f3c75600640 1 -- 172.21.15.5:0/2344785021 shutdown_connections 2024-04-24T06:51:55.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.291+0000 7f3c75600640 1 --2- 172.21.15.5:0/2344785021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c70108ee0 0x7f3c701152b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.291+0000 7f3c75600640 1 --2- 172.21.15.5:0/2344785021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c70108540 0x7f3c701089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.291+0000 7f3c75600640 1 --2- 172.21.15.5:0/2344785021 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c7010e4f0 0x7f3c7010e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.291+0000 7f3c75600640 1 -- 172.21.15.5:0/2344785021 >> 172.21.15.5:0/2344785021 conn(0x7f3c70077e10 msgr2=0x7f3c70106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:55.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.291+0000 7f3c75600640 1 -- 172.21.15.5:0/2344785021 shutdown_connections 2024-04-24T06:51:55.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.291+0000 7f3c75600640 1 -- 172.21.15.5:0/2344785021 wait complete. 2024-04-24T06:51:55.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.292+0000 7f3c75600640 1 Processor -- start 2024-04-24T06:51:55.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.292+0000 7f3c75600640 1 -- start start 2024-04-24T06:51:55.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.292+0000 7f3c75600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c70108540 0x7f3c701a4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.292+0000 7f3c75600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c70108ee0 0x7f3c701a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.292+0000 7f3c75600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c7010e4f0 0x7f3c7019eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.292+0000 7f3c75600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3c701187a0 con 0x7f3c70108540 2024-04-24T06:51:55.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.292+0000 7f3c75600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3c70118620 con 0x7f3c70108ee0 2024-04-24T06:51:55.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.292+0000 7f3c75600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3c70118920 con 0x7f3c7010e4f0 2024-04-24T06:51:55.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.293+0000 7f3c6fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c70108540 0x7f3c701a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.293+0000 7f3c67e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c70108ee0 0x7f3c701a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.293+0000 7f3c6fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c70108540 0x7f3c701a4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54380/0 (socket says 172.21.15.5:54380) 2024-04-24T06:51:55.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.293+0000 7f3c74c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c7010e4f0 0x7f3c7019eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.293+0000 7f3c6fe00640 1 -- 172.21.15.5:0/4207208684 learned_addr learned my addr 172.21.15.5:0/4207208684 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:55.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.293+0000 7f3c67e00640 1 -- 172.21.15.5:0/4207208684 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c7010e4f0 msgr2=0x7f3c7019eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:55.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.293+0000 7f3c67e00640 1 --2- 172.21.15.5:0/4207208684 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c7010e4f0 0x7f3c7019eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.293+0000 7f3c67e00640 1 -- 172.21.15.5:0/4207208684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c70108540 msgr2=0x7f3c701a4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:55.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.293+0000 7f3c67e00640 1 --2- 172.21.15.5:0/4207208684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c70108540 0x7f3c701a4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.293+0000 7f3c67e00640 1 -- 172.21.15.5:0/4207208684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c7019f280 con 0x7f3c70108ee0 2024-04-24T06:51:55.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.293+0000 7f3c6fe00640 1 --2- 172.21.15.5:0/4207208684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c70108540 0x7f3c701a4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:51:55.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.293+0000 7f3c67e00640 1 --2- 172.21.15.5:0/4207208684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c70108ee0 0x7f3c701a4e90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3c5800e9f0 tx=0x7f3c5800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:55.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.294+0000 7f3c6d600640 1 -- 172.21.15.5:0/4207208684 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5800ccc0 con 0x7f3c70108ee0 2024-04-24T06:51:55.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.294+0000 7f3c75600640 1 -- 172.21.15.5:0/4207208684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c7019f570 con 0x7f3c70108ee0 2024-04-24T06:51:55.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.294+0000 7f3c6d600640 1 -- 172.21.15.5:0/4207208684 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c58010910 con 0x7f3c70108ee0 2024-04-24T06:51:55.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.294+0000 7f3c6d600640 1 -- 172.21.15.5:0/4207208684 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c58018960 con 0x7f3c70108ee0 2024-04-24T06:51:55.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.294+0000 7f3c75600640 1 -- 172.21.15.5:0/4207208684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c701ab810 con 0x7f3c70108ee0 2024-04-24T06:51:55.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.297+0000 7f3c799fc740 1 Processor -- start 2024-04-24T06:51:55.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.297+0000 7f3c799fc740 1 -- start start 2024-04-24T06:51:55.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.297+0000 7f3c6d600640 1 -- 172.21.15.5:0/4207208684 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f3c58010ac0 con 0x7f3c70108ee0 2024-04-24T06:51:55.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.298+0000 7f3c799fc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611eef69b40 0x5611eef69f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.298+0000 7f3c799fc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611eef6a460 0x5611eef745f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.298+0000 7f3c799fc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611eef74b30 0x5611eef76f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.298+0000 7f3c799fc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5611eeecf1d0 con 0x5611eef6a460 2024-04-24T06:51:55.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.298+0000 7f3c799fc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5611eee75450 con 0x5611eef69b40 2024-04-24T06:51:55.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.298+0000 7f3c799fc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5611eeecb190 con 0x5611eef74b30 2024-04-24T06:51:55.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.298+0000 7f3c74c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611eef69b40 0x5611eef69f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.298+0000 7f3c6fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611eef6a460 0x5611eef745f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.298+0000 7f3c74c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611eef69b40 0x5611eef69f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:44524/0 (socket says 172.21.15.5:44524) 2024-04-24T06:51:55.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.298+0000 7f3c74c00640 1 -- 172.21.15.5:0/3781117587 learned_addr learned my addr 172.21.15.5:0/3781117587 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:55.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.298+0000 7f3c67e00640 1 --2- 172.21.15.5:0/3781117587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611eef74b30 0x5611eef76f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.298+0000 7f3c74c00640 1 -- 172.21.15.5:0/3781117587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611eef74b30 msgr2=0x5611eef76f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:55.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.298+0000 7f3c74c00640 1 --2- 172.21.15.5:0/3781117587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611eef74b30 0x5611eef76f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.298+0000 7f3c74c00640 1 -- 172.21.15.5:0/3781117587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611eef6a460 msgr2=0x5611eef745f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:55.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.298+0000 7f3c74c00640 1 --2- 172.21.15.5:0/3781117587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611eef6a460 0x5611eef745f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.298+0000 7f3c74c00640 1 -- 172.21.15.5:0/3781117587 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611eef32310 con 0x5611eef69b40 2024-04-24T06:51:55.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.299+0000 7f3c74c00640 1 --2- 172.21.15.5:0/3781117587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611eef69b40 0x5611eef69f20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3c68009be0 tx=0x7f3c68030ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f78de2df2a4326b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:55.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.299+0000 7f3c65200640 1 -- 172.21.15.5:0/3781117587 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c68041030 con 0x5611eef69b40 2024-04-24T06:51:55.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.299+0000 7f3c65200640 1 -- 172.21.15.5:0/3781117587 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c68030c30 con 0x5611eef69b40 2024-04-24T06:51:55.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.299+0000 7f3c65200640 1 -- 172.21.15.5:0/3781117587 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c68036040 con 0x5611eef69b40 2024-04-24T06:51:55.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.299+0000 7f3c6d600640 1 --2- 172.21.15.5:0/4207208684 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f3c540a3da0 0x7f3c540a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.300+0000 7f3c6d600640 1 -- 172.21.15.5:0/4207208684 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f3c580d0b50 con 0x7f3c70108ee0 2024-04-24T06:51:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.300+0000 7f3c6fe00640 1 --2- 172.21.15.5:0/4207208684 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f3c540a3da0 0x7f3c540a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.300+0000 7f3c799fc740 1 -- 172.21.15.5:0/3781117587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611eef69b40 msgr2=0x5611eef69f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.300+0000 7f3c799fc740 1 --2- 172.21.15.5:0/3781117587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611eef69b40 0x5611eef69f20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3c68009be0 tx=0x7f3c68030ac0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.300+0000 7f3c799fc740 1 -- 172.21.15.5:0/3781117587 shutdown_connections 2024-04-24T06:51:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.300+0000 7f3c799fc740 1 --2- 172.21.15.5:0/3781117587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611eef74b30 0x5611eef76f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.300+0000 7f3c799fc740 1 --2- 172.21.15.5:0/3781117587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611eef6a460 0x5611eef745f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.300+0000 7f3c799fc740 1 --2- 172.21.15.5:0/3781117587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611eef69b40 0x5611eef69f20 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.300+0000 7f3c799fc740 1 -- 172.21.15.5:0/3781117587 >> 172.21.15.5:0/3781117587 conn(0x5611eef65e60 msgr2=0x5611eef66240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.300+0000 7f3c799fc740 1 -- 172.21.15.5:0/3781117587 shutdown_connections 2024-04-24T06:51:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.300+0000 7f3c799fc740 1 -- 172.21.15.5:0/3781117587 wait complete. 2024-04-24T06:51:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.300+0000 7f3c6fe00640 1 --2- 172.21.15.5:0/4207208684 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f3c540a3da0 0x7f3c540a6260 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x5611eef6b100 tx=0x7f3c60009260 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.301+0000 7f3c799fc740 1 -- start start 2024-04-24T06:51:55.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.301+0000 7f3c799fc740 1 Processor -- start 2024-04-24T06:51:55.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.301+0000 7f3c799fc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611eef69b40 0x5611eef745e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.301+0000 7f3c799fc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611eef6a460 0x5611eef718d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.301+0000 7f3c799fc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611eef74b30 0x5611eef72010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.301+0000 7f3c799fc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5611eef77960 con 0x5611eef6a460 2024-04-24T06:51:55.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.301+0000 7f3c799fc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5611eee88810 con 0x5611eef69b40 2024-04-24T06:51:55.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.301+0000 7f3c799fc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5611eef77ae0 con 0x5611eef74b30 2024-04-24T06:51:55.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.302+0000 7f3c799fc740 10 client.0 _ll_register_callbacks cb 0x5611eee56140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:51:55.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.302+0000 7f3c67e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611eef6a460 0x5611eef718d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.302+0000 7f3c6fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611eef69b40 0x5611eef745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.302+0000 7f3c74c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611eef74b30 0x5611eef72010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.302+0000 7f3c74c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611eef74b30 0x5611eef72010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:41080/0 (socket says 172.21.15.5:41080) 2024-04-24T06:51:55.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.302+0000 7f3c6fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611eef69b40 0x5611eef745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:44540/0 (socket says 172.21.15.5:44540) 2024-04-24T06:51:55.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.302+0000 7f3c74c00640 1 -- 172.21.15.5:0/2728985798 learned_addr learned my addr 172.21.15.5:0/2728985798 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:55.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.302+0000 7f3c74c00640 1 -- 172.21.15.5:0/2728985798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611eef69b40 msgr2=0x5611eef745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.302+0000 7f3c74c00640 1 --2- 172.21.15.5:0/2728985798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611eef69b40 0x5611eef745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.302+0000 7f3c74c00640 1 -- 172.21.15.5:0/2728985798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611eef6a460 msgr2=0x5611eef718d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.302+0000 7f3c74c00640 1 --2- 172.21.15.5:0/2728985798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611eef6a460 0x5611eef718d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.302+0000 7f3c74c00640 1 -- 172.21.15.5:0/2728985798 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611eee993f0 con 0x5611eef74b30 2024-04-24T06:51:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.302+0000 7f3c74c00640 1 --2- 172.21.15.5:0/2728985798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611eef74b30 0x5611eef72010 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3c68003d20 tx=0x7f3c68009940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85b7b8802a8729b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.302+0000 7f3c5d600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:51:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.303+0000 7f3c5d600640 1 -- 172.21.15.5:0/2728985798 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6804a070 con 0x5611eef74b30 2024-04-24T06:51:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.303+0000 7f3c5d600640 1 -- 172.21.15.5:0/2728985798 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f3c680303a0 con 0x5611eef74b30 2024-04-24T06:51:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.303+0000 7f3c799fc740 1 -- 172.21.15.5:0/2728985798 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c68030c30 con 0x5611eef74b30 2024-04-24T06:51:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.303+0000 7f3c5d600640 1 -- 172.21.15.5:0/2728985798 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3c68036070 con 0x5611eef74b30 2024-04-24T06:51:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.303+0000 7f3c799fc740 10 client.26959 fetch_fsmap learned FSMap version 27 2024-04-24T06:51:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.303+0000 7f3c799fc740 1 -- 172.21.15.5:0/2728985798 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5611eebe13f0 con 0x5611eef74b30 2024-04-24T06:51:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.304+0000 7f3c5d600640 1 -- 172.21.15.5:0/2728985798 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c68030c30 con 0x5611eef74b30 2024-04-24T06:51:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.304+0000 7f3c5d600640 1 -- 172.21.15.5:0/2728985798 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f3c6803d460 con 0x5611eef74b30 2024-04-24T06:51:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.304+0000 7f3c799fc740 10 client.26959 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:51:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.304+0000 7f3c799fc740 10 client.26959 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T06:51:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.304+0000 7f3c799fc740 20 client.26959 populate_metadata read hostname 'smithi005' 2024-04-24T06:51:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.304+0000 7f3c799fc740 1 --2- 172.21.15.5:0/2728985798 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x5611eef84bc0 0x5611eef86fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.304+0000 7f3c799fc740 4 client.26959 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:51:55.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.304+0000 7f3c799fc740 1 -- 172.21.15.5:0/2728985798 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611eeedcbf0 con 0x5611eef84bc0 2024-04-24T06:51:55.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.304+0000 7f3c6fe00640 1 --2- 172.21.15.5:0/2728985798 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x5611eef84bc0 0x5611eef86fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.305+0000 7f3c6fe00640 1 --2- 172.21.15.5:0/2728985798 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x5611eef84bc0 0x5611eef86fb0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e2148b125d1e233d server_cookie=4379bc834ca8d2ad in_seq=0 out_seq=0 2024-04-24T06:51:55.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.305+0000 7f3c5d600640 10 client.26959 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:51:55.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.306+0000 7f3c5d600640 1 -- 172.21.15.5:0/2728985798 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5611eeedcbf0 con 0x5611eef84bc0 2024-04-24T06:51:55.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.306+0000 7f3c5d600640 10 client.26959 handle_command_reply: tid=0 2024-04-24T06:51:55.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.306+0000 7f3c799fc740 1 client.26959 shutdown 2024-04-24T06:51:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.306+0000 7f3c799fc740 1 -- 172.21.15.5:0/2728985798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611eef74b30 msgr2=0x5611eef72010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.306+0000 7f3c799fc740 1 --2- 172.21.15.5:0/2728985798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611eef74b30 0x5611eef72010 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3c68003d20 tx=0x7f3c68009940 comp rx=0 tx=0).stop 2024-04-24T06:51:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.306+0000 7f3c799fc740 1 -- 172.21.15.5:0/2728985798 shutdown_connections 2024-04-24T06:51:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.306+0000 7f3c799fc740 1 --2- 172.21.15.5:0/2728985798 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x5611eef84bc0 0x5611eef86fb0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.306+0000 7f3c799fc740 1 --2- 172.21.15.5:0/2728985798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611eef74b30 0x5611eef72010 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.306+0000 7f3c799fc740 1 --2- 172.21.15.5:0/2728985798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611eef6a460 0x5611eef718d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.306+0000 7f3c5d600640 0 client.26959 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:51:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.306+0000 7f3c799fc740 1 --2- 172.21.15.5:0/2728985798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611eef69b40 0x5611eef745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.306+0000 7f3c799fc740 1 -- 172.21.15.5:0/2728985798 >> 172.21.15.5:0/2728985798 conn(0x5611eef65e60 msgr2=0x5611eef6bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.306+0000 7f3c799fc740 1 -- 172.21.15.5:0/2728985798 shutdown_connections 2024-04-24T06:51:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.306+0000 7f3c799fc740 1 -- 172.21.15.5:0/2728985798 wait complete. 2024-04-24T06:51:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.307+0000 7f3c799fc740 20 client.26959 trim_cache size 0 max 16384 2024-04-24T06:51:55.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.322+0000 7f3c799fc740 1 Processor -- start 2024-04-24T06:51:55.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.322+0000 7f3c799fc740 1 -- start start 2024-04-24T06:51:55.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.322+0000 7f3c799fc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611eef69b40 0x5611eef70a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.322+0000 7f3c799fc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611eef6a460 0x5611eef64dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.322+0000 7f3c799fc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611eef74b30 0x5611eef74110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.322+0000 7f3c799fc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3c701b6aa0 con 0x5611eef74b30 2024-04-24T06:51:55.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.322+0000 7f3c799fc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3c701b4370 con 0x5611eef69b40 2024-04-24T06:51:55.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.322+0000 7f3c799fc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3c701b21e0 con 0x5611eef6a460 2024-04-24T06:51:55.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.322+0000 7f3c74c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611eef74b30 0x5611eef74110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.322+0000 7f3c67e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611eef6a460 0x5611eef64dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.322+0000 7f3c67e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611eef6a460 0x5611eef64dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:41088/0 (socket says 172.21.15.5:41088) 2024-04-24T06:51:55.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.322+0000 7f3c67e00640 1 -- 172.21.15.5:0/3976713066 learned_addr learned my addr 172.21.15.5:0/3976713066 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:55.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.322+0000 7f3c6fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611eef69b40 0x5611eef70a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.323+0000 7f3c74c00640 1 -- 172.21.15.5:0/3976713066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611eef6a460 msgr2=0x5611eef64dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:55.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.323+0000 7f3c74c00640 1 --2- 172.21.15.5:0/3976713066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611eef6a460 0x5611eef64dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.323+0000 7f3c74c00640 1 -- 172.21.15.5:0/3976713066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611eef69b40 msgr2=0x5611eef70a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:55.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.323+0000 7f3c74c00640 1 --2- 172.21.15.5:0/3976713066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611eef69b40 0x5611eef70a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.323+0000 7f3c74c00640 1 -- 172.21.15.5:0/3976713066 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611eef6a290 con 0x5611eef74b30 2024-04-24T06:51:55.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.323+0000 7f3c74c00640 1 --2- 172.21.15.5:0/3976713066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611eef74b30 0x5611eef74110 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f3c68009b30 tx=0x7f3c68005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d047b41ee5f28f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:55.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.323+0000 7f3c5fe00640 1 -- 172.21.15.5:0/3976713066 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6804a070 con 0x5611eef74b30 2024-04-24T06:51:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.323+0000 7f3c5fe00640 1 -- 172.21.15.5:0/3976713066 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c68030ac0 con 0x5611eef74b30 2024-04-24T06:51:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.323+0000 7f3c5fe00640 1 -- 172.21.15.5:0/3976713066 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c68056030 con 0x5611eef74b30 2024-04-24T06:51:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.324+0000 7f3c799fc740 1 -- 172.21.15.5:0/3976713066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611eef74b30 msgr2=0x5611eef74110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.324+0000 7f3c799fc740 1 --2- 172.21.15.5:0/3976713066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611eef74b30 0x5611eef74110 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f3c68009b30 tx=0x7f3c68005fd0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.324+0000 7f3c799fc740 1 -- 172.21.15.5:0/3976713066 shutdown_connections 2024-04-24T06:51:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.324+0000 7f3c799fc740 1 --2- 172.21.15.5:0/3976713066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611eef74b30 0x5611eef74110 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.324+0000 7f3c799fc740 1 --2- 172.21.15.5:0/3976713066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611eef6a460 0x5611eef64dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.324+0000 7f3c799fc740 1 --2- 172.21.15.5:0/3976713066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611eef69b40 0x5611eef70a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.324+0000 7f3c799fc740 1 -- 172.21.15.5:0/3976713066 >> 172.21.15.5:0/3976713066 conn(0x5611eef65e60 msgr2=0x5611eef646c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:55.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.324+0000 7f3c799fc740 1 -- 172.21.15.5:0/3976713066 shutdown_connections 2024-04-24T06:51:55.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.324+0000 7f3c799fc740 1 -- 172.21.15.5:0/3976713066 wait complete. 2024-04-24T06:51:55.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.325+0000 7f3c799fc740 1 -- start start 2024-04-24T06:51:55.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.325+0000 7f3c799fc740 1 Processor -- start 2024-04-24T06:51:55.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.325+0000 7f3c799fc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611eef69b40 0x5611eef7cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.325+0000 7f3c799fc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611eef6a460 0x5611eef7d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.325+0000 7f3c799fc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611eef74b30 0x5611eef77d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.325+0000 7f3c799fc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3c701b2830 con 0x5611eef6a460 2024-04-24T06:51:55.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.325+0000 7f3c799fc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3c701b3ca0 con 0x5611eef69b40 2024-04-24T06:51:55.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.325+0000 7f3c799fc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3c701b2f00 con 0x5611eef74b30 2024-04-24T06:51:55.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.325+0000 7f3c799fc740 10 client.0 _ll_register_callbacks cb 0x7f3c60009b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:51:55.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.326+0000 7f3c67e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611eef6a460 0x5611eef7d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.326+0000 7f3c6fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611eef69b40 0x5611eef7cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.326+0000 7f3c67e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611eef6a460 0x5611eef7d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54434/0 (socket says 172.21.15.5:54434) 2024-04-24T06:51:55.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.326+0000 7f3c67e00640 1 -- 172.21.15.5:0/3054673693 learned_addr learned my addr 172.21.15.5:0/3054673693 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:55.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.326+0000 7f3c74c00640 1 --2- 172.21.15.5:0/3054673693 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611eef74b30 0x5611eef77d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.326+0000 7f3c6fe00640 1 -- 172.21.15.5:0/3054673693 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611eef74b30 msgr2=0x5611eef77d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:55.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.326+0000 7f3c6fe00640 1 --2- 172.21.15.5:0/3054673693 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611eef74b30 0x5611eef77d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.326+0000 7f3c6fe00640 1 -- 172.21.15.5:0/3054673693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611eef6a460 msgr2=0x5611eef7d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:55.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.326+0000 7f3c6fe00640 1 --2- 172.21.15.5:0/3054673693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611eef6a460 0x5611eef7d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.326+0000 7f3c6fe00640 1 -- 172.21.15.5:0/3054673693 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611eecab7d0 con 0x5611eef69b40 2024-04-24T06:51:55.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.326+0000 7f3c67e00640 1 --2- 172.21.15.5:0/3054673693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611eef6a460 0x5611eef7d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:51:55.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.326+0000 7f3c6fe00640 1 --2- 172.21.15.5:0/3054673693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611eef69b40 0x5611eef7cd70 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x5611eef72a10 tx=0x7f3c600046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9015038a6722d08 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:55.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.326+0000 7f3c5d600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:51:55.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.327+0000 7f3c5d600640 1 -- 172.21.15.5:0/3054673693 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6002e070 con 0x5611eef69b40 2024-04-24T06:51:55.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.327+0000 7f3c5d600640 1 -- 172.21.15.5:0/3054673693 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f3c6002a380 con 0x5611eef69b40 2024-04-24T06:51:55.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.327+0000 7f3c799fc740 1 -- 172.21.15.5:0/3054673693 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c68030ac0 con 0x5611eef69b40 2024-04-24T06:51:55.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.327+0000 7f3c5d600640 1 -- 172.21.15.5:0/3054673693 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3c6000fe50 con 0x5611eef69b40 2024-04-24T06:51:55.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.327+0000 7f3c799fc740 10 client.26951 fetch_fsmap learned FSMap version 27 2024-04-24T06:51:55.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.327+0000 7f3c799fc740 1 -- 172.21.15.5:0/3054673693 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5611eefeb570 con 0x5611eef69b40 2024-04-24T06:51:55.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.327+0000 7f3c5d600640 1 -- 172.21.15.5:0/3054673693 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c68030ac0 con 0x5611eef69b40 2024-04-24T06:51:55.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.327+0000 7f3c5d600640 1 -- 172.21.15.5:0/3054673693 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f3c60032590 con 0x5611eef69b40 2024-04-24T06:51:55.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.328+0000 7f3c799fc740 10 client.26951 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:51:55.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.328+0000 7f3c799fc740 10 client.26951 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T06:51:55.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.328+0000 7f3c799fc740 20 client.26951 populate_metadata read hostname 'smithi005' 2024-04-24T06:51:55.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.328+0000 7f3c799fc740 1 --2- 172.21.15.5:0/3054673693 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x5611eefebc40 0x5611eefee060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.328+0000 7f3c799fc740 4 client.26951 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:51:55.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.328+0000 7f3c799fc740 1 -- 172.21.15.5:0/3054673693 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5611eeed6a60 con 0x5611eefebc40 2024-04-24T06:51:55.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.328+0000 7f3c67e00640 1 --2- 172.21.15.5:0/3054673693 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x5611eefebc40 0x5611eefee060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.328+0000 7f3c67e00640 1 --2- 172.21.15.5:0/3054673693 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x5611eefebc40 0x5611eefee060 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fe8c68a0ee2478c7 server_cookie=12e94949d7f06271 in_seq=0 out_seq=0 2024-04-24T06:51:55.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.328+0000 7f3c5d600640 10 client.26951 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:51:55.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.329+0000 7f3c5d600640 1 -- 172.21.15.5:0/3054673693 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5611eeed6a60 con 0x5611eefebc40 2024-04-24T06:51:55.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.329+0000 7f3c5d600640 10 client.26951 handle_command_reply: tid=0 2024-04-24T06:51:55.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.329+0000 7f3c799fc740 1 client.26951 shutdown 2024-04-24T06:51:55.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.329+0000 7f3c799fc740 1 -- 172.21.15.5:0/3054673693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611eef69b40 msgr2=0x5611eef7cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:55.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.329+0000 7f3c799fc740 1 --2- 172.21.15.5:0/3054673693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611eef69b40 0x5611eef7cd70 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x5611eef72a10 tx=0x7f3c600046c0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.329+0000 7f3c799fc740 1 -- 172.21.15.5:0/3054673693 shutdown_connections 2024-04-24T06:51:55.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.329+0000 7f3c799fc740 1 --2- 172.21.15.5:0/3054673693 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x5611eefebc40 0x5611eefee060 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.329+0000 7f3c799fc740 1 --2- 172.21.15.5:0/3054673693 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611eef74b30 0x5611eef77d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.329+0000 7f3c799fc740 1 --2- 172.21.15.5:0/3054673693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611eef6a460 0x5611eef7d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.329+0000 7f3c5d600640 0 client.26951 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:51:55.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.329+0000 7f3c799fc740 1 --2- 172.21.15.5:0/3054673693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5611eef69b40 0x5611eef7cd70 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.329+0000 7f3c799fc740 1 -- 172.21.15.5:0/3054673693 >> 172.21.15.5:0/3054673693 conn(0x5611eef65e60 msgr2=0x5611eef64f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:55.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.329+0000 7f3c799fc740 1 -- 172.21.15.5:0/3054673693 shutdown_connections 2024-04-24T06:51:55.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.329+0000 7f3c799fc740 1 -- 172.21.15.5:0/3054673693 wait complete. 2024-04-24T06:51:55.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.329+0000 7f3c799fc740 20 client.26951 trim_cache size 0 max 16384 2024-04-24T06:51:55.338 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:51:55.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.330+0000 7f3c75600640 1 -- 172.21.15.5:0/4207208684 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f3c540a3da0 msgr2=0x7f3c540a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:55.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.330+0000 7f3c75600640 1 --2- 172.21.15.5:0/4207208684 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f3c540a3da0 0x7f3c540a6260 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x5611eef6b100 tx=0x7f3c60009260 comp rx=0 tx=0).stop 2024-04-24T06:51:55.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.330+0000 7f3c75600640 1 -- 172.21.15.5:0/4207208684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c70108ee0 msgr2=0x7f3c701a4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:55.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.330+0000 7f3c75600640 1 --2- 172.21.15.5:0/4207208684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c70108ee0 0x7f3c701a4e90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3c5800e9f0 tx=0x7f3c5800eeb0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.330+0000 7f3c75600640 1 -- 172.21.15.5:0/4207208684 shutdown_connections 2024-04-24T06:51:55.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.330+0000 7f3c75600640 1 --2- 172.21.15.5:0/4207208684 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f3c540a3da0 0x7f3c540a6260 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.330+0000 7f3c75600640 1 --2- 172.21.15.5:0/4207208684 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c7010e4f0 0x7f3c7019eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.330+0000 7f3c75600640 1 --2- 172.21.15.5:0/4207208684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c70108ee0 0x7f3c701a4e90 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.330+0000 7f3c75600640 1 --2- 172.21.15.5:0/4207208684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c70108540 0x7f3c701a4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.330+0000 7f3c75600640 1 -- 172.21.15.5:0/4207208684 >> 172.21.15.5:0/4207208684 conn(0x7f3c70077e10 msgr2=0x7f3c70078490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:55.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.330+0000 7f3c75600640 1 -- 172.21.15.5:0/4207208684 shutdown_connections 2024-04-24T06:51:55.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.330+0000 7f3c75600640 1 -- 172.21.15.5:0/4207208684 wait complete. 2024-04-24T06:51:55.351 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T06:51:55.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.565+0000 7f6644800640 1 -- 172.21.15.5:0/2155402266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f663c104450 msgr2=0x7f663c104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:55.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.565+0000 7f6644800640 1 --2- 172.21.15.5:0/2155402266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f663c104450 0x7f663c104830 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6634009ca0 tx=0x7f663402a8c0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.565+0000 7f6644800640 1 -- 172.21.15.5:0/2155402266 shutdown_connections 2024-04-24T06:51:55.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.565+0000 7f6644800640 1 --2- 172.21.15.5:0/2155402266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f663c109520 0x7f663c1110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.565+0000 7f6644800640 1 --2- 172.21.15.5:0/2155402266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f663c104e00 0x7f663c108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.565+0000 7f6644800640 1 --2- 172.21.15.5:0/2155402266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f663c104450 0x7f663c104830 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.565+0000 7f6644800640 1 -- 172.21.15.5:0/2155402266 >> 172.21.15.5:0/2155402266 conn(0x7f663c0ffc90 msgr2=0x7f663c1020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:55.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.565+0000 7f6644800640 1 -- 172.21.15.5:0/2155402266 shutdown_connections 2024-04-24T06:51:55.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.565+0000 7f6644800640 1 -- 172.21.15.5:0/2155402266 wait complete. 2024-04-24T06:51:55.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.566+0000 7f6644800640 1 Processor -- start 2024-04-24T06:51:55.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.566+0000 7f6644800640 1 -- start start 2024-04-24T06:51:55.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.566+0000 7f6644800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f663c104450 0x7f663c1a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.567+0000 7f6644800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f663c109520 0x7f663c1a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.567+0000 7f6644800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f663c19aa10 0x7f663c19aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.567+0000 7f6644800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f663c114630 con 0x7f663c109520 2024-04-24T06:51:55.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.567+0000 7f6644800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f663c1144b0 con 0x7f663c104450 2024-04-24T06:51:55.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.567+0000 7f6644800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f663c1147b0 con 0x7f663c19aa10 2024-04-24T06:51:55.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.567+0000 7f6641400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f663c109520 0x7f663c1a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.567+0000 7f6641400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f663c109520 0x7f663c1a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54454/0 (socket says 172.21.15.5:54454) 2024-04-24T06:51:55.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.567+0000 7f6641400640 1 -- 172.21.15.5:0/176867190 learned_addr learned my addr 172.21.15.5:0/176867190 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:55.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.567+0000 7f6641e00640 1 --2- 172.21.15.5:0/176867190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f663c104450 0x7f663c1a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.567+0000 7f6642800640 1 --2- 172.21.15.5:0/176867190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f663c19aa10 0x7f663c19aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.567+0000 7f6641400640 1 -- 172.21.15.5:0/176867190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f663c19aa10 msgr2=0x7f663c19aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:55.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.567+0000 7f6641400640 1 --2- 172.21.15.5:0/176867190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f663c19aa10 0x7f663c19aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.567+0000 7f6641400640 1 -- 172.21.15.5:0/176867190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f663c104450 msgr2=0x7f663c1a0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:55.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.567+0000 7f6641400640 1 --2- 172.21.15.5:0/176867190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f663c104450 0x7f663c1a0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.567+0000 7f6641400640 1 -- 172.21.15.5:0/176867190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f663c19b530 con 0x7f663c109520 2024-04-24T06:51:55.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.568+0000 7f6641400640 1 --2- 172.21.15.5:0/176867190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f663c109520 0x7f663c1a0d50 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f662c00e9a0 tx=0x7f662c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:55.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.568+0000 7f6632a00640 1 -- 172.21.15.5:0/176867190 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c00cd80 con 0x7f663c109520 2024-04-24T06:51:55.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.568+0000 7f6644800640 1 -- 172.21.15.5:0/176867190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f663c19b820 con 0x7f663c109520 2024-04-24T06:51:55.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.568+0000 7f6632a00640 1 -- 172.21.15.5:0/176867190 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f662c0035f0 con 0x7f663c109520 2024-04-24T06:51:55.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.568+0000 7f6632a00640 1 -- 172.21.15.5:0/176867190 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c010670 con 0x7f663c109520 2024-04-24T06:51:55.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.568+0000 7f6644800640 1 -- 172.21.15.5:0/176867190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f663c1a7a70 con 0x7f663c109520 2024-04-24T06:51:55.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.571+0000 7f66475d7740 1 Processor -- start 2024-04-24T06:51:55.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.572+0000 7f6632a00640 1 -- 172.21.15.5:0/176867190 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f662c0037a0 con 0x7f663c109520 2024-04-24T06:51:55.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.572+0000 7f66475d7740 1 -- start start 2024-04-24T06:51:55.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.572+0000 7f66475d7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564545c17950 0x564545c17d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.572+0000 7f66475d7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564545c18270 0x564545c22400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.572+0000 7f66475d7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564545c22940 0x564545c24d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.572+0000 7f66475d7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564545b7d1d0 con 0x564545c22940 2024-04-24T06:51:55.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.572+0000 7f66475d7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564545b23450 con 0x564545c17950 2024-04-24T06:51:55.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.572+0000 7f66475d7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564545b79190 con 0x564545c18270 2024-04-24T06:51:55.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.572+0000 7f6642800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564545c17950 0x564545c17d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.572+0000 7f6641400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564545c22940 0x564545c24d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.572+0000 7f6642800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564545c17950 0x564545c17d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:44580/0 (socket says 172.21.15.5:44580) 2024-04-24T06:51:55.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.572+0000 7f6642800640 1 -- 172.21.15.5:0/2372506576 learned_addr learned my addr 172.21.15.5:0/2372506576 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:55.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.572+0000 7f6641e00640 1 --2- 172.21.15.5:0/2372506576 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564545c18270 0x564545c22400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.573+0000 7f6642800640 1 -- 172.21.15.5:0/2372506576 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564545c18270 msgr2=0x564545c22400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:55.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.573+0000 7f6642800640 1 --2- 172.21.15.5:0/2372506576 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564545c18270 0x564545c22400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.573+0000 7f6642800640 1 -- 172.21.15.5:0/2372506576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564545c22940 msgr2=0x564545c24d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:55.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.573+0000 7f6642800640 1 --2- 172.21.15.5:0/2372506576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564545c22940 0x564545c24d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.573+0000 7f6642800640 1 -- 172.21.15.5:0/2372506576 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564545be0310 con 0x564545c17950 2024-04-24T06:51:55.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.573+0000 7f6642800640 1 --2- 172.21.15.5:0/2372506576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564545c17950 0x564545c17d30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6638009b90 tx=0x7f66380117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=552e07872d29e1a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:55.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.573+0000 7f6627000640 1 -- 172.21.15.5:0/2372506576 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638012450 con 0x564545c17950 2024-04-24T06:51:55.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.573+0000 7f6627000640 1 -- 172.21.15.5:0/2372506576 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6638012ad0 con 0x564545c17950 2024-04-24T06:51:55.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.573+0000 7f6627000640 1 -- 172.21.15.5:0/2372506576 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663800fab0 con 0x564545c17950 2024-04-24T06:51:55.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.573+0000 7f6632a00640 1 --2- 172.21.15.5:0/176867190 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f66280a3da0 0x7f66280a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.574+0000 7f6632a00640 1 -- 172.21.15.5:0/176867190 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f662c0d0b30 con 0x7f663c109520 2024-04-24T06:51:55.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.574+0000 7f6641e00640 1 --2- 172.21.15.5:0/176867190 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f66280a3da0 0x7f66280a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.574+0000 7f66475d7740 1 -- 172.21.15.5:0/2372506576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564545c17950 msgr2=0x564545c17d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:55.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.574+0000 7f66475d7740 1 --2- 172.21.15.5:0/2372506576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564545c17950 0x564545c17d30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6638009b90 tx=0x7f66380117b0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.574+0000 7f66475d7740 1 -- 172.21.15.5:0/2372506576 shutdown_connections 2024-04-24T06:51:55.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.574+0000 7f66475d7740 1 --2- 172.21.15.5:0/2372506576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564545c22940 0x564545c24d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.574+0000 7f66475d7740 1 --2- 172.21.15.5:0/2372506576 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564545c18270 0x564545c22400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.574+0000 7f66475d7740 1 --2- 172.21.15.5:0/2372506576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564545c17950 0x564545c17d30 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.574+0000 7f66475d7740 1 -- 172.21.15.5:0/2372506576 >> 172.21.15.5:0/2372506576 conn(0x564545c13c70 msgr2=0x564545c14050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:55.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.574+0000 7f66475d7740 1 -- 172.21.15.5:0/2372506576 shutdown_connections 2024-04-24T06:51:55.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.574+0000 7f66475d7740 1 -- 172.21.15.5:0/2372506576 wait complete. 2024-04-24T06:51:55.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.575+0000 7f6641e00640 1 --2- 172.21.15.5:0/176867190 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f66280a3da0 0x7f66280a6260 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f663c100090 tx=0x7f6634009800 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:55.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.575+0000 7f66475d7740 1 -- start start 2024-04-24T06:51:55.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.575+0000 7f66475d7740 1 Processor -- start 2024-04-24T06:51:55.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.576+0000 7f66475d7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564545c17950 0x564545c22300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.576+0000 7f66475d7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564545c18270 0x564545c1f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.576+0000 7f66475d7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564545c22940 0x564545c1fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.576+0000 7f66475d7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564545c257d0 con 0x564545c17950 2024-04-24T06:51:55.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.576+0000 7f66475d7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564545b36810 con 0x564545c18270 2024-04-24T06:51:55.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.576+0000 7f66475d7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564545c25950 con 0x564545c22940 2024-04-24T06:51:55.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.576+0000 7f66475d7740 10 client.0 _ll_register_callbacks cb 0x564545b04140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:51:55.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.576+0000 7f6642800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564545c22940 0x564545c1fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.576+0000 7f6641e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564545c17950 0x564545c22300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.576+0000 7f6642800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564545c22940 0x564545c1fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:41122/0 (socket says 172.21.15.5:41122) 2024-04-24T06:51:55.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.576+0000 7f6642800640 1 -- 172.21.15.5:0/810213555 learned_addr learned my addr 172.21.15.5:0/810213555 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:55.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.576+0000 7f6641400640 1 --2- 172.21.15.5:0/810213555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564545c18270 0x564545c1f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.576+0000 7f6642800640 1 -- 172.21.15.5:0/810213555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564545c18270 msgr2=0x564545c1f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:55.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.576+0000 7f6642800640 1 --2- 172.21.15.5:0/810213555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564545c18270 0x564545c1f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.576+0000 7f6642800640 1 -- 172.21.15.5:0/810213555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564545c17950 msgr2=0x564545c22300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:55.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.576+0000 7f6642800640 1 --2- 172.21.15.5:0/810213555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564545c17950 0x564545c22300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.577+0000 7f6642800640 1 -- 172.21.15.5:0/810213555 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564545b473f0 con 0x564545c22940 2024-04-24T06:51:55.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.577+0000 7f6641e00640 1 --2- 172.21.15.5:0/810213555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564545c17950 0x564545c22300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:51:55.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.577+0000 7f6642800640 1 --2- 172.21.15.5:0/810213555 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564545c22940 0x564545c1fc00 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6638009b60 tx=0x7f6638009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48409bcf72698931 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:55.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.577+0000 7f661b400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:51:55.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.577+0000 7f661b400640 1 -- 172.21.15.5:0/810213555 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638029020 con 0x564545c22940 2024-04-24T06:51:55.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.577+0000 7f66475d7740 1 -- 172.21.15.5:0/810213555 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6638012ad0 con 0x564545c22940 2024-04-24T06:51:55.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.577+0000 7f661b400640 1 -- 172.21.15.5:0/810213555 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f663800fa00 con 0x564545c22940 2024-04-24T06:51:55.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.578+0000 7f661b400640 1 -- 172.21.15.5:0/810213555 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6638034050 con 0x564545c22940 2024-04-24T06:51:55.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.578+0000 7f66475d7740 10 client.26977 fetch_fsmap learned FSMap version 27 2024-04-24T06:51:55.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.578+0000 7f66475d7740 1 -- 172.21.15.5:0/810213555 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56454588f3f0 con 0x564545c22940 2024-04-24T06:51:55.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.578+0000 7f661b400640 1 -- 172.21.15.5:0/810213555 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6638012ad0 con 0x564545c22940 2024-04-24T06:51:55.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.578+0000 7f661b400640 1 -- 172.21.15.5:0/810213555 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f66380046c0 con 0x564545c22940 2024-04-24T06:51:55.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.578+0000 7f66475d7740 10 client.26977 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:51:55.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.578+0000 7f66475d7740 10 client.26977 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:51:55.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.578+0000 7f66475d7740 20 client.26977 populate_metadata read hostname 'smithi005' 2024-04-24T06:51:55.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.578+0000 7f66475d7740 1 --2- 172.21.15.5:0/810213555 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x564545c32a90 0x564545c34e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.578+0000 7f66475d7740 4 client.26977 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:51:55.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.578+0000 7f66475d7740 1 -- 172.21.15.5:0/810213555 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564545b8abf0 con 0x564545c32a90 2024-04-24T06:51:55.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.579+0000 7f6641e00640 1 --2- 172.21.15.5:0/810213555 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x564545c32a90 0x564545c34e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.579+0000 7f6641e00640 1 --2- 172.21.15.5:0/810213555 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x564545c32a90 0x564545c34e80 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d5bc20250ceb22c server_cookie=294c9c62242c07fd in_seq=0 out_seq=0 2024-04-24T06:51:55.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.579+0000 7f661b400640 10 client.26977 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:51:55.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.580+0000 7f661b400640 1 -- 172.21.15.5:0/810213555 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564545b8abf0 con 0x564545c32a90 2024-04-24T06:51:55.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.580+0000 7f661b400640 10 client.26977 handle_command_reply: tid=0 2024-04-24T06:51:55.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.580+0000 7f66475d7740 1 client.26977 shutdown 2024-04-24T06:51:55.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.581+0000 7f66475d7740 1 -- 172.21.15.5:0/810213555 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564545c22940 msgr2=0x564545c1fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:55.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.581+0000 7f66475d7740 1 --2- 172.21.15.5:0/810213555 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564545c22940 0x564545c1fc00 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6638009b60 tx=0x7f6638009840 comp rx=0 tx=0).stop 2024-04-24T06:51:55.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.581+0000 7f66475d7740 1 -- 172.21.15.5:0/810213555 shutdown_connections 2024-04-24T06:51:55.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.581+0000 7f66475d7740 1 --2- 172.21.15.5:0/810213555 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x564545c32a90 0x564545c34e80 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.581+0000 7f66475d7740 1 --2- 172.21.15.5:0/810213555 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564545c22940 0x564545c1fc00 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.581+0000 7f66475d7740 1 --2- 172.21.15.5:0/810213555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564545c18270 0x564545c1f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.581+0000 7f661b400640 0 client.26977 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:51:55.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.581+0000 7f66475d7740 1 --2- 172.21.15.5:0/810213555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564545c17950 0x564545c22300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.581+0000 7f66475d7740 1 -- 172.21.15.5:0/810213555 >> 172.21.15.5:0/810213555 conn(0x564545c13c70 msgr2=0x564545c19970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:55.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.581+0000 7f66475d7740 1 -- 172.21.15.5:0/810213555 shutdown_connections 2024-04-24T06:51:55.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.581+0000 7f66475d7740 1 -- 172.21.15.5:0/810213555 wait complete. 2024-04-24T06:51:55.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.581+0000 7f66475d7740 20 client.26977 trim_cache size 0 max 16384 2024-04-24T06:51:55.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.596+0000 7f66475d7740 1 Processor -- start 2024-04-24T06:51:55.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.596+0000 7f66475d7740 1 -- start start 2024-04-24T06:51:55.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.596+0000 7f66475d7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564545c17950 0x564545c24c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.596+0000 7f66475d7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564545c18270 0x564545c13570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.596+0000 7f66475d7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564545c22940 0x564545c21e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.596+0000 7f66475d7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f663c1b2990 con 0x564545c17950 2024-04-24T06:51:55.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.596+0000 7f66475d7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f663c1b0260 con 0x564545c22940 2024-04-24T06:51:55.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.596+0000 7f66475d7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f663c1ae0d0 con 0x564545c18270 2024-04-24T06:51:55.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.597+0000 7f6641e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564545c17950 0x564545c24c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.597+0000 7f6641400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564545c18270 0x564545c13570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.597+0000 7f6641e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564545c17950 0x564545c24c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54478/0 (socket says 172.21.15.5:54478) 2024-04-24T06:51:55.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.597+0000 7f6641e00640 1 -- 172.21.15.5:0/3257631665 learned_addr learned my addr 172.21.15.5:0/3257631665 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:55.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.597+0000 7f6642800640 1 --2- 172.21.15.5:0/3257631665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564545c22940 0x564545c21e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.597+0000 7f6641e00640 1 -- 172.21.15.5:0/3257631665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564545c18270 msgr2=0x564545c13570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:55.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.597+0000 7f6641e00640 1 --2- 172.21.15.5:0/3257631665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564545c18270 0x564545c13570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.597+0000 7f6641e00640 1 -- 172.21.15.5:0/3257631665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564545c22940 msgr2=0x564545c21e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:55.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.597+0000 7f6641e00640 1 --2- 172.21.15.5:0/3257631665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564545c22940 0x564545c21e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.597+0000 7f6641e00640 1 -- 172.21.15.5:0/3257631665 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645459597d0 con 0x564545c17950 2024-04-24T06:51:55.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.597+0000 7f6642800640 1 --2- 172.21.15.5:0/3257631665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564545c22940 0x564545c21e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:51:55.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.597+0000 7f6641e00640 1 --2- 172.21.15.5:0/3257631665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564545c17950 0x564545c24c40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6634003ac0 tx=0x7f6634041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adf29f75efbe39ba server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:55.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.598+0000 7f6626600640 1 -- 172.21.15.5:0/3257631665 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663405c070 con 0x564545c17950 2024-04-24T06:51:55.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.598+0000 7f6626600640 1 -- 172.21.15.5:0/3257631665 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6634036050 con 0x564545c17950 2024-04-24T06:51:55.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.598+0000 7f6626600640 1 -- 172.21.15.5:0/3257631665 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6634056320 con 0x564545c17950 2024-04-24T06:51:55.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.598+0000 7f66475d7740 1 -- 172.21.15.5:0/3257631665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564545c17950 msgr2=0x564545c24c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:55.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.598+0000 7f66475d7740 1 --2- 172.21.15.5:0/3257631665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564545c17950 0x564545c24c40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6634003ac0 tx=0x7f6634041000 comp rx=0 tx=0).stop 2024-04-24T06:51:55.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.598+0000 7f66475d7740 1 -- 172.21.15.5:0/3257631665 shutdown_connections 2024-04-24T06:51:55.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.598+0000 7f66475d7740 1 --2- 172.21.15.5:0/3257631665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564545c22940 0x564545c21e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.598+0000 7f66475d7740 1 --2- 172.21.15.5:0/3257631665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564545c18270 0x564545c13570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.598+0000 7f66475d7740 1 --2- 172.21.15.5:0/3257631665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564545c17950 0x564545c24c40 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.599+0000 7f66475d7740 1 -- 172.21.15.5:0/3257631665 >> 172.21.15.5:0/3257631665 conn(0x564545c13c70 msgr2=0x564545c19930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:55.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.599+0000 7f66475d7740 1 -- 172.21.15.5:0/3257631665 shutdown_connections 2024-04-24T06:51:55.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.599+0000 7f66475d7740 1 -- 172.21.15.5:0/3257631665 wait complete. 2024-04-24T06:51:55.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.599+0000 7f66475d7740 1 -- start start 2024-04-24T06:51:55.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.599+0000 7f66475d7740 1 Processor -- start 2024-04-24T06:51:55.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.600+0000 7f66475d7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564545c17950 0x564545c2b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.600+0000 7f66475d7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564545c18270 0x564545c1c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.600+0000 7f66475d7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564545c22940 0x564545c1c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.600+0000 7f66475d7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f663c1ae720 con 0x564545c17950 2024-04-24T06:51:55.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.600+0000 7f66475d7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f663c1afb90 con 0x564545c22940 2024-04-24T06:51:55.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.600+0000 7f66475d7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f663c1aedf0 con 0x564545c18270 2024-04-24T06:51:55.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.600+0000 7f66475d7740 10 client.0 _ll_register_callbacks cb 0x7f662c0d44b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:51:55.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.600+0000 7f6641e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564545c17950 0x564545c2b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.600+0000 7f6641e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564545c17950 0x564545c2b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54494/0 (socket says 172.21.15.5:54494) 2024-04-24T06:51:55.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.600+0000 7f6641e00640 1 -- 172.21.15.5:0/1150686799 learned_addr learned my addr 172.21.15.5:0/1150686799 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:55.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.600+0000 7f6642800640 1 --2- 172.21.15.5:0/1150686799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564545c22940 0x564545c1c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.600+0000 7f6641400640 1 --2- 172.21.15.5:0/1150686799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564545c18270 0x564545c1c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.600+0000 7f6641e00640 1 -- 172.21.15.5:0/1150686799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564545c18270 msgr2=0x564545c1c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:55.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.600+0000 7f6641e00640 1 --2- 172.21.15.5:0/1150686799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564545c18270 0x564545c1c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.600+0000 7f6641e00640 1 -- 172.21.15.5:0/1150686799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564545c22940 msgr2=0x564545c1c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:55.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.600+0000 7f6641e00640 1 --2- 172.21.15.5:0/1150686799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564545c22940 0x564545c1c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.600+0000 7f6641e00640 1 -- 172.21.15.5:0/1150686799 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564545c1cde0 con 0x564545c17950 2024-04-24T06:51:55.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.600+0000 7f6642800640 1 --2- 172.21.15.5:0/1150686799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564545c22940 0x564545c1c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:51:55.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.601+0000 7f6641e00640 1 --2- 172.21.15.5:0/1150686799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564545c17950 0x564545c2b0d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6634004af0 tx=0x7f6634031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c4cdedfc446d47f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:55.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.601+0000 7f661b400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:51:55.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.601+0000 7f661b400640 1 -- 172.21.15.5:0/1150686799 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663405c040 con 0x564545c17950 2024-04-24T06:51:55.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.601+0000 7f66475d7740 1 -- 172.21.15.5:0/1150686799 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6634036050 con 0x564545c17950 2024-04-24T06:51:55.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.601+0000 7f661b400640 1 -- 172.21.15.5:0/1150686799 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f663402f610 con 0x564545c17950 2024-04-24T06:51:55.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.601+0000 7f661b400640 1 -- 172.21.15.5:0/1150686799 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f663402f810 con 0x564545c17950 2024-04-24T06:51:55.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.601+0000 7f66475d7740 10 client.16833 fetch_fsmap learned FSMap version 27 2024-04-24T06:51:55.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.601+0000 7f66475d7740 1 -- 172.21.15.5:0/1150686799 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564545c98db0 con 0x564545c17950 2024-04-24T06:51:55.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.602+0000 7f661b400640 1 -- 172.21.15.5:0/1150686799 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6634036050 con 0x564545c17950 2024-04-24T06:51:55.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.602+0000 7f661b400640 1 -- 172.21.15.5:0/1150686799 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f66340305c0 con 0x564545c17950 2024-04-24T06:51:55.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.602+0000 7f66475d7740 10 client.16833 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:51:55.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.602+0000 7f66475d7740 10 client.16833 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:51:55.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.602+0000 7f66475d7740 20 client.16833 populate_metadata read hostname 'smithi005' 2024-04-24T06:51:55.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.602+0000 7f66475d7740 1 --2- 172.21.15.5:0/1150686799 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x564545c994d0 0x564545c9b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:55.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.602+0000 7f66475d7740 4 client.16833 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:51:55.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.602+0000 7f66475d7740 1 -- 172.21.15.5:0/1150686799 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564545b84a60 con 0x564545c994d0 2024-04-24T06:51:55.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.602+0000 7f6641400640 1 --2- 172.21.15.5:0/1150686799 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x564545c994d0 0x564545c9b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:55.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.603+0000 7f6641400640 1 --2- 172.21.15.5:0/1150686799 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x564545c994d0 0x564545c9b8c0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b39e89240c0f1e6 server_cookie=c4fccc7bcdb9a8bf in_seq=0 out_seq=0 2024-04-24T06:51:55.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.603+0000 7f661b400640 10 client.16833 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:51:55.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.603+0000 7f661b400640 1 -- 172.21.15.5:0/1150686799 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564545b84a60 con 0x564545c994d0 2024-04-24T06:51:55.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.603+0000 7f661b400640 10 client.16833 handle_command_reply: tid=0 2024-04-24T06:51:55.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.603+0000 7f66475d7740 1 client.16833 shutdown 2024-04-24T06:51:55.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.603+0000 7f66475d7740 1 -- 172.21.15.5:0/1150686799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564545c17950 msgr2=0x564545c2b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:55.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.603+0000 7f66475d7740 1 --2- 172.21.15.5:0/1150686799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564545c17950 0x564545c2b0d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6634004af0 tx=0x7f6634031920 comp rx=0 tx=0).stop 2024-04-24T06:51:55.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.603+0000 7f66475d7740 1 -- 172.21.15.5:0/1150686799 shutdown_connections 2024-04-24T06:51:55.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.604+0000 7f66475d7740 1 --2- 172.21.15.5:0/1150686799 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x564545c994d0 0x564545c9b8c0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.604+0000 7f66475d7740 1 --2- 172.21.15.5:0/1150686799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564545c22940 0x564545c1c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.604+0000 7f66475d7740 1 --2- 172.21.15.5:0/1150686799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564545c18270 0x564545c1c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.604+0000 7f661b400640 0 client.16833 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:51:55.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.604+0000 7f66475d7740 1 --2- 172.21.15.5:0/1150686799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564545c17950 0x564545c2b0d0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.604+0000 7f66475d7740 1 -- 172.21.15.5:0/1150686799 >> 172.21.15.5:0/1150686799 conn(0x564545c13c70 msgr2=0x564545c1db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:55.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.604+0000 7f66475d7740 1 -- 172.21.15.5:0/1150686799 shutdown_connections 2024-04-24T06:51:55.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.604+0000 7f66475d7740 1 -- 172.21.15.5:0/1150686799 wait complete. 2024-04-24T06:51:55.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.604+0000 7f66475d7740 20 client.16833 trim_cache size 0 max 16384 2024-04-24T06:51:55.614 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:51:55.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.604+0000 7f6644800640 1 -- 172.21.15.5:0/176867190 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f66280a3da0 msgr2=0x7f66280a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:55.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.604+0000 7f6644800640 1 --2- 172.21.15.5:0/176867190 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f66280a3da0 0x7f66280a6260 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f663c100090 tx=0x7f6634009800 comp rx=0 tx=0).stop 2024-04-24T06:51:55.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.604+0000 7f6644800640 1 -- 172.21.15.5:0/176867190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f663c109520 msgr2=0x7f663c1a0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:55.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.604+0000 7f6644800640 1 --2- 172.21.15.5:0/176867190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f663c109520 0x7f663c1a0d50 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f662c00e9a0 tx=0x7f662c00ee60 comp rx=0 tx=0).stop 2024-04-24T06:51:55.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.605+0000 7f6644800640 1 -- 172.21.15.5:0/176867190 shutdown_connections 2024-04-24T06:51:55.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.605+0000 7f6644800640 1 --2- 172.21.15.5:0/176867190 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f66280a3da0 0x7f66280a6260 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.605+0000 7f6644800640 1 --2- 172.21.15.5:0/176867190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f663c19aa10 0x7f663c19aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.605+0000 7f6644800640 1 --2- 172.21.15.5:0/176867190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f663c109520 0x7f663c1a0d50 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.605+0000 7f6644800640 1 --2- 172.21.15.5:0/176867190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f663c104450 0x7f663c1a0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:55.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.605+0000 7f6644800640 1 -- 172.21.15.5:0/176867190 >> 172.21.15.5:0/176867190 conn(0x7f663c0ffc90 msgr2=0x7f663c101fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:55.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.605+0000 7f6644800640 1 -- 172.21.15.5:0/176867190 shutdown_connections 2024-04-24T06:51:55.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:55.605+0000 7f6644800640 1 -- 172.21.15.5:0/176867190 wait complete. 2024-04-24T06:51:55.621 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:55 smithi005 ceph-mon[27798]: pgmap v241: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:51:55.622 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:55 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/516575692' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:51:55.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:55 smithi137 ceph-mon[33017]: pgmap v241: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:51:55.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:55 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/516575692' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:51:55.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:55 smithi037 ceph-mon[32864]: pgmap v241: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:51:55.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:55 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/516575692' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:51:56.626 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T06:51:56.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:51:56.627 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T06:51:56.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.855+0000 7ff3f6200640 1 -- 172.21.15.5:0/1325100105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3f0104c10 msgr2=0x7ff3f0108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:56.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.855+0000 7ff3f6200640 1 --2- 172.21.15.5:0/1325100105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3f0104c10 0x7ff3f0108c50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc009900 tx=0x7ff3dc02a400 comp rx=0 tx=0).stop 2024-04-24T06:51:56.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.855+0000 7ff3f6200640 1 -- 172.21.15.5:0/1325100105 shutdown_connections 2024-04-24T06:51:56.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.855+0000 7ff3f6200640 1 --2- 172.21.15.5:0/1325100105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3f0109330 0x7ff3f0110f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:56.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.855+0000 7ff3f6200640 1 --2- 172.21.15.5:0/1325100105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3f0104c10 0x7ff3f0108c50 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:56.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.855+0000 7ff3f6200640 1 --2- 172.21.15.5:0/1325100105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3f0104260 0x7ff3f0104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:56.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.855+0000 7ff3f6200640 1 -- 172.21.15.5:0/1325100105 >> 172.21.15.5:0/1325100105 conn(0x7ff3f00ffaa0 msgr2=0x7ff3f0101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:56.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.856+0000 7ff3f6200640 1 -- 172.21.15.5:0/1325100105 shutdown_connections 2024-04-24T06:51:56.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.856+0000 7ff3f6200640 1 -- 172.21.15.5:0/1325100105 wait complete. 2024-04-24T06:51:56.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.857+0000 7ff3f6200640 1 Processor -- start 2024-04-24T06:51:56.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.857+0000 7ff3f6200640 1 -- start start 2024-04-24T06:51:56.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.857+0000 7ff3f6200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3f0104260 0x7ff3f01a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:56.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.857+0000 7ff3f6200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3f0109330 0x7ff3f01a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:56.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.857+0000 7ff3f6200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3f019a760 0x7ff3f019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:56.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.857+0000 7ff3f6200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f01143a0 con 0x7ff3f0104260 2024-04-24T06:51:56.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.857+0000 7ff3f6200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f0114220 con 0x7ff3f0109330 2024-04-24T06:51:56.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.857+0000 7ff3f6200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f0114520 con 0x7ff3f019a760 2024-04-24T06:51:56.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.857+0000 7ff3f4e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3f0104260 0x7ff3f01a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:56.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.857+0000 7ff3efe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3f0109330 0x7ff3f01a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:56.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.857+0000 7ff3f4e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3f0104260 0x7ff3f01a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54522/0 (socket says 172.21.15.5:54522) 2024-04-24T06:51:56.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.857+0000 7ff3f4e00640 1 -- 172.21.15.5:0/2674831392 learned_addr learned my addr 172.21.15.5:0/2674831392 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:56.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.858+0000 7ff3f5800640 1 --2- 172.21.15.5:0/2674831392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3f019a760 0x7ff3f019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:56.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.858+0000 7ff3f4e00640 1 -- 172.21.15.5:0/2674831392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3f019a760 msgr2=0x7ff3f019abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:56.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.858+0000 7ff3f4e00640 1 --2- 172.21.15.5:0/2674831392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3f019a760 0x7ff3f019abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:56.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.858+0000 7ff3f4e00640 1 -- 172.21.15.5:0/2674831392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3f0109330 msgr2=0x7ff3f01a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:56.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.858+0000 7ff3f4e00640 1 --2- 172.21.15.5:0/2674831392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3f0109330 0x7ff3f01a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:56.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.858+0000 7ff3f4e00640 1 -- 172.21.15.5:0/2674831392 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3f019b360 con 0x7ff3f0104260 2024-04-24T06:51:56.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.858+0000 7ff3efe00640 1 --2- 172.21.15.5:0/2674831392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3f0109330 0x7ff3f01a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:51:56.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.858+0000 7ff3f4e00640 1 --2- 172.21.15.5:0/2674831392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3f0104260 0x7ff3f01a0650 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff3e000e9f0 tx=0x7ff3e000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:56.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.859+0000 7ff3ed600640 1 -- 172.21.15.5:0/2674831392 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e000cd80 con 0x7ff3f0104260 2024-04-24T06:51:56.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.859+0000 7ff3ed600640 1 -- 172.21.15.5:0/2674831392 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3e0010910 con 0x7ff3f0104260 2024-04-24T06:51:56.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.859+0000 7ff3f6200640 1 -- 172.21.15.5:0/2674831392 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3f01a73f0 con 0x7ff3f0104260 2024-04-24T06:51:56.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.859+0000 7ff3ed600640 1 -- 172.21.15.5:0/2674831392 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e0018990 con 0x7ff3f0104260 2024-04-24T06:51:56.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.859+0000 7ff3f6200640 1 -- 172.21.15.5:0/2674831392 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3f01a77a0 con 0x7ff3f0104260 2024-04-24T06:51:56.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.862+0000 7ff3fa7a3740 1 Processor -- start 2024-04-24T06:51:56.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.862+0000 7ff3ed600640 1 -- 172.21.15.5:0/2674831392 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7ff3e0010ac0 con 0x7ff3f0104260 2024-04-24T06:51:56.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.862+0000 7ff3fa7a3740 1 -- start start 2024-04-24T06:51:56.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.863+0000 7ff3fa7a3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6a8aae270 0x55e6a8aae650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:56.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.863+0000 7ff3fa7a3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8aaeb90 0x55e6a8ab8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:56.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.863+0000 7ff3fa7a3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6a8ab9260 0x55e6a8abb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:56.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.863+0000 7ff3fa7a3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e6a8a13e70 con 0x55e6a8aae270 2024-04-24T06:51:56.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.863+0000 7ff3fa7a3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e6a89ba380 con 0x55e6a8ab9260 2024-04-24T06:51:56.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.863+0000 7ff3fa7a3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e6a8a0fe30 con 0x55e6a8aaeb90 2024-04-24T06:51:56.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.863+0000 7ff3efe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6a8ab9260 0x55e6a8abb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:56.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.863+0000 7ff3f5800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6a8aae270 0x55e6a8aae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:56.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.863+0000 7ff3f4e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8aaeb90 0x55e6a8ab8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:56.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.863+0000 7ff3efe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6a8ab9260 0x55e6a8abb650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:44638/0 (socket says 172.21.15.5:44638) 2024-04-24T06:51:56.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.863+0000 7ff3efe00640 1 -- 172.21.15.5:0/3093599238 learned_addr learned my addr 172.21.15.5:0/3093599238 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:56.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.863+0000 7ff3efe00640 1 -- 172.21.15.5:0/3093599238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8aaeb90 msgr2=0x55e6a8ab8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:56.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.863+0000 7ff3efe00640 1 --2- 172.21.15.5:0/3093599238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8aaeb90 0x55e6a8ab8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:56.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.863+0000 7ff3efe00640 1 -- 172.21.15.5:0/3093599238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6a8aae270 msgr2=0x55e6a8aae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:56.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.863+0000 7ff3efe00640 1 --2- 172.21.15.5:0/3093599238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6a8aae270 0x55e6a8aae650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:56.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.863+0000 7ff3efe00640 1 -- 172.21.15.5:0/3093599238 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6a87ee7e0 con 0x55e6a8ab9260 2024-04-24T06:51:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.864+0000 7ff3efe00640 1 --2- 172.21.15.5:0/3093599238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6a8ab9260 0x55e6a8abb650 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff3f006b570 tx=0x7ff3dc005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0cc7a391daa3833 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.864+0000 7ff3ed600640 1 --2- 172.21.15.5:0/2674831392 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ff3d40a3d20 0x7ff3d40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.864+0000 7ff3ed600640 1 -- 172.21.15.5:0/2674831392 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7ff3e00d0b90 con 0x7ff3f0104260 2024-04-24T06:51:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.864+0000 7ff3e6000640 1 -- 172.21.15.5:0/3093599238 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc035040 con 0x55e6a8ab9260 2024-04-24T06:51:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.864+0000 7ff3e6000640 1 -- 172.21.15.5:0/3093599238 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3dc031870 con 0x55e6a8ab9260 2024-04-24T06:51:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.864+0000 7ff3e6000640 1 -- 172.21.15.5:0/3093599238 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc0049c0 con 0x55e6a8ab9260 2024-04-24T06:51:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.864+0000 7ff3f5800640 1 --2- 172.21.15.5:0/2674831392 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ff3d40a3d20 0x7ff3d40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.865+0000 7ff3fa7a3740 1 -- 172.21.15.5:0/3093599238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6a8ab9260 msgr2=0x55e6a8abb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:56.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.865+0000 7ff3fa7a3740 1 --2- 172.21.15.5:0/3093599238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6a8ab9260 0x55e6a8abb650 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff3f006b570 tx=0x7ff3dc005be0 comp rx=0 tx=0).stop 2024-04-24T06:51:56.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.865+0000 7ff3f5800640 1 --2- 172.21.15.5:0/2674831392 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ff3d40a3d20 0x7ff3d40a61e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8005fd0 tx=0x7ff3e8009500 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:56.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.865+0000 7ff3fa7a3740 1 -- 172.21.15.5:0/3093599238 shutdown_connections 2024-04-24T06:51:56.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.865+0000 7ff3fa7a3740 1 --2- 172.21.15.5:0/3093599238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6a8ab9260 0x55e6a8abb650 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:56.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.865+0000 7ff3fa7a3740 1 --2- 172.21.15.5:0/3093599238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8aaeb90 0x55e6a8ab8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:56.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.865+0000 7ff3fa7a3740 1 --2- 172.21.15.5:0/3093599238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6a8aae270 0x55e6a8aae650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:56.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.865+0000 7ff3fa7a3740 1 -- 172.21.15.5:0/3093599238 >> 172.21.15.5:0/3093599238 conn(0x55e6a8aaa590 msgr2=0x55e6a8aaa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:56.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.865+0000 7ff3fa7a3740 1 -- 172.21.15.5:0/3093599238 shutdown_connections 2024-04-24T06:51:56.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.865+0000 7ff3fa7a3740 1 -- 172.21.15.5:0/3093599238 wait complete. 2024-04-24T06:51:56.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.866+0000 7ff3fa7a3740 1 -- start start 2024-04-24T06:51:56.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.866+0000 7ff3fa7a3740 1 Processor -- start 2024-04-24T06:51:56.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.867+0000 7ff3fa7a3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6a8aae270 0x55e6a8ab6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:56.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.867+0000 7ff3fa7a3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8aaeb90 0x55e6a8ab66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:56.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.867+0000 7ff3fa7a3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6a8ab9260 0x55e6a8ab6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:56.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.867+0000 7ff3fa7a3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e6a8abbff0 con 0x55e6a8ab9260 2024-04-24T06:51:56.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.867+0000 7ff3fa7a3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e6a89cd660 con 0x55e6a8aae270 2024-04-24T06:51:56.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.867+0000 7ff3fa7a3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e6a8abc170 con 0x55e6a8aaeb90 2024-04-24T06:51:56.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.867+0000 7ff3fa7a3740 10 client.0 _ll_register_callbacks cb 0x55e6a89bbaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:51:56.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.867+0000 7ff3f4e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6a8aae270 0x55e6a8ab6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:56.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.867+0000 7ff3f4e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6a8aae270 0x55e6a8ab6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:44644/0 (socket says 172.21.15.5:44644) 2024-04-24T06:51:56.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.867+0000 7ff3f5800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6a8ab9260 0x55e6a8ab6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:56.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.867+0000 7ff3f4e00640 1 -- 172.21.15.5:0/988032820 learned_addr learned my addr 172.21.15.5:0/988032820 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:56.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.867+0000 7ff3efe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8aaeb90 0x55e6a8ab66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:56.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.867+0000 7ff3f4e00640 1 -- 172.21.15.5:0/988032820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8aaeb90 msgr2=0x55e6a8ab66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:56.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.867+0000 7ff3f4e00640 1 --2- 172.21.15.5:0/988032820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8aaeb90 0x55e6a8ab66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:56.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.867+0000 7ff3f4e00640 1 -- 172.21.15.5:0/988032820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6a8ab9260 msgr2=0x55e6a8ab6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:56.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.867+0000 7ff3f4e00640 1 --2- 172.21.15.5:0/988032820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6a8ab9260 0x55e6a8ab6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:56.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.867+0000 7ff3f4e00640 1 -- 172.21.15.5:0/988032820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6a884ef70 con 0x55e6a8aae270 2024-04-24T06:51:56.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.868+0000 7ff3f4e00640 1 --2- 172.21.15.5:0/988032820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6a8aae270 0x55e6a8ab6160 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff3e000c6a0 tx=0x7ff3e0009770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fe117e9464ebe88 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:56.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.868+0000 7ff3daa00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:51:56.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.868+0000 7ff3daa00640 1 -- 172.21.15.5:0/988032820 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e00d5040 con 0x55e6a8aae270 2024-04-24T06:51:56.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.868+0000 7ff3daa00640 1 -- 172.21.15.5:0/988032820 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7ff3e0018b30 con 0x55e6a8aae270 2024-04-24T06:51:56.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.868+0000 7ff3fa7a3740 1 -- 172.21.15.5:0/988032820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3dc031870 con 0x55e6a8aae270 2024-04-24T06:51:56.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.868+0000 7ff3daa00640 1 -- 172.21.15.5:0/988032820 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff3e00ce6f0 con 0x55e6a8aae270 2024-04-24T06:51:56.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.868+0000 7ff3fa7a3740 10 client.26981 fetch_fsmap learned FSMap version 27 2024-04-24T06:51:56.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.868+0000 7ff3fa7a3740 1 -- 172.21.15.5:0/988032820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e6a86e6170 con 0x55e6a8aae270 2024-04-24T06:51:56.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.868+0000 7ff3daa00640 1 -- 172.21.15.5:0/988032820 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3dc031870 con 0x55e6a8aae270 2024-04-24T06:51:56.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.869+0000 7ff3daa00640 1 -- 172.21.15.5:0/988032820 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7ff3e009c050 con 0x55e6a8aae270 2024-04-24T06:51:56.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.869+0000 7ff3fa7a3740 10 client.26981 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:51:56.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.869+0000 7ff3fa7a3740 10 client.26981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:51:56.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.869+0000 7ff3fa7a3740 20 client.26981 populate_metadata read hostname 'smithi005' 2024-04-24T06:51:56.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.869+0000 7ff3fa7a3740 1 --2- 172.21.15.5:0/988032820 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55e6a8ac9560 0x55e6a8acb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:56.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.869+0000 7ff3fa7a3740 4 client.26981 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:51:56.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.869+0000 7ff3fa7a3740 1 -- 172.21.15.5:0/988032820 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6a8a21890 con 0x55e6a8ac9560 2024-04-24T06:51:56.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.869+0000 7ff3efe00640 1 --2- 172.21.15.5:0/988032820 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55e6a8ac9560 0x55e6a8acb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.870+0000 7ff3efe00640 1 --2- 172.21.15.5:0/988032820 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55e6a8ac9560 0x55e6a8acb990 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24887e6268215719 server_cookie=486373b9d0f8b0f8 in_seq=0 out_seq=0 2024-04-24T06:51:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.870+0000 7ff3daa00640 10 client.26981 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:51:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.871+0000 7ff3daa00640 1 -- 172.21.15.5:0/988032820 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e6a8a21890 con 0x55e6a8ac9560 2024-04-24T06:51:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.871+0000 7ff3daa00640 10 client.26981 handle_command_reply: tid=0 2024-04-24T06:51:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.871+0000 7ff3fa7a3740 1 client.26981 shutdown 2024-04-24T06:51:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.871+0000 7ff3fa7a3740 1 -- 172.21.15.5:0/988032820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6a8aae270 msgr2=0x55e6a8ab6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:56.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.871+0000 7ff3fa7a3740 1 --2- 172.21.15.5:0/988032820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6a8aae270 0x55e6a8ab6160 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff3e000c6a0 tx=0x7ff3e0009770 comp rx=0 tx=0).stop 2024-04-24T06:51:56.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.871+0000 7ff3fa7a3740 1 -- 172.21.15.5:0/988032820 shutdown_connections 2024-04-24T06:51:56.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.871+0000 7ff3fa7a3740 1 --2- 172.21.15.5:0/988032820 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55e6a8ac9560 0x55e6a8acb990 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:56.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.871+0000 7ff3fa7a3740 1 --2- 172.21.15.5:0/988032820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6a8ab9260 0x55e6a8ab6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:56.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.871+0000 7ff3fa7a3740 1 --2- 172.21.15.5:0/988032820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8aaeb90 0x55e6a8ab66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:56.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.871+0000 7ff3fa7a3740 1 --2- 172.21.15.5:0/988032820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6a8aae270 0x55e6a8ab6160 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:56.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.871+0000 7ff3fa7a3740 1 -- 172.21.15.5:0/988032820 >> 172.21.15.5:0/988032820 conn(0x55e6a8aaa590 msgr2=0x55e6a8ab0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:56.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.871+0000 7ff3daa00640 0 client.26981 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:51:56.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.871+0000 7ff3fa7a3740 1 -- 172.21.15.5:0/988032820 shutdown_connections 2024-04-24T06:51:56.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.871+0000 7ff3fa7a3740 1 -- 172.21.15.5:0/988032820 wait complete. 2024-04-24T06:51:56.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.871+0000 7ff3fa7a3740 20 client.26981 trim_cache size 0 max 16384 2024-04-24T06:51:56.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.891+0000 7ff3fa7a3740 1 Processor -- start 2024-04-24T06:51:56.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.891+0000 7ff3fa7a3740 1 -- start start 2024-04-24T06:51:56.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.892+0000 7ff3fa7a3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8aae270 0x55e6a8ac4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:56.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.892+0000 7ff3fa7a3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6a8aaeb90 0x55e6a8aa9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:56.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.892+0000 7ff3fa7a3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6a8ab9260 0x55e6a8ad1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:56.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.892+0000 7ff3fa7a3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f01b2620 con 0x55e6a8aaeb90 2024-04-24T06:51:56.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.892+0000 7ff3fa7a3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f01afef0 con 0x55e6a8ab9260 2024-04-24T06:51:56.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.892+0000 7ff3fa7a3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f01add60 con 0x55e6a8aae270 2024-04-24T06:51:56.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.892+0000 7ff3efe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6a8aaeb90 0x55e6a8aa9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:56.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.892+0000 7ff3efe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6a8aaeb90 0x55e6a8aa9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54556/0 (socket says 172.21.15.5:54556) 2024-04-24T06:51:56.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.892+0000 7ff3efe00640 1 -- 172.21.15.5:0/1159129957 learned_addr learned my addr 172.21.15.5:0/1159129957 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:56.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.892+0000 7ff3f4e00640 1 --2- 172.21.15.5:0/1159129957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8aae270 0x55e6a8ac4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:56.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.892+0000 7ff3f5800640 1 --2- 172.21.15.5:0/1159129957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6a8ab9260 0x55e6a8ad1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:56.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.893+0000 7ff3efe00640 1 -- 172.21.15.5:0/1159129957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8aae270 msgr2=0x55e6a8ac4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:56.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.893+0000 7ff3efe00640 1 --2- 172.21.15.5:0/1159129957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8aae270 0x55e6a8ac4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:56.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.893+0000 7ff3efe00640 1 -- 172.21.15.5:0/1159129957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6a8ab9260 msgr2=0x55e6a8ad1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:56.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.893+0000 7ff3efe00640 1 --2- 172.21.15.5:0/1159129957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6a8ab9260 0x55e6a8ad1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:56.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.893+0000 7ff3efe00640 1 -- 172.21.15.5:0/1159129957 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6a8aa5e80 con 0x55e6a8aaeb90 2024-04-24T06:51:56.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.893+0000 7ff3f4e00640 1 --2- 172.21.15.5:0/1159129957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8aae270 0x55e6a8ac4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:51:56.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.893+0000 7ff3efe00640 1 --2- 172.21.15.5:0/1159129957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6a8aaeb90 0x55e6a8aa9570 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc0096c0 tx=0x7ff3dc0054d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d894dbce4785a3d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:56.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.893+0000 7ff3e5600640 1 -- 172.21.15.5:0/1159129957 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc057070 con 0x55e6a8aaeb90 2024-04-24T06:51:56.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.893+0000 7ff3e5600640 1 -- 172.21.15.5:0/1159129957 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3dc052890 con 0x55e6a8aaeb90 2024-04-24T06:51:56.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.893+0000 7ff3e5600640 1 -- 172.21.15.5:0/1159129957 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc035040 con 0x55e6a8aaeb90 2024-04-24T06:51:56.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.894+0000 7ff3fa7a3740 1 -- 172.21.15.5:0/1159129957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6a8aaeb90 msgr2=0x55e6a8aa9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:56.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.894+0000 7ff3fa7a3740 1 --2- 172.21.15.5:0/1159129957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6a8aaeb90 0x55e6a8aa9570 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc0096c0 tx=0x7ff3dc0054d0 comp rx=0 tx=0).stop 2024-04-24T06:51:56.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.894+0000 7ff3fa7a3740 1 -- 172.21.15.5:0/1159129957 shutdown_connections 2024-04-24T06:51:56.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.894+0000 7ff3fa7a3740 1 --2- 172.21.15.5:0/1159129957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6a8ab9260 0x55e6a8ad1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:56.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.894+0000 7ff3fa7a3740 1 --2- 172.21.15.5:0/1159129957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6a8aaeb90 0x55e6a8aa9570 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:56.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.894+0000 7ff3fa7a3740 1 --2- 172.21.15.5:0/1159129957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8aae270 0x55e6a8ac4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:56.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.894+0000 7ff3fa7a3740 1 -- 172.21.15.5:0/1159129957 >> 172.21.15.5:0/1159129957 conn(0x55e6a8aaa590 msgr2=0x55e6a8aa8e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:56.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.894+0000 7ff3fa7a3740 1 -- 172.21.15.5:0/1159129957 shutdown_connections 2024-04-24T06:51:56.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.894+0000 7ff3fa7a3740 1 -- 172.21.15.5:0/1159129957 wait complete. 2024-04-24T06:51:56.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.895+0000 7ff3fa7a3740 1 -- start start 2024-04-24T06:51:56.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.895+0000 7ff3fa7a3740 1 Processor -- start 2024-04-24T06:51:56.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.895+0000 7ff3fa7a3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6a8aae270 0x55e6a8abce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:56.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.895+0000 7ff3fa7a3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8aaeb90 0x55e6a8abd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:56.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.895+0000 7ff3fa7a3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6a8ab9260 0x55e6a8abd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:56.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.895+0000 7ff3fa7a3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f01ae3b0 con 0x55e6a8aae270 2024-04-24T06:51:56.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.895+0000 7ff3fa7a3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f01af820 con 0x55e6a8ab9260 2024-04-24T06:51:56.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.895+0000 7ff3fa7a3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f01aea80 con 0x55e6a8aaeb90 2024-04-24T06:51:56.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.895+0000 7ff3fa7a3740 10 client.0 _ll_register_callbacks cb 0x7ff3e000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:51:56.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.896+0000 7ff3f4e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6a8aae270 0x55e6a8abce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:56.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.896+0000 7ff3f4e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6a8aae270 0x55e6a8abce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54564/0 (socket says 172.21.15.5:54564) 2024-04-24T06:51:56.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.896+0000 7ff3f4e00640 1 -- 172.21.15.5:0/244146068 learned_addr learned my addr 172.21.15.5:0/244146068 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:56.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.896+0000 7ff3efe00640 1 --2- 172.21.15.5:0/244146068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8aaeb90 0x55e6a8abd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:56.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.896+0000 7ff3f5800640 1 --2- 172.21.15.5:0/244146068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6a8ab9260 0x55e6a8abd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:56.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.896+0000 7ff3efe00640 1 -- 172.21.15.5:0/244146068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6a8ab9260 msgr2=0x55e6a8abd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:56.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.896+0000 7ff3efe00640 1 --2- 172.21.15.5:0/244146068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6a8ab9260 0x55e6a8abd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:56.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.896+0000 7ff3efe00640 1 -- 172.21.15.5:0/244146068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6a8aae270 msgr2=0x55e6a8abce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:56.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.896+0000 7ff3efe00640 1 --2- 172.21.15.5:0/244146068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6a8aae270 0x55e6a8abce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:56.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.896+0000 7ff3efe00640 1 -- 172.21.15.5:0/244146068 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6a8b372e0 con 0x55e6a8aaeb90 2024-04-24T06:51:56.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.896+0000 7ff3f4e00640 1 --2- 172.21.15.5:0/244146068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6a8aae270 0x55e6a8abce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:51:56.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.896+0000 7ff3efe00640 1 --2- 172.21.15.5:0/244146068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8aaeb90 0x55e6a8abd350 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x55e6a8ab71a0 tx=0x7ff3dc02ab00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66a356fb40bb4714 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:56.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.897+0000 7ff3daa00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:51:56.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.897+0000 7ff3daa00640 1 -- 172.21.15.5:0/244146068 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc057070 con 0x55e6a8aaeb90 2024-04-24T06:51:56.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.897+0000 7ff3fa7a3740 1 -- 172.21.15.5:0/244146068 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3dc052890 con 0x55e6a8aaeb90 2024-04-24T06:51:56.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.897+0000 7ff3daa00640 1 -- 172.21.15.5:0/244146068 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7ff3dc004020 con 0x55e6a8aaeb90 2024-04-24T06:51:56.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.897+0000 7ff3daa00640 1 -- 172.21.15.5:0/244146068 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff3dc035050 con 0x55e6a8aaeb90 2024-04-24T06:51:56.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.897+0000 7ff3fa7a3740 10 client.27001 fetch_fsmap learned FSMap version 27 2024-04-24T06:51:56.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.897+0000 7ff3fa7a3740 1 -- 172.21.15.5:0/244146068 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e6a8b38600 con 0x55e6a8aaeb90 2024-04-24T06:51:56.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.897+0000 7ff3daa00640 1 -- 172.21.15.5:0/244146068 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3dc052890 con 0x55e6a8aaeb90 2024-04-24T06:51:56.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.898+0000 7ff3daa00640 1 -- 172.21.15.5:0/244146068 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7ff3dc0038f0 con 0x55e6a8aaeb90 2024-04-24T06:51:56.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.898+0000 7ff3fa7a3740 10 client.27001 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:51:56.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.898+0000 7ff3fa7a3740 10 client.27001 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:51:56.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.898+0000 7ff3fa7a3740 20 client.27001 populate_metadata read hostname 'smithi005' 2024-04-24T06:51:56.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.898+0000 7ff3fa7a3740 1 --2- 172.21.15.5:0/244146068 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55e6a8b38950 0x55e6a8b3ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:56.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.898+0000 7ff3fa7a3740 4 client.27001 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T06:51:56.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.898+0000 7ff3fa7a3740 1 -- 172.21.15.5:0/244146068 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e6a8a1b700 con 0x55e6a8b38950 2024-04-24T06:51:56.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.898+0000 7ff3f4e00640 1 --2- 172.21.15.5:0/244146068 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55e6a8b38950 0x55e6a8b3ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:56.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.899+0000 7ff3f4e00640 1 --2- 172.21.15.5:0/244146068 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55e6a8b38950 0x55e6a8b3ad40 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94394ce22aea42e9 server_cookie=efca0aa533dd4acb in_seq=0 out_seq=0 2024-04-24T06:51:56.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.899+0000 7ff3daa00640 10 client.27001 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:51:56.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.900+0000 7ff3daa00640 1 -- 172.21.15.5:0/244146068 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e6a8a1b700 con 0x55e6a8b38950 2024-04-24T06:51:56.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.900+0000 7ff3daa00640 10 client.27001 handle_command_reply: tid=0 2024-04-24T06:51:56.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.900+0000 7ff3fa7a3740 1 client.27001 shutdown 2024-04-24T06:51:56.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.900+0000 7ff3fa7a3740 1 -- 172.21.15.5:0/244146068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8aaeb90 msgr2=0x55e6a8abd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:56.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.900+0000 7ff3fa7a3740 1 --2- 172.21.15.5:0/244146068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8aaeb90 0x55e6a8abd350 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x55e6a8ab71a0 tx=0x7ff3dc02ab00 comp rx=0 tx=0).stop 2024-04-24T06:51:56.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.900+0000 7ff3fa7a3740 1 -- 172.21.15.5:0/244146068 shutdown_connections 2024-04-24T06:51:56.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.900+0000 7ff3fa7a3740 1 --2- 172.21.15.5:0/244146068 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55e6a8b38950 0x55e6a8b3ad40 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:56.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.900+0000 7ff3fa7a3740 1 --2- 172.21.15.5:0/244146068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6a8ab9260 0x55e6a8abd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:56.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.900+0000 7ff3daa00640 0 client.27001 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:51:56.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.900+0000 7ff3fa7a3740 1 --2- 172.21.15.5:0/244146068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8aaeb90 0x55e6a8abd350 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:56.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.900+0000 7ff3fa7a3740 1 --2- 172.21.15.5:0/244146068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6a8aae270 0x55e6a8abce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:56.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.900+0000 7ff3fa7a3740 1 -- 172.21.15.5:0/244146068 >> 172.21.15.5:0/244146068 conn(0x55e6a8aaa590 msgr2=0x55e6a8ab69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:56.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.900+0000 7ff3fa7a3740 1 -- 172.21.15.5:0/244146068 shutdown_connections 2024-04-24T06:51:56.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.900+0000 7ff3fa7a3740 1 -- 172.21.15.5:0/244146068 wait complete. 2024-04-24T06:51:56.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.901+0000 7ff3fa7a3740 20 client.27001 trim_cache size 0 max 16384 2024-04-24T06:51:56.909 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:51:56.909 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-24T06:51:56.909 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "8c06531e-b9e6-488e-a85a-f7030333b8e2", 2024-04-24T06:51:56.909 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-24T06:51:56.909 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:51:56.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.901+0000 7ff3f6200640 1 -- 172.21.15.5:0/2674831392 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ff3d40a3d20 msgr2=0x7ff3d40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:56.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.901+0000 7ff3f6200640 1 --2- 172.21.15.5:0/2674831392 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ff3d40a3d20 0x7ff3d40a61e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8005fd0 tx=0x7ff3e8009500 comp rx=0 tx=0).stop 2024-04-24T06:51:56.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.901+0000 7ff3f6200640 1 -- 172.21.15.5:0/2674831392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3f0104260 msgr2=0x7ff3f01a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:56.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.901+0000 7ff3f6200640 1 --2- 172.21.15.5:0/2674831392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3f0104260 0x7ff3f01a0650 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff3e000e9f0 tx=0x7ff3e000eeb0 comp rx=0 tx=0).stop 2024-04-24T06:51:56.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.901+0000 7ff3f6200640 1 -- 172.21.15.5:0/2674831392 shutdown_connections 2024-04-24T06:51:56.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.901+0000 7ff3f6200640 1 --2- 172.21.15.5:0/2674831392 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ff3d40a3d20 0x7ff3d40a61e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:56.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.901+0000 7ff3f6200640 1 --2- 172.21.15.5:0/2674831392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3f019a760 0x7ff3f019abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:56.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.901+0000 7ff3f6200640 1 --2- 172.21.15.5:0/2674831392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3f0109330 0x7ff3f01a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:56.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.901+0000 7ff3f6200640 1 --2- 172.21.15.5:0/2674831392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3f0104260 0x7ff3f01a0650 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:56.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.901+0000 7ff3f6200640 1 -- 172.21.15.5:0/2674831392 >> 172.21.15.5:0/2674831392 conn(0x7ff3f00ffaa0 msgr2=0x7ff3f0100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:56.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.902+0000 7ff3f6200640 1 -- 172.21.15.5:0/2674831392 shutdown_connections 2024-04-24T06:51:56.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:56.902+0000 7ff3f6200640 1 -- 172.21.15.5:0/2674831392 wait complete. 2024-04-24T06:51:56.926 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T06:51:56.926 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T06:51:57.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.152+0000 7fe209a00640 1 -- 172.21.15.5:0/2427159415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe204104280 msgr2=0x7fe204104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:57.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.152+0000 7fe209a00640 1 --2- 172.21.15.5:0/2427159415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe204104280 0x7fe204104660 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4009900 tx=0x7fe1f402a400 comp rx=0 tx=0).stop 2024-04-24T06:51:57.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.152+0000 7fe209a00640 1 -- 172.21.15.5:0/2427159415 shutdown_connections 2024-04-24T06:51:57.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.152+0000 7fe209a00640 1 --2- 172.21.15.5:0/2427159415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe204109350 0x7fe204110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:57.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.152+0000 7fe209a00640 1 --2- 172.21.15.5:0/2427159415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe204104c30 0x7fe204108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:57.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.152+0000 7fe209a00640 1 --2- 172.21.15.5:0/2427159415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe204104280 0x7fe204104660 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:57.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.152+0000 7fe209a00640 1 -- 172.21.15.5:0/2427159415 >> 172.21.15.5:0/2427159415 conn(0x7fe2040ffae0 msgr2=0x7fe204101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:57.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.152+0000 7fe209a00640 1 -- 172.21.15.5:0/2427159415 shutdown_connections 2024-04-24T06:51:57.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.152+0000 7fe209a00640 1 -- 172.21.15.5:0/2427159415 wait complete. 2024-04-24T06:51:57.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.153+0000 7fe209a00640 1 Processor -- start 2024-04-24T06:51:57.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.153+0000 7fe209a00640 1 -- start start 2024-04-24T06:51:57.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.154+0000 7fe209a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe204104280 0x7fe204110e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:57.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.154+0000 7fe209a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe204109350 0x7fe20410df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:57.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.154+0000 7fe209a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe20410e480 0x7fe20410e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:57.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.154+0000 7fe209a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe204114430 con 0x7fe204104280 2024-04-24T06:51:57.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.154+0000 7fe209a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe2041142b0 con 0x7fe20410e480 2024-04-24T06:51:57.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.154+0000 7fe209a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe2041145b0 con 0x7fe204109350 2024-04-24T06:51:57.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.154+0000 7fe203e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe204104280 0x7fe204110e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:57.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.154+0000 7fe1fbe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe204109350 0x7fe20410df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:57.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.154+0000 7fe209000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe20410e480 0x7fe20410e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:57.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.154+0000 7fe203e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe204104280 0x7fe204110e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54584/0 (socket says 172.21.15.5:54584) 2024-04-24T06:51:57.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.154+0000 7fe203e00640 1 -- 172.21.15.5:0/598230365 learned_addr learned my addr 172.21.15.5:0/598230365 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:57.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.155+0000 7fe1fbe00640 1 -- 172.21.15.5:0/598230365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe20410e480 msgr2=0x7fe20410e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:57.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.155+0000 7fe1fbe00640 1 --2- 172.21.15.5:0/598230365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe20410e480 0x7fe20410e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:57.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.155+0000 7fe1fbe00640 1 -- 172.21.15.5:0/598230365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe204104280 msgr2=0x7fe204110e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:57.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.155+0000 7fe1fbe00640 1 --2- 172.21.15.5:0/598230365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe204104280 0x7fe204110e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:57.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.155+0000 7fe1fbe00640 1 -- 172.21.15.5:0/598230365 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2041ab3b0 con 0x7fe204109350 2024-04-24T06:51:57.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.155+0000 7fe1fbe00640 1 --2- 172.21.15.5:0/598230365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe204109350 0x7fe20410df40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec00e9c0 tx=0x7fe1ec00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:57.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.155+0000 7fe201600640 1 -- 172.21.15.5:0/598230365 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec00ccc0 con 0x7fe204109350 2024-04-24T06:51:57.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.155+0000 7fe201600640 1 -- 172.21.15.5:0/598230365 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1ec010910 con 0x7fe204109350 2024-04-24T06:51:57.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.155+0000 7fe209a00640 1 -- 172.21.15.5:0/598230365 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2041ab6a0 con 0x7fe204109350 2024-04-24T06:51:57.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.155+0000 7fe201600640 1 -- 172.21.15.5:0/598230365 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec0188c0 con 0x7fe204109350 2024-04-24T06:51:57.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.156+0000 7fe209a00640 1 -- 172.21.15.5:0/598230365 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2041abc40 con 0x7fe204109350 2024-04-24T06:51:57.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.160+0000 7fe20de20740 1 Processor -- start 2024-04-24T06:51:57.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.160+0000 7fe201600640 1 -- 172.21.15.5:0/598230365 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fe1ec010ac0 con 0x7fe204109350 2024-04-24T06:51:57.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.160+0000 7fe20de20740 1 -- start start 2024-04-24T06:51:57.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.161+0000 7fe20de20740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f8c2dba30 0x559f8c2dbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:57.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.161+0000 7fe20de20740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8c2dc350 0x559f8c2e64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:57.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.161+0000 7fe20de20740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8c2e6a20 0x559f8c2e8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:57.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.161+0000 7fe20de20740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559f8c2411f0 con 0x559f8c2e6a20 2024-04-24T06:51:57.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.161+0000 7fe20de20740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559f8c1e7350 con 0x559f8c2dba30 2024-04-24T06:51:57.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.161+0000 7fe20de20740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559f8c23d1b0 con 0x559f8c2dc350 2024-04-24T06:51:57.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.161+0000 7fe203e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8c2dc350 0x559f8c2e64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:57.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.161+0000 7fe209000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f8c2dba30 0x559f8c2dbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:57.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.161+0000 7fe203e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8c2dc350 0x559f8c2e64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:41236/0 (socket says 172.21.15.5:41236) 2024-04-24T06:51:57.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.161+0000 7fe203e00640 1 -- 172.21.15.5:0/3355095841 learned_addr learned my addr 172.21.15.5:0/3355095841 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:57.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.161+0000 7fe209000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f8c2dba30 0x559f8c2dbe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:44686/0 (socket says 172.21.15.5:44686) 2024-04-24T06:51:57.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.161+0000 7fe1fbe00640 1 --2- 172.21.15.5:0/3355095841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8c2e6a20 0x559f8c2e8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:57.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.161+0000 7fe209000640 1 -- 172.21.15.5:0/3355095841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8c2dc350 msgr2=0x559f8c2e64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:57.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.161+0000 7fe209000640 1 --2- 172.21.15.5:0/3355095841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8c2dc350 0x559f8c2e64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:57.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.161+0000 7fe209000640 1 -- 172.21.15.5:0/3355095841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8c2e6a20 msgr2=0x559f8c2e8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:57.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.162+0000 7fe209000640 1 --2- 172.21.15.5:0/3355095841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8c2e6a20 0x559f8c2e8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:57.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.162+0000 7fe209000640 1 -- 172.21.15.5:0/3355095841 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f8c23bf50 con 0x559f8c2dba30 2024-04-24T06:51:57.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.162+0000 7fe1fbe00640 1 --2- 172.21.15.5:0/3355095841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8c2e6a20 0x559f8c2e8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:51:57.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.162+0000 7fe209000640 1 --2- 172.21.15.5:0/3355095841 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f8c2dba30 0x559f8c2dbe10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc009eb0 tx=0x7fe1fc010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58e8d523d8606a34 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:57.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.162+0000 7fe1f9600640 1 -- 172.21.15.5:0/3355095841 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc016070 con 0x559f8c2dba30 2024-04-24T06:51:57.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.162+0000 7fe1f9600640 1 -- 172.21.15.5:0/3355095841 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1fc004d20 con 0x559f8c2dba30 2024-04-24T06:51:57.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.162+0000 7fe1f9600640 1 -- 172.21.15.5:0/3355095841 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc012850 con 0x559f8c2dba30 2024-04-24T06:51:57.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.163+0000 7fe201600640 1 --2- 172.21.15.5:0/598230365 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fe1e80a3da0 0x7fe1e80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:57.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.163+0000 7fe201600640 1 -- 172.21.15.5:0/598230365 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7fe1ec0d0b60 con 0x7fe204109350 2024-04-24T06:51:57.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.163+0000 7fe203e00640 1 --2- 172.21.15.5:0/598230365 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fe1e80a3da0 0x7fe1e80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:57.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.163+0000 7fe20de20740 1 -- 172.21.15.5:0/3355095841 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f8c2dba30 msgr2=0x559f8c2dbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:57.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.163+0000 7fe20de20740 1 --2- 172.21.15.5:0/3355095841 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f8c2dba30 0x559f8c2dbe10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc009eb0 tx=0x7fe1fc010740 comp rx=0 tx=0).stop 2024-04-24T06:51:57.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.164+0000 7fe20de20740 1 -- 172.21.15.5:0/3355095841 shutdown_connections 2024-04-24T06:51:57.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.164+0000 7fe20de20740 1 --2- 172.21.15.5:0/3355095841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8c2e6a20 0x559f8c2e8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:57.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.164+0000 7fe20de20740 1 --2- 172.21.15.5:0/3355095841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8c2dc350 0x559f8c2e64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:57.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.164+0000 7fe20de20740 1 --2- 172.21.15.5:0/3355095841 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f8c2dba30 0x559f8c2dbe10 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:57.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.164+0000 7fe20de20740 1 -- 172.21.15.5:0/3355095841 >> 172.21.15.5:0/3355095841 conn(0x559f8c2d7d50 msgr2=0x559f8c2d8130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:57.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.164+0000 7fe20de20740 1 -- 172.21.15.5:0/3355095841 shutdown_connections 2024-04-24T06:51:57.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.164+0000 7fe20de20740 1 -- 172.21.15.5:0/3355095841 wait complete. 2024-04-24T06:51:57.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.164+0000 7fe203e00640 1 --2- 172.21.15.5:0/598230365 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fe1e80a3da0 0x7fe1e80a6260 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe1f40096c0 tx=0x7fe1f40046c0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:57.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.165+0000 7fe20de20740 1 -- start start 2024-04-24T06:51:57.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.165+0000 7fe20de20740 1 Processor -- start 2024-04-24T06:51:57.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.166+0000 7fe20de20740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8c2dba30 0x559f8c2e3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:57.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.166+0000 7fe20de20740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f8c2dc350 0x559f8c2e3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:57.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.166+0000 7fe20de20740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8c2e6a20 0x559f8c2e44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:57.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.166+0000 7fe20de20740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559f8c2e9810 con 0x559f8c2e6a20 2024-04-24T06:51:57.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.166+0000 7fe20de20740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559f8c1fa690 con 0x559f8c2dc350 2024-04-24T06:51:57.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.166+0000 7fe20de20740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559f8c2e9990 con 0x559f8c2dba30 2024-04-24T06:51:57.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.166+0000 7fe20de20740 10 client.0 _ll_register_callbacks cb 0x559f8c236650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:51:57.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.166+0000 7fe203e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8c2dba30 0x559f8c2e3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:57.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.166+0000 7fe203e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8c2dba30 0x559f8c2e3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:41242/0 (socket says 172.21.15.5:41242) 2024-04-24T06:51:57.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.166+0000 7fe203e00640 1 -- 172.21.15.5:0/3423721137 learned_addr learned my addr 172.21.15.5:0/3423721137 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:57.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.166+0000 7fe1fbe00640 1 --2- 172.21.15.5:0/3423721137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f8c2dc350 0x559f8c2e3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:57.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.166+0000 7fe203e00640 1 -- 172.21.15.5:0/3423721137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f8c2dc350 msgr2=0x559f8c2e3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:57.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.166+0000 7fe203e00640 1 --2- 172.21.15.5:0/3423721137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f8c2dc350 0x559f8c2e3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:57.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.166+0000 7fe203e00640 1 -- 172.21.15.5:0/3423721137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8c2e6a20 msgr2=0x559f8c2e44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:57.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.166+0000 7fe203e00640 1 --2- 172.21.15.5:0/3423721137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8c2e6a20 0x559f8c2e44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:57.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.166+0000 7fe203e00640 1 -- 172.21.15.5:0/3423721137 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f8c1f9920 con 0x559f8c2dba30 2024-04-24T06:51:57.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.167+0000 7fe203e00640 1 --2- 172.21.15.5:0/3423721137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8c2dba30 0x559f8c2e3a30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4031600 tx=0x7fe1f4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99d915eb8efa41ed server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:57.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.167+0000 7fe1f2000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:51:57.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.167+0000 7fe1f2000640 1 -- 172.21.15.5:0/3423721137 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f404e070 con 0x559f8c2dba30 2024-04-24T06:51:57.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.167+0000 7fe1f2000640 1 -- 172.21.15.5:0/3423721137 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7fe1f40037f0 con 0x559f8c2dba30 2024-04-24T06:51:57.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.167+0000 7fe20de20740 1 -- 172.21.15.5:0/3423721137 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1fc004d20 con 0x559f8c2dba30 2024-04-24T06:51:57.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.167+0000 7fe1f2000640 1 -- 172.21.15.5:0/3423721137 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe1f405a050 con 0x559f8c2dba30 2024-04-24T06:51:57.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.167+0000 7fe20de20740 10 client.27013 fetch_fsmap learned FSMap version 27 2024-04-24T06:51:57.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.167+0000 7fe20de20740 1 -- 172.21.15.5:0/3423721137 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559f8c01d7d0 con 0x559f8c2dba30 2024-04-24T06:51:57.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.168+0000 7fe1f2000640 1 -- 172.21.15.5:0/3423721137 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1fc004d20 con 0x559f8c2dba30 2024-04-24T06:51:57.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.168+0000 7fe1f2000640 1 -- 172.21.15.5:0/3423721137 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7fe1f404ac70 con 0x559f8c2dba30 2024-04-24T06:51:57.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.168+0000 7fe20de20740 10 client.27013 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:51:57.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.168+0000 7fe20de20740 10 client.27013 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:51:57.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.168+0000 7fe20de20740 20 client.27013 populate_metadata read hostname 'smithi005' 2024-04-24T06:51:57.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.168+0000 7fe20de20740 1 --2- 172.21.15.5:0/3423721137 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x559f8c2f6d40 0x559f8c2f91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:57.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.168+0000 7fe20de20740 4 client.27013 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:51:57.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.168+0000 7fe20de20740 1 -- 172.21.15.5:0/3423721137 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f8c24ec10 con 0x559f8c2f6d40 2024-04-24T06:51:57.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.169+0000 7fe1fbe00640 1 --2- 172.21.15.5:0/3423721137 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x559f8c2f6d40 0x559f8c2f91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:57.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.169+0000 7fe1fbe00640 1 --2- 172.21.15.5:0/3423721137 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x559f8c2f6d40 0x559f8c2f91d0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=714c4bf3caed8cfe server_cookie=c69c44d8e06a0c4b in_seq=0 out_seq=0 2024-04-24T06:51:57.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.169+0000 7fe1f2000640 10 client.27013 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:51:57.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.171+0000 7fe1f2000640 1 -- 172.21.15.5:0/3423721137 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559f8c24ec10 con 0x559f8c2f6d40 2024-04-24T06:51:57.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.171+0000 7fe1f2000640 10 client.27013 handle_command_reply: tid=0 2024-04-24T06:51:57.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.171+0000 7fe20de20740 1 client.27013 shutdown 2024-04-24T06:51:57.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.171+0000 7fe20de20740 1 -- 172.21.15.5:0/3423721137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8c2dba30 msgr2=0x559f8c2e3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:57.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.171+0000 7fe20de20740 1 --2- 172.21.15.5:0/3423721137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8c2dba30 0x559f8c2e3a30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4031600 tx=0x7fe1f4009260 comp rx=0 tx=0).stop 2024-04-24T06:51:57.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.171+0000 7fe20de20740 1 -- 172.21.15.5:0/3423721137 shutdown_connections 2024-04-24T06:51:57.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.171+0000 7fe20de20740 1 --2- 172.21.15.5:0/3423721137 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x559f8c2f6d40 0x559f8c2f91d0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:57.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.171+0000 7fe20de20740 1 --2- 172.21.15.5:0/3423721137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8c2e6a20 0x559f8c2e44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:57.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.171+0000 7fe20de20740 1 --2- 172.21.15.5:0/3423721137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f8c2dc350 0x559f8c2e3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:57.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.171+0000 7fe20de20740 1 --2- 172.21.15.5:0/3423721137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8c2dba30 0x559f8c2e3a30 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:57.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.171+0000 7fe20de20740 1 -- 172.21.15.5:0/3423721137 >> 172.21.15.5:0/3423721137 conn(0x559f8c2d7d50 msgr2=0x559f8c2ddb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:57.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.171+0000 7fe1f2000640 0 client.27013 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:51:57.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.171+0000 7fe20de20740 1 -- 172.21.15.5:0/3423721137 shutdown_connections 2024-04-24T06:51:57.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.171+0000 7fe20de20740 1 -- 172.21.15.5:0/3423721137 wait complete. 2024-04-24T06:51:57.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.172+0000 7fe20de20740 20 client.27013 trim_cache size 0 max 16384 2024-04-24T06:51:57.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.196+0000 7fe20de20740 1 Processor -- start 2024-04-24T06:51:57.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.196+0000 7fe20de20740 1 -- start start 2024-04-24T06:51:57.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.196+0000 7fe20de20740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8c2dba30 0x559f8c2f26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:57.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.196+0000 7fe20de20740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8c2dc350 0x559f8c2feaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:57.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.197+0000 7fe20de20740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f8c2e6a20 0x559f8c2fefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:57.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.197+0000 7fe20de20740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe2041b6b10 con 0x559f8c2dc350 2024-04-24T06:51:57.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.197+0000 7fe20de20740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe2041b43e0 con 0x559f8c2e6a20 2024-04-24T06:51:57.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.197+0000 7fe20de20740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe2041b2270 con 0x559f8c2dba30 2024-04-24T06:51:57.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.197+0000 7fe209000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f8c2e6a20 0x559f8c2fefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:57.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.197+0000 7fe203e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8c2dba30 0x559f8c2f26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:57.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.197+0000 7fe1fbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8c2dc350 0x559f8c2feaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:57.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.197+0000 7fe203e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8c2dba30 0x559f8c2f26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:41250/0 (socket says 172.21.15.5:41250) 2024-04-24T06:51:57.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.197+0000 7fe203e00640 1 -- 172.21.15.5:0/1110618312 learned_addr learned my addr 172.21.15.5:0/1110618312 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:57.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.197+0000 7fe209000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f8c2e6a20 0x559f8c2fefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:44710/0 (socket says 172.21.15.5:44710) 2024-04-24T06:51:57.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.197+0000 7fe203e00640 1 -- 172.21.15.5:0/1110618312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f8c2e6a20 msgr2=0x559f8c2fefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:57.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.197+0000 7fe203e00640 1 --2- 172.21.15.5:0/1110618312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f8c2e6a20 0x559f8c2fefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:57.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.197+0000 7fe203e00640 1 -- 172.21.15.5:0/1110618312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8c2dc350 msgr2=0x559f8c2feaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:57.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.197+0000 7fe203e00640 1 --2- 172.21.15.5:0/1110618312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8c2dc350 0x559f8c2feaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:57.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.197+0000 7fe203e00640 1 -- 172.21.15.5:0/1110618312 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f8bf533f0 con 0x559f8c2dba30 2024-04-24T06:51:57.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.198+0000 7fe203e00640 1 --2- 172.21.15.5:0/1110618312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8c2dba30 0x559f8c2f26d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4006010 tx=0x7fe1f4004190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=579062a4b238531d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:57.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.198+0000 7fe1f8c00640 1 -- 172.21.15.5:0/1110618312 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f404e070 con 0x559f8c2dba30 2024-04-24T06:51:57.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.198+0000 7fe1f8c00640 1 -- 172.21.15.5:0/1110618312 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1f4063070 con 0x559f8c2dba30 2024-04-24T06:51:57.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.198+0000 7fe1f8c00640 1 -- 172.21.15.5:0/1110618312 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4035040 con 0x559f8c2dba30 2024-04-24T06:51:57.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.199+0000 7fe20de20740 1 -- 172.21.15.5:0/1110618312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8c2dba30 msgr2=0x559f8c2f26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:57.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.199+0000 7fe20de20740 1 --2- 172.21.15.5:0/1110618312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8c2dba30 0x559f8c2f26d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4006010 tx=0x7fe1f4004190 comp rx=0 tx=0).stop 2024-04-24T06:51:57.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.199+0000 7fe20de20740 1 -- 172.21.15.5:0/1110618312 shutdown_connections 2024-04-24T06:51:57.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.199+0000 7fe20de20740 1 --2- 172.21.15.5:0/1110618312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f8c2e6a20 0x559f8c2fefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:57.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.199+0000 7fe20de20740 1 --2- 172.21.15.5:0/1110618312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8c2dc350 0x559f8c2feaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:57.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.199+0000 7fe20de20740 1 --2- 172.21.15.5:0/1110618312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8c2dba30 0x559f8c2f26d0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:57.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.199+0000 7fe20de20740 1 -- 172.21.15.5:0/1110618312 >> 172.21.15.5:0/1110618312 conn(0x559f8c2d7d50 msgr2=0x559f8c2d6740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:57.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.199+0000 7fe20de20740 1 -- 172.21.15.5:0/1110618312 shutdown_connections 2024-04-24T06:51:57.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.199+0000 7fe20de20740 1 -- 172.21.15.5:0/1110618312 wait complete. 2024-04-24T06:51:57.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.199+0000 7fe20de20740 1 -- start start 2024-04-24T06:51:57.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.200+0000 7fe20de20740 1 Processor -- start 2024-04-24T06:51:57.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.200+0000 7fe20de20740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8c2dba30 0x559f8c2ef110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:57.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.200+0000 7fe20de20740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f8c2dc350 0x559f8c2ea480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:57.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.200+0000 7fe20de20740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8c2e6a20 0x559f8c2eaba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:57.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.200+0000 7fe20de20740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe2041b28c0 con 0x559f8c2dba30 2024-04-24T06:51:57.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.200+0000 7fe20de20740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe2041b3d10 con 0x559f8c2dc350 2024-04-24T06:51:57.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.200+0000 7fe20de20740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe2041b2f70 con 0x559f8c2e6a20 2024-04-24T06:51:57.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.200+0000 7fe20de20740 10 client.0 _ll_register_callbacks cb 0x7fe1f4005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:51:57.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.200+0000 7fe1fbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f8c2dc350 0x559f8c2ea480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:57.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.201+0000 7fe1fbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f8c2dc350 0x559f8c2ea480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:44714/0 (socket says 172.21.15.5:44714) 2024-04-24T06:51:57.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.201+0000 7fe1fbe00640 1 -- 172.21.15.5:0/4216054354 learned_addr learned my addr 172.21.15.5:0/4216054354 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:51:57.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.201+0000 7fe209000640 1 --2- 172.21.15.5:0/4216054354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8c2e6a20 0x559f8c2eaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:57.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.201+0000 7fe203e00640 1 --2- 172.21.15.5:0/4216054354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8c2dba30 0x559f8c2ef110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:57.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.201+0000 7fe1fbe00640 1 -- 172.21.15.5:0/4216054354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8c2e6a20 msgr2=0x559f8c2eaba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:57.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.201+0000 7fe1fbe00640 1 --2- 172.21.15.5:0/4216054354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8c2e6a20 0x559f8c2eaba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:57.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.201+0000 7fe1fbe00640 1 -- 172.21.15.5:0/4216054354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8c2dba30 msgr2=0x559f8c2ef110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:51:57.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.201+0000 7fe1fbe00640 1 --2- 172.21.15.5:0/4216054354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8c2dba30 0x559f8c2ef110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:57.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.201+0000 7fe1fbe00640 1 -- 172.21.15.5:0/4216054354 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f8bf130f0 con 0x559f8c2dc350 2024-04-24T06:51:57.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.201+0000 7fe209000640 1 --2- 172.21.15.5:0/4216054354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8c2e6a20 0x559f8c2eaba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:51:57.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.201+0000 7fe203e00640 1 --2- 172.21.15.5:0/4216054354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8c2dba30 0x559f8c2ef110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:51:57.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.201+0000 7fe1fbe00640 1 --2- 172.21.15.5:0/4216054354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f8c2dc350 0x559f8c2ea480 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec00c6a0 tx=0x7fe1ec0d2fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52e1036a19cb9640 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:51:57.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.201+0000 7fe1f2000640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:51:57.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.202+0000 7fe1f2000640 1 -- 172.21.15.5:0/4216054354 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec0991b0 con 0x559f8c2dc350 2024-04-24T06:51:57.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.202+0000 7fe20de20740 1 -- 172.21.15.5:0/4216054354 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1f4063070 con 0x559f8c2dc350 2024-04-24T06:51:57.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.202+0000 7fe1f2000640 1 -- 172.21.15.5:0/4216054354 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7fe1ec0ce4c0 con 0x559f8c2dc350 2024-04-24T06:51:57.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.202+0000 7fe1f2000640 1 -- 172.21.15.5:0/4216054354 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe1ec099350 con 0x559f8c2dc350 2024-04-24T06:51:57.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.202+0000 7fe20de20740 10 client.27005 fetch_fsmap learned FSMap version 27 2024-04-24T06:51:57.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.202+0000 7fe20de20740 1 -- 172.21.15.5:0/4216054354 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559f8c206810 con 0x559f8c2dc350 2024-04-24T06:51:57.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.202+0000 7fe1f2000640 1 -- 172.21.15.5:0/4216054354 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1f4063070 con 0x559f8c2dc350 2024-04-24T06:51:57.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.202+0000 7fe1f2000640 1 -- 172.21.15.5:0/4216054354 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7fe1ec0989a0 con 0x559f8c2dc350 2024-04-24T06:51:57.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.203+0000 7fe20de20740 10 client.27005 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:51:57.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.203+0000 7fe20de20740 10 client.27005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:51:57.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.203+0000 7fe20de20740 20 client.27005 populate_metadata read hostname 'smithi005' 2024-04-24T06:51:57.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.203+0000 7fe20de20740 1 --2- 172.21.15.5:0/4216054354 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x559f8c35d880 0x559f8c35fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:51:57.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.203+0000 7fe20de20740 4 client.27005 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:51:57.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.203+0000 7fe20de20740 1 -- 172.21.15.5:0/4216054354 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559f8c248a80 con 0x559f8c35d880 2024-04-24T06:51:57.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.203+0000 7fe203e00640 1 --2- 172.21.15.5:0/4216054354 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x559f8c35d880 0x559f8c35fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:51:57.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.204+0000 7fe203e00640 1 --2- 172.21.15.5:0/4216054354 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x559f8c35d880 0x559f8c35fca0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6336c435343d8455 server_cookie=b923e50062923ebe in_seq=0 out_seq=0 2024-04-24T06:51:57.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.204+0000 7fe1f2000640 10 client.27005 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:51:57.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.204+0000 7fe1f2000640 1 -- 172.21.15.5:0/4216054354 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x559f8c248a80 con 0x559f8c35d880 2024-04-24T06:51:57.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.204+0000 7fe1f2000640 10 client.27005 handle_command_reply: tid=0 2024-04-24T06:51:57.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.204+0000 7fe20de20740 1 client.27005 shutdown 2024-04-24T06:51:57.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.205+0000 7fe20de20740 1 -- 172.21.15.5:0/4216054354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f8c2dc350 msgr2=0x559f8c2ea480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:57.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.205+0000 7fe20de20740 1 --2- 172.21.15.5:0/4216054354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f8c2dc350 0x559f8c2ea480 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec00c6a0 tx=0x7fe1ec0d2fb0 comp rx=0 tx=0).stop 2024-04-24T06:51:57.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.205+0000 7fe20de20740 1 -- 172.21.15.5:0/4216054354 shutdown_connections 2024-04-24T06:51:57.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.205+0000 7fe20de20740 1 --2- 172.21.15.5:0/4216054354 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x559f8c35d880 0x559f8c35fca0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:57.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.205+0000 7fe20de20740 1 --2- 172.21.15.5:0/4216054354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8c2e6a20 0x559f8c2eaba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:57.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.205+0000 7fe20de20740 1 --2- 172.21.15.5:0/4216054354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f8c2dc350 0x559f8c2ea480 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:57.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.205+0000 7fe20de20740 1 --2- 172.21.15.5:0/4216054354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8c2dba30 0x559f8c2ef110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:57.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.205+0000 7fe20de20740 1 -- 172.21.15.5:0/4216054354 >> 172.21.15.5:0/4216054354 conn(0x559f8c2d7d50 msgr2=0x559f8c2e1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:57.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.205+0000 7fe1f2000640 0 client.27005 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:51:57.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.205+0000 7fe20de20740 1 -- 172.21.15.5:0/4216054354 shutdown_connections 2024-04-24T06:51:57.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.205+0000 7fe20de20740 1 -- 172.21.15.5:0/4216054354 wait complete. 2024-04-24T06:51:57.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.205+0000 7fe20de20740 20 client.27005 trim_cache size 0 max 16384 2024-04-24T06:51:57.213 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:51:57.213 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (936 inodes in the stack)", 2024-04-24T06:51:57.213 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-24T06:51:57.214 INFO:teuthology.orchestra.run.smithi005.stdout: "8c06531e-b9e6-488e-a85a-f7030333b8e2": { 2024-04-24T06:51:57.214 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-24T06:51:57.214 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "8c06531e-b9e6-488e-a85a-f7030333b8e2", 2024-04-24T06:51:57.214 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-04-24T06:51:57.214 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:51:57.214 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:51:57.214 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:51:57.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.206+0000 7fe209a00640 1 -- 172.21.15.5:0/598230365 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fe1e80a3da0 msgr2=0x7fe1e80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:57.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.206+0000 7fe209a00640 1 --2- 172.21.15.5:0/598230365 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fe1e80a3da0 0x7fe1e80a6260 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe1f40096c0 tx=0x7fe1f40046c0 comp rx=0 tx=0).stop 2024-04-24T06:51:57.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.206+0000 7fe209a00640 1 -- 172.21.15.5:0/598230365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe204109350 msgr2=0x7fe20410df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:51:57.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.206+0000 7fe209a00640 1 --2- 172.21.15.5:0/598230365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe204109350 0x7fe20410df40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec00e9c0 tx=0x7fe1ec00ee80 comp rx=0 tx=0).stop 2024-04-24T06:51:57.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.206+0000 7fe209a00640 1 -- 172.21.15.5:0/598230365 shutdown_connections 2024-04-24T06:51:57.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.206+0000 7fe209a00640 1 --2- 172.21.15.5:0/598230365 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fe1e80a3da0 0x7fe1e80a6260 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:57.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.206+0000 7fe209a00640 1 --2- 172.21.15.5:0/598230365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe20410e480 0x7fe20410e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:57.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.206+0000 7fe209a00640 1 --2- 172.21.15.5:0/598230365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe204109350 0x7fe20410df40 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:57.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.206+0000 7fe209a00640 1 --2- 172.21.15.5:0/598230365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe204104280 0x7fe204110e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:51:57.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.206+0000 7fe209a00640 1 -- 172.21.15.5:0/598230365 >> 172.21.15.5:0/598230365 conn(0x7fe2040ffae0 msgr2=0x7fe204101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:51:57.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.206+0000 7fe209a00640 1 -- 172.21.15.5:0/598230365 shutdown_connections 2024-04-24T06:51:57.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:51:57.206+0000 7fe209a00640 1 -- 172.21.15.5:0/598230365 wait complete. 2024-04-24T06:51:57.229 INFO:tasks.cephfs.filesystem:scrub status for tag:8c06531e-b9e6-488e-a85a-f7030333b8e2 - {'path': '/', 'tag': '8c06531e-b9e6-488e-a85a-f7030333b8e2', 'options': 'recursive,force'} 2024-04-24T06:51:57.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:57 smithi137 ceph-mon[33017]: pgmap v242: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:51:57.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:57 smithi005 ceph-mon[27798]: pgmap v242: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:51:57.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:57 smithi037 ceph-mon[32864]: pgmap v242: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:51:58.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:58 smithi137 ceph-mon[33017]: scrub queued for path: / 2024-04-24T06:51:58.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:58 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:51:58.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:58 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:51:58.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:58 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:51:58.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:58 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:51:58.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:58 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:51:58.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:58 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:51:58.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:58 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:51:58.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:58 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:51:58.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:58 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:51:58.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:58 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:51:58.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:58 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:51:58.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:58 smithi005 ceph-mon[27798]: scrub queued for path: / 2024-04-24T06:51:58.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:58 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:51:58.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:58 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:51:58.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:58 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:51:58.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:58 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:51:58.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:58 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:51:58.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:58 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:51:58.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:58 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:51:58.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:58 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:51:58.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:58 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:51:58.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:58 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:51:58.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:58 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:51:58.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:58 smithi037 ceph-mon[32864]: scrub queued for path: / 2024-04-24T06:51:58.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:58 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:51:58.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:58 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:51:58.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:58 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:51:58.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:58 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:51:58.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:58 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:51:58.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:58 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:51:58.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:58 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:51:58.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:58 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:51:58.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:58 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:51:58.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:58 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:51:58.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:58 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:51:59.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:51:59 smithi137 ceph-mon[33017]: pgmap v243: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-04-24T06:51:59.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:51:59 smithi005 ceph-mon[27798]: pgmap v243: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-04-24T06:51:59.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:51:59 smithi037 ceph-mon[32864]: pgmap v243: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-04-24T06:52:01.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:01 smithi137 ceph-mon[33017]: pgmap v244: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:52:01.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:01 smithi005 ceph-mon[27798]: pgmap v244: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:52:01.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:01 smithi037 ceph-mon[32864]: pgmap v244: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:52:03.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:03 smithi137 ceph-mon[33017]: pgmap v245: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:52:03.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:03 smithi005 ceph-mon[27798]: pgmap v245: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:52:03.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:03 smithi037 ceph-mon[32864]: pgmap v245: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:52:05.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:05 smithi137 ceph-mon[33017]: pgmap v246: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-24T06:52:05.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:05 smithi005 ceph-mon[27798]: pgmap v246: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-24T06:52:05.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:05 smithi037 ceph-mon[32864]: pgmap v246: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-24T06:52:07.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:07 smithi137 ceph-mon[33017]: pgmap v247: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-24T06:52:07.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:07 smithi005 ceph-mon[27798]: pgmap v247: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-24T06:52:07.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:07 smithi037 ceph-mon[32864]: pgmap v247: 97 pgs: 97 active+clean; 88 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-24T06:52:08.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:08 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:52:08.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:08 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:52:08.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:08 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:52:08.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:08 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:52:08.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:08 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:52:08.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:08 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:52:08.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:08 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:52:08.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:08 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:52:08.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:08 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:52:09.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:09 smithi137 ceph-mon[33017]: pgmap v248: 97 pgs: 97 active+clean; 88 MiB data, 748 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-24T06:52:09.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:09 smithi137 ceph-mon[33017]: osdmap e92: 12 total, 12 up, 12 in 2024-04-24T06:52:09.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:09 smithi005 ceph-mon[27798]: pgmap v248: 97 pgs: 97 active+clean; 88 MiB data, 748 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-24T06:52:09.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:09 smithi005 ceph-mon[27798]: osdmap e92: 12 total, 12 up, 12 in 2024-04-24T06:52:09.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:09 smithi037 ceph-mon[32864]: pgmap v248: 97 pgs: 97 active+clean; 88 MiB data, 748 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-24T06:52:09.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:09 smithi037 ceph-mon[32864]: osdmap e92: 12 total, 12 up, 12 in 2024-04-24T06:52:11.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:11 smithi137 ceph-mon[33017]: pgmap v250: 97 pgs: 2 active+clean+snaptrim, 90 active+clean, 5 active+clean+snaptrim_wait; 88 MiB data, 748 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-24T06:52:11.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:11 smithi005 ceph-mon[27798]: pgmap v250: 97 pgs: 2 active+clean+snaptrim, 90 active+clean, 5 active+clean+snaptrim_wait; 88 MiB data, 748 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-24T06:52:11.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:11 smithi037 ceph-mon[32864]: pgmap v250: 97 pgs: 2 active+clean+snaptrim, 90 active+clean, 5 active+clean+snaptrim_wait; 88 MiB data, 748 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-24T06:52:13.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:13 smithi137 ceph-mon[33017]: pgmap v251: 97 pgs: 2 active+clean+snaptrim, 90 active+clean, 5 active+clean+snaptrim_wait; 88 MiB data, 748 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-24T06:52:13.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:13 smithi005 ceph-mon[27798]: pgmap v251: 97 pgs: 2 active+clean+snaptrim, 90 active+clean, 5 active+clean+snaptrim_wait; 88 MiB data, 748 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-24T06:52:13.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:13 smithi037 ceph-mon[32864]: pgmap v251: 97 pgs: 2 active+clean+snaptrim, 90 active+clean, 5 active+clean+snaptrim_wait; 88 MiB data, 748 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-24T06:52:15.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:15 smithi137 ceph-mon[33017]: pgmap v252: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-24T06:52:15.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:15 smithi005 ceph-mon[27798]: pgmap v252: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-24T06:52:15.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:15 smithi037 ceph-mon[32864]: pgmap v252: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-24T06:52:17.445 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:17 smithi137 ceph-mon[33017]: osdmap e93: 12 total, 12 up, 12 in 2024-04-24T06:52:17.445 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:17 smithi137 ceph-mon[33017]: pgmap v254: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-24T06:52:17.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:17 smithi005 ceph-mon[27798]: osdmap e93: 12 total, 12 up, 12 in 2024-04-24T06:52:17.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:17 smithi005 ceph-mon[27798]: pgmap v254: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-24T06:52:17.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:17 smithi037 ceph-mon[32864]: osdmap e93: 12 total, 12 up, 12 in 2024-04-24T06:52:17.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:17 smithi037 ceph-mon[32864]: pgmap v254: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-24T06:52:19.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:19 smithi137 ceph-mon[33017]: pgmap v255: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 104 B/s wr, 6 op/s 2024-04-24T06:52:19.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:19 smithi005 ceph-mon[27798]: pgmap v255: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 104 B/s wr, 6 op/s 2024-04-24T06:52:19.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:19 smithi037 ceph-mon[32864]: pgmap v255: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 104 B/s wr, 6 op/s 2024-04-24T06:52:21.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:21 smithi137 ceph-mon[33017]: pgmap v256: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-24T06:52:21.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:21 smithi005 ceph-mon[27798]: pgmap v256: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-24T06:52:21.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:21 smithi037 ceph-mon[32864]: pgmap v256: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-24T06:52:23.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:23 smithi137 ceph-mon[33017]: pgmap v257: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-24T06:52:23.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:23 smithi005 ceph-mon[27798]: pgmap v257: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-24T06:52:23.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:23 smithi037 ceph-mon[32864]: pgmap v257: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-24T06:52:24.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:24 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:52:24.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:24 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:52:24.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:24 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:52:25.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:25 smithi137 ceph-mon[33017]: pgmap v258: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-24T06:52:25.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:25 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:52:25.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:25 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:52:25.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:25 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:52:25.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:25 smithi005 ceph-mon[27798]: pgmap v258: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-24T06:52:25.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:25 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:52:25.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:25 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:52:25.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:25 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:52:25.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:25 smithi037 ceph-mon[32864]: pgmap v258: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-24T06:52:25.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:25 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:52:25.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:25 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:52:25.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:25 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:52:27.231 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T06:52:27.231 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T06:52:27.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.461+0000 7f0067e00640 1 -- 172.21.15.5:0/3380904699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0068107140 msgr2=0x7f006810ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:27.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.461+0000 7f0067e00640 1 --2- 172.21.15.5:0/3380904699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0068107140 0x7f006810ed10 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0060009900 tx=0x7f006002a410 comp rx=0 tx=0).stop 2024-04-24T06:52:27.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.462+0000 7f0067e00640 1 -- 172.21.15.5:0/3380904699 shutdown_connections 2024-04-24T06:52:27.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.462+0000 7f0067e00640 1 --2- 172.21.15.5:0/3380904699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0068107140 0x7f006810ed10 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:27.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.462+0000 7f0067e00640 1 --2- 172.21.15.5:0/3380904699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0068102a20 0x7f0068106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:27.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.462+0000 7f0067e00640 1 --2- 172.21.15.5:0/3380904699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0068102070 0x7f0068102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:27.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.462+0000 7f0067e00640 1 -- 172.21.15.5:0/3380904699 >> 172.21.15.5:0/3380904699 conn(0x7f00680fd950 msgr2=0x7f00680ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:52:27.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.462+0000 7f0067e00640 1 -- 172.21.15.5:0/3380904699 shutdown_connections 2024-04-24T06:52:27.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.462+0000 7f0067e00640 1 -- 172.21.15.5:0/3380904699 wait complete. 2024-04-24T06:52:27.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.463+0000 7f0067e00640 1 Processor -- start 2024-04-24T06:52:27.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.463+0000 7f0067e00640 1 -- start start 2024-04-24T06:52:27.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.463+0000 7f0067e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0068102070 0x7f00681a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:27.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.463+0000 7f0067e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0068107140 0x7f00681a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:27.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.463+0000 7f0067e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f006819a770 0x7f006819abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:27.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.463+0000 7f0067e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0068112250 con 0x7f0068107140 2024-04-24T06:52:27.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.463+0000 7f0067e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f00681120d0 con 0x7f006819a770 2024-04-24T06:52:27.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.463+0000 7f0067e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f00681123d0 con 0x7f0068102070 2024-04-24T06:52:27.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.464+0000 7f0066000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0068107140 0x7f00681a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:27.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.464+0000 7f0066000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0068107140 0x7f00681a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38000/0 (socket says 172.21.15.5:38000) 2024-04-24T06:52:27.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.464+0000 7f0066000640 1 -- 172.21.15.5:0/1526175530 learned_addr learned my addr 172.21.15.5:0/1526175530 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:52:27.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.464+0000 7f0067400640 1 --2- 172.21.15.5:0/1526175530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f006819a770 0x7f006819abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:27.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.464+0000 7f0066a00640 1 --2- 172.21.15.5:0/1526175530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0068102070 0x7f00681a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:27.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.464+0000 7f0066000640 1 -- 172.21.15.5:0/1526175530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0068102070 msgr2=0x7f00681a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:27.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.464+0000 7f0066000640 1 --2- 172.21.15.5:0/1526175530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0068102070 0x7f00681a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:27.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.464+0000 7f0066000640 1 -- 172.21.15.5:0/1526175530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f006819a770 msgr2=0x7f006819abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:27.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.464+0000 7f0066000640 1 --2- 172.21.15.5:0/1526175530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f006819a770 0x7f006819abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:27.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.464+0000 7f0066000640 1 -- 172.21.15.5:0/1526175530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f006819b370 con 0x7f0068107140 2024-04-24T06:52:27.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.464+0000 7f0067400640 1 --2- 172.21.15.5:0/1526175530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f006819a770 0x7f006819abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:52:27.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.465+0000 7f0066000640 1 --2- 172.21.15.5:0/1526175530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0068107140 0x7f00681a0bf0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f005000e9f0 tx=0x7f005000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:52:27.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.465+0000 7f005f400640 1 -- 172.21.15.5:0/1526175530 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005000ccc0 con 0x7f0068107140 2024-04-24T06:52:27.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.465+0000 7f005f400640 1 -- 172.21.15.5:0/1526175530 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0050005500 con 0x7f0068107140 2024-04-24T06:52:27.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.465+0000 7f005f400640 1 -- 172.21.15.5:0/1526175530 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0050010690 con 0x7f0068107140 2024-04-24T06:52:27.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.465+0000 7f0067e00640 1 -- 172.21.15.5:0/1526175530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00681a7450 con 0x7f0068107140 2024-04-24T06:52:27.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.465+0000 7f0067e00640 1 -- 172.21.15.5:0/1526175530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00681a7800 con 0x7f0068107140 2024-04-24T06:52:27.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.468+0000 7f005f400640 1 -- 172.21.15.5:0/1526175530 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f00500046c0 con 0x7f0068107140 2024-04-24T06:52:27.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.468+0000 7f0070576740 1 Processor -- start 2024-04-24T06:52:27.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.468+0000 7f0070576740 1 -- start start 2024-04-24T06:52:27.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.469+0000 7f0070576740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf17d3a30 0x55daf17d3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:27.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.469+0000 7f0070576740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf17d4350 0x55daf17de4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:27.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.469+0000 7f0070576740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf17dea20 0x55daf17e0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:27.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.469+0000 7f0070576740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55daf17391f0 con 0x55daf17d4350 2024-04-24T06:52:27.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.469+0000 7f0070576740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55daf16df350 con 0x55daf17d3a30 2024-04-24T06:52:27.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.469+0000 7f0070576740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55daf17351b0 con 0x55daf17dea20 2024-04-24T06:52:27.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.469+0000 7f0067400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf17d3a30 0x55daf17d3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:27.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.469+0000 7f0067400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf17d3a30 0x55daf17d3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:53448/0 (socket says 172.21.15.5:53448) 2024-04-24T06:52:27.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.469+0000 7f0067400640 1 -- 172.21.15.5:0/682177903 learned_addr learned my addr 172.21.15.5:0/682177903 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:52:27.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.469+0000 7f0066a00640 1 --2- 172.21.15.5:0/682177903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf17d4350 0x55daf17de4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:27.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.469+0000 7f0066000640 1 --2- 172.21.15.5:0/682177903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf17dea20 0x55daf17e0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:27.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.470+0000 7f0067400640 1 -- 172.21.15.5:0/682177903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf17dea20 msgr2=0x55daf17e0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:27.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.470+0000 7f0067400640 1 --2- 172.21.15.5:0/682177903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf17dea20 0x55daf17e0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:27.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.470+0000 7f0067400640 1 -- 172.21.15.5:0/682177903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf17d4350 msgr2=0x55daf17de4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:27.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.470+0000 7f0067400640 1 --2- 172.21.15.5:0/682177903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf17d4350 0x55daf17de4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:27.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.470+0000 7f0067400640 1 -- 172.21.15.5:0/682177903 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55daf1733f50 con 0x55daf17d3a30 2024-04-24T06:52:27.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.470+0000 7f005f400640 1 --2- 172.21.15.5:0/1526175530 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f004c0a3d20 0x7f004c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:27.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.470+0000 7f0067400640 1 --2- 172.21.15.5:0/682177903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf17d3a30 0x55daf17d3e10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0060009900 tx=0x7f0060005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=381ab5e362ccbb1c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:52:27.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.470+0000 7f005f400640 1 -- 172.21.15.5:0/1526175530 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f00500d0b30 con 0x7f0068107140 2024-04-24T06:52:27.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.470+0000 7f0066a00640 1 --2- 172.21.15.5:0/1526175530 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f004c0a3d20 0x7f004c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:27.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.470+0000 7f0057400640 1 -- 172.21.15.5:0/682177903 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0060040070 con 0x55daf17d3a30 2024-04-24T06:52:27.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.470+0000 7f0057400640 1 -- 172.21.15.5:0/682177903 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0060003cd0 con 0x55daf17d3a30 2024-04-24T06:52:27.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.471+0000 7f0057400640 1 -- 172.21.15.5:0/682177903 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0060035040 con 0x55daf17d3a30 2024-04-24T06:52:27.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.471+0000 7f0070576740 1 -- 172.21.15.5:0/682177903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf17d3a30 msgr2=0x55daf17d3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:27.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.471+0000 7f0066a00640 1 --2- 172.21.15.5:0/1526175530 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f004c0a3d20 0x7f004c0a61e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x55daf17d4ff0 tx=0x7f0058009500 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:52:27.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.471+0000 7f0070576740 1 --2- 172.21.15.5:0/682177903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf17d3a30 0x55daf17d3e10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0060009900 tx=0x7f0060005d50 comp rx=0 tx=0).stop 2024-04-24T06:52:27.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.471+0000 7f0070576740 1 -- 172.21.15.5:0/682177903 shutdown_connections 2024-04-24T06:52:27.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.471+0000 7f0070576740 1 --2- 172.21.15.5:0/682177903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf17dea20 0x55daf17e0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:27.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.471+0000 7f0070576740 1 --2- 172.21.15.5:0/682177903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf17d4350 0x55daf17de4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:27.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.471+0000 7f0070576740 1 --2- 172.21.15.5:0/682177903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf17d3a30 0x55daf17d3e10 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:27.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.471+0000 7f0070576740 1 -- 172.21.15.5:0/682177903 >> 172.21.15.5:0/682177903 conn(0x55daf17cfd50 msgr2=0x55daf17d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:52:27.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.471+0000 7f0070576740 1 -- 172.21.15.5:0/682177903 shutdown_connections 2024-04-24T06:52:27.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.472+0000 7f0070576740 1 -- 172.21.15.5:0/682177903 wait complete. 2024-04-24T06:52:27.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.472+0000 7f0070576740 1 -- start start 2024-04-24T06:52:27.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.472+0000 7f0070576740 1 Processor -- start 2024-04-24T06:52:27.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.473+0000 7f0070576740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf17d3a30 0x55daf17dba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:27.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.473+0000 7f0070576740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf17d4350 0x55daf17dbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:27.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.473+0000 7f0070576740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf17dea20 0x55daf17dc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:27.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.473+0000 7f0070576740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55daf17e17b0 con 0x55daf17dea20 2024-04-24T06:52:27.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.473+0000 7f0070576740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55daf16f2690 con 0x55daf17d3a30 2024-04-24T06:52:27.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.473+0000 7f0070576740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55daf17e1930 con 0x55daf17d4350 2024-04-24T06:52:27.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.473+0000 7f0070576740 10 client.0 _ll_register_callbacks cb 0x55daf172e650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:52:27.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.473+0000 7f0066000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf17d4350 0x55daf17dbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:27.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.473+0000 7f0067400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf17dea20 0x55daf17dc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:27.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.473+0000 7f0066000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf17d4350 0x55daf17dbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:38858/0 (socket says 172.21.15.5:38858) 2024-04-24T06:52:27.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.473+0000 7f0066000640 1 -- 172.21.15.5:0/1887557947 learned_addr learned my addr 172.21.15.5:0/1887557947 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:52:27.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.473+0000 7f0066a00640 1 --2- 172.21.15.5:0/1887557947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf17d3a30 0x55daf17dba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:27.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.473+0000 7f0066000640 1 -- 172.21.15.5:0/1887557947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf17d3a30 msgr2=0x55daf17dba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:27.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.473+0000 7f0066000640 1 --2- 172.21.15.5:0/1887557947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf17d3a30 0x55daf17dba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:27.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.473+0000 7f0066000640 1 -- 172.21.15.5:0/1887557947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf17dea20 msgr2=0x55daf17dc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:27.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.473+0000 7f0066000640 1 --2- 172.21.15.5:0/1887557947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf17dea20 0x55daf17dc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:27.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.473+0000 7f0066000640 1 -- 172.21.15.5:0/1887557947 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55daf16f1920 con 0x55daf17d4350 2024-04-24T06:52:27.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.473+0000 7f0067400640 1 --2- 172.21.15.5:0/1887557947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf17dea20 0x55daf17dc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:52:27.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.474+0000 7f0066000640 1 --2- 172.21.15.5:0/1887557947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf17d4350 0x55daf17dbf70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f005000c6a0 tx=0x7f0050009740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5d1ae30bac1e2cc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:52:27.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.474+0000 7f004be00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:52:27.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.474+0000 7f004be00640 1 -- 172.21.15.5:0/1887557947 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005009c5a0 con 0x55daf17d4350 2024-04-24T06:52:27.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.474+0000 7f0070576740 1 -- 172.21.15.5:0/1887557947 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0060003cd0 con 0x55daf17d4350 2024-04-24T06:52:27.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.474+0000 7f004be00640 1 -- 172.21.15.5:0/1887557947 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f00500a5240 con 0x55daf17d4350 2024-04-24T06:52:27.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.474+0000 7f004be00640 1 -- 172.21.15.5:0/1887557947 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f00500056b0 con 0x55daf17d4350 2024-04-24T06:52:27.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.474+0000 7f0070576740 10 client.27031 fetch_fsmap learned FSMap version 27 2024-04-24T06:52:27.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.474+0000 7f0070576740 1 -- 172.21.15.5:0/1887557947 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55daf15157d0 con 0x55daf17d4350 2024-04-24T06:52:27.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.474+0000 7f004be00640 1 -- 172.21.15.5:0/1887557947 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0060003cd0 con 0x55daf17d4350 2024-04-24T06:52:27.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.475+0000 7f004be00640 1 -- 172.21.15.5:0/1887557947 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f005009beb0 con 0x55daf17d4350 2024-04-24T06:52:27.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.475+0000 7f0070576740 10 client.27031 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:52:27.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.475+0000 7f0070576740 10 client.27031 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:52:27.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.475+0000 7f0070576740 20 client.27031 populate_metadata read hostname 'smithi005' 2024-04-24T06:52:27.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.475+0000 7f0070576740 1 --2- 172.21.15.5:0/1887557947 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55daf17eee60 0x55daf17f1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:27.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.475+0000 7f0070576740 4 client.27031 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:52:27.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.475+0000 7f0070576740 1 -- 172.21.15.5:0/1887557947 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55daf1746c10 con 0x55daf17eee60 2024-04-24T06:52:27.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.475+0000 7f0066a00640 1 --2- 172.21.15.5:0/1887557947 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55daf17eee60 0x55daf17f1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:27.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.476+0000 7f0066a00640 1 --2- 172.21.15.5:0/1887557947 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55daf17eee60 0x55daf17f1250 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b309955fa837d0f1 server_cookie=576ffe9c95ab0f06 in_seq=0 out_seq=0 2024-04-24T06:52:27.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.476+0000 7f004be00640 10 client.27031 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:52:27.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.477+0000 7f004be00640 1 -- 172.21.15.5:0/1887557947 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55daf1746c10 con 0x55daf17eee60 2024-04-24T06:52:27.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.477+0000 7f004be00640 10 client.27031 handle_command_reply: tid=0 2024-04-24T06:52:27.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.477+0000 7f0070576740 1 client.27031 shutdown 2024-04-24T06:52:27.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.477+0000 7f0070576740 1 -- 172.21.15.5:0/1887557947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf17d4350 msgr2=0x55daf17dbf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:27.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.477+0000 7f0070576740 1 --2- 172.21.15.5:0/1887557947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf17d4350 0x55daf17dbf70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f005000c6a0 tx=0x7f0050009740 comp rx=0 tx=0).stop 2024-04-24T06:52:27.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.477+0000 7f0070576740 1 -- 172.21.15.5:0/1887557947 shutdown_connections 2024-04-24T06:52:27.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.477+0000 7f0070576740 1 --2- 172.21.15.5:0/1887557947 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55daf17eee60 0x55daf17f1250 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:27.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.477+0000 7f0070576740 1 --2- 172.21.15.5:0/1887557947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf17dea20 0x55daf17dc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:27.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.477+0000 7f0070576740 1 --2- 172.21.15.5:0/1887557947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf17d4350 0x55daf17dbf70 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:27.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.477+0000 7f0070576740 1 --2- 172.21.15.5:0/1887557947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf17d3a30 0x55daf17dba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:27.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.477+0000 7f0070576740 1 -- 172.21.15.5:0/1887557947 >> 172.21.15.5:0/1887557947 conn(0x55daf17cfd50 msgr2=0x55daf17cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:52:27.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.477+0000 7f004be00640 0 client.27031 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:52:27.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.477+0000 7f0070576740 1 -- 172.21.15.5:0/1887557947 shutdown_connections 2024-04-24T06:52:27.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.477+0000 7f0070576740 1 -- 172.21.15.5:0/1887557947 wait complete. 2024-04-24T06:52:27.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.477+0000 7f0070576740 20 client.27031 trim_cache size 0 max 16384 2024-04-24T06:52:27.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.497+0000 7f0070576740 1 Processor -- start 2024-04-24T06:52:27.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.497+0000 7f0070576740 1 -- start start 2024-04-24T06:52:27.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.497+0000 7f0070576740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf17d3a30 0x55daf17cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:27.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.497+0000 7f0070576740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf17d4350 0x55daf17ea6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:27.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.497+0000 7f0070576740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf17dea20 0x55daf17f6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:27.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.497+0000 7f0070576740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f00681b2640 con 0x55daf17d4350 2024-04-24T06:52:27.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.497+0000 7f0070576740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f00681aff10 con 0x55daf17dea20 2024-04-24T06:52:27.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.497+0000 7f0070576740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f00681add80 con 0x55daf17d3a30 2024-04-24T06:52:27.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.498+0000 7f0067400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf17dea20 0x55daf17f6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:27.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.498+0000 7f0066000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf17d4350 0x55daf17ea6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:27.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.498+0000 7f0066a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf17d3a30 0x55daf17cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:27.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.498+0000 7f0066a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf17d3a30 0x55daf17cf750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:38870/0 (socket says 172.21.15.5:38870) 2024-04-24T06:52:27.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.498+0000 7f0067400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf17dea20 0x55daf17f6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:53458/0 (socket says 172.21.15.5:53458) 2024-04-24T06:52:27.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.498+0000 7f0066a00640 1 -- 172.21.15.5:0/319281934 learned_addr learned my addr 172.21.15.5:0/319281934 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:52:27.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.498+0000 7f0066a00640 1 -- 172.21.15.5:0/319281934 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf17dea20 msgr2=0x55daf17f6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:27.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.498+0000 7f0066a00640 1 --2- 172.21.15.5:0/319281934 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf17dea20 0x55daf17f6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:27.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.498+0000 7f0066a00640 1 -- 172.21.15.5:0/319281934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf17d4350 msgr2=0x55daf17ea6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:27.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.498+0000 7f0066a00640 1 --2- 172.21.15.5:0/319281934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf17d4350 0x55daf17ea6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:27.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.498+0000 7f0066a00640 1 -- 172.21.15.5:0/319281934 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55daf144b3f0 con 0x55daf17d3a30 2024-04-24T06:52:27.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.498+0000 7f0067400640 1 --2- 172.21.15.5:0/319281934 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf17dea20 0x55daf17f6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:52:27.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.498+0000 7f0066a00640 1 --2- 172.21.15.5:0/319281934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf17d3a30 0x55daf17cf750 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0058003c90 tx=0x7f005801a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d23c1fda89b16ccc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:52:27.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.499+0000 7f0056a00640 1 -- 172.21.15.5:0/319281934 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005800fd20 con 0x55daf17d3a30 2024-04-24T06:52:27.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.499+0000 7f0056a00640 1 -- 172.21.15.5:0/319281934 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f005802a420 con 0x55daf17d3a30 2024-04-24T06:52:27.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.499+0000 7f0056a00640 1 -- 172.21.15.5:0/319281934 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0058004330 con 0x55daf17d3a30 2024-04-24T06:52:27.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.499+0000 7f0070576740 1 -- 172.21.15.5:0/319281934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf17d3a30 msgr2=0x55daf17cf750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:27.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.499+0000 7f0070576740 1 --2- 172.21.15.5:0/319281934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf17d3a30 0x55daf17cf750 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0058003c90 tx=0x7f005801a000 comp rx=0 tx=0).stop 2024-04-24T06:52:27.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.499+0000 7f0070576740 1 -- 172.21.15.5:0/319281934 shutdown_connections 2024-04-24T06:52:27.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.499+0000 7f0070576740 1 --2- 172.21.15.5:0/319281934 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf17dea20 0x55daf17f6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:27.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.499+0000 7f0070576740 1 --2- 172.21.15.5:0/319281934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf17d4350 0x55daf17ea6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:27.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.499+0000 7f0070576740 1 --2- 172.21.15.5:0/319281934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf17d3a30 0x55daf17cf750 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:27.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.499+0000 7f0070576740 1 -- 172.21.15.5:0/319281934 >> 172.21.15.5:0/319281934 conn(0x55daf17cfd50 msgr2=0x55daf17d5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:52:27.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.500+0000 7f0070576740 1 -- 172.21.15.5:0/319281934 shutdown_connections 2024-04-24T06:52:27.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.500+0000 7f0070576740 1 -- 172.21.15.5:0/319281934 wait complete. 2024-04-24T06:52:27.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.500+0000 7f0070576740 1 -- start start 2024-04-24T06:52:27.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.500+0000 7f0070576740 1 Processor -- start 2024-04-24T06:52:27.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.501+0000 7f0070576740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf17d3a30 0x55daf17e6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:27.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.501+0000 7f0070576740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf17d4350 0x55daf17d8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:27.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.501+0000 7f0070576740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf17dea20 0x55daf17d8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:27.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.501+0000 7f0070576740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f00681ae3d0 con 0x55daf17d3a30 2024-04-24T06:52:27.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.501+0000 7f0070576740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f00681af840 con 0x55daf17dea20 2024-04-24T06:52:27.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.501+0000 7f0070576740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f00681aeaa0 con 0x55daf17d4350 2024-04-24T06:52:27.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.501+0000 7f0070576740 10 client.0 _ll_register_callbacks cb 0x7f005000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:52:27.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.501+0000 7f0066a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf17d3a30 0x55daf17e6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:27.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.501+0000 7f0066000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf17d4350 0x55daf17d8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:27.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.501+0000 7f0066a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf17d3a30 0x55daf17e6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38038/0 (socket says 172.21.15.5:38038) 2024-04-24T06:52:27.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.501+0000 7f0066a00640 1 -- 172.21.15.5:0/70112824 learned_addr learned my addr 172.21.15.5:0/70112824 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:52:27.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.501+0000 7f0067400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf17dea20 0x55daf17d8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:27.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.501+0000 7f0066000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf17d4350 0x55daf17d8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:38874/0 (socket says 172.21.15.5:38874) 2024-04-24T06:52:27.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.501+0000 7f0066000640 1 -- 172.21.15.5:0/70112824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf17dea20 msgr2=0x55daf17d8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:27.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.501+0000 7f0066000640 1 --2- 172.21.15.5:0/70112824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf17dea20 0x55daf17d8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:27.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.501+0000 7f0066000640 1 -- 172.21.15.5:0/70112824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf17d3a30 msgr2=0x55daf17e6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:27.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.501+0000 7f0066000640 1 --2- 172.21.15.5:0/70112824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf17d3a30 0x55daf17e6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:27.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.501+0000 7f0066000640 1 -- 172.21.15.5:0/70112824 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55daf140b0f0 con 0x55daf17d4350 2024-04-24T06:52:27.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.501+0000 7f0066a00640 1 --2- 172.21.15.5:0/70112824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf17d3a30 0x55daf17e6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:52:27.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.501+0000 7f0067400640 1 --2- 172.21.15.5:0/70112824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf17dea20 0x55daf17d8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:52:27.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.502+0000 7f0066000640 1 --2- 172.21.15.5:0/70112824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf17d4350 0x55daf17d8340 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f005009c760 tx=0x7f00500097b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3889259305dfd2c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:52:27.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.502+0000 7f004be00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:52:27.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.502+0000 7f004be00640 1 -- 172.21.15.5:0/70112824 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005009c3c0 con 0x55daf17d4350 2024-04-24T06:52:27.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.502+0000 7f0070576740 1 -- 172.21.15.5:0/70112824 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f005802a420 con 0x55daf17d4350 2024-04-24T06:52:27.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.502+0000 7f004be00640 1 -- 172.21.15.5:0/70112824 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f00500987b0 con 0x55daf17d4350 2024-04-24T06:52:27.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.502+0000 7f004be00640 1 -- 172.21.15.5:0/70112824 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0050098d00 con 0x55daf17d4350 2024-04-24T06:52:27.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.502+0000 7f0070576740 10 client.27037 fetch_fsmap learned FSMap version 27 2024-04-24T06:52:27.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.502+0000 7f0070576740 1 -- 172.21.15.5:0/70112824 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55daf16fe810 con 0x55daf17d4350 2024-04-24T06:52:27.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.502+0000 7f004be00640 1 -- 172.21.15.5:0/70112824 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f005802a420 con 0x55daf17d4350 2024-04-24T06:52:27.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.503+0000 7f004be00640 1 -- 172.21.15.5:0/70112824 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f00500989b0 con 0x55daf17d4350 2024-04-24T06:52:27.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.503+0000 7f0070576740 10 client.27037 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:52:27.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.503+0000 7f0070576740 10 client.27037 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:52:27.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.503+0000 7f0070576740 20 client.27037 populate_metadata read hostname 'smithi005' 2024-04-24T06:52:27.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.503+0000 7f0070576740 1 --2- 172.21.15.5:0/70112824 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55daf185dbe0 0x55daf185ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:27.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.503+0000 7f0070576740 4 client.27037 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:52:27.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.503+0000 7f0070576740 1 -- 172.21.15.5:0/70112824 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55daf1740a80 con 0x55daf185dbe0 2024-04-24T06:52:27.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.503+0000 7f0066a00640 1 --2- 172.21.15.5:0/70112824 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55daf185dbe0 0x55daf185ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:27.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.504+0000 7f0066a00640 1 --2- 172.21.15.5:0/70112824 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55daf185dbe0 0x55daf185ffd0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9162a2b8ffca1526 server_cookie=face8f743321e894 in_seq=0 out_seq=0 2024-04-24T06:52:27.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.504+0000 7f004be00640 10 client.27037 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:52:27.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.504+0000 7f004be00640 1 -- 172.21.15.5:0/70112824 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55daf1740a80 con 0x55daf185dbe0 2024-04-24T06:52:27.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.504+0000 7f004be00640 10 client.27037 handle_command_reply: tid=0 2024-04-24T06:52:27.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.504+0000 7f0070576740 1 client.27037 shutdown 2024-04-24T06:52:27.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.504+0000 7f0070576740 1 -- 172.21.15.5:0/70112824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf17d4350 msgr2=0x55daf17d8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:27.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.504+0000 7f0070576740 1 --2- 172.21.15.5:0/70112824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf17d4350 0x55daf17d8340 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f005009c760 tx=0x7f00500097b0 comp rx=0 tx=0).stop 2024-04-24T06:52:27.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.504+0000 7f0070576740 1 -- 172.21.15.5:0/70112824 shutdown_connections 2024-04-24T06:52:27.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.504+0000 7f0070576740 1 --2- 172.21.15.5:0/70112824 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55daf185dbe0 0x55daf185ffd0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:27.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.504+0000 7f0070576740 1 --2- 172.21.15.5:0/70112824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55daf17dea20 0x55daf17d8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:27.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.504+0000 7f0070576740 1 --2- 172.21.15.5:0/70112824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55daf17d4350 0x55daf17d8340 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:27.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.504+0000 7f0070576740 1 --2- 172.21.15.5:0/70112824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daf17d3a30 0x55daf17e6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:27.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.504+0000 7f004be00640 0 client.27037 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:52:27.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.504+0000 7f0070576740 1 -- 172.21.15.5:0/70112824 >> 172.21.15.5:0/70112824 conn(0x55daf17cfd50 msgr2=0x55daf17d51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:52:27.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.504+0000 7f0070576740 1 -- 172.21.15.5:0/70112824 shutdown_connections 2024-04-24T06:52:27.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.504+0000 7f0070576740 1 -- 172.21.15.5:0/70112824 wait complete. 2024-04-24T06:52:27.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.504+0000 7f0070576740 20 client.27037 trim_cache size 0 max 16384 2024-04-24T06:52:27.513 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:52:27.513 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-24T06:52:27.514 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-24T06:52:27.514 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:52:27.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.505+0000 7f0067e00640 1 -- 172.21.15.5:0/1526175530 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f004c0a3d20 msgr2=0x7f004c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:27.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.505+0000 7f0067e00640 1 --2- 172.21.15.5:0/1526175530 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f004c0a3d20 0x7f004c0a61e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x55daf17d4ff0 tx=0x7f0058009500 comp rx=0 tx=0).stop 2024-04-24T06:52:27.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.505+0000 7f0067e00640 1 -- 172.21.15.5:0/1526175530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0068107140 msgr2=0x7f00681a0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:27.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.505+0000 7f0067e00640 1 --2- 172.21.15.5:0/1526175530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0068107140 0x7f00681a0bf0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f005000e9f0 tx=0x7f005000eeb0 comp rx=0 tx=0).stop 2024-04-24T06:52:27.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.505+0000 7f0067e00640 1 -- 172.21.15.5:0/1526175530 shutdown_connections 2024-04-24T06:52:27.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.505+0000 7f0067e00640 1 --2- 172.21.15.5:0/1526175530 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f004c0a3d20 0x7f004c0a61e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:27.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.505+0000 7f0067e00640 1 --2- 172.21.15.5:0/1526175530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f006819a770 0x7f006819abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:27.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.505+0000 7f0067e00640 1 --2- 172.21.15.5:0/1526175530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0068107140 0x7f00681a0bf0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:27.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.505+0000 7f0067e00640 1 --2- 172.21.15.5:0/1526175530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0068102070 0x7f00681a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:27.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.505+0000 7f0067e00640 1 -- 172.21.15.5:0/1526175530 >> 172.21.15.5:0/1526175530 conn(0x7f00680fd950 msgr2=0x7f00680ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:52:27.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.505+0000 7f0067e00640 1 -- 172.21.15.5:0/1526175530 shutdown_connections 2024-04-24T06:52:27.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.505+0000 7f0067e00640 1 -- 172.21.15.5:0/1526175530 wait complete. 2024-04-24T06:52:27.524 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T06:52:27.524 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:52:27.593 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:27 smithi005 ceph-mon[27798]: pgmap v259: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-24T06:52:27.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:27 smithi137 ceph-mon[33017]: pgmap v259: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-24T06:52:27.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.734+0000 7fa2de000640 1 -- 172.21.15.5:0/847748550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2d81093e0 msgr2=0x7fa2d8110fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:27.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.734+0000 7fa2de000640 1 --2- 172.21.15.5:0/847748550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2d81093e0 0x7fa2d8110fb0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0009ca0 tx=0x7fa2d002a8c0 comp rx=0 tx=0).stop 2024-04-24T06:52:27.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.734+0000 7fa2de000640 1 -- 172.21.15.5:0/847748550 shutdown_connections 2024-04-24T06:52:27.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.734+0000 7fa2de000640 1 --2- 172.21.15.5:0/847748550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2d81093e0 0x7fa2d8110fb0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:27.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.734+0000 7fa2de000640 1 --2- 172.21.15.5:0/847748550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2d8104cc0 0x7fa2d8108d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:27.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.734+0000 7fa2de000640 1 --2- 172.21.15.5:0/847748550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d8104310 0x7fa2d81046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:27.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.734+0000 7fa2de000640 1 -- 172.21.15.5:0/847748550 >> 172.21.15.5:0/847748550 conn(0x7fa2d80ffb70 msgr2=0x7fa2d8101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:52:27.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.734+0000 7fa2de000640 1 -- 172.21.15.5:0/847748550 shutdown_connections 2024-04-24T06:52:27.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.734+0000 7fa2de000640 1 -- 172.21.15.5:0/847748550 wait complete. 2024-04-24T06:52:27.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.735+0000 7fa2de000640 1 Processor -- start 2024-04-24T06:52:27.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.735+0000 7fa2de000640 1 -- start start 2024-04-24T06:52:27.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.736+0000 7fa2de000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d8104310 0x7fa2d81a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:27.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.736+0000 7fa2de000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2d81093e0 0x7fa2d81a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:27.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.736+0000 7fa2de000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2d819a7c0 0x7fa2d819ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:27.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.736+0000 7fa2de000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d81144a0 con 0x7fa2d81093e0 2024-04-24T06:52:27.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.736+0000 7fa2de000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d8114320 con 0x7fa2d8104310 2024-04-24T06:52:27.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.736+0000 7fa2de000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d8114620 con 0x7fa2d819a7c0 2024-04-24T06:52:27.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.736+0000 7fa2dcc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d8104310 0x7fa2d81a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:27.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.736+0000 7fa2dcc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d8104310 0x7fa2d81a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:53494/0 (socket says 172.21.15.5:53494) 2024-04-24T06:52:27.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.736+0000 7fa2dcc00640 1 -- 172.21.15.5:0/1470269803 learned_addr learned my addr 172.21.15.5:0/1470269803 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:52:27.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.736+0000 7fa2d7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2d81093e0 0x7fa2d81a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:27.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.736+0000 7fa2dd600640 1 --2- 172.21.15.5:0/1470269803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2d819a7c0 0x7fa2d819ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:27.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.736+0000 7fa2dcc00640 1 -- 172.21.15.5:0/1470269803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2d819a7c0 msgr2=0x7fa2d819ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:27.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.737+0000 7fa2dcc00640 1 --2- 172.21.15.5:0/1470269803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2d819a7c0 0x7fa2d819ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:27.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.737+0000 7fa2dcc00640 1 -- 172.21.15.5:0/1470269803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2d81093e0 msgr2=0x7fa2d81a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:27.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.737+0000 7fa2dcc00640 1 --2- 172.21.15.5:0/1470269803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2d81093e0 0x7fa2d81a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:27.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.737+0000 7fa2dcc00640 1 -- 172.21.15.5:0/1470269803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d819b350 con 0x7fa2d8104310 2024-04-24T06:52:27.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.737+0000 7fa2dcc00640 1 --2- 172.21.15.5:0/1470269803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d8104310 0x7fa2d81a0650 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc00e9f0 tx=0x7fa2cc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:52:27.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.737+0000 7fa2d5600640 1 -- 172.21.15.5:0/1470269803 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc00cd80 con 0x7fa2d8104310 2024-04-24T06:52:27.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.737+0000 7fa2de000640 1 -- 172.21.15.5:0/1470269803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2d819b640 con 0x7fa2d8104310 2024-04-24T06:52:27.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.737+0000 7fa2d5600640 1 -- 172.21.15.5:0/1470269803 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2cc010910 con 0x7fa2d8104310 2024-04-24T06:52:27.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.737+0000 7fa2d5600640 1 -- 172.21.15.5:0/1470269803 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc0188f0 con 0x7fa2d8104310 2024-04-24T06:52:27.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.737+0000 7fa2de000640 1 -- 172.21.15.5:0/1470269803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2d81a7870 con 0x7fa2d8104310 2024-04-24T06:52:27.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.738+0000 7fa2de000640 1 -- 172.21.15.5:0/1470269803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2d81057e0 con 0x7fa2d8104310 2024-04-24T06:52:27.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.741+0000 7fa2d5600640 1 -- 172.21.15.5:0/1470269803 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fa2cc010ac0 con 0x7fa2d8104310 2024-04-24T06:52:27.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.743+0000 7fa2d5600640 1 --2- 172.21.15.5:0/1470269803 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fa2c00a3d90 0x7fa2c00a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:27.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.743+0000 7fa2d5600640 1 -- 172.21.15.5:0/1470269803 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7fa2cc0d0e30 con 0x7fa2d8104310 2024-04-24T06:52:27.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.743+0000 7fa2d7e00640 1 --2- 172.21.15.5:0/1470269803 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fa2c00a3d90 0x7fa2c00a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:27.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.744+0000 7fa2d7e00640 1 --2- 172.21.15.5:0/1470269803 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fa2c00a3d90 0x7fa2c00a6250 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa2d81032c0 tx=0x7fa2c4005f90 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:52:27.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.745+0000 7fa2d5600640 1 -- 172.21.15.5:0/1470269803 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa2d819b640 con 0x7fa2d8104310 2024-04-24T06:52:27.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:27 smithi037 ceph-mon[32864]: pgmap v259: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-24T06:52:27.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:27.999+0000 7fa2de000640 1 -- 172.21.15.5:0/1470269803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2d804fe00 con 0x7fa2d8104310 2024-04-24T06:52:28.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.000+0000 7fa2d5600640 1 -- 172.21.15.5:0/1470269803 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11856 (secure 0 0 0) 0x7fa2cc09d460 con 0x7fa2d8104310 2024-04-24T06:52:28.001 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:52:28.001 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1735462750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1735462750},{"type":"v1","addr":"172.21.15.5:6837","nonce":1735462750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1499469370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1499469370},{"type":"v1","addr":"172.21.15.5:6841","nonce":1499469370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3705638847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3705638847},{"type":"v1","addr":"172.21.15.37:6835","nonce":3705638847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T06:41:34.764490+0000","modified":"2024-04-24T06:48:57.768410+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24550,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6837/1390103296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1390103296},{"type":"v1","addr":"172.21.15.37:6837","nonce":1390103296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3217754215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3217754215},{"type":"v1","addr":"172.21.15.137:6835","nonce":3217754215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6837/3044222976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3044222976},{"type":"v1","addr":"172.21.15.137:6837","nonce":3044222976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.5:6839/3173201459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3173201459},{"type":"v1","addr":"172.21.15.5:6839","nonce":3173201459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6839/1889774232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1889774232},{"type":"v1","addr":"172.21.15.137:6839","nonce":1889774232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6839/3536721753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":3536721753},{"type":"v1","addr":"172.21.15.37:6839","nonce":3536721753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24550,24569,24539]},"id":1}]} 2024-04-24T06:52:28.002 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-04-24T06:52:28.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.003+0000 7fa2de000640 1 -- 172.21.15.5:0/1470269803 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fa2c00a3d90 msgr2=0x7fa2c00a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:28.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.003+0000 7fa2de000640 1 --2- 172.21.15.5:0/1470269803 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fa2c00a3d90 0x7fa2c00a6250 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa2d81032c0 tx=0x7fa2c4005f90 comp rx=0 tx=0).stop 2024-04-24T06:52:28.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.003+0000 7fa2de000640 1 -- 172.21.15.5:0/1470269803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d8104310 msgr2=0x7fa2d81a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:28.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.003+0000 7fa2de000640 1 --2- 172.21.15.5:0/1470269803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d8104310 0x7fa2d81a0650 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc00e9f0 tx=0x7fa2cc00eeb0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.003+0000 7fa2de000640 1 -- 172.21.15.5:0/1470269803 shutdown_connections 2024-04-24T06:52:28.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.003+0000 7fa2de000640 1 --2- 172.21.15.5:0/1470269803 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fa2c00a3d90 0x7fa2c00a6250 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.003+0000 7fa2de000640 1 --2- 172.21.15.5:0/1470269803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2d819a7c0 0x7fa2d819ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.003+0000 7fa2de000640 1 --2- 172.21.15.5:0/1470269803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2d81093e0 0x7fa2d81a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.003+0000 7fa2de000640 1 --2- 172.21.15.5:0/1470269803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d8104310 0x7fa2d81a0650 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.003+0000 7fa2de000640 1 -- 172.21.15.5:0/1470269803 >> 172.21.15.5:0/1470269803 conn(0x7fa2d80ffb70 msgr2=0x7fa2d8101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:52:28.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.004+0000 7fa2de000640 1 -- 172.21.15.5:0/1470269803 shutdown_connections 2024-04-24T06:52:28.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.004+0000 7fa2de000640 1 -- 172.21.15.5:0/1470269803 wait complete. 2024-04-24T06:52:28.025 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T06:52:28.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.239+0000 7f25d1200640 1 -- 172.21.15.5:0/990292113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25cc103900 msgr2=0x7f25cc10bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:28.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.239+0000 7f25d1200640 1 --2- 172.21.15.5:0/990292113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25cc103900 0x7f25cc10bde0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f25b8009900 tx=0x7f25b802a480 comp rx=0 tx=0).stop 2024-04-24T06:52:28.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.239+0000 7f25d1200640 1 -- 172.21.15.5:0/990292113 shutdown_connections 2024-04-24T06:52:28.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.239+0000 7f25d1200640 1 --2- 172.21.15.5:0/990292113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25cc10c400 0x7f25cc10e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.239+0000 7f25d1200640 1 --2- 172.21.15.5:0/990292113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25cc103900 0x7f25cc10bde0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.239+0000 7f25d1200640 1 --2- 172.21.15.5:0/990292113 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25cc102fe0 0x7f25cc1033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.239+0000 7f25d1200640 1 -- 172.21.15.5:0/990292113 >> 172.21.15.5:0/990292113 conn(0x7f25cc077ab0 msgr2=0x7f25cc0fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:52:28.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.240+0000 7f25d1200640 1 -- 172.21.15.5:0/990292113 shutdown_connections 2024-04-24T06:52:28.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.240+0000 7f25d1200640 1 -- 172.21.15.5:0/990292113 wait complete. 2024-04-24T06:52:28.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.241+0000 7f25d1200640 1 Processor -- start 2024-04-24T06:52:28.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.241+0000 7f25d1200640 1 -- start start 2024-04-24T06:52:28.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.241+0000 7f25d1200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25cc102fe0 0x7f25cc1a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.241+0000 7f25d1200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25cc103900 0x7f25cc1a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.241+0000 7f25d1200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25cc10c400 0x7f25cc19a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.241+0000 7f25d1200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f25cc112200 con 0x7f25cc102fe0 2024-04-24T06:52:28.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.241+0000 7f25d1200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f25cc112080 con 0x7f25cc103900 2024-04-24T06:52:28.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.241+0000 7f25d1200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f25cc112380 con 0x7f25cc10c400 2024-04-24T06:52:28.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.241+0000 7f25cb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25cc102fe0 0x7f25cc1a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.241+0000 7f25cb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25cc102fe0 0x7f25cc1a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38074/0 (socket says 172.21.15.5:38074) 2024-04-24T06:52:28.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.242+0000 7f25cb400640 1 -- 172.21.15.5:0/2735508875 learned_addr learned my addr 172.21.15.5:0/2735508875 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:52:28.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.242+0000 7f25caa00640 1 --2- 172.21.15.5:0/2735508875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25cc103900 0x7f25cc1a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.242+0000 7f25cbe00640 1 --2- 172.21.15.5:0/2735508875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25cc10c400 0x7f25cc19a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.242+0000 7f25cb400640 1 -- 172.21.15.5:0/2735508875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25cc10c400 msgr2=0x7f25cc19a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:28.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.242+0000 7f25cb400640 1 --2- 172.21.15.5:0/2735508875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25cc10c400 0x7f25cc19a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.242+0000 7f25cb400640 1 -- 172.21.15.5:0/2735508875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25cc103900 msgr2=0x7f25cc1a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:28.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.242+0000 7f25cb400640 1 --2- 172.21.15.5:0/2735508875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25cc103900 0x7f25cc1a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.242+0000 7f25cb400640 1 -- 172.21.15.5:0/2735508875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25cc19af90 con 0x7f25cc102fe0 2024-04-24T06:52:28.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.242+0000 7f25cb400640 1 --2- 172.21.15.5:0/2735508875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25cc102fe0 0x7f25cc1a0650 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f25bc00e9f0 tx=0x7f25bc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:52:28.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.243+0000 7f25c3e00640 1 -- 172.21.15.5:0/2735508875 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25bc00cd80 con 0x7f25cc102fe0 2024-04-24T06:52:28.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.243+0000 7f25c3e00640 1 -- 172.21.15.5:0/2735508875 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25bc005500 con 0x7f25cc102fe0 2024-04-24T06:52:28.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.243+0000 7f25d1200640 1 -- 172.21.15.5:0/2735508875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25cc19b280 con 0x7f25cc102fe0 2024-04-24T06:52:28.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.243+0000 7f25c3e00640 1 -- 172.21.15.5:0/2735508875 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25bc010690 con 0x7f25cc102fe0 2024-04-24T06:52:28.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.243+0000 7f25d1200640 1 -- 172.21.15.5:0/2735508875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25cc1a74a0 con 0x7f25cc102fe0 2024-04-24T06:52:28.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.246+0000 7f25c3e00640 1 -- 172.21.15.5:0/2735508875 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f25bc0046c0 con 0x7f25cc102fe0 2024-04-24T06:52:28.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.246+0000 7f25d5582740 1 Processor -- start 2024-04-24T06:52:28.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.247+0000 7f25d5582740 1 -- start start 2024-04-24T06:52:28.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.247+0000 7f25d5582740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562add795b40 0x562add795f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.247+0000 7f25d5582740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562add796460 0x562add7a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.247+0000 7f25d5582740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562add7a0b30 0x562add7a2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.247+0000 7f25d5582740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562add6fb1d0 con 0x562add7a0b30 2024-04-24T06:52:28.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.247+0000 7f25d5582740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562add6a1450 con 0x562add795b40 2024-04-24T06:52:28.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.247+0000 7f25d5582740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562add6f7190 con 0x562add796460 2024-04-24T06:52:28.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.247+0000 7f25cbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562add795b40 0x562add795f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.247+0000 7f25cbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562add795b40 0x562add795f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:53530/0 (socket says 172.21.15.5:53530) 2024-04-24T06:52:28.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.247+0000 7f25cbe00640 1 -- 172.21.15.5:0/1941121452 learned_addr learned my addr 172.21.15.5:0/1941121452 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:52:28.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.247+0000 7f25caa00640 1 --2- 172.21.15.5:0/1941121452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562add7a0b30 0x562add7a2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.248+0000 7f25cb400640 1 --2- 172.21.15.5:0/1941121452 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562add796460 0x562add7a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.248+0000 7f25cbe00640 1 -- 172.21.15.5:0/1941121452 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562add796460 msgr2=0x562add7a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.248+0000 7f25cbe00640 1 --2- 172.21.15.5:0/1941121452 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562add796460 0x562add7a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.248+0000 7f25cbe00640 1 -- 172.21.15.5:0/1941121452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562add7a0b30 msgr2=0x562add7a2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.248+0000 7f25cbe00640 1 --2- 172.21.15.5:0/1941121452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562add7a0b30 0x562add7a2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.248+0000 7f25cbe00640 1 -- 172.21.15.5:0/1941121452 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562add75e310 con 0x562add795b40 2024-04-24T06:52:28.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.248+0000 7f25cbe00640 1 --2- 172.21.15.5:0/1941121452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562add795b40 0x562add795f20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f25c4009f50 tx=0x7f25c40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3d87bae03e13416 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:52:28.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.248+0000 7f25c3e00640 1 --2- 172.21.15.5:0/2735508875 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f25b00a3da0 0x7f25b00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.249+0000 7f25c0c00640 1 -- 172.21.15.5:0/1941121452 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c4004d20 con 0x562add795b40 2024-04-24T06:52:28.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.249+0000 7f25c3e00640 1 -- 172.21.15.5:0/2735508875 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f25bc0d0ba0 con 0x7f25cc102fe0 2024-04-24T06:52:28.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.249+0000 7f25c0c00640 1 -- 172.21.15.5:0/1941121452 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25c4004050 con 0x562add795b40 2024-04-24T06:52:28.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.249+0000 7f25c0c00640 1 -- 172.21.15.5:0/1941121452 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c4005280 con 0x562add795b40 2024-04-24T06:52:28.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.249+0000 7f25cb400640 1 --2- 172.21.15.5:0/2735508875 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f25b00a3da0 0x7f25b00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.249+0000 7f25d5582740 1 -- 172.21.15.5:0/1941121452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562add795b40 msgr2=0x562add795f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:28.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.249+0000 7f25cb400640 1 --2- 172.21.15.5:0/2735508875 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f25b00a3da0 0x7f25b00a6260 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f25bc0ce390 tx=0x7f25bc0d46f0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:52:28.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.249+0000 7f25d5582740 1 --2- 172.21.15.5:0/1941121452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562add795b40 0x562add795f20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f25c4009f50 tx=0x7f25c40118d0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.249+0000 7f25d5582740 1 -- 172.21.15.5:0/1941121452 shutdown_connections 2024-04-24T06:52:28.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.249+0000 7f25d5582740 1 --2- 172.21.15.5:0/1941121452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562add7a0b30 0x562add7a2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.250+0000 7f25d5582740 1 --2- 172.21.15.5:0/1941121452 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562add796460 0x562add7a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.250+0000 7f25d5582740 1 --2- 172.21.15.5:0/1941121452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562add795b40 0x562add795f20 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.250+0000 7f25d5582740 1 -- 172.21.15.5:0/1941121452 >> 172.21.15.5:0/1941121452 conn(0x562add791e60 msgr2=0x562add792240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:52:28.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.250+0000 7f25d5582740 1 -- 172.21.15.5:0/1941121452 shutdown_connections 2024-04-24T06:52:28.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.250+0000 7f25d5582740 1 -- 172.21.15.5:0/1941121452 wait complete. 2024-04-24T06:52:28.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.250+0000 7f25d5582740 1 -- start start 2024-04-24T06:52:28.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.250+0000 7f25d5582740 1 Processor -- start 2024-04-24T06:52:28.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.251+0000 7f25d5582740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562add795b40 0x562add79d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.251+0000 7f25d5582740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562add796460 0x562add79de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.251+0000 7f25d5582740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562add7a0b30 0x562add79e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.251+0000 7f25d5582740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562add7a39c0 con 0x562add795b40 2024-04-24T06:52:28.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.251+0000 7f25d5582740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562add6b4810 con 0x562add7a0b30 2024-04-24T06:52:28.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.251+0000 7f25d5582740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562add7a3b40 con 0x562add796460 2024-04-24T06:52:28.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.251+0000 7f25d5582740 10 client.0 _ll_register_callbacks cb 0x562add682140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:52:28.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.251+0000 7f25cbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562add7a0b30 0x562add79e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.251+0000 7f25cbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562add7a0b30 0x562add79e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:53536/0 (socket says 172.21.15.5:53536) 2024-04-24T06:52:28.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.251+0000 7f25cbe00640 1 -- 172.21.15.5:0/1865977478 learned_addr learned my addr 172.21.15.5:0/1865977478 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:52:28.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.251+0000 7f25caa00640 1 --2- 172.21.15.5:0/1865977478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562add796460 0x562add79de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.252+0000 7f25cbe00640 1 -- 172.21.15.5:0/1865977478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562add796460 msgr2=0x562add79de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:28.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.252+0000 7f25cbe00640 1 --2- 172.21.15.5:0/1865977478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562add796460 0x562add79de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.252+0000 7f25cbe00640 1 -- 172.21.15.5:0/1865977478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562add795b40 msgr2=0x562add79d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:28.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.252+0000 7f25cbe00640 1 --2- 172.21.15.5:0/1865977478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562add795b40 0x562add79d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.252+0000 7f25cbe00640 1 -- 172.21.15.5:0/1865977478 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562add6c53f0 con 0x562add7a0b30 2024-04-24T06:52:28.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.252+0000 7f25cbe00640 1 --2- 172.21.15.5:0/1865977478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562add7a0b30 0x562add79e380 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f25c4009250 tx=0x7f25c4004de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53bab852c04d1d7a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:52:28.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.252+0000 7f25b5600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:52:28.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.252+0000 7f25b5600640 1 -- 172.21.15.5:0/1865977478 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c4028070 con 0x562add7a0b30 2024-04-24T06:52:28.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.253+0000 7f25d5582740 1 -- 172.21.15.5:0/1865977478 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25c4004050 con 0x562add7a0b30 2024-04-24T06:52:28.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.253+0000 7f25b5600640 1 -- 172.21.15.5:0/1865977478 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f25c4003c30 con 0x562add7a0b30 2024-04-24T06:52:28.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.253+0000 7f25b5600640 1 -- 172.21.15.5:0/1865977478 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f25c4034050 con 0x562add7a0b30 2024-04-24T06:52:28.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.253+0000 7f25d5582740 10 client.27041 fetch_fsmap learned FSMap version 27 2024-04-24T06:52:28.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.253+0000 7f25d5582740 1 -- 172.21.15.5:0/1865977478 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562add40d3f0 con 0x562add7a0b30 2024-04-24T06:52:28.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.253+0000 7f25b5600640 1 -- 172.21.15.5:0/1865977478 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25c4004050 con 0x562add7a0b30 2024-04-24T06:52:28.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.253+0000 7f25b5600640 1 -- 172.21.15.5:0/1865977478 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f25c400f480 con 0x562add7a0b30 2024-04-24T06:52:28.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.253+0000 7f25d5582740 10 client.27041 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:52:28.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.253+0000 7f25d5582740 10 client.27041 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-24T06:52:28.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.254+0000 7f25d5582740 20 client.27041 populate_metadata read hostname 'smithi005' 2024-04-24T06:52:28.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.254+0000 7f25d5582740 1 --2- 172.21.15.5:0/1865977478 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x562add7b0ce0 0x562add7b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.254+0000 7f25d5582740 4 client.27041 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:52:28.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.254+0000 7f25d5582740 1 -- 172.21.15.5:0/1865977478 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562add708bf0 con 0x562add7b0ce0 2024-04-24T06:52:28.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.254+0000 7f25caa00640 1 --2- 172.21.15.5:0/1865977478 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x562add7b0ce0 0x562add7b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.255+0000 7f25b5600640 10 client.27041 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:52:28.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.255+0000 7f25caa00640 1 --2- 172.21.15.5:0/1865977478 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x562add7b0ce0 0x562add7b30d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f24fae27d12a9a40 server_cookie=c6a3f0884f7c7b8e in_seq=0 out_seq=0 2024-04-24T06:52:28.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.255+0000 7f25b5600640 1 -- 172.21.15.5:0/1865977478 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562add708bf0 con 0x562add7b0ce0 2024-04-24T06:52:28.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.255+0000 7f25b5600640 10 client.27041 handle_command_reply: tid=0 2024-04-24T06:52:28.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.255+0000 7f25d5582740 1 client.27041 shutdown 2024-04-24T06:52:28.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.256+0000 7f25d5582740 1 -- 172.21.15.5:0/1865977478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562add7a0b30 msgr2=0x562add79e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:28.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.256+0000 7f25d5582740 1 --2- 172.21.15.5:0/1865977478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562add7a0b30 0x562add79e380 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f25c4009250 tx=0x7f25c4004de0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.256+0000 7f25d5582740 1 -- 172.21.15.5:0/1865977478 shutdown_connections 2024-04-24T06:52:28.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.256+0000 7f25d5582740 1 --2- 172.21.15.5:0/1865977478 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x562add7b0ce0 0x562add7b30d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.256+0000 7f25d5582740 1 --2- 172.21.15.5:0/1865977478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562add7a0b30 0x562add79e380 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.256+0000 7f25d5582740 1 --2- 172.21.15.5:0/1865977478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562add796460 0x562add79de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.256+0000 7f25d5582740 1 --2- 172.21.15.5:0/1865977478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562add795b40 0x562add79d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.256+0000 7f25b5600640 0 client.27041 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:52:28.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.256+0000 7f25d5582740 1 -- 172.21.15.5:0/1865977478 >> 172.21.15.5:0/1865977478 conn(0x562add791e60 msgr2=0x562add797c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:52:28.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.256+0000 7f25d5582740 1 -- 172.21.15.5:0/1865977478 shutdown_connections 2024-04-24T06:52:28.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.256+0000 7f25d5582740 1 -- 172.21.15.5:0/1865977478 wait complete. 2024-04-24T06:52:28.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.256+0000 7f25d5582740 20 client.27041 trim_cache size 0 max 16384 2024-04-24T06:52:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.271+0000 7f25d5582740 1 Processor -- start 2024-04-24T06:52:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.271+0000 7f25d5582740 1 -- start start 2024-04-24T06:52:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.271+0000 7f25d5582740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562add795b40 0x562add7ac880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.271+0000 7f25d5582740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562add796460 0x562add797bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.271+0000 7f25d5582740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562add7a0b30 0x562add7b8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.272+0000 7f25d5582740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f25cc1b2790 con 0x562add7a0b30 2024-04-24T06:52:28.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.272+0000 7f25d5582740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f25cc1b0060 con 0x562add796460 2024-04-24T06:52:28.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.272+0000 7f25d5582740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f25cc1aded0 con 0x562add795b40 2024-04-24T06:52:28.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.272+0000 7f25caa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562add796460 0x562add797bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.272+0000 7f25cbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562add7a0b30 0x562add7b8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.272+0000 7f25caa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562add796460 0x562add797bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:53542/0 (socket says 172.21.15.5:53542) 2024-04-24T06:52:28.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.272+0000 7f25caa00640 1 -- 172.21.15.5:0/2505999940 learned_addr learned my addr 172.21.15.5:0/2505999940 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:52:28.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.272+0000 7f25cbe00640 1 --2- 172.21.15.5:0/2505999940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562add795b40 0x562add7ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.272+0000 7f25caa00640 1 -- 172.21.15.5:0/2505999940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562add795b40 msgr2=0x562add7ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:28.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.272+0000 7f25caa00640 1 --2- 172.21.15.5:0/2505999940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562add795b40 0x562add7ac880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.272+0000 7f25caa00640 1 -- 172.21.15.5:0/2505999940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562add7a0b30 msgr2=0x562add7b8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:28.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.272+0000 7f25caa00640 1 --2- 172.21.15.5:0/2505999940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562add7a0b30 0x562add7b8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.272+0000 7f25caa00640 1 -- 172.21.15.5:0/2505999940 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562add4d77d0 con 0x562add796460 2024-04-24T06:52:28.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.272+0000 7f25caa00640 1 --2- 172.21.15.5:0/2505999940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562add796460 0x562add797bc0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f25b8009230 tx=0x7f25b8005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f461091d4bba87d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:52:28.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.273+0000 7f25b7e00640 1 -- 172.21.15.5:0/2505999940 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b8035040 con 0x562add796460 2024-04-24T06:52:28.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.273+0000 7f25b7e00640 1 -- 172.21.15.5:0/2505999940 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25b804b070 con 0x562add796460 2024-04-24T06:52:28.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.273+0000 7f25b7e00640 1 -- 172.21.15.5:0/2505999940 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b80053c0 con 0x562add796460 2024-04-24T06:52:28.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.273+0000 7f25d5582740 1 -- 172.21.15.5:0/2505999940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562add796460 msgr2=0x562add797bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:28.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.273+0000 7f25d5582740 1 --2- 172.21.15.5:0/2505999940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562add796460 0x562add797bc0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f25b8009230 tx=0x7f25b8005b50 comp rx=0 tx=0).stop 2024-04-24T06:52:28.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.274+0000 7f25d5582740 1 -- 172.21.15.5:0/2505999940 shutdown_connections 2024-04-24T06:52:28.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.274+0000 7f25d5582740 1 --2- 172.21.15.5:0/2505999940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562add7a0b30 0x562add7b8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.274+0000 7f25d5582740 1 --2- 172.21.15.5:0/2505999940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562add796460 0x562add797bc0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.274+0000 7f25d5582740 1 --2- 172.21.15.5:0/2505999940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562add795b40 0x562add7ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.274+0000 7f25d5582740 1 -- 172.21.15.5:0/2505999940 >> 172.21.15.5:0/2505999940 conn(0x562add791e60 msgr2=0x562add792240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:52:28.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.274+0000 7f25d5582740 1 -- 172.21.15.5:0/2505999940 shutdown_connections 2024-04-24T06:52:28.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.274+0000 7f25d5582740 1 -- 172.21.15.5:0/2505999940 wait complete. 2024-04-24T06:52:28.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.274+0000 7f25d5582740 1 -- start start 2024-04-24T06:52:28.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.274+0000 7f25d5582740 1 Processor -- start 2024-04-24T06:52:28.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.275+0000 7f25d5582740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562add795b40 0x562add7a8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.275+0000 7f25d5582740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562add796460 0x562add7a92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.275+0000 7f25d5582740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562add7a0b30 0x562add79ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.275+0000 7f25d5582740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f25cc1ae520 con 0x562add7a0b30 2024-04-24T06:52:28.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.275+0000 7f25d5582740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f25cc1af990 con 0x562add796460 2024-04-24T06:52:28.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.275+0000 7f25d5582740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f25cc1aebf0 con 0x562add795b40 2024-04-24T06:52:28.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.275+0000 7f25d5582740 10 client.0 _ll_register_callbacks cb 0x562add7a0a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:52:28.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.275+0000 7f25cbe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562add795b40 0x562add7a8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.275+0000 7f25cbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562add7a0b30 0x562add79ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.275+0000 7f25caa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562add796460 0x562add7a92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.275+0000 7f25cbe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562add795b40 0x562add7a8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:38972/0 (socket says 172.21.15.5:38972) 2024-04-24T06:52:28.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.275+0000 7f25caa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562add796460 0x562add7a92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:53558/0 (socket says 172.21.15.5:53558) 2024-04-24T06:52:28.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.275+0000 7f25cbe00640 1 -- 172.21.15.5:0/838721452 learned_addr learned my addr 172.21.15.5:0/838721452 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:52:28.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.276+0000 7f25cbe00640 1 -- 172.21.15.5:0/838721452 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562add795b40 msgr2=0x562add7a8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:28.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.276+0000 7f25cbe00640 1 --2- 172.21.15.5:0/838721452 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562add795b40 0x562add7a8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.276+0000 7f25cbe00640 1 -- 172.21.15.5:0/838721452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562add796460 msgr2=0x562add7a92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:28.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.276+0000 7f25cbe00640 1 --2- 172.21.15.5:0/838721452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562add796460 0x562add7a92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.276+0000 7f25cbe00640 1 -- 172.21.15.5:0/838721452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562add79b140 con 0x562add7a0b30 2024-04-24T06:52:28.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.276+0000 7f25caa00640 1 --2- 172.21.15.5:0/838721452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562add796460 0x562add7a92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:52:28.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.276+0000 7f25cbe00640 1 --2- 172.21.15.5:0/838721452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562add7a0b30 0x562add79ab80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f25c4011910 tx=0x7f25c4009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e1236d54353dd2f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:52:28.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.276+0000 7f25b5600640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:52:28.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.276+0000 7f25b5600640 1 -- 172.21.15.5:0/838721452 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c4028040 con 0x562add7a0b30 2024-04-24T06:52:28.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.276+0000 7f25b5600640 1 -- 172.21.15.5:0/838721452 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f25c400f480 con 0x562add7a0b30 2024-04-24T06:52:28.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.276+0000 7f25d5582740 1 -- 172.21.15.5:0/838721452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25b804b070 con 0x562add7a0b30 2024-04-24T06:52:28.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.276+0000 7f25b5600640 1 -- 172.21.15.5:0/838721452 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f25c400f680 con 0x562add7a0b30 2024-04-24T06:52:28.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.276+0000 7f25d5582740 10 client.16902 fetch_fsmap learned FSMap version 27 2024-04-24T06:52:28.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.276+0000 7f25d5582740 1 -- 172.21.15.5:0/838721452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562add7a17f0 con 0x562add7a0b30 2024-04-24T06:52:28.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.277+0000 7f25b5600640 1 -- 172.21.15.5:0/838721452 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25b804b070 con 0x562add7a0b30 2024-04-24T06:52:28.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.277+0000 7f25b5600640 1 -- 172.21.15.5:0/838721452 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f25c402dae0 con 0x562add7a0b30 2024-04-24T06:52:28.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.277+0000 7f25d5582740 10 client.16902 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:52:28.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.277+0000 7f25d5582740 10 client.16902 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-24T06:52:28.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.277+0000 7f25d5582740 20 client.16902 populate_metadata read hostname 'smithi005' 2024-04-24T06:52:28.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.277+0000 7f25d5582740 1 --2- 172.21.15.5:0/838721452 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x562add8170b0 0x562add8194a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.277+0000 7f25d5582740 4 client.16902 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:52:28.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.277+0000 7f25d5582740 1 -- 172.21.15.5:0/838721452 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562add702a60 con 0x562add8170b0 2024-04-24T06:52:28.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.277+0000 7f25caa00640 1 --2- 172.21.15.5:0/838721452 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x562add8170b0 0x562add8194a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.278+0000 7f25caa00640 1 --2- 172.21.15.5:0/838721452 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x562add8170b0 0x562add8194a0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=892ba68c7cda625c server_cookie=b178a289880e1b7a in_seq=0 out_seq=0 2024-04-24T06:52:28.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.278+0000 7f25b5600640 10 client.16902 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:52:28.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.278+0000 7f25b5600640 1 -- 172.21.15.5:0/838721452 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562add702a60 con 0x562add8170b0 2024-04-24T06:52:28.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.278+0000 7f25b5600640 10 client.16902 handle_command_reply: tid=0 2024-04-24T06:52:28.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.278+0000 7f25d5582740 1 client.16902 shutdown 2024-04-24T06:52:28.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.278+0000 7f25d5582740 1 -- 172.21.15.5:0/838721452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562add7a0b30 msgr2=0x562add79ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:28.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.278+0000 7f25d5582740 1 --2- 172.21.15.5:0/838721452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562add7a0b30 0x562add79ab80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f25c4011910 tx=0x7f25c4009500 comp rx=0 tx=0).stop 2024-04-24T06:52:28.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.278+0000 7f25d5582740 1 -- 172.21.15.5:0/838721452 shutdown_connections 2024-04-24T06:52:28.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.278+0000 7f25d5582740 1 --2- 172.21.15.5:0/838721452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562add7a0b30 0x562add79ab80 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.278+0000 7f25d5582740 1 --2- 172.21.15.5:0/838721452 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x562add8170b0 0x562add8194a0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.279+0000 7f25d5582740 1 --2- 172.21.15.5:0/838721452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562add796460 0x562add7a92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.279+0000 7f25d5582740 1 --2- 172.21.15.5:0/838721452 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562add795b40 0x562add7a8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.279+0000 7f25b5600640 0 client.16902 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:52:28.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.279+0000 7f25d5582740 1 -- 172.21.15.5:0/838721452 >> 172.21.15.5:0/838721452 conn(0x562add791e60 msgr2=0x562add79e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:52:28.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.279+0000 7f25d5582740 1 -- 172.21.15.5:0/838721452 shutdown_connections 2024-04-24T06:52:28.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.279+0000 7f25d5582740 1 -- 172.21.15.5:0/838721452 wait complete. 2024-04-24T06:52:28.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.279+0000 7f25d5582740 20 client.16902 trim_cache size 0 max 16384 2024-04-24T06:52:28.287 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:52:28.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.279+0000 7f25d1200640 1 -- 172.21.15.5:0/2735508875 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f25b00a3da0 msgr2=0x7f25b00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:28.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.279+0000 7f25d1200640 1 --2- 172.21.15.5:0/2735508875 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f25b00a3da0 0x7f25b00a6260 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f25bc0ce390 tx=0x7f25bc0d46f0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.279+0000 7f25d1200640 1 -- 172.21.15.5:0/2735508875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25cc102fe0 msgr2=0x7f25cc1a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:28.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.279+0000 7f25d1200640 1 --2- 172.21.15.5:0/2735508875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25cc102fe0 0x7f25cc1a0650 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f25bc00e9f0 tx=0x7f25bc00edf0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.280+0000 7f25d1200640 1 -- 172.21.15.5:0/2735508875 shutdown_connections 2024-04-24T06:52:28.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.280+0000 7f25d1200640 1 --2- 172.21.15.5:0/2735508875 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f25b00a3da0 0x7f25b00a6260 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.280+0000 7f25d1200640 1 --2- 172.21.15.5:0/2735508875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25cc10c400 0x7f25cc19a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.280+0000 7f25d1200640 1 --2- 172.21.15.5:0/2735508875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25cc103900 0x7f25cc1a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.280+0000 7f25d1200640 1 --2- 172.21.15.5:0/2735508875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25cc102fe0 0x7f25cc1a0650 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.280+0000 7f25d1200640 1 -- 172.21.15.5:0/2735508875 >> 172.21.15.5:0/2735508875 conn(0x7f25cc077ab0 msgr2=0x7f25cc0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:52:28.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.280+0000 7f25d1200640 1 -- 172.21.15.5:0/2735508875 shutdown_connections 2024-04-24T06:52:28.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.280+0000 7f25d1200640 1 -- 172.21.15.5:0/2735508875 wait complete. 2024-04-24T06:52:28.300 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T06:52:28.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.512+0000 7f7363600640 1 -- 172.21.15.5:0/461097422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f735c109510 msgr2=0x7f735c1110e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:28.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.512+0000 7f7363600640 1 --2- 172.21.15.5:0/461097422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f735c109510 0x7f735c1110e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7354009ca0 tx=0x7f735402a8c0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.513+0000 7f7363600640 1 -- 172.21.15.5:0/461097422 shutdown_connections 2024-04-24T06:52:28.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.513+0000 7f7363600640 1 --2- 172.21.15.5:0/461097422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f735c109510 0x7f735c1110e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.513+0000 7f7363600640 1 --2- 172.21.15.5:0/461097422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f735c104df0 0x7f735c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.513+0000 7f7363600640 1 --2- 172.21.15.5:0/461097422 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f735c104440 0x7f735c104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.513+0000 7f7363600640 1 -- 172.21.15.5:0/461097422 >> 172.21.15.5:0/461097422 conn(0x7f735c0ffc80 msgr2=0x7f735c1020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:52:28.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.513+0000 7f7363600640 1 -- 172.21.15.5:0/461097422 shutdown_connections 2024-04-24T06:52:28.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.513+0000 7f7363600640 1 -- 172.21.15.5:0/461097422 wait complete. 2024-04-24T06:52:28.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.514+0000 7f7363600640 1 Processor -- start 2024-04-24T06:52:28.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.514+0000 7f7363600640 1 -- start start 2024-04-24T06:52:28.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.514+0000 7f7363600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f735c104440 0x7f735c1a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.514+0000 7f7363600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f735c109510 0x7f735c1a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.514+0000 7f7363600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f735c19a8b0 0x7f735c19ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.514+0000 7f7363600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f735c1145b0 con 0x7f735c104440 2024-04-24T06:52:28.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.514+0000 7f7363600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f735c114430 con 0x7f735c19a8b0 2024-04-24T06:52:28.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.514+0000 7f7363600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f735c114730 con 0x7f735c109510 2024-04-24T06:52:28.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.515+0000 7f7360c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f735c104440 0x7f735c1a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.515+0000 7f735be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f735c109510 0x7f735c1a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.515+0000 7f7360c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f735c104440 0x7f735c1a0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38148/0 (socket says 172.21.15.5:38148) 2024-04-24T06:52:28.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.515+0000 7f7360c00640 1 -- 172.21.15.5:0/768807491 learned_addr learned my addr 172.21.15.5:0/768807491 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:52:28.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.515+0000 7f7361600640 1 --2- 172.21.15.5:0/768807491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f735c19a8b0 0x7f735c19ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.515+0000 7f735be00640 1 -- 172.21.15.5:0/768807491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f735c19a8b0 msgr2=0x7f735c19ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:28.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.515+0000 7f735be00640 1 --2- 172.21.15.5:0/768807491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f735c19a8b0 0x7f735c19ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.515+0000 7f735be00640 1 -- 172.21.15.5:0/768807491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f735c104440 msgr2=0x7f735c1a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:28.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.515+0000 7f735be00640 1 --2- 172.21.15.5:0/768807491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f735c104440 0x7f735c1a0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.515+0000 7f735be00640 1 -- 172.21.15.5:0/768807491 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f735c19b3d0 con 0x7f735c109510 2024-04-24T06:52:28.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.516+0000 7f735be00640 1 --2- 172.21.15.5:0/768807491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f735c109510 0x7f735c1a0c40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f734800e9c0 tx=0x7f734800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:52:28.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.516+0000 7f7359600640 1 -- 172.21.15.5:0/768807491 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734800ccc0 con 0x7f735c109510 2024-04-24T06:52:28.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.516+0000 7f7359600640 1 -- 172.21.15.5:0/768807491 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7348010910 con 0x7f735c109510 2024-04-24T06:52:28.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.516+0000 7f7359600640 1 -- 172.21.15.5:0/768807491 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73480188c0 con 0x7f735c109510 2024-04-24T06:52:28.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.516+0000 7f7363600640 1 -- 172.21.15.5:0/768807491 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f735c19b6c0 con 0x7f735c109510 2024-04-24T06:52:28.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.516+0000 7f7363600640 1 -- 172.21.15.5:0/768807491 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f735c1a7960 con 0x7f735c109510 2024-04-24T06:52:28.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.519+0000 7f73665b0740 1 Processor -- start 2024-04-24T06:52:28.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.519+0000 7f73665b0740 1 -- start start 2024-04-24T06:52:28.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.519+0000 7f7359600640 1 -- 172.21.15.5:0/768807491 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f7348010ac0 con 0x7f735c109510 2024-04-24T06:52:28.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.520+0000 7f73665b0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596286ac950 0x5596286acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.520+0000 7f73665b0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596286ad270 0x5596286b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.520+0000 7f73665b0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596286b7940 0x5596286b9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.520+0000 7f73665b0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5596286121d0 con 0x5596286ad270 2024-04-24T06:52:28.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.520+0000 7f73665b0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5596285b8450 con 0x5596286b7940 2024-04-24T06:52:28.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.520+0000 7f73665b0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55962860e190 con 0x5596286ac950 2024-04-24T06:52:28.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.520+0000 7f7360c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596286ad270 0x5596286b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.520+0000 7f7361600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596286ac950 0x5596286acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.520+0000 7f7360c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596286ad270 0x5596286b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38150/0 (socket says 172.21.15.5:38150) 2024-04-24T06:52:28.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.520+0000 7f7360c00640 1 -- 172.21.15.5:0/2500630509 learned_addr learned my addr 172.21.15.5:0/2500630509 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:52:28.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.520+0000 7f735be00640 1 --2- 172.21.15.5:0/2500630509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596286b7940 0x5596286b9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.521+0000 7f7360c00640 1 -- 172.21.15.5:0/2500630509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596286ac950 msgr2=0x5596286acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:28.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.521+0000 7f7360c00640 1 --2- 172.21.15.5:0/2500630509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596286ac950 0x5596286acd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.521+0000 7f7360c00640 1 -- 172.21.15.5:0/2500630509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596286b7940 msgr2=0x5596286b9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:28.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.521+0000 7f7360c00640 1 --2- 172.21.15.5:0/2500630509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596286b7940 0x5596286b9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.521+0000 7f7360c00640 1 -- 172.21.15.5:0/2500630509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559628675310 con 0x5596286ad270 2024-04-24T06:52:28.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.521+0000 7f7361600640 1 --2- 172.21.15.5:0/2500630509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596286ac950 0x5596286acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:52:28.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.521+0000 7f7360c00640 1 --2- 172.21.15.5:0/2500630509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596286ad270 0x5596286b7400 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7350009d70 tx=0x7f7350011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d5ccd9cfa48f37 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:52:28.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.521+0000 7f734dc00640 1 -- 172.21.15.5:0/2500630509 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735001e070 con 0x5596286ad270 2024-04-24T06:52:28.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.521+0000 7f7359600640 1 --2- 172.21.15.5:0/768807491 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f73440a3da0 0x7f73440a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.522+0000 7f734dc00640 1 -- 172.21.15.5:0/2500630509 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7350019b10 con 0x5596286ad270 2024-04-24T06:52:28.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.522+0000 7f734dc00640 1 -- 172.21.15.5:0/2500630509 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735000ebb0 con 0x5596286ad270 2024-04-24T06:52:28.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.522+0000 7f7359600640 1 -- 172.21.15.5:0/768807491 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f73480d0af0 con 0x7f735c109510 2024-04-24T06:52:28.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.522+0000 7f7361600640 1 --2- 172.21.15.5:0/768807491 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f73440a3da0 0x7f73440a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.522+0000 7f73665b0740 1 -- 172.21.15.5:0/2500630509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596286ad270 msgr2=0x5596286b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:28.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.522+0000 7f73665b0740 1 --2- 172.21.15.5:0/2500630509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596286ad270 0x5596286b7400 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7350009d70 tx=0x7f7350011810 comp rx=0 tx=0).stop 2024-04-24T06:52:28.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.522+0000 7f73665b0740 1 -- 172.21.15.5:0/2500630509 shutdown_connections 2024-04-24T06:52:28.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.522+0000 7f73665b0740 1 --2- 172.21.15.5:0/2500630509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596286b7940 0x5596286b9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.522+0000 7f73665b0740 1 --2- 172.21.15.5:0/2500630509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596286ad270 0x5596286b7400 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.522+0000 7f73665b0740 1 --2- 172.21.15.5:0/2500630509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596286ac950 0x5596286acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.522+0000 7f73665b0740 1 -- 172.21.15.5:0/2500630509 >> 172.21.15.5:0/2500630509 conn(0x5596286a8c70 msgr2=0x5596286a9050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:52:28.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.523+0000 7f7361600640 1 --2- 172.21.15.5:0/768807491 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f73440a3da0 0x7f73440a6260 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7354004af0 tx=0x7f7354004a80 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:52:28.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.523+0000 7f73665b0740 1 -- 172.21.15.5:0/2500630509 shutdown_connections 2024-04-24T06:52:28.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.523+0000 7f73665b0740 1 -- 172.21.15.5:0/2500630509 wait complete. 2024-04-24T06:52:28.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.523+0000 7f73665b0740 1 -- start start 2024-04-24T06:52:28.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.523+0000 7f73665b0740 1 Processor -- start 2024-04-24T06:52:28.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.524+0000 7f73665b0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596286ac950 0x5596286b7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.524+0000 7f73665b0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596286ad270 0x5596286b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.524+0000 7f73665b0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596286b7940 0x5596286b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.524+0000 7f73665b0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5596286ba7d0 con 0x5596286ad270 2024-04-24T06:52:28.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.524+0000 7f73665b0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5596285cb810 con 0x5596286b7940 2024-04-24T06:52:28.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.524+0000 7f73665b0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5596286ba950 con 0x5596286ac950 2024-04-24T06:52:28.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.524+0000 7f73665b0740 10 client.0 _ll_register_callbacks cb 0x559628599140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:52:28.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.524+0000 7f7360c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596286ac950 0x5596286b7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.524+0000 7f735be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596286ad270 0x5596286b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.524+0000 7f7361600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596286b7940 0x5596286b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.524+0000 7f735be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596286ad270 0x5596286b45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38158/0 (socket says 172.21.15.5:38158) 2024-04-24T06:52:28.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.524+0000 7f735be00640 1 -- 172.21.15.5:0/1906153122 learned_addr learned my addr 172.21.15.5:0/1906153122 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:52:28.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.525+0000 7f735be00640 1 -- 172.21.15.5:0/1906153122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596286ac950 msgr2=0x5596286b7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:28.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.525+0000 7f735be00640 1 --2- 172.21.15.5:0/1906153122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596286ac950 0x5596286b7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.525+0000 7f735be00640 1 -- 172.21.15.5:0/1906153122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596286b7940 msgr2=0x5596286b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:28.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.525+0000 7f735be00640 1 --2- 172.21.15.5:0/1906153122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596286b7940 0x5596286b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.525+0000 7f735be00640 1 -- 172.21.15.5:0/1906153122 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596285dc3f0 con 0x5596286ad270 2024-04-24T06:52:28.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.525+0000 7f7361600640 1 --2- 172.21.15.5:0/1906153122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596286b7940 0x5596286b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:52:28.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.525+0000 7f735be00640 1 --2- 172.21.15.5:0/1906153122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596286ad270 0x5596286b45f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f734809ca60 tx=0x7f7348098e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ff31287ded9e06 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:52:28.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.525+0000 7f7342000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:52:28.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.525+0000 7f7342000640 1 -- 172.21.15.5:0/1906153122 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734809c430 con 0x5596286ad270 2024-04-24T06:52:28.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.525+0000 7f7342000640 1 -- 172.21.15.5:0/1906153122 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f73480ce450 con 0x5596286ad270 2024-04-24T06:52:28.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.525+0000 7f73665b0740 1 -- 172.21.15.5:0/1906153122 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7350019b10 con 0x5596286ad270 2024-04-24T06:52:28.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.526+0000 7f7342000640 1 -- 172.21.15.5:0/1906153122 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f73480ce650 con 0x5596286ad270 2024-04-24T06:52:28.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.526+0000 7f73665b0740 10 client.16914 fetch_fsmap learned FSMap version 27 2024-04-24T06:52:28.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.526+0000 7f73665b0740 1 -- 172.21.15.5:0/1906153122 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5596283243f0 con 0x5596286ad270 2024-04-24T06:52:28.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.526+0000 7f7342000640 1 -- 172.21.15.5:0/1906153122 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7350019b10 con 0x5596286ad270 2024-04-24T06:52:28.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.526+0000 7f7342000640 1 -- 172.21.15.5:0/1906153122 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f734809bf20 con 0x5596286ad270 2024-04-24T06:52:28.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.526+0000 7f73665b0740 10 client.16914 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:52:28.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.526+0000 7f73665b0740 10 client.16914 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T06:52:28.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.526+0000 7f73665b0740 20 client.16914 populate_metadata read hostname 'smithi005' 2024-04-24T06:52:28.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.526+0000 7f73665b0740 1 --2- 172.21.15.5:0/1906153122 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x5596286c7b40 0x5596286c9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.526+0000 7f73665b0740 4 client.16914 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:52:28.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.526+0000 7f73665b0740 1 -- 172.21.15.5:0/1906153122 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55962861fbf0 con 0x5596286c7b40 2024-04-24T06:52:28.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.527+0000 7f7360c00640 1 --2- 172.21.15.5:0/1906153122 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x5596286c7b40 0x5596286c9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.527+0000 7f7360c00640 1 --2- 172.21.15.5:0/1906153122 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x5596286c7b40 0x5596286c9f30 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f36f0242ac427a8f server_cookie=6788c82b6bf9d185 in_seq=0 out_seq=0 2024-04-24T06:52:28.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.527+0000 7f7342000640 10 client.16914 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:52:28.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.528+0000 7f7342000640 1 -- 172.21.15.5:0/1906153122 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55962861fbf0 con 0x5596286c7b40 2024-04-24T06:52:28.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.528+0000 7f7342000640 10 client.16914 handle_command_reply: tid=0 2024-04-24T06:52:28.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.528+0000 7f73665b0740 1 client.16914 shutdown 2024-04-24T06:52:28.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.529+0000 7f73665b0740 1 -- 172.21.15.5:0/1906153122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596286ad270 msgr2=0x5596286b45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:28.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.529+0000 7f73665b0740 1 --2- 172.21.15.5:0/1906153122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596286ad270 0x5596286b45f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f734809ca60 tx=0x7f7348098e30 comp rx=0 tx=0).stop 2024-04-24T06:52:28.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.529+0000 7f73665b0740 1 -- 172.21.15.5:0/1906153122 shutdown_connections 2024-04-24T06:52:28.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.529+0000 7f73665b0740 1 --2- 172.21.15.5:0/1906153122 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x5596286c7b40 0x5596286c9f30 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.529+0000 7f73665b0740 1 --2- 172.21.15.5:0/1906153122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596286b7940 0x5596286b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.529+0000 7f73665b0740 1 --2- 172.21.15.5:0/1906153122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596286ad270 0x5596286b45f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.529+0000 7f7342000640 0 client.16914 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:52:28.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.529+0000 7f73665b0740 1 --2- 172.21.15.5:0/1906153122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596286ac950 0x5596286b7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.529+0000 7f73665b0740 1 -- 172.21.15.5:0/1906153122 >> 172.21.15.5:0/1906153122 conn(0x5596286a8c70 msgr2=0x5596286aea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:52:28.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.529+0000 7f73665b0740 1 -- 172.21.15.5:0/1906153122 shutdown_connections 2024-04-24T06:52:28.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.529+0000 7f73665b0740 1 -- 172.21.15.5:0/1906153122 wait complete. 2024-04-24T06:52:28.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.529+0000 7f73665b0740 20 client.16914 trim_cache size 0 max 16384 2024-04-24T06:52:28.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.544+0000 7f73665b0740 1 Processor -- start 2024-04-24T06:52:28.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.544+0000 7f73665b0740 1 -- start start 2024-04-24T06:52:28.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.544+0000 7f73665b0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596286ac950 0x5596286c3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.544+0000 7f73665b0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596286ad270 0x5596286cf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.544+0000 7f73665b0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596286b7940 0x5596286cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.545+0000 7f73665b0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f735c1b2950 con 0x5596286ad270 2024-04-24T06:52:28.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.545+0000 7f73665b0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f735c1b0220 con 0x5596286b7940 2024-04-24T06:52:28.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.545+0000 7f73665b0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f735c1ae090 con 0x5596286ac950 2024-04-24T06:52:28.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.545+0000 7f735be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596286ad270 0x5596286cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.545+0000 7f735be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596286ad270 0x5596286cf760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38164/0 (socket says 172.21.15.5:38164) 2024-04-24T06:52:28.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.545+0000 7f735be00640 1 -- 172.21.15.5:0/790803440 learned_addr learned my addr 172.21.15.5:0/790803440 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:52:28.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.545+0000 7f7361600640 1 --2- 172.21.15.5:0/790803440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596286b7940 0x5596286cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.545+0000 7f7360c00640 1 --2- 172.21.15.5:0/790803440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596286ac950 0x5596286c3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.545+0000 7f735be00640 1 -- 172.21.15.5:0/790803440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596286ac950 msgr2=0x5596286c3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:28.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.545+0000 7f735be00640 1 --2- 172.21.15.5:0/790803440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596286ac950 0x5596286c3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.545+0000 7f735be00640 1 -- 172.21.15.5:0/790803440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596286b7940 msgr2=0x5596286cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:28.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.545+0000 7f735be00640 1 --2- 172.21.15.5:0/790803440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596286b7940 0x5596286cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.545+0000 7f735be00640 1 -- 172.21.15.5:0/790803440 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596283ee7d0 con 0x5596286ad270 2024-04-24T06:52:28.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.545+0000 7f7361600640 1 --2- 172.21.15.5:0/790803440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596286b7940 0x5596286cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:52:28.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.546+0000 7f735be00640 1 --2- 172.21.15.5:0/790803440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596286ad270 0x5596286cf760 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f734809ad80 tx=0x7f734809afe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e146af08ca03ab2b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:52:28.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.546+0000 7f734d200640 1 -- 172.21.15.5:0/790803440 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73480ce650 con 0x5596286ad270 2024-04-24T06:52:28.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.546+0000 7f734d200640 1 -- 172.21.15.5:0/790803440 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73480bbbe0 con 0x5596286ad270 2024-04-24T06:52:28.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.546+0000 7f734d200640 1 -- 172.21.15.5:0/790803440 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734809c120 con 0x5596286ad270 2024-04-24T06:52:28.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.547+0000 7f73665b0740 1 -- 172.21.15.5:0/790803440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596286ad270 msgr2=0x5596286cf760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:28.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.547+0000 7f73665b0740 1 --2- 172.21.15.5:0/790803440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596286ad270 0x5596286cf760 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f734809ad80 tx=0x7f734809afe0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.547+0000 7f73665b0740 1 -- 172.21.15.5:0/790803440 shutdown_connections 2024-04-24T06:52:28.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.547+0000 7f73665b0740 1 --2- 172.21.15.5:0/790803440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596286b7940 0x5596286cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.547+0000 7f73665b0740 1 --2- 172.21.15.5:0/790803440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596286ad270 0x5596286cf760 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.547+0000 7f73665b0740 1 --2- 172.21.15.5:0/790803440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596286ac950 0x5596286c3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.547+0000 7f73665b0740 1 -- 172.21.15.5:0/790803440 >> 172.21.15.5:0/790803440 conn(0x5596286a8c70 msgr2=0x5596286af8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:52:28.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.547+0000 7f73665b0740 1 -- 172.21.15.5:0/790803440 shutdown_connections 2024-04-24T06:52:28.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.547+0000 7f73665b0740 1 -- 172.21.15.5:0/790803440 wait complete. 2024-04-24T06:52:28.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.547+0000 7f73665b0740 1 -- start start 2024-04-24T06:52:28.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.547+0000 7f73665b0740 1 Processor -- start 2024-04-24T06:52:28.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.548+0000 7f73665b0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596286ac950 0x5596286bfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.548+0000 7f73665b0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596286ad270 0x5596286b8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.548+0000 7f73665b0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596286b7940 0x5596286b87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.548+0000 7f73665b0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f735c1ae6e0 con 0x5596286ac950 2024-04-24T06:52:28.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.548+0000 7f73665b0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f735c1afb50 con 0x5596286b7940 2024-04-24T06:52:28.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.548+0000 7f73665b0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f735c1aedb0 con 0x5596286ad270 2024-04-24T06:52:28.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.548+0000 7f73665b0740 10 client.0 _ll_register_callbacks cb 0x7f7348010e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:52:28.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.548+0000 7f7360c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596286ac950 0x5596286bfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.548+0000 7f7360c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596286ac950 0x5596286bfef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38180/0 (socket says 172.21.15.5:38180) 2024-04-24T06:52:28.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.548+0000 7f7361600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596286b7940 0x5596286b87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.548+0000 7f7360c00640 1 -- 172.21.15.5:0/635900017 learned_addr learned my addr 172.21.15.5:0/635900017 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:52:28.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.548+0000 7f735be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596286ad270 0x5596286b8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.549+0000 7f7361600640 1 -- 172.21.15.5:0/635900017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596286ad270 msgr2=0x5596286b8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:28.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.549+0000 7f7361600640 1 --2- 172.21.15.5:0/635900017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596286ad270 0x5596286b8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.549+0000 7f7361600640 1 -- 172.21.15.5:0/635900017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596286ac950 msgr2=0x5596286bfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:28.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.549+0000 7f7361600640 1 --2- 172.21.15.5:0/635900017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596286ac950 0x5596286bfef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.549+0000 7f7361600640 1 -- 172.21.15.5:0/635900017 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596286b8ff0 con 0x5596286b7940 2024-04-24T06:52:28.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.549+0000 7f735be00640 1 --2- 172.21.15.5:0/635900017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596286ad270 0x5596286b8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:52:28.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.549+0000 7f7360c00640 1 --2- 172.21.15.5:0/635900017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596286ac950 0x5596286bfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:52:28.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.549+0000 7f7361600640 1 --2- 172.21.15.5:0/635900017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596286b7940 0x5596286b87c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7354006010 tx=0x7f73540095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=552580f1cdd479e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:52:28.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.549+0000 7f7342000640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:52:28.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.549+0000 7f7342000640 1 -- 172.21.15.5:0/635900017 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7354059060 con 0x5596286b7940 2024-04-24T06:52:28.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.549+0000 7f73665b0740 1 -- 172.21.15.5:0/635900017 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73480bbbe0 con 0x5596286b7940 2024-04-24T06:52:28.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.549+0000 7f7342000640 1 -- 172.21.15.5:0/635900017 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f7354004590 con 0x5596286b7940 2024-04-24T06:52:28.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.549+0000 7f7342000640 1 -- 172.21.15.5:0/635900017 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7354004790 con 0x5596286b7940 2024-04-24T06:52:28.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.550+0000 7f73665b0740 10 client.27065 fetch_fsmap learned FSMap version 27 2024-04-24T06:52:28.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.550+0000 7f73665b0740 1 -- 172.21.15.5:0/635900017 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55962872dfd0 con 0x5596286b7940 2024-04-24T06:52:28.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.550+0000 7f7342000640 1 -- 172.21.15.5:0/635900017 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73480bbbe0 con 0x5596286b7940 2024-04-24T06:52:28.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.550+0000 7f7342000640 1 -- 172.21.15.5:0/635900017 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f7354030780 con 0x5596286b7940 2024-04-24T06:52:28.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.550+0000 7f73665b0740 10 client.27065 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:52:28.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.550+0000 7f73665b0740 10 client.27065 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T06:52:28.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.550+0000 7f73665b0740 20 client.27065 populate_metadata read hostname 'smithi005' 2024-04-24T06:52:28.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.550+0000 7f73665b0740 1 --2- 172.21.15.5:0/635900017 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55962872e6a0 0x559628730a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.550+0000 7f73665b0740 4 client.27065 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:52:28.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.550+0000 7f73665b0740 1 -- 172.21.15.5:0/635900017 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559628619a60 con 0x55962872e6a0 2024-04-24T06:52:28.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.551+0000 7f7360c00640 1 --2- 172.21.15.5:0/635900017 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55962872e6a0 0x559628730a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.551+0000 7f7360c00640 1 --2- 172.21.15.5:0/635900017 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55962872e6a0 0x559628730a90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=19a86a0918f19d92 server_cookie=1a856a7f70106436 in_seq=0 out_seq=0 2024-04-24T06:52:28.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.551+0000 7f7342000640 10 client.27065 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:52:28.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.551+0000 7f7342000640 1 -- 172.21.15.5:0/635900017 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559628619a60 con 0x55962872e6a0 2024-04-24T06:52:28.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.551+0000 7f7342000640 10 client.27065 handle_command_reply: tid=0 2024-04-24T06:52:28.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.551+0000 7f73665b0740 1 client.27065 shutdown 2024-04-24T06:52:28.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.551+0000 7f73665b0740 1 -- 172.21.15.5:0/635900017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596286b7940 msgr2=0x5596286b87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:28.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.551+0000 7f73665b0740 1 --2- 172.21.15.5:0/635900017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596286b7940 0x5596286b87c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7354006010 tx=0x7f73540095c0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.552+0000 7f73665b0740 1 -- 172.21.15.5:0/635900017 shutdown_connections 2024-04-24T06:52:28.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.552+0000 7f73665b0740 1 --2- 172.21.15.5:0/635900017 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55962872e6a0 0x559628730a90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.552+0000 7f73665b0740 1 --2- 172.21.15.5:0/635900017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596286b7940 0x5596286b87c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.552+0000 7f73665b0740 1 --2- 172.21.15.5:0/635900017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596286ad270 0x5596286b8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.552+0000 7f7342000640 0 client.27065 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:52:28.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.552+0000 7f73665b0740 1 --2- 172.21.15.5:0/635900017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596286ac950 0x5596286bfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.552+0000 7f73665b0740 1 -- 172.21.15.5:0/635900017 >> 172.21.15.5:0/635900017 conn(0x5596286a8c70 msgr2=0x5596286b2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:52:28.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.552+0000 7f73665b0740 1 -- 172.21.15.5:0/635900017 shutdown_connections 2024-04-24T06:52:28.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.552+0000 7f73665b0740 1 -- 172.21.15.5:0/635900017 wait complete. 2024-04-24T06:52:28.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.552+0000 7f73665b0740 20 client.27065 trim_cache size 0 max 16384 2024-04-24T06:52:28.561 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:52:28.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.553+0000 7f7363600640 1 -- 172.21.15.5:0/768807491 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f73440a3da0 msgr2=0x7f73440a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:28.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.553+0000 7f7363600640 1 --2- 172.21.15.5:0/768807491 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f73440a3da0 0x7f73440a6260 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7354004af0 tx=0x7f7354004a80 comp rx=0 tx=0).stop 2024-04-24T06:52:28.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.553+0000 7f7363600640 1 -- 172.21.15.5:0/768807491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f735c109510 msgr2=0x7f735c1a0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:28.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.553+0000 7f7363600640 1 --2- 172.21.15.5:0/768807491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f735c109510 0x7f735c1a0c40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f734800e9c0 tx=0x7f734800ee80 comp rx=0 tx=0).stop 2024-04-24T06:52:28.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.553+0000 7f7363600640 1 -- 172.21.15.5:0/768807491 shutdown_connections 2024-04-24T06:52:28.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.553+0000 7f7363600640 1 --2- 172.21.15.5:0/768807491 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f73440a3da0 0x7f73440a6260 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.553+0000 7f7363600640 1 --2- 172.21.15.5:0/768807491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f735c19a8b0 0x7f735c19ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.553+0000 7f7363600640 1 --2- 172.21.15.5:0/768807491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f735c109510 0x7f735c1a0c40 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.553+0000 7f7363600640 1 --2- 172.21.15.5:0/768807491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f735c104440 0x7f735c1a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.553+0000 7f7363600640 1 -- 172.21.15.5:0/768807491 >> 172.21.15.5:0/768807491 conn(0x7f735c0ffc80 msgr2=0x7f735c101f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:52:28.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.553+0000 7f7363600640 1 -- 172.21.15.5:0/768807491 shutdown_connections 2024-04-24T06:52:28.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.553+0000 7f7363600640 1 -- 172.21.15.5:0/768807491 wait complete. 2024-04-24T06:52:28.568 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:28 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1470269803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:52:28.571 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T06:52:28.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:28 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/1470269803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:52:28.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:28 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/1470269803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:52:28.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.783+0000 7fec02400640 1 -- 172.21.15.5:0/389016918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febfc104c10 msgr2=0x7febfc108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:28.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.783+0000 7fec02400640 1 --2- 172.21.15.5:0/389016918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febfc104c10 0x7febfc108c50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7febe4009900 tx=0x7febe402a3f0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.783+0000 7fec02400640 1 -- 172.21.15.5:0/389016918 shutdown_connections 2024-04-24T06:52:28.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.783+0000 7fec02400640 1 --2- 172.21.15.5:0/389016918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febfc109330 0x7febfc110f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.783+0000 7fec02400640 1 --2- 172.21.15.5:0/389016918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febfc104c10 0x7febfc108c50 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.783+0000 7fec02400640 1 --2- 172.21.15.5:0/389016918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febfc104260 0x7febfc104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.783+0000 7fec02400640 1 -- 172.21.15.5:0/389016918 >> 172.21.15.5:0/389016918 conn(0x7febfc0ffaa0 msgr2=0x7febfc101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:52:28.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.783+0000 7fec02400640 1 -- 172.21.15.5:0/389016918 shutdown_connections 2024-04-24T06:52:28.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.783+0000 7fec02400640 1 -- 172.21.15.5:0/389016918 wait complete. 2024-04-24T06:52:28.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.784+0000 7fec02400640 1 Processor -- start 2024-04-24T06:52:28.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.784+0000 7fec02400640 1 -- start start 2024-04-24T06:52:28.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.785+0000 7fec02400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febfc104260 0x7febfc1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.785+0000 7fec02400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febfc109330 0x7febfc1a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.785+0000 7fec02400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febfc19a730 0x7febfc19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.785+0000 7fec02400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7febfc1143f0 con 0x7febfc104260 2024-04-24T06:52:28.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.785+0000 7fec02400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7febfc114270 con 0x7febfc19a730 2024-04-24T06:52:28.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.785+0000 7fec02400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7febfc114570 con 0x7febfc109330 2024-04-24T06:52:28.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.785+0000 7fec01a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febfc19a730 0x7febfc19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.785+0000 7fec01000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febfc104260 0x7febfc1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.785+0000 7fec01a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febfc19a730 0x7febfc19abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:53638/0 (socket says 172.21.15.5:53638) 2024-04-24T06:52:28.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.785+0000 7fec01a00640 1 -- 172.21.15.5:0/281485475 learned_addr learned my addr 172.21.15.5:0/281485475 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:52:28.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.785+0000 7febf3e00640 1 --2- 172.21.15.5:0/281485475 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febfc109330 0x7febfc1a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.785+0000 7fec01000640 1 -- 172.21.15.5:0/281485475 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febfc109330 msgr2=0x7febfc1a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:28.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.785+0000 7fec01000640 1 --2- 172.21.15.5:0/281485475 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febfc109330 0x7febfc1a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.786+0000 7fec01000640 1 -- 172.21.15.5:0/281485475 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febfc19a730 msgr2=0x7febfc19abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:28.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.786+0000 7fec01000640 1 --2- 172.21.15.5:0/281485475 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febfc19a730 0x7febfc19abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.786+0000 7fec01000640 1 -- 172.21.15.5:0/281485475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febfc19b330 con 0x7febfc104260 2024-04-24T06:52:28.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.786+0000 7fec01a00640 1 --2- 172.21.15.5:0/281485475 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febfc19a730 0x7febfc19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:52:28.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.786+0000 7fec01000640 1 --2- 172.21.15.5:0/281485475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febfc104260 0x7febfc1a0670 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7febec00eb00 tx=0x7febec00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:52:28.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.786+0000 7febfa000640 1 -- 172.21.15.5:0/281485475 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febec010430 con 0x7febfc104260 2024-04-24T06:52:28.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.786+0000 7fec02400640 1 -- 172.21.15.5:0/281485475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febfc1a7410 con 0x7febfc104260 2024-04-24T06:52:28.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.786+0000 7febfa000640 1 -- 172.21.15.5:0/281485475 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febec010ab0 con 0x7febfc104260 2024-04-24T06:52:28.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.786+0000 7febfa000640 1 -- 172.21.15.5:0/281485475 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febec003b70 con 0x7febfc104260 2024-04-24T06:52:28.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.786+0000 7fec02400640 1 -- 172.21.15.5:0/281485475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febfc1a77c0 con 0x7febfc104260 2024-04-24T06:52:28.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.790+0000 7fec06808740 1 Processor -- start 2024-04-24T06:52:28.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.790+0000 7febfa000640 1 -- 172.21.15.5:0/281485475 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7febec010c60 con 0x7febfc104260 2024-04-24T06:52:28.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.790+0000 7fec06808740 1 -- start start 2024-04-24T06:52:28.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.790+0000 7fec06808740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b1da4b40 0x5601b1da4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.790+0000 7fec06808740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601b1da5460 0x5601b1daf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.790+0000 7fec06808740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601b1dafb30 0x5601b1db1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.790+0000 7fec06808740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5601b1d0a1d0 con 0x5601b1dafb30 2024-04-24T06:52:28.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.790+0000 7fec06808740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5601b1cb0450 con 0x5601b1da5460 2024-04-24T06:52:28.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.790+0000 7fec06808740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5601b1d06190 con 0x5601b1da4b40 2024-04-24T06:52:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.790+0000 7fec01000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601b1da5460 0x5601b1daf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.791+0000 7fec01a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b1da4b40 0x5601b1da4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.791+0000 7febf3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601b1dafb30 0x5601b1db1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.791+0000 7fec01a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b1da4b40 0x5601b1da4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:39044/0 (socket says 172.21.15.5:39044) 2024-04-24T06:52:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.791+0000 7fec01a00640 1 -- 172.21.15.5:0/1222262464 learned_addr learned my addr 172.21.15.5:0/1222262464 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:52:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.791+0000 7fec01a00640 1 -- 172.21.15.5:0/1222262464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601b1da5460 msgr2=0x5601b1daf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.791+0000 7fec01a00640 1 --2- 172.21.15.5:0/1222262464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601b1da5460 0x5601b1daf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.791+0000 7fec01a00640 1 -- 172.21.15.5:0/1222262464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601b1dafb30 msgr2=0x5601b1db1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.791+0000 7fec01a00640 1 --2- 172.21.15.5:0/1222262464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601b1dafb30 0x5601b1db1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.791+0000 7fec01a00640 1 -- 172.21.15.5:0/1222262464 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601b1d6d310 con 0x5601b1da4b40 2024-04-24T06:52:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.791+0000 7fec01a00640 1 --2- 172.21.15.5:0/1222262464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b1da4b40 0x5601b1da4f20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7febf4011570 tx=0x7febf4011a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7f741b5341caf74 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:52:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.791+0000 7febfa000640 1 --2- 172.21.15.5:0/281485475 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7febe00a3d20 0x7febe00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.792+0000 7febf2000640 1 -- 172.21.15.5:0/1222262464 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf401e070 con 0x5601b1da4b40 2024-04-24T06:52:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.792+0000 7febfa000640 1 -- 172.21.15.5:0/281485475 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7febec0d1180 con 0x7febfc104260 2024-04-24T06:52:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.792+0000 7fec01000640 1 --2- 172.21.15.5:0/281485475 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7febe00a3d20 0x7febe00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.792+0000 7febf2000640 1 -- 172.21.15.5:0/1222262464 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febf400fb20 con 0x5601b1da4b40 2024-04-24T06:52:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.792+0000 7febf2000640 1 -- 172.21.15.5:0/1222262464 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf4019c70 con 0x5601b1da4b40 2024-04-24T06:52:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.793+0000 7fec06808740 1 -- 172.21.15.5:0/1222262464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b1da4b40 msgr2=0x5601b1da4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.793+0000 7fec01000640 1 --2- 172.21.15.5:0/281485475 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7febe00a3d20 0x7febe00a61e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x5601b1da6100 tx=0x7febec0d4c30 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:52:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.793+0000 7fec06808740 1 --2- 172.21.15.5:0/1222262464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b1da4b40 0x5601b1da4f20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7febf4011570 tx=0x7febf4011a30 comp rx=0 tx=0).stop 2024-04-24T06:52:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.793+0000 7fec06808740 1 -- 172.21.15.5:0/1222262464 shutdown_connections 2024-04-24T06:52:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.793+0000 7fec06808740 1 --2- 172.21.15.5:0/1222262464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601b1dafb30 0x5601b1db1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.793+0000 7fec06808740 1 --2- 172.21.15.5:0/1222262464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601b1da5460 0x5601b1daf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.793+0000 7fec06808740 1 --2- 172.21.15.5:0/1222262464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b1da4b40 0x5601b1da4f20 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.793+0000 7fec06808740 1 -- 172.21.15.5:0/1222262464 >> 172.21.15.5:0/1222262464 conn(0x5601b1da0e60 msgr2=0x5601b1da1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:52:28.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.793+0000 7fec06808740 1 -- 172.21.15.5:0/1222262464 shutdown_connections 2024-04-24T06:52:28.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.793+0000 7fec06808740 1 -- 172.21.15.5:0/1222262464 wait complete. 2024-04-24T06:52:28.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.793+0000 7fec06808740 1 -- start start 2024-04-24T06:52:28.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.793+0000 7fec06808740 1 Processor -- start 2024-04-24T06:52:28.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.794+0000 7fec06808740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b1da4b40 0x5601b1daf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.794+0000 7fec06808740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601b1da5460 0x5601b1dac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.794+0000 7fec06808740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601b1dafb30 0x5601b1dacfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.794+0000 7fec06808740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5601b1db29c0 con 0x5601b1da5460 2024-04-24T06:52:28.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.794+0000 7fec06808740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5601b1cc3810 con 0x5601b1dafb30 2024-04-24T06:52:28.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.794+0000 7fec06808740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5601b1db2b40 con 0x5601b1da4b40 2024-04-24T06:52:28.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.794+0000 7fec06808740 10 client.0 _ll_register_callbacks cb 0x5601b1c91140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:52:28.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.794+0000 7fec01a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b1da4b40 0x5601b1daf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.794+0000 7febf3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601b1da5460 0x5601b1dac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.794+0000 7fec01a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b1da4b40 0x5601b1daf5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:39060/0 (socket says 172.21.15.5:39060) 2024-04-24T06:52:28.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.794+0000 7fec01a00640 1 -- 172.21.15.5:0/1588212823 learned_addr learned my addr 172.21.15.5:0/1588212823 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:52:28.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.794+0000 7fec01a00640 1 --2- 172.21.15.5:0/1588212823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601b1dafb30 0x5601b1dacfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.795+0000 7fec01a00640 1 -- 172.21.15.5:0/1588212823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601b1dafb30 msgr2=0x5601b1dacfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:28.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.795+0000 7fec01a00640 1 --2- 172.21.15.5:0/1588212823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601b1dafb30 0x5601b1dacfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.795+0000 7fec01a00640 1 -- 172.21.15.5:0/1588212823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601b1da5460 msgr2=0x5601b1dac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:28.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.795+0000 7fec01a00640 1 --2- 172.21.15.5:0/1588212823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601b1da5460 0x5601b1dac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.795+0000 7fec01a00640 1 -- 172.21.15.5:0/1588212823 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601b1cd43f0 con 0x5601b1da4b40 2024-04-24T06:52:28.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.795+0000 7febf3e00640 1 --2- 172.21.15.5:0/1588212823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601b1da5460 0x5601b1dac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:52:28.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.795+0000 7fec01a00640 1 --2- 172.21.15.5:0/1588212823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b1da4b40 0x5601b1daf5c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7febf4011540 tx=0x7febf40198f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f3f9e6856750e7a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:52:28.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.795+0000 7febeaa00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:52:28.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.795+0000 7febeaa00640 1 -- 172.21.15.5:0/1588212823 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf401e070 con 0x5601b1da4b40 2024-04-24T06:52:28.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.795+0000 7febeaa00640 1 -- 172.21.15.5:0/1588212823 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7febf40199f0 con 0x5601b1da4b40 2024-04-24T06:52:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.795+0000 7fec06808740 1 -- 172.21.15.5:0/1588212823 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febf400fb20 con 0x5601b1da4b40 2024-04-24T06:52:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.796+0000 7febeaa00640 1 -- 172.21.15.5:0/1588212823 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7febf4019bf0 con 0x5601b1da4b40 2024-04-24T06:52:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.796+0000 7fec06808740 10 client.27091 fetch_fsmap learned FSMap version 27 2024-04-24T06:52:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.796+0000 7fec06808740 1 -- 172.21.15.5:0/1588212823 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5601b1a1c3f0 con 0x5601b1da4b40 2024-04-24T06:52:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.796+0000 7febeaa00640 1 -- 172.21.15.5:0/1588212823 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febf400fb20 con 0x5601b1da4b40 2024-04-24T06:52:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.796+0000 7febeaa00640 1 -- 172.21.15.5:0/1588212823 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7febf40049a0 con 0x5601b1da4b40 2024-04-24T06:52:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.796+0000 7fec06808740 10 client.27091 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:52:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.796+0000 7fec06808740 10 client.27091 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:52:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.796+0000 7fec06808740 20 client.27091 populate_metadata read hostname 'smithi005' 2024-04-24T06:52:28.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.796+0000 7fec06808740 1 --2- 172.21.15.5:0/1588212823 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5601b1dbfc30 0x5601b1dc2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.796+0000 7fec06808740 4 client.27091 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:52:28.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.796+0000 7fec06808740 1 -- 172.21.15.5:0/1588212823 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601b1d17bf0 con 0x5601b1dbfc30 2024-04-24T06:52:28.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.797+0000 7febf3e00640 1 --2- 172.21.15.5:0/1588212823 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5601b1dbfc30 0x5601b1dc2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.797+0000 7febf3e00640 1 --2- 172.21.15.5:0/1588212823 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5601b1dbfc30 0x5601b1dc2020 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c51e30866d923510 server_cookie=af3668017cf595d9 in_seq=0 out_seq=0 2024-04-24T06:52:28.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.797+0000 7febeaa00640 10 client.27091 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:52:28.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.798+0000 7febeaa00640 1 -- 172.21.15.5:0/1588212823 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5601b1d17bf0 con 0x5601b1dbfc30 2024-04-24T06:52:28.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.798+0000 7febeaa00640 10 client.27091 handle_command_reply: tid=0 2024-04-24T06:52:28.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.798+0000 7fec06808740 1 client.27091 shutdown 2024-04-24T06:52:28.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.799+0000 7fec06808740 1 -- 172.21.15.5:0/1588212823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b1da4b40 msgr2=0x5601b1daf5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:28.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.799+0000 7fec06808740 1 --2- 172.21.15.5:0/1588212823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b1da4b40 0x5601b1daf5c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7febf4011540 tx=0x7febf40198f0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.799+0000 7fec06808740 1 -- 172.21.15.5:0/1588212823 shutdown_connections 2024-04-24T06:52:28.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.799+0000 7fec06808740 1 --2- 172.21.15.5:0/1588212823 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5601b1dbfc30 0x5601b1dc2020 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.799+0000 7fec06808740 1 --2- 172.21.15.5:0/1588212823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601b1dafb30 0x5601b1dacfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.799+0000 7fec06808740 1 --2- 172.21.15.5:0/1588212823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601b1da5460 0x5601b1dac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.799+0000 7fec06808740 1 --2- 172.21.15.5:0/1588212823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b1da4b40 0x5601b1daf5c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.799+0000 7febeaa00640 0 client.27091 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:52:28.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.799+0000 7fec06808740 1 -- 172.21.15.5:0/1588212823 >> 172.21.15.5:0/1588212823 conn(0x5601b1da0e60 msgr2=0x5601b1da6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:52:28.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.799+0000 7fec06808740 1 -- 172.21.15.5:0/1588212823 shutdown_connections 2024-04-24T06:52:28.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.799+0000 7fec06808740 1 -- 172.21.15.5:0/1588212823 wait complete. 2024-04-24T06:52:28.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.799+0000 7fec06808740 20 client.27091 trim_cache size 0 max 16384 2024-04-24T06:52:28.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.814+0000 7fec06808740 1 Processor -- start 2024-04-24T06:52:28.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.814+0000 7fec06808740 1 -- start start 2024-04-24T06:52:28.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.814+0000 7fec06808740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b1da4b40 0x5601b1dac850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.814+0000 7fec06808740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601b1da5460 0x5601b1dbb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.814+0000 7fec06808740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601b1dafb30 0x5601b1daf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.814+0000 7fec06808740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7febfc1b27e0 con 0x5601b1da5460 2024-04-24T06:52:28.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.814+0000 7fec06808740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7febfc1b00b0 con 0x5601b1dafb30 2024-04-24T06:52:28.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.814+0000 7fec06808740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7febfc1adf20 con 0x5601b1da4b40 2024-04-24T06:52:28.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.815+0000 7febf3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601b1da5460 0x5601b1dbb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.815+0000 7fec01a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601b1dafb30 0x5601b1daf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.815+0000 7fec01a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601b1dafb30 0x5601b1daf0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:53662/0 (socket says 172.21.15.5:53662) 2024-04-24T06:52:28.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.815+0000 7fec01a00640 1 -- 172.21.15.5:0/1128587990 learned_addr learned my addr 172.21.15.5:0/1128587990 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:52:28.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.815+0000 7fec01a00640 1 --2- 172.21.15.5:0/1128587990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b1da4b40 0x5601b1dac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.815+0000 7fec01a00640 1 -- 172.21.15.5:0/1128587990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b1da4b40 msgr2=0x5601b1dac850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:28.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.815+0000 7fec01a00640 1 --2- 172.21.15.5:0/1128587990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b1da4b40 0x5601b1dac850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.815+0000 7fec01a00640 1 -- 172.21.15.5:0/1128587990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601b1da5460 msgr2=0x5601b1dbb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:28.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.815+0000 7fec01a00640 1 --2- 172.21.15.5:0/1128587990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601b1da5460 0x5601b1dbb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.815+0000 7fec01a00640 1 -- 172.21.15.5:0/1128587990 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601b1dad310 con 0x5601b1dafb30 2024-04-24T06:52:28.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.815+0000 7fec01a00640 1 --2- 172.21.15.5:0/1128587990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601b1dafb30 0x5601b1daf0f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7febf4019590 tx=0x7febf40198f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d56ec8bb62cc6b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:52:28.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.816+0000 7febf1600640 1 -- 172.21.15.5:0/1128587990 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf401e040 con 0x5601b1dafb30 2024-04-24T06:52:28.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.816+0000 7febf1600640 1 -- 172.21.15.5:0/1128587990 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febf4004b60 con 0x5601b1dafb30 2024-04-24T06:52:28.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.816+0000 7febf1600640 1 -- 172.21.15.5:0/1128587990 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf40054c0 con 0x5601b1dafb30 2024-04-24T06:52:28.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.816+0000 7fec06808740 1 -- 172.21.15.5:0/1128587990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601b1dafb30 msgr2=0x5601b1daf0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:28.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.816+0000 7fec06808740 1 --2- 172.21.15.5:0/1128587990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601b1dafb30 0x5601b1daf0f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7febf4019590 tx=0x7febf40198f0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.816+0000 7fec06808740 1 -- 172.21.15.5:0/1128587990 shutdown_connections 2024-04-24T06:52:28.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.816+0000 7fec06808740 1 --2- 172.21.15.5:0/1128587990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601b1dafb30 0x5601b1daf0f0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.816+0000 7fec06808740 1 --2- 172.21.15.5:0/1128587990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601b1da5460 0x5601b1dbb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.816+0000 7fec06808740 1 --2- 172.21.15.5:0/1128587990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b1da4b40 0x5601b1dac850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.816+0000 7fec06808740 1 -- 172.21.15.5:0/1128587990 >> 172.21.15.5:0/1128587990 conn(0x5601b1da0e60 msgr2=0x5601b1dac140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:52:28.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.816+0000 7fec06808740 1 -- 172.21.15.5:0/1128587990 shutdown_connections 2024-04-24T06:52:28.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.816+0000 7fec06808740 1 -- 172.21.15.5:0/1128587990 wait complete. 2024-04-24T06:52:28.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.817+0000 7fec06808740 1 -- start start 2024-04-24T06:52:28.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.817+0000 7fec06808740 1 Processor -- start 2024-04-24T06:52:28.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.817+0000 7fec06808740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601b1da4b40 0x5601b1db1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.817+0000 7fec06808740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b1da5460 0x5601b1daff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.817+0000 7fec06808740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601b1dafb30 0x5601b1db0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.817+0000 7fec06808740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7febfc1ae570 con 0x5601b1da4b40 2024-04-24T06:52:28.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.817+0000 7fec06808740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7febfc1af9e0 con 0x5601b1dafb30 2024-04-24T06:52:28.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.817+0000 7fec06808740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7febfc1aec40 con 0x5601b1da5460 2024-04-24T06:52:28.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.818+0000 7fec06808740 10 client.0 _ll_register_callbacks cb 0x7febf401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:52:28.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.818+0000 7fec01a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601b1da4b40 0x5601b1db1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.818+0000 7fec01a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601b1da4b40 0x5601b1db1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38228/0 (socket says 172.21.15.5:38228) 2024-04-24T06:52:28.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.818+0000 7fec01a00640 1 -- 172.21.15.5:0/2214846618 learned_addr learned my addr 172.21.15.5:0/2214846618 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:52:28.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.818+0000 7febf3e00640 1 --2- 172.21.15.5:0/2214846618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b1da5460 0x5601b1daff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.818+0000 7fec01a00640 1 --2- 172.21.15.5:0/2214846618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601b1dafb30 0x5601b1db0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.818+0000 7fec01a00640 1 -- 172.21.15.5:0/2214846618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b1da5460 msgr2=0x5601b1daff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:28.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.818+0000 7fec01a00640 1 --2- 172.21.15.5:0/2214846618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b1da5460 0x5601b1daff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.818+0000 7fec01a00640 1 -- 172.21.15.5:0/2214846618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601b1dafb30 msgr2=0x5601b1db0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:28.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.818+0000 7fec01a00640 1 --2- 172.21.15.5:0/2214846618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601b1dafb30 0x5601b1db0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.818+0000 7fec01a00640 1 -- 172.21.15.5:0/2214846618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601b1ae67d0 con 0x5601b1da4b40 2024-04-24T06:52:28.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.819+0000 7fec01a00640 1 --2- 172.21.15.5:0/2214846618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601b1da4b40 0x5601b1db1920 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7febf4004ee0 tx=0x7febf4005080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6152ccc780e6093 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:52:28.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.819+0000 7febeaa00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:52:28.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.819+0000 7febeaa00640 1 -- 172.21.15.5:0/2214846618 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf4041030 con 0x5601b1da4b40 2024-04-24T06:52:28.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.819+0000 7febeaa00640 1 -- 172.21.15.5:0/2214846618 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7febf4005120 con 0x5601b1da4b40 2024-04-24T06:52:28.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.819+0000 7fec06808740 1 -- 172.21.15.5:0/2214846618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febf4004b60 con 0x5601b1da4b40 2024-04-24T06:52:28.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.819+0000 7febeaa00640 1 -- 172.21.15.5:0/2214846618 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7febf401e070 con 0x5601b1da4b40 2024-04-24T06:52:28.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.819+0000 7fec06808740 10 client.16938 fetch_fsmap learned FSMap version 27 2024-04-24T06:52:28.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.819+0000 7fec06808740 1 -- 172.21.15.5:0/2214846618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5601b1db7d00 con 0x5601b1da4b40 2024-04-24T06:52:28.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.820+0000 7febeaa00640 1 -- 172.21.15.5:0/2214846618 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febf4004b60 con 0x5601b1da4b40 2024-04-24T06:52:28.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.820+0000 7febeaa00640 1 -- 172.21.15.5:0/2214846618 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7febf40277e0 con 0x5601b1da4b40 2024-04-24T06:52:28.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.820+0000 7fec06808740 10 client.16938 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:52:28.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.820+0000 7fec06808740 10 client.16938 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:52:28.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.820+0000 7fec06808740 20 client.16938 populate_metadata read hostname 'smithi005' 2024-04-24T06:52:28.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.820+0000 7fec06808740 1 --2- 172.21.15.5:0/2214846618 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5601b1e25fd0 0x5601b1e283c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:28.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.820+0000 7fec06808740 4 client.16938 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:52:28.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.820+0000 7fec06808740 1 -- 172.21.15.5:0/2214846618 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5601b1d11a60 con 0x5601b1e25fd0 2024-04-24T06:52:28.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.820+0000 7febf3e00640 1 --2- 172.21.15.5:0/2214846618 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5601b1e25fd0 0x5601b1e283c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:28.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.821+0000 7febf3e00640 1 --2- 172.21.15.5:0/2214846618 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5601b1e25fd0 0x5601b1e283c0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1d50191476360b2 server_cookie=c354990b40c83707 in_seq=0 out_seq=0 2024-04-24T06:52:28.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.821+0000 7febeaa00640 10 client.16938 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:52:28.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.821+0000 7febeaa00640 1 -- 172.21.15.5:0/2214846618 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5601b1d11a60 con 0x5601b1e25fd0 2024-04-24T06:52:28.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.821+0000 7febeaa00640 10 client.16938 handle_command_reply: tid=0 2024-04-24T06:52:28.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.821+0000 7fec06808740 1 client.16938 shutdown 2024-04-24T06:52:28.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.821+0000 7fec06808740 1 -- 172.21.15.5:0/2214846618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601b1da4b40 msgr2=0x5601b1db1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:28.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.821+0000 7fec06808740 1 --2- 172.21.15.5:0/2214846618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601b1da4b40 0x5601b1db1920 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7febf4004ee0 tx=0x7febf4005080 comp rx=0 tx=0).stop 2024-04-24T06:52:28.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.821+0000 7fec06808740 1 -- 172.21.15.5:0/2214846618 shutdown_connections 2024-04-24T06:52:28.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.821+0000 7fec06808740 1 --2- 172.21.15.5:0/2214846618 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5601b1e25fd0 0x5601b1e283c0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.821+0000 7fec06808740 1 --2- 172.21.15.5:0/2214846618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601b1dafb30 0x5601b1db0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.821+0000 7fec06808740 1 --2- 172.21.15.5:0/2214846618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5601b1da5460 0x5601b1daff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.821+0000 7fec06808740 1 --2- 172.21.15.5:0/2214846618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601b1da4b40 0x5601b1db1920 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.821+0000 7febeaa00640 0 client.16938 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:52:28.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.821+0000 7fec06808740 1 -- 172.21.15.5:0/2214846618 >> 172.21.15.5:0/2214846618 conn(0x5601b1da0e60 msgr2=0x5601b1dab180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:52:28.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.822+0000 7fec06808740 1 -- 172.21.15.5:0/2214846618 shutdown_connections 2024-04-24T06:52:28.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.822+0000 7fec06808740 1 -- 172.21.15.5:0/2214846618 wait complete. 2024-04-24T06:52:28.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.822+0000 7fec06808740 20 client.16938 trim_cache size 0 max 16384 2024-04-24T06:52:28.831 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:52:28.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.822+0000 7fec02400640 1 -- 172.21.15.5:0/281485475 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7febe00a3d20 msgr2=0x7febe00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:28.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.822+0000 7fec02400640 1 --2- 172.21.15.5:0/281485475 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7febe00a3d20 0x7febe00a61e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x5601b1da6100 tx=0x7febec0d4c30 comp rx=0 tx=0).stop 2024-04-24T06:52:28.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.822+0000 7fec02400640 1 -- 172.21.15.5:0/281485475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febfc104260 msgr2=0x7febfc1a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:28.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.822+0000 7fec02400640 1 --2- 172.21.15.5:0/281485475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febfc104260 0x7febfc1a0670 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7febec00eb00 tx=0x7febec00c2d0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.822+0000 7fec02400640 1 -- 172.21.15.5:0/281485475 shutdown_connections 2024-04-24T06:52:28.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.822+0000 7fec02400640 1 --2- 172.21.15.5:0/281485475 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7febe00a3d20 0x7febe00a61e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.822+0000 7fec02400640 1 --2- 172.21.15.5:0/281485475 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febfc19a730 0x7febfc19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.822+0000 7fec02400640 1 --2- 172.21.15.5:0/281485475 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febfc109330 0x7febfc1a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.822+0000 7fec02400640 1 --2- 172.21.15.5:0/281485475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febfc104260 0x7febfc1a0670 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:28.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.822+0000 7fec02400640 1 -- 172.21.15.5:0/281485475 >> 172.21.15.5:0/281485475 conn(0x7febfc0ffaa0 msgr2=0x7febfc100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:52:28.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.823+0000 7fec02400640 1 -- 172.21.15.5:0/281485475 shutdown_connections 2024-04-24T06:52:28.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:28.823+0000 7fec02400640 1 -- 172.21.15.5:0/281485475 wait complete. 2024-04-24T06:52:29.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:29 smithi137 ceph-mon[33017]: pgmap v260: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-04-24T06:52:29.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:29 smithi005 ceph-mon[27798]: pgmap v260: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-04-24T06:52:29.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:29 smithi037 ceph-mon[32864]: pgmap v260: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-04-24T06:52:29.842 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T06:52:29.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:52:29.843 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T06:52:30.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.076+0000 7f839de00640 1 -- 172.21.15.5:0/2223941775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8398100df0 msgr2=0x7f83981011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:30.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.076+0000 7f839de00640 1 --2- 172.21.15.5:0/2223941775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8398100df0 0x7f83981011d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8388009900 tx=0x7f838802a3f0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.077+0000 7f839de00640 1 -- 172.21.15.5:0/2223941775 shutdown_connections 2024-04-24T06:52:30.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.077+0000 7f839de00640 1 --2- 172.21.15.5:0/2223941775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f839810e9a0 0x7f8398110d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.077+0000 7f839de00640 1 --2- 172.21.15.5:0/2223941775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8398101710 0x7f839810e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.077+0000 7f839de00640 1 --2- 172.21.15.5:0/2223941775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8398100df0 0x7f83981011d0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.077+0000 7f839de00640 1 -- 172.21.15.5:0/2223941775 >> 172.21.15.5:0/2223941775 conn(0x7f83980780b0 msgr2=0x7f83980784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:52:30.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.077+0000 7f839de00640 1 -- 172.21.15.5:0/2223941775 shutdown_connections 2024-04-24T06:52:30.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.077+0000 7f839de00640 1 -- 172.21.15.5:0/2223941775 wait complete. 2024-04-24T06:52:30.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.078+0000 7f839de00640 1 Processor -- start 2024-04-24T06:52:30.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.078+0000 7f839de00640 1 -- start start 2024-04-24T06:52:30.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.078+0000 7f839de00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8398100df0 0x7f83981a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:30.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.078+0000 7f839de00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8398101710 0x7f83981a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:30.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.078+0000 7f839de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f839810e9a0 0x7f839819a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:30.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.078+0000 7f839de00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8398114600 con 0x7f839810e9a0 2024-04-24T06:52:30.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.078+0000 7f839de00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8398114480 con 0x7f8398100df0 2024-04-24T06:52:30.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.078+0000 7f839de00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8398114780 con 0x7f8398101710 2024-04-24T06:52:30.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.079+0000 7f8397400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f839810e9a0 0x7f839819a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:30.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.079+0000 7f8396a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8398100df0 0x7f83981a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:30.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.079+0000 7f8397400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f839810e9a0 0x7f839819a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38248/0 (socket says 172.21.15.5:38248) 2024-04-24T06:52:30.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.079+0000 7f8397400640 1 -- 172.21.15.5:0/375923737 learned_addr learned my addr 172.21.15.5:0/375923737 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:52:30.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.079+0000 7f838e000640 1 --2- 172.21.15.5:0/375923737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8398101710 0x7f83981a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:30.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.079+0000 7f8396a00640 1 -- 172.21.15.5:0/375923737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8398101710 msgr2=0x7f83981a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:30.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.079+0000 7f8396a00640 1 --2- 172.21.15.5:0/375923737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8398101710 0x7f83981a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.079+0000 7f8396a00640 1 -- 172.21.15.5:0/375923737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f839810e9a0 msgr2=0x7f839819a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:30.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.079+0000 7f8396a00640 1 --2- 172.21.15.5:0/375923737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f839810e9a0 0x7f839819a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.079+0000 7f8396a00640 1 -- 172.21.15.5:0/375923737 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f839819b060 con 0x7f8398100df0 2024-04-24T06:52:30.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.079+0000 7f8397400640 1 --2- 172.21.15.5:0/375923737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f839810e9a0 0x7f839819a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:52:30.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.079+0000 7f8396a00640 1 --2- 172.21.15.5:0/375923737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8398100df0 0x7f83981a0730 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f838802a910 tx=0x7f8388003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:52:30.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.080+0000 7f839ca00640 1 -- 172.21.15.5:0/375923737 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838803f070 con 0x7f8398100df0 2024-04-24T06:52:30.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.080+0000 7f839de00640 1 -- 172.21.15.5:0/375923737 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f839819b2f0 con 0x7f8398100df0 2024-04-24T06:52:30.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.080+0000 7f839de00640 1 -- 172.21.15.5:0/375923737 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83981a75f0 con 0x7f8398100df0 2024-04-24T06:52:30.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.080+0000 7f839ca00640 1 -- 172.21.15.5:0/375923737 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8388031c90 con 0x7f8398100df0 2024-04-24T06:52:30.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.080+0000 7f839ca00640 1 -- 172.21.15.5:0/375923737 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8388049060 con 0x7f8398100df0 2024-04-24T06:52:30.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.083+0000 7f83a0c16740 1 Processor -- start 2024-04-24T06:52:30.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.083+0000 7f83a0c16740 1 -- start start 2024-04-24T06:52:30.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.083+0000 7f839ca00640 1 -- 172.21.15.5:0/375923737 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f8388042070 con 0x7f8398100df0 2024-04-24T06:52:30.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.084+0000 7f83a0c16740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561ae507090 0x5561ae507470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:30.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.084+0000 7f83a0c16740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561ae5079b0 0x5561ae511b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:30.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.084+0000 7f83a0c16740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561ae512080 0x5561ae514470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:30.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.084+0000 7f83a0c16740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5561ae46ce70 con 0x5561ae5079b0 2024-04-24T06:52:30.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.084+0000 7f83a0c16740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5561ae413380 con 0x5561ae512080 2024-04-24T06:52:30.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.084+0000 7f83a0c16740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5561ae468e30 con 0x5561ae507090 2024-04-24T06:52:30.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.084+0000 7f8397400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561ae507090 0x5561ae507470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:30.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.084+0000 7f8396a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561ae5079b0 0x5561ae511b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:30.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.084+0000 7f8397400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561ae507090 0x5561ae507470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:39090/0 (socket says 172.21.15.5:39090) 2024-04-24T06:52:30.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.084+0000 7f8397400640 1 -- 172.21.15.5:0/1398505643 learned_addr learned my addr 172.21.15.5:0/1398505643 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:52:30.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.084+0000 7f838e000640 1 --2- 172.21.15.5:0/1398505643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561ae512080 0x5561ae514470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:30.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.084+0000 7f838e000640 1 -- 172.21.15.5:0/1398505643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561ae507090 msgr2=0x5561ae507470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:30.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.084+0000 7f838e000640 1 --2- 172.21.15.5:0/1398505643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561ae507090 0x5561ae507470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.084+0000 7f838e000640 1 -- 172.21.15.5:0/1398505643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561ae5079b0 msgr2=0x5561ae511b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:30.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.084+0000 7f838e000640 1 --2- 172.21.15.5:0/1398505643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561ae5079b0 0x5561ae511b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.085+0000 7f838e000640 1 -- 172.21.15.5:0/1398505643 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561ae2477e0 con 0x5561ae512080 2024-04-24T06:52:30.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.085+0000 7f8397400640 1 --2- 172.21.15.5:0/1398505643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561ae507090 0x5561ae507470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:52:30.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.085+0000 7f838e000640 1 --2- 172.21.15.5:0/1398505643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561ae512080 0x5561ae514470 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f839819c0d0 tx=0x7f8380011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49e1110986c81ba0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:52:30.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.085+0000 7f8387e00640 1 -- 172.21.15.5:0/1398505643 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8380009d70 con 0x5561ae512080 2024-04-24T06:52:30.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.085+0000 7f8387e00640 1 -- 172.21.15.5:0/1398505643 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8380005020 con 0x5561ae512080 2024-04-24T06:52:30.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.085+0000 7f8387e00640 1 -- 172.21.15.5:0/1398505643 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8380004b90 con 0x5561ae512080 2024-04-24T06:52:30.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.085+0000 7f839ca00640 1 --2- 172.21.15.5:0/375923737 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f837c0a3da0 0x7f837c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:30.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.085+0000 7f839ca00640 1 -- 172.21.15.5:0/375923737 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f83880f06a0 con 0x7f8398100df0 2024-04-24T06:52:30.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.086+0000 7f8397400640 1 --2- 172.21.15.5:0/375923737 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f837c0a3da0 0x7f837c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:30.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.086+0000 7f83a0c16740 1 -- 172.21.15.5:0/1398505643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561ae512080 msgr2=0x5561ae514470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:30.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.086+0000 7f83a0c16740 1 --2- 172.21.15.5:0/1398505643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561ae512080 0x5561ae514470 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f839819c0d0 tx=0x7f8380011810 comp rx=0 tx=0).stop 2024-04-24T06:52:30.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.086+0000 7f83a0c16740 1 -- 172.21.15.5:0/1398505643 shutdown_connections 2024-04-24T06:52:30.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.086+0000 7f83a0c16740 1 --2- 172.21.15.5:0/1398505643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561ae512080 0x5561ae514470 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.086+0000 7f83a0c16740 1 --2- 172.21.15.5:0/1398505643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561ae5079b0 0x5561ae511b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.086+0000 7f83a0c16740 1 --2- 172.21.15.5:0/1398505643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561ae507090 0x5561ae507470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.086+0000 7f83a0c16740 1 -- 172.21.15.5:0/1398505643 >> 172.21.15.5:0/1398505643 conn(0x5561ae5033b0 msgr2=0x5561ae503790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:52:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.086+0000 7f8397400640 1 --2- 172.21.15.5:0/375923737 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f837c0a3da0 0x7f837c0a6260 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f83900094d0 tx=0x7f8390009500 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:52:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.086+0000 7f83a0c16740 1 -- 172.21.15.5:0/1398505643 shutdown_connections 2024-04-24T06:52:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.086+0000 7f83a0c16740 1 -- 172.21.15.5:0/1398505643 wait complete. 2024-04-24T06:52:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.087+0000 7f83a0c16740 1 -- start start 2024-04-24T06:52:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.087+0000 7f83a0c16740 1 Processor -- start 2024-04-24T06:52:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.087+0000 7f83a0c16740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561ae507090 0x5561ae50eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:30.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.087+0000 7f83a0c16740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561ae5079b0 0x5561ae50f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:30.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.087+0000 7f83a0c16740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561ae512080 0x5561ae50f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:30.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.087+0000 7f83a0c16740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5561ae514e70 con 0x5561ae512080 2024-04-24T06:52:30.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.087+0000 7f83a0c16740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5561ae426660 con 0x5561ae5079b0 2024-04-24T06:52:30.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.087+0000 7f83a0c16740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5561ae514ff0 con 0x5561ae507090 2024-04-24T06:52:30.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.088+0000 7f83a0c16740 10 client.0 _ll_register_callbacks cb 0x5561ae414aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:52:30.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.088+0000 7f8397400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561ae512080 0x5561ae50f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:30.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.088+0000 7f8397400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561ae512080 0x5561ae50f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38268/0 (socket says 172.21.15.5:38268) 2024-04-24T06:52:30.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.088+0000 7f8397400640 1 -- 172.21.15.5:0/3037688414 learned_addr learned my addr 172.21.15.5:0/3037688414 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:52:30.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.088+0000 7f8396a00640 1 --2- 172.21.15.5:0/3037688414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561ae507090 0x5561ae50eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:30.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.088+0000 7f838e000640 1 --2- 172.21.15.5:0/3037688414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561ae5079b0 0x5561ae50f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:30.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.088+0000 7f838e000640 1 -- 172.21.15.5:0/3037688414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561ae507090 msgr2=0x5561ae50eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:30.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.088+0000 7f838e000640 1 --2- 172.21.15.5:0/3037688414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561ae507090 0x5561ae50eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.088+0000 7f838e000640 1 -- 172.21.15.5:0/3037688414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561ae512080 msgr2=0x5561ae50f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:30.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.088+0000 7f838e000640 1 --2- 172.21.15.5:0/3037688414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561ae512080 0x5561ae50f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.088+0000 7f838e000640 1 -- 172.21.15.5:0/3037688414 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561ae2a7f70 con 0x5561ae5079b0 2024-04-24T06:52:30.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.089+0000 7f838e000640 1 --2- 172.21.15.5:0/3037688414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561ae5079b0 0x5561ae50f3f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8380000c00 tx=0x7f8380005600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98cb4c8532eed75d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:52:30.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.089+0000 7f8384c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:52:30.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.089+0000 7f8384c00640 1 -- 172.21.15.5:0/3037688414 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8380029020 con 0x5561ae5079b0 2024-04-24T06:52:30.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.089+0000 7f8384c00640 1 -- 172.21.15.5:0/3037688414 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f8380004b20 con 0x5561ae5079b0 2024-04-24T06:52:30.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.089+0000 7f83a0c16740 1 -- 172.21.15.5:0/3037688414 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8380005020 con 0x5561ae5079b0 2024-04-24T06:52:30.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.089+0000 7f8384c00640 1 -- 172.21.15.5:0/3037688414 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8380034050 con 0x5561ae5079b0 2024-04-24T06:52:30.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.089+0000 7f83a0c16740 10 client.27095 fetch_fsmap learned FSMap version 27 2024-04-24T06:52:30.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.089+0000 7f83a0c16740 1 -- 172.21.15.5:0/3037688414 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5561ae13f170 con 0x5561ae5079b0 2024-04-24T06:52:30.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.090+0000 7f8384c00640 1 -- 172.21.15.5:0/3037688414 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8380005020 con 0x5561ae5079b0 2024-04-24T06:52:30.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.090+0000 7f8384c00640 1 -- 172.21.15.5:0/3037688414 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f8380023420 con 0x5561ae5079b0 2024-04-24T06:52:30.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.090+0000 7f83a0c16740 10 client.27095 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:52:30.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.090+0000 7f83a0c16740 10 client.27095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:52:30.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.090+0000 7f83a0c16740 20 client.27095 populate_metadata read hostname 'smithi005' 2024-04-24T06:52:30.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.090+0000 7f83a0c16740 1 --2- 172.21.15.5:0/3037688414 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5561ae522350 0x5561ae524740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:30.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.090+0000 7f83a0c16740 4 client.27095 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:52:30.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.090+0000 7f83a0c16740 1 -- 172.21.15.5:0/3037688414 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561ae47a890 con 0x5561ae522350 2024-04-24T06:52:30.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.090+0000 7f8396a00640 1 --2- 172.21.15.5:0/3037688414 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5561ae522350 0x5561ae524740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:30.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.091+0000 7f8396a00640 1 --2- 172.21.15.5:0/3037688414 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5561ae522350 0x5561ae524740 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=469fba1f266af900 server_cookie=a7943c0fcb980410 in_seq=0 out_seq=0 2024-04-24T06:52:30.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.091+0000 7f8384c00640 10 client.27095 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:52:30.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.092+0000 7f8384c00640 1 -- 172.21.15.5:0/3037688414 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5561ae47a890 con 0x5561ae522350 2024-04-24T06:52:30.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.092+0000 7f8384c00640 10 client.27095 handle_command_reply: tid=0 2024-04-24T06:52:30.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.092+0000 7f83a0c16740 1 client.27095 shutdown 2024-04-24T06:52:30.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.092+0000 7f83a0c16740 1 -- 172.21.15.5:0/3037688414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561ae5079b0 msgr2=0x5561ae50f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:30.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.092+0000 7f83a0c16740 1 --2- 172.21.15.5:0/3037688414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561ae5079b0 0x5561ae50f3f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8380000c00 tx=0x7f8380005600 comp rx=0 tx=0).stop 2024-04-24T06:52:30.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.092+0000 7f83a0c16740 1 -- 172.21.15.5:0/3037688414 shutdown_connections 2024-04-24T06:52:30.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.092+0000 7f83a0c16740 1 --2- 172.21.15.5:0/3037688414 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5561ae522350 0x5561ae524740 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.092+0000 7f83a0c16740 1 --2- 172.21.15.5:0/3037688414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561ae512080 0x5561ae50f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.092+0000 7f83a0c16740 1 --2- 172.21.15.5:0/3037688414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561ae5079b0 0x5561ae50f3f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.092+0000 7f8384c00640 0 client.27095 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:52:30.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.092+0000 7f83a0c16740 1 --2- 172.21.15.5:0/3037688414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561ae507090 0x5561ae50eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.092+0000 7f83a0c16740 1 -- 172.21.15.5:0/3037688414 >> 172.21.15.5:0/3037688414 conn(0x5561ae5033b0 msgr2=0x5561ae5090b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:52:30.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.092+0000 7f83a0c16740 1 -- 172.21.15.5:0/3037688414 shutdown_connections 2024-04-24T06:52:30.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.092+0000 7f83a0c16740 1 -- 172.21.15.5:0/3037688414 wait complete. 2024-04-24T06:52:30.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.092+0000 7f83a0c16740 20 client.27095 trim_cache size 0 max 16384 2024-04-24T06:52:30.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.112+0000 7f83a0c16740 1 Processor -- start 2024-04-24T06:52:30.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.112+0000 7f83a0c16740 1 -- start start 2024-04-24T06:52:30.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.112+0000 7f83a0c16740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561ae507090 0x5561ae51ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:30.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.112+0000 7f83a0c16740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561ae5079b0 0x5561ae508f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:30.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.112+0000 7f83a0c16740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561ae512080 0x5561ae52a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:30.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.112+0000 7f83a0c16740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8374011090 con 0x5561ae507090 2024-04-24T06:52:30.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.112+0000 7f83a0c16740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f837400e960 con 0x5561ae5079b0 2024-04-24T06:52:30.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.112+0000 7f83a0c16740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f837400c7d0 con 0x5561ae512080 2024-04-24T06:52:30.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.113+0000 7f8396a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561ae507090 0x5561ae51ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:30.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.113+0000 7f8396a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561ae507090 0x5561ae51ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38272/0 (socket says 172.21.15.5:38272) 2024-04-24T06:52:30.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.113+0000 7f8396a00640 1 -- 172.21.15.5:0/1024201406 learned_addr learned my addr 172.21.15.5:0/1024201406 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:52:30.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.113+0000 7f838e000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561ae5079b0 0x5561ae508f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:30.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.113+0000 7f8397400640 1 --2- 172.21.15.5:0/1024201406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561ae512080 0x5561ae52a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:30.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.113+0000 7f8396a00640 1 -- 172.21.15.5:0/1024201406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561ae512080 msgr2=0x5561ae52a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:30.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.113+0000 7f8396a00640 1 --2- 172.21.15.5:0/1024201406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561ae512080 0x5561ae52a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.113+0000 7f8396a00640 1 -- 172.21.15.5:0/1024201406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561ae5079b0 msgr2=0x5561ae508f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:30.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.113+0000 7f8396a00640 1 --2- 172.21.15.5:0/1024201406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561ae5079b0 0x5561ae508f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.113+0000 7f8396a00640 1 -- 172.21.15.5:0/1024201406 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561ae50de60 con 0x5561ae507090 2024-04-24T06:52:30.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.113+0000 7f838e000640 1 --2- 172.21.15.5:0/1024201406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561ae5079b0 0x5561ae508f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:52:30.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.113+0000 7f8397400640 1 --2- 172.21.15.5:0/1024201406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561ae512080 0x5561ae52a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:52:30.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.113+0000 7f8396a00640 1 --2- 172.21.15.5:0/1024201406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561ae507090 0x5561ae51ddc0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8388009900 tx=0x7f83880f6530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79863caed1dcdc7c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:52:30.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.114+0000 7f8387400640 1 -- 172.21.15.5:0/1024201406 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83880bc1c0 con 0x5561ae507090 2024-04-24T06:52:30.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.114+0000 7f8387400640 1 -- 172.21.15.5:0/1024201406 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8388031e40 con 0x5561ae507090 2024-04-24T06:52:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.114+0000 7f8387400640 1 -- 172.21.15.5:0/1024201406 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8388005d50 con 0x5561ae507090 2024-04-24T06:52:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.114+0000 7f83a0c16740 1 -- 172.21.15.5:0/1024201406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561ae507090 msgr2=0x5561ae51ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.114+0000 7f83a0c16740 1 --2- 172.21.15.5:0/1024201406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561ae507090 0x5561ae51ddc0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8388009900 tx=0x7f83880f6530 comp rx=0 tx=0).stop 2024-04-24T06:52:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.114+0000 7f83a0c16740 1 -- 172.21.15.5:0/1024201406 shutdown_connections 2024-04-24T06:52:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.114+0000 7f83a0c16740 1 --2- 172.21.15.5:0/1024201406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561ae512080 0x5561ae52a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.114+0000 7f83a0c16740 1 --2- 172.21.15.5:0/1024201406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561ae5079b0 0x5561ae508f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.114+0000 7f83a0c16740 1 --2- 172.21.15.5:0/1024201406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561ae507090 0x5561ae51ddc0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.114+0000 7f83a0c16740 1 -- 172.21.15.5:0/1024201406 >> 172.21.15.5:0/1024201406 conn(0x5561ae5033b0 msgr2=0x5561ae508820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:52:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.115+0000 7f83a0c16740 1 -- 172.21.15.5:0/1024201406 shutdown_connections 2024-04-24T06:52:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.115+0000 7f83a0c16740 1 -- 172.21.15.5:0/1024201406 wait complete. 2024-04-24T06:52:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.115+0000 7f83a0c16740 1 -- start start 2024-04-24T06:52:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.115+0000 7f83a0c16740 1 Processor -- start 2024-04-24T06:52:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.115+0000 7f83a0c16740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561ae507090 0x5561ae513610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.116+0000 7f83a0c16740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561ae5079b0 0x5561ae513b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.116+0000 7f83a0c16740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561ae512080 0x5561ae518960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:30.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.116+0000 7f83a0c16740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f837400ce20 con 0x5561ae512080 2024-04-24T06:52:30.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.116+0000 7f83a0c16740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f837400e290 con 0x5561ae507090 2024-04-24T06:52:30.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.116+0000 7f83a0c16740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f837400d4f0 con 0x5561ae5079b0 2024-04-24T06:52:30.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.116+0000 7f83a0c16740 10 client.0 _ll_register_callbacks cb 0x7f83880f6400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:52:30.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.116+0000 7f8397400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561ae512080 0x5561ae518960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:30.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.116+0000 7f8396a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561ae507090 0x5561ae513610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:30.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.116+0000 7f8396a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561ae507090 0x5561ae513610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:53738/0 (socket says 172.21.15.5:53738) 2024-04-24T06:52:30.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.116+0000 7f8397400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561ae512080 0x5561ae518960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38284/0 (socket says 172.21.15.5:38284) 2024-04-24T06:52:30.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.116+0000 7f8396a00640 1 -- 172.21.15.5:0/84087314 learned_addr learned my addr 172.21.15.5:0/84087314 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:52:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.116+0000 7f838e000640 1 --2- 172.21.15.5:0/84087314 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561ae5079b0 0x5561ae513b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.116+0000 7f8397400640 1 -- 172.21.15.5:0/84087314 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561ae5079b0 msgr2=0x5561ae513b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.116+0000 7f8397400640 1 --2- 172.21.15.5:0/84087314 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561ae5079b0 0x5561ae513b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.116+0000 7f8397400640 1 -- 172.21.15.5:0/84087314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561ae507090 msgr2=0x5561ae513610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.116+0000 7f8397400640 1 --2- 172.21.15.5:0/84087314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561ae507090 0x5561ae513610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.116+0000 7f8397400640 1 -- 172.21.15.5:0/84087314 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561ae524ab0 con 0x5561ae512080 2024-04-24T06:52:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.116+0000 7f8396a00640 1 --2- 172.21.15.5:0/84087314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561ae507090 0x5561ae513610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:52:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.117+0000 7f8397400640 1 --2- 172.21.15.5:0/84087314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561ae512080 0x5561ae518960 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8390003f20 tx=0x7f8390004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74a6dac49a2f8a33 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:52:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.117+0000 7f8384c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:52:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.117+0000 7f8384c00640 1 -- 172.21.15.5:0/84087314 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8390010eb0 con 0x5561ae512080 2024-04-24T06:52:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.117+0000 7f8384c00640 1 -- 172.21.15.5:0/84087314 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f8390017030 con 0x5561ae512080 2024-04-24T06:52:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.117+0000 7f83a0c16740 1 -- 172.21.15.5:0/84087314 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5561ae474700 con 0x5561ae512080 2024-04-24T06:52:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.117+0000 7f8384c00640 1 -- 172.21.15.5:0/84087314 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8390004250 con 0x5561ae512080 2024-04-24T06:52:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.117+0000 7f83a0c16740 10 client.16956 fetch_fsmap learned FSMap version 27 2024-04-24T06:52:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.117+0000 7f83a0c16740 1 -- 172.21.15.5:0/84087314 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5561ae588f10 con 0x5561ae512080 2024-04-24T06:52:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.118+0000 7f8384c00640 1 -- 172.21.15.5:0/84087314 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5561ae474700 con 0x5561ae512080 2024-04-24T06:52:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.118+0000 7f8384c00640 1 -- 172.21.15.5:0/84087314 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f8390031ba0 con 0x5561ae512080 2024-04-24T06:52:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.118+0000 7f83a0c16740 10 client.16956 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:52:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.118+0000 7f83a0c16740 10 client.16956 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:52:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.118+0000 7f83a0c16740 20 client.16956 populate_metadata read hostname 'smithi005' 2024-04-24T06:52:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.118+0000 7f83a0c16740 1 --2- 172.21.15.5:0/84087314 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5561ae5896e0 0x5561ae58bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.118+0000 7f83a0c16740 4 client.16956 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T06:52:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.118+0000 7f83a0c16740 1 -- 172.21.15.5:0/84087314 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5561ae3f0980 con 0x5561ae5896e0 2024-04-24T06:52:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.118+0000 7f8396a00640 1 --2- 172.21.15.5:0/84087314 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5561ae5896e0 0x5561ae58bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.119+0000 7f8396a00640 1 --2- 172.21.15.5:0/84087314 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5561ae5896e0 0x5561ae58bb00 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89999a056039aec7 server_cookie=a5942bcb2f4040b6 in_seq=0 out_seq=0 2024-04-24T06:52:30.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.119+0000 7f8384c00640 10 client.16956 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:52:30.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.120+0000 7f8384c00640 1 -- 172.21.15.5:0/84087314 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5561ae3f0980 con 0x5561ae5896e0 2024-04-24T06:52:30.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.120+0000 7f8384c00640 10 client.16956 handle_command_reply: tid=0 2024-04-24T06:52:30.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.120+0000 7f83a0c16740 1 client.16956 shutdown 2024-04-24T06:52:30.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.120+0000 7f83a0c16740 1 -- 172.21.15.5:0/84087314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561ae512080 msgr2=0x5561ae518960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:30.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.120+0000 7f83a0c16740 1 --2- 172.21.15.5:0/84087314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561ae512080 0x5561ae518960 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8390003f20 tx=0x7f8390004ab0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.120+0000 7f83a0c16740 1 -- 172.21.15.5:0/84087314 shutdown_connections 2024-04-24T06:52:30.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.120+0000 7f83a0c16740 1 --2- 172.21.15.5:0/84087314 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5561ae5896e0 0x5561ae58bb00 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.120+0000 7f83a0c16740 1 --2- 172.21.15.5:0/84087314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561ae512080 0x5561ae518960 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.120+0000 7f83a0c16740 1 --2- 172.21.15.5:0/84087314 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561ae5079b0 0x5561ae513b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.120+0000 7f83a0c16740 1 --2- 172.21.15.5:0/84087314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561ae507090 0x5561ae513610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.120+0000 7f83a0c16740 1 -- 172.21.15.5:0/84087314 >> 172.21.15.5:0/84087314 conn(0x5561ae5033b0 msgr2=0x5561ae509070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:52:30.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.120+0000 7f8384c00640 0 client.16956 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:52:30.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.120+0000 7f83a0c16740 1 -- 172.21.15.5:0/84087314 shutdown_connections 2024-04-24T06:52:30.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.120+0000 7f83a0c16740 1 -- 172.21.15.5:0/84087314 wait complete. 2024-04-24T06:52:30.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.121+0000 7f83a0c16740 20 client.16956 trim_cache size 0 max 16384 2024-04-24T06:52:30.130 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:52:30.130 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-24T06:52:30.130 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "28a88227-5708-40cc-991c-5b397b878e62", 2024-04-24T06:52:30.130 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-24T06:52:30.130 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:52:30.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.121+0000 7f839de00640 1 -- 172.21.15.5:0/375923737 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f837c0a3da0 msgr2=0x7f837c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:30.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.121+0000 7f839de00640 1 --2- 172.21.15.5:0/375923737 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f837c0a3da0 0x7f837c0a6260 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f83900094d0 tx=0x7f8390009500 comp rx=0 tx=0).stop 2024-04-24T06:52:30.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.121+0000 7f839de00640 1 -- 172.21.15.5:0/375923737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8398100df0 msgr2=0x7f83981a0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:30.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.121+0000 7f839de00640 1 --2- 172.21.15.5:0/375923737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8398100df0 0x7f83981a0730 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f838802a910 tx=0x7f8388003b20 comp rx=0 tx=0).stop 2024-04-24T06:52:30.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.121+0000 7f839de00640 1 -- 172.21.15.5:0/375923737 shutdown_connections 2024-04-24T06:52:30.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.121+0000 7f839de00640 1 --2- 172.21.15.5:0/375923737 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f837c0a3da0 0x7f837c0a6260 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.121+0000 7f839de00640 1 --2- 172.21.15.5:0/375923737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f839810e9a0 0x7f839819a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.121+0000 7f839de00640 1 --2- 172.21.15.5:0/375923737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8398101710 0x7f83981a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.121+0000 7f839de00640 1 --2- 172.21.15.5:0/375923737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8398100df0 0x7f83981a0730 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.121+0000 7f839de00640 1 -- 172.21.15.5:0/375923737 >> 172.21.15.5:0/375923737 conn(0x7f83980780b0 msgr2=0x7f839810fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:52:30.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.121+0000 7f839de00640 1 -- 172.21.15.5:0/375923737 shutdown_connections 2024-04-24T06:52:30.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.122+0000 7f839de00640 1 -- 172.21.15.5:0/375923737 wait complete. 2024-04-24T06:52:30.142 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T06:52:30.142 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T06:52:30.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.377+0000 7fc3bcc00640 1 -- 172.21.15.5:0/1591191855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3b8100df0 msgr2=0x7fc3b81011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:30.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.377+0000 7fc3bcc00640 1 --2- 172.21.15.5:0/1591191855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3b8100df0 0x7fc3b81011d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8009900 tx=0x7fc3a802a400 comp rx=0 tx=0).stop 2024-04-24T06:52:30.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.377+0000 7fc3bcc00640 1 -- 172.21.15.5:0/1591191855 shutdown_connections 2024-04-24T06:52:30.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.377+0000 7fc3bcc00640 1 --2- 172.21.15.5:0/1591191855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3b810e9a0 0x7fc3b8110d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.377+0000 7fc3bcc00640 1 --2- 172.21.15.5:0/1591191855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3b8101710 0x7fc3b810e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.377+0000 7fc3bcc00640 1 --2- 172.21.15.5:0/1591191855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3b8100df0 0x7fc3b81011d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.377+0000 7fc3bcc00640 1 -- 172.21.15.5:0/1591191855 >> 172.21.15.5:0/1591191855 conn(0x7fc3b80780b0 msgr2=0x7fc3b80784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:52:30.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.378+0000 7fc3bcc00640 1 -- 172.21.15.5:0/1591191855 shutdown_connections 2024-04-24T06:52:30.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.378+0000 7fc3bcc00640 1 -- 172.21.15.5:0/1591191855 wait complete. 2024-04-24T06:52:30.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.379+0000 7fc3bcc00640 1 Processor -- start 2024-04-24T06:52:30.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.379+0000 7fc3bcc00640 1 -- start start 2024-04-24T06:52:30.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.380+0000 7fc3bcc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3b8100df0 0x7fc3b81a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:30.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.380+0000 7fc3bcc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3b8101710 0x7fc3b81a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:30.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.380+0000 7fc3bcc00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3b810e9a0 0x7fc3b819a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:30.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.380+0000 7fc3bcc00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b8114600 con 0x7fc3b8101710 2024-04-24T06:52:30.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.380+0000 7fc3bcc00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b8114480 con 0x7fc3b810e9a0 2024-04-24T06:52:30.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.380+0000 7fc3bcc00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b8114780 con 0x7fc3b8100df0 2024-04-24T06:52:30.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.380+0000 7fc3b5e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3b8100df0 0x7fc3b81a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:30.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.380+0000 7fc3b6800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3b810e9a0 0x7fc3b819a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:30.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.380+0000 7fc3afe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3b8101710 0x7fc3b81a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:30.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.380+0000 7fc3b6800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3b810e9a0 0x7fc3b819a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:53766/0 (socket says 172.21.15.5:53766) 2024-04-24T06:52:30.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.381+0000 7fc3b6800640 1 -- 172.21.15.5:0/4208486096 learned_addr learned my addr 172.21.15.5:0/4208486096 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:52:30.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.381+0000 7fc3b5e00640 1 -- 172.21.15.5:0/4208486096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3b810e9a0 msgr2=0x7fc3b819a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:30.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.381+0000 7fc3b5e00640 1 --2- 172.21.15.5:0/4208486096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3b810e9a0 0x7fc3b819a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.381+0000 7fc3b5e00640 1 -- 172.21.15.5:0/4208486096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3b8101710 msgr2=0x7fc3b81a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:30.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.381+0000 7fc3b5e00640 1 --2- 172.21.15.5:0/4208486096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3b8101710 0x7fc3b81a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.381+0000 7fc3b5e00640 1 -- 172.21.15.5:0/4208486096 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b819b060 con 0x7fc3b8100df0 2024-04-24T06:52:30.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.381+0000 7fc3b6800640 1 --2- 172.21.15.5:0/4208486096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3b810e9a0 0x7fc3b819a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:52:30.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.382+0000 7fc3b5e00640 1 --2- 172.21.15.5:0/4208486096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3b8100df0 0x7fc3b81a0730 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc3a802a920 tx=0x7fc3a8003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:52:30.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.383+0000 7fc3aea00640 1 -- 172.21.15.5:0/4208486096 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a803f070 con 0x7fc3b8100df0 2024-04-24T06:52:30.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.383+0000 7fc3aea00640 1 -- 172.21.15.5:0/4208486096 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3a8031c90 con 0x7fc3b8100df0 2024-04-24T06:52:30.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.383+0000 7fc3aea00640 1 -- 172.21.15.5:0/4208486096 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8035030 con 0x7fc3b8100df0 2024-04-24T06:52:30.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.383+0000 7fc3bcc00640 1 -- 172.21.15.5:0/4208486096 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3b819b2f0 con 0x7fc3b8100df0 2024-04-24T06:52:30.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.383+0000 7fc3bcc00640 1 -- 172.21.15.5:0/4208486096 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3b81a75f0 con 0x7fc3b8100df0 2024-04-24T06:52:30.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.388+0000 7fc3bfa33740 1 Processor -- start 2024-04-24T06:52:30.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.388+0000 7fc3aea00640 1 -- 172.21.15.5:0/4208486096 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fc3a8042070 con 0x7fc3b8100df0 2024-04-24T06:52:30.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.388+0000 7fc3bfa33740 1 -- start start 2024-04-24T06:52:30.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.388+0000 7fc3bfa33740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556308ab860 0x5556308abc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:30.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.388+0000 7fc3bfa33740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556308ac180 0x5556308b6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:30.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.389+0000 7fc3bfa33740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556308b6850 0x5556308b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:30.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.389+0000 7fc3bfa33740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5556308111f0 con 0x5556308ac180 2024-04-24T06:52:30.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.389+0000 7fc3bfa33740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5556307b7350 con 0x5556308ab860 2024-04-24T06:52:30.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.389+0000 7fc3bfa33740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55563080d1b0 con 0x5556308b6850 2024-04-24T06:52:30.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.389+0000 7fc3b6800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556308ab860 0x5556308abc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:30.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.389+0000 7fc3afe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556308b6850 0x5556308b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:30.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.389+0000 7fc3b6800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556308ab860 0x5556308abc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:53778/0 (socket says 172.21.15.5:53778) 2024-04-24T06:52:30.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.389+0000 7fc3b6800640 1 -- 172.21.15.5:0/1772554338 learned_addr learned my addr 172.21.15.5:0/1772554338 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:52:30.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.389+0000 7fc3b5e00640 1 --2- 172.21.15.5:0/1772554338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556308ac180 0x5556308b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:30.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.389+0000 7fc3b6800640 1 -- 172.21.15.5:0/1772554338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556308b6850 msgr2=0x5556308b8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:30.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.389+0000 7fc3b6800640 1 --2- 172.21.15.5:0/1772554338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556308b6850 0x5556308b8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.389+0000 7fc3b6800640 1 -- 172.21.15.5:0/1772554338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556308ac180 msgr2=0x5556308b6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:30.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.389+0000 7fc3b6800640 1 --2- 172.21.15.5:0/1772554338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556308ac180 0x5556308b6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.389+0000 7fc3b6800640 1 -- 172.21.15.5:0/1772554338 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55563080bf50 con 0x5556308ab860 2024-04-24T06:52:30.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.390+0000 7fc3b6800640 1 --2- 172.21.15.5:0/1772554338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556308ab860 0x5556308abc40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0011820 tx=0x7fc3b0011ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dcc73435c158e4a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:52:30.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.390+0000 7fc3aea00640 1 --2- 172.21.15.5:0/4208486096 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc39c0a3da0 0x7fc39c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:30.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.390+0000 7fc3aea00640 1 -- 172.21.15.5:0/4208486096 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7fc3a80f1070 con 0x7fc3b8100df0 2024-04-24T06:52:30.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.390+0000 7fc3a7000640 1 -- 172.21.15.5:0/1772554338 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b0011330 con 0x5556308ab860 2024-04-24T06:52:30.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.390+0000 7fc3a7000640 1 -- 172.21.15.5:0/1772554338 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3b000fe80 con 0x5556308ab860 2024-04-24T06:52:30.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.390+0000 7fc3a7000640 1 -- 172.21.15.5:0/1772554338 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b00193e0 con 0x5556308ab860 2024-04-24T06:52:30.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.390+0000 7fc3b5e00640 1 --2- 172.21.15.5:0/4208486096 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc39c0a3da0 0x7fc39c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:30.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.391+0000 7fc3bfa33740 1 -- 172.21.15.5:0/1772554338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556308ab860 msgr2=0x5556308abc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:30.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.391+0000 7fc3bfa33740 1 --2- 172.21.15.5:0/1772554338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556308ab860 0x5556308abc40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0011820 tx=0x7fc3b0011ce0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.392+0000 7fc3bfa33740 1 -- 172.21.15.5:0/1772554338 shutdown_connections 2024-04-24T06:52:30.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.392+0000 7fc3bfa33740 1 --2- 172.21.15.5:0/1772554338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556308b6850 0x5556308b8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.392+0000 7fc3bfa33740 1 --2- 172.21.15.5:0/1772554338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556308ac180 0x5556308b6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.392+0000 7fc3bfa33740 1 --2- 172.21.15.5:0/1772554338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556308ab860 0x5556308abc40 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.392+0000 7fc3bfa33740 1 -- 172.21.15.5:0/1772554338 >> 172.21.15.5:0/1772554338 conn(0x5556308a7b80 msgr2=0x5556308a7f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:52:30.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.392+0000 7fc3b5e00640 1 --2- 172.21.15.5:0/4208486096 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc39c0a3da0 0x7fc39c0a6260 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc3a80ee860 tx=0x7fc3a803b720 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:52:30.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.392+0000 7fc3bfa33740 1 -- 172.21.15.5:0/1772554338 shutdown_connections 2024-04-24T06:52:30.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.392+0000 7fc3bfa33740 1 -- 172.21.15.5:0/1772554338 wait complete. 2024-04-24T06:52:30.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.392+0000 7fc3bfa33740 1 -- start start 2024-04-24T06:52:30.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.392+0000 7fc3bfa33740 1 Processor -- start 2024-04-24T06:52:30.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.393+0000 7fc3bfa33740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556308ab860 0x5556308b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:30.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.393+0000 7fc3bfa33740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556308ac180 0x5556308b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:30.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.393+0000 7fc3bfa33740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556308b6850 0x5556308b4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:30.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.393+0000 7fc3bfa33740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5556308b9640 con 0x5556308ab860 2024-04-24T06:52:30.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.393+0000 7fc3bfa33740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5556307ca690 con 0x5556308b6850 2024-04-24T06:52:30.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.393+0000 7fc3bfa33740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5556308b97c0 con 0x5556308ac180 2024-04-24T06:52:30.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.393+0000 7fc3bfa33740 10 client.0 _ll_register_callbacks cb 0x555630806650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:52:30.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.393+0000 7fc3afe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556308ac180 0x5556308b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:30.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.393+0000 7fc3b6800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556308ab860 0x5556308b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:30.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.393+0000 7fc3b6800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556308ab860 0x5556308b37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38310/0 (socket says 172.21.15.5:38310) 2024-04-24T06:52:30.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.393+0000 7fc3b6800640 1 -- 172.21.15.5:0/346782500 learned_addr learned my addr 172.21.15.5:0/346782500 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:52:30.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.394+0000 7fc3b6800640 1 --2- 172.21.15.5:0/346782500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556308b6850 0x5556308b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:30.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.394+0000 7fc3afe00640 1 -- 172.21.15.5:0/346782500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556308b6850 msgr2=0x5556308b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:30.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.394+0000 7fc3afe00640 1 --2- 172.21.15.5:0/346782500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556308b6850 0x5556308b4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.394+0000 7fc3afe00640 1 -- 172.21.15.5:0/346782500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556308ab860 msgr2=0x5556308b37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:30.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.394+0000 7fc3afe00640 1 --2- 172.21.15.5:0/346782500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556308ab860 0x5556308b37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.394+0000 7fc3afe00640 1 -- 172.21.15.5:0/346782500 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556307c9920 con 0x5556308ac180 2024-04-24T06:52:30.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.394+0000 7fc3b6800640 1 --2- 172.21.15.5:0/346782500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556308ab860 0x5556308b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:52:30.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.394+0000 7fc3afe00640 1 --2- 172.21.15.5:0/346782500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556308ac180 0x5556308b3cf0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc3b81027e0 tx=0x7fc3a0012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd6ab83e9d24f2d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:52:30.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.394+0000 7fc39b400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:52:30.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.395+0000 7fc39b400640 1 -- 172.21.15.5:0/346782500 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a000f8f0 con 0x5556308ac180 2024-04-24T06:52:30.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.395+0000 7fc3bfa33740 1 -- 172.21.15.5:0/346782500 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55563081ec10 con 0x5556308ac180 2024-04-24T06:52:30.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.395+0000 7fc39b400640 1 -- 172.21.15.5:0/346782500 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7fc3a000fa90 con 0x5556308ac180 2024-04-24T06:52:30.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.395+0000 7fc39b400640 1 -- 172.21.15.5:0/346782500 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc3a000fc90 con 0x5556308ac180 2024-04-24T06:52:30.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.395+0000 7fc3bfa33740 10 client.27127 fetch_fsmap learned FSMap version 27 2024-04-24T06:52:30.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.395+0000 7fc3bfa33740 1 -- 172.21.15.5:0/346782500 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5556305ed7d0 con 0x5556308ac180 2024-04-24T06:52:30.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.396+0000 7fc39b400640 1 -- 172.21.15.5:0/346782500 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55563081ec10 con 0x5556308ac180 2024-04-24T06:52:30.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.396+0000 7fc39b400640 1 -- 172.21.15.5:0/346782500 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7fc3a001c420 con 0x5556308ac180 2024-04-24T06:52:30.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.396+0000 7fc3bfa33740 10 client.27127 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:52:30.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.396+0000 7fc3bfa33740 10 client.27127 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:52:30.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.396+0000 7fc3bfa33740 20 client.27127 populate_metadata read hostname 'smithi005' 2024-04-24T06:52:30.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.396+0000 7fc3bfa33740 1 --2- 172.21.15.5:0/346782500 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5556308cf400 0x5556308d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:30.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.396+0000 7fc3bfa33740 4 client.27127 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:52:30.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.396+0000 7fc3bfa33740 1 -- 172.21.15.5:0/346782500 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555630818a80 con 0x5556308cf400 2024-04-24T06:52:30.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.396+0000 7fc3b6800640 1 --2- 172.21.15.5:0/346782500 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5556308cf400 0x5556308d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:30.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.397+0000 7fc3b6800640 1 --2- 172.21.15.5:0/346782500 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5556308cf400 0x5556308d1830 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91cd9ee6ddc284e0 server_cookie=561df0beb6dab325 in_seq=0 out_seq=0 2024-04-24T06:52:30.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.397+0000 7fc39b400640 10 client.27127 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:52:30.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.398+0000 7fc39b400640 1 -- 172.21.15.5:0/346782500 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555630818a80 con 0x5556308cf400 2024-04-24T06:52:30.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.398+0000 7fc39b400640 10 client.27127 handle_command_reply: tid=0 2024-04-24T06:52:30.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.398+0000 7fc3bfa33740 1 client.27127 shutdown 2024-04-24T06:52:30.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.399+0000 7fc3bfa33740 1 -- 172.21.15.5:0/346782500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556308ac180 msgr2=0x5556308b3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:30.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.399+0000 7fc3bfa33740 1 --2- 172.21.15.5:0/346782500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556308ac180 0x5556308b3cf0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc3b81027e0 tx=0x7fc3a0012b60 comp rx=0 tx=0).stop 2024-04-24T06:52:30.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.399+0000 7fc3bfa33740 1 -- 172.21.15.5:0/346782500 shutdown_connections 2024-04-24T06:52:30.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.399+0000 7fc3bfa33740 1 --2- 172.21.15.5:0/346782500 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5556308cf400 0x5556308d1830 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.399+0000 7fc3bfa33740 1 --2- 172.21.15.5:0/346782500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556308b6850 0x5556308b4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.399+0000 7fc3bfa33740 1 --2- 172.21.15.5:0/346782500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556308ac180 0x5556308b3cf0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.399+0000 7fc3bfa33740 1 --2- 172.21.15.5:0/346782500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556308ab860 0x5556308b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.399+0000 7fc39b400640 0 client.27127 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:52:30.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.399+0000 7fc3bfa33740 1 -- 172.21.15.5:0/346782500 >> 172.21.15.5:0/346782500 conn(0x5556308a7b80 msgr2=0x5556308ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:52:30.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.399+0000 7fc3bfa33740 1 -- 172.21.15.5:0/346782500 shutdown_connections 2024-04-24T06:52:30.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.399+0000 7fc3bfa33740 1 -- 172.21.15.5:0/346782500 wait complete. 2024-04-24T06:52:30.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.399+0000 7fc3bfa33740 20 client.27127 trim_cache size 0 max 16384 2024-04-24T06:52:30.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.422+0000 7fc3bfa33740 1 Processor -- start 2024-04-24T06:52:30.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.422+0000 7fc3bfa33740 1 -- start start 2024-04-24T06:52:30.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.422+0000 7fc3bfa33740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556308ab860 0x5556308c2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:30.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.422+0000 7fc3bfa33740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556308ac180 0x5556308d7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:30.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.422+0000 7fc3bfa33740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556308b6850 0x5556308d7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:30.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.422+0000 7fc3bfa33740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b81ae580 con 0x5556308ab860 2024-04-24T06:52:30.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.422+0000 7fc3bfa33740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b81abe50 con 0x5556308ac180 2024-04-24T06:52:30.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.422+0000 7fc3bfa33740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b81a9cc0 con 0x5556308b6850 2024-04-24T06:52:30.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.423+0000 7fc3b6800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556308ab860 0x5556308c2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:30.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.423+0000 7fc3afe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556308ac180 0x5556308d7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:30.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.423+0000 7fc3afe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556308ac180 0x5556308d7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:53800/0 (socket says 172.21.15.5:53800) 2024-04-24T06:52:30.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.423+0000 7fc3afe00640 1 -- 172.21.15.5:0/2432859465 learned_addr learned my addr 172.21.15.5:0/2432859465 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:52:30.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.423+0000 7fc3b6800640 1 --2- 172.21.15.5:0/2432859465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556308b6850 0x5556308d7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:30.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.423+0000 7fc3afe00640 1 -- 172.21.15.5:0/2432859465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556308b6850 msgr2=0x5556308d7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:30.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.423+0000 7fc3afe00640 1 --2- 172.21.15.5:0/2432859465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556308b6850 0x5556308d7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.423+0000 7fc3afe00640 1 -- 172.21.15.5:0/2432859465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556308ab860 msgr2=0x5556308c2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:30.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.423+0000 7fc3afe00640 1 --2- 172.21.15.5:0/2432859465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556308ab860 0x5556308c2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.423+0000 7fc3afe00640 1 -- 172.21.15.5:0/2432859465 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556305233f0 con 0x5556308ac180 2024-04-24T06:52:30.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.423+0000 7fc3b6800640 1 --2- 172.21.15.5:0/2432859465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556308ab860 0x5556308c2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:52:30.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.424+0000 7fc3afe00640 1 --2- 172.21.15.5:0/2432859465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556308ac180 0x5556308d7140 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0009e00 tx=0x7fc3a00098c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5d10e3468b5b00 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:52:30.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.424+0000 7fc3a6600640 1 -- 172.21.15.5:0/2432859465 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0000e60 con 0x5556308ac180 2024-04-24T06:52:30.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.424+0000 7fc3a6600640 1 -- 172.21.15.5:0/2432859465 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3a000fe30 con 0x5556308ac180 2024-04-24T06:52:30.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.424+0000 7fc3a6600640 1 -- 172.21.15.5:0/2432859465 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a001c680 con 0x5556308ac180 2024-04-24T06:52:30.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.425+0000 7fc3bfa33740 1 -- 172.21.15.5:0/2432859465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556308ac180 msgr2=0x5556308d7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:30.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.425+0000 7fc3bfa33740 1 --2- 172.21.15.5:0/2432859465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556308ac180 0x5556308d7140 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0009e00 tx=0x7fc3a00098c0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.425+0000 7fc3bfa33740 1 -- 172.21.15.5:0/2432859465 shutdown_connections 2024-04-24T06:52:30.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.425+0000 7fc3bfa33740 1 --2- 172.21.15.5:0/2432859465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556308b6850 0x5556308d7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.425+0000 7fc3bfa33740 1 --2- 172.21.15.5:0/2432859465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556308ac180 0x5556308d7140 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.425+0000 7fc3bfa33740 1 --2- 172.21.15.5:0/2432859465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556308ab860 0x5556308c2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.425+0000 7fc3bfa33740 1 -- 172.21.15.5:0/2432859465 >> 172.21.15.5:0/2432859465 conn(0x5556308a7b80 msgr2=0x5556308a7f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:52:30.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.425+0000 7fc3bfa33740 1 -- 172.21.15.5:0/2432859465 shutdown_connections 2024-04-24T06:52:30.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.425+0000 7fc3bfa33740 1 -- 172.21.15.5:0/2432859465 wait complete. 2024-04-24T06:52:30.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.426+0000 7fc3bfa33740 1 -- start start 2024-04-24T06:52:30.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.426+0000 7fc3bfa33740 1 Processor -- start 2024-04-24T06:52:30.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.426+0000 7fc3bfa33740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556308ab860 0x5556308bec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:30.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.426+0000 7fc3bfa33740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556308ac180 0x5556308b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:30.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.426+0000 7fc3bfa33740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556308b6850 0x5556308b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:30.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.426+0000 7fc3bfa33740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b81aa310 con 0x5556308ab860 2024-04-24T06:52:30.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.426+0000 7fc3bfa33740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b81ab780 con 0x5556308b6850 2024-04-24T06:52:30.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.426+0000 7fc3bfa33740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b81aa9e0 con 0x5556308ac180 2024-04-24T06:52:30.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.427+0000 7fc3bfa33740 10 client.0 _ll_register_callbacks cb 0x7fc3a0009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:52:30.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.427+0000 7fc3afe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556308ac180 0x5556308b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:30.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.427+0000 7fc3afe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556308ac180 0x5556308b0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:39180/0 (socket says 172.21.15.5:39180) 2024-04-24T06:52:30.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.427+0000 7fc3b6800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556308ab860 0x5556308bec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:30.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.427+0000 7fc3afe00640 1 -- 172.21.15.5:0/1277543648 learned_addr learned my addr 172.21.15.5:0/1277543648 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:52:30.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.427+0000 7fc3b6800640 1 --2- 172.21.15.5:0/1277543648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556308b6850 0x5556308b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:30.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.427+0000 7fc3afe00640 1 -- 172.21.15.5:0/1277543648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556308b6850 msgr2=0x5556308b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:30.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.427+0000 7fc3afe00640 1 --2- 172.21.15.5:0/1277543648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556308b6850 0x5556308b0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.427+0000 7fc3afe00640 1 -- 172.21.15.5:0/1277543648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556308ab860 msgr2=0x5556308bec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:52:30.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.427+0000 7fc3afe00640 1 --2- 172.21.15.5:0/1277543648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556308ab860 0x5556308bec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.427+0000 7fc3afe00640 1 -- 172.21.15.5:0/1277543648 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556304e30f0 con 0x5556308ac180 2024-04-24T06:52:30.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.428+0000 7fc3afe00640 1 --2- 172.21.15.5:0/1277543648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556308ac180 0x5556308b0000 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc3a000f930 tx=0x7fc3a001cc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65ec8fd703fe49f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:52:30.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.428+0000 7fc39b400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:52:30.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.428+0000 7fc39b400640 1 -- 172.21.15.5:0/1277543648 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a00052c0 con 0x5556308ac180 2024-04-24T06:52:30.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.428+0000 7fc39b400640 1 -- 172.21.15.5:0/1277543648 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7fc3a0005460 con 0x5556308ac180 2024-04-24T06:52:30.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.428+0000 7fc3bfa33740 1 -- 172.21.15.5:0/1277543648 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3a000fe30 con 0x5556308ac180 2024-04-24T06:52:30.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.428+0000 7fc39b400640 1 -- 172.21.15.5:0/1277543648 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc3a003d050 con 0x5556308ac180 2024-04-24T06:52:30.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.428+0000 7fc3bfa33740 10 client.27133 fetch_fsmap learned FSMap version 27 2024-04-24T06:52:30.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.428+0000 7fc3bfa33740 1 -- 172.21.15.5:0/1277543648 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5556307d6810 con 0x5556308ac180 2024-04-24T06:52:30.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.429+0000 7fc39b400640 1 -- 172.21.15.5:0/1277543648 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3a000fe30 con 0x5556308ac180 2024-04-24T06:52:30.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.429+0000 7fc39b400640 1 -- 172.21.15.5:0/1277543648 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7fc3a002b020 con 0x5556308ac180 2024-04-24T06:52:30.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.429+0000 7fc3bfa33740 10 client.27133 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:52:30.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.429+0000 7fc3bfa33740 10 client.27133 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:52:30.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.429+0000 7fc3bfa33740 20 client.27133 populate_metadata read hostname 'smithi005' 2024-04-24T06:52:30.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.429+0000 7fc3bfa33740 1 --2- 172.21.15.5:0/1277543648 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x555630935b50 0x555630937f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:52:30.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.429+0000 7fc3bfa33740 4 client.27133 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:52:30.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.429+0000 7fc3bfa33740 1 -- 172.21.15.5:0/1277543648 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555630794360 con 0x555630935b50 2024-04-24T06:52:30.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.429+0000 7fc3b6800640 1 --2- 172.21.15.5:0/1277543648 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x555630935b50 0x555630937f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:52:30.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.430+0000 7fc39b400640 10 client.27133 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:52:30.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.430+0000 7fc3b6800640 1 --2- 172.21.15.5:0/1277543648 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x555630935b50 0x555630937f70 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb030b337da05a96 server_cookie=a92093d00f10e4cc in_seq=0 out_seq=0 2024-04-24T06:52:30.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.430+0000 7fc39b400640 1 -- 172.21.15.5:0/1277543648 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x555630794360 con 0x555630935b50 2024-04-24T06:52:30.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.430+0000 7fc39b400640 10 client.27133 handle_command_reply: tid=0 2024-04-24T06:52:30.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.430+0000 7fc3bfa33740 1 client.27133 shutdown 2024-04-24T06:52:30.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.431+0000 7fc3bfa33740 1 -- 172.21.15.5:0/1277543648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556308ac180 msgr2=0x5556308b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:30.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.431+0000 7fc3bfa33740 1 --2- 172.21.15.5:0/1277543648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556308ac180 0x5556308b0000 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc3a000f930 tx=0x7fc3a001cc90 comp rx=0 tx=0).stop 2024-04-24T06:52:30.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.431+0000 7fc3bfa33740 1 -- 172.21.15.5:0/1277543648 shutdown_connections 2024-04-24T06:52:30.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.431+0000 7fc3bfa33740 1 --2- 172.21.15.5:0/1277543648 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x555630935b50 0x555630937f70 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.431+0000 7fc3bfa33740 1 --2- 172.21.15.5:0/1277543648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556308b6850 0x5556308b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.431+0000 7fc3bfa33740 1 --2- 172.21.15.5:0/1277543648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556308ac180 0x5556308b0000 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.431+0000 7fc3bfa33740 1 --2- 172.21.15.5:0/1277543648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556308ab860 0x5556308bec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.431+0000 7fc3bfa33740 1 -- 172.21.15.5:0/1277543648 >> 172.21.15.5:0/1277543648 conn(0x5556308a7b80 msgr2=0x5556308b3770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:52:30.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.431+0000 7fc39b400640 0 client.27133 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:52:30.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.431+0000 7fc3bfa33740 1 -- 172.21.15.5:0/1277543648 shutdown_connections 2024-04-24T06:52:30.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.431+0000 7fc3bfa33740 1 -- 172.21.15.5:0/1277543648 wait complete. 2024-04-24T06:52:30.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.431+0000 7fc3bfa33740 20 client.27133 trim_cache size 0 max 16384 2024-04-24T06:52:30.439 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:52:30.439 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (903 inodes in the stack)", 2024-04-24T06:52:30.440 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-24T06:52:30.440 INFO:teuthology.orchestra.run.smithi005.stdout: "28a88227-5708-40cc-991c-5b397b878e62": { 2024-04-24T06:52:30.440 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-24T06:52:30.440 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "28a88227-5708-40cc-991c-5b397b878e62", 2024-04-24T06:52:30.440 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-04-24T06:52:30.440 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:52:30.440 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:52:30.440 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:52:30.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.432+0000 7fc3bcc00640 1 -- 172.21.15.5:0/4208486096 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc39c0a3da0 msgr2=0x7fc39c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:30.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.432+0000 7fc3bcc00640 1 --2- 172.21.15.5:0/4208486096 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc39c0a3da0 0x7fc39c0a6260 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc3a80ee860 tx=0x7fc3a803b720 comp rx=0 tx=0).stop 2024-04-24T06:52:30.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.432+0000 7fc3bcc00640 1 -- 172.21.15.5:0/4208486096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3b8100df0 msgr2=0x7fc3b81a0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:52:30.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.432+0000 7fc3bcc00640 1 --2- 172.21.15.5:0/4208486096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3b8100df0 0x7fc3b81a0730 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc3a802a920 tx=0x7fc3a8003b20 comp rx=0 tx=0).stop 2024-04-24T06:52:30.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.432+0000 7fc3bcc00640 1 -- 172.21.15.5:0/4208486096 shutdown_connections 2024-04-24T06:52:30.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.432+0000 7fc3bcc00640 1 --2- 172.21.15.5:0/4208486096 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc39c0a3da0 0x7fc39c0a6260 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.432+0000 7fc3bcc00640 1 --2- 172.21.15.5:0/4208486096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3b810e9a0 0x7fc3b819a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.432+0000 7fc3bcc00640 1 --2- 172.21.15.5:0/4208486096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3b8101710 0x7fc3b81a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.432+0000 7fc3bcc00640 1 --2- 172.21.15.5:0/4208486096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3b8100df0 0x7fc3b81a0730 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:52:30.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.432+0000 7fc3bcc00640 1 -- 172.21.15.5:0/4208486096 >> 172.21.15.5:0/4208486096 conn(0x7fc3b80780b0 msgr2=0x7fc3b810fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:52:30.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.433+0000 7fc3bcc00640 1 -- 172.21.15.5:0/4208486096 shutdown_connections 2024-04-24T06:52:30.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:52:30.433+0000 7fc3bcc00640 1 -- 172.21.15.5:0/4208486096 wait complete. 2024-04-24T06:52:30.453 INFO:tasks.cephfs.filesystem:scrub status for tag:28a88227-5708-40cc-991c-5b397b878e62 - {'path': '/', 'tag': '28a88227-5708-40cc-991c-5b397b878e62', 'options': 'recursive,force'} 2024-04-24T06:52:31.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:31 smithi137 ceph-mon[33017]: scrub queued for path: / 2024-04-24T06:52:31.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:31 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:52:31.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:31 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:52:31.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:31 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:52:31.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:31 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:52:31.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:31 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:52:31.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:31 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:52:31.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:31 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:52:31.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:31 smithi137 ceph-mon[33017]: pgmap v261: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:52:31.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:31 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:52:31.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:31 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:52:31.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:31 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:52:31.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:31 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:52:31.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:31 smithi005 ceph-mon[27798]: scrub queued for path: / 2024-04-24T06:52:31.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:31 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:52:31.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:31 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:52:31.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:31 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:52:31.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:31 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:52:31.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:31 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:52:31.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:31 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:52:31.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:31 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:52:31.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:31 smithi005 ceph-mon[27798]: pgmap v261: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:52:31.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:31 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:52:31.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:31 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:52:31.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:31 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:52:31.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:31 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:52:31.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:31 smithi037 ceph-mon[32864]: scrub queued for path: / 2024-04-24T06:52:31.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:31 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:52:31.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:31 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:52:31.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:31 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:52:31.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:31 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:52:31.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:31 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:52:31.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:31 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:52:31.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:31 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:52:31.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:31 smithi037 ceph-mon[32864]: pgmap v261: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:52:31.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:31 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:52:31.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:31 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:52:31.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:31 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:52:31.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:31 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:52:33.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:33 smithi137 ceph-mon[33017]: pgmap v262: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:52:33.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:33 smithi005 ceph-mon[27798]: pgmap v262: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:52:33.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:33 smithi037 ceph-mon[32864]: pgmap v262: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:52:35.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:35 smithi137 ceph-mon[33017]: pgmap v263: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:52:35.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:35 smithi005 ceph-mon[27798]: pgmap v263: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:52:35.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:35 smithi037 ceph-mon[32864]: pgmap v263: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:52:37.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:37 smithi137 ceph-mon[33017]: pgmap v264: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:52:37.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:37 smithi005 ceph-mon[27798]: pgmap v264: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:52:37.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:37 smithi037 ceph-mon[32864]: pgmap v264: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:52:39.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:39 smithi137 ceph-mon[33017]: pgmap v265: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:52:39.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:39 smithi005 ceph-mon[27798]: pgmap v265: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:52:39.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:39 smithi037 ceph-mon[32864]: pgmap v265: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:52:41.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:41 smithi137 ceph-mon[33017]: pgmap v266: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:52:41.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:41 smithi005 ceph-mon[27798]: pgmap v266: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:52:41.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:41 smithi037 ceph-mon[32864]: pgmap v266: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:52:43.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:43 smithi137 ceph-mon[33017]: pgmap v267: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:52:43.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:43 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:52:43.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:43 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:52:43.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:43 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:52:43.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:43 smithi005 ceph-mon[27798]: pgmap v267: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:52:43.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:43 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:52:43.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:43 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:52:43.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:43 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:52:43.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:43 smithi037 ceph-mon[32864]: pgmap v267: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:52:43.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:43 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:52:43.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:43 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:52:43.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:43 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:52:45.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:45 smithi137 ceph-mon[33017]: pgmap v268: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:52:45.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:45 smithi005 ceph-mon[27798]: pgmap v268: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:52:45.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:45 smithi037 ceph-mon[32864]: pgmap v268: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:52:47.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:47 smithi137 ceph-mon[33017]: pgmap v269: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:52:47.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:47 smithi005 ceph-mon[27798]: pgmap v269: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:52:47.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:47 smithi037 ceph-mon[32864]: pgmap v269: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:52:49.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:49 smithi137 ceph-mon[33017]: pgmap v270: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:52:49.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:49 smithi005 ceph-mon[27798]: pgmap v270: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:52:49.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:49 smithi037 ceph-mon[32864]: pgmap v270: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:52:51.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:51 smithi137 ceph-mon[33017]: pgmap v271: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:52:51.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:51 smithi005 ceph-mon[27798]: pgmap v271: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:52:51.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:51 smithi037 ceph-mon[32864]: pgmap v271: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:52:53.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:53 smithi137 ceph-mon[33017]: pgmap v272: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:52:53.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:53 smithi005 ceph-mon[27798]: pgmap v272: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:52:53.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:53 smithi037 ceph-mon[32864]: pgmap v272: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:52:55.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:55 smithi137 ceph-mon[33017]: pgmap v273: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:52:55.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:55 smithi005 ceph-mon[27798]: pgmap v273: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:52:55.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:55 smithi037 ceph-mon[32864]: pgmap v273: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:52:57.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:57 smithi137 ceph-mon[33017]: pgmap v274: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:52:57.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:57 smithi005 ceph-mon[27798]: pgmap v274: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:52:57.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:57 smithi037 ceph-mon[32864]: pgmap v274: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:52:59.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:52:59 smithi137 ceph-mon[33017]: pgmap v275: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:52:59.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:52:59 smithi005 ceph-mon[27798]: pgmap v275: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:52:59.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:52:59 smithi037 ceph-mon[32864]: pgmap v275: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:53:00.455 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T06:53:00.455 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T06:53:00.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.689+0000 7faeff800640 1 -- 172.21.15.5:0/4047067784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faef8074cf0 msgr2=0x7faef810e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:00.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.689+0000 7faeff800640 1 --2- 172.21.15.5:0/4047067784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faef8074cf0 0x7faef810e320 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7faee8009860 tx=0x7faee802a3d0 comp rx=0 tx=0).stop 2024-04-24T06:53:00.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.689+0000 7faeff800640 1 -- 172.21.15.5:0/4047067784 shutdown_connections 2024-04-24T06:53:00.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.689+0000 7faeff800640 1 --2- 172.21.15.5:0/4047067784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faef810e860 0x7faef8110c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:00.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.689+0000 7faeff800640 1 --2- 172.21.15.5:0/4047067784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faef8074cf0 0x7faef810e320 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:00.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.689+0000 7faeff800640 1 --2- 172.21.15.5:0/4047067784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faef80743d0 0x7faef80747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:00.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.689+0000 7faeff800640 1 -- 172.21.15.5:0/4047067784 >> 172.21.15.5:0/4047067784 conn(0x7faef80fdab0 msgr2=0x7faef80ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:00.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.690+0000 7faeff800640 1 -- 172.21.15.5:0/4047067784 shutdown_connections 2024-04-24T06:53:00.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.690+0000 7faeff800640 1 -- 172.21.15.5:0/4047067784 wait complete. 2024-04-24T06:53:00.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.691+0000 7faeff800640 1 Processor -- start 2024-04-24T06:53:00.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.691+0000 7faeff800640 1 -- start start 2024-04-24T06:53:00.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.691+0000 7faeff800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faef80743d0 0x7faef81a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:00.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.691+0000 7faeff800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faef8074cf0 0x7faef81a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:00.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.691+0000 7faeff800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faef810e860 0x7faef819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:00.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.692+0000 7faeff800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faef8114610 con 0x7faef80743d0 2024-04-24T06:53:00.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.692+0000 7faeff800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faef8114490 con 0x7faef810e860 2024-04-24T06:53:00.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.692+0000 7faeff800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faef8114790 con 0x7faef8074cf0 2024-04-24T06:53:00.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.692+0000 7faef7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faef8074cf0 0x7faef81a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:00.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.692+0000 7faefce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faef80743d0 0x7faef81a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:00.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.692+0000 7faef7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faef8074cf0 0x7faef81a0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:58668/0 (socket says 172.21.15.5:58668) 2024-04-24T06:53:00.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.692+0000 7faef7e00640 1 -- 172.21.15.5:0/1160455728 learned_addr learned my addr 172.21.15.5:0/1160455728 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:00.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.692+0000 7faefd800640 1 --2- 172.21.15.5:0/1160455728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faef810e860 0x7faef819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:00.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.692+0000 7faefce00640 1 -- 172.21.15.5:0/1160455728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faef8074cf0 msgr2=0x7faef81a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:00.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.692+0000 7faefce00640 1 --2- 172.21.15.5:0/1160455728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faef8074cf0 0x7faef81a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:00.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.692+0000 7faefce00640 1 -- 172.21.15.5:0/1160455728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faef810e860 msgr2=0x7faef819a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:00.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.692+0000 7faefce00640 1 --2- 172.21.15.5:0/1160455728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faef810e860 0x7faef819a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:00.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.692+0000 7faefce00640 1 -- 172.21.15.5:0/1160455728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef819b150 con 0x7faef80743d0 2024-04-24T06:53:00.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.692+0000 7faefd800640 1 --2- 172.21.15.5:0/1160455728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faef810e860 0x7faef819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:53:00.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.693+0000 7faefce00640 1 --2- 172.21.15.5:0/1160455728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faef80743d0 0x7faef81a0820 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7faeec005fd0 tx=0x7faeec00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:00.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.693+0000 7faef5600640 1 -- 172.21.15.5:0/1160455728 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec00cd00 con 0x7faef80743d0 2024-04-24T06:53:00.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.693+0000 7faef5600640 1 -- 172.21.15.5:0/1160455728 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faeec0035f0 con 0x7faef80743d0 2024-04-24T06:53:00.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.693+0000 7faeff800640 1 -- 172.21.15.5:0/1160455728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faef819b440 con 0x7faef80743d0 2024-04-24T06:53:00.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.694+0000 7faef5600640 1 -- 172.21.15.5:0/1160455728 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec010670 con 0x7faef80743d0 2024-04-24T06:53:00.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.694+0000 7faeff800640 1 -- 172.21.15.5:0/1160455728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faef81a76e0 con 0x7faef80743d0 2024-04-24T06:53:00.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.697+0000 7faf025b9740 1 Processor -- start 2024-04-24T06:53:00.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.698+0000 7faf025b9740 1 -- start start 2024-04-24T06:53:00.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.698+0000 7faef5600640 1 -- 172.21.15.5:0/1160455728 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7faeec0037a0 con 0x7faef80743d0 2024-04-24T06:53:00.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.698+0000 7faf025b9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c7a00a860 0x564c7a00ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:00.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.698+0000 7faf025b9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564c7a00b180 0x564c7a015310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:00.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.698+0000 7faf025b9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7a015850 0x564c7a017c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:00.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.698+0000 7faf025b9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564c79f701f0 con 0x564c7a015850 2024-04-24T06:53:00.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.698+0000 7faf025b9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564c79f16350 con 0x564c7a00b180 2024-04-24T06:53:00.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.698+0000 7faf025b9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564c79f6c1b0 con 0x564c7a00a860 2024-04-24T06:53:00.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.698+0000 7faefce00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564c7a00b180 0x564c7a015310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:00.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.698+0000 7faefd800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c7a00a860 0x564c7a00ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:00.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.698+0000 7faefce00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564c7a00b180 0x564c7a015310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:35994/0 (socket says 172.21.15.5:35994) 2024-04-24T06:53:00.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.698+0000 7faefce00640 1 -- 172.21.15.5:0/14899591 learned_addr learned my addr 172.21.15.5:0/14899591 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:00.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.698+0000 7faef7e00640 1 --2- 172.21.15.5:0/14899591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7a015850 0x564c7a017c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:00.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.699+0000 7faefd800640 1 -- 172.21.15.5:0/14899591 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564c7a00b180 msgr2=0x564c7a015310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:00.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.699+0000 7faefd800640 1 --2- 172.21.15.5:0/14899591 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564c7a00b180 0x564c7a015310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:00.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.699+0000 7faefd800640 1 -- 172.21.15.5:0/14899591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7a015850 msgr2=0x564c7a017c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:00.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.699+0000 7faefd800640 1 --2- 172.21.15.5:0/14899591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7a015850 0x564c7a017c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:00.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.699+0000 7faefd800640 1 -- 172.21.15.5:0/14899591 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c79f6af50 con 0x564c7a00a860 2024-04-24T06:53:00.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.699+0000 7faefd800640 1 --2- 172.21.15.5:0/14899591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c7a00a860 0x564c7a00ac40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7faef806b7b0 tx=0x7faef0011bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=339a9655059dabcd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:00.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.700+0000 7faee6000640 1 -- 172.21.15.5:0/14899591 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef0004d20 con 0x564c7a00a860 2024-04-24T06:53:00.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.700+0000 7faee6000640 1 -- 172.21.15.5:0/14899591 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faef000f970 con 0x564c7a00a860 2024-04-24T06:53:00.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.700+0000 7faee6000640 1 -- 172.21.15.5:0/14899591 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef0004510 con 0x564c7a00a860 2024-04-24T06:53:00.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.700+0000 7faef5600640 1 --2- 172.21.15.5:0/1160455728 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7faee00a3da0 0x7faee00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:00.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.700+0000 7faef5600640 1 -- 172.21.15.5:0/1160455728 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7faeec0d0950 con 0x7faef80743d0 2024-04-24T06:53:00.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.701+0000 7faefce00640 1 --2- 172.21.15.5:0/1160455728 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7faee00a3da0 0x7faee00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:00.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.701+0000 7faf025b9740 1 -- 172.21.15.5:0/14899591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c7a00a860 msgr2=0x564c7a00ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:00.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.701+0000 7faf025b9740 1 --2- 172.21.15.5:0/14899591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c7a00a860 0x564c7a00ac40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7faef806b7b0 tx=0x7faef0011bf0 comp rx=0 tx=0).stop 2024-04-24T06:53:00.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.701+0000 7faf025b9740 1 -- 172.21.15.5:0/14899591 shutdown_connections 2024-04-24T06:53:00.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.701+0000 7faf025b9740 1 --2- 172.21.15.5:0/14899591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7a015850 0x564c7a017c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:00.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.701+0000 7faf025b9740 1 --2- 172.21.15.5:0/14899591 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564c7a00b180 0x564c7a015310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:00.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.701+0000 7faf025b9740 1 --2- 172.21.15.5:0/14899591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c7a00a860 0x564c7a00ac40 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:00.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.701+0000 7faf025b9740 1 -- 172.21.15.5:0/14899591 >> 172.21.15.5:0/14899591 conn(0x564c7a006b80 msgr2=0x564c7a006f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:00.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.701+0000 7faf025b9740 1 -- 172.21.15.5:0/14899591 shutdown_connections 2024-04-24T06:53:00.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.701+0000 7faf025b9740 1 -- 172.21.15.5:0/14899591 wait complete. 2024-04-24T06:53:00.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.701+0000 7faefce00640 1 --2- 172.21.15.5:0/1160455728 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7faee00a3da0 0x7faee00a6260 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faeec09c850 tx=0x7faeec0d6530 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:00.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.702+0000 7faf025b9740 1 -- start start 2024-04-24T06:53:00.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.702+0000 7faf025b9740 1 Processor -- start 2024-04-24T06:53:00.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.703+0000 7faf025b9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564c7a00a860 0x564c7a012800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:00.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.703+0000 7faf025b9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7a00b180 0x564c7a012d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:00.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.703+0000 7faf025b9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c7a015850 0x564c7a013280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:00.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.703+0000 7faf025b9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564c7a018640 con 0x564c7a00b180 2024-04-24T06:53:00.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.703+0000 7faf025b9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564c79f29690 con 0x564c7a00a860 2024-04-24T06:53:00.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.703+0000 7faf025b9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564c7a0187c0 con 0x564c7a015850 2024-04-24T06:53:00.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.703+0000 7faf025b9740 10 client.0 _ll_register_callbacks cb 0x564c79f65650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:53:00.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.703+0000 7faefd800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564c7a00a860 0x564c7a012800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:00.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.703+0000 7faef7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7a00b180 0x564c7a012d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:00.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.703+0000 7faefd800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564c7a00a860 0x564c7a012800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:35998/0 (socket says 172.21.15.5:35998) 2024-04-24T06:53:00.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.703+0000 7faefd800640 1 -- 172.21.15.5:0/3130346794 learned_addr learned my addr 172.21.15.5:0/3130346794 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:00.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.703+0000 7faefd800640 1 --2- 172.21.15.5:0/3130346794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c7a015850 0x564c7a013280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:00.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.704+0000 7faef7e00640 1 -- 172.21.15.5:0/3130346794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c7a015850 msgr2=0x564c7a013280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:00.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.704+0000 7faef7e00640 1 --2- 172.21.15.5:0/3130346794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c7a015850 0x564c7a013280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:00.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.704+0000 7faef7e00640 1 -- 172.21.15.5:0/3130346794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564c7a00a860 msgr2=0x564c7a012800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:00.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.704+0000 7faef7e00640 1 --2- 172.21.15.5:0/3130346794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564c7a00a860 0x564c7a012800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:00.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.704+0000 7faef7e00640 1 -- 172.21.15.5:0/3130346794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c79f28920 con 0x564c7a00b180 2024-04-24T06:53:00.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.704+0000 7faef7e00640 1 --2- 172.21.15.5:0/3130346794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7a00b180 0x564c7a012d40 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7faee802a9c0 tx=0x7faee80305d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3acc3b1e014a3048 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:00.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.704+0000 7faedaa00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:53:00.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.705+0000 7faedaa00640 1 -- 172.21.15.5:0/3130346794 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee8005230 con 0x564c7a00b180 2024-04-24T06:53:00.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.705+0000 7faedaa00640 1 -- 172.21.15.5:0/3130346794 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7faee80053d0 con 0x564c7a00b180 2024-04-24T06:53:00.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.705+0000 7faf025b9740 1 -- 172.21.15.5:0/3130346794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faef000f970 con 0x564c7a00b180 2024-04-24T06:53:00.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.705+0000 7faedaa00640 1 -- 172.21.15.5:0/3130346794 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faee80055d0 con 0x564c7a00b180 2024-04-24T06:53:00.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.705+0000 7faf025b9740 10 client.16983 fetch_fsmap learned FSMap version 27 2024-04-24T06:53:00.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.705+0000 7faf025b9740 1 -- 172.21.15.5:0/3130346794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564c79d4c7d0 con 0x564c7a00b180 2024-04-24T06:53:00.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.705+0000 7faedaa00640 1 -- 172.21.15.5:0/3130346794 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faef000f970 con 0x564c7a00b180 2024-04-24T06:53:00.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.706+0000 7faedaa00640 1 -- 172.21.15.5:0/3130346794 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7faee803d790 con 0x564c7a00b180 2024-04-24T06:53:00.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.706+0000 7faf025b9740 10 client.16983 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:53:00.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.706+0000 7faf025b9740 10 client.16983 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:53:00.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.706+0000 7faf025b9740 20 client.16983 populate_metadata read hostname 'smithi005' 2024-04-24T06:53:00.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.706+0000 7faf025b9740 1 --2- 172.21.15.5:0/3130346794 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x564c7a02e430 0x564c7a030820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:00.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.706+0000 7faf025b9740 4 client.16983 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:53:00.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.706+0000 7faf025b9740 1 -- 172.21.15.5:0/3130346794 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c79f7dc10 con 0x564c7a02e430 2024-04-24T06:53:00.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.706+0000 7faefd800640 1 --2- 172.21.15.5:0/3130346794 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x564c7a02e430 0x564c7a030820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:00.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.707+0000 7faefd800640 1 --2- 172.21.15.5:0/3130346794 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x564c7a02e430 0x564c7a030820 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dbf0a065b5ca6bb server_cookie=2c46de88905cddac in_seq=0 out_seq=0 2024-04-24T06:53:00.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.707+0000 7faedaa00640 10 client.16983 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:53:00.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.708+0000 7faedaa00640 1 -- 172.21.15.5:0/3130346794 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564c79f7dc10 con 0x564c7a02e430 2024-04-24T06:53:00.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.708+0000 7faedaa00640 10 client.16983 handle_command_reply: tid=0 2024-04-24T06:53:00.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.708+0000 7faf025b9740 1 client.16983 shutdown 2024-04-24T06:53:00.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.708+0000 7faf025b9740 1 -- 172.21.15.5:0/3130346794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7a00b180 msgr2=0x564c7a012d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:00.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.708+0000 7faf025b9740 1 --2- 172.21.15.5:0/3130346794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7a00b180 0x564c7a012d40 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7faee802a9c0 tx=0x7faee80305d0 comp rx=0 tx=0).stop 2024-04-24T06:53:00.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.708+0000 7faf025b9740 1 -- 172.21.15.5:0/3130346794 shutdown_connections 2024-04-24T06:53:00.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.708+0000 7faf025b9740 1 --2- 172.21.15.5:0/3130346794 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x564c7a02e430 0x564c7a030820 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:00.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.708+0000 7faf025b9740 1 --2- 172.21.15.5:0/3130346794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c7a015850 0x564c7a013280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:00.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.708+0000 7faf025b9740 1 --2- 172.21.15.5:0/3130346794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7a00b180 0x564c7a012d40 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:00.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.708+0000 7faf025b9740 1 --2- 172.21.15.5:0/3130346794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564c7a00a860 0x564c7a012800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:00.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.708+0000 7faf025b9740 1 -- 172.21.15.5:0/3130346794 >> 172.21.15.5:0/3130346794 conn(0x564c7a006b80 msgr2=0x564c7a00c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:00.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.708+0000 7faedaa00640 0 client.16983 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:53:00.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.709+0000 7faf025b9740 1 -- 172.21.15.5:0/3130346794 shutdown_connections 2024-04-24T06:53:00.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.709+0000 7faf025b9740 1 -- 172.21.15.5:0/3130346794 wait complete. 2024-04-24T06:53:00.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.709+0000 7faf025b9740 20 client.16983 trim_cache size 0 max 16384 2024-04-24T06:53:00.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.735+0000 7faf025b9740 1 Processor -- start 2024-04-24T06:53:00.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.735+0000 7faf025b9740 1 -- start start 2024-04-24T06:53:00.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.735+0000 7faf025b9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7a00a860 0x564c7a011740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:00.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.735+0000 7faf025b9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564c7a00b180 0x564c7a036080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:00.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.735+0000 7faf025b9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c7a015850 0x564c7a0365c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:00.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.735+0000 7faf025b9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faef81ae510 con 0x564c7a00a860 2024-04-24T06:53:00.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.735+0000 7faf025b9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faef81abde0 con 0x564c7a00b180 2024-04-24T06:53:00.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.735+0000 7faf025b9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faef81a9c70 con 0x564c7a015850 2024-04-24T06:53:00.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.735+0000 7faef7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564c7a00b180 0x564c7a036080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:00.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.735+0000 7faef7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564c7a00b180 0x564c7a036080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:36008/0 (socket says 172.21.15.5:36008) 2024-04-24T06:53:00.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.736+0000 7faef7e00640 1 -- 172.21.15.5:0/2901999161 learned_addr learned my addr 172.21.15.5:0/2901999161 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:00.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.736+0000 7faefd800640 1 --2- 172.21.15.5:0/2901999161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c7a015850 0x564c7a0365c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:00.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.736+0000 7faef7e00640 1 -- 172.21.15.5:0/2901999161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c7a015850 msgr2=0x564c7a0365c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:00.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.736+0000 7faef7e00640 1 --2- 172.21.15.5:0/2901999161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c7a015850 0x564c7a0365c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:00.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.736+0000 7faef7e00640 1 -- 172.21.15.5:0/2901999161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7a00a860 msgr2=0x564c7a011740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:00.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.736+0000 7faef7e00640 1 --2- 172.21.15.5:0/2901999161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7a00a860 0x564c7a011740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:00.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.736+0000 7faef7e00640 1 -- 172.21.15.5:0/2901999161 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c79c823f0 con 0x564c7a00b180 2024-04-24T06:53:00.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.736+0000 7faef7e00640 1 --2- 172.21.15.5:0/2901999161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564c7a00b180 0x564c7a036080 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7faee803dda0 tx=0x7faee802a3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d806937dcf6c429 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:00.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.737+0000 7faee5600640 1 -- 172.21.15.5:0/2901999161 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee8053030 con 0x564c7a00b180 2024-04-24T06:53:00.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.737+0000 7faee5600640 1 -- 172.21.15.5:0/2901999161 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faee8005850 con 0x564c7a00b180 2024-04-24T06:53:00.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.737+0000 7faee5600640 1 -- 172.21.15.5:0/2901999161 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee803c8f0 con 0x564c7a00b180 2024-04-24T06:53:00.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.738+0000 7faf025b9740 1 -- 172.21.15.5:0/2901999161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564c7a00b180 msgr2=0x564c7a036080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:00.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.738+0000 7faf025b9740 1 --2- 172.21.15.5:0/2901999161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564c7a00b180 0x564c7a036080 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7faee803dda0 tx=0x7faee802a3d0 comp rx=0 tx=0).stop 2024-04-24T06:53:00.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.738+0000 7faf025b9740 1 -- 172.21.15.5:0/2901999161 shutdown_connections 2024-04-24T06:53:00.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.738+0000 7faf025b9740 1 --2- 172.21.15.5:0/2901999161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c7a015850 0x564c7a0365c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:00.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.738+0000 7faf025b9740 1 --2- 172.21.15.5:0/2901999161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564c7a00b180 0x564c7a036080 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:00.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.738+0000 7faf025b9740 1 --2- 172.21.15.5:0/2901999161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7a00a860 0x564c7a011740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:00.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.738+0000 7faf025b9740 1 -- 172.21.15.5:0/2901999161 >> 172.21.15.5:0/2901999161 conn(0x564c7a006b80 msgr2=0x564c7a00c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:00.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.738+0000 7faf025b9740 1 -- 172.21.15.5:0/2901999161 shutdown_connections 2024-04-24T06:53:00.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.738+0000 7faf025b9740 1 -- 172.21.15.5:0/2901999161 wait complete. 2024-04-24T06:53:00.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.738+0000 7faf025b9740 1 -- start start 2024-04-24T06:53:00.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.739+0000 7faf025b9740 1 Processor -- start 2024-04-24T06:53:00.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.739+0000 7faf025b9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564c7a00a860 0x564c7a00f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:00.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.739+0000 7faf025b9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7a00b180 0x564c7a00f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:00.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.739+0000 7faf025b9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c7a015850 0x564c7a00fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:00.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.739+0000 7faf025b9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faef81aa2c0 con 0x564c7a00b180 2024-04-24T06:53:00.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.739+0000 7faf025b9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faef81ab710 con 0x564c7a00a860 2024-04-24T06:53:00.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.739+0000 7faf025b9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faef81aa970 con 0x564c7a015850 2024-04-24T06:53:00.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.739+0000 7faf025b9740 10 client.0 _ll_register_callbacks cb 0x7faee80096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:53:00.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.740+0000 7faef7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7a00b180 0x564c7a00f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:00.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.740+0000 7faefd800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c7a015850 0x564c7a00fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:00.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.740+0000 7faef7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7a00b180 0x564c7a00f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51822/0 (socket says 172.21.15.5:51822) 2024-04-24T06:53:00.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.740+0000 7faef7e00640 1 -- 172.21.15.5:0/1397433150 learned_addr learned my addr 172.21.15.5:0/1397433150 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:00.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.740+0000 7faefd800640 1 --2- 172.21.15.5:0/1397433150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564c7a00a860 0x564c7a00f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:00.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.740+0000 7faefd800640 1 -- 172.21.15.5:0/1397433150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564c7a00a860 msgr2=0x564c7a00f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:00.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.740+0000 7faefd800640 1 --2- 172.21.15.5:0/1397433150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564c7a00a860 0x564c7a00f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:00.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.740+0000 7faefd800640 1 -- 172.21.15.5:0/1397433150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7a00b180 msgr2=0x564c7a00f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:00.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.740+0000 7faefd800640 1 --2- 172.21.15.5:0/1397433150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7a00b180 0x564c7a00f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:00.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.740+0000 7faefd800640 1 -- 172.21.15.5:0/1397433150 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c79c420f0 con 0x564c7a015850 2024-04-24T06:53:00.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.740+0000 7faefd800640 1 --2- 172.21.15.5:0/1397433150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c7a015850 0x564c7a00fe30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7faef00041f0 tx=0x7faef001b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5abee0a96825112d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:00.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.741+0000 7faedaa00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:53:00.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.741+0000 7faedaa00640 1 -- 172.21.15.5:0/1397433150 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef0003b50 con 0x564c7a015850 2024-04-24T06:53:00.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.741+0000 7faedaa00640 1 -- 172.21.15.5:0/1397433150 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7faef001e040 con 0x564c7a015850 2024-04-24T06:53:00.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.741+0000 7faf025b9740 1 -- 172.21.15.5:0/1397433150 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faee8005850 con 0x564c7a015850 2024-04-24T06:53:00.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.741+0000 7faedaa00640 1 -- 172.21.15.5:0/1397433150 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faef0003cf0 con 0x564c7a015850 2024-04-24T06:53:00.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.741+0000 7faf025b9740 10 client.27151 fetch_fsmap learned FSMap version 27 2024-04-24T06:53:00.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.741+0000 7faf025b9740 1 -- 172.21.15.5:0/1397433150 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564c79f35810 con 0x564c7a015850 2024-04-24T06:53:00.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.742+0000 7faedaa00640 1 -- 172.21.15.5:0/1397433150 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faee8005850 con 0x564c7a015850 2024-04-24T06:53:00.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.742+0000 7faedaa00640 1 -- 172.21.15.5:0/1397433150 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7faef0018070 con 0x564c7a015850 2024-04-24T06:53:00.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.742+0000 7faf025b9740 10 client.27151 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:53:00.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.742+0000 7faf025b9740 10 client.27151 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:53:00.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.742+0000 7faf025b9740 20 client.27151 populate_metadata read hostname 'smithi005' 2024-04-24T06:53:00.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.742+0000 7faf025b9740 1 --2- 172.21.15.5:0/1397433150 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x564c7a094d10 0x564c7a097130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:00.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.742+0000 7faf025b9740 4 client.27151 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:53:00.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.742+0000 7faf025b9740 1 -- 172.21.15.5:0/1397433150 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564c79f77a80 con 0x564c7a094d10 2024-04-24T06:53:00.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.742+0000 7faef7e00640 1 --2- 172.21.15.5:0/1397433150 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x564c7a094d10 0x564c7a097130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:00.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.743+0000 7faef7e00640 1 --2- 172.21.15.5:0/1397433150 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x564c7a094d10 0x564c7a097130 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfc19898bf10e3e server_cookie=ab56348c23d19168 in_seq=0 out_seq=0 2024-04-24T06:53:00.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.743+0000 7faedaa00640 10 client.27151 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:53:00.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.743+0000 7faedaa00640 1 -- 172.21.15.5:0/1397433150 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564c79f77a80 con 0x564c7a094d10 2024-04-24T06:53:00.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.743+0000 7faedaa00640 10 client.27151 handle_command_reply: tid=0 2024-04-24T06:53:00.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.743+0000 7faf025b9740 1 client.27151 shutdown 2024-04-24T06:53:00.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.743+0000 7faf025b9740 1 -- 172.21.15.5:0/1397433150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c7a015850 msgr2=0x564c7a00fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:00.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.743+0000 7faf025b9740 1 --2- 172.21.15.5:0/1397433150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c7a015850 0x564c7a00fe30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7faef00041f0 tx=0x7faef001b040 comp rx=0 tx=0).stop 2024-04-24T06:53:00.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.743+0000 7faf025b9740 1 -- 172.21.15.5:0/1397433150 shutdown_connections 2024-04-24T06:53:00.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.743+0000 7faf025b9740 1 --2- 172.21.15.5:0/1397433150 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x564c7a094d10 0x564c7a097130 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:00.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.743+0000 7faf025b9740 1 --2- 172.21.15.5:0/1397433150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c7a015850 0x564c7a00fe30 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:00.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.743+0000 7faf025b9740 1 --2- 172.21.15.5:0/1397433150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7a00b180 0x564c7a00f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:00.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.743+0000 7faf025b9740 1 --2- 172.21.15.5:0/1397433150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564c7a00a860 0x564c7a00f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:00.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.743+0000 7faf025b9740 1 -- 172.21.15.5:0/1397433150 >> 172.21.15.5:0/1397433150 conn(0x564c7a006b80 msgr2=0x564c7a005390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:00.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.743+0000 7faedaa00640 0 client.27151 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:53:00.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.744+0000 7faf025b9740 1 -- 172.21.15.5:0/1397433150 shutdown_connections 2024-04-24T06:53:00.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.744+0000 7faf025b9740 1 -- 172.21.15.5:0/1397433150 wait complete. 2024-04-24T06:53:00.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.744+0000 7faf025b9740 20 client.27151 trim_cache size 0 max 16384 2024-04-24T06:53:00.752 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:53:00.752 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-24T06:53:00.753 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-24T06:53:00.753 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:53:00.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.744+0000 7faeff800640 1 -- 172.21.15.5:0/1160455728 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7faee00a3da0 msgr2=0x7faee00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:00.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.744+0000 7faeff800640 1 --2- 172.21.15.5:0/1160455728 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7faee00a3da0 0x7faee00a6260 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faeec09c850 tx=0x7faeec0d6530 comp rx=0 tx=0).stop 2024-04-24T06:53:00.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.745+0000 7faeff800640 1 -- 172.21.15.5:0/1160455728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faef80743d0 msgr2=0x7faef81a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:00.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.745+0000 7faeff800640 1 --2- 172.21.15.5:0/1160455728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faef80743d0 0x7faef81a0820 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7faeec005fd0 tx=0x7faeec00ee50 comp rx=0 tx=0).stop 2024-04-24T06:53:00.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.745+0000 7faeff800640 1 -- 172.21.15.5:0/1160455728 shutdown_connections 2024-04-24T06:53:00.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.745+0000 7faeff800640 1 --2- 172.21.15.5:0/1160455728 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7faee00a3da0 0x7faee00a6260 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:00.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.745+0000 7faeff800640 1 --2- 172.21.15.5:0/1160455728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faef810e860 0x7faef819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:00.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.745+0000 7faeff800640 1 --2- 172.21.15.5:0/1160455728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faef8074cf0 0x7faef81a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:00.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.745+0000 7faeff800640 1 --2- 172.21.15.5:0/1160455728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faef80743d0 0x7faef81a0820 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:00.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.745+0000 7faeff800640 1 -- 172.21.15.5:0/1160455728 >> 172.21.15.5:0/1160455728 conn(0x7faef80fdab0 msgr2=0x7faef80ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:00.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.745+0000 7faeff800640 1 -- 172.21.15.5:0/1160455728 shutdown_connections 2024-04-24T06:53:00.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.745+0000 7faeff800640 1 -- 172.21.15.5:0/1160455728 wait complete. 2024-04-24T06:53:00.765 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T06:53:00.765 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:53:00.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.956+0000 7f0c54a00640 1 -- 172.21.15.5:0/1800693956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c50104460 msgr2=0x7f0c50104840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:00.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.956+0000 7f0c54a00640 1 --2- 172.21.15.5:0/1800693956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c50104460 0x7f0c50104840 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0c40009ca0 tx=0x7f0c4002a8c0 comp rx=0 tx=0).stop 2024-04-24T06:53:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.956+0000 7f0c54a00640 1 -- 172.21.15.5:0/1800693956 shutdown_connections 2024-04-24T06:53:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.956+0000 7f0c54a00640 1 --2- 172.21.15.5:0/1800693956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c50109530 0x7f0c50111100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.956+0000 7f0c54a00640 1 --2- 172.21.15.5:0/1800693956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c50104e10 0x7f0c50108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.956+0000 7f0c54a00640 1 --2- 172.21.15.5:0/1800693956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c50104460 0x7f0c50104840 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.956+0000 7f0c54a00640 1 -- 172.21.15.5:0/1800693956 >> 172.21.15.5:0/1800693956 conn(0x7f0c500ffcc0 msgr2=0x7f0c501020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.956+0000 7f0c54a00640 1 -- 172.21.15.5:0/1800693956 shutdown_connections 2024-04-24T06:53:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.956+0000 7f0c54a00640 1 -- 172.21.15.5:0/1800693956 wait complete. 2024-04-24T06:53:00.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.957+0000 7f0c54a00640 1 Processor -- start 2024-04-24T06:53:00.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.958+0000 7f0c54a00640 1 -- start start 2024-04-24T06:53:00.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.958+0000 7f0c54a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c50104460 0x7f0c501a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:00.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.958+0000 7f0c54a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c50109530 0x7f0c501a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:00.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.958+0000 7f0c54a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c5019a900 0x7f0c5019adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:00.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.958+0000 7f0c54a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0c50114640 con 0x7f0c50104460 2024-04-24T06:53:00.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.958+0000 7f0c54a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0c501144c0 con 0x7f0c50109530 2024-04-24T06:53:00.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.958+0000 7f0c54a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0c501147c0 con 0x7f0c5019a900 2024-04-24T06:53:00.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.958+0000 7f0c4de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c50104460 0x7f0c501a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:00.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.958+0000 7f0c4de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c50104460 0x7f0c501a0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51830/0 (socket says 172.21.15.5:51830) 2024-04-24T06:53:00.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.958+0000 7f0c4de00640 1 -- 172.21.15.5:0/4089171527 learned_addr learned my addr 172.21.15.5:0/4089171527 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:00.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.958+0000 7f0c4e800640 1 --2- 172.21.15.5:0/4089171527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c5019a900 0x7f0c5019adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:00.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.958+0000 7f0c45400640 1 --2- 172.21.15.5:0/4089171527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c50109530 0x7f0c501a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:00.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.959+0000 7f0c4de00640 1 -- 172.21.15.5:0/4089171527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c5019a900 msgr2=0x7f0c5019adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:00.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.959+0000 7f0c4de00640 1 --2- 172.21.15.5:0/4089171527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c5019a900 0x7f0c5019adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:00.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.959+0000 7f0c4de00640 1 -- 172.21.15.5:0/4089171527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c50109530 msgr2=0x7f0c501a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:00.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.959+0000 7f0c4de00640 1 --2- 172.21.15.5:0/4089171527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c50109530 0x7f0c501a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:00.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.959+0000 7f0c4de00640 1 -- 172.21.15.5:0/4089171527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c5019b420 con 0x7f0c50104460 2024-04-24T06:53:00.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.959+0000 7f0c4e800640 1 --2- 172.21.15.5:0/4089171527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c5019a900 0x7f0c5019adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:53:00.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.959+0000 7f0c4de00640 1 --2- 172.21.15.5:0/4089171527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c50104460 0x7f0c501a0750 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0c40005e30 tx=0x7f0c40003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:00.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.960+0000 7f0c47400640 1 -- 172.21.15.5:0/4089171527 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c40031650 con 0x7f0c50104460 2024-04-24T06:53:00.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.960+0000 7f0c47400640 1 -- 172.21.15.5:0/4089171527 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c400317f0 con 0x7f0c50104460 2024-04-24T06:53:00.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.960+0000 7f0c54a00640 1 -- 172.21.15.5:0/4089171527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c5019b6b0 con 0x7f0c50104460 2024-04-24T06:53:00.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.960+0000 7f0c47400640 1 -- 172.21.15.5:0/4089171527 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4002fa30 con 0x7f0c50104460 2024-04-24T06:53:00.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.960+0000 7f0c54a00640 1 -- 172.21.15.5:0/4089171527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c501a7900 con 0x7f0c50104460 2024-04-24T06:53:00.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.961+0000 7f0c54a00640 1 -- 172.21.15.5:0/4089171527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c50069300 con 0x7f0c50104460 2024-04-24T06:53:00.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.963+0000 7f0c47400640 1 -- 172.21.15.5:0/4089171527 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f0c4002fc30 con 0x7f0c50104460 2024-04-24T06:53:00.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.964+0000 7f0c47400640 1 --2- 172.21.15.5:0/4089171527 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f0c340a3d20 0x7f0c340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:00.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.965+0000 7f0c47400640 1 -- 172.21.15.5:0/4089171527 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f0c400f1ac0 con 0x7f0c50104460 2024-04-24T06:53:00.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.965+0000 7f0c45400640 1 --2- 172.21.15.5:0/4089171527 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f0c340a3d20 0x7f0c340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:00.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.966+0000 7f0c45400640 1 --2- 172.21.15.5:0/4089171527 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f0c340a3d20 0x7f0c340a61e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0c501033f0 tx=0x7f0c38005f90 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:00.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:00.969+0000 7f0c47400640 1 -- 172.21.15.5:0/4089171527 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0c400319a0 con 0x7f0c50104460 2024-04-24T06:53:01.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.221+0000 7f0c54a00640 1 -- 172.21.15.5:0/4089171527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c5019be30 con 0x7f0c50104460 2024-04-24T06:53:01.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.222+0000 7f0c47400640 1 -- 172.21.15.5:0/4089171527 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11856 (secure 0 0 0) 0x7f0c400be200 con 0x7f0c50104460 2024-04-24T06:53:01.223 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:53:01.224 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1735462750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1735462750},{"type":"v1","addr":"172.21.15.5:6837","nonce":1735462750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1499469370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1499469370},{"type":"v1","addr":"172.21.15.5:6841","nonce":1499469370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3705638847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3705638847},{"type":"v1","addr":"172.21.15.37:6835","nonce":3705638847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T06:41:34.764490+0000","modified":"2024-04-24T06:48:57.768410+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24550,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6837/1390103296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1390103296},{"type":"v1","addr":"172.21.15.37:6837","nonce":1390103296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3217754215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3217754215},{"type":"v1","addr":"172.21.15.137:6835","nonce":3217754215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6837/3044222976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3044222976},{"type":"v1","addr":"172.21.15.137:6837","nonce":3044222976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.5:6839/3173201459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3173201459},{"type":"v1","addr":"172.21.15.5:6839","nonce":3173201459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6839/1889774232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1889774232},{"type":"v1","addr":"172.21.15.137:6839","nonce":1889774232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6839/3536721753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":3536721753},{"type":"v1","addr":"172.21.15.37:6839","nonce":3536721753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24550,24569,24539]},"id":1}]} 2024-04-24T06:53:01.224 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-04-24T06:53:01.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.225+0000 7f0c54a00640 1 -- 172.21.15.5:0/4089171527 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f0c340a3d20 msgr2=0x7f0c340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:01.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.225+0000 7f0c54a00640 1 --2- 172.21.15.5:0/4089171527 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f0c340a3d20 0x7f0c340a61e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0c501033f0 tx=0x7f0c38005f90 comp rx=0 tx=0).stop 2024-04-24T06:53:01.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.225+0000 7f0c54a00640 1 -- 172.21.15.5:0/4089171527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c50104460 msgr2=0x7f0c501a0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:01.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.225+0000 7f0c54a00640 1 --2- 172.21.15.5:0/4089171527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c50104460 0x7f0c501a0750 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0c40005e30 tx=0x7f0c40003f90 comp rx=0 tx=0).stop 2024-04-24T06:53:01.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.225+0000 7f0c54a00640 1 -- 172.21.15.5:0/4089171527 shutdown_connections 2024-04-24T06:53:01.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.225+0000 7f0c54a00640 1 --2- 172.21.15.5:0/4089171527 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f0c340a3d20 0x7f0c340a61e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.225+0000 7f0c54a00640 1 --2- 172.21.15.5:0/4089171527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c5019a900 0x7f0c5019adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.225+0000 7f0c54a00640 1 --2- 172.21.15.5:0/4089171527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c50109530 0x7f0c501a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.225+0000 7f0c54a00640 1 --2- 172.21.15.5:0/4089171527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c50104460 0x7f0c501a0750 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.225+0000 7f0c54a00640 1 -- 172.21.15.5:0/4089171527 >> 172.21.15.5:0/4089171527 conn(0x7f0c500ffcc0 msgr2=0x7f0c50101fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:01.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.226+0000 7f0c54a00640 1 -- 172.21.15.5:0/4089171527 shutdown_connections 2024-04-24T06:53:01.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.226+0000 7f0c54a00640 1 -- 172.21.15.5:0/4089171527 wait complete. 2024-04-24T06:53:01.246 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T06:53:01.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.461+0000 7f1cc9200640 1 -- 172.21.15.5:0/1641746155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cc4105e10 msgr2=0x7f1cc4106270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:01.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.461+0000 7f1cc9200640 1 --2- 172.21.15.5:0/1641746155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cc4105e10 0x7f1cc4106270 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1cac009890 tx=0x7f1cac02a360 comp rx=0 tx=0).stop 2024-04-24T06:53:01.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.461+0000 7f1cc9200640 1 -- 172.21.15.5:0/1641746155 shutdown_connections 2024-04-24T06:53:01.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.461+0000 7f1cc9200640 1 --2- 172.21.15.5:0/1641746155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1cc41067b0 0x7f1cc41111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.461+0000 7f1cc9200640 1 --2- 172.21.15.5:0/1641746155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cc4105e10 0x7f1cc4106270 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.461+0000 7f1cc9200640 1 --2- 172.21.15.5:0/1641746155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cc4104530 0x7f1cc4104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.461+0000 7f1cc9200640 1 -- 172.21.15.5:0/1641746155 >> 172.21.15.5:0/1641746155 conn(0x7f1cc40ffc90 msgr2=0x7f1cc41020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:01.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.461+0000 7f1cc9200640 1 -- 172.21.15.5:0/1641746155 shutdown_connections 2024-04-24T06:53:01.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.461+0000 7f1cc9200640 1 -- 172.21.15.5:0/1641746155 wait complete. 2024-04-24T06:53:01.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.463+0000 7f1cc9200640 1 Processor -- start 2024-04-24T06:53:01.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.463+0000 7f1cc9200640 1 -- start start 2024-04-24T06:53:01.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.463+0000 7f1cc9200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cc4104530 0x7f1cc41a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:01.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.463+0000 7f1cc9200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1cc4105e10 0x7f1cc41a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:01.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.463+0000 7f1cc9200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cc41067b0 0x7f1cc419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:01.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.464+0000 7f1cc9200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc4114680 con 0x7f1cc4104530 2024-04-24T06:53:01.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.464+0000 7f1cc9200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc4114500 con 0x7f1cc41067b0 2024-04-24T06:53:01.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.464+0000 7f1cc9200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc4114800 con 0x7f1cc4105e10 2024-04-24T06:53:01.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.464+0000 7f1cc1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cc4104530 0x7f1cc41a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:01.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.464+0000 7f1cbbe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1cc4105e10 0x7f1cc41a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:01.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.464+0000 7f1cc1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cc4104530 0x7f1cc41a07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51838/0 (socket says 172.21.15.5:51838) 2024-04-24T06:53:01.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.464+0000 7f1cc1e00640 1 -- 172.21.15.5:0/3983966016 learned_addr learned my addr 172.21.15.5:0/3983966016 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:01.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.464+0000 7f1cc2800640 1 --2- 172.21.15.5:0/3983966016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cc41067b0 0x7f1cc419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:01.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.464+0000 7f1cc1e00640 1 -- 172.21.15.5:0/3983966016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1cc4105e10 msgr2=0x7f1cc41a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:01.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.464+0000 7f1cc1e00640 1 --2- 172.21.15.5:0/3983966016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1cc4105e10 0x7f1cc41a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.464+0000 7f1cc1e00640 1 -- 172.21.15.5:0/3983966016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cc41067b0 msgr2=0x7f1cc419a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:01.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.464+0000 7f1cc1e00640 1 --2- 172.21.15.5:0/3983966016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cc41067b0 0x7f1cc419a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.464+0000 7f1cc1e00640 1 -- 172.21.15.5:0/3983966016 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc419b140 con 0x7f1cc4104530 2024-04-24T06:53:01.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.465+0000 7f1cc1e00640 1 --2- 172.21.15.5:0/3983966016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cc4104530 0x7f1cc41a07c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1cb400e9c0 tx=0x7f1cb400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:01.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.465+0000 7f1cbaa00640 1 -- 172.21.15.5:0/3983966016 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb400ccc0 con 0x7f1cc4104530 2024-04-24T06:53:01.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.465+0000 7f1cbaa00640 1 -- 172.21.15.5:0/3983966016 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cb4005500 con 0x7f1cc4104530 2024-04-24T06:53:01.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.465+0000 7f1cbaa00640 1 -- 172.21.15.5:0/3983966016 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4010430 con 0x7f1cc4104530 2024-04-24T06:53:01.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.465+0000 7f1cc9200640 1 -- 172.21.15.5:0/3983966016 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cc419b430 con 0x7f1cc4104530 2024-04-24T06:53:01.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.465+0000 7f1cc9200640 1 -- 172.21.15.5:0/3983966016 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cc41a7680 con 0x7f1cc4104530 2024-04-24T06:53:01.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.468+0000 7f1ccc020740 1 Processor -- start 2024-04-24T06:53:01.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.468+0000 7f1cbaa00640 1 -- 172.21.15.5:0/3983966016 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f1cb40046c0 con 0x7f1cc4104530 2024-04-24T06:53:01.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.468+0000 7f1ccc020740 1 -- start start 2024-04-24T06:53:01.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.469+0000 7f1ccc020740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c80a94950 0x563c80a94d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:01.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.469+0000 7f1ccc020740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c80a95270 0x563c80a9f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:01.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.469+0000 7f1ccc020740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c80a9f940 0x563c80aa1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:01.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.469+0000 7f1ccc020740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563c809fa1d0 con 0x563c80a9f940 2024-04-24T06:53:01.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.469+0000 7f1ccc020740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563c809a0450 con 0x563c80a95270 2024-04-24T06:53:01.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.469+0000 7f1ccc020740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563c809f6190 con 0x563c80a94950 2024-04-24T06:53:01.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.469+0000 7f1cbbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c80a9f940 0x563c80aa1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:01.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.469+0000 7f1cc2800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c80a94950 0x563c80a94d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:01.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.469+0000 7f1cc1e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c80a95270 0x563c80a9f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:01.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.469+0000 7f1cc2800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c80a94950 0x563c80a94d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:58760/0 (socket says 172.21.15.5:58760) 2024-04-24T06:53:01.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.469+0000 7f1cbbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c80a9f940 0x563c80aa1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51846/0 (socket says 172.21.15.5:51846) 2024-04-24T06:53:01.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.469+0000 7f1cc2800640 1 -- 172.21.15.5:0/4091782257 learned_addr learned my addr 172.21.15.5:0/4091782257 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:01.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.470+0000 7f1cc2800640 1 -- 172.21.15.5:0/4091782257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c80a95270 msgr2=0x563c80a9f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:01.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.470+0000 7f1cc2800640 1 --2- 172.21.15.5:0/4091782257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c80a95270 0x563c80a9f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.470+0000 7f1cc2800640 1 -- 172.21.15.5:0/4091782257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c80a9f940 msgr2=0x563c80aa1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:01.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.470+0000 7f1cc2800640 1 --2- 172.21.15.5:0/4091782257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c80a9f940 0x563c80aa1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.470+0000 7f1cc2800640 1 -- 172.21.15.5:0/4091782257 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c80a5d310 con 0x563c80a94950 2024-04-24T06:53:01.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.470+0000 7f1cc2800640 1 --2- 172.21.15.5:0/4091782257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c80a94950 0x563c80a94d30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009fd0 tx=0x7f1cbc0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21ba6693a25fc708 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:01.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.470+0000 7f1cbaa00640 1 --2- 172.21.15.5:0/3983966016 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1ca40a3da0 0x7f1ca40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:01.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.470+0000 7f1cb3400640 1 -- 172.21.15.5:0/4091782257 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc004d20 con 0x563c80a94950 2024-04-24T06:53:01.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.470+0000 7f1cbaa00640 1 -- 172.21.15.5:0/3983966016 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f1cb40d0a80 con 0x7f1cc4104530 2024-04-24T06:53:01.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.470+0000 7f1cc1e00640 1 --2- 172.21.15.5:0/3983966016 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1ca40a3da0 0x7f1ca40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:01.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.471+0000 7f1cb3400640 1 -- 172.21.15.5:0/4091782257 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cbc004050 con 0x563c80a94950 2024-04-24T06:53:01.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.471+0000 7f1cb3400640 1 -- 172.21.15.5:0/4091782257 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc005220 con 0x563c80a94950 2024-04-24T06:53:01.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.471+0000 7f1ccc020740 1 -- 172.21.15.5:0/4091782257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c80a94950 msgr2=0x563c80a94d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:01.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.471+0000 7f1ccc020740 1 --2- 172.21.15.5:0/4091782257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c80a94950 0x563c80a94d30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009fd0 tx=0x7f1cbc0118d0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.471+0000 7f1cc1e00640 1 --2- 172.21.15.5:0/3983966016 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1ca40a3da0 0x7f1ca40a6260 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x563c80a95f10 tx=0x7f1cb40d4570 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:01.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.471+0000 7f1ccc020740 1 -- 172.21.15.5:0/4091782257 shutdown_connections 2024-04-24T06:53:01.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.471+0000 7f1ccc020740 1 --2- 172.21.15.5:0/4091782257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c80a9f940 0x563c80aa1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.471+0000 7f1ccc020740 1 --2- 172.21.15.5:0/4091782257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c80a95270 0x563c80a9f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.471+0000 7f1ccc020740 1 --2- 172.21.15.5:0/4091782257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c80a94950 0x563c80a94d30 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.471+0000 7f1ccc020740 1 -- 172.21.15.5:0/4091782257 >> 172.21.15.5:0/4091782257 conn(0x563c80a90c70 msgr2=0x563c80a91050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:01.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.472+0000 7f1ccc020740 1 -- 172.21.15.5:0/4091782257 shutdown_connections 2024-04-24T06:53:01.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.472+0000 7f1ccc020740 1 -- 172.21.15.5:0/4091782257 wait complete. 2024-04-24T06:53:01.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.472+0000 7f1ccc020740 1 -- start start 2024-04-24T06:53:01.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.472+0000 7f1ccc020740 1 Processor -- start 2024-04-24T06:53:01.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.473+0000 7f1ccc020740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c80a94950 0x563c80a9f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:01.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.473+0000 7f1ccc020740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c80a95270 0x563c80a9c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:01.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.473+0000 7f1ccc020740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c80a9f940 0x563c80a9cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:01.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.473+0000 7f1ccc020740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563c80aa27d0 con 0x563c80a9f940 2024-04-24T06:53:01.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.473+0000 7f1ccc020740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563c809b3810 con 0x563c80a95270 2024-04-24T06:53:01.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.473+0000 7f1ccc020740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563c80aa2950 con 0x563c80a94950 2024-04-24T06:53:01.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.473+0000 7f1ccc020740 10 client.0 _ll_register_callbacks cb 0x563c80981140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:53:01.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.473+0000 7f1cc2800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c80a9f940 0x563c80a9cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:01.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.473+0000 7f1cbbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c80a95270 0x563c80a9c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:01.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.473+0000 7f1cbbe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c80a95270 0x563c80a9c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:36094/0 (socket says 172.21.15.5:36094) 2024-04-24T06:53:01.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.473+0000 7f1cbbe00640 1 -- 172.21.15.5:0/2975530343 learned_addr learned my addr 172.21.15.5:0/2975530343 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:01.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.473+0000 7f1cc2800640 1 --2- 172.21.15.5:0/2975530343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c80a94950 0x563c80a9f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:01.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.473+0000 7f1cbbe00640 1 -- 172.21.15.5:0/2975530343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c80a94950 msgr2=0x563c80a9f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:01.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.473+0000 7f1cbbe00640 1 --2- 172.21.15.5:0/2975530343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c80a94950 0x563c80a9f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.473+0000 7f1cbbe00640 1 -- 172.21.15.5:0/2975530343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c80a9f940 msgr2=0x563c80a9cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:01.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.473+0000 7f1cbbe00640 1 --2- 172.21.15.5:0/2975530343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c80a9f940 0x563c80a9cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.473+0000 7f1cbbe00640 1 -- 172.21.15.5:0/2975530343 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c809c43f0 con 0x563c80a95270 2024-04-24T06:53:01.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.473+0000 7f1cc2800640 1 --2- 172.21.15.5:0/2975530343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c80a9f940 0x563c80a9cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:53:01.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.474+0000 7f1cbbe00640 1 --2- 172.21.15.5:0/2975530343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c80a95270 0x563c80a9c680 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1cac000c00 tx=0x7f1cac030430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2b0616308d527cf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:01.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.474+0000 7f1cabe00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:53:01.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.474+0000 7f1cabe00640 1 -- 172.21.15.5:0/2975530343 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac041070 con 0x563c80a95270 2024-04-24T06:53:01.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.474+0000 7f1cabe00640 1 -- 172.21.15.5:0/2975530343 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f1cac030580 con 0x563c80a95270 2024-04-24T06:53:01.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.474+0000 7f1ccc020740 1 -- 172.21.15.5:0/2975530343 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cbc004050 con 0x563c80a95270 2024-04-24T06:53:01.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.474+0000 7f1cabe00640 1 -- 172.21.15.5:0/2975530343 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1cac04d050 con 0x563c80a95270 2024-04-24T06:53:01.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.474+0000 7f1ccc020740 10 client.27155 fetch_fsmap learned FSMap version 27 2024-04-24T06:53:01.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.474+0000 7f1ccc020740 1 -- 172.21.15.5:0/2975530343 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563c8070c3f0 con 0x563c80a95270 2024-04-24T06:53:01.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.475+0000 7f1cabe00640 1 -- 172.21.15.5:0/2975530343 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cbc004050 con 0x563c80a95270 2024-04-24T06:53:01.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.475+0000 7f1cabe00640 1 -- 172.21.15.5:0/2975530343 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f1cac03d450 con 0x563c80a95270 2024-04-24T06:53:01.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.475+0000 7f1ccc020740 10 client.27155 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:53:01.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.475+0000 7f1ccc020740 10 client.27155 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-24T06:53:01.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.475+0000 7f1ccc020740 20 client.27155 populate_metadata read hostname 'smithi005' 2024-04-24T06:53:01.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.475+0000 7f1ccc020740 1 --2- 172.21.15.5:0/2975530343 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x563c80aafab0 0x563c80ab1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:01.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.475+0000 7f1ccc020740 4 client.27155 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:53:01.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.475+0000 7f1ccc020740 1 -- 172.21.15.5:0/2975530343 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c80a07bf0 con 0x563c80aafab0 2024-04-24T06:53:01.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.476+0000 7f1cc2800640 1 --2- 172.21.15.5:0/2975530343 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x563c80aafab0 0x563c80ab1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:01.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.476+0000 7f1cc2800640 1 --2- 172.21.15.5:0/2975530343 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x563c80aafab0 0x563c80ab1f10 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1d82dd727f288b1 server_cookie=6d1837506d4fe6eb in_seq=0 out_seq=0 2024-04-24T06:53:01.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.476+0000 7f1cabe00640 10 client.27155 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:53:01.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.477+0000 7f1cabe00640 1 -- 172.21.15.5:0/2975530343 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563c80a07bf0 con 0x563c80aafab0 2024-04-24T06:53:01.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.477+0000 7f1cabe00640 10 client.27155 handle_command_reply: tid=0 2024-04-24T06:53:01.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.477+0000 7f1ccc020740 1 client.27155 shutdown 2024-04-24T06:53:01.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.477+0000 7f1ccc020740 1 -- 172.21.15.5:0/2975530343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c80a95270 msgr2=0x563c80a9c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:01.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.477+0000 7f1ccc020740 1 --2- 172.21.15.5:0/2975530343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c80a95270 0x563c80a9c680 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1cac000c00 tx=0x7f1cac030430 comp rx=0 tx=0).stop 2024-04-24T06:53:01.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.477+0000 7f1ccc020740 1 -- 172.21.15.5:0/2975530343 shutdown_connections 2024-04-24T06:53:01.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.477+0000 7f1ccc020740 1 --2- 172.21.15.5:0/2975530343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c80a9f940 0x563c80a9cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.477+0000 7f1ccc020740 1 --2- 172.21.15.5:0/2975530343 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x563c80aafab0 0x563c80ab1f10 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.477+0000 7f1ccc020740 1 --2- 172.21.15.5:0/2975530343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c80a95270 0x563c80a9c680 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.477+0000 7f1ccc020740 1 --2- 172.21.15.5:0/2975530343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c80a94950 0x563c80a9f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.477+0000 7f1cabe00640 0 client.27155 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:53:01.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.477+0000 7f1ccc020740 1 -- 172.21.15.5:0/2975530343 >> 172.21.15.5:0/2975530343 conn(0x563c80a90c70 msgr2=0x563c80a96a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:01.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.477+0000 7f1ccc020740 1 -- 172.21.15.5:0/2975530343 shutdown_connections 2024-04-24T06:53:01.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.477+0000 7f1ccc020740 1 -- 172.21.15.5:0/2975530343 wait complete. 2024-04-24T06:53:01.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.478+0000 7f1ccc020740 20 client.27155 trim_cache size 0 max 16384 2024-04-24T06:53:01.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.492+0000 7f1ccc020740 1 Processor -- start 2024-04-24T06:53:01.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.493+0000 7f1ccc020740 1 -- start start 2024-04-24T06:53:01.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.493+0000 7f1ccc020740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c80a94950 0x563c80aab690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:01.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.493+0000 7f1ccc020740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c80a95270 0x563c80a9eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:01.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.493+0000 7f1ccc020740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c80a9f940 0x563c80a9f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:01.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.493+0000 7f1ccc020740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41b2860 con 0x563c80a95270 2024-04-24T06:53:01.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.493+0000 7f1ccc020740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41b0130 con 0x563c80a94950 2024-04-24T06:53:01.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.493+0000 7f1ccc020740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41adfa0 con 0x563c80a9f940 2024-04-24T06:53:01.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.493+0000 7f1cbbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c80a95270 0x563c80a9eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:01.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.493+0000 7f1cbbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c80a95270 0x563c80a9eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51868/0 (socket says 172.21.15.5:51868) 2024-04-24T06:53:01.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.493+0000 7f1cbbe00640 1 -- 172.21.15.5:0/2077818227 learned_addr learned my addr 172.21.15.5:0/2077818227 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:01.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.493+0000 7f1cc2800640 1 --2- 172.21.15.5:0/2077818227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c80a94950 0x563c80aab690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:01.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.493+0000 7f1cc2800640 1 --2- 172.21.15.5:0/2077818227 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c80a9f940 0x563c80a9f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:01.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.494+0000 7f1cbbe00640 1 -- 172.21.15.5:0/2077818227 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c80a9f940 msgr2=0x563c80a9f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:01.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.494+0000 7f1cbbe00640 1 --2- 172.21.15.5:0/2077818227 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c80a9f940 0x563c80a9f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.494+0000 7f1cbbe00640 1 -- 172.21.15.5:0/2077818227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c80a94950 msgr2=0x563c80aab690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:01.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.494+0000 7f1cbbe00640 1 --2- 172.21.15.5:0/2077818227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c80a94950 0x563c80aab690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.494+0000 7f1cbbe00640 1 -- 172.21.15.5:0/2077818227 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c807d67d0 con 0x563c80a95270 2024-04-24T06:53:01.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.494+0000 7f1cc2800640 1 --2- 172.21.15.5:0/2077818227 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c80a9f940 0x563c80a9f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:53:01.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.494+0000 7f1cbbe00640 1 --2- 172.21.15.5:0/2077818227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c80a95270 0x563c80a9eec0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1cac02a880 tx=0x7f1cac005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fcfab9376d6829b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:01.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.494+0000 7f1cb2a00640 1 -- 172.21.15.5:0/2077818227 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac04a070 con 0x563c80a95270 2024-04-24T06:53:01.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.494+0000 7f1cb2a00640 1 -- 172.21.15.5:0/2077818227 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cac03d450 con 0x563c80a95270 2024-04-24T06:53:01.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.494+0000 7f1cb2a00640 1 -- 172.21.15.5:0/2077818227 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac041050 con 0x563c80a95270 2024-04-24T06:53:01.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.495+0000 7f1ccc020740 1 -- 172.21.15.5:0/2077818227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c80a95270 msgr2=0x563c80a9eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:01.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.495+0000 7f1ccc020740 1 --2- 172.21.15.5:0/2077818227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c80a95270 0x563c80a9eec0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1cac02a880 tx=0x7f1cac005d80 comp rx=0 tx=0).stop 2024-04-24T06:53:01.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.495+0000 7f1ccc020740 1 -- 172.21.15.5:0/2077818227 shutdown_connections 2024-04-24T06:53:01.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.495+0000 7f1ccc020740 1 --2- 172.21.15.5:0/2077818227 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c80a9f940 0x563c80a9f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.495+0000 7f1ccc020740 1 --2- 172.21.15.5:0/2077818227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c80a95270 0x563c80a9eec0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.495+0000 7f1ccc020740 1 --2- 172.21.15.5:0/2077818227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c80a94950 0x563c80aab690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.495+0000 7f1ccc020740 1 -- 172.21.15.5:0/2077818227 >> 172.21.15.5:0/2077818227 conn(0x563c80a90c70 msgr2=0x563c80a91050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:01.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.495+0000 7f1ccc020740 1 -- 172.21.15.5:0/2077818227 shutdown_connections 2024-04-24T06:53:01.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.495+0000 7f1ccc020740 1 -- 172.21.15.5:0/2077818227 wait complete. 2024-04-24T06:53:01.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.495+0000 7f1ccc020740 1 -- start start 2024-04-24T06:53:01.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.496+0000 7f1ccc020740 1 Processor -- start 2024-04-24T06:53:01.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.496+0000 7f1ccc020740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c80a94950 0x563c80aa09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:01.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.496+0000 7f1ccc020740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c80a95270 0x563c80aa0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:01.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.496+0000 7f1ccc020740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c80a9f940 0x563c80aa60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:01.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.496+0000 7f1ccc020740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41ae5f0 con 0x563c80a95270 2024-04-24T06:53:01.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.496+0000 7f1ccc020740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41afa60 con 0x563c80a94950 2024-04-24T06:53:01.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.496+0000 7f1ccc020740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41aecc0 con 0x563c80a9f940 2024-04-24T06:53:01.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.496+0000 7f1ccc020740 10 client.0 _ll_register_callbacks cb 0x7f1cbc0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:53:01.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.496+0000 7f1cc2800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c80a94950 0x563c80aa09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:01.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.496+0000 7f1cbbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c80a95270 0x563c80aa0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:01.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.496+0000 7f1cc2800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c80a94950 0x563c80aa09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:36118/0 (socket says 172.21.15.5:36118) 2024-04-24T06:53:01.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.496+0000 7f1cc2800640 1 -- 172.21.15.5:0/1306167690 learned_addr learned my addr 172.21.15.5:0/1306167690 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:01.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.496+0000 7f1cc2800640 1 --2- 172.21.15.5:0/1306167690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c80a9f940 0x563c80aa60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:01.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.497+0000 7f1cbbe00640 1 -- 172.21.15.5:0/1306167690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c80a9f940 msgr2=0x563c80aa60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:01.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.497+0000 7f1cbbe00640 1 --2- 172.21.15.5:0/1306167690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c80a9f940 0x563c80aa60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.497+0000 7f1cbbe00640 1 -- 172.21.15.5:0/1306167690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c80a94950 msgr2=0x563c80aa09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:01.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.497+0000 7f1cbbe00640 1 --2- 172.21.15.5:0/1306167690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c80a94950 0x563c80aa09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.497+0000 7f1cbbe00640 1 -- 172.21.15.5:0/1306167690 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c80aa1800 con 0x563c80a95270 2024-04-24T06:53:01.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.497+0000 7f1cbbe00640 1 --2- 172.21.15.5:0/1306167690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c80a95270 0x563c80aa0f00 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x563c80a9d3c0 tx=0x7f1cac0058c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a518e1c574d0cfb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:01.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.497+0000 7f1cabe00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:53:01.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.497+0000 7f1cabe00640 1 -- 172.21.15.5:0/1306167690 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac04a070 con 0x563c80a95270 2024-04-24T06:53:01.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.497+0000 7f1cabe00640 1 -- 172.21.15.5:0/1306167690 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f1cac030780 con 0x563c80a95270 2024-04-24T06:53:01.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.497+0000 7f1ccc020740 1 -- 172.21.15.5:0/1306167690 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cac03d450 con 0x563c80a95270 2024-04-24T06:53:01.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.498+0000 7f1cabe00640 1 -- 172.21.15.5:0/1306167690 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1cac061050 con 0x563c80a95270 2024-04-24T06:53:01.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.498+0000 7f1ccc020740 10 client.17010 fetch_fsmap learned FSMap version 27 2024-04-24T06:53:01.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.498+0000 7f1ccc020740 1 -- 172.21.15.5:0/1306167690 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563c80b16680 con 0x563c80a95270 2024-04-24T06:53:01.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.498+0000 7f1cabe00640 1 -- 172.21.15.5:0/1306167690 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cac03d450 con 0x563c80a95270 2024-04-24T06:53:01.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.498+0000 7f1cabe00640 1 -- 172.21.15.5:0/1306167690 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f1cac0047c0 con 0x563c80a95270 2024-04-24T06:53:01.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.498+0000 7f1ccc020740 10 client.17010 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:53:01.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.498+0000 7f1ccc020740 10 client.17010 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-24T06:53:01.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.498+0000 7f1ccc020740 20 client.17010 populate_metadata read hostname 'smithi005' 2024-04-24T06:53:01.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.498+0000 7f1ccc020740 1 --2- 172.21.15.5:0/1306167690 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x563c80b16d30 0x563c80b19150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:01.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.498+0000 7f1ccc020740 4 client.17010 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:53:01.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.498+0000 7f1ccc020740 1 -- 172.21.15.5:0/1306167690 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563c80a01a60 con 0x563c80b16d30 2024-04-24T06:53:01.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.499+0000 7f1cc2800640 1 --2- 172.21.15.5:0/1306167690 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x563c80b16d30 0x563c80b19150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:01.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.499+0000 7f1cc2800640 1 --2- 172.21.15.5:0/1306167690 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x563c80b16d30 0x563c80b19150 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=91d9a04405ed80ce server_cookie=fd765ec2d6b9037e in_seq=0 out_seq=0 2024-04-24T06:53:01.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.499+0000 7f1cabe00640 10 client.17010 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:53:01.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.499+0000 7f1cabe00640 1 -- 172.21.15.5:0/1306167690 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563c80a01a60 con 0x563c80b16d30 2024-04-24T06:53:01.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.499+0000 7f1cabe00640 10 client.17010 handle_command_reply: tid=0 2024-04-24T06:53:01.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.499+0000 7f1ccc020740 1 client.17010 shutdown 2024-04-24T06:53:01.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.500+0000 7f1ccc020740 1 -- 172.21.15.5:0/1306167690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c80a95270 msgr2=0x563c80aa0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:01.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.500+0000 7f1ccc020740 1 --2- 172.21.15.5:0/1306167690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c80a95270 0x563c80aa0f00 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x563c80a9d3c0 tx=0x7f1cac0058c0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.500+0000 7f1ccc020740 1 -- 172.21.15.5:0/1306167690 shutdown_connections 2024-04-24T06:53:01.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.500+0000 7f1ccc020740 1 --2- 172.21.15.5:0/1306167690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c80a9f940 0x563c80aa60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.500+0000 7f1ccc020740 1 --2- 172.21.15.5:0/1306167690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c80a95270 0x563c80aa0f00 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.500+0000 7f1ccc020740 1 --2- 172.21.15.5:0/1306167690 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x563c80b16d30 0x563c80b19150 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.500+0000 7f1ccc020740 1 --2- 172.21.15.5:0/1306167690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c80a94950 0x563c80aa09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.500+0000 7f1ccc020740 1 -- 172.21.15.5:0/1306167690 >> 172.21.15.5:0/1306167690 conn(0x563c80a90c70 msgr2=0x563c80a8f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:01.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.500+0000 7f1cabe00640 0 client.17010 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:53:01.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.500+0000 7f1ccc020740 1 -- 172.21.15.5:0/1306167690 shutdown_connections 2024-04-24T06:53:01.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.500+0000 7f1ccc020740 1 -- 172.21.15.5:0/1306167690 wait complete. 2024-04-24T06:53:01.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.500+0000 7f1ccc020740 20 client.17010 trim_cache size 0 max 16384 2024-04-24T06:53:01.511 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:53:01.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.501+0000 7f1cc9200640 1 -- 172.21.15.5:0/3983966016 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1ca40a3da0 msgr2=0x7f1ca40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:01.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.501+0000 7f1cc9200640 1 --2- 172.21.15.5:0/3983966016 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1ca40a3da0 0x7f1ca40a6260 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x563c80a95f10 tx=0x7f1cb40d4570 comp rx=0 tx=0).stop 2024-04-24T06:53:01.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.501+0000 7f1cc9200640 1 -- 172.21.15.5:0/3983966016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cc4104530 msgr2=0x7f1cc41a07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:01.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.501+0000 7f1cc9200640 1 --2- 172.21.15.5:0/3983966016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cc4104530 0x7f1cc41a07c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1cb400e9c0 tx=0x7f1cb400ee80 comp rx=0 tx=0).stop 2024-04-24T06:53:01.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.501+0000 7f1cc9200640 1 -- 172.21.15.5:0/3983966016 shutdown_connections 2024-04-24T06:53:01.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.501+0000 7f1cc9200640 1 --2- 172.21.15.5:0/3983966016 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1ca40a3da0 0x7f1ca40a6260 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.501+0000 7f1cc9200640 1 --2- 172.21.15.5:0/3983966016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cc41067b0 0x7f1cc419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.501+0000 7f1cc9200640 1 --2- 172.21.15.5:0/3983966016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1cc4105e10 0x7f1cc41a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.501+0000 7f1cc9200640 1 --2- 172.21.15.5:0/3983966016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cc4104530 0x7f1cc41a07c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.501+0000 7f1cc9200640 1 -- 172.21.15.5:0/3983966016 >> 172.21.15.5:0/3983966016 conn(0x7f1cc40ffc90 msgr2=0x7f1cc4100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:01.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.501+0000 7f1cc9200640 1 -- 172.21.15.5:0/3983966016 shutdown_connections 2024-04-24T06:53:01.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.501+0000 7f1cc9200640 1 -- 172.21.15.5:0/3983966016 wait complete. 2024-04-24T06:53:01.515 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:01 smithi005 ceph-mon[27798]: pgmap v276: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:53:01.515 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:01 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/4089171527' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:53:01.522 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T06:53:01.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:01 smithi137 ceph-mon[33017]: pgmap v276: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:53:01.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:01 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/4089171527' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:53:01.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.738+0000 7f04ce800640 1 -- 172.21.15.5:0/2344382810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04c8104530 msgr2=0x7f04c8104910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:01.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.738+0000 7f04ce800640 1 --2- 172.21.15.5:0/2344382810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04c8104530 0x7f04c8104910 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f04b8009930 tx=0x7f04b802a420 comp rx=0 tx=0).stop 2024-04-24T06:53:01.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.738+0000 7f04ce800640 1 -- 172.21.15.5:0/2344382810 shutdown_connections 2024-04-24T06:53:01.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.738+0000 7f04ce800640 1 --2- 172.21.15.5:0/2344382810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04c81067b0 0x7f04c81111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.738+0000 7f04ce800640 1 --2- 172.21.15.5:0/2344382810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04c8105e10 0x7f04c8106270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.738+0000 7f04ce800640 1 --2- 172.21.15.5:0/2344382810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04c8104530 0x7f04c8104910 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.738+0000 7f04ce800640 1 -- 172.21.15.5:0/2344382810 >> 172.21.15.5:0/2344382810 conn(0x7f04c80ffc90 msgr2=0x7f04c81020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:01.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.738+0000 7f04ce800640 1 -- 172.21.15.5:0/2344382810 shutdown_connections 2024-04-24T06:53:01.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.738+0000 7f04ce800640 1 -- 172.21.15.5:0/2344382810 wait complete. 2024-04-24T06:53:01.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.740+0000 7f04ce800640 1 Processor -- start 2024-04-24T06:53:01.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.740+0000 7f04ce800640 1 -- start start 2024-04-24T06:53:01.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.740+0000 7f04ce800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04c8104530 0x7f04c81a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:01.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.740+0000 7f04ce800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04c8105e10 0x7f04c81a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:01.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.741+0000 7f04ce800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04c81067b0 0x7f04c819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:01.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.741+0000 7f04ce800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f04c81146d0 con 0x7f04c81067b0 2024-04-24T06:53:01.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.741+0000 7f04ce800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f04c8114550 con 0x7f04c8104530 2024-04-24T06:53:01.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.741+0000 7f04ce800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f04c8114850 con 0x7f04c8105e10 2024-04-24T06:53:01.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.741+0000 7f04c7400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04c8104530 0x7f04c81a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:01.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.741+0000 7f04bfe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04c8105e10 0x7f04c81a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:01.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.741+0000 7f04c7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04c81067b0 0x7f04c819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:01.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.741+0000 7f04c7400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04c8104530 0x7f04c81a07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:36134/0 (socket says 172.21.15.5:36134) 2024-04-24T06:53:01.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.741+0000 7f04c7400640 1 -- 172.21.15.5:0/557999570 learned_addr learned my addr 172.21.15.5:0/557999570 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:01.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.741+0000 7f04bfe00640 1 -- 172.21.15.5:0/557999570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04c8104530 msgr2=0x7f04c81a07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:01.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.741+0000 7f04bfe00640 1 --2- 172.21.15.5:0/557999570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04c8104530 0x7f04c81a07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.741+0000 7f04bfe00640 1 -- 172.21.15.5:0/557999570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04c81067b0 msgr2=0x7f04c819a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:01.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.741+0000 7f04bfe00640 1 --2- 172.21.15.5:0/557999570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04c81067b0 0x7f04c819a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.741+0000 7f04bfe00640 1 -- 172.21.15.5:0/557999570 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c819b150 con 0x7f04c8105e10 2024-04-24T06:53:01.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.741+0000 7f04c7e00640 1 --2- 172.21.15.5:0/557999570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04c81067b0 0x7f04c819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:53:01.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.742+0000 7f04bfe00640 1 --2- 172.21.15.5:0/557999570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04c8105e10 0x7f04c81a0d10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f04b000e9f0 tx=0x7f04b000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:01.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.742+0000 7f04c4c00640 1 -- 172.21.15.5:0/557999570 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b000ccc0 con 0x7f04c8105e10 2024-04-24T06:53:01.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.742+0000 7f04c4c00640 1 -- 172.21.15.5:0/557999570 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04b0010910 con 0x7f04c8105e10 2024-04-24T06:53:01.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.742+0000 7f04c4c00640 1 -- 172.21.15.5:0/557999570 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b0018960 con 0x7f04c8105e10 2024-04-24T06:53:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.742+0000 7f04ce800640 1 -- 172.21.15.5:0/557999570 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04c819b440 con 0x7f04c8105e10 2024-04-24T06:53:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.742+0000 7f04ce800640 1 -- 172.21.15.5:0/557999570 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04c81a7690 con 0x7f04c8105e10 2024-04-24T06:53:01.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.745+0000 7f04d1643740 1 Processor -- start 2024-04-24T06:53:01.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.746+0000 7f04d1643740 1 -- start start 2024-04-24T06:53:01.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.746+0000 7f04c4c00640 1 -- 172.21.15.5:0/557999570 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f04b0010ac0 con 0x7f04c8105e10 2024-04-24T06:53:01.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.746+0000 7f04d1643740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e5e55f950 0x558e5e55fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:01.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.746+0000 7f04d1643740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e5e560270 0x558e5e56a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:01.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.746+0000 7f04d1643740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e5e56a940 0x558e5e56cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:01.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.746+0000 7f04d1643740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558e5e4c51d0 con 0x558e5e560270 2024-04-24T06:53:01.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.746+0000 7f04d1643740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558e5e46b450 con 0x558e5e56a940 2024-04-24T06:53:01.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.746+0000 7f04d1643740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558e5e4c1190 con 0x558e5e55f950 2024-04-24T06:53:01.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.746+0000 7f04c7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e5e560270 0x558e5e56a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:01.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.746+0000 7f04c7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e5e55f950 0x558e5e55fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:01.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.746+0000 7f04c7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e5e55f950 0x558e5e55fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:58832/0 (socket says 172.21.15.5:58832) 2024-04-24T06:53:01.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.746+0000 7f04c7e00640 1 -- 172.21.15.5:0/1756805745 learned_addr learned my addr 172.21.15.5:0/1756805745 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:01.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.746+0000 7f04c7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e5e560270 0x558e5e56a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51908/0 (socket says 172.21.15.5:51908) 2024-04-24T06:53:01.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.746+0000 7f04bfe00640 1 --2- 172.21.15.5:0/1756805745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e5e56a940 0x558e5e56cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:01.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.747+0000 7f04c7e00640 1 -- 172.21.15.5:0/1756805745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e5e56a940 msgr2=0x558e5e56cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:01.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.747+0000 7f04c7e00640 1 --2- 172.21.15.5:0/1756805745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e5e56a940 0x558e5e56cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.747+0000 7f04c7e00640 1 -- 172.21.15.5:0/1756805745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e5e560270 msgr2=0x558e5e56a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:01.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.747+0000 7f04c7e00640 1 --2- 172.21.15.5:0/1756805745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e5e560270 0x558e5e56a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.747+0000 7f04c7e00640 1 -- 172.21.15.5:0/1756805745 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e5e528310 con 0x558e5e55f950 2024-04-24T06:53:01.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.747+0000 7f04c7400640 1 --2- 172.21.15.5:0/1756805745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e5e560270 0x558e5e56a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:53:01.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.747+0000 7f04c7e00640 1 --2- 172.21.15.5:0/1756805745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e5e55f950 0x558e5e55fd30 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f04c00107e0 tx=0x7f04c0010ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5457e8c3dd86f1c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:01.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.747+0000 7f04bcc00640 1 -- 172.21.15.5:0/1756805745 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c0003b70 con 0x558e5e55f950 2024-04-24T06:53:01.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.747+0000 7f04bcc00640 1 -- 172.21.15.5:0/1756805745 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04c000ee10 con 0x558e5e55f950 2024-04-24T06:53:01.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.747+0000 7f04bcc00640 1 -- 172.21.15.5:0/1756805745 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c00047f0 con 0x558e5e55f950 2024-04-24T06:53:01.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.748+0000 7f04c4c00640 1 --2- 172.21.15.5:0/557999570 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f04ac0a3da0 0x7f04ac0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:01.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.748+0000 7f04c4c00640 1 -- 172.21.15.5:0/557999570 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f04b00d0af0 con 0x7f04c8105e10 2024-04-24T06:53:01.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.748+0000 7f04c7400640 1 --2- 172.21.15.5:0/557999570 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f04ac0a3da0 0x7f04ac0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:01.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.748+0000 7f04d1643740 1 -- 172.21.15.5:0/1756805745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e5e55f950 msgr2=0x558e5e55fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:01.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.748+0000 7f04d1643740 1 --2- 172.21.15.5:0/1756805745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e5e55f950 0x558e5e55fd30 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f04c00107e0 tx=0x7f04c0010ca0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.748+0000 7f04d1643740 1 -- 172.21.15.5:0/1756805745 shutdown_connections 2024-04-24T06:53:01.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.748+0000 7f04d1643740 1 --2- 172.21.15.5:0/1756805745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e5e56a940 0x558e5e56cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.748+0000 7f04d1643740 1 --2- 172.21.15.5:0/1756805745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e5e560270 0x558e5e56a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.748+0000 7f04d1643740 1 --2- 172.21.15.5:0/1756805745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e5e55f950 0x558e5e55fd30 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.748+0000 7f04d1643740 1 -- 172.21.15.5:0/1756805745 >> 172.21.15.5:0/1756805745 conn(0x558e5e55bc70 msgr2=0x558e5e55c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:01.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.748+0000 7f04d1643740 1 -- 172.21.15.5:0/1756805745 shutdown_connections 2024-04-24T06:53:01.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.749+0000 7f04d1643740 1 -- 172.21.15.5:0/1756805745 wait complete. 2024-04-24T06:53:01.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.749+0000 7f04c7400640 1 --2- 172.21.15.5:0/557999570 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f04ac0a3da0 0x7f04ac0a6260 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f04b8031550 tx=0x7f04b8005b50 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:01.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.749+0000 7f04d1643740 1 -- start start 2024-04-24T06:53:01.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.749+0000 7f04d1643740 1 Processor -- start 2024-04-24T06:53:01.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.750+0000 7f04d1643740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e5e55f950 0x558e5e56a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:01.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.750+0000 7f04d1643740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e5e560270 0x558e5e567680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:01.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.750+0000 7f04d1643740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e5e56a940 0x558e5e567dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:01.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.750+0000 7f04d1643740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558e5e56d7d0 con 0x558e5e560270 2024-04-24T06:53:01.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.750+0000 7f04d1643740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558e5e47e810 con 0x558e5e55f950 2024-04-24T06:53:01.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.750+0000 7f04d1643740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558e5e56d950 con 0x558e5e56a940 2024-04-24T06:53:01.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.750+0000 7f04d1643740 10 client.0 _ll_register_callbacks cb 0x558e5e44c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:53:01.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.750+0000 7f04bfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e5e560270 0x558e5e567680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:01.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.750+0000 7f04c7400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e5e55f950 0x558e5e56a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:01.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.750+0000 7f04bfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e5e560270 0x558e5e567680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51920/0 (socket says 172.21.15.5:51920) 2024-04-24T06:53:01.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.750+0000 7f04c7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e5e56a940 0x558e5e567dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:01.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.750+0000 7f04bfe00640 1 -- 172.21.15.5:0/1493731933 learned_addr learned my addr 172.21.15.5:0/1493731933 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:01.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.750+0000 7f04bfe00640 1 -- 172.21.15.5:0/1493731933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e5e56a940 msgr2=0x558e5e567dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:01.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.750+0000 7f04bfe00640 1 --2- 172.21.15.5:0/1493731933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e5e56a940 0x558e5e567dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.750+0000 7f04bfe00640 1 -- 172.21.15.5:0/1493731933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e5e55f950 msgr2=0x558e5e56a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:01.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.750+0000 7f04bfe00640 1 --2- 172.21.15.5:0/1493731933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e5e55f950 0x558e5e56a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.751+0000 7f04bfe00640 1 -- 172.21.15.5:0/1493731933 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e5e48f3f0 con 0x558e5e560270 2024-04-24T06:53:01.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.751+0000 7f04c7e00640 1 --2- 172.21.15.5:0/1493731933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e5e56a940 0x558e5e567dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:53:01.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.751+0000 7f04bfe00640 1 --2- 172.21.15.5:0/1493731933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e5e560270 0x558e5e567680 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f04b000c6a0 tx=0x7f04b00d4230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dcc4886462b419a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:01.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.751+0000 7f04b5600640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:53:01.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.751+0000 7f04b5600640 1 -- 172.21.15.5:0/1493731933 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b009c900 con 0x558e5e560270 2024-04-24T06:53:01.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.751+0000 7f04b5600640 1 -- 172.21.15.5:0/1493731933 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f04b00991a0 con 0x558e5e560270 2024-04-24T06:53:01.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.751+0000 7f04d1643740 1 -- 172.21.15.5:0/1493731933 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04c000ee10 con 0x558e5e560270 2024-04-24T06:53:01.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.751+0000 7f04b5600640 1 -- 172.21.15.5:0/1493731933 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f04b009c490 con 0x558e5e560270 2024-04-24T06:53:01.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.751+0000 7f04d1643740 10 client.17022 fetch_fsmap learned FSMap version 27 2024-04-24T06:53:01.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.751+0000 7f04d1643740 1 -- 172.21.15.5:0/1493731933 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558e5e1d73f0 con 0x558e5e560270 2024-04-24T06:53:01.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.752+0000 7f04b5600640 1 -- 172.21.15.5:0/1493731933 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04c000ee10 con 0x558e5e560270 2024-04-24T06:53:01.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.752+0000 7f04b5600640 1 -- 172.21.15.5:0/1493731933 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f04b009c010 con 0x558e5e560270 2024-04-24T06:53:01.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.752+0000 7f04d1643740 10 client.17022 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:53:01.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.752+0000 7f04d1643740 10 client.17022 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T06:53:01.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.752+0000 7f04d1643740 20 client.17022 populate_metadata read hostname 'smithi005' 2024-04-24T06:53:01.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.752+0000 7f04d1643740 1 --2- 172.21.15.5:0/1493731933 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x558e5e57aa60 0x558e5e57ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:01.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.752+0000 7f04d1643740 4 client.17022 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:53:01.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.752+0000 7f04d1643740 1 -- 172.21.15.5:0/1493731933 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e5e4d2bf0 con 0x558e5e57aa60 2024-04-24T06:53:01.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.752+0000 7f04c7400640 1 --2- 172.21.15.5:0/1493731933 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x558e5e57aa60 0x558e5e57ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:01.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.753+0000 7f04c7400640 1 --2- 172.21.15.5:0/1493731933 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x558e5e57aa60 0x558e5e57ce70 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=99a7aa7536b34c6 server_cookie=b76443c751ad7779 in_seq=0 out_seq=0 2024-04-24T06:53:01.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.753+0000 7f04b5600640 10 client.17022 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:53:01.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.754+0000 7f04b5600640 1 -- 172.21.15.5:0/1493731933 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558e5e4d2bf0 con 0x558e5e57aa60 2024-04-24T06:53:01.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.754+0000 7f04b5600640 10 client.17022 handle_command_reply: tid=0 2024-04-24T06:53:01.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.754+0000 7f04d1643740 1 client.17022 shutdown 2024-04-24T06:53:01.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.754+0000 7f04d1643740 1 -- 172.21.15.5:0/1493731933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e5e560270 msgr2=0x558e5e567680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:01.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.754+0000 7f04d1643740 1 --2- 172.21.15.5:0/1493731933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e5e560270 0x558e5e567680 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f04b000c6a0 tx=0x7f04b00d4230 comp rx=0 tx=0).stop 2024-04-24T06:53:01.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.754+0000 7f04d1643740 1 -- 172.21.15.5:0/1493731933 shutdown_connections 2024-04-24T06:53:01.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.754+0000 7f04d1643740 1 --2- 172.21.15.5:0/1493731933 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x558e5e57aa60 0x558e5e57ce70 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.754+0000 7f04d1643740 1 --2- 172.21.15.5:0/1493731933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e5e56a940 0x558e5e567dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.754+0000 7f04b5600640 0 client.17022 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:53:01.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.754+0000 7f04d1643740 1 --2- 172.21.15.5:0/1493731933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e5e560270 0x558e5e567680 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.754+0000 7f04d1643740 1 --2- 172.21.15.5:0/1493731933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e5e55f950 0x558e5e56a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.754+0000 7f04d1643740 1 -- 172.21.15.5:0/1493731933 >> 172.21.15.5:0/1493731933 conn(0x558e5e55bc70 msgr2=0x558e5e561a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:01.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.755+0000 7f04d1643740 1 -- 172.21.15.5:0/1493731933 shutdown_connections 2024-04-24T06:53:01.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.755+0000 7f04d1643740 1 -- 172.21.15.5:0/1493731933 wait complete. 2024-04-24T06:53:01.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.755+0000 7f04d1643740 20 client.17022 trim_cache size 0 max 16384 2024-04-24T06:53:01.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:01 smithi037 ceph-mon[32864]: pgmap v276: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:53:01.771 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:01 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/4089171527' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:53:01.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.770+0000 7f04d1643740 1 Processor -- start 2024-04-24T06:53:01.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.770+0000 7f04d1643740 1 -- start start 2024-04-24T06:53:01.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.770+0000 7f04d1643740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e5e55f950 0x558e5e57ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:01.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.770+0000 7f04d1643740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e5e560270 0x558e5e576690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:01.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.770+0000 7f04d1643740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e5e56a940 0x558e5e566660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:01.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.770+0000 7f04d1643740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f04c81b2880 con 0x558e5e55f950 2024-04-24T06:53:01.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.770+0000 7f04d1643740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f04c81b0150 con 0x558e5e56a940 2024-04-24T06:53:01.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.770+0000 7f04d1643740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f04c81adfc0 con 0x558e5e560270 2024-04-24T06:53:01.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.770+0000 7f04c7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e5e55f950 0x558e5e57ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:01.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.771+0000 7f04c7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e5e56a940 0x558e5e566660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:01.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.771+0000 7f04c7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e5e55f950 0x558e5e57ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51926/0 (socket says 172.21.15.5:51926) 2024-04-24T06:53:01.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.771+0000 7f04c7400640 1 -- 172.21.15.5:0/4142496561 learned_addr learned my addr 172.21.15.5:0/4142496561 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:01.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.771+0000 7f04bfe00640 1 --2- 172.21.15.5:0/4142496561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e5e560270 0x558e5e576690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:01.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.771+0000 7f04c7400640 1 -- 172.21.15.5:0/4142496561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e5e560270 msgr2=0x558e5e576690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:01.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.771+0000 7f04c7400640 1 --2- 172.21.15.5:0/4142496561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e5e560270 0x558e5e576690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.771+0000 7f04c7400640 1 -- 172.21.15.5:0/4142496561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e5e56a940 msgr2=0x558e5e566660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:01.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.771+0000 7f04c7400640 1 --2- 172.21.15.5:0/4142496561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e5e56a940 0x558e5e566660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.771+0000 7f04c7400640 1 -- 172.21.15.5:0/4142496561 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e5e2a17d0 con 0x558e5e55f950 2024-04-24T06:53:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.771+0000 7f04c7e00640 1 --2- 172.21.15.5:0/4142496561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e5e56a940 0x558e5e566660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:53:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.771+0000 7f04c7400640 1 --2- 172.21.15.5:0/4142496561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e5e55f950 0x558e5e57ce50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f04c8101db0 tx=0x7f04b803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=129c458358c0c48 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.772+0000 7f04b7e00640 1 -- 172.21.15.5:0/4142496561 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b8035040 con 0x558e5e55f950 2024-04-24T06:53:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.772+0000 7f04b7e00640 1 -- 172.21.15.5:0/4142496561 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04b805a070 con 0x558e5e55f950 2024-04-24T06:53:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.772+0000 7f04b7e00640 1 -- 172.21.15.5:0/4142496561 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b80325c0 con 0x558e5e55f950 2024-04-24T06:53:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.772+0000 7f04d1643740 1 -- 172.21.15.5:0/4142496561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e5e55f950 msgr2=0x558e5e57ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.772+0000 7f04d1643740 1 --2- 172.21.15.5:0/4142496561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e5e55f950 0x558e5e57ce50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f04c8101db0 tx=0x7f04b803f000 comp rx=0 tx=0).stop 2024-04-24T06:53:01.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.773+0000 7f04d1643740 1 -- 172.21.15.5:0/4142496561 shutdown_connections 2024-04-24T06:53:01.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.773+0000 7f04d1643740 1 --2- 172.21.15.5:0/4142496561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e5e56a940 0x558e5e566660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.773+0000 7f04d1643740 1 --2- 172.21.15.5:0/4142496561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e5e560270 0x558e5e576690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.773+0000 7f04d1643740 1 --2- 172.21.15.5:0/4142496561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e5e55f950 0x558e5e57ce50 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.773+0000 7f04d1643740 1 -- 172.21.15.5:0/4142496561 >> 172.21.15.5:0/4142496561 conn(0x558e5e55bc70 msgr2=0x558e5e565f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:01.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.773+0000 7f04d1643740 1 -- 172.21.15.5:0/4142496561 shutdown_connections 2024-04-24T06:53:01.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.773+0000 7f04d1643740 1 -- 172.21.15.5:0/4142496561 wait complete. 2024-04-24T06:53:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.773+0000 7f04d1643740 1 -- start start 2024-04-24T06:53:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.773+0000 7f04d1643740 1 Processor -- start 2024-04-24T06:53:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.774+0000 7f04d1643740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e5e55f950 0x558e5e572ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.774+0000 7f04d1643740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e5e560270 0x558e5e5730e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.774+0000 7f04d1643740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e5e56a940 0x558e5e564900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:01.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.774+0000 7f04d1643740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f04c81ae610 con 0x558e5e560270 2024-04-24T06:53:01.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.774+0000 7f04d1643740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f04c81afa80 con 0x558e5e56a940 2024-04-24T06:53:01.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.774+0000 7f04d1643740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f04c81aece0 con 0x558e5e55f950 2024-04-24T06:53:01.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.774+0000 7f04d1643740 10 client.0 _ll_register_callbacks cb 0x7f04b000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:53:01.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.774+0000 7f04bfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e5e560270 0x558e5e5730e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:01.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.774+0000 7f04bfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e5e560270 0x558e5e5730e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51942/0 (socket says 172.21.15.5:51942) 2024-04-24T06:53:01.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.774+0000 7f04bfe00640 1 -- 172.21.15.5:0/1238103699 learned_addr learned my addr 172.21.15.5:0/1238103699 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:01.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.774+0000 7f04c7400640 1 --2- 172.21.15.5:0/1238103699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e5e55f950 0x558e5e572ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:01.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.774+0000 7f04c7e00640 1 --2- 172.21.15.5:0/1238103699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e5e56a940 0x558e5e564900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:01.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.775+0000 7f04bfe00640 1 -- 172.21.15.5:0/1238103699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e5e55f950 msgr2=0x558e5e572ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:01.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.775+0000 7f04bfe00640 1 --2- 172.21.15.5:0/1238103699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e5e55f950 0x558e5e572ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.775+0000 7f04bfe00640 1 -- 172.21.15.5:0/1238103699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e5e56a940 msgr2=0x558e5e564900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:01.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.775+0000 7f04bfe00640 1 --2- 172.21.15.5:0/1238103699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e5e56a940 0x558e5e564900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.775+0000 7f04bfe00640 1 -- 172.21.15.5:0/1238103699 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e5e565100 con 0x558e5e560270 2024-04-24T06:53:01.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.775+0000 7f04c7e00640 1 --2- 172.21.15.5:0/1238103699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e5e56a940 0x558e5e564900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:53:01.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.775+0000 7f04bfe00640 1 --2- 172.21.15.5:0/1238103699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e5e560270 0x558e5e5730e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f04b009c420 tx=0x7f04b00d0e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f69a78b95471a136 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:01.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.775+0000 7f04b5600640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:53:01.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.775+0000 7f04b5600640 1 -- 172.21.15.5:0/1238103699 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b009c670 con 0x558e5e560270 2024-04-24T06:53:01.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.775+0000 7f04b5600640 1 -- 172.21.15.5:0/1238103699 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f04b0010430 con 0x558e5e560270 2024-04-24T06:53:01.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.775+0000 7f04d1643740 1 -- 172.21.15.5:0/1238103699 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04b805a070 con 0x558e5e560270 2024-04-24T06:53:01.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.775+0000 7f04b5600640 1 -- 172.21.15.5:0/1238103699 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f04b009c900 con 0x558e5e560270 2024-04-24T06:53:01.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.775+0000 7f04d1643740 10 client.17028 fetch_fsmap learned FSMap version 27 2024-04-24T06:53:01.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.775+0000 7f04d1643740 1 -- 172.21.15.5:0/1238103699 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558e5e56b950 con 0x558e5e560270 2024-04-24T06:53:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.776+0000 7f04b5600640 1 -- 172.21.15.5:0/1238103699 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04b805a070 con 0x558e5e560270 2024-04-24T06:53:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.776+0000 7f04b5600640 1 -- 172.21.15.5:0/1238103699 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f04b009b9f0 con 0x558e5e560270 2024-04-24T06:53:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.776+0000 7f04d1643740 10 client.17028 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:53:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.776+0000 7f04d1643740 10 client.17028 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T06:53:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.776+0000 7f04d1643740 20 client.17028 populate_metadata read hostname 'smithi005' 2024-04-24T06:53:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.776+0000 7f04d1643740 1 --2- 172.21.15.5:0/1238103699 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x558e5e5e1210 0x558e5e5e3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.776+0000 7f04d1643740 4 client.17028 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:53:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.776+0000 7f04d1643740 1 -- 172.21.15.5:0/1238103699 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558e5e4cca60 con 0x558e5e5e1210 2024-04-24T06:53:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.777+0000 7f04c7400640 1 --2- 172.21.15.5:0/1238103699 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x558e5e5e1210 0x558e5e5e3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.777+0000 7f04c7400640 1 --2- 172.21.15.5:0/1238103699 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x558e5e5e1210 0x558e5e5e3630 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7e4c9107305a8c77 server_cookie=62950d54c4e9c54a in_seq=0 out_seq=0 2024-04-24T06:53:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.777+0000 7f04b5600640 10 client.17028 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:53:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.777+0000 7f04b5600640 1 -- 172.21.15.5:0/1238103699 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558e5e4cca60 con 0x558e5e5e1210 2024-04-24T06:53:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.777+0000 7f04b5600640 10 client.17028 handle_command_reply: tid=0 2024-04-24T06:53:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.777+0000 7f04d1643740 1 client.17028 shutdown 2024-04-24T06:53:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.778+0000 7f04d1643740 1 -- 172.21.15.5:0/1238103699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e5e560270 msgr2=0x558e5e5730e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:01.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.778+0000 7f04d1643740 1 --2- 172.21.15.5:0/1238103699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e5e560270 0x558e5e5730e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f04b009c420 tx=0x7f04b00d0e80 comp rx=0 tx=0).stop 2024-04-24T06:53:01.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.778+0000 7f04d1643740 1 -- 172.21.15.5:0/1238103699 shutdown_connections 2024-04-24T06:53:01.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.778+0000 7f04d1643740 1 --2- 172.21.15.5:0/1238103699 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x558e5e5e1210 0x558e5e5e3630 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.778+0000 7f04d1643740 1 --2- 172.21.15.5:0/1238103699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e5e56a940 0x558e5e564900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.778+0000 7f04d1643740 1 --2- 172.21.15.5:0/1238103699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e5e560270 0x558e5e5730e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.778+0000 7f04d1643740 1 --2- 172.21.15.5:0/1238103699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558e5e55f950 0x558e5e572ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.778+0000 7f04b5600640 0 client.17028 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:53:01.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.778+0000 7f04d1643740 1 -- 172.21.15.5:0/1238103699 >> 172.21.15.5:0/1238103699 conn(0x558e5e55bc70 msgr2=0x558e5e55a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:01.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.778+0000 7f04d1643740 1 -- 172.21.15.5:0/1238103699 shutdown_connections 2024-04-24T06:53:01.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.778+0000 7f04d1643740 1 -- 172.21.15.5:0/1238103699 wait complete. 2024-04-24T06:53:01.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.778+0000 7f04d1643740 20 client.17028 trim_cache size 0 max 16384 2024-04-24T06:53:01.789 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:53:01.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.779+0000 7f04ce800640 1 -- 172.21.15.5:0/557999570 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f04ac0a3da0 msgr2=0x7f04ac0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:01.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.779+0000 7f04ce800640 1 --2- 172.21.15.5:0/557999570 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f04ac0a3da0 0x7f04ac0a6260 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f04b8031550 tx=0x7f04b8005b50 comp rx=0 tx=0).stop 2024-04-24T06:53:01.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.779+0000 7f04ce800640 1 -- 172.21.15.5:0/557999570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04c8105e10 msgr2=0x7f04c81a0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:01.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.779+0000 7f04ce800640 1 --2- 172.21.15.5:0/557999570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04c8105e10 0x7f04c81a0d10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f04b000e9f0 tx=0x7f04b000eeb0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.779+0000 7f04ce800640 1 -- 172.21.15.5:0/557999570 shutdown_connections 2024-04-24T06:53:01.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.779+0000 7f04ce800640 1 --2- 172.21.15.5:0/557999570 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f04ac0a3da0 0x7f04ac0a6260 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.779+0000 7f04ce800640 1 --2- 172.21.15.5:0/557999570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04c81067b0 0x7f04c819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.779+0000 7f04ce800640 1 --2- 172.21.15.5:0/557999570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04c8105e10 0x7f04c81a0d10 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.779+0000 7f04ce800640 1 --2- 172.21.15.5:0/557999570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04c8104530 0x7f04c81a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:01.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.779+0000 7f04ce800640 1 -- 172.21.15.5:0/557999570 >> 172.21.15.5:0/557999570 conn(0x7f04c80ffc90 msgr2=0x7f04c8100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:01.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.779+0000 7f04ce800640 1 -- 172.21.15.5:0/557999570 shutdown_connections 2024-04-24T06:53:01.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:01.779+0000 7f04ce800640 1 -- 172.21.15.5:0/557999570 wait complete. 2024-04-24T06:53:01.798 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T06:53:02.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.011+0000 7f1414a00640 1 -- 172.21.15.5:0/3131726385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1410074430 msgr2=0x7f1410074810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:02.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.011+0000 7f1414a00640 1 --2- 172.21.15.5:0/3131726385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1410074430 0x7f1410074810 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1400009900 tx=0x7f140002a480 comp rx=0 tx=0).stop 2024-04-24T06:53:02.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.011+0000 7f1414a00640 1 -- 172.21.15.5:0/3131726385 shutdown_connections 2024-04-24T06:53:02.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.011+0000 7f1414a00640 1 --2- 172.21.15.5:0/3131726385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f141010e610 0x7f1410110a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:02.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.011+0000 7f1414a00640 1 --2- 172.21.15.5:0/3131726385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1410074d50 0x7f141010e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:02.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.011+0000 7f1414a00640 1 --2- 172.21.15.5:0/3131726385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1410074430 0x7f1410074810 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:02.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.011+0000 7f1414a00640 1 -- 172.21.15.5:0/3131726385 >> 172.21.15.5:0/3131726385 conn(0x7f14100fd8e0 msgr2=0x7f14100ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:02.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.011+0000 7f1414a00640 1 -- 172.21.15.5:0/3131726385 shutdown_connections 2024-04-24T06:53:02.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.011+0000 7f1414a00640 1 -- 172.21.15.5:0/3131726385 wait complete. 2024-04-24T06:53:02.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.012+0000 7f1414a00640 1 Processor -- start 2024-04-24T06:53:02.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.012+0000 7f1414a00640 1 -- start start 2024-04-24T06:53:02.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.013+0000 7f1414a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1410074430 0x7f14101a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:02.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.013+0000 7f1414a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1410074d50 0x7f14101a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:02.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.013+0000 7f1414a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f141010e610 0x7f141019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:02.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.013+0000 7f1414a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f14101143f0 con 0x7f1410074430 2024-04-24T06:53:02.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.013+0000 7f1414a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1410114270 con 0x7f141010e610 2024-04-24T06:53:02.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.013+0000 7f1414a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1410114570 con 0x7f1410074d50 2024-04-24T06:53:02.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.013+0000 7f140e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1410074430 0x7f14101a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:02.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.013+0000 7f140de00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1410074d50 0x7f14101a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:02.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.013+0000 7f140e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1410074430 0x7f14101a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51952/0 (socket says 172.21.15.5:51952) 2024-04-24T06:53:02.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.013+0000 7f140e800640 1 -- 172.21.15.5:0/3931037208 learned_addr learned my addr 172.21.15.5:0/3931037208 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:02.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.013+0000 7f140f200640 1 --2- 172.21.15.5:0/3931037208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f141010e610 0x7f141019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:02.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.013+0000 7f140de00640 1 -- 172.21.15.5:0/3931037208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f141010e610 msgr2=0x7f141019a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:02.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.013+0000 7f140de00640 1 --2- 172.21.15.5:0/3931037208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f141010e610 0x7f141019a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:02.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.013+0000 7f140de00640 1 -- 172.21.15.5:0/3931037208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1410074430 msgr2=0x7f14101a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:02.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.013+0000 7f140de00640 1 --2- 172.21.15.5:0/3931037208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1410074430 0x7f14101a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:02.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.013+0000 7f140de00640 1 -- 172.21.15.5:0/3931037208 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f141019b030 con 0x7f1410074d50 2024-04-24T06:53:02.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.014+0000 7f140de00640 1 --2- 172.21.15.5:0/3931037208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1410074d50 0x7f14101a0bf0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f13fc00e9f0 tx=0x7f13fc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:02.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.014+0000 7f1407400640 1 -- 172.21.15.5:0/3931037208 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13fc00ccc0 con 0x7f1410074d50 2024-04-24T06:53:02.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.014+0000 7f1407400640 1 -- 172.21.15.5:0/3931037208 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13fc005020 con 0x7f1410074d50 2024-04-24T06:53:02.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.014+0000 7f1414a00640 1 -- 172.21.15.5:0/3931037208 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f141019b320 con 0x7f1410074d50 2024-04-24T06:53:02.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.014+0000 7f1407400640 1 -- 172.21.15.5:0/3931037208 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13fc0049e0 con 0x7f1410074d50 2024-04-24T06:53:02.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.014+0000 7f1414a00640 1 -- 172.21.15.5:0/3931037208 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14101a7630 con 0x7f1410074d50 2024-04-24T06:53:02.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.017+0000 7f1418229740 1 Processor -- start 2024-04-24T06:53:02.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.017+0000 7f1418229740 1 -- start start 2024-04-24T06:53:02.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.018+0000 7f1418229740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558cde667b40 0x558cde667f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:02.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.018+0000 7f1418229740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cde668460 0x558cde6725f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:02.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.018+0000 7f1418229740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cde672b30 0x558cde674f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:02.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.018+0000 7f1418229740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558cde5cd1d0 con 0x558cde672b30 2024-04-24T06:53:02.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.018+0000 7f1418229740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558cde573450 con 0x558cde668460 2024-04-24T06:53:02.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.018+0000 7f1418229740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558cde5c9190 con 0x558cde667b40 2024-04-24T06:53:02.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.018+0000 7f140e800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cde668460 0x558cde6725f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:02.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.018+0000 7f140e800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cde668460 0x558cde6725f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:36194/0 (socket says 172.21.15.5:36194) 2024-04-24T06:53:02.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.018+0000 7f140e800640 1 -- 172.21.15.5:0/2874210629 learned_addr learned my addr 172.21.15.5:0/2874210629 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:02.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.018+0000 7f1407400640 1 -- 172.21.15.5:0/3931037208 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f13fc005560 con 0x7f1410074d50 2024-04-24T06:53:02.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.018+0000 7f140f200640 1 --2- 172.21.15.5:0/2874210629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558cde667b40 0x558cde667f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:02.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.018+0000 7f140e800640 1 -- 172.21.15.5:0/2874210629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558cde667b40 msgr2=0x558cde667f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:02.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.018+0000 7f140e800640 1 --2- 172.21.15.5:0/2874210629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558cde667b40 0x558cde667f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:02.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.019+0000 7f140e800640 1 -- 172.21.15.5:0/2874210629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cde672b30 msgr2=0x558cde674f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:02.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.019+0000 7f140de00640 1 --2- 172.21.15.5:0/2874210629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cde672b30 0x558cde674f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:02.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.019+0000 7f140e800640 1 --2- 172.21.15.5:0/2874210629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cde672b30 0x558cde674f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:02.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.019+0000 7f140e800640 1 -- 172.21.15.5:0/2874210629 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cde630310 con 0x558cde668460 2024-04-24T06:53:02.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.019+0000 7f140de00640 1 --2- 172.21.15.5:0/2874210629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cde672b30 0x558cde674f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:53:02.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.019+0000 7f140e800640 1 --2- 172.21.15.5:0/2874210629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cde668460 0x558cde6725f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1400005dd0 tx=0x7f1400005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56ca4bc81b429f6f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:02.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.019+0000 7f13fb400640 1 -- 172.21.15.5:0/2874210629 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1400040070 con 0x558cde668460 2024-04-24T06:53:02.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.019+0000 7f13fb400640 1 -- 172.21.15.5:0/2874210629 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1400004400 con 0x558cde668460 2024-04-24T06:53:02.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.019+0000 7f13fb400640 1 -- 172.21.15.5:0/2874210629 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1400035040 con 0x558cde668460 2024-04-24T06:53:02.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.020+0000 7f1407400640 1 --2- 172.21.15.5:0/3931037208 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f13f40a3d20 0x7f13f40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:02.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.020+0000 7f1418229740 1 -- 172.21.15.5:0/2874210629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cde668460 msgr2=0x558cde6725f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:02.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.020+0000 7f1418229740 1 --2- 172.21.15.5:0/2874210629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cde668460 0x558cde6725f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1400005dd0 tx=0x7f1400005c00 comp rx=0 tx=0).stop 2024-04-24T06:53:02.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.020+0000 7f1407400640 1 -- 172.21.15.5:0/3931037208 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f13fc0d07f0 con 0x7f1410074d50 2024-04-24T06:53:02.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.020+0000 7f1418229740 1 -- 172.21.15.5:0/2874210629 shutdown_connections 2024-04-24T06:53:02.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.020+0000 7f1418229740 1 --2- 172.21.15.5:0/2874210629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cde672b30 0x558cde674f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:02.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.020+0000 7f1418229740 1 --2- 172.21.15.5:0/2874210629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cde668460 0x558cde6725f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:02.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.020+0000 7f1418229740 1 --2- 172.21.15.5:0/2874210629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558cde667b40 0x558cde667f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:02.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.020+0000 7f1418229740 1 -- 172.21.15.5:0/2874210629 >> 172.21.15.5:0/2874210629 conn(0x558cde663e60 msgr2=0x558cde664240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:02.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.020+0000 7f140f200640 1 --2- 172.21.15.5:0/3931037208 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f13f40a3d20 0x7f13f40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:02.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.020+0000 7f1418229740 1 -- 172.21.15.5:0/2874210629 shutdown_connections 2024-04-24T06:53:02.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.020+0000 7f1418229740 1 -- 172.21.15.5:0/2874210629 wait complete. 2024-04-24T06:53:02.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.021+0000 7f1418229740 1 -- start start 2024-04-24T06:53:02.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.021+0000 7f140f200640 1 --2- 172.21.15.5:0/3931037208 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f13f40a3d20 0x7f13f40a61e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1408005fd0 tx=0x7f1408009500 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:02.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.021+0000 7f1418229740 1 Processor -- start 2024-04-24T06:53:02.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.022+0000 7f1418229740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558cde667b40 0x558cde672560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:02.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.022+0000 7f1418229740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cde668460 0x558cde66f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:02.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.022+0000 7f1418229740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cde672b30 0x558cde66ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:02.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.022+0000 7f1418229740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558cde675960 con 0x558cde672b30 2024-04-24T06:53:02.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.022+0000 7f1418229740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558cde586810 con 0x558cde668460 2024-04-24T06:53:02.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.022+0000 7f1418229740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558cde675ae0 con 0x558cde667b40 2024-04-24T06:53:02.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.022+0000 7f1418229740 10 client.0 _ll_register_callbacks cb 0x558cde554140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:53:02.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.022+0000 7f140e800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558cde667b40 0x558cde672560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:02.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.022+0000 7f140e800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558cde667b40 0x558cde672560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:58888/0 (socket says 172.21.15.5:58888) 2024-04-24T06:53:02.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.022+0000 7f140e800640 1 -- 172.21.15.5:0/102766284 learned_addr learned my addr 172.21.15.5:0/102766284 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:02.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.022+0000 7f140f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cde672b30 0x558cde66ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:02.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.022+0000 7f140de00640 1 --2- 172.21.15.5:0/102766284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cde668460 0x558cde66f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:02.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.022+0000 7f140e800640 1 -- 172.21.15.5:0/102766284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cde668460 msgr2=0x558cde66f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:02.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.022+0000 7f140e800640 1 --2- 172.21.15.5:0/102766284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cde668460 0x558cde66f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:02.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.022+0000 7f140e800640 1 -- 172.21.15.5:0/102766284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cde672b30 msgr2=0x558cde66ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:02.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.022+0000 7f140e800640 1 --2- 172.21.15.5:0/102766284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cde672b30 0x558cde66ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:02.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.022+0000 7f140e800640 1 -- 172.21.15.5:0/102766284 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cde5973f0 con 0x558cde667b40 2024-04-24T06:53:02.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.023+0000 7f140e800640 1 --2- 172.21.15.5:0/102766284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558cde667b40 0x558cde672560 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f140002aa70 tx=0x7f1400005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4522037c639fca0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:02.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.023+0000 7f13efe00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:53:02.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.023+0000 7f13efe00640 1 -- 172.21.15.5:0/102766284 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1400040070 con 0x558cde667b40 2024-04-24T06:53:02.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.023+0000 7f1418229740 1 -- 172.21.15.5:0/102766284 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1400004400 con 0x558cde667b40 2024-04-24T06:53:02.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.023+0000 7f13efe00640 1 -- 172.21.15.5:0/102766284 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f1400005230 con 0x558cde667b40 2024-04-24T06:53:02.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.024+0000 7f13efe00640 1 -- 172.21.15.5:0/102766284 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1400056050 con 0x558cde667b40 2024-04-24T06:53:02.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.024+0000 7f1418229740 10 client.27205 fetch_fsmap learned FSMap version 27 2024-04-24T06:53:02.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.024+0000 7f1418229740 1 -- 172.21.15.5:0/102766284 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558cde2df3f0 con 0x558cde667b40 2024-04-24T06:53:02.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.024+0000 7f13efe00640 1 -- 172.21.15.5:0/102766284 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1400004400 con 0x558cde667b40 2024-04-24T06:53:02.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.024+0000 7f13efe00640 1 -- 172.21.15.5:0/102766284 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f14000035f0 con 0x558cde667b40 2024-04-24T06:53:02.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.024+0000 7f1418229740 10 client.27205 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:53:02.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.024+0000 7f1418229740 10 client.27205 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:53:02.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.024+0000 7f1418229740 20 client.27205 populate_metadata read hostname 'smithi005' 2024-04-24T06:53:02.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.024+0000 7f1418229740 1 --2- 172.21.15.5:0/102766284 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x558cde682a70 0x558cde684e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:02.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.024+0000 7f1418229740 4 client.27205 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:53:02.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.024+0000 7f1418229740 1 -- 172.21.15.5:0/102766284 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558cde5dabf0 con 0x558cde682a70 2024-04-24T06:53:02.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.025+0000 7f140de00640 1 --2- 172.21.15.5:0/102766284 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x558cde682a70 0x558cde684e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:02.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.025+0000 7f140de00640 1 --2- 172.21.15.5:0/102766284 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x558cde682a70 0x558cde684e60 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7a1764514343815 server_cookie=bbc9d2150eda46e2 in_seq=0 out_seq=0 2024-04-24T06:53:02.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.025+0000 7f13efe00640 10 client.27205 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:53:02.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.026+0000 7f13efe00640 1 -- 172.21.15.5:0/102766284 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558cde5dabf0 con 0x558cde682a70 2024-04-24T06:53:02.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.026+0000 7f13efe00640 10 client.27205 handle_command_reply: tid=0 2024-04-24T06:53:02.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.026+0000 7f1418229740 1 client.27205 shutdown 2024-04-24T06:53:02.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.026+0000 7f1418229740 1 -- 172.21.15.5:0/102766284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558cde667b40 msgr2=0x558cde672560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:02.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.026+0000 7f1418229740 1 --2- 172.21.15.5:0/102766284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558cde667b40 0x558cde672560 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f140002aa70 tx=0x7f1400005850 comp rx=0 tx=0).stop 2024-04-24T06:53:02.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.026+0000 7f1418229740 1 -- 172.21.15.5:0/102766284 shutdown_connections 2024-04-24T06:53:02.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.026+0000 7f1418229740 1 --2- 172.21.15.5:0/102766284 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x558cde682a70 0x558cde684e60 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:02.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.026+0000 7f1418229740 1 --2- 172.21.15.5:0/102766284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cde672b30 0x558cde66ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:02.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.026+0000 7f1418229740 1 --2- 172.21.15.5:0/102766284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cde668460 0x558cde66f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:02.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.026+0000 7f1418229740 1 --2- 172.21.15.5:0/102766284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558cde667b40 0x558cde672560 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:02.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.026+0000 7f13efe00640 0 client.27205 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:53:02.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.026+0000 7f1418229740 1 -- 172.21.15.5:0/102766284 >> 172.21.15.5:0/102766284 conn(0x558cde663e60 msgr2=0x558cde669c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:02.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.027+0000 7f1418229740 1 -- 172.21.15.5:0/102766284 shutdown_connections 2024-04-24T06:53:02.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.027+0000 7f1418229740 1 -- 172.21.15.5:0/102766284 wait complete. 2024-04-24T06:53:02.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.027+0000 7f1418229740 20 client.27205 trim_cache size 0 max 16384 2024-04-24T06:53:02.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.042+0000 7f1418229740 1 Processor -- start 2024-04-24T06:53:02.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.042+0000 7f1418229740 1 -- start start 2024-04-24T06:53:02.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.042+0000 7f1418229740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558cde667b40 0x558cde67e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:02.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.042+0000 7f1418229740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cde668460 0x558cde66e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:02.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.042+0000 7f1418229740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cde672b30 0x558cde672090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:02.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.042+0000 7f1418229740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f14101b26c0 con 0x558cde672b30 2024-04-24T06:53:02.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.042+0000 7f1418229740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f14101aff90 con 0x558cde668460 2024-04-24T06:53:02.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.042+0000 7f1418229740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f14101ade00 con 0x558cde667b40 2024-04-24T06:53:02.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.043+0000 7f140e800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558cde667b40 0x558cde67e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:02.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.043+0000 7f140f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cde672b30 0x558cde672090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:02.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.043+0000 7f140e800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558cde667b40 0x558cde67e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:58892/0 (socket says 172.21.15.5:58892) 2024-04-24T06:53:02.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.043+0000 7f140e800640 1 -- 172.21.15.5:0/2957087071 learned_addr learned my addr 172.21.15.5:0/2957087071 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:02.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.043+0000 7f140de00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cde668460 0x558cde66e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:02.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.043+0000 7f140e800640 1 -- 172.21.15.5:0/2957087071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cde668460 msgr2=0x558cde66e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:02.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.043+0000 7f140e800640 1 --2- 172.21.15.5:0/2957087071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cde668460 0x558cde66e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:02.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.043+0000 7f140e800640 1 -- 172.21.15.5:0/2957087071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cde672b30 msgr2=0x558cde672090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:02.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.043+0000 7f140e800640 1 --2- 172.21.15.5:0/2957087071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cde672b30 0x558cde672090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:02.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.043+0000 7f140e800640 1 -- 172.21.15.5:0/2957087071 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cde6753d0 con 0x558cde667b40 2024-04-24T06:53:02.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.043+0000 7f140e800640 1 --2- 172.21.15.5:0/2957087071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558cde667b40 0x558cde67e820 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f140002a4c0 tx=0x7f1400005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91a4d874f46cd1c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:02.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.044+0000 7f13faa00640 1 -- 172.21.15.5:0/2957087071 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140005f070 con 0x558cde667b40 2024-04-24T06:53:02.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.044+0000 7f13faa00640 1 -- 172.21.15.5:0/2957087071 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f140004d070 con 0x558cde667b40 2024-04-24T06:53:02.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.044+0000 7f13faa00640 1 -- 172.21.15.5:0/2957087071 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1400040040 con 0x558cde667b40 2024-04-24T06:53:02.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.044+0000 7f1418229740 1 -- 172.21.15.5:0/2957087071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558cde667b40 msgr2=0x558cde67e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:02.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.044+0000 7f1418229740 1 --2- 172.21.15.5:0/2957087071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558cde667b40 0x558cde67e820 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f140002a4c0 tx=0x7f1400005c00 comp rx=0 tx=0).stop 2024-04-24T06:53:02.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.045+0000 7f1418229740 1 -- 172.21.15.5:0/2957087071 shutdown_connections 2024-04-24T06:53:02.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.045+0000 7f1418229740 1 --2- 172.21.15.5:0/2957087071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cde672b30 0x558cde672090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:02.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.045+0000 7f1418229740 1 --2- 172.21.15.5:0/2957087071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cde668460 0x558cde66e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:02.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.045+0000 7f1418229740 1 --2- 172.21.15.5:0/2957087071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558cde667b40 0x558cde67e820 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:02.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.045+0000 7f1418229740 1 -- 172.21.15.5:0/2957087071 >> 172.21.15.5:0/2957087071 conn(0x558cde663e60 msgr2=0x558cde66e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:02.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.045+0000 7f1418229740 1 -- 172.21.15.5:0/2957087071 shutdown_connections 2024-04-24T06:53:02.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.045+0000 7f1418229740 1 -- 172.21.15.5:0/2957087071 wait complete. 2024-04-24T06:53:02.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.045+0000 7f1418229740 1 -- start start 2024-04-24T06:53:02.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.045+0000 7f1418229740 1 Processor -- start 2024-04-24T06:53:02.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.046+0000 7f1418229740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cde667b40 0x558cde67b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:02.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.046+0000 7f1418229740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558cde668460 0x558cde66cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:02.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.046+0000 7f1418229740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cde672b30 0x558cde66d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:02.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.046+0000 7f1418229740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f14101ae450 con 0x558cde667b40 2024-04-24T06:53:02.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.046+0000 7f1418229740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f14101af8c0 con 0x558cde672b30 2024-04-24T06:53:02.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.046+0000 7f1418229740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f14101aeb20 con 0x558cde668460 2024-04-24T06:53:02.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.046+0000 7f1418229740 10 client.0 _ll_register_callbacks cb 0x7f1408009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:53:02.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.046+0000 7f140e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cde667b40 0x558cde67b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:02.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.046+0000 7f140f200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cde672b30 0x558cde66d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:02.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.046+0000 7f140f200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cde672b30 0x558cde66d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:36218/0 (socket says 172.21.15.5:36218) 2024-04-24T06:53:02.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.046+0000 7f140f200640 1 -- 172.21.15.5:0/3667133840 learned_addr learned my addr 172.21.15.5:0/3667133840 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:02.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.046+0000 7f140de00640 1 --2- 172.21.15.5:0/3667133840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558cde668460 0x558cde66cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:02.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.047+0000 7f140f200640 1 -- 172.21.15.5:0/3667133840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558cde668460 msgr2=0x558cde66cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:02.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.047+0000 7f140f200640 1 --2- 172.21.15.5:0/3667133840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558cde668460 0x558cde66cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:02.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.047+0000 7f140f200640 1 -- 172.21.15.5:0/3667133840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cde667b40 msgr2=0x558cde67b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:02.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.047+0000 7f140f200640 1 --2- 172.21.15.5:0/3667133840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cde667b40 0x558cde67b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:02.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.047+0000 7f140f200640 1 -- 172.21.15.5:0/3667133840 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cde670300 con 0x558cde672b30 2024-04-24T06:53:02.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.047+0000 7f140f200640 1 --2- 172.21.15.5:0/3667133840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cde672b30 0x558cde66d050 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f14080041b0 tx=0x7f1408004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57b9b9ad65b516f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:02.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.047+0000 7f13efe00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:53:02.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.047+0000 7f13efe00640 1 -- 172.21.15.5:0/3667133840 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1408004b20 con 0x558cde672b30 2024-04-24T06:53:02.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.047+0000 7f1418229740 1 -- 172.21.15.5:0/3667133840 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f140004d070 con 0x558cde672b30 2024-04-24T06:53:02.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.047+0000 7f13efe00640 1 -- 172.21.15.5:0/3667133840 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f1408004cc0 con 0x558cde672b30 2024-04-24T06:53:02.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.047+0000 7f13efe00640 1 -- 172.21.15.5:0/3667133840 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f140800f990 con 0x558cde672b30 2024-04-24T06:53:02.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.047+0000 7f1418229740 10 client.27197 fetch_fsmap learned FSMap version 27 2024-04-24T06:53:02.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.047+0000 7f1418229740 1 -- 172.21.15.5:0/3667133840 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558cde3a97d0 con 0x558cde672b30 2024-04-24T06:53:02.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.048+0000 7f13efe00640 1 -- 172.21.15.5:0/3667133840 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f140004d070 con 0x558cde672b30 2024-04-24T06:53:02.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.048+0000 7f13efe00640 1 -- 172.21.15.5:0/3667133840 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f140801e430 con 0x558cde672b30 2024-04-24T06:53:02.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.048+0000 7f1418229740 10 client.27197 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:53:02.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.048+0000 7f1418229740 10 client.27197 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:53:02.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.048+0000 7f1418229740 20 client.27197 populate_metadata read hostname 'smithi005' 2024-04-24T06:53:02.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.048+0000 7f1418229740 1 --2- 172.21.15.5:0/3667133840 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x558cde6e90e0 0x558cde6eb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:02.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.048+0000 7f1418229740 4 client.27197 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:53:02.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.048+0000 7f1418229740 1 -- 172.21.15.5:0/3667133840 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558cde5d4a60 con 0x558cde6e90e0 2024-04-24T06:53:02.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.048+0000 7f140e800640 1 --2- 172.21.15.5:0/3667133840 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x558cde6e90e0 0x558cde6eb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:02.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.049+0000 7f140e800640 1 --2- 172.21.15.5:0/3667133840 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x558cde6e90e0 0x558cde6eb4d0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d69728a761b6f3 server_cookie=24ffb8e38d888583 in_seq=0 out_seq=0 2024-04-24T06:53:02.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.049+0000 7f13efe00640 10 client.27197 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:53:02.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.049+0000 7f13efe00640 1 -- 172.21.15.5:0/3667133840 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558cde5d4a60 con 0x558cde6e90e0 2024-04-24T06:53:02.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.049+0000 7f13efe00640 10 client.27197 handle_command_reply: tid=0 2024-04-24T06:53:02.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.049+0000 7f1418229740 1 client.27197 shutdown 2024-04-24T06:53:02.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.049+0000 7f1418229740 1 -- 172.21.15.5:0/3667133840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cde672b30 msgr2=0x558cde66d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:02.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.049+0000 7f1418229740 1 --2- 172.21.15.5:0/3667133840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cde672b30 0x558cde66d050 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f14080041b0 tx=0x7f1408004a40 comp rx=0 tx=0).stop 2024-04-24T06:53:02.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.049+0000 7f1418229740 1 -- 172.21.15.5:0/3667133840 shutdown_connections 2024-04-24T06:53:02.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.049+0000 7f1418229740 1 --2- 172.21.15.5:0/3667133840 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x558cde6e90e0 0x558cde6eb4d0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:02.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.049+0000 7f1418229740 1 --2- 172.21.15.5:0/3667133840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cde672b30 0x558cde66d050 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:02.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.049+0000 7f1418229740 1 --2- 172.21.15.5:0/3667133840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558cde668460 0x558cde66cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:02.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.049+0000 7f1418229740 1 --2- 172.21.15.5:0/3667133840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cde667b40 0x558cde67b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:02.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.049+0000 7f1418229740 1 -- 172.21.15.5:0/3667133840 >> 172.21.15.5:0/3667133840 conn(0x558cde663e60 msgr2=0x558cde662f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:02.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.049+0000 7f13efe00640 0 client.27197 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:53:02.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.049+0000 7f1418229740 1 -- 172.21.15.5:0/3667133840 shutdown_connections 2024-04-24T06:53:02.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.049+0000 7f1418229740 1 -- 172.21.15.5:0/3667133840 wait complete. 2024-04-24T06:53:02.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.050+0000 7f1418229740 20 client.27197 trim_cache size 0 max 16384 2024-04-24T06:53:02.061 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:53:02.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.050+0000 7f1414a00640 1 -- 172.21.15.5:0/3931037208 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f13f40a3d20 msgr2=0x7f13f40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:02.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.050+0000 7f1414a00640 1 --2- 172.21.15.5:0/3931037208 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f13f40a3d20 0x7f13f40a61e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1408005fd0 tx=0x7f1408009500 comp rx=0 tx=0).stop 2024-04-24T06:53:02.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.050+0000 7f1414a00640 1 -- 172.21.15.5:0/3931037208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1410074d50 msgr2=0x7f14101a0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:02.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.050+0000 7f1414a00640 1 --2- 172.21.15.5:0/3931037208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1410074d50 0x7f14101a0bf0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f13fc00e9f0 tx=0x7f13fc00eeb0 comp rx=0 tx=0).stop 2024-04-24T06:53:02.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.050+0000 7f1414a00640 1 -- 172.21.15.5:0/3931037208 shutdown_connections 2024-04-24T06:53:02.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.050+0000 7f1414a00640 1 --2- 172.21.15.5:0/3931037208 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f13f40a3d20 0x7f13f40a61e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:02.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.050+0000 7f1414a00640 1 --2- 172.21.15.5:0/3931037208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f141010e610 0x7f141019a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:02.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.050+0000 7f1414a00640 1 --2- 172.21.15.5:0/3931037208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1410074d50 0x7f14101a0bf0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:02.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.050+0000 7f1414a00640 1 --2- 172.21.15.5:0/3931037208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1410074430 0x7f14101a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:02.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.050+0000 7f1414a00640 1 -- 172.21.15.5:0/3931037208 >> 172.21.15.5:0/3931037208 conn(0x7f14100fd8e0 msgr2=0x7f14100ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:02.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.050+0000 7f1414a00640 1 -- 172.21.15.5:0/3931037208 shutdown_connections 2024-04-24T06:53:02.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:02.051+0000 7f1414a00640 1 -- 172.21.15.5:0/3931037208 wait complete. 2024-04-24T06:53:03.069 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T06:53:03.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:53:03.070 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T06:53:03.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.306+0000 7fa6dc400640 1 -- 172.21.15.5:0/3958322958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6d4104e00 msgr2=0x7fa6d4108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:03.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.306+0000 7fa6dc400640 1 --2- 172.21.15.5:0/3958322958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6d4104e00 0x7fa6d4108e40 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0009900 tx=0x7fa6c002a410 comp rx=0 tx=0).stop 2024-04-24T06:53:03.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.306+0000 7fa6dc400640 1 -- 172.21.15.5:0/3958322958 shutdown_connections 2024-04-24T06:53:03.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.306+0000 7fa6dc400640 1 --2- 172.21.15.5:0/3958322958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6d4109520 0x7fa6d41110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.306+0000 7fa6dc400640 1 --2- 172.21.15.5:0/3958322958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6d4104e00 0x7fa6d4108e40 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.306+0000 7fa6dc400640 1 --2- 172.21.15.5:0/3958322958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6d4104450 0x7fa6d4104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.306+0000 7fa6dc400640 1 -- 172.21.15.5:0/3958322958 >> 172.21.15.5:0/3958322958 conn(0x7fa6d40ffc90 msgr2=0x7fa6d41020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:03.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.306+0000 7fa6dc400640 1 -- 172.21.15.5:0/3958322958 shutdown_connections 2024-04-24T06:53:03.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.306+0000 7fa6dc400640 1 -- 172.21.15.5:0/3958322958 wait complete. 2024-04-24T06:53:03.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.307+0000 7fa6dc400640 1 Processor -- start 2024-04-24T06:53:03.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.308+0000 7fa6dc400640 1 -- start start 2024-04-24T06:53:03.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.308+0000 7fa6dc400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6d4104450 0x7fa6d41a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:03.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.308+0000 7fa6dc400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6d4109520 0x7fa6d41a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:03.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.308+0000 7fa6dc400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6d419a980 0x7fa6d419ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:03.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.308+0000 7fa6dc400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d4114600 con 0x7fa6d4109520 2024-04-24T06:53:03.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.308+0000 7fa6dc400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d4114480 con 0x7fa6d4104450 2024-04-24T06:53:03.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.308+0000 7fa6dc400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d4114780 con 0x7fa6d419a980 2024-04-24T06:53:03.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.308+0000 7fa6d9a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6d4104450 0x7fa6d41a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:03.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.308+0000 7fa6d9a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6d4104450 0x7fa6d41a0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:36250/0 (socket says 172.21.15.5:36250) 2024-04-24T06:53:03.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.308+0000 7fa6da400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6d419a980 0x7fa6d419ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:03.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.308+0000 7fa6d9a00640 1 -- 172.21.15.5:0/2119017471 learned_addr learned my addr 172.21.15.5:0/2119017471 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:03.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.308+0000 7fa6d9000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6d4109520 0x7fa6d41a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:03.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.309+0000 7fa6d9a00640 1 -- 172.21.15.5:0/2119017471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6d419a980 msgr2=0x7fa6d419ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:03.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.309+0000 7fa6d9a00640 1 --2- 172.21.15.5:0/2119017471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6d419a980 0x7fa6d419ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.309+0000 7fa6d9a00640 1 -- 172.21.15.5:0/2119017471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6d4109520 msgr2=0x7fa6d41a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:03.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.309+0000 7fa6d9a00640 1 --2- 172.21.15.5:0/2119017471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6d4109520 0x7fa6d41a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.309+0000 7fa6d9a00640 1 -- 172.21.15.5:0/2119017471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d419b4a0 con 0x7fa6d4104450 2024-04-24T06:53:03.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.309+0000 7fa6d9a00640 1 --2- 172.21.15.5:0/2119017471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6d4104450 0x7fa6d41a0780 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa6c800e9f0 tx=0x7fa6c800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:03.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.309+0000 7fa6d2000640 1 -- 172.21.15.5:0/2119017471 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c800ccc0 con 0x7fa6d4104450 2024-04-24T06:53:03.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.309+0000 7fa6dc400640 1 -- 172.21.15.5:0/2119017471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6d419b790 con 0x7fa6d4104450 2024-04-24T06:53:03.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.309+0000 7fa6d2000640 1 -- 172.21.15.5:0/2119017471 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6c800ce60 con 0x7fa6d4104450 2024-04-24T06:53:03.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.309+0000 7fa6d2000640 1 -- 172.21.15.5:0/2119017471 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c8010690 con 0x7fa6d4104450 2024-04-24T06:53:03.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.309+0000 7fa6dc400640 1 -- 172.21.15.5:0/2119017471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6d41a7990 con 0x7fa6d4104450 2024-04-24T06:53:03.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.313+0000 7fa6df3cf740 1 Processor -- start 2024-04-24T06:53:03.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.313+0000 7fa6df3cf740 1 -- start start 2024-04-24T06:53:03.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.313+0000 7fa6df3cf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55656cb20090 0x55656cb20470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:03.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.313+0000 7fa6df3cf740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55656cb209b0 0x55656cb2ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:03.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.313+0000 7fa6df3cf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55656cb2b080 0x55656cb2d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:03.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.313+0000 7fa6df3cf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55656ca85e70 con 0x55656cb2b080 2024-04-24T06:53:03.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.314+0000 7fa6df3cf740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55656ca2c380 con 0x55656cb209b0 2024-04-24T06:53:03.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.314+0000 7fa6df3cf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55656ca81e30 con 0x55656cb20090 2024-04-24T06:53:03.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.314+0000 7fa6da400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55656cb20090 0x55656cb20470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:03.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.314+0000 7fa6da400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55656cb20090 0x55656cb20470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:58922/0 (socket says 172.21.15.5:58922) 2024-04-24T06:53:03.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.314+0000 7fa6da400640 1 -- 172.21.15.5:0/1862085150 learned_addr learned my addr 172.21.15.5:0/1862085150 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:03.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.314+0000 7fa6d2000640 1 -- 172.21.15.5:0/2119017471 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fa6c80046c0 con 0x7fa6d4104450 2024-04-24T06:53:03.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.314+0000 7fa6d9000640 1 --2- 172.21.15.5:0/1862085150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55656cb2b080 0x55656cb2d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:03.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.314+0000 7fa6da400640 1 -- 172.21.15.5:0/1862085150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55656cb209b0 msgr2=0x55656cb2ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:03.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.314+0000 7fa6da400640 1 --2- 172.21.15.5:0/1862085150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55656cb209b0 0x55656cb2ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.314+0000 7fa6da400640 1 -- 172.21.15.5:0/1862085150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55656cb2b080 msgr2=0x55656cb2d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:03.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.314+0000 7fa6da400640 1 --2- 172.21.15.5:0/1862085150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55656cb2b080 0x55656cb2d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.314+0000 7fa6da400640 1 -- 172.21.15.5:0/1862085150 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55656c8607e0 con 0x55656cb20090 2024-04-24T06:53:03.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.315+0000 7fa6da400640 1 --2- 172.21.15.5:0/1862085150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55656cb20090 0x55656cb20470 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc009f60 tx=0x7fa6cc009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17f50e87d346258a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:03.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.315+0000 7fa6c6a00640 1 -- 172.21.15.5:0/1862085150 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc01e070 con 0x55656cb20090 2024-04-24T06:53:03.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.315+0000 7fa6c6a00640 1 -- 172.21.15.5:0/1862085150 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6cc003c30 con 0x55656cb20090 2024-04-24T06:53:03.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.315+0000 7fa6c6a00640 1 -- 172.21.15.5:0/1862085150 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc00f920 con 0x55656cb20090 2024-04-24T06:53:03.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.316+0000 7fa6df3cf740 1 -- 172.21.15.5:0/1862085150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55656cb20090 msgr2=0x55656cb20470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:03.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.316+0000 7fa6df3cf740 1 --2- 172.21.15.5:0/1862085150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55656cb20090 0x55656cb20470 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc009f60 tx=0x7fa6cc009f90 comp rx=0 tx=0).stop 2024-04-24T06:53:03.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.316+0000 7fa6d2000640 1 --2- 172.21.15.5:0/2119017471 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fa6bc0a3d20 0x7fa6bc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:03.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.316+0000 7fa6df3cf740 1 -- 172.21.15.5:0/1862085150 shutdown_connections 2024-04-24T06:53:03.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.316+0000 7fa6df3cf740 1 --2- 172.21.15.5:0/1862085150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55656cb2b080 0x55656cb2d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.316+0000 7fa6df3cf740 1 --2- 172.21.15.5:0/1862085150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55656cb209b0 0x55656cb2ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.316+0000 7fa6df3cf740 1 --2- 172.21.15.5:0/1862085150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55656cb20090 0x55656cb20470 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.316+0000 7fa6df3cf740 1 -- 172.21.15.5:0/1862085150 >> 172.21.15.5:0/1862085150 conn(0x55656cb1c3b0 msgr2=0x55656cb1c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:03.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.316+0000 7fa6d2000640 1 -- 172.21.15.5:0/2119017471 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7fa6c80d0bd0 con 0x7fa6d4104450 2024-04-24T06:53:03.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.316+0000 7fa6d9a00640 1 --2- 172.21.15.5:0/2119017471 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fa6bc0a3d20 0x7fa6bc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:03.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.316+0000 7fa6df3cf740 1 -- 172.21.15.5:0/1862085150 shutdown_connections 2024-04-24T06:53:03.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.316+0000 7fa6df3cf740 1 -- 172.21.15.5:0/1862085150 wait complete. 2024-04-24T06:53:03.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.317+0000 7fa6df3cf740 1 -- start start 2024-04-24T06:53:03.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.317+0000 7fa6d9a00640 1 --2- 172.21.15.5:0/2119017471 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fa6bc0a3d20 0x7fa6bc0a61e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x55656cb21650 tx=0x7fa6c8004a00 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:03.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.317+0000 7fa6df3cf740 1 Processor -- start 2024-04-24T06:53:03.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.317+0000 7fa6df3cf740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55656cb20090 0x55656cb28050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:03.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.317+0000 7fa6df3cf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55656cb209b0 0x55656cb28590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:03.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.317+0000 7fa6df3cf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55656cb2b080 0x55656cb28ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:03.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.317+0000 7fa6df3cf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55656cb2de70 con 0x55656cb2b080 2024-04-24T06:53:03.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.318+0000 7fa6df3cf740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55656ca3f660 con 0x55656cb20090 2024-04-24T06:53:03.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.318+0000 7fa6df3cf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55656cb2dff0 con 0x55656cb209b0 2024-04-24T06:53:03.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.318+0000 7fa6df3cf740 10 client.0 _ll_register_callbacks cb 0x55656ca2daa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:53:03.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.318+0000 7fa6d9000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55656cb209b0 0x55656cb28590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:03.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.318+0000 7fa6d9000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55656cb209b0 0x55656cb28590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:58932/0 (socket says 172.21.15.5:58932) 2024-04-24T06:53:03.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.318+0000 7fa6d9000640 1 -- 172.21.15.5:0/2500279897 learned_addr learned my addr 172.21.15.5:0/2500279897 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:03.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.318+0000 7fa6da400640 1 --2- 172.21.15.5:0/2500279897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55656cb2b080 0x55656cb28ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:03.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.318+0000 7fa6da400640 1 --2- 172.21.15.5:0/2500279897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55656cb20090 0x55656cb28050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:03.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.318+0000 7fa6d9000640 1 -- 172.21.15.5:0/2500279897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55656cb20090 msgr2=0x55656cb28050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:03.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.318+0000 7fa6d9000640 1 --2- 172.21.15.5:0/2500279897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55656cb20090 0x55656cb28050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.318+0000 7fa6d9000640 1 -- 172.21.15.5:0/2500279897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55656cb2b080 msgr2=0x55656cb28ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:03.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.318+0000 7fa6d9000640 1 --2- 172.21.15.5:0/2500279897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55656cb2b080 0x55656cb28ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.318+0000 7fa6d9000640 1 -- 172.21.15.5:0/2500279897 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55656c8c0f70 con 0x55656cb209b0 2024-04-24T06:53:03.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.318+0000 7fa6da400640 1 --2- 172.21.15.5:0/2500279897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55656cb2b080 0x55656cb28ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:53:03.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.319+0000 7fa6d9000640 1 --2- 172.21.15.5:0/2500279897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55656cb209b0 0x55656cb28590 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0033000 tx=0x7fa6c00043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cc6c76b29831dc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:03.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.319+0000 7fa6bb400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:53:03.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.319+0000 7fa6bb400640 1 -- 172.21.15.5:0/2500279897 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0035040 con 0x55656cb209b0 2024-04-24T06:53:03.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.319+0000 7fa6bb400640 1 -- 172.21.15.5:0/2500279897 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7fa6c0005450 con 0x55656cb209b0 2024-04-24T06:53:03.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.319+0000 7fa6df3cf740 1 -- 172.21.15.5:0/2500279897 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6cc003c30 con 0x55656cb209b0 2024-04-24T06:53:03.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.319+0000 7fa6bb400640 1 -- 172.21.15.5:0/2500279897 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa6c0005650 con 0x55656cb209b0 2024-04-24T06:53:03.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.319+0000 7fa6df3cf740 10 client.27223 fetch_fsmap learned FSMap version 27 2024-04-24T06:53:03.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.319+0000 7fa6df3cf740 1 -- 172.21.15.5:0/2500279897 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55656c758170 con 0x55656cb209b0 2024-04-24T06:53:03.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.320+0000 7fa6bb400640 1 -- 172.21.15.5:0/2500279897 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6cc003c30 con 0x55656cb209b0 2024-04-24T06:53:03.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.320+0000 7fa6bb400640 1 -- 172.21.15.5:0/2500279897 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7fa6c0031a20 con 0x55656cb209b0 2024-04-24T06:53:03.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.320+0000 7fa6df3cf740 10 client.27223 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:53:03.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.320+0000 7fa6df3cf740 10 client.27223 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:53:03.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.320+0000 7fa6df3cf740 20 client.27223 populate_metadata read hostname 'smithi005' 2024-04-24T06:53:03.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.320+0000 7fa6df3cf740 1 --2- 172.21.15.5:0/2500279897 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55656cb3b450 0x55656cb3d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:03.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.320+0000 7fa6df3cf740 4 client.27223 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:53:03.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.320+0000 7fa6df3cf740 1 -- 172.21.15.5:0/2500279897 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55656ca93890 con 0x55656cb3b450 2024-04-24T06:53:03.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.320+0000 7fa6da400640 1 --2- 172.21.15.5:0/2500279897 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55656cb3b450 0x55656cb3d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:03.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.321+0000 7fa6da400640 1 --2- 172.21.15.5:0/2500279897 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55656cb3b450 0x55656cb3d890 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a2ab34eea199e54 server_cookie=a449a1e7ab24f920 in_seq=0 out_seq=0 2024-04-24T06:53:03.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.321+0000 7fa6bb400640 10 client.27223 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:53:03.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.322+0000 7fa6bb400640 1 -- 172.21.15.5:0/2500279897 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55656ca93890 con 0x55656cb3b450 2024-04-24T06:53:03.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.322+0000 7fa6bb400640 10 client.27223 handle_command_reply: tid=0 2024-04-24T06:53:03.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.322+0000 7fa6df3cf740 1 client.27223 shutdown 2024-04-24T06:53:03.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.322+0000 7fa6df3cf740 1 -- 172.21.15.5:0/2500279897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55656cb209b0 msgr2=0x55656cb28590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:03.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.322+0000 7fa6df3cf740 1 --2- 172.21.15.5:0/2500279897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55656cb209b0 0x55656cb28590 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0033000 tx=0x7fa6c00043b0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.322+0000 7fa6df3cf740 1 -- 172.21.15.5:0/2500279897 shutdown_connections 2024-04-24T06:53:03.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.322+0000 7fa6df3cf740 1 --2- 172.21.15.5:0/2500279897 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55656cb3b450 0x55656cb3d890 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.322+0000 7fa6df3cf740 1 --2- 172.21.15.5:0/2500279897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55656cb2b080 0x55656cb28ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.322+0000 7fa6df3cf740 1 --2- 172.21.15.5:0/2500279897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55656cb209b0 0x55656cb28590 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.322+0000 7fa6df3cf740 1 --2- 172.21.15.5:0/2500279897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55656cb20090 0x55656cb28050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.322+0000 7fa6bb400640 0 client.27223 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:53:03.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.322+0000 7fa6df3cf740 1 -- 172.21.15.5:0/2500279897 >> 172.21.15.5:0/2500279897 conn(0x55656cb1c3b0 msgr2=0x55656cb221b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:03.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.322+0000 7fa6df3cf740 1 -- 172.21.15.5:0/2500279897 shutdown_connections 2024-04-24T06:53:03.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.322+0000 7fa6df3cf740 1 -- 172.21.15.5:0/2500279897 wait complete. 2024-04-24T06:53:03.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.323+0000 7fa6df3cf740 20 client.27223 trim_cache size 0 max 16384 2024-04-24T06:53:03.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.345+0000 7fa6df3cf740 1 Processor -- start 2024-04-24T06:53:03.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.345+0000 7fa6df3cf740 1 -- start start 2024-04-24T06:53:03.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.346+0000 7fa6df3cf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55656cb20090 0x55656cb36d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:03.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.346+0000 7fa6df3cf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55656cb209b0 0x55656cb1b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:03.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.346+0000 7fa6df3cf740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55656cb2b080 0x55656cb43160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.346+0000 7fa6df3cf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d41b2870 con 0x55656cb209b0 2024-04-24T06:53:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.346+0000 7fa6df3cf740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d41b0140 con 0x55656cb2b080 2024-04-24T06:53:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.346+0000 7fa6df3cf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d41adfb0 con 0x55656cb20090 2024-04-24T06:53:03.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.346+0000 7fa6da400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55656cb20090 0x55656cb36d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:03.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.346+0000 7fa6da400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55656cb20090 0x55656cb36d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:58946/0 (socket says 172.21.15.5:58946) 2024-04-24T06:53:03.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.346+0000 7fa6d9000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55656cb209b0 0x55656cb1b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:03.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.346+0000 7fa6da400640 1 -- 172.21.15.5:0/2422422120 learned_addr learned my addr 172.21.15.5:0/2422422120 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:03.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.346+0000 7fa6da400640 1 --2- 172.21.15.5:0/2422422120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55656cb2b080 0x55656cb43160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:03.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.346+0000 7fa6d9000640 1 -- 172.21.15.5:0/2422422120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55656cb20090 msgr2=0x55656cb36d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.347+0000 7fa6d9000640 1 --2- 172.21.15.5:0/2422422120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55656cb20090 0x55656cb36d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.347+0000 7fa6d9000640 1 -- 172.21.15.5:0/2422422120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55656cb2b080 msgr2=0x55656cb43160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.347+0000 7fa6d9000640 1 --2- 172.21.15.5:0/2422422120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55656cb2b080 0x55656cb43160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.347+0000 7fa6d9000640 1 -- 172.21.15.5:0/2422422120 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55656cb27040 con 0x55656cb209b0 2024-04-24T06:53:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.347+0000 7fa6d9000640 1 --2- 172.21.15.5:0/2422422120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55656cb209b0 0x55656cb1b390 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0031df0 tx=0x7fa6c00038f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f815b660d2baa2a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.347+0000 7fa6c6000640 1 -- 172.21.15.5:0/2422422120 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0035030 con 0x55656cb209b0 2024-04-24T06:53:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.347+0000 7fa6c6000640 1 -- 172.21.15.5:0/2422422120 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6c0041070 con 0x55656cb209b0 2024-04-24T06:53:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.347+0000 7fa6c6000640 1 -- 172.21.15.5:0/2422422120 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0052070 con 0x55656cb209b0 2024-04-24T06:53:03.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.348+0000 7fa6df3cf740 1 -- 172.21.15.5:0/2422422120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55656cb209b0 msgr2=0x55656cb1b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:03.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.348+0000 7fa6df3cf740 1 --2- 172.21.15.5:0/2422422120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55656cb209b0 0x55656cb1b390 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0031df0 tx=0x7fa6c00038f0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.348+0000 7fa6df3cf740 1 -- 172.21.15.5:0/2422422120 shutdown_connections 2024-04-24T06:53:03.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.348+0000 7fa6df3cf740 1 --2- 172.21.15.5:0/2422422120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55656cb2b080 0x55656cb43160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.348+0000 7fa6df3cf740 1 --2- 172.21.15.5:0/2422422120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55656cb209b0 0x55656cb1b390 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.348+0000 7fa6df3cf740 1 --2- 172.21.15.5:0/2422422120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55656cb20090 0x55656cb36d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.348+0000 7fa6df3cf740 1 -- 172.21.15.5:0/2422422120 >> 172.21.15.5:0/2422422120 conn(0x55656cb1c3b0 msgr2=0x55656cb1ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:03.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.348+0000 7fa6df3cf740 1 -- 172.21.15.5:0/2422422120 shutdown_connections 2024-04-24T06:53:03.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.348+0000 7fa6df3cf740 1 -- 172.21.15.5:0/2422422120 wait complete. 2024-04-24T06:53:03.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.349+0000 7fa6df3cf740 1 -- start start 2024-04-24T06:53:03.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.349+0000 7fa6df3cf740 1 Processor -- start 2024-04-24T06:53:03.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.349+0000 7fa6df3cf740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55656cb20090 0x55656cb253a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:03.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.349+0000 7fa6df3cf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55656cb209b0 0x55656cb258e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:03.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.349+0000 7fa6df3cf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55656cb2b080 0x55656cba09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:03.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.350+0000 7fa6df3cf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d41ae600 con 0x55656cb209b0 2024-04-24T06:53:03.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.350+0000 7fa6df3cf740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d41afa70 con 0x55656cb20090 2024-04-24T06:53:03.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.350+0000 7fa6df3cf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d41aecd0 con 0x55656cb2b080 2024-04-24T06:53:03.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.350+0000 7fa6df3cf740 10 client.0 _ll_register_callbacks cb 0x7fa6c0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:53:03.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.350+0000 7fa6d9000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55656cb209b0 0x55656cb258e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:03.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.350+0000 7fa6d9000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55656cb209b0 0x55656cb258e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52046/0 (socket says 172.21.15.5:52046) 2024-04-24T06:53:03.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.350+0000 7fa6d9000640 1 -- 172.21.15.5:0/2069669156 learned_addr learned my addr 172.21.15.5:0/2069669156 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:03.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.350+0000 7fa6da400640 1 --2- 172.21.15.5:0/2069669156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55656cb20090 0x55656cb253a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:03.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.350+0000 7fa6da400640 1 --2- 172.21.15.5:0/2069669156 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55656cb2b080 0x55656cba09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:03.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.350+0000 7fa6d9000640 1 -- 172.21.15.5:0/2069669156 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55656cb2b080 msgr2=0x55656cba09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:03.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.350+0000 7fa6d9000640 1 --2- 172.21.15.5:0/2069669156 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55656cb2b080 0x55656cba09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.350+0000 7fa6d9000640 1 -- 172.21.15.5:0/2069669156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55656cb20090 msgr2=0x55656cb253a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:03.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.350+0000 7fa6d9000640 1 --2- 172.21.15.5:0/2069669156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55656cb20090 0x55656cb253a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.350+0000 7fa6d9000640 1 -- 172.21.15.5:0/2069669156 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55656cba0f10 con 0x55656cb209b0 2024-04-24T06:53:03.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.351+0000 7fa6d9000640 1 --2- 172.21.15.5:0/2069669156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55656cb209b0 0x55656cb258e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa6c00314c0 tx=0x7fa6c00043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8147f402b0ac2060 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:03.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.351+0000 7fa6bb400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:53:03.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.351+0000 7fa6bb400640 1 -- 172.21.15.5:0/2069669156 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0052070 con 0x55656cb209b0 2024-04-24T06:53:03.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.351+0000 7fa6bb400640 1 -- 172.21.15.5:0/2069669156 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7fa6c0003e70 con 0x55656cb209b0 2024-04-24T06:53:03.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.351+0000 7fa6df3cf740 1 -- 172.21.15.5:0/2069669156 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6c0041070 con 0x55656cb209b0 2024-04-24T06:53:03.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.351+0000 7fa6bb400640 1 -- 172.21.15.5:0/2069669156 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa6c0066050 con 0x55656cb209b0 2024-04-24T06:53:03.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.351+0000 7fa6df3cf740 10 client.17061 fetch_fsmap learned FSMap version 27 2024-04-24T06:53:03.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.351+0000 7fa6df3cf740 1 -- 172.21.15.5:0/2069669156 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55656cba2160 con 0x55656cb209b0 2024-04-24T06:53:03.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.352+0000 7fa6bb400640 1 -- 172.21.15.5:0/2069669156 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6c0041070 con 0x55656cb209b0 2024-04-24T06:53:03.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.352+0000 7fa6bb400640 1 -- 172.21.15.5:0/2069669156 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7fa6c00048e0 con 0x55656cb209b0 2024-04-24T06:53:03.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.352+0000 7fa6df3cf740 10 client.17061 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:53:03.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.352+0000 7fa6df3cf740 10 client.17061 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:53:03.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.352+0000 7fa6df3cf740 20 client.17061 populate_metadata read hostname 'smithi005' 2024-04-24T06:53:03.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.352+0000 7fa6df3cf740 1 --2- 172.21.15.5:0/2069669156 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55656cba2760 0x55656cba4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:03.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.352+0000 7fa6df3cf740 4 client.17061 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T06:53:03.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.352+0000 7fa6df3cf740 1 -- 172.21.15.5:0/2069669156 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55656ca8d700 con 0x55656cba2760 2024-04-24T06:53:03.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.352+0000 7fa6da400640 1 --2- 172.21.15.5:0/2069669156 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55656cba2760 0x55656cba4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:03.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.353+0000 7fa6da400640 1 --2- 172.21.15.5:0/2069669156 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55656cba2760 0x55656cba4b80 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e83d631c8b7e168 server_cookie=6aff32f784de2be in_seq=0 out_seq=0 2024-04-24T06:53:03.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.353+0000 7fa6bb400640 10 client.17061 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:53:03.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.354+0000 7fa6bb400640 1 -- 172.21.15.5:0/2069669156 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55656ca8d700 con 0x55656cba2760 2024-04-24T06:53:03.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.354+0000 7fa6bb400640 10 client.17061 handle_command_reply: tid=0 2024-04-24T06:53:03.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.354+0000 7fa6df3cf740 1 client.17061 shutdown 2024-04-24T06:53:03.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.354+0000 7fa6df3cf740 1 -- 172.21.15.5:0/2069669156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55656cb209b0 msgr2=0x55656cb258e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:03.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.354+0000 7fa6df3cf740 1 --2- 172.21.15.5:0/2069669156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55656cb209b0 0x55656cb258e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa6c00314c0 tx=0x7fa6c00043b0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.354+0000 7fa6df3cf740 1 -- 172.21.15.5:0/2069669156 shutdown_connections 2024-04-24T06:53:03.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.354+0000 7fa6df3cf740 1 --2- 172.21.15.5:0/2069669156 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55656cba2760 0x55656cba4b80 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.354+0000 7fa6df3cf740 1 --2- 172.21.15.5:0/2069669156 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55656cb2b080 0x55656cba09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.354+0000 7fa6df3cf740 1 --2- 172.21.15.5:0/2069669156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55656cb209b0 0x55656cb258e0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.354+0000 7fa6df3cf740 1 --2- 172.21.15.5:0/2069669156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55656cb20090 0x55656cb253a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.354+0000 7fa6df3cf740 1 -- 172.21.15.5:0/2069669156 >> 172.21.15.5:0/2069669156 conn(0x55656cb1c3b0 msgr2=0x55656cb2a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:03.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.354+0000 7fa6bb400640 0 client.17061 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:53:03.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.355+0000 7fa6df3cf740 1 -- 172.21.15.5:0/2069669156 shutdown_connections 2024-04-24T06:53:03.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.355+0000 7fa6df3cf740 1 -- 172.21.15.5:0/2069669156 wait complete. 2024-04-24T06:53:03.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.355+0000 7fa6df3cf740 20 client.17061 trim_cache size 0 max 16384 2024-04-24T06:53:03.365 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:53:03.365 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-24T06:53:03.365 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "e1da777c-44eb-422f-ab09-99ff986fd7b6", 2024-04-24T06:53:03.366 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-24T06:53:03.366 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:53:03.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.355+0000 7fa6dc400640 1 -- 172.21.15.5:0/2119017471 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fa6bc0a3d20 msgr2=0x7fa6bc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:03.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.355+0000 7fa6dc400640 1 --2- 172.21.15.5:0/2119017471 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fa6bc0a3d20 0x7fa6bc0a61e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x55656cb21650 tx=0x7fa6c8004a00 comp rx=0 tx=0).stop 2024-04-24T06:53:03.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.355+0000 7fa6dc400640 1 -- 172.21.15.5:0/2119017471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6d4104450 msgr2=0x7fa6d41a0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:03.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.355+0000 7fa6dc400640 1 --2- 172.21.15.5:0/2119017471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6d4104450 0x7fa6d41a0780 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa6c800e9f0 tx=0x7fa6c800edf0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.356+0000 7fa6dc400640 1 -- 172.21.15.5:0/2119017471 shutdown_connections 2024-04-24T06:53:03.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.356+0000 7fa6dc400640 1 --2- 172.21.15.5:0/2119017471 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fa6bc0a3d20 0x7fa6bc0a61e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.356+0000 7fa6dc400640 1 --2- 172.21.15.5:0/2119017471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6d419a980 0x7fa6d419ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.356+0000 7fa6dc400640 1 --2- 172.21.15.5:0/2119017471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6d4109520 0x7fa6d41a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.356+0000 7fa6dc400640 1 --2- 172.21.15.5:0/2119017471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6d4104450 0x7fa6d41a0780 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.356+0000 7fa6dc400640 1 -- 172.21.15.5:0/2119017471 >> 172.21.15.5:0/2119017471 conn(0x7fa6d40ffc90 msgr2=0x7fa6d4100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:03.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.356+0000 7fa6dc400640 1 -- 172.21.15.5:0/2119017471 shutdown_connections 2024-04-24T06:53:03.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.356+0000 7fa6dc400640 1 -- 172.21.15.5:0/2119017471 wait complete. 2024-04-24T06:53:03.381 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T06:53:03.381 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T06:53:03.444 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:03 smithi005 ceph-mon[27798]: pgmap v277: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:53:03.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.611+0000 7f1b1aa00640 1 -- 172.21.15.5:0/4273685610 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b14102230 msgr2=0x7f1b14102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:03.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.611+0000 7f1b1aa00640 1 --2- 172.21.15.5:0/4273685610 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b14102230 0x7f1b14102610 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1b04009900 tx=0x7f1b0402a3f0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.611+0000 7f1b1aa00640 1 -- 172.21.15.5:0/4273685610 shutdown_connections 2024-04-24T06:53:03.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.611+0000 7f1b1aa00640 1 --2- 172.21.15.5:0/4273685610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b14107300 0x7f1b1410eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.611+0000 7f1b1aa00640 1 --2- 172.21.15.5:0/4273685610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b14102be0 0x7f1b14106c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.611+0000 7f1b1aa00640 1 --2- 172.21.15.5:0/4273685610 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b14102230 0x7f1b14102610 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.611+0000 7f1b1aa00640 1 -- 172.21.15.5:0/4273685610 >> 172.21.15.5:0/4273685610 conn(0x7f1b140fdab0 msgr2=0x7f1b140ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:03.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.611+0000 7f1b1aa00640 1 -- 172.21.15.5:0/4273685610 shutdown_connections 2024-04-24T06:53:03.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.611+0000 7f1b1aa00640 1 -- 172.21.15.5:0/4273685610 wait complete. 2024-04-24T06:53:03.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.612+0000 7f1b1aa00640 1 Processor -- start 2024-04-24T06:53:03.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.612+0000 7f1b1aa00640 1 -- start start 2024-04-24T06:53:03.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.613+0000 7f1b1aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b14102230 0x7f1b141a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:03.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.613+0000 7f1b1aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b14107300 0x7f1b141a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:03.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.613+0000 7f1b1aa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b1419a930 0x7f1b1419ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:03.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.613+0000 7f1b1aa00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1b141123e0 con 0x7f1b14107300 2024-04-24T06:53:03.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.613+0000 7f1b1aa00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1b14112260 con 0x7f1b14102230 2024-04-24T06:53:03.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.613+0000 7f1b1aa00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1b14112560 con 0x7f1b1419a930 2024-04-24T06:53:03.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.613+0000 7f1b13e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b14102230 0x7f1b141a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:03.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.613+0000 7f1b18a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b1419a930 0x7f1b1419ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:03.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.613+0000 7f1b13e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b14102230 0x7f1b141a0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:36288/0 (socket says 172.21.15.5:36288) 2024-04-24T06:53:03.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.613+0000 7f1b13e00640 1 -- 172.21.15.5:0/2083590185 learned_addr learned my addr 172.21.15.5:0/2083590185 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:03.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.613+0000 7f1b13400640 1 --2- 172.21.15.5:0/2083590185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b14107300 0x7f1b141a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:03.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.614+0000 7f1b18a00640 1 -- 172.21.15.5:0/2083590185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b14102230 msgr2=0x7f1b141a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:03.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.614+0000 7f1b18a00640 1 --2- 172.21.15.5:0/2083590185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b14102230 0x7f1b141a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.614+0000 7f1b18a00640 1 -- 172.21.15.5:0/2083590185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b14107300 msgr2=0x7f1b141a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:03.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.614+0000 7f1b18a00640 1 --2- 172.21.15.5:0/2083590185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b14107300 0x7f1b141a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.614+0000 7f1b18a00640 1 -- 172.21.15.5:0/2083590185 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b1419b450 con 0x7f1b1419a930 2024-04-24T06:53:03.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.614+0000 7f1b18a00640 1 --2- 172.21.15.5:0/2083590185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b1419a930 0x7f1b1419ade0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c009fd0 tx=0x7f1b0c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:03.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.614+0000 7f1b10c00640 1 -- 172.21.15.5:0/2083590185 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c00ce20 con 0x7f1b1419a930 2024-04-24T06:53:03.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.615+0000 7f1b10c00640 1 -- 172.21.15.5:0/2083590185 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b0c005020 con 0x7f1b1419a930 2024-04-24T06:53:03.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.615+0000 7f1b10c00640 1 -- 172.21.15.5:0/2083590185 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c010720 con 0x7f1b1419a930 2024-04-24T06:53:03.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.615+0000 7f1b1aa00640 1 -- 172.21.15.5:0/2083590185 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b1419b740 con 0x7f1b1419a930 2024-04-24T06:53:03.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.615+0000 7f1b1aa00640 1 -- 172.21.15.5:0/2083590185 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b141a7940 con 0x7f1b1419a930 2024-04-24T06:53:03.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.619+0000 7f1b10c00640 1 -- 172.21.15.5:0/2083590185 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f1b0c005540 con 0x7f1b1419a930 2024-04-24T06:53:03.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.619+0000 7f1b1d834740 1 Processor -- start 2024-04-24T06:53:03.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.619+0000 7f1b1d834740 1 -- start start 2024-04-24T06:53:03.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.620+0000 7f1b1d834740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557463e5860 0x5557463e5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:03.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.620+0000 7f1b1d834740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557463e6180 0x5557463f0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:03.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.620+0000 7f1b1d834740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557463f0850 0x5557463f2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:03.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.620+0000 7f1b1d834740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55574634b1f0 con 0x5557463f0850 2024-04-24T06:53:03.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.620+0000 7f1b1d834740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5557462f1350 con 0x5557463e6180 2024-04-24T06:53:03.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.620+0000 7f1b1d834740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5557463471b0 con 0x5557463e5860 2024-04-24T06:53:03.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.620+0000 7f1b13400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557463e5860 0x5557463e5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:03.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.620+0000 7f1b13e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557463e6180 0x5557463f0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:03.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.620+0000 7f1b13e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557463e6180 0x5557463f0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:36296/0 (socket says 172.21.15.5:36296) 2024-04-24T06:53:03.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.620+0000 7f1b13e00640 1 -- 172.21.15.5:0/835751344 learned_addr learned my addr 172.21.15.5:0/835751344 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:03.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.620+0000 7f1b13400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557463e5860 0x5557463e5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:58992/0 (socket says 172.21.15.5:58992) 2024-04-24T06:53:03.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.620+0000 7f1b13400640 1 --2- 172.21.15.5:0/835751344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557463f0850 0x5557463f2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:03.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.621+0000 7f1b13e00640 1 -- 172.21.15.5:0/835751344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557463e5860 msgr2=0x5557463e5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:03.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.621+0000 7f1b13e00640 1 --2- 172.21.15.5:0/835751344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557463e5860 0x5557463e5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.621+0000 7f1b13e00640 1 -- 172.21.15.5:0/835751344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557463f0850 msgr2=0x5557463f2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:03.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.621+0000 7f1b13e00640 1 --2- 172.21.15.5:0/835751344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557463f0850 0x5557463f2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.621+0000 7f1b10c00640 1 --2- 172.21.15.5:0/2083590185 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1af80a3da0 0x7f1af80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:03.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.621+0000 7f1b13e00640 1 -- 172.21.15.5:0/835751344 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555746345f50 con 0x5557463e6180 2024-04-24T06:53:03.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.621+0000 7f1b10c00640 1 -- 172.21.15.5:0/2083590185 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f1b0c0d0e40 con 0x7f1b1419a930 2024-04-24T06:53:03.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.621+0000 7f1b18a00640 1 --2- 172.21.15.5:0/2083590185 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1af80a3da0 0x7f1af80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:03.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.621+0000 7f1b13e00640 1 --2- 172.21.15.5:0/835751344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557463e6180 0x5557463f0310 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1b04006010 tx=0x7f1b04005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89b387022bfd655e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:03.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.622+0000 7f1b09200640 1 -- 172.21.15.5:0/835751344 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b04040070 con 0x5557463e6180 2024-04-24T06:53:03.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.622+0000 7f1b09200640 1 -- 172.21.15.5:0/835751344 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b04004400 con 0x5557463e6180 2024-04-24T06:53:03.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.622+0000 7f1b09200640 1 -- 172.21.15.5:0/835751344 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b04035040 con 0x5557463e6180 2024-04-24T06:53:03.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.623+0000 7f1b1d834740 1 -- 172.21.15.5:0/835751344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557463e6180 msgr2=0x5557463f0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:03.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.623+0000 7f1b18a00640 1 --2- 172.21.15.5:0/2083590185 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1af80a3da0 0x7f1af80a6260 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c0a54b0 tx=0x7f1b0c005f70 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:03.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.623+0000 7f1b1d834740 1 --2- 172.21.15.5:0/835751344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557463e6180 0x5557463f0310 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1b04006010 tx=0x7f1b04005dd0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.623+0000 7f1b1d834740 1 -- 172.21.15.5:0/835751344 shutdown_connections 2024-04-24T06:53:03.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.623+0000 7f1b1d834740 1 --2- 172.21.15.5:0/835751344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557463f0850 0x5557463f2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.623+0000 7f1b1d834740 1 --2- 172.21.15.5:0/835751344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557463e6180 0x5557463f0310 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.623+0000 7f1b1d834740 1 --2- 172.21.15.5:0/835751344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557463e5860 0x5557463e5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.623+0000 7f1b1d834740 1 -- 172.21.15.5:0/835751344 >> 172.21.15.5:0/835751344 conn(0x5557463e1b80 msgr2=0x5557463e1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:03.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.623+0000 7f1b1d834740 1 -- 172.21.15.5:0/835751344 shutdown_connections 2024-04-24T06:53:03.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.623+0000 7f1b1d834740 1 -- 172.21.15.5:0/835751344 wait complete. 2024-04-24T06:53:03.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.624+0000 7f1b1d834740 1 -- start start 2024-04-24T06:53:03.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.624+0000 7f1b1d834740 1 Processor -- start 2024-04-24T06:53:03.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.624+0000 7f1b1d834740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557463e5860 0x5557463ed7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:03.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.624+0000 7f1b1d834740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557463e6180 0x5557463edcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:03.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.624+0000 7f1b1d834740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557463f0850 0x5557463ee230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:03.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.624+0000 7f1b1d834740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5557463f35e0 con 0x5557463e6180 2024-04-24T06:53:03.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.624+0000 7f1b1d834740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555746304690 con 0x5557463e5860 2024-04-24T06:53:03.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.624+0000 7f1b1d834740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5557463f3760 con 0x5557463f0850 2024-04-24T06:53:03.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.624+0000 7f1b1d834740 10 client.0 _ll_register_callbacks cb 0x555746340650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:53:03.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.625+0000 7f1b13400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557463e5860 0x5557463ed7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:03.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.625+0000 7f1b13400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557463e5860 0x5557463ed7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:36300/0 (socket says 172.21.15.5:36300) 2024-04-24T06:53:03.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.625+0000 7f1b13400640 1 -- 172.21.15.5:0/2033566686 learned_addr learned my addr 172.21.15.5:0/2033566686 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:03.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.625+0000 7f1b13400640 1 --2- 172.21.15.5:0/2033566686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557463f0850 0x5557463ee230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:03.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.625+0000 7f1b13e00640 1 --2- 172.21.15.5:0/2033566686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557463e6180 0x5557463edcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:03.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.625+0000 7f1b13400640 1 -- 172.21.15.5:0/2033566686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557463f0850 msgr2=0x5557463ee230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:03.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.625+0000 7f1b13400640 1 --2- 172.21.15.5:0/2033566686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557463f0850 0x5557463ee230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.625+0000 7f1b13400640 1 -- 172.21.15.5:0/2033566686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557463e6180 msgr2=0x5557463edcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:03.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.625+0000 7f1b13400640 1 --2- 172.21.15.5:0/2033566686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557463e6180 0x5557463edcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.625+0000 7f1b13400640 1 -- 172.21.15.5:0/2033566686 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555746303920 con 0x5557463e5860 2024-04-24T06:53:03.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.625+0000 7f1b13400640 1 --2- 172.21.15.5:0/2033566686 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557463e5860 0x5557463ed7b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1b1406b7b0 tx=0x7f1b00004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49cb416b573bfc38 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:03.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.625+0000 7f1afd600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:53:03.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.626+0000 7f1afd600640 1 -- 172.21.15.5:0/2033566686 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0000fc50 con 0x5557463e5860 2024-04-24T06:53:03.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.626+0000 7f1afd600640 1 -- 172.21.15.5:0/2033566686 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f1b0000fdf0 con 0x5557463e5860 2024-04-24T06:53:03.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.626+0000 7f1b1d834740 1 -- 172.21.15.5:0/2033566686 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b04004400 con 0x5557463e5860 2024-04-24T06:53:03.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.626+0000 7f1afd600640 1 -- 172.21.15.5:0/2033566686 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1b00004240 con 0x5557463e5860 2024-04-24T06:53:03.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.626+0000 7f1b1d834740 10 client.27224 fetch_fsmap learned FSMap version 27 2024-04-24T06:53:03.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.626+0000 7f1b1d834740 1 -- 172.21.15.5:0/2033566686 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5557461277d0 con 0x5557463e5860 2024-04-24T06:53:03.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.626+0000 7f1afd600640 1 -- 172.21.15.5:0/2033566686 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b04004400 con 0x5557463e5860 2024-04-24T06:53:03.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.627+0000 7f1afd600640 1 -- 172.21.15.5:0/2033566686 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f1b00022420 con 0x5557463e5860 2024-04-24T06:53:03.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.627+0000 7f1b1d834740 10 client.27224 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:53:03.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.627+0000 7f1b1d834740 10 client.27224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:53:03.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.627+0000 7f1b1d834740 20 client.27224 populate_metadata read hostname 'smithi005' 2024-04-24T06:53:03.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.627+0000 7f1b1d834740 1 --2- 172.21.15.5:0/2033566686 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x555746400b30 0x555746402f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:03.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.627+0000 7f1b1d834740 4 client.27224 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:53:03.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.627+0000 7f1b1d834740 1 -- 172.21.15.5:0/2033566686 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555746358c10 con 0x555746400b30 2024-04-24T06:53:03.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.627+0000 7f1b13e00640 1 --2- 172.21.15.5:0/2033566686 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x555746400b30 0x555746402f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:03.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.628+0000 7f1b13e00640 1 --2- 172.21.15.5:0/2033566686 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x555746400b30 0x555746402f60 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54dfe3899a287fe7 server_cookie=fbf0e07b1683eac4 in_seq=0 out_seq=0 2024-04-24T06:53:03.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.628+0000 7f1afd600640 10 client.27224 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:53:03.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.629+0000 7f1afd600640 1 -- 172.21.15.5:0/2033566686 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555746358c10 con 0x555746400b30 2024-04-24T06:53:03.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.629+0000 7f1afd600640 10 client.27224 handle_command_reply: tid=0 2024-04-24T06:53:03.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.629+0000 7f1b1d834740 1 client.27224 shutdown 2024-04-24T06:53:03.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.629+0000 7f1b1d834740 1 -- 172.21.15.5:0/2033566686 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557463e5860 msgr2=0x5557463ed7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:03.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.629+0000 7f1b1d834740 1 --2- 172.21.15.5:0/2033566686 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557463e5860 0x5557463ed7b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1b1406b7b0 tx=0x7f1b00004730 comp rx=0 tx=0).stop 2024-04-24T06:53:03.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.629+0000 7f1b1d834740 1 -- 172.21.15.5:0/2033566686 shutdown_connections 2024-04-24T06:53:03.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.629+0000 7f1b1d834740 1 --2- 172.21.15.5:0/2033566686 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x555746400b30 0x555746402f60 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.629+0000 7f1b1d834740 1 --2- 172.21.15.5:0/2033566686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557463f0850 0x5557463ee230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.629+0000 7f1b1d834740 1 --2- 172.21.15.5:0/2033566686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557463e6180 0x5557463edcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.629+0000 7f1b1d834740 1 --2- 172.21.15.5:0/2033566686 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557463e5860 0x5557463ed7b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.629+0000 7f1b1d834740 1 -- 172.21.15.5:0/2033566686 >> 172.21.15.5:0/2033566686 conn(0x5557463e1b80 msgr2=0x5557463e7980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:03.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.629+0000 7f1afd600640 0 client.27224 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:53:03.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.629+0000 7f1b1d834740 1 -- 172.21.15.5:0/2033566686 shutdown_connections 2024-04-24T06:53:03.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.629+0000 7f1b1d834740 1 -- 172.21.15.5:0/2033566686 wait complete. 2024-04-24T06:53:03.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.630+0000 7f1b1d834740 20 client.27224 trim_cache size 0 max 16384 2024-04-24T06:53:03.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.654+0000 7f1b1d834740 1 Processor -- start 2024-04-24T06:53:03.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.654+0000 7f1b1d834740 1 -- start start 2024-04-24T06:53:03.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.654+0000 7f1b1d834740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557463e5860 0x5557463fc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:03.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.654+0000 7f1b1d834740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557463e6180 0x5557463e8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:03.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.654+0000 7f1b1d834740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557463f0850 0x555746408830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:03.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.654+0000 7f1b1d834740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1b141b28b0 con 0x5557463f0850 2024-04-24T06:53:03.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.654+0000 7f1b1d834740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1b141b0180 con 0x5557463e5860 2024-04-24T06:53:03.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.654+0000 7f1b1d834740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1b141adff0 con 0x5557463e6180 2024-04-24T06:53:03.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.655+0000 7f1b13e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557463e6180 0x5557463e8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:03.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.655+0000 7f1b13e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557463e6180 0x5557463e8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:59010/0 (socket says 172.21.15.5:59010) 2024-04-24T06:53:03.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.655+0000 7f1b13e00640 1 -- 172.21.15.5:0/3010002105 learned_addr learned my addr 172.21.15.5:0/3010002105 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:03.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.655+0000 7f1b13400640 1 --2- 172.21.15.5:0/3010002105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557463e5860 0x5557463fc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:03.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.655+0000 7f1b13400640 1 --2- 172.21.15.5:0/3010002105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557463f0850 0x555746408830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:03.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.655+0000 7f1b13e00640 1 -- 172.21.15.5:0/3010002105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557463e5860 msgr2=0x5557463fc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:03.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.655+0000 7f1b13e00640 1 --2- 172.21.15.5:0/3010002105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557463e5860 0x5557463fc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.655+0000 7f1b13e00640 1 -- 172.21.15.5:0/3010002105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557463f0850 msgr2=0x555746408830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:03.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.655+0000 7f1b13e00640 1 --2- 172.21.15.5:0/3010002105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557463f0850 0x555746408830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.655+0000 7f1b13e00640 1 -- 172.21.15.5:0/3010002105 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55574605d3f0 con 0x5557463e6180 2024-04-24T06:53:03.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.655+0000 7f1b13e00640 1 --2- 172.21.15.5:0/3010002105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557463e6180 0x5557463e8000 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1b04004b30 tx=0x7f1b0402a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e089154c0d89c46c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:03.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.656+0000 7f1affe00640 1 -- 172.21.15.5:0/3010002105 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b04035040 con 0x5557463e6180 2024-04-24T06:53:03.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.656+0000 7f1affe00640 1 -- 172.21.15.5:0/3010002105 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b04004200 con 0x5557463e6180 2024-04-24T06:53:03.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.656+0000 7f1affe00640 1 -- 172.21.15.5:0/3010002105 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b04040040 con 0x5557463e6180 2024-04-24T06:53:03.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.657+0000 7f1b1d834740 1 -- 172.21.15.5:0/3010002105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557463e6180 msgr2=0x5557463e8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:03.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.657+0000 7f1b1d834740 1 --2- 172.21.15.5:0/3010002105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557463e6180 0x5557463e8000 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1b04004b30 tx=0x7f1b0402a3f0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.657+0000 7f1b1d834740 1 -- 172.21.15.5:0/3010002105 shutdown_connections 2024-04-24T06:53:03.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.657+0000 7f1b1d834740 1 --2- 172.21.15.5:0/3010002105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557463f0850 0x555746408830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.657+0000 7f1b1d834740 1 --2- 172.21.15.5:0/3010002105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557463e6180 0x5557463e8000 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.657+0000 7f1b1d834740 1 --2- 172.21.15.5:0/3010002105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557463e5860 0x5557463fc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.657+0000 7f1b1d834740 1 -- 172.21.15.5:0/3010002105 >> 172.21.15.5:0/3010002105 conn(0x5557463e1b80 msgr2=0x5557463e0390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:03.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.657+0000 7f1b1d834740 1 -- 172.21.15.5:0/3010002105 shutdown_connections 2024-04-24T06:53:03.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.657+0000 7f1b1d834740 1 -- 172.21.15.5:0/3010002105 wait complete. 2024-04-24T06:53:03.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.658+0000 7f1b1d834740 1 -- start start 2024-04-24T06:53:03.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.658+0000 7f1b1d834740 1 Processor -- start 2024-04-24T06:53:03.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.658+0000 7f1b1d834740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557463e5860 0x5557463f8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:03.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.658+0000 7f1b1d834740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557463e6180 0x5557463f8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:03.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.658+0000 7f1b1d834740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557463f0850 0x5557463f3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:03.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.658+0000 7f1b1d834740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1b141ae640 con 0x5557463f0850 2024-04-24T06:53:03.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.658+0000 7f1b1d834740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1b141afab0 con 0x5557463e5860 2024-04-24T06:53:03.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.658+0000 7f1b1d834740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1b141aed10 con 0x5557463e6180 2024-04-24T06:53:03.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.658+0000 7f1b1d834740 10 client.0 _ll_register_callbacks cb 0x7f1b00011ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:53:03.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.659+0000 7f1b13e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557463e6180 0x5557463f8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:03.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.659+0000 7f1b13e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557463e6180 0x5557463f8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:59020/0 (socket says 172.21.15.5:59020) 2024-04-24T06:53:03.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.659+0000 7f1b13e00640 1 -- 172.21.15.5:0/3962198805 learned_addr learned my addr 172.21.15.5:0/3962198805 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:03.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.659+0000 7f1b13400640 1 --2- 172.21.15.5:0/3962198805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557463e5860 0x5557463f8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:03.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.659+0000 7f1b13400640 1 --2- 172.21.15.5:0/3962198805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557463f0850 0x5557463f3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:03.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.659+0000 7f1b13e00640 1 -- 172.21.15.5:0/3962198805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557463e5860 msgr2=0x5557463f8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:03.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.659+0000 7f1b13e00640 1 --2- 172.21.15.5:0/3962198805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557463e5860 0x5557463f8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.659+0000 7f1b13e00640 1 -- 172.21.15.5:0/3962198805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557463f0850 msgr2=0x5557463f3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:03.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.659+0000 7f1b13e00640 1 --2- 172.21.15.5:0/3962198805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557463f0850 0x5557463f3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.659+0000 7f1b13e00640 1 -- 172.21.15.5:0/3962198805 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55574601d0f0 con 0x5557463e6180 2024-04-24T06:53:03.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.659+0000 7f1b13400640 1 --2- 172.21.15.5:0/3962198805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557463f0850 0x5557463f3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:53:03.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.659+0000 7f1b13e00640 1 --2- 172.21.15.5:0/3962198805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557463e6180 0x5557463f8cb0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1b04000c00 tx=0x7f1b040095e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a25d8814cbcf189 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:03.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.659+0000 7f1afd600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:53:03.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.660+0000 7f1afd600640 1 -- 172.21.15.5:0/3962198805 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b04055070 con 0x5557463e6180 2024-04-24T06:53:03.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.660+0000 7f1b1d834740 1 -- 172.21.15.5:0/3962198805 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b04004200 con 0x5557463e6180 2024-04-24T06:53:03.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.660+0000 7f1afd600640 1 -- 172.21.15.5:0/3962198805 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f1b040502d0 con 0x5557463e6180 2024-04-24T06:53:03.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.660+0000 7f1afd600640 1 -- 172.21.15.5:0/3962198805 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1b04035050 con 0x5557463e6180 2024-04-24T06:53:03.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.660+0000 7f1b1d834740 10 client.27247 fetch_fsmap learned FSMap version 27 2024-04-24T06:53:03.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.660+0000 7f1b1d834740 1 -- 172.21.15.5:0/3962198805 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555746310810 con 0x5557463e6180 2024-04-24T06:53:03.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.660+0000 7f1afd600640 1 -- 172.21.15.5:0/3962198805 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b04004200 con 0x5557463e6180 2024-04-24T06:53:03.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.660+0000 7f1afd600640 1 -- 172.21.15.5:0/3962198805 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f1b040035f0 con 0x5557463e6180 2024-04-24T06:53:03.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.661+0000 7f1b1d834740 10 client.27247 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:53:03.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.661+0000 7f1b1d834740 10 client.27247 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:53:03.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.661+0000 7f1b1d834740 20 client.27247 populate_metadata read hostname 'smithi005' 2024-04-24T06:53:03.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.661+0000 7f1b1d834740 1 --2- 172.21.15.5:0/3962198805 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x555746466a20 0x555746468e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:03.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.661+0000 7f1b1d834740 4 client.27247 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:53:03.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.661+0000 7f1b1d834740 1 -- 172.21.15.5:0/3962198805 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555746352a80 con 0x555746466a20 2024-04-24T06:53:03.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.661+0000 7f1b13400640 1 --2- 172.21.15.5:0/3962198805 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x555746466a20 0x555746468e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:03.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.661+0000 7f1b13400640 1 --2- 172.21.15.5:0/3962198805 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x555746466a20 0x555746468e10 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79ed06b188f2891e server_cookie=809a284e8f3221a6 in_seq=0 out_seq=0 2024-04-24T06:53:03.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.661+0000 7f1afd600640 10 client.27247 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:53:03.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.662+0000 7f1afd600640 1 -- 172.21.15.5:0/3962198805 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x555746352a80 con 0x555746466a20 2024-04-24T06:53:03.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.662+0000 7f1afd600640 10 client.27247 handle_command_reply: tid=0 2024-04-24T06:53:03.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.662+0000 7f1b1d834740 1 client.27247 shutdown 2024-04-24T06:53:03.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.662+0000 7f1b1d834740 1 -- 172.21.15.5:0/3962198805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557463e6180 msgr2=0x5557463f8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:03.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.662+0000 7f1b1d834740 1 --2- 172.21.15.5:0/3962198805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557463e6180 0x5557463f8cb0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1b04000c00 tx=0x7f1b040095e0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.662+0000 7f1b1d834740 1 -- 172.21.15.5:0/3962198805 shutdown_connections 2024-04-24T06:53:03.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.662+0000 7f1b1d834740 1 --2- 172.21.15.5:0/3962198805 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x555746466a20 0x555746468e10 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.663+0000 7f1b1d834740 1 --2- 172.21.15.5:0/3962198805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557463f0850 0x5557463f3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.663+0000 7f1b1d834740 1 --2- 172.21.15.5:0/3962198805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557463e6180 0x5557463f8cb0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.663+0000 7f1b1d834740 1 --2- 172.21.15.5:0/3962198805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557463e5860 0x5557463f8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.663+0000 7f1b1d834740 1 -- 172.21.15.5:0/3962198805 >> 172.21.15.5:0/3962198805 conn(0x5557463e1b80 msgr2=0x5557463e0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:03.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.663+0000 7f1afd600640 0 client.27247 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:53:03.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.663+0000 7f1b1d834740 1 -- 172.21.15.5:0/3962198805 shutdown_connections 2024-04-24T06:53:03.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.663+0000 7f1b1d834740 1 -- 172.21.15.5:0/3962198805 wait complete. 2024-04-24T06:53:03.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.663+0000 7f1b1d834740 20 client.27247 trim_cache size 0 max 16384 2024-04-24T06:53:03.671 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:53:03.671 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (887 inodes in the stack)", 2024-04-24T06:53:03.672 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-24T06:53:03.672 INFO:teuthology.orchestra.run.smithi005.stdout: "e1da777c-44eb-422f-ab09-99ff986fd7b6": { 2024-04-24T06:53:03.672 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-24T06:53:03.672 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "e1da777c-44eb-422f-ab09-99ff986fd7b6", 2024-04-24T06:53:03.672 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-04-24T06:53:03.672 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:53:03.673 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:53:03.673 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:53:03.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.664+0000 7f1b1aa00640 1 -- 172.21.15.5:0/2083590185 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1af80a3da0 msgr2=0x7f1af80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:03.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.664+0000 7f1b1aa00640 1 --2- 172.21.15.5:0/2083590185 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1af80a3da0 0x7f1af80a6260 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c0a54b0 tx=0x7f1b0c005f70 comp rx=0 tx=0).stop 2024-04-24T06:53:03.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.664+0000 7f1b1aa00640 1 -- 172.21.15.5:0/2083590185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b1419a930 msgr2=0x7f1b1419ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:03.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.664+0000 7f1b1aa00640 1 --2- 172.21.15.5:0/2083590185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b1419a930 0x7f1b1419ade0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c009fd0 tx=0x7f1b0c00ef10 comp rx=0 tx=0).stop 2024-04-24T06:53:03.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.664+0000 7f1b1aa00640 1 -- 172.21.15.5:0/2083590185 shutdown_connections 2024-04-24T06:53:03.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.664+0000 7f1b1aa00640 1 --2- 172.21.15.5:0/2083590185 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1af80a3da0 0x7f1af80a6260 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.664+0000 7f1b1aa00640 1 --2- 172.21.15.5:0/2083590185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b1419a930 0x7f1b1419ade0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.664+0000 7f1b1aa00640 1 --2- 172.21.15.5:0/2083590185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b14107300 0x7f1b141a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.664+0000 7f1b1aa00640 1 --2- 172.21.15.5:0/2083590185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b14102230 0x7f1b141a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:03.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.664+0000 7f1b1aa00640 1 -- 172.21.15.5:0/2083590185 >> 172.21.15.5:0/2083590185 conn(0x7f1b140fdab0 msgr2=0x7f1b140fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:03.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.664+0000 7f1b1aa00640 1 -- 172.21.15.5:0/2083590185 shutdown_connections 2024-04-24T06:53:03.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:03.664+0000 7f1b1aa00640 1 -- 172.21.15.5:0/2083590185 wait complete. 2024-04-24T06:53:03.686 INFO:tasks.cephfs.filesystem:scrub status for tag:e1da777c-44eb-422f-ab09-99ff986fd7b6 - {'path': '/', 'tag': 'e1da777c-44eb-422f-ab09-99ff986fd7b6', 'options': 'recursive,force'} 2024-04-24T06:53:03.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:03 smithi137 ceph-mon[33017]: pgmap v277: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:53:03.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:03 smithi037 ceph-mon[32864]: pgmap v277: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:53:04.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:04 smithi137 ceph-mon[33017]: scrub queued for path: / 2024-04-24T06:53:04.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:04 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:53:04.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:04 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:53:04.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:04 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:53:04.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:04 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:53:04.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:04 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:53:04.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:04 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:53:04.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:04 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:53:04.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:04 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:53:04.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:04 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:53:04.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:04 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:53:04.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:04 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:53:04.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:04 smithi005 ceph-mon[27798]: scrub queued for path: / 2024-04-24T06:53:04.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:04 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:53:04.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:04 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:53:04.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:04 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:53:04.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:04 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:53:04.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:04 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:53:04.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:04 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:53:04.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:04 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:53:04.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:04 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:53:04.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:04 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:53:04.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:04 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:53:04.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:04 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:53:04.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:04 smithi037 ceph-mon[32864]: scrub queued for path: / 2024-04-24T06:53:04.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:04 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:53:04.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:04 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:53:04.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:04 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:53:04.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:04 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:53:04.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:04 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:53:04.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:04 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:53:04.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:04 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:53:04.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:04 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:53:04.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:04 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:53:04.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:04 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:53:04.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:04 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:53:05.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:05 smithi137 ceph-mon[33017]: pgmap v278: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-24T06:53:05.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:05 smithi005 ceph-mon[27798]: pgmap v278: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-24T06:53:05.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:05 smithi037 ceph-mon[32864]: pgmap v278: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-24T06:53:07.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:07 smithi137 ceph-mon[33017]: pgmap v279: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-24T06:53:07.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:07 smithi005 ceph-mon[27798]: pgmap v279: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-24T06:53:07.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:07 smithi037 ceph-mon[32864]: pgmap v279: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-24T06:53:09.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:09 smithi137 ceph-mon[33017]: pgmap v280: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-24T06:53:09.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:09 smithi137 ceph-mon[33017]: osdmap e94: 12 total, 12 up, 12 in 2024-04-24T06:53:09.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:09 smithi005 ceph-mon[27798]: pgmap v280: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-24T06:53:09.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:09 smithi005 ceph-mon[27798]: osdmap e94: 12 total, 12 up, 12 in 2024-04-24T06:53:09.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:09 smithi037 ceph-mon[32864]: pgmap v280: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-24T06:53:09.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:09 smithi037 ceph-mon[32864]: osdmap e94: 12 total, 12 up, 12 in 2024-04-24T06:53:11.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:11 smithi137 ceph-mon[33017]: pgmap v282: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-24T06:53:11.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:11 smithi005 ceph-mon[27798]: pgmap v282: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-24T06:53:11.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:11 smithi037 ceph-mon[32864]: pgmap v282: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-24T06:53:13.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:13 smithi137 ceph-mon[33017]: pgmap v283: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-24T06:53:13.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:13 smithi005 ceph-mon[27798]: pgmap v283: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-24T06:53:13.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:13 smithi037 ceph-mon[32864]: pgmap v283: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-24T06:53:15.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:15 smithi005 ceph-mon[27798]: pgmap v284: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-24T06:53:15.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:15 smithi037 ceph-mon[32864]: pgmap v284: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-24T06:53:15.945 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:15 smithi137 ceph-mon[33017]: pgmap v284: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-24T06:53:17.445 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:17 smithi137 ceph-mon[33017]: osdmap e95: 12 total, 12 up, 12 in 2024-04-24T06:53:17.445 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:17 smithi137 ceph-mon[33017]: pgmap v286: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-24T06:53:17.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:17 smithi005 ceph-mon[27798]: osdmap e95: 12 total, 12 up, 12 in 2024-04-24T06:53:17.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:17 smithi005 ceph-mon[27798]: pgmap v286: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-24T06:53:17.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:17 smithi037 ceph-mon[32864]: osdmap e95: 12 total, 12 up, 12 in 2024-04-24T06:53:17.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:17 smithi037 ceph-mon[32864]: pgmap v286: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-24T06:53:18.445 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:18 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:53:18.445 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:18 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:53:18.446 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:18 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:53:18.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:18 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:53:18.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:18 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:53:18.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:18 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:53:18.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:18 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:53:18.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:18 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:53:18.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:18 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:53:19.445 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:19 smithi137 ceph-mon[33017]: pgmap v287: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 104 B/s wr, 6 op/s 2024-04-24T06:53:19.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:19 smithi005 ceph-mon[27798]: pgmap v287: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 104 B/s wr, 6 op/s 2024-04-24T06:53:19.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:19 smithi037 ceph-mon[32864]: pgmap v287: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 104 B/s wr, 6 op/s 2024-04-24T06:53:21.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:21 smithi137 ceph-mon[33017]: pgmap v288: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-24T06:53:21.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:21 smithi005 ceph-mon[27798]: pgmap v288: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-24T06:53:21.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:21 smithi037 ceph-mon[32864]: pgmap v288: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-24T06:53:23.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:23 smithi137 ceph-mon[33017]: pgmap v289: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-24T06:53:23.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:23 smithi005 ceph-mon[27798]: pgmap v289: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-24T06:53:23.767 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:23 smithi037 ceph-mon[32864]: pgmap v289: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-24T06:53:25.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:25 smithi137 ceph-mon[33017]: pgmap v290: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-24T06:53:25.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:25 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:53:25.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:25 smithi005 ceph-mon[27798]: pgmap v290: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-24T06:53:25.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:25 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:53:25.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:25 smithi037 ceph-mon[32864]: pgmap v290: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-24T06:53:25.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:25 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:53:27.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:27 smithi137 ceph-mon[33017]: pgmap v291: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:53:27.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:27 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:53:27.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:27 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:53:27.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:27 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:53:27.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:27 smithi005 ceph-mon[27798]: pgmap v291: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:53:27.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:27 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:53:27.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:27 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:53:27.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:27 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:53:27.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:27 smithi037 ceph-mon[32864]: pgmap v291: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:53:27.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:27 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:53:27.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:27 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:53:27.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:27 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:53:29.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:29 smithi137 ceph-mon[33017]: pgmap v292: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:53:29.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:29 smithi005 ceph-mon[27798]: pgmap v292: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:53:29.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:29 smithi037 ceph-mon[32864]: pgmap v292: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:53:31.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:31 smithi137 ceph-mon[33017]: pgmap v293: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:53:31.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:31 smithi005 ceph-mon[27798]: pgmap v293: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:53:31.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:31 smithi037 ceph-mon[32864]: pgmap v293: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:53:33.687 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T06:53:33.688 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T06:53:33.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:33 smithi137 ceph-mon[33017]: pgmap v294: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:53:33.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:33 smithi005 ceph-mon[27798]: pgmap v294: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:53:33.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:33 smithi037 ceph-mon[32864]: pgmap v294: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:53:33.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.912+0000 7f24eea00640 1 -- 172.21.15.5:0/3202402246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24e8100fb0 msgr2=0x7f24e81053c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:33.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.912+0000 7f24eea00640 1 --2- 172.21.15.5:0/3202402246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24e8100fb0 0x7f24e81053c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f24d4009860 tx=0x7f24d402a340 comp rx=0 tx=0).stop 2024-04-24T06:53:33.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.912+0000 7f24eea00640 1 -- 172.21.15.5:0/3202402246 shutdown_connections 2024-04-24T06:53:33.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.912+0000 7f24eea00640 1 --2- 172.21.15.5:0/3202402246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24e810b120 0x7f24e810d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:33.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.912+0000 7f24eea00640 1 --2- 172.21.15.5:0/3202402246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24e8100fb0 0x7f24e81053c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:33.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.912+0000 7f24eea00640 1 --2- 172.21.15.5:0/3202402246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24e8068620 0x7f24e8104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:33.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.912+0000 7f24eea00640 1 -- 172.21.15.5:0/3202402246 >> 172.21.15.5:0/3202402246 conn(0x7f24e8075da0 msgr2=0x7f24e80761b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:33.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.912+0000 7f24eea00640 1 -- 172.21.15.5:0/3202402246 shutdown_connections 2024-04-24T06:53:33.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.912+0000 7f24eea00640 1 -- 172.21.15.5:0/3202402246 wait complete. 2024-04-24T06:53:33.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.913+0000 7f24eea00640 1 Processor -- start 2024-04-24T06:53:33.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.914+0000 7f24eea00640 1 -- start start 2024-04-24T06:53:33.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.914+0000 7f24eea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24e8068620 0x7f24e81a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:33.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.914+0000 7f24eea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24e8100fb0 0x7f24e81a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:33.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.914+0000 7f24eea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24e810b120 0x7f24e819a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:33.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.914+0000 7f24eea00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f24e81123c0 con 0x7f24e8068620 2024-04-24T06:53:33.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.914+0000 7f24eea00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f24e8112240 con 0x7f24e810b120 2024-04-24T06:53:33.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.914+0000 7f24eea00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f24e8112540 con 0x7f24e8100fb0 2024-04-24T06:53:33.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.915+0000 7f24e7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24e8068620 0x7f24e81a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:33.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.915+0000 7f24eca00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24e810b120 0x7f24e819a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:33.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.915+0000 7f24eca00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24e810b120 0x7f24e819a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:55172/0 (socket says 172.21.15.5:55172) 2024-04-24T06:53:33.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.915+0000 7f24e7400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24e8100fb0 0x7f24e81a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:33.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.915+0000 7f24eca00640 1 -- 172.21.15.5:0/2858031954 learned_addr learned my addr 172.21.15.5:0/2858031954 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:33.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.915+0000 7f24e7400640 1 -- 172.21.15.5:0/2858031954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24e810b120 msgr2=0x7f24e819a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:33.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.915+0000 7f24e7400640 1 --2- 172.21.15.5:0/2858031954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24e810b120 0x7f24e819a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:33.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.915+0000 7f24e7400640 1 -- 172.21.15.5:0/2858031954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24e8068620 msgr2=0x7f24e81a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:33.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.915+0000 7f24e7400640 1 --2- 172.21.15.5:0/2858031954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24e8068620 0x7f24e81a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:33.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.915+0000 7f24e7400640 1 -- 172.21.15.5:0/2858031954 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e819b060 con 0x7f24e8100fb0 2024-04-24T06:53:33.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.915+0000 7f24eca00640 1 --2- 172.21.15.5:0/2858031954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24e810b120 0x7f24e819a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:53:33.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.915+0000 7f24e7400640 1 --2- 172.21.15.5:0/2858031954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24e8100fb0 0x7f24e81a0c70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f24d402a860 tx=0x7f24d4033b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:33.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.916+0000 7f24e4c00640 1 -- 172.21.15.5:0/2858031954 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d4037040 con 0x7f24e8100fb0 2024-04-24T06:53:33.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.916+0000 7f24e4c00640 1 -- 172.21.15.5:0/2858031954 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24d4033d50 con 0x7f24e8100fb0 2024-04-24T06:53:33.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.916+0000 7f24eea00640 1 -- 172.21.15.5:0/2858031954 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24e819b2f0 con 0x7f24e8100fb0 2024-04-24T06:53:33.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.916+0000 7f24e4c00640 1 -- 172.21.15.5:0/2858031954 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d4003920 con 0x7f24e8100fb0 2024-04-24T06:53:33.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.916+0000 7f24eea00640 1 -- 172.21.15.5:0/2858031954 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24e81a75f0 con 0x7f24e8100fb0 2024-04-24T06:53:33.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.919+0000 7f24f19b6740 1 Processor -- start 2024-04-24T06:53:33.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.920+0000 7f24e4c00640 1 -- 172.21.15.5:0/2858031954 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f24d4042020 con 0x7f24e8100fb0 2024-04-24T06:53:33.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.920+0000 7f24f19b6740 1 -- start start 2024-04-24T06:53:33.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.920+0000 7f24f19b6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a93a4e5860 0x55a93a4e5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:33.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.920+0000 7f24f19b6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a93a4e6180 0x55a93a4f0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:33.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.920+0000 7f24f19b6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a93a4f0850 0x55a93a4f2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:33.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.920+0000 7f24f19b6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a93a44b1f0 con 0x55a93a4e6180 2024-04-24T06:53:33.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.920+0000 7f24f19b6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a93a3f1350 con 0x55a93a4f0850 2024-04-24T06:53:33.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.920+0000 7f24f19b6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a93a4471b0 con 0x55a93a4e5860 2024-04-24T06:53:33.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.920+0000 7f24e7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a93a4e6180 0x55a93a4f0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:33.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.920+0000 7f24e7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a93a4e6180 0x55a93a4f0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34172/0 (socket says 172.21.15.5:34172) 2024-04-24T06:53:33.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.920+0000 7f24e7e00640 1 -- 172.21.15.5:0/2152696288 learned_addr learned my addr 172.21.15.5:0/2152696288 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:33.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.920+0000 7f24e7400640 1 --2- 172.21.15.5:0/2152696288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a93a4f0850 0x55a93a4f2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:33.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.920+0000 7f24eca00640 1 --2- 172.21.15.5:0/2152696288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a93a4e5860 0x55a93a4e5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:33.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.921+0000 7f24e7e00640 1 -- 172.21.15.5:0/2152696288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a93a4e5860 msgr2=0x55a93a4e5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:33.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.921+0000 7f24e7e00640 1 --2- 172.21.15.5:0/2152696288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a93a4e5860 0x55a93a4e5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:33.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.921+0000 7f24e7e00640 1 -- 172.21.15.5:0/2152696288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a93a4f0850 msgr2=0x55a93a4f2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:33.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.921+0000 7f24e7e00640 1 --2- 172.21.15.5:0/2152696288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a93a4f0850 0x55a93a4f2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:33.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.921+0000 7f24e7e00640 1 -- 172.21.15.5:0/2152696288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a93a445f50 con 0x55a93a4e6180 2024-04-24T06:53:33.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.921+0000 7f24e7400640 1 --2- 172.21.15.5:0/2152696288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a93a4f0850 0x55a93a4f2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:53:33.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.921+0000 7f24e7e00640 1 --2- 172.21.15.5:0/2152696288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a93a4e6180 0x55a93a4f0310 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f24d8009c80 tx=0x7f24d8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d71313f5eebe1436 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:33.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.921+0000 7f24e4c00640 1 --2- 172.21.15.5:0/2858031954 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f24cc0a3d20 0x7f24cc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:33.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.921+0000 7f24dd200640 1 -- 172.21.15.5:0/2152696288 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d8004220 con 0x55a93a4e6180 2024-04-24T06:53:33.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.922+0000 7f24dd200640 1 -- 172.21.15.5:0/2152696288 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24d80043c0 con 0x55a93a4e6180 2024-04-24T06:53:33.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.922+0000 7f24dd200640 1 -- 172.21.15.5:0/2152696288 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d8019890 con 0x55a93a4e6180 2024-04-24T06:53:33.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.922+0000 7f24e4c00640 1 -- 172.21.15.5:0/2858031954 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f24d40efbd0 con 0x7f24e8100fb0 2024-04-24T06:53:33.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.922+0000 7f24eca00640 1 --2- 172.21.15.5:0/2858031954 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f24cc0a3d20 0x7f24cc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:33.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.922+0000 7f24f19b6740 1 -- 172.21.15.5:0/2152696288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a93a4e6180 msgr2=0x55a93a4f0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:33.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.922+0000 7f24f19b6740 1 --2- 172.21.15.5:0/2152696288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a93a4e6180 0x55a93a4f0310 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f24d8009c80 tx=0x7f24d8011810 comp rx=0 tx=0).stop 2024-04-24T06:53:33.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.923+0000 7f24f19b6740 1 -- 172.21.15.5:0/2152696288 shutdown_connections 2024-04-24T06:53:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.923+0000 7f24f19b6740 1 --2- 172.21.15.5:0/2152696288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a93a4f0850 0x55a93a4f2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.923+0000 7f24f19b6740 1 --2- 172.21.15.5:0/2152696288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a93a4e6180 0x55a93a4f0310 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.923+0000 7f24f19b6740 1 --2- 172.21.15.5:0/2152696288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a93a4e5860 0x55a93a4e5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.923+0000 7f24f19b6740 1 -- 172.21.15.5:0/2152696288 >> 172.21.15.5:0/2152696288 conn(0x55a93a4e1b80 msgr2=0x55a93a4e1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.923+0000 7f24eca00640 1 --2- 172.21.15.5:0/2858031954 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f24cc0a3d20 0x7f24cc0a61e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f24e000e8f0 tx=0x7f24e0012040 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.923+0000 7f24f19b6740 1 -- 172.21.15.5:0/2152696288 shutdown_connections 2024-04-24T06:53:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.923+0000 7f24f19b6740 1 -- 172.21.15.5:0/2152696288 wait complete. 2024-04-24T06:53:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.923+0000 7f24f19b6740 1 -- start start 2024-04-24T06:53:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.923+0000 7f24f19b6740 1 Processor -- start 2024-04-24T06:53:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.924+0000 7f24f19b6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a93a4e5860 0x55a93a4ed770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.924+0000 7f24f19b6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a93a4e6180 0x55a93a4edcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.924+0000 7f24f19b6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a93a4f0850 0x55a93a4ee1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.924+0000 7f24f19b6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a93a4f3640 con 0x55a93a4e5860 2024-04-24T06:53:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.924+0000 7f24f19b6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a93a404690 con 0x55a93a4f0850 2024-04-24T06:53:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.924+0000 7f24f19b6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a93a4f37c0 con 0x55a93a4e6180 2024-04-24T06:53:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.924+0000 7f24f19b6740 10 client.0 _ll_register_callbacks cb 0x55a93a440650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:53:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.924+0000 7f24e7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a93a4e5860 0x55a93a4ed770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.924+0000 7f24e7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a93a4e5860 0x55a93a4ed770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34174/0 (socket says 172.21.15.5:34174) 2024-04-24T06:53:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.924+0000 7f24e7e00640 1 -- 172.21.15.5:0/3707069636 learned_addr learned my addr 172.21.15.5:0/3707069636 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.924+0000 7f24eca00640 1 --2- 172.21.15.5:0/3707069636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a93a4f0850 0x55a93a4ee1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.924+0000 7f24e7400640 1 --2- 172.21.15.5:0/3707069636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a93a4e6180 0x55a93a4edcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.925+0000 7f24eca00640 1 -- 172.21.15.5:0/3707069636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a93a4e6180 msgr2=0x55a93a4edcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.925+0000 7f24eca00640 1 --2- 172.21.15.5:0/3707069636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a93a4e6180 0x55a93a4edcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.925+0000 7f24eca00640 1 -- 172.21.15.5:0/3707069636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a93a4e5860 msgr2=0x55a93a4ed770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.925+0000 7f24eca00640 1 --2- 172.21.15.5:0/3707069636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a93a4e5860 0x55a93a4ed770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.925+0000 7f24eca00640 1 -- 172.21.15.5:0/3707069636 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a93a403920 con 0x55a93a4f0850 2024-04-24T06:53:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.925+0000 7f24e7e00640 1 --2- 172.21.15.5:0/3707069636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a93a4e5860 0x55a93a4ed770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:53:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.925+0000 7f24eca00640 1 --2- 172.21.15.5:0/3707069636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a93a4f0850 0x55a93a4ee1f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f24e001c650 tx=0x7f24e0004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2e608a12cec4a5b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.925+0000 7f24d1600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:53:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.925+0000 7f24d1600640 1 -- 172.21.15.5:0/3707069636 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e002c060 con 0x55a93a4f0850 2024-04-24T06:53:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.925+0000 7f24f19b6740 1 -- 172.21.15.5:0/3707069636 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24d80043c0 con 0x55a93a4f0850 2024-04-24T06:53:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.925+0000 7f24d1600640 1 -- 172.21.15.5:0/3707069636 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f24e000f480 con 0x55a93a4f0850 2024-04-24T06:53:33.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.926+0000 7f24d1600640 1 -- 172.21.15.5:0/3707069636 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f24e000f680 con 0x55a93a4f0850 2024-04-24T06:53:33.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.926+0000 7f24f19b6740 10 client.27239 fetch_fsmap learned FSMap version 27 2024-04-24T06:53:33.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.926+0000 7f24f19b6740 1 -- 172.21.15.5:0/3707069636 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a93a2277d0 con 0x55a93a4f0850 2024-04-24T06:53:33.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.926+0000 7f24d1600640 1 -- 172.21.15.5:0/3707069636 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24d80043c0 con 0x55a93a4f0850 2024-04-24T06:53:33.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.927+0000 7f24d1600640 1 -- 172.21.15.5:0/3707069636 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f24e0027950 con 0x55a93a4f0850 2024-04-24T06:53:33.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.927+0000 7f24f19b6740 10 client.27239 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:53:33.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.927+0000 7f24f19b6740 10 client.27239 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:53:33.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.927+0000 7f24f19b6740 20 client.27239 populate_metadata read hostname 'smithi005' 2024-04-24T06:53:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.927+0000 7f24f19b6740 1 --2- 172.21.15.5:0/3707069636 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a93a500b90 0x55a93a502fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.927+0000 7f24f19b6740 4 client.27239 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:53:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.927+0000 7f24f19b6740 1 -- 172.21.15.5:0/3707069636 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a93a458c10 con 0x55a93a500b90 2024-04-24T06:53:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.927+0000 7f24e7e00640 1 --2- 172.21.15.5:0/3707069636 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a93a500b90 0x55a93a502fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.928+0000 7f24e7e00640 1 --2- 172.21.15.5:0/3707069636 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a93a500b90 0x55a93a502fc0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20e50af7b6d27715 server_cookie=898d6dcaa212b4ac in_seq=0 out_seq=0 2024-04-24T06:53:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.928+0000 7f24d1600640 10 client.27239 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:53:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.929+0000 7f24d1600640 1 -- 172.21.15.5:0/3707069636 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a93a458c10 con 0x55a93a500b90 2024-04-24T06:53:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.929+0000 7f24d1600640 10 client.27239 handle_command_reply: tid=0 2024-04-24T06:53:33.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.929+0000 7f24f19b6740 1 client.27239 shutdown 2024-04-24T06:53:33.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.929+0000 7f24f19b6740 1 -- 172.21.15.5:0/3707069636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a93a4f0850 msgr2=0x55a93a4ee1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:33.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.929+0000 7f24f19b6740 1 --2- 172.21.15.5:0/3707069636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a93a4f0850 0x55a93a4ee1f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f24e001c650 tx=0x7f24e0004ab0 comp rx=0 tx=0).stop 2024-04-24T06:53:33.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.929+0000 7f24f19b6740 1 -- 172.21.15.5:0/3707069636 shutdown_connections 2024-04-24T06:53:33.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.929+0000 7f24f19b6740 1 --2- 172.21.15.5:0/3707069636 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a93a500b90 0x55a93a502fc0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:33.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.929+0000 7f24f19b6740 1 --2- 172.21.15.5:0/3707069636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a93a4f0850 0x55a93a4ee1f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:33.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.929+0000 7f24f19b6740 1 --2- 172.21.15.5:0/3707069636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a93a4e6180 0x55a93a4edcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:33.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.929+0000 7f24f19b6740 1 --2- 172.21.15.5:0/3707069636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a93a4e5860 0x55a93a4ed770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:33.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.929+0000 7f24d1600640 0 client.27239 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:53:33.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.929+0000 7f24f19b6740 1 -- 172.21.15.5:0/3707069636 >> 172.21.15.5:0/3707069636 conn(0x55a93a4e1b80 msgr2=0x55a93a4e7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:33.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.929+0000 7f24f19b6740 1 -- 172.21.15.5:0/3707069636 shutdown_connections 2024-04-24T06:53:33.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.929+0000 7f24f19b6740 1 -- 172.21.15.5:0/3707069636 wait complete. 2024-04-24T06:53:33.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.929+0000 7f24f19b6740 20 client.27239 trim_cache size 0 max 16384 2024-04-24T06:53:33.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.949+0000 7f24f19b6740 1 Processor -- start 2024-04-24T06:53:33.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.949+0000 7f24f19b6740 1 -- start start 2024-04-24T06:53:33.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.949+0000 7f24f19b6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a93a4e5860 0x55a93a4fc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:33.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.949+0000 7f24f19b6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a93a4e6180 0x55a93a508890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:33.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.949+0000 7f24f19b6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a93a4f0850 0x55a93a508dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:33.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.949+0000 7f24f19b6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f24e81b2830 con 0x55a93a4e5860 2024-04-24T06:53:33.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.949+0000 7f24f19b6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f24e81b0100 con 0x55a93a4f0850 2024-04-24T06:53:33.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.949+0000 7f24f19b6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f24e81adf70 con 0x55a93a4e6180 2024-04-24T06:53:33.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.949+0000 7f24e7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a93a4e5860 0x55a93a4fc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:33.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.949+0000 7f24eca00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a93a4f0850 0x55a93a508dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:33.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.949+0000 7f24e7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a93a4e5860 0x55a93a4fc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34182/0 (socket says 172.21.15.5:34182) 2024-04-24T06:53:33.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.949+0000 7f24eca00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a93a4f0850 0x55a93a508dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:55188/0 (socket says 172.21.15.5:55188) 2024-04-24T06:53:33.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.949+0000 7f24e7e00640 1 -- 172.21.15.5:0/950022242 learned_addr learned my addr 172.21.15.5:0/950022242 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:33.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.950+0000 7f24e7400640 1 --2- 172.21.15.5:0/950022242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a93a4e6180 0x55a93a508890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:33.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.950+0000 7f24eca00640 1 -- 172.21.15.5:0/950022242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a93a4e6180 msgr2=0x55a93a508890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:33.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.950+0000 7f24eca00640 1 --2- 172.21.15.5:0/950022242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a93a4e6180 0x55a93a508890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:33.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.950+0000 7f24eca00640 1 -- 172.21.15.5:0/950022242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a93a4e5860 msgr2=0x55a93a4fc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:33.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.950+0000 7f24eca00640 1 --2- 172.21.15.5:0/950022242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a93a4e5860 0x55a93a4fc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:33.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.950+0000 7f24eca00640 1 -- 172.21.15.5:0/950022242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a93a15d3f0 con 0x55a93a4f0850 2024-04-24T06:53:33.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.950+0000 7f24e7e00640 1 --2- 172.21.15.5:0/950022242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a93a4e5860 0x55a93a4fc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:53:33.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.950+0000 7f24eca00640 1 --2- 172.21.15.5:0/950022242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a93a4f0850 0x55a93a508dd0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f24e0000f80 tx=0x7f24e0009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6345881d46e919b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:33.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.951+0000 7f24d3e00640 1 -- 172.21.15.5:0/950022242 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e000f480 con 0x55a93a4f0850 2024-04-24T06:53:33.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.951+0000 7f24d3e00640 1 -- 172.21.15.5:0/950022242 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24e0003e50 con 0x55a93a4f0850 2024-04-24T06:53:33.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.951+0000 7f24d3e00640 1 -- 172.21.15.5:0/950022242 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e0027620 con 0x55a93a4f0850 2024-04-24T06:53:33.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.951+0000 7f24f19b6740 1 -- 172.21.15.5:0/950022242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a93a4f0850 msgr2=0x55a93a508dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:33.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.951+0000 7f24f19b6740 1 --2- 172.21.15.5:0/950022242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a93a4f0850 0x55a93a508dd0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f24e0000f80 tx=0x7f24e0009580 comp rx=0 tx=0).stop 2024-04-24T06:53:33.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.951+0000 7f24f19b6740 1 -- 172.21.15.5:0/950022242 shutdown_connections 2024-04-24T06:53:33.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.951+0000 7f24f19b6740 1 --2- 172.21.15.5:0/950022242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a93a4f0850 0x55a93a508dd0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:33.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.951+0000 7f24f19b6740 1 --2- 172.21.15.5:0/950022242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a93a4e6180 0x55a93a508890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:33.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.951+0000 7f24f19b6740 1 --2- 172.21.15.5:0/950022242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a93a4e5860 0x55a93a4fc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:33.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.951+0000 7f24f19b6740 1 -- 172.21.15.5:0/950022242 >> 172.21.15.5:0/950022242 conn(0x55a93a4e1b80 msgr2=0x55a93a4e1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:33.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.951+0000 7f24f19b6740 1 -- 172.21.15.5:0/950022242 shutdown_connections 2024-04-24T06:53:33.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.951+0000 7f24f19b6740 1 -- 172.21.15.5:0/950022242 wait complete. 2024-04-24T06:53:33.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.952+0000 7f24f19b6740 1 -- start start 2024-04-24T06:53:33.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.952+0000 7f24f19b6740 1 Processor -- start 2024-04-24T06:53:33.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.952+0000 7f24f19b6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a93a4e5860 0x55a93a4ea660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:33.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.952+0000 7f24f19b6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a93a4e6180 0x55a93a4eaba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:33.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.952+0000 7f24f19b6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a93a4f0850 0x55a93a4eb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:33.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.952+0000 7f24f19b6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f24e81ae5c0 con 0x55a93a4f0850 2024-04-24T06:53:33.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.953+0000 7f24f19b6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f24e81afa30 con 0x55a93a4e6180 2024-04-24T06:53:33.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.953+0000 7f24f19b6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f24e81aec90 con 0x55a93a4e5860 2024-04-24T06:53:33.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.953+0000 7f24f19b6740 10 client.0 _ll_register_callbacks cb 0x7f24e0009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:53:33.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.953+0000 7f24eca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a93a4f0850 0x55a93a4eb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:33.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.953+0000 7f24e7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a93a4e5860 0x55a93a4ea660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:33.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.953+0000 7f24e7400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a93a4e6180 0x55a93a4eaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:33.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.953+0000 7f24e7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a93a4e5860 0x55a93a4ea660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:50552/0 (socket says 172.21.15.5:50552) 2024-04-24T06:53:33.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.953+0000 7f24e7e00640 1 -- 172.21.15.5:0/3228210176 learned_addr learned my addr 172.21.15.5:0/3228210176 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:33.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.953+0000 7f24e7400640 1 -- 172.21.15.5:0/3228210176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a93a4e5860 msgr2=0x55a93a4ea660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:33.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.953+0000 7f24e7400640 1 --2- 172.21.15.5:0/3228210176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a93a4e5860 0x55a93a4ea660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:33.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.953+0000 7f24e7400640 1 -- 172.21.15.5:0/3228210176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a93a4f0850 msgr2=0x55a93a4eb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:33.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.953+0000 7f24e7400640 1 --2- 172.21.15.5:0/3228210176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a93a4f0850 0x55a93a4eb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:33.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.953+0000 7f24e7400640 1 -- 172.21.15.5:0/3228210176 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a93a11d0f0 con 0x55a93a4e6180 2024-04-24T06:53:33.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.953+0000 7f24e7e00640 1 --2- 172.21.15.5:0/3228210176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a93a4e5860 0x55a93a4ea660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:53:33.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.954+0000 7f24e7400640 1 --2- 172.21.15.5:0/3228210176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a93a4e6180 0x55a93a4eaba0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f24d40ed3c0 tx=0x7f24d40f3290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1adf0b01671b5026 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:33.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.954+0000 7f24d1600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:53:33.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.954+0000 7f24d1600640 1 -- 172.21.15.5:0/3228210176 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d40bb750 con 0x55a93a4e6180 2024-04-24T06:53:33.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.954+0000 7f24d1600640 1 -- 172.21.15.5:0/3228210176 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f24d40bbe70 con 0x55a93a4e6180 2024-04-24T06:53:33.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.954+0000 7f24f19b6740 1 -- 172.21.15.5:0/3228210176 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24e0003e50 con 0x55a93a4e6180 2024-04-24T06:53:33.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.954+0000 7f24d1600640 1 -- 172.21.15.5:0/3228210176 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f24d40bb570 con 0x55a93a4e6180 2024-04-24T06:53:33.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.954+0000 7f24f19b6740 10 client.27245 fetch_fsmap learned FSMap version 27 2024-04-24T06:53:33.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.954+0000 7f24f19b6740 1 -- 172.21.15.5:0/3228210176 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a93a410810 con 0x55a93a4e6180 2024-04-24T06:53:33.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.954+0000 7f24d1600640 1 -- 172.21.15.5:0/3228210176 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24e0003e50 con 0x55a93a4e6180 2024-04-24T06:53:33.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.954+0000 7f24d1600640 1 -- 172.21.15.5:0/3228210176 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f24d403a070 con 0x55a93a4e6180 2024-04-24T06:53:33.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.954+0000 7f24f19b6740 10 client.27245 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:53:33.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.954+0000 7f24f19b6740 10 client.27245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:53:33.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.954+0000 7f24f19b6740 20 client.27245 populate_metadata read hostname 'smithi005' 2024-04-24T06:53:33.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.954+0000 7f24f19b6740 1 --2- 172.21.15.5:0/3228210176 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a93a567aa0 0x55a93a569e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:33.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.954+0000 7f24f19b6740 4 client.27245 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:53:33.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.955+0000 7f24f19b6740 1 -- 172.21.15.5:0/3228210176 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a93a452a80 con 0x55a93a567aa0 2024-04-24T06:53:33.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.955+0000 7f24e7e00640 1 --2- 172.21.15.5:0/3228210176 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a93a567aa0 0x55a93a569e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:33.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.955+0000 7f24e7e00640 1 --2- 172.21.15.5:0/3228210176 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a93a567aa0 0x55a93a569e90 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=642f60ddb4450789 server_cookie=4d9efeaecc4e18f in_seq=0 out_seq=0 2024-04-24T06:53:33.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.955+0000 7f24d1600640 10 client.27245 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:53:33.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.956+0000 7f24d1600640 1 -- 172.21.15.5:0/3228210176 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a93a452a80 con 0x55a93a567aa0 2024-04-24T06:53:33.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.956+0000 7f24d1600640 10 client.27245 handle_command_reply: tid=0 2024-04-24T06:53:33.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.956+0000 7f24f19b6740 1 client.27245 shutdown 2024-04-24T06:53:33.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.956+0000 7f24f19b6740 1 -- 172.21.15.5:0/3228210176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a93a4e6180 msgr2=0x55a93a4eaba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:33.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.956+0000 7f24f19b6740 1 --2- 172.21.15.5:0/3228210176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a93a4e6180 0x55a93a4eaba0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f24d40ed3c0 tx=0x7f24d40f3290 comp rx=0 tx=0).stop 2024-04-24T06:53:33.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.956+0000 7f24f19b6740 1 -- 172.21.15.5:0/3228210176 shutdown_connections 2024-04-24T06:53:33.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.956+0000 7f24f19b6740 1 --2- 172.21.15.5:0/3228210176 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a93a567aa0 0x55a93a569e90 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:33.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.956+0000 7f24f19b6740 1 --2- 172.21.15.5:0/3228210176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a93a4f0850 0x55a93a4eb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:33.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.956+0000 7f24f19b6740 1 --2- 172.21.15.5:0/3228210176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a93a4e6180 0x55a93a4eaba0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:33.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.956+0000 7f24f19b6740 1 --2- 172.21.15.5:0/3228210176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a93a4e5860 0x55a93a4ea660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:33.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.956+0000 7f24d1600640 0 client.27245 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:53:33.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.956+0000 7f24f19b6740 1 -- 172.21.15.5:0/3228210176 >> 172.21.15.5:0/3228210176 conn(0x55a93a4e1b80 msgr2=0x55a93a4e6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:33.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.956+0000 7f24f19b6740 1 -- 172.21.15.5:0/3228210176 shutdown_connections 2024-04-24T06:53:33.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.956+0000 7f24f19b6740 1 -- 172.21.15.5:0/3228210176 wait complete. 2024-04-24T06:53:33.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.956+0000 7f24f19b6740 20 client.27245 trim_cache size 0 max 16384 2024-04-24T06:53:33.970 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:53:33.970 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-24T06:53:33.970 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-24T06:53:33.970 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:53:33.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.957+0000 7f24eea00640 1 -- 172.21.15.5:0/2858031954 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f24cc0a3d20 msgr2=0x7f24cc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:33.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.957+0000 7f24eea00640 1 --2- 172.21.15.5:0/2858031954 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f24cc0a3d20 0x7f24cc0a61e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f24e000e8f0 tx=0x7f24e0012040 comp rx=0 tx=0).stop 2024-04-24T06:53:33.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.957+0000 7f24eea00640 1 -- 172.21.15.5:0/2858031954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24e8100fb0 msgr2=0x7f24e81a0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:33.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.957+0000 7f24eea00640 1 --2- 172.21.15.5:0/2858031954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24e8100fb0 0x7f24e81a0c70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f24d402a860 tx=0x7f24d4033b60 comp rx=0 tx=0).stop 2024-04-24T06:53:33.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.957+0000 7f24eea00640 1 -- 172.21.15.5:0/2858031954 shutdown_connections 2024-04-24T06:53:33.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.957+0000 7f24eea00640 1 --2- 172.21.15.5:0/2858031954 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f24cc0a3d20 0x7f24cc0a61e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:33.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.957+0000 7f24eea00640 1 --2- 172.21.15.5:0/2858031954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24e810b120 0x7f24e819a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:33.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.957+0000 7f24eea00640 1 --2- 172.21.15.5:0/2858031954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24e8100fb0 0x7f24e81a0c70 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:33.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.957+0000 7f24eea00640 1 --2- 172.21.15.5:0/2858031954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24e8068620 0x7f24e81a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:33.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.957+0000 7f24eea00640 1 -- 172.21.15.5:0/2858031954 >> 172.21.15.5:0/2858031954 conn(0x7f24e8075da0 msgr2=0x7f24e80fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:33.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.957+0000 7f24eea00640 1 -- 172.21.15.5:0/2858031954 shutdown_connections 2024-04-24T06:53:33.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:33.957+0000 7f24eea00640 1 -- 172.21.15.5:0/2858031954 wait complete. 2024-04-24T06:53:33.981 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T06:53:33.982 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:53:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.195+0000 7fdbff200640 1 -- 172.21.15.5:0/1139091174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc00104350 msgr2=0x7fdc00104730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.195+0000 7fdbff200640 1 --2- 172.21.15.5:0/1139091174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc00104350 0x7fdc00104730 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0009900 tx=0x7fdbf002a3f0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.195+0000 7fdbff200640 1 -- 172.21.15.5:0/1139091174 shutdown_connections 2024-04-24T06:53:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.195+0000 7fdbff200640 1 --2- 172.21.15.5:0/1139091174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc00109420 0x7fdc00110ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.195+0000 7fdbff200640 1 --2- 172.21.15.5:0/1139091174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc00104d00 0x7fdc00108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.195+0000 7fdbff200640 1 --2- 172.21.15.5:0/1139091174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc00104350 0x7fdc00104730 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.195+0000 7fdbff200640 1 -- 172.21.15.5:0/1139091174 >> 172.21.15.5:0/1139091174 conn(0x7fdc000ffb70 msgr2=0x7fdc00101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.196+0000 7fdbff200640 1 -- 172.21.15.5:0/1139091174 shutdown_connections 2024-04-24T06:53:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.196+0000 7fdbff200640 1 -- 172.21.15.5:0/1139091174 wait complete. 2024-04-24T06:53:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.197+0000 7fdbff200640 1 Processor -- start 2024-04-24T06:53:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.197+0000 7fdbff200640 1 -- start start 2024-04-24T06:53:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.197+0000 7fdbff200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc00104350 0x7fdc001a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.197+0000 7fdbff200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc00109420 0x7fdc001a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.198+0000 7fdbff200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc0019a8b0 0x7fdc0019ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.198+0000 7fdbff200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdc001144b0 con 0x7fdc0019a8b0 2024-04-24T06:53:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.198+0000 7fdbff200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdc00114330 con 0x7fdc00109420 2024-04-24T06:53:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.198+0000 7fdbff200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdc00114630 con 0x7fdc00104350 2024-04-24T06:53:34.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.198+0000 7fdbfde00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc00104350 0x7fdc001a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:34.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.198+0000 7fdbfe800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc0019a8b0 0x7fdc0019ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:34.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.198+0000 7fdbfd400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc00109420 0x7fdc001a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:34.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.198+0000 7fdbfde00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc00104350 0x7fdc001a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:50568/0 (socket says 172.21.15.5:50568) 2024-04-24T06:53:34.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.198+0000 7fdbfde00640 1 -- 172.21.15.5:0/4260039415 learned_addr learned my addr 172.21.15.5:0/4260039415 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:34.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.198+0000 7fdbfd400640 1 -- 172.21.15.5:0/4260039415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc00104350 msgr2=0x7fdc001a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:34.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.198+0000 7fdbfd400640 1 --2- 172.21.15.5:0/4260039415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc00104350 0x7fdc001a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.198+0000 7fdbfd400640 1 -- 172.21.15.5:0/4260039415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc0019a8b0 msgr2=0x7fdc0019ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:34.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.198+0000 7fdbfd400640 1 --2- 172.21.15.5:0/4260039415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc0019a8b0 0x7fdc0019ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.198+0000 7fdbfd400640 1 -- 172.21.15.5:0/4260039415 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc0019b440 con 0x7fdc00109420 2024-04-24T06:53:34.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.199+0000 7fdbfd400640 1 --2- 172.21.15.5:0/4260039415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc00109420 0x7fdc001a0be0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fdbe800d6a0 tx=0x7fdbe800db60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:34.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.199+0000 7fdbf6a00640 1 -- 172.21.15.5:0/4260039415 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8017070 con 0x7fdc00109420 2024-04-24T06:53:34.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.199+0000 7fdbf6a00640 1 -- 172.21.15.5:0/4260039415 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbe8009820 con 0x7fdc00109420 2024-04-24T06:53:34.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.199+0000 7fdbf6a00640 1 -- 172.21.15.5:0/4260039415 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8004d00 con 0x7fdc00109420 2024-04-24T06:53:34.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.199+0000 7fdbff200640 1 -- 172.21.15.5:0/4260039415 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc0019b730 con 0x7fdc00109420 2024-04-24T06:53:34.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.199+0000 7fdbff200640 1 -- 172.21.15.5:0/4260039415 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc001a7820 con 0x7fdc00109420 2024-04-24T06:53:34.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.200+0000 7fdbff200640 1 -- 172.21.15.5:0/4260039415 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc00105820 con 0x7fdc00109420 2024-04-24T06:53:34.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.202+0000 7fdbf6a00640 1 -- 172.21.15.5:0/4260039415 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fdbe80099d0 con 0x7fdc00109420 2024-04-24T06:53:34.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.204+0000 7fdbf6a00640 1 --2- 172.21.15.5:0/4260039415 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fdbe40a3d90 0x7fdbe40a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:34.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.204+0000 7fdbf6a00640 1 -- 172.21.15.5:0/4260039415 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7fdbe80d04a0 con 0x7fdc00109420 2024-04-24T06:53:34.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.204+0000 7fdbfde00640 1 --2- 172.21.15.5:0/4260039415 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fdbe40a3d90 0x7fdbe40a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:34.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.205+0000 7fdbfde00640 1 --2- 172.21.15.5:0/4260039415 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fdbe40a3d90 0x7fdbe40a6250 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdbf00096c0 tx=0x7fdbf0005a20 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.206+0000 7fdbf6a00640 1 -- 172.21.15.5:0/4260039415 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdc0019b730 con 0x7fdc00109420 2024-04-24T06:53:34.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.466+0000 7fdbff200640 1 -- 172.21.15.5:0/4260039415 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc0010c650 con 0x7fdc00109420 2024-04-24T06:53:34.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.467+0000 7fdbf6a00640 1 -- 172.21.15.5:0/4260039415 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11856 (secure 0 0 0) 0x7fdbe809c500 con 0x7fdc00109420 2024-04-24T06:53:34.469 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:53:34.470 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1735462750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1735462750},{"type":"v1","addr":"172.21.15.5:6837","nonce":1735462750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1499469370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1499469370},{"type":"v1","addr":"172.21.15.5:6841","nonce":1499469370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3705638847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3705638847},{"type":"v1","addr":"172.21.15.37:6835","nonce":3705638847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T06:41:34.764490+0000","modified":"2024-04-24T06:48:57.768410+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24550,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6837/1390103296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1390103296},{"type":"v1","addr":"172.21.15.37:6837","nonce":1390103296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3217754215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3217754215},{"type":"v1","addr":"172.21.15.137:6835","nonce":3217754215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6837/3044222976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3044222976},{"type":"v1","addr":"172.21.15.137:6837","nonce":3044222976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.5:6839/3173201459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3173201459},{"type":"v1","addr":"172.21.15.5:6839","nonce":3173201459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6839/1889774232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1889774232},{"type":"v1","addr":"172.21.15.137:6839","nonce":1889774232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6839/3536721753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":3536721753},{"type":"v1","addr":"172.21.15.37:6839","nonce":3536721753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24550,24569,24539]},"id":1}]} 2024-04-24T06:53:34.470 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-04-24T06:53:34.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.470+0000 7fdbff200640 1 -- 172.21.15.5:0/4260039415 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fdbe40a3d90 msgr2=0x7fdbe40a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:34.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.471+0000 7fdbff200640 1 --2- 172.21.15.5:0/4260039415 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fdbe40a3d90 0x7fdbe40a6250 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdbf00096c0 tx=0x7fdbf0005a20 comp rx=0 tx=0).stop 2024-04-24T06:53:34.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.471+0000 7fdbff200640 1 -- 172.21.15.5:0/4260039415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc00109420 msgr2=0x7fdc001a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:34.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.471+0000 7fdbff200640 1 --2- 172.21.15.5:0/4260039415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc00109420 0x7fdc001a0be0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fdbe800d6a0 tx=0x7fdbe800db60 comp rx=0 tx=0).stop 2024-04-24T06:53:34.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.471+0000 7fdbff200640 1 -- 172.21.15.5:0/4260039415 shutdown_connections 2024-04-24T06:53:34.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.471+0000 7fdbff200640 1 --2- 172.21.15.5:0/4260039415 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fdbe40a3d90 0x7fdbe40a6250 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.471+0000 7fdbff200640 1 --2- 172.21.15.5:0/4260039415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc0019a8b0 0x7fdc0019ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.471+0000 7fdbff200640 1 --2- 172.21.15.5:0/4260039415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc00109420 0x7fdc001a0be0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.471+0000 7fdbff200640 1 --2- 172.21.15.5:0/4260039415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc00104350 0x7fdc001a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.471+0000 7fdbff200640 1 -- 172.21.15.5:0/4260039415 >> 172.21.15.5:0/4260039415 conn(0x7fdc000ffb70 msgr2=0x7fdc00100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:34.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.471+0000 7fdbff200640 1 -- 172.21.15.5:0/4260039415 shutdown_connections 2024-04-24T06:53:34.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.471+0000 7fdbff200640 1 -- 172.21.15.5:0/4260039415 wait complete. 2024-04-24T06:53:34.494 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T06:53:34.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.713+0000 7fcc26000640 1 -- 172.21.15.5:0/2488033518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc20108ee0 msgr2=0x7fcc201152b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:34.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.713+0000 7fcc26000640 1 --2- 172.21.15.5:0/2488033518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc20108ee0 0x7fcc201152b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fcc18009be0 tx=0x7fcc1802a750 comp rx=0 tx=0).stop 2024-04-24T06:53:34.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.713+0000 7fcc26000640 1 -- 172.21.15.5:0/2488033518 shutdown_connections 2024-04-24T06:53:34.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.713+0000 7fcc26000640 1 --2- 172.21.15.5:0/2488033518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc20108ee0 0x7fcc201152b0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.713+0000 7fcc26000640 1 --2- 172.21.15.5:0/2488033518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc20108540 0x7fcc201089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.713+0000 7fcc26000640 1 --2- 172.21.15.5:0/2488033518 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc2010e4f0 0x7fcc2010e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.713+0000 7fcc26000640 1 -- 172.21.15.5:0/2488033518 >> 172.21.15.5:0/2488033518 conn(0x7fcc20077e10 msgr2=0x7fcc20106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:34.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.713+0000 7fcc26000640 1 -- 172.21.15.5:0/2488033518 shutdown_connections 2024-04-24T06:53:34.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.714+0000 7fcc26000640 1 -- 172.21.15.5:0/2488033518 wait complete. 2024-04-24T06:53:34.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.714+0000 7fcc26000640 1 Processor -- start 2024-04-24T06:53:34.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.715+0000 7fcc26000640 1 -- start start 2024-04-24T06:53:34.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.715+0000 7fcc26000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc20108540 0x7fcc201a4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:34.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.715+0000 7fcc26000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc20108ee0 0x7fcc201a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:34.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.715+0000 7fcc26000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc2010e4f0 0x7fcc2019eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:34.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.715+0000 7fcc26000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcc201187a0 con 0x7fcc20108540 2024-04-24T06:53:34.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.715+0000 7fcc26000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcc20118620 con 0x7fcc2010e4f0 2024-04-24T06:53:34.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.715+0000 7fcc26000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcc20118920 con 0x7fcc20108ee0 2024-04-24T06:53:34.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.715+0000 7fcc24c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc20108540 0x7fcc201a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:34.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.715+0000 7fcc24c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc20108540 0x7fcc201a4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34238/0 (socket says 172.21.15.5:34238) 2024-04-24T06:53:34.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.715+0000 7fcc24c00640 1 -- 172.21.15.5:0/4199596317 learned_addr learned my addr 172.21.15.5:0/4199596317 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:34.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.715+0000 7fcc1fe00640 1 --2- 172.21.15.5:0/4199596317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc20108ee0 0x7fcc201a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:34.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.716+0000 7fcc25600640 1 --2- 172.21.15.5:0/4199596317 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc2010e4f0 0x7fcc2019eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:34.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.716+0000 7fcc24c00640 1 -- 172.21.15.5:0/4199596317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc20108ee0 msgr2=0x7fcc201a4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:34.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.716+0000 7fcc24c00640 1 --2- 172.21.15.5:0/4199596317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc20108ee0 0x7fcc201a4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.716+0000 7fcc24c00640 1 -- 172.21.15.5:0/4199596317 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc2010e4f0 msgr2=0x7fcc2019eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:34.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.716+0000 7fcc24c00640 1 --2- 172.21.15.5:0/4199596317 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc2010e4f0 0x7fcc2019eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.716+0000 7fcc24c00640 1 -- 172.21.15.5:0/4199596317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc2019f280 con 0x7fcc20108540 2024-04-24T06:53:34.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.716+0000 7fcc24c00640 1 --2- 172.21.15.5:0/4199596317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc20108540 0x7fcc201a4950 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fcc1400e9c0 tx=0x7fcc1400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:34.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.716+0000 7fcc1d600640 1 -- 172.21.15.5:0/4199596317 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1400cde0 con 0x7fcc20108540 2024-04-24T06:53:34.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.716+0000 7fcc1d600640 1 -- 172.21.15.5:0/4199596317 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc140035f0 con 0x7fcc20108540 2024-04-24T06:53:34.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.717+0000 7fcc26000640 1 -- 172.21.15.5:0/4199596317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc2019f570 con 0x7fcc20108540 2024-04-24T06:53:34.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.717+0000 7fcc1d600640 1 -- 172.21.15.5:0/4199596317 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc14010690 con 0x7fcc20108540 2024-04-24T06:53:34.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.717+0000 7fcc26000640 1 -- 172.21.15.5:0/4199596317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc201ab810 con 0x7fcc20108540 2024-04-24T06:53:34.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.721+0000 7fcc2a3a9740 1 Processor -- start 2024-04-24T06:53:34.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.721+0000 7fcc2a3a9740 1 -- start start 2024-04-24T06:53:34.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.721+0000 7fcc2a3a9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d29fbb40 0x55b2d29fbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:34.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.722+0000 7fcc2a3a9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b2d29fc460 0x55b2d2a065f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:34.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.722+0000 7fcc2a3a9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2d2a06b30 0x55b2d2a08f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:34.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.722+0000 7fcc2a3a9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b2d29611d0 con 0x55b2d29fbb40 2024-04-24T06:53:34.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.722+0000 7fcc2a3a9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b2d2907450 con 0x55b2d29fc460 2024-04-24T06:53:34.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.722+0000 7fcc2a3a9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b2d295d190 con 0x55b2d2a06b30 2024-04-24T06:53:34.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.722+0000 7fcc1d600640 1 -- 172.21.15.5:0/4199596317 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fcc140037a0 con 0x7fcc20108540 2024-04-24T06:53:34.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.722+0000 7fcc25600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d29fbb40 0x55b2d29fbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:34.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.722+0000 7fcc25600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d29fbb40 0x55b2d29fbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34244/0 (socket says 172.21.15.5:34244) 2024-04-24T06:53:34.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.722+0000 7fcc25600640 1 -- 172.21.15.5:0/720334647 learned_addr learned my addr 172.21.15.5:0/720334647 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:34.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.722+0000 7fcc1fe00640 1 --2- 172.21.15.5:0/720334647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2d2a06b30 0x55b2d2a08f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:34.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.722+0000 7fcc24c00640 1 --2- 172.21.15.5:0/720334647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b2d29fc460 0x55b2d2a065f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:34.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.722+0000 7fcc25600640 1 -- 172.21.15.5:0/720334647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2d2a06b30 msgr2=0x55b2d2a08f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:34.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.722+0000 7fcc25600640 1 --2- 172.21.15.5:0/720334647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2d2a06b30 0x55b2d2a08f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.722+0000 7fcc25600640 1 -- 172.21.15.5:0/720334647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b2d29fc460 msgr2=0x55b2d2a065f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:34.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.723+0000 7fcc25600640 1 --2- 172.21.15.5:0/720334647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b2d29fc460 0x55b2d2a065f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.723+0000 7fcc25600640 1 -- 172.21.15.5:0/720334647 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2d29c4310 con 0x55b2d29fbb40 2024-04-24T06:53:34.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.723+0000 7fcc1fe00640 1 --2- 172.21.15.5:0/720334647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2d2a06b30 0x55b2d2a08f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:53:34.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.723+0000 7fcc24c00640 1 --2- 172.21.15.5:0/720334647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b2d29fc460 0x55b2d2a065f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:53:34.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.723+0000 7fcc25600640 1 --2- 172.21.15.5:0/720334647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d29fbb40 0x55b2d29fbf20 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fcc18009be0 tx=0x7fcc18030410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2548134528b28a7c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:34.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.723+0000 7fcc11c00640 1 -- 172.21.15.5:0/720334647 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc18041030 con 0x55b2d29fbb40 2024-04-24T06:53:34.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.723+0000 7fcc11c00640 1 -- 172.21.15.5:0/720334647 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc180305c0 con 0x55b2d29fbb40 2024-04-24T06:53:34.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.723+0000 7fcc11c00640 1 -- 172.21.15.5:0/720334647 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc18036040 con 0x55b2d29fbb40 2024-04-24T06:53:34.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.724+0000 7fcc2a3a9740 1 -- 172.21.15.5:0/720334647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d29fbb40 msgr2=0x55b2d29fbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:34.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.724+0000 7fcc2a3a9740 1 --2- 172.21.15.5:0/720334647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d29fbb40 0x55b2d29fbf20 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fcc18009be0 tx=0x7fcc18030410 comp rx=0 tx=0).stop 2024-04-24T06:53:34.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.724+0000 7fcc1d600640 1 --2- 172.21.15.5:0/4199596317 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fcc080a3d20 0x7fcc080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:34.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.724+0000 7fcc2a3a9740 1 -- 172.21.15.5:0/720334647 shutdown_connections 2024-04-24T06:53:34.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.724+0000 7fcc2a3a9740 1 --2- 172.21.15.5:0/720334647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2d2a06b30 0x55b2d2a08f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.724+0000 7fcc2a3a9740 1 --2- 172.21.15.5:0/720334647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b2d29fc460 0x55b2d2a065f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.724+0000 7fcc2a3a9740 1 --2- 172.21.15.5:0/720334647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d29fbb40 0x55b2d29fbf20 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.724+0000 7fcc2a3a9740 1 -- 172.21.15.5:0/720334647 >> 172.21.15.5:0/720334647 conn(0x55b2d29f7e60 msgr2=0x55b2d29f8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:34.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.724+0000 7fcc1d600640 1 -- 172.21.15.5:0/4199596317 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7fcc140d0b10 con 0x7fcc20108540 2024-04-24T06:53:34.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.725+0000 7fcc2a3a9740 1 -- 172.21.15.5:0/720334647 shutdown_connections 2024-04-24T06:53:34.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.725+0000 7fcc25600640 1 --2- 172.21.15.5:0/4199596317 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fcc080a3d20 0x7fcc080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:34.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.725+0000 7fcc2a3a9740 1 -- 172.21.15.5:0/720334647 wait complete. 2024-04-24T06:53:34.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.725+0000 7fcc2a3a9740 1 -- start start 2024-04-24T06:53:34.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.725+0000 7fcc25600640 1 --2- 172.21.15.5:0/4199596317 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fcc080a3d20 0x7fcc080a61e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fcc18003c10 tx=0x7fcc1802a750 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:34.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.725+0000 7fcc2a3a9740 1 Processor -- start 2024-04-24T06:53:34.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.726+0000 7fcc2a3a9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2d29fbb40 0x55b2d2a06570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:34.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.726+0000 7fcc2a3a9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b2d29fc460 0x55b2d2a03860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:34.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.726+0000 7fcc2a3a9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d2a06b30 0x55b2d2a03fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:34.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.726+0000 7fcc2a3a9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b2d2a09960 con 0x55b2d2a06b30 2024-04-24T06:53:34.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.726+0000 7fcc2a3a9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b2d291a810 con 0x55b2d29fc460 2024-04-24T06:53:34.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.726+0000 7fcc2a3a9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b2d2a09ae0 con 0x55b2d29fbb40 2024-04-24T06:53:34.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.726+0000 7fcc2a3a9740 10 client.0 _ll_register_callbacks cb 0x55b2d28e8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:53:34.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.726+0000 7fcc25600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d2a06b30 0x55b2d2a03fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:34.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.726+0000 7fcc25600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d2a06b30 0x55b2d2a03fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34252/0 (socket says 172.21.15.5:34252) 2024-04-24T06:53:34.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.726+0000 7fcc24c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2d29fbb40 0x55b2d2a06570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:34.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.726+0000 7fcc25600640 1 -- 172.21.15.5:0/2650494386 learned_addr learned my addr 172.21.15.5:0/2650494386 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:34.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.726+0000 7fcc1fe00640 1 --2- 172.21.15.5:0/2650494386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b2d29fc460 0x55b2d2a03860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:34.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.727+0000 7fcc25600640 1 -- 172.21.15.5:0/2650494386 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2d29fbb40 msgr2=0x55b2d2a06570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:34.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.727+0000 7fcc25600640 1 --2- 172.21.15.5:0/2650494386 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2d29fbb40 0x55b2d2a06570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.727+0000 7fcc25600640 1 -- 172.21.15.5:0/2650494386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b2d29fc460 msgr2=0x55b2d2a03860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:34.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.727+0000 7fcc25600640 1 --2- 172.21.15.5:0/2650494386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b2d29fc460 0x55b2d2a03860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.727+0000 7fcc25600640 1 -- 172.21.15.5:0/2650494386 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2d292b3f0 con 0x55b2d2a06b30 2024-04-24T06:53:34.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.727+0000 7fcc24c00640 1 --2- 172.21.15.5:0/2650494386 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2d29fbb40 0x55b2d2a06570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:53:34.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.727+0000 7fcc25600640 1 --2- 172.21.15.5:0/2650494386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d2a06b30 0x55b2d2a03fa0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fcc18044000 tx=0x7fcc18003c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64866dfe592674c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:34.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.727+0000 7fcc06000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:53:34.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.727+0000 7fcc06000640 1 -- 172.21.15.5:0/2650494386 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc18057070 con 0x55b2d2a06b30 2024-04-24T06:53:34.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.727+0000 7fcc06000640 1 -- 172.21.15.5:0/2650494386 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7fcc18004230 con 0x55b2d2a06b30 2024-04-24T06:53:34.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.727+0000 7fcc2a3a9740 1 -- 172.21.15.5:0/2650494386 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc180305c0 con 0x55b2d2a06b30 2024-04-24T06:53:34.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.728+0000 7fcc06000640 1 -- 172.21.15.5:0/2650494386 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcc18063050 con 0x55b2d2a06b30 2024-04-24T06:53:34.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.728+0000 7fcc2a3a9740 10 client.17115 fetch_fsmap learned FSMap version 27 2024-04-24T06:53:34.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.728+0000 7fcc2a3a9740 1 -- 172.21.15.5:0/2650494386 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b2d26733f0 con 0x55b2d2a06b30 2024-04-24T06:53:34.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.728+0000 7fcc06000640 1 -- 172.21.15.5:0/2650494386 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc180305c0 con 0x55b2d2a06b30 2024-04-24T06:53:34.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.728+0000 7fcc06000640 1 -- 172.21.15.5:0/2650494386 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7fcc18032c00 con 0x55b2d2a06b30 2024-04-24T06:53:34.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.728+0000 7fcc2a3a9740 10 client.17115 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:53:34.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.728+0000 7fcc2a3a9740 10 client.17115 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-24T06:53:34.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.728+0000 7fcc2a3a9740 20 client.17115 populate_metadata read hostname 'smithi005' 2024-04-24T06:53:34.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.728+0000 7fcc2a3a9740 1 --2- 172.21.15.5:0/2650494386 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55b2d2a16b40 0x55b2d2a18f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:34.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.728+0000 7fcc2a3a9740 4 client.17115 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:53:34.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.729+0000 7fcc2a3a9740 1 -- 172.21.15.5:0/2650494386 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2d296ebf0 con 0x55b2d2a16b40 2024-04-24T06:53:34.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.729+0000 7fcc24c00640 1 --2- 172.21.15.5:0/2650494386 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55b2d2a16b40 0x55b2d2a18f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:34.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.729+0000 7fcc24c00640 1 --2- 172.21.15.5:0/2650494386 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55b2d2a16b40 0x55b2d2a18f30 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ddcc7366bc2a84c6 server_cookie=51dd0a89c64b0476 in_seq=0 out_seq=0 2024-04-24T06:53:34.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.729+0000 7fcc06000640 10 client.17115 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:53:34.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.730+0000 7fcc06000640 1 -- 172.21.15.5:0/2650494386 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b2d296ebf0 con 0x55b2d2a16b40 2024-04-24T06:53:34.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.730+0000 7fcc06000640 10 client.17115 handle_command_reply: tid=0 2024-04-24T06:53:34.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.730+0000 7fcc2a3a9740 1 client.17115 shutdown 2024-04-24T06:53:34.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.730+0000 7fcc2a3a9740 1 -- 172.21.15.5:0/2650494386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d2a06b30 msgr2=0x55b2d2a03fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:34.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.730+0000 7fcc2a3a9740 1 --2- 172.21.15.5:0/2650494386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d2a06b30 0x55b2d2a03fa0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fcc18044000 tx=0x7fcc18003c70 comp rx=0 tx=0).stop 2024-04-24T06:53:34.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.730+0000 7fcc2a3a9740 1 -- 172.21.15.5:0/2650494386 shutdown_connections 2024-04-24T06:53:34.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.730+0000 7fcc2a3a9740 1 --2- 172.21.15.5:0/2650494386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d2a06b30 0x55b2d2a03fa0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.730+0000 7fcc2a3a9740 1 --2- 172.21.15.5:0/2650494386 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55b2d2a16b40 0x55b2d2a18f30 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.730+0000 7fcc2a3a9740 1 --2- 172.21.15.5:0/2650494386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b2d29fc460 0x55b2d2a03860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.730+0000 7fcc2a3a9740 1 --2- 172.21.15.5:0/2650494386 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2d29fbb40 0x55b2d2a06570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.730+0000 7fcc2a3a9740 1 -- 172.21.15.5:0/2650494386 >> 172.21.15.5:0/2650494386 conn(0x55b2d29f7e60 msgr2=0x55b2d29fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:34.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.730+0000 7fcc06000640 0 client.17115 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:53:34.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.731+0000 7fcc2a3a9740 1 -- 172.21.15.5:0/2650494386 shutdown_connections 2024-04-24T06:53:34.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.731+0000 7fcc2a3a9740 1 -- 172.21.15.5:0/2650494386 wait complete. 2024-04-24T06:53:34.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.731+0000 7fcc2a3a9740 20 client.17115 trim_cache size 0 max 16384 2024-04-24T06:53:34.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.746+0000 7fcc2a3a9740 1 Processor -- start 2024-04-24T06:53:34.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.746+0000 7fcc2a3a9740 1 -- start start 2024-04-24T06:53:34.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.746+0000 7fcc2a3a9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2d29fbb40 0x55b2d2a12850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:34.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.746+0000 7fcc2a3a9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d29fc460 0x55b2d2a060a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:34.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.746+0000 7fcc2a3a9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b2d2a06b30 0x55b2d2a065e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:34.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.746+0000 7fcc2a3a9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcc201b69d0 con 0x55b2d29fc460 2024-04-24T06:53:34.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.746+0000 7fcc2a3a9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcc201b42a0 con 0x55b2d2a06b30 2024-04-24T06:53:34.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.746+0000 7fcc2a3a9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcc201b2110 con 0x55b2d29fbb40 2024-04-24T06:53:34.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.746+0000 7fcc1fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d29fc460 0x55b2d2a060a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:34.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.746+0000 7fcc24c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2d29fbb40 0x55b2d2a12850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:34.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.746+0000 7fcc1fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d29fc460 0x55b2d2a060a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34254/0 (socket says 172.21.15.5:34254) 2024-04-24T06:53:34.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.747+0000 7fcc1fe00640 1 -- 172.21.15.5:0/3948594797 learned_addr learned my addr 172.21.15.5:0/3948594797 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:34.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.747+0000 7fcc25600640 1 --2- 172.21.15.5:0/3948594797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b2d2a06b30 0x55b2d2a065e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:34.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.747+0000 7fcc24c00640 1 -- 172.21.15.5:0/3948594797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b2d2a06b30 msgr2=0x55b2d2a065e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:34.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.747+0000 7fcc24c00640 1 --2- 172.21.15.5:0/3948594797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b2d2a06b30 0x55b2d2a065e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.747+0000 7fcc24c00640 1 -- 172.21.15.5:0/3948594797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d29fc460 msgr2=0x55b2d2a060a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:34.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.747+0000 7fcc24c00640 1 --2- 172.21.15.5:0/3948594797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d29fc460 0x55b2d2a060a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.747+0000 7fcc24c00640 1 -- 172.21.15.5:0/3948594797 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2d273d7d0 con 0x55b2d29fbb40 2024-04-24T06:53:34.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.747+0000 7fcc24c00640 1 --2- 172.21.15.5:0/3948594797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2d29fbb40 0x55b2d2a12850 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fcc140ce1f0 tx=0x7fcc14000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f178fc78b65568c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:34.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.747+0000 7fcc11200640 1 -- 172.21.15.5:0/3948594797 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1409c3a0 con 0x55b2d29fbb40 2024-04-24T06:53:34.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.748+0000 7fcc11200640 1 -- 172.21.15.5:0/3948594797 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc140d8040 con 0x55b2d29fbb40 2024-04-24T06:53:34.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.748+0000 7fcc11200640 1 -- 172.21.15.5:0/3948594797 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc14003b10 con 0x55b2d29fbb40 2024-04-24T06:53:34.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.748+0000 7fcc2a3a9740 1 -- 172.21.15.5:0/3948594797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2d29fbb40 msgr2=0x55b2d2a12850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:34.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.748+0000 7fcc2a3a9740 1 --2- 172.21.15.5:0/3948594797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2d29fbb40 0x55b2d2a12850 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fcc140ce1f0 tx=0x7fcc14000bc0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.748+0000 7fcc2a3a9740 1 -- 172.21.15.5:0/3948594797 shutdown_connections 2024-04-24T06:53:34.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.748+0000 7fcc2a3a9740 1 --2- 172.21.15.5:0/3948594797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b2d2a06b30 0x55b2d2a065e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.748+0000 7fcc2a3a9740 1 --2- 172.21.15.5:0/3948594797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d29fc460 0x55b2d2a060a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.748+0000 7fcc2a3a9740 1 --2- 172.21.15.5:0/3948594797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2d29fbb40 0x55b2d2a12850 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.748+0000 7fcc2a3a9740 1 -- 172.21.15.5:0/3948594797 >> 172.21.15.5:0/3948594797 conn(0x55b2d29f7e60 msgr2=0x55b2d29f8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:34.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.749+0000 7fcc2a3a9740 1 -- 172.21.15.5:0/3948594797 shutdown_connections 2024-04-24T06:53:34.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.749+0000 7fcc2a3a9740 1 -- 172.21.15.5:0/3948594797 wait complete. 2024-04-24T06:53:34.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.749+0000 7fcc2a3a9740 1 -- start start 2024-04-24T06:53:34.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.749+0000 7fcc2a3a9740 1 Processor -- start 2024-04-24T06:53:34.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.750+0000 7fcc2a3a9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d29fc460 0x55b2d2a0f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:34.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.750+0000 7fcc2a3a9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2d2a06b30 0x55b2d2a00e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:34.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.750+0000 7fcc2a3a9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b2d2a013a0 0x55b2d2a07970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:34.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.750+0000 7fcc2a3a9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcc201b2760 con 0x55b2d29fc460 2024-04-24T06:53:34.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.750+0000 7fcc2a3a9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcc201b3bd0 con 0x55b2d2a013a0 2024-04-24T06:53:34.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.750+0000 7fcc2a3a9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcc201b2e30 con 0x55b2d2a06b30 2024-04-24T06:53:34.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.750+0000 7fcc2a3a9740 10 client.0 _ll_register_callbacks cb 0x7fcc1804a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:53:34.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.750+0000 7fcc24c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d29fc460 0x55b2d2a0f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:34.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.750+0000 7fcc24c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d29fc460 0x55b2d2a0f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34268/0 (socket says 172.21.15.5:34268) 2024-04-24T06:53:34.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.750+0000 7fcc24c00640 1 -- 172.21.15.5:0/1131049959 learned_addr learned my addr 172.21.15.5:0/1131049959 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:34.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.750+0000 7fcc1fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2d2a06b30 0x55b2d2a00e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:34.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.750+0000 7fcc25600640 1 --2- 172.21.15.5:0/1131049959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b2d2a013a0 0x55b2d2a07970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:34.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.750+0000 7fcc24c00640 1 -- 172.21.15.5:0/1131049959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2d2a06b30 msgr2=0x55b2d2a00e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:34.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.750+0000 7fcc24c00640 1 --2- 172.21.15.5:0/1131049959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2d2a06b30 0x55b2d2a00e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.750+0000 7fcc24c00640 1 -- 172.21.15.5:0/1131049959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b2d2a013a0 msgr2=0x55b2d2a07970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:34.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.750+0000 7fcc24c00640 1 --2- 172.21.15.5:0/1131049959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b2d2a013a0 0x55b2d2a07970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.750+0000 7fcc24c00640 1 -- 172.21.15.5:0/1131049959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2d2a07f30 con 0x55b2d29fc460 2024-04-24T06:53:34.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.751+0000 7fcc24c00640 1 --2- 172.21.15.5:0/1131049959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d29fc460 0x55b2d2a0f240 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fcc1400c6a0 tx=0x7fcc1409ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e764e77585a35f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:34.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.751+0000 7fcc06000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:53:34.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.751+0000 7fcc06000640 1 -- 172.21.15.5:0/1131049959 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc140b5020 con 0x55b2d29fc460 2024-04-24T06:53:34.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.751+0000 7fcc06000640 1 -- 172.21.15.5:0/1131049959 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7fcc14003b10 con 0x55b2d29fc460 2024-04-24T06:53:34.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.751+0000 7fcc2a3a9740 1 -- 172.21.15.5:0/1131049959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc140d8040 con 0x55b2d29fc460 2024-04-24T06:53:34.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.751+0000 7fcc06000640 1 -- 172.21.15.5:0/1131049959 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcc140a56b0 con 0x55b2d29fc460 2024-04-24T06:53:34.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.751+0000 7fcc2a3a9740 10 client.17121 fetch_fsmap learned FSMap version 27 2024-04-24T06:53:34.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.751+0000 7fcc2a3a9740 1 -- 172.21.15.5:0/1131049959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b2d2a7d650 con 0x55b2d29fc460 2024-04-24T06:53:34.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.751+0000 7fcc06000640 1 -- 172.21.15.5:0/1131049959 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc140d8040 con 0x55b2d29fc460 2024-04-24T06:53:34.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.752+0000 7fcc06000640 1 -- 172.21.15.5:0/1131049959 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7fcc14098790 con 0x55b2d29fc460 2024-04-24T06:53:34.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.752+0000 7fcc2a3a9740 10 client.17121 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:53:34.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.752+0000 7fcc2a3a9740 10 client.17121 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-24T06:53:34.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.752+0000 7fcc2a3a9740 20 client.17121 populate_metadata read hostname 'smithi005' 2024-04-24T06:53:34.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.752+0000 7fcc2a3a9740 1 --2- 172.21.15.5:0/1131049959 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55b2d2a7dcc0 0x55b2d2a800e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:34.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.752+0000 7fcc2a3a9740 4 client.17121 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:53:34.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.752+0000 7fcc2a3a9740 1 -- 172.21.15.5:0/1131049959 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b2d2968a60 con 0x55b2d2a7dcc0 2024-04-24T06:53:34.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.752+0000 7fcc1fe00640 1 --2- 172.21.15.5:0/1131049959 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55b2d2a7dcc0 0x55b2d2a800e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:34.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.752+0000 7fcc1fe00640 1 --2- 172.21.15.5:0/1131049959 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55b2d2a7dcc0 0x55b2d2a800e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dca32d234747a992 server_cookie=2f634124fd07779a in_seq=0 out_seq=0 2024-04-24T06:53:34.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.753+0000 7fcc06000640 10 client.17121 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:53:34.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.753+0000 7fcc06000640 1 -- 172.21.15.5:0/1131049959 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b2d2968a60 con 0x55b2d2a7dcc0 2024-04-24T06:53:34.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.753+0000 7fcc06000640 10 client.17121 handle_command_reply: tid=0 2024-04-24T06:53:34.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.753+0000 7fcc2a3a9740 1 client.17121 shutdown 2024-04-24T06:53:34.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.753+0000 7fcc2a3a9740 1 -- 172.21.15.5:0/1131049959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d29fc460 msgr2=0x55b2d2a0f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:34.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.753+0000 7fcc2a3a9740 1 --2- 172.21.15.5:0/1131049959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d29fc460 0x55b2d2a0f240 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fcc1400c6a0 tx=0x7fcc1409ca40 comp rx=0 tx=0).stop 2024-04-24T06:53:34.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.753+0000 7fcc2a3a9740 1 -- 172.21.15.5:0/1131049959 shutdown_connections 2024-04-24T06:53:34.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.753+0000 7fcc2a3a9740 1 --2- 172.21.15.5:0/1131049959 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x55b2d2a7dcc0 0x55b2d2a800e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.753+0000 7fcc2a3a9740 1 --2- 172.21.15.5:0/1131049959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b2d2a013a0 0x55b2d2a07970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.753+0000 7fcc2a3a9740 1 --2- 172.21.15.5:0/1131049959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2d2a06b30 0x55b2d2a00e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.753+0000 7fcc2a3a9740 1 --2- 172.21.15.5:0/1131049959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2d29fc460 0x55b2d2a0f240 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.753+0000 7fcc06000640 0 client.17121 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:53:34.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.753+0000 7fcc2a3a9740 1 -- 172.21.15.5:0/1131049959 >> 172.21.15.5:0/1131049959 conn(0x55b2d29f7e60 msgr2=0x55b2d29f66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:34.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.753+0000 7fcc2a3a9740 1 -- 172.21.15.5:0/1131049959 shutdown_connections 2024-04-24T06:53:34.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.753+0000 7fcc2a3a9740 1 -- 172.21.15.5:0/1131049959 wait complete. 2024-04-24T06:53:34.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.753+0000 7fcc2a3a9740 20 client.17121 trim_cache size 0 max 16384 2024-04-24T06:53:34.763 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:53:34.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.754+0000 7fcc26000640 1 -- 172.21.15.5:0/4199596317 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fcc080a3d20 msgr2=0x7fcc080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:34.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.754+0000 7fcc26000640 1 --2- 172.21.15.5:0/4199596317 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fcc080a3d20 0x7fcc080a61e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fcc18003c10 tx=0x7fcc1802a750 comp rx=0 tx=0).stop 2024-04-24T06:53:34.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.754+0000 7fcc26000640 1 -- 172.21.15.5:0/4199596317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc20108540 msgr2=0x7fcc201a4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:34.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.754+0000 7fcc26000640 1 --2- 172.21.15.5:0/4199596317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc20108540 0x7fcc201a4950 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fcc1400e9c0 tx=0x7fcc1400ee80 comp rx=0 tx=0).stop 2024-04-24T06:53:34.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.754+0000 7fcc26000640 1 -- 172.21.15.5:0/4199596317 shutdown_connections 2024-04-24T06:53:34.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.754+0000 7fcc26000640 1 --2- 172.21.15.5:0/4199596317 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fcc080a3d20 0x7fcc080a61e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.754+0000 7fcc26000640 1 --2- 172.21.15.5:0/4199596317 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc2010e4f0 0x7fcc2019eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.754+0000 7fcc26000640 1 --2- 172.21.15.5:0/4199596317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc20108ee0 0x7fcc201a4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.754+0000 7fcc26000640 1 --2- 172.21.15.5:0/4199596317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc20108540 0x7fcc201a4950 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.754+0000 7fcc26000640 1 -- 172.21.15.5:0/4199596317 >> 172.21.15.5:0/4199596317 conn(0x7fcc20077e10 msgr2=0x7fcc20078490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:34.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.754+0000 7fcc26000640 1 -- 172.21.15.5:0/4199596317 shutdown_connections 2024-04-24T06:53:34.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.755+0000 7fcc26000640 1 -- 172.21.15.5:0/4199596317 wait complete. 2024-04-24T06:53:34.775 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T06:53:34.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.984+0000 7fed15000640 1 -- 172.21.15.5:0/2170404020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed10076530 msgr2=0x7fed10076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:34.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.984+0000 7fed15000640 1 --2- 172.21.15.5:0/2170404020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed10076530 0x7fed10076910 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fed00009900 tx=0x7fed0002a410 comp rx=0 tx=0).stop 2024-04-24T06:53:34.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.984+0000 7fed15000640 1 -- 172.21.15.5:0/2170404020 shutdown_connections 2024-04-24T06:53:34.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.984+0000 7fed15000640 1 --2- 172.21.15.5:0/2170404020 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed10112c90 0x7fed10115080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.984+0000 7fed15000640 1 --2- 172.21.15.5:0/2170404020 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed10076e50 0x7fed10112750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.984+0000 7fed15000640 1 --2- 172.21.15.5:0/2170404020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed10076530 0x7fed10076910 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.984+0000 7fed15000640 1 -- 172.21.15.5:0/2170404020 >> 172.21.15.5:0/2170404020 conn(0x7fed100ffaa0 msgr2=0x7fed10101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:34.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.984+0000 7fed15000640 1 -- 172.21.15.5:0/2170404020 shutdown_connections 2024-04-24T06:53:34.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.984+0000 7fed15000640 1 -- 172.21.15.5:0/2170404020 wait complete. 2024-04-24T06:53:34.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.985+0000 7fed15000640 1 Processor -- start 2024-04-24T06:53:34.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.985+0000 7fed15000640 1 -- start start 2024-04-24T06:53:34.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.986+0000 7fed15000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed10076530 0x7fed1019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:34.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.986+0000 7fed15000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed10076e50 0x7fed1019f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:34.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.986+0000 7fed15000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed10112c90 0x7fed101a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:34.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.986+0000 7fed15000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fed10118850 con 0x7fed10076e50 2024-04-24T06:53:34.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.986+0000 7fed15000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fed101186d0 con 0x7fed10112c90 2024-04-24T06:53:34.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.986+0000 7fed15000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fed101189d0 con 0x7fed10076530 2024-04-24T06:53:34.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.986+0000 7fed07e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed10076e50 0x7fed1019f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:34.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.986+0000 7fed07e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed10076e50 0x7fed1019f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34288/0 (socket says 172.21.15.5:34288) 2024-04-24T06:53:34.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.986+0000 7fed07e00640 1 -- 172.21.15.5:0/1103456120 learned_addr learned my addr 172.21.15.5:0/1103456120 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:34.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.986+0000 7fed0fe00640 1 --2- 172.21.15.5:0/1103456120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed10112c90 0x7fed101a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:34.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.986+0000 7fed0f400640 1 --2- 172.21.15.5:0/1103456120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed10076530 0x7fed1019ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:34.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.987+0000 7fed07e00640 1 -- 172.21.15.5:0/1103456120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed10076530 msgr2=0x7fed1019ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:34.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.987+0000 7fed07e00640 1 --2- 172.21.15.5:0/1103456120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed10076530 0x7fed1019ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.987+0000 7fed07e00640 1 -- 172.21.15.5:0/1103456120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed10112c90 msgr2=0x7fed101a3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:34.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.987+0000 7fed07e00640 1 --2- 172.21.15.5:0/1103456120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed10112c90 0x7fed101a3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.987+0000 7fed07e00640 1 -- 172.21.15.5:0/1103456120 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed101a3d80 con 0x7fed10076e50 2024-04-24T06:53:34.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.987+0000 7fed07e00640 1 --2- 172.21.15.5:0/1103456120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed10076e50 0x7fed1019f2d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fecf800e9f0 tx=0x7fecf800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:34.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.987+0000 7fed0cc00640 1 -- 172.21.15.5:0/1103456120 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf800cd80 con 0x7fed10076e50 2024-04-24T06:53:34.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.987+0000 7fed0cc00640 1 -- 172.21.15.5:0/1103456120 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecf8005500 con 0x7fed10076e50 2024-04-24T06:53:34.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.987+0000 7fed0cc00640 1 -- 172.21.15.5:0/1103456120 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf8010690 con 0x7fed10076e50 2024-04-24T06:53:34.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.987+0000 7fed15000640 1 -- 172.21.15.5:0/1103456120 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed101a4070 con 0x7fed10076e50 2024-04-24T06:53:34.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.988+0000 7fed15000640 1 -- 172.21.15.5:0/1103456120 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed101ab900 con 0x7fed10076e50 2024-04-24T06:53:34.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.991+0000 7fed1939c740 1 Processor -- start 2024-04-24T06:53:34.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.991+0000 7fed0cc00640 1 -- 172.21.15.5:0/1103456120 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fecf80046c0 con 0x7fed10076e50 2024-04-24T06:53:34.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.991+0000 7fed1939c740 1 -- start start 2024-04-24T06:53:34.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.991+0000 7fed1939c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a740e21b40 0x55a740e21f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:34.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.991+0000 7fed1939c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a740e22460 0x55a740e2c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:34.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.991+0000 7fed1939c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a740e2cb30 0x55a740e2ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:34.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.991+0000 7fed1939c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a740d871d0 con 0x55a740e2cb30 2024-04-24T06:53:34.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.991+0000 7fed1939c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a740d2d450 con 0x55a740e21b40 2024-04-24T06:53:34.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.991+0000 7fed1939c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a740d83190 con 0x55a740e22460 2024-04-24T06:53:34.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.992+0000 7fed07e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a740e2cb30 0x55a740e2ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:34.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.992+0000 7fed0f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a740e22460 0x55a740e2c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:34.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.992+0000 7fed0fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a740e21b40 0x55a740e21f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:34.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.992+0000 7fed0f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a740e22460 0x55a740e2c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:50654/0 (socket says 172.21.15.5:50654) 2024-04-24T06:53:34.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.992+0000 7fed0f400640 1 -- 172.21.15.5:0/2057510721 learned_addr learned my addr 172.21.15.5:0/2057510721 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:34.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.992+0000 7fed0f400640 1 -- 172.21.15.5:0/2057510721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a740e21b40 msgr2=0x55a740e21f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:34.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.992+0000 7fed0f400640 1 --2- 172.21.15.5:0/2057510721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a740e21b40 0x55a740e21f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.992+0000 7fed0f400640 1 -- 172.21.15.5:0/2057510721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a740e2cb30 msgr2=0x55a740e2ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:34.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.992+0000 7fed0f400640 1 --2- 172.21.15.5:0/2057510721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a740e2cb30 0x55a740e2ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:34.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.992+0000 7fed0f400640 1 -- 172.21.15.5:0/2057510721 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a740dea310 con 0x55a740e22460 2024-04-24T06:53:34.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.992+0000 7fed0fe00640 1 --2- 172.21.15.5:0/2057510721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a740e21b40 0x55a740e21f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:53:34.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.992+0000 7fed0f400640 1 --2- 172.21.15.5:0/2057510721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a740e22460 0x55a740e2c5f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fed00009230 tx=0x7fed00005e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6b3f0db84474e42 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:34.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.993+0000 7fed0cc00640 1 --2- 172.21.15.5:0/1103456120 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fecf40a3d20 0x7fecf40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:34.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.993+0000 7fed04c00640 1 -- 172.21.15.5:0/2057510721 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed00035040 con 0x55a740e22460 2024-04-24T06:53:34.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.993+0000 7fed04c00640 1 -- 172.21.15.5:0/2057510721 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed00004400 con 0x55a740e22460 2024-04-24T06:53:34.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.993+0000 7fed04c00640 1 -- 172.21.15.5:0/2057510721 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed000035f0 con 0x55a740e22460 2024-04-24T06:53:34.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.993+0000 7fed0cc00640 1 -- 172.21.15.5:0/1103456120 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7fecf80d0c00 con 0x7fed10076e50 2024-04-24T06:53:34.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.993+0000 7fed0fe00640 1 --2- 172.21.15.5:0/1103456120 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fecf40a3d20 0x7fecf40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:35.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.994+0000 7fed1939c740 1 -- 172.21.15.5:0/2057510721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a740e22460 msgr2=0x55a740e2c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:35.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.994+0000 7fed1939c740 1 --2- 172.21.15.5:0/2057510721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a740e22460 0x55a740e2c5f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fed00009230 tx=0x7fed00005e00 comp rx=0 tx=0).stop 2024-04-24T06:53:35.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.994+0000 7fed1939c740 1 -- 172.21.15.5:0/2057510721 shutdown_connections 2024-04-24T06:53:35.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.994+0000 7fed1939c740 1 --2- 172.21.15.5:0/2057510721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a740e2cb30 0x55a740e2ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.994+0000 7fed1939c740 1 --2- 172.21.15.5:0/2057510721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a740e22460 0x55a740e2c5f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.994+0000 7fed1939c740 1 --2- 172.21.15.5:0/2057510721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a740e21b40 0x55a740e21f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.994+0000 7fed1939c740 1 -- 172.21.15.5:0/2057510721 >> 172.21.15.5:0/2057510721 conn(0x55a740e1de60 msgr2=0x55a740e1e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:35.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.994+0000 7fed0fe00640 1 --2- 172.21.15.5:0/1103456120 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fecf40a3d20 0x7fecf40a61e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fed08009fd0 tx=0x7fed08009500 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:35.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.994+0000 7fed1939c740 1 -- 172.21.15.5:0/2057510721 shutdown_connections 2024-04-24T06:53:35.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.994+0000 7fed1939c740 1 -- 172.21.15.5:0/2057510721 wait complete. 2024-04-24T06:53:35.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.994+0000 7fed1939c740 1 -- start start 2024-04-24T06:53:35.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.995+0000 7fed1939c740 1 Processor -- start 2024-04-24T06:53:35.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.995+0000 7fed1939c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a740e21b40 0x55a740e2c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:35.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.995+0000 7fed1939c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a740e22460 0x55a740e297f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:35.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.995+0000 7fed1939c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a740e2cb30 0x55a740e29e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:35.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.995+0000 7fed1939c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a740e2f960 con 0x55a740e21b40 2024-04-24T06:53:35.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.995+0000 7fed1939c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a740d40810 con 0x55a740e2cb30 2024-04-24T06:53:35.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.995+0000 7fed1939c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a740e2fae0 con 0x55a740e22460 2024-04-24T06:53:35.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.995+0000 7fed1939c740 10 client.0 _ll_register_callbacks cb 0x55a740d0e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:53:35.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.995+0000 7fed07e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a740e22460 0x55a740e297f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:35.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.996+0000 7fed07e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a740e22460 0x55a740e297f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:50662/0 (socket says 172.21.15.5:50662) 2024-04-24T06:53:35.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.996+0000 7fed07e00640 1 -- 172.21.15.5:0/3104011725 learned_addr learned my addr 172.21.15.5:0/3104011725 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:35.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.996+0000 7fed0f400640 1 --2- 172.21.15.5:0/3104011725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a740e21b40 0x55a740e2c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:35.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.996+0000 7fed0fe00640 1 --2- 172.21.15.5:0/3104011725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a740e2cb30 0x55a740e29e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:35.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.996+0000 7fed07e00640 1 -- 172.21.15.5:0/3104011725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a740e2cb30 msgr2=0x55a740e29e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:35.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.996+0000 7fed07e00640 1 --2- 172.21.15.5:0/3104011725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a740e2cb30 0x55a740e29e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.996+0000 7fed07e00640 1 -- 172.21.15.5:0/3104011725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a740e21b40 msgr2=0x55a740e2c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:35.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.996+0000 7fed07e00640 1 --2- 172.21.15.5:0/3104011725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a740e21b40 0x55a740e2c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.996+0000 7fed07e00640 1 -- 172.21.15.5:0/3104011725 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a740d513f0 con 0x55a740e22460 2024-04-24T06:53:35.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.996+0000 7fed0f400640 1 --2- 172.21.15.5:0/3104011725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a740e21b40 0x55a740e2c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:53:35.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.996+0000 7fed07e00640 1 --2- 172.21.15.5:0/3104011725 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a740e22460 0x55a740e297f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fecf80ce3a0 tx=0x7fecf8009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c50224fe8b9047a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:35.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.996+0000 7fecfd600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:53:35.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.997+0000 7fecfd600640 1 -- 172.21.15.5:0/3104011725 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf809c6d0 con 0x55a740e22460 2024-04-24T06:53:35.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.997+0000 7fecfd600640 1 -- 172.21.15.5:0/3104011725 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7fecf80ce5c0 con 0x55a740e22460 2024-04-24T06:53:35.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.997+0000 7fed1939c740 1 -- 172.21.15.5:0/3104011725 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed00004400 con 0x55a740e22460 2024-04-24T06:53:35.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.997+0000 7fecfd600640 1 -- 172.21.15.5:0/3104011725 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fecf809c4f0 con 0x55a740e22460 2024-04-24T06:53:35.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.997+0000 7fed1939c740 10 client.27301 fetch_fsmap learned FSMap version 27 2024-04-24T06:53:35.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.997+0000 7fed1939c740 1 -- 172.21.15.5:0/3104011725 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a740a993f0 con 0x55a740e22460 2024-04-24T06:53:35.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.997+0000 7fecfd600640 1 -- 172.21.15.5:0/3104011725 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed00004400 con 0x55a740e22460 2024-04-24T06:53:35.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.997+0000 7fecfd600640 1 -- 172.21.15.5:0/3104011725 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7fecf80a5370 con 0x55a740e22460 2024-04-24T06:53:35.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.997+0000 7fed1939c740 10 client.27301 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:53:35.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.997+0000 7fed1939c740 10 client.27301 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T06:53:35.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.997+0000 7fed1939c740 20 client.27301 populate_metadata read hostname 'smithi005' 2024-04-24T06:53:35.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.997+0000 7fed1939c740 1 --2- 172.21.15.5:0/3104011725 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55a740e3cd00 0x55a740e3f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:35.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.998+0000 7fed1939c740 4 client.27301 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:53:35.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.998+0000 7fed1939c740 1 -- 172.21.15.5:0/3104011725 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a740d94bf0 con 0x55a740e3cd00 2024-04-24T06:53:35.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.998+0000 7fed0f400640 1 --2- 172.21.15.5:0/3104011725 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55a740e3cd00 0x55a740e3f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:35.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.998+0000 7fed0f400640 1 --2- 172.21.15.5:0/3104011725 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55a740e3cd00 0x55a740e3f110 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d6785d1f16753471 server_cookie=ac7fc7d940368ddb in_seq=0 out_seq=0 2024-04-24T06:53:35.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.998+0000 7fecfd600640 10 client.27301 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:53:35.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.999+0000 7fecfd600640 1 -- 172.21.15.5:0/3104011725 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a740d94bf0 con 0x55a740e3cd00 2024-04-24T06:53:35.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.999+0000 7fecfd600640 10 client.27301 handle_command_reply: tid=0 2024-04-24T06:53:35.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:34.999+0000 7fed1939c740 1 client.27301 shutdown 2024-04-24T06:53:35.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.000+0000 7fed1939c740 1 -- 172.21.15.5:0/3104011725 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a740e22460 msgr2=0x55a740e297f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:35.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.000+0000 7fed1939c740 1 --2- 172.21.15.5:0/3104011725 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a740e22460 0x55a740e297f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fecf80ce3a0 tx=0x7fecf8009310 comp rx=0 tx=0).stop 2024-04-24T06:53:35.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.000+0000 7fed1939c740 1 -- 172.21.15.5:0/3104011725 shutdown_connections 2024-04-24T06:53:35.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.000+0000 7fed1939c740 1 --2- 172.21.15.5:0/3104011725 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55a740e3cd00 0x55a740e3f110 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.000+0000 7fed1939c740 1 --2- 172.21.15.5:0/3104011725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a740e2cb30 0x55a740e29e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.000+0000 7fed1939c740 1 --2- 172.21.15.5:0/3104011725 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a740e22460 0x55a740e297f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.000+0000 7fed1939c740 1 --2- 172.21.15.5:0/3104011725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a740e21b40 0x55a740e2c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.000+0000 7fed1939c740 1 -- 172.21.15.5:0/3104011725 >> 172.21.15.5:0/3104011725 conn(0x55a740e1de60 msgr2=0x55a740e23c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:35.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.000+0000 7fecfd600640 0 client.27301 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:53:35.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.000+0000 7fed1939c740 1 -- 172.21.15.5:0/3104011725 shutdown_connections 2024-04-24T06:53:35.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.000+0000 7fed1939c740 1 -- 172.21.15.5:0/3104011725 wait complete. 2024-04-24T06:53:35.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.000+0000 7fed1939c740 20 client.27301 trim_cache size 0 max 16384 2024-04-24T06:53:35.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.015+0000 7fed1939c740 1 Processor -- start 2024-04-24T06:53:35.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.015+0000 7fed1939c740 1 -- start start 2024-04-24T06:53:35.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.015+0000 7fed1939c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a740e21b40 0x55a740e38850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:35.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.015+0000 7fed1939c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a740e22460 0x55a740e2c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:35.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.015+0000 7fed1939c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a740e2cb30 0x55a740e44a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:35.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.015+0000 7fed1939c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fed101b6b00 con 0x55a740e2cb30 2024-04-24T06:53:35.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.015+0000 7fed1939c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fed101b43d0 con 0x55a740e22460 2024-04-24T06:53:35.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.015+0000 7fed1939c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fed101b2240 con 0x55a740e21b40 2024-04-24T06:53:35.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.016+0000 7fed0fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a740e2cb30 0x55a740e44a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:35.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.016+0000 7fed0fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a740e2cb30 0x55a740e44a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34310/0 (socket says 172.21.15.5:34310) 2024-04-24T06:53:35.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.016+0000 7fed0fe00640 1 -- 172.21.15.5:0/593883649 learned_addr learned my addr 172.21.15.5:0/593883649 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:35.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.016+0000 7fed0f400640 1 --2- 172.21.15.5:0/593883649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a740e21b40 0x55a740e38850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:35.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.016+0000 7fed07e00640 1 --2- 172.21.15.5:0/593883649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a740e22460 0x55a740e2c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:35.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.016+0000 7fed0f400640 1 -- 172.21.15.5:0/593883649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a740e22460 msgr2=0x55a740e2c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:35.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.016+0000 7fed0f400640 1 --2- 172.21.15.5:0/593883649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a740e22460 0x55a740e2c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.016+0000 7fed0f400640 1 -- 172.21.15.5:0/593883649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a740e2cb30 msgr2=0x55a740e44a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:35.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.016+0000 7fed0f400640 1 --2- 172.21.15.5:0/593883649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a740e2cb30 0x55a740e44a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.016+0000 7fed0f400640 1 -- 172.21.15.5:0/593883649 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a740b637d0 con 0x55a740e21b40 2024-04-24T06:53:35.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.016+0000 7fed0f400640 1 --2- 172.21.15.5:0/593883649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a740e21b40 0x55a740e38850 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fed00009900 tx=0x7fed00004bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b273cb7ed6a12b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:35.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.017+0000 7fecffe00640 1 -- 172.21.15.5:0/593883649 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed00035040 con 0x55a740e21b40 2024-04-24T06:53:35.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.017+0000 7fecffe00640 1 -- 172.21.15.5:0/593883649 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed000054e0 con 0x55a740e21b40 2024-04-24T06:53:35.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.017+0000 7fecffe00640 1 -- 172.21.15.5:0/593883649 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed00054430 con 0x55a740e21b40 2024-04-24T06:53:35.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.017+0000 7fed1939c740 1 -- 172.21.15.5:0/593883649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a740e21b40 msgr2=0x55a740e38850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:35.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.017+0000 7fed1939c740 1 --2- 172.21.15.5:0/593883649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a740e21b40 0x55a740e38850 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fed00009900 tx=0x7fed00004bd0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.017+0000 7fed1939c740 1 -- 172.21.15.5:0/593883649 shutdown_connections 2024-04-24T06:53:35.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.017+0000 7fed1939c740 1 --2- 172.21.15.5:0/593883649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a740e2cb30 0x55a740e44a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.017+0000 7fed1939c740 1 --2- 172.21.15.5:0/593883649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a740e22460 0x55a740e2c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.017+0000 7fed1939c740 1 --2- 172.21.15.5:0/593883649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a740e21b40 0x55a740e38850 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.017+0000 7fed1939c740 1 -- 172.21.15.5:0/593883649 >> 172.21.15.5:0/593883649 conn(0x55a740e1de60 msgr2=0x55a740e24b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:35.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.018+0000 7fed1939c740 1 -- 172.21.15.5:0/593883649 shutdown_connections 2024-04-24T06:53:35.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.018+0000 7fed1939c740 1 -- 172.21.15.5:0/593883649 wait complete. 2024-04-24T06:53:35.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.018+0000 7fed1939c740 1 -- start start 2024-04-24T06:53:35.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.018+0000 7fed1939c740 1 Processor -- start 2024-04-24T06:53:35.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.018+0000 7fed1939c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a740e21b40 0x55a740e2d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:35.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.018+0000 7fed1939c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a740e22460 0x55a740e2d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:35.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.018+0000 7fed1939c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a740e2cb30 0x55a740e33d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:35.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.019+0000 7fed1939c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fed101b2890 con 0x55a740e21b40 2024-04-24T06:53:35.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.019+0000 7fed1939c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fed101b3d00 con 0x55a740e2cb30 2024-04-24T06:53:35.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.019+0000 7fed1939c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fed101b2f60 con 0x55a740e22460 2024-04-24T06:53:35.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.019+0000 7fed1939c740 10 client.0 _ll_register_callbacks cb 0x7fecf80d4430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:53:35.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.019+0000 7fed0f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a740e21b40 0x55a740e2d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:35.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.019+0000 7fed07e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a740e22460 0x55a740e2d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:35.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.019+0000 7fed0f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a740e21b40 0x55a740e2d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34324/0 (socket says 172.21.15.5:34324) 2024-04-24T06:53:35.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.019+0000 7fed0f400640 1 -- 172.21.15.5:0/270878949 learned_addr learned my addr 172.21.15.5:0/270878949 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:35.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.019+0000 7fed0fe00640 1 --2- 172.21.15.5:0/270878949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a740e2cb30 0x55a740e33d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:35.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.019+0000 7fed07e00640 1 -- 172.21.15.5:0/270878949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a740e2cb30 msgr2=0x55a740e33d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:35.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.019+0000 7fed07e00640 1 --2- 172.21.15.5:0/270878949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a740e2cb30 0x55a740e33d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.019+0000 7fed07e00640 1 -- 172.21.15.5:0/270878949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a740e21b40 msgr2=0x55a740e2d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:35.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.019+0000 7fed07e00640 1 --2- 172.21.15.5:0/270878949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a740e21b40 0x55a740e2d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.019+0000 7fed07e00640 1 -- 172.21.15.5:0/270878949 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a740e342a0 con 0x55a740e22460 2024-04-24T06:53:35.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.019+0000 7fed0f400640 1 --2- 172.21.15.5:0/270878949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a740e21b40 0x55a740e2d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:53:35.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.020+0000 7fed07e00640 1 --2- 172.21.15.5:0/270878949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a740e22460 0x55a740e2d820 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fecf8009380 tx=0x7fecf8005e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acc103a7b4ac834f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:35.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.020+0000 7fecfd600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:53:35.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.020+0000 7fecfd600640 1 -- 172.21.15.5:0/270878949 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf809c6d0 con 0x55a740e22460 2024-04-24T06:53:35.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.020+0000 7fecfd600640 1 -- 172.21.15.5:0/270878949 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7fecf80988e0 con 0x55a740e22460 2024-04-24T06:53:35.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.020+0000 7fed1939c740 1 -- 172.21.15.5:0/270878949 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed000054e0 con 0x55a740e22460 2024-04-24T06:53:35.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.020+0000 7fecfd600640 1 -- 172.21.15.5:0/270878949 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fecf80ce7c0 con 0x55a740e22460 2024-04-24T06:53:35.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.020+0000 7fed1939c740 10 client.27307 fetch_fsmap learned FSMap version 27 2024-04-24T06:53:35.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.020+0000 7fed1939c740 1 -- 172.21.15.5:0/270878949 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a740e354c0 con 0x55a740e22460 2024-04-24T06:53:35.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.020+0000 7fecfd600640 1 -- 172.21.15.5:0/270878949 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed000054e0 con 0x55a740e22460 2024-04-24T06:53:35.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.020+0000 7fecfd600640 1 -- 172.21.15.5:0/270878949 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7fecf8098ae0 con 0x55a740e22460 2024-04-24T06:53:35.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.021+0000 7fed1939c740 10 client.27307 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:53:35.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.021+0000 7fed1939c740 10 client.27307 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T06:53:35.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.021+0000 7fed1939c740 20 client.27307 populate_metadata read hostname 'smithi005' 2024-04-24T06:53:35.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.021+0000 7fed1939c740 1 --2- 172.21.15.5:0/270878949 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55a740ea3650 0x55a740ea5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:35.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.021+0000 7fed1939c740 4 client.27307 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:53:35.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.021+0000 7fed1939c740 1 -- 172.21.15.5:0/270878949 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a740d8ea60 con 0x55a740ea3650 2024-04-24T06:53:35.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.021+0000 7fed0f400640 1 --2- 172.21.15.5:0/270878949 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55a740ea3650 0x55a740ea5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:35.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.021+0000 7fed0f400640 1 --2- 172.21.15.5:0/270878949 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55a740ea3650 0x55a740ea5a70 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a196ac5ecfd404 server_cookie=53eb43d925bb0467 in_seq=0 out_seq=0 2024-04-24T06:53:35.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.021+0000 7fecfd600640 10 client.27307 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:53:35.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.022+0000 7fecfd600640 1 -- 172.21.15.5:0/270878949 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a740d8ea60 con 0x55a740ea3650 2024-04-24T06:53:35.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.022+0000 7fecfd600640 10 client.27307 handle_command_reply: tid=0 2024-04-24T06:53:35.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.022+0000 7fed1939c740 1 client.27307 shutdown 2024-04-24T06:53:35.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.022+0000 7fed1939c740 1 -- 172.21.15.5:0/270878949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a740e22460 msgr2=0x55a740e2d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:35.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.022+0000 7fed1939c740 1 --2- 172.21.15.5:0/270878949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a740e22460 0x55a740e2d820 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fecf8009380 tx=0x7fecf8005e30 comp rx=0 tx=0).stop 2024-04-24T06:53:35.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.022+0000 7fed1939c740 1 -- 172.21.15.5:0/270878949 shutdown_connections 2024-04-24T06:53:35.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.022+0000 7fed1939c740 1 --2- 172.21.15.5:0/270878949 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55a740ea3650 0x55a740ea5a70 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.022+0000 7fed1939c740 1 --2- 172.21.15.5:0/270878949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a740e2cb30 0x55a740e33d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.022+0000 7fed1939c740 1 --2- 172.21.15.5:0/270878949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a740e22460 0x55a740e2d820 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.022+0000 7fed1939c740 1 --2- 172.21.15.5:0/270878949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a740e21b40 0x55a740e2d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.022+0000 7fecfd600640 0 client.27307 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:53:35.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.022+0000 7fed1939c740 1 -- 172.21.15.5:0/270878949 >> 172.21.15.5:0/270878949 conn(0x55a740e1de60 msgr2=0x55a740e28080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:35.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.022+0000 7fed1939c740 1 -- 172.21.15.5:0/270878949 shutdown_connections 2024-04-24T06:53:35.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.022+0000 7fed1939c740 1 -- 172.21.15.5:0/270878949 wait complete. 2024-04-24T06:53:35.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.022+0000 7fed1939c740 20 client.27307 trim_cache size 0 max 16384 2024-04-24T06:53:35.031 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:53:35.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.023+0000 7fed15000640 1 -- 172.21.15.5:0/1103456120 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fecf40a3d20 msgr2=0x7fecf40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:35.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.023+0000 7fed15000640 1 --2- 172.21.15.5:0/1103456120 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fecf40a3d20 0x7fecf40a61e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fed08009fd0 tx=0x7fed08009500 comp rx=0 tx=0).stop 2024-04-24T06:53:35.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.023+0000 7fed15000640 1 -- 172.21.15.5:0/1103456120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed10076e50 msgr2=0x7fed1019f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:35.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.023+0000 7fed15000640 1 --2- 172.21.15.5:0/1103456120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed10076e50 0x7fed1019f2d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fecf800e9f0 tx=0x7fecf800edf0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.023+0000 7fed15000640 1 -- 172.21.15.5:0/1103456120 shutdown_connections 2024-04-24T06:53:35.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.023+0000 7fed15000640 1 --2- 172.21.15.5:0/1103456120 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fecf40a3d20 0x7fecf40a61e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.023+0000 7fed15000640 1 --2- 172.21.15.5:0/1103456120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed10112c90 0x7fed101a3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.023+0000 7fed15000640 1 --2- 172.21.15.5:0/1103456120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed10076e50 0x7fed1019f2d0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.023+0000 7fed15000640 1 --2- 172.21.15.5:0/1103456120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed10076530 0x7fed1019ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.023+0000 7fed15000640 1 -- 172.21.15.5:0/1103456120 >> 172.21.15.5:0/1103456120 conn(0x7fed100ffaa0 msgr2=0x7fed10114340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:35.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.023+0000 7fed15000640 1 -- 172.21.15.5:0/1103456120 shutdown_connections 2024-04-24T06:53:35.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.023+0000 7fed15000640 1 -- 172.21.15.5:0/1103456120 wait complete. 2024-04-24T06:53:35.042 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T06:53:35.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.243+0000 7f8949400640 1 -- 172.21.15.5:0/4149370060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89440743a0 msgr2=0x7f8944074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:35.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.243+0000 7f8949400640 1 --2- 172.21.15.5:0/4149370060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89440743a0 0x7f8944074780 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8934009be0 tx=0x7f893402a750 comp rx=0 tx=0).stop 2024-04-24T06:53:35.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.243+0000 7f8949400640 1 -- 172.21.15.5:0/4149370060 shutdown_connections 2024-04-24T06:53:35.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.243+0000 7f8949400640 1 --2- 172.21.15.5:0/4149370060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8944112e10 0x7f8944115200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.243+0000 7f8949400640 1 --2- 172.21.15.5:0/4149370060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8944074cc0 0x7f89441128d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.243+0000 7f8949400640 1 --2- 172.21.15.5:0/4149370060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89440743a0 0x7f8944074780 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.243+0000 7f8949400640 1 -- 172.21.15.5:0/4149370060 >> 172.21.15.5:0/4149370060 conn(0x7f89440fd950 msgr2=0x7f89440ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:35.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.244+0000 7f8949400640 1 -- 172.21.15.5:0/4149370060 shutdown_connections 2024-04-24T06:53:35.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.244+0000 7f8949400640 1 -- 172.21.15.5:0/4149370060 wait complete. 2024-04-24T06:53:35.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.245+0000 7f8949400640 1 Processor -- start 2024-04-24T06:53:35.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.245+0000 7f8949400640 1 -- start start 2024-04-24T06:53:35.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.246+0000 7f8949400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89440743a0 0x7f894419ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:35.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.246+0000 7f8949400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8944074cc0 0x7f894419f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:35.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.246+0000 7f8949400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8944112e10 0x7f89441a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:35.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.246+0000 7f8949400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f89441187e0 con 0x7f89440743a0 2024-04-24T06:53:35.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.246+0000 7f8949400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8944118660 con 0x7f8944074cc0 2024-04-24T06:53:35.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.246+0000 7f8949400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8944118960 con 0x7f8944112e10 2024-04-24T06:53:35.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.246+0000 7f8943e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89440743a0 0x7f894419ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:35.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.246+0000 7f8943e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89440743a0 0x7f894419ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34344/0 (socket says 172.21.15.5:34344) 2024-04-24T06:53:35.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.246+0000 7f8943e00640 1 -- 172.21.15.5:0/2202159629 learned_addr learned my addr 172.21.15.5:0/2202159629 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:35.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.246+0000 7f893b400640 1 --2- 172.21.15.5:0/2202159629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8944074cc0 0x7f894419f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:35.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.246+0000 7f8948a00640 1 --2- 172.21.15.5:0/2202159629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8944112e10 0x7f89441a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:35.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.247+0000 7f8943e00640 1 -- 172.21.15.5:0/2202159629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8944112e10 msgr2=0x7f89441a36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:35.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.247+0000 7f8943e00640 1 --2- 172.21.15.5:0/2202159629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8944112e10 0x7f89441a36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.247+0000 7f8943e00640 1 -- 172.21.15.5:0/2202159629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8944074cc0 msgr2=0x7f894419f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:35.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.247+0000 7f8943e00640 1 --2- 172.21.15.5:0/2202159629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8944074cc0 0x7f894419f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.247+0000 7f8943e00640 1 -- 172.21.15.5:0/2202159629 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89441a3df0 con 0x7f89440743a0 2024-04-24T06:53:35.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.247+0000 7f893b400640 1 --2- 172.21.15.5:0/2202159629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8944074cc0 0x7f894419f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:53:35.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.247+0000 7f8943e00640 1 --2- 172.21.15.5:0/2202159629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89440743a0 0x7f894419ee00 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8934009d00 tx=0x7f89340305d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:35.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.247+0000 7f8941600640 1 -- 172.21.15.5:0/2202159629 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8934041020 con 0x7f89440743a0 2024-04-24T06:53:35.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.247+0000 7f8941600640 1 -- 172.21.15.5:0/2202159629 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8934003e50 con 0x7f89440743a0 2024-04-24T06:53:35.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.248+0000 7f8949400640 1 -- 172.21.15.5:0/2202159629 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89441a4080 con 0x7f89440743a0 2024-04-24T06:53:35.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.248+0000 7f8941600640 1 -- 172.21.15.5:0/2202159629 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8934036040 con 0x7f89440743a0 2024-04-24T06:53:35.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.248+0000 7f8949400640 1 -- 172.21.15.5:0/2202159629 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89441ab970 con 0x7f89440743a0 2024-04-24T06:53:35.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.251+0000 7f894d9c9740 1 Processor -- start 2024-04-24T06:53:35.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.251+0000 7f8941600640 1 -- 172.21.15.5:0/2202159629 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f8934044020 con 0x7f89440743a0 2024-04-24T06:53:35.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.251+0000 7f894d9c9740 1 -- start start 2024-04-24T06:53:35.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.251+0000 7f894d9c9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd4027bb40 0x55fd4027bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:35.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.252+0000 7f894d9c9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd4027c460 0x55fd402865f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:35.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.252+0000 7f894d9c9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd40286b30 0x55fd40288f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:35.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.252+0000 7f894d9c9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fd401e11d0 con 0x55fd4027c460 2024-04-24T06:53:35.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.252+0000 7f894d9c9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55fd40187450 con 0x55fd40286b30 2024-04-24T06:53:35.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.252+0000 7f894d9c9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fd401dd190 con 0x55fd4027bb40 2024-04-24T06:53:35.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.252+0000 7f8943e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd4027c460 0x55fd402865f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:35.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.252+0000 7f8948a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd4027bb40 0x55fd4027bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:35.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.252+0000 7f893b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd40286b30 0x55fd40288f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:35.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.252+0000 7f8948a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd4027bb40 0x55fd4027bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:50712/0 (socket says 172.21.15.5:50712) 2024-04-24T06:53:35.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.252+0000 7f8948a00640 1 -- 172.21.15.5:0/3831277290 learned_addr learned my addr 172.21.15.5:0/3831277290 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:35.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.252+0000 7f893b400640 1 -- 172.21.15.5:0/3831277290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd4027bb40 msgr2=0x55fd4027bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:35.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.252+0000 7f893b400640 1 --2- 172.21.15.5:0/3831277290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd4027bb40 0x55fd4027bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.252+0000 7f893b400640 1 -- 172.21.15.5:0/3831277290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd4027c460 msgr2=0x55fd402865f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:35.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.252+0000 7f893b400640 1 --2- 172.21.15.5:0/3831277290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd4027c460 0x55fd402865f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.252+0000 7f893b400640 1 -- 172.21.15.5:0/3831277290 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd40244310 con 0x55fd40286b30 2024-04-24T06:53:35.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.253+0000 7f8948a00640 1 --2- 172.21.15.5:0/3831277290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd4027bb40 0x55fd4027bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:53:35.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.253+0000 7f8941600640 1 --2- 172.21.15.5:0/2202159629 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f89280a3d60 0x7f89280a6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:35.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.253+0000 7f893b400640 1 --2- 172.21.15.5:0/3831277290 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd40286b30 0x55fd40288f20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f892c009fa0 tx=0x7f892c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9c54082430b0e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:35.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.253+0000 7f8941600640 1 -- 172.21.15.5:0/2202159629 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f89340f1370 con 0x7f89440743a0 2024-04-24T06:53:35.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.253+0000 7f8948a00640 1 --2- 172.21.15.5:0/2202159629 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f89280a3d60 0x7f89280a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:35.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.253+0000 7f8939200640 1 -- 172.21.15.5:0/3831277290 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892c004030 con 0x55fd40286b30 2024-04-24T06:53:35.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.253+0000 7f8939200640 1 -- 172.21.15.5:0/3831277290 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f892c005500 con 0x55fd40286b30 2024-04-24T06:53:35.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.253+0000 7f8939200640 1 -- 172.21.15.5:0/3831277290 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892c004e10 con 0x55fd40286b30 2024-04-24T06:53:35.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.254+0000 7f894d9c9740 1 -- 172.21.15.5:0/3831277290 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd40286b30 msgr2=0x55fd40288f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:35.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.254+0000 7f894d9c9740 1 --2- 172.21.15.5:0/3831277290 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd40286b30 0x55fd40288f20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f892c009fa0 tx=0x7f892c0118d0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.254+0000 7f8948a00640 1 --2- 172.21.15.5:0/2202159629 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f89280a3d60 0x7f89280a6220 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f893c011440 tx=0x7f893c009260 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:35.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.254+0000 7f894d9c9740 1 -- 172.21.15.5:0/3831277290 shutdown_connections 2024-04-24T06:53:35.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.254+0000 7f894d9c9740 1 --2- 172.21.15.5:0/3831277290 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd40286b30 0x55fd40288f20 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.254+0000 7f894d9c9740 1 --2- 172.21.15.5:0/3831277290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd4027c460 0x55fd402865f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.254+0000 7f894d9c9740 1 --2- 172.21.15.5:0/3831277290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd4027bb40 0x55fd4027bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.254+0000 7f894d9c9740 1 -- 172.21.15.5:0/3831277290 >> 172.21.15.5:0/3831277290 conn(0x55fd40277e60 msgr2=0x55fd40278240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:35.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.254+0000 7f894d9c9740 1 -- 172.21.15.5:0/3831277290 shutdown_connections 2024-04-24T06:53:35.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.254+0000 7f894d9c9740 1 -- 172.21.15.5:0/3831277290 wait complete. 2024-04-24T06:53:35.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.255+0000 7f894d9c9740 1 -- start start 2024-04-24T06:53:35.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.255+0000 7f894d9c9740 1 Processor -- start 2024-04-24T06:53:35.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.255+0000 7f894d9c9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd4027bb40 0x55fd402864e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:35.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.255+0000 7f894d9c9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd4027c460 0x55fd402837d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:35.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.255+0000 7f894d9c9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd40286b30 0x55fd40283d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:35.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.255+0000 7f894d9c9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fd402899c0 con 0x55fd40286b30 2024-04-24T06:53:35.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.256+0000 7f894d9c9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55fd4019a810 con 0x55fd4027c460 2024-04-24T06:53:35.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.256+0000 7f894d9c9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fd40289b40 con 0x55fd4027bb40 2024-04-24T06:53:35.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.256+0000 7f894d9c9740 10 client.0 _ll_register_callbacks cb 0x55fd40168140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:53:35.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.256+0000 7f8948a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd40286b30 0x55fd40283d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:35.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.256+0000 7f8943e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd4027bb40 0x55fd402864e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:35.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.256+0000 7f8948a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd40286b30 0x55fd40283d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34374/0 (socket says 172.21.15.5:34374) 2024-04-24T06:53:35.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.256+0000 7f8948a00640 1 -- 172.21.15.5:0/3950782582 learned_addr learned my addr 172.21.15.5:0/3950782582 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:35.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.256+0000 7f893b400640 1 --2- 172.21.15.5:0/3950782582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd4027c460 0x55fd402837d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:35.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.256+0000 7f8943e00640 1 -- 172.21.15.5:0/3950782582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd4027c460 msgr2=0x55fd402837d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:35.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.256+0000 7f8943e00640 1 --2- 172.21.15.5:0/3950782582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd4027c460 0x55fd402837d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.256+0000 7f8943e00640 1 -- 172.21.15.5:0/3950782582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd40286b30 msgr2=0x55fd40283d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:35.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.256+0000 7f8943e00640 1 --2- 172.21.15.5:0/3950782582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd40286b30 0x55fd40283d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.256+0000 7f8943e00640 1 -- 172.21.15.5:0/3950782582 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd401ab3f0 con 0x55fd4027bb40 2024-04-24T06:53:35.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.256+0000 7f8948a00640 1 --2- 172.21.15.5:0/3950782582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd40286b30 0x55fd40283d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:53:35.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.257+0000 7f8943e00640 1 --2- 172.21.15.5:0/3950782582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd4027bb40 0x55fd402864e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f89340eea50 tx=0x7f893402a750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44a07621f2d8e61 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:35.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.257+0000 7f8931600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:53:35.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.257+0000 7f8931600640 1 -- 172.21.15.5:0/3950782582 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89340bcde0 con 0x55fd4027bb40 2024-04-24T06:53:35.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.257+0000 7f8931600640 1 -- 172.21.15.5:0/3950782582 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f89340eecd0 con 0x55fd4027bb40 2024-04-24T06:53:35.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.257+0000 7f894d9c9740 1 -- 172.21.15.5:0/3950782582 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f892c005500 con 0x55fd4027bb40 2024-04-24T06:53:35.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.257+0000 7f8931600640 1 -- 172.21.15.5:0/3950782582 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f89340bcc00 con 0x55fd4027bb40 2024-04-24T06:53:35.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.257+0000 7f894d9c9740 10 client.27319 fetch_fsmap learned FSMap version 27 2024-04-24T06:53:35.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.257+0000 7f894d9c9740 1 -- 172.21.15.5:0/3950782582 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fd3fef33f0 con 0x55fd4027bb40 2024-04-24T06:53:35.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.257+0000 7f8931600640 1 -- 172.21.15.5:0/3950782582 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f892c005500 con 0x55fd4027bb40 2024-04-24T06:53:35.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.258+0000 7f8931600640 1 -- 172.21.15.5:0/3950782582 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f89340c5a80 con 0x55fd4027bb40 2024-04-24T06:53:35.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.258+0000 7f894d9c9740 10 client.27319 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:53:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.258+0000 7f894d9c9740 10 client.27319 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:53:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.258+0000 7f894d9c9740 20 client.27319 populate_metadata read hostname 'smithi005' 2024-04-24T06:53:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.258+0000 7f894d9c9740 1 --2- 172.21.15.5:0/3950782582 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55fd40296b50 0x55fd40298f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.258+0000 7f894d9c9740 4 client.27319 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:53:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.258+0000 7f894d9c9740 1 -- 172.21.15.5:0/3950782582 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd401eebf0 con 0x55fd40296b50 2024-04-24T06:53:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.258+0000 7f893b400640 1 --2- 172.21.15.5:0/3950782582 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55fd40296b50 0x55fd40298f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.259+0000 7f893b400640 1 --2- 172.21.15.5:0/3950782582 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55fd40296b50 0x55fd40298f40 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22a6c2e6d9f0f0f8 server_cookie=3dd657ec8ef4743d in_seq=0 out_seq=0 2024-04-24T06:53:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.259+0000 7f8931600640 10 client.27319 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:53:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.260+0000 7f8931600640 1 -- 172.21.15.5:0/3950782582 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fd401eebf0 con 0x55fd40296b50 2024-04-24T06:53:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.260+0000 7f8931600640 10 client.27319 handle_command_reply: tid=0 2024-04-24T06:53:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.260+0000 7f894d9c9740 1 client.27319 shutdown 2024-04-24T06:53:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.260+0000 7f894d9c9740 1 -- 172.21.15.5:0/3950782582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd4027bb40 msgr2=0x55fd402864e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.260+0000 7f894d9c9740 1 --2- 172.21.15.5:0/3950782582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd4027bb40 0x55fd402864e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f89340eea50 tx=0x7f893402a750 comp rx=0 tx=0).stop 2024-04-24T06:53:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.260+0000 7f894d9c9740 1 -- 172.21.15.5:0/3950782582 shutdown_connections 2024-04-24T06:53:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.260+0000 7f894d9c9740 1 --2- 172.21.15.5:0/3950782582 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55fd40296b50 0x55fd40298f40 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.260+0000 7f894d9c9740 1 --2- 172.21.15.5:0/3950782582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd40286b30 0x55fd40283d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.260+0000 7f894d9c9740 1 --2- 172.21.15.5:0/3950782582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd4027c460 0x55fd402837d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.260+0000 7f894d9c9740 1 --2- 172.21.15.5:0/3950782582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd4027bb40 0x55fd402864e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.260+0000 7f894d9c9740 1 -- 172.21.15.5:0/3950782582 >> 172.21.15.5:0/3950782582 conn(0x55fd40277e60 msgr2=0x55fd4027db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.260+0000 7f8931600640 0 client.27319 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:53:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.260+0000 7f894d9c9740 1 -- 172.21.15.5:0/3950782582 shutdown_connections 2024-04-24T06:53:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.260+0000 7f894d9c9740 1 -- 172.21.15.5:0/3950782582 wait complete. 2024-04-24T06:53:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.260+0000 7f894d9c9740 20 client.27319 trim_cache size 0 max 16384 2024-04-24T06:53:35.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.275+0000 7f894d9c9740 1 Processor -- start 2024-04-24T06:53:35.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.275+0000 7f894d9c9740 1 -- start start 2024-04-24T06:53:35.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.276+0000 7f894d9c9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd4027bb40 0x55fd402928e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:35.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.276+0000 7f894d9c9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd4027c460 0x55fd402827b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:35.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.276+0000 7f894d9c9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd40286b30 0x55fd40286010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:35.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.276+0000 7f894d9c9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f89441b6ae0 con 0x55fd4027c460 2024-04-24T06:53:35.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.276+0000 7f894d9c9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f89441b43b0 con 0x55fd40286b30 2024-04-24T06:53:35.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.276+0000 7f894d9c9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f89441b2220 con 0x55fd4027bb40 2024-04-24T06:53:35.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.276+0000 7f893b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd4027c460 0x55fd402827b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:35.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.276+0000 7f893b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd4027c460 0x55fd402827b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34382/0 (socket says 172.21.15.5:34382) 2024-04-24T06:53:35.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.276+0000 7f893b400640 1 -- 172.21.15.5:0/3184668306 learned_addr learned my addr 172.21.15.5:0/3184668306 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:35.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.276+0000 7f8943e00640 1 --2- 172.21.15.5:0/3184668306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd4027bb40 0x55fd402928e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:35.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.276+0000 7f8948a00640 1 --2- 172.21.15.5:0/3184668306 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd40286b30 0x55fd40286010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:35.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.277+0000 7f8948a00640 1 -- 172.21.15.5:0/3184668306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd4027bb40 msgr2=0x55fd402928e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:35.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.277+0000 7f8948a00640 1 --2- 172.21.15.5:0/3184668306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd4027bb40 0x55fd402928e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.277+0000 7f8948a00640 1 -- 172.21.15.5:0/3184668306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd4027c460 msgr2=0x55fd402827b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:35.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.277+0000 7f8948a00640 1 --2- 172.21.15.5:0/3184668306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd4027c460 0x55fd402827b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.277+0000 7f8948a00640 1 -- 172.21.15.5:0/3184668306 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd3ffbd7d0 con 0x55fd40286b30 2024-04-24T06:53:35.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.277+0000 7f893b400640 1 --2- 172.21.15.5:0/3184668306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd4027c460 0x55fd402827b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:53:35.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.277+0000 7f8948a00640 1 --2- 172.21.15.5:0/3184668306 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd40286b30 0x55fd40286010 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x55fd402724e0 tx=0x7f893c01a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5d764ce68de798 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:35.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.277+0000 7f8933e00640 1 -- 172.21.15.5:0/3184668306 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893c01e5f0 con 0x55fd40286b30 2024-04-24T06:53:35.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.277+0000 7f8933e00640 1 -- 172.21.15.5:0/3184668306 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f893c01ec70 con 0x55fd40286b30 2024-04-24T06:53:35.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.277+0000 7f8933e00640 1 -- 172.21.15.5:0/3184668306 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893c029de0 con 0x55fd40286b30 2024-04-24T06:53:35.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.278+0000 7f894d9c9740 1 -- 172.21.15.5:0/3184668306 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd40286b30 msgr2=0x55fd40286010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:35.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.278+0000 7f894d9c9740 1 --2- 172.21.15.5:0/3184668306 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd40286b30 0x55fd40286010 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x55fd402724e0 tx=0x7f893c01a000 comp rx=0 tx=0).stop 2024-04-24T06:53:35.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.278+0000 7f894d9c9740 1 -- 172.21.15.5:0/3184668306 shutdown_connections 2024-04-24T06:53:35.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.278+0000 7f894d9c9740 1 --2- 172.21.15.5:0/3184668306 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd40286b30 0x55fd40286010 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.278+0000 7f894d9c9740 1 --2- 172.21.15.5:0/3184668306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd4027c460 0x55fd402827b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.278+0000 7f894d9c9740 1 --2- 172.21.15.5:0/3184668306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd4027bb40 0x55fd402928e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.278+0000 7f894d9c9740 1 -- 172.21.15.5:0/3184668306 >> 172.21.15.5:0/3184668306 conn(0x55fd40277e60 msgr2=0x55fd402820a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:35.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.278+0000 7f894d9c9740 1 -- 172.21.15.5:0/3184668306 shutdown_connections 2024-04-24T06:53:35.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.278+0000 7f894d9c9740 1 -- 172.21.15.5:0/3184668306 wait complete. 2024-04-24T06:53:35.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.279+0000 7f894d9c9740 1 -- start start 2024-04-24T06:53:35.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.279+0000 7f894d9c9740 1 Processor -- start 2024-04-24T06:53:35.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.279+0000 7f894d9c9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd4027bb40 0x55fd4028edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:35.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.279+0000 7f894d9c9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd4027c460 0x55fd4028f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:35.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.279+0000 7f894d9c9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd40286b30 0x55fd40280a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:35.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.279+0000 7f894d9c9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f89441b2870 con 0x55fd4027bb40 2024-04-24T06:53:35.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.279+0000 7f894d9c9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f89441b3ce0 con 0x55fd4027c460 2024-04-24T06:53:35.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.279+0000 7f894d9c9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f89441b2f40 con 0x55fd40286b30 2024-04-24T06:53:35.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.279+0000 7f894d9c9740 10 client.0 _ll_register_callbacks cb 0x7f89340f4b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:53:35.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.279+0000 7f8943e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd4027bb40 0x55fd4028edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:35.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.280+0000 7f8943e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd4027bb40 0x55fd4028edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34398/0 (socket says 172.21.15.5:34398) 2024-04-24T06:53:35.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.280+0000 7f8943e00640 1 -- 172.21.15.5:0/1117847758 learned_addr learned my addr 172.21.15.5:0/1117847758 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:35.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.280+0000 7f893b400640 1 --2- 172.21.15.5:0/1117847758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd4027c460 0x55fd4028f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:35.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.280+0000 7f8948a00640 1 --2- 172.21.15.5:0/1117847758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd40286b30 0x55fd40280a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:35.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.280+0000 7f893b400640 1 -- 172.21.15.5:0/1117847758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd40286b30 msgr2=0x55fd40280a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:35.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.280+0000 7f893b400640 1 --2- 172.21.15.5:0/1117847758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd40286b30 0x55fd40280a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.280+0000 7f893b400640 1 -- 172.21.15.5:0/1117847758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd4027bb40 msgr2=0x55fd4028edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:35.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.280+0000 7f893b400640 1 --2- 172.21.15.5:0/1117847758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd4027bb40 0x55fd4028edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.280+0000 7f893b400640 1 -- 172.21.15.5:0/1117847758 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd402812c0 con 0x55fd4027c460 2024-04-24T06:53:35.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.280+0000 7f8943e00640 1 --2- 172.21.15.5:0/1117847758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd4027bb40 0x55fd4028edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:53:35.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.280+0000 7f8948a00640 1 --2- 172.21.15.5:0/1117847758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd40286b30 0x55fd40280a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:53:35.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.280+0000 7f893b400640 1 --2- 172.21.15.5:0/1117847758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd4027c460 0x55fd4028f330 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f892c004940 tx=0x7f892c004970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b36c2203b2aca664 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:35.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.280+0000 7f8931600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:53:35.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.281+0000 7f8931600640 1 -- 172.21.15.5:0/1117847758 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892c01d040 con 0x55fd4027c460 2024-04-24T06:53:35.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.281+0000 7f894d9c9740 1 -- 172.21.15.5:0/1117847758 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f893c01ec70 con 0x55fd4027c460 2024-04-24T06:53:35.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.281+0000 7f8931600640 1 -- 172.21.15.5:0/1117847758 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f892c02bb50 con 0x55fd4027c460 2024-04-24T06:53:35.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.281+0000 7f8931600640 1 -- 172.21.15.5:0/1117847758 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f892c02bd50 con 0x55fd4027c460 2024-04-24T06:53:35.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.281+0000 7f894d9c9740 10 client.27305 fetch_fsmap learned FSMap version 27 2024-04-24T06:53:35.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.281+0000 7f894d9c9740 1 -- 172.21.15.5:0/1117847758 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fd40287c70 con 0x55fd4027c460 2024-04-24T06:53:35.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.281+0000 7f8931600640 1 -- 172.21.15.5:0/1117847758 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f893c01ec70 con 0x55fd4027c460 2024-04-24T06:53:35.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.281+0000 7f8931600640 1 -- 172.21.15.5:0/1117847758 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f892c0035f0 con 0x55fd4027c460 2024-04-24T06:53:35.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.282+0000 7f894d9c9740 10 client.27305 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:53:35.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.282+0000 7f894d9c9740 10 client.27305 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:53:35.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.282+0000 7f894d9c9740 20 client.27305 populate_metadata read hostname 'smithi005' 2024-04-24T06:53:35.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.282+0000 7f894d9c9740 1 --2- 172.21.15.5:0/1117847758 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55fd402fd120 0x55fd402ff510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:35.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.282+0000 7f894d9c9740 4 client.27305 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:53:35.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.282+0000 7f894d9c9740 1 -- 172.21.15.5:0/1117847758 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fd401e8a60 con 0x55fd402fd120 2024-04-24T06:53:35.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.282+0000 7f8943e00640 1 --2- 172.21.15.5:0/1117847758 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55fd402fd120 0x55fd402ff510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:35.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.283+0000 7f8943e00640 1 --2- 172.21.15.5:0/1117847758 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55fd402fd120 0x55fd402ff510 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5324675aa96a8b8a server_cookie=f79fee9beb589d3b in_seq=0 out_seq=0 2024-04-24T06:53:35.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.283+0000 7f8931600640 10 client.27305 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:53:35.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.283+0000 7f8931600640 1 -- 172.21.15.5:0/1117847758 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fd401e8a60 con 0x55fd402fd120 2024-04-24T06:53:35.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.283+0000 7f8931600640 10 client.27305 handle_command_reply: tid=0 2024-04-24T06:53:35.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.283+0000 7f894d9c9740 1 client.27305 shutdown 2024-04-24T06:53:35.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.283+0000 7f894d9c9740 1 -- 172.21.15.5:0/1117847758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd4027c460 msgr2=0x55fd4028f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:35.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.283+0000 7f894d9c9740 1 --2- 172.21.15.5:0/1117847758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd4027c460 0x55fd4028f330 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f892c004940 tx=0x7f892c004970 comp rx=0 tx=0).stop 2024-04-24T06:53:35.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.283+0000 7f894d9c9740 1 -- 172.21.15.5:0/1117847758 shutdown_connections 2024-04-24T06:53:35.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.283+0000 7f894d9c9740 1 --2- 172.21.15.5:0/1117847758 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55fd402fd120 0x55fd402ff510 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.283+0000 7f894d9c9740 1 --2- 172.21.15.5:0/1117847758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd40286b30 0x55fd40280a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.283+0000 7f894d9c9740 1 --2- 172.21.15.5:0/1117847758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fd4027c460 0x55fd4028f330 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.283+0000 7f894d9c9740 1 --2- 172.21.15.5:0/1117847758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd4027bb40 0x55fd4028edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.283+0000 7f8931600640 0 client.27305 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:53:35.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.283+0000 7f894d9c9740 1 -- 172.21.15.5:0/1117847758 >> 172.21.15.5:0/1117847758 conn(0x55fd40277e60 msgr2=0x55fd40282930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:35.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.283+0000 7f894d9c9740 1 -- 172.21.15.5:0/1117847758 shutdown_connections 2024-04-24T06:53:35.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.283+0000 7f894d9c9740 1 -- 172.21.15.5:0/1117847758 wait complete. 2024-04-24T06:53:35.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.283+0000 7f894d9c9740 20 client.27305 trim_cache size 0 max 16384 2024-04-24T06:53:35.293 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:53:35.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.284+0000 7f8949400640 1 -- 172.21.15.5:0/2202159629 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f89280a3d60 msgr2=0x7f89280a6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:35.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.284+0000 7f8949400640 1 --2- 172.21.15.5:0/2202159629 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f89280a3d60 0x7f89280a6220 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f893c011440 tx=0x7f893c009260 comp rx=0 tx=0).stop 2024-04-24T06:53:35.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.284+0000 7f8949400640 1 -- 172.21.15.5:0/2202159629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89440743a0 msgr2=0x7f894419ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:35.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.284+0000 7f8949400640 1 --2- 172.21.15.5:0/2202159629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89440743a0 0x7f894419ee00 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8934009d00 tx=0x7f89340305d0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.284+0000 7f8949400640 1 -- 172.21.15.5:0/2202159629 shutdown_connections 2024-04-24T06:53:35.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.284+0000 7f8949400640 1 --2- 172.21.15.5:0/2202159629 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f89280a3d60 0x7f89280a6220 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.284+0000 7f8949400640 1 --2- 172.21.15.5:0/2202159629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8944112e10 0x7f89441a36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.284+0000 7f8949400640 1 --2- 172.21.15.5:0/2202159629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8944074cc0 0x7f894419f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.284+0000 7f8949400640 1 --2- 172.21.15.5:0/2202159629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89440743a0 0x7f894419ee00 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:35.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.284+0000 7f8949400640 1 -- 172.21.15.5:0/2202159629 >> 172.21.15.5:0/2202159629 conn(0x7f89440fd950 msgr2=0x7f89440fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:35.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.284+0000 7f8949400640 1 -- 172.21.15.5:0/2202159629 shutdown_connections 2024-04-24T06:53:35.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:35.284+0000 7f8949400640 1 -- 172.21.15.5:0/2202159629 wait complete. 2024-04-24T06:53:35.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:35 smithi137 ceph-mon[33017]: pgmap v295: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:53:35.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:35 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/4260039415' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:53:35.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:35 smithi005 ceph-mon[27798]: pgmap v295: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:53:35.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:35 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/4260039415' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:53:35.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:35 smithi037 ceph-mon[32864]: pgmap v295: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:53:35.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:35 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/4260039415' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:53:36.304 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T06:53:36.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:53:36.304 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T06:53:36.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.533+0000 7f39ef000640 1 -- 172.21.15.5:0/736104293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39e8104450 msgr2=0x7f39e8104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:36.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.533+0000 7f39ef000640 1 --2- 172.21.15.5:0/736104293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39e8104450 0x7f39e8104830 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f39d8009900 tx=0x7f39d802a3f0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.533+0000 7f39ef000640 1 -- 172.21.15.5:0/736104293 shutdown_connections 2024-04-24T06:53:36.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.533+0000 7f39ef000640 1 --2- 172.21.15.5:0/736104293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39e8109520 0x7f39e81110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.533+0000 7f39ef000640 1 --2- 172.21.15.5:0/736104293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39e8104e00 0x7f39e8108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.533+0000 7f39ef000640 1 --2- 172.21.15.5:0/736104293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39e8104450 0x7f39e8104830 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.533+0000 7f39ef000640 1 -- 172.21.15.5:0/736104293 >> 172.21.15.5:0/736104293 conn(0x7f39e80ffc90 msgr2=0x7f39e81020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:36.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.533+0000 7f39ef000640 1 -- 172.21.15.5:0/736104293 shutdown_connections 2024-04-24T06:53:36.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.533+0000 7f39ef000640 1 -- 172.21.15.5:0/736104293 wait complete. 2024-04-24T06:53:36.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.534+0000 7f39ef000640 1 Processor -- start 2024-04-24T06:53:36.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.535+0000 7f39ef000640 1 -- start start 2024-04-24T06:53:36.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.535+0000 7f39ef000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39e8104450 0x7f39e81a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:36.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.535+0000 7f39ef000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39e8109520 0x7f39e81a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:36.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.535+0000 7f39ef000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39e819a960 0x7f39e819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:36.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.535+0000 7f39ef000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39e8114620 con 0x7f39e8109520 2024-04-24T06:53:36.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.535+0000 7f39ef000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f39e81144a0 con 0x7f39e819a960 2024-04-24T06:53:36.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.535+0000 7f39ef000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f39e81147a0 con 0x7f39e8104450 2024-04-24T06:53:36.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.535+0000 7f39e7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39e8104450 0x7f39e81a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:36.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.535+0000 7f39e7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39e8104450 0x7f39e81a0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:50758/0 (socket says 172.21.15.5:50758) 2024-04-24T06:53:36.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.535+0000 7f39e7e00640 1 -- 172.21.15.5:0/4182826153 learned_addr learned my addr 172.21.15.5:0/4182826153 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:36.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.535+0000 7f39e7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39e8109520 0x7f39e81a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:36.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.536+0000 7f39ed000640 1 --2- 172.21.15.5:0/4182826153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39e819a960 0x7f39e819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:36.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.536+0000 7f39e7e00640 1 -- 172.21.15.5:0/4182826153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39e819a960 msgr2=0x7f39e819ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:36.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.536+0000 7f39e7e00640 1 --2- 172.21.15.5:0/4182826153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39e819a960 0x7f39e819ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.536+0000 7f39e7e00640 1 -- 172.21.15.5:0/4182826153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39e8109520 msgr2=0x7f39e81a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:36.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.536+0000 7f39e7e00640 1 --2- 172.21.15.5:0/4182826153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39e8109520 0x7f39e81a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.536+0000 7f39e7e00640 1 -- 172.21.15.5:0/4182826153 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e819b480 con 0x7f39e8104450 2024-04-24T06:53:36.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.536+0000 7f39e7e00640 1 --2- 172.21.15.5:0/4182826153 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39e8104450 0x7f39e81a0760 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f39d8004770 tx=0x7f39d8003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:36.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.537+0000 7f39e4c00640 1 -- 172.21.15.5:0/4182826153 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d803f070 con 0x7f39e8104450 2024-04-24T06:53:36.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.537+0000 7f39e4c00640 1 -- 172.21.15.5:0/4182826153 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39d8031ca0 con 0x7f39e8104450 2024-04-24T06:53:36.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.537+0000 7f39ef000640 1 -- 172.21.15.5:0/4182826153 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39e819b710 con 0x7f39e8104450 2024-04-24T06:53:36.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.537+0000 7f39e4c00640 1 -- 172.21.15.5:0/4182826153 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d8035030 con 0x7f39e8104450 2024-04-24T06:53:36.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.537+0000 7f39ef000640 1 -- 172.21.15.5:0/4182826153 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39e81a7910 con 0x7f39e8104450 2024-04-24T06:53:36.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.540+0000 7f39f1e33740 1 Processor -- start 2024-04-24T06:53:36.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.540+0000 7f39f1e33740 1 -- start start 2024-04-24T06:53:36.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.540+0000 7f39f1e33740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570598ca090 0x5570598ca470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:36.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.540+0000 7f39f1e33740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570598ca9b0 0x5570598d4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:36.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.540+0000 7f39f1e33740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570598d5080 0x5570598d7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:36.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.540+0000 7f39f1e33740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55705982fe70 con 0x5570598ca9b0 2024-04-24T06:53:36.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.540+0000 7f39e4c00640 1 -- 172.21.15.5:0/4182826153 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f39d8042070 con 0x7f39e8104450 2024-04-24T06:53:36.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.540+0000 7f39f1e33740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5570597d6380 con 0x5570598d5080 2024-04-24T06:53:36.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.540+0000 7f39f1e33740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55705982be30 con 0x5570598ca090 2024-04-24T06:53:36.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.541+0000 7f39e7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570598ca9b0 0x5570598d4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:36.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.541+0000 7f39e7400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570598d5080 0x5570598d7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:36.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.541+0000 7f39e7400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570598d5080 0x5570598d7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:55382/0 (socket says 172.21.15.5:55382) 2024-04-24T06:53:36.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.541+0000 7f39e7400640 1 -- 172.21.15.5:0/1812489061 learned_addr learned my addr 172.21.15.5:0/1812489061 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:36.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.541+0000 7f39ed000640 1 --2- 172.21.15.5:0/1812489061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570598ca090 0x5570598ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:36.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.541+0000 7f39e7400640 1 -- 172.21.15.5:0/1812489061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570598ca090 msgr2=0x5570598ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:36.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.541+0000 7f39e7400640 1 --2- 172.21.15.5:0/1812489061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570598ca090 0x5570598ca470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.541+0000 7f39e7400640 1 -- 172.21.15.5:0/1812489061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570598ca9b0 msgr2=0x5570598d4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:36.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.541+0000 7f39e7400640 1 --2- 172.21.15.5:0/1812489061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570598ca9b0 0x5570598d4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.541+0000 7f39e7400640 1 -- 172.21.15.5:0/1812489061 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55705960a7e0 con 0x5570598d5080 2024-04-24T06:53:36.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.541+0000 7f39e7400640 1 --2- 172.21.15.5:0/1812489061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570598d5080 0x5570598d7470 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f39d0009c20 tx=0x7f39d0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e74d0b2d49010166 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:36.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.542+0000 7f39dd600640 1 -- 172.21.15.5:0/1812489061 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d0004230 con 0x5570598d5080 2024-04-24T06:53:36.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.542+0000 7f39dd600640 1 -- 172.21.15.5:0/1812489061 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39d00043d0 con 0x5570598d5080 2024-04-24T06:53:36.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.542+0000 7f39dd600640 1 -- 172.21.15.5:0/1812489061 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d0004dc0 con 0x5570598d5080 2024-04-24T06:53:36.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.542+0000 7f39e4c00640 1 --2- 172.21.15.5:0/4182826153 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f39cc0a3da0 0x7f39cc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:36.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.542+0000 7f39e4c00640 1 -- 172.21.15.5:0/4182826153 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f39d80f0d70 con 0x7f39e8104450 2024-04-24T06:53:36.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.542+0000 7f39f1e33740 1 -- 172.21.15.5:0/1812489061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570598d5080 msgr2=0x5570598d7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:36.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.542+0000 7f39f1e33740 1 --2- 172.21.15.5:0/1812489061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570598d5080 0x5570598d7470 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f39d0009c20 tx=0x7f39d0011810 comp rx=0 tx=0).stop 2024-04-24T06:53:36.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.542+0000 7f39ed000640 1 --2- 172.21.15.5:0/4182826153 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f39cc0a3da0 0x7f39cc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:36.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.542+0000 7f39f1e33740 1 -- 172.21.15.5:0/1812489061 shutdown_connections 2024-04-24T06:53:36.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.542+0000 7f39f1e33740 1 --2- 172.21.15.5:0/1812489061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570598d5080 0x5570598d7470 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.542+0000 7f39f1e33740 1 --2- 172.21.15.5:0/1812489061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570598ca9b0 0x5570598d4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.542+0000 7f39f1e33740 1 --2- 172.21.15.5:0/1812489061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570598ca090 0x5570598ca470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.542+0000 7f39f1e33740 1 -- 172.21.15.5:0/1812489061 >> 172.21.15.5:0/1812489061 conn(0x5570598c63b0 msgr2=0x5570598c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:36.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.543+0000 7f39f1e33740 1 -- 172.21.15.5:0/1812489061 shutdown_connections 2024-04-24T06:53:36.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.543+0000 7f39f1e33740 1 -- 172.21.15.5:0/1812489061 wait complete. 2024-04-24T06:53:36.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.543+0000 7f39f1e33740 1 -- start start 2024-04-24T06:53:36.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.543+0000 7f39ed000640 1 --2- 172.21.15.5:0/4182826153 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f39cc0a3da0 0x7f39cc0a6260 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f39e0005fd0 tx=0x7f39e0009500 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:36.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.543+0000 7f39f1e33740 1 Processor -- start 2024-04-24T06:53:36.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.544+0000 7f39f1e33740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570598ca090 0x5570598d1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:36.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.544+0000 7f39f1e33740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570598ca9b0 0x5570598d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:36.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.544+0000 7f39f1e33740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570598d5080 0x5570598d2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:36.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.544+0000 7f39f1e33740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5570598d7e70 con 0x5570598d5080 2024-04-24T06:53:36.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.544+0000 7f39f1e33740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5570597e9660 con 0x5570598ca090 2024-04-24T06:53:36.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.544+0000 7f39f1e33740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5570598d7ff0 con 0x5570598ca9b0 2024-04-24T06:53:36.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.544+0000 7f39f1e33740 10 client.0 _ll_register_callbacks cb 0x5570597d7aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:53:36.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.544+0000 7f39e7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570598ca090 0x5570598d1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:36.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.544+0000 7f39ed000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570598d5080 0x5570598d2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:36.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.544+0000 7f39e7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570598ca090 0x5570598d1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:55388/0 (socket says 172.21.15.5:55388) 2024-04-24T06:53:36.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.544+0000 7f39e7e00640 1 -- 172.21.15.5:0/2728259754 learned_addr learned my addr 172.21.15.5:0/2728259754 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:36.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.544+0000 7f39e7400640 1 --2- 172.21.15.5:0/2728259754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570598ca9b0 0x5570598d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:36.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.544+0000 7f39e7e00640 1 -- 172.21.15.5:0/2728259754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570598ca9b0 msgr2=0x5570598d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:36.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.544+0000 7f39e7e00640 1 --2- 172.21.15.5:0/2728259754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570598ca9b0 0x5570598d2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.544+0000 7f39e7e00640 1 -- 172.21.15.5:0/2728259754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570598d5080 msgr2=0x5570598d2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:36.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.544+0000 7f39e7e00640 1 --2- 172.21.15.5:0/2728259754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570598d5080 0x5570598d2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.545+0000 7f39e7e00640 1 -- 172.21.15.5:0/2728259754 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55705966af70 con 0x5570598ca090 2024-04-24T06:53:36.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.545+0000 7f39e7e00640 1 --2- 172.21.15.5:0/2728259754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570598ca090 0x5570598d1fe0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f39d8031550 tx=0x7f39d8009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17688eadc53ca1f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:36.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.545+0000 7f39d6000640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:53:36.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.545+0000 7f39d6000640 1 -- 172.21.15.5:0/2728259754 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d8049030 con 0x5570598ca090 2024-04-24T06:53:36.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.545+0000 7f39d6000640 1 -- 172.21.15.5:0/2728259754 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f39d803b4f0 con 0x5570598ca090 2024-04-24T06:53:36.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.545+0000 7f39f1e33740 1 -- 172.21.15.5:0/2728259754 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39d00043d0 con 0x5570598ca090 2024-04-24T06:53:36.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.545+0000 7f39d6000640 1 -- 172.21.15.5:0/2728259754 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f39d80dbe60 con 0x5570598ca090 2024-04-24T06:53:36.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.545+0000 7f39f1e33740 10 client.27317 fetch_fsmap learned FSMap version 27 2024-04-24T06:53:36.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.545+0000 7f39f1e33740 1 -- 172.21.15.5:0/2728259754 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557059502170 con 0x5570598ca090 2024-04-24T06:53:36.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.546+0000 7f39d6000640 1 -- 172.21.15.5:0/2728259754 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39d00043d0 con 0x5570598ca090 2024-04-24T06:53:36.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.546+0000 7f39d6000640 1 -- 172.21.15.5:0/2728259754 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f39d80ee6d0 con 0x5570598ca090 2024-04-24T06:53:36.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.546+0000 7f39f1e33740 10 client.27317 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:53:36.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.546+0000 7f39f1e33740 10 client.27317 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:53:36.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.546+0000 7f39f1e33740 20 client.27317 populate_metadata read hostname 'smithi005' 2024-04-24T06:53:36.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.546+0000 7f39f1e33740 1 --2- 172.21.15.5:0/2728259754 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5570598e5340 0x5570598e7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:36.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.546+0000 7f39f1e33740 4 client.27317 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:53:36.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.546+0000 7f39f1e33740 1 -- 172.21.15.5:0/2728259754 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55705983d890 con 0x5570598e5340 2024-04-24T06:53:36.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.546+0000 7f39e7400640 1 --2- 172.21.15.5:0/2728259754 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5570598e5340 0x5570598e7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:36.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.547+0000 7f39e7400640 1 --2- 172.21.15.5:0/2728259754 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5570598e5340 0x5570598e7770 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b8dfdd9fff3431d server_cookie=7b7390e6528f9d01 in_seq=0 out_seq=0 2024-04-24T06:53:36.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.547+0000 7f39d6000640 10 client.27317 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:53:36.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.548+0000 7f39d6000640 1 -- 172.21.15.5:0/2728259754 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55705983d890 con 0x5570598e5340 2024-04-24T06:53:36.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.548+0000 7f39d6000640 10 client.27317 handle_command_reply: tid=0 2024-04-24T06:53:36.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.548+0000 7f39f1e33740 1 client.27317 shutdown 2024-04-24T06:53:36.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.548+0000 7f39f1e33740 1 -- 172.21.15.5:0/2728259754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570598ca090 msgr2=0x5570598d1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:36.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.548+0000 7f39f1e33740 1 --2- 172.21.15.5:0/2728259754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570598ca090 0x5570598d1fe0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f39d8031550 tx=0x7f39d8009560 comp rx=0 tx=0).stop 2024-04-24T06:53:36.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.548+0000 7f39f1e33740 1 -- 172.21.15.5:0/2728259754 shutdown_connections 2024-04-24T06:53:36.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.548+0000 7f39f1e33740 1 --2- 172.21.15.5:0/2728259754 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5570598e5340 0x5570598e7770 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.548+0000 7f39f1e33740 1 --2- 172.21.15.5:0/2728259754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570598d5080 0x5570598d2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.548+0000 7f39f1e33740 1 --2- 172.21.15.5:0/2728259754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570598ca9b0 0x5570598d2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.548+0000 7f39f1e33740 1 --2- 172.21.15.5:0/2728259754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570598ca090 0x5570598d1fe0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.548+0000 7f39f1e33740 1 -- 172.21.15.5:0/2728259754 >> 172.21.15.5:0/2728259754 conn(0x5570598c63b0 msgr2=0x5570598cc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:36.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.548+0000 7f39d6000640 0 client.27317 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:53:36.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.549+0000 7f39f1e33740 1 -- 172.21.15.5:0/2728259754 shutdown_connections 2024-04-24T06:53:36.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.549+0000 7f39f1e33740 1 -- 172.21.15.5:0/2728259754 wait complete. 2024-04-24T06:53:36.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.549+0000 7f39f1e33740 20 client.27317 trim_cache size 0 max 16384 2024-04-24T06:53:36.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.568+0000 7f39f1e33740 1 Processor -- start 2024-04-24T06:53:36.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.568+0000 7f39f1e33740 1 -- start start 2024-04-24T06:53:36.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.569+0000 7f39f1e33740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570598ca090 0x5570598e0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:36.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.569+0000 7f39f1e33740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570598ca9b0 0x5570598cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:36.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.569+0000 7f39f1e33740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570598d5080 0x5570598c5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:36.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.569+0000 7f39f1e33740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39e81b2790 con 0x5570598ca090 2024-04-24T06:53:36.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.569+0000 7f39f1e33740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f39e81b0060 con 0x5570598d5080 2024-04-24T06:53:36.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.569+0000 7f39f1e33740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f39e81aded0 con 0x5570598ca9b0 2024-04-24T06:53:36.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.569+0000 7f39e7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570598ca090 0x5570598e0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:36.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.569+0000 7f39ed000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570598d5080 0x5570598c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:36.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.569+0000 7f39e7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570598ca090 0x5570598e0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34440/0 (socket says 172.21.15.5:34440) 2024-04-24T06:53:36.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.569+0000 7f39e7e00640 1 -- 172.21.15.5:0/2547122049 learned_addr learned my addr 172.21.15.5:0/2547122049 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:36.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.569+0000 7f39e7400640 1 --2- 172.21.15.5:0/2547122049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570598ca9b0 0x5570598cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:36.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.569+0000 7f39e7e00640 1 -- 172.21.15.5:0/2547122049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570598ca9b0 msgr2=0x5570598cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:36.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.569+0000 7f39e7e00640 1 --2- 172.21.15.5:0/2547122049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570598ca9b0 0x5570598cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.569+0000 7f39e7e00640 1 -- 172.21.15.5:0/2547122049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570598d5080 msgr2=0x5570598c5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:36.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.569+0000 7f39e7e00640 1 --2- 172.21.15.5:0/2547122049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570598d5080 0x5570598c5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.569+0000 7f39e7e00640 1 -- 172.21.15.5:0/2547122049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570598d0fb0 con 0x5570598ca090 2024-04-24T06:53:36.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.570+0000 7f39e7e00640 1 --2- 172.21.15.5:0/2547122049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570598ca090 0x5570598e0d30 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f39d80c54a0 tx=0x7f39d80bb320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4667317128226c7a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:36.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.570+0000 7f39dcc00640 1 -- 172.21.15.5:0/2547122049 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d8049070 con 0x5570598ca090 2024-04-24T06:53:36.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.570+0000 7f39dcc00640 1 -- 172.21.15.5:0/2547122049 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39d80bce10 con 0x5570598ca090 2024-04-24T06:53:36.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.570+0000 7f39dcc00640 1 -- 172.21.15.5:0/2547122049 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d80bc770 con 0x5570598ca090 2024-04-24T06:53:36.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.571+0000 7f39f1e33740 1 -- 172.21.15.5:0/2547122049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570598ca090 msgr2=0x5570598e0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:36.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.571+0000 7f39f1e33740 1 --2- 172.21.15.5:0/2547122049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570598ca090 0x5570598e0d30 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f39d80c54a0 tx=0x7f39d80bb320 comp rx=0 tx=0).stop 2024-04-24T06:53:36.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.571+0000 7f39f1e33740 1 -- 172.21.15.5:0/2547122049 shutdown_connections 2024-04-24T06:53:36.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.571+0000 7f39f1e33740 1 --2- 172.21.15.5:0/2547122049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570598d5080 0x5570598c5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.571+0000 7f39f1e33740 1 --2- 172.21.15.5:0/2547122049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570598ca9b0 0x5570598cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.571+0000 7f39f1e33740 1 --2- 172.21.15.5:0/2547122049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570598ca090 0x5570598e0d30 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.571+0000 7f39f1e33740 1 -- 172.21.15.5:0/2547122049 >> 172.21.15.5:0/2547122049 conn(0x5570598c63b0 msgr2=0x5570598c4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:36.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.571+0000 7f39f1e33740 1 -- 172.21.15.5:0/2547122049 shutdown_connections 2024-04-24T06:53:36.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.571+0000 7f39f1e33740 1 -- 172.21.15.5:0/2547122049 wait complete. 2024-04-24T06:53:36.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.571+0000 7f39f1e33740 1 -- start start 2024-04-24T06:53:36.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.572+0000 7f39f1e33740 1 Processor -- start 2024-04-24T06:53:36.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.572+0000 7f39f1e33740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570598ca090 0x5570598cee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:36.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.572+0000 7f39f1e33740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570598ca9b0 0x5570598cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:36.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.572+0000 7f39f1e33740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570598d5080 0x5570598cf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:36.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.572+0000 7f39f1e33740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39e81ae520 con 0x5570598ca9b0 2024-04-24T06:53:36.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.572+0000 7f39f1e33740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f39e81af990 con 0x5570598ca090 2024-04-24T06:53:36.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.572+0000 7f39f1e33740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f39e81aebf0 con 0x5570598d5080 2024-04-24T06:53:36.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.572+0000 7f39f1e33740 10 client.0 _ll_register_callbacks cb 0x7f39d80f4510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:53:36.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.572+0000 7f39e7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570598ca9b0 0x5570598cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:36.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.572+0000 7f39e7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570598ca090 0x5570598cee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:36.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.572+0000 7f39e7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570598ca090 0x5570598cee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:55406/0 (socket says 172.21.15.5:55406) 2024-04-24T06:53:36.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.572+0000 7f39e7e00640 1 -- 172.21.15.5:0/2513926789 learned_addr learned my addr 172.21.15.5:0/2513926789 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:36.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.572+0000 7f39ed000640 1 --2- 172.21.15.5:0/2513926789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570598d5080 0x5570598cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:36.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.573+0000 7f39e7e00640 1 -- 172.21.15.5:0/2513926789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570598d5080 msgr2=0x5570598cf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:36.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.573+0000 7f39e7e00640 1 --2- 172.21.15.5:0/2513926789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570598d5080 0x5570598cf910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.573+0000 7f39e7e00640 1 -- 172.21.15.5:0/2513926789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570598ca9b0 msgr2=0x5570598cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:36.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.573+0000 7f39e7e00640 1 --2- 172.21.15.5:0/2513926789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570598ca9b0 0x5570598cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.573+0000 7f39e7e00640 1 -- 172.21.15.5:0/2513926789 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55705994aab0 con 0x5570598ca090 2024-04-24T06:53:36.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.573+0000 7f39ed000640 1 --2- 172.21.15.5:0/2513926789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570598d5080 0x5570598cf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:53:36.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.573+0000 7f39e7e00640 1 --2- 172.21.15.5:0/2513926789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570598ca090 0x5570598cee90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f39d80f4320 tx=0x7f39d80f45c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48ca6a6e9efaa0b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:36.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.573+0000 7f39d6000640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:53:36.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.573+0000 7f39d6000640 1 -- 172.21.15.5:0/2513926789 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d8049070 con 0x5570598ca090 2024-04-24T06:53:36.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.573+0000 7f39d6000640 1 -- 172.21.15.5:0/2513926789 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f39d80c5540 con 0x5570598ca090 2024-04-24T06:53:36.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.573+0000 7f39f1e33740 1 -- 172.21.15.5:0/2513926789 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39d80bce10 con 0x5570598ca090 2024-04-24T06:53:36.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.573+0000 7f39d6000640 1 -- 172.21.15.5:0/2513926789 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f39d80b6050 con 0x5570598ca090 2024-04-24T06:53:36.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.573+0000 7f39f1e33740 10 client.27323 fetch_fsmap learned FSMap version 27 2024-04-24T06:53:36.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.573+0000 7f39f1e33740 1 -- 172.21.15.5:0/2513926789 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55705994aab0 con 0x5570598ca090 2024-04-24T06:53:36.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.574+0000 7f39d6000640 1 -- 172.21.15.5:0/2513926789 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39d80bce10 con 0x5570598ca090 2024-04-24T06:53:36.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.574+0000 7f39d6000640 1 -- 172.21.15.5:0/2513926789 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f39d80ae020 con 0x5570598ca090 2024-04-24T06:53:36.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.574+0000 7f39f1e33740 10 client.27323 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:53:36.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.574+0000 7f39f1e33740 10 client.27323 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:53:36.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.574+0000 7f39f1e33740 20 client.27323 populate_metadata read hostname 'smithi005' 2024-04-24T06:53:36.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.574+0000 7f39f1e33740 1 --2- 172.21.15.5:0/2513926789 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55705994c050 0x55705994e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:36.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.574+0000 7f39f1e33740 4 client.27323 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T06:53:36.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.574+0000 7f39f1e33740 1 -- 172.21.15.5:0/2513926789 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557059837700 con 0x55705994c050 2024-04-24T06:53:36.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.574+0000 7f39e7400640 1 --2- 172.21.15.5:0/2513926789 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55705994c050 0x55705994e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:36.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.575+0000 7f39e7400640 1 --2- 172.21.15.5:0/2513926789 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55705994c050 0x55705994e440 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aaebc6948ec820f0 server_cookie=de2934174d9489aa in_seq=0 out_seq=0 2024-04-24T06:53:36.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.575+0000 7f39d6000640 10 client.27323 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:53:36.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.576+0000 7f39d6000640 1 -- 172.21.15.5:0/2513926789 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557059837700 con 0x55705994c050 2024-04-24T06:53:36.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.576+0000 7f39d6000640 10 client.27323 handle_command_reply: tid=0 2024-04-24T06:53:36.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.576+0000 7f39f1e33740 1 client.27323 shutdown 2024-04-24T06:53:36.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.576+0000 7f39f1e33740 1 -- 172.21.15.5:0/2513926789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570598ca090 msgr2=0x5570598cee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:36.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.576+0000 7f39f1e33740 1 --2- 172.21.15.5:0/2513926789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570598ca090 0x5570598cee90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f39d80f4320 tx=0x7f39d80f45c0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.576+0000 7f39f1e33740 1 -- 172.21.15.5:0/2513926789 shutdown_connections 2024-04-24T06:53:36.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.576+0000 7f39f1e33740 1 --2- 172.21.15.5:0/2513926789 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55705994c050 0x55705994e440 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.576+0000 7f39f1e33740 1 --2- 172.21.15.5:0/2513926789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570598d5080 0x5570598cf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.576+0000 7f39f1e33740 1 --2- 172.21.15.5:0/2513926789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570598ca9b0 0x5570598cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.576+0000 7f39f1e33740 1 --2- 172.21.15.5:0/2513926789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570598ca090 0x5570598cee90 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.576+0000 7f39d6000640 0 client.27323 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:53:36.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.576+0000 7f39f1e33740 1 -- 172.21.15.5:0/2513926789 >> 172.21.15.5:0/2513926789 conn(0x5570598c63b0 msgr2=0x5570598d2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:36.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.577+0000 7f39f1e33740 1 -- 172.21.15.5:0/2513926789 shutdown_connections 2024-04-24T06:53:36.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.577+0000 7f39f1e33740 1 -- 172.21.15.5:0/2513926789 wait complete. 2024-04-24T06:53:36.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.577+0000 7f39f1e33740 20 client.27323 trim_cache size 0 max 16384 2024-04-24T06:53:36.585 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:53:36.586 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-24T06:53:36.586 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "87fa6eb6-3c41-4998-bb07-6d2d633fd960", 2024-04-24T06:53:36.586 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-24T06:53:36.586 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:53:36.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.577+0000 7f39ef000640 1 -- 172.21.15.5:0/4182826153 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f39cc0a3da0 msgr2=0x7f39cc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:36.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.577+0000 7f39ef000640 1 --2- 172.21.15.5:0/4182826153 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f39cc0a3da0 0x7f39cc0a6260 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f39e0005fd0 tx=0x7f39e0009500 comp rx=0 tx=0).stop 2024-04-24T06:53:36.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.577+0000 7f39ef000640 1 -- 172.21.15.5:0/4182826153 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39e8104450 msgr2=0x7f39e81a0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:36.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.577+0000 7f39ef000640 1 --2- 172.21.15.5:0/4182826153 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39e8104450 0x7f39e81a0760 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f39d8004770 tx=0x7f39d8003c10 comp rx=0 tx=0).stop 2024-04-24T06:53:36.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.577+0000 7f39ef000640 1 -- 172.21.15.5:0/4182826153 shutdown_connections 2024-04-24T06:53:36.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.577+0000 7f39ef000640 1 --2- 172.21.15.5:0/4182826153 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f39cc0a3da0 0x7f39cc0a6260 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.577+0000 7f39ef000640 1 --2- 172.21.15.5:0/4182826153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39e819a960 0x7f39e819ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.577+0000 7f39ef000640 1 --2- 172.21.15.5:0/4182826153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39e8109520 0x7f39e81a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.577+0000 7f39ef000640 1 --2- 172.21.15.5:0/4182826153 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39e8104450 0x7f39e81a0760 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.577+0000 7f39ef000640 1 -- 172.21.15.5:0/4182826153 >> 172.21.15.5:0/4182826153 conn(0x7f39e80ffc90 msgr2=0x7f39e8102010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:36.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.578+0000 7f39ef000640 1 -- 172.21.15.5:0/4182826153 shutdown_connections 2024-04-24T06:53:36.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.578+0000 7f39ef000640 1 -- 172.21.15.5:0/4182826153 wait complete. 2024-04-24T06:53:36.599 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T06:53:36.600 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T06:53:36.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.823+0000 7fec85e00640 1 -- 172.21.15.5:0/857584179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec800743b0 msgr2=0x7fec80074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:36.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.823+0000 7fec85e00640 1 --2- 172.21.15.5:0/857584179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec800743b0 0x7fec80074790 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fec74009900 tx=0x7fec7402a480 comp rx=0 tx=0).stop 2024-04-24T06:53:36.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.823+0000 7fec85e00640 1 -- 172.21.15.5:0/857584179 shutdown_connections 2024-04-24T06:53:36.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.823+0000 7fec85e00640 1 --2- 172.21.15.5:0/857584179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec8010e6c0 0x7fec80110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.823+0000 7fec85e00640 1 --2- 172.21.15.5:0/857584179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec80074cd0 0x7fec8010e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.823+0000 7fec85e00640 1 --2- 172.21.15.5:0/857584179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec800743b0 0x7fec80074790 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.823+0000 7fec85e00640 1 -- 172.21.15.5:0/857584179 >> 172.21.15.5:0/857584179 conn(0x7fec800fd950 msgr2=0x7fec800ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:36.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.823+0000 7fec85e00640 1 -- 172.21.15.5:0/857584179 shutdown_connections 2024-04-24T06:53:36.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.823+0000 7fec85e00640 1 -- 172.21.15.5:0/857584179 wait complete. 2024-04-24T06:53:36.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.824+0000 7fec85e00640 1 Processor -- start 2024-04-24T06:53:36.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.824+0000 7fec85e00640 1 -- start start 2024-04-24T06:53:36.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.825+0000 7fec85e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec800743b0 0x7fec801a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:36.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.825+0000 7fec85e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec80074cd0 0x7fec801a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:36.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.825+0000 7fec85e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec8010e6c0 0x7fec8019a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:36.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.825+0000 7fec85e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fec80114420 con 0x7fec80074cd0 2024-04-24T06:53:36.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.825+0000 7fec85e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fec801142a0 con 0x7fec8010e6c0 2024-04-24T06:53:36.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.825+0000 7fec85e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fec801145a0 con 0x7fec800743b0 2024-04-24T06:53:36.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.825+0000 7fec84a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec800743b0 0x7fec801a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:36.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.825+0000 7fec84a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec800743b0 0x7fec801a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:50804/0 (socket says 172.21.15.5:50804) 2024-04-24T06:53:36.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.825+0000 7fec84a00640 1 -- 172.21.15.5:0/2848862186 learned_addr learned my addr 172.21.15.5:0/2848862186 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:36.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.825+0000 7fec85400640 1 --2- 172.21.15.5:0/2848862186 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec8010e6c0 0x7fec8019a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:36.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.825+0000 7fec7fe00640 1 --2- 172.21.15.5:0/2848862186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec80074cd0 0x7fec801a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:36.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.826+0000 7fec84a00640 1 -- 172.21.15.5:0/2848862186 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec8010e6c0 msgr2=0x7fec8019a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:36.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.826+0000 7fec84a00640 1 --2- 172.21.15.5:0/2848862186 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec8010e6c0 0x7fec8019a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.826+0000 7fec84a00640 1 -- 172.21.15.5:0/2848862186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec80074cd0 msgr2=0x7fec801a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:36.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.826+0000 7fec84a00640 1 --2- 172.21.15.5:0/2848862186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec80074cd0 0x7fec801a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.826+0000 7fec84a00640 1 -- 172.21.15.5:0/2848862186 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec8019afe0 con 0x7fec800743b0 2024-04-24T06:53:36.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.826+0000 7fec7fe00640 1 --2- 172.21.15.5:0/2848862186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec80074cd0 0x7fec801a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:53:36.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.826+0000 7fec85400640 1 --2- 172.21.15.5:0/2848862186 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec8010e6c0 0x7fec8019a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:53:36.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.826+0000 7fec84a00640 1 --2- 172.21.15.5:0/2848862186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec800743b0 0x7fec801a0660 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fec74004770 tx=0x7fec74003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:36.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.826+0000 7fec7d600640 1 -- 172.21.15.5:0/2848862186 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7403f070 con 0x7fec800743b0 2024-04-24T06:53:36.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.827+0000 7fec7d600640 1 -- 172.21.15.5:0/2848862186 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec74031ca0 con 0x7fec800743b0 2024-04-24T06:53:36.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.827+0000 7fec85e00640 1 -- 172.21.15.5:0/2848862186 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec8019b270 con 0x7fec800743b0 2024-04-24T06:53:36.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.827+0000 7fec7d600640 1 -- 172.21.15.5:0/2848862186 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec74035030 con 0x7fec800743b0 2024-04-24T06:53:36.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.827+0000 7fec85e00640 1 -- 172.21.15.5:0/2848862186 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec801a75e0 con 0x7fec800743b0 2024-04-24T06:53:36.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.830+0000 7fec8a19d740 1 Processor -- start 2024-04-24T06:53:36.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.830+0000 7fec8a19d740 1 -- start start 2024-04-24T06:53:36.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.831+0000 7fec8a19d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f15f885a30 0x55f15f885e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:36.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.831+0000 7fec8a19d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f15f886350 0x55f15f8904e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:36.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.831+0000 7fec8a19d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f15f890a20 0x55f15f892e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:36.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.831+0000 7fec8a19d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f15f7eb1f0 con 0x55f15f886350 2024-04-24T06:53:36.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.831+0000 7fec8a19d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f15f791350 con 0x55f15f890a20 2024-04-24T06:53:36.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.831+0000 7fec8a19d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f15f7e71b0 con 0x55f15f885a30 2024-04-24T06:53:36.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.831+0000 7fec7d600640 1 -- 172.21.15.5:0/2848862186 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fec74042070 con 0x7fec800743b0 2024-04-24T06:53:36.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.831+0000 7fec85400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f15f885a30 0x55f15f885e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:36.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.831+0000 7fec7fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f15f890a20 0x55f15f892e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:36.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.831+0000 7fec85400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f15f885a30 0x55f15f885e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:50812/0 (socket says 172.21.15.5:50812) 2024-04-24T06:53:36.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.831+0000 7fec85400640 1 -- 172.21.15.5:0/1247688611 learned_addr learned my addr 172.21.15.5:0/1247688611 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:36.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.831+0000 7fec84a00640 1 --2- 172.21.15.5:0/1247688611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f15f886350 0x55f15f8904e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:36.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.832+0000 7fec85400640 1 -- 172.21.15.5:0/1247688611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f15f890a20 msgr2=0x55f15f892e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:36.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.832+0000 7fec85400640 1 --2- 172.21.15.5:0/1247688611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f15f890a20 0x55f15f892e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.832+0000 7fec85400640 1 -- 172.21.15.5:0/1247688611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f15f886350 msgr2=0x55f15f8904e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:36.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.832+0000 7fec85400640 1 --2- 172.21.15.5:0/1247688611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f15f886350 0x55f15f8904e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.832+0000 7fec85400640 1 -- 172.21.15.5:0/1247688611 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f15f7e5f50 con 0x55f15f885a30 2024-04-24T06:53:36.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.832+0000 7fec7fe00640 1 --2- 172.21.15.5:0/1247688611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f15f890a20 0x55f15f892e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:53:36.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.832+0000 7fec85400640 1 --2- 172.21.15.5:0/1247688611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f15f885a30 0x55f15f885e10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fec78011790 tx=0x7fec78011c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e098efe62efbc300 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:36.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.832+0000 7fec71c00640 1 -- 172.21.15.5:0/1247688611 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec78003c30 con 0x55f15f885a30 2024-04-24T06:53:36.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.832+0000 7fec71c00640 1 -- 172.21.15.5:0/1247688611 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec7800fe80 con 0x55f15f885a30 2024-04-24T06:53:36.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.832+0000 7fec71c00640 1 -- 172.21.15.5:0/1247688611 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec780193e0 con 0x55f15f885a30 2024-04-24T06:53:36.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.833+0000 7fec7d600640 1 --2- 172.21.15.5:0/2848862186 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fec680a3da0 0x7fec680a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:36.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.833+0000 7fec8a19d740 1 -- 172.21.15.5:0/1247688611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f15f885a30 msgr2=0x55f15f885e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:36.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.833+0000 7fec8a19d740 1 --2- 172.21.15.5:0/1247688611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f15f885a30 0x55f15f885e10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fec78011790 tx=0x7fec78011c50 comp rx=0 tx=0).stop 2024-04-24T06:53:36.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.833+0000 7fec7d600640 1 -- 172.21.15.5:0/2848862186 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7fec740f0df0 con 0x7fec800743b0 2024-04-24T06:53:36.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.833+0000 7fec84a00640 1 --2- 172.21.15.5:0/2848862186 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fec680a3da0 0x7fec680a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:36.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.834+0000 7fec8a19d740 1 -- 172.21.15.5:0/1247688611 shutdown_connections 2024-04-24T06:53:36.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.834+0000 7fec8a19d740 1 --2- 172.21.15.5:0/1247688611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f15f890a20 0x55f15f892e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.834+0000 7fec8a19d740 1 --2- 172.21.15.5:0/1247688611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f15f886350 0x55f15f8904e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.834+0000 7fec8a19d740 1 --2- 172.21.15.5:0/1247688611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f15f885a30 0x55f15f885e10 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.834+0000 7fec8a19d740 1 -- 172.21.15.5:0/1247688611 >> 172.21.15.5:0/1247688611 conn(0x55f15f881d50 msgr2=0x55f15f882130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:36.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.834+0000 7fec8a19d740 1 -- 172.21.15.5:0/1247688611 shutdown_connections 2024-04-24T06:53:36.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.834+0000 7fec8a19d740 1 -- 172.21.15.5:0/1247688611 wait complete. 2024-04-24T06:53:36.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.834+0000 7fec84a00640 1 --2- 172.21.15.5:0/2848862186 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fec680a3da0 0x7fec680a6260 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x55f15f886ff0 tx=0x7fec74009260 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:36.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.835+0000 7fec8a19d740 1 -- start start 2024-04-24T06:53:36.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.835+0000 7fec8a19d740 1 Processor -- start 2024-04-24T06:53:36.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.836+0000 7fec8a19d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f15f885a30 0x55f15f88da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:36.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.836+0000 7fec8a19d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f15f886350 0x55f15f88df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:36.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.836+0000 7fec8a19d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f15f890a20 0x55f15f88e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:36.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.836+0000 7fec8a19d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f15f893810 con 0x55f15f886350 2024-04-24T06:53:36.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.836+0000 7fec8a19d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f15f7a4690 con 0x55f15f890a20 2024-04-24T06:53:36.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.836+0000 7fec8a19d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f15f893990 con 0x55f15f885a30 2024-04-24T06:53:36.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.836+0000 7fec8a19d740 10 client.0 _ll_register_callbacks cb 0x55f15f7e0650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:53:36.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.836+0000 7fec7fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f15f886350 0x55f15f88df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:36.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.836+0000 7fec7fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f15f886350 0x55f15f88df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34502/0 (socket says 172.21.15.5:34502) 2024-04-24T06:53:36.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.836+0000 7fec7fe00640 1 -- 172.21.15.5:0/719952427 learned_addr learned my addr 172.21.15.5:0/719952427 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:36.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.836+0000 7fec85400640 1 --2- 172.21.15.5:0/719952427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f15f885a30 0x55f15f88da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:36.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.836+0000 7fec85400640 1 --2- 172.21.15.5:0/719952427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f15f890a20 0x55f15f88e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:36.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.836+0000 7fec7fe00640 1 -- 172.21.15.5:0/719952427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f15f885a30 msgr2=0x55f15f88da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:36.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.836+0000 7fec7fe00640 1 --2- 172.21.15.5:0/719952427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f15f885a30 0x55f15f88da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.836+0000 7fec7fe00640 1 -- 172.21.15.5:0/719952427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f15f890a20 msgr2=0x55f15f88e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:36.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.836+0000 7fec7fe00640 1 --2- 172.21.15.5:0/719952427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f15f890a20 0x55f15f88e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.836+0000 7fec7fe00640 1 -- 172.21.15.5:0/719952427 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f15f7a3920 con 0x55f15f886350 2024-04-24T06:53:36.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.837+0000 7fec7fe00640 1 --2- 172.21.15.5:0/719952427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f15f886350 0x55f15f88df80 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fec6c0126b0 tx=0x7fec6c012b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=755bb9c2dc943bfe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:36.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.837+0000 7fec66000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:53:36.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.837+0000 7fec66000640 1 -- 172.21.15.5:0/719952427 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6c011c70 con 0x55f15f886350 2024-04-24T06:53:36.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.837+0000 7fec66000640 1 -- 172.21.15.5:0/719952427 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7fec6c011e10 con 0x55f15f886350 2024-04-24T06:53:36.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.837+0000 7fec8a19d740 1 -- 172.21.15.5:0/719952427 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f15f7f8c10 con 0x55f15f886350 2024-04-24T06:53:36.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.838+0000 7fec66000640 1 -- 172.21.15.5:0/719952427 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fec6c004030 con 0x55f15f886350 2024-04-24T06:53:36.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.838+0000 7fec8a19d740 10 client.17187 fetch_fsmap learned FSMap version 27 2024-04-24T06:53:36.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.838+0000 7fec8a19d740 1 -- 172.21.15.5:0/719952427 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f15f5c77d0 con 0x55f15f886350 2024-04-24T06:53:36.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.838+0000 7fec66000640 1 -- 172.21.15.5:0/719952427 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f15f7f8c10 con 0x55f15f886350 2024-04-24T06:53:36.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.839+0000 7fec66000640 1 -- 172.21.15.5:0/719952427 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7fec6c01c460 con 0x55f15f886350 2024-04-24T06:53:36.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.839+0000 7fec8a19d740 10 client.17187 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:53:36.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.839+0000 7fec8a19d740 10 client.17187 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:53:36.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.839+0000 7fec8a19d740 20 client.17187 populate_metadata read hostname 'smithi005' 2024-04-24T06:53:36.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.839+0000 7fec8a19d740 1 --2- 172.21.15.5:0/719952427 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55f15f8a0ee0 0x55f15f8a3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:36.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.839+0000 7fec8a19d740 4 client.17187 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:53:36.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.839+0000 7fec8a19d740 1 -- 172.21.15.5:0/719952427 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f15f7f2a80 con 0x55f15f8a0ee0 2024-04-24T06:53:36.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.839+0000 7fec85400640 1 --2- 172.21.15.5:0/719952427 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55f15f8a0ee0 0x55f15f8a3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:36.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.840+0000 7fec85400640 1 --2- 172.21.15.5:0/719952427 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55f15f8a0ee0 0x55f15f8a3300 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2e72a10c15c44bd server_cookie=d82d8420c7c882c6 in_seq=0 out_seq=0 2024-04-24T06:53:36.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.840+0000 7fec66000640 10 client.17187 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:53:36.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.840+0000 7fec66000640 1 -- 172.21.15.5:0/719952427 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f15f7f2a80 con 0x55f15f8a0ee0 2024-04-24T06:53:36.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.840+0000 7fec66000640 10 client.17187 handle_command_reply: tid=0 2024-04-24T06:53:36.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.841+0000 7fec8a19d740 1 client.17187 shutdown 2024-04-24T06:53:36.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.841+0000 7fec8a19d740 1 -- 172.21.15.5:0/719952427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f15f886350 msgr2=0x55f15f88df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:36.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.841+0000 7fec8a19d740 1 --2- 172.21.15.5:0/719952427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f15f886350 0x55f15f88df80 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fec6c0126b0 tx=0x7fec6c012b70 comp rx=0 tx=0).stop 2024-04-24T06:53:36.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.841+0000 7fec8a19d740 1 -- 172.21.15.5:0/719952427 shutdown_connections 2024-04-24T06:53:36.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.841+0000 7fec8a19d740 1 --2- 172.21.15.5:0/719952427 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55f15f8a0ee0 0x55f15f8a3300 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.841+0000 7fec8a19d740 1 --2- 172.21.15.5:0/719952427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f15f890a20 0x55f15f88e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.841+0000 7fec8a19d740 1 --2- 172.21.15.5:0/719952427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f15f886350 0x55f15f88df80 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.841+0000 7fec8a19d740 1 --2- 172.21.15.5:0/719952427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f15f885a30 0x55f15f88da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.841+0000 7fec8a19d740 1 -- 172.21.15.5:0/719952427 >> 172.21.15.5:0/719952427 conn(0x55f15f881d50 msgr2=0x55f15f887b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:36.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.841+0000 7fec66000640 0 client.17187 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:53:36.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.841+0000 7fec8a19d740 1 -- 172.21.15.5:0/719952427 shutdown_connections 2024-04-24T06:53:36.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.841+0000 7fec8a19d740 1 -- 172.21.15.5:0/719952427 wait complete. 2024-04-24T06:53:36.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.841+0000 7fec8a19d740 20 client.17187 trim_cache size 0 max 16384 2024-04-24T06:53:36.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.867+0000 7fec8a19d740 1 Processor -- start 2024-04-24T06:53:36.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.867+0000 7fec8a19d740 1 -- start start 2024-04-24T06:53:36.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.867+0000 7fec8a19d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f15f885a30 0x55f15f880fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:36.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.867+0000 7fec8a19d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f15f886350 0x55f15f8879a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:36.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.867+0000 7fec8a19d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f15f890a20 0x55f15f88e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:36.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.867+0000 7fec8a19d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fec801b26e0 con 0x55f15f890a20 2024-04-24T06:53:36.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.867+0000 7fec8a19d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fec801affb0 con 0x55f15f885a30 2024-04-24T06:53:36.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.867+0000 7fec8a19d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fec801ade20 con 0x55f15f886350 2024-04-24T06:53:36.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.868+0000 7fec7fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f15f886350 0x55f15f8879a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:36.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.868+0000 7fec7fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f15f886350 0x55f15f8879a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:50828/0 (socket says 172.21.15.5:50828) 2024-04-24T06:53:36.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.868+0000 7fec7fe00640 1 -- 172.21.15.5:0/2125915682 learned_addr learned my addr 172.21.15.5:0/2125915682 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:36.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.868+0000 7fec85400640 1 --2- 172.21.15.5:0/2125915682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f15f885a30 0x55f15f880fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:36.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.868+0000 7fec85400640 1 --2- 172.21.15.5:0/2125915682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f15f890a20 0x55f15f88e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:36.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.868+0000 7fec7fe00640 1 -- 172.21.15.5:0/2125915682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f15f885a30 msgr2=0x55f15f880fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:36.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.868+0000 7fec7fe00640 1 --2- 172.21.15.5:0/2125915682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f15f885a30 0x55f15f880fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.868+0000 7fec7fe00640 1 -- 172.21.15.5:0/2125915682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f15f890a20 msgr2=0x55f15f88e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:36.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.868+0000 7fec7fe00640 1 --2- 172.21.15.5:0/2125915682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f15f890a20 0x55f15f88e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.868+0000 7fec7fe00640 1 -- 172.21.15.5:0/2125915682 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f15f4fd3f0 con 0x55f15f886350 2024-04-24T06:53:36.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.868+0000 7fec7fe00640 1 --2- 172.21.15.5:0/2125915682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f15f886350 0x55f15f8879a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fec6c000c00 tx=0x7fec6c009280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0e16383971664d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:36.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.869+0000 7fec71200640 1 -- 172.21.15.5:0/2125915682 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6c028040 con 0x55f15f886350 2024-04-24T06:53:36.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.869+0000 7fec71200640 1 -- 172.21.15.5:0/2125915682 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec6c01cc30 con 0x55f15f886350 2024-04-24T06:53:36.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.869+0000 7fec71200640 1 -- 172.21.15.5:0/2125915682 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6c003700 con 0x55f15f886350 2024-04-24T06:53:36.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.870+0000 7fec8a19d740 1 -- 172.21.15.5:0/2125915682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f15f886350 msgr2=0x55f15f8879a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:36.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.870+0000 7fec8a19d740 1 --2- 172.21.15.5:0/2125915682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f15f886350 0x55f15f8879a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fec6c000c00 tx=0x7fec6c009280 comp rx=0 tx=0).stop 2024-04-24T06:53:36.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.870+0000 7fec8a19d740 1 -- 172.21.15.5:0/2125915682 shutdown_connections 2024-04-24T06:53:36.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.870+0000 7fec8a19d740 1 --2- 172.21.15.5:0/2125915682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f15f890a20 0x55f15f88e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.870+0000 7fec8a19d740 1 --2- 172.21.15.5:0/2125915682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f15f886350 0x55f15f8879a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.870+0000 7fec8a19d740 1 --2- 172.21.15.5:0/2125915682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f15f885a30 0x55f15f880fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.870+0000 7fec8a19d740 1 -- 172.21.15.5:0/2125915682 >> 172.21.15.5:0/2125915682 conn(0x55f15f881d50 msgr2=0x55f15f887290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:36.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.870+0000 7fec8a19d740 1 -- 172.21.15.5:0/2125915682 shutdown_connections 2024-04-24T06:53:36.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.870+0000 7fec8a19d740 1 -- 172.21.15.5:0/2125915682 wait complete. 2024-04-24T06:53:36.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.871+0000 7fec8a19d740 1 -- start start 2024-04-24T06:53:36.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.871+0000 7fec8a19d740 1 Processor -- start 2024-04-24T06:53:36.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.871+0000 7fec8a19d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f15f885a30 0x55f15f894690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:36.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.871+0000 7fec8a19d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f15f886350 0x55f15f894bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:36.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.871+0000 7fec8a19d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f15f890a20 0x55f15f895110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:36.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.871+0000 7fec8a19d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fec801ae470 con 0x55f15f885a30 2024-04-24T06:53:36.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.871+0000 7fec8a19d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fec801af8e0 con 0x55f15f886350 2024-04-24T06:53:36.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.871+0000 7fec8a19d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fec801aeb40 con 0x55f15f890a20 2024-04-24T06:53:36.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.871+0000 7fec8a19d740 10 client.0 _ll_register_callbacks cb 0x7fec6c009840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:53:36.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.872+0000 7fec7fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f15f886350 0x55f15f894bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:36.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.872+0000 7fec85400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f15f885a30 0x55f15f894690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:36.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.872+0000 7fec7fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f15f886350 0x55f15f894bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:55456/0 (socket says 172.21.15.5:55456) 2024-04-24T06:53:36.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.872+0000 7fec7fe00640 1 -- 172.21.15.5:0/2168760994 learned_addr learned my addr 172.21.15.5:0/2168760994 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:53:36.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.872+0000 7fec7fe00640 1 -- 172.21.15.5:0/2168760994 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f15f890a20 msgr2=0x55f15f895110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:36.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.872+0000 7fec85400640 1 --2- 172.21.15.5:0/2168760994 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f15f890a20 0x55f15f895110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:36.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.872+0000 7fec7fe00640 1 --2- 172.21.15.5:0/2168760994 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f15f890a20 0x55f15f895110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.872+0000 7fec7fe00640 1 -- 172.21.15.5:0/2168760994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f15f885a30 msgr2=0x55f15f894690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:53:36.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.872+0000 7fec7fe00640 1 --2- 172.21.15.5:0/2168760994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f15f885a30 0x55f15f894690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.872+0000 7fec7fe00640 1 -- 172.21.15.5:0/2168760994 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f15f4bd0f0 con 0x55f15f886350 2024-04-24T06:53:36.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.872+0000 7fec85400640 1 --2- 172.21.15.5:0/2168760994 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f15f890a20 0x55f15f895110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:53:36.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.873+0000 7fec7fe00640 1 --2- 172.21.15.5:0/2168760994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f15f886350 0x55f15f894bd0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fec6c01c720 tx=0x7fec6c005970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f32b9f131d06800f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:53:36.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.873+0000 7fec66000640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:53:36.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.873+0000 7fec66000640 1 -- 172.21.15.5:0/2168760994 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6c028030 con 0x55f15f886350 2024-04-24T06:53:36.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.873+0000 7fec8a19d740 1 -- 172.21.15.5:0/2168760994 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec6c01cc30 con 0x55f15f886350 2024-04-24T06:53:36.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.873+0000 7fec66000640 1 -- 172.21.15.5:0/2168760994 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7fec6c032040 con 0x55f15f886350 2024-04-24T06:53:36.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.873+0000 7fec66000640 1 -- 172.21.15.5:0/2168760994 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fec6c043050 con 0x55f15f886350 2024-04-24T06:53:36.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.873+0000 7fec8a19d740 10 client.27341 fetch_fsmap learned FSMap version 27 2024-04-24T06:53:36.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.873+0000 7fec8a19d740 1 -- 172.21.15.5:0/2168760994 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f15f7b0810 con 0x55f15f886350 2024-04-24T06:53:36.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.873+0000 7fec66000640 1 -- 172.21.15.5:0/2168760994 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec6c01cc30 con 0x55f15f886350 2024-04-24T06:53:36.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.874+0000 7fec66000640 1 -- 172.21.15.5:0/2168760994 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7fec6c02c420 con 0x55f15f886350 2024-04-24T06:53:36.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.874+0000 7fec8a19d740 10 client.27341 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:53:36.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.874+0000 7fec8a19d740 10 client.27341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:53:36.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.874+0000 7fec8a19d740 20 client.27341 populate_metadata read hostname 'smithi005' 2024-04-24T06:53:36.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.874+0000 7fec8a19d740 1 --2- 172.21.15.5:0/2168760994 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55f15f907b20 0x55f15f909f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:53:36.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.874+0000 7fec8a19d740 4 client.27341 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:53:36.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.874+0000 7fec8a19d740 1 -- 172.21.15.5:0/2168760994 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f15f76e360 con 0x55f15f907b20 2024-04-24T06:53:36.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.874+0000 7fec85400640 1 --2- 172.21.15.5:0/2168760994 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55f15f907b20 0x55f15f909f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:53:36.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.875+0000 7fec85400640 1 --2- 172.21.15.5:0/2168760994 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55f15f907b20 0x55f15f909f40 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f12179aed4f58310 server_cookie=2de0e450d7d824c6 in_seq=0 out_seq=0 2024-04-24T06:53:36.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.875+0000 7fec66000640 10 client.27341 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:53:36.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.875+0000 7fec66000640 1 -- 172.21.15.5:0/2168760994 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55f15f76e360 con 0x55f15f907b20 2024-04-24T06:53:36.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.875+0000 7fec66000640 10 client.27341 handle_command_reply: tid=0 2024-04-24T06:53:36.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.875+0000 7fec8a19d740 1 client.27341 shutdown 2024-04-24T06:53:36.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.875+0000 7fec8a19d740 1 -- 172.21.15.5:0/2168760994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f15f886350 msgr2=0x55f15f894bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:36.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.875+0000 7fec8a19d740 1 --2- 172.21.15.5:0/2168760994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f15f886350 0x55f15f894bd0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fec6c01c720 tx=0x7fec6c005970 comp rx=0 tx=0).stop 2024-04-24T06:53:36.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.876+0000 7fec8a19d740 1 -- 172.21.15.5:0/2168760994 shutdown_connections 2024-04-24T06:53:36.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.876+0000 7fec8a19d740 1 --2- 172.21.15.5:0/2168760994 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55f15f907b20 0x55f15f909f40 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.876+0000 7fec8a19d740 1 --2- 172.21.15.5:0/2168760994 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f15f890a20 0x55f15f895110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.876+0000 7fec8a19d740 1 --2- 172.21.15.5:0/2168760994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f15f886350 0x55f15f894bd0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.876+0000 7fec8a19d740 1 --2- 172.21.15.5:0/2168760994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f15f885a30 0x55f15f894690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.876+0000 7fec8a19d740 1 -- 172.21.15.5:0/2168760994 >> 172.21.15.5:0/2168760994 conn(0x55f15f881d50 msgr2=0x55f15f887a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:36.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.876+0000 7fec66000640 0 client.27341 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:53:36.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.876+0000 7fec8a19d740 1 -- 172.21.15.5:0/2168760994 shutdown_connections 2024-04-24T06:53:36.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.876+0000 7fec8a19d740 1 -- 172.21.15.5:0/2168760994 wait complete. 2024-04-24T06:53:36.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.876+0000 7fec8a19d740 20 client.27341 trim_cache size 0 max 16384 2024-04-24T06:53:36.888 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:53:36.888 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1045 inodes in the stack)", 2024-04-24T06:53:36.888 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-24T06:53:36.889 INFO:teuthology.orchestra.run.smithi005.stdout: "87fa6eb6-3c41-4998-bb07-6d2d633fd960": { 2024-04-24T06:53:36.889 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-24T06:53:36.889 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "87fa6eb6-3c41-4998-bb07-6d2d633fd960", 2024-04-24T06:53:36.889 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-04-24T06:53:36.889 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:53:36.889 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:53:36.889 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:53:36.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.877+0000 7fec85e00640 1 -- 172.21.15.5:0/2848862186 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fec680a3da0 msgr2=0x7fec680a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:36.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.877+0000 7fec85e00640 1 --2- 172.21.15.5:0/2848862186 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fec680a3da0 0x7fec680a6260 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x55f15f886ff0 tx=0x7fec74009260 comp rx=0 tx=0).stop 2024-04-24T06:53:36.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.877+0000 7fec85e00640 1 -- 172.21.15.5:0/2848862186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec800743b0 msgr2=0x7fec801a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:53:36.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.877+0000 7fec85e00640 1 --2- 172.21.15.5:0/2848862186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec800743b0 0x7fec801a0660 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fec74004770 tx=0x7fec74003c10 comp rx=0 tx=0).stop 2024-04-24T06:53:36.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.877+0000 7fec85e00640 1 -- 172.21.15.5:0/2848862186 shutdown_connections 2024-04-24T06:53:36.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.877+0000 7fec85e00640 1 --2- 172.21.15.5:0/2848862186 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fec680a3da0 0x7fec680a6260 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.877+0000 7fec85e00640 1 --2- 172.21.15.5:0/2848862186 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec8010e6c0 0x7fec8019a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.877+0000 7fec85e00640 1 --2- 172.21.15.5:0/2848862186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec80074cd0 0x7fec801a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.877+0000 7fec85e00640 1 --2- 172.21.15.5:0/2848862186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec800743b0 0x7fec801a0660 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:53:36.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.877+0000 7fec85e00640 1 -- 172.21.15.5:0/2848862186 >> 172.21.15.5:0/2848862186 conn(0x7fec800fd950 msgr2=0x7fec800fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:53:36.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.877+0000 7fec85e00640 1 -- 172.21.15.5:0/2848862186 shutdown_connections 2024-04-24T06:53:36.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:53:36.877+0000 7fec85e00640 1 -- 172.21.15.5:0/2848862186 wait complete. 2024-04-24T06:53:36.899 INFO:tasks.cephfs.filesystem:scrub status for tag:87fa6eb6-3c41-4998-bb07-6d2d633fd960 - {'path': '/', 'tag': '87fa6eb6-3c41-4998-bb07-6d2d633fd960', 'options': 'recursive,force'} 2024-04-24T06:53:37.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:37 smithi137 ceph-mon[33017]: pgmap v296: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:53:37.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:37 smithi137 ceph-mon[33017]: scrub queued for path: / 2024-04-24T06:53:37.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:37 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:53:37.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:37 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:53:37.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:37 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:53:37.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:37 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:53:37.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:37 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:53:37.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:37 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:53:37.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:37 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:53:37.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:37 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:53:37.761 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:37 smithi005 ceph-mon[27798]: pgmap v296: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:53:37.761 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:37 smithi005 ceph-mon[27798]: scrub queued for path: / 2024-04-24T06:53:37.761 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:37 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:53:37.761 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:37 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:53:37.761 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:37 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:53:37.762 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:37 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:53:37.762 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:37 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:53:37.762 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:37 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:53:37.762 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:37 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:53:37.762 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:37 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:53:37.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:37 smithi037 ceph-mon[32864]: pgmap v296: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:53:37.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:37 smithi037 ceph-mon[32864]: scrub queued for path: / 2024-04-24T06:53:37.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:37 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:53:37.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:37 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:53:37.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:37 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:53:37.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:37 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:53:37.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:37 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:53:37.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:37 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:53:37.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:37 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:53:37.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:37 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:53:38.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:38 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:53:38.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:38 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:53:38.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:38 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:53:38.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:38 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:53:38.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:38 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:53:38.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:38 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:53:38.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:38 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:53:38.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:38 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:53:38.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:38 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:53:39.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:39 smithi137 ceph-mon[33017]: pgmap v297: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:53:39.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:39 smithi005 ceph-mon[27798]: pgmap v297: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:53:39.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:39 smithi037 ceph-mon[32864]: pgmap v297: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:53:41.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:41 smithi137 ceph-mon[33017]: pgmap v298: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:53:41.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:41 smithi005 ceph-mon[27798]: pgmap v298: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:53:41.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:41 smithi037 ceph-mon[32864]: pgmap v298: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:53:43.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:43 smithi137 ceph-mon[33017]: pgmap v299: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:53:43.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:43 smithi005 ceph-mon[27798]: pgmap v299: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:53:43.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:43 smithi037 ceph-mon[32864]: pgmap v299: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:53:45.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:45 smithi137 ceph-mon[33017]: pgmap v300: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:53:45.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:45 smithi005 ceph-mon[27798]: pgmap v300: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:53:45.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:45 smithi037 ceph-mon[32864]: pgmap v300: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:53:47.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:47 smithi137 ceph-mon[33017]: pgmap v301: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:53:47.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:47 smithi005 ceph-mon[27798]: pgmap v301: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:53:47.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:47 smithi037 ceph-mon[32864]: pgmap v301: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-24T06:53:48.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:48 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:53:48.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:48 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:53:48.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:48 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:53:48.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:48 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:53:48.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:48 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:53:48.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:48 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:53:48.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:48 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:53:48.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:48 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:53:48.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:48 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:53:49.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:49 smithi137 ceph-mon[33017]: pgmap v302: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:53:49.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:49 smithi005 ceph-mon[27798]: pgmap v302: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:53:49.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:49 smithi037 ceph-mon[32864]: pgmap v302: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-24T06:53:51.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:51 smithi137 ceph-mon[33017]: pgmap v303: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:53:51.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:51 smithi005 ceph-mon[27798]: pgmap v303: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:53:51.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:51 smithi037 ceph-mon[32864]: pgmap v303: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:53:53.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:53 smithi137 ceph-mon[33017]: pgmap v304: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:53:53.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:53 smithi005 ceph-mon[27798]: pgmap v304: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:53:53.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:53 smithi037 ceph-mon[32864]: pgmap v304: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:53:55.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:55 smithi137 ceph-mon[33017]: pgmap v305: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:53:55.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:55 smithi005 ceph-mon[27798]: pgmap v305: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:53:55.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:55 smithi037 ceph-mon[32864]: pgmap v305: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:53:57.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:57 smithi137 ceph-mon[33017]: pgmap v306: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:53:57.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:57 smithi005 ceph-mon[27798]: pgmap v306: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:53:57.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:57 smithi037 ceph-mon[32864]: pgmap v306: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:53:59.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:53:59 smithi137 ceph-mon[33017]: pgmap v307: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:53:59.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:53:59 smithi005 ceph-mon[27798]: pgmap v307: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:53:59.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:53:59 smithi037 ceph-mon[32864]: pgmap v307: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:54:01.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:01 smithi137 ceph-mon[33017]: pgmap v308: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:54:01.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:01 smithi005 ceph-mon[27798]: pgmap v308: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:54:01.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:01 smithi037 ceph-mon[32864]: pgmap v308: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:54:03.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:03 smithi137 ceph-mon[33017]: pgmap v309: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:54:03.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:03 smithi005 ceph-mon[27798]: pgmap v309: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:54:03.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:03 smithi037 ceph-mon[32864]: pgmap v309: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:54:05.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:05 smithi137 ceph-mon[33017]: pgmap v310: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-24T06:54:05.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:05 smithi005 ceph-mon[27798]: pgmap v310: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-24T06:54:05.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:05 smithi037 ceph-mon[32864]: pgmap v310: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-24T06:54:06.900 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T06:54:06.901 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T06:54:07.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.130+0000 7f1852a00640 1 -- 172.21.15.5:0/3535005426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f184c108ee0 msgr2=0x7f184c1152b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:07.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.130+0000 7f1852a00640 1 --2- 172.21.15.5:0/3535005426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f184c108ee0 0x7f184c1152b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1844009c10 tx=0x7f184402a780 comp rx=0 tx=0).stop 2024-04-24T06:54:07.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.130+0000 7f1852a00640 1 -- 172.21.15.5:0/3535005426 shutdown_connections 2024-04-24T06:54:07.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.130+0000 7f1852a00640 1 --2- 172.21.15.5:0/3535005426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f184c108ee0 0x7f184c1152b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.130+0000 7f1852a00640 1 --2- 172.21.15.5:0/3535005426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f184c108540 0x7f184c1089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.130+0000 7f1852a00640 1 --2- 172.21.15.5:0/3535005426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f184c10e4f0 0x7f184c10e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.130+0000 7f1852a00640 1 -- 172.21.15.5:0/3535005426 >> 172.21.15.5:0/3535005426 conn(0x7f184c077e10 msgr2=0x7f184c106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:07.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.130+0000 7f1852a00640 1 -- 172.21.15.5:0/3535005426 shutdown_connections 2024-04-24T06:54:07.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.130+0000 7f1852a00640 1 -- 172.21.15.5:0/3535005426 wait complete. 2024-04-24T06:54:07.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.132+0000 7f1852a00640 1 Processor -- start 2024-04-24T06:54:07.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.132+0000 7f1852a00640 1 -- start start 2024-04-24T06:54:07.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.132+0000 7f1852a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f184c108540 0x7f184c1a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:07.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.132+0000 7f1852a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f184c108ee0 0x7f184c1a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:07.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.132+0000 7f1852a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f184c10e4f0 0x7f184c19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:07.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.132+0000 7f1852a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f184c118750 con 0x7f184c108ee0 2024-04-24T06:54:07.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.132+0000 7f1852a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f184c1185d0 con 0x7f184c10e4f0 2024-04-24T06:54:07.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.132+0000 7f1852a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f184c1188d0 con 0x7f184c108540 2024-04-24T06:54:07.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.133+0000 7f1850c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f184c108ee0 0x7f184c1a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:07.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.133+0000 7f1851600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f184c108540 0x7f184c1a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:07.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.133+0000 7f1850c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f184c108ee0 0x7f184c1a4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48232/0 (socket says 172.21.15.5:48232) 2024-04-24T06:54:07.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.133+0000 7f1850c00640 1 -- 172.21.15.5:0/3403081761 learned_addr learned my addr 172.21.15.5:0/3403081761 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:07.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.133+0000 7f1852000640 1 --2- 172.21.15.5:0/3403081761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f184c10e4f0 0x7f184c19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:07.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.133+0000 7f1850c00640 1 -- 172.21.15.5:0/3403081761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f184c108540 msgr2=0x7f184c1a4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:07.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.133+0000 7f1850c00640 1 --2- 172.21.15.5:0/3403081761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f184c108540 0x7f184c1a4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.133+0000 7f1850c00640 1 -- 172.21.15.5:0/3403081761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f184c10e4f0 msgr2=0x7f184c19eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:07.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.133+0000 7f1850c00640 1 --2- 172.21.15.5:0/3403081761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f184c10e4f0 0x7f184c19eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.133+0000 7f1850c00640 1 -- 172.21.15.5:0/3403081761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f184c19f270 con 0x7f184c108ee0 2024-04-24T06:54:07.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.134+0000 7f1850c00640 1 --2- 172.21.15.5:0/3403081761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f184c108ee0 0x7f184c1a4e80 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f183800e9c0 tx=0x7f183800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:07.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.134+0000 7f184a000640 1 -- 172.21.15.5:0/3403081761 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183800cde0 con 0x7f184c108ee0 2024-04-24T06:54:07.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.134+0000 7f1852a00640 1 -- 172.21.15.5:0/3403081761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f184c19f560 con 0x7f184c108ee0 2024-04-24T06:54:07.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.134+0000 7f184a000640 1 -- 172.21.15.5:0/3403081761 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18380035f0 con 0x7f184c108ee0 2024-04-24T06:54:07.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.134+0000 7f184a000640 1 -- 172.21.15.5:0/3403081761 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1838010650 con 0x7f184c108ee0 2024-04-24T06:54:07.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.134+0000 7f1852a00640 1 -- 172.21.15.5:0/3403081761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f184c1ab800 con 0x7f184c108ee0 2024-04-24T06:54:07.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.138+0000 7f184a000640 1 -- 172.21.15.5:0/3403081761 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f18380037a0 con 0x7f184c108ee0 2024-04-24T06:54:07.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.138+0000 7f1856e1b740 1 Processor -- start 2024-04-24T06:54:07.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.138+0000 7f1856e1b740 1 -- start start 2024-04-24T06:54:07.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.138+0000 7f1856e1b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab51132a30 0x55ab51132e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:07.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.138+0000 7f1856e1b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab51133350 0x55ab5113d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:07.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.138+0000 7f1856e1b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab5113da20 0x55ab5113fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:07.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.138+0000 7f1856e1b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ab510981f0 con 0x55ab5113da20 2024-04-24T06:54:07.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.138+0000 7f1856e1b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ab5103e350 con 0x55ab51133350 2024-04-24T06:54:07.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.138+0000 7f1856e1b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ab510941b0 con 0x55ab51132a30 2024-04-24T06:54:07.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.139+0000 7f1850c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab5113da20 0x55ab5113fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:07.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.139+0000 7f1850c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab5113da20 0x55ab5113fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48248/0 (socket says 172.21.15.5:48248) 2024-04-24T06:54:07.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.139+0000 7f1850c00640 1 -- 172.21.15.5:0/174608700 learned_addr learned my addr 172.21.15.5:0/174608700 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:07.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.139+0000 7f1852000640 1 --2- 172.21.15.5:0/174608700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab51132a30 0x55ab51132e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:07.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.139+0000 7f1851600640 1 --2- 172.21.15.5:0/174608700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab51133350 0x55ab5113d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:07.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.139+0000 7f1850c00640 1 -- 172.21.15.5:0/174608700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab51132a30 msgr2=0x55ab51132e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:07.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.139+0000 7f1850c00640 1 --2- 172.21.15.5:0/174608700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab51132a30 0x55ab51132e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.139+0000 7f1850c00640 1 -- 172.21.15.5:0/174608700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab51133350 msgr2=0x55ab5113d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:07.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.139+0000 7f1850c00640 1 --2- 172.21.15.5:0/174608700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab51133350 0x55ab5113d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.139+0000 7f1850c00640 1 -- 172.21.15.5:0/174608700 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab51092f50 con 0x55ab5113da20 2024-04-24T06:54:07.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.139+0000 7f184a000640 1 --2- 172.21.15.5:0/3403081761 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f18340a3d20 0x7f18340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:07.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.139+0000 7f1850c00640 1 --2- 172.21.15.5:0/174608700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab5113da20 0x55ab5113fe10 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f18380ce190 tx=0x7f18380d9260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80debc853adcdd2d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:07.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.140+0000 7f184a000640 1 -- 172.21.15.5:0/3403081761 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f18380d0b70 con 0x7f184c108ee0 2024-04-24T06:54:07.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.140+0000 7f1852000640 1 --2- 172.21.15.5:0/3403081761 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f18340a3d20 0x7f18340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:07.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.140+0000 7f183e600640 1 -- 172.21.15.5:0/174608700 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1838098d40 con 0x55ab5113da20 2024-04-24T06:54:07.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.140+0000 7f183e600640 1 -- 172.21.15.5:0/174608700 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f183809d1a0 con 0x55ab5113da20 2024-04-24T06:54:07.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.140+0000 7f183e600640 1 -- 172.21.15.5:0/174608700 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183809c400 con 0x55ab5113da20 2024-04-24T06:54:07.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.141+0000 7f1856e1b740 1 -- 172.21.15.5:0/174608700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab5113da20 msgr2=0x55ab5113fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:07.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.141+0000 7f1856e1b740 1 --2- 172.21.15.5:0/174608700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab5113da20 0x55ab5113fe10 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f18380ce190 tx=0x7f18380d9260 comp rx=0 tx=0).stop 2024-04-24T06:54:07.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.141+0000 7f1852000640 1 --2- 172.21.15.5:0/3403081761 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f18340a3d20 0x7f18340a61e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1844009370 tx=0x7f18440098a0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:07.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.141+0000 7f1856e1b740 1 -- 172.21.15.5:0/174608700 shutdown_connections 2024-04-24T06:54:07.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.141+0000 7f1856e1b740 1 --2- 172.21.15.5:0/174608700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab5113da20 0x55ab5113fe10 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.141+0000 7f1856e1b740 1 --2- 172.21.15.5:0/174608700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab51133350 0x55ab5113d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.141+0000 7f1856e1b740 1 --2- 172.21.15.5:0/174608700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab51132a30 0x55ab51132e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.141+0000 7f1856e1b740 1 -- 172.21.15.5:0/174608700 >> 172.21.15.5:0/174608700 conn(0x55ab5112ed50 msgr2=0x55ab5112f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:07.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.141+0000 7f1856e1b740 1 -- 172.21.15.5:0/174608700 shutdown_connections 2024-04-24T06:54:07.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.141+0000 7f1856e1b740 1 -- 172.21.15.5:0/174608700 wait complete. 2024-04-24T06:54:07.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.141+0000 7f1856e1b740 1 -- start start 2024-04-24T06:54:07.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.142+0000 7f1856e1b740 1 Processor -- start 2024-04-24T06:54:07.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.142+0000 7f1856e1b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab51132a30 0x55ab5113a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:07.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.142+0000 7f1856e1b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab51133350 0x55ab5113af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:07.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.142+0000 7f1856e1b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab5113da20 0x55ab5113b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:07.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.142+0000 7f1856e1b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ab511407b0 con 0x55ab51133350 2024-04-24T06:54:07.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.142+0000 7f1856e1b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ab51051690 con 0x55ab51132a30 2024-04-24T06:54:07.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.142+0000 7f1856e1b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ab51140930 con 0x55ab5113da20 2024-04-24T06:54:07.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.142+0000 7f1856e1b740 10 client.0 _ll_register_callbacks cb 0x55ab5108d650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:54:07.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.142+0000 7f1850c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab51133350 0x55ab5113af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:07.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.142+0000 7f1851600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab51132a30 0x55ab5113a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:07.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.143+0000 7f1850c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab51133350 0x55ab5113af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48258/0 (socket says 172.21.15.5:48258) 2024-04-24T06:54:07.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.143+0000 7f1850c00640 1 -- 172.21.15.5:0/3098993696 learned_addr learned my addr 172.21.15.5:0/3098993696 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:07.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.143+0000 7f1852000640 1 --2- 172.21.15.5:0/3098993696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab5113da20 0x55ab5113b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:07.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.143+0000 7f1851600640 1 -- 172.21.15.5:0/3098993696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab5113da20 msgr2=0x55ab5113b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:07.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.143+0000 7f1851600640 1 --2- 172.21.15.5:0/3098993696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab5113da20 0x55ab5113b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.143+0000 7f1851600640 1 -- 172.21.15.5:0/3098993696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab51133350 msgr2=0x55ab5113af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:07.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.143+0000 7f1851600640 1 --2- 172.21.15.5:0/3098993696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab51133350 0x55ab5113af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.143+0000 7f1851600640 1 -- 172.21.15.5:0/3098993696 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab51050920 con 0x55ab51132a30 2024-04-24T06:54:07.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.143+0000 7f1852000640 1 --2- 172.21.15.5:0/3098993696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab5113da20 0x55ab5113b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:54:07.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.143+0000 7f1850c00640 1 --2- 172.21.15.5:0/3098993696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab51133350 0x55ab5113af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:54:07.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.143+0000 7f1851600640 1 --2- 172.21.15.5:0/3098993696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab51132a30 0x55ab5113a9d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f18400126d0 tx=0x7f1840012b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f99f996d53a93457 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:07.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.143+0000 7f1832a00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:54:07.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.143+0000 7f1832a00640 1 -- 172.21.15.5:0/3098993696 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1840004030 con 0x55ab51132a30 2024-04-24T06:54:07.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.144+0000 7f1856e1b740 1 -- 172.21.15.5:0/3098993696 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f183809d1a0 con 0x55ab51132a30 2024-04-24T06:54:07.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.144+0000 7f1832a00640 1 -- 172.21.15.5:0/3098993696 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f18400041d0 con 0x55ab51132a30 2024-04-24T06:54:07.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.144+0000 7f1832a00640 1 -- 172.21.15.5:0/3098993696 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f18400043d0 con 0x55ab51132a30 2024-04-24T06:54:07.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.144+0000 7f1856e1b740 10 client.27353 fetch_fsmap learned FSMap version 27 2024-04-24T06:54:07.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.144+0000 7f1856e1b740 1 -- 172.21.15.5:0/3098993696 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ab50e747d0 con 0x55ab51132a30 2024-04-24T06:54:07.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.144+0000 7f1832a00640 1 -- 172.21.15.5:0/3098993696 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f183809d1a0 con 0x55ab51132a30 2024-04-24T06:54:07.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.145+0000 7f1832a00640 1 -- 172.21.15.5:0/3098993696 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f1840003af0 con 0x55ab51132a30 2024-04-24T06:54:07.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.145+0000 7f1856e1b740 10 client.27353 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:54:07.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.145+0000 7f1856e1b740 10 client.27353 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:54:07.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.145+0000 7f1856e1b740 20 client.27353 populate_metadata read hostname 'smithi005' 2024-04-24T06:54:07.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.145+0000 7f1856e1b740 1 --2- 172.21.15.5:0/3098993696 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55ab5114ddc0 0x55ab51150230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:07.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.145+0000 7f1856e1b740 4 client.27353 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:54:07.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.145+0000 7f1856e1b740 1 -- 172.21.15.5:0/3098993696 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab510a5c10 con 0x55ab5114ddc0 2024-04-24T06:54:07.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.145+0000 7f1850c00640 1 --2- 172.21.15.5:0/3098993696 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55ab5114ddc0 0x55ab51150230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:07.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.145+0000 7f1850c00640 1 --2- 172.21.15.5:0/3098993696 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55ab5114ddc0 0x55ab51150230 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca369b21652e5c18 server_cookie=d651223377f7a9b6 in_seq=0 out_seq=0 2024-04-24T06:54:07.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.145+0000 7f1832a00640 10 client.27353 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:54:07.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.146+0000 7f1832a00640 1 -- 172.21.15.5:0/3098993696 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ab510a5c10 con 0x55ab5114ddc0 2024-04-24T06:54:07.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.146+0000 7f1832a00640 10 client.27353 handle_command_reply: tid=0 2024-04-24T06:54:07.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.146+0000 7f1856e1b740 1 client.27353 shutdown 2024-04-24T06:54:07.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.147+0000 7f1856e1b740 1 -- 172.21.15.5:0/3098993696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab51132a30 msgr2=0x55ab5113a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:07.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.147+0000 7f1856e1b740 1 --2- 172.21.15.5:0/3098993696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab51132a30 0x55ab5113a9d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f18400126d0 tx=0x7f1840012b90 comp rx=0 tx=0).stop 2024-04-24T06:54:07.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.147+0000 7f1856e1b740 1 -- 172.21.15.5:0/3098993696 shutdown_connections 2024-04-24T06:54:07.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.147+0000 7f1856e1b740 1 --2- 172.21.15.5:0/3098993696 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55ab5114ddc0 0x55ab51150230 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.147+0000 7f1856e1b740 1 --2- 172.21.15.5:0/3098993696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab5113da20 0x55ab5113b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.147+0000 7f1856e1b740 1 --2- 172.21.15.5:0/3098993696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab51133350 0x55ab5113af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.147+0000 7f1856e1b740 1 --2- 172.21.15.5:0/3098993696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab51132a30 0x55ab5113a9d0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.147+0000 7f1856e1b740 1 -- 172.21.15.5:0/3098993696 >> 172.21.15.5:0/3098993696 conn(0x55ab5112ed50 msgr2=0x55ab51134b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:07.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.147+0000 7f1832a00640 0 client.27353 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:54:07.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.147+0000 7f1856e1b740 1 -- 172.21.15.5:0/3098993696 shutdown_connections 2024-04-24T06:54:07.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.147+0000 7f1856e1b740 1 -- 172.21.15.5:0/3098993696 wait complete. 2024-04-24T06:54:07.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.147+0000 7f1856e1b740 20 client.27353 trim_cache size 0 max 16384 2024-04-24T06:54:07.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.167+0000 7f1856e1b740 1 Processor -- start 2024-04-24T06:54:07.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.167+0000 7f1856e1b740 1 -- start start 2024-04-24T06:54:07.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.167+0000 7f1856e1b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab51132a30 0x55ab51149670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:07.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.167+0000 7f1856e1b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab51133350 0x55ab511352a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:07.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.167+0000 7f1856e1b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab5113da20 0x55ab51155b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:07.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.167+0000 7f1856e1b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f184c1b6b30 con 0x55ab51132a30 2024-04-24T06:54:07.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.167+0000 7f1856e1b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f184c1b4400 con 0x55ab5113da20 2024-04-24T06:54:07.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.167+0000 7f1856e1b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f184c1b2270 con 0x55ab51133350 2024-04-24T06:54:07.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.167+0000 7f1851600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab51132a30 0x55ab51149670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:07.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.167+0000 7f1851600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab51132a30 0x55ab51149670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48272/0 (socket says 172.21.15.5:48272) 2024-04-24T06:54:07.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.167+0000 7f1851600640 1 -- 172.21.15.5:0/3892050017 learned_addr learned my addr 172.21.15.5:0/3892050017 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:07.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.167+0000 7f1850c00640 1 --2- 172.21.15.5:0/3892050017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab51133350 0x55ab511352a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:07.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.167+0000 7f1852000640 1 --2- 172.21.15.5:0/3892050017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab5113da20 0x55ab51155b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:07.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.168+0000 7f1851600640 1 -- 172.21.15.5:0/3892050017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab51133350 msgr2=0x55ab511352a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:07.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.168+0000 7f1851600640 1 --2- 172.21.15.5:0/3892050017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab51133350 0x55ab511352a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.168+0000 7f1851600640 1 -- 172.21.15.5:0/3892050017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab5113da20 msgr2=0x55ab51155b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:07.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.168+0000 7f1851600640 1 --2- 172.21.15.5:0/3892050017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab5113da20 0x55ab51155b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.168+0000 7f1851600640 1 -- 172.21.15.5:0/3892050017 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab51133180 con 0x55ab51132a30 2024-04-24T06:54:07.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.168+0000 7f1852000640 1 --2- 172.21.15.5:0/3892050017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab5113da20 0x55ab51155b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:54:07.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.168+0000 7f1851600640 1 --2- 172.21.15.5:0/3892050017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab51132a30 0x55ab51149670 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f18400059f0 tx=0x7f184000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98143134cb06bfdc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:07.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.168+0000 7f183dc00640 1 -- 172.21.15.5:0/3892050017 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1840028070 con 0x55ab51132a30 2024-04-24T06:54:07.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.168+0000 7f183dc00640 1 -- 172.21.15.5:0/3892050017 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f184001c580 con 0x55ab51132a30 2024-04-24T06:54:07.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.168+0000 7f183dc00640 1 -- 172.21.15.5:0/3892050017 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184000fb80 con 0x55ab51132a30 2024-04-24T06:54:07.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.169+0000 7f1856e1b740 1 -- 172.21.15.5:0/3892050017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab51132a30 msgr2=0x55ab51149670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:07.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.169+0000 7f1856e1b740 1 --2- 172.21.15.5:0/3892050017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab51132a30 0x55ab51149670 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f18400059f0 tx=0x7f184000f790 comp rx=0 tx=0).stop 2024-04-24T06:54:07.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.169+0000 7f1856e1b740 1 -- 172.21.15.5:0/3892050017 shutdown_connections 2024-04-24T06:54:07.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.169+0000 7f1856e1b740 1 --2- 172.21.15.5:0/3892050017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab5113da20 0x55ab51155b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.169+0000 7f1856e1b740 1 --2- 172.21.15.5:0/3892050017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab51133350 0x55ab511352a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.169+0000 7f1856e1b740 1 --2- 172.21.15.5:0/3892050017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab51132a30 0x55ab51149670 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.169+0000 7f1856e1b740 1 -- 172.21.15.5:0/3892050017 >> 172.21.15.5:0/3892050017 conn(0x55ab5112ed50 msgr2=0x55ab5112dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:07.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.169+0000 7f1856e1b740 1 -- 172.21.15.5:0/3892050017 shutdown_connections 2024-04-24T06:54:07.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.169+0000 7f1856e1b740 1 -- 172.21.15.5:0/3892050017 wait complete. 2024-04-24T06:54:07.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.169+0000 7f1856e1b740 1 -- start start 2024-04-24T06:54:07.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.170+0000 7f1856e1b740 1 Processor -- start 2024-04-24T06:54:07.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.170+0000 7f1856e1b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab51132a30 0x55ab51141700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:07.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.170+0000 7f1856e1b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab51133350 0x55ab51141c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:07.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.170+0000 7f1856e1b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab5113da20 0x55ab51142180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:07.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.170+0000 7f1856e1b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f184c1b28c0 con 0x55ab5113da20 2024-04-24T06:54:07.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.170+0000 7f1856e1b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f184c1b3d30 con 0x55ab51132a30 2024-04-24T06:54:07.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.170+0000 7f1856e1b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f184c1b2f90 con 0x55ab51133350 2024-04-24T06:54:07.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.170+0000 7f1856e1b740 10 client.0 _ll_register_callbacks cb 0x7f1840009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:54:07.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.170+0000 7f1852000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab5113da20 0x55ab51142180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:07.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.170+0000 7f1852000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab5113da20 0x55ab51142180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48274/0 (socket says 172.21.15.5:48274) 2024-04-24T06:54:07.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.170+0000 7f1852000640 1 -- 172.21.15.5:0/421369269 learned_addr learned my addr 172.21.15.5:0/421369269 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:07.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.171+0000 7f1851600640 1 --2- 172.21.15.5:0/421369269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab51132a30 0x55ab51141700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:07.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.171+0000 7f1850c00640 1 --2- 172.21.15.5:0/421369269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab51133350 0x55ab51141c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:07.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.171+0000 7f1851600640 1 -- 172.21.15.5:0/421369269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab51133350 msgr2=0x55ab51141c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:07.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.171+0000 7f1851600640 1 --2- 172.21.15.5:0/421369269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab51133350 0x55ab51141c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.171+0000 7f1851600640 1 -- 172.21.15.5:0/421369269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab5113da20 msgr2=0x55ab51142180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:07.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.171+0000 7f1851600640 1 --2- 172.21.15.5:0/421369269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab5113da20 0x55ab51142180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.171+0000 7f1851600640 1 -- 172.21.15.5:0/421369269 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab50daa3f0 con 0x55ab51132a30 2024-04-24T06:54:07.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.171+0000 7f1852000640 1 --2- 172.21.15.5:0/421369269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab5113da20 0x55ab51142180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:54:07.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.171+0000 7f1851600640 1 --2- 172.21.15.5:0/421369269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab51132a30 0x55ab51141700 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f18400059f0 tx=0x7f18400096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=438c039eac547333 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:07.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.171+0000 7f1832a00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:54:07.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.172+0000 7f1832a00640 1 -- 172.21.15.5:0/421369269 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1840028040 con 0x55ab51132a30 2024-04-24T06:54:07.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.172+0000 7f1856e1b740 1 -- 172.21.15.5:0/421369269 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f184001c580 con 0x55ab51132a30 2024-04-24T06:54:07.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.172+0000 7f1832a00640 1 -- 172.21.15.5:0/421369269 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f1840005280 con 0x55ab51132a30 2024-04-24T06:54:07.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.172+0000 7f1832a00640 1 -- 172.21.15.5:0/421369269 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f184003d050 con 0x55ab51132a30 2024-04-24T06:54:07.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.172+0000 7f1856e1b740 10 client.27356 fetch_fsmap learned FSMap version 27 2024-04-24T06:54:07.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.172+0000 7f1856e1b740 1 -- 172.21.15.5:0/421369269 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ab50d6a0f0 con 0x55ab51132a30 2024-04-24T06:54:07.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.172+0000 7f1832a00640 1 -- 172.21.15.5:0/421369269 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f184001c580 con 0x55ab51132a30 2024-04-24T06:54:07.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.172+0000 7f1832a00640 1 -- 172.21.15.5:0/421369269 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f184001c850 con 0x55ab51132a30 2024-04-24T06:54:07.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.172+0000 7f1856e1b740 10 client.27356 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:54:07.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.172+0000 7f1856e1b740 10 client.27356 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:54:07.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.172+0000 7f1856e1b740 20 client.27356 populate_metadata read hostname 'smithi005' 2024-04-24T06:54:07.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.172+0000 7f1856e1b740 1 --2- 172.21.15.5:0/421369269 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55ab511b4950 0x55ab511b6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:07.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.173+0000 7f1856e1b740 4 client.27356 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:54:07.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.173+0000 7f1856e1b740 1 -- 172.21.15.5:0/421369269 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ab5109fa80 con 0x55ab511b4950 2024-04-24T06:54:07.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.173+0000 7f1850c00640 1 --2- 172.21.15.5:0/421369269 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55ab511b4950 0x55ab511b6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:07.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.173+0000 7f1850c00640 1 --2- 172.21.15.5:0/421369269 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55ab511b4950 0x55ab511b6d70 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ff5c53140f5204f server_cookie=b617daf1d7f1e7ed in_seq=0 out_seq=0 2024-04-24T06:54:07.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.173+0000 7f1832a00640 10 client.27356 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:54:07.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.173+0000 7f1832a00640 1 -- 172.21.15.5:0/421369269 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ab5109fa80 con 0x55ab511b4950 2024-04-24T06:54:07.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.173+0000 7f1832a00640 10 client.27356 handle_command_reply: tid=0 2024-04-24T06:54:07.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.174+0000 7f1856e1b740 1 client.27356 shutdown 2024-04-24T06:54:07.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.174+0000 7f1856e1b740 1 -- 172.21.15.5:0/421369269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab51132a30 msgr2=0x55ab51141700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:07.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.174+0000 7f1856e1b740 1 --2- 172.21.15.5:0/421369269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab51132a30 0x55ab51141700 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f18400059f0 tx=0x7f18400096c0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.174+0000 7f1856e1b740 1 -- 172.21.15.5:0/421369269 shutdown_connections 2024-04-24T06:54:07.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.174+0000 7f1856e1b740 1 --2- 172.21.15.5:0/421369269 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55ab511b4950 0x55ab511b6d70 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.174+0000 7f1856e1b740 1 --2- 172.21.15.5:0/421369269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab5113da20 0x55ab51142180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.174+0000 7f1856e1b740 1 --2- 172.21.15.5:0/421369269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab51133350 0x55ab51141c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.174+0000 7f1832a00640 0 client.27356 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:54:07.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.174+0000 7f1856e1b740 1 --2- 172.21.15.5:0/421369269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab51132a30 0x55ab51141700 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.174+0000 7f1856e1b740 1 -- 172.21.15.5:0/421369269 >> 172.21.15.5:0/421369269 conn(0x55ab5112ed50 msgr2=0x55ab5113a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:07.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.174+0000 7f1856e1b740 1 -- 172.21.15.5:0/421369269 shutdown_connections 2024-04-24T06:54:07.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.174+0000 7f1856e1b740 1 -- 172.21.15.5:0/421369269 wait complete. 2024-04-24T06:54:07.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.174+0000 7f1856e1b740 20 client.27356 trim_cache size 0 max 16384 2024-04-24T06:54:07.186 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:54:07.186 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-24T06:54:07.186 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-24T06:54:07.186 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:54:07.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.175+0000 7f1852a00640 1 -- 172.21.15.5:0/3403081761 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f18340a3d20 msgr2=0x7f18340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:07.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.175+0000 7f1852a00640 1 --2- 172.21.15.5:0/3403081761 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f18340a3d20 0x7f18340a61e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1844009370 tx=0x7f18440098a0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.175+0000 7f1852a00640 1 -- 172.21.15.5:0/3403081761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f184c108ee0 msgr2=0x7f184c1a4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:07.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.175+0000 7f1852a00640 1 --2- 172.21.15.5:0/3403081761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f184c108ee0 0x7f184c1a4e80 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f183800e9c0 tx=0x7f183800ee80 comp rx=0 tx=0).stop 2024-04-24T06:54:07.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.175+0000 7f1852a00640 1 -- 172.21.15.5:0/3403081761 shutdown_connections 2024-04-24T06:54:07.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.175+0000 7f1852a00640 1 --2- 172.21.15.5:0/3403081761 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f18340a3d20 0x7f18340a61e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.175+0000 7f1852a00640 1 --2- 172.21.15.5:0/3403081761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f184c10e4f0 0x7f184c19eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.175+0000 7f1852a00640 1 --2- 172.21.15.5:0/3403081761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f184c108ee0 0x7f184c1a4e80 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.175+0000 7f1852a00640 1 --2- 172.21.15.5:0/3403081761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f184c108540 0x7f184c1a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.175+0000 7f1852a00640 1 -- 172.21.15.5:0/3403081761 >> 172.21.15.5:0/3403081761 conn(0x7f184c077e10 msgr2=0x7f184c078490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:07.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.175+0000 7f1852a00640 1 -- 172.21.15.5:0/3403081761 shutdown_connections 2024-04-24T06:54:07.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.175+0000 7f1852a00640 1 -- 172.21.15.5:0/3403081761 wait complete. 2024-04-24T06:54:07.193 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T06:54:07.193 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:54:07.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.397+0000 7fd89e800640 1 -- 172.21.15.5:0/2390248199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8981087b0 msgr2=0x7fd898108b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:07.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.397+0000 7fd89e800640 1 --2- 172.21.15.5:0/2390248199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8981087b0 0x7fd898108b90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd88c009930 tx=0x7fd88c02a420 comp rx=0 tx=0).stop 2024-04-24T06:54:07.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.397+0000 7fd89e800640 1 -- 172.21.15.5:0/2390248199 shutdown_connections 2024-04-24T06:54:07.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.397+0000 7fd89e800640 1 --2- 172.21.15.5:0/2390248199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd89810d8a0 0x7fd898115420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.397+0000 7fd89e800640 1 --2- 172.21.15.5:0/2390248199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd898109160 0x7fd89810d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.397+0000 7fd89e800640 1 --2- 172.21.15.5:0/2390248199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8981087b0 0x7fd898108b90 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.397+0000 7fd89e800640 1 -- 172.21.15.5:0/2390248199 >> 172.21.15.5:0/2390248199 conn(0x7fd898077dd0 msgr2=0x7fd898106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:07.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.398+0000 7fd89e800640 1 -- 172.21.15.5:0/2390248199 shutdown_connections 2024-04-24T06:54:07.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.398+0000 7fd89e800640 1 -- 172.21.15.5:0/2390248199 wait complete. 2024-04-24T06:54:07.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.399+0000 7fd89e800640 1 Processor -- start 2024-04-24T06:54:07.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.399+0000 7fd89e800640 1 -- start start 2024-04-24T06:54:07.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.399+0000 7fd89e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8981087b0 0x7fd89819ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:07.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.399+0000 7fd89e800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd89810d8a0 0x7fd89819f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:07.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.399+0000 7fd89e800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8981a36e0 0x7fd8981a3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:07.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.399+0000 7fd89e800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd8981188a0 con 0x7fd8981087b0 2024-04-24T06:54:07.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.399+0000 7fd89e800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd898118720 con 0x7fd89810d8a0 2024-04-24T06:54:07.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.399+0000 7fd89e800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd898118a20 con 0x7fd8981a36e0 2024-04-24T06:54:07.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.399+0000 7fd89ca00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd89810d8a0 0x7fd89819f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:07.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.399+0000 7fd89d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8981087b0 0x7fd89819ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:07.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.400+0000 7fd89d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8981087b0 0x7fd89819ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48282/0 (socket says 172.21.15.5:48282) 2024-04-24T06:54:07.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.400+0000 7fd89d400640 1 -- 172.21.15.5:0/1825797670 learned_addr learned my addr 172.21.15.5:0/1825797670 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:07.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.400+0000 7fd89ca00640 1 --2- 172.21.15.5:0/1825797670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd89810d8a0 0x7fd89819f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:48798/0 (socket says 172.21.15.5:48798) 2024-04-24T06:54:07.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.400+0000 7fd89de00640 1 --2- 172.21.15.5:0/1825797670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8981a36e0 0x7fd8981a3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:07.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.400+0000 7fd89ca00640 1 -- 172.21.15.5:0/1825797670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8981a36e0 msgr2=0x7fd8981a3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:07.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.400+0000 7fd89ca00640 1 --2- 172.21.15.5:0/1825797670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8981a36e0 0x7fd8981a3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.400+0000 7fd89ca00640 1 -- 172.21.15.5:0/1825797670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8981087b0 msgr2=0x7fd89819ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:07.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.400+0000 7fd89ca00640 1 --2- 172.21.15.5:0/1825797670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8981087b0 0x7fd89819ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.400+0000 7fd89ca00640 1 -- 172.21.15.5:0/1825797670 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8981a41d0 con 0x7fd89810d8a0 2024-04-24T06:54:07.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.400+0000 7fd89d400640 1 --2- 172.21.15.5:0/1825797670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8981087b0 0x7fd89819ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:54:07.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.400+0000 7fd89ca00640 1 --2- 172.21.15.5:0/1825797670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd89810d8a0 0x7fd89819f360 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd88400e9c0 tx=0x7fd88400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:07.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.401+0000 7fd896000640 1 -- 172.21.15.5:0/1825797670 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88400ccc0 con 0x7fd89810d8a0 2024-04-24T06:54:07.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.401+0000 7fd89e800640 1 -- 172.21.15.5:0/1825797670 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8981a44c0 con 0x7fd89810d8a0 2024-04-24T06:54:07.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.401+0000 7fd896000640 1 -- 172.21.15.5:0/1825797670 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd884010910 con 0x7fd89810d8a0 2024-04-24T06:54:07.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.401+0000 7fd896000640 1 -- 172.21.15.5:0/1825797670 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd884018880 con 0x7fd89810d8a0 2024-04-24T06:54:07.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.401+0000 7fd89e800640 1 -- 172.21.15.5:0/1825797670 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8981abd00 con 0x7fd89810d8a0 2024-04-24T06:54:07.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.402+0000 7fd89e800640 1 -- 172.21.15.5:0/1825797670 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd89804fe00 con 0x7fd89810d8a0 2024-04-24T06:54:07.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.405+0000 7fd896000640 1 -- 172.21.15.5:0/1825797670 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fd884010ac0 con 0x7fd89810d8a0 2024-04-24T06:54:07.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.406+0000 7fd896000640 1 --2- 172.21.15.5:0/1825797670 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fd8800a3d90 0x7fd8800a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:07.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.406+0000 7fd896000640 1 -- 172.21.15.5:0/1825797670 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7fd8840d0fe0 con 0x7fd89810d8a0 2024-04-24T06:54:07.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.407+0000 7fd89d400640 1 --2- 172.21.15.5:0/1825797670 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fd8800a3d90 0x7fd8800a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:07.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.407+0000 7fd89d400640 1 --2- 172.21.15.5:0/1825797670 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fd8800a3d90 0x7fd8800a6250 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd88c000c00 tx=0x7fd88c005a20 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:07.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.408+0000 7fd896000640 1 -- 172.21.15.5:0/1825797670 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd88409d020 con 0x7fd89810d8a0 2024-04-24T06:54:07.505 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:07 smithi005 ceph-mon[27798]: pgmap v311: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-24T06:54:07.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.658+0000 7fd89e800640 1 -- 172.21.15.5:0/1825797670 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd898110a80 con 0x7fd89810d8a0 2024-04-24T06:54:07.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.659+0000 7fd896000640 1 -- 172.21.15.5:0/1825797670 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11856 (secure 0 0 0) 0x7fd8840a1320 con 0x7fd89810d8a0 2024-04-24T06:54:07.660 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:54:07.660 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1735462750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1735462750},{"type":"v1","addr":"172.21.15.5:6837","nonce":1735462750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1499469370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1499469370},{"type":"v1","addr":"172.21.15.5:6841","nonce":1499469370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3705638847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3705638847},{"type":"v1","addr":"172.21.15.37:6835","nonce":3705638847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T06:41:34.764490+0000","modified":"2024-04-24T06:48:57.768410+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24550,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6837/1390103296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1390103296},{"type":"v1","addr":"172.21.15.37:6837","nonce":1390103296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3217754215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3217754215},{"type":"v1","addr":"172.21.15.137:6835","nonce":3217754215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6837/3044222976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3044222976},{"type":"v1","addr":"172.21.15.137:6837","nonce":3044222976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.5:6839/3173201459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3173201459},{"type":"v1","addr":"172.21.15.5:6839","nonce":3173201459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6839/1889774232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1889774232},{"type":"v1","addr":"172.21.15.137:6839","nonce":1889774232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6839/3536721753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":3536721753},{"type":"v1","addr":"172.21.15.37:6839","nonce":3536721753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24550,24569,24539]},"id":1}]} 2024-04-24T06:54:07.660 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-04-24T06:54:07.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.662+0000 7fd89e800640 1 -- 172.21.15.5:0/1825797670 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fd8800a3d90 msgr2=0x7fd8800a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:07.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.662+0000 7fd89e800640 1 --2- 172.21.15.5:0/1825797670 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fd8800a3d90 0x7fd8800a6250 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd88c000c00 tx=0x7fd88c005a20 comp rx=0 tx=0).stop 2024-04-24T06:54:07.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.662+0000 7fd89e800640 1 -- 172.21.15.5:0/1825797670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd89810d8a0 msgr2=0x7fd89819f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:07.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.662+0000 7fd89e800640 1 --2- 172.21.15.5:0/1825797670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd89810d8a0 0x7fd89819f360 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd88400e9c0 tx=0x7fd88400ee80 comp rx=0 tx=0).stop 2024-04-24T06:54:07.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.662+0000 7fd89e800640 1 -- 172.21.15.5:0/1825797670 shutdown_connections 2024-04-24T06:54:07.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.662+0000 7fd89e800640 1 --2- 172.21.15.5:0/1825797670 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fd8800a3d90 0x7fd8800a6250 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.662+0000 7fd89e800640 1 --2- 172.21.15.5:0/1825797670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8981a36e0 0x7fd8981a3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.662+0000 7fd89e800640 1 --2- 172.21.15.5:0/1825797670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd89810d8a0 0x7fd89819f360 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.662+0000 7fd89e800640 1 --2- 172.21.15.5:0/1825797670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8981087b0 0x7fd89819ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.662+0000 7fd89e800640 1 -- 172.21.15.5:0/1825797670 >> 172.21.15.5:0/1825797670 conn(0x7fd898077dd0 msgr2=0x7fd898078a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:07.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.662+0000 7fd89e800640 1 -- 172.21.15.5:0/1825797670 shutdown_connections 2024-04-24T06:54:07.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.662+0000 7fd89e800640 1 -- 172.21.15.5:0/1825797670 wait complete. 2024-04-24T06:54:07.681 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T06:54:07.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:07 smithi137 ceph-mon[33017]: pgmap v311: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-24T06:54:07.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:07 smithi037 ceph-mon[32864]: pgmap v311: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-24T06:54:07.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.896+0000 7ff7f3600640 1 -- 172.21.15.5:0/3176087345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7ec104440 msgr2=0x7ff7ec104820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:07.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.896+0000 7ff7f3600640 1 --2- 172.21.15.5:0/3176087345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7ec104440 0x7ff7ec104820 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0009860 tx=0x7ff7e002a340 comp rx=0 tx=0).stop 2024-04-24T06:54:07.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.896+0000 7ff7f3600640 1 -- 172.21.15.5:0/3176087345 shutdown_connections 2024-04-24T06:54:07.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.896+0000 7ff7f3600640 1 --2- 172.21.15.5:0/3176087345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7ec109510 0x7ff7ec1110e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.896+0000 7ff7f3600640 1 --2- 172.21.15.5:0/3176087345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7ec104df0 0x7ff7ec108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.896+0000 7ff7f3600640 1 --2- 172.21.15.5:0/3176087345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7ec104440 0x7ff7ec104820 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.896+0000 7ff7f3600640 1 -- 172.21.15.5:0/3176087345 >> 172.21.15.5:0/3176087345 conn(0x7ff7ec0ffcc0 msgr2=0x7ff7ec1020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:07.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.896+0000 7ff7f3600640 1 -- 172.21.15.5:0/3176087345 shutdown_connections 2024-04-24T06:54:07.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.896+0000 7ff7f3600640 1 -- 172.21.15.5:0/3176087345 wait complete. 2024-04-24T06:54:07.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.897+0000 7ff7f3600640 1 Processor -- start 2024-04-24T06:54:07.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.897+0000 7ff7f3600640 1 -- start start 2024-04-24T06:54:07.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.898+0000 7ff7f3600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7ec104440 0x7ff7ec1a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:07.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.898+0000 7ff7f3600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7ec109510 0x7ff7ec1a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:07.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.898+0000 7ff7f3600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7ec19a940 0x7ff7ec19adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:07.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.898+0000 7ff7f3600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ec1145a0 con 0x7ff7ec109510 2024-04-24T06:54:07.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.898+0000 7ff7f3600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ec114420 con 0x7ff7ec19a940 2024-04-24T06:54:07.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.898+0000 7ff7f3600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ec114720 con 0x7ff7ec104440 2024-04-24T06:54:07.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.898+0000 7ff7ebe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7ec109510 0x7ff7ec1a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:07.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.898+0000 7ff7f0c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7ec104440 0x7ff7ec1a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:07.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.898+0000 7ff7ebe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7ec109510 0x7ff7ec1a0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48298/0 (socket says 172.21.15.5:48298) 2024-04-24T06:54:07.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.898+0000 7ff7ebe00640 1 -- 172.21.15.5:0/3373758647 learned_addr learned my addr 172.21.15.5:0/3373758647 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:07.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.898+0000 7ff7f1600640 1 --2- 172.21.15.5:0/3373758647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7ec19a940 0x7ff7ec19adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:07.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.899+0000 7ff7f0c00640 1 -- 172.21.15.5:0/3373758647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7ec19a940 msgr2=0x7ff7ec19adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:07.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.899+0000 7ff7f0c00640 1 --2- 172.21.15.5:0/3373758647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7ec19a940 0x7ff7ec19adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.899+0000 7ff7f0c00640 1 -- 172.21.15.5:0/3373758647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7ec109510 msgr2=0x7ff7ec1a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:07.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.899+0000 7ff7f0c00640 1 --2- 172.21.15.5:0/3373758647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7ec109510 0x7ff7ec1a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.899+0000 7ff7f0c00640 1 -- 172.21.15.5:0/3373758647 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7ec19b460 con 0x7ff7ec104440 2024-04-24T06:54:07.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.899+0000 7ff7f1600640 1 --2- 172.21.15.5:0/3373758647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7ec19a940 0x7ff7ec19adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:54:07.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.899+0000 7ff7ebe00640 1 --2- 172.21.15.5:0/3373758647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7ec109510 0x7ff7ec1a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:54:07.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.899+0000 7ff7f0c00640 1 --2- 172.21.15.5:0/3373758647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7ec104440 0x7ff7ec1a0790 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff7e002a860 tx=0x7ff7e0033b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:07.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.899+0000 7ff7e9600640 1 -- 172.21.15.5:0/3373758647 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e0037040 con 0x7ff7ec104440 2024-04-24T06:54:07.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.899+0000 7ff7f3600640 1 -- 172.21.15.5:0/3373758647 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7ec19b6f0 con 0x7ff7ec104440 2024-04-24T06:54:07.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.899+0000 7ff7e9600640 1 -- 172.21.15.5:0/3373758647 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7e0033d50 con 0x7ff7ec104440 2024-04-24T06:54:07.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.900+0000 7ff7e9600640 1 -- 172.21.15.5:0/3373758647 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e0003960 con 0x7ff7ec104440 2024-04-24T06:54:07.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.900+0000 7ff7f3600640 1 -- 172.21.15.5:0/3373758647 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7ec1a78e0 con 0x7ff7ec104440 2024-04-24T06:54:07.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.903+0000 7ff7f65e8740 1 Processor -- start 2024-04-24T06:54:07.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.903+0000 7ff7f65e8740 1 -- start start 2024-04-24T06:54:07.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.903+0000 7ff7e9600640 1 -- 172.21.15.5:0/3373758647 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7ff7e0042020 con 0x7ff7ec104440 2024-04-24T06:54:07.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.903+0000 7ff7f65e8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a595cd950 0x562a595cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:07.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.903+0000 7ff7f65e8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a595ce270 0x562a595d8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:07.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.903+0000 7ff7f65e8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a595d8940 0x562a595dad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:07.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.903+0000 7ff7f65e8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562a595331d0 con 0x562a595d8940 2024-04-24T06:54:07.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.903+0000 7ff7f65e8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562a594d9450 con 0x562a595ce270 2024-04-24T06:54:07.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.903+0000 7ff7f65e8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562a5952f190 con 0x562a595cd950 2024-04-24T06:54:07.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.904+0000 7ff7ebe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a595d8940 0x562a595dad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:07.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.904+0000 7ff7ebe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a595d8940 0x562a595dad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48302/0 (socket says 172.21.15.5:48302) 2024-04-24T06:54:07.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.904+0000 7ff7ebe00640 1 -- 172.21.15.5:0/1404781172 learned_addr learned my addr 172.21.15.5:0/1404781172 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:07.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.904+0000 7ff7f1600640 1 --2- 172.21.15.5:0/1404781172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a595cd950 0x562a595cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:07.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.904+0000 7ff7f0c00640 1 --2- 172.21.15.5:0/1404781172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a595ce270 0x562a595d8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:07.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.904+0000 7ff7ebe00640 1 -- 172.21.15.5:0/1404781172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a595cd950 msgr2=0x562a595cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:07.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.904+0000 7ff7ebe00640 1 --2- 172.21.15.5:0/1404781172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a595cd950 0x562a595cdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.904+0000 7ff7ebe00640 1 -- 172.21.15.5:0/1404781172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a595ce270 msgr2=0x562a595d8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:07.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.904+0000 7ff7ebe00640 1 --2- 172.21.15.5:0/1404781172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a595ce270 0x562a595d8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.904+0000 7ff7ebe00640 1 -- 172.21.15.5:0/1404781172 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a59596310 con 0x562a595d8940 2024-04-24T06:54:07.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.905+0000 7ff7ebe00640 1 --2- 172.21.15.5:0/1404781172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a595d8940 0x562a595dad30 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8011510 tx=0x7ff7d80119d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a8948a74c0e3ff server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:07.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.905+0000 7ff7e9600640 1 --2- 172.21.15.5:0/3373758647 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ff7d40a3d20 0x7ff7d40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:07.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.905+0000 7ff7ddc00640 1 -- 172.21.15.5:0/1404781172 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d801e070 con 0x562a595d8940 2024-04-24T06:54:07.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.905+0000 7ff7ddc00640 1 -- 172.21.15.5:0/1404781172 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7d80035f0 con 0x562a595d8940 2024-04-24T06:54:07.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.905+0000 7ff7ddc00640 1 -- 172.21.15.5:0/1404781172 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d800f820 con 0x562a595d8940 2024-04-24T06:54:07.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.905+0000 7ff7e9600640 1 -- 172.21.15.5:0/3373758647 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7ff7e00efd00 con 0x7ff7ec104440 2024-04-24T06:54:07.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.905+0000 7ff7f1600640 1 --2- 172.21.15.5:0/3373758647 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ff7d40a3d20 0x7ff7d40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:07.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.906+0000 7ff7f65e8740 1 -- 172.21.15.5:0/1404781172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a595d8940 msgr2=0x562a595dad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:07.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.906+0000 7ff7f65e8740 1 --2- 172.21.15.5:0/1404781172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a595d8940 0x562a595dad30 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8011510 tx=0x7ff7d80119d0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.906+0000 7ff7f1600640 1 --2- 172.21.15.5:0/3373758647 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ff7d40a3d20 0x7ff7d40a61e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4005fd0 tx=0x7ff7e4011040 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:07.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.906+0000 7ff7f65e8740 1 -- 172.21.15.5:0/1404781172 shutdown_connections 2024-04-24T06:54:07.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.906+0000 7ff7f65e8740 1 --2- 172.21.15.5:0/1404781172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a595d8940 0x562a595dad30 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.906+0000 7ff7f65e8740 1 --2- 172.21.15.5:0/1404781172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a595ce270 0x562a595d8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.906+0000 7ff7f65e8740 1 --2- 172.21.15.5:0/1404781172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a595cd950 0x562a595cdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.906+0000 7ff7f65e8740 1 -- 172.21.15.5:0/1404781172 >> 172.21.15.5:0/1404781172 conn(0x562a595c9c70 msgr2=0x562a595ca050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:07.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.906+0000 7ff7f65e8740 1 -- 172.21.15.5:0/1404781172 shutdown_connections 2024-04-24T06:54:07.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.906+0000 7ff7f65e8740 1 -- 172.21.15.5:0/1404781172 wait complete. 2024-04-24T06:54:07.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.906+0000 7ff7f65e8740 1 -- start start 2024-04-24T06:54:07.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.907+0000 7ff7f65e8740 1 Processor -- start 2024-04-24T06:54:07.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.907+0000 7ff7f65e8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a595cd950 0x562a595d83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:07.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.907+0000 7ff7f65e8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a595ce270 0x562a595d56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:07.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.907+0000 7ff7f65e8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a595d8940 0x562a595d5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:07.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.907+0000 7ff7f65e8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562a595db7d0 con 0x562a595cd950 2024-04-24T06:54:07.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.907+0000 7ff7f65e8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562a594ec810 con 0x562a595d8940 2024-04-24T06:54:07.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.907+0000 7ff7f65e8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562a595db950 con 0x562a595ce270 2024-04-24T06:54:07.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.907+0000 7ff7f65e8740 10 client.0 _ll_register_callbacks cb 0x562a594ba140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:54:07.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.907+0000 7ff7f0c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a595cd950 0x562a595d83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:07.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.908+0000 7ff7f0c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a595cd950 0x562a595d83b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48310/0 (socket says 172.21.15.5:48310) 2024-04-24T06:54:07.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.908+0000 7ff7f0c00640 1 -- 172.21.15.5:0/2849648670 learned_addr learned my addr 172.21.15.5:0/2849648670 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:07.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.908+0000 7ff7ebe00640 1 --2- 172.21.15.5:0/2849648670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a595ce270 0x562a595d56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:07.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.908+0000 7ff7f1600640 1 --2- 172.21.15.5:0/2849648670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a595d8940 0x562a595d5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:07.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.908+0000 7ff7f0c00640 1 -- 172.21.15.5:0/2849648670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a595ce270 msgr2=0x562a595d56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:07.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.908+0000 7ff7f0c00640 1 --2- 172.21.15.5:0/2849648670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a595ce270 0x562a595d56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.908+0000 7ff7f0c00640 1 -- 172.21.15.5:0/2849648670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a595d8940 msgr2=0x562a595d5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:07.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.908+0000 7ff7f0c00640 1 --2- 172.21.15.5:0/2849648670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a595d8940 0x562a595d5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.908+0000 7ff7f0c00640 1 -- 172.21.15.5:0/2849648670 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a594fd3f0 con 0x562a595cd950 2024-04-24T06:54:07.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.908+0000 7ff7f1600640 1 --2- 172.21.15.5:0/2849648670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a595d8940 0x562a595d5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:54:07.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.908+0000 7ff7f0c00640 1 --2- 172.21.15.5:0/2849648670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a595cd950 0x562a595d83b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0009780 tx=0x7ff7e0044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e4fa84946260b8e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:07.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.908+0000 7ff7d2000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:54:07.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.908+0000 7ff7d2000640 1 -- 172.21.15.5:0/2849648670 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e00bb8e0 con 0x562a595cd950 2024-04-24T06:54:07.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.909+0000 7ff7d2000640 1 -- 172.21.15.5:0/2849648670 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7ff7e00bc200 con 0x562a595cd950 2024-04-24T06:54:07.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.909+0000 7ff7f65e8740 1 -- 172.21.15.5:0/2849648670 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7d80035f0 con 0x562a595cd950 2024-04-24T06:54:07.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.909+0000 7ff7d2000640 1 -- 172.21.15.5:0/2849648670 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff7e00ed8d0 con 0x562a595cd950 2024-04-24T06:54:07.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.909+0000 7ff7f65e8740 10 client.17229 fetch_fsmap learned FSMap version 27 2024-04-24T06:54:07.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.909+0000 7ff7f65e8740 1 -- 172.21.15.5:0/2849648670 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562a592453f0 con 0x562a595cd950 2024-04-24T06:54:07.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.909+0000 7ff7d2000640 1 -- 172.21.15.5:0/2849648670 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7d80035f0 con 0x562a595cd950 2024-04-24T06:54:07.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.909+0000 7ff7d2000640 1 -- 172.21.15.5:0/2849648670 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7ff7e003a070 con 0x562a595cd950 2024-04-24T06:54:07.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.909+0000 7ff7f65e8740 10 client.17229 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:54:07.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.909+0000 7ff7f65e8740 10 client.17229 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-24T06:54:07.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.909+0000 7ff7f65e8740 20 client.17229 populate_metadata read hostname 'smithi005' 2024-04-24T06:54:07.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.909+0000 7ff7f65e8740 1 --2- 172.21.15.5:0/2849648670 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x562a595e8af0 0x562a595eaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:07.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.909+0000 7ff7f65e8740 4 client.17229 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:54:07.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.909+0000 7ff7f65e8740 1 -- 172.21.15.5:0/2849648670 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a59540bf0 con 0x562a595e8af0 2024-04-24T06:54:07.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.910+0000 7ff7ebe00640 1 --2- 172.21.15.5:0/2849648670 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x562a595e8af0 0x562a595eaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:07.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.910+0000 7ff7ebe00640 1 --2- 172.21.15.5:0/2849648670 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x562a595e8af0 0x562a595eaf20 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dbbf0763c603c0cd server_cookie=ed7e61cfd2997bdf in_seq=0 out_seq=0 2024-04-24T06:54:07.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.910+0000 7ff7d2000640 10 client.17229 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:54:07.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.911+0000 7ff7d2000640 1 -- 172.21.15.5:0/2849648670 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562a59540bf0 con 0x562a595e8af0 2024-04-24T06:54:07.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.911+0000 7ff7d2000640 10 client.17229 handle_command_reply: tid=0 2024-04-24T06:54:07.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.911+0000 7ff7f65e8740 1 client.17229 shutdown 2024-04-24T06:54:07.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.911+0000 7ff7f65e8740 1 -- 172.21.15.5:0/2849648670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a595cd950 msgr2=0x562a595d83b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:07.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.911+0000 7ff7f65e8740 1 --2- 172.21.15.5:0/2849648670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a595cd950 0x562a595d83b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0009780 tx=0x7ff7e0044000 comp rx=0 tx=0).stop 2024-04-24T06:54:07.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.911+0000 7ff7f65e8740 1 -- 172.21.15.5:0/2849648670 shutdown_connections 2024-04-24T06:54:07.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.911+0000 7ff7f65e8740 1 --2- 172.21.15.5:0/2849648670 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x562a595e8af0 0x562a595eaf20 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.911+0000 7ff7f65e8740 1 --2- 172.21.15.5:0/2849648670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a595d8940 0x562a595d5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.911+0000 7ff7f65e8740 1 --2- 172.21.15.5:0/2849648670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a595ce270 0x562a595d56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.911+0000 7ff7f65e8740 1 --2- 172.21.15.5:0/2849648670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a595cd950 0x562a595d83b0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.911+0000 7ff7f65e8740 1 -- 172.21.15.5:0/2849648670 >> 172.21.15.5:0/2849648670 conn(0x562a595c9c70 msgr2=0x562a595cfa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:07.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.911+0000 7ff7d2000640 0 client.17229 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:54:07.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.912+0000 7ff7f65e8740 1 -- 172.21.15.5:0/2849648670 shutdown_connections 2024-04-24T06:54:07.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.912+0000 7ff7f65e8740 1 -- 172.21.15.5:0/2849648670 wait complete. 2024-04-24T06:54:07.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.912+0000 7ff7f65e8740 20 client.17229 trim_cache size 0 max 16384 2024-04-24T06:54:07.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.927+0000 7ff7f65e8740 1 Processor -- start 2024-04-24T06:54:07.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.927+0000 7ff7f65e8740 1 -- start start 2024-04-24T06:54:07.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.927+0000 7ff7f65e8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a595cd950 0x562a595d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:07.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.927+0000 7ff7f65e8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a595ce270 0x562a595d4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:07.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.927+0000 7ff7f65e8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a595d8940 0x562a595d7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:07.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.927+0000 7ff7f65e8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ec1b2810 con 0x562a595cd950 2024-04-24T06:54:07.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.927+0000 7ff7f65e8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ec1b00e0 con 0x562a595d8940 2024-04-24T06:54:07.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.927+0000 7ff7f65e8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ec1adf50 con 0x562a595ce270 2024-04-24T06:54:07.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.927+0000 7ff7f0c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a595cd950 0x562a595d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:07.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.927+0000 7ff7f0c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a595cd950 0x562a595d00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48322/0 (socket says 172.21.15.5:48322) 2024-04-24T06:54:07.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.927+0000 7ff7f0c00640 1 -- 172.21.15.5:0/4107836764 learned_addr learned my addr 172.21.15.5:0/4107836764 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:07.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.928+0000 7ff7ebe00640 1 --2- 172.21.15.5:0/4107836764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a595ce270 0x562a595d4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:07.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.928+0000 7ff7f1600640 1 --2- 172.21.15.5:0/4107836764 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a595d8940 0x562a595d7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:07.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.928+0000 7ff7f0c00640 1 -- 172.21.15.5:0/4107836764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a595ce270 msgr2=0x562a595d4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:07.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.928+0000 7ff7f0c00640 1 --2- 172.21.15.5:0/4107836764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a595ce270 0x562a595d4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.928+0000 7ff7f0c00640 1 -- 172.21.15.5:0/4107836764 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a595d8940 msgr2=0x562a595d7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:07.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.928+0000 7ff7f0c00640 1 --2- 172.21.15.5:0/4107836764 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a595d8940 0x562a595d7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.928+0000 7ff7f0c00640 1 -- 172.21.15.5:0/4107836764 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a5930f7d0 con 0x562a595cd950 2024-04-24T06:54:07.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.928+0000 7ff7ebe00640 1 --2- 172.21.15.5:0/4107836764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a595ce270 0x562a595d4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:54:07.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.928+0000 7ff7f1600640 1 --2- 172.21.15.5:0/4107836764 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a595d8940 0x562a595d7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:54:07.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.928+0000 7ff7f0c00640 1 --2- 172.21.15.5:0/4107836764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a595cd950 0x562a595d00b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff7e00bbaa0 tx=0x7ff7e00edbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89af30d672a43a47 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:07.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.928+0000 7ff7dd200640 1 -- 172.21.15.5:0/4107836764 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e00bb700 con 0x562a595cd950 2024-04-24T06:54:07.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.928+0000 7ff7dd200640 1 -- 172.21.15.5:0/4107836764 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7e00f1070 con 0x562a595cd950 2024-04-24T06:54:07.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.928+0000 7ff7dd200640 1 -- 172.21.15.5:0/4107836764 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e00bc260 con 0x562a595cd950 2024-04-24T06:54:07.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.929+0000 7ff7f65e8740 1 -- 172.21.15.5:0/4107836764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a595cd950 msgr2=0x562a595d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:07.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.929+0000 7ff7f65e8740 1 --2- 172.21.15.5:0/4107836764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a595cd950 0x562a595d00b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff7e00bbaa0 tx=0x7ff7e00edbe0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.929+0000 7ff7f65e8740 1 -- 172.21.15.5:0/4107836764 shutdown_connections 2024-04-24T06:54:07.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.929+0000 7ff7f65e8740 1 --2- 172.21.15.5:0/4107836764 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a595d8940 0x562a595d7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.929+0000 7ff7f65e8740 1 --2- 172.21.15.5:0/4107836764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a595ce270 0x562a595d4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.929+0000 7ff7f65e8740 1 --2- 172.21.15.5:0/4107836764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a595cd950 0x562a595d00b0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.929+0000 7ff7f65e8740 1 -- 172.21.15.5:0/4107836764 >> 172.21.15.5:0/4107836764 conn(0x562a595c9c70 msgr2=0x562a595e3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:07.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.929+0000 7ff7f65e8740 1 -- 172.21.15.5:0/4107836764 shutdown_connections 2024-04-24T06:54:07.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.929+0000 7ff7f65e8740 1 -- 172.21.15.5:0/4107836764 wait complete. 2024-04-24T06:54:07.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.930+0000 7ff7f65e8740 1 -- start start 2024-04-24T06:54:07.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.930+0000 7ff7f65e8740 1 Processor -- start 2024-04-24T06:54:07.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.930+0000 7ff7f65e8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a595cd950 0x562a595e10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:07.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.930+0000 7ff7f65e8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a595ce270 0x562a595d9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:07.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.930+0000 7ff7f65e8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a595d8940 0x562a595d9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:07.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.930+0000 7ff7f65e8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ec1ae5a0 con 0x562a595ce270 2024-04-24T06:54:07.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.930+0000 7ff7f65e8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ec1afa10 con 0x562a595cd950 2024-04-24T06:54:07.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.930+0000 7ff7f65e8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ec1aec70 con 0x562a595d8940 2024-04-24T06:54:07.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.931+0000 7ff7f65e8740 10 client.0 _ll_register_callbacks cb 0x7ff7e00f3530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:54:07.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.931+0000 7ff7ebe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a595ce270 0x562a595d9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:07.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.931+0000 7ff7ebe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a595ce270 0x562a595d9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48338/0 (socket says 172.21.15.5:48338) 2024-04-24T06:54:07.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.931+0000 7ff7ebe00640 1 -- 172.21.15.5:0/3215078343 learned_addr learned my addr 172.21.15.5:0/3215078343 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:07.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.931+0000 7ff7f0c00640 1 --2- 172.21.15.5:0/3215078343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a595cd950 0x562a595e10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:07.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.931+0000 7ff7f1600640 1 --2- 172.21.15.5:0/3215078343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a595d8940 0x562a595d9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:07.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.931+0000 7ff7ebe00640 1 -- 172.21.15.5:0/3215078343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a595d8940 msgr2=0x562a595d9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:07.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.931+0000 7ff7ebe00640 1 --2- 172.21.15.5:0/3215078343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a595d8940 0x562a595d9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.931+0000 7ff7ebe00640 1 -- 172.21.15.5:0/3215078343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a595cd950 msgr2=0x562a595e10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:07.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.931+0000 7ff7ebe00640 1 --2- 172.21.15.5:0/3215078343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a595cd950 0x562a595e10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.931+0000 7ff7ebe00640 1 -- 172.21.15.5:0/3215078343 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a595dec00 con 0x562a595ce270 2024-04-24T06:54:07.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.932+0000 7ff7ebe00640 1 --2- 172.21.15.5:0/3215078343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a595ce270 0x562a595d9560 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff7d801c040 tx=0x7ff7d80093b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f220c22cc0d8f282 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:07.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.932+0000 7ff7d2000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:54:07.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.932+0000 7ff7d2000640 1 -- 172.21.15.5:0/3215078343 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d801e040 con 0x562a595ce270 2024-04-24T06:54:07.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.932+0000 7ff7d2000640 1 -- 172.21.15.5:0/3215078343 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7ff7d80041b0 con 0x562a595ce270 2024-04-24T06:54:07.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.932+0000 7ff7f65e8740 1 -- 172.21.15.5:0/3215078343 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7e00f1070 con 0x562a595ce270 2024-04-24T06:54:07.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.932+0000 7ff7d2000640 1 -- 172.21.15.5:0/3215078343 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff7d803a050 con 0x562a595ce270 2024-04-24T06:54:07.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.932+0000 7ff7f65e8740 10 client.17235 fetch_fsmap learned FSMap version 27 2024-04-24T06:54:07.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.932+0000 7ff7f65e8740 1 -- 172.21.15.5:0/3215078343 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562a5964f160 con 0x562a595ce270 2024-04-24T06:54:07.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.932+0000 7ff7d2000640 1 -- 172.21.15.5:0/3215078343 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7e00f1070 con 0x562a595ce270 2024-04-24T06:54:07.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.933+0000 7ff7d2000640 1 -- 172.21.15.5:0/3215078343 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7ff7d8004980 con 0x562a595ce270 2024-04-24T06:54:07.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.933+0000 7ff7f65e8740 10 client.17235 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:54:07.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.933+0000 7ff7f65e8740 10 client.17235 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-24T06:54:07.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.933+0000 7ff7f65e8740 20 client.17235 populate_metadata read hostname 'smithi005' 2024-04-24T06:54:07.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.933+0000 7ff7f65e8740 1 --2- 172.21.15.5:0/3215078343 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x562a5964f780 0x562a59651ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:07.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.933+0000 7ff7f65e8740 4 client.17235 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:54:07.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.933+0000 7ff7f65e8740 1 -- 172.21.15.5:0/3215078343 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562a5953aa60 con 0x562a5964f780 2024-04-24T06:54:07.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.933+0000 7ff7f0c00640 1 --2- 172.21.15.5:0/3215078343 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x562a5964f780 0x562a59651ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:07.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.934+0000 7ff7f0c00640 1 --2- 172.21.15.5:0/3215078343 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x562a5964f780 0x562a59651ba0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dbb98084f066a751 server_cookie=a0822fbd8412771c in_seq=0 out_seq=0 2024-04-24T06:54:07.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.934+0000 7ff7d2000640 10 client.17235 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:54:07.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.934+0000 7ff7d2000640 1 -- 172.21.15.5:0/3215078343 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562a5953aa60 con 0x562a5964f780 2024-04-24T06:54:07.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.934+0000 7ff7d2000640 10 client.17235 handle_command_reply: tid=0 2024-04-24T06:54:07.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.934+0000 7ff7f65e8740 1 client.17235 shutdown 2024-04-24T06:54:07.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.934+0000 7ff7f65e8740 1 -- 172.21.15.5:0/3215078343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a595ce270 msgr2=0x562a595d9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:07.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.934+0000 7ff7f65e8740 1 --2- 172.21.15.5:0/3215078343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a595ce270 0x562a595d9560 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff7d801c040 tx=0x7ff7d80093b0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.934+0000 7ff7f65e8740 1 -- 172.21.15.5:0/3215078343 shutdown_connections 2024-04-24T06:54:07.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.934+0000 7ff7f65e8740 1 --2- 172.21.15.5:0/3215078343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a595d8940 0x562a595d9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.934+0000 7ff7f65e8740 1 --2- 172.21.15.5:0/3215078343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a595ce270 0x562a595d9560 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.934+0000 7ff7f65e8740 1 --2- 172.21.15.5:0/3215078343 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x562a5964f780 0x562a59651ba0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.934+0000 7ff7f65e8740 1 --2- 172.21.15.5:0/3215078343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a595cd950 0x562a595e10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.934+0000 7ff7f65e8740 1 -- 172.21.15.5:0/3215078343 >> 172.21.15.5:0/3215078343 conn(0x562a595c9c70 msgr2=0x562a595cf0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:07.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.934+0000 7ff7d2000640 0 client.17235 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:54:07.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.934+0000 7ff7f65e8740 1 -- 172.21.15.5:0/3215078343 shutdown_connections 2024-04-24T06:54:07.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.934+0000 7ff7f65e8740 1 -- 172.21.15.5:0/3215078343 wait complete. 2024-04-24T06:54:07.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.934+0000 7ff7f65e8740 20 client.17235 trim_cache size 0 max 16384 2024-04-24T06:54:07.943 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:54:07.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.935+0000 7ff7f3600640 1 -- 172.21.15.5:0/3373758647 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ff7d40a3d20 msgr2=0x7ff7d40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:07.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.935+0000 7ff7f3600640 1 --2- 172.21.15.5:0/3373758647 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ff7d40a3d20 0x7ff7d40a61e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4005fd0 tx=0x7ff7e4011040 comp rx=0 tx=0).stop 2024-04-24T06:54:07.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.935+0000 7ff7f3600640 1 -- 172.21.15.5:0/3373758647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7ec104440 msgr2=0x7ff7ec1a0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:07.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.935+0000 7ff7f3600640 1 --2- 172.21.15.5:0/3373758647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7ec104440 0x7ff7ec1a0790 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff7e002a860 tx=0x7ff7e0033b60 comp rx=0 tx=0).stop 2024-04-24T06:54:07.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.935+0000 7ff7f3600640 1 -- 172.21.15.5:0/3373758647 shutdown_connections 2024-04-24T06:54:07.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.935+0000 7ff7f3600640 1 --2- 172.21.15.5:0/3373758647 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7ff7d40a3d20 0x7ff7d40a61e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.935+0000 7ff7f3600640 1 --2- 172.21.15.5:0/3373758647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7ec19a940 0x7ff7ec19adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.935+0000 7ff7f3600640 1 --2- 172.21.15.5:0/3373758647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7ec109510 0x7ff7ec1a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.935+0000 7ff7f3600640 1 --2- 172.21.15.5:0/3373758647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7ec104440 0x7ff7ec1a0790 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:07.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.935+0000 7ff7f3600640 1 -- 172.21.15.5:0/3373758647 >> 172.21.15.5:0/3373758647 conn(0x7ff7ec0ffcc0 msgr2=0x7ff7ec101030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:07.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.935+0000 7ff7f3600640 1 -- 172.21.15.5:0/3373758647 shutdown_connections 2024-04-24T06:54:07.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:07.935+0000 7ff7f3600640 1 -- 172.21.15.5:0/3373758647 wait complete. 2024-04-24T06:54:07.955 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T06:54:08.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.171+0000 7f5a3fe00640 1 -- 172.21.15.5:0/2464942058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a40108ee0 msgr2=0x7f5a401152b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:08.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.171+0000 7f5a3fe00640 1 --2- 172.21.15.5:0/2464942058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a40108ee0 0x7f5a401152b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5a38009cd0 tx=0x7f5a3802a980 comp rx=0 tx=0).stop 2024-04-24T06:54:08.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.171+0000 7f5a3fe00640 1 -- 172.21.15.5:0/2464942058 shutdown_connections 2024-04-24T06:54:08.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.171+0000 7f5a3fe00640 1 --2- 172.21.15.5:0/2464942058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a40108ee0 0x7f5a401152b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.171+0000 7f5a3fe00640 1 --2- 172.21.15.5:0/2464942058 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a40108540 0x7f5a401089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.171+0000 7f5a3fe00640 1 --2- 172.21.15.5:0/2464942058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a4010e4f0 0x7f5a4010e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.171+0000 7f5a3fe00640 1 -- 172.21.15.5:0/2464942058 >> 172.21.15.5:0/2464942058 conn(0x7f5a40077e10 msgr2=0x7f5a40106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:08.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.171+0000 7f5a3fe00640 1 -- 172.21.15.5:0/2464942058 shutdown_connections 2024-04-24T06:54:08.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.172+0000 7f5a3fe00640 1 -- 172.21.15.5:0/2464942058 wait complete. 2024-04-24T06:54:08.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.173+0000 7f5a3fe00640 1 Processor -- start 2024-04-24T06:54:08.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.173+0000 7f5a3fe00640 1 -- start start 2024-04-24T06:54:08.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.173+0000 7f5a3fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a40108540 0x7f5a401a4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:08.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.173+0000 7f5a3fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a40108ee0 0x7f5a401a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:08.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.173+0000 7f5a3fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a4010e4f0 0x7f5a4019eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:08.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.174+0000 7f5a3fe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5a40118810 con 0x7f5a40108ee0 2024-04-24T06:54:08.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.174+0000 7f5a3fe00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5a40118690 con 0x7f5a40108540 2024-04-24T06:54:08.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.174+0000 7f5a3fe00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5a40118990 con 0x7f5a4010e4f0 2024-04-24T06:54:08.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.174+0000 7f5a3e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a40108ee0 0x7f5a401a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:08.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.174+0000 7f5a3ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a40108540 0x7f5a401a4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:08.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.174+0000 7f5a3e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a40108ee0 0x7f5a401a4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48360/0 (socket says 172.21.15.5:48360) 2024-04-24T06:54:08.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.174+0000 7f5a3e000640 1 -- 172.21.15.5:0/1846652157 learned_addr learned my addr 172.21.15.5:0/1846652157 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:08.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.174+0000 7f5a3f400640 1 --2- 172.21.15.5:0/1846652157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a4010e4f0 0x7f5a4019eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:08.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.174+0000 7f5a3ea00640 1 -- 172.21.15.5:0/1846652157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a4010e4f0 msgr2=0x7f5a4019eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:08.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.174+0000 7f5a3ea00640 1 --2- 172.21.15.5:0/1846652157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a4010e4f0 0x7f5a4019eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.174+0000 7f5a3ea00640 1 -- 172.21.15.5:0/1846652157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a40108ee0 msgr2=0x7f5a401a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:08.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.174+0000 7f5a3ea00640 1 --2- 172.21.15.5:0/1846652157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a40108ee0 0x7f5a401a4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.174+0000 7f5a3ea00640 1 -- 172.21.15.5:0/1846652157 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a4019f3f0 con 0x7f5a40108540 2024-04-24T06:54:08.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.174+0000 7f5a3e000640 1 --2- 172.21.15.5:0/1846652157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a40108ee0 0x7f5a401a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:54:08.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.175+0000 7f5a3ea00640 1 --2- 172.21.15.5:0/1846652157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a40108540 0x7f5a401a4a70 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5a3000e9f0 tx=0x7f5a3000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:08.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.175+0000 7f5a37400640 1 -- 172.21.15.5:0/1846652157 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3000ccc0 con 0x7f5a40108540 2024-04-24T06:54:08.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.175+0000 7f5a37400640 1 -- 172.21.15.5:0/1846652157 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a30010910 con 0x7f5a40108540 2024-04-24T06:54:08.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.175+0000 7f5a37400640 1 -- 172.21.15.5:0/1846652157 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a30018960 con 0x7f5a40108540 2024-04-24T06:54:08.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.175+0000 7f5a3fe00640 1 -- 172.21.15.5:0/1846652157 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a4019f6e0 con 0x7f5a40108540 2024-04-24T06:54:08.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.175+0000 7f5a3fe00640 1 -- 172.21.15.5:0/1846652157 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a401ab9f0 con 0x7f5a40108540 2024-04-24T06:54:08.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.179+0000 7f5a48823740 1 Processor -- start 2024-04-24T06:54:08.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.179+0000 7f5a37400640 1 -- 172.21.15.5:0/1846652157 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f5a30010ac0 con 0x7f5a40108540 2024-04-24T06:54:08.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.179+0000 7f5a48823740 1 -- start start 2024-04-24T06:54:08.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.179+0000 7f5a48823740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561820cecb40 0x561820cecf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:08.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.179+0000 7f5a48823740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561820ced460 0x561820cf75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:08.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.179+0000 7f5a48823740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561820cf7b30 0x561820cf9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:08.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.179+0000 7f5a48823740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561820c521d0 con 0x561820cf7b30 2024-04-24T06:54:08.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.179+0000 7f5a48823740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561820bf8450 con 0x561820ced460 2024-04-24T06:54:08.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.179+0000 7f5a48823740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561820c4e190 con 0x561820cecb40 2024-04-24T06:54:08.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.179+0000 7f5a3e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561820cf7b30 0x561820cf9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:08.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.179+0000 7f5a3f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561820cecb40 0x561820cecf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:08.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.179+0000 7f5a3ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561820ced460 0x561820cf75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:08.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.180+0000 7f5a3e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561820cf7b30 0x561820cf9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48366/0 (socket says 172.21.15.5:48366) 2024-04-24T06:54:08.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.180+0000 7f5a3e000640 1 -- 172.21.15.5:0/2986339332 learned_addr learned my addr 172.21.15.5:0/2986339332 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:08.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.180+0000 7f5a3f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561820cecb40 0x561820cecf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:54634/0 (socket says 172.21.15.5:54634) 2024-04-24T06:54:08.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.180+0000 7f5a3e000640 1 -- 172.21.15.5:0/2986339332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561820cecb40 msgr2=0x561820cecf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:08.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.180+0000 7f5a3e000640 1 --2- 172.21.15.5:0/2986339332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561820cecb40 0x561820cecf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.180+0000 7f5a3e000640 1 -- 172.21.15.5:0/2986339332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561820ced460 msgr2=0x561820cf75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:08.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.180+0000 7f5a3e000640 1 --2- 172.21.15.5:0/2986339332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561820ced460 0x561820cf75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.180+0000 7f5a3e000640 1 -- 172.21.15.5:0/2986339332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561820cb5310 con 0x561820cf7b30 2024-04-24T06:54:08.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.180+0000 7f5a37400640 1 --2- 172.21.15.5:0/1846652157 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f5a240a3da0 0x7f5a240a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:08.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.180+0000 7f5a3e000640 1 --2- 172.21.15.5:0/2986339332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561820cf7b30 0x561820cf9f20 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c0115f0 tx=0x7f5a2c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d262227d0807576c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:08.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.181+0000 7f5a37400640 1 -- 172.21.15.5:0/1846652157 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f5a300d0ac0 con 0x7f5a40108540 2024-04-24T06:54:08.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.181+0000 7f5a3f400640 1 --2- 172.21.15.5:0/1846652157 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f5a240a3da0 0x7f5a240a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:08.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.181+0000 7f5a2be00640 1 -- 172.21.15.5:0/2986339332 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c01e070 con 0x561820cf7b30 2024-04-24T06:54:08.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.181+0000 7f5a2be00640 1 -- 172.21.15.5:0/2986339332 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a2c00fba0 con 0x561820cf7b30 2024-04-24T06:54:08.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.181+0000 7f5a2be00640 1 -- 172.21.15.5:0/2986339332 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c019cd0 con 0x561820cf7b30 2024-04-24T06:54:08.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.181+0000 7f5a48823740 1 -- 172.21.15.5:0/2986339332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561820cf7b30 msgr2=0x561820cf9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:08.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.181+0000 7f5a48823740 1 --2- 172.21.15.5:0/2986339332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561820cf7b30 0x561820cf9f20 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c0115f0 tx=0x7f5a2c011ab0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.182+0000 7f5a3f400640 1 --2- 172.21.15.5:0/1846652157 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f5a240a3da0 0x7f5a240a6260 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5a38004af0 tx=0x7f5a38005d80 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:08.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.182+0000 7f5a48823740 1 -- 172.21.15.5:0/2986339332 shutdown_connections 2024-04-24T06:54:08.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.182+0000 7f5a48823740 1 --2- 172.21.15.5:0/2986339332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561820cf7b30 0x561820cf9f20 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.182+0000 7f5a48823740 1 --2- 172.21.15.5:0/2986339332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561820ced460 0x561820cf75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.182+0000 7f5a48823740 1 --2- 172.21.15.5:0/2986339332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561820cecb40 0x561820cecf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.182+0000 7f5a48823740 1 -- 172.21.15.5:0/2986339332 >> 172.21.15.5:0/2986339332 conn(0x561820ce8e60 msgr2=0x561820ce9240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:08.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.182+0000 7f5a48823740 1 -- 172.21.15.5:0/2986339332 shutdown_connections 2024-04-24T06:54:08.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.182+0000 7f5a48823740 1 -- 172.21.15.5:0/2986339332 wait complete. 2024-04-24T06:54:08.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.182+0000 7f5a48823740 1 -- start start 2024-04-24T06:54:08.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.182+0000 7f5a48823740 1 Processor -- start 2024-04-24T06:54:08.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.183+0000 7f5a48823740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561820cecb40 0x561820cf75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:08.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.183+0000 7f5a48823740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561820ced460 0x561820cf4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:08.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.183+0000 7f5a48823740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561820cf7b30 0x561820cf4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:08.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.183+0000 7f5a48823740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561820cfa9c0 con 0x561820cecb40 2024-04-24T06:54:08.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.183+0000 7f5a48823740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561820c0b810 con 0x561820cf7b30 2024-04-24T06:54:08.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.183+0000 7f5a48823740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561820cfab40 con 0x561820ced460 2024-04-24T06:54:08.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.183+0000 7f5a48823740 10 client.0 _ll_register_callbacks cb 0x561820bd9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:54:08.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.183+0000 7f5a3ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561820cecb40 0x561820cf75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:08.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.183+0000 7f5a3e000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561820ced460 0x561820cf4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:08.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.183+0000 7f5a3ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561820cecb40 0x561820cf75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48376/0 (socket says 172.21.15.5:48376) 2024-04-24T06:54:08.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.183+0000 7f5a3ea00640 1 -- 172.21.15.5:0/3484947080 learned_addr learned my addr 172.21.15.5:0/3484947080 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:08.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.183+0000 7f5a3f400640 1 --2- 172.21.15.5:0/3484947080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561820cf7b30 0x561820cf4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:08.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.184+0000 7f5a3ea00640 1 -- 172.21.15.5:0/3484947080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561820ced460 msgr2=0x561820cf4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:08.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.184+0000 7f5a3ea00640 1 --2- 172.21.15.5:0/3484947080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561820ced460 0x561820cf4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.184+0000 7f5a3ea00640 1 -- 172.21.15.5:0/3484947080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561820cf7b30 msgr2=0x561820cf4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:08.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.184+0000 7f5a3ea00640 1 --2- 172.21.15.5:0/3484947080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561820cf7b30 0x561820cf4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.184+0000 7f5a3ea00640 1 -- 172.21.15.5:0/3484947080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561820c1c3f0 con 0x561820cecb40 2024-04-24T06:54:08.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.184+0000 7f5a3e000640 1 --2- 172.21.15.5:0/3484947080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561820ced460 0x561820cf4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:54:08.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.184+0000 7f5a3ea00640 1 --2- 172.21.15.5:0/3484947080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561820cecb40 0x561820cf75a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5a3000e9c0 tx=0x7f5a30098e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd5b769c0a24ec84 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:08.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.184+0000 7f5a28c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:54:08.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.184+0000 7f5a28c00640 1 -- 172.21.15.5:0/3484947080 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a300bd070 con 0x561820cecb40 2024-04-24T06:54:08.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.184+0000 7f5a28c00640 1 -- 172.21.15.5:0/3484947080 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f5a300bbdb0 con 0x561820cecb40 2024-04-24T06:54:08.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.184+0000 7f5a48823740 1 -- 172.21.15.5:0/3484947080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a2c00fba0 con 0x561820cecb40 2024-04-24T06:54:08.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.184+0000 7f5a28c00640 1 -- 172.21.15.5:0/3484947080 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5a300ce630 con 0x561820cecb40 2024-04-24T06:54:08.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.184+0000 7f5a48823740 10 client.17247 fetch_fsmap learned FSMap version 27 2024-04-24T06:54:08.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.185+0000 7f5a48823740 1 -- 172.21.15.5:0/3484947080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5618209643f0 con 0x561820cecb40 2024-04-24T06:54:08.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.185+0000 7f5a28c00640 1 -- 172.21.15.5:0/3484947080 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a2c00fba0 con 0x561820cecb40 2024-04-24T06:54:08.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.185+0000 7f5a28c00640 1 -- 172.21.15.5:0/3484947080 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f5a30018b00 con 0x561820cecb40 2024-04-24T06:54:08.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.185+0000 7f5a48823740 10 client.17247 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:54:08.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.185+0000 7f5a48823740 10 client.17247 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T06:54:08.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.185+0000 7f5a48823740 20 client.17247 populate_metadata read hostname 'smithi005' 2024-04-24T06:54:08.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.185+0000 7f5a48823740 1 --2- 172.21.15.5:0/3484947080 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x561820d07cc0 0x561820d0a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:08.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.185+0000 7f5a48823740 4 client.17247 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:54:08.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.185+0000 7f5a48823740 1 -- 172.21.15.5:0/3484947080 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561820c5fbf0 con 0x561820d07cc0 2024-04-24T06:54:08.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.186+0000 7f5a3e000640 1 --2- 172.21.15.5:0/3484947080 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x561820d07cc0 0x561820d0a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:08.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.186+0000 7f5a3e000640 1 --2- 172.21.15.5:0/3484947080 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x561820d07cc0 0x561820d0a110 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=632a654b5c1856a4 server_cookie=14f98734405149d8 in_seq=0 out_seq=0 2024-04-24T06:54:08.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.186+0000 7f5a28c00640 10 client.17247 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:54:08.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.187+0000 7f5a28c00640 1 -- 172.21.15.5:0/3484947080 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561820c5fbf0 con 0x561820d07cc0 2024-04-24T06:54:08.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.187+0000 7f5a28c00640 10 client.17247 handle_command_reply: tid=0 2024-04-24T06:54:08.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.187+0000 7f5a48823740 1 client.17247 shutdown 2024-04-24T06:54:08.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.187+0000 7f5a48823740 1 -- 172.21.15.5:0/3484947080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561820cecb40 msgr2=0x561820cf75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:08.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.187+0000 7f5a48823740 1 --2- 172.21.15.5:0/3484947080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561820cecb40 0x561820cf75a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5a3000e9c0 tx=0x7f5a30098e00 comp rx=0 tx=0).stop 2024-04-24T06:54:08.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.187+0000 7f5a48823740 1 -- 172.21.15.5:0/3484947080 shutdown_connections 2024-04-24T06:54:08.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.187+0000 7f5a48823740 1 --2- 172.21.15.5:0/3484947080 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x561820d07cc0 0x561820d0a110 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.187+0000 7f5a48823740 1 --2- 172.21.15.5:0/3484947080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561820cf7b30 0x561820cf4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.187+0000 7f5a48823740 1 --2- 172.21.15.5:0/3484947080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561820ced460 0x561820cf4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.187+0000 7f5a48823740 1 --2- 172.21.15.5:0/3484947080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561820cecb40 0x561820cf75a0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.187+0000 7f5a48823740 1 -- 172.21.15.5:0/3484947080 >> 172.21.15.5:0/3484947080 conn(0x561820ce8e60 msgr2=0x561820ceec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:08.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.187+0000 7f5a28c00640 0 client.17247 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:54:08.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.187+0000 7f5a48823740 1 -- 172.21.15.5:0/3484947080 shutdown_connections 2024-04-24T06:54:08.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.188+0000 7f5a48823740 1 -- 172.21.15.5:0/3484947080 wait complete. 2024-04-24T06:54:08.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.188+0000 7f5a48823740 20 client.17247 trim_cache size 0 max 16384 2024-04-24T06:54:08.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.202+0000 7f5a48823740 1 Processor -- start 2024-04-24T06:54:08.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.203+0000 7f5a48823740 1 -- start start 2024-04-24T06:54:08.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.203+0000 7f5a48823740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561820cecb40 0x561820d038b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:08.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.203+0000 7f5a48823740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561820ced460 0x561820cf3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:08.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.203+0000 7f5a48823740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561820cf7b30 0x561820cf70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:08.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.203+0000 7f5a48823740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5a401b6b10 con 0x561820ced460 2024-04-24T06:54:08.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.203+0000 7f5a48823740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5a401b43e0 con 0x561820cecb40 2024-04-24T06:54:08.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.203+0000 7f5a48823740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5a401b2250 con 0x561820cf7b30 2024-04-24T06:54:08.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.203+0000 7f5a3e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561820ced460 0x561820cf3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:08.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.203+0000 7f5a3e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561820ced460 0x561820cf3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48386/0 (socket says 172.21.15.5:48386) 2024-04-24T06:54:08.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.203+0000 7f5a3e000640 1 -- 172.21.15.5:0/24768033 learned_addr learned my addr 172.21.15.5:0/24768033 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:08.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.203+0000 7f5a3ea00640 1 --2- 172.21.15.5:0/24768033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561820cecb40 0x561820d038b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:08.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.203+0000 7f5a3f400640 1 --2- 172.21.15.5:0/24768033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561820cf7b30 0x561820cf70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:08.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.204+0000 7f5a3ea00640 1 -- 172.21.15.5:0/24768033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561820cf7b30 msgr2=0x561820cf70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:08.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.204+0000 7f5a3ea00640 1 --2- 172.21.15.5:0/24768033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561820cf7b30 0x561820cf70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.204+0000 7f5a3ea00640 1 -- 172.21.15.5:0/24768033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561820ced460 msgr2=0x561820cf3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:08.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.204+0000 7f5a3ea00640 1 --2- 172.21.15.5:0/24768033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561820ced460 0x561820cf3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.204+0000 7f5a3ea00640 1 -- 172.21.15.5:0/24768033 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561820a2e7d0 con 0x561820cecb40 2024-04-24T06:54:08.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.204+0000 7f5a3e000640 1 --2- 172.21.15.5:0/24768033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561820ced460 0x561820cf3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:54:08.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.204+0000 7f5a3ea00640 1 --2- 172.21.15.5:0/24768033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561820cecb40 0x561820d038b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5a30099420 tx=0x7f5a300ce280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5979ad99212d6d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:08.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.204+0000 7f5a2b400640 1 -- 172.21.15.5:0/24768033 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a300a0da0 con 0x561820cecb40 2024-04-24T06:54:08.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.204+0000 7f5a2b400640 1 -- 172.21.15.5:0/24768033 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a300a5660 con 0x561820cecb40 2024-04-24T06:54:08.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.204+0000 7f5a2b400640 1 -- 172.21.15.5:0/24768033 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a300bbe10 con 0x561820cecb40 2024-04-24T06:54:08.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.205+0000 7f5a48823740 1 -- 172.21.15.5:0/24768033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561820cecb40 msgr2=0x561820d038b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:08.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.205+0000 7f5a48823740 1 --2- 172.21.15.5:0/24768033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561820cecb40 0x561820d038b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5a30099420 tx=0x7f5a300ce280 comp rx=0 tx=0).stop 2024-04-24T06:54:08.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.205+0000 7f5a48823740 1 -- 172.21.15.5:0/24768033 shutdown_connections 2024-04-24T06:54:08.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.205+0000 7f5a48823740 1 --2- 172.21.15.5:0/24768033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561820cf7b30 0x561820cf70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.205+0000 7f5a48823740 1 --2- 172.21.15.5:0/24768033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561820ced460 0x561820cf3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.205+0000 7f5a48823740 1 --2- 172.21.15.5:0/24768033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561820cecb40 0x561820d038b0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.205+0000 7f5a48823740 1 -- 172.21.15.5:0/24768033 >> 172.21.15.5:0/24768033 conn(0x561820ce8e60 msgr2=0x561820cf3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:08.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.205+0000 7f5a48823740 1 -- 172.21.15.5:0/24768033 shutdown_connections 2024-04-24T06:54:08.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.205+0000 7f5a48823740 1 -- 172.21.15.5:0/24768033 wait complete. 2024-04-24T06:54:08.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.205+0000 7f5a48823740 1 -- start start 2024-04-24T06:54:08.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.206+0000 7f5a48823740 1 Processor -- start 2024-04-24T06:54:08.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.206+0000 7f5a48823740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561820cecb40 0x561820cf8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:08.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.206+0000 7f5a48823740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561820ced460 0x561820cf8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:08.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.206+0000 7f5a48823740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561820cf7b30 0x561820cfedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:08.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.206+0000 7f5a48823740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5a401b28a0 con 0x561820ced460 2024-04-24T06:54:08.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.206+0000 7f5a48823740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5a401b3d10 con 0x561820cecb40 2024-04-24T06:54:08.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.206+0000 7f5a48823740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5a401b2f70 con 0x561820cf7b30 2024-04-24T06:54:08.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.206+0000 7f5a48823740 10 client.0 _ll_register_callbacks cb 0x7f5a300d4260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:54:08.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.206+0000 7f5a3e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561820ced460 0x561820cf8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:08.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.206+0000 7f5a3e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561820ced460 0x561820cf8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48402/0 (socket says 172.21.15.5:48402) 2024-04-24T06:54:08.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.206+0000 7f5a3e000640 1 -- 172.21.15.5:0/4244775687 learned_addr learned my addr 172.21.15.5:0/4244775687 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:08.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.206+0000 7f5a3ea00640 1 --2- 172.21.15.5:0/4244775687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561820cecb40 0x561820cf8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:08.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.207+0000 7f5a3f400640 1 --2- 172.21.15.5:0/4244775687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561820cf7b30 0x561820cfedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:08.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.207+0000 7f5a3e000640 1 -- 172.21.15.5:0/4244775687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561820cf7b30 msgr2=0x561820cfedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:08.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.207+0000 7f5a3e000640 1 --2- 172.21.15.5:0/4244775687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561820cf7b30 0x561820cfedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.207+0000 7f5a3e000640 1 -- 172.21.15.5:0/4244775687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561820cecb40 msgr2=0x561820cf8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:08.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.207+0000 7f5a3e000640 1 --2- 172.21.15.5:0/4244775687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561820cecb40 0x561820cf8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.207+0000 7f5a3e000640 1 -- 172.21.15.5:0/4244775687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561820cf9160 con 0x561820ced460 2024-04-24T06:54:08.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.207+0000 7f5a3f400640 1 --2- 172.21.15.5:0/4244775687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561820cf7b30 0x561820cfedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:54:08.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.207+0000 7f5a3e000640 1 --2- 172.21.15.5:0/4244775687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561820ced460 0x561820cf8860 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c0115f0 tx=0x7f5a2c005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319f20f145dbd082 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:08.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.207+0000 7f5a28c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:54:08.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.207+0000 7f5a28c00640 1 -- 172.21.15.5:0/4244775687 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c01e040 con 0x561820ced460 2024-04-24T06:54:08.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.207+0000 7f5a28c00640 1 -- 172.21.15.5:0/4244775687 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f5a2c019410 con 0x561820ced460 2024-04-24T06:54:08.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.207+0000 7f5a48823740 1 -- 172.21.15.5:0/4244775687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a300a5660 con 0x561820ced460 2024-04-24T06:54:08.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.208+0000 7f5a28c00640 1 -- 172.21.15.5:0/4244775687 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5a2c039050 con 0x561820ced460 2024-04-24T06:54:08.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.208+0000 7f5a48823740 10 client.17253 fetch_fsmap learned FSMap version 27 2024-04-24T06:54:08.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.208+0000 7f5a48823740 1 -- 172.21.15.5:0/4244775687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561820d00460 con 0x561820ced460 2024-04-24T06:54:08.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.208+0000 7f5a28c00640 1 -- 172.21.15.5:0/4244775687 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a300a5660 con 0x561820ced460 2024-04-24T06:54:08.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.208+0000 7f5a28c00640 1 -- 172.21.15.5:0/4244775687 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f5a2c005040 con 0x561820ced460 2024-04-24T06:54:08.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.208+0000 7f5a48823740 10 client.17253 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:54:08.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.208+0000 7f5a48823740 10 client.17253 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T06:54:08.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.208+0000 7f5a48823740 20 client.17253 populate_metadata read hostname 'smithi005' 2024-04-24T06:54:08.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.208+0000 7f5a48823740 1 --2- 172.21.15.5:0/4244775687 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x561820d6e500 0x561820d70920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:08.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.208+0000 7f5a48823740 4 client.17253 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:54:08.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.208+0000 7f5a48823740 1 -- 172.21.15.5:0/4244775687 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561820c59a60 con 0x561820d6e500 2024-04-24T06:54:08.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.209+0000 7f5a3ea00640 1 --2- 172.21.15.5:0/4244775687 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x561820d6e500 0x561820d70920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:08.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.209+0000 7f5a3ea00640 1 --2- 172.21.15.5:0/4244775687 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x561820d6e500 0x561820d70920 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fb457f62324a0dda server_cookie=851776627a5ef409 in_seq=0 out_seq=0 2024-04-24T06:54:08.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.209+0000 7f5a28c00640 10 client.17253 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:54:08.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.209+0000 7f5a28c00640 1 -- 172.21.15.5:0/4244775687 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561820c59a60 con 0x561820d6e500 2024-04-24T06:54:08.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.209+0000 7f5a28c00640 10 client.17253 handle_command_reply: tid=0 2024-04-24T06:54:08.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.209+0000 7f5a48823740 1 client.17253 shutdown 2024-04-24T06:54:08.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.210+0000 7f5a48823740 1 -- 172.21.15.5:0/4244775687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561820ced460 msgr2=0x561820cf8860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:08.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.210+0000 7f5a48823740 1 --2- 172.21.15.5:0/4244775687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561820ced460 0x561820cf8860 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c0115f0 tx=0x7f5a2c005da0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.210+0000 7f5a48823740 1 -- 172.21.15.5:0/4244775687 shutdown_connections 2024-04-24T06:54:08.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.210+0000 7f5a48823740 1 --2- 172.21.15.5:0/4244775687 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x561820d6e500 0x561820d70920 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.210+0000 7f5a48823740 1 --2- 172.21.15.5:0/4244775687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561820cf7b30 0x561820cfedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.210+0000 7f5a48823740 1 --2- 172.21.15.5:0/4244775687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561820ced460 0x561820cf8860 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.210+0000 7f5a48823740 1 --2- 172.21.15.5:0/4244775687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561820cecb40 0x561820cf8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.210+0000 7f5a28c00640 0 client.17253 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:54:08.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.210+0000 7f5a48823740 1 -- 172.21.15.5:0/4244775687 >> 172.21.15.5:0/4244775687 conn(0x561820ce8e60 msgr2=0x561820cf39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:08.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.210+0000 7f5a48823740 1 -- 172.21.15.5:0/4244775687 shutdown_connections 2024-04-24T06:54:08.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.210+0000 7f5a48823740 1 -- 172.21.15.5:0/4244775687 wait complete. 2024-04-24T06:54:08.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.210+0000 7f5a48823740 20 client.17253 trim_cache size 0 max 16384 2024-04-24T06:54:08.219 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:54:08.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.211+0000 7f5a3fe00640 1 -- 172.21.15.5:0/1846652157 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f5a240a3da0 msgr2=0x7f5a240a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:08.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.211+0000 7f5a3fe00640 1 --2- 172.21.15.5:0/1846652157 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f5a240a3da0 0x7f5a240a6260 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5a38004af0 tx=0x7f5a38005d80 comp rx=0 tx=0).stop 2024-04-24T06:54:08.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.211+0000 7f5a3fe00640 1 -- 172.21.15.5:0/1846652157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a40108540 msgr2=0x7f5a401a4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:08.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.211+0000 7f5a3fe00640 1 --2- 172.21.15.5:0/1846652157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a40108540 0x7f5a401a4a70 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5a3000e9f0 tx=0x7f5a3000eeb0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.211+0000 7f5a3fe00640 1 -- 172.21.15.5:0/1846652157 shutdown_connections 2024-04-24T06:54:08.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.211+0000 7f5a3fe00640 1 --2- 172.21.15.5:0/1846652157 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f5a240a3da0 0x7f5a240a6260 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.211+0000 7f5a3fe00640 1 --2- 172.21.15.5:0/1846652157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a4010e4f0 0x7f5a4019eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.211+0000 7f5a3fe00640 1 --2- 172.21.15.5:0/1846652157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a40108ee0 0x7f5a401a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.211+0000 7f5a3fe00640 1 --2- 172.21.15.5:0/1846652157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a40108540 0x7f5a401a4a70 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.211+0000 7f5a3fe00640 1 -- 172.21.15.5:0/1846652157 >> 172.21.15.5:0/1846652157 conn(0x7f5a40077e10 msgr2=0x7f5a40078430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:08.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.211+0000 7f5a3fe00640 1 -- 172.21.15.5:0/1846652157 shutdown_connections 2024-04-24T06:54:08.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.211+0000 7f5a3fe00640 1 -- 172.21.15.5:0/1846652157 wait complete. 2024-04-24T06:54:08.229 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T06:54:08.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.442+0000 7fdfae600640 1 -- 172.21.15.5:0/1948571915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfa8100560 msgr2=0x7fdfa8100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:08.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.442+0000 7fdfae600640 1 --2- 172.21.15.5:0/1948571915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfa8100560 0x7fdfa8100940 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c009900 tx=0x7fdf9c02a3f0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.442+0000 7fdfae600640 1 -- 172.21.15.5:0/1948571915 shutdown_connections 2024-04-24T06:54:08.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.442+0000 7fdfae600640 1 --2- 172.21.15.5:0/1948571915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfa8107120 0x7fdfa810ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.442+0000 7fdfae600640 1 --2- 172.21.15.5:0/1948571915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfa8100f10 0x7fdfa8106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.442+0000 7fdfae600640 1 --2- 172.21.15.5:0/1948571915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfa8100560 0x7fdfa8100940 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.442+0000 7fdfae600640 1 -- 172.21.15.5:0/1948571915 >> 172.21.15.5:0/1948571915 conn(0x7fdfa8077240 msgr2=0x7fdfa80fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:08.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.442+0000 7fdfae600640 1 -- 172.21.15.5:0/1948571915 shutdown_connections 2024-04-24T06:54:08.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.442+0000 7fdfae600640 1 -- 172.21.15.5:0/1948571915 wait complete. 2024-04-24T06:54:08.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.443+0000 7fdfae600640 1 Processor -- start 2024-04-24T06:54:08.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.444+0000 7fdfae600640 1 -- start start 2024-04-24T06:54:08.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.444+0000 7fdfae600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfa8100560 0x7fdfa81a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:08.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.444+0000 7fdfae600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfa8107120 0x7fdfa81a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:08.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.444+0000 7fdfae600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfa819a760 0x7fdfa819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:08.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.444+0000 7fdfae600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa8112200 con 0x7fdfa8100560 2024-04-24T06:54:08.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.444+0000 7fdfae600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa8112080 con 0x7fdfa8107120 2024-04-24T06:54:08.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.444+0000 7fdfae600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa8112380 con 0x7fdfa819a760 2024-04-24T06:54:08.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.445+0000 7fdfa7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfa8107120 0x7fdfa81a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:08.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.445+0000 7fdfad200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfa8100560 0x7fdfa81a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:08.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.445+0000 7fdfa7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfa8107120 0x7fdfa81a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:48906/0 (socket says 172.21.15.5:48906) 2024-04-24T06:54:08.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.445+0000 7fdfa7e00640 1 -- 172.21.15.5:0/306139375 learned_addr learned my addr 172.21.15.5:0/306139375 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:08.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.445+0000 7fdfadc00640 1 --2- 172.21.15.5:0/306139375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfa819a760 0x7fdfa819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:08.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.445+0000 7fdfa7e00640 1 -- 172.21.15.5:0/306139375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfa819a760 msgr2=0x7fdfa819abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:08.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.445+0000 7fdfa7e00640 1 --2- 172.21.15.5:0/306139375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfa819a760 0x7fdfa819abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.445+0000 7fdfa7e00640 1 -- 172.21.15.5:0/306139375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfa8100560 msgr2=0x7fdfa81a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:08.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.445+0000 7fdfa7e00640 1 --2- 172.21.15.5:0/306139375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfa8100560 0x7fdfa81a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.445+0000 7fdfa7e00640 1 -- 172.21.15.5:0/306139375 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa819b360 con 0x7fdfa8107120 2024-04-24T06:54:08.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.446+0000 7fdfa7e00640 1 --2- 172.21.15.5:0/306139375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfa8107120 0x7fdfa81a0be0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fdf9400e9c0 tx=0x7fdf9400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:08.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.446+0000 7fdfa5600640 1 -- 172.21.15.5:0/306139375 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9400cd80 con 0x7fdfa8107120 2024-04-24T06:54:08.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.446+0000 7fdfa5600640 1 -- 172.21.15.5:0/306139375 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf94005500 con 0x7fdfa8107120 2024-04-24T06:54:08.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.446+0000 7fdfa5600640 1 -- 172.21.15.5:0/306139375 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf94010690 con 0x7fdfa8107120 2024-04-24T06:54:08.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.446+0000 7fdfae600640 1 -- 172.21.15.5:0/306139375 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfa81a7440 con 0x7fdfa8107120 2024-04-24T06:54:08.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.446+0000 7fdfae600640 1 -- 172.21.15.5:0/306139375 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfa81a77f0 con 0x7fdfa8107120 2024-04-24T06:54:08.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.451+0000 7fdfb2a35740 1 Processor -- start 2024-04-24T06:54:08.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.451+0000 7fdfb2a35740 1 -- start start 2024-04-24T06:54:08.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.452+0000 7fdfb2a35740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363949e350 0x56363949e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:08.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.452+0000 7fdfb2a35740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56363949ec70 0x5636394a8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:08.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.452+0000 7fdfb2a35740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636394a9340 0x5636394ab730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:08.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.452+0000 7fdfb2a35740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5636393fb1d0 con 0x56363949ec70 2024-04-24T06:54:08.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.452+0000 7fdfb2a35740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5636393a1450 con 0x5636394a9340 2024-04-24T06:54:08.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.452+0000 7fdfb2a35740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5636393f7190 con 0x56363949e350 2024-04-24T06:54:08.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.452+0000 7fdfadc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363949e350 0x56363949e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:08.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.452+0000 7fdfadc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363949e350 0x56363949e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:54684/0 (socket says 172.21.15.5:54684) 2024-04-24T06:54:08.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.452+0000 7fdfadc00640 1 -- 172.21.15.5:0/1223325625 learned_addr learned my addr 172.21.15.5:0/1223325625 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:08.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.453+0000 7fdfad200640 1 --2- 172.21.15.5:0/1223325625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56363949ec70 0x5636394a8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:08.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.453+0000 7fdfa7e00640 1 --2- 172.21.15.5:0/1223325625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636394a9340 0x5636394ab730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:08.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.453+0000 7fdfadc00640 1 -- 172.21.15.5:0/1223325625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636394a9340 msgr2=0x5636394ab730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:08.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.453+0000 7fdfadc00640 1 --2- 172.21.15.5:0/1223325625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636394a9340 0x5636394ab730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.453+0000 7fdfadc00640 1 -- 172.21.15.5:0/1223325625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56363949ec70 msgr2=0x5636394a8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:08.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.453+0000 7fdfadc00640 1 --2- 172.21.15.5:0/1223325625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56363949ec70 0x5636394a8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.453+0000 7fdfadc00640 1 -- 172.21.15.5:0/1223325625 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56363945e310 con 0x56363949e350 2024-04-24T06:54:08.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.453+0000 7fdfad200640 1 --2- 172.21.15.5:0/1223325625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56363949ec70 0x5636394a8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:54:08.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.454+0000 7fdfadc00640 1 --2- 172.21.15.5:0/1223325625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363949e350 0x56363949e730 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0009f50 tx=0x7fdfa00118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11abfe881ef77f17 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:08.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.454+0000 7fdf9a000640 1 -- 172.21.15.5:0/1223325625 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa0004d20 con 0x56363949e350 2024-04-24T06:54:08.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.454+0000 7fdf9a000640 1 -- 172.21.15.5:0/1223325625 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfa0004050 con 0x56363949e350 2024-04-24T06:54:08.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.454+0000 7fdf9a000640 1 -- 172.21.15.5:0/1223325625 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa0005280 con 0x56363949e350 2024-04-24T06:54:08.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.455+0000 7fdfb2a35740 1 -- 172.21.15.5:0/1223325625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363949e350 msgr2=0x56363949e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:08.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.455+0000 7fdfb2a35740 1 --2- 172.21.15.5:0/1223325625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363949e350 0x56363949e730 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0009f50 tx=0x7fdfa00118d0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.456+0000 7fdfb2a35740 1 -- 172.21.15.5:0/1223325625 shutdown_connections 2024-04-24T06:54:08.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.456+0000 7fdfb2a35740 1 --2- 172.21.15.5:0/1223325625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636394a9340 0x5636394ab730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.456+0000 7fdfb2a35740 1 --2- 172.21.15.5:0/1223325625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56363949ec70 0x5636394a8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.456+0000 7fdfb2a35740 1 --2- 172.21.15.5:0/1223325625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363949e350 0x56363949e730 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.456+0000 7fdfb2a35740 1 -- 172.21.15.5:0/1223325625 >> 172.21.15.5:0/1223325625 conn(0x56363949a670 msgr2=0x56363949aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:08.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.456+0000 7fdfa5600640 1 -- 172.21.15.5:0/306139375 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fdf940046c0 con 0x7fdfa8107120 2024-04-24T06:54:08.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.456+0000 7fdfb2a35740 1 -- 172.21.15.5:0/1223325625 shutdown_connections 2024-04-24T06:54:08.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.456+0000 7fdfb2a35740 1 -- 172.21.15.5:0/1223325625 wait complete. 2024-04-24T06:54:08.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.457+0000 7fdfb2a35740 1 -- start start 2024-04-24T06:54:08.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.457+0000 7fdfb2a35740 1 Processor -- start 2024-04-24T06:54:08.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.458+0000 7fdfb2a35740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363949e350 0x5636394a8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:08.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.458+0000 7fdfb2a35740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56363949ec70 0x5636394a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:08.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.458+0000 7fdfb2a35740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636394a9340 0x5636394a6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:08.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.458+0000 7fdfb2a35740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5636394ac1d0 con 0x56363949ec70 2024-04-24T06:54:08.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.458+0000 7fdfa5600640 1 --2- 172.21.15.5:0/306139375 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fdf900a3cb0 0x7fdf900a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:08.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.458+0000 7fdfb2a35740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5636393b4810 con 0x5636394a9340 2024-04-24T06:54:08.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.458+0000 7fdfb2a35740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5636394ac350 con 0x56363949e350 2024-04-24T06:54:08.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.458+0000 7fdfb2a35740 10 client.0 _ll_register_callbacks cb 0x563639382140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:54:08.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.458+0000 7fdfa7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636394a9340 0x5636394a6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:08.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.458+0000 7fdfa5600640 1 -- 172.21.15.5:0/306139375 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8586+0+0 (secure 0 0 0) 0x7fdf940d0b70 con 0x7fdfa8107120 2024-04-24T06:54:08.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.458+0000 7fdfad200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56363949ec70 0x5636394a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:08.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.458+0000 7fdfadc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363949e350 0x5636394a8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:08.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.458+0000 7fdfa7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636394a9340 0x5636394a6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:48922/0 (socket says 172.21.15.5:48922) 2024-04-24T06:54:08.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.458+0000 7fdfa7e00640 1 -- 172.21.15.5:0/3781270331 learned_addr learned my addr 172.21.15.5:0/3781270331 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:08.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.458+0000 7fdfadc00640 1 --2- 172.21.15.5:0/306139375 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fdf900a3cb0 0x7fdf900a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:08.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.459+0000 7fdfa7e00640 1 -- 172.21.15.5:0/3781270331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363949e350 msgr2=0x5636394a8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:08.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.459+0000 7fdfa7e00640 1 --2- 172.21.15.5:0/3781270331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363949e350 0x5636394a8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.459+0000 7fdfa7e00640 1 -- 172.21.15.5:0/3781270331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56363949ec70 msgr2=0x5636394a60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:08.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.459+0000 7fdfa7e00640 1 --2- 172.21.15.5:0/3781270331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56363949ec70 0x5636394a60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.459+0000 7fdfa7e00640 1 -- 172.21.15.5:0/3781270331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636393c53f0 con 0x5636394a9340 2024-04-24T06:54:08.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.459+0000 7fdfa7e00640 1 --2- 172.21.15.5:0/3781270331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636394a9340 0x5636394a6820 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fdf94000c00 tx=0x7fdf9409c5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f8b99274dba4066 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:08.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.459+0000 7fdf8ea00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:54:08.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.459+0000 7fdf8ea00640 1 -- 172.21.15.5:0/3781270331 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf940ce6a0 con 0x5636394a9340 2024-04-24T06:54:08.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.459+0000 7fdfadc00640 1 --2- 172.21.15.5:0/306139375 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fdf900a3cb0 0x7fdf900a6170 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0012fd0 tx=0x7fdfa0005fd0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:08.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.459+0000 7fdf8ea00640 1 -- 172.21.15.5:0/3781270331 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8586+0+0 (secure 0 0 0) 0x7fdf9409cd10 con 0x5636394a9340 2024-04-24T06:54:08.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.459+0000 7fdfb2a35740 1 -- 172.21.15.5:0/3781270331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfa0004050 con 0x5636394a9340 2024-04-24T06:54:08.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.459+0000 7fdf8ea00640 1 -- 172.21.15.5:0/3781270331 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdf9409cf90 con 0x5636394a9340 2024-04-24T06:54:08.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.459+0000 7fdfb2a35740 10 client.27407 fetch_fsmap learned FSMap version 27 2024-04-24T06:54:08.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.459+0000 7fdfb2a35740 1 -- 172.21.15.5:0/3781270331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56363910d3f0 con 0x5636394a9340 2024-04-24T06:54:08.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.460+0000 7fdf8ea00640 1 -- 172.21.15.5:0/3781270331 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfa0004050 con 0x5636394a9340 2024-04-24T06:54:08.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.460+0000 7fdf8ea00640 1 -- 172.21.15.5:0/3781270331 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7fdf9409bf00 con 0x5636394a9340 2024-04-24T06:54:08.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.460+0000 7fdfb2a35740 10 client.27407 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:54:08.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.460+0000 7fdfb2a35740 10 client.27407 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:54:08.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.460+0000 7fdfb2a35740 20 client.27407 populate_metadata read hostname 'smithi005' 2024-04-24T06:54:08.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.460+0000 7fdfb2a35740 1 --2- 172.21.15.5:0/3781270331 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5636394b94c0 0x5636394bb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:08.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.460+0000 7fdfb2a35740 4 client.27407 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:54:08.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.460+0000 7fdfb2a35740 1 -- 172.21.15.5:0/3781270331 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563639408bf0 con 0x5636394b94c0 2024-04-24T06:54:08.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.460+0000 7fdfad200640 1 --2- 172.21.15.5:0/3781270331 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5636394b94c0 0x5636394bb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:08.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.461+0000 7fdfad200640 1 --2- 172.21.15.5:0/3781270331 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5636394b94c0 0x5636394bb8b0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bac7593d004ccf69 server_cookie=70c712bbf8235a45 in_seq=0 out_seq=0 2024-04-24T06:54:08.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.461+0000 7fdf8ea00640 10 client.27407 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:54:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.462+0000 7fdf8ea00640 1 -- 172.21.15.5:0/3781270331 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563639408bf0 con 0x5636394b94c0 2024-04-24T06:54:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.462+0000 7fdf8ea00640 10 client.27407 handle_command_reply: tid=0 2024-04-24T06:54:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.462+0000 7fdfb2a35740 1 client.27407 shutdown 2024-04-24T06:54:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.462+0000 7fdfb2a35740 1 -- 172.21.15.5:0/3781270331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636394a9340 msgr2=0x5636394a6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.462+0000 7fdfb2a35740 1 --2- 172.21.15.5:0/3781270331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636394a9340 0x5636394a6820 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fdf94000c00 tx=0x7fdf9409c5f0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.462+0000 7fdfb2a35740 1 -- 172.21.15.5:0/3781270331 shutdown_connections 2024-04-24T06:54:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.462+0000 7fdfb2a35740 1 --2- 172.21.15.5:0/3781270331 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5636394b94c0 0x5636394bb8b0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.462+0000 7fdfb2a35740 1 --2- 172.21.15.5:0/3781270331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636394a9340 0x5636394a6820 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.462+0000 7fdfb2a35740 1 --2- 172.21.15.5:0/3781270331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56363949ec70 0x5636394a60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.462+0000 7fdfb2a35740 1 --2- 172.21.15.5:0/3781270331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363949e350 0x5636394a8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.462+0000 7fdf8ea00640 0 client.27407 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:54:08.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.462+0000 7fdfb2a35740 1 -- 172.21.15.5:0/3781270331 >> 172.21.15.5:0/3781270331 conn(0x56363949a670 msgr2=0x563639490f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:08.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.462+0000 7fdfb2a35740 1 -- 172.21.15.5:0/3781270331 shutdown_connections 2024-04-24T06:54:08.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.462+0000 7fdfb2a35740 1 -- 172.21.15.5:0/3781270331 wait complete. 2024-04-24T06:54:08.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.462+0000 7fdfb2a35740 20 client.27407 trim_cache size 0 max 16384 2024-04-24T06:54:08.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.478+0000 7fdfb2a35740 1 Processor -- start 2024-04-24T06:54:08.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.478+0000 7fdfb2a35740 1 -- start start 2024-04-24T06:54:08.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.478+0000 7fdfb2a35740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363949e350 0x5636394b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:08.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.478+0000 7fdfb2a35740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56363949ec70 0x5636394a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:08.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.478+0000 7fdfb2a35740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636394a9340 0x5636394a8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:08.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.478+0000 7fdfb2a35740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa81ae4e0 con 0x5636394a9340 2024-04-24T06:54:08.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.478+0000 7fdfb2a35740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa81abdb0 con 0x56363949ec70 2024-04-24T06:54:08.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.478+0000 7fdfb2a35740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa81a9c60 con 0x56363949e350 2024-04-24T06:54:08.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.478+0000 7fdfa7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56363949ec70 0x5636394a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:08.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.478+0000 7fdfa7e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56363949ec70 0x5636394a50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:48926/0 (socket says 172.21.15.5:48926) 2024-04-24T06:54:08.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.478+0000 7fdfa7e00640 1 -- 172.21.15.5:0/2637417030 learned_addr learned my addr 172.21.15.5:0/2637417030 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:08.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.478+0000 7fdfadc00640 1 --2- 172.21.15.5:0/2637417030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636394a9340 0x5636394a8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:08.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.478+0000 7fdfad200640 1 --2- 172.21.15.5:0/2637417030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363949e350 0x5636394b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:08.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.479+0000 7fdfa7e00640 1 -- 172.21.15.5:0/2637417030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363949e350 msgr2=0x5636394b50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:08.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.479+0000 7fdfa7e00640 1 --2- 172.21.15.5:0/2637417030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363949e350 0x5636394b50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.479+0000 7fdfa7e00640 1 -- 172.21.15.5:0/2637417030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636394a9340 msgr2=0x5636394a8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:08.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.479+0000 7fdfa7e00640 1 --2- 172.21.15.5:0/2637417030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636394a9340 0x5636394a8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.479+0000 7fdfa7e00640 1 -- 172.21.15.5:0/2637417030 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636391d77d0 con 0x56363949ec70 2024-04-24T06:54:08.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.479+0000 7fdfa7e00640 1 --2- 172.21.15.5:0/2637417030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56363949ec70 0x5636394a50c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fdf940bd260 tx=0x7fdf9409afc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1859154c1fa6a83e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:08.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.479+0000 7fdf99600640 1 -- 172.21.15.5:0/2637417030 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf940056b0 con 0x56363949ec70 2024-04-24T06:54:08.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.480+0000 7fdf99600640 1 -- 172.21.15.5:0/2637417030 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf9409b970 con 0x56363949ec70 2024-04-24T06:54:08.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.480+0000 7fdf99600640 1 -- 172.21.15.5:0/2637417030 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9409c410 con 0x56363949ec70 2024-04-24T06:54:08.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.480+0000 7fdfb2a35740 1 -- 172.21.15.5:0/2637417030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56363949ec70 msgr2=0x5636394a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:08.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.480+0000 7fdfb2a35740 1 --2- 172.21.15.5:0/2637417030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56363949ec70 0x5636394a50c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fdf940bd260 tx=0x7fdf9409afc0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.480+0000 7fdfb2a35740 1 -- 172.21.15.5:0/2637417030 shutdown_connections 2024-04-24T06:54:08.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.480+0000 7fdfb2a35740 1 --2- 172.21.15.5:0/2637417030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636394a9340 0x5636394a8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.480+0000 7fdfb2a35740 1 --2- 172.21.15.5:0/2637417030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56363949ec70 0x5636394a50c0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.480+0000 7fdfb2a35740 1 --2- 172.21.15.5:0/2637417030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363949e350 0x5636394b50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.480+0000 7fdfb2a35740 1 -- 172.21.15.5:0/2637417030 >> 172.21.15.5:0/2637417030 conn(0x56363949a670 msgr2=0x5636394a49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:08.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.480+0000 7fdfb2a35740 1 -- 172.21.15.5:0/2637417030 shutdown_connections 2024-04-24T06:54:08.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.480+0000 7fdfb2a35740 1 -- 172.21.15.5:0/2637417030 wait complete. 2024-04-24T06:54:08.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.481+0000 7fdfb2a35740 1 -- start start 2024-04-24T06:54:08.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.481+0000 7fdfb2a35740 1 Processor -- start 2024-04-24T06:54:08.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.481+0000 7fdfb2a35740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56363949e350 0x5636394b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:08.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.481+0000 7fdfb2a35740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363949ec70 0x5636394a9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:08.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.481+0000 7fdfb2a35740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636394a9340 0x5636394aa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:08.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.481+0000 7fdfb2a35740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa81aa2b0 con 0x5636394a9340 2024-04-24T06:54:08.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.481+0000 7fdfb2a35740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa81ab6e0 con 0x56363949e350 2024-04-24T06:54:08.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.481+0000 7fdfb2a35740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa81aa940 con 0x56363949ec70 2024-04-24T06:54:08.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.482+0000 7fdfb2a35740 10 client.0 _ll_register_callbacks cb 0x7fdf940ce040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:54:08.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.482+0000 7fdfad200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56363949e350 0x5636394b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:08.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.482+0000 7fdfa7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363949ec70 0x5636394a9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:08.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.482+0000 7fdfadc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636394a9340 0x5636394aa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:08.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.482+0000 7fdfad200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56363949e350 0x5636394b18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:48930/0 (socket says 172.21.15.5:48930) 2024-04-24T06:54:08.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.482+0000 7fdfad200640 1 -- 172.21.15.5:0/1392758583 learned_addr learned my addr 172.21.15.5:0/1392758583 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:08.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.482+0000 7fdfad200640 1 -- 172.21.15.5:0/1392758583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363949ec70 msgr2=0x5636394a9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:08.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.482+0000 7fdfad200640 1 --2- 172.21.15.5:0/1392758583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363949ec70 0x5636394a9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.482+0000 7fdfad200640 1 -- 172.21.15.5:0/1392758583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636394a9340 msgr2=0x5636394aa2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:08.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.482+0000 7fdfad200640 1 --2- 172.21.15.5:0/1392758583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636394a9340 0x5636394aa2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.482+0000 7fdfad200640 1 -- 172.21.15.5:0/1392758583 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636394aaa00 con 0x56363949e350 2024-04-24T06:54:08.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.483+0000 7fdfad200640 1 --2- 172.21.15.5:0/1392758583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56363949e350 0x5636394b18b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c009230 tx=0x7fdf9c005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a792dfac18ec64cc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:08.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.483+0000 7fdf8ea00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:54:08.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.483+0000 7fdf8ea00640 1 -- 172.21.15.5:0/1392758583 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c035040 con 0x56363949e350 2024-04-24T06:54:08.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.483+0000 7fdfb2a35740 1 -- 172.21.15.5:0/1392758583 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf9409b970 con 0x56363949e350 2024-04-24T06:54:08.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.483+0000 7fdf8ea00640 1 -- 172.21.15.5:0/1392758583 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8586+0+0 (secure 0 0 0) 0x7fdf9c005040 con 0x56363949e350 2024-04-24T06:54:08.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.483+0000 7fdf8ea00640 1 -- 172.21.15.5:0/1392758583 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdf9c005240 con 0x56363949e350 2024-04-24T06:54:08.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.483+0000 7fdfb2a35740 10 client.27413 fetch_fsmap learned FSMap version 27 2024-04-24T06:54:08.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.483+0000 7fdfb2a35740 1 -- 172.21.15.5:0/1392758583 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56363951fa00 con 0x56363949e350 2024-04-24T06:54:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.483+0000 7fdf8ea00640 1 -- 172.21.15.5:0/1392758583 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf9409b970 con 0x56363949e350 2024-04-24T06:54:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.484+0000 7fdf8ea00640 1 -- 172.21.15.5:0/1392758583 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7fdf9c004850 con 0x56363949e350 2024-04-24T06:54:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.484+0000 7fdfb2a35740 10 client.27413 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:54:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.484+0000 7fdfb2a35740 10 client.27413 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:54:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.484+0000 7fdfb2a35740 20 client.27413 populate_metadata read hostname 'smithi005' 2024-04-24T06:54:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.484+0000 7fdfb2a35740 1 --2- 172.21.15.5:0/1392758583 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x563639520080 0x563639522470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.484+0000 7fdfb2a35740 4 client.27413 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:54:08.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.484+0000 7fdfb2a35740 1 -- 172.21.15.5:0/1392758583 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563639402a60 con 0x563639520080 2024-04-24T06:54:08.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.484+0000 7fdfa7e00640 1 --2- 172.21.15.5:0/1392758583 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x563639520080 0x563639522470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:08.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.485+0000 7fdfa7e00640 1 --2- 172.21.15.5:0/1392758583 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x563639520080 0x563639522470 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be36a0e5f02304c9 server_cookie=c4bb846b52fa6237 in_seq=0 out_seq=0 2024-04-24T06:54:08.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.485+0000 7fdf8ea00640 10 client.27413 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:54:08.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.485+0000 7fdf8ea00640 1 -- 172.21.15.5:0/1392758583 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563639402a60 con 0x563639520080 2024-04-24T06:54:08.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.485+0000 7fdf8ea00640 10 client.27413 handle_command_reply: tid=0 2024-04-24T06:54:08.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.485+0000 7fdfb2a35740 1 client.27413 shutdown 2024-04-24T06:54:08.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.485+0000 7fdfb2a35740 1 -- 172.21.15.5:0/1392758583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56363949e350 msgr2=0x5636394b18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:08.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.485+0000 7fdfb2a35740 1 --2- 172.21.15.5:0/1392758583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56363949e350 0x5636394b18b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c009230 tx=0x7fdf9c005c00 comp rx=0 tx=0).stop 2024-04-24T06:54:08.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.485+0000 7fdfb2a35740 1 -- 172.21.15.5:0/1392758583 shutdown_connections 2024-04-24T06:54:08.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.485+0000 7fdfb2a35740 1 --2- 172.21.15.5:0/1392758583 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x563639520080 0x563639522470 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.485+0000 7fdfb2a35740 1 --2- 172.21.15.5:0/1392758583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636394a9340 0x5636394aa2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.485+0000 7fdfb2a35740 1 --2- 172.21.15.5:0/1392758583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56363949ec70 0x5636394a9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.485+0000 7fdfb2a35740 1 --2- 172.21.15.5:0/1392758583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56363949e350 0x5636394b18b0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.485+0000 7fdf8ea00640 0 client.27413 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:54:08.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.485+0000 7fdfb2a35740 1 -- 172.21.15.5:0/1392758583 >> 172.21.15.5:0/1392758583 conn(0x56363949a670 msgr2=0x5636394a5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:08.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.485+0000 7fdfb2a35740 1 -- 172.21.15.5:0/1392758583 shutdown_connections 2024-04-24T06:54:08.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.485+0000 7fdfb2a35740 1 -- 172.21.15.5:0/1392758583 wait complete. 2024-04-24T06:54:08.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.485+0000 7fdfb2a35740 20 client.27413 trim_cache size 0 max 16384 2024-04-24T06:54:08.495 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:54:08.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.486+0000 7fdfae600640 1 -- 172.21.15.5:0/306139375 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fdf900a3cb0 msgr2=0x7fdf900a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:08.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.486+0000 7fdfae600640 1 --2- 172.21.15.5:0/306139375 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fdf900a3cb0 0x7fdf900a6170 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0012fd0 tx=0x7fdfa0005fd0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.486+0000 7fdfae600640 1 -- 172.21.15.5:0/306139375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfa8107120 msgr2=0x7fdfa81a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:08.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.486+0000 7fdfae600640 1 --2- 172.21.15.5:0/306139375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfa8107120 0x7fdfa81a0be0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fdf9400e9c0 tx=0x7fdf9400ee80 comp rx=0 tx=0).stop 2024-04-24T06:54:08.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.486+0000 7fdfae600640 1 -- 172.21.15.5:0/306139375 shutdown_connections 2024-04-24T06:54:08.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.486+0000 7fdfae600640 1 --2- 172.21.15.5:0/306139375 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fdf900a3cb0 0x7fdf900a6170 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.486+0000 7fdfae600640 1 --2- 172.21.15.5:0/306139375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfa819a760 0x7fdfa819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.486+0000 7fdfae600640 1 --2- 172.21.15.5:0/306139375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfa8107120 0x7fdfa81a0be0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.486+0000 7fdfae600640 1 --2- 172.21.15.5:0/306139375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfa8100560 0x7fdfa81a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:08.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.486+0000 7fdfae600640 1 -- 172.21.15.5:0/306139375 >> 172.21.15.5:0/306139375 conn(0x7fdfa8077240 msgr2=0x7fdfa8105600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:08.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.486+0000 7fdfae600640 1 -- 172.21.15.5:0/306139375 shutdown_connections 2024-04-24T06:54:08.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:08.486+0000 7fdfae600640 1 -- 172.21.15.5:0/306139375 wait complete. 2024-04-24T06:54:08.503 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:08 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1825797670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:54:08.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:08 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/1825797670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:54:08.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:08 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/1825797670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:54:09.558 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T06:54:09.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:54:09.559 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T06:54:09.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:09 smithi005 ceph-mon[27798]: pgmap v312: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-24T06:54:09.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:09 smithi005 ceph-mon[27798]: osdmap e96: 12 total, 12 up, 12 in 2024-04-24T06:54:09.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:09 smithi037 ceph-mon[32864]: pgmap v312: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-24T06:54:09.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:09 smithi037 ceph-mon[32864]: osdmap e96: 12 total, 12 up, 12 in 2024-04-24T06:54:09.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.793+0000 7f0843200640 1 -- 172.21.15.5:0/3720258630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0844104290 msgr2=0x7f0844104670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:09.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.793+0000 7f0843200640 1 --2- 172.21.15.5:0/3720258630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0844104290 0x7f0844104670 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0834009900 tx=0x7f083402a410 comp rx=0 tx=0).stop 2024-04-24T06:54:09.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.794+0000 7f0843200640 1 -- 172.21.15.5:0/3720258630 shutdown_connections 2024-04-24T06:54:09.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.794+0000 7f0843200640 1 --2- 172.21.15.5:0/3720258630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0844109360 0x7f0844110f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:09.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.794+0000 7f0843200640 1 --2- 172.21.15.5:0/3720258630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0844104c40 0x7f0844108c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:09.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.794+0000 7f0843200640 1 --2- 172.21.15.5:0/3720258630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0844104290 0x7f0844104670 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:09.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.794+0000 7f0843200640 1 -- 172.21.15.5:0/3720258630 >> 172.21.15.5:0/3720258630 conn(0x7f08440ffaf0 msgr2=0x7f0844101f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:09.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.794+0000 7f0843200640 1 -- 172.21.15.5:0/3720258630 shutdown_connections 2024-04-24T06:54:09.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.794+0000 7f0843200640 1 -- 172.21.15.5:0/3720258630 wait complete. 2024-04-24T06:54:09.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.795+0000 7f0843200640 1 Processor -- start 2024-04-24T06:54:09.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.795+0000 7f0843200640 1 -- start start 2024-04-24T06:54:09.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.795+0000 7f0843200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0844104290 0x7f08441a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:09.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.795+0000 7f0843200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0844109360 0x7f08441a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:09.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.795+0000 7f0843200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f084419a7d0 0x7f084419ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:09.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.795+0000 7f0843200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0844114410 con 0x7f0844109360 2024-04-24T06:54:09.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.795+0000 7f0843200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0844114290 con 0x7f084419a7d0 2024-04-24T06:54:09.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.795+0000 7f0843200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0844114590 con 0x7f0844104290 2024-04-24T06:54:09.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.796+0000 7f0841400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0844109360 0x7f08441a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:09.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.796+0000 7f0841400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0844109360 0x7f08441a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48484/0 (socket says 172.21.15.5:48484) 2024-04-24T06:54:09.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.796+0000 7f0841400640 1 -- 172.21.15.5:0/3084235425 learned_addr learned my addr 172.21.15.5:0/3084235425 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:09.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.796+0000 7f0841e00640 1 --2- 172.21.15.5:0/3084235425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0844104290 0x7f08441a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:09.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.796+0000 7f0842800640 1 --2- 172.21.15.5:0/3084235425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f084419a7d0 0x7f084419ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:09.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.796+0000 7f0841400640 1 -- 172.21.15.5:0/3084235425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0844104290 msgr2=0x7f08441a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:09.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.796+0000 7f0841400640 1 --2- 172.21.15.5:0/3084235425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0844104290 0x7f08441a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:09.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.796+0000 7f0841400640 1 -- 172.21.15.5:0/3084235425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f084419a7d0 msgr2=0x7f084419ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:09.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.796+0000 7f0841400640 1 --2- 172.21.15.5:0/3084235425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f084419a7d0 0x7f084419ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:09.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.796+0000 7f0841400640 1 -- 172.21.15.5:0/3084235425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f084419b360 con 0x7f0844109360 2024-04-24T06:54:09.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.796+0000 7f0841400640 1 --2- 172.21.15.5:0/3084235425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0844109360 0x7f08441a0ba0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f083000e9f0 tx=0x7f083000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:09.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.797+0000 7f083aa00640 1 -- 172.21.15.5:0/3084235425 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083000ccc0 con 0x7f0844109360 2024-04-24T06:54:09.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.797+0000 7f083aa00640 1 -- 172.21.15.5:0/3084235425 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0830005500 con 0x7f0844109360 2024-04-24T06:54:09.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.797+0000 7f0843200640 1 -- 172.21.15.5:0/3084235425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f084419b650 con 0x7f0844109360 2024-04-24T06:54:09.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.797+0000 7f083aa00640 1 -- 172.21.15.5:0/3084235425 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0830010690 con 0x7f0844109360 2024-04-24T06:54:09.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.797+0000 7f0843200640 1 -- 172.21.15.5:0/3084235425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08441a77e0 con 0x7f0844109360 2024-04-24T06:54:09.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.801+0000 7f084b9aa740 1 Processor -- start 2024-04-24T06:54:09.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.801+0000 7f084b9aa740 1 -- start start 2024-04-24T06:54:09.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.802+0000 7f083aa00640 1 -- 172.21.15.5:0/3084235425 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f08300046c0 con 0x7f0844109360 2024-04-24T06:54:09.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.802+0000 7f084b9aa740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f2b9d3fc0 0x563f2b9d43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:09.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.802+0000 7f084b9aa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f2b9d4970 0x563f2b9e7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:09.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.802+0000 7f084b9aa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f2b9e7a70 0x563f2b9e9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:09.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.802+0000 7f084b9aa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563f2b939e70 con 0x563f2b9e7a70 2024-04-24T06:54:09.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.802+0000 7f084b9aa740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563f2b8e0380 con 0x563f2b9d3fc0 2024-04-24T06:54:09.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.802+0000 7f084b9aa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563f2b935e30 con 0x563f2b9d4970 2024-04-24T06:54:09.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.802+0000 7f0841e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f2b9d4970 0x563f2b9e7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:09.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.802+0000 7f0841400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f2b9e7a70 0x563f2b9e9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:09.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.802+0000 7f0841e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f2b9d4970 0x563f2b9e7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:54746/0 (socket says 172.21.15.5:54746) 2024-04-24T06:54:09.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.802+0000 7f0841e00640 1 -- 172.21.15.5:0/1156008374 learned_addr learned my addr 172.21.15.5:0/1156008374 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:09.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.802+0000 7f0842800640 1 --2- 172.21.15.5:0/1156008374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f2b9d3fc0 0x563f2b9d43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:09.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.803+0000 7f0841e00640 1 -- 172.21.15.5:0/1156008374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f2b9d3fc0 msgr2=0x563f2b9d43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:09.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.803+0000 7f0841e00640 1 --2- 172.21.15.5:0/1156008374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f2b9d3fc0 0x563f2b9d43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:09.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.803+0000 7f0841e00640 1 -- 172.21.15.5:0/1156008374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f2b9e7a70 msgr2=0x563f2b9e9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:09.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.803+0000 7f0841e00640 1 --2- 172.21.15.5:0/1156008374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f2b9e7a70 0x563f2b9e9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:09.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.803+0000 7f0841e00640 1 -- 172.21.15.5:0/1156008374 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f2b7147e0 con 0x563f2b9d4970 2024-04-24T06:54:09.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.803+0000 7f0841e00640 1 --2- 172.21.15.5:0/1156008374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f2b9d4970 0x563f2b9e7530 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0834009900 tx=0x7f0834005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5d1edf39b757cb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:09.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.803+0000 7f083aa00640 1 --2- 172.21.15.5:0/3084235425 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f08280a3d20 0x7f08280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:09.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.803+0000 7f082f000640 1 -- 172.21.15.5:0/1156008374 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0834040070 con 0x563f2b9d4970 2024-04-24T06:54:09.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.804+0000 7f082f000640 1 -- 172.21.15.5:0/1156008374 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0834031870 con 0x563f2b9d4970 2024-04-24T06:54:09.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.804+0000 7f082f000640 1 -- 172.21.15.5:0/1156008374 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0834035040 con 0x563f2b9d4970 2024-04-24T06:54:09.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.804+0000 7f083aa00640 1 -- 172.21.15.5:0/3084235425 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8586+0+0 (secure 0 0 0) 0x7f08300d0b80 con 0x7f0844109360 2024-04-24T06:54:09.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.804+0000 7f0842800640 1 --2- 172.21.15.5:0/3084235425 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f08280a3d20 0x7f08280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:09.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.804+0000 7f084b9aa740 1 -- 172.21.15.5:0/1156008374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f2b9d4970 msgr2=0x563f2b9e7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:09.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.804+0000 7f084b9aa740 1 --2- 172.21.15.5:0/1156008374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f2b9d4970 0x563f2b9e7530 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0834009900 tx=0x7f0834005be0 comp rx=0 tx=0).stop 2024-04-24T06:54:09.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.805+0000 7f0842800640 1 --2- 172.21.15.5:0/3084235425 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f08280a3d20 0x7f08280a61e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f083c005fd0 tx=0x7f083c009500 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:09.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.805+0000 7f084b9aa740 1 -- 172.21.15.5:0/1156008374 shutdown_connections 2024-04-24T06:54:09.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.805+0000 7f084b9aa740 1 --2- 172.21.15.5:0/1156008374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f2b9e7a70 0x563f2b9e9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:09.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.805+0000 7f084b9aa740 1 --2- 172.21.15.5:0/1156008374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f2b9d4970 0x563f2b9e7530 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:09.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.805+0000 7f084b9aa740 1 --2- 172.21.15.5:0/1156008374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f2b9d3fc0 0x563f2b9d43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:09.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.805+0000 7f084b9aa740 1 -- 172.21.15.5:0/1156008374 >> 172.21.15.5:0/1156008374 conn(0x563f2b9d0590 msgr2=0x563f2b9d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:09.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.805+0000 7f084b9aa740 1 -- 172.21.15.5:0/1156008374 shutdown_connections 2024-04-24T06:54:09.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.805+0000 7f084b9aa740 1 -- 172.21.15.5:0/1156008374 wait complete. 2024-04-24T06:54:09.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.805+0000 7f084b9aa740 1 -- start start 2024-04-24T06:54:09.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.805+0000 7f084b9aa740 1 Processor -- start 2024-04-24T06:54:09.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.806+0000 7f084b9aa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f2b9d3fc0 0x563f2b9e4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:09.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.806+0000 7f084b9aa740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f2b9d4970 0x563f2b9e4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:09.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.806+0000 7f084b9aa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f2b9e7a70 0x563f2b9e5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:09.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.806+0000 7f084b9aa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563f2b9ea7d0 con 0x563f2b9d3fc0 2024-04-24T06:54:09.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.806+0000 7f084b9aa740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563f2b8f3660 con 0x563f2b9d4970 2024-04-24T06:54:09.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.806+0000 7f084b9aa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563f2b9ea950 con 0x563f2b9e7a70 2024-04-24T06:54:09.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.806+0000 7f084b9aa740 10 client.0 _ll_register_callbacks cb 0x563f2b8e1aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:54:09.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.806+0000 7f0841400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f2b9d4970 0x563f2b9e4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:09.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.806+0000 7f0841e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f2b9d3fc0 0x563f2b9e4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:09.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.806+0000 7f0841400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f2b9d4970 0x563f2b9e4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:48964/0 (socket says 172.21.15.5:48964) 2024-04-24T06:54:09.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.806+0000 7f0841400640 1 -- 172.21.15.5:0/682796648 learned_addr learned my addr 172.21.15.5:0/682796648 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:09.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.806+0000 7f0842800640 1 --2- 172.21.15.5:0/682796648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f2b9e7a70 0x563f2b9e5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:09.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.807+0000 7f0841400640 1 -- 172.21.15.5:0/682796648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f2b9e7a70 msgr2=0x563f2b9e5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:09.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.807+0000 7f0841400640 1 --2- 172.21.15.5:0/682796648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f2b9e7a70 0x563f2b9e5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:09.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.807+0000 7f0841400640 1 -- 172.21.15.5:0/682796648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f2b9d3fc0 msgr2=0x563f2b9e4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:09.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.807+0000 7f0841400640 1 --2- 172.21.15.5:0/682796648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f2b9d3fc0 0x563f2b9e4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:09.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.807+0000 7f0841400640 1 -- 172.21.15.5:0/682796648 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f2b774f70 con 0x563f2b9d4970 2024-04-24T06:54:09.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.807+0000 7f0842800640 1 --2- 172.21.15.5:0/682796648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f2b9e7a70 0x563f2b9e5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:54:09.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.807+0000 7f0841e00640 1 --2- 172.21.15.5:0/682796648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f2b9d3fc0 0x563f2b9e4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:54:09.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.807+0000 7f0841400640 1 --2- 172.21.15.5:0/682796648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f2b9d4970 0x563f2b9e4f40 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f08300ce2f0 tx=0x7f083000cf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e07966ef02c7c67a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:09.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.807+0000 7f0823400640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:54:09.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.807+0000 7f0823400640 1 -- 172.21.15.5:0/682796648 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083009c620 con 0x563f2b9d4970 2024-04-24T06:54:09.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.807+0000 7f084b9aa740 1 -- 172.21.15.5:0/682796648 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0834031870 con 0x563f2b9d4970 2024-04-24T06:54:09.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.807+0000 7f0823400640 1 -- 172.21.15.5:0/682796648 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8586+0+0 (secure 0 0 0) 0x7f0830099150 con 0x563f2b9d4970 2024-04-24T06:54:09.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.807+0000 7f0823400640 1 -- 172.21.15.5:0/682796648 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f08300056b0 con 0x563f2b9d4970 2024-04-24T06:54:09.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.807+0000 7f084b9aa740 10 client.27425 fetch_fsmap learned FSMap version 27 2024-04-24T06:54:09.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.807+0000 7f084b9aa740 1 -- 172.21.15.5:0/682796648 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563f2b60c170 con 0x563f2b9d4970 2024-04-24T06:54:09.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.808+0000 7f0823400640 1 -- 172.21.15.5:0/682796648 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0834031870 con 0x563f2b9d4970 2024-04-24T06:54:09.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.808+0000 7f0823400640 1 -- 172.21.15.5:0/682796648 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f08300da040 con 0x563f2b9d4970 2024-04-24T06:54:09.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.808+0000 7f084b9aa740 10 client.27425 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:54:09.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.808+0000 7f084b9aa740 10 client.27425 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:54:09.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.808+0000 7f084b9aa740 20 client.27425 populate_metadata read hostname 'smithi005' 2024-04-24T06:54:09.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.808+0000 7f084b9aa740 1 --2- 172.21.15.5:0/682796648 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x563f2b9f7e50 0x563f2b9fa240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:09.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.808+0000 7f084b9aa740 4 client.27425 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:54:09.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.808+0000 7f084b9aa740 1 -- 172.21.15.5:0/682796648 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f2b947890 con 0x563f2b9f7e50 2024-04-24T06:54:09.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.808+0000 7f0841e00640 1 --2- 172.21.15.5:0/682796648 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x563f2b9f7e50 0x563f2b9fa240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:09.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.809+0000 7f0841e00640 1 --2- 172.21.15.5:0/682796648 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x563f2b9f7e50 0x563f2b9fa240 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2fc79d74f56f185 server_cookie=a57405da329c67 in_seq=0 out_seq=0 2024-04-24T06:54:09.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.809+0000 7f0823400640 10 client.27425 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:54:09.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.810+0000 7f0823400640 1 -- 172.21.15.5:0/682796648 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563f2b947890 con 0x563f2b9f7e50 2024-04-24T06:54:09.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.810+0000 7f0823400640 10 client.27425 handle_command_reply: tid=0 2024-04-24T06:54:09.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.810+0000 7f084b9aa740 1 client.27425 shutdown 2024-04-24T06:54:09.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.810+0000 7f084b9aa740 1 -- 172.21.15.5:0/682796648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f2b9d4970 msgr2=0x563f2b9e4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:09.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.810+0000 7f084b9aa740 1 --2- 172.21.15.5:0/682796648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f2b9d4970 0x563f2b9e4f40 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f08300ce2f0 tx=0x7f083000cf60 comp rx=0 tx=0).stop 2024-04-24T06:54:09.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.810+0000 7f084b9aa740 1 -- 172.21.15.5:0/682796648 shutdown_connections 2024-04-24T06:54:09.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.810+0000 7f084b9aa740 1 --2- 172.21.15.5:0/682796648 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x563f2b9f7e50 0x563f2b9fa240 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:09.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.810+0000 7f084b9aa740 1 --2- 172.21.15.5:0/682796648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f2b9e7a70 0x563f2b9e5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:09.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.810+0000 7f084b9aa740 1 --2- 172.21.15.5:0/682796648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f2b9d4970 0x563f2b9e4f40 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:09.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.810+0000 7f084b9aa740 1 --2- 172.21.15.5:0/682796648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f2b9d3fc0 0x563f2b9e4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:09.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.810+0000 7f0823400640 0 client.27425 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:54:09.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.810+0000 7f084b9aa740 1 -- 172.21.15.5:0/682796648 >> 172.21.15.5:0/682796648 conn(0x563f2b9d0590 msgr2=0x563f2b9d6390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:09.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.810+0000 7f084b9aa740 1 -- 172.21.15.5:0/682796648 shutdown_connections 2024-04-24T06:54:09.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.810+0000 7f084b9aa740 1 -- 172.21.15.5:0/682796648 wait complete. 2024-04-24T06:54:09.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.810+0000 7f084b9aa740 20 client.27425 trim_cache size 0 max 16384 2024-04-24T06:54:09.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.831+0000 7f084b9aa740 1 Processor -- start 2024-04-24T06:54:09.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.831+0000 7f084b9aa740 1 -- start start 2024-04-24T06:54:09.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.831+0000 7f084b9aa740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f2b9d3fc0 0x563f2b9f36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:09.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.831+0000 7f084b9aa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f2b9d4970 0x563f2b9cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:09.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.831+0000 7f084b9aa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f2b9e7a70 0x563f2b9ffa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:09.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.831+0000 7f084b9aa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f08441ae3d0 con 0x563f2b9d4970 2024-04-24T06:54:09.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.831+0000 7f084b9aa740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f08441abca0 con 0x563f2b9d3fc0 2024-04-24T06:54:09.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.831+0000 7f084b9aa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f08441a9b30 con 0x563f2b9e7a70 2024-04-24T06:54:09.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.832+0000 7f0841e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f2b9d3fc0 0x563f2b9f36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:09.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.832+0000 7f0841400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f2b9d4970 0x563f2b9cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:09.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.832+0000 7f0841e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f2b9d3fc0 0x563f2b9f36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:48980/0 (socket says 172.21.15.5:48980) 2024-04-24T06:54:09.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.832+0000 7f0841e00640 1 -- 172.21.15.5:0/2166402397 learned_addr learned my addr 172.21.15.5:0/2166402397 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:09.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.832+0000 7f0842800640 1 --2- 172.21.15.5:0/2166402397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f2b9e7a70 0x563f2b9ffa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:09.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.832+0000 7f0841e00640 1 -- 172.21.15.5:0/2166402397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f2b9e7a70 msgr2=0x563f2b9ffa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:09.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.832+0000 7f0841e00640 1 --2- 172.21.15.5:0/2166402397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f2b9e7a70 0x563f2b9ffa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:09.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.832+0000 7f0841e00640 1 -- 172.21.15.5:0/2166402397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f2b9d4970 msgr2=0x563f2b9cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:09.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.832+0000 7f0841e00640 1 --2- 172.21.15.5:0/2166402397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f2b9d4970 0x563f2b9cf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:09.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.832+0000 7f0841e00640 1 -- 172.21.15.5:0/2166402397 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f2b9e39b0 con 0x563f2b9d3fc0 2024-04-24T06:54:09.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.832+0000 7f0841e00640 1 --2- 172.21.15.5:0/2166402397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f2b9d3fc0 0x563f2b9f36d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0834005940 tx=0x7f08340054a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47d5a62817c9cfd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:09.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.833+0000 7f082e600640 1 -- 172.21.15.5:0/2166402397 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0834057070 con 0x563f2b9d3fc0 2024-04-24T06:54:09.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.833+0000 7f082e600640 1 -- 172.21.15.5:0/2166402397 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08340035f0 con 0x563f2b9d3fc0 2024-04-24T06:54:09.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.833+0000 7f082e600640 1 -- 172.21.15.5:0/2166402397 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0834035040 con 0x563f2b9d3fc0 2024-04-24T06:54:09.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.833+0000 7f084b9aa740 1 -- 172.21.15.5:0/2166402397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f2b9d3fc0 msgr2=0x563f2b9f36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:09.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.833+0000 7f084b9aa740 1 --2- 172.21.15.5:0/2166402397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f2b9d3fc0 0x563f2b9f36d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0834005940 tx=0x7f08340054a0 comp rx=0 tx=0).stop 2024-04-24T06:54:09.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.833+0000 7f084b9aa740 1 -- 172.21.15.5:0/2166402397 shutdown_connections 2024-04-24T06:54:09.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.833+0000 7f084b9aa740 1 --2- 172.21.15.5:0/2166402397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f2b9e7a70 0x563f2b9ffa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:09.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.833+0000 7f084b9aa740 1 --2- 172.21.15.5:0/2166402397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f2b9d4970 0x563f2b9cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:09.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.833+0000 7f084b9aa740 1 --2- 172.21.15.5:0/2166402397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f2b9d3fc0 0x563f2b9f36d0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:09.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.833+0000 7f084b9aa740 1 -- 172.21.15.5:0/2166402397 >> 172.21.15.5:0/2166402397 conn(0x563f2b9d0590 msgr2=0x563f2b9cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:09.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.833+0000 7f084b9aa740 1 -- 172.21.15.5:0/2166402397 shutdown_connections 2024-04-24T06:54:09.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.834+0000 7f084b9aa740 1 -- 172.21.15.5:0/2166402397 wait complete. 2024-04-24T06:54:09.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.834+0000 7f084b9aa740 1 -- start start 2024-04-24T06:54:09.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.834+0000 7f084b9aa740 1 Processor -- start 2024-04-24T06:54:09.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.835+0000 7f084b9aa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f2b9d3fc0 0x563f2b9ef840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:09.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.835+0000 7f084b9aa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f2b9d4970 0x563f2b9efd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:09.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.835+0000 7f084b9aa740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f2b9e7a70 0x563f2b9ebd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:09.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.835+0000 7f084b9aa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f08441aa180 con 0x563f2b9d4970 2024-04-24T06:54:09.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.835+0000 7f084b9aa740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f08441ab5d0 con 0x563f2b9e7a70 2024-04-24T06:54:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.835+0000 7f084b9aa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f08441aa830 con 0x563f2b9d3fc0 2024-04-24T06:54:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.835+0000 7f084b9aa740 10 client.0 _ll_register_callbacks cb 0x7f083000ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:54:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.835+0000 7f0841e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f2b9d3fc0 0x563f2b9ef840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.835+0000 7f0841400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f2b9d4970 0x563f2b9efd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.835+0000 7f0842800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f2b9e7a70 0x563f2b9ebd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.835+0000 7f0841e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f2b9d3fc0 0x563f2b9ef840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:54788/0 (socket says 172.21.15.5:54788) 2024-04-24T06:54:09.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.835+0000 7f0841e00640 1 -- 172.21.15.5:0/1407987677 learned_addr learned my addr 172.21.15.5:0/1407987677 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:09.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.835+0000 7f0841e00640 1 -- 172.21.15.5:0/1407987677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f2b9e7a70 msgr2=0x563f2b9ebd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:09.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.835+0000 7f0841e00640 1 --2- 172.21.15.5:0/1407987677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f2b9e7a70 0x563f2b9ebd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:09.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.835+0000 7f0841e00640 1 -- 172.21.15.5:0/1407987677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f2b9d4970 msgr2=0x563f2b9efd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:09.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.835+0000 7f0841e00640 1 --2- 172.21.15.5:0/1407987677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f2b9d4970 0x563f2b9efd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:09.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.836+0000 7f0841e00640 1 -- 172.21.15.5:0/1407987677 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f2b9ec450 con 0x563f2b9d3fc0 2024-04-24T06:54:09.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.836+0000 7f0841e00640 1 --2- 172.21.15.5:0/1407987677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f2b9d3fc0 0x563f2b9ef840 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0834031470 tx=0x7f083402abc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b2029f24cc49bab server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:09.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.836+0000 7f0823400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:54:09.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.836+0000 7f0823400640 1 -- 172.21.15.5:0/1407987677 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0834060060 con 0x563f2b9d3fc0 2024-04-24T06:54:09.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.836+0000 7f0823400640 1 -- 172.21.15.5:0/1407987677 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8586+0+0 (secure 0 0 0) 0x7f0834004000 con 0x563f2b9d3fc0 2024-04-24T06:54:09.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.836+0000 7f084b9aa740 1 -- 172.21.15.5:0/1407987677 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08340035f0 con 0x563f2b9d3fc0 2024-04-24T06:54:09.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.836+0000 7f0823400640 1 -- 172.21.15.5:0/1407987677 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0834057040 con 0x563f2b9d3fc0 2024-04-24T06:54:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.836+0000 7f084b9aa740 10 client.27451 fetch_fsmap learned FSMap version 27 2024-04-24T06:54:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.836+0000 7f084b9aa740 1 -- 172.21.15.5:0/1407987677 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563f2ba5ed30 con 0x563f2b9d3fc0 2024-04-24T06:54:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.837+0000 7f0823400640 1 -- 172.21.15.5:0/1407987677 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08340035f0 con 0x563f2b9d3fc0 2024-04-24T06:54:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.837+0000 7f0823400640 1 -- 172.21.15.5:0/1407987677 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f083404d070 con 0x563f2b9d3fc0 2024-04-24T06:54:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.837+0000 7f084b9aa740 10 client.27451 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:54:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.837+0000 7f084b9aa740 10 client.27451 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:54:09.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.837+0000 7f084b9aa740 20 client.27451 populate_metadata read hostname 'smithi005' 2024-04-24T06:54:09.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.837+0000 7f084b9aa740 1 --2- 172.21.15.5:0/1407987677 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x563f2ba5ef60 0x563f2ba61350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:09.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.837+0000 7f084b9aa740 4 client.27451 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T06:54:09.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.837+0000 7f084b9aa740 1 -- 172.21.15.5:0/1407987677 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563f2b941700 con 0x563f2ba5ef60 2024-04-24T06:54:09.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.837+0000 7f0841400640 1 --2- 172.21.15.5:0/1407987677 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x563f2ba5ef60 0x563f2ba61350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:09.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.838+0000 7f0841400640 1 --2- 172.21.15.5:0/1407987677 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x563f2ba5ef60 0x563f2ba61350 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f3e6002b32b14a2 server_cookie=27735af16ee28a13 in_seq=0 out_seq=0 2024-04-24T06:54:09.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.838+0000 7f0823400640 10 client.27451 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:54:09.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.839+0000 7f0823400640 1 -- 172.21.15.5:0/1407987677 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563f2b941700 con 0x563f2ba5ef60 2024-04-24T06:54:09.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.839+0000 7f0823400640 10 client.27451 handle_command_reply: tid=0 2024-04-24T06:54:09.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.839+0000 7f084b9aa740 1 client.27451 shutdown 2024-04-24T06:54:09.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.839+0000 7f084b9aa740 1 -- 172.21.15.5:0/1407987677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f2b9d3fc0 msgr2=0x563f2b9ef840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:09.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.839+0000 7f084b9aa740 1 --2- 172.21.15.5:0/1407987677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f2b9d3fc0 0x563f2b9ef840 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0834031470 tx=0x7f083402abc0 comp rx=0 tx=0).stop 2024-04-24T06:54:09.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.839+0000 7f084b9aa740 1 -- 172.21.15.5:0/1407987677 shutdown_connections 2024-04-24T06:54:09.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.839+0000 7f084b9aa740 1 --2- 172.21.15.5:0/1407987677 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x563f2ba5ef60 0x563f2ba61350 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:09.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.839+0000 7f084b9aa740 1 --2- 172.21.15.5:0/1407987677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f2b9e7a70 0x563f2b9ebd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:09.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.839+0000 7f0823400640 0 client.27451 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:54:09.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.839+0000 7f084b9aa740 1 --2- 172.21.15.5:0/1407987677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f2b9d4970 0x563f2b9efd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:09.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.839+0000 7f084b9aa740 1 --2- 172.21.15.5:0/1407987677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f2b9d3fc0 0x563f2b9ef840 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:09.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.839+0000 7f084b9aa740 1 -- 172.21.15.5:0/1407987677 >> 172.21.15.5:0/1407987677 conn(0x563f2b9d0590 msgr2=0x563f2b9cf6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:09.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.840+0000 7f084b9aa740 1 -- 172.21.15.5:0/1407987677 shutdown_connections 2024-04-24T06:54:09.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.840+0000 7f084b9aa740 1 -- 172.21.15.5:0/1407987677 wait complete. 2024-04-24T06:54:09.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.840+0000 7f084b9aa740 20 client.27451 trim_cache size 0 max 16384 2024-04-24T06:54:09.855 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:54:09.855 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-24T06:54:09.856 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "45125394-fdb7-4c39-bae3-743f7795c7b1", 2024-04-24T06:54:09.856 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-24T06:54:09.856 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:54:09.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.840+0000 7f0843200640 1 -- 172.21.15.5:0/3084235425 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f08280a3d20 msgr2=0x7f08280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:09.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.840+0000 7f0843200640 1 --2- 172.21.15.5:0/3084235425 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f08280a3d20 0x7f08280a61e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f083c005fd0 tx=0x7f083c009500 comp rx=0 tx=0).stop 2024-04-24T06:54:09.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.840+0000 7f0843200640 1 -- 172.21.15.5:0/3084235425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0844109360 msgr2=0x7f08441a0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:09.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.840+0000 7f0843200640 1 --2- 172.21.15.5:0/3084235425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0844109360 0x7f08441a0ba0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f083000e9f0 tx=0x7f083000edf0 comp rx=0 tx=0).stop 2024-04-24T06:54:09.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.841+0000 7f0843200640 1 -- 172.21.15.5:0/3084235425 shutdown_connections 2024-04-24T06:54:09.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.841+0000 7f0843200640 1 --2- 172.21.15.5:0/3084235425 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f08280a3d20 0x7f08280a61e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:09.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.841+0000 7f0843200640 1 --2- 172.21.15.5:0/3084235425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f084419a7d0 0x7f084419ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:09.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.841+0000 7f0843200640 1 --2- 172.21.15.5:0/3084235425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0844109360 0x7f08441a0ba0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:09.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.841+0000 7f0843200640 1 --2- 172.21.15.5:0/3084235425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0844104290 0x7f08441a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:09.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.841+0000 7f0843200640 1 -- 172.21.15.5:0/3084235425 >> 172.21.15.5:0/3084235425 conn(0x7f08440ffaf0 msgr2=0x7f0844100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:09.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.841+0000 7f0843200640 1 -- 172.21.15.5:0/3084235425 shutdown_connections 2024-04-24T06:54:09.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:09.841+0000 7f0843200640 1 -- 172.21.15.5:0/3084235425 wait complete. 2024-04-24T06:54:09.866 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T06:54:09.866 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T06:54:09.945 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:09 smithi137 ceph-mon[33017]: pgmap v312: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-24T06:54:09.945 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:09 smithi137 ceph-mon[33017]: osdmap e96: 12 total, 12 up, 12 in 2024-04-24T06:54:10.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.101+0000 7f2e16000640 1 -- 172.21.15.5:0/2445759454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e10112c80 msgr2=0x7f2e10115070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:10.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.101+0000 7f2e16000640 1 --2- 172.21.15.5:0/2445759454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e10112c80 0x7f2e10115070 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2e08009cd0 tx=0x7f2e0802a8f0 comp rx=0 tx=0).stop 2024-04-24T06:54:10.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.101+0000 7f2e16000640 1 -- 172.21.15.5:0/2445759454 shutdown_connections 2024-04-24T06:54:10.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.101+0000 7f2e16000640 1 --2- 172.21.15.5:0/2445759454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e10112c80 0x7f2e10115070 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:10.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.101+0000 7f2e16000640 1 --2- 172.21.15.5:0/2445759454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e10076e50 0x7f2e10112740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:10.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.101+0000 7f2e16000640 1 --2- 172.21.15.5:0/2445759454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e10076530 0x7f2e10076910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:10.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.101+0000 7f2e16000640 1 -- 172.21.15.5:0/2445759454 >> 172.21.15.5:0/2445759454 conn(0x7f2e100ffad0 msgr2=0x7f2e10101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:10.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.102+0000 7f2e16000640 1 -- 172.21.15.5:0/2445759454 shutdown_connections 2024-04-24T06:54:10.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.102+0000 7f2e16000640 1 -- 172.21.15.5:0/2445759454 wait complete. 2024-04-24T06:54:10.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.103+0000 7f2e16000640 1 Processor -- start 2024-04-24T06:54:10.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.103+0000 7f2e16000640 1 -- start start 2024-04-24T06:54:10.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.103+0000 7f2e16000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e10076530 0x7f2e1019ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:10.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.103+0000 7f2e16000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e10076e50 0x7f2e1019f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:10.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.103+0000 7f2e16000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e10112c80 0x7f2e101a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:10.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.103+0000 7f2e16000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2e101187f0 con 0x7f2e10112c80 2024-04-24T06:54:10.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.103+0000 7f2e16000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2e10118670 con 0x7f2e10076530 2024-04-24T06:54:10.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.103+0000 7f2e16000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2e10118970 con 0x7f2e10076e50 2024-04-24T06:54:10.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.104+0000 7f2e14c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e10076530 0x7f2e1019ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:10.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.104+0000 7f2e14c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e10076530 0x7f2e1019ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:49012/0 (socket says 172.21.15.5:49012) 2024-04-24T06:54:10.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.104+0000 7f2e14c00640 1 -- 172.21.15.5:0/1658484233 learned_addr learned my addr 172.21.15.5:0/1658484233 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:10.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.104+0000 7f2e0fe00640 1 --2- 172.21.15.5:0/1658484233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e10076e50 0x7f2e1019f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:10.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.104+0000 7f2e15600640 1 --2- 172.21.15.5:0/1658484233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e10112c80 0x7f2e101a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:10.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.104+0000 7f2e14c00640 1 -- 172.21.15.5:0/1658484233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e10076e50 msgr2=0x7f2e1019f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:10.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.104+0000 7f2e14c00640 1 --2- 172.21.15.5:0/1658484233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e10076e50 0x7f2e1019f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:10.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.104+0000 7f2e14c00640 1 -- 172.21.15.5:0/1658484233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e10112c80 msgr2=0x7f2e101a35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:10.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.104+0000 7f2e14c00640 1 --2- 172.21.15.5:0/1658484233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e10112c80 0x7f2e101a35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:10.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.104+0000 7f2e14c00640 1 -- 172.21.15.5:0/1658484233 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e101a3d20 con 0x7f2e10076530 2024-04-24T06:54:10.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.104+0000 7f2e15600640 1 --2- 172.21.15.5:0/1658484233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e10112c80 0x7f2e101a35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:54:10.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.105+0000 7f2e14c00640 1 --2- 172.21.15.5:0/1658484233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e10076530 0x7f2e1019ed30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2e0400ea60 tx=0x7f2e0400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:10.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.105+0000 7f2e0d600640 1 -- 172.21.15.5:0/1658484233 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0400cde0 con 0x7f2e10076530 2024-04-24T06:54:10.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.105+0000 7f2e0d600640 1 -- 172.21.15.5:0/1658484233 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e040035f0 con 0x7f2e10076530 2024-04-24T06:54:10.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.105+0000 7f2e0d600640 1 -- 172.21.15.5:0/1658484233 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e04010690 con 0x7f2e10076530 2024-04-24T06:54:10.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.105+0000 7f2e16000640 1 -- 172.21.15.5:0/1658484233 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e101a4010 con 0x7f2e10076530 2024-04-24T06:54:10.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.105+0000 7f2e16000640 1 -- 172.21.15.5:0/1658484233 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e101ab8a0 con 0x7f2e10076530 2024-04-24T06:54:10.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.110+0000 7f2e1a415740 1 Processor -- start 2024-04-24T06:54:10.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.110+0000 7f2e1a415740 1 -- start start 2024-04-24T06:54:10.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.110+0000 7f2e0d600640 1 -- 172.21.15.5:0/1658484233 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f2e040037a0 con 0x7f2e10076530 2024-04-24T06:54:10.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.110+0000 7f2e1a415740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5d94b6a30 0x55b5d94b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:10.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.110+0000 7f2e1a415740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5d94b7350 0x55b5d94c14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:10.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.110+0000 7f2e1a415740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5d94c1a20 0x55b5d94c3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:10.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.110+0000 7f2e1a415740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b5d941c1f0 con 0x55b5d94c1a20 2024-04-24T06:54:10.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.110+0000 7f2e1a415740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b5d93c2350 con 0x55b5d94b7350 2024-04-24T06:54:10.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.110+0000 7f2e1a415740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b5d94181b0 con 0x55b5d94b6a30 2024-04-24T06:54:10.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.111+0000 7f2e15600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5d94b6a30 0x55b5d94b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:10.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.111+0000 7f2e14c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5d94b7350 0x55b5d94c14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:10.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.111+0000 7f2e15600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5d94b6a30 0x55b5d94b6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:54818/0 (socket says 172.21.15.5:54818) 2024-04-24T06:54:10.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.111+0000 7f2e15600640 1 -- 172.21.15.5:0/521446776 learned_addr learned my addr 172.21.15.5:0/521446776 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:10.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.111+0000 7f2e0fe00640 1 --2- 172.21.15.5:0/521446776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5d94c1a20 0x55b5d94c3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:10.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.111+0000 7f2e15600640 1 -- 172.21.15.5:0/521446776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5d94b7350 msgr2=0x55b5d94c14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:10.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.111+0000 7f2e15600640 1 --2- 172.21.15.5:0/521446776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5d94b7350 0x55b5d94c14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:10.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.111+0000 7f2e15600640 1 -- 172.21.15.5:0/521446776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5d94c1a20 msgr2=0x55b5d94c3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:10.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.111+0000 7f2e15600640 1 --2- 172.21.15.5:0/521446776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5d94c1a20 0x55b5d94c3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:10.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.111+0000 7f2e15600640 1 -- 172.21.15.5:0/521446776 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5d9416f50 con 0x55b5d94b6a30 2024-04-24T06:54:10.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.112+0000 7f2e15600640 1 --2- 172.21.15.5:0/521446776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5d94b6a30 0x55b5d94b6e10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2e1006b360 tx=0x7f2e08009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1b10b695606ba65 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:10.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.112+0000 7f2e01c00640 1 -- 172.21.15.5:0/521446776 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e08030720 con 0x55b5d94b6a30 2024-04-24T06:54:10.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.112+0000 7f2e01c00640 1 -- 172.21.15.5:0/521446776 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e08036040 con 0x55b5d94b6a30 2024-04-24T06:54:10.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.112+0000 7f2e0d600640 1 --2- 172.21.15.5:0/1658484233 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f2df80a3d20 0x7f2df80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:10.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.112+0000 7f2e01c00640 1 -- 172.21.15.5:0/521446776 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e080043c0 con 0x55b5d94b6a30 2024-04-24T06:54:10.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.112+0000 7f2e0d600640 1 -- 172.21.15.5:0/1658484233 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8586+0+0 (secure 0 0 0) 0x7f2e040d0bb0 con 0x7f2e10076530 2024-04-24T06:54:10.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.113+0000 7f2e14c00640 1 --2- 172.21.15.5:0/1658484233 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f2df80a3d20 0x7f2df80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:10.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.113+0000 7f2e1a415740 1 -- 172.21.15.5:0/521446776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5d94b6a30 msgr2=0x55b5d94b6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:10.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.113+0000 7f2e1a415740 1 --2- 172.21.15.5:0/521446776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5d94b6a30 0x55b5d94b6e10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2e1006b360 tx=0x7f2e08009a20 comp rx=0 tx=0).stop 2024-04-24T06:54:10.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.113+0000 7f2e14c00640 1 --2- 172.21.15.5:0/1658484233 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f2df80a3d20 0x7f2df80a61e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x55b5d94b7ff0 tx=0x7f2e040d4660 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:10.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.113+0000 7f2e1a415740 1 -- 172.21.15.5:0/521446776 shutdown_connections 2024-04-24T06:54:10.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.113+0000 7f2e1a415740 1 --2- 172.21.15.5:0/521446776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5d94c1a20 0x55b5d94c3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:10.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.113+0000 7f2e1a415740 1 --2- 172.21.15.5:0/521446776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5d94b7350 0x55b5d94c14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:10.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.113+0000 7f2e1a415740 1 --2- 172.21.15.5:0/521446776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5d94b6a30 0x55b5d94b6e10 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:10.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.113+0000 7f2e1a415740 1 -- 172.21.15.5:0/521446776 >> 172.21.15.5:0/521446776 conn(0x55b5d94b2d50 msgr2=0x55b5d94b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:10.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.114+0000 7f2e1a415740 1 -- 172.21.15.5:0/521446776 shutdown_connections 2024-04-24T06:54:10.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.114+0000 7f2e1a415740 1 -- 172.21.15.5:0/521446776 wait complete. 2024-04-24T06:54:10.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.114+0000 7f2e1a415740 1 -- start start 2024-04-24T06:54:10.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.115+0000 7f2e1a415740 1 Processor -- start 2024-04-24T06:54:10.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.115+0000 7f2e1a415740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5d94b6a30 0x55b5d94bea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:10.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.115+0000 7f2e1a415740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5d94b7350 0x55b5d94bef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:10.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.115+0000 7f2e1a415740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5d94c1a20 0x55b5d94bf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:10.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.115+0000 7f2e1a415740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b5d94c47b0 con 0x55b5d94c1a20 2024-04-24T06:54:10.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.115+0000 7f2e1a415740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b5d93d5690 con 0x55b5d94b6a30 2024-04-24T06:54:10.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.115+0000 7f2e1a415740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b5d94c4930 con 0x55b5d94b7350 2024-04-24T06:54:10.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.115+0000 7f2e1a415740 10 client.0 _ll_register_callbacks cb 0x55b5d9411650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:54:10.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.115+0000 7f2e15600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5d94b6a30 0x55b5d94bea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:10.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.115+0000 7f2e15600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5d94b6a30 0x55b5d94bea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:49032/0 (socket says 172.21.15.5:49032) 2024-04-24T06:54:10.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.116+0000 7f2e15600640 1 -- 172.21.15.5:0/3454370933 learned_addr learned my addr 172.21.15.5:0/3454370933 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:10.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.116+0000 7f2e0fe00640 1 --2- 172.21.15.5:0/3454370933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5d94b7350 0x55b5d94bef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:10.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.116+0000 7f2e15600640 1 --2- 172.21.15.5:0/3454370933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5d94c1a20 0x55b5d94bf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:10.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.116+0000 7f2e0fe00640 1 -- 172.21.15.5:0/3454370933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5d94b6a30 msgr2=0x55b5d94bea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:10.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.116+0000 7f2e0fe00640 1 --2- 172.21.15.5:0/3454370933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5d94b6a30 0x55b5d94bea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:10.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.116+0000 7f2e0fe00640 1 -- 172.21.15.5:0/3454370933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5d94c1a20 msgr2=0x55b5d94bf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:10.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.116+0000 7f2e0fe00640 1 --2- 172.21.15.5:0/3454370933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5d94c1a20 0x55b5d94bf4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:10.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.116+0000 7f2e0fe00640 1 -- 172.21.15.5:0/3454370933 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5d93d4920 con 0x55b5d94b7350 2024-04-24T06:54:10.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.116+0000 7f2e15600640 1 --2- 172.21.15.5:0/3454370933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5d94b6a30 0x55b5d94bea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:54:10.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.117+0000 7f2e0fe00640 1 --2- 172.21.15.5:0/3454370933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5d94b7350 0x55b5d94bef80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2e10101fa0 tx=0x7f2dfc012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f4426810968deaa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:10.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.117+0000 7f2df6000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:54:10.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.117+0000 7f2df6000640 1 -- 172.21.15.5:0/3454370933 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc004030 con 0x55b5d94b7350 2024-04-24T06:54:10.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.117+0000 7f2e1a415740 1 -- 172.21.15.5:0/3454370933 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e08036040 con 0x55b5d94b7350 2024-04-24T06:54:10.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.117+0000 7f2df6000640 1 -- 172.21.15.5:0/3454370933 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8586+0+0 (secure 0 0 0) 0x7f2dfc0041d0 con 0x55b5d94b7350 2024-04-24T06:54:10.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.117+0000 7f2df6000640 1 -- 172.21.15.5:0/3454370933 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2dfc0043d0 con 0x55b5d94b7350 2024-04-24T06:54:10.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.117+0000 7f2e1a415740 10 client.27463 fetch_fsmap learned FSMap version 27 2024-04-24T06:54:10.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.117+0000 7f2e1a415740 1 -- 172.21.15.5:0/3454370933 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b5d91f87d0 con 0x55b5d94b7350 2024-04-24T06:54:10.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.118+0000 7f2df6000640 1 -- 172.21.15.5:0/3454370933 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e08036040 con 0x55b5d94b7350 2024-04-24T06:54:10.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.118+0000 7f2df6000640 1 -- 172.21.15.5:0/3454370933 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f2dfc003c40 con 0x55b5d94b7350 2024-04-24T06:54:10.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.118+0000 7f2e1a415740 10 client.27463 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:54:10.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.118+0000 7f2e1a415740 10 client.27463 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:54:10.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.118+0000 7f2e1a415740 20 client.27463 populate_metadata read hostname 'smithi005' 2024-04-24T06:54:10.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.118+0000 7f2e1a415740 1 --2- 172.21.15.5:0/3454370933 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55b5d94d1e20 0x55b5d94d4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:10.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.118+0000 7f2e1a415740 4 client.27463 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:54:10.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.118+0000 7f2e1a415740 1 -- 172.21.15.5:0/3454370933 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5d9429c10 con 0x55b5d94d1e20 2024-04-24T06:54:10.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.118+0000 7f2e15600640 1 --2- 172.21.15.5:0/3454370933 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55b5d94d1e20 0x55b5d94d4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:10.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.119+0000 7f2e15600640 1 --2- 172.21.15.5:0/3454370933 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55b5d94d1e20 0x55b5d94d4240 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6bc25fd19d7415b server_cookie=e606a3a6e3ad3fb6 in_seq=0 out_seq=0 2024-04-24T06:54:10.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.119+0000 7f2df6000640 10 client.27463 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:54:10.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.120+0000 7f2df6000640 1 -- 172.21.15.5:0/3454370933 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b5d9429c10 con 0x55b5d94d1e20 2024-04-24T06:54:10.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.120+0000 7f2df6000640 10 client.27463 handle_command_reply: tid=0 2024-04-24T06:54:10.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.120+0000 7f2e1a415740 1 client.27463 shutdown 2024-04-24T06:54:10.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.120+0000 7f2e1a415740 1 -- 172.21.15.5:0/3454370933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5d94b7350 msgr2=0x55b5d94bef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:10.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.120+0000 7f2e1a415740 1 --2- 172.21.15.5:0/3454370933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5d94b7350 0x55b5d94bef80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2e10101fa0 tx=0x7f2dfc012b60 comp rx=0 tx=0).stop 2024-04-24T06:54:10.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.120+0000 7f2e1a415740 1 -- 172.21.15.5:0/3454370933 shutdown_connections 2024-04-24T06:54:10.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.120+0000 7f2e1a415740 1 --2- 172.21.15.5:0/3454370933 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55b5d94d1e20 0x55b5d94d4240 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:10.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.120+0000 7f2e1a415740 1 --2- 172.21.15.5:0/3454370933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5d94c1a20 0x55b5d94bf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:10.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.120+0000 7f2e1a415740 1 --2- 172.21.15.5:0/3454370933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5d94b7350 0x55b5d94bef80 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:10.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.120+0000 7f2e1a415740 1 --2- 172.21.15.5:0/3454370933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5d94b6a30 0x55b5d94bea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:10.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.120+0000 7f2e1a415740 1 -- 172.21.15.5:0/3454370933 >> 172.21.15.5:0/3454370933 conn(0x55b5d94b2d50 msgr2=0x55b5d94b8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:10.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.120+0000 7f2df6000640 0 client.27463 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:54:10.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.121+0000 7f2e1a415740 1 -- 172.21.15.5:0/3454370933 shutdown_connections 2024-04-24T06:54:10.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.121+0000 7f2e1a415740 1 -- 172.21.15.5:0/3454370933 wait complete. 2024-04-24T06:54:10.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.121+0000 7f2e1a415740 20 client.27463 trim_cache size 0 max 16384 2024-04-24T06:54:10.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.145+0000 7f2e1a415740 1 Processor -- start 2024-04-24T06:54:10.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.145+0000 7f2e1a415740 1 -- start start 2024-04-24T06:54:10.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.145+0000 7f2e1a415740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5d94b6a30 0x55b5d94cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:10.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.145+0000 7f2e1a415740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5d94b7350 0x55b5d94d9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:10.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.145+0000 7f2e1a415740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5d94c1a20 0x55b5d94da010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:10.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.145+0000 7f2e1a415740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2e101b6cf0 con 0x55b5d94b6a30 2024-04-24T06:54:10.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.145+0000 7f2e1a415740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2e101b45c0 con 0x55b5d94c1a20 2024-04-24T06:54:10.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.145+0000 7f2e1a415740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2e101b2430 con 0x55b5d94b7350 2024-04-24T06:54:10.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.146+0000 7f2e0fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5d94b7350 0x55b5d94d9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:10.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.146+0000 7f2e0fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5d94b7350 0x55b5d94d9ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:54840/0 (socket says 172.21.15.5:54840) 2024-04-24T06:54:10.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.146+0000 7f2e0fe00640 1 -- 172.21.15.5:0/3695684892 learned_addr learned my addr 172.21.15.5:0/3695684892 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:10.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.146+0000 7f2e15600640 1 --2- 172.21.15.5:0/3695684892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5d94b6a30 0x55b5d94cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:10.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.146+0000 7f2e0fe00640 1 -- 172.21.15.5:0/3695684892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5d94c1a20 msgr2=0x55b5d94da010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:10.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.146+0000 7f2e15600640 1 --2- 172.21.15.5:0/3695684892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5d94c1a20 0x55b5d94da010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:10.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.146+0000 7f2e0fe00640 1 --2- 172.21.15.5:0/3695684892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5d94c1a20 0x55b5d94da010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:10.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.146+0000 7f2e0fe00640 1 -- 172.21.15.5:0/3695684892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5d94b6a30 msgr2=0x55b5d94cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:10.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.146+0000 7f2e0fe00640 1 --2- 172.21.15.5:0/3695684892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5d94b6a30 0x55b5d94cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:10.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.146+0000 7f2e0fe00640 1 -- 172.21.15.5:0/3695684892 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5d94bd9f0 con 0x55b5d94b7350 2024-04-24T06:54:10.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.146+0000 7f2e15600640 1 --2- 172.21.15.5:0/3695684892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5d94c1a20 0x55b5d94da010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:54:10.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.146+0000 7f2e0fe00640 1 --2- 172.21.15.5:0/3695684892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5d94b7350 0x55b5d94d9ad0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc010670 tx=0x7f2dfc005e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=177bef87b5662f0e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:10.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.147+0000 7f2e01200640 1 -- 172.21.15.5:0/3695684892 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc00f920 con 0x55b5d94b7350 2024-04-24T06:54:10.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.147+0000 7f2e01200640 1 -- 172.21.15.5:0/3695684892 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dfc0106f0 con 0x55b5d94b7350 2024-04-24T06:54:10.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.147+0000 7f2e01200640 1 -- 172.21.15.5:0/3695684892 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc02d6b0 con 0x55b5d94b7350 2024-04-24T06:54:10.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.148+0000 7f2e1a415740 1 -- 172.21.15.5:0/3695684892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5d94b7350 msgr2=0x55b5d94d9ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:10.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.148+0000 7f2e1a415740 1 --2- 172.21.15.5:0/3695684892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5d94b7350 0x55b5d94d9ad0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc010670 tx=0x7f2dfc005e30 comp rx=0 tx=0).stop 2024-04-24T06:54:10.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.148+0000 7f2e1a415740 1 -- 172.21.15.5:0/3695684892 shutdown_connections 2024-04-24T06:54:10.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.148+0000 7f2e1a415740 1 --2- 172.21.15.5:0/3695684892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5d94c1a20 0x55b5d94da010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:10.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.148+0000 7f2e1a415740 1 --2- 172.21.15.5:0/3695684892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5d94b7350 0x55b5d94d9ad0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:10.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.148+0000 7f2e1a415740 1 --2- 172.21.15.5:0/3695684892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5d94b6a30 0x55b5d94cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:10.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.148+0000 7f2e1a415740 1 -- 172.21.15.5:0/3695684892 >> 172.21.15.5:0/3695684892 conn(0x55b5d94b2d50 msgr2=0x55b5d94bf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:10.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.148+0000 7f2e1a415740 1 -- 172.21.15.5:0/3695684892 shutdown_connections 2024-04-24T06:54:10.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.148+0000 7f2e1a415740 1 -- 172.21.15.5:0/3695684892 wait complete. 2024-04-24T06:54:10.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.148+0000 7f2e1a415740 1 -- start start 2024-04-24T06:54:10.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.148+0000 7f2e1a415740 1 Processor -- start 2024-04-24T06:54:10.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.149+0000 7f2e1a415740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5d94b6a30 0x55b5d94bbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:10.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.149+0000 7f2e1a415740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5d94b7350 0x55b5d94bc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:10.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.149+0000 7f2e1a415740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5d94c1a20 0x55b5d9537340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:10.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.149+0000 7f2e1a415740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2e101b2a80 con 0x55b5d94c1a20 2024-04-24T06:54:10.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.149+0000 7f2e1a415740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2e101b3ef0 con 0x55b5d94b6a30 2024-04-24T06:54:10.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.149+0000 7f2e1a415740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2e101b3150 con 0x55b5d94b7350 2024-04-24T06:54:10.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.149+0000 7f2e1a415740 10 client.0 _ll_register_callbacks cb 0x7f2dfc0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:54:10.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.149+0000 7f2e0fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5d94b7350 0x55b5d94bc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:10.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.149+0000 7f2e0fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5d94b7350 0x55b5d94bc0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:54852/0 (socket says 172.21.15.5:54852) 2024-04-24T06:54:10.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.149+0000 7f2e0fe00640 1 -- 172.21.15.5:0/138517635 learned_addr learned my addr 172.21.15.5:0/138517635 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:10.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.150+0000 7f2e15600640 1 --2- 172.21.15.5:0/138517635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5d94b6a30 0x55b5d94bbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:10.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.150+0000 7f2e15600640 1 --2- 172.21.15.5:0/138517635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5d94c1a20 0x55b5d9537340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:10.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.150+0000 7f2e0fe00640 1 -- 172.21.15.5:0/138517635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5d94b6a30 msgr2=0x55b5d94bbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:10.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.150+0000 7f2e0fe00640 1 --2- 172.21.15.5:0/138517635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5d94b6a30 0x55b5d94bbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:10.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.150+0000 7f2e0fe00640 1 -- 172.21.15.5:0/138517635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5d94c1a20 msgr2=0x55b5d9537340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:10.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.150+0000 7f2e0fe00640 1 --2- 172.21.15.5:0/138517635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5d94c1a20 0x55b5d9537340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:10.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.150+0000 7f2e0fe00640 1 -- 172.21.15.5:0/138517635 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5d912e3f0 con 0x55b5d94b7350 2024-04-24T06:54:10.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.150+0000 7f2e0fe00640 1 --2- 172.21.15.5:0/138517635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5d94b7350 0x55b5d94bc0f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc012420 tx=0x7f2dfc00ffd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcea7e4c9a6dbed3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:10.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.150+0000 7f2df6000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:54:10.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.151+0000 7f2df6000640 1 -- 172.21.15.5:0/138517635 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc005250 con 0x55b5d94b7350 2024-04-24T06:54:10.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.151+0000 7f2df6000640 1 -- 172.21.15.5:0/138517635 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8586+0+0 (secure 0 0 0) 0x7f2dfc0053f0 con 0x55b5d94b7350 2024-04-24T06:54:10.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.151+0000 7f2e1a415740 1 -- 172.21.15.5:0/138517635 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dfc0106f0 con 0x55b5d94b7350 2024-04-24T06:54:10.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.151+0000 7f2df6000640 1 -- 172.21.15.5:0/138517635 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2dfc0108a0 con 0x55b5d94b7350 2024-04-24T06:54:10.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.151+0000 7f2e1a415740 10 client.27469 fetch_fsmap learned FSMap version 27 2024-04-24T06:54:10.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.151+0000 7f2e1a415740 1 -- 172.21.15.5:0/138517635 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b5d90ee0f0 con 0x55b5d94b7350 2024-04-24T06:54:10.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.151+0000 7f2df6000640 1 -- 172.21.15.5:0/138517635 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dfc0106f0 con 0x55b5d94b7350 2024-04-24T06:54:10.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.152+0000 7f2df6000640 1 -- 172.21.15.5:0/138517635 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f2dfc036030 con 0x55b5d94b7350 2024-04-24T06:54:10.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.152+0000 7f2e1a415740 10 client.27469 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:54:10.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.152+0000 7f2e1a415740 10 client.27469 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:54:10.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.152+0000 7f2e1a415740 20 client.27469 populate_metadata read hostname 'smithi005' 2024-04-24T06:54:10.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.152+0000 7f2e1a415740 1 --2- 172.21.15.5:0/138517635 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55b5d9538cf0 0x55b5d953b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:10.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.152+0000 7f2e1a415740 4 client.27469 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:54:10.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.152+0000 7f2e1a415740 1 -- 172.21.15.5:0/138517635 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b5d9423a80 con 0x55b5d9538cf0 2024-04-24T06:54:10.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.152+0000 7f2e15600640 1 --2- 172.21.15.5:0/138517635 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55b5d9538cf0 0x55b5d953b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:10.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.153+0000 7f2e15600640 1 --2- 172.21.15.5:0/138517635 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55b5d9538cf0 0x55b5d953b110 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fea560b1bc4ce0cb server_cookie=ccd7f097b68c74e6 in_seq=0 out_seq=0 2024-04-24T06:54:10.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.153+0000 7f2df6000640 10 client.27469 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:54:10.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.154+0000 7f2df6000640 1 -- 172.21.15.5:0/138517635 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55b5d9423a80 con 0x55b5d9538cf0 2024-04-24T06:54:10.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.154+0000 7f2df6000640 10 client.27469 handle_command_reply: tid=0 2024-04-24T06:54:10.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.154+0000 7f2e1a415740 1 client.27469 shutdown 2024-04-24T06:54:10.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.154+0000 7f2e1a415740 1 -- 172.21.15.5:0/138517635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5d94b7350 msgr2=0x55b5d94bc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:10.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.154+0000 7f2e1a415740 1 --2- 172.21.15.5:0/138517635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5d94b7350 0x55b5d94bc0f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc012420 tx=0x7f2dfc00ffd0 comp rx=0 tx=0).stop 2024-04-24T06:54:10.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.154+0000 7f2e1a415740 1 -- 172.21.15.5:0/138517635 shutdown_connections 2024-04-24T06:54:10.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.154+0000 7f2e1a415740 1 --2- 172.21.15.5:0/138517635 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55b5d9538cf0 0x55b5d953b110 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:10.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.154+0000 7f2e1a415740 1 --2- 172.21.15.5:0/138517635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5d94c1a20 0x55b5d9537340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:10.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.154+0000 7f2e1a415740 1 --2- 172.21.15.5:0/138517635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5d94b7350 0x55b5d94bc0f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:10.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.154+0000 7f2e1a415740 1 --2- 172.21.15.5:0/138517635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5d94b6a30 0x55b5d94bbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:10.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.154+0000 7f2df6000640 0 client.27469 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:54:10.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.154+0000 7f2e1a415740 1 -- 172.21.15.5:0/138517635 >> 172.21.15.5:0/138517635 conn(0x55b5d94b2d50 msgr2=0x55b5d94b1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:10.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.154+0000 7f2e1a415740 1 -- 172.21.15.5:0/138517635 shutdown_connections 2024-04-24T06:54:10.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.154+0000 7f2e1a415740 1 -- 172.21.15.5:0/138517635 wait complete. 2024-04-24T06:54:10.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.154+0000 7f2e1a415740 20 client.27469 trim_cache size 0 max 16384 2024-04-24T06:54:10.161 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:54:10.161 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1000 inodes in the stack)", 2024-04-24T06:54:10.161 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-24T06:54:10.161 INFO:teuthology.orchestra.run.smithi005.stdout: "45125394-fdb7-4c39-bae3-743f7795c7b1": { 2024-04-24T06:54:10.161 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-24T06:54:10.162 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "45125394-fdb7-4c39-bae3-743f7795c7b1", 2024-04-24T06:54:10.162 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-04-24T06:54:10.162 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:10.162 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:10.162 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:54:10.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.155+0000 7f2e16000640 1 -- 172.21.15.5:0/1658484233 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f2df80a3d20 msgr2=0x7f2df80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:10.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.155+0000 7f2e16000640 1 --2- 172.21.15.5:0/1658484233 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f2df80a3d20 0x7f2df80a61e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x55b5d94b7ff0 tx=0x7f2e040d4660 comp rx=0 tx=0).stop 2024-04-24T06:54:10.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.155+0000 7f2e16000640 1 -- 172.21.15.5:0/1658484233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e10076530 msgr2=0x7f2e1019ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:10.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.155+0000 7f2e16000640 1 --2- 172.21.15.5:0/1658484233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e10076530 0x7f2e1019ed30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2e0400ea60 tx=0x7f2e0400ef20 comp rx=0 tx=0).stop 2024-04-24T06:54:10.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.155+0000 7f2e16000640 1 -- 172.21.15.5:0/1658484233 shutdown_connections 2024-04-24T06:54:10.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.155+0000 7f2e16000640 1 --2- 172.21.15.5:0/1658484233 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f2df80a3d20 0x7f2df80a61e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:10.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.155+0000 7f2e16000640 1 --2- 172.21.15.5:0/1658484233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e10112c80 0x7f2e101a35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:10.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.155+0000 7f2e16000640 1 --2- 172.21.15.5:0/1658484233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e10076e50 0x7f2e1019f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:10.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.155+0000 7f2e16000640 1 --2- 172.21.15.5:0/1658484233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e10076530 0x7f2e1019ed30 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:10.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.155+0000 7f2e16000640 1 -- 172.21.15.5:0/1658484233 >> 172.21.15.5:0/1658484233 conn(0x7f2e100ffad0 msgr2=0x7f2e101143a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:10.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.155+0000 7f2e16000640 1 -- 172.21.15.5:0/1658484233 shutdown_connections 2024-04-24T06:54:10.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:10.155+0000 7f2e16000640 1 -- 172.21.15.5:0/1658484233 wait complete. 2024-04-24T06:54:10.177 INFO:tasks.cephfs.filesystem:scrub status for tag:45125394-fdb7-4c39-bae3-743f7795c7b1 - {'path': '/', 'tag': '45125394-fdb7-4c39-bae3-743f7795c7b1', 'options': 'recursive,force'} 2024-04-24T06:54:10.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:10 smithi005 ceph-mon[27798]: scrub queued for path: / 2024-04-24T06:54:10.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:10 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:54:10.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:10 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:54:10.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:10 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:54:10.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:10 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:54:10.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:10 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:54:10.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:10 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:54:10.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:10 smithi037 ceph-mon[32864]: scrub queued for path: / 2024-04-24T06:54:10.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:10 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:54:10.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:10 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:54:10.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:10 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:54:10.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:10 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:54:10.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:10 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:54:10.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:10 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:54:10.945 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:10 smithi137 ceph-mon[33017]: scrub queued for path: / 2024-04-24T06:54:10.945 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:10 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:54:10.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:10 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:54:10.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:10 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:54:10.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:10 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:54:10.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:10 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:54:10.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:10 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:54:11.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:11 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:54:11.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:11 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:54:11.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:11 smithi005 ceph-mon[27798]: pgmap v314: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-24T06:54:11.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:11 smithi005 ceph-mon[27798]: scrub summary: active paths [/] 2024-04-24T06:54:11.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:11 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:54:11.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:11 smithi005 ceph-mon[27798]: scrub summary: idle+waiting paths [/] 2024-04-24T06:54:11.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:11 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:54:11.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:11 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:54:11.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:11 smithi037 ceph-mon[32864]: pgmap v314: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-24T06:54:11.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:11 smithi037 ceph-mon[32864]: scrub summary: active paths [/] 2024-04-24T06:54:11.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:11 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:54:11.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:11 smithi037 ceph-mon[32864]: scrub summary: idle+waiting paths [/] 2024-04-24T06:54:11.945 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:11 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:54:11.945 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:11 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:54:11.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:11 smithi137 ceph-mon[33017]: pgmap v314: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-24T06:54:11.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:11 smithi137 ceph-mon[33017]: scrub summary: active paths [/] 2024-04-24T06:54:11.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:11 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:54:11.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:11 smithi137 ceph-mon[33017]: scrub summary: idle+waiting paths [/] 2024-04-24T06:54:13.136 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-24T06:54:13.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.353+0000 7f404ba00640 1 -- 172.21.15.5:0/4160051035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4044104490 msgr2=0x7f4044104870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:13.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.353+0000 7f404ba00640 1 --2- 172.21.15.5:0/4160051035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4044104490 0x7f4044104870 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4038009900 tx=0x7f403802a400 comp rx=0 tx=0).stop 2024-04-24T06:54:13.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.353+0000 7f404ba00640 1 -- 172.21.15.5:0/4160051035 shutdown_connections 2024-04-24T06:54:13.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.353+0000 7f404ba00640 1 --2- 172.21.15.5:0/4160051035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4044109560 0x7f4044111130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:13.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.353+0000 7f404ba00640 1 --2- 172.21.15.5:0/4160051035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4044104e40 0x7f4044108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:13.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.353+0000 7f404ba00640 1 --2- 172.21.15.5:0/4160051035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4044104490 0x7f4044104870 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:13.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.353+0000 7f404ba00640 1 -- 172.21.15.5:0/4160051035 >> 172.21.15.5:0/4160051035 conn(0x7f40440ffcb0 msgr2=0x7f40441020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:13.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.353+0000 7f404ba00640 1 -- 172.21.15.5:0/4160051035 shutdown_connections 2024-04-24T06:54:13.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.353+0000 7f404ba00640 1 -- 172.21.15.5:0/4160051035 wait complete. 2024-04-24T06:54:13.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.354+0000 7f404ba00640 1 Processor -- start 2024-04-24T06:54:13.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.355+0000 7f404ba00640 1 -- start start 2024-04-24T06:54:13.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.355+0000 7f404ba00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4044104490 0x7f40441a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:13.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.355+0000 7f404ba00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4044109560 0x7f40441a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:13.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.355+0000 7f404ba00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f404419a990 0x7f404419ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:13.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.355+0000 7f404ba00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f40441145f0 con 0x7f404419a990 2024-04-24T06:54:13.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.355+0000 7f404ba00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4044114470 con 0x7f4044104490 2024-04-24T06:54:13.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.355+0000 7f404ba00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4044114770 con 0x7f4044109560 2024-04-24T06:54:13.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.355+0000 7f4049000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4044104490 0x7f40441a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:13.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.355+0000 7f4049a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f404419a990 0x7f404419ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:13.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.355+0000 7f4049000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4044104490 0x7f40441a0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:49062/0 (socket says 172.21.15.5:49062) 2024-04-24T06:54:13.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.355+0000 7f4049000640 1 -- 172.21.15.5:0/676192396 learned_addr learned my addr 172.21.15.5:0/676192396 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:13.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.355+0000 7f4043e00640 1 --2- 172.21.15.5:0/676192396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4044109560 0x7f40441a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:13.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.356+0000 7f4049000640 1 -- 172.21.15.5:0/676192396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4044109560 msgr2=0x7f40441a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:13.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.356+0000 7f4049000640 1 --2- 172.21.15.5:0/676192396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4044109560 0x7f40441a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:13.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.356+0000 7f4049000640 1 -- 172.21.15.5:0/676192396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f404419a990 msgr2=0x7f404419ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:13.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.356+0000 7f4049000640 1 --2- 172.21.15.5:0/676192396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f404419a990 0x7f404419ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:13.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.356+0000 7f4049000640 1 -- 172.21.15.5:0/676192396 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f404419b4b0 con 0x7f4044104490 2024-04-24T06:54:13.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.356+0000 7f4049000640 1 --2- 172.21.15.5:0/676192396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4044104490 0x7f40441a0790 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4038006010 tx=0x7f40380042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:13.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.356+0000 7f4041600640 1 -- 172.21.15.5:0/676192396 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403803f030 con 0x7f4044104490 2024-04-24T06:54:13.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.357+0000 7f4041600640 1 -- 172.21.15.5:0/676192396 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4038003ce0 con 0x7f4044104490 2024-04-24T06:54:13.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.357+0000 7f4041600640 1 -- 172.21.15.5:0/676192396 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4038035070 con 0x7f4044104490 2024-04-24T06:54:13.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.357+0000 7f404ba00640 1 -- 172.21.15.5:0/676192396 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f404419b740 con 0x7f4044104490 2024-04-24T06:54:13.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.357+0000 7f404ba00640 1 -- 172.21.15.5:0/676192396 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40441a7890 con 0x7f4044104490 2024-04-24T06:54:13.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.358+0000 7f404ba00640 1 -- 172.21.15.5:0/676192396 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f404419be70 con 0x7f4044104490 2024-04-24T06:54:13.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.360+0000 7f4041600640 1 -- 172.21.15.5:0/676192396 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f4038042030 con 0x7f4044104490 2024-04-24T06:54:13.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.361+0000 7f4041600640 1 --2- 172.21.15.5:0/676192396 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f402c0a3d10 0x7f402c0a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:13.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.362+0000 7f4041600640 1 -- 172.21.15.5:0/676192396 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8586+0+0 (secure 0 0 0) 0x7f40380f0c70 con 0x7f4044104490 2024-04-24T06:54:13.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.362+0000 7f4043e00640 1 --2- 172.21.15.5:0/676192396 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f402c0a3d10 0x7f402c0a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:13.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.362+0000 7f4043e00640 1 --2- 172.21.15.5:0/676192396 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f402c0a3d10 0x7f402c0a61d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4030009590 tx=0x7f4030005f30 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:13.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.366+0000 7f4041600640 1 -- 172.21.15.5:0/676192396 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f40380bcc20 con 0x7f4044104490 2024-04-24T06:54:13.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.598+0000 7f404ba00640 1 -- 172.21.15.5:0/676192396 --> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f4044104f00 con 0x7f402c0a3d10 2024-04-24T06:54:13.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.614+0000 7f4041600640 1 -- 172.21.15.5:0/676192396 <== mgr.25202 v2:172.21.15.5:6800/3154624651 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f4044104f00 con 0x7f402c0a3d10 2024-04-24T06:54:13.615 INFO:teuthology.orchestra.run.smithi005.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-24T00:00:00", "created": "2024-04-24T06:44:12", "first": "2024-04-24T06:45:00", "last": "2024-04-24T06:54:00", "last_pruned": "2024-04-24T06:54:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-04-24T06:54:13.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.617+0000 7f404ba00640 1 -- 172.21.15.5:0/676192396 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f402c0a3d10 msgr2=0x7f402c0a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:13.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.617+0000 7f404ba00640 1 --2- 172.21.15.5:0/676192396 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f402c0a3d10 0x7f402c0a61d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4030009590 tx=0x7f4030005f30 comp rx=0 tx=0).stop 2024-04-24T06:54:13.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.617+0000 7f404ba00640 1 -- 172.21.15.5:0/676192396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4044104490 msgr2=0x7f40441a0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:13.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.617+0000 7f404ba00640 1 --2- 172.21.15.5:0/676192396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4044104490 0x7f40441a0790 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4038006010 tx=0x7f40380042f0 comp rx=0 tx=0).stop 2024-04-24T06:54:13.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.617+0000 7f404ba00640 1 -- 172.21.15.5:0/676192396 shutdown_connections 2024-04-24T06:54:13.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.618+0000 7f404ba00640 1 --2- 172.21.15.5:0/676192396 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f402c0a3d10 0x7f402c0a61d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:13.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.618+0000 7f404ba00640 1 --2- 172.21.15.5:0/676192396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f404419a990 0x7f404419ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:13.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.618+0000 7f404ba00640 1 --2- 172.21.15.5:0/676192396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4044109560 0x7f40441a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:13.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.618+0000 7f404ba00640 1 --2- 172.21.15.5:0/676192396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4044104490 0x7f40441a0790 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:13.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.618+0000 7f404ba00640 1 -- 172.21.15.5:0/676192396 >> 172.21.15.5:0/676192396 conn(0x7f40440ffcb0 msgr2=0x7f4044101060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:13.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.618+0000 7f404ba00640 1 -- 172.21.15.5:0/676192396 shutdown_connections 2024-04-24T06:54:13.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.618+0000 7f404ba00640 1 -- 172.21.15.5:0/676192396 wait complete. 2024-04-24T06:54:13.638 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-24T06:54:13.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:13 smithi005 ceph-mon[27798]: pgmap v315: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-24T06:54:13.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:13 smithi037 ceph-mon[32864]: pgmap v315: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-24T06:54:13.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.846+0000 7fe5eb400640 1 -- 172.21.15.5:0/1935724878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5e4103b90 msgr2=0x7fe5e410c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:13.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.846+0000 7fe5eb400640 1 --2- 172.21.15.5:0/1935724878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5e4103b90 0x7fe5e410c070 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc009900 tx=0x7fe5cc02a480 comp rx=0 tx=0).stop 2024-04-24T06:54:13.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.846+0000 7fe5eb400640 1 -- 172.21.15.5:0/1935724878 shutdown_connections 2024-04-24T06:54:13.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.846+0000 7fe5eb400640 1 --2- 172.21.15.5:0/1935724878 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5e410c640 0x7fe5e410ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:13.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.846+0000 7fe5eb400640 1 --2- 172.21.15.5:0/1935724878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5e4103b90 0x7fe5e410c070 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:13.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.846+0000 7fe5eb400640 1 --2- 172.21.15.5:0/1935724878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5e4103270 0x7fe5e4103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:13.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.846+0000 7fe5eb400640 1 -- 172.21.15.5:0/1935724878 >> 172.21.15.5:0/1935724878 conn(0x7fe5e4078020 msgr2=0x7fe5e4078430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:13.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.846+0000 7fe5eb400640 1 -- 172.21.15.5:0/1935724878 shutdown_connections 2024-04-24T06:54:13.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.846+0000 7fe5eb400640 1 -- 172.21.15.5:0/1935724878 wait complete. 2024-04-24T06:54:13.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.847+0000 7fe5eb400640 1 Processor -- start 2024-04-24T06:54:13.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.848+0000 7fe5eb400640 1 -- start start 2024-04-24T06:54:13.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.848+0000 7fe5eb400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5e4103270 0x7fe5e419e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:13.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.848+0000 7fe5eb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5e4103b90 0x7fe5e419ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:13.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.848+0000 7fe5eb400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5e410c640 0x7fe5e4198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:13.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.848+0000 7fe5eb400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e4112420 con 0x7fe5e4103b90 2024-04-24T06:54:13.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.848+0000 7fe5eb400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e41122a0 con 0x7fe5e410c640 2024-04-24T06:54:13.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.848+0000 7fe5eb400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e41125a0 con 0x7fe5e4103270 2024-04-24T06:54:13.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.848+0000 7fe5e3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5e4103b90 0x7fe5e419ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:13.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.848+0000 7fe5e9400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5e410c640 0x7fe5e4198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:13.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.848+0000 7fe5e9400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5e410c640 0x7fe5e4198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:49080/0 (socket says 172.21.15.5:49080) 2024-04-24T06:54:13.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.848+0000 7fe5e9400640 1 -- 172.21.15.5:0/3839811482 learned_addr learned my addr 172.21.15.5:0/3839811482 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:13.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.848+0000 7fe5e8a00640 1 --2- 172.21.15.5:0/3839811482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5e4103270 0x7fe5e419e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:13.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.849+0000 7fe5e9400640 1 -- 172.21.15.5:0/3839811482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5e4103270 msgr2=0x7fe5e419e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:13.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.849+0000 7fe5e9400640 1 --2- 172.21.15.5:0/3839811482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5e4103270 0x7fe5e419e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:13.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.849+0000 7fe5e9400640 1 -- 172.21.15.5:0/3839811482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5e4103b90 msgr2=0x7fe5e419ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:13.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.849+0000 7fe5e9400640 1 --2- 172.21.15.5:0/3839811482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5e4103b90 0x7fe5e419ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:13.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.849+0000 7fe5e9400640 1 -- 172.21.15.5:0/3839811482 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5e4199040 con 0x7fe5e410c640 2024-04-24T06:54:13.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.849+0000 7fe5e3e00640 1 --2- 172.21.15.5:0/3839811482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5e4103b90 0x7fe5e419ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:54:13.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.849+0000 7fe5e9400640 1 --2- 172.21.15.5:0/3839811482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5e410c640 0x7fe5e4198780 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc00e970 tx=0x7fe5dc00ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:13.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.849+0000 7fe5e1600640 1 -- 172.21.15.5:0/3839811482 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc00ccc0 con 0x7fe5e410c640 2024-04-24T06:54:13.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.849+0000 7fe5eb400640 1 -- 172.21.15.5:0/3839811482 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5e4199330 con 0x7fe5e410c640 2024-04-24T06:54:13.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.849+0000 7fe5e1600640 1 -- 172.21.15.5:0/3839811482 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5dc005500 con 0x7fe5e410c640 2024-04-24T06:54:13.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.849+0000 7fe5e1600640 1 -- 172.21.15.5:0/3839811482 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc010690 con 0x7fe5e410c640 2024-04-24T06:54:13.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.850+0000 7fe5eb400640 1 -- 172.21.15.5:0/3839811482 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5e41a55f0 con 0x7fe5e410c640 2024-04-24T06:54:13.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.851+0000 7fe5eb400640 1 -- 172.21.15.5:0/3839811482 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5e406a040 con 0x7fe5e410c640 2024-04-24T06:54:13.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.853+0000 7fe5e1600640 1 -- 172.21.15.5:0/3839811482 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fe5dc0046c0 con 0x7fe5e410c640 2024-04-24T06:54:13.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.854+0000 7fe5e1600640 1 --2- 172.21.15.5:0/3839811482 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fe5c80a3d10 0x7fe5c80a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:13.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.854+0000 7fe5e1600640 1 -- 172.21.15.5:0/3839811482 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8586+0+0 (secure 0 0 0) 0x7fe5dc0d0b40 con 0x7fe5e410c640 2024-04-24T06:54:13.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.854+0000 7fe5e8a00640 1 --2- 172.21.15.5:0/3839811482 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fe5c80a3d10 0x7fe5c80a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:13.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.855+0000 7fe5e8a00640 1 --2- 172.21.15.5:0/3839811482 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fe5c80a3d10 0x7fe5c80a61d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4005fd0 tx=0x7fe5d4009500 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:13.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:13.858+0000 7fe5e1600640 1 -- 172.21.15.5:0/3839811482 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe5dc09d1a0 con 0x7fe5e410c640 2024-04-24T06:54:13.945 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:13 smithi137 ceph-mon[33017]: pgmap v315: 97 pgs: 97 active+clean; 85 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-24T06:54:14.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:14.096+0000 7fe5eb400640 1 -- 172.21.15.5:0/3839811482 --> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fe5e4062440 con 0x7fe5c80a3d10 2024-04-24T06:54:14.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:14.109+0000 7fe5e1600640 1 -- 172.21.15.5:0/3839811482 <== mgr.25202 v2:172.21.15.5:6800/3154624651 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fe5e4062440 con 0x7fe5c80a3d10 2024-04-24T06:54:14.110 INFO:teuthology.orchestra.run.smithi005.stdout:/ 1m 6m3h 2024-04-24T06:54:14.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:14.112+0000 7fe5eb400640 1 -- 172.21.15.5:0/3839811482 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fe5c80a3d10 msgr2=0x7fe5c80a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:14.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:14.112+0000 7fe5eb400640 1 --2- 172.21.15.5:0/3839811482 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fe5c80a3d10 0x7fe5c80a61d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4005fd0 tx=0x7fe5d4009500 comp rx=0 tx=0).stop 2024-04-24T06:54:14.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:14.112+0000 7fe5eb400640 1 -- 172.21.15.5:0/3839811482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5e410c640 msgr2=0x7fe5e4198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:14.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:14.112+0000 7fe5eb400640 1 --2- 172.21.15.5:0/3839811482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5e410c640 0x7fe5e4198780 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc00e970 tx=0x7fe5dc00ee30 comp rx=0 tx=0).stop 2024-04-24T06:54:14.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:14.112+0000 7fe5eb400640 1 -- 172.21.15.5:0/3839811482 shutdown_connections 2024-04-24T06:54:14.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:14.113+0000 7fe5eb400640 1 --2- 172.21.15.5:0/3839811482 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fe5c80a3d10 0x7fe5c80a61d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:14.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:14.113+0000 7fe5eb400640 1 --2- 172.21.15.5:0/3839811482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5e410c640 0x7fe5e4198780 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:14.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:14.113+0000 7fe5eb400640 1 --2- 172.21.15.5:0/3839811482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5e4103b90 0x7fe5e419ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:14.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:14.113+0000 7fe5eb400640 1 --2- 172.21.15.5:0/3839811482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5e4103270 0x7fe5e419e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:14.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:14.113+0000 7fe5eb400640 1 -- 172.21.15.5:0/3839811482 >> 172.21.15.5:0/3839811482 conn(0x7fe5e4078020 msgr2=0x7fe5e40fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:14.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:14.113+0000 7fe5eb400640 1 -- 172.21.15.5:0/3839811482 shutdown_connections 2024-04-24T06:54:14.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:14.113+0000 7fe5eb400640 1 -- 172.21.15.5:0/3839811482 wait complete. 2024-04-24T06:54:14.134 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-04-24T06:54:14.146 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-04-24T06:54:14.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:14 smithi005 ceph-mon[27798]: from='client.27452 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:54:14.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:14 smithi037 ceph-mon[32864]: from='client.27452 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:54:14.945 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:14 smithi137 ceph-mon[33017]: from='client.27452 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:54:15.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:15 smithi005 ceph-mon[27798]: from='client.27458 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:54:15.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:15 smithi005 ceph-mon[27798]: pgmap v316: 97 pgs: 97 active+clean; 85 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-24T06:54:15.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:15 smithi037 ceph-mon[32864]: from='client.27458 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:54:15.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:15 smithi037 ceph-mon[32864]: pgmap v316: 97 pgs: 97 active+clean; 85 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-24T06:54:15.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:15 smithi137 ceph-mon[33017]: from='client.27458 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T06:54:15.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:15 smithi137 ceph-mon[33017]: pgmap v316: 97 pgs: 97 active+clean; 85 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-24T06:54:17.445 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:17 smithi137 ceph-mon[33017]: osdmap e97: 12 total, 12 up, 12 in 2024-04-24T06:54:17.445 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:17 smithi137 ceph-mon[33017]: pgmap v318: 97 pgs: 97 active+clean; 85 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 127 B/s wr, 4 op/s 2024-04-24T06:54:17.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:17 smithi005 ceph-mon[27798]: osdmap e97: 12 total, 12 up, 12 in 2024-04-24T06:54:17.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:17 smithi005 ceph-mon[27798]: pgmap v318: 97 pgs: 97 active+clean; 85 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 127 B/s wr, 4 op/s 2024-04-24T06:54:17.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:17 smithi037 ceph-mon[32864]: osdmap e97: 12 total, 12 up, 12 in 2024-04-24T06:54:17.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:17 smithi037 ceph-mon[32864]: pgmap v318: 97 pgs: 97 active+clean; 85 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 127 B/s wr, 4 op/s 2024-04-24T06:54:19.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:19 smithi137 ceph-mon[33017]: pgmap v319: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 104 B/s wr, 9 op/s 2024-04-24T06:54:19.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:19 smithi005 ceph-mon[27798]: pgmap v319: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 104 B/s wr, 9 op/s 2024-04-24T06:54:19.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:19 smithi037 ceph-mon[32864]: pgmap v319: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 104 B/s wr, 9 op/s 2024-04-24T06:54:21.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:21 smithi137 ceph-mon[33017]: pgmap v320: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 9 op/s 2024-04-24T06:54:21.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:21 smithi005 ceph-mon[27798]: pgmap v320: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 9 op/s 2024-04-24T06:54:21.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:21 smithi037 ceph-mon[32864]: pgmap v320: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 9 op/s 2024-04-24T06:54:23.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:23 smithi137 ceph-mon[33017]: pgmap v321: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 9 op/s 2024-04-24T06:54:23.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:23 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:54:23.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:23 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:54:23.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:23 smithi137 ceph-mon[33017]: scrub summary: idle 2024-04-24T06:54:23.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:23 smithi005 ceph-mon[27798]: pgmap v321: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 9 op/s 2024-04-24T06:54:23.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:23 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:54:23.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:23 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:54:23.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:23 smithi005 ceph-mon[27798]: scrub summary: idle 2024-04-24T06:54:23.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:23 smithi037 ceph-mon[32864]: pgmap v321: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 9 op/s 2024-04-24T06:54:23.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:23 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:54:23.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:23 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:54:23.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:23 smithi037 ceph-mon[32864]: scrub summary: idle 2024-04-24T06:54:25.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:25 smithi137 ceph-mon[33017]: pgmap v322: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-24T06:54:25.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:25 smithi005 ceph-mon[27798]: pgmap v322: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-24T06:54:25.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:25 smithi037 ceph-mon[32864]: pgmap v322: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-24T06:54:27.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:27 smithi137 ceph-mon[33017]: pgmap v323: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-24T06:54:27.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:27 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:54:27.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:27 smithi005 ceph-mon[27798]: pgmap v323: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-24T06:54:27.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:27 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:54:27.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:27 smithi037 ceph-mon[32864]: pgmap v323: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-24T06:54:27.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:27 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T06:54:28.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:28 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:54:28.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:28 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:54:28.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:28 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:54:28.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:28 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:54:28.945 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:28 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:54:28.945 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:28 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:54:28.945 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:28 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:54:28.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:28 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:54:29.005 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:28 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:54:29.005 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:28 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:54:29.005 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:28 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:54:29.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:28 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:54:29.945 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:29 smithi137 ceph-mon[33017]: pgmap v324: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-24T06:54:29.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:29 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:54:29.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:29 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:54:29.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:29 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:54:29.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:29 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:54:29.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:29 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:29.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:29 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:29.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:29 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:29.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:29 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:29.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:29 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:29.948 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:29 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:29.948 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:29 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:29.948 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:29 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:30.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:29 smithi005 ceph-mon[27798]: pgmap v324: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-24T06:54:30.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:29 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:54:30.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:29 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:54:30.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:29 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:54:30.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:29 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:54:30.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:29 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:30.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:29 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:30.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:29 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:30.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:29 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:30.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:29 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:30.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:29 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:30.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:29 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:30.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:29 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:30.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:29 smithi037 ceph-mon[32864]: pgmap v324: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-24T06:54:30.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:29 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:54:30.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:29 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:54:30.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:29 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:54:30.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:29 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:54:30.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:29 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:30.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:29 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:30.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:29 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:30.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:29 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:30.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:29 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:30.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:29 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:30.020 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:29 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:30.020 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:29 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:30.945 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:30 smithi137 ceph-mon[33017]: Adjusting osd_memory_target on smithi137 to 176.2M 2024-04-24T06:54:30.945 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:30 smithi137 ceph-mon[33017]: Unable to set osd_memory_target on smithi137 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-24T06:54:30.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:30 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:30.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:30 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:30.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:30 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:30.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:30 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:30.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:30 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:30.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:30 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:30.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:30 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:30.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:30 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:31.005 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:30 smithi005 ceph-mon[27798]: Adjusting osd_memory_target on smithi137 to 176.2M 2024-04-24T06:54:31.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:30 smithi005 ceph-mon[27798]: Unable to set osd_memory_target on smithi137 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-24T06:54:31.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:30 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:31.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:30 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:31.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:30 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:31.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:30 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:31.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:30 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:31.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:30 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:31.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:30 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:31.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:30 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:31.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:30 smithi037 ceph-mon[32864]: Adjusting osd_memory_target on smithi137 to 176.2M 2024-04-24T06:54:31.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:30 smithi037 ceph-mon[32864]: Unable to set osd_memory_target on smithi137 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-24T06:54:31.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:30 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:31.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:30 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:31.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:30 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:31.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:30 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:31.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:30 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:31.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:30 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:31.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:30 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:31.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:30 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:31.945 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:31 smithi137 ceph-mon[33017]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-04-24T06:54:31.945 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:31 smithi137 ceph-mon[33017]: Unable to set osd_memory_target on smithi037 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-24T06:54:31.945 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:31 smithi137 ceph-mon[33017]: pgmap v325: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:54:31.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:31 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:54:31.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:31 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:54:32.005 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:31 smithi005 ceph-mon[27798]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-04-24T06:54:32.005 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:31 smithi005 ceph-mon[27798]: Unable to set osd_memory_target on smithi037 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-24T06:54:32.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:31 smithi005 ceph-mon[27798]: pgmap v325: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:54:32.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:31 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:54:32.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:31 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:54:32.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:31 smithi037 ceph-mon[32864]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-04-24T06:54:32.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:31 smithi037 ceph-mon[32864]: Unable to set osd_memory_target on smithi037 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-24T06:54:32.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:31 smithi037 ceph-mon[32864]: pgmap v325: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:54:32.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:31 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:54:32.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:31 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:54:32.945 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:32 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:32.945 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:32 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:32.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:32 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:32.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:32 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:32.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:32 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:32.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:32 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:32.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:32 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:32.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:32 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:32.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:32 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:54:32.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:32 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:54:32.947 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:32 smithi137 ceph-mon[33017]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:54:33.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:32 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:33.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:32 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:33.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:32 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:33.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:32 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:33.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:32 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:33.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:32 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:33.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:32 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:33.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:32 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:33.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:32 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:54:33.007 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:32 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:54:33.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:32 smithi005 ceph-mon[27798]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:54:33.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:32 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:33.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:32 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:33.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:32 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:33.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:32 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:33.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:32 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:33.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:32 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:33.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:32 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:33.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:32 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T06:54:33.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:32 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T06:54:33.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:32 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T06:54:33.019 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:32 smithi037 ceph-mon[32864]: from='mgr.25202 ' entity='mgr.x' 2024-04-24T06:54:33.945 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:33 smithi137 ceph-mon[33017]: pgmap v326: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:54:33.945 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:33 smithi137 ceph-mon[33017]: Adjusting osd_memory_target on smithi005 to 177.5M 2024-04-24T06:54:33.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:33 smithi137 ceph-mon[33017]: Unable to set osd_memory_target on smithi005 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-24T06:54:34.005 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:33 smithi005 ceph-mon[27798]: pgmap v326: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:54:34.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:33 smithi005 ceph-mon[27798]: Adjusting osd_memory_target on smithi005 to 177.5M 2024-04-24T06:54:34.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:33 smithi005 ceph-mon[27798]: Unable to set osd_memory_target on smithi005 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-24T06:54:34.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:33 smithi037 ceph-mon[32864]: pgmap v326: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:54:34.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:33 smithi037 ceph-mon[32864]: Adjusting osd_memory_target on smithi005 to 177.5M 2024-04-24T06:54:34.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:33 smithi037 ceph-mon[32864]: Unable to set osd_memory_target on smithi005 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-24T06:54:35.945 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:35 smithi137 ceph-mon[33017]: pgmap v327: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:54:36.005 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:35 smithi005 ceph-mon[27798]: pgmap v327: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:54:36.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:35 smithi037 ceph-mon[32864]: pgmap v327: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:54:37.945 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:37 smithi137 ceph-mon[33017]: pgmap v328: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:54:38.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:37 smithi005 ceph-mon[27798]: pgmap v328: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:54:38.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:37 smithi037 ceph-mon[32864]: pgmap v328: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:54:40.005 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:39 smithi005 ceph-mon[27798]: pgmap v329: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:54:40.017 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:39 smithi037 ceph-mon[32864]: pgmap v329: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:54:40.178 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T06:54:40.185 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T06:54:40.195 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:39 smithi137 ceph-mon[33017]: pgmap v329: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:54:40.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.412+0000 7f2d27e00640 1 -- 172.21.15.5:0/1752266512 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d28100dd0 msgr2=0x7f2d281011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:40.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.412+0000 7f2d27e00640 1 --2- 172.21.15.5:0/1752266512 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d28100dd0 0x7f2d281011b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2d18009930 tx=0x7f2d1802a4b0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.413+0000 7f2d27e00640 1 -- 172.21.15.5:0/1752266512 shutdown_connections 2024-04-24T06:54:40.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.413+0000 7f2d27e00640 1 --2- 172.21.15.5:0/1752266512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d281061d0 0x7f2d28106630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.413+0000 7f2d27e00640 1 --2- 172.21.15.5:0/1752266512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d281016f0 0x7f2d28105bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.413+0000 7f2d27e00640 1 --2- 172.21.15.5:0/1752266512 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d28100dd0 0x7f2d281011b0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.413+0000 7f2d27e00640 1 -- 172.21.15.5:0/1752266512 >> 172.21.15.5:0/1752266512 conn(0x7f2d28077ab0 msgr2=0x7f2d280fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:40.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.413+0000 7f2d27e00640 1 -- 172.21.15.5:0/1752266512 shutdown_connections 2024-04-24T06:54:40.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.413+0000 7f2d27e00640 1 -- 172.21.15.5:0/1752266512 wait complete. 2024-04-24T06:54:40.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.414+0000 7f2d27e00640 1 Processor -- start 2024-04-24T06:54:40.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.414+0000 7f2d27e00640 1 -- start start 2024-04-24T06:54:40.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.415+0000 7f2d27e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d28100dd0 0x7f2d2819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:40.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.415+0000 7f2d27e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d281016f0 0x7f2d2819f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:40.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.415+0000 7f2d27e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d281061d0 0x7f2d281a3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:40.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.415+0000 7f2d27e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2d28118810 con 0x7f2d28100dd0 2024-04-24T06:54:40.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.415+0000 7f2d27e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2d28118690 con 0x7f2d281016f0 2024-04-24T06:54:40.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.415+0000 7f2d27e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2d28118990 con 0x7f2d281061d0 2024-04-24T06:54:40.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.415+0000 7f2d26a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d28100dd0 0x7f2d2819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:40.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.415+0000 7f2d26a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d28100dd0 0x7f2d2819edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50428/0 (socket says 172.21.15.5:50428) 2024-04-24T06:54:40.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.415+0000 7f2d26a00640 1 -- 172.21.15.5:0/3946420234 learned_addr learned my addr 172.21.15.5:0/3946420234 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:40.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.415+0000 7f2d26000640 1 --2- 172.21.15.5:0/3946420234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d281016f0 0x7f2d2819f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:40.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.415+0000 7f2d27400640 1 --2- 172.21.15.5:0/3946420234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d281061d0 0x7f2d281a3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:40.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.416+0000 7f2d26000640 1 -- 172.21.15.5:0/3946420234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d281061d0 msgr2=0x7f2d281a3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:40.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.416+0000 7f2d26000640 1 --2- 172.21.15.5:0/3946420234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d281061d0 0x7f2d281a3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.416+0000 7f2d26000640 1 -- 172.21.15.5:0/3946420234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d28100dd0 msgr2=0x7f2d2819edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:40.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.416+0000 7f2d26000640 1 --2- 172.21.15.5:0/3946420234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d28100dd0 0x7f2d2819edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.416+0000 7f2d26000640 1 -- 172.21.15.5:0/3946420234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d281a3d50 con 0x7f2d281016f0 2024-04-24T06:54:40.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.416+0000 7f2d26000640 1 --2- 172.21.15.5:0/3946420234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d281016f0 0x7f2d2819f2f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400e9f0 tx=0x7f2d1400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:40.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.416+0000 7f2d1f400640 1 -- 172.21.15.5:0/3946420234 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1400ccc0 con 0x7f2d281016f0 2024-04-24T06:54:40.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.416+0000 7f2d1f400640 1 -- 172.21.15.5:0/3946420234 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d14010910 con 0x7f2d281016f0 2024-04-24T06:54:40.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.416+0000 7f2d27e00640 1 -- 172.21.15.5:0/3946420234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d281a4040 con 0x7f2d281016f0 2024-04-24T06:54:40.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.416+0000 7f2d1f400640 1 -- 172.21.15.5:0/3946420234 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14018960 con 0x7f2d281016f0 2024-04-24T06:54:40.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.417+0000 7f2d27e00640 1 -- 172.21.15.5:0/3946420234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d281ab8d0 con 0x7f2d281016f0 2024-04-24T06:54:40.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.421+0000 7f2d3079c740 1 Processor -- start 2024-04-24T06:54:40.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.421+0000 7f2d3079c740 1 -- start start 2024-04-24T06:54:40.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.421+0000 7f2d3079c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630c2d1ba30 0x5630c2d1be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:40.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.422+0000 7f2d3079c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630c2d1c350 0x5630c2d264e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:40.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.422+0000 7f2d3079c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630c2d26a20 0x5630c2d28e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:40.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.422+0000 7f2d3079c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5630c2c811f0 con 0x5630c2d1c350 2024-04-24T06:54:40.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.422+0000 7f2d3079c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5630c2c27350 con 0x5630c2d26a20 2024-04-24T06:54:40.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.422+0000 7f2d3079c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5630c2c7d1b0 con 0x5630c2d1ba30 2024-04-24T06:54:40.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.422+0000 7f2d26a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630c2d1c350 0x5630c2d264e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:40.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.422+0000 7f2d27400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630c2d1ba30 0x5630c2d1be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:40.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.422+0000 7f2d27400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630c2d1ba30 0x5630c2d1be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:32974/0 (socket says 172.21.15.5:32974) 2024-04-24T06:54:40.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.422+0000 7f2d27400640 1 -- 172.21.15.5:0/2059632651 learned_addr learned my addr 172.21.15.5:0/2059632651 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:40.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.422+0000 7f2d1f400640 1 -- 172.21.15.5:0/3946420234 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f2d14010ac0 con 0x7f2d281016f0 2024-04-24T06:54:40.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.422+0000 7f2d27400640 1 -- 172.21.15.5:0/2059632651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630c2d26a20 msgr2=0x5630c2d28e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:40.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.422+0000 7f2d27400640 1 --2- 172.21.15.5:0/2059632651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630c2d26a20 0x5630c2d28e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.422+0000 7f2d27400640 1 -- 172.21.15.5:0/2059632651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630c2d1c350 msgr2=0x5630c2d264e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:40.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.422+0000 7f2d27400640 1 --2- 172.21.15.5:0/2059632651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630c2d1c350 0x5630c2d264e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.422+0000 7f2d27400640 1 -- 172.21.15.5:0/2059632651 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630c2c7bf50 con 0x5630c2d1ba30 2024-04-24T06:54:40.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.423+0000 7f2d27400640 1 --2- 172.21.15.5:0/2059632651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630c2d1ba30 0x5630c2d1be10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2d20009f50 tx=0x7f2d200118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86b9ce280636654d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:40.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.423+0000 7f2d13e00640 1 -- 172.21.15.5:0/2059632651 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d20004d20 con 0x5630c2d1ba30 2024-04-24T06:54:40.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.423+0000 7f2d13e00640 1 -- 172.21.15.5:0/2059632651 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d20004050 con 0x5630c2d1ba30 2024-04-24T06:54:40.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.423+0000 7f2d13e00640 1 -- 172.21.15.5:0/2059632651 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d20005280 con 0x5630c2d1ba30 2024-04-24T06:54:40.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.424+0000 7f2d1f400640 1 --2- 172.21.15.5:0/3946420234 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f2d0c0a3da0 0x7f2d0c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:40.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.424+0000 7f2d3079c740 1 -- 172.21.15.5:0/2059632651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630c2d1ba30 msgr2=0x5630c2d1be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:40.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.424+0000 7f2d3079c740 1 --2- 172.21.15.5:0/2059632651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630c2d1ba30 0x5630c2d1be10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2d20009f50 tx=0x7f2d200118d0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.424+0000 7f2d3079c740 1 -- 172.21.15.5:0/2059632651 shutdown_connections 2024-04-24T06:54:40.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.424+0000 7f2d3079c740 1 --2- 172.21.15.5:0/2059632651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630c2d26a20 0x5630c2d28e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.424+0000 7f2d3079c740 1 --2- 172.21.15.5:0/2059632651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630c2d1c350 0x5630c2d264e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.424+0000 7f2d3079c740 1 --2- 172.21.15.5:0/2059632651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630c2d1ba30 0x5630c2d1be10 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.424+0000 7f2d3079c740 1 -- 172.21.15.5:0/2059632651 >> 172.21.15.5:0/2059632651 conn(0x5630c2d17d50 msgr2=0x5630c2d18130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:40.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.424+0000 7f2d1f400640 1 -- 172.21.15.5:0/3946420234 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f2d140d0b50 con 0x7f2d281016f0 2024-04-24T06:54:40.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.424+0000 7f2d3079c740 1 -- 172.21.15.5:0/2059632651 shutdown_connections 2024-04-24T06:54:40.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.424+0000 7f2d26a00640 1 --2- 172.21.15.5:0/3946420234 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f2d0c0a3da0 0x7f2d0c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:40.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.424+0000 7f2d3079c740 1 -- 172.21.15.5:0/2059632651 wait complete. 2024-04-24T06:54:40.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.425+0000 7f2d3079c740 1 -- start start 2024-04-24T06:54:40.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.425+0000 7f2d3079c740 1 Processor -- start 2024-04-24T06:54:40.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.425+0000 7f2d26a00640 1 --2- 172.21.15.5:0/3946420234 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f2d0c0a3da0 0x7f2d0c0a6260 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2d180096c0 tx=0x7f2d18005b50 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:40.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.425+0000 7f2d3079c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630c2d1ba30 0x5630c2d23a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:40.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.425+0000 7f2d3079c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630c2d1c350 0x5630c2d23f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:40.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.425+0000 7f2d3079c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630c2d26a20 0x5630c2d244c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:40.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.425+0000 7f2d3079c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5630c2d29810 con 0x5630c2d1ba30 2024-04-24T06:54:40.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.425+0000 7f2d3079c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5630c2c3a690 con 0x5630c2d1c350 2024-04-24T06:54:40.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.425+0000 7f2d3079c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5630c2d29990 con 0x5630c2d26a20 2024-04-24T06:54:40.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.425+0000 7f2d3079c740 10 client.0 _ll_register_callbacks cb 0x5630c2c76650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:54:40.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.426+0000 7f2d26a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630c2d1ba30 0x5630c2d23a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:40.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.426+0000 7f2d26a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630c2d1ba30 0x5630c2d23a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50442/0 (socket says 172.21.15.5:50442) 2024-04-24T06:54:40.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.426+0000 7f2d26a00640 1 -- 172.21.15.5:0/492847721 learned_addr learned my addr 172.21.15.5:0/492847721 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:40.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.426+0000 7f2d26000640 1 --2- 172.21.15.5:0/492847721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630c2d1c350 0x5630c2d23f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:40.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.426+0000 7f2d27400640 1 --2- 172.21.15.5:0/492847721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630c2d26a20 0x5630c2d244c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:40.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.426+0000 7f2d26000640 1 -- 172.21.15.5:0/492847721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630c2d26a20 msgr2=0x5630c2d244c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:40.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.426+0000 7f2d26000640 1 --2- 172.21.15.5:0/492847721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630c2d26a20 0x5630c2d244c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.426+0000 7f2d26000640 1 -- 172.21.15.5:0/492847721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630c2d1ba30 msgr2=0x5630c2d23a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:40.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.426+0000 7f2d26000640 1 --2- 172.21.15.5:0/492847721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630c2d1ba30 0x5630c2d23a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.426+0000 7f2d26000640 1 -- 172.21.15.5:0/492847721 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630c2c39920 con 0x5630c2d1c350 2024-04-24T06:54:40.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.426+0000 7f2d26a00640 1 --2- 172.21.15.5:0/492847721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630c2d1ba30 0x5630c2d23a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:54:40.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.426+0000 7f2d26000640 1 --2- 172.21.15.5:0/492847721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630c2d1c350 0x5630c2d23f80 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2d140ce230 tx=0x7f2d1409c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f80caba0af80d01d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:40.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.426+0000 7f2d10c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:54:40.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.427+0000 7f2d10c00640 1 -- 172.21.15.5:0/492847721 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d140ce6b0 con 0x5630c2d1c350 2024-04-24T06:54:40.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.427+0000 7f2d10c00640 1 -- 172.21.15.5:0/492847721 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f2d1409bf80 con 0x5630c2d1c350 2024-04-24T06:54:40.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.427+0000 7f2d3079c740 1 -- 172.21.15.5:0/492847721 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d20004050 con 0x5630c2d1c350 2024-04-24T06:54:40.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.427+0000 7f2d10c00640 1 -- 172.21.15.5:0/492847721 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2d140a56f0 con 0x5630c2d1c350 2024-04-24T06:54:40.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.427+0000 7f2d3079c740 10 client.27467 fetch_fsmap learned FSMap version 27 2024-04-24T06:54:40.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.427+0000 7f2d3079c740 1 -- 172.21.15.5:0/492847721 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5630c2a5d7d0 con 0x5630c2d1c350 2024-04-24T06:54:40.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.427+0000 7f2d10c00640 1 -- 172.21.15.5:0/492847721 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d20004050 con 0x5630c2d1c350 2024-04-24T06:54:40.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.428+0000 7f2d10c00640 1 -- 172.21.15.5:0/492847721 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f2d14098880 con 0x5630c2d1c350 2024-04-24T06:54:40.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.428+0000 7f2d3079c740 10 client.27467 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:54:40.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.428+0000 7f2d3079c740 10 client.27467 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:54:40.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.428+0000 7f2d3079c740 20 client.27467 populate_metadata read hostname 'smithi005' 2024-04-24T06:54:40.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.428+0000 7f2d3079c740 1 --2- 172.21.15.5:0/492847721 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5630c2d36e60 0x5630c2d39250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:40.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.428+0000 7f2d3079c740 4 client.27467 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:54:40.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.428+0000 7f2d3079c740 1 -- 172.21.15.5:0/492847721 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630c2c8ec10 con 0x5630c2d36e60 2024-04-24T06:54:40.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.428+0000 7f2d26a00640 1 --2- 172.21.15.5:0/492847721 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5630c2d36e60 0x5630c2d39250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:40.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.429+0000 7f2d26a00640 1 --2- 172.21.15.5:0/492847721 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5630c2d36e60 0x5630c2d39250 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47a25e8d4b52cbd5 server_cookie=35b80764b2fc934 in_seq=0 out_seq=0 2024-04-24T06:54:40.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.429+0000 7f2d10c00640 10 client.27467 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:54:40.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.430+0000 7f2d10c00640 1 -- 172.21.15.5:0/492847721 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5630c2c8ec10 con 0x5630c2d36e60 2024-04-24T06:54:40.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.430+0000 7f2d10c00640 10 client.27467 handle_command_reply: tid=0 2024-04-24T06:54:40.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.430+0000 7f2d3079c740 1 client.27467 shutdown 2024-04-24T06:54:40.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.430+0000 7f2d3079c740 1 -- 172.21.15.5:0/492847721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630c2d1c350 msgr2=0x5630c2d23f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:40.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.430+0000 7f2d3079c740 1 --2- 172.21.15.5:0/492847721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630c2d1c350 0x5630c2d23f80 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2d140ce230 tx=0x7f2d1409c490 comp rx=0 tx=0).stop 2024-04-24T06:54:40.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.430+0000 7f2d3079c740 1 -- 172.21.15.5:0/492847721 shutdown_connections 2024-04-24T06:54:40.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.430+0000 7f2d3079c740 1 --2- 172.21.15.5:0/492847721 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5630c2d36e60 0x5630c2d39250 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.430+0000 7f2d3079c740 1 --2- 172.21.15.5:0/492847721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630c2d26a20 0x5630c2d244c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.430+0000 7f2d3079c740 1 --2- 172.21.15.5:0/492847721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630c2d1c350 0x5630c2d23f80 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.430+0000 7f2d3079c740 1 --2- 172.21.15.5:0/492847721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630c2d1ba30 0x5630c2d23a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.430+0000 7f2d10c00640 0 client.27467 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:54:40.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.430+0000 7f2d3079c740 1 -- 172.21.15.5:0/492847721 >> 172.21.15.5:0/492847721 conn(0x5630c2d17d50 msgr2=0x5630c2d1d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:40.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.430+0000 7f2d3079c740 1 -- 172.21.15.5:0/492847721 shutdown_connections 2024-04-24T06:54:40.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.430+0000 7f2d3079c740 1 -- 172.21.15.5:0/492847721 wait complete. 2024-04-24T06:54:40.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.430+0000 7f2d3079c740 20 client.27467 trim_cache size 0 max 16384 2024-04-24T06:54:40.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.450+0000 7f2d3079c740 1 Processor -- start 2024-04-24T06:54:40.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.450+0000 7f2d3079c740 1 -- start start 2024-04-24T06:54:40.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.450+0000 7f2d3079c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630c2d1ba30 0x5630c2d32700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:40.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.450+0000 7f2d3079c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630c2d1c350 0x5630c2d1e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:40.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.450+0000 7f2d3079c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630c2d26a20 0x5630c2d3eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:40.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.450+0000 7f2d3079c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2d281b6b70 con 0x5630c2d1c350 2024-04-24T06:54:40.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.450+0000 7f2d3079c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2d281b4440 con 0x5630c2d1ba30 2024-04-24T06:54:40.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.450+0000 7f2d3079c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2d281b22b0 con 0x5630c2d26a20 2024-04-24T06:54:40.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.450+0000 7f2d26000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630c2d1c350 0x5630c2d1e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:40.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.450+0000 7f2d26a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630c2d1ba30 0x5630c2d32700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:40.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.450+0000 7f2d26a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630c2d1ba30 0x5630c2d32700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:57486/0 (socket says 172.21.15.5:57486) 2024-04-24T06:54:40.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.450+0000 7f2d26a00640 1 -- 172.21.15.5:0/1471483745 learned_addr learned my addr 172.21.15.5:0/1471483745 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:40.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.450+0000 7f2d26000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630c2d1c350 0x5630c2d1e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50448/0 (socket says 172.21.15.5:50448) 2024-04-24T06:54:40.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.451+0000 7f2d27400640 1 --2- 172.21.15.5:0/1471483745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630c2d26a20 0x5630c2d3eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:40.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.451+0000 7f2d26a00640 1 -- 172.21.15.5:0/1471483745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630c2d26a20 msgr2=0x5630c2d3eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:40.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.451+0000 7f2d26a00640 1 --2- 172.21.15.5:0/1471483745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630c2d26a20 0x5630c2d3eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.451+0000 7f2d26a00640 1 -- 172.21.15.5:0/1471483745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630c2d1c350 msgr2=0x5630c2d1e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:40.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.451+0000 7f2d26a00640 1 --2- 172.21.15.5:0/1471483745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630c2d1c350 0x5630c2d1e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.451+0000 7f2d26a00640 1 -- 172.21.15.5:0/1471483745 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630c2d395c0 con 0x5630c2d1ba30 2024-04-24T06:54:40.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.451+0000 7f2d26000640 1 --2- 172.21.15.5:0/1471483745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630c2d1c350 0x5630c2d1e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:54:40.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.451+0000 7f2d26a00640 1 --2- 172.21.15.5:0/1471483745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630c2d1ba30 0x5630c2d32700 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x5630c2d39a90 tx=0x7f2d1803f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24e16b5c831a075e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:40.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.451+0000 7f2d13400640 1 -- 172.21.15.5:0/1471483745 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d18035040 con 0x5630c2d1ba30 2024-04-24T06:54:40.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.451+0000 7f2d13400640 1 -- 172.21.15.5:0/1471483745 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d18005610 con 0x5630c2d1ba30 2024-04-24T06:54:40.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.451+0000 7f2d13400640 1 -- 172.21.15.5:0/1471483745 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d18004e90 con 0x5630c2d1ba30 2024-04-24T06:54:40.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.452+0000 7f2d3079c740 1 -- 172.21.15.5:0/1471483745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630c2d1ba30 msgr2=0x5630c2d32700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:40.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.452+0000 7f2d3079c740 1 --2- 172.21.15.5:0/1471483745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630c2d1ba30 0x5630c2d32700 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x5630c2d39a90 tx=0x7f2d1803f000 comp rx=0 tx=0).stop 2024-04-24T06:54:40.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.452+0000 7f2d3079c740 1 -- 172.21.15.5:0/1471483745 shutdown_connections 2024-04-24T06:54:40.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.452+0000 7f2d3079c740 1 --2- 172.21.15.5:0/1471483745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630c2d26a20 0x5630c2d3eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.452+0000 7f2d3079c740 1 --2- 172.21.15.5:0/1471483745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630c2d1c350 0x5630c2d1e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.452+0000 7f2d3079c740 1 --2- 172.21.15.5:0/1471483745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630c2d1ba30 0x5630c2d32700 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.452+0000 7f2d3079c740 1 -- 172.21.15.5:0/1471483745 >> 172.21.15.5:0/1471483745 conn(0x5630c2d17d50 msgr2=0x5630c2d25270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:40.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.452+0000 7f2d3079c740 1 -- 172.21.15.5:0/1471483745 shutdown_connections 2024-04-24T06:54:40.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.452+0000 7f2d3079c740 1 -- 172.21.15.5:0/1471483745 wait complete. 2024-04-24T06:54:40.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.453+0000 7f2d3079c740 1 -- start start 2024-04-24T06:54:40.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.453+0000 7f2d3079c740 1 Processor -- start 2024-04-24T06:54:40.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.453+0000 7f2d3079c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630c2d1ba30 0x5630c2d208a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:40.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.453+0000 7f2d3079c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630c2d1c350 0x5630c2d20de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:40.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.453+0000 7f2d3079c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630c2d26a20 0x5630c2d21320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:40.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.453+0000 7f2d3079c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2d281b2900 con 0x5630c2d26a20 2024-04-24T06:54:40.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.453+0000 7f2d3079c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2d281b3d70 con 0x5630c2d1ba30 2024-04-24T06:54:40.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.453+0000 7f2d3079c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2d281b2fd0 con 0x5630c2d1c350 2024-04-24T06:54:40.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.453+0000 7f2d3079c740 10 client.0 _ll_register_callbacks cb 0x7f2d1400ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:54:40.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.454+0000 7f2d26a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630c2d1ba30 0x5630c2d208a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:40.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.454+0000 7f2d26a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630c2d1ba30 0x5630c2d208a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:57496/0 (socket says 172.21.15.5:57496) 2024-04-24T06:54:40.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.454+0000 7f2d26a00640 1 -- 172.21.15.5:0/3628014486 learned_addr learned my addr 172.21.15.5:0/3628014486 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:40.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.454+0000 7f2d27400640 1 --2- 172.21.15.5:0/3628014486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630c2d26a20 0x5630c2d21320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:40.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.454+0000 7f2d26000640 1 --2- 172.21.15.5:0/3628014486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630c2d1c350 0x5630c2d20de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:40.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.454+0000 7f2d26a00640 1 -- 172.21.15.5:0/3628014486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630c2d1c350 msgr2=0x5630c2d20de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:40.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.454+0000 7f2d26a00640 1 --2- 172.21.15.5:0/3628014486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630c2d1c350 0x5630c2d20de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.454+0000 7f2d26a00640 1 -- 172.21.15.5:0/3628014486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630c2d26a20 msgr2=0x5630c2d21320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:40.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.454+0000 7f2d26a00640 1 --2- 172.21.15.5:0/3628014486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630c2d26a20 0x5630c2d21320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.454+0000 7f2d26a00640 1 -- 172.21.15.5:0/3628014486 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630c29933f0 con 0x5630c2d1ba30 2024-04-24T06:54:40.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.454+0000 7f2d26a00640 1 --2- 172.21.15.5:0/3628014486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630c2d1ba30 0x5630c2d208a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2d18005890 tx=0x7f2d18005140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bcdc920b769193f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:40.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.454+0000 7f2d10c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:54:40.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.455+0000 7f2d10c00640 1 -- 172.21.15.5:0/3628014486 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d18061070 con 0x5630c2d1ba30 2024-04-24T06:54:40.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.455+0000 7f2d3079c740 1 -- 172.21.15.5:0/3628014486 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d18005610 con 0x5630c2d1ba30 2024-04-24T06:54:40.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.455+0000 7f2d10c00640 1 -- 172.21.15.5:0/3628014486 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f2d180046c0 con 0x5630c2d1ba30 2024-04-24T06:54:40.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.455+0000 7f2d10c00640 1 -- 172.21.15.5:0/3628014486 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2d180048c0 con 0x5630c2d1ba30 2024-04-24T06:54:40.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.455+0000 7f2d3079c740 10 client.27473 fetch_fsmap learned FSMap version 27 2024-04-24T06:54:40.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.455+0000 7f2d3079c740 1 -- 172.21.15.5:0/3628014486 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5630c29530f0 con 0x5630c2d1ba30 2024-04-24T06:54:40.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.455+0000 7f2d10c00640 1 -- 172.21.15.5:0/3628014486 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d18005610 con 0x5630c2d1ba30 2024-04-24T06:54:40.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.455+0000 7f2d10c00640 1 -- 172.21.15.5:0/3628014486 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f2d18032b40 con 0x5630c2d1ba30 2024-04-24T06:54:40.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.456+0000 7f2d3079c740 10 client.27473 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:54:40.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.456+0000 7f2d3079c740 10 client.27473 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:54:40.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.456+0000 7f2d3079c740 20 client.27473 populate_metadata read hostname 'smithi005' 2024-04-24T06:54:40.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.456+0000 7f2d3079c740 1 --2- 172.21.15.5:0/3628014486 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5630c2d9db10 0x5630c2d9ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:40.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.456+0000 7f2d3079c740 4 client.27473 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T06:54:40.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.456+0000 7f2d3079c740 1 -- 172.21.15.5:0/3628014486 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5630c2c88a80 con 0x5630c2d9db10 2024-04-24T06:54:40.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.456+0000 7f2d26000640 1 --2- 172.21.15.5:0/3628014486 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5630c2d9db10 0x5630c2d9ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:40.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.456+0000 7f2d26000640 1 --2- 172.21.15.5:0/3628014486 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5630c2d9db10 0x5630c2d9ff00 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c55fc5068f84a253 server_cookie=da1e741ea23cde30 in_seq=0 out_seq=0 2024-04-24T06:54:40.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.456+0000 7f2d10c00640 10 client.27473 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:54:40.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.456+0000 7f2d10c00640 1 -- 172.21.15.5:0/3628014486 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5630c2c88a80 con 0x5630c2d9db10 2024-04-24T06:54:40.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.457+0000 7f2d10c00640 10 client.27473 handle_command_reply: tid=0 2024-04-24T06:54:40.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.457+0000 7f2d3079c740 1 client.27473 shutdown 2024-04-24T06:54:40.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.457+0000 7f2d3079c740 1 -- 172.21.15.5:0/3628014486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630c2d1ba30 msgr2=0x5630c2d208a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:40.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.457+0000 7f2d3079c740 1 --2- 172.21.15.5:0/3628014486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630c2d1ba30 0x5630c2d208a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2d18005890 tx=0x7f2d18005140 comp rx=0 tx=0).stop 2024-04-24T06:54:40.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.457+0000 7f2d3079c740 1 -- 172.21.15.5:0/3628014486 shutdown_connections 2024-04-24T06:54:40.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.457+0000 7f2d3079c740 1 --2- 172.21.15.5:0/3628014486 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x5630c2d9db10 0x5630c2d9ff00 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.457+0000 7f2d3079c740 1 --2- 172.21.15.5:0/3628014486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630c2d26a20 0x5630c2d21320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.457+0000 7f2d3079c740 1 --2- 172.21.15.5:0/3628014486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630c2d1c350 0x5630c2d20de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.457+0000 7f2d3079c740 1 --2- 172.21.15.5:0/3628014486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630c2d1ba30 0x5630c2d208a0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.457+0000 7f2d10c00640 0 client.27473 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:54:40.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.457+0000 7f2d3079c740 1 -- 172.21.15.5:0/3628014486 >> 172.21.15.5:0/3628014486 conn(0x5630c2d17d50 msgr2=0x5630c2d25b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:40.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.457+0000 7f2d3079c740 1 -- 172.21.15.5:0/3628014486 shutdown_connections 2024-04-24T06:54:40.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.457+0000 7f2d3079c740 1 -- 172.21.15.5:0/3628014486 wait complete. 2024-04-24T06:54:40.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.457+0000 7f2d3079c740 20 client.27473 trim_cache size 0 max 16384 2024-04-24T06:54:40.469 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:54:40.470 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-24T06:54:40.470 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-24T06:54:40.470 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:54:40.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.458+0000 7f2d27e00640 1 -- 172.21.15.5:0/3946420234 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f2d0c0a3da0 msgr2=0x7f2d0c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:40.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.458+0000 7f2d27e00640 1 --2- 172.21.15.5:0/3946420234 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f2d0c0a3da0 0x7f2d0c0a6260 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2d180096c0 tx=0x7f2d18005b50 comp rx=0 tx=0).stop 2024-04-24T06:54:40.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.458+0000 7f2d27e00640 1 -- 172.21.15.5:0/3946420234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d281016f0 msgr2=0x7f2d2819f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:40.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.458+0000 7f2d27e00640 1 --2- 172.21.15.5:0/3946420234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d281016f0 0x7f2d2819f2f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400e9f0 tx=0x7f2d1400eeb0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.458+0000 7f2d27e00640 1 -- 172.21.15.5:0/3946420234 shutdown_connections 2024-04-24T06:54:40.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.458+0000 7f2d27e00640 1 --2- 172.21.15.5:0/3946420234 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f2d0c0a3da0 0x7f2d0c0a6260 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.458+0000 7f2d27e00640 1 --2- 172.21.15.5:0/3946420234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d281061d0 0x7f2d281a3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.458+0000 7f2d27e00640 1 --2- 172.21.15.5:0/3946420234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d281016f0 0x7f2d2819f2f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.458+0000 7f2d27e00640 1 --2- 172.21.15.5:0/3946420234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d28100dd0 0x7f2d2819edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.458+0000 7f2d27e00640 1 -- 172.21.15.5:0/3946420234 >> 172.21.15.5:0/3946420234 conn(0x7f2d28077ab0 msgr2=0x7f2d28103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:40.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.458+0000 7f2d27e00640 1 -- 172.21.15.5:0/3946420234 shutdown_connections 2024-04-24T06:54:40.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.458+0000 7f2d27e00640 1 -- 172.21.15.5:0/3946420234 wait complete. 2024-04-24T06:54:40.476 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T06:54:40.476 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:54:40.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.667+0000 7f8326600640 1 -- 172.21.15.5:0/2265245593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83200743b0 msgr2=0x7f8320074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.667+0000 7f8326600640 1 --2- 172.21.15.5:0/2265245593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83200743b0 0x7f8320074790 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8314009900 tx=0x7f831402a480 comp rx=0 tx=0).stop 2024-04-24T06:54:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.667+0000 7f8326600640 1 -- 172.21.15.5:0/2265245593 shutdown_connections 2024-04-24T06:54:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.667+0000 7f8326600640 1 --2- 172.21.15.5:0/2265245593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f832010e760 0x7f8320110b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.667+0000 7f8326600640 1 --2- 172.21.15.5:0/2265245593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8320074cd0 0x7f832010e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.667+0000 7f8326600640 1 --2- 172.21.15.5:0/2265245593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83200743b0 0x7f8320074790 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.667+0000 7f8326600640 1 -- 172.21.15.5:0/2265245593 >> 172.21.15.5:0/2265245593 conn(0x7f83200fda10 msgr2=0x7f83200ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:40.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.667+0000 7f8326600640 1 -- 172.21.15.5:0/2265245593 shutdown_connections 2024-04-24T06:54:40.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.667+0000 7f8326600640 1 -- 172.21.15.5:0/2265245593 wait complete. 2024-04-24T06:54:40.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.668+0000 7f8326600640 1 Processor -- start 2024-04-24T06:54:40.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.668+0000 7f8326600640 1 -- start start 2024-04-24T06:54:40.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.668+0000 7f8326600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83200743b0 0x7f83201a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:40.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.668+0000 7f8326600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8320074cd0 0x7f83201a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:40.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.669+0000 7f8326600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f832010e760 0x7f832019a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:40.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.669+0000 7f8326600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f83201144b0 con 0x7f83200743b0 2024-04-24T06:54:40.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.669+0000 7f8326600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8320114330 con 0x7f832010e760 2024-04-24T06:54:40.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.669+0000 7f8326600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8320114630 con 0x7f8320074cd0 2024-04-24T06:54:40.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.669+0000 7f8325200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83200743b0 0x7f83201a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:40.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.669+0000 7f831fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8320074cd0 0x7f83201a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:40.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.669+0000 7f8325200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83200743b0 0x7f83201a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50468/0 (socket says 172.21.15.5:50468) 2024-04-24T06:54:40.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.669+0000 7f8325200640 1 -- 172.21.15.5:0/2542534160 learned_addr learned my addr 172.21.15.5:0/2542534160 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:40.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.669+0000 7f8325c00640 1 --2- 172.21.15.5:0/2542534160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f832010e760 0x7f832019a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:40.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.669+0000 7f8325200640 1 -- 172.21.15.5:0/2542534160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8320074cd0 msgr2=0x7f83201a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:40.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.669+0000 7f8325200640 1 --2- 172.21.15.5:0/2542534160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8320074cd0 0x7f83201a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.669+0000 7f8325200640 1 -- 172.21.15.5:0/2542534160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f832010e760 msgr2=0x7f832019a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:40.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.669+0000 7f8325200640 1 --2- 172.21.15.5:0/2542534160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f832010e760 0x7f832019a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.669+0000 7f8325200640 1 -- 172.21.15.5:0/2542534160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f832019b020 con 0x7f83200743b0 2024-04-24T06:54:40.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.669+0000 7f8325c00640 1 --2- 172.21.15.5:0/2542534160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f832010e760 0x7f832019a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:54:40.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.669+0000 7f831fe00640 1 --2- 172.21.15.5:0/2542534160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8320074cd0 0x7f83201a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:54:40.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.670+0000 7f8325200640 1 --2- 172.21.15.5:0/2542534160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83200743b0 0x7f83201a0690 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8314004770 tx=0x7f8314003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:40.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.670+0000 7f831d600640 1 -- 172.21.15.5:0/2542534160 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831403f070 con 0x7f83200743b0 2024-04-24T06:54:40.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.670+0000 7f831d600640 1 -- 172.21.15.5:0/2542534160 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8314031ca0 con 0x7f83200743b0 2024-04-24T06:54:40.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.670+0000 7f8326600640 1 -- 172.21.15.5:0/2542534160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f832019b2b0 con 0x7f83200743b0 2024-04-24T06:54:40.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.670+0000 7f831d600640 1 -- 172.21.15.5:0/2542534160 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8314035030 con 0x7f83200743b0 2024-04-24T06:54:40.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.670+0000 7f8326600640 1 -- 172.21.15.5:0/2542534160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83201a74e0 con 0x7f83200743b0 2024-04-24T06:54:40.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.672+0000 7f8326600640 1 -- 172.21.15.5:0/2542534160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8320075a60 con 0x7f83200743b0 2024-04-24T06:54:40.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.674+0000 7f831d600640 1 -- 172.21.15.5:0/2542534160 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f8314042070 con 0x7f83200743b0 2024-04-24T06:54:40.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.675+0000 7f831d600640 1 --2- 172.21.15.5:0/2542534160 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f83080a3d90 0x7f83080a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:40.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.675+0000 7f831d600640 1 -- 172.21.15.5:0/2542534160 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f83140f0d90 con 0x7f83200743b0 2024-04-24T06:54:40.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.675+0000 7f831fe00640 1 --2- 172.21.15.5:0/2542534160 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f83080a3d90 0x7f83080a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:40.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.676+0000 7f831fe00640 1 --2- 172.21.15.5:0/2542534160 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f83080a3d90 0x7f83080a6250 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f83201000a0 tx=0x7f830c005f90 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:40.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.680+0000 7f831d600640 1 -- 172.21.15.5:0/2542534160 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f83140bce30 con 0x7f83200743b0 2024-04-24T06:54:40.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.930+0000 7f8326600640 1 -- 172.21.15.5:0/2542534160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83201098e0 con 0x7f83200743b0 2024-04-24T06:54:40.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.931+0000 7f831d600640 1 -- 172.21.15.5:0/2542534160 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11856 (secure 0 0 0) 0x7f83140c1130 con 0x7f83200743b0 2024-04-24T06:54:40.932 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:54:40.932 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1735462750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1735462750},{"type":"v1","addr":"172.21.15.5:6837","nonce":1735462750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1499469370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1499469370},{"type":"v1","addr":"172.21.15.5:6841","nonce":1499469370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3705638847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3705638847},{"type":"v1","addr":"172.21.15.37:6835","nonce":3705638847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T06:41:34.764490+0000","modified":"2024-04-24T06:48:57.768410+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24550,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6837/1390103296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1390103296},{"type":"v1","addr":"172.21.15.37:6837","nonce":1390103296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3217754215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3217754215},{"type":"v1","addr":"172.21.15.137:6835","nonce":3217754215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6837/3044222976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3044222976},{"type":"v1","addr":"172.21.15.137:6837","nonce":3044222976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.5:6839/3173201459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3173201459},{"type":"v1","addr":"172.21.15.5:6839","nonce":3173201459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6839/1889774232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1889774232},{"type":"v1","addr":"172.21.15.137:6839","nonce":1889774232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6839/3536721753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":3536721753},{"type":"v1","addr":"172.21.15.37:6839","nonce":3536721753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24550,24569,24539]},"id":1}]} 2024-04-24T06:54:40.933 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-04-24T06:54:40.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.934+0000 7f8326600640 1 -- 172.21.15.5:0/2542534160 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f83080a3d90 msgr2=0x7f83080a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:40.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.934+0000 7f8326600640 1 --2- 172.21.15.5:0/2542534160 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f83080a3d90 0x7f83080a6250 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f83201000a0 tx=0x7f830c005f90 comp rx=0 tx=0).stop 2024-04-24T06:54:40.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.934+0000 7f8326600640 1 -- 172.21.15.5:0/2542534160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83200743b0 msgr2=0x7f83201a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:40.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.934+0000 7f8326600640 1 --2- 172.21.15.5:0/2542534160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83200743b0 0x7f83201a0690 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8314004770 tx=0x7f8314003c10 comp rx=0 tx=0).stop 2024-04-24T06:54:40.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.934+0000 7f8326600640 1 -- 172.21.15.5:0/2542534160 shutdown_connections 2024-04-24T06:54:40.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.934+0000 7f8326600640 1 --2- 172.21.15.5:0/2542534160 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f83080a3d90 0x7f83080a6250 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.934+0000 7f8326600640 1 --2- 172.21.15.5:0/2542534160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f832010e760 0x7f832019a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.934+0000 7f8326600640 1 --2- 172.21.15.5:0/2542534160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8320074cd0 0x7f83201a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.934+0000 7f8326600640 1 --2- 172.21.15.5:0/2542534160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83200743b0 0x7f83201a0690 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:40.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.934+0000 7f8326600640 1 -- 172.21.15.5:0/2542534160 >> 172.21.15.5:0/2542534160 conn(0x7f83200fda10 msgr2=0x7f83200ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:40.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.935+0000 7f8326600640 1 -- 172.21.15.5:0/2542534160 shutdown_connections 2024-04-24T06:54:40.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:40.935+0000 7f8326600640 1 -- 172.21.15.5:0/2542534160 wait complete. 2024-04-24T06:54:40.954 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T06:54:41.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.166+0000 7f1ac2600640 1 -- 172.21.15.5:0/3951163512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1abc104c00 msgr2=0x7f1abc108c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:41.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.166+0000 7f1ac2600640 1 --2- 172.21.15.5:0/3951163512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1abc104c00 0x7f1abc108c40 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8009860 tx=0x7f1aa802a340 comp rx=0 tx=0).stop 2024-04-24T06:54:41.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.166+0000 7f1ac2600640 1 -- 172.21.15.5:0/3951163512 shutdown_connections 2024-04-24T06:54:41.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.166+0000 7f1ac2600640 1 --2- 172.21.15.5:0/3951163512 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1abc109320 0x7f1abc110ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.166+0000 7f1ac2600640 1 --2- 172.21.15.5:0/3951163512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1abc104c00 0x7f1abc108c40 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.166+0000 7f1ac2600640 1 --2- 172.21.15.5:0/3951163512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1abc104250 0x7f1abc104630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.166+0000 7f1ac2600640 1 -- 172.21.15.5:0/3951163512 >> 172.21.15.5:0/3951163512 conn(0x7f1abc0ffad0 msgr2=0x7f1abc101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:41.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.166+0000 7f1ac2600640 1 -- 172.21.15.5:0/3951163512 shutdown_connections 2024-04-24T06:54:41.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.166+0000 7f1ac2600640 1 -- 172.21.15.5:0/3951163512 wait complete. 2024-04-24T06:54:41.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.167+0000 7f1ac2600640 1 Processor -- start 2024-04-24T06:54:41.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.167+0000 7f1ac2600640 1 -- start start 2024-04-24T06:54:41.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.167+0000 7f1ac2600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1abc104250 0x7f1abc1a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.167+0000 7f1ac2600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1abc109320 0x7f1abc1a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.167+0000 7f1ac2600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1abc19a750 0x7f1abc19abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.168+0000 7f1ac2600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc1143b0 con 0x7f1abc104250 2024-04-24T06:54:41.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.168+0000 7f1ac2600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc114230 con 0x7f1abc109320 2024-04-24T06:54:41.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.168+0000 7f1ac2600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc114530 con 0x7f1abc19a750 2024-04-24T06:54:41.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.168+0000 7f1ac1200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1abc104250 0x7f1abc1a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.168+0000 7f1ac1200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1abc104250 0x7f1abc1a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50488/0 (socket says 172.21.15.5:50488) 2024-04-24T06:54:41.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.168+0000 7f1ac1200640 1 -- 172.21.15.5:0/993759167 learned_addr learned my addr 172.21.15.5:0/993759167 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:41.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.168+0000 7f1ac1c00640 1 --2- 172.21.15.5:0/993759167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1abc19a750 0x7f1abc19abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.168+0000 7f1abbe00640 1 --2- 172.21.15.5:0/993759167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1abc109320 0x7f1abc1a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.168+0000 7f1ac1200640 1 -- 172.21.15.5:0/993759167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1abc19a750 msgr2=0x7f1abc19abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:41.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.168+0000 7f1ac1200640 1 --2- 172.21.15.5:0/993759167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1abc19a750 0x7f1abc19abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.168+0000 7f1ac1200640 1 -- 172.21.15.5:0/993759167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1abc109320 msgr2=0x7f1abc1a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.168+0000 7f1ac1200640 1 --2- 172.21.15.5:0/993759167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1abc109320 0x7f1abc1a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.168+0000 7f1ac1200640 1 -- 172.21.15.5:0/993759167 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1abc19b350 con 0x7f1abc104250 2024-04-24T06:54:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.169+0000 7f1ac1200640 1 --2- 172.21.15.5:0/993759167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1abc104250 0x7f1abc1a0690 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1ab000e9c0 tx=0x7f1ab000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.169+0000 7f1ab9600640 1 -- 172.21.15.5:0/993759167 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab000ccc0 con 0x7f1abc104250 2024-04-24T06:54:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.169+0000 7f1ab9600640 1 -- 172.21.15.5:0/993759167 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ab0005500 con 0x7f1abc104250 2024-04-24T06:54:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.169+0000 7f1ac2600640 1 -- 172.21.15.5:0/993759167 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1abc1a7430 con 0x7f1abc104250 2024-04-24T06:54:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.169+0000 7f1ab9600640 1 -- 172.21.15.5:0/993759167 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0010430 con 0x7f1abc104250 2024-04-24T06:54:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.169+0000 7f1ac2600640 1 -- 172.21.15.5:0/993759167 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1abc1a77e0 con 0x7f1abc104250 2024-04-24T06:54:41.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.172+0000 7f1ac6b82740 1 Processor -- start 2024-04-24T06:54:41.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.172+0000 7f1ac6b82740 1 -- start start 2024-04-24T06:54:41.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.173+0000 7f1ab9600640 1 -- 172.21.15.5:0/993759167 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f1ab00046c0 con 0x7f1abc104250 2024-04-24T06:54:41.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.173+0000 7f1ac6b82740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a81054b40 0x561a81054f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.173+0000 7f1ac6b82740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a81055460 0x561a8105f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.173+0000 7f1ac6b82740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a8105fb30 0x561a81061f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.173+0000 7f1ac6b82740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561a80fba1d0 con 0x561a81054b40 2024-04-24T06:54:41.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.173+0000 7f1ac6b82740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561a80f60450 con 0x561a8105fb30 2024-04-24T06:54:41.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.173+0000 7f1ac6b82740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561a80fb6190 con 0x561a81055460 2024-04-24T06:54:41.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.173+0000 7f1ac1c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a81054b40 0x561a81054f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.173+0000 7f1ac1c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a81054b40 0x561a81054f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50494/0 (socket says 172.21.15.5:50494) 2024-04-24T06:54:41.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.173+0000 7f1ac1c00640 1 -- 172.21.15.5:0/2128561958 learned_addr learned my addr 172.21.15.5:0/2128561958 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:41.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.173+0000 7f1ac1200640 1 --2- 172.21.15.5:0/2128561958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a81055460 0x561a8105f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.173+0000 7f1abbe00640 1 --2- 172.21.15.5:0/2128561958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a8105fb30 0x561a81061f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.174+0000 7f1ac1c00640 1 -- 172.21.15.5:0/2128561958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a81055460 msgr2=0x561a8105f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:41.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.174+0000 7f1ac1c00640 1 --2- 172.21.15.5:0/2128561958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a81055460 0x561a8105f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.174+0000 7f1ac1c00640 1 -- 172.21.15.5:0/2128561958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a8105fb30 msgr2=0x561a81061f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:41.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.174+0000 7f1ac1c00640 1 --2- 172.21.15.5:0/2128561958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a8105fb30 0x561a81061f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.174+0000 7f1ac1c00640 1 -- 172.21.15.5:0/2128561958 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a8101d310 con 0x561a81054b40 2024-04-24T06:54:41.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.174+0000 7f1abbe00640 1 --2- 172.21.15.5:0/2128561958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a8105fb30 0x561a81061f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:54:41.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.174+0000 7f1ac1c00640 1 --2- 172.21.15.5:0/2128561958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a81054b40 0x561a81054f20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4009f50 tx=0x7f1ab40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b86163bda233d93e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:41.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.174+0000 7f1aae000640 1 -- 172.21.15.5:0/2128561958 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4004d20 con 0x561a81054b40 2024-04-24T06:54:41.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.174+0000 7f1aae000640 1 -- 172.21.15.5:0/2128561958 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ab4004050 con 0x561a81054b40 2024-04-24T06:54:41.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.174+0000 7f1aae000640 1 -- 172.21.15.5:0/2128561958 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4005240 con 0x561a81054b40 2024-04-24T06:54:41.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.175+0000 7f1ab9600640 1 --2- 172.21.15.5:0/993759167 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1aa40a3d20 0x7f1aa40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.175+0000 7f1ab9600640 1 -- 172.21.15.5:0/993759167 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f1ab00d0b40 con 0x7f1abc104250 2024-04-24T06:54:41.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.175+0000 7f1ac6b82740 1 -- 172.21.15.5:0/2128561958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a81054b40 msgr2=0x561a81054f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:41.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.175+0000 7f1ac6b82740 1 --2- 172.21.15.5:0/2128561958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a81054b40 0x561a81054f20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4009f50 tx=0x7f1ab40118d0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.175+0000 7f1ac1200640 1 --2- 172.21.15.5:0/993759167 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1aa40a3d20 0x7f1aa40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.175+0000 7f1ac6b82740 1 -- 172.21.15.5:0/2128561958 shutdown_connections 2024-04-24T06:54:41.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.175+0000 7f1ac6b82740 1 --2- 172.21.15.5:0/2128561958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a8105fb30 0x561a81061f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.175+0000 7f1ac6b82740 1 --2- 172.21.15.5:0/2128561958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a81055460 0x561a8105f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.175+0000 7f1ac6b82740 1 --2- 172.21.15.5:0/2128561958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a81054b40 0x561a81054f20 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.175+0000 7f1ac6b82740 1 -- 172.21.15.5:0/2128561958 >> 172.21.15.5:0/2128561958 conn(0x561a81050e60 msgr2=0x561a81051240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:41.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.175+0000 7f1ac6b82740 1 -- 172.21.15.5:0/2128561958 shutdown_connections 2024-04-24T06:54:41.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.175+0000 7f1ac6b82740 1 -- 172.21.15.5:0/2128561958 wait complete. 2024-04-24T06:54:41.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.176+0000 7f1ac1200640 1 --2- 172.21.15.5:0/993759167 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1aa40a3d20 0x7f1aa40a61e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x561a81056100 tx=0x7f1ab00d45f0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:41.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.176+0000 7f1ac6b82740 1 -- start start 2024-04-24T06:54:41.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.176+0000 7f1ac6b82740 1 Processor -- start 2024-04-24T06:54:41.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.176+0000 7f1ac6b82740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a81054b40 0x561a8105f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.176+0000 7f1ac6b82740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a81055460 0x561a8105c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.176+0000 7f1ac6b82740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a8105fb30 0x561a8105cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.176+0000 7f1ac6b82740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561a810629c0 con 0x561a81054b40 2024-04-24T06:54:41.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.176+0000 7f1ac6b82740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561a80f73810 con 0x561a8105fb30 2024-04-24T06:54:41.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.176+0000 7f1ac6b82740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561a81062b40 con 0x561a81055460 2024-04-24T06:54:41.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.177+0000 7f1ac6b82740 10 client.0 _ll_register_callbacks cb 0x561a80f41140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:54:41.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.177+0000 7f1ac1c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a81054b40 0x561a8105f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.177+0000 7f1abbe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a81055460 0x561a8105c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.177+0000 7f1ac1c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a81054b40 0x561a8105f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50500/0 (socket says 172.21.15.5:50500) 2024-04-24T06:54:41.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.177+0000 7f1ac1c00640 1 -- 172.21.15.5:0/914819573 learned_addr learned my addr 172.21.15.5:0/914819573 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:41.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.177+0000 7f1ac1c00640 1 --2- 172.21.15.5:0/914819573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a8105fb30 0x561a8105cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.177+0000 7f1abbe00640 1 -- 172.21.15.5:0/914819573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a8105fb30 msgr2=0x561a8105cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:41.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.177+0000 7f1abbe00640 1 --2- 172.21.15.5:0/914819573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a8105fb30 0x561a8105cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.177+0000 7f1abbe00640 1 -- 172.21.15.5:0/914819573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a81054b40 msgr2=0x561a8105f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:41.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.177+0000 7f1abbe00640 1 --2- 172.21.15.5:0/914819573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a81054b40 0x561a8105f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.177+0000 7f1abbe00640 1 -- 172.21.15.5:0/914819573 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a80f843f0 con 0x561a81055460 2024-04-24T06:54:41.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.177+0000 7f1ac1c00640 1 --2- 172.21.15.5:0/914819573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a81054b40 0x561a8105f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:54:41.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.178+0000 7f1abbe00640 1 --2- 172.21.15.5:0/914819573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a81055460 0x561a8105c880 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1aa802a860 tx=0x7f1aa8009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad7abc6f73ea45de server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:41.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.178+0000 7f1aa2a00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-24T06:54:41.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.178+0000 7f1aa2a00640 1 -- 172.21.15.5:0/914819573 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8037040 con 0x561a81055460 2024-04-24T06:54:41.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.178+0000 7f1ac6b82740 1 -- 172.21.15.5:0/914819573 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ab4004050 con 0x561a81055460 2024-04-24T06:54:41.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.178+0000 7f1aa2a00640 1 -- 172.21.15.5:0/914819573 <== mon.1 v2:172.21.15.137:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f1aa8004450 con 0x561a81055460 2024-04-24T06:54:41.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.178+0000 7f1aa2a00640 1 -- 172.21.15.5:0/914819573 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1aa8030550 con 0x561a81055460 2024-04-24T06:54:41.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.178+0000 7f1ac6b82740 10 client.27517 fetch_fsmap learned FSMap version 27 2024-04-24T06:54:41.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.178+0000 7f1ac6b82740 1 -- 172.21.15.5:0/914819573 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561a80ccc3f0 con 0x561a81055460 2024-04-24T06:54:41.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.179+0000 7f1aa2a00640 1 -- 172.21.15.5:0/914819573 <== mon.1 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ab4004050 con 0x561a81055460 2024-04-24T06:54:41.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.179+0000 7f1aa2a00640 1 -- 172.21.15.5:0/914819573 <== mon.1 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f1aa8032440 con 0x561a81055460 2024-04-24T06:54:41.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.179+0000 7f1ac6b82740 10 client.27517 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:54:41.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.179+0000 7f1ac6b82740 10 client.27517 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-24T06:54:41.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.179+0000 7f1ac6b82740 20 client.27517 populate_metadata read hostname 'smithi005' 2024-04-24T06:54:41.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.179+0000 7f1ac6b82740 1 --2- 172.21.15.5:0/914819573 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x561a8106fcc0 0x561a810720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.179+0000 7f1ac6b82740 4 client.27517 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:54:41.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.179+0000 7f1ac6b82740 1 -- 172.21.15.5:0/914819573 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a80fc7bf0 con 0x561a8106fcc0 2024-04-24T06:54:41.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.179+0000 7f1ac1c00640 1 --2- 172.21.15.5:0/914819573 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x561a8106fcc0 0x561a810720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.180+0000 7f1aa2a00640 10 client.27517 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:54:41.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.180+0000 7f1ac1c00640 1 --2- 172.21.15.5:0/914819573 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x561a8106fcc0 0x561a810720b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=31f45117f61f6f7a server_cookie=f30af3af5be10aab in_seq=0 out_seq=0 2024-04-24T06:54:41.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.181+0000 7f1aa2a00640 1 -- 172.21.15.5:0/914819573 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561a80fc7bf0 con 0x561a8106fcc0 2024-04-24T06:54:41.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.181+0000 7f1aa2a00640 10 client.27517 handle_command_reply: tid=0 2024-04-24T06:54:41.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.181+0000 7f1ac6b82740 1 client.27517 shutdown 2024-04-24T06:54:41.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.181+0000 7f1ac6b82740 1 -- 172.21.15.5:0/914819573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a81055460 msgr2=0x561a8105c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:41.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.181+0000 7f1ac6b82740 1 --2- 172.21.15.5:0/914819573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a81055460 0x561a8105c880 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1aa802a860 tx=0x7f1aa8009510 comp rx=0 tx=0).stop 2024-04-24T06:54:41.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.181+0000 7f1ac6b82740 1 -- 172.21.15.5:0/914819573 shutdown_connections 2024-04-24T06:54:41.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.181+0000 7f1ac6b82740 1 --2- 172.21.15.5:0/914819573 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x561a8106fcc0 0x561a810720b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.181+0000 7f1ac6b82740 1 --2- 172.21.15.5:0/914819573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a8105fb30 0x561a8105cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.181+0000 7f1ac6b82740 1 --2- 172.21.15.5:0/914819573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a81055460 0x561a8105c880 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.181+0000 7f1ac6b82740 1 --2- 172.21.15.5:0/914819573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a81054b40 0x561a8105f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.181+0000 7f1ac6b82740 1 -- 172.21.15.5:0/914819573 >> 172.21.15.5:0/914819573 conn(0x561a81050e60 msgr2=0x561a81056c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:41.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.181+0000 7f1aa2a00640 0 client.27517 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:54:41.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.181+0000 7f1ac6b82740 1 -- 172.21.15.5:0/914819573 shutdown_connections 2024-04-24T06:54:41.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.181+0000 7f1ac6b82740 1 -- 172.21.15.5:0/914819573 wait complete. 2024-04-24T06:54:41.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.181+0000 7f1ac6b82740 20 client.27517 trim_cache size 0 max 16384 2024-04-24T06:54:41.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.195+0000 7f1ac6b82740 1 Processor -- start 2024-04-24T06:54:41.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.196+0000 7f1ac6b82740 1 -- start start 2024-04-24T06:54:41.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.196+0000 7f1ac6b82740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a81054b40 0x561a8105b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.196+0000 7f1ac6b82740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a81055460 0x561a8105f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.196+0000 7f1ac6b82740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a8105fb30 0x561a8104fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.196+0000 7f1ac6b82740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc1b27b0 con 0x561a81055460 2024-04-24T06:54:41.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.196+0000 7f1ac6b82740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc1b0080 con 0x561a81054b40 2024-04-24T06:54:41.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.196+0000 7f1ac6b82740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc1adef0 con 0x561a8105fb30 2024-04-24T06:54:41.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.196+0000 7f1abbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a81055460 0x561a8105f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.196+0000 7f1abbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a81055460 0x561a8105f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50502/0 (socket says 172.21.15.5:50502) 2024-04-24T06:54:41.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.196+0000 7f1abbe00640 1 -- 172.21.15.5:0/4118337769 learned_addr learned my addr 172.21.15.5:0/4118337769 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:41.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.196+0000 7f1ac1c00640 1 --2- 172.21.15.5:0/4118337769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a81054b40 0x561a8105b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.196+0000 7f1abbe00640 1 -- 172.21.15.5:0/4118337769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a8105fb30 msgr2=0x561a8104fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:41.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.197+0000 7f1abbe00640 1 --2- 172.21.15.5:0/4118337769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a8105fb30 0x561a8104fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.197+0000 7f1abbe00640 1 -- 172.21.15.5:0/4118337769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a81054b40 msgr2=0x561a8105b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:41.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.197+0000 7f1abbe00640 1 --2- 172.21.15.5:0/4118337769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a81054b40 0x561a8105b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.197+0000 7f1abbe00640 1 -- 172.21.15.5:0/4118337769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a80d967d0 con 0x561a81055460 2024-04-24T06:54:41.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.197+0000 7f1abbe00640 1 --2- 172.21.15.5:0/4118337769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a81055460 0x561a8105f0c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8009780 tx=0x7f1aa8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=867d6bd30e70797d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:41.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.197+0000 7f1aad600640 1 -- 172.21.15.5:0/4118337769 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8053070 con 0x561a81055460 2024-04-24T06:54:41.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.197+0000 7f1aad600640 1 -- 172.21.15.5:0/4118337769 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1aa8032d20 con 0x561a81055460 2024-04-24T06:54:41.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.197+0000 7f1aad600640 1 -- 172.21.15.5:0/4118337769 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8037040 con 0x561a81055460 2024-04-24T06:54:41.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.198+0000 7f1ac6b82740 1 -- 172.21.15.5:0/4118337769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a81055460 msgr2=0x561a8105f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:41.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.198+0000 7f1ac6b82740 1 --2- 172.21.15.5:0/4118337769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a81055460 0x561a8105f0c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8009780 tx=0x7f1aa8000bc0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.198+0000 7f1ac6b82740 1 -- 172.21.15.5:0/4118337769 shutdown_connections 2024-04-24T06:54:41.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.198+0000 7f1ac6b82740 1 --2- 172.21.15.5:0/4118337769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a8105fb30 0x561a8104fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.198+0000 7f1ac6b82740 1 --2- 172.21.15.5:0/4118337769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a81055460 0x561a8105f0c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.198+0000 7f1ac6b82740 1 --2- 172.21.15.5:0/4118337769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a81054b40 0x561a8105b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.198+0000 7f1ac6b82740 1 -- 172.21.15.5:0/4118337769 >> 172.21.15.5:0/4118337769 conn(0x561a81050e60 msgr2=0x561a8104f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:41.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.198+0000 7f1ac6b82740 1 -- 172.21.15.5:0/4118337769 shutdown_connections 2024-04-24T06:54:41.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.198+0000 7f1ac6b82740 1 -- 172.21.15.5:0/4118337769 wait complete. 2024-04-24T06:54:41.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.198+0000 7f1ac6b82740 1 -- start start 2024-04-24T06:54:41.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.198+0000 7f1ac6b82740 1 Processor -- start 2024-04-24T06:54:41.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.199+0000 7f1ac6b82740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a81054b40 0x561a81067da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.199+0000 7f1ac6b82740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a81055460 0x561a810682e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.199+0000 7f1ac6b82740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a8105fb30 0x561a81062d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.199+0000 7f1ac6b82740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc1ae540 con 0x561a81054b40 2024-04-24T06:54:41.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.199+0000 7f1ac6b82740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc1af9b0 con 0x561a81055460 2024-04-24T06:54:41.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.199+0000 7f1ac6b82740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc1aec10 con 0x561a8105fb30 2024-04-24T06:54:41.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.199+0000 7f1ac6b82740 10 client.0 _ll_register_callbacks cb 0x7f1ab0005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:54:41.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.199+0000 7f1ac1c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a81054b40 0x561a81067da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.199+0000 7f1ac1c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a81054b40 0x561a81067da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50512/0 (socket says 172.21.15.5:50512) 2024-04-24T06:54:41.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.199+0000 7f1ac1c00640 1 -- 172.21.15.5:0/1413448880 learned_addr learned my addr 172.21.15.5:0/1413448880 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:41.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.199+0000 7f1abbe00640 1 --2- 172.21.15.5:0/1413448880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a81055460 0x561a810682e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.199+0000 7f1ac1c00640 1 --2- 172.21.15.5:0/1413448880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a8105fb30 0x561a81062d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.200+0000 7f1ac1c00640 1 -- 172.21.15.5:0/1413448880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a8105fb30 msgr2=0x561a81062d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:41.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.200+0000 7f1ac1c00640 1 --2- 172.21.15.5:0/1413448880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a8105fb30 0x561a81062d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.200+0000 7f1ac1c00640 1 -- 172.21.15.5:0/1413448880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a81055460 msgr2=0x561a810682e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:41.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.200+0000 7f1ac1c00640 1 --2- 172.21.15.5:0/1413448880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a81055460 0x561a810682e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.200+0000 7f1ac1c00640 1 -- 172.21.15.5:0/1413448880 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a81063550 con 0x561a81054b40 2024-04-24T06:54:41.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.200+0000 7f1ac1c00640 1 --2- 172.21.15.5:0/1413448880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a81054b40 0x561a81067da0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4015040 tx=0x7f1ab4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d85576ca44920fa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:41.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.200+0000 7f1aa2a00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:54:41.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.200+0000 7f1aa2a00640 1 -- 172.21.15.5:0/1413448880 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4032020 con 0x561a81054b40 2024-04-24T06:54:41.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.200+0000 7f1aa2a00640 1 -- 172.21.15.5:0/1413448880 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f1ab4004b10 con 0x561a81054b40 2024-04-24T06:54:41.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.200+0000 7f1ac6b82740 1 -- 172.21.15.5:0/1413448880 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1aa8032d20 con 0x561a81054b40 2024-04-24T06:54:41.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.200+0000 7f1aa2a00640 1 -- 172.21.15.5:0/1413448880 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1ab40046c0 con 0x561a81054b40 2024-04-24T06:54:41.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.200+0000 7f1ac6b82740 10 client.17361 fetch_fsmap learned FSMap version 27 2024-04-24T06:54:41.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.200+0000 7f1ac6b82740 1 -- 172.21.15.5:0/1413448880 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561a810d6b70 con 0x561a81054b40 2024-04-24T06:54:41.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.201+0000 7f1aa2a00640 1 -- 172.21.15.5:0/1413448880 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1aa8032d20 con 0x561a81054b40 2024-04-24T06:54:41.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.201+0000 7f1aa2a00640 1 -- 172.21.15.5:0/1413448880 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f1ab4005030 con 0x561a81054b40 2024-04-24T06:54:41.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.201+0000 7f1ac6b82740 10 client.17361 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:54:41.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.201+0000 7f1ac6b82740 10 client.17361 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-04-24T06:54:41.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.201+0000 7f1ac6b82740 20 client.17361 populate_metadata read hostname 'smithi005' 2024-04-24T06:54:41.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.201+0000 7f1ac6b82740 1 --2- 172.21.15.5:0/1413448880 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x561a810d7110 0x561a810d9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.201+0000 7f1ac6b82740 4 client.17361 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:54:41.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.201+0000 7f1ac6b82740 1 -- 172.21.15.5:0/1413448880 --> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561a80fc1a60 con 0x561a810d7110 2024-04-24T06:54:41.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.201+0000 7f1abbe00640 1 --2- 172.21.15.5:0/1413448880 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x561a810d7110 0x561a810d9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.202+0000 7f1abbe00640 1 --2- 172.21.15.5:0/1413448880 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x561a810d7110 0x561a810d9530 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7a4a8be87f263064 server_cookie=ee0e290a0f1bc136 in_seq=0 out_seq=0 2024-04-24T06:54:41.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.202+0000 7f1aa2a00640 10 client.17361 ms_handle_connect on v2:172.21.15.5:6838/3173201459 2024-04-24T06:54:41.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.202+0000 7f1aa2a00640 1 -- 172.21.15.5:0/1413448880 <== mds.2 v2:172.21.15.5:6838/3173201459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561a80fc1a60 con 0x561a810d7110 2024-04-24T06:54:41.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.202+0000 7f1aa2a00640 10 client.17361 handle_command_reply: tid=0 2024-04-24T06:54:41.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.202+0000 7f1ac6b82740 1 client.17361 shutdown 2024-04-24T06:54:41.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.202+0000 7f1ac6b82740 1 -- 172.21.15.5:0/1413448880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a81054b40 msgr2=0x561a81067da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:41.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.202+0000 7f1ac6b82740 1 --2- 172.21.15.5:0/1413448880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a81054b40 0x561a81067da0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4015040 tx=0x7f1ab4009bf0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.202+0000 7f1ac6b82740 1 -- 172.21.15.5:0/1413448880 shutdown_connections 2024-04-24T06:54:41.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.202+0000 7f1ac6b82740 1 --2- 172.21.15.5:0/1413448880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a8105fb30 0x561a81062d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.202+0000 7f1ac6b82740 1 --2- 172.21.15.5:0/1413448880 >> [v2:172.21.15.5:6838/3173201459,v1:172.21.15.5:6839/3173201459] conn(0x561a810d7110 0x561a810d9530 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.202+0000 7f1ac6b82740 1 --2- 172.21.15.5:0/1413448880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a81055460 0x561a810682e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.202+0000 7f1ac6b82740 1 --2- 172.21.15.5:0/1413448880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a81054b40 0x561a81067da0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.202+0000 7f1aa2a00640 0 client.17361 ms_handle_reset on v2:172.21.15.5:6838/3173201459 2024-04-24T06:54:41.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.202+0000 7f1ac6b82740 1 -- 172.21.15.5:0/1413448880 >> 172.21.15.5:0/1413448880 conn(0x561a81050e60 msgr2=0x561a8104f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:41.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.202+0000 7f1ac6b82740 1 -- 172.21.15.5:0/1413448880 shutdown_connections 2024-04-24T06:54:41.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.202+0000 7f1ac6b82740 1 -- 172.21.15.5:0/1413448880 wait complete. 2024-04-24T06:54:41.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.202+0000 7f1ac6b82740 20 client.17361 trim_cache size 0 max 16384 2024-04-24T06:54:41.212 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:54:41.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.203+0000 7f1ac2600640 1 -- 172.21.15.5:0/993759167 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1aa40a3d20 msgr2=0x7f1aa40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:41.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.203+0000 7f1ac2600640 1 --2- 172.21.15.5:0/993759167 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1aa40a3d20 0x7f1aa40a61e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x561a81056100 tx=0x7f1ab00d45f0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.203+0000 7f1ac2600640 1 -- 172.21.15.5:0/993759167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1abc104250 msgr2=0x7f1abc1a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:41.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.203+0000 7f1ac2600640 1 --2- 172.21.15.5:0/993759167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1abc104250 0x7f1abc1a0690 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1ab000e9c0 tx=0x7f1ab000ee80 comp rx=0 tx=0).stop 2024-04-24T06:54:41.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.203+0000 7f1ac2600640 1 -- 172.21.15.5:0/993759167 shutdown_connections 2024-04-24T06:54:41.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.203+0000 7f1ac2600640 1 --2- 172.21.15.5:0/993759167 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1aa40a3d20 0x7f1aa40a61e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.203+0000 7f1ac2600640 1 --2- 172.21.15.5:0/993759167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1abc19a750 0x7f1abc19abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.203+0000 7f1ac2600640 1 --2- 172.21.15.5:0/993759167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1abc109320 0x7f1abc1a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.203+0000 7f1ac2600640 1 --2- 172.21.15.5:0/993759167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1abc104250 0x7f1abc1a0690 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.203+0000 7f1ac2600640 1 -- 172.21.15.5:0/993759167 >> 172.21.15.5:0/993759167 conn(0x7f1abc0ffad0 msgr2=0x7f1abc100ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:41.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.203+0000 7f1ac2600640 1 -- 172.21.15.5:0/993759167 shutdown_connections 2024-04-24T06:54:41.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.203+0000 7f1ac2600640 1 -- 172.21.15.5:0/993759167 wait complete. 2024-04-24T06:54:41.221 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T06:54:41.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.423+0000 7f82e2400640 1 -- 172.21.15.5:0/1580742030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82dc102fe0 msgr2=0x7f82dc1033c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:41.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.423+0000 7f82e2400640 1 --2- 172.21.15.5:0/1580742030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82dc102fe0 0x7f82dc1033c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009900 tx=0x7f82cc02a480 comp rx=0 tx=0).stop 2024-04-24T06:54:41.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.423+0000 7f82e2400640 1 -- 172.21.15.5:0/1580742030 shutdown_connections 2024-04-24T06:54:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.423+0000 7f82e2400640 1 --2- 172.21.15.5:0/1580742030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82dc10c400 0x7f82dc10e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.423+0000 7f82e2400640 1 --2- 172.21.15.5:0/1580742030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82dc103900 0x7f82dc10bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.423+0000 7f82e2400640 1 --2- 172.21.15.5:0/1580742030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82dc102fe0 0x7f82dc1033c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.423+0000 7f82e2400640 1 -- 172.21.15.5:0/1580742030 >> 172.21.15.5:0/1580742030 conn(0x7f82dc077ab0 msgr2=0x7f82dc0fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.424+0000 7f82e2400640 1 -- 172.21.15.5:0/1580742030 shutdown_connections 2024-04-24T06:54:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.424+0000 7f82e2400640 1 -- 172.21.15.5:0/1580742030 wait complete. 2024-04-24T06:54:41.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.425+0000 7f82e2400640 1 Processor -- start 2024-04-24T06:54:41.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.425+0000 7f82e2400640 1 -- start start 2024-04-24T06:54:41.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.426+0000 7f82e2400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82dc102fe0 0x7f82dc100c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.426+0000 7f82e2400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82dc103900 0x7f82dc101170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.426+0000 7f82e2400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82dc10c400 0x7f82dc1016b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.426+0000 7f82e2400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc112200 con 0x7f82dc102fe0 2024-04-24T06:54:41.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.426+0000 7f82e2400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc112080 con 0x7f82dc10c400 2024-04-24T06:54:41.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.426+0000 7f82e2400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc112380 con 0x7f82dc103900 2024-04-24T06:54:41.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.426+0000 7f82dbe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82dc103900 0x7f82dc101170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.426+0000 7f82e1000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82dc102fe0 0x7f82dc100c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.426+0000 7f82dbe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82dc103900 0x7f82dc101170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:33080/0 (socket says 172.21.15.5:33080) 2024-04-24T06:54:41.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.426+0000 7f82dbe00640 1 -- 172.21.15.5:0/2806560066 learned_addr learned my addr 172.21.15.5:0/2806560066 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:41.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.426+0000 7f82e1a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82dc10c400 0x7f82dc1016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.426+0000 7f82dbe00640 1 -- 172.21.15.5:0/2806560066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82dc10c400 msgr2=0x7f82dc1016b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:41.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.426+0000 7f82dbe00640 1 --2- 172.21.15.5:0/2806560066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82dc10c400 0x7f82dc1016b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.426+0000 7f82dbe00640 1 -- 172.21.15.5:0/2806560066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82dc102fe0 msgr2=0x7f82dc100c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:41.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.426+0000 7f82dbe00640 1 --2- 172.21.15.5:0/2806560066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82dc102fe0 0x7f82dc100c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.427+0000 7f82dbe00640 1 -- 172.21.15.5:0/2806560066 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82dc1a6c60 con 0x7f82dc103900 2024-04-24T06:54:41.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.427+0000 7f82dbe00640 1 --2- 172.21.15.5:0/2806560066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82dc103900 0x7f82dc101170 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f82c400eb90 tx=0x7f82c400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:41.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.427+0000 7f82d9600640 1 -- 172.21.15.5:0/2806560066 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c4004d20 con 0x7f82dc103900 2024-04-24T06:54:41.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.427+0000 7f82d9600640 1 -- 172.21.15.5:0/2806560066 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c400ee90 con 0x7f82dc103900 2024-04-24T06:54:41.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.427+0000 7f82d9600640 1 -- 172.21.15.5:0/2806560066 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c4010830 con 0x7f82dc103900 2024-04-24T06:54:41.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.427+0000 7f82e2400640 1 -- 172.21.15.5:0/2806560066 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82dc1a6f50 con 0x7f82dc103900 2024-04-24T06:54:41.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.427+0000 7f82e2400640 1 -- 172.21.15.5:0/2806560066 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82dc1a7360 con 0x7f82dc103900 2024-04-24T06:54:41.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.431+0000 7f82e69b0740 1 Processor -- start 2024-04-24T06:54:41.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.431+0000 7f82e69b0740 1 -- start start 2024-04-24T06:54:41.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.431+0000 7f82e69b0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3a0624b40 0x55a3a0624f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.431+0000 7f82e69b0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3a0625460 0x55a3a062f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.431+0000 7f82e69b0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3a062fb30 0x55a3a0631f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.431+0000 7f82e69b0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a3a058a1d0 con 0x55a3a0624b40 2024-04-24T06:54:41.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.431+0000 7f82e69b0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a3a0530450 con 0x55a3a062fb30 2024-04-24T06:54:41.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.431+0000 7f82e69b0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a3a0586190 con 0x55a3a0625460 2024-04-24T06:54:41.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.431+0000 7f82d9600640 1 -- 172.21.15.5:0/2806560066 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f82c4005020 con 0x7f82dc103900 2024-04-24T06:54:41.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.431+0000 7f82e1a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3a0624b40 0x55a3a0624f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.431+0000 7f82e1a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3a0624b40 0x55a3a0624f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50552/0 (socket says 172.21.15.5:50552) 2024-04-24T06:54:41.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.431+0000 7f82e1a00640 1 -- 172.21.15.5:0/3419267731 learned_addr learned my addr 172.21.15.5:0/3419267731 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:41.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.432+0000 7f82e1000640 1 --2- 172.21.15.5:0/3419267731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3a0625460 0x55a3a062f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.432+0000 7f82dbe00640 1 --2- 172.21.15.5:0/3419267731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3a062fb30 0x55a3a0631f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.432+0000 7f82e1a00640 1 -- 172.21.15.5:0/3419267731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3a0625460 msgr2=0x55a3a062f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:41.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.432+0000 7f82e1a00640 1 --2- 172.21.15.5:0/3419267731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3a0625460 0x55a3a062f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.432+0000 7f82e1a00640 1 -- 172.21.15.5:0/3419267731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3a062fb30 msgr2=0x55a3a0631f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:41.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.432+0000 7f82e1a00640 1 --2- 172.21.15.5:0/3419267731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3a062fb30 0x55a3a0631f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.432+0000 7f82e1a00640 1 -- 172.21.15.5:0/3419267731 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3a05ed310 con 0x55a3a0624b40 2024-04-24T06:54:41.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.432+0000 7f82dbe00640 1 --2- 172.21.15.5:0/3419267731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3a062fb30 0x55a3a0631f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:54:41.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.432+0000 7f82e1a00640 1 --2- 172.21.15.5:0/3419267731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3a0624b40 0x55a3a0624f20 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f82d4009c20 tx=0x7f82d4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c66625e7a211ab server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:41.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.432+0000 7f82d2000640 1 -- 172.21.15.5:0/3419267731 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d401e070 con 0x55a3a0624b40 2024-04-24T06:54:41.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.432+0000 7f82d2000640 1 -- 172.21.15.5:0/3419267731 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82d4019b10 con 0x55a3a0624b40 2024-04-24T06:54:41.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.433+0000 7f82d2000640 1 -- 172.21.15.5:0/3419267731 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d400ea80 con 0x55a3a0624b40 2024-04-24T06:54:41.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.433+0000 7f82d9600640 1 --2- 172.21.15.5:0/2806560066 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f82bc0a3da0 0x7f82bc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.433+0000 7f82e69b0740 1 -- 172.21.15.5:0/3419267731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3a0624b40 msgr2=0x55a3a0624f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:41.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.433+0000 7f82e69b0740 1 --2- 172.21.15.5:0/3419267731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3a0624b40 0x55a3a0624f20 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f82d4009c20 tx=0x7f82d4011810 comp rx=0 tx=0).stop 2024-04-24T06:54:41.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.433+0000 7f82e69b0740 1 -- 172.21.15.5:0/3419267731 shutdown_connections 2024-04-24T06:54:41.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.433+0000 7f82e69b0740 1 --2- 172.21.15.5:0/3419267731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3a062fb30 0x55a3a0631f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.433+0000 7f82e69b0740 1 --2- 172.21.15.5:0/3419267731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3a0625460 0x55a3a062f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.433+0000 7f82e69b0740 1 --2- 172.21.15.5:0/3419267731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3a0624b40 0x55a3a0624f20 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.433+0000 7f82e69b0740 1 -- 172.21.15.5:0/3419267731 >> 172.21.15.5:0/3419267731 conn(0x55a3a0620e60 msgr2=0x55a3a0621240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:41.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.433+0000 7f82d9600640 1 -- 172.21.15.5:0/2806560066 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f82c40d0e40 con 0x7f82dc103900 2024-04-24T06:54:41.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.433+0000 7f82e69b0740 1 -- 172.21.15.5:0/3419267731 shutdown_connections 2024-04-24T06:54:41.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.433+0000 7f82e1000640 1 --2- 172.21.15.5:0/2806560066 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f82bc0a3da0 0x7f82bc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.434+0000 7f82e69b0740 1 -- 172.21.15.5:0/3419267731 wait complete. 2024-04-24T06:54:41.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.434+0000 7f82e69b0740 1 -- start start 2024-04-24T06:54:41.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.434+0000 7f82e1000640 1 --2- 172.21.15.5:0/2806560066 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f82bc0a3da0 0x7f82bc0a6260 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f82cc000c00 tx=0x7f82cc005b50 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:41.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.434+0000 7f82e69b0740 1 Processor -- start 2024-04-24T06:54:41.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.435+0000 7f82e69b0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3a0624b40 0x55a3a062f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.435+0000 7f82e69b0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3a0625460 0x55a3a062c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.435+0000 7f82e69b0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3a062fb30 0x55a3a062cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.435+0000 7f82e69b0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a3a06329c0 con 0x55a3a062fb30 2024-04-24T06:54:41.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.435+0000 7f82e69b0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a3a0543810 con 0x55a3a0624b40 2024-04-24T06:54:41.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.435+0000 7f82e69b0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a3a0632b40 con 0x55a3a0625460 2024-04-24T06:54:41.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.435+0000 7f82e69b0740 10 client.0 _ll_register_callbacks cb 0x55a3a0511140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:54:41.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.435+0000 7f82e1000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3a0624b40 0x55a3a062f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.435+0000 7f82e1000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3a0624b40 0x55a3a062f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:57598/0 (socket says 172.21.15.5:57598) 2024-04-24T06:54:41.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.435+0000 7f82e1000640 1 -- 172.21.15.5:0/518996682 learned_addr learned my addr 172.21.15.5:0/518996682 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:41.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.435+0000 7f82e1a00640 1 --2- 172.21.15.5:0/518996682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3a062fb30 0x55a3a062cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.435+0000 7f82dbe00640 1 --2- 172.21.15.5:0/518996682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3a0625460 0x55a3a062c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.435+0000 7f82e1000640 1 -- 172.21.15.5:0/518996682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3a0625460 msgr2=0x55a3a062c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:41.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.435+0000 7f82e1000640 1 --2- 172.21.15.5:0/518996682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3a0625460 0x55a3a062c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.435+0000 7f82e1000640 1 -- 172.21.15.5:0/518996682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3a062fb30 msgr2=0x55a3a062cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:41.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.435+0000 7f82e1000640 1 --2- 172.21.15.5:0/518996682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3a062fb30 0x55a3a062cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.435+0000 7f82e1000640 1 -- 172.21.15.5:0/518996682 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3a05543f0 con 0x55a3a0624b40 2024-04-24T06:54:41.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.436+0000 7f82e1000640 1 --2- 172.21.15.5:0/518996682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3a0624b40 0x55a3a062f5d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009900 tx=0x7f82cc009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fb3681a78241684 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:41.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.436+0000 7f82caa00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-24T06:54:41.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.436+0000 7f82caa00640 1 -- 172.21.15.5:0/518996682 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc04e070 con 0x55a3a0624b40 2024-04-24T06:54:41.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.436+0000 7f82e69b0740 1 -- 172.21.15.5:0/518996682 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82d4019b10 con 0x55a3a0624b40 2024-04-24T06:54:41.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.436+0000 7f82caa00640 1 -- 172.21.15.5:0/518996682 <== mon.2 v2:172.21.15.37:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f82cc004ad0 con 0x55a3a0624b40 2024-04-24T06:54:41.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.436+0000 7f82caa00640 1 -- 172.21.15.5:0/518996682 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f82cc05a050 con 0x55a3a0624b40 2024-04-24T06:54:41.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.436+0000 7f82e69b0740 10 client.27509 fetch_fsmap learned FSMap version 27 2024-04-24T06:54:41.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.436+0000 7f82e69b0740 1 -- 172.21.15.5:0/518996682 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a3a029c3f0 con 0x55a3a0624b40 2024-04-24T06:54:41.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.436+0000 7f82caa00640 1 -- 172.21.15.5:0/518996682 <== mon.2 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82d4019b10 con 0x55a3a0624b40 2024-04-24T06:54:41.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.437+0000 7f82caa00640 1 -- 172.21.15.5:0/518996682 <== mon.2 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f82cc0035f0 con 0x55a3a0624b40 2024-04-24T06:54:41.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.437+0000 7f82e69b0740 10 client.27509 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:54:41.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.437+0000 7f82e69b0740 10 client.27509 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T06:54:41.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.437+0000 7f82e69b0740 20 client.27509 populate_metadata read hostname 'smithi005' 2024-04-24T06:54:41.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.437+0000 7f82e69b0740 1 --2- 172.21.15.5:0/518996682 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55a3a063fce0 0x55a3a06420f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.437+0000 7f82e69b0740 4 client.27509 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:54:41.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.437+0000 7f82e69b0740 1 -- 172.21.15.5:0/518996682 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3a0597bf0 con 0x55a3a063fce0 2024-04-24T06:54:41.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.437+0000 7f82dbe00640 1 --2- 172.21.15.5:0/518996682 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55a3a063fce0 0x55a3a06420f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.438+0000 7f82dbe00640 1 --2- 172.21.15.5:0/518996682 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55a3a063fce0 0x55a3a06420f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8bfd2c3f84d5a3fe server_cookie=1bea78870d744a4f in_seq=0 out_seq=0 2024-04-24T06:54:41.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.438+0000 7f82caa00640 10 client.27509 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:54:41.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.438+0000 7f82caa00640 1 -- 172.21.15.5:0/518996682 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a3a0597bf0 con 0x55a3a063fce0 2024-04-24T06:54:41.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.438+0000 7f82caa00640 10 client.27509 handle_command_reply: tid=0 2024-04-24T06:54:41.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.439+0000 7f82e69b0740 1 client.27509 shutdown 2024-04-24T06:54:41.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.439+0000 7f82e69b0740 1 -- 172.21.15.5:0/518996682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3a0624b40 msgr2=0x55a3a062f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:41.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.439+0000 7f82e69b0740 1 --2- 172.21.15.5:0/518996682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3a0624b40 0x55a3a062f5d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009900 tx=0x7f82cc009260 comp rx=0 tx=0).stop 2024-04-24T06:54:41.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.439+0000 7f82e69b0740 1 -- 172.21.15.5:0/518996682 shutdown_connections 2024-04-24T06:54:41.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.439+0000 7f82e69b0740 1 --2- 172.21.15.5:0/518996682 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55a3a063fce0 0x55a3a06420f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.439+0000 7f82e69b0740 1 --2- 172.21.15.5:0/518996682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3a062fb30 0x55a3a062cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.439+0000 7f82e69b0740 1 --2- 172.21.15.5:0/518996682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3a0625460 0x55a3a062c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.439+0000 7f82caa00640 0 client.27509 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:54:41.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.439+0000 7f82e69b0740 1 --2- 172.21.15.5:0/518996682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3a0624b40 0x55a3a062f5d0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.439+0000 7f82e69b0740 1 -- 172.21.15.5:0/518996682 >> 172.21.15.5:0/518996682 conn(0x55a3a0620e60 msgr2=0x55a3a0626c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:41.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.439+0000 7f82e69b0740 1 -- 172.21.15.5:0/518996682 shutdown_connections 2024-04-24T06:54:41.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.439+0000 7f82e69b0740 1 -- 172.21.15.5:0/518996682 wait complete. 2024-04-24T06:54:41.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.439+0000 7f82e69b0740 20 client.27509 trim_cache size 0 max 16384 2024-04-24T06:54:41.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.454+0000 7f82e69b0740 1 Processor -- start 2024-04-24T06:54:41.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.454+0000 7f82e69b0740 1 -- start start 2024-04-24T06:54:41.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.454+0000 7f82e69b0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3a0624b40 0x55a3a06272a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.454+0000 7f82e69b0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3a0625460 0x55a3a062f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.454+0000 7f82e69b0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3a062fb30 0x55a3a06479c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.454+0000 7f82e69b0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc1b2630 con 0x55a3a0625460 2024-04-24T06:54:41.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.454+0000 7f82e69b0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc1aff00 con 0x55a3a062fb30 2024-04-24T06:54:41.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.454+0000 7f82e69b0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc1add70 con 0x55a3a0624b40 2024-04-24T06:54:41.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.454+0000 7f82dbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3a0625460 0x55a3a062f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.454+0000 7f82e1000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3a0624b40 0x55a3a06272a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.454+0000 7f82dbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3a0625460 0x55a3a062f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50580/0 (socket says 172.21.15.5:50580) 2024-04-24T06:54:41.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.454+0000 7f82dbe00640 1 -- 172.21.15.5:0/3885022671 learned_addr learned my addr 172.21.15.5:0/3885022671 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:41.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.455+0000 7f82e1a00640 1 --2- 172.21.15.5:0/3885022671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3a062fb30 0x55a3a06479c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.455+0000 7f82dbe00640 1 -- 172.21.15.5:0/3885022671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3a0624b40 msgr2=0x55a3a06272a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:41.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.455+0000 7f82dbe00640 1 --2- 172.21.15.5:0/3885022671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3a0624b40 0x55a3a06272a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.455+0000 7f82dbe00640 1 -- 172.21.15.5:0/3885022671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3a062fb30 msgr2=0x55a3a06479c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:41.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.455+0000 7f82dbe00640 1 --2- 172.21.15.5:0/3885022671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3a062fb30 0x55a3a06479c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.455+0000 7f82dbe00640 1 -- 172.21.15.5:0/3885022671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3a062d320 con 0x55a3a0625460 2024-04-24T06:54:41.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.455+0000 7f82dbe00640 1 --2- 172.21.15.5:0/3885022671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3a0625460 0x55a3a062f100 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f82c400cba0 tx=0x7f82c40d0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b483253d4cdcd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:41.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.455+0000 7f82d1600640 1 -- 172.21.15.5:0/3885022671 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c409c8b0 con 0x55a3a0625460 2024-04-24T06:54:41.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.455+0000 7f82d1600640 1 -- 172.21.15.5:0/3885022671 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c409d470 con 0x55a3a0625460 2024-04-24T06:54:41.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.455+0000 7f82d1600640 1 -- 172.21.15.5:0/3885022671 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c409c6d0 con 0x55a3a0625460 2024-04-24T06:54:41.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.456+0000 7f82e69b0740 1 -- 172.21.15.5:0/3885022671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3a0625460 msgr2=0x55a3a062f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:41.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.456+0000 7f82e69b0740 1 --2- 172.21.15.5:0/3885022671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3a0625460 0x55a3a062f100 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f82c400cba0 tx=0x7f82c40d0040 comp rx=0 tx=0).stop 2024-04-24T06:54:41.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.456+0000 7f82e69b0740 1 -- 172.21.15.5:0/3885022671 shutdown_connections 2024-04-24T06:54:41.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.456+0000 7f82e69b0740 1 --2- 172.21.15.5:0/3885022671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3a062fb30 0x55a3a06479c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.456+0000 7f82e69b0740 1 --2- 172.21.15.5:0/3885022671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3a0625460 0x55a3a062f100 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.456+0000 7f82e69b0740 1 --2- 172.21.15.5:0/3885022671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3a0624b40 0x55a3a06272a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.456+0000 7f82e69b0740 1 -- 172.21.15.5:0/3885022671 >> 172.21.15.5:0/3885022671 conn(0x55a3a0620e60 msgr2=0x55a3a0627bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:41.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.456+0000 7f82e69b0740 1 -- 172.21.15.5:0/3885022671 shutdown_connections 2024-04-24T06:54:41.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.456+0000 7f82e69b0740 1 -- 172.21.15.5:0/3885022671 wait complete. 2024-04-24T06:54:41.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.457+0000 7f82e69b0740 1 -- start start 2024-04-24T06:54:41.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.457+0000 7f82e69b0740 1 Processor -- start 2024-04-24T06:54:41.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.457+0000 7f82e69b0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3a0624b40 0x55a3a0638220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.457+0000 7f82e69b0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3a0625460 0x55a3a0629c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.457+0000 7f82e69b0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3a062fb30 0x55a3a062a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.457+0000 7f82e69b0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc1ae3c0 con 0x55a3a062fb30 2024-04-24T06:54:41.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.457+0000 7f82e69b0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc1af830 con 0x55a3a0624b40 2024-04-24T06:54:41.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.457+0000 7f82e69b0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc1aea90 con 0x55a3a0625460 2024-04-24T06:54:41.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.457+0000 7f82e69b0740 10 client.0 _ll_register_callbacks cb 0x7f82cc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:54:41.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.457+0000 7f82e1a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3a062fb30 0x55a3a062a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.458+0000 7f82e1a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3a062fb30 0x55a3a062a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50588/0 (socket says 172.21.15.5:50588) 2024-04-24T06:54:41.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.458+0000 7f82e1a00640 1 -- 172.21.15.5:0/3842165234 learned_addr learned my addr 172.21.15.5:0/3842165234 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:41.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.458+0000 7f82dbe00640 1 --2- 172.21.15.5:0/3842165234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3a0625460 0x55a3a0629c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.458+0000 7f82e1000640 1 --2- 172.21.15.5:0/3842165234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3a0624b40 0x55a3a0638220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.458+0000 7f82e1a00640 1 -- 172.21.15.5:0/3842165234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3a0625460 msgr2=0x55a3a0629c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:41.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.458+0000 7f82e1a00640 1 --2- 172.21.15.5:0/3842165234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3a0625460 0x55a3a0629c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.458+0000 7f82e1a00640 1 -- 172.21.15.5:0/3842165234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3a0624b40 msgr2=0x55a3a0638220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:41.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.458+0000 7f82e1a00640 1 --2- 172.21.15.5:0/3842165234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3a0624b40 0x55a3a0638220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.458+0000 7f82e1a00640 1 -- 172.21.15.5:0/3842165234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3a03667d0 con 0x55a3a062fb30 2024-04-24T06:54:41.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.458+0000 7f82e1000640 1 --2- 172.21.15.5:0/3842165234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3a0624b40 0x55a3a0638220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:54:41.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.458+0000 7f82e1a00640 1 --2- 172.21.15.5:0/3842165234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3a062fb30 0x55a3a062a250 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f82d401c040 tx=0x7f82d4005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b1308b9d1336ed server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:41.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.458+0000 7f82caa00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:54:41.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.459+0000 7f82caa00640 1 -- 172.21.15.5:0/3842165234 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d401e040 con 0x55a3a062fb30 2024-04-24T06:54:41.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.459+0000 7f82e69b0740 1 -- 172.21.15.5:0/3842165234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82c409d470 con 0x55a3a062fb30 2024-04-24T06:54:41.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.459+0000 7f82caa00640 1 -- 172.21.15.5:0/3842165234 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f82d4005540 con 0x55a3a062fb30 2024-04-24T06:54:41.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.459+0000 7f82caa00640 1 -- 172.21.15.5:0/3842165234 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f82d4035050 con 0x55a3a062fb30 2024-04-24T06:54:41.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.459+0000 7f82e69b0740 10 client.17379 fetch_fsmap learned FSMap version 27 2024-04-24T06:54:41.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.459+0000 7f82e69b0740 1 -- 172.21.15.5:0/3842165234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a3a06a62e0 con 0x55a3a062fb30 2024-04-24T06:54:41.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.459+0000 7f82caa00640 1 -- 172.21.15.5:0/3842165234 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c409d470 con 0x55a3a062fb30 2024-04-24T06:54:41.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.459+0000 7f82caa00640 1 -- 172.21.15.5:0/3842165234 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f82d4004bc0 con 0x55a3a062fb30 2024-04-24T06:54:41.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.459+0000 7f82e69b0740 10 client.17379 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:54:41.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.459+0000 7f82e69b0740 10 client.17379 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T06:54:41.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.459+0000 7f82e69b0740 20 client.17379 populate_metadata read hostname 'smithi005' 2024-04-24T06:54:41.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.459+0000 7f82e69b0740 1 --2- 172.21.15.5:0/3842165234 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55a3a06a6870 0x55a3a06a8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.459+0000 7f82e69b0740 4 client.17379 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:54:41.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.459+0000 7f82e69b0740 1 -- 172.21.15.5:0/3842165234 --> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a3a0591a60 con 0x55a3a06a6870 2024-04-24T06:54:41.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.460+0000 7f82e1000640 1 --2- 172.21.15.5:0/3842165234 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55a3a06a6870 0x55a3a06a8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.461+0000 7f82e1000640 1 --2- 172.21.15.5:0/3842165234 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55a3a06a6870 0x55a3a06a8c90 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=25eaa9278a64c61c server_cookie=80af9e2790b43d76 in_seq=0 out_seq=0 2024-04-24T06:54:41.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.461+0000 7f82caa00640 10 client.17379 ms_handle_connect on v2:172.21.15.137:6838/1889774232 2024-04-24T06:54:41.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.461+0000 7f82caa00640 1 -- 172.21.15.5:0/3842165234 <== mds.1 v2:172.21.15.137:6838/1889774232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a3a0591a60 con 0x55a3a06a6870 2024-04-24T06:54:41.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.461+0000 7f82caa00640 10 client.17379 handle_command_reply: tid=0 2024-04-24T06:54:41.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.461+0000 7f82e69b0740 1 client.17379 shutdown 2024-04-24T06:54:41.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.461+0000 7f82e69b0740 1 -- 172.21.15.5:0/3842165234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3a062fb30 msgr2=0x55a3a062a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:41.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.461+0000 7f82e69b0740 1 --2- 172.21.15.5:0/3842165234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3a062fb30 0x55a3a062a250 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f82d401c040 tx=0x7f82d4005040 comp rx=0 tx=0).stop 2024-04-24T06:54:41.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.461+0000 7f82e69b0740 1 -- 172.21.15.5:0/3842165234 shutdown_connections 2024-04-24T06:54:41.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.461+0000 7f82e69b0740 1 --2- 172.21.15.5:0/3842165234 >> [v2:172.21.15.137:6838/1889774232,v1:172.21.15.137:6839/1889774232] conn(0x55a3a06a6870 0x55a3a06a8c90 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.461+0000 7f82e69b0740 1 --2- 172.21.15.5:0/3842165234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3a062fb30 0x55a3a062a250 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.461+0000 7f82e69b0740 1 --2- 172.21.15.5:0/3842165234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3a0625460 0x55a3a0629c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.461+0000 7f82e69b0740 1 --2- 172.21.15.5:0/3842165234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3a0624b40 0x55a3a0638220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.461+0000 7f82e69b0740 1 -- 172.21.15.5:0/3842165234 >> 172.21.15.5:0/3842165234 conn(0x55a3a0620e60 msgr2=0x55a3a061f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:41.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.461+0000 7f82caa00640 0 client.17379 ms_handle_reset on v2:172.21.15.137:6838/1889774232 2024-04-24T06:54:41.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.461+0000 7f82e69b0740 1 -- 172.21.15.5:0/3842165234 shutdown_connections 2024-04-24T06:54:41.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.461+0000 7f82e69b0740 1 -- 172.21.15.5:0/3842165234 wait complete. 2024-04-24T06:54:41.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.461+0000 7f82e69b0740 20 client.17379 trim_cache size 0 max 16384 2024-04-24T06:54:41.470 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:54:41.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.462+0000 7f82e2400640 1 -- 172.21.15.5:0/2806560066 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f82bc0a3da0 msgr2=0x7f82bc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:41.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.462+0000 7f82e2400640 1 --2- 172.21.15.5:0/2806560066 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f82bc0a3da0 0x7f82bc0a6260 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f82cc000c00 tx=0x7f82cc005b50 comp rx=0 tx=0).stop 2024-04-24T06:54:41.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.462+0000 7f82e2400640 1 -- 172.21.15.5:0/2806560066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82dc103900 msgr2=0x7f82dc101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:41.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.462+0000 7f82e2400640 1 --2- 172.21.15.5:0/2806560066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82dc103900 0x7f82dc101170 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f82c400eb90 tx=0x7f82c400c6a0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.462+0000 7f82e2400640 1 -- 172.21.15.5:0/2806560066 shutdown_connections 2024-04-24T06:54:41.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.462+0000 7f82e2400640 1 --2- 172.21.15.5:0/2806560066 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f82bc0a3da0 0x7f82bc0a6260 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.462+0000 7f82e2400640 1 --2- 172.21.15.5:0/2806560066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82dc10c400 0x7f82dc1016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.462+0000 7f82e2400640 1 --2- 172.21.15.5:0/2806560066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82dc103900 0x7f82dc101170 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.462+0000 7f82e2400640 1 --2- 172.21.15.5:0/2806560066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82dc102fe0 0x7f82dc100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.462+0000 7f82e2400640 1 -- 172.21.15.5:0/2806560066 >> 172.21.15.5:0/2806560066 conn(0x7f82dc077ab0 msgr2=0x7f82dc0fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:41.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.462+0000 7f82e2400640 1 -- 172.21.15.5:0/2806560066 shutdown_connections 2024-04-24T06:54:41.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.462+0000 7f82e2400640 1 -- 172.21.15.5:0/2806560066 wait complete. 2024-04-24T06:54:41.478 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T06:54:41.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.667+0000 7f56c3e00640 1 -- 172.21.15.5:0/810151883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56c4108540 msgr2=0x7f56c41089a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:41.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.667+0000 7f56c3e00640 1 --2- 172.21.15.5:0/810151883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56c4108540 0x7f56c41089a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f56ac009930 tx=0x7f56ac02a4d0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.667+0000 7f56c3e00640 1 -- 172.21.15.5:0/810151883 shutdown_connections 2024-04-24T06:54:41.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.667+0000 7f56c3e00640 1 --2- 172.21.15.5:0/810151883 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56c4108ee0 0x7f56c41152b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.667+0000 7f56c3e00640 1 --2- 172.21.15.5:0/810151883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56c4108540 0x7f56c41089a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.667+0000 7f56c3e00640 1 --2- 172.21.15.5:0/810151883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56c410e4f0 0x7f56c410e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.667+0000 7f56c3e00640 1 -- 172.21.15.5:0/810151883 >> 172.21.15.5:0/810151883 conn(0x7f56c4077e10 msgr2=0x7f56c4106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:41.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.667+0000 7f56c3e00640 1 -- 172.21.15.5:0/810151883 shutdown_connections 2024-04-24T06:54:41.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.667+0000 7f56c3e00640 1 -- 172.21.15.5:0/810151883 wait complete. 2024-04-24T06:54:41.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.669+0000 7f56c3e00640 1 Processor -- start 2024-04-24T06:54:41.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.669+0000 7f56c3e00640 1 -- start start 2024-04-24T06:54:41.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.669+0000 7f56c3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56c4108540 0x7f56c419edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.669+0000 7f56c3e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56c4108ee0 0x7f56c419f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.669+0000 7f56c3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56c410e4f0 0x7f56c41a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.669+0000 7f56c3e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f56c4118810 con 0x7f56c410e4f0 2024-04-24T06:54:41.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.669+0000 7f56c3e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f56c4118690 con 0x7f56c4108ee0 2024-04-24T06:54:41.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.669+0000 7f56c3e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f56c4118990 con 0x7f56c4108540 2024-04-24T06:54:41.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.670+0000 7f56c3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56c410e4f0 0x7f56c41a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.670+0000 7f56c2a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56c4108540 0x7f56c419edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.670+0000 7f56c2a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56c4108540 0x7f56c419edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:33144/0 (socket says 172.21.15.5:33144) 2024-04-24T06:54:41.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.670+0000 7f56c3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56c410e4f0 0x7f56c41a3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50604/0 (socket says 172.21.15.5:50604) 2024-04-24T06:54:41.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.670+0000 7f56c2a00640 1 -- 172.21.15.5:0/1989301627 learned_addr learned my addr 172.21.15.5:0/1989301627 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:41.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.670+0000 7f56ba000640 1 --2- 172.21.15.5:0/1989301627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56c4108ee0 0x7f56c419f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.670+0000 7f56c2a00640 1 -- 172.21.15.5:0/1989301627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56c4108ee0 msgr2=0x7f56c419f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:41.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.670+0000 7f56c2a00640 1 --2- 172.21.15.5:0/1989301627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56c4108ee0 0x7f56c419f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.670+0000 7f56c2a00640 1 -- 172.21.15.5:0/1989301627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56c410e4f0 msgr2=0x7f56c41a3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:41.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.670+0000 7f56c2a00640 1 --2- 172.21.15.5:0/1989301627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56c410e4f0 0x7f56c41a3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.670+0000 7f56c2a00640 1 -- 172.21.15.5:0/1989301627 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c41a3dc0 con 0x7f56c4108540 2024-04-24T06:54:41.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.671+0000 7f56c2a00640 1 --2- 172.21.15.5:0/1989301627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56c4108540 0x7f56c419edd0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f56b400d950 tx=0x7f56b400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:41.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.671+0000 7f56bbe00640 1 -- 172.21.15.5:0/1989301627 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b4017070 con 0x7f56c4108540 2024-04-24T06:54:41.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.671+0000 7f56bbe00640 1 -- 172.21.15.5:0/1989301627 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56b4009c30 con 0x7f56c4108540 2024-04-24T06:54:41.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.671+0000 7f56c3e00640 1 -- 172.21.15.5:0/1989301627 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56c41a40b0 con 0x7f56c4108540 2024-04-24T06:54:41.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.671+0000 7f56bbe00640 1 -- 172.21.15.5:0/1989301627 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b400f6b0 con 0x7f56c4108540 2024-04-24T06:54:41.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.671+0000 7f56c3e00640 1 -- 172.21.15.5:0/1989301627 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56c41ab940 con 0x7f56c4108540 2024-04-24T06:54:41.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.674+0000 7f56cca22740 1 Processor -- start 2024-04-24T06:54:41.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.674+0000 7f56cca22740 1 -- start start 2024-04-24T06:54:41.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.674+0000 7f56bbe00640 1 -- 172.21.15.5:0/1989301627 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f56b40109d0 con 0x7f56c4108540 2024-04-24T06:54:41.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.675+0000 7f56cca22740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a227666b40 0x55a227666f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.675+0000 7f56cca22740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a227667460 0x55a2276715f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.675+0000 7f56cca22740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a227671b30 0x55a227673f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.675+0000 7f56cca22740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a2275cc1d0 con 0x55a227666b40 2024-04-24T06:54:41.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.675+0000 7f56cca22740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a227572450 con 0x55a227667460 2024-04-24T06:54:41.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.675+0000 7f56cca22740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a2275c8190 con 0x55a227671b30 2024-04-24T06:54:41.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.675+0000 7f56c3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a227666b40 0x55a227666f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.675+0000 7f56c3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a227666b40 0x55a227666f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50606/0 (socket says 172.21.15.5:50606) 2024-04-24T06:54:41.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.675+0000 7f56c3400640 1 -- 172.21.15.5:0/2537199469 learned_addr learned my addr 172.21.15.5:0/2537199469 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:41.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.675+0000 7f56c2a00640 1 --2- 172.21.15.5:0/2537199469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a227667460 0x55a2276715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.675+0000 7f56ba000640 1 --2- 172.21.15.5:0/2537199469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a227671b30 0x55a227673f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.675+0000 7f56c3400640 1 -- 172.21.15.5:0/2537199469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a227671b30 msgr2=0x55a227673f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:41.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.675+0000 7f56c3400640 1 --2- 172.21.15.5:0/2537199469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a227671b30 0x55a227673f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.675+0000 7f56c3400640 1 -- 172.21.15.5:0/2537199469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a227667460 msgr2=0x55a2276715f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:41.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.676+0000 7f56c3400640 1 --2- 172.21.15.5:0/2537199469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a227667460 0x55a2276715f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.676+0000 7f56c3400640 1 -- 172.21.15.5:0/2537199469 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a22762f310 con 0x55a227666b40 2024-04-24T06:54:41.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.676+0000 7f56c3400640 1 --2- 172.21.15.5:0/2537199469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a227666b40 0x55a227666f20 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f56bc009fd0 tx=0x7f56bc0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60cdf7cbe1586edd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:41.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.676+0000 7f56b3e00640 1 -- 172.21.15.5:0/2537199469 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc004d20 con 0x55a227666b40 2024-04-24T06:54:41.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.676+0000 7f56b3e00640 1 -- 172.21.15.5:0/2537199469 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56bc004050 con 0x55a227666b40 2024-04-24T06:54:41.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.676+0000 7f56b3e00640 1 -- 172.21.15.5:0/2537199469 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc005220 con 0x55a227666b40 2024-04-24T06:54:41.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.676+0000 7f56bbe00640 1 --2- 172.21.15.5:0/1989301627 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f56a80a3d20 0x7f56a80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.677+0000 7f56c2a00640 1 --2- 172.21.15.5:0/1989301627 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f56a80a3d20 0x7f56a80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.677+0000 7f56bbe00640 1 -- 172.21.15.5:0/1989301627 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f56b40d0310 con 0x7f56c4108540 2024-04-24T06:54:41.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.677+0000 7f56cca22740 1 -- 172.21.15.5:0/2537199469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a227666b40 msgr2=0x55a227666f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:41.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.677+0000 7f56cca22740 1 --2- 172.21.15.5:0/2537199469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a227666b40 0x55a227666f20 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f56bc009fd0 tx=0x7f56bc0118d0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.677+0000 7f56c2a00640 1 --2- 172.21.15.5:0/1989301627 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f56a80a3d20 0x7f56a80a61e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f56b40cdb00 tx=0x7f56b40d3e60 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:41.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.677+0000 7f56cca22740 1 -- 172.21.15.5:0/2537199469 shutdown_connections 2024-04-24T06:54:41.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.677+0000 7f56cca22740 1 --2- 172.21.15.5:0/2537199469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a227671b30 0x55a227673f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.677+0000 7f56cca22740 1 --2- 172.21.15.5:0/2537199469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a227667460 0x55a2276715f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.677+0000 7f56cca22740 1 --2- 172.21.15.5:0/2537199469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a227666b40 0x55a227666f20 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.677+0000 7f56cca22740 1 -- 172.21.15.5:0/2537199469 >> 172.21.15.5:0/2537199469 conn(0x55a227662e60 msgr2=0x55a227663240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:41.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.677+0000 7f56cca22740 1 -- 172.21.15.5:0/2537199469 shutdown_connections 2024-04-24T06:54:41.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.677+0000 7f56cca22740 1 -- 172.21.15.5:0/2537199469 wait complete. 2024-04-24T06:54:41.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.678+0000 7f56cca22740 1 -- start start 2024-04-24T06:54:41.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.678+0000 7f56cca22740 1 Processor -- start 2024-04-24T06:54:41.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.678+0000 7f56cca22740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a227666b40 0x55a2276715c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.678+0000 7f56cca22740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a227667460 0x55a22766e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.678+0000 7f56cca22740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a227671b30 0x55a22766efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.678+0000 7f56cca22740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a2276749c0 con 0x55a227667460 2024-04-24T06:54:41.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.678+0000 7f56cca22740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a227585810 con 0x55a227666b40 2024-04-24T06:54:41.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.678+0000 7f56cca22740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a227674b40 con 0x55a227671b30 2024-04-24T06:54:41.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.679+0000 7f56cca22740 10 client.0 _ll_register_callbacks cb 0x55a227553140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:54:41.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.679+0000 7f56ba000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a227667460 0x55a22766e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.679+0000 7f56ba000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a227667460 0x55a22766e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50608/0 (socket says 172.21.15.5:50608) 2024-04-24T06:54:41.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.679+0000 7f56ba000640 1 -- 172.21.15.5:0/1483393662 learned_addr learned my addr 172.21.15.5:0/1483393662 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:41.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.679+0000 7f56c3400640 1 --2- 172.21.15.5:0/1483393662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a227666b40 0x55a2276715c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.679+0000 7f56c3400640 1 --2- 172.21.15.5:0/1483393662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a227671b30 0x55a22766efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.679+0000 7f56ba000640 1 -- 172.21.15.5:0/1483393662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a227671b30 msgr2=0x55a22766efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:41.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.679+0000 7f56ba000640 1 --2- 172.21.15.5:0/1483393662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a227671b30 0x55a22766efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.679+0000 7f56ba000640 1 -- 172.21.15.5:0/1483393662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a227666b40 msgr2=0x55a2276715c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:41.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.679+0000 7f56ba000640 1 --2- 172.21.15.5:0/1483393662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a227666b40 0x55a2276715c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.679+0000 7f56ba000640 1 -- 172.21.15.5:0/1483393662 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2275963f0 con 0x55a227667460 2024-04-24T06:54:41.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.679+0000 7f56c3400640 1 --2- 172.21.15.5:0/1483393662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a227671b30 0x55a22766efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T06:54:41.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.679+0000 7f56ba000640 1 --2- 172.21.15.5:0/1483393662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a227667460 0x55a22766e8b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f56ac009930 tx=0x7f56ac004440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=407d2eac3312ca77 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:41.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.679+0000 7f56b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:54:41.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.680+0000 7f56b0c00640 1 -- 172.21.15.5:0/1483393662 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac035040 con 0x55a227667460 2024-04-24T06:54:41.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.680+0000 7f56b0c00640 1 -- 172.21.15.5:0/1483393662 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f56ac005450 con 0x55a227667460 2024-04-24T06:54:41.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.680+0000 7f56cca22740 1 -- 172.21.15.5:0/1483393662 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56bc004050 con 0x55a227667460 2024-04-24T06:54:41.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.680+0000 7f56b0c00640 1 -- 172.21.15.5:0/1483393662 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f56ac005650 con 0x55a227667460 2024-04-24T06:54:41.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.680+0000 7f56cca22740 10 client.17391 fetch_fsmap learned FSMap version 27 2024-04-24T06:54:41.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.680+0000 7f56cca22740 1 -- 172.21.15.5:0/1483393662 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a2272de3f0 con 0x55a227667460 2024-04-24T06:54:41.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.680+0000 7f56b0c00640 1 -- 172.21.15.5:0/1483393662 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56bc004050 con 0x55a227667460 2024-04-24T06:54:41.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.681+0000 7f56b0c00640 1 -- 172.21.15.5:0/1483393662 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f56ac003a10 con 0x55a227667460 2024-04-24T06:54:41.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.681+0000 7f56cca22740 10 client.17391 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:54:41.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.681+0000 7f56cca22740 10 client.17391 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:54:41.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.681+0000 7f56cca22740 20 client.17391 populate_metadata read hostname 'smithi005' 2024-04-24T06:54:41.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.681+0000 7f56cca22740 1 --2- 172.21.15.5:0/1483393662 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a227681d40 0x55a227684150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.681+0000 7f56cca22740 4 client.17391 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T06:54:41.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.681+0000 7f56cca22740 1 -- 172.21.15.5:0/1483393662 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2275d9bf0 con 0x55a227681d40 2024-04-24T06:54:41.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.681+0000 7f56c3400640 1 --2- 172.21.15.5:0/1483393662 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a227681d40 0x55a227684150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.682+0000 7f56c3400640 1 --2- 172.21.15.5:0/1483393662 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a227681d40 0x55a227684150 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e7cd8a532f5be74 server_cookie=c46477aeae4c3de1 in_seq=0 out_seq=0 2024-04-24T06:54:41.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.682+0000 7f56b0c00640 10 client.17391 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:54:41.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.682+0000 7f56b0c00640 1 -- 172.21.15.5:0/1483393662 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a2275d9bf0 con 0x55a227681d40 2024-04-24T06:54:41.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.682+0000 7f56b0c00640 10 client.17391 handle_command_reply: tid=0 2024-04-24T06:54:41.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.682+0000 7f56cca22740 1 client.17391 shutdown 2024-04-24T06:54:41.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.683+0000 7f56cca22740 1 -- 172.21.15.5:0/1483393662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a227667460 msgr2=0x55a22766e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:41.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.683+0000 7f56cca22740 1 --2- 172.21.15.5:0/1483393662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a227667460 0x55a22766e8b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f56ac009930 tx=0x7f56ac004440 comp rx=0 tx=0).stop 2024-04-24T06:54:41.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.683+0000 7f56cca22740 1 -- 172.21.15.5:0/1483393662 shutdown_connections 2024-04-24T06:54:41.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.683+0000 7f56cca22740 1 --2- 172.21.15.5:0/1483393662 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a227681d40 0x55a227684150 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.683+0000 7f56cca22740 1 --2- 172.21.15.5:0/1483393662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a227671b30 0x55a22766efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.683+0000 7f56cca22740 1 --2- 172.21.15.5:0/1483393662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a227667460 0x55a22766e8b0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.683+0000 7f56cca22740 1 --2- 172.21.15.5:0/1483393662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a227666b40 0x55a2276715c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.683+0000 7f56b0c00640 0 client.17391 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:54:41.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.683+0000 7f56cca22740 1 -- 172.21.15.5:0/1483393662 >> 172.21.15.5:0/1483393662 conn(0x55a227662e60 msgr2=0x55a227668c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:41.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.683+0000 7f56cca22740 1 -- 172.21.15.5:0/1483393662 shutdown_connections 2024-04-24T06:54:41.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.683+0000 7f56cca22740 1 -- 172.21.15.5:0/1483393662 wait complete. 2024-04-24T06:54:41.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.683+0000 7f56cca22740 20 client.17391 trim_cache size 0 max 16384 2024-04-24T06:54:41.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.698+0000 7f56cca22740 1 Processor -- start 2024-04-24T06:54:41.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.698+0000 7f56cca22740 1 -- start start 2024-04-24T06:54:41.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.698+0000 7f56cca22740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a227666b40 0x55a227661dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.698+0000 7f56cca22740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a227667460 0x55a2276710f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.698+0000 7f56cca22740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a227671b30 0x55a227689a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.698+0000 7f56cca22740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f56c41b6b10 con 0x55a227671b30 2024-04-24T06:54:41.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.698+0000 7f56cca22740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f56c41b43e0 con 0x55a227667460 2024-04-24T06:54:41.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.698+0000 7f56cca22740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f56c41b2250 con 0x55a227666b40 2024-04-24T06:54:41.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.699+0000 7f56ba000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a227667460 0x55a2276710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.699+0000 7f56c3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a227671b30 0x55a227689a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.699+0000 7f56ba000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a227667460 0x55a2276710f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:57676/0 (socket says 172.21.15.5:57676) 2024-04-24T06:54:41.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.699+0000 7f56ba000640 1 -- 172.21.15.5:0/2350874043 learned_addr learned my addr 172.21.15.5:0/2350874043 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:41.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.699+0000 7f56c3400640 1 --2- 172.21.15.5:0/2350874043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a227666b40 0x55a227661dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.699+0000 7f56ba000640 1 -- 172.21.15.5:0/2350874043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a227666b40 msgr2=0x55a227661dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:41.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.699+0000 7f56ba000640 1 --2- 172.21.15.5:0/2350874043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a227666b40 0x55a227661dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.699+0000 7f56ba000640 1 -- 172.21.15.5:0/2350874043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a227671b30 msgr2=0x55a227689a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:41.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.699+0000 7f56ba000640 1 --2- 172.21.15.5:0/2350874043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a227671b30 0x55a227689a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.699+0000 7f56ba000640 1 -- 172.21.15.5:0/2350874043 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2273a87d0 con 0x55a227667460 2024-04-24T06:54:41.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.700+0000 7f56ba000640 1 --2- 172.21.15.5:0/2350874043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a227667460 0x55a2276710f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f56ac02a9f0 tx=0x7f56ac02a4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29bf7a3d0ce73ac1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:41.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.703+0000 7f56b3400640 1 -- 172.21.15.5:0/2350874043 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac000e60 con 0x55a227667460 2024-04-24T06:54:41.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.703+0000 7f56b3400640 1 -- 172.21.15.5:0/2350874043 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56ac041070 con 0x55a227667460 2024-04-24T06:54:41.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.703+0000 7f56b3400640 1 -- 172.21.15.5:0/2350874043 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac035070 con 0x55a227667460 2024-04-24T06:54:41.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.704+0000 7f56cca22740 1 -- 172.21.15.5:0/2350874043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a227667460 msgr2=0x55a2276710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:41.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.704+0000 7f56cca22740 1 --2- 172.21.15.5:0/2350874043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a227667460 0x55a2276710f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f56ac02a9f0 tx=0x7f56ac02a4d0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.704+0000 7f56cca22740 1 -- 172.21.15.5:0/2350874043 shutdown_connections 2024-04-24T06:54:41.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.704+0000 7f56cca22740 1 --2- 172.21.15.5:0/2350874043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a227671b30 0x55a227689a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.704+0000 7f56cca22740 1 --2- 172.21.15.5:0/2350874043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a227667460 0x55a2276710f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.704+0000 7f56cca22740 1 --2- 172.21.15.5:0/2350874043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a227666b40 0x55a227661dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.704+0000 7f56cca22740 1 -- 172.21.15.5:0/2350874043 >> 172.21.15.5:0/2350874043 conn(0x55a227662e60 msgr2=0x55a2276616c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:41.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.704+0000 7f56cca22740 1 -- 172.21.15.5:0/2350874043 shutdown_connections 2024-04-24T06:54:41.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.704+0000 7f56cca22740 1 -- 172.21.15.5:0/2350874043 wait complete. 2024-04-24T06:54:41.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.704+0000 7f56cca22740 1 -- start start 2024-04-24T06:54:41.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.704+0000 7f56cca22740 1 Processor -- start 2024-04-24T06:54:41.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.705+0000 7f56cca22740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a227666b40 0x55a227679d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.705+0000 7f56cca22740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a227667460 0x55a22767a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.705+0000 7f56cca22740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a227671b30 0x55a227672e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.705+0000 7f56cca22740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f56c41b28a0 con 0x55a227667460 2024-04-24T06:54:41.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.705+0000 7f56cca22740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f56c41b3d10 con 0x55a227666b40 2024-04-24T06:54:41.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.705+0000 7f56cca22740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f56c41b2f70 con 0x55a227671b30 2024-04-24T06:54:41.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.705+0000 7f56cca22740 10 client.0 _ll_register_callbacks cb 0x7f56ac009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T06:54:41.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.705+0000 7f56ba000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a227667460 0x55a22767a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.705+0000 7f56c3400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a227666b40 0x55a227679d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.705+0000 7f56ba000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a227667460 0x55a22767a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50624/0 (socket says 172.21.15.5:50624) 2024-04-24T06:54:41.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.705+0000 7f56ba000640 1 -- 172.21.15.5:0/836008641 learned_addr learned my addr 172.21.15.5:0/836008641 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:41.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.705+0000 7f56c3400640 1 --2- 172.21.15.5:0/836008641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a227671b30 0x55a227672e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.706+0000 7f56ba000640 1 -- 172.21.15.5:0/836008641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a227671b30 msgr2=0x55a227672e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:41.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.706+0000 7f56ba000640 1 --2- 172.21.15.5:0/836008641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a227671b30 0x55a227672e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.706+0000 7f56ba000640 1 -- 172.21.15.5:0/836008641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a227666b40 msgr2=0x55a227679d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T06:54:41.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.706+0000 7f56ba000640 1 --2- 172.21.15.5:0/836008641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a227666b40 0x55a227679d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.706+0000 7f56ba000640 1 -- 172.21.15.5:0/836008641 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a227673400 con 0x55a227667460 2024-04-24T06:54:41.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.706+0000 7f56c3400640 1 --2- 172.21.15.5:0/836008641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a227666b40 0x55a227679d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:54:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.706+0000 7f56ba000640 1 --2- 172.21.15.5:0/836008641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a227667460 0x55a22767a2d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f56ac000c00 tx=0x7f56ac003ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a486b81a7f2f631 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.706+0000 7f56b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-24T06:54:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.706+0000 7f56b0c00640 1 -- 172.21.15.5:0/836008641 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac035070 con 0x55a227667460 2024-04-24T06:54:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.706+0000 7f56cca22740 1 -- 172.21.15.5:0/836008641 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56ac041070 con 0x55a227667460 2024-04-24T06:54:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.706+0000 7f56b0c00640 1 -- 172.21.15.5:0/836008641 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f56ac031d20 con 0x55a227667460 2024-04-24T06:54:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.707+0000 7f56b0c00640 1 -- 172.21.15.5:0/836008641 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f56ac065050 con 0x55a227667460 2024-04-24T06:54:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.707+0000 7f56cca22740 10 client.17397 fetch_fsmap learned FSMap version 27 2024-04-24T06:54:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.707+0000 7f56cca22740 1 -- 172.21.15.5:0/836008641 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a2276e8b10 con 0x55a227667460 2024-04-24T06:54:41.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.707+0000 7f56b0c00640 1 -- 172.21.15.5:0/836008641 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56ac041070 con 0x55a227667460 2024-04-24T06:54:41.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.707+0000 7f56b0c00640 1 -- 172.21.15.5:0/836008641 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f56ac046470 con 0x55a227667460 2024-04-24T06:54:41.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.707+0000 7f56cca22740 10 client.17397 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T06:54:41.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.707+0000 7f56cca22740 10 client.17397 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T06:54:41.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.707+0000 7f56cca22740 20 client.17397 populate_metadata read hostname 'smithi005' 2024-04-24T06:54:41.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.707+0000 7f56cca22740 1 --2- 172.21.15.5:0/836008641 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a2276e9140 0x55a2276eb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:41.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.707+0000 7f56cca22740 4 client.17397 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T06:54:41.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.707+0000 7f56cca22740 1 -- 172.21.15.5:0/836008641 --> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a2275d3a60 con 0x55a2276e9140 2024-04-24T06:54:41.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.708+0000 7f56c3400640 1 --2- 172.21.15.5:0/836008641 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a2276e9140 0x55a2276eb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:41.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.708+0000 7f56c3400640 1 --2- 172.21.15.5:0/836008641 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a2276e9140 0x55a2276eb560 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f12c1abb4887840 server_cookie=2f4bbbc4ecfac9c2 in_seq=0 out_seq=0 2024-04-24T06:54:41.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.708+0000 7f56b0c00640 10 client.17397 ms_handle_connect on v2:172.21.15.37:6838/3536721753 2024-04-24T06:54:41.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.708+0000 7f56b0c00640 1 -- 172.21.15.5:0/836008641 <== mds.0 v2:172.21.15.37:6838/3536721753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a2275d3a60 con 0x55a2276e9140 2024-04-24T06:54:41.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.708+0000 7f56b0c00640 10 client.17397 handle_command_reply: tid=0 2024-04-24T06:54:41.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.708+0000 7f56cca22740 1 client.17397 shutdown 2024-04-24T06:54:41.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.708+0000 7f56cca22740 1 -- 172.21.15.5:0/836008641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a227667460 msgr2=0x55a22767a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:41.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.708+0000 7f56cca22740 1 --2- 172.21.15.5:0/836008641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a227667460 0x55a22767a2d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f56ac000c00 tx=0x7f56ac003ad0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.708+0000 7f56cca22740 1 -- 172.21.15.5:0/836008641 shutdown_connections 2024-04-24T06:54:41.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.708+0000 7f56cca22740 1 --2- 172.21.15.5:0/836008641 >> [v2:172.21.15.37:6838/3536721753,v1:172.21.15.37:6839/3536721753] conn(0x55a2276e9140 0x55a2276eb560 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.709+0000 7f56cca22740 1 --2- 172.21.15.5:0/836008641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a227671b30 0x55a227672e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.709+0000 7f56cca22740 1 --2- 172.21.15.5:0/836008641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a227667460 0x55a22767a2d0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.709+0000 7f56b0c00640 0 client.17397 ms_handle_reset on v2:172.21.15.37:6838/3536721753 2024-04-24T06:54:41.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.709+0000 7f56cca22740 1 --2- 172.21.15.5:0/836008641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a227666b40 0x55a227679d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.709+0000 7f56cca22740 1 -- 172.21.15.5:0/836008641 >> 172.21.15.5:0/836008641 conn(0x55a227662e60 msgr2=0x55a227661f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:41.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.709+0000 7f56cca22740 1 -- 172.21.15.5:0/836008641 shutdown_connections 2024-04-24T06:54:41.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.709+0000 7f56cca22740 1 -- 172.21.15.5:0/836008641 wait complete. 2024-04-24T06:54:41.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.709+0000 7f56cca22740 20 client.17397 trim_cache size 0 max 16384 2024-04-24T06:54:41.716 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-24T06:54:41.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.709+0000 7f56c3e00640 1 -- 172.21.15.5:0/1989301627 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f56a80a3d20 msgr2=0x7f56a80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:41.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.709+0000 7f56c3e00640 1 --2- 172.21.15.5:0/1989301627 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f56a80a3d20 0x7f56a80a61e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f56b40cdb00 tx=0x7f56b40d3e60 comp rx=0 tx=0).stop 2024-04-24T06:54:41.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.709+0000 7f56c3e00640 1 -- 172.21.15.5:0/1989301627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56c4108540 msgr2=0x7f56c419edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:41.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.709+0000 7f56c3e00640 1 --2- 172.21.15.5:0/1989301627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56c4108540 0x7f56c419edd0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f56b400d950 tx=0x7f56b400de10 comp rx=0 tx=0).stop 2024-04-24T06:54:41.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.710+0000 7f56c3e00640 1 -- 172.21.15.5:0/1989301627 shutdown_connections 2024-04-24T06:54:41.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.710+0000 7f56c3e00640 1 --2- 172.21.15.5:0/1989301627 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f56a80a3d20 0x7f56a80a61e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.710+0000 7f56c3e00640 1 --2- 172.21.15.5:0/1989301627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56c410e4f0 0x7f56c41a3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.710+0000 7f56c3e00640 1 --2- 172.21.15.5:0/1989301627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56c4108ee0 0x7f56c419f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.710+0000 7f56c3e00640 1 --2- 172.21.15.5:0/1989301627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56c4108540 0x7f56c419edd0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:41.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.710+0000 7f56c3e00640 1 -- 172.21.15.5:0/1989301627 >> 172.21.15.5:0/1989301627 conn(0x7f56c4077e10 msgr2=0x7f56c40784d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:41.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.710+0000 7f56c3e00640 1 -- 172.21.15.5:0/1989301627 shutdown_connections 2024-04-24T06:54:41.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:41.710+0000 7f56c3e00640 1 -- 172.21.15.5:0/1989301627 wait complete. 2024-04-24T06:54:41.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:41 smithi005 ceph-mon[27798]: pgmap v330: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:54:41.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:41 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/2542534160' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:54:42.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:41 smithi037 ceph-mon[32864]: pgmap v330: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:54:42.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:41 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/2542534160' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:54:42.195 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:41 smithi137 ceph-mon[33017]: pgmap v330: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:54:42.195 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:41 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/2542534160' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:54:42.728 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-04-24T06:54:42.729 INFO:tasks.fwd_scrub:done joining 2024-04-24T06:54:42.729 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-24T06:54:42.740 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:54:42.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:42.967+0000 7f4313e00640 1 -- 172.21.15.5:0/594182775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4314076e50 msgr2=0x7f4314112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:42.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:42.967+0000 7f4313e00640 1 --2- 172.21.15.5:0/594182775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4314076e50 0x7f4314112840 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4300009930 tx=0x7f430002a440 comp rx=0 tx=0).stop 2024-04-24T06:54:42.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:42.967+0000 7f4313e00640 1 -- 172.21.15.5:0/594182775 shutdown_connections 2024-04-24T06:54:42.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:42.967+0000 7f4313e00640 1 --2- 172.21.15.5:0/594182775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4314112d80 0x7f4314115170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:42.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:42.967+0000 7f4313e00640 1 --2- 172.21.15.5:0/594182775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4314076e50 0x7f4314112840 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:42.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:42.967+0000 7f4313e00640 1 --2- 172.21.15.5:0/594182775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4314076530 0x7f4314076910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:42.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:42.967+0000 7f4313e00640 1 -- 172.21.15.5:0/594182775 >> 172.21.15.5:0/594182775 conn(0x7f43140ffb70 msgr2=0x7f4314101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:42.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:42.967+0000 7f4313e00640 1 -- 172.21.15.5:0/594182775 shutdown_connections 2024-04-24T06:54:42.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:42.967+0000 7f4313e00640 1 -- 172.21.15.5:0/594182775 wait complete. 2024-04-24T06:54:42.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:42.968+0000 7f4313e00640 1 Processor -- start 2024-04-24T06:54:42.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:42.969+0000 7f4313e00640 1 -- start start 2024-04-24T06:54:42.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:42.969+0000 7f4313e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4314076530 0x7f431419ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:42.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:42.969+0000 7f4313e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4314076e50 0x7f431419f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:42.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:42.969+0000 7f4313e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4314112d80 0x7f43141a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:42.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:42.969+0000 7f4313e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4314118930 con 0x7f4314076e50 2024-04-24T06:54:42.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:42.969+0000 7f4313e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f43141187b0 con 0x7f4314112d80 2024-04-24T06:54:42.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:42.969+0000 7f4313e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4314118ab0 con 0x7f4314076530 2024-04-24T06:54:42.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:42.969+0000 7f4312000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4314076e50 0x7f431419f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:42.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:42.969+0000 7f4312a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4314076530 0x7f431419ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:42.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:42.969+0000 7f4312000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4314076e50 0x7f431419f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50642/0 (socket says 172.21.15.5:50642) 2024-04-24T06:54:42.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:42.969+0000 7f4312000640 1 -- 172.21.15.5:0/3512001828 learned_addr learned my addr 172.21.15.5:0/3512001828 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:42.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:42.969+0000 7f4313400640 1 --2- 172.21.15.5:0/3512001828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4314112d80 0x7f43141a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:42.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:42.970+0000 7f4312a00640 1 -- 172.21.15.5:0/3512001828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4314112d80 msgr2=0x7f43141a36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:42.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:42.970+0000 7f4312a00640 1 --2- 172.21.15.5:0/3512001828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4314112d80 0x7f43141a36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:42.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:42.970+0000 7f4312a00640 1 -- 172.21.15.5:0/3512001828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4314076e50 msgr2=0x7f431419f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:42.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:42.970+0000 7f4312a00640 1 --2- 172.21.15.5:0/3512001828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4314076e50 0x7f431419f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:42.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:42.970+0000 7f4312a00640 1 -- 172.21.15.5:0/3512001828 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43141a3e20 con 0x7f4314076530 2024-04-24T06:54:42.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:42.970+0000 7f4312a00640 1 --2- 172.21.15.5:0/3512001828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4314076530 0x7f431419ee30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f430400e9f0 tx=0x7f430400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:42.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:42.970+0000 7f4318a00640 1 -- 172.21.15.5:0/3512001828 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430400ccc0 con 0x7f4314076530 2024-04-24T06:54:42.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:42.970+0000 7f4318a00640 1 -- 172.21.15.5:0/3512001828 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4304010910 con 0x7f4314076530 2024-04-24T06:54:42.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:42.970+0000 7f4318a00640 1 -- 172.21.15.5:0/3512001828 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4304018960 con 0x7f4314076530 2024-04-24T06:54:42.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:42.970+0000 7f4313e00640 1 -- 172.21.15.5:0/3512001828 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43141a4110 con 0x7f4314076530 2024-04-24T06:54:42.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:42.970+0000 7f4313e00640 1 -- 172.21.15.5:0/3512001828 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43141ab9a0 con 0x7f4314076530 2024-04-24T06:54:42.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:42.971+0000 7f4313e00640 1 -- 172.21.15.5:0/3512001828 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f431404fe00 con 0x7f4314076530 2024-04-24T06:54:42.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:42.974+0000 7f4318a00640 1 -- 172.21.15.5:0/3512001828 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f4304010ac0 con 0x7f4314076530 2024-04-24T06:54:42.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:42.975+0000 7f4318a00640 1 --2- 172.21.15.5:0/3512001828 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f42f80a3d10 0x7f42f80a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:42.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:42.975+0000 7f4318a00640 1 -- 172.21.15.5:0/3512001828 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f43040d0ba0 con 0x7f4314076530 2024-04-24T06:54:42.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:42.975+0000 7f4312000640 1 --2- 172.21.15.5:0/3512001828 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f42f80a3d10 0x7f42f80a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:42.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:42.976+0000 7f4312000640 1 --2- 172.21.15.5:0/3512001828 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f42f80a3d10 0x7f42f80a61d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4300005c40 tx=0x7f4300005b50 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:42.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:42.979+0000 7f4318a00640 1 -- 172.21.15.5:0/3512001828 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f430409cb80 con 0x7f4314076530 2024-04-24T06:54:43.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.190+0000 7f4313e00640 1 -- 172.21.15.5:0/3512001828 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f431410df10 con 0x7f4314076530 2024-04-24T06:54:43.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.192+0000 7f4318a00640 1 -- 172.21.15.5:0/3512001828 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+22974 (secure 0 0 0) 0x7f43040a0e80 con 0x7f4314076530 2024-04-24T06:54:43.194 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:54:43.194 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":97,"fsid":"43e9bad0-0204-11ef-bc93-c7b262605968","created":"2024-04-24T06:32:26.340555+0000","modified":"2024-04-24T06:54:16.077627+0000","last_up_change":"2024-04-24T06:40:36.790934+0000","last_in_change":"2024-04-24T06:40:17.803257+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:36:27.320342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:41:32.254502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:41:33.739067+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":96,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c2e974c-745c-4dd5-8c3d-8e8da2eb1ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6803","nonce":2155858515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6805","nonce":2155858515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6809","nonce":2155858515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6807","nonce":2155858515}]},"public_addr":"172.21.15.5:6803/2155858515","cluster_addr":"172.21.15.5:6805/2155858515","heartbeat_back_addr":"172.21.15.5:6809/2155858515","heartbeat_front_addr":"172.21.15.5:6807/2155858515","state":["exists","up"]},{"osd":1,"uuid":"50467744-db20-4721-bf2d-1c5db938297b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6801","nonce":1302362650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6803","nonce":1302362650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6807","nonce":1302362650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6805","nonce":1302362650}]},"public_addr":"172.21.15.37:6801/1302362650","cluster_addr":"172.21.15.37:6803/1302362650","heartbeat_back_addr":"172.21.15.37:6807/1302362650","heartbeat_front_addr":"172.21.15.37:6805/1302362650","state":["exists","up"]},{"osd":2,"uuid":"0e190b15-d04c-4288-8d48-ecf6664392fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6801","nonce":2885027256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6803","nonce":2885027256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6807","nonce":2885027256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6805","nonce":2885027256}]},"public_addr":"172.21.15.137:6801/2885027256","cluster_addr":"172.21.15.137:6803/2885027256","heartbeat_back_addr":"172.21.15.137:6807/2885027256","heartbeat_front_addr":"172.21.15.137:6805/2885027256","state":["exists","up"]},{"osd":3,"uuid":"e05592ba-d96a-472a-9f63-09a535d04dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6811","nonce":3701747260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6813","nonce":3701747260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6817","nonce":3701747260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6815","nonce":3701747260}]},"public_addr":"172.21.15.5:6811/3701747260","cluster_addr":"172.21.15.5:6813/3701747260","heartbeat_back_addr":"172.21.15.5:6817/3701747260","heartbeat_front_addr":"172.21.15.5:6815/3701747260","state":["exists","up"]},{"osd":4,"uuid":"c3f36e51-7a3b-4dd9-af4b-a10558830c4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6809","nonce":759253844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6811","nonce":759253844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6815","nonce":759253844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6813","nonce":759253844}]},"public_addr":"172.21.15.37:6809/759253844","cluster_addr":"172.21.15.37:6811/759253844","heartbeat_back_addr":"172.21.15.37:6815/759253844","heartbeat_front_addr":"172.21.15.37:6813/759253844","state":["exists","up"]},{"osd":5,"uuid":"97a0500c-cbb8-4ff5-b476-8634b26b3821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6809","nonce":3771921560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6811","nonce":3771921560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6815","nonce":3771921560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6813","nonce":3771921560}]},"public_addr":"172.21.15.137:6809/3771921560","cluster_addr":"172.21.15.137:6811/3771921560","heartbeat_back_addr":"172.21.15.137:6815/3771921560","heartbeat_front_addr":"172.21.15.137:6813/3771921560","state":["exists","up"]},{"osd":6,"uuid":"33a33d62-bbb3-4ed9-834e-19ae93a97670","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6819","nonce":2064636084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6821","nonce":2064636084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6825","nonce":2064636084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6823","nonce":2064636084}]},"public_addr":"172.21.15.5:6819/2064636084","cluster_addr":"172.21.15.5:6821/2064636084","heartbeat_back_addr":"172.21.15.5:6825/2064636084","heartbeat_front_addr":"172.21.15.5:6823/2064636084","state":["exists","up"]},{"osd":7,"uuid":"c0f4e036-4717-4514-bb59-ab11fbaab063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6817","nonce":2952750659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6819","nonce":2952750659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6823","nonce":2952750659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6821","nonce":2952750659}]},"public_addr":"172.21.15.37:6817/2952750659","cluster_addr":"172.21.15.37:6819/2952750659","heartbeat_back_addr":"172.21.15.37:6823/2952750659","heartbeat_front_addr":"172.21.15.37:6821/2952750659","state":["exists","up"]},{"osd":8,"uuid":"5391fd87-44ea-440f-80da-c54d0327c04c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6817","nonce":29724945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6819","nonce":29724945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6823","nonce":29724945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6821","nonce":29724945}]},"public_addr":"172.21.15.137:6817/29724945","cluster_addr":"172.21.15.137:6819/29724945","heartbeat_back_addr":"172.21.15.137:6823/29724945","heartbeat_front_addr":"172.21.15.137:6821/29724945","state":["exists","up"]},{"osd":9,"uuid":"d94c9dbc-f43d-4237-ac7f-47a6a5ede1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6827","nonce":2257102107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6829","nonce":2257102107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6833","nonce":2257102107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6831","nonce":2257102107}]},"public_addr":"172.21.15.5:6827/2257102107","cluster_addr":"172.21.15.5:6829/2257102107","heartbeat_back_addr":"172.21.15.5:6833/2257102107","heartbeat_front_addr":"172.21.15.5:6831/2257102107","state":["exists","up"]},{"osd":10,"uuid":"e2455332-2a5e-44b1-85d3-8e47da399f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6825","nonce":2451570637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6827","nonce":2451570637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6831","nonce":2451570637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6829","nonce":2451570637}]},"public_addr":"172.21.15.37:6825/2451570637","cluster_addr":"172.21.15.37:6827/2451570637","heartbeat_back_addr":"172.21.15.37:6831/2451570637","heartbeat_front_addr":"172.21.15.37:6829/2451570637","state":["exists","up"]},{"osd":11,"uuid":"49d9396b-704a-4cef-8210-75d30c28e924","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6825","nonce":90600596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6827","nonce":90600596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6831","nonce":90600596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6829","nonce":90600596}]},"public_addr":"172.21.15.137:6825/90600596","cluster_addr":"172.21.15.137:6827/90600596","heartbeat_back_addr":"172.21.15.137:6831/90600596","heartbeat_front_addr":"172.21.15.137:6829/90600596","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:35:35.591619+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:00.568817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:24.195643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:56.451724+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:22.109293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:46.161749+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:19.278305+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:45.620256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:10.041165+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:43.813697+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:10.736026+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:35.384377+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6800/4042936965":"2024-04-25T06:44:01.965881+0000","172.21.15.5:0/2912680052":"2024-04-25T06:44:01.965881+0000","172.21.15.5:6801/4042936965":"2024-04-25T06:44:01.965881+0000","172.21.15.5:0/3478347847":"2024-04-25T06:44:01.965881+0000","172.21.15.5:0/23307164":"2024-04-25T06:33:41.161852+0000","172.21.15.5:0/1012004002":"2024-04-25T06:44:01.965881+0000","172.21.15.5:6801/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:0/230785941":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/805658012":"2024-04-25T06:44:01.965881+0000","172.21.15.5:0/2940023597":"2024-04-25T06:33:07.040369+0000","172.21.15.5:6801/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3886613904":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/4199165437":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3459593499":"2024-04-25T06:33:41.161852+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:54:43.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.196+0000 7f4313e00640 1 -- 172.21.15.5:0/3512001828 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f42f80a3d10 msgr2=0x7f42f80a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:43.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.196+0000 7f4313e00640 1 --2- 172.21.15.5:0/3512001828 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f42f80a3d10 0x7f42f80a61d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4300005c40 tx=0x7f4300005b50 comp rx=0 tx=0).stop 2024-04-24T06:54:43.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.196+0000 7f4313e00640 1 -- 172.21.15.5:0/3512001828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4314076530 msgr2=0x7f431419ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:43.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.196+0000 7f4313e00640 1 --2- 172.21.15.5:0/3512001828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4314076530 0x7f431419ee30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f430400e9f0 tx=0x7f430400eeb0 comp rx=0 tx=0).stop 2024-04-24T06:54:43.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.196+0000 7f4313e00640 1 -- 172.21.15.5:0/3512001828 shutdown_connections 2024-04-24T06:54:43.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.196+0000 7f4313e00640 1 --2- 172.21.15.5:0/3512001828 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f42f80a3d10 0x7f42f80a61d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:43.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.196+0000 7f4313e00640 1 --2- 172.21.15.5:0/3512001828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4314112d80 0x7f43141a36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:43.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.196+0000 7f4313e00640 1 --2- 172.21.15.5:0/3512001828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4314076e50 0x7f431419f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:43.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.196+0000 7f4313e00640 1 --2- 172.21.15.5:0/3512001828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4314076530 0x7f431419ee30 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:43.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.196+0000 7f4313e00640 1 -- 172.21.15.5:0/3512001828 >> 172.21.15.5:0/3512001828 conn(0x7f43140ffb70 msgr2=0x7f43141143a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:43.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.196+0000 7f4313e00640 1 -- 172.21.15.5:0/3512001828 shutdown_connections 2024-04-24T06:54:43.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.196+0000 7f4313e00640 1 -- 172.21.15.5:0/3512001828 wait complete. 2024-04-24T06:54:43.217 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:36:27.320342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:41:32.254502+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:41:33.739067+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 96, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:54:43.218 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:54:43.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.431+0000 7f31e3200640 1 -- 172.21.15.5:0/4072826420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31e4103900 msgr2=0x7f31e410bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:43.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.431+0000 7f31e3200640 1 --2- 172.21.15.5:0/4072826420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31e4103900 0x7f31e410bde0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f31d0009ca0 tx=0x7f31d002a950 comp rx=0 tx=0).stop 2024-04-24T06:54:43.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.431+0000 7f31e3200640 1 -- 172.21.15.5:0/4072826420 shutdown_connections 2024-04-24T06:54:43.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.431+0000 7f31e3200640 1 --2- 172.21.15.5:0/4072826420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f31e410c400 0x7f31e410e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:43.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.431+0000 7f31e3200640 1 --2- 172.21.15.5:0/4072826420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31e4103900 0x7f31e410bde0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:43.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.431+0000 7f31e3200640 1 --2- 172.21.15.5:0/4072826420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31e4102fe0 0x7f31e41033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:43.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.431+0000 7f31e3200640 1 -- 172.21.15.5:0/4072826420 >> 172.21.15.5:0/4072826420 conn(0x7f31e4077ab0 msgr2=0x7f31e40fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:43.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.431+0000 7f31e3200640 1 -- 172.21.15.5:0/4072826420 shutdown_connections 2024-04-24T06:54:43.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.431+0000 7f31e3200640 1 -- 172.21.15.5:0/4072826420 wait complete. 2024-04-24T06:54:43.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.433+0000 7f31e3200640 1 Processor -- start 2024-04-24T06:54:43.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.433+0000 7f31e3200640 1 -- start start 2024-04-24T06:54:43.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.433+0000 7f31e3200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f31e4102fe0 0x7f31e4198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:43.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.433+0000 7f31e3200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31e4103900 0x7f31e4198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:43.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.433+0000 7f31e3200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31e410c400 0x7f31e419d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:43.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.433+0000 7f31e3200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f31e41121b0 con 0x7f31e410c400 2024-04-24T06:54:43.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.433+0000 7f31e3200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f31e4112030 con 0x7f31e4103900 2024-04-24T06:54:43.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.433+0000 7f31e3200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f31e4112330 con 0x7f31e4102fe0 2024-04-24T06:54:43.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.434+0000 7f31e2800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31e410c400 0x7f31e419d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:43.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.434+0000 7f31e2800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31e410c400 0x7f31e419d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50668/0 (socket says 172.21.15.5:50668) 2024-04-24T06:54:43.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.434+0000 7f31e2800640 1 -- 172.21.15.5:0/132559352 learned_addr learned my addr 172.21.15.5:0/132559352 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:43.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.434+0000 7f31e1e00640 1 --2- 172.21.15.5:0/132559352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f31e4102fe0 0x7f31e4198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:43.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.434+0000 7f31e1400640 1 --2- 172.21.15.5:0/132559352 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31e4103900 0x7f31e4198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:43.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.434+0000 7f31e2800640 1 -- 172.21.15.5:0/132559352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f31e4102fe0 msgr2=0x7f31e4198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:43.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.434+0000 7f31e2800640 1 --2- 172.21.15.5:0/132559352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f31e4102fe0 0x7f31e4198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:43.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.434+0000 7f31e2800640 1 -- 172.21.15.5:0/132559352 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31e4103900 msgr2=0x7f31e4198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:43.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.434+0000 7f31e2800640 1 --2- 172.21.15.5:0/132559352 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31e4103900 0x7f31e4198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:43.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.434+0000 7f31e2800640 1 -- 172.21.15.5:0/132559352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e419d740 con 0x7f31e410c400 2024-04-24T06:54:43.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.435+0000 7f31e2800640 1 --2- 172.21.15.5:0/132559352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31e410c400 0x7f31e419d010 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f31dc00e970 tx=0x7f31dc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:43.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.435+0000 7f31daa00640 1 -- 172.21.15.5:0/132559352 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31dc00ccc0 con 0x7f31e410c400 2024-04-24T06:54:43.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.435+0000 7f31e3200640 1 -- 172.21.15.5:0/132559352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31e419da30 con 0x7f31e410c400 2024-04-24T06:54:43.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.435+0000 7f31daa00640 1 -- 172.21.15.5:0/132559352 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31dc00ce60 con 0x7f31e410c400 2024-04-24T06:54:43.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.435+0000 7f31daa00640 1 -- 172.21.15.5:0/132559352 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31dc010450 con 0x7f31e410c400 2024-04-24T06:54:43.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.435+0000 7f31e3200640 1 -- 172.21.15.5:0/132559352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31e41a52c0 con 0x7f31e410c400 2024-04-24T06:54:43.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.436+0000 7f31e3200640 1 -- 172.21.15.5:0/132559352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31e404fd90 con 0x7f31e410c400 2024-04-24T06:54:43.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.438+0000 7f31daa00640 1 -- 172.21.15.5:0/132559352 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f31dc005020 con 0x7f31e410c400 2024-04-24T06:54:43.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.440+0000 7f31daa00640 1 --2- 172.21.15.5:0/132559352 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f31c80a3da0 0x7f31c80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:43.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.440+0000 7f31daa00640 1 -- 172.21.15.5:0/132559352 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f31dc0d0b80 con 0x7f31e410c400 2024-04-24T06:54:43.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.440+0000 7f31e1e00640 1 --2- 172.21.15.5:0/132559352 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f31c80a3da0 0x7f31c80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:43.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.441+0000 7f31e1e00640 1 --2- 172.21.15.5:0/132559352 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f31c80a3da0 0x7f31c80a6260 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f31d40095f0 tx=0x7f31d4005f90 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:43.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.445+0000 7f31daa00640 1 -- 172.21.15.5:0/132559352 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f31dc09d1b0 con 0x7f31e410c400 2024-04-24T06:54:43.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.648+0000 7f31e3200640 1 -- 172.21.15.5:0/132559352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f31e402d620 con 0x7f31e410c400 2024-04-24T06:54:43.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.649+0000 7f31daa00640 1 -- 172.21.15.5:0/132559352 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7f31dc09cab0 con 0x7f31e410c400 2024-04-24T06:54:43.649 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-24T06:54:43.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.651+0000 7f31e3200640 1 -- 172.21.15.5:0/132559352 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f31c80a3da0 msgr2=0x7f31c80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:43.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.651+0000 7f31e3200640 1 --2- 172.21.15.5:0/132559352 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f31c80a3da0 0x7f31c80a6260 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f31d40095f0 tx=0x7f31d4005f90 comp rx=0 tx=0).stop 2024-04-24T06:54:43.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.651+0000 7f31e3200640 1 -- 172.21.15.5:0/132559352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31e410c400 msgr2=0x7f31e419d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:43.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.651+0000 7f31e3200640 1 --2- 172.21.15.5:0/132559352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31e410c400 0x7f31e419d010 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f31dc00e970 tx=0x7f31dc00ed70 comp rx=0 tx=0).stop 2024-04-24T06:54:43.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.651+0000 7f31e3200640 1 -- 172.21.15.5:0/132559352 shutdown_connections 2024-04-24T06:54:43.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.651+0000 7f31e3200640 1 --2- 172.21.15.5:0/132559352 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f31c80a3da0 0x7f31c80a6260 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:43.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.651+0000 7f31e3200640 1 --2- 172.21.15.5:0/132559352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31e410c400 0x7f31e419d010 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:43.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.651+0000 7f31e3200640 1 --2- 172.21.15.5:0/132559352 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31e4103900 0x7f31e4198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:43.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.651+0000 7f31e3200640 1 --2- 172.21.15.5:0/132559352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f31e4102fe0 0x7f31e4198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:43.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.651+0000 7f31e3200640 1 -- 172.21.15.5:0/132559352 >> 172.21.15.5:0/132559352 conn(0x7f31e4077ab0 msgr2=0x7f31e40fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:43.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.652+0000 7f31e3200640 1 -- 172.21.15.5:0/132559352 shutdown_connections 2024-04-24T06:54:43.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.652+0000 7f31e3200640 1 -- 172.21.15.5:0/132559352 wait complete. 2024-04-24T06:54:43.672 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:54:43.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.887+0000 7fc96f600640 1 -- 172.21.15.5:0/1523921781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc968100df0 msgr2=0x7fc9681011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:43.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.887+0000 7fc96f600640 1 --2- 172.21.15.5:0/1523921781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc968100df0 0x7fc9681011d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc95c009900 tx=0x7fc95c02a400 comp rx=0 tx=0).stop 2024-04-24T06:54:43.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.887+0000 7fc96f600640 1 -- 172.21.15.5:0/1523921781 shutdown_connections 2024-04-24T06:54:43.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.887+0000 7fc96f600640 1 --2- 172.21.15.5:0/1523921781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc96810e9a0 0x7fc968110d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:43.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.887+0000 7fc96f600640 1 --2- 172.21.15.5:0/1523921781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc968101710 0x7fc96810e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:43.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.887+0000 7fc96f600640 1 --2- 172.21.15.5:0/1523921781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc968100df0 0x7fc9681011d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:43.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.887+0000 7fc96f600640 1 -- 172.21.15.5:0/1523921781 >> 172.21.15.5:0/1523921781 conn(0x7fc9680780b0 msgr2=0x7fc9680784c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:43.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.887+0000 7fc96f600640 1 -- 172.21.15.5:0/1523921781 shutdown_connections 2024-04-24T06:54:43.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.888+0000 7fc96f600640 1 -- 172.21.15.5:0/1523921781 wait complete. 2024-04-24T06:54:43.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.888+0000 7fc96f600640 1 Processor -- start 2024-04-24T06:54:43.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.889+0000 7fc96f600640 1 -- start start 2024-04-24T06:54:43.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.889+0000 7fc96f600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc968100df0 0x7fc9681a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:43.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.889+0000 7fc96f600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc968101710 0x7fc9681a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:43.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.889+0000 7fc96f600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc96810e9a0 0x7fc96819a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:43.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.889+0000 7fc96f600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc968114600 con 0x7fc968101710 2024-04-24T06:54:43.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.889+0000 7fc96f600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc968114480 con 0x7fc96810e9a0 2024-04-24T06:54:43.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.889+0000 7fc96f600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc968114780 con 0x7fc968100df0 2024-04-24T06:54:43.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.889+0000 7fc967e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc968101710 0x7fc9681a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:43.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.889+0000 7fc967e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc968101710 0x7fc9681a0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52758/0 (socket says 172.21.15.5:52758) 2024-04-24T06:54:43.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.889+0000 7fc967e00640 1 -- 172.21.15.5:0/1142645919 learned_addr learned my addr 172.21.15.5:0/1142645919 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:43.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.889+0000 7fc96cc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc968100df0 0x7fc9681a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:43.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.890+0000 7fc96d600640 1 --2- 172.21.15.5:0/1142645919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc96810e9a0 0x7fc96819a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:43.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.890+0000 7fc967e00640 1 -- 172.21.15.5:0/1142645919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc968100df0 msgr2=0x7fc9681a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:43.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.890+0000 7fc967e00640 1 --2- 172.21.15.5:0/1142645919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc968100df0 0x7fc9681a0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:43.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.890+0000 7fc967e00640 1 -- 172.21.15.5:0/1142645919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc96810e9a0 msgr2=0x7fc96819a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:43.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.890+0000 7fc967e00640 1 --2- 172.21.15.5:0/1142645919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc96810e9a0 0x7fc96819a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:43.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.890+0000 7fc967e00640 1 -- 172.21.15.5:0/1142645919 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc96819b0a0 con 0x7fc968101710 2024-04-24T06:54:43.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.890+0000 7fc967e00640 1 --2- 172.21.15.5:0/1142645919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc968101710 0x7fc9681a0c60 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc95800e9f0 tx=0x7fc95800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:43.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.890+0000 7fc965600640 1 -- 172.21.15.5:0/1142645919 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95800cd80 con 0x7fc968101710 2024-04-24T06:54:43.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.890+0000 7fc96f600640 1 -- 172.21.15.5:0/1142645919 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc96819b390 con 0x7fc968101710 2024-04-24T06:54:43.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.891+0000 7fc965600640 1 -- 172.21.15.5:0/1142645919 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc958010910 con 0x7fc968101710 2024-04-24T06:54:43.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.891+0000 7fc965600640 1 -- 172.21.15.5:0/1142645919 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9580188f0 con 0x7fc968101710 2024-04-24T06:54:43.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.891+0000 7fc96f600640 1 -- 172.21.15.5:0/1142645919 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9681a75e0 con 0x7fc968101710 2024-04-24T06:54:43.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.892+0000 7fc96f600640 1 -- 172.21.15.5:0/1142645919 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc96819be00 con 0x7fc968101710 2024-04-24T06:54:43.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.894+0000 7fc965600640 1 -- 172.21.15.5:0/1142645919 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fc958010ac0 con 0x7fc968101710 2024-04-24T06:54:43.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.895+0000 7fc965600640 1 --2- 172.21.15.5:0/1142645919 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc9500a3da0 0x7fc9500a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:43.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.895+0000 7fc965600640 1 -- 172.21.15.5:0/1142645919 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7fc9580d0b50 con 0x7fc968101710 2024-04-24T06:54:43.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.896+0000 7fc96cc00640 1 --2- 172.21.15.5:0/1142645919 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc9500a3da0 0x7fc9500a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:43.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.896+0000 7fc96cc00640 1 --2- 172.21.15.5:0/1142645919 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc9500a3da0 0x7fc9500a6260 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc95c006010 tx=0x7fc95c000bc0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:43.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:43.899+0000 7fc965600640 1 -- 172.21.15.5:0/1142645919 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc95809cb30 con 0x7fc968101710 2024-04-24T06:54:44.005 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:43 smithi005 ceph-mon[27798]: pgmap v331: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:54:44.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:43 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/3512001828' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:54:44.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:43 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/132559352' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T06:54:44.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:43 smithi037 ceph-mon[32864]: pgmap v331: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:54:44.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:43 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/3512001828' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:54:44.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:43 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/132559352' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T06:54:44.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.108+0000 7fc96f600640 1 -- 172.21.15.5:0/1142645919 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc96802d620 con 0x7fc968101710 2024-04-24T06:54:44.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.109+0000 7fc965600640 1 -- 172.21.15.5:0/1142645919 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) ==== 104+0+11 (secure 0 0 0) 0x7fc9580a0e30 con 0x7fc968101710 2024-04-24T06:54:44.110 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-24T06:54:44.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.112+0000 7fc96f600640 1 -- 172.21.15.5:0/1142645919 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc9500a3da0 msgr2=0x7fc9500a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:44.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.112+0000 7fc96f600640 1 --2- 172.21.15.5:0/1142645919 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc9500a3da0 0x7fc9500a6260 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc95c006010 tx=0x7fc95c000bc0 comp rx=0 tx=0).stop 2024-04-24T06:54:44.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.112+0000 7fc96f600640 1 -- 172.21.15.5:0/1142645919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc968101710 msgr2=0x7fc9681a0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:44.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.112+0000 7fc96f600640 1 --2- 172.21.15.5:0/1142645919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc968101710 0x7fc9681a0c60 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc95800e9f0 tx=0x7fc95800eeb0 comp rx=0 tx=0).stop 2024-04-24T06:54:44.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.112+0000 7fc96f600640 1 -- 172.21.15.5:0/1142645919 shutdown_connections 2024-04-24T06:54:44.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.112+0000 7fc96f600640 1 --2- 172.21.15.5:0/1142645919 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fc9500a3da0 0x7fc9500a6260 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:44.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.112+0000 7fc96f600640 1 --2- 172.21.15.5:0/1142645919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc96810e9a0 0x7fc96819a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:44.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.112+0000 7fc96f600640 1 --2- 172.21.15.5:0/1142645919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc968101710 0x7fc9681a0c60 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:44.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.112+0000 7fc96f600640 1 --2- 172.21.15.5:0/1142645919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc968100df0 0x7fc9681a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:44.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.112+0000 7fc96f600640 1 -- 172.21.15.5:0/1142645919 >> 172.21.15.5:0/1142645919 conn(0x7fc9680780b0 msgr2=0x7fc96810ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:44.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.112+0000 7fc96f600640 1 -- 172.21.15.5:0/1142645919 shutdown_connections 2024-04-24T06:54:44.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.112+0000 7fc96f600640 1 -- 172.21.15.5:0/1142645919 wait complete. 2024-04-24T06:54:44.133 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:54:44.195 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:43 smithi137 ceph-mon[33017]: pgmap v331: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:54:44.195 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:43 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/3512001828' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:54:44.196 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:43 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/132559352' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T06:54:44.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.352+0000 7fbb1fa00640 1 -- 172.21.15.5:0/156214699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb18103220 msgr2=0x7fbb18103600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:44.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.352+0000 7fbb1fa00640 1 --2- 172.21.15.5:0/156214699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb18103220 0x7fbb18103600 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fbb08009900 tx=0x7fbb0802a480 comp rx=0 tx=0).stop 2024-04-24T06:54:44.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.352+0000 7fbb1fa00640 1 -- 172.21.15.5:0/156214699 shutdown_connections 2024-04-24T06:54:44.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.352+0000 7fbb1fa00640 1 --2- 172.21.15.5:0/156214699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb1810c5f0 0x7fbb1810ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:44.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.352+0000 7fbb1fa00640 1 --2- 172.21.15.5:0/156214699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb18103b40 0x7fbb1810c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:44.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.352+0000 7fbb1fa00640 1 --2- 172.21.15.5:0/156214699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb18103220 0x7fbb18103600 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:44.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.352+0000 7fbb1fa00640 1 -- 172.21.15.5:0/156214699 >> 172.21.15.5:0/156214699 conn(0x7fbb18078020 msgr2=0x7fbb18078430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:44.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.353+0000 7fbb1fa00640 1 -- 172.21.15.5:0/156214699 shutdown_connections 2024-04-24T06:54:44.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.353+0000 7fbb1fa00640 1 -- 172.21.15.5:0/156214699 wait complete. 2024-04-24T06:54:44.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.354+0000 7fbb1fa00640 1 Processor -- start 2024-04-24T06:54:44.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.354+0000 7fbb1fa00640 1 -- start start 2024-04-24T06:54:44.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.354+0000 7fbb1fa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb18103220 0x7fbb181a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:44.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.354+0000 7fbb1fa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb18103b40 0x7fbb181a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:44.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.354+0000 7fbb1fa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb1810c5f0 0x7fbb1819a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:44.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.354+0000 7fbb1fa00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbb18112380 con 0x7fbb18103220 2024-04-24T06:54:44.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.354+0000 7fbb1fa00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbb18112200 con 0x7fbb1810c5f0 2024-04-24T06:54:44.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.354+0000 7fbb1fa00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbb18112500 con 0x7fbb18103b40 2024-04-24T06:54:44.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.354+0000 7fbb1d000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb18103220 0x7fbb181a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:44.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.354+0000 7fbb1d000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb18103220 0x7fbb181a0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52778/0 (socket says 172.21.15.5:52778) 2024-04-24T06:54:44.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.354+0000 7fbb1d000640 1 -- 172.21.15.5:0/936009100 learned_addr learned my addr 172.21.15.5:0/936009100 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:44.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.355+0000 7fbb1da00640 1 --2- 172.21.15.5:0/936009100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb1810c5f0 0x7fbb1819a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:44.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.355+0000 7fbb17e00640 1 --2- 172.21.15.5:0/936009100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb18103b40 0x7fbb181a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:44.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.355+0000 7fbb1d000640 1 -- 172.21.15.5:0/936009100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb18103b40 msgr2=0x7fbb181a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:44.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.355+0000 7fbb1d000640 1 --2- 172.21.15.5:0/936009100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb18103b40 0x7fbb181a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:44.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.355+0000 7fbb1d000640 1 -- 172.21.15.5:0/936009100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb1810c5f0 msgr2=0x7fbb1819a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:44.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.355+0000 7fbb1d000640 1 --2- 172.21.15.5:0/936009100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb1810c5f0 0x7fbb1819a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:44.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.355+0000 7fbb1d000640 1 -- 172.21.15.5:0/936009100 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb1819b0e0 con 0x7fbb18103220 2024-04-24T06:54:44.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.355+0000 7fbb1da00640 1 --2- 172.21.15.5:0/936009100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb1810c5f0 0x7fbb1819a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:54:44.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.355+0000 7fbb1d000640 1 --2- 172.21.15.5:0/936009100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb18103220 0x7fbb181a0760 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fbb08006010 tx=0x7fbb080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:44.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.355+0000 7fbb15600640 1 -- 172.21.15.5:0/936009100 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0803f030 con 0x7fbb18103220 2024-04-24T06:54:44.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.356+0000 7fbb1fa00640 1 -- 172.21.15.5:0/936009100 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb1819b370 con 0x7fbb18103220 2024-04-24T06:54:44.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.356+0000 7fbb15600640 1 -- 172.21.15.5:0/936009100 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb08003ce0 con 0x7fbb18103220 2024-04-24T06:54:44.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.356+0000 7fbb1fa00640 1 -- 172.21.15.5:0/936009100 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb181a7620 con 0x7fbb18103220 2024-04-24T06:54:44.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.356+0000 7fbb15600640 1 -- 172.21.15.5:0/936009100 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb08049060 con 0x7fbb18103220 2024-04-24T06:54:44.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.357+0000 7fbb1fa00640 1 -- 172.21.15.5:0/936009100 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb1806af10 con 0x7fbb18103220 2024-04-24T06:54:44.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.359+0000 7fbb15600640 1 -- 172.21.15.5:0/936009100 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fbb08042030 con 0x7fbb18103220 2024-04-24T06:54:44.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.360+0000 7fbb15600640 1 --2- 172.21.15.5:0/936009100 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fbafc0a3d20 0x7fbafc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:44.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.361+0000 7fbb15600640 1 -- 172.21.15.5:0/936009100 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7fbb080f1400 con 0x7fbb18103220 2024-04-24T06:54:44.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.361+0000 7fbb17e00640 1 --2- 172.21.15.5:0/936009100 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fbafc0a3d20 0x7fbafc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:44.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.361+0000 7fbb17e00640 1 --2- 172.21.15.5:0/936009100 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fbafc0a3d20 0x7fbafc0a61e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbb1810bfa0 tx=0x7fbb00005f90 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:44.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.363+0000 7fbb15600640 1 -- 172.21.15.5:0/936009100 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbb080bdaf0 con 0x7fbb18103220 2024-04-24T06:54:44.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.572+0000 7fbb1fa00640 1 -- 172.21.15.5:0/936009100 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbb181a77f0 con 0x7fbb18103220 2024-04-24T06:54:44.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.573+0000 7fbb15600640 1 -- 172.21.15.5:0/936009100 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) ==== 100+0+11 (secure 0 0 0) 0x7fbb080bd3f0 con 0x7fbb18103220 2024-04-24T06:54:44.573 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-24T06:54:44.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.575+0000 7fbb1fa00640 1 -- 172.21.15.5:0/936009100 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fbafc0a3d20 msgr2=0x7fbafc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:44.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.575+0000 7fbb1fa00640 1 --2- 172.21.15.5:0/936009100 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fbafc0a3d20 0x7fbafc0a61e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbb1810bfa0 tx=0x7fbb00005f90 comp rx=0 tx=0).stop 2024-04-24T06:54:44.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.575+0000 7fbb1fa00640 1 -- 172.21.15.5:0/936009100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb18103220 msgr2=0x7fbb181a0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:44.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.575+0000 7fbb1fa00640 1 --2- 172.21.15.5:0/936009100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb18103220 0x7fbb181a0760 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fbb08006010 tx=0x7fbb080042f0 comp rx=0 tx=0).stop 2024-04-24T06:54:44.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.575+0000 7fbb1fa00640 1 -- 172.21.15.5:0/936009100 shutdown_connections 2024-04-24T06:54:44.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.575+0000 7fbb1fa00640 1 --2- 172.21.15.5:0/936009100 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fbafc0a3d20 0x7fbafc0a61e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:44.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.575+0000 7fbb1fa00640 1 --2- 172.21.15.5:0/936009100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb1810c5f0 0x7fbb1819a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:44.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.575+0000 7fbb1fa00640 1 --2- 172.21.15.5:0/936009100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb18103b40 0x7fbb181a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:44.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.575+0000 7fbb1fa00640 1 --2- 172.21.15.5:0/936009100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb18103220 0x7fbb181a0760 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:44.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.575+0000 7fbb1fa00640 1 -- 172.21.15.5:0/936009100 >> 172.21.15.5:0/936009100 conn(0x7fbb18078020 msgr2=0x7fbb18108510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:44.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.576+0000 7fbb1fa00640 1 -- 172.21.15.5:0/936009100 shutdown_connections 2024-04-24T06:54:44.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.576+0000 7fbb1fa00640 1 -- 172.21.15.5:0/936009100 wait complete. 2024-04-24T06:54:44.596 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-24T06:54:44.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.809+0000 7fca25000640 1 -- 172.21.15.5:0/113596217 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca201087b0 msgr2=0x7fca20108b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:44.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.809+0000 7fca25000640 1 --2- 172.21.15.5:0/113596217 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca201087b0 0x7fca20108b90 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fca10009cd0 tx=0x7fca1002a8e0 comp rx=0 tx=0).stop 2024-04-24T06:54:44.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.809+0000 7fca25000640 1 -- 172.21.15.5:0/113596217 shutdown_connections 2024-04-24T06:54:44.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.809+0000 7fca25000640 1 --2- 172.21.15.5:0/113596217 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca2010d8a0 0x7fca20115420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:44.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.809+0000 7fca25000640 1 --2- 172.21.15.5:0/113596217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca20109160 0x7fca2010d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:44.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.809+0000 7fca25000640 1 --2- 172.21.15.5:0/113596217 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca201087b0 0x7fca20108b90 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:44.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.809+0000 7fca25000640 1 -- 172.21.15.5:0/113596217 >> 172.21.15.5:0/113596217 conn(0x7fca20077dd0 msgr2=0x7fca20106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:44.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.809+0000 7fca25000640 1 -- 172.21.15.5:0/113596217 shutdown_connections 2024-04-24T06:54:44.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.809+0000 7fca25000640 1 -- 172.21.15.5:0/113596217 wait complete. 2024-04-24T06:54:44.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.810+0000 7fca25000640 1 Processor -- start 2024-04-24T06:54:44.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.810+0000 7fca25000640 1 -- start start 2024-04-24T06:54:44.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.810+0000 7fca25000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca201087b0 0x7fca201a4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:44.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.810+0000 7fca25000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca2010d8a0 0x7fca201a4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:44.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.810+0000 7fca25000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca2019eb60 0x7fca2019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:44.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.811+0000 7fca25000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fca20118840 con 0x7fca201087b0 2024-04-24T06:54:44.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.811+0000 7fca25000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fca201186c0 con 0x7fca2019eb60 2024-04-24T06:54:44.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.811+0000 7fca25000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fca201189c0 con 0x7fca2010d8a0 2024-04-24T06:54:44.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.811+0000 7fca1f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca201087b0 0x7fca201a4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:44.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.811+0000 7fca1f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca201087b0 0x7fca201a4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52796/0 (socket says 172.21.15.5:52796) 2024-04-24T06:54:44.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.811+0000 7fca1f400640 1 -- 172.21.15.5:0/3415290759 learned_addr learned my addr 172.21.15.5:0/3415290759 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:54:44.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.811+0000 7fca1fe00640 1 --2- 172.21.15.5:0/3415290759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca2019eb60 0x7fca2019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:44.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.811+0000 7fca16a00640 1 --2- 172.21.15.5:0/3415290759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca2010d8a0 0x7fca201a4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:44.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.811+0000 7fca1fe00640 1 -- 172.21.15.5:0/3415290759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca2010d8a0 msgr2=0x7fca201a4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:44.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.811+0000 7fca1fe00640 1 --2- 172.21.15.5:0/3415290759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca2010d8a0 0x7fca201a4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:44.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.811+0000 7fca1fe00640 1 -- 172.21.15.5:0/3415290759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca201087b0 msgr2=0x7fca201a4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:44.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.811+0000 7fca1fe00640 1 --2- 172.21.15.5:0/3415290759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca201087b0 0x7fca201a4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:44.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.811+0000 7fca1fe00640 1 -- 172.21.15.5:0/3415290759 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca2019f760 con 0x7fca2019eb60 2024-04-24T06:54:44.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.811+0000 7fca1f400640 1 --2- 172.21.15.5:0/3415290759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca201087b0 0x7fca201a4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:54:44.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.812+0000 7fca1fe00640 1 --2- 172.21.15.5:0/3415290759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca2019eb60 0x7fca2019efe0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fca1800e970 tx=0x7fca1800ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:44.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.812+0000 7fca1cc00640 1 -- 172.21.15.5:0/3415290759 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1800ccc0 con 0x7fca2019eb60 2024-04-24T06:54:44.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.812+0000 7fca25000640 1 -- 172.21.15.5:0/3415290759 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca201ab7f0 con 0x7fca2019eb60 2024-04-24T06:54:44.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.812+0000 7fca1cc00640 1 -- 172.21.15.5:0/3415290759 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca18005020 con 0x7fca2019eb60 2024-04-24T06:54:44.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.812+0000 7fca1cc00640 1 -- 172.21.15.5:0/3415290759 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca18004a00 con 0x7fca2019eb60 2024-04-24T06:54:44.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.812+0000 7fca25000640 1 -- 172.21.15.5:0/3415290759 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca201abbf0 con 0x7fca2019eb60 2024-04-24T06:54:44.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.813+0000 7fca25000640 1 -- 172.21.15.5:0/3415290759 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca20109c30 con 0x7fca2019eb60 2024-04-24T06:54:44.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.815+0000 7fca1cc00640 1 -- 172.21.15.5:0/3415290759 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fca180051d0 con 0x7fca2019eb60 2024-04-24T06:54:44.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.816+0000 7fca1cc00640 1 --2- 172.21.15.5:0/3415290759 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fca040a3d10 0x7fca040a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:54:44.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.817+0000 7fca1cc00640 1 -- 172.21.15.5:0/3415290759 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7fca180d07f0 con 0x7fca2019eb60 2024-04-24T06:54:44.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.817+0000 7fca1f400640 1 --2- 172.21.15.5:0/3415290759 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fca040a3d10 0x7fca040a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:54:44.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.818+0000 7fca1f400640 1 --2- 172.21.15.5:0/3415290759 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fca040a3d10 0x7fca040a61d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fca10009ca0 tx=0x7fca10005d80 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:54:44.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:44.821+0000 7fca1cc00640 1 -- 172.21.15.5:0/3415290759 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fca1809ce20 con 0x7fca2019eb60 2024-04-24T06:54:44.945 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:44 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/1142645919' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T06:54:44.946 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:44 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/936009100' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T06:54:45.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:44 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1142645919' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T06:54:45.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:44 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/936009100' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T06:54:45.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:44 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/1142645919' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T06:54:45.018 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:44 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/936009100' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T06:54:45.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:45.082+0000 7fca25000640 1 -- 172.21.15.5:0/3415290759 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fca2004fe00 con 0x7fca2019eb60 2024-04-24T06:54:45.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:45.094+0000 7fca1cc00640 1 -- 172.21.15.5:0/3415290759 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v27) ==== 81+0+746052 (secure 0 0 0) 0x7fca180870e0 con 0x7fca2019eb60 2024-04-24T06:54:45.108 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:54:45.108 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:54:45.108 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 27, 2024-04-24T06:54:45.108 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.109 INFO:teuthology.orchestra.run.smithi005.stdout: "active_gid": 25202, 2024-04-24T06:54:45.109 INFO:teuthology.orchestra.run.smithi005.stdout: "active_name": "x", 2024-04-24T06:54:45.109 INFO:teuthology.orchestra.run.smithi005.stdout: "active_addrs": { 2024-04-24T06:54:45.109 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-04-24T06:54:45.109 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:45.109 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-04-24T06:54:45.109 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6800", 2024-04-24T06:54:45.109 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3154624651 2024-04-24T06:54:45.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:45.110 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-04-24T06:54:45.110 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6801", 2024-04-24T06:54:45.110 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3154624651 2024-04-24T06:54:45.110 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.110 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-04-24T06:54:45.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.111 INFO:teuthology.orchestra.run.smithi005.stdout: "active_addr": "172.21.15.5:6801/3154624651", 2024-04-24T06:54:45.111 INFO:teuthology.orchestra.run.smithi005.stdout: "active_change": "2024-04-24T06:44:01.966470+0000", 2024-04-24T06:54:45.111 INFO:teuthology.orchestra.run.smithi005.stdout: "active_mgr_features": 4540701547738038271, 2024-04-24T06:54:45.111 INFO:teuthology.orchestra.run.smithi005.stdout: "available": true, 2024-04-24T06:54:45.111 INFO:teuthology.orchestra.run.smithi005.stdout: "standbys": [ 2024-04-24T06:54:45.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:45.111 INFO:teuthology.orchestra.run.smithi005.stdout: "gid": 25196, 2024-04-24T06:54:45.111 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "y", 2024-04-24T06:54:45.111 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_features": 4540701547738038271, 2024-04-24T06:54:45.112 INFO:teuthology.orchestra.run.smithi005.stdout: "available_modules": [ 2024-04-24T06:54:45.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:45.112 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "alerts", 2024-04-24T06:54:45.112 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:45.112 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:45.112 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:45.112 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-04-24T06:54:45.112 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-04-24T06:54:45.113 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:45.113 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.113 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.113 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-04-24T06:54:45.113 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.113 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.113 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.113 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How frequently to reexamine health status", 2024-04-24T06:54:45.114 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.114 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.114 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.114 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:45.114 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:45.114 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.114 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.114 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.115 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.115 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.115 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.115 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.115 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.115 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.115 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.115 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.116 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.116 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.116 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.116 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.116 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.116 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.116 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.116 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:45.117 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:45.117 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.117 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.117 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.117 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.117 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.117 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.117 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.118 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.118 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.118 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.118 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.118 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:45.118 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:45.119 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.119 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.119 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.119 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:45.119 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.119 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.119 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.119 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.119 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.120 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.120 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.120 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.120 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.120 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.120 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.120 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.120 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.121 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.121 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:45.121 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:45.121 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.121 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.121 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.121 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.122 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.122 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.122 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.122 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.122 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.122 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.122 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.123 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_destination": { 2024-04-24T06:54:45.123 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_destination", 2024-04-24T06:54:45.123 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.123 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.123 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.123 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.123 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.123 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.123 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.124 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email address to send alerts to", 2024-04-24T06:54:45.124 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.124 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.124 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.124 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_from_name": { 2024-04-24T06:54:45.124 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_from_name", 2024-04-24T06:54:45.124 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.125 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.125 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.125 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "Ceph", 2024-04-24T06:54:45.125 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.125 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.125 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.125 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email From: name", 2024-04-24T06:54:45.125 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.126 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.126 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.126 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_host": { 2024-04-24T06:54:45.126 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_host", 2024-04-24T06:54:45.126 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.126 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.126 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.126 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.127 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.127 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.127 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.127 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP server", 2024-04-24T06:54:45.127 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.127 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.127 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.128 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_password": { 2024-04-24T06:54:45.128 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_password", 2024-04-24T06:54:45.128 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.128 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.128 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.128 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.128 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.128 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.129 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.129 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Password to authenticate with", 2024-04-24T06:54:45.129 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.129 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.129 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.129 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_port": { 2024-04-24T06:54:45.129 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_port", 2024-04-24T06:54:45.129 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:45.130 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.130 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.130 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "465", 2024-04-24T06:54:45.130 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.130 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.130 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.130 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP port", 2024-04-24T06:54:45.130 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.131 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.131 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.131 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_sender": { 2024-04-24T06:54:45.131 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_sender", 2024-04-24T06:54:45.131 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.131 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.131 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.132 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.132 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.132 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.132 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.132 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP envelope sender", 2024-04-24T06:54:45.132 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.132 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.133 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.133 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_ssl": { 2024-04-24T06:54:45.133 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_ssl", 2024-04-24T06:54:45.133 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.133 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.133 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.134 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:45.134 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.134 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.134 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.134 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-24T06:54:45.134 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.134 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.134 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.135 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_user": { 2024-04-24T06:54:45.135 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_user", 2024-04-24T06:54:45.135 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.135 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.135 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.135 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.135 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.135 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.136 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.136 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "User to authenticate as", 2024-04-24T06:54:45.136 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.136 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.136 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.136 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.136 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:45.137 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "balancer", 2024-04-24T06:54:45.137 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:45.137 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:45.137 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:45.137 INFO:teuthology.orchestra.run.smithi005.stdout: "active": { 2024-04-24T06:54:45.137 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "active", 2024-04-24T06:54:45.137 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.138 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.138 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.138 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:45.138 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.138 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.138 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.138 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically balance PGs across cluster", 2024-04-24T06:54:45.138 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.139 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.139 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.139 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_time": { 2024-04-24T06:54:45.139 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_time", 2024-04-24T06:54:45.139 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.139 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.139 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.140 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0000", 2024-04-24T06:54:45.140 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.140 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.140 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.140 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "beginning time of day to automatically balance", 2024-04-24T06:54:45.140 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T06:54:45.140 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.140 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.141 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_weekday": { 2024-04-24T06:54:45.141 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_weekday", 2024-04-24T06:54:45.141 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-04-24T06:54:45.141 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.141 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.141 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-04-24T06:54:45.141 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-04-24T06:54:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-04-24T06:54:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-24T06:54:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T06:54:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_max_iterations": { 2024-04-24T06:54:45.143 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_max_iterations", 2024-04-24T06:54:45.143 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-04-24T06:54:45.143 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.146 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.146 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "25", 2024-04-24T06:54:45.146 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-04-24T06:54:45.146 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "250", 2024-04-24T06:54:45.146 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.146 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-24T06:54:45.147 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.147 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.147 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.147 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_metrics": { 2024-04-24T06:54:45.147 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_metrics", 2024-04-24T06:54:45.147 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.148 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.148 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.148 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "pgs,objects,bytes", 2024-04-24T06:54:45.148 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.148 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.148 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.148 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-24T06:54:45.148 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-24T06:54:45.149 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.149 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.149 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_step": { 2024-04-24T06:54:45.149 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_step", 2024-04-24T06:54:45.149 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-24T06:54:45.149 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.149 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.149 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.5", 2024-04-24T06:54:45.150 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0.001", 2024-04-24T06:54:45.150 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "0.999", 2024-04-24T06:54:45.150 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.150 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "aggressiveness of optimization", 2024-04-24T06:54:45.150 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-24T06:54:45.150 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.150 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.151 INFO:teuthology.orchestra.run.smithi005.stdout: "end_time": { 2024-04-24T06:54:45.151 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_time", 2024-04-24T06:54:45.151 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.151 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.151 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.151 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2359", 2024-04-24T06:54:45.151 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.152 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.152 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.152 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ending time of day to automatically balance", 2024-04-24T06:54:45.152 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T06:54:45.152 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.152 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.152 INFO:teuthology.orchestra.run.smithi005.stdout: "end_weekday": { 2024-04-24T06:54:45.153 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_weekday", 2024-04-24T06:54:45.153 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-04-24T06:54:45.153 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.153 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.153 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-04-24T06:54:45.153 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-04-24T06:54:45.153 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-04-24T06:54:45.153 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.154 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-24T06:54:45.154 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T06:54:45.154 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.154 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.154 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:45.154 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:45.154 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.155 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.155 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.155 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.155 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.155 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.155 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.155 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.156 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.156 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.156 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.156 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.156 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.156 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.156 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.156 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.157 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.157 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.157 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:45.157 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:45.157 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.157 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.157 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.157 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.158 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.158 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.158 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.158 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.158 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.158 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.158 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.159 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:45.159 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:45.159 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.159 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.159 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.159 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:45.159 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.160 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.160 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.160 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.160 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.160 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.160 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.160 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.160 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.160 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.161 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.161 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.161 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.161 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.161 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:45.161 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:45.161 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.162 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.162 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.162 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.162 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.162 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.162 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.162 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.162 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.163 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.163 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.163 INFO:teuthology.orchestra.run.smithi005.stdout: "min_score": { 2024-04-24T06:54:45.163 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_score", 2024-04-24T06:54:45.163 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-24T06:54:45.163 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.163 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.164 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-04-24T06:54:45.164 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.164 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.164 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.164 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-24T06:54:45.164 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.164 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.164 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.165 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-04-24T06:54:45.165 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-04-24T06:54:45.165 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.165 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.165 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.165 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "upmap", 2024-04-24T06:54:45.165 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.165 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.166 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.166 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-compat", 2024-04-24T06:54:45.166 INFO:teuthology.orchestra.run.smithi005.stdout: "none", 2024-04-24T06:54:45.166 INFO:teuthology.orchestra.run.smithi005.stdout: "read", 2024-04-24T06:54:45.166 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap", 2024-04-24T06:54:45.166 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap-read" 2024-04-24T06:54:45.166 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.166 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Balancer mode", 2024-04-24T06:54:45.167 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.167 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.167 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.167 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_ids": { 2024-04-24T06:54:45.167 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_ids", 2024-04-24T06:54:45.167 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.167 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.168 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.168 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.168 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.168 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.168 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.168 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-24T06:54:45.168 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.168 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.169 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.169 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-04-24T06:54:45.169 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-04-24T06:54:45.169 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:45.169 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.169 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.169 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-04-24T06:54:45.169 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.170 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.170 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.170 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-24T06:54:45.170 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.170 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.170 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.170 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_deviation": { 2024-04-24T06:54:45.171 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_deviation", 2024-04-24T06:54:45.171 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:45.171 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.171 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.171 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-04-24T06:54:45.171 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-04-24T06:54:45.171 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.172 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.172 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-24T06:54:45.172 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-24T06:54:45.172 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.172 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.172 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_optimizations": { 2024-04-24T06:54:45.172 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_optimizations", 2024-04-24T06:54:45.173 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-04-24T06:54:45.173 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.173 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.173 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-24T06:54:45.173 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.173 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.173 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.173 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-24T06:54:45.173 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.174 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.174 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.174 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.174 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:45.174 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm", 2024-04-24T06:54:45.174 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:45.175 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:45.175 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:45.175 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_down_multiplier": { 2024-04-24T06:54:45.175 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_down_multiplier", 2024-04-24T06:54:45.175 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-24T06:54:45.175 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.175 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.175 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-04-24T06:54:45.176 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.176 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.176 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.176 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-24T06:54:45.176 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.176 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.176 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.177 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_refresh_rate": { 2024-04-24T06:54:45.177 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_refresh_rate", 2024-04-24T06:54:45.177 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:45.177 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.177 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.177 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "20", 2024-04-24T06:54:45.177 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.178 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.178 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.178 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-24T06:54:45.178 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.178 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.178 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.179 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_starting_port": { 2024-04-24T06:54:45.179 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_starting_port", 2024-04-24T06:54:45.179 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:45.179 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.179 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.179 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4721", 2024-04-24T06:54:45.179 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.179 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.180 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.180 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-24T06:54:45.180 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.180 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.180 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.181 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_ptrace": { 2024-04-24T06:54:45.181 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_ptrace", 2024-04-24T06:54:45.181 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.181 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.181 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.181 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.181 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.181 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.182 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.182 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-24T06:54:45.182 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-24T06:54:45.182 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.182 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.182 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_interval": { 2024-04-24T06:54:45.183 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_interval", 2024-04-24T06:54:45.183 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:45.183 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.183 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.183 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-04-24T06:54:45.183 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.183 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.183 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.184 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-24T06:54:45.184 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.184 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.184 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.184 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_memory_target_ratio": { 2024-04-24T06:54:45.184 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_memory_target_ratio", 2024-04-24T06:54:45.184 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-24T06:54:45.185 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.185 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.185 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.7", 2024-04-24T06:54:45.185 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.185 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.185 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.185 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-24T06:54:45.185 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.186 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.186 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.186 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm_log_destination": { 2024-04-24T06:54:45.186 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm_log_destination", 2024-04-24T06:54:45.186 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.186 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.186 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.187 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.187 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.187 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.187 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.187 INFO:teuthology.orchestra.run.smithi005.stdout: "file", 2024-04-24T06:54:45.187 INFO:teuthology.orchestra.run.smithi005.stdout: "file,syslog", 2024-04-24T06:54:45.187 INFO:teuthology.orchestra.run.smithi005.stdout: "syslog" 2024-04-24T06:54:45.188 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.188 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-24T06:54:45.188 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.188 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.188 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.188 INFO:teuthology.orchestra.run.smithi005.stdout: "cgroups_split": { 2024-04-24T06:54:45.188 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cgroups_split", 2024-04-24T06:54:45.189 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.189 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.189 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.189 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:45.189 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.189 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.189 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.189 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-24T06:54:45.190 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.190 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.190 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.190 INFO:teuthology.orchestra.run.smithi005.stdout: "config_checks_enabled": { 2024-04-24T06:54:45.190 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_checks_enabled", 2024-04-24T06:54:45.190 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.190 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.191 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.191 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.191 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.191 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.191 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.191 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-24T06:54:45.191 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.191 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.192 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.192 INFO:teuthology.orchestra.run.smithi005.stdout: "config_dashboard": { 2024-04-24T06:54:45.192 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_dashboard", 2024-04-24T06:54:45.192 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.192 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.192 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.192 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:45.192 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.193 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.193 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.193 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-24T06:54:45.193 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.193 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.193 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.193 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_alertmanager": { 2024-04-24T06:54:45.194 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_alertmanager", 2024-04-24T06:54:45.194 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.194 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.194 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.194 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-24T06:54:45.194 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.194 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.194 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.195 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-04-24T06:54:45.195 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.195 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.195 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.195 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_base": { 2024-04-24T06:54:45.195 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_base", 2024-04-24T06:54:45.195 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.196 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.196 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.196 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-24T06:54:45.196 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.196 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.196 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.196 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Container image name, without the tag", 2024-04-24T06:54:45.196 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.197 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.197 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.197 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_elasticsearch": { 2024-04-24T06:54:45.197 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_elasticsearch", 2024-04-24T06:54:45.197 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.197 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.197 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.198 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-24T06:54:45.198 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.198 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.198 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.198 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "elasticsearch container image", 2024-04-24T06:54:45.198 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.198 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.199 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.199 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_grafana": { 2024-04-24T06:54:45.199 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_grafana", 2024-04-24T06:54:45.199 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.199 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.199 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.199 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-24T06:54:45.200 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.200 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.200 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.200 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-04-24T06:54:45.200 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.200 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.200 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.201 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_haproxy": { 2024-04-24T06:54:45.201 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_haproxy", 2024-04-24T06:54:45.201 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.201 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.201 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.201 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-24T06:54:45.201 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.202 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.202 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.202 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "HAproxy container image", 2024-04-24T06:54:45.202 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.202 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.202 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.203 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_agent": { 2024-04-24T06:54:45.203 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_agent", 2024-04-24T06:54:45.203 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.203 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.203 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.203 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-24T06:54:45.203 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.204 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.204 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.204 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger agent container image", 2024-04-24T06:54:45.204 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.204 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.204 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.205 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_collector": { 2024-04-24T06:54:45.205 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_collector", 2024-04-24T06:54:45.205 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.205 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.205 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.205 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-24T06:54:45.205 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.205 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.206 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.206 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger collector container image", 2024-04-24T06:54:45.206 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.206 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.206 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.206 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_query": { 2024-04-24T06:54:45.206 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_query", 2024-04-24T06:54:45.207 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.207 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.207 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.207 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-24T06:54:45.207 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.207 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.207 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.207 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger query container image", 2024-04-24T06:54:45.208 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.208 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.208 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.208 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_keepalived": { 2024-04-24T06:54:45.208 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_keepalived", 2024-04-24T06:54:45.208 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.208 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.208 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.209 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-24T06:54:45.209 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.209 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.209 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.209 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Keepalived container image", 2024-04-24T06:54:45.209 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.209 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.209 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.210 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_loki": { 2024-04-24T06:54:45.210 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_loki", 2024-04-24T06:54:45.210 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.210 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.210 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.210 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-24T06:54:45.210 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.211 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.211 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.211 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Loki container image", 2024-04-24T06:54:45.212 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.212 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.212 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.212 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_node_exporter": { 2024-04-24T06:54:45.212 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_node_exporter", 2024-04-24T06:54:45.212 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.213 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.213 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.213 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-24T06:54:45.213 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.213 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.213 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.213 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-04-24T06:54:45.214 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.214 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.214 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.214 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_nvmeof": { 2024-04-24T06:54:45.214 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_nvmeof", 2024-04-24T06:54:45.214 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.214 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.215 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.215 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-24T06:54:45.215 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.215 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.215 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.215 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Nvme-of container image", 2024-04-24T06:54:45.215 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.215 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.216 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.216 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_prometheus": { 2024-04-24T06:54:45.216 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_prometheus", 2024-04-24T06:54:45.216 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.216 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.216 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.216 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-24T06:54:45.217 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.217 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.217 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.217 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-04-24T06:54:45.217 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.217 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.217 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.218 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_promtail": { 2024-04-24T06:54:45.218 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_promtail", 2024-04-24T06:54:45.218 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.218 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.218 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.218 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-24T06:54:45.218 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.219 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.219 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.219 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Promtail container image", 2024-04-24T06:54:45.219 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.219 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.220 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.220 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_samba": { 2024-04-24T06:54:45.220 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_samba", 2024-04-24T06:54:45.220 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.220 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.220 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.220 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-24T06:54:45.220 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.221 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.221 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.221 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Samba/SMB container image", 2024-04-24T06:54:45.221 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.221 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.221 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.221 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_snmp_gateway": { 2024-04-24T06:54:45.222 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_snmp_gateway", 2024-04-24T06:54:45.222 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.222 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.222 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.222 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-24T06:54:45.222 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.222 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.222 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.223 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SNMP Gateway container image", 2024-04-24T06:54:45.223 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.223 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.223 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.223 INFO:teuthology.orchestra.run.smithi005.stdout: "container_init": { 2024-04-24T06:54:45.223 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_init", 2024-04-24T06:54:45.223 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.224 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.224 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.224 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:45.224 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.224 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.224 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.225 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Run podman/docker with `--init`", 2024-04-24T06:54:45.225 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.225 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.225 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.225 INFO:teuthology.orchestra.run.smithi005.stdout: "daemon_cache_timeout": { 2024-04-24T06:54:45.225 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "daemon_cache_timeout", 2024-04-24T06:54:45.225 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:45.226 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.226 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.226 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-04-24T06:54:45.226 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.226 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.226 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.226 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-24T06:54:45.227 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.227 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.227 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.227 INFO:teuthology.orchestra.run.smithi005.stdout: "default_cephadm_command_timeout": { 2024-04-24T06:54:45.227 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_cephadm_command_timeout", 2024-04-24T06:54:45.227 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:45.227 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.228 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.228 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "900", 2024-04-24T06:54:45.228 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.228 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.228 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.228 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-24T06:54:45.228 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.228 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.229 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.229 INFO:teuthology.orchestra.run.smithi005.stdout: "default_registry": { 2024-04-24T06:54:45.229 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_registry", 2024-04-24T06:54:45.229 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.229 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.229 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.229 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io", 2024-04-24T06:54:45.229 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.230 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.230 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.230 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-24T06:54:45.230 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.230 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.230 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.231 INFO:teuthology.orchestra.run.smithi005.stdout: "device_cache_timeout": { 2024-04-24T06:54:45.231 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_cache_timeout", 2024-04-24T06:54:45.231 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:45.231 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.231 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.231 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1800", 2024-04-24T06:54:45.231 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.231 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.232 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.232 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache device inventory", 2024-04-24T06:54:45.232 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.232 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.232 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.232 INFO:teuthology.orchestra.run.smithi005.stdout: "device_enhanced_scan": { 2024-04-24T06:54:45.232 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_enhanced_scan", 2024-04-24T06:54:45.233 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.233 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.233 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.233 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.233 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.233 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.234 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.234 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-24T06:54:45.234 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.234 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.234 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.234 INFO:teuthology.orchestra.run.smithi005.stdout: "facts_cache_timeout": { 2024-04-24T06:54:45.234 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "facts_cache_timeout", 2024-04-24T06:54:45.235 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:45.235 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.235 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.235 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-04-24T06:54:45.235 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.235 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.235 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.235 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache host facts data", 2024-04-24T06:54:45.236 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.236 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.236 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.236 INFO:teuthology.orchestra.run.smithi005.stdout: "grafana_dashboards_path": { 2024-04-24T06:54:45.236 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "grafana_dashboards_path", 2024-04-24T06:54:45.236 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.236 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.237 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.237 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-24T06:54:45.237 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.237 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.237 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.237 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-24T06:54:45.237 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.237 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.238 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.238 INFO:teuthology.orchestra.run.smithi005.stdout: "host_check_interval": { 2024-04-24T06:54:45.238 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "host_check_interval", 2024-04-24T06:54:45.238 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:45.239 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.239 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.239 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-04-24T06:54:45.239 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.239 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.239 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.239 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to perform a host check", 2024-04-24T06:54:45.240 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.240 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.240 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.240 INFO:teuthology.orchestra.run.smithi005.stdout: "hw_monitoring": { 2024-04-24T06:54:45.240 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hw_monitoring", 2024-04-24T06:54:45.240 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.240 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.241 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.241 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.241 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.241 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.241 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.241 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-24T06:54:45.241 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.241 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.242 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.242 INFO:teuthology.orchestra.run.smithi005.stdout: "inventory_list_all": { 2024-04-24T06:54:45.242 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "inventory_list_all", 2024-04-24T06:54:45.242 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.242 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.242 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.242 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.243 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.243 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.243 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.243 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-24T06:54:45.243 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.243 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.243 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.244 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:45.244 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:45.244 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.244 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.244 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.244 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.244 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.245 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.245 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.245 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.245 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.245 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.245 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.245 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.245 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.246 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.246 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.246 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.246 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.246 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.246 INFO:teuthology.orchestra.run.smithi005.stdout: "log_refresh_metadata": { 2024-04-24T06:54:45.246 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_refresh_metadata", 2024-04-24T06:54:45.247 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.247 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.247 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.247 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.248 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.248 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.248 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.248 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-24T06:54:45.248 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.248 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.248 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.249 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:45.249 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:45.249 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.249 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.249 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.249 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:45.249 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.249 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.250 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.250 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-24T06:54:45.250 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.250 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.250 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.250 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:45.251 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:45.251 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.251 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.251 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.251 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:45.251 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.251 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.252 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.252 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.252 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.252 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.252 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.252 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.252 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.252 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.253 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.253 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.253 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.253 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.253 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:45.253 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:45.253 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.254 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.254 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.254 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.254 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.254 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.254 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.254 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.254 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.255 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.255 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.255 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-24T06:54:45.255 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-24T06:54:45.255 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.255 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.255 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.256 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.256 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.256 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.256 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.256 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-24T06:54:45.256 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.256 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.256 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.257 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-24T06:54:45.257 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-24T06:54:45.257 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.257 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.257 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.257 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "*", 2024-04-24T06:54:45.257 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.258 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.258 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.258 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-24T06:54:45.258 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.258 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.258 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.258 INFO:teuthology.orchestra.run.smithi005.stdout: "max_count_per_host": { 2024-04-24T06:54:45.259 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_count_per_host", 2024-04-24T06:54:45.259 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:45.259 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.259 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.259 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-24T06:54:45.259 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.259 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.259 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.260 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of daemons per service per host", 2024-04-24T06:54:45.260 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.260 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.260 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.260 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd_draining_count": { 2024-04-24T06:54:45.260 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_osd_draining_count", 2024-04-24T06:54:45.260 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:45.260 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.261 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.261 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-24T06:54:45.261 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.261 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.261 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.261 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-24T06:54:45.261 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.262 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.262 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.262 INFO:teuthology.orchestra.run.smithi005.stdout: "migration_current": { 2024-04-24T06:54:45.262 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "migration_current", 2024-04-24T06:54:45.262 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:45.262 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.262 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.262 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.263 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.263 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.263 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.263 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "internal - do not modify", 2024-04-24T06:54:45.263 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.263 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.263 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.264 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-04-24T06:54:45.264 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-04-24T06:54:45.264 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.264 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.264 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.264 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "root", 2024-04-24T06:54:45.264 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.264 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.265 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.265 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm-package", 2024-04-24T06:54:45.265 INFO:teuthology.orchestra.run.smithi005.stdout: "root" 2024-04-24T06:54:45.265 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.265 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "mode for remote execution of cephadm", 2024-04-24T06:54:45.265 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.265 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.265 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.266 INFO:teuthology.orchestra.run.smithi005.stdout: "oob_default_addr": { 2024-04-24T06:54:45.266 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "oob_default_addr", 2024-04-24T06:54:45.266 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.266 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.266 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.266 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "169.254.1.1", 2024-04-24T06:54:45.267 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.267 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.267 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.267 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-24T06:54:45.267 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.267 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.267 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.268 INFO:teuthology.orchestra.run.smithi005.stdout: "prometheus_alerts_path": { 2024-04-24T06:54:45.268 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus_alerts_path", 2024-04-24T06:54:45.268 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.268 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.268 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.268 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-24T06:54:45.268 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.268 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.269 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.269 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-24T06:54:45.269 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.269 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.269 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.269 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_insecure": { 2024-04-24T06:54:45.269 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_insecure", 2024-04-24T06:54:45.270 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.270 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.270 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.270 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.270 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.270 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.270 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.271 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-24T06:54:45.271 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.271 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.271 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.271 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_password": { 2024-04-24T06:54:45.271 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_password", 2024-04-24T06:54:45.271 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.272 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.272 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.272 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.272 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.272 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.272 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.272 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-24T06:54:45.272 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.273 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.273 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.273 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_url": { 2024-04-24T06:54:45.273 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_url", 2024-04-24T06:54:45.273 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.273 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.273 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.274 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.274 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.274 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.274 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.274 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-24T06:54:45.274 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.274 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.274 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.275 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_username": { 2024-04-24T06:54:45.275 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_username", 2024-04-24T06:54:45.275 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.275 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.275 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.275 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.276 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.276 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.276 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.276 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-24T06:54:45.276 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.276 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.276 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.277 INFO:teuthology.orchestra.run.smithi005.stdout: "secure_monitoring_stack": { 2024-04-24T06:54:45.277 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "secure_monitoring_stack", 2024-04-24T06:54:45.277 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.277 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.277 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.277 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.277 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.278 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.278 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.278 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-24T06:54:45.278 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.278 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.278 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.278 INFO:teuthology.orchestra.run.smithi005.stdout: "service_discovery_port": { 2024-04-24T06:54:45.279 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "service_discovery_port", 2024-04-24T06:54:45.279 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:45.279 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.279 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.279 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8765", 2024-04-24T06:54:45.279 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.279 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.279 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.280 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "cephadm service discovery port", 2024-04-24T06:54:45.280 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.280 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.280 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.280 INFO:teuthology.orchestra.run.smithi005.stdout: "ssh_config_file": { 2024-04-24T06:54:45.280 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssh_config_file", 2024-04-24T06:54:45.280 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.281 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.281 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.281 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.281 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.281 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.281 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.281 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-24T06:54:45.281 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.282 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.282 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.282 INFO:teuthology.orchestra.run.smithi005.stdout: "use_agent": { 2024-04-24T06:54:45.282 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_agent", 2024-04-24T06:54:45.282 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.282 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.282 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.283 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.283 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.283 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.283 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.283 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-24T06:54:45.283 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.283 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.283 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.284 INFO:teuthology.orchestra.run.smithi005.stdout: "use_repo_digest": { 2024-04-24T06:54:45.284 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_repo_digest", 2024-04-24T06:54:45.284 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.284 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.284 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.284 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:45.284 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.284 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.284 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.285 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-24T06:54:45.285 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.285 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.285 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.285 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_failed_host_check": { 2024-04-24T06:54:45.285 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_failed_host_check", 2024-04-24T06:54:45.285 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.286 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.286 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.286 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:45.286 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.286 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.286 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.286 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if the host check fails", 2024-04-24T06:54:45.287 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.287 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.287 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.287 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_daemons": { 2024-04-24T06:54:45.287 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_daemons", 2024-04-24T06:54:45.287 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.287 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.288 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.288 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:45.288 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.288 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.288 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.288 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-24T06:54:45.288 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.288 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.289 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.289 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_hosts": { 2024-04-24T06:54:45.289 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_hosts", 2024-04-24T06:54:45.289 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.289 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.289 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.289 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:45.290 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.290 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.290 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.290 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-24T06:54:45.290 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.290 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.290 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.291 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.291 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.291 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.291 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:45.291 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crash", 2024-04-24T06:54:45.291 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:45.291 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:45.292 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:45.292 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:45.292 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:45.292 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.292 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.292 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.292 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.293 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.293 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.293 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.293 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.293 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.293 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.293 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.293 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.294 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.294 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.294 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.294 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.294 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.294 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.295 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:45.295 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:45.295 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.295 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.295 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.295 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.295 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.296 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.296 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.296 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.296 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.296 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.296 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.297 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:45.297 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:45.297 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.298 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.298 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.298 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:45.298 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.298 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.299 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.299 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.299 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.299 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.299 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.300 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.300 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.300 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.300 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.300 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.300 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.301 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.301 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:45.301 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:45.301 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.301 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.302 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.302 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.302 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.302 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.302 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.303 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.303 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.303 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.303 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.304 INFO:teuthology.orchestra.run.smithi005.stdout: "retain_interval": { 2024-04-24T06:54:45.304 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retain_interval", 2024-04-24T06:54:45.304 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:45.304 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.304 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.304 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "31536000", 2024-04-24T06:54:45.305 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.305 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.305 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.305 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-24T06:54:45.305 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.305 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.306 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.306 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_recent_interval": { 2024-04-24T06:54:45.306 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_recent_interval", 2024-04-24T06:54:45.307 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:45.307 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.307 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.307 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1209600", 2024-04-24T06:54:45.308 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.308 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.308 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.308 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-24T06:54:45.308 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.308 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.309 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.309 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.309 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.309 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.309 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:45.309 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dashboard", 2024-04-24T06:54:45.310 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:45.310 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:45.310 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:45.310 INFO:teuthology.orchestra.run.smithi005.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-24T06:54:45.310 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-24T06:54:45.311 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:45.311 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.311 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.311 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-24T06:54:45.311 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.311 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.312 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.312 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.312 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.312 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.312 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.313 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_HOST": { 2024-04-24T06:54:45.313 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-24T06:54:45.313 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.313 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.313 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.314 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.314 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.314 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.314 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.314 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.315 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.315 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.315 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.315 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-24T06:54:45.315 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-24T06:54:45.316 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.316 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.316 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.316 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:45.316 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.316 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.317 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.317 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.317 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.317 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.317 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.318 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_ENABLED": { 2024-04-24T06:54:45.318 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_ENABLED", 2024-04-24T06:54:45.318 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.318 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.318 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.318 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.319 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.319 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.319 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.319 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.319 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.319 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.320 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.320 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-24T06:54:45.320 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-24T06:54:45.320 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.320 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.321 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.321 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:45.321 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.321 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.321 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.321 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.322 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.322 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.322 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.322 INFO:teuthology.orchestra.run.smithi005.stdout: "ENABLE_BROWSABLE_API": { 2024-04-24T06:54:45.322 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-24T06:54:45.323 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.323 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.323 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.323 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:45.323 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.323 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.323 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.324 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.324 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.324 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.325 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.325 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-24T06:54:45.325 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-24T06:54:45.325 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.325 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.326 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.326 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:45.326 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.326 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.326 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.327 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.327 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.327 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.327 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.327 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-24T06:54:45.327 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-24T06:54:45.328 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.328 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.328 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.328 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:45.328 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.328 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.328 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.329 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.329 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.329 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.329 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.329 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-24T06:54:45.330 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-24T06:54:45.330 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.330 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.330 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.331 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:45.331 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.331 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.331 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.331 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.331 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.332 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.332 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.332 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-24T06:54:45.332 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-24T06:54:45.332 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.332 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.333 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.333 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:45.333 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.333 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.333 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.334 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.334 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.334 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.334 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.334 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-24T06:54:45.335 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-24T06:54:45.335 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.335 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.335 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.335 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:45.336 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.336 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.336 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.336 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.336 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.337 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.337 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.337 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-24T06:54:45.337 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-24T06:54:45.337 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.338 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.338 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.338 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:45.339 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.339 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.339 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.339 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.339 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.340 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.340 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.340 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-24T06:54:45.340 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-24T06:54:45.341 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.341 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.341 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.341 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:45.341 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.341 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.342 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.342 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.342 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.342 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.342 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.343 INFO:teuthology.orchestra.run.smithi005.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-24T06:54:45.343 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-24T06:54:45.343 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.343 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.343 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.344 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.344 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.344 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.344 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.344 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.344 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.345 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.345 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.345 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_PASSWORD": { 2024-04-24T06:54:45.345 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-24T06:54:45.346 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.346 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.346 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.346 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-04-24T06:54:45.346 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.346 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.347 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.347 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.347 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.347 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.350 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.351 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-24T06:54:45.351 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-24T06:54:45.351 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.351 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.351 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.351 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:45.351 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.352 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.352 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.352 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.352 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.352 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.352 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.353 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_URL": { 2024-04-24T06:54:45.353 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_URL", 2024-04-24T06:54:45.353 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.354 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.354 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.354 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.354 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.355 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.355 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.355 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.355 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.355 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.356 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.356 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_USERNAME": { 2024-04-24T06:54:45.356 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-24T06:54:45.357 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.357 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.357 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.357 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-04-24T06:54:45.357 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.358 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.358 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.358 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.358 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.358 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.359 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.359 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-24T06:54:45.359 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-24T06:54:45.359 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.359 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.359 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.360 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.360 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.360 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.360 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.360 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.360 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.360 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.361 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.361 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-24T06:54:45.361 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-24T06:54:45.362 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.362 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.362 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.362 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.362 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.363 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.363 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.363 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.363 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.363 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.364 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.364 INFO:teuthology.orchestra.run.smithi005.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-24T06:54:45.364 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-24T06:54:45.364 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.364 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.365 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.365 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:45.365 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.365 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.365 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.366 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.366 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.366 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.366 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.367 INFO:teuthology.orchestra.run.smithi005.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-24T06:54:45.367 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-24T06:54:45.367 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.367 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.368 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.368 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.368 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.368 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.368 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.368 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.369 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.369 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.369 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.369 INFO:teuthology.orchestra.run.smithi005.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-24T06:54:45.370 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-24T06:54:45.370 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.370 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.370 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.370 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "[]", 2024-04-24T06:54:45.371 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.371 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.371 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.371 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.371 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.371 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.371 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.372 INFO:teuthology.orchestra.run.smithi005.stdout: "MULTICLUSTER_CONFIG": { 2024-04-24T06:54:45.372 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-24T06:54:45.372 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.372 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.373 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.373 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "{}", 2024-04-24T06:54:45.373 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.373 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.374 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.374 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.374 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.374 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.374 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.375 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_HOST": { 2024-04-24T06:54:45.375 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-24T06:54:45.375 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.375 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.376 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.376 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.376 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.376 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.377 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.377 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.377 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.377 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.378 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.378 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-24T06:54:45.378 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-24T06:54:45.379 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.379 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.379 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.379 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:45.379 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.379 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.380 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.380 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.380 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.380 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.380 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.381 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-24T06:54:45.381 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-24T06:54:45.381 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.381 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.381 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.381 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.382 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.382 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.382 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.382 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.382 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.382 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.383 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.383 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-24T06:54:45.383 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-24T06:54:45.383 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.383 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.384 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.384 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.384 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.384 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.384 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.384 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.385 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.385 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.385 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.385 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-24T06:54:45.386 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-24T06:54:45.386 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.386 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.387 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.387 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:45.387 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.387 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.388 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.388 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.388 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.388 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.388 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.389 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-24T06:54:45.389 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-24T06:54:45.389 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.389 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.390 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.390 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:45.390 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.390 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.390 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.391 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.391 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.391 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.391 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.392 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-24T06:54:45.392 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-24T06:54:45.392 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.392 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.393 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.393 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.393 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.393 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.393 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.394 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.394 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.394 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.394 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.395 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-24T06:54:45.395 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-24T06:54:45.395 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.395 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.395 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.395 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.396 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.396 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.396 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.396 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.396 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.396 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.397 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.397 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-24T06:54:45.398 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-24T06:54:45.398 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.398 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.398 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.398 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.399 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.399 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.399 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.399 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.399 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.400 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.400 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.400 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_ENABLED": { 2024-04-24T06:54:45.400 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-24T06:54:45.401 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.401 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.401 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.401 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:45.402 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.402 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.402 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.402 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.402 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.402 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.403 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.403 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-24T06:54:45.403 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-24T06:54:45.403 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.403 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.404 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.404 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-24T06:54:45.404 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.404 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.404 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.404 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.404 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.405 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.405 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.405 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-24T06:54:45.405 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-24T06:54:45.406 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:45.406 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.406 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.406 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-24T06:54:45.406 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.406 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.407 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.407 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.407 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.407 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.407 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.407 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-24T06:54:45.408 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-24T06:54:45.408 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:45.408 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.408 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.408 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8", 2024-04-24T06:54:45.408 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.409 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.409 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.409 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.409 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.410 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.410 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.410 INFO:teuthology.orchestra.run.smithi005.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-24T06:54:45.410 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-24T06:54:45.411 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:45.411 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.411 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.411 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "45", 2024-04-24T06:54:45.411 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.411 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.411 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.411 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.412 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.412 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.412 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.412 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ACCESS_KEY": { 2024-04-24T06:54:45.412 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-24T06:54:45.412 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.412 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.413 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.413 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.413 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.413 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.413 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.413 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.413 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.414 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.414 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.414 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-24T06:54:45.414 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-24T06:54:45.414 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.414 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.415 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.415 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-04-24T06:54:45.415 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.415 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.415 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.415 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.416 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.416 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.416 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.416 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SECRET_KEY": { 2024-04-24T06:54:45.416 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-24T06:54:45.417 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.417 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.417 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.417 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.417 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.417 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.417 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.418 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.418 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.418 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.418 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.418 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SSL_VERIFY": { 2024-04-24T06:54:45.419 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-24T06:54:45.419 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.419 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.419 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.420 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:45.420 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.420 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.420 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.420 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.420 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.420 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.421 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.421 INFO:teuthology.orchestra.run.smithi005.stdout: "UNSAFE_TLS_v1_2": { 2024-04-24T06:54:45.421 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-24T06:54:45.421 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.421 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.421 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.421 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.422 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.422 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.422 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.422 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.422 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.422 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.422 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.423 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-24T06:54:45.423 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-24T06:54:45.423 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:45.423 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.423 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.423 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-04-24T06:54:45.424 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.424 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.424 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.424 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.424 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.424 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.425 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.425 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-24T06:54:45.425 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-24T06:54:45.425 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:45.426 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.426 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.426 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-24T06:54:45.426 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.426 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.426 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.427 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.427 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.427 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.427 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.428 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-24T06:54:45.428 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-24T06:54:45.428 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:45.428 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.429 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.429 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-04-24T06:54:45.429 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.430 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.430 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.430 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.430 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.430 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.430 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.431 INFO:teuthology.orchestra.run.smithi005.stdout: "cross_origin_url": { 2024-04-24T06:54:45.431 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cross_origin_url", 2024-04-24T06:54:45.431 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.431 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.431 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.432 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.432 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.432 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.432 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.432 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.432 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.432 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.433 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.433 INFO:teuthology.orchestra.run.smithi005.stdout: "crt_file": { 2024-04-24T06:54:45.433 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crt_file", 2024-04-24T06:54:45.433 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.433 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.433 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.434 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.434 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.434 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.434 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.434 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.434 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.434 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.435 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.435 INFO:teuthology.orchestra.run.smithi005.stdout: "debug": { 2024-04-24T06:54:45.435 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "debug", 2024-04-24T06:54:45.435 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.435 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.435 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.436 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.436 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.436 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.436 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.436 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable/disable debug options", 2024-04-24T06:54:45.436 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.436 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.436 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.437 INFO:teuthology.orchestra.run.smithi005.stdout: "jwt_token_ttl": { 2024-04-24T06:54:45.437 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "jwt_token_ttl", 2024-04-24T06:54:45.437 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:45.437 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.437 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.437 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "28800", 2024-04-24T06:54:45.437 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.437 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.438 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.438 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.438 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.438 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.438 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.438 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-04-24T06:54:45.439 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-04-24T06:54:45.439 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.439 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.439 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.439 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.439 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.439 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.439 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.440 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.440 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.440 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.440 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.440 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:45.440 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:45.440 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.441 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.441 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.441 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.441 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.441 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.441 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.441 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.441 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.442 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.442 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.442 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.442 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.443 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.443 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.443 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.443 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.443 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.444 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:45.444 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:45.444 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.444 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.445 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.445 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.445 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.445 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.445 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.446 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.446 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.446 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.446 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.446 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:45.447 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:45.447 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.447 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.447 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.447 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:45.448 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.448 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.448 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.448 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.448 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.448 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.448 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.448 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.449 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.449 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.449 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.449 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.449 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.449 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.449 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.449 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:45.450 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:45.450 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.450 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.450 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.450 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.450 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.451 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.451 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.451 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.451 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.451 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.451 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.451 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.451 INFO:teuthology.orchestra.run.smithi005.stdout: "motd": { 2024-04-24T06:54:45.452 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "motd", 2024-04-24T06:54:45.452 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.452 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.452 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.452 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.452 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.452 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.452 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.453 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "The message of the day", 2024-04-24T06:54:45.453 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.453 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.453 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.453 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect_resolve_ip_addr": { 2024-04-24T06:54:45.454 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "redirect_resolve_ip_addr", 2024-04-24T06:54:45.454 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.454 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.454 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.454 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.454 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.454 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.455 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.455 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.455 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.455 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.455 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.455 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-04-24T06:54:45.456 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-04-24T06:54:45.456 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.456 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.456 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.456 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-04-24T06:54:45.456 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.456 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.456 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.457 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.457 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.457 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.457 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.457 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.457 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-04-24T06:54:45.458 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-04-24T06:54:45.458 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:45.458 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.458 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.458 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8080", 2024-04-24T06:54:45.458 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.460 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-04-24T06:54:45.460 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-04-24T06:54:45.460 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.460 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.462 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.462 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.462 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.462 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl_server_port": { 2024-04-24T06:54:45.462 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl_server_port", 2024-04-24T06:54:45.462 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8443", 2024-04-24T06:54:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-04-24T06:54:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-04-24T06:54:45.465 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.465 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.465 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.465 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "redirect", 2024-04-24T06:54:45.465 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.465 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.466 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.466 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.466 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect" 2024-04-24T06:54:45.466 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.466 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.466 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-04-24T06:54:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-04-24T06:54:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-04-24T06:54:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-04-24T06:54:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-04-24T06:54:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.469 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.469 INFO:teuthology.orchestra.run.smithi005.stdout: "url_prefix": { 2024-04-24T06:54:45.469 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url_prefix", 2024-04-24T06:54:45.469 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.469 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.470 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.470 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.470 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.470 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.470 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.470 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.471 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.471 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.471 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.471 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.472 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.472 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:45.472 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2024-04-24T06:54:45.473 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:45.473 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:45.473 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:45.473 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_monitoring": { 2024-04-24T06:54:45.474 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_monitoring", 2024-04-24T06:54:45.474 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.474 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.474 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.475 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:45.475 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.475 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.475 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.475 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "monitor device health metrics", 2024-04-24T06:54:45.476 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.476 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.476 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.476 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:45.476 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:45.477 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.477 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.477 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.477 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.477 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.477 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.478 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.478 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.478 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.478 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.479 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.479 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.479 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.479 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.479 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.480 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.480 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.480 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.480 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:45.481 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:45.481 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.481 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.482 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.482 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.482 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.482 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.483 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.483 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.483 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.483 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.483 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.484 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:45.484 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:45.484 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.484 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.484 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "mark_out_threshold": { 2024-04-24T06:54:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mark_out_threshold", 2024-04-24T06:54:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2419200", 2024-04-24T06:54:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-24T06:54:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": { 2024-04-24T06:54:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_name", 2024-04-24T06:54:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "device_health_metrics", 2024-04-24T06:54:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-24T06:54:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "retention_period": { 2024-04-24T06:54:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retention_period", 2024-04-24T06:54:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15552000", 2024-04-24T06:54:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain device health metrics", 2024-04-24T06:54:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_frequency": { 2024-04-24T06:54:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_frequency", 2024-04-24T06:54:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-04-24T06:54:45.497 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.497 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.497 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.497 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-24T06:54:45.497 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.497 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.498 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.498 INFO:teuthology.orchestra.run.smithi005.stdout: "self_heal": { 2024-04-24T06:54:45.498 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "self_heal", 2024-04-24T06:54:45.498 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.498 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.498 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.499 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:45.499 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.499 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.499 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.499 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-24T06:54:45.499 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.499 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.499 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.500 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-04-24T06:54:45.500 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-04-24T06:54:45.500 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:45.500 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.500 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.500 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-04-24T06:54:45.501 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.501 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.501 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.501 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and check device health", 2024-04-24T06:54:45.501 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.501 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.502 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.502 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_threshold": { 2024-04-24T06:54:45.502 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_threshold", 2024-04-24T06:54:45.502 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:45.503 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.503 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.503 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7257600", 2024-04-24T06:54:45.503 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.503 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.503 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.504 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-24T06:54:45.504 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.504 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.504 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.504 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.504 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.505 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:45.505 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "diskprediction_local", 2024-04-24T06:54:45.505 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:45.505 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:45.505 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:45.505 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:45.505 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:45.506 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.506 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.506 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.506 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.506 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.506 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.507 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.507 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.507 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.507 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.507 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.507 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.507 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.508 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.508 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.508 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.508 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.508 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.509 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:45.509 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:45.509 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.509 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.510 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.510 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.510 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.510 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.511 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.511 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.511 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.511 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.512 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.512 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:45.512 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:45.513 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.513 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.513 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.513 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:45.514 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.514 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.514 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.515 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.515 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.515 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.515 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.516 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.516 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.516 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.516 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.516 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.517 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.517 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.517 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:45.517 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:45.518 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.518 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.518 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.518 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.518 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.519 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.519 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.519 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.519 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.519 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.519 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: "predict_interval": { 2024-04-24T06:54:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predict_interval", 2024-04-24T06:54:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-04-24T06:54:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.522 INFO:teuthology.orchestra.run.smithi005.stdout: "predictor_model": { 2024-04-24T06:54:45.522 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predictor_model", 2024-04-24T06:54:45.522 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.522 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.522 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.522 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "prophetstor", 2024-04-24T06:54:45.523 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.523 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.523 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.523 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.523 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.523 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.523 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.524 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-04-24T06:54:45.524 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-04-24T06:54:45.524 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.524 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.524 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.524 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-04-24T06:54:45.524 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.525 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.525 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.525 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.525 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.525 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.525 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.526 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.526 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:45.526 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "influx", 2024-04-24T06:54:45.526 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": false, 2024-04-24T06:54:45.527 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "influxdb python module not found", 2024-04-24T06:54:45.527 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:45.527 INFO:teuthology.orchestra.run.smithi005.stdout: "batch_size": { 2024-04-24T06:54:45.527 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "batch_size", 2024-04-24T06:54:45.527 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:45.528 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.528 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.528 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5000", 2024-04-24T06:54:45.528 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.529 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.529 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.529 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-24T06:54:45.529 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.529 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.530 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.530 INFO:teuthology.orchestra.run.smithi005.stdout: "database": { 2024-04-24T06:54:45.530 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "database", 2024-04-24T06:54:45.531 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.531 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.531 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.531 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "ceph", 2024-04-24T06:54:45.532 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.532 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.532 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.532 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-24T06:54:45.532 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.533 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.533 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.533 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": { 2024-04-24T06:54:45.533 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hostname", 2024-04-24T06:54:45.533 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.534 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.534 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.534 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.534 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.534 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.534 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.535 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server hostname", 2024-04-24T06:54:45.535 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.535 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.535 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.535 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-04-24T06:54:45.536 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-04-24T06:54:45.536 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:45.536 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.536 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.536 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "30", 2024-04-24T06:54:45.536 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "5", 2024-04-24T06:54:45.536 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.537 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.537 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-24T06:54:45.537 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.537 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.537 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.538 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:45.538 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:45.538 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.538 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.538 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.538 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.538 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.539 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.539 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.539 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.539 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.539 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.540 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.540 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.540 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.540 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.540 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.540 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.541 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.541 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.541 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:45.541 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:45.541 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.542 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.542 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.542 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.542 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.542 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.542 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.543 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.543 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.543 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.543 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.543 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:45.544 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:45.544 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.544 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.544 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.544 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:45.544 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.544 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.545 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.545 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.545 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.545 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.545 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.545 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.546 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.546 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.546 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.546 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.546 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.546 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.547 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:45.547 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:45.547 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.547 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.547 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.548 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.548 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.548 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.548 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.548 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.548 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.548 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.548 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.549 INFO:teuthology.orchestra.run.smithi005.stdout: "password": { 2024-04-24T06:54:45.549 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "password", 2024-04-24T06:54:45.549 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.549 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.549 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.549 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.549 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.550 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.550 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.550 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "password of InfluxDB server user", 2024-04-24T06:54:45.550 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.550 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.550 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.553 INFO:teuthology.orchestra.run.smithi005.stdout: "port": { 2024-04-24T06:54:45.554 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "port", 2024-04-24T06:54:45.554 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:45.554 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.554 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.554 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8086", 2024-04-24T06:54:45.554 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.554 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.555 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.555 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server port", 2024-04-24T06:54:45.555 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.555 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.555 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.555 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-04-24T06:54:45.555 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-04-24T06:54:45.556 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.556 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.556 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.556 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "false", 2024-04-24T06:54:45.556 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.556 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.556 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.557 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T06:54:45.557 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.557 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.557 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.557 INFO:teuthology.orchestra.run.smithi005.stdout: "threads": { 2024-04-24T06:54:45.558 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threads", 2024-04-24T06:54:45.558 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:45.558 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.558 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.558 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-04-24T06:54:45.558 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-04-24T06:54:45.559 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "32", 2024-04-24T06:54:45.559 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.559 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-24T06:54:45.559 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.559 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.559 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.560 INFO:teuthology.orchestra.run.smithi005.stdout: "username": { 2024-04-24T06:54:45.560 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "username", 2024-04-24T06:54:45.560 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.560 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.560 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.560 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.561 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.561 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.561 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.561 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "username of InfluxDB server user", 2024-04-24T06:54:45.561 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.561 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.562 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.562 INFO:teuthology.orchestra.run.smithi005.stdout: "verify_ssl": { 2024-04-24T06:54:45.562 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "verify_ssl", 2024-04-24T06:54:45.562 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.563 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.563 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.563 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "true", 2024-04-24T06:54:45.563 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.563 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.564 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.564 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T06:54:45.564 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.564 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.564 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.565 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.565 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:45.565 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "insights", 2024-04-24T06:54:45.565 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:45.566 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:45.566 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:45.566 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:45.566 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:45.566 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.566 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.567 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.567 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.567 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.567 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.567 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.567 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.567 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.568 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.568 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.568 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.568 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.568 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.568 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.568 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.569 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.569 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.569 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:45.569 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:45.569 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.570 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.570 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.570 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.570 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.570 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.571 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.571 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.571 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.571 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.571 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.572 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:45.572 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:45.572 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.572 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.572 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.572 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:45.572 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.572 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.573 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.573 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.573 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.573 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.573 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.573 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.573 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.574 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.574 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.574 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.574 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.574 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.575 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:45.575 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:45.575 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.575 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.575 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.575 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.576 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.576 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.576 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.576 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.576 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.576 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.577 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.577 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.577 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:45.577 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "iostat", 2024-04-24T06:54:45.578 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:45.578 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:45.578 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:45.578 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:45.578 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:45.578 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.578 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.579 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.579 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.579 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.579 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.579 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.579 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.579 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.579 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.579 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.580 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.580 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.580 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.580 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.580 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.580 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.580 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.581 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:45.581 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:45.581 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.581 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.581 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.581 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.581 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.581 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.581 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.582 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.582 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.582 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.582 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.582 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:45.582 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:45.582 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.583 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.583 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.583 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:45.583 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.583 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.583 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.583 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.584 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.584 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.584 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.584 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.584 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.584 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.585 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.585 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.585 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.585 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.586 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:45.586 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:45.586 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.586 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.586 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.586 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.587 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.587 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.587 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.587 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.588 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.588 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.588 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.588 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.588 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:45.589 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "k8sevents", 2024-04-24T06:54:45.589 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:45.589 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:45.590 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:45.590 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_event_retention_days": { 2024-04-24T06:54:45.590 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ceph_event_retention_days", 2024-04-24T06:54:45.590 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:45.590 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.591 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.591 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7", 2024-04-24T06:54:45.591 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.591 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.591 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.591 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-24T06:54:45.591 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.592 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.592 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.592 INFO:teuthology.orchestra.run.smithi005.stdout: "config_check_secs": { 2024-04-24T06:54:45.592 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_check_secs", 2024-04-24T06:54:45.592 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:45.592 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.593 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.593 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-24T06:54:45.593 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "10", 2024-04-24T06:54:45.593 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.593 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.593 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-24T06:54:45.593 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.593 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.594 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.594 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:45.594 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:45.594 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.594 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.595 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.595 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.595 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.595 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.595 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.595 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.596 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.596 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.596 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.596 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.596 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.596 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.596 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.597 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.597 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.597 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.597 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:45.597 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:45.598 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.598 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.598 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.598 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.598 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.598 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.599 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.599 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.599 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.599 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.599 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.599 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:45.600 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:45.600 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.600 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.600 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.600 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:45.600 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.600 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.601 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.601 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.601 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.601 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.601 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.601 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.601 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.601 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.602 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.602 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.602 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.602 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.602 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:45.602 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:45.603 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.603 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.603 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.603 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.603 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.603 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.603 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.604 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.604 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.604 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.604 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.604 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.604 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:45.605 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "localpool", 2024-04-24T06:54:45.605 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:45.605 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:45.605 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:45.605 INFO:teuthology.orchestra.run.smithi005.stdout: "failure_domain": { 2024-04-24T06:54:45.605 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "failure_domain", 2024-04-24T06:54:45.606 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.606 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.606 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.606 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "host", 2024-04-24T06:54:45.606 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.606 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.606 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.606 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "failure domain for any created local pool", 2024-04-24T06:54:45.607 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-24T06:54:45.607 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.607 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.607 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:45.607 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:45.607 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.607 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.608 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.608 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.608 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.608 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.608 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.608 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.608 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.608 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.608 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.609 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.609 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.609 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.609 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.609 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.609 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.609 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.609 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:45.610 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:45.610 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.610 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.610 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.610 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.610 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.610 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.610 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.611 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.611 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.611 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.611 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.611 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:45.611 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:45.611 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.612 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.612 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.612 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:45.612 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.612 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.612 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.612 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.612 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.612 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.613 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.613 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.613 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.613 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.613 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.614 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.614 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.614 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.614 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:45.614 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:45.615 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.615 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.615 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.615 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.615 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.615 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.616 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.616 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.616 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.616 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.616 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.617 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": { 2024-04-24T06:54:45.617 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_size", 2024-04-24T06:54:45.617 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:45.617 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.617 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.618 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.618 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.618 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.618 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.618 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default min_size for any created local pool", 2024-04-24T06:54:45.618 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-24T06:54:45.619 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.619 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.619 INFO:teuthology.orchestra.run.smithi005.stdout: "num_rep": { 2024-04-24T06:54:45.619 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "num_rep", 2024-04-24T06:54:45.620 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:45.620 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.620 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.620 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3", 2024-04-24T06:54:45.620 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.620 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.620 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.621 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default replica count for any created local pool", 2024-04-24T06:54:45.621 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.621 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.621 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.621 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": { 2024-04-24T06:54:45.622 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_num", 2024-04-24T06:54:45.622 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:45.622 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.622 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.622 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "128", 2024-04-24T06:54:45.622 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.623 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.623 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.623 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default pg_num for any created local pool", 2024-04-24T06:54:45.623 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.623 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.623 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.624 INFO:teuthology.orchestra.run.smithi005.stdout: "prefix": { 2024-04-24T06:54:45.624 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prefix", 2024-04-24T06:54:45.624 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.624 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.624 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.625 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.625 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.625 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.625 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.625 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name prefix for any created local pool", 2024-04-24T06:54:45.625 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.626 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.626 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.626 INFO:teuthology.orchestra.run.smithi005.stdout: "subtree": { 2024-04-24T06:54:45.626 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "subtree", 2024-04-24T06:54:45.626 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.626 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.626 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.627 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "rack", 2024-04-24T06:54:45.627 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.627 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.627 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.627 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-24T06:54:45.627 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-24T06:54:45.627 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.627 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.628 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.628 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:45.628 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mds_autoscaler", 2024-04-24T06:54:45.628 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:45.628 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:45.628 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:45.629 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:45.629 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:45.629 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.629 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.629 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.629 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.629 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.630 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.630 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.630 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.630 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.630 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.630 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.630 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.630 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.631 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.631 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.631 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.631 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.631 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.631 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:45.631 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:45.631 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.632 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.632 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.632 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.632 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.632 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.632 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.632 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.632 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.633 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.633 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.633 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:45.633 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:45.633 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.633 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.633 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.633 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:45.634 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.634 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.634 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.634 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.634 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.634 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.634 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.634 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.635 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.635 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.635 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.635 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.635 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.635 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.635 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:45.636 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:45.636 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.636 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.636 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.636 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.636 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.636 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.637 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.637 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.637 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.637 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.637 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.637 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.637 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:45.638 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirroring", 2024-04-24T06:54:45.638 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:45.638 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:45.638 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:45.638 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:45.638 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:45.638 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.638 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.639 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.639 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.639 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.639 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.639 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.639 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.639 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.639 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.640 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.640 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.640 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.640 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.640 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.640 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.640 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.640 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.641 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:45.641 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:45.641 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.641 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.641 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.641 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.641 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.642 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.642 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.642 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.642 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.642 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.642 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.643 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:45.643 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:45.643 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.643 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.643 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.643 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:45.643 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.644 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.644 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.644 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.644 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.644 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.644 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.644 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.645 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.645 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.645 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.645 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.645 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.646 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.646 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:45.646 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:45.646 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.646 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.647 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.647 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.647 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.647 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.647 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.648 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.648 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.648 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.648 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.648 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.648 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.649 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:45.649 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "nfs", 2024-04-24T06:54:45.649 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:45.649 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:45.650 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:45.650 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:45.650 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:45.650 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.650 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.650 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.650 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.651 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.651 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.651 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.651 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.651 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.651 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.651 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.652 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.652 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.652 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.652 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.652 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.652 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.652 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.653 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:45.653 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:45.653 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.653 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.653 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.653 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.654 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.654 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.654 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.654 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.654 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.654 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.654 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.655 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:45.655 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:45.655 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.655 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.655 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.655 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:45.656 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.656 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.656 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.656 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.656 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.656 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.656 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.657 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.657 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.657 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.657 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.657 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.657 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.657 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.658 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:45.658 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:45.658 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.658 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.658 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.658 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.658 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.659 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.659 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.659 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.659 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.659 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.659 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.659 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.660 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.660 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:45.660 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-04-24T06:54:45.660 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:45.660 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:45.661 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:45.661 INFO:teuthology.orchestra.run.smithi005.stdout: "fail_fs": { 2024-04-24T06:54:45.661 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "fail_fs", 2024-04-24T06:54:45.661 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.661 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.661 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.661 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.662 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.662 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.662 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.662 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-24T06:54:45.662 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.662 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.662 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.663 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:45.663 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:45.663 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.663 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.663 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.663 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.664 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.664 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.664 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.664 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.664 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.664 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.664 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.664 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.665 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.665 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.665 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.665 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.665 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.666 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.666 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.666 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:45.666 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:45.666 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.666 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.666 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.666 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.667 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.667 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.667 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.667 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.667 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.667 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.667 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.668 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:45.668 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:45.668 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.668 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.668 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.668 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:45.668 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.668 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.668 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.669 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.669 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.669 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.669 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.669 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.669 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.669 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.670 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.670 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.670 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.670 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.670 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.670 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:45.670 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:45.670 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.671 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.671 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.671 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.671 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.671 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.671 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.671 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.672 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.672 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.672 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.672 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator": { 2024-04-24T06:54:45.672 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-04-24T06:54:45.672 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.672 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.673 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.673 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.673 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.673 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.673 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.673 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2024-04-24T06:54:45.673 INFO:teuthology.orchestra.run.smithi005.stdout: "rook", 2024-04-24T06:54:45.674 INFO:teuthology.orchestra.run.smithi005.stdout: "test_orchestrator" 2024-04-24T06:54:45.674 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.674 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Orchestrator backend", 2024-04-24T06:54:45.674 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.674 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.674 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.674 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.674 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.675 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.675 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:45.675 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_perf_query", 2024-04-24T06:54:45.675 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:45.675 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:45.675 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:45.675 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:45.675 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:45.676 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.676 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.676 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.676 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.676 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.676 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.676 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.676 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.676 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.677 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.677 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.677 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.677 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.677 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.677 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.677 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.677 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.678 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.678 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:45.678 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:45.678 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.678 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.678 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.678 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.679 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.679 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.679 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.679 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.679 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.679 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.679 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.680 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:45.680 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:45.680 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.680 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.680 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.680 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:45.681 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.681 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.681 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.681 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.681 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.681 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.682 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.682 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.682 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.682 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.682 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.682 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.682 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.683 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.683 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:45.683 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:45.683 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.683 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.684 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.684 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.684 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.684 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.684 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.684 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.685 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.685 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.685 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.685 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.685 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:45.685 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_support", 2024-04-24T06:54:45.686 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:45.686 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:45.686 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:45.686 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:45.686 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:45.686 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.686 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.687 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.687 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.687 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.687 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.687 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.687 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.688 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.688 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.688 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.688 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.688 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.688 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.688 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.688 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.689 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.689 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.689 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:45.689 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:45.689 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.689 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.690 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.690 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.690 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.690 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.690 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.690 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.691 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.691 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.691 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.691 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:45.691 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:45.692 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.692 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.692 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.692 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:45.692 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.692 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.692 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.693 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.693 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.693 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.693 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.693 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.693 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.693 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.693 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.694 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.694 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.694 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.694 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:45.694 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:45.694 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.694 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.694 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.695 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.695 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.695 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.695 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.695 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.695 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.695 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.696 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.696 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.696 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.696 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:45.696 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_autoscaler", 2024-04-24T06:54:45.696 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:45.696 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:45.697 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:45.697 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:45.697 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:45.697 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.697 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.697 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.697 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.697 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.697 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.698 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.698 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.698 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.698 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.698 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.698 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.698 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.698 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.699 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.699 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.699 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.699 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.699 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:45.699 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:45.699 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.700 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.700 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.700 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.700 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.700 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.700 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.700 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.700 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.700 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.701 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.701 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:45.701 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:45.701 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.701 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.701 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.702 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:45.702 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.702 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.702 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.702 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.702 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.702 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.702 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.703 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.703 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.703 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.703 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.703 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.703 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.703 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.703 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:45.704 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:45.704 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.704 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.704 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.704 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.704 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.704 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.705 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.705 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.705 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.705 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.705 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.705 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-04-24T06:54:45.706 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-04-24T06:54:45.706 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:45.706 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.706 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.706 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-04-24T06:54:45.706 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.706 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.707 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.707 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.707 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.707 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.707 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.707 INFO:teuthology.orchestra.run.smithi005.stdout: "threshold": { 2024-04-24T06:54:45.707 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threshold", 2024-04-24T06:54:45.708 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-24T06:54:45.708 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.708 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.708 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-04-24T06:54:45.708 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1.0", 2024-04-24T06:54:45.708 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.708 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.708 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "scaling threshold", 2024-04-24T06:54:45.709 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-24T06:54:45.709 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.709 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.709 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.709 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.709 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:45.709 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "progress", 2024-04-24T06:54:45.709 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:45.710 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:45.710 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:45.710 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_pg_recovery_event": { 2024-04-24T06:54:45.710 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_pg_recovery_event", 2024-04-24T06:54:45.710 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.710 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.710 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.710 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.711 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.711 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.711 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.711 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-24T06:54:45.711 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.711 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.711 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.712 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-04-24T06:54:45.712 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-04-24T06:54:45.712 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.712 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.712 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.712 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:45.712 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.712 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.713 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.713 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.713 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.713 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.713 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.713 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:45.713 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:45.714 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.714 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.714 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.714 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.714 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.714 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.714 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.715 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.715 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.715 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.715 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.715 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.715 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.715 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.715 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.716 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.716 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.716 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.716 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:45.716 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:45.716 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.716 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.716 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.717 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.717 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.717 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.717 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.717 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.717 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.717 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.717 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.718 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:45.718 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:45.718 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.718 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.718 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.718 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:45.718 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.718 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.719 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.719 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.719 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.719 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.719 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.719 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.719 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.719 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.720 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.720 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.720 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.720 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.720 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:45.720 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:45.720 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.721 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.721 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.721 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.721 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.721 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.721 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.721 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.721 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.721 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.722 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.722 INFO:teuthology.orchestra.run.smithi005.stdout: "max_completed_events": { 2024-04-24T06:54:45.722 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_completed_events", 2024-04-24T06:54:45.722 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:45.722 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.722 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.722 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "50", 2024-04-24T06:54:45.723 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.723 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.723 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.723 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "number of past completed events to remember", 2024-04-24T06:54:45.723 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.723 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.723 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.724 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-04-24T06:54:45.724 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-04-24T06:54:45.724 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:45.724 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.724 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.724 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-04-24T06:54:45.724 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.725 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.725 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.725 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long the module is going to sleep", 2024-04-24T06:54:45.725 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.725 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.725 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.726 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.726 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.726 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:45.726 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus", 2024-04-24T06:54:45.726 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:45.726 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:45.726 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:45.727 INFO:teuthology.orchestra.run.smithi005.stdout: "cache": { 2024-04-24T06:54:45.727 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cache", 2024-04-24T06:54:45.727 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.727 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.727 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.727 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:45.727 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.727 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.727 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.728 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.728 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.728 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.728 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.728 INFO:teuthology.orchestra.run.smithi005.stdout: "exclude_perf_counters": { 2024-04-24T06:54:45.728 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "exclude_perf_counters", 2024-04-24T06:54:45.728 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.729 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.729 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.729 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:45.729 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.729 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.729 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.729 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-24T06:54:45.729 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-24T06:54:45.730 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.730 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.730 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:45.730 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:45.730 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.731 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.731 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.731 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.731 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.731 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.731 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.731 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.731 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.732 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.732 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.732 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.732 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.732 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.732 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.732 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.732 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.733 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.733 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:45.733 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:45.733 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.733 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.733 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.733 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.734 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.734 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.734 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.734 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.734 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.734 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.734 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.735 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.735 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:45.735 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:45.735 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.735 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.735 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.735 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:45.735 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.736 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.736 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.736 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.736 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.736 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.736 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.736 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.736 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.737 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.737 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.737 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.737 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.737 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.737 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:45.737 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:45.738 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.738 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.738 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.738 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.738 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.738 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.738 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.739 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.739 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.739 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.739 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.739 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools": { 2024-04-24T06:54:45.739 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools", 2024-04-24T06:54:45.739 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.740 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.740 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.740 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.740 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.740 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.740 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.740 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.741 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.741 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.741 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.741 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-24T06:54:45.741 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-24T06:54:45.741 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:45.741 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.742 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.742 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "300", 2024-04-24T06:54:45.742 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.742 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.742 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.742 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.742 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.742 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.743 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.743 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_interval": { 2024-04-24T06:54:45.743 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_interval", 2024-04-24T06:54:45.743 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-24T06:54:45.743 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.743 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.744 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15.0", 2024-04-24T06:54:45.744 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.744 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.744 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.744 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.744 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.744 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.744 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.745 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-04-24T06:54:45.745 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-04-24T06:54:45.745 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.745 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.745 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.745 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-04-24T06:54:45.746 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.746 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.746 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.746 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-24T06:54:45.746 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.746 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.746 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.747 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-04-24T06:54:45.747 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-04-24T06:54:45.747 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:45.747 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.747 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.747 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "9283", 2024-04-24T06:54:45.748 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.748 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.748 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.748 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-24T06:54:45.748 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.748 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.748 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.749 INFO:teuthology.orchestra.run.smithi005.stdout: "stale_cache_strategy": { 2024-04-24T06:54:45.749 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stale_cache_strategy", 2024-04-24T06:54:45.749 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.749 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.749 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.749 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "log", 2024-04-24T06:54:45.749 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.750 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.750 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.750 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.750 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.750 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.750 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.750 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.750 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-04-24T06:54:45.750 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-04-24T06:54:45.751 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.751 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.751 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.751 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "default", 2024-04-24T06:54:45.751 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.751 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.751 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.751 INFO:teuthology.orchestra.run.smithi005.stdout: "default", 2024-04-24T06:54:45.752 INFO:teuthology.orchestra.run.smithi005.stdout: "error" 2024-04-24T06:54:45.752 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.752 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.752 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.752 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.752 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.752 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-04-24T06:54:45.753 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-04-24T06:54:45.753 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:45.753 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.753 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.753 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-04-24T06:54:45.753 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-04-24T06:54:45.753 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-04-24T06:54:45.753 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.753 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.754 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.754 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.754 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.754 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.754 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.754 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:45.754 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2024-04-24T06:54:45.755 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:45.755 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:45.755 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:45.755 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:45.755 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:45.755 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.755 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.755 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.755 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.758 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.759 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.759 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.759 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.759 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.759 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.759 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.759 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.759 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.759 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.760 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.760 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.760 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.760 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.760 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:45.760 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:45.760 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.761 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.761 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.761 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.761 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.761 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.761 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.761 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.761 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.762 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.762 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.762 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:45.762 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:45.762 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.762 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.762 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.763 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:45.763 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.763 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.763 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.763 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.763 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.763 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.763 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.764 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.764 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.764 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.764 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.764 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.764 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.764 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.764 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:45.765 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:45.765 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.765 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.765 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.765 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.765 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.765 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.765 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.766 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.766 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.766 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.766 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.766 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_snap_create": { 2024-04-24T06:54:45.766 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_snap_create", 2024-04-24T06:54:45.766 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:45.766 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.767 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.767 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-24T06:54:45.767 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.767 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.767 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.767 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.767 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.767 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.768 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.768 INFO:teuthology.orchestra.run.smithi005.stdout: "mirror_snapshot_schedule": { 2024-04-24T06:54:45.768 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirror_snapshot_schedule", 2024-04-24T06:54:45.768 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.768 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.768 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.768 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.769 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.769 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.769 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.769 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.769 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.769 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.769 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.770 INFO:teuthology.orchestra.run.smithi005.stdout: "trash_purge_schedule": { 2024-04-24T06:54:45.770 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "trash_purge_schedule", 2024-04-24T06:54:45.770 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.770 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.770 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.771 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.771 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.771 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.771 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.771 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.771 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.771 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.771 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.772 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.772 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.772 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:45.772 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "restful", 2024-04-24T06:54:45.772 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:45.772 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:45.772 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:45.772 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_auth": { 2024-04-24T06:54:45.773 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_auth", 2024-04-24T06:54:45.773 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.773 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.773 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.773 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:45.773 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.773 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.773 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.774 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.774 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.774 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.774 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.774 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-04-24T06:54:45.774 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-04-24T06:54:45.774 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.775 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.775 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.775 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.775 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.775 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.775 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.775 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.775 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.776 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.776 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.776 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:45.776 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:45.776 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.776 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.776 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.777 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.777 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.777 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.777 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.777 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.777 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.777 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.777 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.778 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.778 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.778 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.778 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.778 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.778 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.779 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.779 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:45.779 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:45.779 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.779 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.779 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.780 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.780 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.780 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.780 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.780 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.780 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.780 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.780 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.781 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:45.781 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:45.781 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.781 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.781 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.781 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:45.781 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.781 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.782 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.782 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.782 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.782 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.782 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.782 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.782 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.782 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.783 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.783 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.783 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.783 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.783 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:45.783 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:45.783 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.784 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.784 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.784 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.784 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.784 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.784 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.784 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.784 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.785 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.785 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.785 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-04-24T06:54:45.785 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-04-24T06:54:45.785 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.785 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.786 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.786 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.786 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.786 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.786 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.786 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.786 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.786 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.787 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.787 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-04-24T06:54:45.787 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-04-24T06:54:45.787 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.787 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.787 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.787 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.788 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.788 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.788 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.788 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.788 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.788 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.788 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.788 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.788 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:45.789 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rgw", 2024-04-24T06:54:45.789 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:45.789 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:45.789 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:45.789 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:45.789 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:45.790 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.790 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.790 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.790 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.790 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.790 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.790 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.790 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.790 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.791 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.791 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.791 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.791 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.791 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.791 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.791 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.791 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.792 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.792 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:45.792 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:45.792 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.792 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.793 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.793 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.793 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.793 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.793 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.793 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.794 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.794 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.794 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.794 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:45.794 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:45.794 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.795 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.795 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.795 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:45.795 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.795 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.795 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.795 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.796 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.796 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.796 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.796 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.796 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.796 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.796 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.797 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.797 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.797 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.797 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:45.797 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:45.797 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.798 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.798 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.798 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.798 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.798 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.798 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.798 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.799 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.799 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.799 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.799 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.799 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:45.799 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rook", 2024-04-24T06:54:45.800 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:45.800 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:45.800 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:45.800 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:45.800 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:45.800 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.800 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.800 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.801 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.801 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.801 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.801 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.801 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.801 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.801 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.802 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.802 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.802 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.802 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.802 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.802 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.802 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.802 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.803 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:45.803 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:45.803 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.803 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.803 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.803 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.804 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.804 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.804 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.804 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.804 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.804 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.804 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.805 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:45.805 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:45.805 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.805 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.805 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.805 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:45.805 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.805 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.806 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.806 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.806 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.806 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.806 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.806 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.806 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.807 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.807 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.807 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.807 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.807 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.807 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:45.808 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:45.808 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.808 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.808 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.808 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.808 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.808 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.809 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.809 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.809 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.809 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.809 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.810 INFO:teuthology.orchestra.run.smithi005.stdout: "storage_class": { 2024-04-24T06:54:45.810 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "storage_class", 2024-04-24T06:54:45.810 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.810 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.810 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.810 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "local", 2024-04-24T06:54:45.810 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.811 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.811 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.811 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-24T06:54:45.811 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.811 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.811 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.811 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.812 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.812 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:45.812 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "selftest", 2024-04-24T06:54:45.812 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:45.812 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:45.812 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:45.812 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:45.813 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:45.813 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.813 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.813 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.813 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.813 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.813 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.814 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.814 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.814 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.814 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.814 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.814 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.815 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.815 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.815 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.815 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.815 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.815 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.815 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:45.816 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:45.816 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.816 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.816 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.816 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.816 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.816 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.816 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.817 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.817 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.817 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.817 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.817 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:45.818 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:45.818 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.818 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.818 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.818 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:45.818 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.818 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.818 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.819 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.819 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.819 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.819 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.819 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.819 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.819 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.820 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.820 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.820 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.820 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.820 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:45.820 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:45.820 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.821 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.821 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.821 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.821 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.821 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.821 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.821 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.821 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.822 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.822 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.822 INFO:teuthology.orchestra.run.smithi005.stdout: "roption1": { 2024-04-24T06:54:45.822 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption1", 2024-04-24T06:54:45.822 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.823 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.823 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.823 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.823 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.823 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.823 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.823 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.823 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.824 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.824 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.824 INFO:teuthology.orchestra.run.smithi005.stdout: "roption2": { 2024-04-24T06:54:45.824 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption2", 2024-04-24T06:54:45.824 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.824 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.824 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.825 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "xyz", 2024-04-24T06:54:45.825 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.825 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.825 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.825 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.825 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.825 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.825 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.826 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption1": { 2024-04-24T06:54:45.826 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption1", 2024-04-24T06:54:45.826 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.826 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.826 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.826 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.827 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.827 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.827 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.827 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.827 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.827 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.827 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.828 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption2": { 2024-04-24T06:54:45.828 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption2", 2024-04-24T06:54:45.828 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:45.828 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.828 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.828 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.828 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.829 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.829 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.829 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.829 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.829 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.829 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.829 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption3": { 2024-04-24T06:54:45.830 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption3", 2024-04-24T06:54:45.830 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-24T06:54:45.830 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.830 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.830 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.830 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.830 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.830 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.831 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.831 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.831 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.831 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.831 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption4": { 2024-04-24T06:54:45.831 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption4", 2024-04-24T06:54:45.832 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.832 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.832 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.832 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.832 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.832 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.832 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.832 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.833 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.833 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.833 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.833 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption5": { 2024-04-24T06:54:45.833 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption5", 2024-04-24T06:54:45.833 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.833 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.834 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.834 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.834 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.834 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.834 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.834 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.834 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.834 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.835 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.835 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption6": { 2024-04-24T06:54:45.835 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption6", 2024-04-24T06:54:45.835 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.835 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.835 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.835 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:45.836 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.836 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.836 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.836 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.836 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.836 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.836 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.837 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption7": { 2024-04-24T06:54:45.837 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption7", 2024-04-24T06:54:45.837 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:45.837 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.837 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.837 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.837 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-04-24T06:54:45.837 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "42", 2024-04-24T06:54:45.838 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.838 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.838 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.838 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.838 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.839 INFO:teuthology.orchestra.run.smithi005.stdout: "testkey": { 2024-04-24T06:54:45.839 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testkey", 2024-04-24T06:54:45.839 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.839 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.839 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.839 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.840 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.840 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.840 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.840 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.840 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.841 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.841 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.841 INFO:teuthology.orchestra.run.smithi005.stdout: "testlkey": { 2024-04-24T06:54:45.841 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testlkey", 2024-04-24T06:54:45.842 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.842 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.842 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.843 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.843 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.843 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.843 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.843 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.844 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.844 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.844 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.844 INFO:teuthology.orchestra.run.smithi005.stdout: "testnewline": { 2024-04-24T06:54:45.845 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testnewline", 2024-04-24T06:54:45.845 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.845 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.845 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.846 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.846 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.846 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.846 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.846 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.847 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.847 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.847 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.847 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.847 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:45.848 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2024-04-24T06:54:45.848 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:45.848 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:45.848 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:45.849 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_m_granularity": { 2024-04-24T06:54:45.849 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_m_granularity", 2024-04-24T06:54:45.849 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.849 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.849 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.850 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.850 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.850 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.850 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.850 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow minute scheduled snapshots", 2024-04-24T06:54:45.851 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.851 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.851 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.851 INFO:teuthology.orchestra.run.smithi005.stdout: "dump_on_update": { 2024-04-24T06:54:45.851 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dump_on_update", 2024-04-24T06:54:45.851 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.852 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.852 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.852 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.852 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.852 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.852 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.853 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "dump database to debug log on update", 2024-04-24T06:54:45.853 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.853 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.853 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.854 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:45.854 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:45.854 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.854 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.855 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.855 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.855 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.855 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.855 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.856 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.856 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.856 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.856 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.856 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.856 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.857 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.857 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.857 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.857 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.857 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.858 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:45.858 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:45.858 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.858 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.858 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.858 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.858 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.859 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.859 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.859 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.859 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.859 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.860 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.860 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:45.860 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:45.860 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.860 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.860 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.861 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:45.861 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.861 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.861 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.861 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.861 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.861 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.862 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.862 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.862 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.862 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.862 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.862 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.863 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.863 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.863 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:45.863 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:45.863 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.863 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.863 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.864 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.864 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.864 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.864 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.864 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.864 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.864 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.865 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.865 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.865 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:45.865 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stats", 2024-04-24T06:54:45.865 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:45.866 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:45.866 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:45.866 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:45.866 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:45.866 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.866 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.867 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.867 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.867 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.867 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.867 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.867 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.867 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.867 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.868 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.868 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.868 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.868 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.868 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.869 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.869 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.869 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.869 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:45.869 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:45.869 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.869 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.870 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.870 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.870 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.870 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.870 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.870 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.870 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.871 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.871 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.871 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:45.871 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:45.871 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.871 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.871 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.872 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:45.872 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.872 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.872 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.872 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.872 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.872 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.872 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.873 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.873 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.873 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.873 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.873 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.873 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.873 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.874 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:45.874 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:45.874 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.874 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.874 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.874 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.875 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.875 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.875 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.875 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.875 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.875 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.875 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.875 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.876 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.876 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:45.876 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "status", 2024-04-24T06:54:45.877 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:45.877 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:45.877 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:45.877 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:45.878 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:45.878 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.878 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.878 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.878 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.878 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.878 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.879 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.879 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.879 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.879 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.879 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.879 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.879 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.879 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.880 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.880 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.880 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.880 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.880 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:45.880 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:45.881 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.881 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.881 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.881 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.881 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.881 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.881 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.882 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.882 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.882 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.882 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.882 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:45.882 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:45.882 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.883 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.883 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.883 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:45.883 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.883 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.883 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.883 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.884 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.884 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.884 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.884 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.884 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.884 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.884 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.885 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.885 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.885 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.885 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:45.885 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:45.885 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.885 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.886 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.886 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.886 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.886 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.889 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.889 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.889 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.890 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.890 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.890 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.890 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.891 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:45.891 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telegraf", 2024-04-24T06:54:45.891 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:45.892 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:45.892 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:45.892 INFO:teuthology.orchestra.run.smithi005.stdout: "address": { 2024-04-24T06:54:45.892 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "address", 2024-04-24T06:54:45.893 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.893 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.893 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.893 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-24T06:54:45.893 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.893 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.893 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.894 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.894 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.894 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.894 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.895 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-04-24T06:54:45.895 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-04-24T06:54:45.895 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:45.895 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.895 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.895 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15", 2024-04-24T06:54:45.895 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.896 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.896 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.896 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.896 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.896 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.896 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.897 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:45.897 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:45.897 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.897 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.897 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.897 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.897 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.898 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.898 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.898 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.898 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.898 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.899 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.899 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.899 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.899 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.899 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.899 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.899 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.899 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.900 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:45.900 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:45.900 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.900 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.900 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.901 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.901 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.901 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.901 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.901 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.901 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.901 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.901 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.902 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:45.902 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:45.902 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.902 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.902 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.902 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:45.903 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.903 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.903 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.903 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.903 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.903 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.904 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.904 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.904 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.904 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.904 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.905 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.905 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.905 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.905 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:45.906 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:45.906 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.906 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.906 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.906 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.906 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.906 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.907 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.907 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.907 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.907 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.907 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.907 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.907 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:45.908 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telemetry", 2024-04-24T06:54:45.908 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:45.908 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:45.908 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:45.908 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_basic": { 2024-04-24T06:54:45.909 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_basic", 2024-04-24T06:54:45.909 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.909 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.909 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.909 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:45.909 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.909 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.909 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.910 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-24T06:54:45.910 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.910 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.910 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.910 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_crash": { 2024-04-24T06:54:45.910 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_crash", 2024-04-24T06:54:45.911 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.911 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.911 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.911 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:45.911 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.911 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.911 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.911 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-24T06:54:45.912 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.912 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.912 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.912 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_device": { 2024-04-24T06:54:45.912 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_device", 2024-04-24T06:54:45.912 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.913 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.913 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.913 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:45.913 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.913 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.913 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.913 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-24T06:54:45.913 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.914 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.914 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.914 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_ident": { 2024-04-24T06:54:45.914 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_ident", 2024-04-24T06:54:45.914 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.915 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.915 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.916 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.916 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.916 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.916 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.916 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-24T06:54:45.916 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.916 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.916 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.917 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_perf": { 2024-04-24T06:54:45.917 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_perf", 2024-04-24T06:54:45.917 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.917 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.917 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.918 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.918 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.918 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.918 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.918 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-24T06:54:45.918 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.918 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.919 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.919 INFO:teuthology.orchestra.run.smithi005.stdout: "contact": { 2024-04-24T06:54:45.919 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "contact", 2024-04-24T06:54:45.919 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.919 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.919 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.919 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.920 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.920 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.920 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.920 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.920 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.920 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.921 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.921 INFO:teuthology.orchestra.run.smithi005.stdout: "description": { 2024-04-24T06:54:45.921 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "description", 2024-04-24T06:54:45.921 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.921 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.921 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.921 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.921 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.922 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.922 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.922 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.922 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.922 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.922 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.923 INFO:teuthology.orchestra.run.smithi005.stdout: "device_url": { 2024-04-24T06:54:45.923 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_url", 2024-04-24T06:54:45.923 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.923 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.923 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.923 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-24T06:54:45.924 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.924 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.924 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.924 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.924 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.924 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.925 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.925 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-04-24T06:54:45.925 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-04-24T06:54:45.925 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.925 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.926 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.926 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.926 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.926 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.926 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.927 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.927 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.927 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.927 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.928 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-04-24T06:54:45.928 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-04-24T06:54:45.928 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:45.928 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.928 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.928 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "24", 2024-04-24T06:54:45.929 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "8", 2024-04-24T06:54:45.929 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.929 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.929 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.929 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.929 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.930 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.930 INFO:teuthology.orchestra.run.smithi005.stdout: "last_opt_revision": { 2024-04-24T06:54:45.930 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "last_opt_revision", 2024-04-24T06:54:45.930 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:45.931 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.931 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.931 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1", 2024-04-24T06:54:45.931 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.931 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.931 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.932 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.932 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.932 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.932 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.932 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard": { 2024-04-24T06:54:45.932 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard", 2024-04-24T06:54:45.933 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.933 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.933 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.933 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.933 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.933 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.934 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.934 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.934 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.934 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.934 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.934 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard_description": { 2024-04-24T06:54:45.935 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard_description", 2024-04-24T06:54:45.935 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.935 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.935 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.935 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.935 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.935 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.935 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.936 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.936 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.936 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.936 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.936 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:45.936 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:45.937 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.937 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.937 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.937 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.937 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.937 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.937 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.938 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.938 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.938 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.938 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.938 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.938 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.938 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.939 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.939 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.939 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.939 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.939 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:45.939 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:45.940 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.940 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.940 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.940 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.940 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.940 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.941 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.941 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.941 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.941 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.941 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.941 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:45.941 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:45.942 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.942 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.942 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.942 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:45.942 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.942 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.942 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.943 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.943 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.943 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.943 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.943 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.943 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.944 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.944 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.944 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.944 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.944 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.945 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:45.945 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:45.945 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.945 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.945 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.945 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.945 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.946 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.946 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.946 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.946 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.946 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.946 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.947 INFO:teuthology.orchestra.run.smithi005.stdout: "organization": { 2024-04-24T06:54:45.947 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "organization", 2024-04-24T06:54:45.947 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.947 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.947 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.947 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.947 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.948 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.948 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.948 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.948 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.948 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.948 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.949 INFO:teuthology.orchestra.run.smithi005.stdout: "proxy": { 2024-04-24T06:54:45.949 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "proxy", 2024-04-24T06:54:45.949 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.949 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.949 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.949 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.950 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.950 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.950 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.950 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.950 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.950 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.950 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.951 INFO:teuthology.orchestra.run.smithi005.stdout: "url": { 2024-04-24T06:54:45.951 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url", 2024-04-24T06:54:45.951 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.951 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.951 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.951 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-24T06:54:45.952 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.952 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.952 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.952 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.952 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.952 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.953 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.953 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.953 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.953 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:45.953 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "test_orchestrator", 2024-04-24T06:54:45.953 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:45.954 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:45.954 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:45.954 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:45.954 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:45.954 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.954 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.954 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.955 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.955 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.955 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.955 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.955 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.955 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.955 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.956 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.956 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.956 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.956 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.956 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.956 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.956 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.957 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.957 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:45.957 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:45.957 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.957 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.958 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.958 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.958 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.958 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.958 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.958 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.958 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.959 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.959 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.959 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:45.959 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:45.959 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.959 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.960 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.960 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:45.960 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.960 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.960 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.960 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.961 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.961 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.961 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.961 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.961 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.961 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.962 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.962 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.962 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.962 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.962 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:45.962 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:45.963 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.963 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.963 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.963 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.963 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.963 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.963 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.964 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.964 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.964 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.964 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.964 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.964 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.965 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:45.965 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2024-04-24T06:54:45.965 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:45.965 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:45.965 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:45.965 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:45.965 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:45.966 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.966 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.966 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.966 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.966 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.966 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.966 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.967 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.967 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.967 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.967 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.967 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.967 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.967 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.968 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.968 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.968 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.968 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.968 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:45.968 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:45.968 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.969 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.969 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.969 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.969 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.969 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.969 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.970 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.970 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.970 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.970 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.970 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:45.971 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:45.971 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.971 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.971 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.971 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:45.972 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.972 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.972 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.972 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.972 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.972 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.972 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.972 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.973 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.973 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.973 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.973 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.973 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.974 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.974 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.974 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:45.974 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:45.974 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.975 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.975 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.975 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.975 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.975 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.975 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.975 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.976 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.976 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.976 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.976 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_clones": { 2024-04-24T06:54:45.976 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_clones", 2024-04-24T06:54:45.976 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:45.977 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.977 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.977 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4", 2024-04-24T06:54:45.977 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.977 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.977 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.978 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-24T06:54:45.978 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.978 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.978 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.979 INFO:teuthology.orchestra.run.smithi005.stdout: "periodic_async_work": { 2024-04-24T06:54:45.979 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "periodic_async_work", 2024-04-24T06:54:45.979 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.980 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.980 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.980 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.980 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.980 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.981 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.981 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Periodically check for async work", 2024-04-24T06:54:45.981 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.981 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.981 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.982 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_delay": { 2024-04-24T06:54:45.982 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_delay", 2024-04-24T06:54:45.982 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:45.982 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.982 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.982 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-04-24T06:54:45.983 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.983 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.983 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.983 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-24T06:54:45.983 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.983 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.983 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.984 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_no_wait": { 2024-04-24T06:54:45.984 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_no_wait", 2024-04-24T06:54:45.984 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.984 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.984 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.985 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:45.985 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.985 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.985 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.985 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-24T06:54:45.985 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.986 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.986 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.986 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.986 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:45.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.986 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:45.986 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix", 2024-04-24T06:54:45.987 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:45.987 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:45.987 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:45.987 INFO:teuthology.orchestra.run.smithi005.stdout: "discovery_interval": { 2024-04-24T06:54:45.987 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "discovery_interval", 2024-04-24T06:54:45.987 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-04-24T06:54:45.987 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.988 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.988 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "100", 2024-04-24T06:54:45.988 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.988 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.988 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.988 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.988 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.989 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.989 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.989 INFO:teuthology.orchestra.run.smithi005.stdout: "identifier": { 2024-04-24T06:54:45.989 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "identifier", 2024-04-24T06:54:45.989 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.989 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.990 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.990 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.990 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.990 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.990 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.990 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.991 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.991 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.991 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.991 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-04-24T06:54:45.991 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-04-24T06:54:45.991 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:45.992 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.992 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:45.992 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-04-24T06:54:45.992 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.992 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.992 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.992 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.992 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.993 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.993 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.993 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:45.993 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:45.993 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.993 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.994 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.994 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:45.994 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.994 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.994 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.994 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:45.995 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:45.995 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:45.995 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:45.995 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:45.995 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:45.995 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:45.995 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.995 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.996 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.996 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.996 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:45.996 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:45.996 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:45.996 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.997 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.997 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:45.997 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.997 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.997 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:45.997 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:45.998 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:45.998 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:45.998 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:45.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:45.998 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:45.998 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:45.998 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:45.999 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:45.999 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:45.999 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:45.999 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:45.999 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:45.999 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:45.999 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.000 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.000 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.000 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.000 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.000 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.000 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.001 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.001 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.001 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.001 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.001 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:46.001 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:46.002 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.002 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.002 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.002 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.002 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.002 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.003 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.003 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.003 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.003 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.003 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.003 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_host": { 2024-04-24T06:54:46.004 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_host", 2024-04-24T06:54:46.004 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.004 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.004 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.004 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.004 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.004 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.005 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.005 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.005 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.005 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.005 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.005 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_port": { 2024-04-24T06:54:46.005 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_port", 2024-04-24T06:54:46.005 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:46.006 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.006 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.006 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10051", 2024-04-24T06:54:46.006 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.006 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.006 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.006 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.007 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.007 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.007 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.007 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.007 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_sender": { 2024-04-24T06:54:46.007 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_sender", 2024-04-24T06:54:46.007 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.008 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.008 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.008 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-24T06:54:46.008 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.008 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.008 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.008 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.009 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.009 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.009 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.009 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.009 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.009 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.010 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-04-24T06:54:46.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:46.010 INFO:teuthology.orchestra.run.smithi005.stdout: "gid": 25216, 2024-04-24T06:54:46.010 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "z", 2024-04-24T06:54:46.010 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_features": 4540701547738038271, 2024-04-24T06:54:46.011 INFO:teuthology.orchestra.run.smithi005.stdout: "available_modules": [ 2024-04-24T06:54:46.011 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:46.011 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "alerts", 2024-04-24T06:54:46.011 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:46.011 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:46.011 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:46.012 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-04-24T06:54:46.012 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-04-24T06:54:46.012 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:46.012 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.012 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.012 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-04-24T06:54:46.012 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.012 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.013 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.013 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How frequently to reexamine health status", 2024-04-24T06:54:46.013 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.013 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.013 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.013 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:46.014 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:46.014 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.014 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.014 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.014 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.015 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.015 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.015 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.015 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.015 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.015 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.015 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.015 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.016 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.016 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.016 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.016 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.016 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.016 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.017 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:46.017 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:46.017 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.017 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.017 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.017 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.017 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.018 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.018 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.018 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.018 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.018 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.018 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.019 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:46.019 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:46.019 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.019 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.019 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.019 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:46.019 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.020 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.020 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.020 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.020 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.020 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.020 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.021 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.021 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.021 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.021 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.021 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.021 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.021 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.022 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:46.022 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:46.022 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.022 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.022 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.022 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.023 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.023 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.023 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.023 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.023 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.023 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.023 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.024 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_destination": { 2024-04-24T06:54:46.024 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_destination", 2024-04-24T06:54:46.024 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.024 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.024 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.024 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.024 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.025 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.025 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.025 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email address to send alerts to", 2024-04-24T06:54:46.025 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.025 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.025 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.025 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_from_name": { 2024-04-24T06:54:46.026 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_from_name", 2024-04-24T06:54:46.026 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.026 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.026 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.026 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "Ceph", 2024-04-24T06:54:46.026 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.026 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.027 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.027 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email From: name", 2024-04-24T06:54:46.027 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.027 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.027 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.027 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_host": { 2024-04-24T06:54:46.027 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_host", 2024-04-24T06:54:46.028 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.028 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.028 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.028 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.028 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.028 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.028 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.029 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP server", 2024-04-24T06:54:46.029 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.029 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.029 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.029 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_password": { 2024-04-24T06:54:46.030 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_password", 2024-04-24T06:54:46.030 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.030 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.030 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.030 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.031 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.031 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.031 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.031 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Password to authenticate with", 2024-04-24T06:54:46.031 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.031 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.032 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.032 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_port": { 2024-04-24T06:54:46.032 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_port", 2024-04-24T06:54:46.032 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:46.032 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.033 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.033 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "465", 2024-04-24T06:54:46.033 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.033 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.033 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.033 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP port", 2024-04-24T06:54:46.033 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.034 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.034 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.034 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.034 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_sender": { 2024-04-24T06:54:46.034 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_sender", 2024-04-24T06:54:46.034 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.035 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.035 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.035 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.035 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.035 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.035 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.035 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP envelope sender", 2024-04-24T06:54:46.036 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.036 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.036 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.036 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_ssl": { 2024-04-24T06:54:46.036 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_ssl", 2024-04-24T06:54:46.036 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.037 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.037 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.037 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:46.037 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.038 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.038 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.038 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-24T06:54:46.038 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.038 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.038 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.039 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_user": { 2024-04-24T06:54:46.039 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_user", 2024-04-24T06:54:46.039 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.039 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.040 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.040 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.040 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.040 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.040 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.040 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "User to authenticate as", 2024-04-24T06:54:46.040 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.041 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.041 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.041 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.041 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.041 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:46.041 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "balancer", 2024-04-24T06:54:46.042 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:46.042 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:46.042 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:46.042 INFO:teuthology.orchestra.run.smithi005.stdout: "active": { 2024-04-24T06:54:46.042 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "active", 2024-04-24T06:54:46.042 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.042 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.042 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.043 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:46.043 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.043 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.043 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.043 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically balance PGs across cluster", 2024-04-24T06:54:46.043 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.043 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.044 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.044 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_time": { 2024-04-24T06:54:46.044 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_time", 2024-04-24T06:54:46.044 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.044 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.044 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.044 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0000", 2024-04-24T06:54:46.045 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.045 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.045 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.045 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "beginning time of day to automatically balance", 2024-04-24T06:54:46.045 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T06:54:46.045 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.045 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.046 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_weekday": { 2024-04-24T06:54:46.046 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_weekday", 2024-04-24T06:54:46.046 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-04-24T06:54:46.046 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.046 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.046 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-04-24T06:54:46.046 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-04-24T06:54:46.047 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-04-24T06:54:46.047 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.047 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-24T06:54:46.047 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T06:54:46.047 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.048 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.048 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_max_iterations": { 2024-04-24T06:54:46.048 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_max_iterations", 2024-04-24T06:54:46.048 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-04-24T06:54:46.049 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.049 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.049 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "25", 2024-04-24T06:54:46.049 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-04-24T06:54:46.049 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "250", 2024-04-24T06:54:46.049 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.050 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-24T06:54:46.050 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.050 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.050 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.050 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_metrics": { 2024-04-24T06:54:46.051 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_metrics", 2024-04-24T06:54:46.051 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.051 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.051 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.051 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "pgs,objects,bytes", 2024-04-24T06:54:46.051 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.051 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.052 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.052 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-24T06:54:46.052 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-24T06:54:46.052 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.052 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.052 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_step": { 2024-04-24T06:54:46.052 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_step", 2024-04-24T06:54:46.053 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-24T06:54:46.053 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.053 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.053 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.5", 2024-04-24T06:54:46.053 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0.001", 2024-04-24T06:54:46.053 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "0.999", 2024-04-24T06:54:46.053 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.054 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "aggressiveness of optimization", 2024-04-24T06:54:46.054 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-24T06:54:46.054 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.054 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.054 INFO:teuthology.orchestra.run.smithi005.stdout: "end_time": { 2024-04-24T06:54:46.054 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_time", 2024-04-24T06:54:46.055 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.055 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.055 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.055 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2359", 2024-04-24T06:54:46.055 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.055 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.055 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.055 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ending time of day to automatically balance", 2024-04-24T06:54:46.056 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T06:54:46.056 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.056 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.056 INFO:teuthology.orchestra.run.smithi005.stdout: "end_weekday": { 2024-04-24T06:54:46.056 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_weekday", 2024-04-24T06:54:46.056 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-04-24T06:54:46.057 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.057 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.057 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-04-24T06:54:46.057 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-04-24T06:54:46.057 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-04-24T06:54:46.057 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.057 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-24T06:54:46.058 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T06:54:46.058 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.058 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.058 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:46.058 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:46.058 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.058 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.059 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.059 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.059 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.059 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.059 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.059 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.059 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.059 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.060 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.060 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.060 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.060 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.060 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.060 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.060 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.061 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.061 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:46.061 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:46.061 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.061 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.061 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.061 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.062 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.062 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.062 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.062 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.062 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.062 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.063 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.063 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:46.063 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:46.063 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.063 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.063 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.063 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:46.064 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.064 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.064 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.064 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.064 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.064 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.064 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.064 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.065 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.065 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.065 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.065 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.065 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.065 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.066 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:46.066 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:46.066 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.066 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.066 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.066 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.067 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.067 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.067 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.067 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.067 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.068 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.068 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.068 INFO:teuthology.orchestra.run.smithi005.stdout: "min_score": { 2024-04-24T06:54:46.068 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_score", 2024-04-24T06:54:46.068 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-24T06:54:46.068 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.069 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.069 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-04-24T06:54:46.069 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.069 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.069 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.070 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-24T06:54:46.070 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.070 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.070 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.071 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-04-24T06:54:46.071 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-04-24T06:54:46.071 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.071 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.071 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.071 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "upmap", 2024-04-24T06:54:46.071 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.072 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.072 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.072 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-compat", 2024-04-24T06:54:46.072 INFO:teuthology.orchestra.run.smithi005.stdout: "none", 2024-04-24T06:54:46.072 INFO:teuthology.orchestra.run.smithi005.stdout: "read", 2024-04-24T06:54:46.072 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap", 2024-04-24T06:54:46.072 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap-read" 2024-04-24T06:54:46.073 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.073 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Balancer mode", 2024-04-24T06:54:46.073 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.073 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.073 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.073 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_ids": { 2024-04-24T06:54:46.073 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_ids", 2024-04-24T06:54:46.074 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.074 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.074 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.074 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.074 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.074 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.074 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.075 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-24T06:54:46.075 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.075 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.075 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.075 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-04-24T06:54:46.076 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-04-24T06:54:46.076 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:46.076 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.076 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.076 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-04-24T06:54:46.076 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.076 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.076 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.077 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-24T06:54:46.077 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.077 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.077 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.077 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_deviation": { 2024-04-24T06:54:46.077 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_deviation", 2024-04-24T06:54:46.078 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:46.078 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.078 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.078 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-04-24T06:54:46.078 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-04-24T06:54:46.078 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.078 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.079 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-24T06:54:46.079 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-24T06:54:46.079 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.079 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.079 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_optimizations": { 2024-04-24T06:54:46.079 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_optimizations", 2024-04-24T06:54:46.080 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-04-24T06:54:46.080 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.080 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.080 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-24T06:54:46.080 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.080 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.080 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.081 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-24T06:54:46.081 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.081 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.081 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.081 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.081 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.081 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:46.081 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm", 2024-04-24T06:54:46.082 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:46.082 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:46.082 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:46.082 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_down_multiplier": { 2024-04-24T06:54:46.082 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_down_multiplier", 2024-04-24T06:54:46.082 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-24T06:54:46.082 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.083 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.083 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-04-24T06:54:46.083 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.083 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.083 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.083 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-24T06:54:46.083 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.083 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.084 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.084 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_refresh_rate": { 2024-04-24T06:54:46.084 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_refresh_rate", 2024-04-24T06:54:46.084 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:46.084 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.084 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.084 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "20", 2024-04-24T06:54:46.085 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.085 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.085 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.085 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-24T06:54:46.085 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.085 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.085 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.086 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_starting_port": { 2024-04-24T06:54:46.086 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_starting_port", 2024-04-24T06:54:46.086 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:46.086 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.086 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.086 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4721", 2024-04-24T06:54:46.086 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.087 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.087 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.087 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-24T06:54:46.087 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.087 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.087 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.087 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_ptrace": { 2024-04-24T06:54:46.088 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_ptrace", 2024-04-24T06:54:46.088 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.088 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.088 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.088 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.088 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.088 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.088 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.089 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-24T06:54:46.089 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-24T06:54:46.089 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.089 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.089 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_interval": { 2024-04-24T06:54:46.089 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_interval", 2024-04-24T06:54:46.090 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:46.090 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.090 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.090 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-04-24T06:54:46.090 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.090 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.090 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.091 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-24T06:54:46.091 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.091 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.091 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.091 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_memory_target_ratio": { 2024-04-24T06:54:46.091 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_memory_target_ratio", 2024-04-24T06:54:46.092 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-24T06:54:46.092 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.092 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.092 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.7", 2024-04-24T06:54:46.092 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.092 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.092 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.093 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-24T06:54:46.093 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.093 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.093 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.093 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm_log_destination": { 2024-04-24T06:54:46.093 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm_log_destination", 2024-04-24T06:54:46.093 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.093 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.094 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.094 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.094 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.094 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.094 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.094 INFO:teuthology.orchestra.run.smithi005.stdout: "file", 2024-04-24T06:54:46.095 INFO:teuthology.orchestra.run.smithi005.stdout: "file,syslog", 2024-04-24T06:54:46.095 INFO:teuthology.orchestra.run.smithi005.stdout: "syslog" 2024-04-24T06:54:46.095 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.095 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-24T06:54:46.095 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.095 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.096 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.096 INFO:teuthology.orchestra.run.smithi005.stdout: "cgroups_split": { 2024-04-24T06:54:46.096 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cgroups_split", 2024-04-24T06:54:46.096 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.097 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.097 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.097 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:46.097 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.097 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.097 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.098 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-24T06:54:46.098 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.098 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.098 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.098 INFO:teuthology.orchestra.run.smithi005.stdout: "config_checks_enabled": { 2024-04-24T06:54:46.099 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_checks_enabled", 2024-04-24T06:54:46.099 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.099 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.099 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.100 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.100 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.100 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.100 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.100 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-24T06:54:46.100 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.101 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.101 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.101 INFO:teuthology.orchestra.run.smithi005.stdout: "config_dashboard": { 2024-04-24T06:54:46.101 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_dashboard", 2024-04-24T06:54:46.101 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.102 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.102 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.102 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:46.102 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.102 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.102 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.103 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-24T06:54:46.103 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.103 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.103 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.103 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_alertmanager": { 2024-04-24T06:54:46.103 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_alertmanager", 2024-04-24T06:54:46.104 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.104 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.104 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.104 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-24T06:54:46.104 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.104 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.105 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.105 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-04-24T06:54:46.105 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.105 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.105 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.106 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_base": { 2024-04-24T06:54:46.106 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_base", 2024-04-24T06:54:46.106 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.106 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.106 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.106 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-24T06:54:46.107 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.107 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.107 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.107 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Container image name, without the tag", 2024-04-24T06:54:46.107 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.108 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.108 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.108 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_elasticsearch": { 2024-04-24T06:54:46.108 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_elasticsearch", 2024-04-24T06:54:46.108 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.108 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.109 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.109 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-24T06:54:46.109 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.109 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.109 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.109 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "elasticsearch container image", 2024-04-24T06:54:46.110 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.110 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.110 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.110 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_grafana": { 2024-04-24T06:54:46.110 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_grafana", 2024-04-24T06:54:46.110 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.111 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.111 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.111 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-24T06:54:46.111 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.111 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.111 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.111 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-04-24T06:54:46.112 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.112 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.112 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.112 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_haproxy": { 2024-04-24T06:54:46.112 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_haproxy", 2024-04-24T06:54:46.113 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.113 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.113 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.113 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-24T06:54:46.113 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.113 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.114 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.114 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "HAproxy container image", 2024-04-24T06:54:46.114 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.114 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.114 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.114 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_agent": { 2024-04-24T06:54:46.115 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_agent", 2024-04-24T06:54:46.115 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.115 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.115 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.116 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-24T06:54:46.116 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.116 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.116 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.116 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger agent container image", 2024-04-24T06:54:46.116 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.116 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.117 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.117 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_collector": { 2024-04-24T06:54:46.117 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_collector", 2024-04-24T06:54:46.117 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.117 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.118 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.118 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-24T06:54:46.118 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.118 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.118 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.118 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger collector container image", 2024-04-24T06:54:46.118 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.119 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.119 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.119 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_query": { 2024-04-24T06:54:46.119 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_query", 2024-04-24T06:54:46.119 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.119 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.120 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.120 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-24T06:54:46.120 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.120 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.120 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.120 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger query container image", 2024-04-24T06:54:46.121 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.121 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.121 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.121 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_keepalived": { 2024-04-24T06:54:46.121 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_keepalived", 2024-04-24T06:54:46.122 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.122 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.122 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.122 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-24T06:54:46.122 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.122 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.122 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.123 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Keepalived container image", 2024-04-24T06:54:46.123 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.123 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.123 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.123 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_loki": { 2024-04-24T06:54:46.124 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_loki", 2024-04-24T06:54:46.124 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.124 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.124 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.124 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-24T06:54:46.124 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.124 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.125 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.125 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Loki container image", 2024-04-24T06:54:46.125 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.125 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.125 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.125 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_node_exporter": { 2024-04-24T06:54:46.126 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_node_exporter", 2024-04-24T06:54:46.126 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.126 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.126 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.126 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-24T06:54:46.127 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.127 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.127 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.127 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-04-24T06:54:46.127 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.127 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.128 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.128 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_nvmeof": { 2024-04-24T06:54:46.128 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_nvmeof", 2024-04-24T06:54:46.128 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.129 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.129 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.129 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-24T06:54:46.129 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.130 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.130 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.130 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Nvme-of container image", 2024-04-24T06:54:46.130 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.130 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.131 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.131 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_prometheus": { 2024-04-24T06:54:46.131 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_prometheus", 2024-04-24T06:54:46.132 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.132 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.132 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.132 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-24T06:54:46.132 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.133 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.133 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.133 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-04-24T06:54:46.133 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.133 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.134 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.134 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_promtail": { 2024-04-24T06:54:46.134 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_promtail", 2024-04-24T06:54:46.135 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.135 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.135 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.135 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-24T06:54:46.135 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.135 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.136 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.136 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Promtail container image", 2024-04-24T06:54:46.136 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.136 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.137 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.137 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_samba": { 2024-04-24T06:54:46.137 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_samba", 2024-04-24T06:54:46.137 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.137 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.138 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.138 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-24T06:54:46.138 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.138 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.139 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.139 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Samba/SMB container image", 2024-04-24T06:54:46.139 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.139 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.139 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.140 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_snmp_gateway": { 2024-04-24T06:54:46.140 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_snmp_gateway", 2024-04-24T06:54:46.140 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.140 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.140 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.141 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-24T06:54:46.141 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.141 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.141 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.141 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SNMP Gateway container image", 2024-04-24T06:54:46.141 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.142 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.142 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.142 INFO:teuthology.orchestra.run.smithi005.stdout: "container_init": { 2024-04-24T06:54:46.142 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_init", 2024-04-24T06:54:46.142 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.143 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.143 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.143 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:46.143 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.143 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.143 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.144 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Run podman/docker with `--init`", 2024-04-24T06:54:46.144 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.144 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.144 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.144 INFO:teuthology.orchestra.run.smithi005.stdout: "daemon_cache_timeout": { 2024-04-24T06:54:46.145 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "daemon_cache_timeout", 2024-04-24T06:54:46.145 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:46.145 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.145 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.145 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-04-24T06:54:46.145 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.146 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.146 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.146 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-24T06:54:46.146 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.147 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.147 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.147 INFO:teuthology.orchestra.run.smithi005.stdout: "default_cephadm_command_timeout": { 2024-04-24T06:54:46.147 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_cephadm_command_timeout", 2024-04-24T06:54:46.148 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:46.148 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.148 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.148 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "900", 2024-04-24T06:54:46.148 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.148 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.149 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.149 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-24T06:54:46.149 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.149 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.149 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.150 INFO:teuthology.orchestra.run.smithi005.stdout: "default_registry": { 2024-04-24T06:54:46.150 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_registry", 2024-04-24T06:54:46.150 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.150 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.150 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.151 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io", 2024-04-24T06:54:46.151 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.151 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.151 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.151 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-24T06:54:46.152 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.152 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.152 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.152 INFO:teuthology.orchestra.run.smithi005.stdout: "device_cache_timeout": { 2024-04-24T06:54:46.152 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_cache_timeout", 2024-04-24T06:54:46.153 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:46.153 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.153 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.153 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1800", 2024-04-24T06:54:46.154 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.154 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.154 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.154 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache device inventory", 2024-04-24T06:54:46.155 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.155 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.155 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.155 INFO:teuthology.orchestra.run.smithi005.stdout: "device_enhanced_scan": { 2024-04-24T06:54:46.155 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_enhanced_scan", 2024-04-24T06:54:46.155 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.156 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.156 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.156 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.156 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.156 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.156 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.157 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-24T06:54:46.157 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.157 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.157 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.157 INFO:teuthology.orchestra.run.smithi005.stdout: "facts_cache_timeout": { 2024-04-24T06:54:46.158 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "facts_cache_timeout", 2024-04-24T06:54:46.158 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:46.158 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.158 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.158 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-04-24T06:54:46.158 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.159 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.159 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.159 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache host facts data", 2024-04-24T06:54:46.159 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.159 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.160 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.160 INFO:teuthology.orchestra.run.smithi005.stdout: "grafana_dashboards_path": { 2024-04-24T06:54:46.160 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "grafana_dashboards_path", 2024-04-24T06:54:46.160 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.161 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.161 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.161 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-24T06:54:46.161 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.161 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.161 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.161 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-24T06:54:46.162 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.162 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.162 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.162 INFO:teuthology.orchestra.run.smithi005.stdout: "host_check_interval": { 2024-04-24T06:54:46.162 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "host_check_interval", 2024-04-24T06:54:46.162 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:46.163 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.163 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.163 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-04-24T06:54:46.163 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.163 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.164 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.164 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to perform a host check", 2024-04-24T06:54:46.164 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.164 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.164 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.165 INFO:teuthology.orchestra.run.smithi005.stdout: "hw_monitoring": { 2024-04-24T06:54:46.165 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hw_monitoring", 2024-04-24T06:54:46.165 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.165 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.165 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.165 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.165 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.166 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.166 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.166 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-24T06:54:46.166 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.166 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.167 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.167 INFO:teuthology.orchestra.run.smithi005.stdout: "inventory_list_all": { 2024-04-24T06:54:46.167 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "inventory_list_all", 2024-04-24T06:54:46.167 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.167 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.167 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.168 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.168 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.168 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.168 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.168 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-24T06:54:46.168 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.169 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.169 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.169 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:46.169 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:46.169 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.170 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.170 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.170 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.170 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.170 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.170 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.171 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.171 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.171 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.171 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.171 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.171 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.171 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.171 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.172 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.172 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.172 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.172 INFO:teuthology.orchestra.run.smithi005.stdout: "log_refresh_metadata": { 2024-04-24T06:54:46.172 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_refresh_metadata", 2024-04-24T06:54:46.172 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.173 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.173 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.173 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.173 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.173 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.173 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.174 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-24T06:54:46.174 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.174 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.174 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.174 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:46.175 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:46.175 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.175 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.175 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.175 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:46.176 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.176 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.176 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.176 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-24T06:54:46.176 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.176 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.176 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.177 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:46.177 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:46.177 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.177 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.177 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.178 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:46.178 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.178 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.178 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.178 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.178 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.179 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.179 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.179 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.179 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.179 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.179 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.179 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.179 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.180 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.180 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:46.180 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:46.180 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.181 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.181 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.181 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.181 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.181 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.181 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.182 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.182 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.182 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.182 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.183 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-24T06:54:46.183 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-24T06:54:46.183 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.183 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.183 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.183 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.183 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.184 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.184 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.184 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-24T06:54:46.184 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.184 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.184 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.185 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-24T06:54:46.185 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-24T06:54:46.185 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.185 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.185 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.185 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "*", 2024-04-24T06:54:46.185 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.186 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.186 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.186 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-24T06:54:46.186 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.186 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.186 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.187 INFO:teuthology.orchestra.run.smithi005.stdout: "max_count_per_host": { 2024-04-24T06:54:46.187 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_count_per_host", 2024-04-24T06:54:46.187 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:46.187 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.188 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.188 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-24T06:54:46.188 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.188 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.188 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.189 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of daemons per service per host", 2024-04-24T06:54:46.189 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.189 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.189 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.190 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd_draining_count": { 2024-04-24T06:54:46.190 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_osd_draining_count", 2024-04-24T06:54:46.190 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:46.190 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.190 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.191 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-24T06:54:46.191 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.191 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.191 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.191 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-24T06:54:46.191 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.191 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.192 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.192 INFO:teuthology.orchestra.run.smithi005.stdout: "migration_current": { 2024-04-24T06:54:46.192 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "migration_current", 2024-04-24T06:54:46.192 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:46.193 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.193 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.193 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.193 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.193 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.193 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.194 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "internal - do not modify", 2024-04-24T06:54:46.194 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.194 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.194 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.195 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-04-24T06:54:46.195 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-04-24T06:54:46.195 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.195 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.196 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.196 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "root", 2024-04-24T06:54:46.196 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.196 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.196 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.196 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm-package", 2024-04-24T06:54:46.196 INFO:teuthology.orchestra.run.smithi005.stdout: "root" 2024-04-24T06:54:46.197 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.197 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "mode for remote execution of cephadm", 2024-04-24T06:54:46.197 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.197 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.197 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.198 INFO:teuthology.orchestra.run.smithi005.stdout: "oob_default_addr": { 2024-04-24T06:54:46.198 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "oob_default_addr", 2024-04-24T06:54:46.198 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.198 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.198 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.198 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "169.254.1.1", 2024-04-24T06:54:46.199 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.199 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.199 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.199 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-24T06:54:46.199 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.200 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.200 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.200 INFO:teuthology.orchestra.run.smithi005.stdout: "prometheus_alerts_path": { 2024-04-24T06:54:46.200 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus_alerts_path", 2024-04-24T06:54:46.200 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.201 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.201 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.201 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-24T06:54:46.201 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.202 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.202 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.202 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-24T06:54:46.202 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.202 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.203 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.203 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_insecure": { 2024-04-24T06:54:46.203 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_insecure", 2024-04-24T06:54:46.204 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.204 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.204 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.204 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.205 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.205 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.205 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.205 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-24T06:54:46.205 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.206 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.206 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.207 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_password": { 2024-04-24T06:54:46.207 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_password", 2024-04-24T06:54:46.207 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.207 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.207 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.208 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.208 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.208 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.208 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.209 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-24T06:54:46.209 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.209 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.209 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.209 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_url": { 2024-04-24T06:54:46.210 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_url", 2024-04-24T06:54:46.210 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.210 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.210 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.211 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.211 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.211 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.212 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.212 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-24T06:54:46.212 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.213 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.213 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.213 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_username": { 2024-04-24T06:54:46.214 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_username", 2024-04-24T06:54:46.214 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.214 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.214 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.215 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.215 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.215 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.215 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.216 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-24T06:54:46.216 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.216 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.216 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.217 INFO:teuthology.orchestra.run.smithi005.stdout: "secure_monitoring_stack": { 2024-04-24T06:54:46.217 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "secure_monitoring_stack", 2024-04-24T06:54:46.217 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.217 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.218 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.218 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.218 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.218 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.219 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.219 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-24T06:54:46.219 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.219 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.220 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.220 INFO:teuthology.orchestra.run.smithi005.stdout: "service_discovery_port": { 2024-04-24T06:54:46.221 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "service_discovery_port", 2024-04-24T06:54:46.221 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:46.221 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.221 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.222 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8765", 2024-04-24T06:54:46.222 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.222 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.222 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.223 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "cephadm service discovery port", 2024-04-24T06:54:46.223 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.223 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.224 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.224 INFO:teuthology.orchestra.run.smithi005.stdout: "ssh_config_file": { 2024-04-24T06:54:46.224 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssh_config_file", 2024-04-24T06:54:46.224 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.225 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.225 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.225 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.225 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.225 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.226 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.226 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-24T06:54:46.226 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.226 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.226 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.227 INFO:teuthology.orchestra.run.smithi005.stdout: "use_agent": { 2024-04-24T06:54:46.227 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_agent", 2024-04-24T06:54:46.227 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.228 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.228 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.228 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.228 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.229 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.229 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.229 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-24T06:54:46.229 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.229 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.230 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.230 INFO:teuthology.orchestra.run.smithi005.stdout: "use_repo_digest": { 2024-04-24T06:54:46.230 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_repo_digest", 2024-04-24T06:54:46.230 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.231 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.231 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.231 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:46.231 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.231 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.231 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.232 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-24T06:54:46.232 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.232 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.232 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.232 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_failed_host_check": { 2024-04-24T06:54:46.233 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_failed_host_check", 2024-04-24T06:54:46.233 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.233 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.233 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.233 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:46.234 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.234 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.234 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.234 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if the host check fails", 2024-04-24T06:54:46.234 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.234 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.234 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.235 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_daemons": { 2024-04-24T06:54:46.235 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_daemons", 2024-04-24T06:54:46.235 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.235 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.236 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.236 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:46.236 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.236 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.236 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.237 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-24T06:54:46.237 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.237 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.237 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.238 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_hosts": { 2024-04-24T06:54:46.238 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_hosts", 2024-04-24T06:54:46.238 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.238 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.238 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.239 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:46.239 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.239 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.239 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.239 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-24T06:54:46.239 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.240 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.240 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.240 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.240 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.241 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:46.241 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crash", 2024-04-24T06:54:46.241 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:46.241 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:46.242 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:46.242 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:46.242 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:46.242 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.242 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.243 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.243 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.243 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.243 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.243 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.243 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.243 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.244 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.244 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.244 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.244 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.244 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.245 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.245 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.245 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.245 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.245 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:46.246 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:46.246 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.246 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.246 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.246 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.247 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.247 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.247 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.247 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.247 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.248 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.248 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.248 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:46.248 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:46.248 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.248 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.249 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.249 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:46.249 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.249 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.249 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.249 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.250 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.250 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.250 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.250 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.250 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.251 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.251 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.251 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.251 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.251 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.252 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:46.252 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:46.252 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.252 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.252 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.252 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.253 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.253 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.253 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.253 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.253 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.254 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.254 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.254 INFO:teuthology.orchestra.run.smithi005.stdout: "retain_interval": { 2024-04-24T06:54:46.254 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retain_interval", 2024-04-24T06:54:46.255 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:46.255 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.255 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.255 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "31536000", 2024-04-24T06:54:46.255 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.255 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.256 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.256 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-24T06:54:46.256 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.256 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.256 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.257 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_recent_interval": { 2024-04-24T06:54:46.257 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_recent_interval", 2024-04-24T06:54:46.257 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:46.257 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.257 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.258 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1209600", 2024-04-24T06:54:46.258 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.258 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.258 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.258 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-24T06:54:46.258 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.259 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.259 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.259 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.259 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.259 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:46.260 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dashboard", 2024-04-24T06:54:46.260 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:46.260 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:46.260 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:46.260 INFO:teuthology.orchestra.run.smithi005.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-24T06:54:46.260 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-24T06:54:46.260 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:46.261 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.261 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.261 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-24T06:54:46.261 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.261 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.261 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.261 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.262 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.262 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.262 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.262 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_HOST": { 2024-04-24T06:54:46.263 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-24T06:54:46.263 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.263 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.263 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.263 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.263 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.263 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.263 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.264 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.264 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.264 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.264 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.264 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-24T06:54:46.264 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-24T06:54:46.264 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.265 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.265 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.265 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:46.265 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.265 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.265 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.265 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.265 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.265 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.266 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.266 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_ENABLED": { 2024-04-24T06:54:46.266 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_ENABLED", 2024-04-24T06:54:46.266 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.266 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.266 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.267 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.267 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.267 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.267 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.267 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.267 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.267 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.267 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.268 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-24T06:54:46.268 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-24T06:54:46.268 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.268 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.268 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.268 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:46.268 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.269 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.269 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.269 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.269 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.269 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.269 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.270 INFO:teuthology.orchestra.run.smithi005.stdout: "ENABLE_BROWSABLE_API": { 2024-04-24T06:54:46.270 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-24T06:54:46.270 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.270 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.270 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.270 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:46.271 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.271 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.271 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.271 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.271 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.271 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.271 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.272 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-24T06:54:46.272 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-24T06:54:46.272 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.272 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.272 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.272 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:46.273 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.273 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.273 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.273 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.273 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.273 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.273 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.274 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-24T06:54:46.274 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-24T06:54:46.274 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.274 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.274 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.275 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:46.275 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.275 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.275 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.275 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.275 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.276 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.276 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.276 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-24T06:54:46.276 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-24T06:54:46.277 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.277 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.277 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.277 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:46.277 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.277 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.278 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.278 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.278 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.278 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.278 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.279 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-24T06:54:46.279 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-24T06:54:46.279 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.279 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.279 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.279 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:46.279 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.280 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.280 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.280 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.280 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.280 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.280 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.281 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-24T06:54:46.281 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-24T06:54:46.281 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.281 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.281 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.282 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:46.282 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.282 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.282 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.282 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.282 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.283 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.283 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.283 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-24T06:54:46.283 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-24T06:54:46.283 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.284 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.284 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.284 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:46.284 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.284 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.285 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.285 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.285 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.285 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.285 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.286 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.286 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-24T06:54:46.286 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-24T06:54:46.286 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.286 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.286 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.287 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:46.287 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.287 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.287 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.287 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.287 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.288 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.288 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.288 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.288 INFO:teuthology.orchestra.run.smithi005.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-24T06:54:46.288 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-24T06:54:46.289 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.289 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.289 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.289 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.289 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.289 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.290 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.290 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.290 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.290 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.291 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.291 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.291 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_PASSWORD": { 2024-04-24T06:54:46.291 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-24T06:54:46.291 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.291 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.292 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.292 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-04-24T06:54:46.292 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.292 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.292 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.292 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.293 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.293 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.293 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.293 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-24T06:54:46.293 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-24T06:54:46.293 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.294 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.294 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.294 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:46.294 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.294 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.294 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.295 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.295 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.295 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.295 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.296 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_URL": { 2024-04-24T06:54:46.296 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_URL", 2024-04-24T06:54:46.296 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.296 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.296 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.296 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.296 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.297 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.297 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.297 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.297 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.298 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.298 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.298 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_USERNAME": { 2024-04-24T06:54:46.298 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-24T06:54:46.299 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.299 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.299 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.300 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-04-24T06:54:46.300 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.300 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.300 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.300 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.300 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.301 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.301 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.301 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-24T06:54:46.301 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-24T06:54:46.301 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.302 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.302 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.302 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.302 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.302 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.303 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.303 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.303 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.303 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.304 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.304 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-24T06:54:46.305 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-24T06:54:46.305 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.305 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.305 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.306 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.306 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.306 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.306 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.306 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.307 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.307 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.307 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.308 INFO:teuthology.orchestra.run.smithi005.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-24T06:54:46.308 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-24T06:54:46.308 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.308 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.309 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.309 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:46.309 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.309 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.309 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.310 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.310 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.310 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.310 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.311 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.311 INFO:teuthology.orchestra.run.smithi005.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-24T06:54:46.311 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-24T06:54:46.311 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.311 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.311 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.312 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.312 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.312 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.312 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.313 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.313 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.313 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.313 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.313 INFO:teuthology.orchestra.run.smithi005.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-24T06:54:46.314 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-24T06:54:46.314 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.314 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.314 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.315 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "[]", 2024-04-24T06:54:46.315 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.315 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.315 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.315 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.316 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.316 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.316 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.316 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.316 INFO:teuthology.orchestra.run.smithi005.stdout: "MULTICLUSTER_CONFIG": { 2024-04-24T06:54:46.317 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-24T06:54:46.317 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.317 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.317 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.318 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "{}", 2024-04-24T06:54:46.318 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.318 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.318 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.318 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.319 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.319 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.319 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.320 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_HOST": { 2024-04-24T06:54:46.320 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-24T06:54:46.320 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.320 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.321 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.321 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.321 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.321 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.321 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.322 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.322 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.322 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.322 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.323 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-24T06:54:46.323 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-24T06:54:46.323 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.323 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.324 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.324 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:46.324 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.324 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.324 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.325 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.325 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.325 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.325 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.326 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-24T06:54:46.326 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-24T06:54:46.326 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.327 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.327 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.327 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.327 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.327 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.327 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.328 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.328 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.328 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.328 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.329 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-24T06:54:46.329 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-24T06:54:46.329 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.329 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.329 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.330 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.330 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.330 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.330 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.331 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.331 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.331 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.331 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.332 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-24T06:54:46.332 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-24T06:54:46.332 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.332 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.333 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.333 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:46.333 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.333 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.333 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.333 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.334 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.334 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.334 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.334 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-24T06:54:46.335 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-24T06:54:46.335 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.335 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.335 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.336 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:46.336 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.336 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.336 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.336 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.337 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.337 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.337 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.337 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-24T06:54:46.338 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-24T06:54:46.338 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.338 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.338 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.339 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.339 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.339 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.339 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.339 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.339 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.340 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.340 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.340 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-24T06:54:46.341 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-24T06:54:46.341 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.341 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.341 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.341 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.342 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.342 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.342 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.342 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.343 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.343 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.343 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.343 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-24T06:54:46.343 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-24T06:54:46.344 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.344 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.344 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.345 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.345 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.345 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.345 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.345 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.346 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.346 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.346 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.347 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_ENABLED": { 2024-04-24T06:54:46.347 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-24T06:54:46.347 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.347 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.347 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.348 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:46.348 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.348 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.348 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.348 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.348 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.349 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.349 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.349 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-24T06:54:46.349 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-24T06:54:46.350 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.350 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.350 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.350 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-24T06:54:46.351 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.351 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.351 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.351 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.352 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.352 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.352 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.353 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-24T06:54:46.353 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-24T06:54:46.353 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:46.353 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.353 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.354 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-24T06:54:46.354 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.354 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.354 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.355 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.355 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.355 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.355 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.356 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-24T06:54:46.356 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-24T06:54:46.356 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:46.356 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.357 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.357 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8", 2024-04-24T06:54:46.357 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.357 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.357 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.358 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.358 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.358 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.358 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.359 INFO:teuthology.orchestra.run.smithi005.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-24T06:54:46.359 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-24T06:54:46.359 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:46.360 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.360 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.360 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "45", 2024-04-24T06:54:46.360 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.360 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.361 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.361 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.361 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.361 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.362 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.362 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ACCESS_KEY": { 2024-04-24T06:54:46.362 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-24T06:54:46.363 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.363 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.363 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.363 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.363 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.364 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.364 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.364 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.364 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.365 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.365 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.365 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-24T06:54:46.365 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-24T06:54:46.366 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.366 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.366 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.366 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-04-24T06:54:46.366 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.367 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.367 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.367 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.367 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.368 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.369 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.370 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SECRET_KEY": { 2024-04-24T06:54:46.370 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-24T06:54:46.370 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.371 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.371 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.371 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.371 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.372 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.372 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.372 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.372 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.372 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.373 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.373 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SSL_VERIFY": { 2024-04-24T06:54:46.373 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-24T06:54:46.373 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.374 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.374 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.374 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:46.374 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.374 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.375 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.375 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.375 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.375 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.376 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.376 INFO:teuthology.orchestra.run.smithi005.stdout: "UNSAFE_TLS_v1_2": { 2024-04-24T06:54:46.376 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-24T06:54:46.376 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.377 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.377 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.377 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.377 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.378 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.378 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.378 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.378 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.378 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.379 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.379 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-24T06:54:46.380 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-24T06:54:46.380 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:46.380 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.380 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.380 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-04-24T06:54:46.381 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.381 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.381 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.381 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.382 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.382 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.382 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.383 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-24T06:54:46.383 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-24T06:54:46.383 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:46.383 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.384 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.384 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-24T06:54:46.384 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.384 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.384 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.385 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.385 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.385 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.385 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.386 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-24T06:54:46.386 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-24T06:54:46.386 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:46.387 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.387 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.387 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-04-24T06:54:46.387 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.388 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.388 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.388 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.388 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.388 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.389 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.389 INFO:teuthology.orchestra.run.smithi005.stdout: "cross_origin_url": { 2024-04-24T06:54:46.389 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cross_origin_url", 2024-04-24T06:54:46.389 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.390 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.390 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.390 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.390 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.390 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.391 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.391 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.391 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.391 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.391 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.392 INFO:teuthology.orchestra.run.smithi005.stdout: "crt_file": { 2024-04-24T06:54:46.392 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crt_file", 2024-04-24T06:54:46.392 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.392 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.392 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.393 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.393 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.393 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.393 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.394 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.394 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.394 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.394 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.395 INFO:teuthology.orchestra.run.smithi005.stdout: "debug": { 2024-04-24T06:54:46.395 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "debug", 2024-04-24T06:54:46.395 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.395 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.395 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.396 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.396 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.396 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.396 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.397 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable/disable debug options", 2024-04-24T06:54:46.397 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.397 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.397 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.398 INFO:teuthology.orchestra.run.smithi005.stdout: "jwt_token_ttl": { 2024-04-24T06:54:46.398 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "jwt_token_ttl", 2024-04-24T06:54:46.398 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:46.398 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.399 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.399 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "28800", 2024-04-24T06:54:46.399 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.399 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.399 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.400 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.400 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.400 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.400 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.401 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-04-24T06:54:46.401 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-04-24T06:54:46.401 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.401 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.402 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.402 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.402 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.402 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.402 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.403 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.403 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.403 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.403 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.404 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:46.404 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:46.404 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.404 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.405 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.405 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.405 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.405 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.405 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.406 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.406 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.406 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.406 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.407 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.407 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.407 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.407 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.407 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.408 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.408 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.408 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:46.409 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:46.409 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.409 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.409 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.409 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.410 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.410 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.410 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.410 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.410 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.411 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.411 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.411 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:46.411 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:46.411 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.412 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.412 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.412 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:46.412 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.412 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.413 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.413 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.413 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.413 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.413 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.413 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.414 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.414 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.414 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.414 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.415 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.415 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.415 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:46.416 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:46.416 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.416 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.416 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.416 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.417 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.417 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.417 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.417 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.417 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.418 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.418 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.418 INFO:teuthology.orchestra.run.smithi005.stdout: "motd": { 2024-04-24T06:54:46.418 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "motd", 2024-04-24T06:54:46.419 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.419 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.419 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.419 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.419 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.420 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.420 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.420 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "The message of the day", 2024-04-24T06:54:46.420 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.421 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.421 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.421 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect_resolve_ip_addr": { 2024-04-24T06:54:46.421 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "redirect_resolve_ip_addr", 2024-04-24T06:54:46.422 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.422 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.422 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.422 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.422 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.423 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.423 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.423 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.423 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.423 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.424 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.424 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-04-24T06:54:46.424 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-04-24T06:54:46.424 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.425 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.425 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.425 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-04-24T06:54:46.425 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.426 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.426 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.426 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.426 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.427 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.427 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.427 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-04-24T06:54:46.427 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-04-24T06:54:46.428 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:46.428 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.428 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.428 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8080", 2024-04-24T06:54:46.428 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.429 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.429 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.429 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.429 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.430 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.430 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.430 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-04-24T06:54:46.430 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-04-24T06:54:46.431 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.431 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.431 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.431 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:46.432 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.432 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.432 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.432 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.432 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.433 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.433 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.433 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl_server_port": { 2024-04-24T06:54:46.433 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl_server_port", 2024-04-24T06:54:46.434 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:46.434 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.434 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.434 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8443", 2024-04-24T06:54:46.434 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.434 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.435 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.435 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.435 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.435 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.435 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.436 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-04-24T06:54:46.436 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-04-24T06:54:46.436 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.436 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.436 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.437 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "redirect", 2024-04-24T06:54:46.437 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.437 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.437 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.437 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.438 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect" 2024-04-24T06:54:46.438 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.438 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.438 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.439 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.439 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.439 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.439 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-04-24T06:54:46.440 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-04-24T06:54:46.440 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:46.440 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.440 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.440 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-04-24T06:54:46.441 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-04-24T06:54:46.441 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-04-24T06:54:46.441 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.441 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.442 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.442 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.442 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.443 INFO:teuthology.orchestra.run.smithi005.stdout: "url_prefix": { 2024-04-24T06:54:46.443 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url_prefix", 2024-04-24T06:54:46.443 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.443 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.443 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.444 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.444 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.444 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.444 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.444 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.445 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.445 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.445 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.445 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.445 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.446 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:46.446 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2024-04-24T06:54:46.446 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:46.446 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:46.447 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:46.447 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_monitoring": { 2024-04-24T06:54:46.447 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_monitoring", 2024-04-24T06:54:46.447 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.448 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.448 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.448 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:46.448 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.448 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.448 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.449 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "monitor device health metrics", 2024-04-24T06:54:46.449 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.449 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.449 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.449 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.449 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:46.450 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:46.450 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.450 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.450 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.450 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.451 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.451 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.451 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.451 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.451 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.452 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.452 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.452 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.452 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.452 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.452 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.453 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.453 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.453 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.453 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:46.453 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:46.454 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.454 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.454 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.454 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.454 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.455 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.455 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.455 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.455 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.455 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.455 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.456 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:46.456 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:46.456 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.456 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.457 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.457 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:46.457 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.457 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.457 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.458 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.458 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.458 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.458 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.458 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.458 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.459 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.459 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.459 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.459 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.460 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.460 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:46.460 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:46.460 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.460 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.461 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.461 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.461 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.461 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.462 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.462 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.462 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.462 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.463 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.463 INFO:teuthology.orchestra.run.smithi005.stdout: "mark_out_threshold": { 2024-04-24T06:54:46.463 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mark_out_threshold", 2024-04-24T06:54:46.463 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:46.464 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.464 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.464 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2419200", 2024-04-24T06:54:46.464 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.464 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.464 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.465 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-24T06:54:46.465 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.465 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.465 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.465 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.466 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": { 2024-04-24T06:54:46.466 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_name", 2024-04-24T06:54:46.466 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.466 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.466 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.467 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "device_health_metrics", 2024-04-24T06:54:46.467 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.467 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.467 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.467 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-24T06:54:46.468 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.468 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.468 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.468 INFO:teuthology.orchestra.run.smithi005.stdout: "retention_period": { 2024-04-24T06:54:46.469 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retention_period", 2024-04-24T06:54:46.469 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:46.469 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.469 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.469 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15552000", 2024-04-24T06:54:46.470 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.470 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.470 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.470 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain device health metrics", 2024-04-24T06:54:46.470 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.470 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.471 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.471 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_frequency": { 2024-04-24T06:54:46.471 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_frequency", 2024-04-24T06:54:46.471 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:46.472 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.472 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.472 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-04-24T06:54:46.472 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.472 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.473 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.473 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-24T06:54:46.473 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.473 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.474 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.474 INFO:teuthology.orchestra.run.smithi005.stdout: "self_heal": { 2024-04-24T06:54:46.474 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "self_heal", 2024-04-24T06:54:46.474 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.475 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.475 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.475 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:46.475 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.475 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.476 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.476 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-24T06:54:46.476 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.476 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.476 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.477 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-04-24T06:54:46.477 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-04-24T06:54:46.477 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:46.478 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.478 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.478 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-04-24T06:54:46.478 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.478 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.479 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.479 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and check device health", 2024-04-24T06:54:46.479 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.479 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.479 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.480 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_threshold": { 2024-04-24T06:54:46.480 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_threshold", 2024-04-24T06:54:46.480 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:46.480 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.481 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.481 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7257600", 2024-04-24T06:54:46.481 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.481 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.481 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.481 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-24T06:54:46.482 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.482 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.482 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.482 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.483 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.483 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:46.483 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "diskprediction_local", 2024-04-24T06:54:46.483 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:46.483 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:46.483 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:46.484 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:46.484 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:46.484 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.484 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.484 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.485 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.485 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.485 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.485 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.485 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.486 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.486 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.486 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.486 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.486 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.487 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.487 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.487 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.487 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.487 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.488 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:46.488 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:46.488 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.488 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.488 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.488 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.489 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.489 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.489 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.489 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.489 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.490 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.490 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.490 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:46.491 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:46.491 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.491 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.491 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.492 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:46.492 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.492 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.492 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.492 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.493 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.493 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.493 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.493 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.493 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.494 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.494 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.494 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.494 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.495 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.495 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:46.495 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:46.496 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.496 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.496 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.496 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.497 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.497 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.497 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.497 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.497 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.498 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.498 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.498 INFO:teuthology.orchestra.run.smithi005.stdout: "predict_interval": { 2024-04-24T06:54:46.498 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predict_interval", 2024-04-24T06:54:46.499 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.499 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.499 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.499 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-04-24T06:54:46.500 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.500 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.500 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.500 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.501 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.501 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.501 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.501 INFO:teuthology.orchestra.run.smithi005.stdout: "predictor_model": { 2024-04-24T06:54:46.502 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predictor_model", 2024-04-24T06:54:46.502 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.502 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.502 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.503 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "prophetstor", 2024-04-24T06:54:46.503 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.503 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.503 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.503 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.504 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.504 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.504 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.505 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-04-24T06:54:46.505 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-04-24T06:54:46.505 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.505 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.505 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.506 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-04-24T06:54:46.506 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.506 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.506 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.506 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.507 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.507 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.507 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.507 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.507 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:46.508 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "influx", 2024-04-24T06:54:46.508 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": false, 2024-04-24T06:54:46.508 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "influxdb python module not found", 2024-04-24T06:54:46.508 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:46.509 INFO:teuthology.orchestra.run.smithi005.stdout: "batch_size": { 2024-04-24T06:54:46.509 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "batch_size", 2024-04-24T06:54:46.509 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:46.509 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.510 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.510 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5000", 2024-04-24T06:54:46.510 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.510 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.510 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.511 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-24T06:54:46.511 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.511 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.511 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.512 INFO:teuthology.orchestra.run.smithi005.stdout: "database": { 2024-04-24T06:54:46.512 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "database", 2024-04-24T06:54:46.512 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.512 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.512 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.513 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "ceph", 2024-04-24T06:54:46.513 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.513 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.513 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.514 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-24T06:54:46.514 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.514 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.514 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.515 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": { 2024-04-24T06:54:46.515 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hostname", 2024-04-24T06:54:46.515 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.515 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.515 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.516 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.516 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.516 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.516 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.516 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server hostname", 2024-04-24T06:54:46.517 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.517 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.517 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.517 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-04-24T06:54:46.518 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-04-24T06:54:46.518 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:46.518 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.518 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.518 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "30", 2024-04-24T06:54:46.519 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "5", 2024-04-24T06:54:46.519 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.519 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.519 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-24T06:54:46.520 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.520 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.520 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.520 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:46.520 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:46.521 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.521 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.521 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.521 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.522 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.522 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.522 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.522 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.522 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.523 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.523 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.523 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.523 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.523 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.524 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.524 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.524 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.524 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.525 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:46.525 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:46.525 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.525 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.525 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.526 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.526 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.526 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.526 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.527 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.527 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.527 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.527 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.528 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:46.528 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:46.528 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.528 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.529 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.529 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:46.529 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.529 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.529 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.530 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.530 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.530 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.530 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.531 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.531 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.531 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.532 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.532 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.532 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.532 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.533 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:46.533 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:46.533 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.533 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.533 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.534 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.534 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.534 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.534 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.535 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.535 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.535 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.535 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.536 INFO:teuthology.orchestra.run.smithi005.stdout: "password": { 2024-04-24T06:54:46.536 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "password", 2024-04-24T06:54:46.536 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.536 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.537 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.537 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.537 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.537 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.537 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.538 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "password of InfluxDB server user", 2024-04-24T06:54:46.538 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.538 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.538 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.539 INFO:teuthology.orchestra.run.smithi005.stdout: "port": { 2024-04-24T06:54:46.539 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "port", 2024-04-24T06:54:46.539 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:46.539 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.540 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.540 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8086", 2024-04-24T06:54:46.541 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.541 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.541 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.541 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server port", 2024-04-24T06:54:46.542 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.542 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.542 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.543 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-04-24T06:54:46.543 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-04-24T06:54:46.543 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.543 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.543 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.543 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "false", 2024-04-24T06:54:46.544 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.544 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.544 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.544 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T06:54:46.544 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.545 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.545 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.545 INFO:teuthology.orchestra.run.smithi005.stdout: "threads": { 2024-04-24T06:54:46.546 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threads", 2024-04-24T06:54:46.546 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:46.546 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.546 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.546 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-04-24T06:54:46.547 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-04-24T06:54:46.547 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "32", 2024-04-24T06:54:46.547 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.547 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-24T06:54:46.547 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.548 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.548 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.548 INFO:teuthology.orchestra.run.smithi005.stdout: "username": { 2024-04-24T06:54:46.549 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "username", 2024-04-24T06:54:46.549 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.549 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.549 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.549 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.550 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.550 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.550 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.550 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "username of InfluxDB server user", 2024-04-24T06:54:46.551 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.551 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.551 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.551 INFO:teuthology.orchestra.run.smithi005.stdout: "verify_ssl": { 2024-04-24T06:54:46.552 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "verify_ssl", 2024-04-24T06:54:46.552 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.552 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.552 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.552 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "true", 2024-04-24T06:54:46.553 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.553 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.553 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.553 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T06:54:46.554 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.554 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.554 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.554 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.554 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:46.555 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "insights", 2024-04-24T06:54:46.555 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:46.555 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:46.556 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:46.556 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:46.556 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:46.556 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.556 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.557 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.557 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.557 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.557 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.558 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.558 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.558 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.558 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.559 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.559 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.559 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.559 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.559 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.560 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.560 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.560 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.561 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:46.561 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:46.561 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.561 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.561 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.562 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.562 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.562 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.562 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.563 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.563 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.563 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.563 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.564 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:46.564 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:46.564 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.564 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.565 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.565 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:46.565 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.565 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.566 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.566 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.566 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.566 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.566 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.567 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.567 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.567 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.567 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.568 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.568 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.568 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.568 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:46.569 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:46.569 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.569 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.569 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.569 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.570 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.570 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.573 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.574 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.574 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.574 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.574 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.574 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.574 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:46.575 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "iostat", 2024-04-24T06:54:46.575 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:46.575 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:46.576 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:46.576 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:46.576 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:46.576 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.576 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.576 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.576 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.577 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.577 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.577 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.577 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.577 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.577 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.577 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.577 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.578 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.578 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.578 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.578 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.578 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.578 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.578 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:46.579 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:46.579 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.579 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.579 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.579 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.579 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.579 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.580 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.580 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.580 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.580 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.580 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.580 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:46.581 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:46.581 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.581 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.581 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.581 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:46.581 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.581 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.581 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.582 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.582 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.582 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.582 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.582 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.582 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.582 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.582 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.583 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.583 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.583 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.583 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:46.583 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:46.583 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.584 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.584 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.584 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.584 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.584 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.584 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.584 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.585 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.585 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.585 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.585 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.585 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:46.585 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "k8sevents", 2024-04-24T06:54:46.586 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:46.586 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:46.586 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:46.586 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_event_retention_days": { 2024-04-24T06:54:46.586 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ceph_event_retention_days", 2024-04-24T06:54:46.586 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:46.586 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.587 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.587 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7", 2024-04-24T06:54:46.587 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.587 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.587 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.587 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-24T06:54:46.587 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.588 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.588 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.588 INFO:teuthology.orchestra.run.smithi005.stdout: "config_check_secs": { 2024-04-24T06:54:46.588 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_check_secs", 2024-04-24T06:54:46.588 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:46.588 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.588 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.589 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-24T06:54:46.589 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "10", 2024-04-24T06:54:46.589 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.589 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.589 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-24T06:54:46.589 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.589 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.589 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.590 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:46.590 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:46.590 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.590 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.590 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.590 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.590 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.591 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.591 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.591 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.591 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.591 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.591 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.591 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.591 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.591 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.592 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.592 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.592 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.592 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.592 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:46.592 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:46.593 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.593 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.593 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.593 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.593 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.593 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.593 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.593 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.594 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.594 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.594 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.594 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:46.594 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:46.594 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.594 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.595 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.595 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:46.595 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.595 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.595 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.595 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.595 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.595 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.596 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.596 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.596 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.596 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.596 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.596 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.596 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.596 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.597 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:46.597 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:46.597 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.597 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.597 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.597 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.597 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.598 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.598 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.598 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.598 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.598 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.598 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.598 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.598 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:46.599 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "localpool", 2024-04-24T06:54:46.599 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:46.599 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:46.599 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:46.599 INFO:teuthology.orchestra.run.smithi005.stdout: "failure_domain": { 2024-04-24T06:54:46.600 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "failure_domain", 2024-04-24T06:54:46.600 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.600 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.600 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.600 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "host", 2024-04-24T06:54:46.600 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.600 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.600 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.601 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "failure domain for any created local pool", 2024-04-24T06:54:46.601 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-24T06:54:46.601 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.601 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.601 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:46.601 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:46.601 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.602 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.602 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.602 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.602 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.602 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.602 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.602 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.602 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.603 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.603 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.603 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.603 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.603 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.603 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.603 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.603 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.604 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.604 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:46.604 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:46.604 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.605 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.605 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.605 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.605 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.605 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.605 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.605 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.605 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.606 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.606 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.606 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:46.606 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:46.606 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.606 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.606 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.607 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:46.607 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.607 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.607 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.607 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.607 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.607 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.607 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.608 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.608 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.608 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.608 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.608 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.608 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.608 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.609 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:46.609 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:46.609 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.609 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.609 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.609 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.609 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.609 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.610 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.610 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.610 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.610 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.610 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.611 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": { 2024-04-24T06:54:46.611 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_size", 2024-04-24T06:54:46.611 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:46.611 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.611 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.611 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.611 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.611 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.612 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.612 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default min_size for any created local pool", 2024-04-24T06:54:46.612 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-24T06:54:46.612 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.612 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.612 INFO:teuthology.orchestra.run.smithi005.stdout: "num_rep": { 2024-04-24T06:54:46.613 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "num_rep", 2024-04-24T06:54:46.613 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:46.613 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.613 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.613 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3", 2024-04-24T06:54:46.613 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.613 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.613 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.614 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default replica count for any created local pool", 2024-04-24T06:54:46.614 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.614 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.614 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.614 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": { 2024-04-24T06:54:46.614 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_num", 2024-04-24T06:54:46.615 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:46.615 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.615 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.615 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "128", 2024-04-24T06:54:46.615 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.615 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.615 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.615 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default pg_num for any created local pool", 2024-04-24T06:54:46.615 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.616 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.616 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.616 INFO:teuthology.orchestra.run.smithi005.stdout: "prefix": { 2024-04-24T06:54:46.616 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prefix", 2024-04-24T06:54:46.616 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.616 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.617 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.617 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.617 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.617 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.617 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.617 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name prefix for any created local pool", 2024-04-24T06:54:46.617 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.617 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.618 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.618 INFO:teuthology.orchestra.run.smithi005.stdout: "subtree": { 2024-04-24T06:54:46.618 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "subtree", 2024-04-24T06:54:46.618 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.618 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.618 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.618 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "rack", 2024-04-24T06:54:46.619 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.619 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.619 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.619 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-24T06:54:46.619 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-24T06:54:46.619 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.619 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.620 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.620 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:46.620 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mds_autoscaler", 2024-04-24T06:54:46.620 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:46.620 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:46.620 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:46.621 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:46.621 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:46.621 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.621 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.621 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.621 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.621 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.621 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.622 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.622 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.622 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.622 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.622 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.622 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.622 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.623 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.623 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.623 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.623 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.623 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.623 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:46.623 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:46.624 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.624 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.624 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.624 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.624 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.624 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.624 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.624 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.625 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.625 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.625 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.625 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:46.625 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:46.625 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.625 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.626 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.626 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:46.626 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.626 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.626 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.626 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.626 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.627 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.627 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.627 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.627 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.627 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.627 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.627 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.627 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.628 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.628 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:46.628 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:46.628 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.628 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.628 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.629 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.629 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.629 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.629 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.629 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.629 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.629 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.629 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.630 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.630 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:46.630 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirroring", 2024-04-24T06:54:46.630 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:46.630 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:46.630 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:46.631 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:46.631 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:46.631 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.631 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.631 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.631 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.631 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.632 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.632 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.632 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.632 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.632 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.632 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.632 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.632 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.633 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.633 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.633 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.633 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.633 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.633 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:46.633 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:46.634 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.634 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.634 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.634 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.634 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.634 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.634 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.635 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.635 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.635 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.635 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.635 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:46.635 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:46.635 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.636 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.636 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.636 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:46.636 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.636 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.636 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.636 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.636 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.637 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.637 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.637 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.637 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.637 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.637 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.637 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.637 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.638 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.638 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:46.638 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:46.638 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.638 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.638 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.638 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.639 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.639 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.639 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.639 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.639 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.639 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.639 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.640 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.640 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:46.640 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "nfs", 2024-04-24T06:54:46.640 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:46.640 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:46.640 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:46.641 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:46.641 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:46.641 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.641 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.641 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.641 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.641 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.642 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.642 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.642 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.642 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.642 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.642 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.642 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.642 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.643 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.643 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.643 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.643 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.643 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.643 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:46.643 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:46.644 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.644 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.644 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.644 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.644 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.644 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.644 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.644 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.645 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.645 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.645 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.645 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:46.645 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:46.645 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.646 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.646 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.646 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:46.646 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.646 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.646 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.646 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.646 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.647 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.647 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.647 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.647 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.647 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.647 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.647 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.647 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.648 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.648 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:46.648 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:46.648 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.648 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.648 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.648 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.649 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.649 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.649 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.649 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.649 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.649 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.649 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.649 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.650 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:46.650 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-04-24T06:54:46.650 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:46.650 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:46.650 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:46.650 INFO:teuthology.orchestra.run.smithi005.stdout: "fail_fs": { 2024-04-24T06:54:46.651 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "fail_fs", 2024-04-24T06:54:46.651 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.651 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.651 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.651 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.651 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.651 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.652 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.652 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-24T06:54:46.652 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.652 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.652 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.652 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:46.653 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:46.653 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.653 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.653 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.653 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.653 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.653 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.654 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.654 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.654 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.654 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.654 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.654 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.654 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.655 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.655 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.655 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.655 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.655 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.655 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:46.655 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:46.656 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.656 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.656 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.656 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.656 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.656 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.656 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.656 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.657 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.657 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.657 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.657 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:46.657 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:46.657 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.657 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.658 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.658 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:46.658 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.658 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.658 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.658 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.659 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.659 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.659 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.659 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.659 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.659 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.659 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.660 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.660 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.660 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.660 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:46.660 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:46.660 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.660 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.661 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.661 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.661 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.661 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.661 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.661 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.661 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.662 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.662 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.662 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.662 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator": { 2024-04-24T06:54:46.662 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-04-24T06:54:46.662 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.662 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.663 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.663 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.663 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.663 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.663 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.663 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2024-04-24T06:54:46.663 INFO:teuthology.orchestra.run.smithi005.stdout: "rook", 2024-04-24T06:54:46.663 INFO:teuthology.orchestra.run.smithi005.stdout: "test_orchestrator" 2024-04-24T06:54:46.664 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.664 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Orchestrator backend", 2024-04-24T06:54:46.664 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.664 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.664 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.664 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.664 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.665 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:46.665 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_perf_query", 2024-04-24T06:54:46.665 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:46.665 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:46.665 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:46.665 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:46.665 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:46.666 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.666 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.666 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.666 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.666 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.666 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.666 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.667 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.667 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.667 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.667 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.667 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.667 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.667 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.667 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.668 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.668 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.668 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.668 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:46.668 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:46.668 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.668 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.669 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.669 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.669 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.669 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.669 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.669 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.669 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.669 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.670 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.670 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.670 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:46.670 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:46.670 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.670 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.670 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.671 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:46.671 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.671 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.671 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.671 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.671 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.671 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.672 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.672 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.672 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.672 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.672 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.672 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.672 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.672 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.673 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:46.673 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:46.673 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.673 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.673 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.673 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.673 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.674 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.674 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.674 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.674 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.674 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.674 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.674 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.675 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.675 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.675 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:46.675 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_support", 2024-04-24T06:54:46.675 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:46.675 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:46.675 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:46.675 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:46.676 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:46.676 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.676 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.676 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.676 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.676 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.676 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.677 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.677 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.677 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.677 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.677 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.677 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.677 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.677 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.678 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.678 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.678 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.678 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.678 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:46.678 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:46.679 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.679 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.679 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.679 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.679 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.679 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.679 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.679 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.680 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.680 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.680 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.680 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:46.680 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:46.680 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.681 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.681 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.681 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:46.681 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.681 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.681 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.681 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.682 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.682 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.682 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.682 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.682 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.682 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.682 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.683 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.683 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.683 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.683 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:46.683 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:46.683 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.683 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.684 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.684 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.684 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.684 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.684 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.684 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.684 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.684 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.685 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.685 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.685 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:46.685 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_autoscaler", 2024-04-24T06:54:46.685 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:46.686 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:46.686 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:46.686 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:46.686 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:46.686 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.686 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.686 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.687 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.687 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.687 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.687 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.687 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.687 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.687 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.687 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.688 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.688 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.688 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.688 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.688 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.688 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.688 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.688 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.689 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:46.689 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:46.689 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.689 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.689 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.689 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.689 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.689 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.690 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.690 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.690 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.690 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.690 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.690 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:46.691 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:46.691 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.691 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.691 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.691 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:46.691 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.691 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.691 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.692 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.692 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.692 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.692 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.692 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.692 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.692 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.692 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.693 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.693 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.693 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.693 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:46.693 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:46.693 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.693 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.694 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.694 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.694 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.694 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.694 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.694 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.694 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.694 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.695 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.695 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-04-24T06:54:46.695 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-04-24T06:54:46.695 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:46.695 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.695 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.695 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-04-24T06:54:46.695 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.696 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.696 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.696 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.696 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.696 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.696 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.696 INFO:teuthology.orchestra.run.smithi005.stdout: "threshold": { 2024-04-24T06:54:46.697 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threshold", 2024-04-24T06:54:46.697 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-24T06:54:46.697 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.697 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.697 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-04-24T06:54:46.697 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1.0", 2024-04-24T06:54:46.697 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.697 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.698 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "scaling threshold", 2024-04-24T06:54:46.698 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-24T06:54:46.698 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.698 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.698 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.698 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.699 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:46.699 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "progress", 2024-04-24T06:54:46.699 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:46.699 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:46.699 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:46.699 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_pg_recovery_event": { 2024-04-24T06:54:46.699 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_pg_recovery_event", 2024-04-24T06:54:46.699 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.700 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.700 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.700 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.700 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.700 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.700 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.700 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-24T06:54:46.700 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.701 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.701 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.701 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-04-24T06:54:46.701 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-04-24T06:54:46.701 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.701 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.702 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.702 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:46.702 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.702 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.702 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.702 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.702 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.702 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.703 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.703 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:46.703 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:46.703 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.703 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.703 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.703 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.704 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.704 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.704 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.704 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.704 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.704 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.704 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.705 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.705 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.705 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.705 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.705 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.705 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.705 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.706 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:46.706 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:46.706 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.706 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.706 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.706 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.707 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.707 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.707 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.707 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.707 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.707 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.707 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.708 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:46.708 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:46.708 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.708 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.708 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.708 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:46.708 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.708 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.709 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.709 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.709 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.709 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.709 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.709 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.709 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.709 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.710 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.710 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.710 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.710 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.710 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:46.710 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:46.710 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.711 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.711 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.711 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.711 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.711 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.711 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.711 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.711 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.712 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.712 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.712 INFO:teuthology.orchestra.run.smithi005.stdout: "max_completed_events": { 2024-04-24T06:54:46.712 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_completed_events", 2024-04-24T06:54:46.712 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:46.712 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.713 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.713 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "50", 2024-04-24T06:54:46.713 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.713 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.713 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.713 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "number of past completed events to remember", 2024-04-24T06:54:46.713 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.713 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.714 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.714 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-04-24T06:54:46.714 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-04-24T06:54:46.714 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:46.714 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.714 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.714 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-04-24T06:54:46.715 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.715 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.715 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.715 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long the module is going to sleep", 2024-04-24T06:54:46.715 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.715 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.715 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.715 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.716 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.716 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:46.716 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus", 2024-04-24T06:54:46.716 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:46.716 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:46.716 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:46.716 INFO:teuthology.orchestra.run.smithi005.stdout: "cache": { 2024-04-24T06:54:46.717 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cache", 2024-04-24T06:54:46.717 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.717 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.717 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.717 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:46.717 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.717 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.717 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.718 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.718 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.718 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.718 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.718 INFO:teuthology.orchestra.run.smithi005.stdout: "exclude_perf_counters": { 2024-04-24T06:54:46.718 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "exclude_perf_counters", 2024-04-24T06:54:46.718 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.719 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.719 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.719 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:46.719 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.719 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.719 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.719 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-24T06:54:46.720 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-24T06:54:46.720 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.720 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.720 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:46.720 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:46.720 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.720 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.721 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.721 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.721 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.721 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.721 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.721 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.721 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.721 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.722 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.722 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.722 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.722 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.722 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.722 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.722 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.723 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.723 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:46.723 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:46.723 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.723 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.723 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.723 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.724 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.724 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.724 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.724 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.724 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.724 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.724 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.725 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:46.725 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:46.725 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.725 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.725 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.725 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:46.725 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.725 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.725 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.726 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.726 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.726 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.726 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.726 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.726 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.726 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.727 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.727 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.727 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.727 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.727 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:46.727 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:46.727 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.728 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.728 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.728 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.728 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.728 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.728 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.728 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.728 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.729 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.729 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.729 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools": { 2024-04-24T06:54:46.729 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools", 2024-04-24T06:54:46.729 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.729 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.729 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.730 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.730 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.730 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.730 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.730 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.730 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.730 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.731 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.731 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-24T06:54:46.731 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-24T06:54:46.731 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:46.731 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.731 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.731 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "300", 2024-04-24T06:54:46.732 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.732 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.732 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.732 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.732 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.732 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.732 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.733 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_interval": { 2024-04-24T06:54:46.733 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_interval", 2024-04-24T06:54:46.733 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-24T06:54:46.733 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.733 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.733 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15.0", 2024-04-24T06:54:46.733 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.734 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.734 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.734 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.734 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.734 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.734 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.734 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-04-24T06:54:46.735 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-04-24T06:54:46.735 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.735 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.735 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.735 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-04-24T06:54:46.735 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.735 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.735 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.736 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-24T06:54:46.736 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.736 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.736 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.736 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-04-24T06:54:46.736 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-04-24T06:54:46.736 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:46.737 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.737 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.737 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "9283", 2024-04-24T06:54:46.737 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.737 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.737 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.737 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-24T06:54:46.738 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.738 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.738 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.738 INFO:teuthology.orchestra.run.smithi005.stdout: "stale_cache_strategy": { 2024-04-24T06:54:46.738 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stale_cache_strategy", 2024-04-24T06:54:46.738 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.738 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.739 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.739 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "log", 2024-04-24T06:54:46.739 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.739 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.739 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.739 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.739 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.739 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.740 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.740 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-04-24T06:54:46.740 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-04-24T06:54:46.740 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.740 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.740 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.740 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "default", 2024-04-24T06:54:46.741 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.741 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.741 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.741 INFO:teuthology.orchestra.run.smithi005.stdout: "default", 2024-04-24T06:54:46.741 INFO:teuthology.orchestra.run.smithi005.stdout: "error" 2024-04-24T06:54:46.741 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.741 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.741 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.742 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.742 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.742 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-04-24T06:54:46.742 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-04-24T06:54:46.742 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:46.742 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.743 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.743 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-04-24T06:54:46.743 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-04-24T06:54:46.743 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-04-24T06:54:46.743 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.743 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.743 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.743 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.743 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.744 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.744 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.744 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:46.744 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2024-04-24T06:54:46.744 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:46.744 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:46.744 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:46.745 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:46.745 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:46.745 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.745 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.745 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.745 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.745 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.745 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.746 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.746 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.746 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.746 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.746 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.746 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.746 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.747 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.747 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.747 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.747 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.747 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.747 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:46.747 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:46.748 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.748 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.748 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.748 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.748 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.748 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.748 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.748 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.749 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.749 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.749 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.749 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:46.749 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:46.749 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.749 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.750 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.750 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:46.750 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.750 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.750 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.750 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.750 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.751 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.751 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.751 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.751 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.751 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.751 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.751 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.751 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.752 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.752 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:46.752 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:46.752 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.752 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.752 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.752 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.753 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.753 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.753 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.753 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.753 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.753 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.753 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.754 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_snap_create": { 2024-04-24T06:54:46.754 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_snap_create", 2024-04-24T06:54:46.754 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:46.754 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.754 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.754 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-24T06:54:46.754 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.755 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.755 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.755 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.755 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.755 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.755 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.756 INFO:teuthology.orchestra.run.smithi005.stdout: "mirror_snapshot_schedule": { 2024-04-24T06:54:46.756 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirror_snapshot_schedule", 2024-04-24T06:54:46.756 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.756 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.756 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.756 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.756 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.756 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.757 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.757 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.757 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.757 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.757 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.757 INFO:teuthology.orchestra.run.smithi005.stdout: "trash_purge_schedule": { 2024-04-24T06:54:46.757 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "trash_purge_schedule", 2024-04-24T06:54:46.758 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.758 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.758 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.758 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.758 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.758 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.758 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.758 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.759 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.759 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.759 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.759 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.759 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.759 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:46.759 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "restful", 2024-04-24T06:54:46.760 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:46.760 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:46.760 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:46.760 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_auth": { 2024-04-24T06:54:46.760 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_auth", 2024-04-24T06:54:46.760 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.760 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.760 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.761 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:46.761 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.761 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.761 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.761 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.761 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.761 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.761 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.762 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-04-24T06:54:46.762 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-04-24T06:54:46.762 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.762 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.762 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.762 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.762 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.763 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.763 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.763 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.763 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.763 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.763 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.763 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:46.764 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:46.764 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.764 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.764 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.764 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.764 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.764 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.764 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.765 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.765 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.765 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.765 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.765 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.765 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.765 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.765 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.766 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.766 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.766 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.766 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:46.766 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:46.766 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.766 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.767 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.767 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.767 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.767 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.767 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.767 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.767 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.767 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.768 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.768 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:46.768 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:46.768 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.768 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.768 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.768 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:46.769 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.769 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.769 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.769 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.769 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.769 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.769 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.769 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.770 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.770 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.770 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.770 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.770 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.770 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.770 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:46.771 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:46.771 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.771 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.771 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.771 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.771 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.771 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.771 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.772 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.772 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.772 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.772 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.772 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-04-24T06:54:46.772 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-04-24T06:54:46.773 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.773 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.773 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.773 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.773 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.773 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.773 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.773 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.774 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.774 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.774 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.774 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-04-24T06:54:46.774 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-04-24T06:54:46.774 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.774 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.775 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.775 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.775 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.775 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.775 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.778 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.778 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.778 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.778 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.778 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.778 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:46.779 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rgw", 2024-04-24T06:54:46.779 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:46.779 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:46.779 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:46.779 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:46.780 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:46.780 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.780 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.780 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.780 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.780 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.780 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.781 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.781 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.781 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.781 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.781 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.782 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.782 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.782 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.782 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.782 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.782 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.782 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.783 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:46.783 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:46.783 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.783 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.783 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.783 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.784 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.784 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.784 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.784 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.784 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.784 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.784 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.785 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:46.785 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:46.785 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.785 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.785 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.785 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:46.785 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.786 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.786 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.786 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.786 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.786 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.786 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.786 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.787 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.787 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.787 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.787 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.787 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.787 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.788 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:46.788 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:46.788 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.788 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.788 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.789 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.789 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.789 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.789 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.790 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.790 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.790 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.791 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.791 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.791 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:46.792 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rook", 2024-04-24T06:54:46.792 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:46.792 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:46.793 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:46.793 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:46.793 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:46.793 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.793 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.793 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.793 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.794 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.794 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.794 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.794 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.794 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.794 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.795 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.795 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.795 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.795 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.795 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.795 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.795 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.795 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.796 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:46.796 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:46.796 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.796 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.796 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.797 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.797 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.797 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.797 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.797 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.797 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.797 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.798 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.798 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:46.798 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:46.798 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.799 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.799 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.799 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:46.799 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.799 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.800 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.800 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.800 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.800 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.800 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.800 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.800 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.801 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.801 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.801 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.801 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.801 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.801 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:46.801 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:46.802 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.802 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.802 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.802 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.802 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.802 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.802 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.803 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.803 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.803 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.803 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.803 INFO:teuthology.orchestra.run.smithi005.stdout: "storage_class": { 2024-04-24T06:54:46.803 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "storage_class", 2024-04-24T06:54:46.804 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.804 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.804 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.804 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "local", 2024-04-24T06:54:46.804 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.804 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.804 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.804 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-24T06:54:46.805 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.805 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.805 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.805 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.805 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.805 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:46.805 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "selftest", 2024-04-24T06:54:46.806 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:46.806 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:46.806 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:46.806 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:46.806 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:46.806 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.806 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.807 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.807 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.807 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.807 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.807 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.807 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.807 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.808 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.808 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.808 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.808 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.808 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.808 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.808 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.808 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.809 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.809 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:46.809 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:46.809 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.809 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.809 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.809 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.810 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.810 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.810 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.810 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.810 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.810 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.810 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.811 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:46.811 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:46.811 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.811 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.811 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.811 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:46.811 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.812 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.812 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.812 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.812 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.812 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.812 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.812 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.813 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.813 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.813 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.813 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.813 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.813 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.813 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:46.814 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:46.814 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.814 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.814 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.814 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.814 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.815 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.815 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.815 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.815 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.815 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.815 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.816 INFO:teuthology.orchestra.run.smithi005.stdout: "roption1": { 2024-04-24T06:54:46.816 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption1", 2024-04-24T06:54:46.816 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.816 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.816 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.816 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.816 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.817 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.817 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.817 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.817 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.817 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.817 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.818 INFO:teuthology.orchestra.run.smithi005.stdout: "roption2": { 2024-04-24T06:54:46.818 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption2", 2024-04-24T06:54:46.818 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.818 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.819 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.819 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "xyz", 2024-04-24T06:54:46.819 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.819 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.819 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.820 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.820 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.820 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.820 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.820 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption1": { 2024-04-24T06:54:46.820 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption1", 2024-04-24T06:54:46.821 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.821 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.821 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.821 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.821 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.821 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.821 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.821 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.821 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.822 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.822 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.822 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption2": { 2024-04-24T06:54:46.822 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption2", 2024-04-24T06:54:46.823 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:46.823 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.823 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.823 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.823 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.823 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.823 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.823 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.824 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.824 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.824 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.824 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption3": { 2024-04-24T06:54:46.824 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption3", 2024-04-24T06:54:46.824 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-24T06:54:46.824 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.825 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.825 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.825 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.825 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.825 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.825 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.825 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.825 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.825 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.826 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption4": { 2024-04-24T06:54:46.826 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption4", 2024-04-24T06:54:46.826 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.826 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.826 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.826 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.827 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.827 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.827 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.827 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.827 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.827 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.827 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.827 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption5": { 2024-04-24T06:54:46.828 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption5", 2024-04-24T06:54:46.828 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.828 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.828 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.828 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.828 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.828 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.828 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.829 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.829 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.829 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.829 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.829 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption6": { 2024-04-24T06:54:46.829 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption6", 2024-04-24T06:54:46.829 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.829 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.830 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.830 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:46.830 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.830 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.830 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.830 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.830 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.831 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.831 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.831 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption7": { 2024-04-24T06:54:46.831 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption7", 2024-04-24T06:54:46.831 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:46.832 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.832 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.832 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.832 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-04-24T06:54:46.832 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "42", 2024-04-24T06:54:46.832 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.832 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.832 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.833 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.833 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.833 INFO:teuthology.orchestra.run.smithi005.stdout: "testkey": { 2024-04-24T06:54:46.833 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testkey", 2024-04-24T06:54:46.833 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.833 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.833 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.834 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.834 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.834 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.834 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.834 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.834 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.834 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.835 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.835 INFO:teuthology.orchestra.run.smithi005.stdout: "testlkey": { 2024-04-24T06:54:46.835 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testlkey", 2024-04-24T06:54:46.835 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.835 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.835 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.835 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.835 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.836 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.836 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.836 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.836 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.836 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.836 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.836 INFO:teuthology.orchestra.run.smithi005.stdout: "testnewline": { 2024-04-24T06:54:46.837 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testnewline", 2024-04-24T06:54:46.837 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.837 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.837 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.837 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.838 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.838 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.838 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.838 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.838 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.838 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.838 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.839 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.839 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.839 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:46.839 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2024-04-24T06:54:46.839 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:46.840 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:46.840 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:46.840 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_m_granularity": { 2024-04-24T06:54:46.840 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_m_granularity", 2024-04-24T06:54:46.841 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.841 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.841 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.841 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.842 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.842 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.842 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.842 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow minute scheduled snapshots", 2024-04-24T06:54:46.842 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.842 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.842 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.843 INFO:teuthology.orchestra.run.smithi005.stdout: "dump_on_update": { 2024-04-24T06:54:46.843 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dump_on_update", 2024-04-24T06:54:46.843 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.843 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.843 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.843 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.843 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.843 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.844 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.844 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "dump database to debug log on update", 2024-04-24T06:54:46.844 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.844 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.844 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.844 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:46.844 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:46.845 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.845 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.845 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.845 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.845 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.845 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.845 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.845 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.846 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.846 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.846 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.846 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.846 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.846 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.846 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.846 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.847 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.847 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.847 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:46.847 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:46.847 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.847 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.847 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.848 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.848 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.848 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.848 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.848 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.848 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.848 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.848 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.849 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:46.849 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:46.849 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.849 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.849 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.849 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:46.849 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.850 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.850 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.850 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.850 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.850 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.850 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.850 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.850 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.851 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.851 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.851 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.851 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.851 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.851 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:46.851 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:46.852 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.852 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.852 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.852 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.852 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.852 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.852 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.852 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.853 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.853 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.853 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.853 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.853 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:46.853 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stats", 2024-04-24T06:54:46.854 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:46.854 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:46.854 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:46.854 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:46.854 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:46.854 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.854 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.854 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.855 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.855 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.855 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.855 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.855 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.855 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.855 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.855 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.856 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.856 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.856 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.856 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.856 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.856 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.856 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.857 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:46.857 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:46.857 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.857 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.857 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.857 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.857 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.857 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.858 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.858 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.858 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.858 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.858 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.858 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:46.858 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:46.859 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.859 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.859 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.859 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:46.859 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.859 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.859 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.859 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.860 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.860 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.860 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.860 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.860 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.860 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.860 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.860 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.860 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.861 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.861 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:46.861 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:46.861 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.861 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.861 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.861 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.862 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.862 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.862 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.862 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.862 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.862 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.862 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.862 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.863 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:46.863 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "status", 2024-04-24T06:54:46.863 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:46.863 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:46.863 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:46.863 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:46.863 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:46.864 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.864 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.864 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.864 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.864 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.864 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.864 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.864 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.864 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.865 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.865 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.865 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.865 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.865 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.865 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.865 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.865 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.866 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.866 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:46.866 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:46.866 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.866 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.866 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.866 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.867 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.867 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.867 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.867 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.867 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.867 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.867 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.867 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:46.868 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:46.868 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.868 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.868 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.868 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:46.868 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.868 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.868 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.869 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.869 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.869 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.869 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.869 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.869 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.869 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.870 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.870 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.870 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.870 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.870 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:46.870 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:46.870 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.871 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.871 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.871 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.871 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.871 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.871 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.871 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.871 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.872 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.872 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.872 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.872 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.872 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:46.872 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telegraf", 2024-04-24T06:54:46.872 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:46.873 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:46.873 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:46.873 INFO:teuthology.orchestra.run.smithi005.stdout: "address": { 2024-04-24T06:54:46.873 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "address", 2024-04-24T06:54:46.873 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.873 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.873 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.873 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-24T06:54:46.874 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.874 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.874 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.874 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.874 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.874 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.874 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.875 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-04-24T06:54:46.875 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-04-24T06:54:46.875 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:46.875 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.875 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.875 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15", 2024-04-24T06:54:46.875 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.876 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.876 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.876 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.876 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.876 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.876 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.876 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:46.877 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:46.877 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.877 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.877 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.877 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.877 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.877 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.877 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.878 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.878 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.878 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.878 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.878 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.878 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.878 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.879 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.879 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.879 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.879 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.879 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:46.879 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:46.880 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.880 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.880 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.880 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.880 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.880 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.880 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.880 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.881 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.881 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.881 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.881 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:46.881 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:46.881 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.881 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.881 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.882 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:46.882 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.882 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.882 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.882 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.882 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.883 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.883 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.883 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.883 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.883 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.883 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.883 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.883 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.884 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.884 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:46.884 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:46.884 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.884 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.884 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.884 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.885 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.885 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.885 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.885 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.885 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.885 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.885 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.885 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.886 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.886 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:46.886 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telemetry", 2024-04-24T06:54:46.886 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:46.886 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:46.886 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:46.886 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_basic": { 2024-04-24T06:54:46.887 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_basic", 2024-04-24T06:54:46.887 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.887 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.887 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.887 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:46.887 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.887 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.887 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.888 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-24T06:54:46.888 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.888 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.888 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.888 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_crash": { 2024-04-24T06:54:46.888 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_crash", 2024-04-24T06:54:46.889 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.889 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.889 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.889 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:46.889 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.889 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.889 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.890 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-24T06:54:46.890 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.890 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.890 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.890 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_device": { 2024-04-24T06:54:46.890 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_device", 2024-04-24T06:54:46.891 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.891 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.891 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.891 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:46.891 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.891 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.891 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.891 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-24T06:54:46.892 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.892 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.892 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.892 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_ident": { 2024-04-24T06:54:46.892 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_ident", 2024-04-24T06:54:46.892 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.892 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.893 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.893 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.893 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.893 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.893 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.893 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-24T06:54:46.893 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.893 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.893 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.894 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_perf": { 2024-04-24T06:54:46.894 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_perf", 2024-04-24T06:54:46.894 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.894 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.894 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.894 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.895 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.895 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.895 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.895 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-24T06:54:46.895 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.895 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.895 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.896 INFO:teuthology.orchestra.run.smithi005.stdout: "contact": { 2024-04-24T06:54:46.896 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "contact", 2024-04-24T06:54:46.896 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.896 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.896 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.896 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.896 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.896 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.897 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.897 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.897 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.897 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.897 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.897 INFO:teuthology.orchestra.run.smithi005.stdout: "description": { 2024-04-24T06:54:46.897 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "description", 2024-04-24T06:54:46.898 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.898 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.898 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.898 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.898 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.898 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.898 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.898 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.899 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.899 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.899 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.899 INFO:teuthology.orchestra.run.smithi005.stdout: "device_url": { 2024-04-24T06:54:46.899 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_url", 2024-04-24T06:54:46.899 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.899 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.899 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.900 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-24T06:54:46.900 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.900 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.900 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.900 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.900 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.900 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.900 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.901 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-04-24T06:54:46.901 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-04-24T06:54:46.901 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.901 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.901 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.901 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.901 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.901 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.902 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.902 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.902 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.902 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.902 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.902 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-04-24T06:54:46.902 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-04-24T06:54:46.903 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:46.903 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.903 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.903 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "24", 2024-04-24T06:54:46.903 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "8", 2024-04-24T06:54:46.903 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.903 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.904 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.904 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.904 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.904 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.904 INFO:teuthology.orchestra.run.smithi005.stdout: "last_opt_revision": { 2024-04-24T06:54:46.905 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "last_opt_revision", 2024-04-24T06:54:46.905 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:46.905 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.905 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.906 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1", 2024-04-24T06:54:46.906 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.906 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.906 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.907 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.907 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.907 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.907 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.908 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard": { 2024-04-24T06:54:46.908 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard", 2024-04-24T06:54:46.908 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.909 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.909 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.909 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.909 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.910 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.910 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.910 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.911 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.911 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.911 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.912 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard_description": { 2024-04-24T06:54:46.912 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard_description", 2024-04-24T06:54:46.912 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.912 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.913 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.913 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.913 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.913 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.914 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.914 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.914 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.914 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.914 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.915 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:46.915 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:46.915 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.916 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.916 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.916 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.916 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.917 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.917 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.917 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.918 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.918 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.918 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.918 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.919 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.919 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.919 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.919 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.920 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.920 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.920 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:46.921 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:46.921 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.921 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.922 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.922 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.922 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.922 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.923 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.923 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.923 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.923 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.924 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.924 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:46.925 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:46.925 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.925 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.925 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.926 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:46.926 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.926 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.926 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.927 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.927 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.927 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.927 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.928 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.928 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.928 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.929 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.929 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.929 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.929 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.930 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:46.930 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:46.931 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.931 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.931 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.931 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.932 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.932 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.932 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.932 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.933 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.933 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.933 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.934 INFO:teuthology.orchestra.run.smithi005.stdout: "organization": { 2024-04-24T06:54:46.934 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "organization", 2024-04-24T06:54:46.934 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.934 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.935 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.935 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.935 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.935 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.935 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.936 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.936 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.936 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.936 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.937 INFO:teuthology.orchestra.run.smithi005.stdout: "proxy": { 2024-04-24T06:54:46.937 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "proxy", 2024-04-24T06:54:46.937 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.937 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.937 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.938 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.938 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.938 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.938 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.939 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.939 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.939 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.939 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.939 INFO:teuthology.orchestra.run.smithi005.stdout: "url": { 2024-04-24T06:54:46.939 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url", 2024-04-24T06:54:46.940 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.940 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.940 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.940 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-24T06:54:46.940 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.940 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.941 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.941 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.941 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.941 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.941 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.941 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.941 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:46.942 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "test_orchestrator", 2024-04-24T06:54:46.942 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:46.942 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:46.942 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:46.943 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:46.943 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:46.943 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.944 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.944 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.944 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.944 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.944 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.944 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.944 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.945 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.945 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.945 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.945 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.945 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.946 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.946 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.946 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.946 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.946 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.946 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.946 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:46.947 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:46.947 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.947 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.947 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.947 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.947 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.947 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.947 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.947 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.948 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.948 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.948 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.948 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:46.948 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:46.948 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.948 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.949 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.949 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:46.949 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.949 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.949 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.949 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.949 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.949 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.949 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.950 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.950 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.950 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.950 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.950 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.950 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.950 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.951 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:46.951 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:46.951 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.951 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.951 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.951 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.951 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.951 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.952 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.952 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.952 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.952 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.952 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.952 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.952 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.952 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:46.953 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2024-04-24T06:54:46.953 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:46.953 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:46.953 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:46.953 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:46.953 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:46.953 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.953 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.953 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.954 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.954 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.954 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.954 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.954 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.954 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.954 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.955 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.955 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.955 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.955 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.955 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.955 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.955 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.956 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.956 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.956 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:46.956 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:46.956 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.956 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.956 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.956 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.957 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.957 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.957 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.957 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.957 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.957 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.957 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.958 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:46.958 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:46.958 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.958 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.958 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.958 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:46.958 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.958 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.959 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.959 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.959 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.959 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.959 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.959 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.959 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.959 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.960 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.960 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.960 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.960 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.960 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.961 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:46.961 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:46.961 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.961 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.961 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.961 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.961 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.961 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.962 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.962 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.962 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.962 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.962 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.962 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_clones": { 2024-04-24T06:54:46.962 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_clones", 2024-04-24T06:54:46.963 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:46.963 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.963 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.963 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4", 2024-04-24T06:54:46.963 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.963 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.963 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.963 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-24T06:54:46.963 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.964 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.964 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.964 INFO:teuthology.orchestra.run.smithi005.stdout: "periodic_async_work": { 2024-04-24T06:54:46.964 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "periodic_async_work", 2024-04-24T06:54:46.964 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.964 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.964 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.965 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.965 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.965 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.965 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.965 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Periodically check for async work", 2024-04-24T06:54:46.965 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.965 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.965 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.966 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_delay": { 2024-04-24T06:54:46.966 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_delay", 2024-04-24T06:54:46.966 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:46.966 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.966 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.966 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-04-24T06:54:46.966 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.967 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.967 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.967 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-24T06:54:46.967 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.967 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.967 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.967 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_no_wait": { 2024-04-24T06:54:46.968 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_no_wait", 2024-04-24T06:54:46.968 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.968 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.968 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.968 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:46.968 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.968 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.968 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.968 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-24T06:54:46.969 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.969 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.969 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.969 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.969 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.969 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.969 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:46.969 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix", 2024-04-24T06:54:46.970 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:46.970 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:46.970 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:46.970 INFO:teuthology.orchestra.run.smithi005.stdout: "discovery_interval": { 2024-04-24T06:54:46.970 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "discovery_interval", 2024-04-24T06:54:46.970 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-04-24T06:54:46.970 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.971 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.971 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "100", 2024-04-24T06:54:46.971 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.971 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.971 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.971 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.971 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.971 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.972 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.972 INFO:teuthology.orchestra.run.smithi005.stdout: "identifier": { 2024-04-24T06:54:46.972 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "identifier", 2024-04-24T06:54:46.972 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.972 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.972 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.972 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.973 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.973 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.973 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.973 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.973 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.973 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.973 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.974 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-04-24T06:54:46.974 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-04-24T06:54:46.974 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:46.974 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.974 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.974 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-04-24T06:54:46.974 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.975 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.975 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.975 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.975 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.975 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.975 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.976 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:46.976 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:46.976 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.976 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.976 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.976 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.976 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.976 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.976 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.977 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.977 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.977 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.977 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.977 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.977 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.977 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.977 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.978 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.978 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.978 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.978 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:46.978 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:46.978 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.979 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.979 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.979 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.979 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.979 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.979 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.979 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.980 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.980 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.980 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.980 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:46.980 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:46.980 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.981 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.981 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.981 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:46.981 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.981 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.981 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.981 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.981 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.982 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.982 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.982 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.982 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.982 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.982 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.983 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.983 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.983 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.983 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:46.983 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:46.983 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.984 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.984 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.984 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.984 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.984 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.984 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.984 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.984 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.985 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.985 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.985 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_host": { 2024-04-24T06:54:46.985 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_host", 2024-04-24T06:54:46.985 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.985 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.985 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.985 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.986 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.986 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.986 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.986 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.986 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.986 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.986 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.987 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_port": { 2024-04-24T06:54:46.987 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_port", 2024-04-24T06:54:46.987 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:46.987 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.987 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.987 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10051", 2024-04-24T06:54:46.987 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.987 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.987 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.988 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.988 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.988 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.988 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.988 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_sender": { 2024-04-24T06:54:46.988 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_sender", 2024-04-24T06:54:46.988 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.988 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.989 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:46.989 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-24T06:54:46.989 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.989 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.989 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.989 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.989 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.989 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.990 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.990 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.990 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.990 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.990 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-04-24T06:54:46.990 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:46.990 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.990 INFO:teuthology.orchestra.run.smithi005.stdout: "modules": [ 2024-04-24T06:54:46.991 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2024-04-24T06:54:46.991 INFO:teuthology.orchestra.run.smithi005.stdout: "dashboard", 2024-04-24T06:54:46.991 INFO:teuthology.orchestra.run.smithi005.stdout: "iostat", 2024-04-24T06:54:46.991 INFO:teuthology.orchestra.run.smithi005.stdout: "nfs", 2024-04-24T06:54:46.991 INFO:teuthology.orchestra.run.smithi005.stdout: "restful", 2024-04-24T06:54:46.991 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_schedule" 2024-04-24T06:54:46.991 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.991 INFO:teuthology.orchestra.run.smithi005.stdout: "available_modules": [ 2024-04-24T06:54:46.991 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:46.992 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "alerts", 2024-04-24T06:54:46.992 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:46.992 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:46.992 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:46.992 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-04-24T06:54:46.992 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-04-24T06:54:46.992 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:46.992 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.993 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.993 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-04-24T06:54:46.993 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.993 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.993 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.993 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How frequently to reexamine health status", 2024-04-24T06:54:46.993 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.994 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.994 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.994 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:46.994 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:46.994 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.994 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.995 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.995 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:46.995 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.995 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.995 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:46.995 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:46.995 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:46.995 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:46.995 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:46.996 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:46.996 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:46.996 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:46.996 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.996 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.996 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.996 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.997 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:46.997 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:46.997 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:46.997 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.997 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.997 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:46.997 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.998 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.998 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:46.998 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:46.998 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:46.998 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:46.998 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:46.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:46.999 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:46.999 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:46.999 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:46.999 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:46.999 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:46.999 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:46.999 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:46.999 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:46.999 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.000 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.000 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.000 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.000 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.000 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.000 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.000 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.000 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.001 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.001 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.001 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.001 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:47.001 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:47.001 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.001 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.001 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.002 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.002 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.002 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.002 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.002 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.002 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.002 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.003 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.003 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_destination": { 2024-04-24T06:54:47.003 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_destination", 2024-04-24T06:54:47.003 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.003 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.003 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.003 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.003 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.004 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.004 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.004 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email address to send alerts to", 2024-04-24T06:54:47.004 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.004 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.004 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.004 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_from_name": { 2024-04-24T06:54:47.004 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_from_name", 2024-04-24T06:54:47.005 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.005 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.005 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.005 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "Ceph", 2024-04-24T06:54:47.005 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.005 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.005 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.005 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email From: name", 2024-04-24T06:54:47.006 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.006 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.006 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.006 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_host": { 2024-04-24T06:54:47.006 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_host", 2024-04-24T06:54:47.006 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.006 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.007 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.007 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.007 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.007 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.007 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.007 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP server", 2024-04-24T06:54:47.007 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.007 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.008 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.008 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_password": { 2024-04-24T06:54:47.008 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_password", 2024-04-24T06:54:47.008 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.008 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.008 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.008 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.009 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.009 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.009 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.009 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Password to authenticate with", 2024-04-24T06:54:47.009 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.009 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.009 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.010 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_port": { 2024-04-24T06:54:47.010 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_port", 2024-04-24T06:54:47.010 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:47.010 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.010 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.010 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "465", 2024-04-24T06:54:47.010 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.010 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.011 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.011 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP port", 2024-04-24T06:54:47.011 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.011 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.011 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.011 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_sender": { 2024-04-24T06:54:47.011 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_sender", 2024-04-24T06:54:47.012 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.012 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.012 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.012 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.012 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.012 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.012 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.012 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP envelope sender", 2024-04-24T06:54:47.012 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.013 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.013 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.013 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_ssl": { 2024-04-24T06:54:47.013 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_ssl", 2024-04-24T06:54:47.013 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.013 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.013 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.014 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:47.014 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.014 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.014 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.014 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-24T06:54:47.014 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.014 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.015 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.015 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_user": { 2024-04-24T06:54:47.015 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_user", 2024-04-24T06:54:47.015 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.015 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.015 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.015 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.016 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.016 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.016 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.016 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "User to authenticate as", 2024-04-24T06:54:47.016 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.016 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.016 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.017 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.017 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.017 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "balancer", 2024-04-24T06:54:47.017 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:47.017 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:47.017 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:47.018 INFO:teuthology.orchestra.run.smithi005.stdout: "active": { 2024-04-24T06:54:47.018 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "active", 2024-04-24T06:54:47.018 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.018 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.018 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.018 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:47.018 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.018 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.019 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.019 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically balance PGs across cluster", 2024-04-24T06:54:47.019 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.019 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.019 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.019 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_time": { 2024-04-24T06:54:47.019 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_time", 2024-04-24T06:54:47.020 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.020 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.020 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.020 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0000", 2024-04-24T06:54:47.020 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.020 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.020 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.020 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "beginning time of day to automatically balance", 2024-04-24T06:54:47.020 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T06:54:47.021 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.021 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.021 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_weekday": { 2024-04-24T06:54:47.021 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_weekday", 2024-04-24T06:54:47.021 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-04-24T06:54:47.021 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.022 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.022 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-04-24T06:54:47.022 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-04-24T06:54:47.022 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-04-24T06:54:47.022 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.022 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-24T06:54:47.023 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T06:54:47.023 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.023 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.023 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_max_iterations": { 2024-04-24T06:54:47.023 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_max_iterations", 2024-04-24T06:54:47.023 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-04-24T06:54:47.023 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.024 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.024 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "25", 2024-04-24T06:54:47.024 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-04-24T06:54:47.024 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "250", 2024-04-24T06:54:47.024 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.024 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-24T06:54:47.024 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.024 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.025 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.025 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_metrics": { 2024-04-24T06:54:47.025 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_metrics", 2024-04-24T06:54:47.025 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.025 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.025 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.025 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "pgs,objects,bytes", 2024-04-24T06:54:47.026 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.026 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.026 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.026 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-24T06:54:47.026 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-24T06:54:47.026 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.026 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.027 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_step": { 2024-04-24T06:54:47.027 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_step", 2024-04-24T06:54:47.027 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-24T06:54:47.027 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.027 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.027 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.5", 2024-04-24T06:54:47.027 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0.001", 2024-04-24T06:54:47.027 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "0.999", 2024-04-24T06:54:47.028 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.028 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "aggressiveness of optimization", 2024-04-24T06:54:47.028 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-24T06:54:47.028 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.028 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.028 INFO:teuthology.orchestra.run.smithi005.stdout: "end_time": { 2024-04-24T06:54:47.028 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_time", 2024-04-24T06:54:47.028 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.029 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.029 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.029 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2359", 2024-04-24T06:54:47.029 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.029 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.030 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.030 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ending time of day to automatically balance", 2024-04-24T06:54:47.030 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T06:54:47.030 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.030 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.031 INFO:teuthology.orchestra.run.smithi005.stdout: "end_weekday": { 2024-04-24T06:54:47.031 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_weekday", 2024-04-24T06:54:47.031 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-04-24T06:54:47.031 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.031 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.031 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-04-24T06:54:47.031 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-04-24T06:54:47.031 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-04-24T06:54:47.032 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.032 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-24T06:54:47.032 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T06:54:47.032 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.032 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.032 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:47.033 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:47.033 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.033 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.033 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.033 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.033 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.033 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.033 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.034 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.034 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.034 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.034 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.034 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.034 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.034 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.034 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.035 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.035 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.035 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.035 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:47.035 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:47.035 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.035 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.035 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.036 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.036 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.036 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.036 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.036 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.036 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.036 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.036 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.037 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:47.037 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:47.037 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.037 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.037 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.037 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:47.037 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.038 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.038 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.038 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.038 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.038 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.038 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.038 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.039 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.039 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.039 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.039 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.039 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.039 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.039 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:47.040 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:47.040 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.040 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.040 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.040 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.040 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.040 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.040 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.040 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.041 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.041 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.041 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.041 INFO:teuthology.orchestra.run.smithi005.stdout: "min_score": { 2024-04-24T06:54:47.041 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_score", 2024-04-24T06:54:47.041 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-24T06:54:47.042 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.042 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.042 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-04-24T06:54:47.042 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.042 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.042 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.042 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-24T06:54:47.042 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.043 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.043 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.043 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-04-24T06:54:47.043 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-04-24T06:54:47.043 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.043 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.043 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.044 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "upmap", 2024-04-24T06:54:47.044 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.044 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.044 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.044 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-compat", 2024-04-24T06:54:47.044 INFO:teuthology.orchestra.run.smithi005.stdout: "none", 2024-04-24T06:54:47.044 INFO:teuthology.orchestra.run.smithi005.stdout: "read", 2024-04-24T06:54:47.044 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap", 2024-04-24T06:54:47.045 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap-read" 2024-04-24T06:54:47.045 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.045 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Balancer mode", 2024-04-24T06:54:47.045 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.045 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.045 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.045 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_ids": { 2024-04-24T06:54:47.045 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_ids", 2024-04-24T06:54:47.046 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.046 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.046 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.046 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.046 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.046 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.046 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.046 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-24T06:54:47.047 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.047 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.047 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.047 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-04-24T06:54:47.047 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-04-24T06:54:47.047 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:47.047 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.047 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.048 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-04-24T06:54:47.048 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.048 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.048 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.048 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-24T06:54:47.048 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.048 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.048 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.049 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_deviation": { 2024-04-24T06:54:47.049 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_deviation", 2024-04-24T06:54:47.049 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:47.049 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.049 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.049 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-04-24T06:54:47.049 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-04-24T06:54:47.049 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.050 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.050 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-24T06:54:47.050 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-24T06:54:47.050 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.050 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.050 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_optimizations": { 2024-04-24T06:54:47.050 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_optimizations", 2024-04-24T06:54:47.051 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-04-24T06:54:47.051 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.051 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.051 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-24T06:54:47.051 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.051 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.051 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.051 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-24T06:54:47.051 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.052 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.052 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.052 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.052 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.052 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.052 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm", 2024-04-24T06:54:47.052 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:47.052 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:47.053 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:47.053 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_down_multiplier": { 2024-04-24T06:54:47.053 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_down_multiplier", 2024-04-24T06:54:47.053 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-24T06:54:47.053 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.053 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.053 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-04-24T06:54:47.053 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.054 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.054 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.054 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-24T06:54:47.054 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.054 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.054 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.055 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_refresh_rate": { 2024-04-24T06:54:47.055 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_refresh_rate", 2024-04-24T06:54:47.055 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:47.055 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.055 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.055 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "20", 2024-04-24T06:54:47.055 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.055 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.056 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.056 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-24T06:54:47.056 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.056 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.056 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.056 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_starting_port": { 2024-04-24T06:54:47.057 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_starting_port", 2024-04-24T06:54:47.057 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:47.057 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.057 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.057 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4721", 2024-04-24T06:54:47.057 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.057 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.057 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.058 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-24T06:54:47.058 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.058 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.058 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.058 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_ptrace": { 2024-04-24T06:54:47.058 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_ptrace", 2024-04-24T06:54:47.058 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.059 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.059 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.059 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.059 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.059 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.059 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.059 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-24T06:54:47.059 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-24T06:54:47.060 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.060 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.060 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_interval": { 2024-04-24T06:54:47.060 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_interval", 2024-04-24T06:54:47.060 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:47.060 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.060 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.061 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-04-24T06:54:47.061 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.061 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.061 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.061 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-24T06:54:47.061 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.061 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.061 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.062 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_memory_target_ratio": { 2024-04-24T06:54:47.062 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_memory_target_ratio", 2024-04-24T06:54:47.062 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-24T06:54:47.062 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.062 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.062 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.7", 2024-04-24T06:54:47.063 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.063 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.063 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.063 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-24T06:54:47.063 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.063 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.063 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.063 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm_log_destination": { 2024-04-24T06:54:47.064 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm_log_destination", 2024-04-24T06:54:47.064 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.064 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.064 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.064 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.065 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.065 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.065 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.065 INFO:teuthology.orchestra.run.smithi005.stdout: "file", 2024-04-24T06:54:47.065 INFO:teuthology.orchestra.run.smithi005.stdout: "file,syslog", 2024-04-24T06:54:47.065 INFO:teuthology.orchestra.run.smithi005.stdout: "syslog" 2024-04-24T06:54:47.065 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.065 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-24T06:54:47.065 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.066 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.066 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.066 INFO:teuthology.orchestra.run.smithi005.stdout: "cgroups_split": { 2024-04-24T06:54:47.066 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cgroups_split", 2024-04-24T06:54:47.066 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.066 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.067 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.067 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:47.067 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.067 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.067 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.067 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-24T06:54:47.067 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.067 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.067 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.068 INFO:teuthology.orchestra.run.smithi005.stdout: "config_checks_enabled": { 2024-04-24T06:54:47.068 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_checks_enabled", 2024-04-24T06:54:47.068 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.068 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.068 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.068 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.068 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.068 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.069 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.069 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-24T06:54:47.069 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.069 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.069 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.069 INFO:teuthology.orchestra.run.smithi005.stdout: "config_dashboard": { 2024-04-24T06:54:47.069 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_dashboard", 2024-04-24T06:54:47.069 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.070 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.070 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.070 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:47.070 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.070 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.070 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.070 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-24T06:54:47.070 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.071 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.071 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.071 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_alertmanager": { 2024-04-24T06:54:47.071 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_alertmanager", 2024-04-24T06:54:47.071 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.071 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.071 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.072 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-24T06:54:47.072 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.072 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.072 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.072 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-04-24T06:54:47.072 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.072 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.072 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.073 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_base": { 2024-04-24T06:54:47.073 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_base", 2024-04-24T06:54:47.073 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.073 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.073 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.073 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-24T06:54:47.073 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.073 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.073 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.074 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Container image name, without the tag", 2024-04-24T06:54:47.074 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.074 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.074 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.074 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_elasticsearch": { 2024-04-24T06:54:47.074 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_elasticsearch", 2024-04-24T06:54:47.075 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.075 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.075 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.075 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-24T06:54:47.075 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.075 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.075 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.076 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "elasticsearch container image", 2024-04-24T06:54:47.076 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.076 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.076 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.076 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_grafana": { 2024-04-24T06:54:47.076 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_grafana", 2024-04-24T06:54:47.076 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.076 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.077 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.077 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-24T06:54:47.077 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.077 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.077 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.077 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-04-24T06:54:47.077 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.077 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.078 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.078 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_haproxy": { 2024-04-24T06:54:47.078 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_haproxy", 2024-04-24T06:54:47.078 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.078 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.079 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.079 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-24T06:54:47.079 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.079 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.079 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.079 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "HAproxy container image", 2024-04-24T06:54:47.079 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.079 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.080 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.080 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_agent": { 2024-04-24T06:54:47.080 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_agent", 2024-04-24T06:54:47.080 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.080 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.080 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.081 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-24T06:54:47.081 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.081 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.081 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.081 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger agent container image", 2024-04-24T06:54:47.081 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.081 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.081 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.082 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_collector": { 2024-04-24T06:54:47.082 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_collector", 2024-04-24T06:54:47.082 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.082 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.082 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.082 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-24T06:54:47.083 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.083 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.083 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.083 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger collector container image", 2024-04-24T06:54:47.083 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.083 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.083 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.084 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_query": { 2024-04-24T06:54:47.084 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_query", 2024-04-24T06:54:47.084 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.084 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.084 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.084 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-24T06:54:47.085 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.085 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.085 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.085 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger query container image", 2024-04-24T06:54:47.085 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.085 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.085 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.086 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_keepalived": { 2024-04-24T06:54:47.086 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_keepalived", 2024-04-24T06:54:47.086 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.086 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.086 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.086 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-24T06:54:47.086 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.087 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.087 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.087 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Keepalived container image", 2024-04-24T06:54:47.087 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.087 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.087 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.087 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_loki": { 2024-04-24T06:54:47.087 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_loki", 2024-04-24T06:54:47.088 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.088 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.088 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.088 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-24T06:54:47.088 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.088 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.088 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.089 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Loki container image", 2024-04-24T06:54:47.089 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.089 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.089 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.090 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_node_exporter": { 2024-04-24T06:54:47.090 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_node_exporter", 2024-04-24T06:54:47.090 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.090 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.090 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.090 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-24T06:54:47.090 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.091 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.091 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.091 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-04-24T06:54:47.091 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.091 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.091 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.091 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_nvmeof": { 2024-04-24T06:54:47.092 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_nvmeof", 2024-04-24T06:54:47.092 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.092 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.092 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.092 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-24T06:54:47.092 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.092 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.093 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.093 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Nvme-of container image", 2024-04-24T06:54:47.093 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.093 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.093 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.093 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_prometheus": { 2024-04-24T06:54:47.093 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_prometheus", 2024-04-24T06:54:47.094 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.094 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.094 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.094 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-24T06:54:47.094 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.094 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.094 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.095 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-04-24T06:54:47.095 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.095 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.095 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.095 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_promtail": { 2024-04-24T06:54:47.095 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_promtail", 2024-04-24T06:54:47.095 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.096 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.096 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.096 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-24T06:54:47.096 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.096 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.096 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.096 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Promtail container image", 2024-04-24T06:54:47.096 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.097 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.097 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.097 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_samba": { 2024-04-24T06:54:47.097 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_samba", 2024-04-24T06:54:47.097 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.098 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.098 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.098 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-24T06:54:47.098 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.098 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.098 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.098 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Samba/SMB container image", 2024-04-24T06:54:47.098 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.099 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.099 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.099 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_snmp_gateway": { 2024-04-24T06:54:47.099 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_snmp_gateway", 2024-04-24T06:54:47.099 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.099 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.099 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.100 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-24T06:54:47.100 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.100 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.100 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.100 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SNMP Gateway container image", 2024-04-24T06:54:47.100 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.100 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.100 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.101 INFO:teuthology.orchestra.run.smithi005.stdout: "container_init": { 2024-04-24T06:54:47.101 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_init", 2024-04-24T06:54:47.101 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.101 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.101 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.101 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:47.101 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.101 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.102 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.102 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Run podman/docker with `--init`", 2024-04-24T06:54:47.102 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.102 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.102 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.102 INFO:teuthology.orchestra.run.smithi005.stdout: "daemon_cache_timeout": { 2024-04-24T06:54:47.102 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "daemon_cache_timeout", 2024-04-24T06:54:47.103 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:47.103 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.103 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.103 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-04-24T06:54:47.103 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.103 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.103 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.103 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-24T06:54:47.104 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.104 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.104 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.104 INFO:teuthology.orchestra.run.smithi005.stdout: "default_cephadm_command_timeout": { 2024-04-24T06:54:47.104 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_cephadm_command_timeout", 2024-04-24T06:54:47.104 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:47.104 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.104 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.105 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "900", 2024-04-24T06:54:47.105 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.105 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.105 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.105 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-24T06:54:47.105 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.105 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.105 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.106 INFO:teuthology.orchestra.run.smithi005.stdout: "default_registry": { 2024-04-24T06:54:47.106 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_registry", 2024-04-24T06:54:47.106 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.106 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.106 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.106 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io", 2024-04-24T06:54:47.106 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.107 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.107 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.107 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-24T06:54:47.107 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.107 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.107 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.107 INFO:teuthology.orchestra.run.smithi005.stdout: "device_cache_timeout": { 2024-04-24T06:54:47.108 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_cache_timeout", 2024-04-24T06:54:47.108 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:47.108 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.108 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.108 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1800", 2024-04-24T06:54:47.108 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.108 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.109 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.109 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache device inventory", 2024-04-24T06:54:47.109 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.109 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.109 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.109 INFO:teuthology.orchestra.run.smithi005.stdout: "device_enhanced_scan": { 2024-04-24T06:54:47.109 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_enhanced_scan", 2024-04-24T06:54:47.109 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.110 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.110 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.110 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.110 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.110 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.110 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.110 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-24T06:54:47.111 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.111 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.111 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.111 INFO:teuthology.orchestra.run.smithi005.stdout: "facts_cache_timeout": { 2024-04-24T06:54:47.111 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "facts_cache_timeout", 2024-04-24T06:54:47.111 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:47.111 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.111 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.112 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-04-24T06:54:47.112 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.112 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.112 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.112 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache host facts data", 2024-04-24T06:54:47.112 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.112 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.112 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.113 INFO:teuthology.orchestra.run.smithi005.stdout: "grafana_dashboards_path": { 2024-04-24T06:54:47.113 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "grafana_dashboards_path", 2024-04-24T06:54:47.113 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.113 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.113 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.113 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-24T06:54:47.113 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.114 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.114 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.114 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-24T06:54:47.114 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.114 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.114 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.115 INFO:teuthology.orchestra.run.smithi005.stdout: "host_check_interval": { 2024-04-24T06:54:47.115 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "host_check_interval", 2024-04-24T06:54:47.115 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:47.115 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.115 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.115 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-04-24T06:54:47.115 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.115 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.116 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.116 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to perform a host check", 2024-04-24T06:54:47.116 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.116 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.116 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.117 INFO:teuthology.orchestra.run.smithi005.stdout: "hw_monitoring": { 2024-04-24T06:54:47.117 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hw_monitoring", 2024-04-24T06:54:47.117 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.117 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.117 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.117 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.117 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.118 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.118 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.118 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-24T06:54:47.118 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.118 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.118 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.118 INFO:teuthology.orchestra.run.smithi005.stdout: "inventory_list_all": { 2024-04-24T06:54:47.119 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "inventory_list_all", 2024-04-24T06:54:47.119 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.119 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.119 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.119 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.119 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.119 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.119 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.120 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-24T06:54:47.120 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.120 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.120 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.120 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:47.120 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:47.120 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.121 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.121 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.121 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.121 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.121 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.121 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.121 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.121 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.122 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.122 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.122 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.122 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.122 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.122 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.122 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.123 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.123 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.123 INFO:teuthology.orchestra.run.smithi005.stdout: "log_refresh_metadata": { 2024-04-24T06:54:47.123 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_refresh_metadata", 2024-04-24T06:54:47.123 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.123 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.123 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.124 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.124 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.124 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.124 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.124 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-24T06:54:47.124 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.124 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.124 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.125 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:47.125 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:47.125 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.125 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.125 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.125 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:47.125 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.125 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.126 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.126 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-24T06:54:47.126 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.126 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.126 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.126 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:47.127 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:47.127 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.127 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.127 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.127 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:47.127 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.127 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.127 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.127 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.128 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.128 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.128 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.128 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.128 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.128 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.128 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.128 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.128 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.129 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.129 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:47.129 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:47.129 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.129 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.129 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.129 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.130 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.130 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.130 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.130 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.130 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.130 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.130 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.131 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-24T06:54:47.131 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-24T06:54:47.131 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.131 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.131 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.131 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.131 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.131 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.132 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.132 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-24T06:54:47.132 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.132 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.132 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.132 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-24T06:54:47.133 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-24T06:54:47.133 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.133 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.133 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.133 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "*", 2024-04-24T06:54:47.133 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.133 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.133 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.134 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-24T06:54:47.134 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.134 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.134 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.134 INFO:teuthology.orchestra.run.smithi005.stdout: "max_count_per_host": { 2024-04-24T06:54:47.134 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_count_per_host", 2024-04-24T06:54:47.135 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:47.135 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.135 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.135 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-24T06:54:47.135 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.135 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.135 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.135 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of daemons per service per host", 2024-04-24T06:54:47.135 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.136 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.136 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.136 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd_draining_count": { 2024-04-24T06:54:47.136 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_osd_draining_count", 2024-04-24T06:54:47.136 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:47.136 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.136 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.137 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-24T06:54:47.137 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.137 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.137 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.137 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-24T06:54:47.137 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.137 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.137 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.138 INFO:teuthology.orchestra.run.smithi005.stdout: "migration_current": { 2024-04-24T06:54:47.138 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "migration_current", 2024-04-24T06:54:47.138 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:47.138 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.138 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.138 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.139 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.139 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.139 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.139 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "internal - do not modify", 2024-04-24T06:54:47.139 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.139 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.139 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.139 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-04-24T06:54:47.140 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-04-24T06:54:47.140 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.140 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.140 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.140 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "root", 2024-04-24T06:54:47.140 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.140 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.140 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.140 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm-package", 2024-04-24T06:54:47.141 INFO:teuthology.orchestra.run.smithi005.stdout: "root" 2024-04-24T06:54:47.141 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.141 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "mode for remote execution of cephadm", 2024-04-24T06:54:47.141 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.141 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.141 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.141 INFO:teuthology.orchestra.run.smithi005.stdout: "oob_default_addr": { 2024-04-24T06:54:47.142 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "oob_default_addr", 2024-04-24T06:54:47.142 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.142 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.142 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.142 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "169.254.1.1", 2024-04-24T06:54:47.142 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.143 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.143 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.143 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-24T06:54:47.143 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.143 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.143 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.144 INFO:teuthology.orchestra.run.smithi005.stdout: "prometheus_alerts_path": { 2024-04-24T06:54:47.144 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus_alerts_path", 2024-04-24T06:54:47.144 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.144 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.144 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.144 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-24T06:54:47.144 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.144 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.145 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.145 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-24T06:54:47.145 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.145 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.145 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.145 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_insecure": { 2024-04-24T06:54:47.145 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_insecure", 2024-04-24T06:54:47.145 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.146 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.146 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.146 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.146 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.146 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.146 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.146 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-24T06:54:47.146 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.147 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.147 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.147 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_password": { 2024-04-24T06:54:47.147 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_password", 2024-04-24T06:54:47.147 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.147 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.147 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.147 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.148 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.148 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.148 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.148 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-24T06:54:47.148 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.148 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.148 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.148 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_url": { 2024-04-24T06:54:47.149 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_url", 2024-04-24T06:54:47.149 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.149 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.149 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.149 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.149 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.149 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.149 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.149 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-24T06:54:47.150 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.150 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.150 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.150 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_username": { 2024-04-24T06:54:47.150 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_username", 2024-04-24T06:54:47.150 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.150 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.151 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.151 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.151 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.151 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.151 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.151 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-24T06:54:47.151 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.151 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.151 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.152 INFO:teuthology.orchestra.run.smithi005.stdout: "secure_monitoring_stack": { 2024-04-24T06:54:47.152 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "secure_monitoring_stack", 2024-04-24T06:54:47.152 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.152 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.152 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.152 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.152 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.152 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.152 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.153 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-24T06:54:47.153 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.153 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.153 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.153 INFO:teuthology.orchestra.run.smithi005.stdout: "service_discovery_port": { 2024-04-24T06:54:47.153 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "service_discovery_port", 2024-04-24T06:54:47.153 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:47.153 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.154 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.154 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8765", 2024-04-24T06:54:47.154 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.154 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.154 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.154 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "cephadm service discovery port", 2024-04-24T06:54:47.154 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.154 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.154 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.155 INFO:teuthology.orchestra.run.smithi005.stdout: "ssh_config_file": { 2024-04-24T06:54:47.155 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssh_config_file", 2024-04-24T06:54:47.155 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.155 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.155 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.155 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.155 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.156 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.156 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.156 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-24T06:54:47.156 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.156 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.156 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.156 INFO:teuthology.orchestra.run.smithi005.stdout: "use_agent": { 2024-04-24T06:54:47.157 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_agent", 2024-04-24T06:54:47.157 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.157 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.157 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.157 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.157 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.157 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.157 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.158 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-24T06:54:47.158 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.158 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.158 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.158 INFO:teuthology.orchestra.run.smithi005.stdout: "use_repo_digest": { 2024-04-24T06:54:47.158 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_repo_digest", 2024-04-24T06:54:47.158 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.159 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.159 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.159 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:47.159 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.159 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.159 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.159 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-24T06:54:47.159 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.159 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.159 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.160 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_failed_host_check": { 2024-04-24T06:54:47.160 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_failed_host_check", 2024-04-24T06:54:47.160 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.160 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.160 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.160 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:47.160 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.160 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.161 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.161 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if the host check fails", 2024-04-24T06:54:47.161 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.161 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.161 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.161 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_daemons": { 2024-04-24T06:54:47.161 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_daemons", 2024-04-24T06:54:47.162 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.162 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.162 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.162 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:47.162 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.162 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.162 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.162 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-24T06:54:47.163 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.163 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.163 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.163 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_hosts": { 2024-04-24T06:54:47.163 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_hosts", 2024-04-24T06:54:47.163 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.163 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.163 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.164 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:47.164 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.164 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.164 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.164 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-24T06:54:47.164 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.164 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.164 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.164 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.165 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.165 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crash", 2024-04-24T06:54:47.165 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:47.165 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:47.165 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:47.165 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:47.166 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:47.166 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.166 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.166 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.166 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.166 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.166 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.167 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.167 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.167 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.167 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.167 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.167 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.167 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.167 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.168 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.168 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.168 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.168 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.168 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:47.168 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:47.168 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.169 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.169 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.169 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.169 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.169 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.169 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.169 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.169 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.170 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.170 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.170 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:47.170 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:47.170 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.170 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.170 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.171 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:47.171 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.171 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.171 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.171 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.171 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.171 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.171 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.172 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.172 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.172 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.172 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.172 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.172 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.172 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.173 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:47.173 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:47.173 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.173 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.173 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.173 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.173 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.173 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.173 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.174 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.174 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.174 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.174 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.174 INFO:teuthology.orchestra.run.smithi005.stdout: "retain_interval": { 2024-04-24T06:54:47.174 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retain_interval", 2024-04-24T06:54:47.174 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:47.175 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.175 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.175 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "31536000", 2024-04-24T06:54:47.175 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.175 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.175 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.176 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-24T06:54:47.176 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.176 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.176 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.176 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_recent_interval": { 2024-04-24T06:54:47.176 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_recent_interval", 2024-04-24T06:54:47.177 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:47.177 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.177 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.177 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1209600", 2024-04-24T06:54:47.177 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.177 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.177 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.177 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-24T06:54:47.178 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.178 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.178 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.178 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.178 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.179 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dashboard", 2024-04-24T06:54:47.179 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:47.179 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:47.179 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:47.179 INFO:teuthology.orchestra.run.smithi005.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-24T06:54:47.179 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-24T06:54:47.179 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:47.179 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.180 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.180 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-24T06:54:47.180 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.180 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.180 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.180 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.180 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.181 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.181 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.181 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_HOST": { 2024-04-24T06:54:47.181 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-24T06:54:47.181 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.181 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.181 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.182 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.182 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.182 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.182 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.182 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.182 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.182 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.182 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.183 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-24T06:54:47.183 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-24T06:54:47.183 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.183 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.183 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.183 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:47.183 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.184 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.184 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.184 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.184 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.184 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.184 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.185 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_ENABLED": { 2024-04-24T06:54:47.185 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_ENABLED", 2024-04-24T06:54:47.185 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.185 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.185 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.185 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.185 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.185 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.185 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.186 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.186 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.186 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.186 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.186 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-24T06:54:47.186 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-24T06:54:47.186 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.187 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.187 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.187 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:47.187 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.187 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.187 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.187 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.187 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.187 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.188 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.188 INFO:teuthology.orchestra.run.smithi005.stdout: "ENABLE_BROWSABLE_API": { 2024-04-24T06:54:47.188 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-24T06:54:47.188 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.188 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.188 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.188 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:47.189 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.189 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.189 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.189 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.189 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.189 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.189 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.190 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-24T06:54:47.190 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-24T06:54:47.190 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.190 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.190 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.190 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:47.191 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.191 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.191 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.191 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.191 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.191 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.191 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.191 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-24T06:54:47.192 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-24T06:54:47.192 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.192 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.192 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.192 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:47.192 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.192 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.192 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.193 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.193 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.193 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.193 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.193 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-24T06:54:47.193 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-24T06:54:47.193 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.194 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.194 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.194 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:47.194 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.194 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.194 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.195 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.195 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.195 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.195 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.195 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-24T06:54:47.195 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-24T06:54:47.195 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.196 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.196 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.196 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:47.196 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.196 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.196 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.196 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.196 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.197 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.197 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.197 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-24T06:54:47.197 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-24T06:54:47.197 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.197 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.197 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.198 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:47.198 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.198 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.198 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.198 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.198 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.198 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.199 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.199 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-24T06:54:47.199 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-24T06:54:47.199 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.199 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.199 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.199 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:47.200 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.200 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.200 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.200 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.200 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.200 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.200 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.200 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-24T06:54:47.201 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-24T06:54:47.201 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.201 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.201 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.201 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:47.201 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.201 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.201 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.201 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.201 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.202 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.202 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.202 INFO:teuthology.orchestra.run.smithi005.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-24T06:54:47.202 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-24T06:54:47.202 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.202 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.202 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.203 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.203 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.203 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.203 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.203 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.203 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.203 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.203 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.204 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_PASSWORD": { 2024-04-24T06:54:47.204 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-24T06:54:47.204 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.204 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.204 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.204 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-04-24T06:54:47.204 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.204 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.204 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.204 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.205 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.205 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.205 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.205 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-24T06:54:47.205 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-24T06:54:47.205 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.205 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.205 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.206 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:47.206 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.206 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.206 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.206 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.206 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.206 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.206 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.207 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_URL": { 2024-04-24T06:54:47.207 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_URL", 2024-04-24T06:54:47.207 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.207 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.207 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.207 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.207 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.207 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.208 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.208 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.208 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.208 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.208 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.208 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_USERNAME": { 2024-04-24T06:54:47.208 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-24T06:54:47.209 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.209 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.209 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.209 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-04-24T06:54:47.209 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.209 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.209 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.209 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.210 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.210 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.210 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.210 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-24T06:54:47.210 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-24T06:54:47.210 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.211 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.211 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.211 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.211 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.211 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.211 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.211 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.211 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.211 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.212 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.212 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-24T06:54:47.212 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-24T06:54:47.212 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.212 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.212 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.212 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.212 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.213 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.213 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.213 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.213 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.213 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.213 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.213 INFO:teuthology.orchestra.run.smithi005.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-24T06:54:47.213 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-24T06:54:47.214 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.214 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.214 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.214 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:47.214 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.214 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.214 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.214 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.215 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.215 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.215 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.215 INFO:teuthology.orchestra.run.smithi005.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-24T06:54:47.215 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-24T06:54:47.215 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.215 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.215 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.216 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.216 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.216 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.216 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.216 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.216 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.216 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.216 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.217 INFO:teuthology.orchestra.run.smithi005.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-24T06:54:47.217 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-24T06:54:47.217 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.217 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.217 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.217 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "[]", 2024-04-24T06:54:47.217 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.218 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.218 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.218 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.218 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.218 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.218 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.218 INFO:teuthology.orchestra.run.smithi005.stdout: "MULTICLUSTER_CONFIG": { 2024-04-24T06:54:47.219 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-24T06:54:47.219 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.219 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.219 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.219 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "{}", 2024-04-24T06:54:47.219 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.219 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.219 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.220 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.220 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.220 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.220 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.220 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_HOST": { 2024-04-24T06:54:47.220 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-24T06:54:47.220 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.220 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.221 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.221 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.221 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.221 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.221 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.221 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.221 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.221 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.221 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.222 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-24T06:54:47.222 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-24T06:54:47.222 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.222 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.222 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.222 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:47.222 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.223 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.223 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.223 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.223 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.223 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.223 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.223 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-24T06:54:47.224 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-24T06:54:47.224 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.224 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.224 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.224 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.224 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.224 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.224 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.224 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.225 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.225 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.225 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.225 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-24T06:54:47.225 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-24T06:54:47.225 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.225 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.225 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.226 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.226 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.226 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.226 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.226 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.226 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.226 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.227 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.227 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-24T06:54:47.227 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-24T06:54:47.227 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.227 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.227 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.227 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:47.228 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.228 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.228 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.228 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.228 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.228 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.228 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.228 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-24T06:54:47.229 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-24T06:54:47.229 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.229 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.229 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.229 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:47.229 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.229 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.229 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.230 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.230 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.230 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.230 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.230 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-24T06:54:47.230 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-24T06:54:47.231 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.231 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.231 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.231 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.231 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.231 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.231 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.231 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.232 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.232 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.232 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.232 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-24T06:54:47.232 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-24T06:54:47.232 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.232 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.233 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.233 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.233 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.233 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.233 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.233 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.233 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.234 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.234 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.234 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-24T06:54:47.234 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-24T06:54:47.234 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.234 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.234 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.235 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.235 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.235 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.235 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.235 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.235 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.235 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.235 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.236 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_ENABLED": { 2024-04-24T06:54:47.236 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-24T06:54:47.236 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.236 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.236 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.236 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:47.236 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.237 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.237 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.237 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.237 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.237 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.237 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.237 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-24T06:54:47.237 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-24T06:54:47.238 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.238 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.238 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.238 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-24T06:54:47.238 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.238 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.238 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.239 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.239 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.239 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.239 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.239 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-24T06:54:47.239 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-24T06:54:47.239 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:47.239 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.240 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.240 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-24T06:54:47.240 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.240 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.240 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.240 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.240 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.240 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.240 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.241 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-24T06:54:47.241 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-24T06:54:47.241 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:47.241 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.241 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.241 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8", 2024-04-24T06:54:47.241 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.242 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.242 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.242 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.242 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.242 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.242 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.243 INFO:teuthology.orchestra.run.smithi005.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-24T06:54:47.243 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-24T06:54:47.243 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:47.243 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.243 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.243 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "45", 2024-04-24T06:54:47.243 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.244 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.244 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.244 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.244 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.244 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.244 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.244 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ACCESS_KEY": { 2024-04-24T06:54:47.244 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-24T06:54:47.245 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.245 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.245 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.245 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.245 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.245 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.245 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.245 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.245 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.246 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.246 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.246 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-24T06:54:47.246 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-24T06:54:47.246 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.246 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.246 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.247 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-04-24T06:54:47.247 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.247 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.247 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.247 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.247 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.247 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.247 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.248 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SECRET_KEY": { 2024-04-24T06:54:47.248 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-24T06:54:47.248 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.248 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.248 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.248 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.248 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.248 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.249 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.249 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.249 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.249 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.249 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.249 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SSL_VERIFY": { 2024-04-24T06:54:47.249 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-24T06:54:47.250 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.250 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.250 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.250 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:47.250 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.250 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.250 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.250 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.251 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.251 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.251 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.251 INFO:teuthology.orchestra.run.smithi005.stdout: "UNSAFE_TLS_v1_2": { 2024-04-24T06:54:47.251 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-24T06:54:47.251 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.251 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.252 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.252 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.252 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.252 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.252 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.252 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.252 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.252 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.253 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.253 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-24T06:54:47.253 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-24T06:54:47.253 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:47.253 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.253 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.253 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-04-24T06:54:47.253 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.254 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.254 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.254 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.254 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.254 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.254 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.255 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-24T06:54:47.255 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-24T06:54:47.255 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:47.255 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.255 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.255 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-24T06:54:47.255 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.255 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.256 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.256 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.256 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.256 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.256 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.256 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-24T06:54:47.256 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-24T06:54:47.257 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:47.257 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.257 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.257 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-04-24T06:54:47.257 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.257 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.257 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.257 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.258 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.258 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.258 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.258 INFO:teuthology.orchestra.run.smithi005.stdout: "cross_origin_url": { 2024-04-24T06:54:47.258 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cross_origin_url", 2024-04-24T06:54:47.258 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.258 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.259 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.259 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.259 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.259 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.259 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.259 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.259 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.259 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.260 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.260 INFO:teuthology.orchestra.run.smithi005.stdout: "crt_file": { 2024-04-24T06:54:47.260 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crt_file", 2024-04-24T06:54:47.260 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.260 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.260 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.260 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.261 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.261 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.261 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.261 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.261 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.261 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.261 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.262 INFO:teuthology.orchestra.run.smithi005.stdout: "debug": { 2024-04-24T06:54:47.262 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "debug", 2024-04-24T06:54:47.262 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.262 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.262 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.262 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.262 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.262 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.263 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.263 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable/disable debug options", 2024-04-24T06:54:47.263 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.263 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.263 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.263 INFO:teuthology.orchestra.run.smithi005.stdout: "jwt_token_ttl": { 2024-04-24T06:54:47.263 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "jwt_token_ttl", 2024-04-24T06:54:47.264 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:47.264 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.264 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.264 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "28800", 2024-04-24T06:54:47.264 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.264 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.264 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.264 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.264 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.265 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.265 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.265 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-04-24T06:54:47.265 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-04-24T06:54:47.265 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.265 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.265 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.265 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.266 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.266 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.266 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.266 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.266 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.266 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.266 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.267 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:47.267 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:47.267 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.267 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.267 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.267 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.267 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.267 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.268 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.268 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.268 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.268 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.268 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.268 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.268 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.268 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.269 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.269 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.269 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.269 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.269 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:47.269 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:47.269 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.270 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.270 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.270 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.270 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.270 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.270 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.270 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.271 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.271 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.271 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.271 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:47.271 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:47.271 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.271 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.271 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.272 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:47.272 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.272 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.272 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.272 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.272 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.272 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.273 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.273 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.273 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.273 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.274 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.274 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.274 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.274 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.275 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:47.275 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:47.275 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.276 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.276 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.276 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.276 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.277 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.277 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.277 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.277 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.278 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.278 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.278 INFO:teuthology.orchestra.run.smithi005.stdout: "motd": { 2024-04-24T06:54:47.278 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "motd", 2024-04-24T06:54:47.279 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.279 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.279 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.279 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.280 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.280 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.280 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.280 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "The message of the day", 2024-04-24T06:54:47.281 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.281 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.281 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.282 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect_resolve_ip_addr": { 2024-04-24T06:54:47.282 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "redirect_resolve_ip_addr", 2024-04-24T06:54:47.282 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.282 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.283 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.283 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.283 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.283 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.283 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.284 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.284 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.284 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.284 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.284 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-04-24T06:54:47.285 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-04-24T06:54:47.285 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.285 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.285 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.285 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-04-24T06:54:47.285 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.285 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.285 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.286 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.286 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.286 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.286 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.286 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.286 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-04-24T06:54:47.286 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-04-24T06:54:47.287 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:47.287 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.287 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.287 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8080", 2024-04-24T06:54:47.287 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.287 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.287 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.287 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.287 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.288 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.288 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.288 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.288 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-04-24T06:54:47.288 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-04-24T06:54:47.288 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.288 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.288 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.288 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:47.289 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.289 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.289 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.289 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.289 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.289 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.289 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.289 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl_server_port": { 2024-04-24T06:54:47.289 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl_server_port", 2024-04-24T06:54:47.290 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:47.290 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.290 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.290 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8443", 2024-04-24T06:54:47.290 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.290 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.290 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.290 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.291 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.291 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.291 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.291 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.291 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-04-24T06:54:47.291 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-04-24T06:54:47.291 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.292 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.292 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.292 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "redirect", 2024-04-24T06:54:47.292 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.292 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.292 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.292 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.292 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect" 2024-04-24T06:54:47.292 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.293 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.293 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.293 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.293 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.293 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-04-24T06:54:47.293 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-04-24T06:54:47.293 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:47.294 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.294 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.294 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-04-24T06:54:47.294 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-04-24T06:54:47.294 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-04-24T06:54:47.294 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.294 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.295 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.295 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.295 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.295 INFO:teuthology.orchestra.run.smithi005.stdout: "url_prefix": { 2024-04-24T06:54:47.295 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url_prefix", 2024-04-24T06:54:47.295 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.295 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.296 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.296 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.296 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.296 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.296 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.296 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.296 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.296 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.297 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.297 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.297 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.297 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.297 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2024-04-24T06:54:47.297 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:47.297 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:47.298 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:47.298 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_monitoring": { 2024-04-24T06:54:47.298 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_monitoring", 2024-04-24T06:54:47.298 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.298 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.298 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.298 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:47.299 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.299 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.299 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.299 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "monitor device health metrics", 2024-04-24T06:54:47.299 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.299 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.299 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.299 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.300 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:47.300 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:47.300 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.300 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.300 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.300 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.300 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.300 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.300 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.301 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.301 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.301 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.301 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.301 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.301 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.301 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.301 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.301 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.302 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.302 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.302 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:47.302 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:47.302 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.302 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.303 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.303 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.303 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.303 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.303 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.303 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.303 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.303 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.303 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.304 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:47.304 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:47.304 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.304 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.304 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.304 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:47.304 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.304 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.305 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.305 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.305 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.305 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.305 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.305 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.305 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.305 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.305 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.306 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.306 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.306 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.306 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:47.306 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:47.306 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.306 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.307 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.307 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.307 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.307 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.307 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.307 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.307 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.307 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.307 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.308 INFO:teuthology.orchestra.run.smithi005.stdout: "mark_out_threshold": { 2024-04-24T06:54:47.308 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mark_out_threshold", 2024-04-24T06:54:47.308 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:47.308 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.308 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.308 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2419200", 2024-04-24T06:54:47.308 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.308 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.308 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.309 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-24T06:54:47.309 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.309 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.309 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.309 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.309 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": { 2024-04-24T06:54:47.309 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_name", 2024-04-24T06:54:47.309 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.310 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.310 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.310 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "device_health_metrics", 2024-04-24T06:54:47.310 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.310 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.310 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.310 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-24T06:54:47.310 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.311 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.311 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.311 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.311 INFO:teuthology.orchestra.run.smithi005.stdout: "retention_period": { 2024-04-24T06:54:47.311 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retention_period", 2024-04-24T06:54:47.311 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:47.311 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.311 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.312 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15552000", 2024-04-24T06:54:47.312 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.312 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.312 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.312 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain device health metrics", 2024-04-24T06:54:47.312 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.312 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.313 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.313 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_frequency": { 2024-04-24T06:54:47.313 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_frequency", 2024-04-24T06:54:47.313 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:47.313 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.313 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.314 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-04-24T06:54:47.314 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.314 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.314 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.314 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-24T06:54:47.314 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.314 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.314 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.315 INFO:teuthology.orchestra.run.smithi005.stdout: "self_heal": { 2024-04-24T06:54:47.315 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "self_heal", 2024-04-24T06:54:47.315 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.315 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.315 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.315 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:47.315 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.315 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.316 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.316 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-24T06:54:47.316 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.316 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.316 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.316 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.316 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-04-24T06:54:47.316 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-04-24T06:54:47.317 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:47.317 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.317 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.317 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-04-24T06:54:47.317 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.317 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.318 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.318 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and check device health", 2024-04-24T06:54:47.318 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.318 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.318 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.318 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_threshold": { 2024-04-24T06:54:47.319 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_threshold", 2024-04-24T06:54:47.319 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:47.319 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.319 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.319 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7257600", 2024-04-24T06:54:47.319 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.319 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.319 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.320 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-24T06:54:47.320 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.320 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.320 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.320 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.320 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.320 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.320 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "diskprediction_local", 2024-04-24T06:54:47.321 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:47.321 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:47.321 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:47.321 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:47.321 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:47.322 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.322 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.322 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.322 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.322 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.322 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.322 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.322 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.323 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.323 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.323 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.323 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.323 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.323 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.323 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.324 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.324 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.324 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.324 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:47.324 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:47.324 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.324 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.324 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.325 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.325 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.325 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.325 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.325 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.325 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.325 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.325 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.326 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:47.326 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:47.326 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.326 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.326 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.326 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:47.327 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.327 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.327 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.327 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.327 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.327 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.327 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.327 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.327 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.328 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.328 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.328 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.328 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.328 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.328 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:47.328 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:47.329 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.329 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.329 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.329 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.329 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.329 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.329 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.329 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.329 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.330 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.330 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.330 INFO:teuthology.orchestra.run.smithi005.stdout: "predict_interval": { 2024-04-24T06:54:47.330 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predict_interval", 2024-04-24T06:54:47.330 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.330 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.330 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.331 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-04-24T06:54:47.331 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.331 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.331 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.331 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.331 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.331 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.332 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.332 INFO:teuthology.orchestra.run.smithi005.stdout: "predictor_model": { 2024-04-24T06:54:47.332 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predictor_model", 2024-04-24T06:54:47.332 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.332 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.332 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.332 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "prophetstor", 2024-04-24T06:54:47.332 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.333 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.333 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.333 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.333 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.333 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.333 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.334 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-04-24T06:54:47.334 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-04-24T06:54:47.334 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.334 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.334 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.334 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-04-24T06:54:47.335 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.335 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.335 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.335 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.335 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.335 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.335 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.335 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.336 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.336 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "influx", 2024-04-24T06:54:47.336 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": false, 2024-04-24T06:54:47.336 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "influxdb python module not found", 2024-04-24T06:54:47.336 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:47.336 INFO:teuthology.orchestra.run.smithi005.stdout: "batch_size": { 2024-04-24T06:54:47.337 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "batch_size", 2024-04-24T06:54:47.337 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:47.337 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.337 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.337 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5000", 2024-04-24T06:54:47.337 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.337 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.337 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.338 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-24T06:54:47.338 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.338 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.338 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.338 INFO:teuthology.orchestra.run.smithi005.stdout: "database": { 2024-04-24T06:54:47.338 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "database", 2024-04-24T06:54:47.338 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.339 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.339 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.339 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "ceph", 2024-04-24T06:54:47.339 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.339 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.339 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.339 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-24T06:54:47.339 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.340 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.340 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.340 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": { 2024-04-24T06:54:47.340 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hostname", 2024-04-24T06:54:47.340 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.340 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.340 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.340 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.341 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.341 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.341 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.341 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server hostname", 2024-04-24T06:54:47.341 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.341 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.341 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.342 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-04-24T06:54:47.342 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-04-24T06:54:47.342 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:47.342 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.342 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.342 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "30", 2024-04-24T06:54:47.342 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "5", 2024-04-24T06:54:47.343 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.343 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.343 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-24T06:54:47.343 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.343 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.343 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.344 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:47.344 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:47.344 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.344 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.344 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.344 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.344 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.345 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.345 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.345 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.345 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.345 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.345 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.345 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.345 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.346 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.346 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.346 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.346 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.346 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.346 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:47.347 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:47.347 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.347 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.347 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.347 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.347 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.347 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.347 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.347 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.348 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.348 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.348 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.348 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:47.348 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:47.348 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.348 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.349 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.349 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:47.349 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.349 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.349 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.349 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.349 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.349 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.349 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.350 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.350 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.350 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.350 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.350 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.350 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.350 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.351 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:47.351 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:47.351 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.351 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.351 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.351 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.351 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.352 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.352 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.352 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.352 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.352 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.352 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.352 INFO:teuthology.orchestra.run.smithi005.stdout: "password": { 2024-04-24T06:54:47.352 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "password", 2024-04-24T06:54:47.353 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.353 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.353 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.353 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.353 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.353 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.353 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.353 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "password of InfluxDB server user", 2024-04-24T06:54:47.354 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.354 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.354 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.354 INFO:teuthology.orchestra.run.smithi005.stdout: "port": { 2024-04-24T06:54:47.354 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "port", 2024-04-24T06:54:47.354 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:47.354 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.355 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.355 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8086", 2024-04-24T06:54:47.355 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.355 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.355 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.355 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server port", 2024-04-24T06:54:47.355 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.355 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.356 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.356 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-04-24T06:54:47.356 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-04-24T06:54:47.356 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.356 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.356 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.356 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "false", 2024-04-24T06:54:47.356 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.357 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.357 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.357 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T06:54:47.357 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.357 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.357 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.357 INFO:teuthology.orchestra.run.smithi005.stdout: "threads": { 2024-04-24T06:54:47.357 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threads", 2024-04-24T06:54:47.358 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:47.358 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.358 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.358 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-04-24T06:54:47.358 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-04-24T06:54:47.358 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "32", 2024-04-24T06:54:47.358 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.358 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-24T06:54:47.359 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.359 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.359 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.359 INFO:teuthology.orchestra.run.smithi005.stdout: "username": { 2024-04-24T06:54:47.359 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "username", 2024-04-24T06:54:47.359 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.359 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.359 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.360 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.360 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.360 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.360 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.360 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "username of InfluxDB server user", 2024-04-24T06:54:47.360 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.360 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.360 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.361 INFO:teuthology.orchestra.run.smithi005.stdout: "verify_ssl": { 2024-04-24T06:54:47.361 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "verify_ssl", 2024-04-24T06:54:47.361 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.361 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.361 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.361 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "true", 2024-04-24T06:54:47.361 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.361 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.361 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.362 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T06:54:47.362 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.362 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.362 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.362 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.362 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.363 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "insights", 2024-04-24T06:54:47.363 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:47.363 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:47.363 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:47.363 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:47.363 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:47.363 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.363 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.364 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.364 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.364 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.364 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.364 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.364 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.364 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.364 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.365 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.365 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.365 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.365 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.365 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.365 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.365 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.366 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.366 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:47.366 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:47.366 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.366 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.366 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.366 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.367 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.367 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.367 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.367 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.367 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.367 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.367 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.367 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:47.368 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:47.368 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.368 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.368 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.368 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:47.368 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.368 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.368 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.369 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.369 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.369 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.369 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.369 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.369 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.369 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.369 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.370 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.370 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.370 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.370 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:47.370 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:47.370 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.371 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.371 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.371 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.371 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.371 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.371 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.371 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.371 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.372 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.372 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.372 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.372 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.372 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "iostat", 2024-04-24T06:54:47.372 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:47.373 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:47.373 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:47.373 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:47.373 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:47.373 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.373 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.373 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.373 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.374 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.374 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.374 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.374 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.374 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.374 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.374 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.375 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.375 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.375 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.375 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.375 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.375 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.375 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.376 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:47.376 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:47.376 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.376 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.376 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.376 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.376 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.377 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.377 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.377 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.377 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.377 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.377 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.377 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:47.378 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:47.378 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.378 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.378 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.378 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:47.378 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.378 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.379 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.379 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.379 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.379 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.379 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.379 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.379 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.379 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.380 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.380 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.380 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.380 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.380 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:47.380 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:47.380 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.381 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.381 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.381 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.381 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.381 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.381 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.381 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.382 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.382 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.382 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.382 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.382 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.382 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "k8sevents", 2024-04-24T06:54:47.383 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:47.383 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:47.383 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:47.383 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_event_retention_days": { 2024-04-24T06:54:47.383 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ceph_event_retention_days", 2024-04-24T06:54:47.383 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:47.383 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.383 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.384 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7", 2024-04-24T06:54:47.384 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.384 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.384 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.384 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-24T06:54:47.384 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.384 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.384 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.385 INFO:teuthology.orchestra.run.smithi005.stdout: "config_check_secs": { 2024-04-24T06:54:47.385 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_check_secs", 2024-04-24T06:54:47.385 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:47.385 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.385 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.385 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-24T06:54:47.385 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "10", 2024-04-24T06:54:47.386 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.386 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.386 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-24T06:54:47.386 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.386 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.386 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.386 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:47.387 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:47.387 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.387 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.387 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.387 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.387 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.387 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.387 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.387 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.388 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.388 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.388 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.388 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.388 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.388 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.388 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.388 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.388 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.389 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.389 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:47.389 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:47.389 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.389 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.389 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.389 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.389 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.390 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.390 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.390 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.390 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.390 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.390 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.390 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:47.391 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:47.391 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.391 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.391 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.391 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:47.391 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.391 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.392 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.392 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.392 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.392 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.392 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.392 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.392 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.392 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.392 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.393 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.393 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.393 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.393 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:47.393 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:47.393 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.393 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.394 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.394 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.394 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.394 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.394 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.394 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.394 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.394 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.399 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "localpool", 2024-04-24T06:54:47.399 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:47.399 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:47.399 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:47.399 INFO:teuthology.orchestra.run.smithi005.stdout: "failure_domain": { 2024-04-24T06:54:47.399 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "failure_domain", 2024-04-24T06:54:47.399 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.399 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.399 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.400 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "host", 2024-04-24T06:54:47.400 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.400 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.400 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.400 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "failure domain for any created local pool", 2024-04-24T06:54:47.400 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-24T06:54:47.400 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.400 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.401 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:47.401 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:47.401 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.401 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.401 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.401 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.401 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.402 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.402 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.402 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.402 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.402 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.402 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.402 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.403 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.403 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.403 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.403 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.403 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.403 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.403 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:47.404 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:47.404 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.404 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.404 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.404 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.404 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.404 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.404 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.405 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.405 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.405 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.405 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.405 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:47.405 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:47.405 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.406 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.406 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.406 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:47.406 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.406 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.406 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.406 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.406 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.407 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.407 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.407 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.407 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.407 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.407 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.407 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.407 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.408 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.408 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:47.408 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:47.408 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.408 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.408 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.408 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.409 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.409 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.409 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.409 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.409 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.409 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.409 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.409 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": { 2024-04-24T06:54:47.410 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_size", 2024-04-24T06:54:47.410 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:47.410 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.410 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.410 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.410 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.410 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.410 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.411 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default min_size for any created local pool", 2024-04-24T06:54:47.411 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-24T06:54:47.411 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.411 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.411 INFO:teuthology.orchestra.run.smithi005.stdout: "num_rep": { 2024-04-24T06:54:47.411 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "num_rep", 2024-04-24T06:54:47.411 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:47.412 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.412 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.412 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3", 2024-04-24T06:54:47.412 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.412 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.412 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.412 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default replica count for any created local pool", 2024-04-24T06:54:47.412 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.412 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": { 2024-04-24T06:54:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_num", 2024-04-24T06:54:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "128", 2024-04-24T06:54:47.414 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.414 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.414 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.414 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default pg_num for any created local pool", 2024-04-24T06:54:47.414 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.414 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.414 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.415 INFO:teuthology.orchestra.run.smithi005.stdout: "prefix": { 2024-04-24T06:54:47.415 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prefix", 2024-04-24T06:54:47.415 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.415 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.415 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.415 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.415 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.415 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.415 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.416 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name prefix for any created local pool", 2024-04-24T06:54:47.416 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.416 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.416 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.416 INFO:teuthology.orchestra.run.smithi005.stdout: "subtree": { 2024-04-24T06:54:47.416 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "subtree", 2024-04-24T06:54:47.416 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.416 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.417 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.417 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "rack", 2024-04-24T06:54:47.417 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.417 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.417 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.417 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-24T06:54:47.417 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-24T06:54:47.417 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.418 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.418 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.418 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.418 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.418 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mds_autoscaler", 2024-04-24T06:54:47.419 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:47.419 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:47.419 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:47.419 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:47.419 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:47.419 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.419 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.419 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.419 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.420 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.420 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.420 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.420 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.420 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.420 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.420 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.420 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.421 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.421 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.421 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.421 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.421 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.421 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.421 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:47.421 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:47.422 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.422 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.422 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.422 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.422 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.422 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.422 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.422 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.422 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.423 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.423 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.423 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:47.423 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:47.423 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.423 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.423 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.424 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:47.424 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.424 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.424 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.424 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.424 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.424 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.425 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.425 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.425 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.425 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.425 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.425 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.425 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.425 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.426 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:47.426 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:47.426 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.426 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.426 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.426 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.426 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.427 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.427 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.427 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.427 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.427 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.427 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.427 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.427 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.428 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.428 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirroring", 2024-04-24T06:54:47.428 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:47.428 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:47.428 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:47.428 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:47.428 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:47.428 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.428 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.429 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.429 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.429 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.429 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.429 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.429 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.429 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.429 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.429 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.430 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.430 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.430 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.430 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.430 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.430 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.430 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.431 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:47.431 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:47.431 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.431 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.431 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.431 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.431 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.431 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.431 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.432 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.432 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.432 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.432 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.432 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:47.432 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:47.432 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.432 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.433 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.433 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:47.433 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.433 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.433 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.433 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.433 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.433 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.433 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.434 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.434 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.434 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.434 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.435 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.435 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.435 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.435 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:47.435 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:47.435 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.435 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.436 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.436 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.436 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.436 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.436 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.436 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.436 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.436 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.436 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.437 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.437 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.437 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.437 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "nfs", 2024-04-24T06:54:47.437 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:47.437 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:47.437 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:47.438 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:47.438 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:47.438 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.438 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.438 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.438 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.438 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.438 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.439 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.439 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.439 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.439 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.439 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.439 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.439 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.439 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.440 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.440 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.440 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.440 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.440 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:47.440 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:47.440 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.441 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.441 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.441 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.441 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.441 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.441 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.441 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.441 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.441 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.442 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.442 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:47.442 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:47.442 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.442 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.442 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.442 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:47.443 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.443 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.443 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.443 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.443 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.443 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.443 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.443 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.444 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.444 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.444 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.444 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.445 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.445 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.445 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:47.445 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:47.445 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.445 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.446 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.446 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.446 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.446 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.446 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.446 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.446 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.446 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.447 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.447 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.447 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.447 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.447 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-04-24T06:54:47.447 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:47.447 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:47.447 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:47.448 INFO:teuthology.orchestra.run.smithi005.stdout: "fail_fs": { 2024-04-24T06:54:47.448 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "fail_fs", 2024-04-24T06:54:47.448 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.448 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.448 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.448 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.448 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.448 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.448 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.449 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-24T06:54:47.449 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.449 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.449 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.449 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.449 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:47.449 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:47.449 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.450 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.450 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.450 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.450 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.450 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.450 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.450 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.450 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.451 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.451 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.451 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.451 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.451 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.451 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.451 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.451 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.451 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.452 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.452 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:47.452 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:47.452 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.452 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.452 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.452 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.452 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.452 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.452 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.453 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.453 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.453 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.453 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.453 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:47.453 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:47.453 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.453 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.454 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.454 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:47.454 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.454 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.454 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.454 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.454 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.454 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.455 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.455 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.455 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.455 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.455 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.455 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.456 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.456 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.456 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:47.456 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:47.456 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.457 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.457 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.457 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.457 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.457 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.457 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.457 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.457 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.457 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.458 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.458 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator": { 2024-04-24T06:54:47.458 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-04-24T06:54:47.458 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.458 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.458 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.459 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.459 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.459 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.459 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.459 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2024-04-24T06:54:47.459 INFO:teuthology.orchestra.run.smithi005.stdout: "rook", 2024-04-24T06:54:47.459 INFO:teuthology.orchestra.run.smithi005.stdout: "test_orchestrator" 2024-04-24T06:54:47.459 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.460 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Orchestrator backend", 2024-04-24T06:54:47.460 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.460 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.460 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.460 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.460 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.460 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.460 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_perf_query", 2024-04-24T06:54:47.461 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:47.461 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:47.461 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:47.461 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:47.461 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:47.461 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.461 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.461 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.461 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.462 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.462 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.462 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.462 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.462 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.462 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.462 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.463 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.463 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.463 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.463 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.463 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.463 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.463 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.464 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:47.464 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:47.464 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.464 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.464 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.464 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.464 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.464 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.465 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.465 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.465 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.465 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.465 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.465 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.465 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:47.465 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:47.466 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.466 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.466 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.466 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:47.466 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.466 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.466 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.467 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.467 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.467 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.467 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.467 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.467 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.467 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.467 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.468 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.468 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.468 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.468 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:47.468 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:47.468 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.468 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.468 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.469 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.469 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.469 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.469 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.469 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.469 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.469 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.469 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.470 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.470 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.470 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.470 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_support", 2024-04-24T06:54:47.470 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:47.470 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:47.470 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:47.471 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:47.471 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:47.471 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.471 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.471 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.471 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.471 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.471 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.471 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.472 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.472 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.472 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.472 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.472 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.472 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.472 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.472 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.472 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.473 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.473 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.473 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:47.473 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:47.473 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.473 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.473 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.474 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.474 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.474 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.474 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.474 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.474 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.474 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.474 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.475 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:47.475 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:47.475 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.475 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.475 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.475 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:47.475 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.476 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.476 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.476 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.476 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.476 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.476 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.477 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.477 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.477 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.477 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.477 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.477 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.477 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.478 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:47.478 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:47.478 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.478 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.478 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.478 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.478 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.478 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.479 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.479 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.479 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.479 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.479 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.479 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.479 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.480 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.480 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_autoscaler", 2024-04-24T06:54:47.480 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:47.480 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:47.480 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:47.480 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:47.480 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:47.481 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.481 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.481 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.481 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.481 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.481 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.481 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.481 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.482 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.482 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.482 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.482 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.482 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.482 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.482 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.483 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.483 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.483 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.483 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:47.483 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:47.483 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.483 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.483 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.483 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.484 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.484 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.484 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.484 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.484 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.484 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.484 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.485 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:47.485 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:47.485 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.485 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.485 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.485 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:47.485 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.485 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.485 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.486 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.486 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.486 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.486 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.486 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.486 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.487 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.487 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.487 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.487 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.487 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.487 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:47.488 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:47.488 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.488 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.488 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.488 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.488 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.488 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.488 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.489 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.489 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.489 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.489 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.489 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-04-24T06:54:47.489 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-04-24T06:54:47.489 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:47.489 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.490 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.490 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-04-24T06:54:47.490 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.490 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.490 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.490 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.490 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.490 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.491 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.491 INFO:teuthology.orchestra.run.smithi005.stdout: "threshold": { 2024-04-24T06:54:47.491 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threshold", 2024-04-24T06:54:47.491 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-24T06:54:47.491 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.491 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.491 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-04-24T06:54:47.492 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1.0", 2024-04-24T06:54:47.492 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.492 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.492 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "scaling threshold", 2024-04-24T06:54:47.492 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-24T06:54:47.492 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.493 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.493 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.493 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.493 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "progress", 2024-04-24T06:54:47.493 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:47.493 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:47.494 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:47.494 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_pg_recovery_event": { 2024-04-24T06:54:47.494 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_pg_recovery_event", 2024-04-24T06:54:47.494 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.494 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.494 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.494 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.494 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.495 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.495 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.495 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-24T06:54:47.495 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.495 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.495 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.495 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-04-24T06:54:47.496 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-04-24T06:54:47.496 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.496 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.496 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.496 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:47.496 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.496 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.496 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.497 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.497 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.497 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.497 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.498 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:47.498 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:47.498 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.498 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.498 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.498 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.498 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.499 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.499 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.499 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.499 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.499 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.499 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.499 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.499 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.500 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.500 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.500 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.500 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.500 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.500 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:47.500 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:47.501 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.501 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.501 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.501 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.501 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.501 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.501 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.501 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.501 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.502 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.502 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.502 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:47.502 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:47.502 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.502 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.503 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.503 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:47.503 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.503 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.503 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.503 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.503 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.503 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.503 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.503 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.504 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.504 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.504 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.504 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.504 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.504 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.504 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:47.505 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:47.505 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.505 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.505 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.505 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.505 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.505 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.505 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.505 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.506 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.506 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.506 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.506 INFO:teuthology.orchestra.run.smithi005.stdout: "max_completed_events": { 2024-04-24T06:54:47.506 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_completed_events", 2024-04-24T06:54:47.506 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:47.506 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.507 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.507 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "50", 2024-04-24T06:54:47.507 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.507 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.507 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.507 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "number of past completed events to remember", 2024-04-24T06:54:47.507 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.508 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.508 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.508 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-04-24T06:54:47.508 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-04-24T06:54:47.509 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:47.509 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.509 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.509 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-04-24T06:54:47.509 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.509 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.509 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.509 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long the module is going to sleep", 2024-04-24T06:54:47.510 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.510 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.510 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.510 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.510 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.510 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus", 2024-04-24T06:54:47.511 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:47.511 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:47.511 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:47.511 INFO:teuthology.orchestra.run.smithi005.stdout: "cache": { 2024-04-24T06:54:47.511 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cache", 2024-04-24T06:54:47.511 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.511 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.511 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.512 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:47.512 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.512 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.512 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.512 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.512 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.512 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.512 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.513 INFO:teuthology.orchestra.run.smithi005.stdout: "exclude_perf_counters": { 2024-04-24T06:54:47.513 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "exclude_perf_counters", 2024-04-24T06:54:47.513 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.513 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.513 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.513 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:47.513 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.513 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.514 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.514 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-24T06:54:47.514 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-24T06:54:47.514 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.514 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.514 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:47.515 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:47.515 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.515 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.515 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.515 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.515 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.515 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.515 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.515 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.516 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.516 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.516 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.516 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.516 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.516 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.516 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.516 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.517 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.517 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.517 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:47.517 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:47.517 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.517 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.517 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.518 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.518 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.518 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.518 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.518 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.519 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.519 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.519 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.519 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:47.519 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:47.519 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.520 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.520 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.520 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:47.520 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.520 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.520 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.520 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.520 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.521 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.521 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.521 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.521 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.521 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.521 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.521 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.521 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.521 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.522 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:47.522 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:47.522 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.522 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.522 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.522 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.523 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.523 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.523 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.523 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.523 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.523 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.523 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.524 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools": { 2024-04-24T06:54:47.524 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools", 2024-04-24T06:54:47.524 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.524 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.524 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.524 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.524 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.524 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.525 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.525 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.525 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.525 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.525 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.525 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-24T06:54:47.525 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-24T06:54:47.526 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:47.526 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.526 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.526 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "300", 2024-04-24T06:54:47.526 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.526 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.526 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.527 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.527 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.527 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.527 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.527 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_interval": { 2024-04-24T06:54:47.527 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_interval", 2024-04-24T06:54:47.527 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-24T06:54:47.528 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.528 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.528 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15.0", 2024-04-24T06:54:47.528 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.528 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.528 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.528 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.528 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.529 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.529 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.529 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-04-24T06:54:47.529 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-04-24T06:54:47.529 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.530 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.530 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.530 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-04-24T06:54:47.530 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.530 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.530 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.530 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-24T06:54:47.530 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.531 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.531 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.531 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-04-24T06:54:47.531 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-04-24T06:54:47.531 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:47.531 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.531 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.532 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "9283", 2024-04-24T06:54:47.532 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.532 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.532 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.532 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-24T06:54:47.532 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.532 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.532 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.533 INFO:teuthology.orchestra.run.smithi005.stdout: "stale_cache_strategy": { 2024-04-24T06:54:47.533 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stale_cache_strategy", 2024-04-24T06:54:47.533 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.533 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.533 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.533 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "log", 2024-04-24T06:54:47.533 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.533 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.534 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.534 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.534 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.534 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.534 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.534 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-04-24T06:54:47.535 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-04-24T06:54:47.535 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.535 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.535 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.535 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "default", 2024-04-24T06:54:47.535 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.535 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.535 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.535 INFO:teuthology.orchestra.run.smithi005.stdout: "default", 2024-04-24T06:54:47.536 INFO:teuthology.orchestra.run.smithi005.stdout: "error" 2024-04-24T06:54:47.536 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.536 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.536 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.536 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.536 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.536 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-04-24T06:54:47.536 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-04-24T06:54:47.537 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:47.537 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.537 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.537 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-04-24T06:54:47.537 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-04-24T06:54:47.537 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-04-24T06:54:47.537 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.537 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.537 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.538 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.538 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.538 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.538 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.538 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2024-04-24T06:54:47.538 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:47.538 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:47.539 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:47.539 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:47.539 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:47.539 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.539 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.539 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.540 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.540 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.540 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.540 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.540 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.540 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.540 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.540 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.540 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.541 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.541 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.541 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.541 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.541 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.541 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.541 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:47.542 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:47.542 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.542 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.542 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.542 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.542 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.542 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.542 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.543 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.543 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.543 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.543 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.543 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:47.543 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:47.543 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.543 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.544 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.544 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:47.544 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.544 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.544 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.544 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.544 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.545 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.545 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.545 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.545 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.545 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.545 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.545 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.546 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.546 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.546 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:47.546 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:47.546 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.546 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.546 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.547 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.547 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.547 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.547 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.547 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.547 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.547 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.548 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.548 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_snap_create": { 2024-04-24T06:54:47.548 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_snap_create", 2024-04-24T06:54:47.548 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:47.548 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.548 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.548 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-04-24T06:54:47.549 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.549 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.549 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.549 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.549 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.549 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.549 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.550 INFO:teuthology.orchestra.run.smithi005.stdout: "mirror_snapshot_schedule": { 2024-04-24T06:54:47.550 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirror_snapshot_schedule", 2024-04-24T06:54:47.550 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.550 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.550 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.550 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.551 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.551 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.551 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.551 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.551 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.551 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.551 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.551 INFO:teuthology.orchestra.run.smithi005.stdout: "trash_purge_schedule": { 2024-04-24T06:54:47.552 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "trash_purge_schedule", 2024-04-24T06:54:47.552 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.552 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.552 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.552 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.552 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.552 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.552 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.553 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.553 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.553 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.553 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.553 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.553 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.554 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "restful", 2024-04-24T06:54:47.554 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:47.554 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:47.554 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:47.554 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_auth": { 2024-04-24T06:54:47.554 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_auth", 2024-04-24T06:54:47.554 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.554 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.555 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.555 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:47.555 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.555 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.555 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.555 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.555 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.555 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.556 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.556 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-04-24T06:54:47.556 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-04-24T06:54:47.556 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.556 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.556 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.556 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.556 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.557 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.557 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.557 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.557 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.557 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.557 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.557 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:47.557 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:47.558 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.558 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.558 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.558 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.558 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.558 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.558 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.558 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.558 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.559 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.559 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.559 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.559 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.559 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.559 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.559 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.559 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.559 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.560 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:47.560 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:47.560 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.560 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.560 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.560 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.561 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.561 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.561 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.561 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.561 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.561 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.561 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.561 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:47.562 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:47.562 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.562 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.562 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.562 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:47.562 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.562 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.562 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.562 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.563 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.563 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.563 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.563 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.563 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.563 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.563 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.563 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.564 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.564 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.564 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:47.564 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:47.564 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.564 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.564 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.564 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.565 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.565 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.565 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.565 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.565 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.565 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.565 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.565 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-04-24T06:54:47.566 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-04-24T06:54:47.566 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.566 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.566 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.566 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.566 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.566 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.566 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.567 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.567 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.567 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.567 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.567 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-04-24T06:54:47.567 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-04-24T06:54:47.567 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.568 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.568 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.568 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.568 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.568 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.568 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.568 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.568 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.568 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.569 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.569 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.569 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.569 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rgw", 2024-04-24T06:54:47.569 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:47.569 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:47.570 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:47.570 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:47.570 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:47.570 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.570 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.570 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.570 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.570 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.571 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.571 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.571 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.571 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.571 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.571 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.571 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.572 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.572 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.572 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.572 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.572 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.572 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.572 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:47.572 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:47.573 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.573 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.573 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.573 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.573 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.573 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.573 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.573 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.574 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.574 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.574 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.574 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:47.574 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:47.574 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.574 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.575 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.575 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:47.575 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.575 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.575 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.575 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.575 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.575 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.575 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.576 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.576 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.576 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.576 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.576 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.576 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.577 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.577 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:47.577 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:47.577 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.577 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.577 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.578 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.578 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.578 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.578 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.578 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.578 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.578 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.578 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.579 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.579 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.579 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rook", 2024-04-24T06:54:47.579 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:47.579 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:47.579 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:47.580 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:47.580 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:47.580 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.580 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.580 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.580 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.580 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.580 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.581 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.581 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.581 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.581 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.581 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.582 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.582 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.582 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.582 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.582 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.582 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.582 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.583 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:47.583 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:47.583 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.583 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.583 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.583 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.583 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.583 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.584 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.584 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.584 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.584 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.584 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.584 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:47.584 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:47.585 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.585 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.585 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.585 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:47.585 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.585 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.585 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.585 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.586 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.586 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.586 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.586 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.586 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.586 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.586 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.586 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.587 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.587 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.587 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:47.587 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:47.587 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.587 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.587 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.588 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.588 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.588 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.588 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.588 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.588 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.588 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.588 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.589 INFO:teuthology.orchestra.run.smithi005.stdout: "storage_class": { 2024-04-24T06:54:47.589 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "storage_class", 2024-04-24T06:54:47.589 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.589 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.589 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.589 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "local", 2024-04-24T06:54:47.589 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.589 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.590 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.590 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-24T06:54:47.590 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.590 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.590 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.590 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.590 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.591 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "selftest", 2024-04-24T06:54:47.591 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:47.591 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:47.591 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:47.591 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:47.591 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:47.591 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.592 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.592 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.592 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.592 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.592 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.592 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.593 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.593 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.593 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.593 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.593 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.593 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.593 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.593 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.594 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.594 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.594 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.594 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:47.594 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:47.594 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.594 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.595 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.595 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.595 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.595 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.595 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.595 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.595 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.595 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.595 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.596 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:47.596 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:47.596 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.596 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.596 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.596 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:47.596 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.596 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.597 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.597 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.597 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.597 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.597 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.597 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.597 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.598 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.598 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.598 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.598 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.598 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.598 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:47.599 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:47.599 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.599 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.599 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.599 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.599 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.599 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.599 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.600 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.600 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.600 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.600 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.600 INFO:teuthology.orchestra.run.smithi005.stdout: "roption1": { 2024-04-24T06:54:47.600 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption1", 2024-04-24T06:54:47.600 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.601 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.601 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.601 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.601 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.601 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.601 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.601 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.601 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.601 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.602 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.602 INFO:teuthology.orchestra.run.smithi005.stdout: "roption2": { 2024-04-24T06:54:47.602 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption2", 2024-04-24T06:54:47.602 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.603 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.603 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.603 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "xyz", 2024-04-24T06:54:47.603 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.603 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.603 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.603 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.603 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.603 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.604 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.604 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption1": { 2024-04-24T06:54:47.604 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption1", 2024-04-24T06:54:47.604 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.604 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.604 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.604 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.605 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.605 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.605 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.605 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.605 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.605 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.605 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.605 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption2": { 2024-04-24T06:54:47.606 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption2", 2024-04-24T06:54:47.606 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:47.606 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.606 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.606 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.606 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.606 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.607 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.607 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.607 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.607 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.607 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.607 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption3": { 2024-04-24T06:54:47.607 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption3", 2024-04-24T06:54:47.608 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-04-24T06:54:47.608 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.608 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.608 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.608 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.608 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.608 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.608 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.608 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.609 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.609 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.609 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption4": { 2024-04-24T06:54:47.609 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption4", 2024-04-24T06:54:47.609 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.609 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.609 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.610 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.610 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.610 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.610 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.610 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.610 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.610 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.610 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.611 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption5": { 2024-04-24T06:54:47.611 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption5", 2024-04-24T06:54:47.611 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.611 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.611 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.611 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.611 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.611 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.612 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.612 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.612 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.612 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.612 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.612 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption6": { 2024-04-24T06:54:47.613 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption6", 2024-04-24T06:54:47.613 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.613 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.613 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.613 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:47.613 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.613 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.613 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.614 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.614 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.614 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.614 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.614 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption7": { 2024-04-24T06:54:47.614 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption7", 2024-04-24T06:54:47.614 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:47.615 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.615 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.615 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.615 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-04-24T06:54:47.615 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "42", 2024-04-24T06:54:47.615 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.615 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.615 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.615 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.616 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.616 INFO:teuthology.orchestra.run.smithi005.stdout: "testkey": { 2024-04-24T06:54:47.616 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testkey", 2024-04-24T06:54:47.616 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.616 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.616 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.616 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.616 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.617 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.617 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.617 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.617 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.617 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.617 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.617 INFO:teuthology.orchestra.run.smithi005.stdout: "testlkey": { 2024-04-24T06:54:47.618 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testlkey", 2024-04-24T06:54:47.618 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.618 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.618 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.618 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.618 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.618 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.619 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.619 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.619 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.619 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.619 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.620 INFO:teuthology.orchestra.run.smithi005.stdout: "testnewline": { 2024-04-24T06:54:47.620 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testnewline", 2024-04-24T06:54:47.620 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.620 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.620 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.620 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.620 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.620 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.621 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.621 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.621 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.621 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.621 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.621 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.621 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.622 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2024-04-24T06:54:47.622 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:47.622 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:47.622 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:47.622 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_m_granularity": { 2024-04-24T06:54:47.622 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_m_granularity", 2024-04-24T06:54:47.622 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.622 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.623 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.623 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.623 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.623 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.623 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.623 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow minute scheduled snapshots", 2024-04-24T06:54:47.624 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.624 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.624 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.624 INFO:teuthology.orchestra.run.smithi005.stdout: "dump_on_update": { 2024-04-24T06:54:47.624 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dump_on_update", 2024-04-24T06:54:47.624 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.625 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.625 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.625 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.625 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.625 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.625 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.625 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "dump database to debug log on update", 2024-04-24T06:54:47.625 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.626 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.626 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.626 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:47.626 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:47.626 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.626 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.626 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.627 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.627 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.627 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.627 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.627 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.627 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.627 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.627 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.628 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.628 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.628 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.628 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.628 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.628 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.628 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.629 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:47.629 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:47.629 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.629 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.629 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.629 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.629 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.629 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.630 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.630 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.630 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.630 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.630 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.630 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:47.630 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:47.631 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.631 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.631 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.631 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:47.631 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.631 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.631 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.631 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.631 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.632 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.632 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.632 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.632 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.632 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.632 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.632 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.632 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.632 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.633 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:47.633 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:47.633 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.633 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.633 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.634 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.634 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.634 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.634 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.634 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.634 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.635 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.635 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.635 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.635 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.635 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.635 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stats", 2024-04-24T06:54:47.635 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:47.636 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:47.636 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:47.636 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:47.636 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:47.636 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.636 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.636 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.636 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.636 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.637 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.637 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.637 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.637 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.637 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.637 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.637 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.637 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.637 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.638 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.638 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.638 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.638 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.638 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:47.638 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:47.638 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.638 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.639 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.639 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.639 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.639 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.639 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.639 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.639 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.639 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.639 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.640 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:47.640 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:47.640 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.640 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.640 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.640 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:47.640 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.641 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.641 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.641 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.641 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.641 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.641 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.641 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.641 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.641 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.642 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.642 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.642 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.642 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.642 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:47.642 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:47.642 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.643 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.643 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.643 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.643 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.643 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.643 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.643 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.643 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.644 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.644 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.644 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.644 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.645 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "status", 2024-04-24T06:54:47.645 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:47.645 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:47.645 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:47.645 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:47.645 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:47.645 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.645 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.646 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.646 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.646 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.646 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.646 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.646 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.646 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.647 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.647 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.647 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.647 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.647 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.647 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.647 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.647 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.647 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.648 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:47.648 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:47.648 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.648 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.648 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.648 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.648 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.649 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.649 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.649 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.649 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.649 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.649 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.649 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:47.649 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:47.650 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.650 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.650 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.650 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:47.650 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.650 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.650 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.650 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.651 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.651 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.651 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.651 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.651 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.651 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.651 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.651 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.652 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.652 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.652 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:47.652 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:47.652 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.652 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.652 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.653 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.653 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.653 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.653 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.653 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.653 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.653 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.653 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.654 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.654 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.654 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.654 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telegraf", 2024-04-24T06:54:47.654 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:47.655 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:47.655 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:47.655 INFO:teuthology.orchestra.run.smithi005.stdout: "address": { 2024-04-24T06:54:47.655 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "address", 2024-04-24T06:54:47.655 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.655 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.655 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.656 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-24T06:54:47.656 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.656 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.656 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.656 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.656 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.656 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.657 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.657 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-04-24T06:54:47.657 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-04-24T06:54:47.657 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:47.657 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.657 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.657 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15", 2024-04-24T06:54:47.657 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.658 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.658 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.658 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.658 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.658 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.658 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.658 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:47.659 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:47.659 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.659 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.659 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.659 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.659 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.659 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.659 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.659 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.660 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.660 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.660 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.660 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.660 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.660 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.660 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.660 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.660 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.661 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.661 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:47.661 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:47.661 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.661 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.661 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.661 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.661 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.662 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.662 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.662 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.662 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.662 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.662 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.662 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.662 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:47.663 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:47.663 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.663 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.663 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.663 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:47.663 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.663 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.663 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.663 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.664 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.664 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.664 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.664 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.664 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.664 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.664 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.664 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.665 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.665 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.665 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:47.665 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:47.665 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.665 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.666 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.666 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.666 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.666 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.666 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.666 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.666 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.666 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.667 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.667 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.667 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.667 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.667 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telemetry", 2024-04-24T06:54:47.667 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:47.667 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:47.667 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:47.668 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_basic": { 2024-04-24T06:54:47.668 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_basic", 2024-04-24T06:54:47.668 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.668 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.668 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.668 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:47.668 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.668 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.668 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.669 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-24T06:54:47.669 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.669 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.669 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.669 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_crash": { 2024-04-24T06:54:47.669 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_crash", 2024-04-24T06:54:47.669 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.669 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.670 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.670 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:47.670 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.670 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.670 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.670 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-24T06:54:47.670 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.671 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.671 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.671 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.671 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_device": { 2024-04-24T06:54:47.671 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_device", 2024-04-24T06:54:47.671 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.671 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.671 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.671 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:47.672 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.672 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.672 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.672 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-24T06:54:47.672 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.672 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.672 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.673 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_ident": { 2024-04-24T06:54:47.673 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_ident", 2024-04-24T06:54:47.673 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.673 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.673 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.673 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.673 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.673 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.673 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.674 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-24T06:54:47.674 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.674 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.674 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.674 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_perf": { 2024-04-24T06:54:47.674 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_perf", 2024-04-24T06:54:47.674 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.675 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.675 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.675 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.675 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.675 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.675 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.676 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-24T06:54:47.676 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.676 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.676 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.676 INFO:teuthology.orchestra.run.smithi005.stdout: "contact": { 2024-04-24T06:54:47.676 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "contact", 2024-04-24T06:54:47.677 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.677 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.677 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.677 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.677 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.677 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.677 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.678 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.678 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.678 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.678 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.678 INFO:teuthology.orchestra.run.smithi005.stdout: "description": { 2024-04-24T06:54:47.678 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "description", 2024-04-24T06:54:47.679 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.679 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.679 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.679 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.679 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.679 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.679 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.680 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.680 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.680 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.680 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.680 INFO:teuthology.orchestra.run.smithi005.stdout: "device_url": { 2024-04-24T06:54:47.680 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_url", 2024-04-24T06:54:47.681 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.681 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.681 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.681 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-24T06:54:47.681 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.681 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.681 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.681 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.682 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.682 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.682 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.682 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-04-24T06:54:47.682 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-04-24T06:54:47.682 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.682 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.683 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.683 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.683 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.683 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.683 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.683 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.683 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.684 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.684 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.684 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-04-24T06:54:47.684 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-04-24T06:54:47.684 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:47.684 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.684 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.685 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "24", 2024-04-24T06:54:47.685 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "8", 2024-04-24T06:54:47.685 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.685 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.685 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.685 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.685 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.685 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.686 INFO:teuthology.orchestra.run.smithi005.stdout: "last_opt_revision": { 2024-04-24T06:54:47.686 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "last_opt_revision", 2024-04-24T06:54:47.686 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:47.686 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.686 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.686 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1", 2024-04-24T06:54:47.687 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.687 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.687 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.687 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.687 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.687 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.687 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.687 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard": { 2024-04-24T06:54:47.688 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard", 2024-04-24T06:54:47.688 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.688 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.688 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.688 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.688 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.688 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.688 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.688 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.689 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.689 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.689 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.689 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard_description": { 2024-04-24T06:54:47.689 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard_description", 2024-04-24T06:54:47.689 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.689 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.689 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.690 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.690 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.690 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.690 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.690 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.690 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.690 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.690 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.691 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:47.691 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:47.691 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.691 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.691 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.691 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.691 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.691 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.692 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.692 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.692 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.692 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.692 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.692 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.692 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.692 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.692 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.693 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.693 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.693 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.693 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:47.693 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:47.693 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.693 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.694 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.694 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.694 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.694 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.694 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.694 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.694 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.695 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.695 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.695 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:47.695 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:47.695 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.695 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.695 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.695 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:47.696 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.696 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.696 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.696 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.696 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.697 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.697 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.697 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.697 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.697 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.697 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.697 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.697 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.697 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.698 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:47.698 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:47.698 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.698 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.698 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.698 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.698 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.699 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.699 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.699 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.699 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.699 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.699 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.700 INFO:teuthology.orchestra.run.smithi005.stdout: "organization": { 2024-04-24T06:54:47.700 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "organization", 2024-04-24T06:54:47.700 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.700 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.700 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.700 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.700 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.700 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.701 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.701 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.701 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.701 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.701 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.701 INFO:teuthology.orchestra.run.smithi005.stdout: "proxy": { 2024-04-24T06:54:47.701 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "proxy", 2024-04-24T06:54:47.702 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.702 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.702 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.702 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.702 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.702 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.702 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.702 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.703 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.703 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.703 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.703 INFO:teuthology.orchestra.run.smithi005.stdout: "url": { 2024-04-24T06:54:47.703 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url", 2024-04-24T06:54:47.703 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.703 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.704 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.704 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-24T06:54:47.704 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.704 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.704 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.704 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.704 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.704 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.705 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.705 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.705 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.705 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.705 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "test_orchestrator", 2024-04-24T06:54:47.705 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:47.705 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:47.706 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:47.706 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:47.706 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:47.706 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.706 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.706 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.707 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.707 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.707 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.707 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.707 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.707 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.707 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.707 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.707 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.708 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.708 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.708 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.708 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.708 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.708 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.708 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:47.709 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:47.709 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.709 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.709 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.709 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.709 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.709 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.709 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.709 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.710 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.710 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.710 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.710 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:47.710 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:47.710 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.711 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.711 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.711 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:47.711 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.711 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.711 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.711 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.711 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.712 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.712 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.712 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.712 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.712 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.712 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.712 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.712 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.713 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.713 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:47.713 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:47.713 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.713 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.713 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.713 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.713 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.714 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.714 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.714 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.714 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.714 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.714 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.714 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.714 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.715 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.715 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2024-04-24T06:54:47.715 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:47.715 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:47.715 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:47.715 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:47.715 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:47.715 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.716 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.716 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.716 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.716 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.716 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.716 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.716 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.716 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.716 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.716 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.717 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.717 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.717 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.717 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.717 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.717 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.717 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.718 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:47.718 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:47.718 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.718 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.718 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.718 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.718 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.719 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.719 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.719 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.719 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.719 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.719 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.720 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:47.720 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:47.720 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.720 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.720 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.720 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:47.720 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.720 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.721 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.721 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.721 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.721 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.721 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.721 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.721 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.721 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.721 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.722 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.722 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.722 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.722 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:47.722 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:47.722 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.722 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.723 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.723 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.723 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.723 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.723 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.723 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.723 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.723 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.724 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.724 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_clones": { 2024-04-24T06:54:47.724 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_clones", 2024-04-24T06:54:47.724 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:47.724 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.724 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.725 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4", 2024-04-24T06:54:47.725 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.725 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.725 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.725 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-24T06:54:47.725 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.725 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.725 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.726 INFO:teuthology.orchestra.run.smithi005.stdout: "periodic_async_work": { 2024-04-24T06:54:47.726 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "periodic_async_work", 2024-04-24T06:54:47.726 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.726 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.726 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.726 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.726 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.727 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.727 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.727 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Periodically check for async work", 2024-04-24T06:54:47.727 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.727 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.727 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.728 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_delay": { 2024-04-24T06:54:47.728 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_delay", 2024-04-24T06:54:47.728 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:47.728 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.728 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.728 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-04-24T06:54:47.728 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.728 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.728 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.729 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-24T06:54:47.729 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.729 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.729 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.729 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_no_wait": { 2024-04-24T06:54:47.729 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_no_wait", 2024-04-24T06:54:47.729 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.729 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.730 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.730 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-04-24T06:54:47.730 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.730 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.730 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.730 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-24T06:54:47.730 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.730 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.731 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.731 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.731 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.731 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.731 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix", 2024-04-24T06:54:47.731 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-04-24T06:54:47.731 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-04-24T06:54:47.731 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-04-24T06:54:47.732 INFO:teuthology.orchestra.run.smithi005.stdout: "discovery_interval": { 2024-04-24T06:54:47.732 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "discovery_interval", 2024-04-24T06:54:47.732 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-04-24T06:54:47.732 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.732 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.732 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "100", 2024-04-24T06:54:47.732 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.732 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.732 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.733 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.733 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.733 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.733 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.733 INFO:teuthology.orchestra.run.smithi005.stdout: "identifier": { 2024-04-24T06:54:47.733 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "identifier", 2024-04-24T06:54:47.733 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.734 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.734 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.734 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.734 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.734 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.734 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.734 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.734 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.735 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.735 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.735 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.735 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-04-24T06:54:47.735 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-04-24T06:54:47.735 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-04-24T06:54:47.736 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.736 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.736 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-04-24T06:54:47.736 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.736 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.736 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.736 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.736 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.736 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.736 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.737 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-04-24T06:54:47.737 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-04-24T06:54:47.737 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.737 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.737 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.737 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.737 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.738 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.738 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.738 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.738 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.738 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.738 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.738 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.739 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.739 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.739 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.739 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.739 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.739 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.739 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-04-24T06:54:47.739 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-04-24T06:54:47.740 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.740 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.740 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.740 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.740 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.740 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.740 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.740 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.740 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.741 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.741 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.741 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-04-24T06:54:47.741 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-04-24T06:54:47.741 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.741 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.742 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.742 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-04-24T06:54:47.742 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.742 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.742 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-04-24T06:54:47.742 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-04-24T06:54:47.742 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-04-24T06:54:47.742 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-04-24T06:54:47.743 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-04-24T06:54:47.743 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-04-24T06:54:47.743 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-04-24T06:54:47.743 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.743 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.743 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.743 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.743 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.744 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-04-24T06:54:47.744 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-04-24T06:54:47.744 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-04-24T06:54:47.744 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.744 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-04-24T06:54:47.744 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-04-24T06:54:47.744 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.744 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.745 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.745 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.745 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.745 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.745 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.745 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_host": { 2024-04-24T06:54:47.745 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_host", 2024-04-24T06:54:47.746 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.746 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.746 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.746 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-04-24T06:54:47.746 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.746 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.746 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.747 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.747 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.747 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.747 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.747 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_port": { 2024-04-24T06:54:47.747 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_port", 2024-04-24T06:54:47.747 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-04-24T06:54:47.747 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.748 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.748 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10051", 2024-04-24T06:54:47.748 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.748 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.748 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.748 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.748 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.749 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.749 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.749 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_sender": { 2024-04-24T06:54:47.749 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_sender", 2024-04-24T06:54:47.749 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-04-24T06:54:47.749 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-04-24T06:54:47.749 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-04-24T06:54:47.750 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-24T06:54:47.750 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-04-24T06:54:47.750 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-04-24T06:54:47.750 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-04-24T06:54:47.750 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-04-24T06:54:47.750 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-04-24T06:54:47.750 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-04-24T06:54:47.750 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-04-24T06:54:47.751 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.751 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.751 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.751 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.751 INFO:teuthology.orchestra.run.smithi005.stdout: "services": { 2024-04-24T06:54:47.751 INFO:teuthology.orchestra.run.smithi005.stdout: "dashboard": "https://172.21.15.5:8443/" 2024-04-24T06:54:47.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.751 INFO:teuthology.orchestra.run.smithi005.stdout: "always_on_modules": { 2024-04-24T06:54:47.751 INFO:teuthology.orchestra.run.smithi005.stdout: "octopus": [ 2024-04-24T06:54:47.752 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-04-24T06:54:47.752 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-04-24T06:54:47.752 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-04-24T06:54:47.752 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-04-24T06:54:47.752 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-04-24T06:54:47.752 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-04-24T06:54:47.752 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-04-24T06:54:47.752 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-04-24T06:54:47.753 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-04-24T06:54:47.753 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-04-24T06:54:47.753 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.753 INFO:teuthology.orchestra.run.smithi005.stdout: "pacific": [ 2024-04-24T06:54:47.753 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-04-24T06:54:47.753 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-04-24T06:54:47.753 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-04-24T06:54:47.753 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-04-24T06:54:47.753 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-04-24T06:54:47.754 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-04-24T06:54:47.754 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-04-24T06:54:47.754 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-04-24T06:54:47.754 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-04-24T06:54:47.754 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-04-24T06:54:47.754 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.754 INFO:teuthology.orchestra.run.smithi005.stdout: "quincy": [ 2024-04-24T06:54:47.754 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-04-24T06:54:47.755 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-04-24T06:54:47.755 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-04-24T06:54:47.755 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-04-24T06:54:47.755 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-04-24T06:54:47.755 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-04-24T06:54:47.755 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-04-24T06:54:47.755 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-04-24T06:54:47.755 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-04-24T06:54:47.756 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-04-24T06:54:47.756 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.756 INFO:teuthology.orchestra.run.smithi005.stdout: "reef": [ 2024-04-24T06:54:47.756 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-04-24T06:54:47.756 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-04-24T06:54:47.756 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-04-24T06:54:47.756 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-04-24T06:54:47.756 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-04-24T06:54:47.757 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-04-24T06:54:47.757 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-04-24T06:54:47.757 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-04-24T06:54:47.757 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-04-24T06:54:47.757 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-04-24T06:54:47.757 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-24T06:54:47.757 INFO:teuthology.orchestra.run.smithi005.stdout: "squid": [ 2024-04-24T06:54:47.757 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-04-24T06:54:47.758 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-04-24T06:54:47.758 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-04-24T06:54:47.758 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-04-24T06:54:47.758 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-04-24T06:54:47.758 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-04-24T06:54:47.758 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-04-24T06:54:47.759 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-04-24T06:54:47.759 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-04-24T06:54:47.759 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-04-24T06:54:47.759 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-04-24T06:54:47.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.759 INFO:teuthology.orchestra.run.smithi005.stdout: "last_failure_osd_epoch": 89, 2024-04-24T06:54:47.759 INFO:teuthology.orchestra.run.smithi005.stdout: "active_clients": [ 2024-04-24T06:54:47.759 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.760 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2024-04-24T06:54:47.760 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-04-24T06:54:47.760 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.760 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-04-24T06:54:47.760 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-04-24T06:54:47.760 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1506965886 2024-04-24T06:54:47.760 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.760 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-04-24T06:54:47.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.761 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.761 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "libcephsqlite", 2024-04-24T06:54:47.761 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-04-24T06:54:47.761 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.761 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-04-24T06:54:47.761 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-04-24T06:54:47.761 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 207486499 2024-04-24T06:54:47.761 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.762 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-04-24T06:54:47.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.762 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.762 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2024-04-24T06:54:47.762 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-04-24T06:54:47.762 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.762 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-04-24T06:54:47.762 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-04-24T06:54:47.763 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4080880795 2024-04-24T06:54:47.763 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.763 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-04-24T06:54:47.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.763 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.763 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2024-04-24T06:54:47.763 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-04-24T06:54:47.763 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.763 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-04-24T06:54:47.764 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-04-24T06:54:47.764 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4223197165 2024-04-24T06:54:47.764 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.764 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-04-24T06:54:47.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.764 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.764 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2024-04-24T06:54:47.764 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-04-24T06:54:47.765 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.765 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-04-24T06:54:47.765 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-04-24T06:54:47.765 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1128641242 2024-04-24T06:54:47.765 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.765 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-04-24T06:54:47.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:47.765 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.765 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2024-04-24T06:54:47.766 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-04-24T06:54:47.766 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-24T06:54:47.766 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-04-24T06:54:47.766 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-04-24T06:54:47.766 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3580091734 2024-04-24T06:54:47.766 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.766 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-04-24T06:54:47.767 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:47.767 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-04-24T06:54:47.767 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:54:47.768 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:45 smithi137 ceph-mon[33017]: pgmap v332: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:54:47.768 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:45 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/3415290759' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-24T06:54:47.769 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:46 smithi137 ceph-mon[33017]: pgmap v333: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:54:47.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:45 smithi037 ceph-mon[32864]: pgmap v332: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:54:47.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:45 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/3415290759' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-24T06:54:47.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:46 smithi037 ceph-mon[32864]: pgmap v333: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:54:47.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:45.109+0000 7fca25000640 1 -- 172.21.15.5:0/3415290759 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fca040a3d10 msgr2=0x7fca040a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:47.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:45.109+0000 7fca25000640 1 --2- 172.21.15.5:0/3415290759 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fca040a3d10 0x7fca040a61d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fca10009ca0 tx=0x7fca10005d80 comp rx=0 tx=0).stop 2024-04-24T06:54:47.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:45.109+0000 7fca25000640 1 -- 172.21.15.5:0/3415290759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca2019eb60 msgr2=0x7fca2019efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:54:47.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:45.109+0000 7fca25000640 1 --2- 172.21.15.5:0/3415290759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca2019eb60 0x7fca2019efe0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fca1800e970 tx=0x7fca1800ee30 comp rx=0 tx=0).stop 2024-04-24T06:54:47.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:45.109+0000 7fca25000640 1 -- 172.21.15.5:0/3415290759 shutdown_connections 2024-04-24T06:54:47.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:45.109+0000 7fca25000640 1 --2- 172.21.15.5:0/3415290759 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fca040a3d10 0x7fca040a61d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:47.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:45.109+0000 7fca25000640 1 --2- 172.21.15.5:0/3415290759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca2019eb60 0x7fca2019efe0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:47.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:45.109+0000 7fca25000640 1 --2- 172.21.15.5:0/3415290759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca2010d8a0 0x7fca201a4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:47.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:45.109+0000 7fca25000640 1 --2- 172.21.15.5:0/3415290759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca201087b0 0x7fca201a4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:54:47.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:45.109+0000 7fca25000640 1 -- 172.21.15.5:0/3415290759 >> 172.21.15.5:0/3415290759 conn(0x7fca20077dd0 msgr2=0x7fca20079a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:54:47.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:45.110+0000 7fca25000640 1 -- 172.21.15.5:0/3415290759 shutdown_connections 2024-04-24T06:54:47.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:54:45.110+0000 7fca25000640 1 -- 172.21.15.5:0/3415290759 wait complete. 2024-04-24T06:54:47.779 DEBUG:tasks.check_counter:Getting stats from a 2024-04-24T06:54:47.780 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-24T06:54:47.780 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:45 smithi005 ceph-mon[27798]: pgmap v332: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:54:47.780 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:45 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/3415290759' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-24T06:54:47.781 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:46 smithi005 ceph-mon[27798]: pgmap v333: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:54:48.242 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:54:49.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:49 smithi137 ceph-mon[33017]: pgmap v334: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:54:49.739 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:49 smithi005 ceph-mon[27798]: pgmap v334: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:54:49.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:49 smithi037 ceph-mon[32864]: pgmap v334: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:54:50.619 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:54:50.619 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T06:54:50.619 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 0, 2024-04-24T06:54:50.620 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 2, 2024-04-24T06:54:50.620 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 0, 2024-04-24T06:54:50.620 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 0, 2024-04-24T06:54:50.620 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 4, 2024-04-24T06:54:50.620 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 0, 2024-04-24T06:54:50.620 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.004658461, 2024-04-24T06:54:50.620 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.000000000, 2024-04-24T06:54:50.620 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.000938400, 2024-04-24T06:54:50.621 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T06:54:50.621 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T06:54:50.621 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-24T06:54:50.621 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-24T06:54:50.621 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-24T06:54:50.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:50.621 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-04-24T06:54:50.621 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-24T06:54:50.621 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-24T06:54:50.622 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-24T06:54:50.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:50.622 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T06:54:50.622 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T06:54:50.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:50.623 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T06:54:50.623 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 232, 2024-04-24T06:54:50.623 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 218, 2024-04-24T06:54:50.623 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 1733072, 2024-04-24T06:54:50.623 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 133472, 2024-04-24T06:54:50.623 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 2, 2024-04-24T06:54:50.623 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 1, 2024-04-24T06:54:50.623 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.105512547, 2024-04-24T06:54:50.624 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.033920535, 2024-04-24T06:54:50.624 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.084547025, 2024-04-24T06:54:50.624 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.010271370, 2024-04-24T06:54:50.624 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T06:54:50.624 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 218, 2024-04-24T06:54:50.624 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.022883630, 2024-04-24T06:54:50.624 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000104970 2024-04-24T06:54:50.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:50.625 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-04-24T06:54:50.625 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-24T06:54:50.625 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-24T06:54:50.625 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-24T06:54:50.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:50.625 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 1733072, 2024-04-24T06:54:50.625 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 133472 2024-04-24T06:54:50.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:50.626 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T06:54:50.626 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 2, 2024-04-24T06:54:50.626 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 165, 2024-04-24T06:54:50.626 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 320, 2024-04-24T06:54:50.626 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 185312, 2024-04-24T06:54:50.626 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 5, 2024-04-24T06:54:50.626 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 3, 2024-04-24T06:54:50.626 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.036183208, 2024-04-24T06:54:50.627 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.026262711, 2024-04-24T06:54:50.627 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.004164848, 2024-04-24T06:54:50.627 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T06:54:50.627 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T06:54:50.627 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 162, 2024-04-24T06:54:50.627 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.035581141, 2024-04-24T06:54:50.627 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000219636 2024-04-24T06:54:50.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:50.628 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-04-24T06:54:50.628 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-24T06:54:50.628 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-24T06:54:50.628 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-24T06:54:50.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:50.628 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-24T06:54:50.628 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 185312 2024-04-24T06:54:50.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:50.628 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2024-04-24T06:54:50.629 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 0, 2024-04-24T06:54:50.629 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2024-04-24T06:54:50.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:50.629 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2024-04-24T06:54:50.629 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2024-04-24T06:54:50.629 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2024-04-24T06:54:50.629 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T06:54:50.629 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2024-04-24T06:54:50.630 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T06:54:50.630 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2024-04-24T06:54:50.630 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T06:54:50.630 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T06:54:50.630 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T06:54:50.630 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T06:54:50.631 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T06:54:50.631 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2024-04-24T06:54:50.631 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T06:54:50.632 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T06:54:50.632 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2024-04-24T06:54:50.632 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2024-04-24T06:54:50.632 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2024-04-24T06:54:50.633 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2024-04-24T06:54:50.633 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T06:54:50.633 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T06:54:50.633 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T06:54:50.633 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T06:54:50.633 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T06:54:50.634 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2024-04-24T06:54:50.634 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2024-04-24T06:54:50.634 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2024-04-24T06:54:50.634 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T06:54:50.634 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T06:54:50.634 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2024-04-24T06:54:50.634 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2024-04-24T06:54:50.634 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2024-04-24T06:54:50.635 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2024-04-24T06:54:50.635 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T06:54:50.635 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2024-04-24T06:54:50.636 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T06:54:50.636 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2024-04-24T06:54:50.636 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 148977, 2024-04-24T06:54:50.636 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 30, 2024-04-24T06:54:50.636 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2024-04-24T06:54:50.636 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2024-04-24T06:54:50.636 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2024-04-24T06:54:50.637 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2024-04-24T06:54:50.637 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2024-04-24T06:54:50.637 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2024-04-24T06:54:50.637 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2024-04-24T06:54:50.637 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2024-04-24T06:54:50.637 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 0, 2024-04-24T06:54:50.637 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 0, 2024-04-24T06:54:50.637 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T06:54:50.638 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2024-04-24T06:54:50.638 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2024-04-24T06:54:50.638 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2024-04-24T06:54:50.638 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 928, 2024-04-24T06:54:50.638 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 2, 2024-04-24T06:54:50.638 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2024-04-24T06:54:50.638 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2024-04-24T06:54:50.638 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2024-04-24T06:54:50.639 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2024-04-24T06:54:50.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:50.639 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T06:54:50.639 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-04-24T06:54:50.639 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-04-24T06:54:50.639 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-04-24T06:54:50.639 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 230, 2024-04-24T06:54:50.639 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 1697902, 2024-04-24T06:54:50.639 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-04-24T06:54:50.640 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 230, 2024-04-24T06:54:50.640 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-04-24T06:54:50.640 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-04-24T06:54:50.640 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 230, 2024-04-24T06:54:50.640 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 1697902, 2024-04-24T06:54:50.640 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-04-24T06:54:50.640 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-24T06:54:50.641 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-24T06:54:50.641 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-24T06:54:50.641 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:50.641 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:50.641 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:54:51.465 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-04-24T06:54:51.466 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-04-24T06:54:51.466 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-04-24T06:54:51.466 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-04-24T06:54:51.466 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-04-24T06:54:51.466 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-04-24T06:54:51.466 DEBUG:tasks.check_counter:Getting stats from d 2024-04-24T06:54:51.466 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-24T06:54:51.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:51 smithi137 ceph-mon[33017]: pgmap v335: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:54:51.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:51 smithi005 ceph-mon[27798]: pgmap v335: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:54:51.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:51 smithi037 ceph-mon[32864]: pgmap v335: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:54:51.949 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:54:53.449 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:53 smithi005 ceph-mon[27798]: pgmap v336: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:54:53.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:53 smithi137 ceph-mon[33017]: pgmap v336: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:54:53.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:53 smithi037 ceph-mon[32864]: pgmap v336: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:54:54.079 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:54:54.080 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T06:54:54.080 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 4, 2024-04-24T06:54:54.080 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 4, 2024-04-24T06:54:54.080 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 3024, 2024-04-24T06:54:54.080 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 448, 2024-04-24T06:54:54.080 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 4, 2024-04-24T06:54:54.081 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 0, 2024-04-24T06:54:54.081 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.004323130, 2024-04-24T06:54:54.081 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.000073841, 2024-04-24T06:54:54.081 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.001235114, 2024-04-24T06:54:54.081 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T06:54:54.081 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T06:54:54.081 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 3, 2024-04-24T06:54:54.081 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.001446001, 2024-04-24T06:54:54.082 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000482000 2024-04-24T06:54:54.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:54.082 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-04-24T06:54:54.082 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-24T06:54:54.082 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-24T06:54:54.082 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-24T06:54:54.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:54.083 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-04-24T06:54:54.083 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 448 2024-04-24T06:54:54.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:54.083 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T06:54:54.083 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 2, 2024-04-24T06:54:54.083 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 170, 2024-04-24T06:54:54.083 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 320, 2024-04-24T06:54:54.083 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 186192, 2024-04-24T06:54:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 5, 2024-04-24T06:54:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 3, 2024-04-24T06:54:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.036292781, 2024-04-24T06:54:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.025593186, 2024-04-24T06:54:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.005434401, 2024-04-24T06:54:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T06:54:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T06:54:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 167, 2024-04-24T06:54:54.085 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.039153430, 2024-04-24T06:54:54.085 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000234451 2024-04-24T06:54:54.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:54.085 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-04-24T06:54:54.085 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-24T06:54:54.085 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-24T06:54:54.085 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-24T06:54:54.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:54.086 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-24T06:54:54.086 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 186192 2024-04-24T06:54:54.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:54.086 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T06:54:54.086 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 234, 2024-04-24T06:54:54.086 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 222, 2024-04-24T06:54:54.086 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 1731680, 2024-04-24T06:54:54.086 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 136864, 2024-04-24T06:54:54.087 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 2, 2024-04-24T06:54:54.087 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 1, 2024-04-24T06:54:54.087 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.099750326, 2024-04-24T06:54:54.087 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.034337794, 2024-04-24T06:54:54.087 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.074679300, 2024-04-24T06:54:54.087 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.010826157, 2024-04-24T06:54:54.087 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T06:54:54.087 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 221, 2024-04-24T06:54:54.088 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.022132344, 2024-04-24T06:54:54.088 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000100146 2024-04-24T06:54:54.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:54.088 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-04-24T06:54:54.088 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-24T06:54:54.088 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-24T06:54:54.088 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-24T06:54:54.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:54.089 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 1731680, 2024-04-24T06:54:54.089 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 136864 2024-04-24T06:54:54.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:54.089 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2024-04-24T06:54:54.089 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 0, 2024-04-24T06:54:54.089 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2024-04-24T06:54:54.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:54.090 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2024-04-24T06:54:54.090 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2024-04-24T06:54:54.090 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2024-04-24T06:54:54.090 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T06:54:54.090 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2024-04-24T06:54:54.090 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T06:54:54.090 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2024-04-24T06:54:54.090 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T06:54:54.091 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T06:54:54.091 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T06:54:54.091 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T06:54:54.091 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T06:54:54.091 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2024-04-24T06:54:54.091 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T06:54:54.091 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T06:54:54.091 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2024-04-24T06:54:54.091 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2024-04-24T06:54:54.092 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2024-04-24T06:54:54.092 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2024-04-24T06:54:54.092 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T06:54:54.092 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T06:54:54.092 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T06:54:54.093 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T06:54:54.093 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T06:54:54.093 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2024-04-24T06:54:54.093 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2024-04-24T06:54:54.093 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2024-04-24T06:54:54.093 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T06:54:54.093 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T06:54:54.093 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2024-04-24T06:54:54.094 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2024-04-24T06:54:54.094 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2024-04-24T06:54:54.094 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2024-04-24T06:54:54.094 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T06:54:54.094 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2024-04-24T06:54:54.095 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T06:54:54.095 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2024-04-24T06:54:54.095 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 148714, 2024-04-24T06:54:54.095 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 30, 2024-04-24T06:54:54.095 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2024-04-24T06:54:54.095 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2024-04-24T06:54:54.095 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2024-04-24T06:54:54.095 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2024-04-24T06:54:54.096 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2024-04-24T06:54:54.096 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2024-04-24T06:54:54.096 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2024-04-24T06:54:54.096 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2024-04-24T06:54:54.096 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 0, 2024-04-24T06:54:54.097 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 0, 2024-04-24T06:54:54.097 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T06:54:54.097 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2024-04-24T06:54:54.097 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2024-04-24T06:54:54.097 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2024-04-24T06:54:54.097 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 928, 2024-04-24T06:54:54.097 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 2, 2024-04-24T06:54:54.097 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2024-04-24T06:54:54.098 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2024-04-24T06:54:54.098 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2024-04-24T06:54:54.098 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2024-04-24T06:54:54.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:54.098 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T06:54:54.099 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-04-24T06:54:54.099 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-04-24T06:54:54.099 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-04-24T06:54:54.099 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 236, 2024-04-24T06:54:54.099 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 1698682, 2024-04-24T06:54:54.099 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-04-24T06:54:54.099 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 236, 2024-04-24T06:54:54.099 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-04-24T06:54:54.100 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-04-24T06:54:54.100 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 236, 2024-04-24T06:54:54.100 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 1698682, 2024-04-24T06:54:54.100 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-04-24T06:54:54.100 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-24T06:54:54.100 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-24T06:54:54.101 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-24T06:54:54.101 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:54.101 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:54.101 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:54:54.876 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-24T06:54:54.877 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-24T06:54:54.877 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-24T06:54:54.877 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-24T06:54:54.877 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-24T06:54:54.877 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-04-24T06:54:54.877 DEBUG:tasks.check_counter:Getting stats from g 2024-04-24T06:54:54.878 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-24T06:54:55.368 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:54:55.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:55 smithi137 ceph-mon[33017]: pgmap v337: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:54:55.732 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:55 smithi005 ceph-mon[27798]: pgmap v337: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:54:55.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:55 smithi037 ceph-mon[32864]: pgmap v337: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:54:57.666 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:57 smithi005 ceph-mon[27798]: pgmap v338: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:54:57.672 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:54:57.672 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T06:54:57.672 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 2036, 2024-04-24T06:54:57.672 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 2079, 2024-04-24T06:54:57.672 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 717888, 2024-04-24T06:54:57.673 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 1457500, 2024-04-24T06:54:57.673 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 12, 2024-04-24T06:54:57.673 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 5, 2024-04-24T06:54:57.673 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.369168039, 2024-04-24T06:54:57.673 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.162222572, 2024-04-24T06:54:57.673 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.172019985, 2024-04-24T06:54:57.673 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.037175311, 2024-04-24T06:54:57.673 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T06:54:57.674 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 2077, 2024-04-24T06:54:57.674 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.197812656, 2024-04-24T06:54:57.674 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000095239 2024-04-24T06:54:57.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.674 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-04-24T06:54:57.674 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 2189, 2024-04-24T06:54:57.674 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000975212, 2024-04-24T06:54:57.674 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000445 2024-04-24T06:54:57.675 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.675 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 3488, 2024-04-24T06:54:57.675 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 11056 2024-04-24T06:54:57.675 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.675 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T06:54:57.675 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 1393, 2024-04-24T06:54:57.675 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 1418, 2024-04-24T06:54:57.675 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 2068853, 2024-04-24T06:54:57.676 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 942352, 2024-04-24T06:54:57.676 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 42, 2024-04-24T06:54:57.676 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 9, 2024-04-24T06:54:57.676 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.258576091, 2024-04-24T06:54:57.676 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.082877398, 2024-04-24T06:54:57.676 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.152065834, 2024-04-24T06:54:57.676 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.039906198, 2024-04-24T06:54:57.676 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T06:54:57.676 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 1418, 2024-04-24T06:54:57.677 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.123179042, 2024-04-24T06:54:57.677 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000086868 2024-04-24T06:54:57.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.677 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-04-24T06:54:57.677 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 196, 2024-04-24T06:54:57.677 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000068725, 2024-04-24T06:54:57.677 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000350 2024-04-24T06:54:57.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.678 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 1791760, 2024-04-24T06:54:57.678 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 157984 2024-04-24T06:54:57.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.678 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T06:54:57.678 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 1642, 2024-04-24T06:54:57.678 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 2091, 2024-04-24T06:54:57.678 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 462014, 2024-04-24T06:54:57.678 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 1321185, 2024-04-24T06:54:57.679 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 18, 2024-04-24T06:54:57.679 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 5, 2024-04-24T06:54:57.679 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.325533221, 2024-04-24T06:54:57.679 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.180166396, 2024-04-24T06:54:57.679 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.112570701, 2024-04-24T06:54:57.679 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.033165022, 2024-04-24T06:54:57.679 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T06:54:57.679 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 2088, 2024-04-24T06:54:57.680 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.199897703, 2024-04-24T06:54:57.680 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000095736 2024-04-24T06:54:57.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.680 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-04-24T06:54:57.680 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 1078, 2024-04-24T06:54:57.680 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000344057, 2024-04-24T06:54:57.681 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000319 2024-04-24T06:54:57.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.681 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 2176, 2024-04-24T06:54:57.681 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 463840 2024-04-24T06:54:57.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.681 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2024-04-24T06:54:57.681 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 1, 2024-04-24T06:54:57.681 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2024-04-24T06:54:57.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.682 INFO:teuthology.orchestra.run.smithi005.stdout: "finisher-MDSRank": { 2024-04-24T06:54:57.682 INFO:teuthology.orchestra.run.smithi005.stdout: "queue_len": 0, 2024-04-24T06:54:57.682 INFO:teuthology.orchestra.run.smithi005.stdout: "complete_latency": { 2024-04-24T06:54:57.682 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 1830, 2024-04-24T06:54:57.682 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.905204220, 2024-04-24T06:54:57.682 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000494647 2024-04-24T06:54:57.682 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:57.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.683 INFO:teuthology.orchestra.run.smithi005.stdout: "finisher-PurgeQueue": { 2024-04-24T06:54:57.683 INFO:teuthology.orchestra.run.smithi005.stdout: "queue_len": 0, 2024-04-24T06:54:57.683 INFO:teuthology.orchestra.run.smithi005.stdout: "complete_latency": { 2024-04-24T06:54:57.683 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 15, 2024-04-24T06:54:57.683 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000915362, 2024-04-24T06:54:57.683 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000061024 2024-04-24T06:54:57.683 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:57.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.684 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": { 2024-04-24T06:54:57.684 INFO:teuthology.orchestra.run.smithi005.stdout: "request": 224, 2024-04-24T06:54:57.684 INFO:teuthology.orchestra.run.smithi005.stdout: "reply": 217, 2024-04-24T06:54:57.684 INFO:teuthology.orchestra.run.smithi005.stdout: "reply_latency": { 2024-04-24T06:54:57.684 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 217, 2024-04-24T06:54:57.684 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.274642154, 2024-04-24T06:54:57.684 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.001265632 2024-04-24T06:54:57.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.685 INFO:teuthology.orchestra.run.smithi005.stdout: "slow_reply": 0, 2024-04-24T06:54:57.685 INFO:teuthology.orchestra.run.smithi005.stdout: "forward": 7, 2024-04-24T06:54:57.685 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_fetch_complete": 0, 2024-04-24T06:54:57.685 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_fetch_keys": 0, 2024-04-24T06:54:57.685 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_commit": 11, 2024-04-24T06:54:57.685 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_split": 0, 2024-04-24T06:54:57.685 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_merge": 0, 2024-04-24T06:54:57.686 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes": 275, 2024-04-24T06:54:57.686 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_top": 11, 2024-04-24T06:54:57.686 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_bottom": 264, 2024-04-24T06:54:57.686 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_pin_tail": 0, 2024-04-24T06:54:57.686 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_pinned": 246, 2024-04-24T06:54:57.686 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_expired": 1, 2024-04-24T06:54:57.686 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_with_caps": 143, 2024-04-24T06:54:57.686 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": 144, 2024-04-24T06:54:57.687 INFO:teuthology.orchestra.run.smithi005.stdout: "subtrees": 5, 2024-04-24T06:54:57.687 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse": 535, 2024-04-24T06:54:57.687 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_hit": 258, 2024-04-24T06:54:57.687 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_forward": 4, 2024-04-24T06:54:57.687 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_discover": 12, 2024-04-24T06:54:57.687 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_dir_fetch": 0, 2024-04-24T06:54:57.687 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_remote_ino": 0, 2024-04-24T06:54:57.687 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_lock": 3, 2024-04-24T06:54:57.688 INFO:teuthology.orchestra.run.smithi005.stdout: "load_cent": 0, 2024-04-24T06:54:57.688 INFO:teuthology.orchestra.run.smithi005.stdout: "q": 0, 2024-04-24T06:54:57.688 INFO:teuthology.orchestra.run.smithi005.stdout: "exported": 0, 2024-04-24T06:54:57.688 INFO:teuthology.orchestra.run.smithi005.stdout: "exported_inodes": 0, 2024-04-24T06:54:57.688 INFO:teuthology.orchestra.run.smithi005.stdout: "imported": 10, 2024-04-24T06:54:57.688 INFO:teuthology.orchestra.run.smithi005.stdout: "imported_inodes": 169, 2024-04-24T06:54:57.688 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_dir_fetch": 0, 2024-04-24T06:54:57.688 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_backtrace_fetch": 2926, 2024-04-24T06:54:57.688 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_peer_discover": 0, 2024-04-24T06:54:57.689 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rfiles": 3454, 2024-04-24T06:54:57.689 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rbytes": 26624153, 2024-04-24T06:54:57.689 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rsnaps": 6, 2024-04-24T06:54:57.689 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_backtrace_fetch": 2926, 2024-04-24T06:54:57.689 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_set_tag": 0, 2024-04-24T06:54:57.689 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T06:54:57.689 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_inotable_repaired": 0, 2024-04-24T06:54:57.689 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dir_inodes": 224, 2024-04-24T06:54:57.690 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T06:54:57.690 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dirfrag_rstats": 224, 2024-04-24T06:54:57.690 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_file_inodes": 2702, 2024-04-24T06:54:57.690 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_inode_file_caps": 0, 2024-04-24T06:54:57.690 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_revoke": 7, 2024-04-24T06:54:57.690 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_grant": 88, 2024-04-24T06:54:57.690 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_trunc": 2, 2024-04-24T06:54:57.690 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T06:54:57.691 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T06:54:57.691 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_caps": 257, 2024-04-24T06:54:57.691 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_caps_dirty": 200, 2024-04-24T06:54:57.691 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_cap_release": 2, 2024-04-24T06:54:57.691 INFO:teuthology.orchestra.run.smithi005.stdout: "process_request_cap_release": 77 2024-04-24T06:54:57.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.691 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cache": { 2024-04-24T06:54:57.691 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_update": 0, 2024-04-24T06:54:57.692 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_update_receipt": 0, 2024-04-24T06:54:57.692 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_try_discover": 0, 2024-04-24T06:54:57.692 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_send_discover": 14, 2024-04-24T06:54:57.692 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_handle_discover": 0, 2024-04-24T06:54:57.692 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays": 0, 2024-04-24T06:54:57.692 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays_delayed": 0, 2024-04-24T06:54:57.692 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays_enqueuing": 0, 2024-04-24T06:54:57.692 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_created": 2, 2024-04-24T06:54:57.693 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_enqueued": 2, 2024-04-24T06:54:57.693 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_reintegrated": 0, 2024-04-24T06:54:57.693 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_migrated": 0, 2024-04-24T06:54:57.693 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_processing": 0, 2024-04-24T06:54:57.693 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_enqueued": 0, 2024-04-24T06:54:57.693 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_prioritized": 0, 2024-04-24T06:54:57.693 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_started": 0, 2024-04-24T06:54:57.693 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_completed": 0, 2024-04-24T06:54:57.693 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_quiesce_path": 0, 2024-04-24T06:54:57.694 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_quiesce_inode": 0, 2024-04-24T06:54:57.694 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T06:54:57.694 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_exportdir": 0, 2024-04-24T06:54:57.694 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_flush": 0, 2024-04-24T06:54:57.694 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_fragmentdir": 0, 2024-04-24T06:54:57.694 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_fragstats": 0, 2024-04-24T06:54:57.694 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_inodestats": 0 2024-04-24T06:54:57.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.695 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log": { 2024-04-24T06:54:57.695 INFO:teuthology.orchestra.run.smithi005.stdout: "evlrg": 0, 2024-04-24T06:54:57.695 INFO:teuthology.orchestra.run.smithi005.stdout: "evadd": 384, 2024-04-24T06:54:57.695 INFO:teuthology.orchestra.run.smithi005.stdout: "evex": 0, 2024-04-24T06:54:57.695 INFO:teuthology.orchestra.run.smithi005.stdout: "evtrm": 0, 2024-04-24T06:54:57.695 INFO:teuthology.orchestra.run.smithi005.stdout: "ev": 384, 2024-04-24T06:54:57.695 INFO:teuthology.orchestra.run.smithi005.stdout: "evexg": 0, 2024-04-24T06:54:57.695 INFO:teuthology.orchestra.run.smithi005.stdout: "evexd": 0, 2024-04-24T06:54:57.695 INFO:teuthology.orchestra.run.smithi005.stdout: "segadd": 2, 2024-04-24T06:54:57.696 INFO:teuthology.orchestra.run.smithi005.stdout: "segex": 0, 2024-04-24T06:54:57.696 INFO:teuthology.orchestra.run.smithi005.stdout: "segtrm": 0, 2024-04-24T06:54:57.696 INFO:teuthology.orchestra.run.smithi005.stdout: "seg": 2, 2024-04-24T06:54:57.696 INFO:teuthology.orchestra.run.smithi005.stdout: "segmjr": 2, 2024-04-24T06:54:57.696 INFO:teuthology.orchestra.run.smithi005.stdout: "segexg": 0, 2024-04-24T06:54:57.696 INFO:teuthology.orchestra.run.smithi005.stdout: "segexd": 0, 2024-04-24T06:54:57.696 INFO:teuthology.orchestra.run.smithi005.stdout: "expos": 4194304, 2024-04-24T06:54:57.696 INFO:teuthology.orchestra.run.smithi005.stdout: "wrpos": 5030217, 2024-04-24T06:54:57.697 INFO:teuthology.orchestra.run.smithi005.stdout: "rdpos": 0, 2024-04-24T06:54:57.697 INFO:teuthology.orchestra.run.smithi005.stdout: "jlat": { 2024-04-24T06:54:57.697 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 76, 2024-04-24T06:54:57.697 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.205890138, 2024-04-24T06:54:57.697 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.002709080 2024-04-24T06:54:57.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.697 INFO:teuthology.orchestra.run.smithi005.stdout: "replayed": 0 2024-04-24T06:54:57.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.698 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_mem": { 2024-04-24T06:54:57.698 INFO:teuthology.orchestra.run.smithi005.stdout: "ino": 234, 2024-04-24T06:54:57.698 INFO:teuthology.orchestra.run.smithi005.stdout: "ino+": 236, 2024-04-24T06:54:57.698 INFO:teuthology.orchestra.run.smithi005.stdout: "ino-": 2, 2024-04-24T06:54:57.698 INFO:teuthology.orchestra.run.smithi005.stdout: "dir": 36, 2024-04-24T06:54:57.698 INFO:teuthology.orchestra.run.smithi005.stdout: "dir+": 37, 2024-04-24T06:54:57.698 INFO:teuthology.orchestra.run.smithi005.stdout: "dir-": 1, 2024-04-24T06:54:57.698 INFO:teuthology.orchestra.run.smithi005.stdout: "dn": 275, 2024-04-24T06:54:57.699 INFO:teuthology.orchestra.run.smithi005.stdout: "dn+": 292, 2024-04-24T06:54:57.699 INFO:teuthology.orchestra.run.smithi005.stdout: "dn-": 17, 2024-04-24T06:54:57.699 INFO:teuthology.orchestra.run.smithi005.stdout: "cap": 144, 2024-04-24T06:54:57.699 INFO:teuthology.orchestra.run.smithi005.stdout: "cap+": 207, 2024-04-24T06:54:57.699 INFO:teuthology.orchestra.run.smithi005.stdout: "cap-": 63, 2024-04-24T06:54:57.699 INFO:teuthology.orchestra.run.smithi005.stdout: "rss": 55756, 2024-04-24T06:54:57.699 INFO:teuthology.orchestra.run.smithi005.stdout: "heap": 223608 2024-04-24T06:54:57.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.700 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_server": { 2024-04-24T06:54:57.700 INFO:teuthology.orchestra.run.smithi005.stdout: "dispatch_client_request": 345, 2024-04-24T06:54:57.700 INFO:teuthology.orchestra.run.smithi005.stdout: "dispatch_server_request": 0, 2024-04-24T06:54:57.700 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_request": 224, 2024-04-24T06:54:57.700 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_session": 25, 2024-04-24T06:54:57.700 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_peer_request": 42, 2024-04-24T06:54:57.700 INFO:teuthology.orchestra.run.smithi005.stdout: "req_create_latency": { 2024-04-24T06:54:57.700 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 44, 2024-04-24T06:54:57.700 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.117903937, 2024-04-24T06:54:57.701 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.002679634 2024-04-24T06:54:57.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.701 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getattr_latency": { 2024-04-24T06:54:57.701 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 5, 2024-04-24T06:54:57.701 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.001575763, 2024-04-24T06:54:57.701 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000315152 2024-04-24T06:54:57.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.701 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getfilelock_latency": { 2024-04-24T06:54:57.702 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-24T06:54:57.702 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-24T06:54:57.702 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-24T06:54:57.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.702 INFO:teuthology.orchestra.run.smithi005.stdout: "req_link_latency": { 2024-04-24T06:54:57.702 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-24T06:54:57.702 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-24T06:54:57.702 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-24T06:54:57.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.703 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookup_latency": { 2024-04-24T06:54:57.703 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 132, 2024-04-24T06:54:57.703 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.065798848, 2024-04-24T06:54:57.703 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000498476 2024-04-24T06:54:57.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.703 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookuphash_latency": { 2024-04-24T06:54:57.704 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-24T06:54:57.704 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-24T06:54:57.704 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-24T06:54:57.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.704 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupino_latency": { 2024-04-24T06:54:57.704 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 1, 2024-04-24T06:54:57.705 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000195398, 2024-04-24T06:54:57.705 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000195398 2024-04-24T06:54:57.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.705 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupname_latency": { 2024-04-24T06:54:57.705 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-24T06:54:57.705 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-24T06:54:57.705 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-24T06:54:57.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.706 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupparent_latency": { 2024-04-24T06:54:57.706 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-24T06:54:57.706 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-24T06:54:57.706 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-24T06:54:57.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.706 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupsnap_latency": { 2024-04-24T06:54:57.706 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-24T06:54:57.706 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-24T06:54:57.707 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-24T06:54:57.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.707 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lssnap_latency": { 2024-04-24T06:54:57.707 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-24T06:54:57.707 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-24T06:54:57.708 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:57 smithi137 ceph-mon[33017]: pgmap v338: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:54:57.708 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-24T06:54:57.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.709 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mkdir_latency": { 2024-04-24T06:54:57.709 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 1, 2024-04-24T06:54:57.709 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.002522746, 2024-04-24T06:54:57.709 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.002522746 2024-04-24T06:54:57.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.709 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mknod_latency": { 2024-04-24T06:54:57.709 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-24T06:54:57.710 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-24T06:54:57.710 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-24T06:54:57.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.710 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mksnap_latency": { 2024-04-24T06:54:57.710 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-24T06:54:57.710 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-24T06:54:57.710 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-24T06:54:57.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.711 INFO:teuthology.orchestra.run.smithi005.stdout: "req_open_latency": { 2024-04-24T06:54:57.711 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 12, 2024-04-24T06:54:57.711 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.006430448, 2024-04-24T06:54:57.711 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000535870 2024-04-24T06:54:57.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.711 INFO:teuthology.orchestra.run.smithi005.stdout: "req_readdir_latency": { 2024-04-24T06:54:57.711 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 4, 2024-04-24T06:54:57.711 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.004746386, 2024-04-24T06:54:57.712 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.001186596 2024-04-24T06:54:57.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.712 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rename_latency": { 2024-04-24T06:54:57.712 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 13, 2024-04-24T06:54:57.712 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.052498021, 2024-04-24T06:54:57.712 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.004038309 2024-04-24T06:54:57.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.712 INFO:teuthology.orchestra.run.smithi005.stdout: "req_renamesnap_latency": { 2024-04-24T06:54:57.713 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-24T06:54:57.713 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-24T06:54:57.713 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-24T06:54:57.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.713 INFO:teuthology.orchestra.run.smithi005.stdout: "req_snapdiff_latency": { 2024-04-24T06:54:57.713 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-24T06:54:57.713 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-24T06:54:57.713 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-24T06:54:57.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.714 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmdir_latency": { 2024-04-24T06:54:57.714 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-24T06:54:57.714 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-24T06:54:57.714 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-24T06:54:57.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.714 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmsnap_latency": { 2024-04-24T06:54:57.714 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-24T06:54:57.715 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-24T06:54:57.715 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-24T06:54:57.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.715 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmxattr_latency": { 2024-04-24T06:54:57.715 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-24T06:54:57.715 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-24T06:54:57.715 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-24T06:54:57.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.716 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setattr_latency": { 2024-04-24T06:54:57.716 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 2, 2024-04-24T06:54:57.716 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.016444572, 2024-04-24T06:54:57.716 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.008222286 2024-04-24T06:54:57.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.716 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setdirlayout_latency": { 2024-04-24T06:54:57.716 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-24T06:54:57.716 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-24T06:54:57.717 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-24T06:54:57.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.717 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setfilelock_latency": { 2024-04-24T06:54:57.717 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-24T06:54:57.717 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-24T06:54:57.717 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-24T06:54:57.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.717 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setlayout_latency": { 2024-04-24T06:54:57.718 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-24T06:54:57.718 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-24T06:54:57.718 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-24T06:54:57.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.718 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setxattr_latency": { 2024-04-24T06:54:57.718 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-24T06:54:57.718 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-24T06:54:57.718 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-24T06:54:57.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.719 INFO:teuthology.orchestra.run.smithi005.stdout: "req_symlink_latency": { 2024-04-24T06:54:57.719 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 1, 2024-04-24T06:54:57.719 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.001777757, 2024-04-24T06:54:57.719 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.001777757 2024-04-24T06:54:57.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.719 INFO:teuthology.orchestra.run.smithi005.stdout: "req_unlink_latency": { 2024-04-24T06:54:57.719 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 2, 2024-04-24T06:54:57.720 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.004748278, 2024-04-24T06:54:57.720 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.002374139 2024-04-24T06:54:57.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.720 INFO:teuthology.orchestra.run.smithi005.stdout: "cap_revoke_eviction": 0, 2024-04-24T06:54:57.720 INFO:teuthology.orchestra.run.smithi005.stdout: "cap_acquisition_throttle": 0, 2024-04-24T06:54:57.720 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getvxattr_latency": { 2024-04-24T06:54:57.720 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-24T06:54:57.720 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-24T06:54:57.721 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-24T06:54:57.721 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:57.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.721 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_sessions": { 2024-04-24T06:54:57.721 INFO:teuthology.orchestra.run.smithi005.stdout: "session_count": 2, 2024-04-24T06:54:57.721 INFO:teuthology.orchestra.run.smithi005.stdout: "session_add": 4, 2024-04-24T06:54:57.721 INFO:teuthology.orchestra.run.smithi005.stdout: "session_remove": 2, 2024-04-24T06:54:57.721 INFO:teuthology.orchestra.run.smithi005.stdout: "sessions_open": 2, 2024-04-24T06:54:57.722 INFO:teuthology.orchestra.run.smithi005.stdout: "sessions_stale": 0, 2024-04-24T06:54:57.722 INFO:teuthology.orchestra.run.smithi005.stdout: "total_load": 201, 2024-04-24T06:54:57.722 INFO:teuthology.orchestra.run.smithi005.stdout: "average_load": 100, 2024-04-24T06:54:57.722 INFO:teuthology.orchestra.run.smithi005.stdout: "avg_session_uptime": 618, 2024-04-24T06:54:57.723 INFO:teuthology.orchestra.run.smithi005.stdout: "mdthresh_evicted": 0 2024-04-24T06:54:57.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.723 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2024-04-24T06:54:57.723 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2024-04-24T06:54:57.723 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2024-04-24T06:54:57.723 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T06:54:57.723 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2024-04-24T06:54:57.723 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T06:54:57.724 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2024-04-24T06:54:57.724 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T06:54:57.724 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T06:54:57.724 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T06:54:57.724 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T06:54:57.724 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T06:54:57.724 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2024-04-24T06:54:57.724 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T06:54:57.725 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T06:54:57.725 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2024-04-24T06:54:57.725 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2024-04-24T06:54:57.725 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2024-04-24T06:54:57.725 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2024-04-24T06:54:57.725 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T06:54:57.725 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T06:54:57.725 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T06:54:57.725 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T06:54:57.726 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T06:54:57.726 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2024-04-24T06:54:57.726 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2024-04-24T06:54:57.726 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2024-04-24T06:54:57.726 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T06:54:57.726 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T06:54:57.726 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2024-04-24T06:54:57.726 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2024-04-24T06:54:57.727 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2024-04-24T06:54:57.727 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2024-04-24T06:54:57.727 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T06:54:57.727 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2024-04-24T06:54:57.727 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T06:54:57.727 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2024-04-24T06:54:57.727 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 308860, 2024-04-24T06:54:57.727 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 353, 2024-04-24T06:54:57.728 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 1672, 2024-04-24T06:54:57.728 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 19, 2024-04-24T06:54:57.728 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2024-04-24T06:54:57.728 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2024-04-24T06:54:57.728 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2024-04-24T06:54:57.728 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2024-04-24T06:54:57.728 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2024-04-24T06:54:57.728 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2024-04-24T06:54:57.729 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 8128, 2024-04-24T06:54:57.729 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 169, 2024-04-24T06:54:57.729 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T06:54:57.729 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2024-04-24T06:54:57.729 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2024-04-24T06:54:57.729 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2024-04-24T06:54:57.729 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 858146, 2024-04-24T06:54:57.729 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 9389, 2024-04-24T06:54:57.730 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2024-04-24T06:54:57.730 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2024-04-24T06:54:57.730 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2024-04-24T06:54:57.730 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2024-04-24T06:54:57.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.731 INFO:teuthology.orchestra.run.smithi005.stdout: "objecter": { 2024-04-24T06:54:57.731 INFO:teuthology.orchestra.run.smithi005.stdout: "op_active": 0, 2024-04-24T06:54:57.731 INFO:teuthology.orchestra.run.smithi005.stdout: "op_laggy": 0, 2024-04-24T06:54:57.731 INFO:teuthology.orchestra.run.smithi005.stdout: "op_send": 3074, 2024-04-24T06:54:57.731 INFO:teuthology.orchestra.run.smithi005.stdout: "op_send_bytes": 882833, 2024-04-24T06:54:57.731 INFO:teuthology.orchestra.run.smithi005.stdout: "op_resend": 29, 2024-04-24T06:54:57.731 INFO:teuthology.orchestra.run.smithi005.stdout: "op_reply": 3074, 2024-04-24T06:54:57.732 INFO:teuthology.orchestra.run.smithi005.stdout: "op_latency": { 2024-04-24T06:54:57.732 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 3074, 2024-04-24T06:54:57.732 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 2.356952907, 2024-04-24T06:54:57.732 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000766738 2024-04-24T06:54:57.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.732 INFO:teuthology.orchestra.run.smithi005.stdout: "op_inflight": 0, 2024-04-24T06:54:57.732 INFO:teuthology.orchestra.run.smithi005.stdout: "oplen_avg": { 2024-04-24T06:54:57.732 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 3074, 2024-04-24T06:54:57.733 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 3112 2024-04-24T06:54:57.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.733 INFO:teuthology.orchestra.run.smithi005.stdout: "op": 3074, 2024-04-24T06:54:57.733 INFO:teuthology.orchestra.run.smithi005.stdout: "op_r": 2927, 2024-04-24T06:54:57.733 INFO:teuthology.orchestra.run.smithi005.stdout: "op_w": 147, 2024-04-24T06:54:57.733 INFO:teuthology.orchestra.run.smithi005.stdout: "op_rmw": 0, 2024-04-24T06:54:57.733 INFO:teuthology.orchestra.run.smithi005.stdout: "op_pg": 0, 2024-04-24T06:54:57.733 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_stat": 0, 2024-04-24T06:54:57.734 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_create": 10, 2024-04-24T06:54:57.734 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_read": 1, 2024-04-24T06:54:57.734 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_write": 77, 2024-04-24T06:54:57.734 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_writefull": 15, 2024-04-24T06:54:57.734 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_writesame": 0, 2024-04-24T06:54:57.734 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_append": 0, 2024-04-24T06:54:57.734 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_zero": 0, 2024-04-24T06:54:57.734 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_truncate": 0, 2024-04-24T06:54:57.735 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_delete": 14, 2024-04-24T06:54:57.735 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_mapext": 0, 2024-04-24T06:54:57.735 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_sparse_read": 0, 2024-04-24T06:54:57.735 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_clonerange": 0, 2024-04-24T06:54:57.735 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_getxattr": 2926, 2024-04-24T06:54:57.735 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_setxattr": 20, 2024-04-24T06:54:57.735 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_cmpxattr": 0, 2024-04-24T06:54:57.735 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_rmxattr": 0, 2024-04-24T06:54:57.736 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_resetxattrs": 0, 2024-04-24T06:54:57.736 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_call": 0, 2024-04-24T06:54:57.736 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_watch": 0, 2024-04-24T06:54:57.736 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_notify": 0, 2024-04-24T06:54:57.736 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T06:54:57.736 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_pgls": 0, 2024-04-24T06:54:57.736 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_pgls_filter": 0, 2024-04-24T06:54:57.736 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_other": 2, 2024-04-24T06:54:57.737 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_active": 0, 2024-04-24T06:54:57.737 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_send": 0, 2024-04-24T06:54:57.737 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_resend": 0, 2024-04-24T06:54:57.737 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_ping": 0, 2024-04-24T06:54:57.737 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_active": 0, 2024-04-24T06:54:57.737 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_send": 0, 2024-04-24T06:54:57.737 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_resend": 0, 2024-04-24T06:54:57.737 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_active": 0, 2024-04-24T06:54:57.737 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_send": 0, 2024-04-24T06:54:57.738 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_resend": 0, 2024-04-24T06:54:57.738 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_active": 0, 2024-04-24T06:54:57.738 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_send": 0, 2024-04-24T06:54:57.738 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_resend": 0, 2024-04-24T06:54:57.738 INFO:teuthology.orchestra.run.smithi005.stdout: "command_active": 0, 2024-04-24T06:54:57.738 INFO:teuthology.orchestra.run.smithi005.stdout: "command_send": 0, 2024-04-24T06:54:57.738 INFO:teuthology.orchestra.run.smithi005.stdout: "command_resend": 0, 2024-04-24T06:54:57.739 INFO:teuthology.orchestra.run.smithi005.stdout: "map_epoch": 97, 2024-04-24T06:54:57.739 INFO:teuthology.orchestra.run.smithi005.stdout: "map_full": 0, 2024-04-24T06:54:57.739 INFO:teuthology.orchestra.run.smithi005.stdout: "map_inc": 11, 2024-04-24T06:54:57.739 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_sessions": 12, 2024-04-24T06:54:57.739 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_session_open": 12, 2024-04-24T06:54:57.739 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_session_close": 0, 2024-04-24T06:54:57.739 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_laggy": 0, 2024-04-24T06:54:57.739 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_wr": 40, 2024-04-24T06:54:57.740 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_rd": 0, 2024-04-24T06:54:57.740 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_del": 7 2024-04-24T06:54:57.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.740 INFO:teuthology.orchestra.run.smithi005.stdout: "oft": { 2024-04-24T06:54:57.740 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_objs": 1, 2024-04-24T06:54:57.740 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_kv_pairs": 0, 2024-04-24T06:54:57.740 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_updates": 0, 2024-04-24T06:54:57.740 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_removes": 0 2024-04-24T06:54:57.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.741 INFO:teuthology.orchestra.run.smithi005.stdout: "osd-slow-ops": { 2024-04-24T06:54:57.741 INFO:teuthology.orchestra.run.smithi005.stdout: "slow_ops_count": 0 2024-04-24T06:54:57.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.741 INFO:teuthology.orchestra.run.smithi005.stdout: "purge_queue": { 2024-04-24T06:54:57.741 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_ops": 0, 2024-04-24T06:54:57.741 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_ops_high_water": 2, 2024-04-24T06:54:57.741 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing": 0, 2024-04-24T06:54:57.741 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_high_water": 2, 2024-04-24T06:54:57.742 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executed_ops": 2, 2024-04-24T06:54:57.742 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executed": 2, 2024-04-24T06:54:57.742 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_item_in_journal": 0 2024-04-24T06:54:57.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.742 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T06:54:57.742 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-04-24T06:54:57.742 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-04-24T06:54:57.743 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-04-24T06:54:57.743 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 5067, 2024-04-24T06:54:57.743 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 2772990, 2024-04-24T06:54:57.743 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-04-24T06:54:57.743 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 5067, 2024-04-24T06:54:57.743 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-04-24T06:54:57.743 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-04-24T06:54:57.743 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 5067, 2024-04-24T06:54:57.743 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 2772990, 2024-04-24T06:54:57.744 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-04-24T06:54:57.744 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-24T06:54:57.744 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-24T06:54:57.744 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-24T06:54:57.744 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:57.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.744 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-objecter_bytes": { 2024-04-24T06:54:57.744 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-04-24T06:54:57.745 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-04-24T06:54:57.745 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-04-24T06:54:57.745 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-04-24T06:54:57.745 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-04-24T06:54:57.745 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-04-24T06:54:57.745 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-04-24T06:54:57.745 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 3074, 2024-04-24T06:54:57.745 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 883035, 2024-04-24T06:54:57.746 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 3074, 2024-04-24T06:54:57.746 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 883035, 2024-04-24T06:54:57.746 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-04-24T06:54:57.746 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-24T06:54:57.746 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-24T06:54:57.746 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-24T06:54:57.746 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:57.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.747 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-objecter_ops": { 2024-04-24T06:54:57.747 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-04-24T06:54:57.747 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 1024, 2024-04-24T06:54:57.747 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-04-24T06:54:57.747 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-04-24T06:54:57.747 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-04-24T06:54:57.747 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-04-24T06:54:57.747 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-04-24T06:54:57.747 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 3074, 2024-04-24T06:54:57.748 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 3074, 2024-04-24T06:54:57.748 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 3074, 2024-04-24T06:54:57.748 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 3074, 2024-04-24T06:54:57.748 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-04-24T06:54:57.748 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-24T06:54:57.748 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-24T06:54:57.748 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-24T06:54:57.748 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:57.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.749 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-write_buf_throttle": { 2024-04-24T06:54:57.749 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-04-24T06:54:57.749 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 3758096384, 2024-04-24T06:54:57.749 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-04-24T06:54:57.749 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 2, 2024-04-24T06:54:57.749 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 202, 2024-04-24T06:54:57.749 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-04-24T06:54:57.750 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 2, 2024-04-24T06:54:57.750 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-04-24T06:54:57.750 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-04-24T06:54:57.750 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 1, 2024-04-24T06:54:57.750 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 202, 2024-04-24T06:54:57.750 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-04-24T06:54:57.750 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-24T06:54:57.750 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-24T06:54:57.751 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-24T06:54:57.751 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:57.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:54:57.751 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-write_buf_throttle-0x557b17916460": { 2024-04-24T06:54:57.751 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-04-24T06:54:57.751 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 3758096384, 2024-04-24T06:54:57.751 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-04-24T06:54:57.751 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 384, 2024-04-24T06:54:57.752 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 835913, 2024-04-24T06:54:57.752 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-04-24T06:54:57.752 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 384, 2024-04-24T06:54:57.752 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-04-24T06:54:57.752 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-04-24T06:54:57.752 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 76, 2024-04-24T06:54:57.752 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 835913, 2024-04-24T06:54:57.752 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-04-24T06:54:57.753 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-24T06:54:57.753 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-24T06:54:57.753 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-24T06:54:57.753 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:57.753 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:54:57.753 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:54:57.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:57 smithi037 ceph-mon[32864]: pgmap v338: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:54:58.535 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-04-24T06:54:58.535 INFO:tasks.check_counter:Daemon mds.g mds.imported=10 2024-04-24T06:54:58.535 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=6 2024-04-24T06:54:58.535 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-04-24T06:54:58.535 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-04-24T06:54:58.535 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=0 2024-04-24T06:54:58.536 DEBUG:tasks.check_counter:Getting stats from j 2024-04-24T06:54:58.536 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-24T06:54:59.015 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T06:54:59.468 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:54:59 smithi005 ceph-mon[27798]: pgmap v339: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:54:59.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:54:59 smithi137 ceph-mon[33017]: pgmap v339: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:54:59.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:54:59 smithi037 ceph-mon[32864]: pgmap v339: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-24T06:55:01.189 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-24T06:55:01.189 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T06:55:01.189 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 0, 2024-04-24T06:55:01.189 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 2, 2024-04-24T06:55:01.189 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 0, 2024-04-24T06:55:01.189 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 0, 2024-04-24T06:55:01.190 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 4, 2024-04-24T06:55:01.190 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 0, 2024-04-24T06:55:01.190 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.004332398, 2024-04-24T06:55:01.190 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.000000000, 2024-04-24T06:55:01.190 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.000995603, 2024-04-24T06:55:01.190 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T06:55:01.190 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T06:55:01.190 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-24T06:55:01.191 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-24T06:55:01.191 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-24T06:55:01.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:55:01.191 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-04-24T06:55:01.191 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-24T06:55:01.191 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-24T06:55:01.191 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-24T06:55:01.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:55:01.192 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T06:55:01.192 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T06:55:01.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:55:01.192 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T06:55:01.192 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 237, 2024-04-24T06:55:01.192 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 225, 2024-04-24T06:55:01.192 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 1732496, 2024-04-24T06:55:01.192 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 138784, 2024-04-24T06:55:01.193 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 2, 2024-04-24T06:55:01.193 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 1, 2024-04-24T06:55:01.193 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.103747738, 2024-04-24T06:55:01.193 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.024918947, 2024-04-24T06:55:01.193 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.099903401, 2024-04-24T06:55:01.193 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.010874043, 2024-04-24T06:55:01.193 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T06:55:01.193 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 224, 2024-04-24T06:55:01.194 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.021862284, 2024-04-24T06:55:01.194 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000097599 2024-04-24T06:55:01.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:55:01.194 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-04-24T06:55:01.194 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-24T06:55:01.194 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-24T06:55:01.194 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-24T06:55:01.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:55:01.195 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 1732496, 2024-04-24T06:55:01.195 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 138784 2024-04-24T06:55:01.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:55:01.195 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T06:55:01.195 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 6, 2024-04-24T06:55:01.195 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 174, 2024-04-24T06:55:01.195 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 3344, 2024-04-24T06:55:01.195 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 186992, 2024-04-24T06:55:01.196 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 5, 2024-04-24T06:55:01.196 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 3, 2024-04-24T06:55:01.196 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.037642216, 2024-04-24T06:55:01.196 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.027363618, 2024-04-24T06:55:01.196 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.005016651, 2024-04-24T06:55:01.196 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T06:55:01.196 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T06:55:01.197 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 172, 2024-04-24T06:55:01.197 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.037964724, 2024-04-24T06:55:01.197 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000220725 2024-04-24T06:55:01.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:55:01.197 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-04-24T06:55:01.197 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-24T06:55:01.197 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-24T06:55:01.197 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-24T06:55:01.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:55:01.198 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-04-24T06:55:01.198 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 186992 2024-04-24T06:55:01.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:55:01.198 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2024-04-24T06:55:01.198 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 0, 2024-04-24T06:55:01.198 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2024-04-24T06:55:01.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:55:01.198 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2024-04-24T06:55:01.199 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2024-04-24T06:55:01.199 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2024-04-24T06:55:01.199 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T06:55:01.199 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2024-04-24T06:55:01.199 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T06:55:01.199 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2024-04-24T06:55:01.199 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T06:55:01.199 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T06:55:01.199 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T06:55:01.200 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T06:55:01.200 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T06:55:01.200 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2024-04-24T06:55:01.200 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T06:55:01.200 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T06:55:01.200 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2024-04-24T06:55:01.200 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2024-04-24T06:55:01.200 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2024-04-24T06:55:01.200 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2024-04-24T06:55:01.201 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T06:55:01.201 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T06:55:01.201 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T06:55:01.201 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T06:55:01.201 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T06:55:01.201 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2024-04-24T06:55:01.201 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2024-04-24T06:55:01.201 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2024-04-24T06:55:01.201 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T06:55:01.202 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T06:55:01.202 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2024-04-24T06:55:01.202 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2024-04-24T06:55:01.202 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2024-04-24T06:55:01.202 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2024-04-24T06:55:01.202 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T06:55:01.202 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2024-04-24T06:55:01.203 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T06:55:01.203 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2024-04-24T06:55:01.203 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 148714, 2024-04-24T06:55:01.203 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 30, 2024-04-24T06:55:01.203 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2024-04-24T06:55:01.203 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2024-04-24T06:55:01.203 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2024-04-24T06:55:01.203 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2024-04-24T06:55:01.203 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2024-04-24T06:55:01.204 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2024-04-24T06:55:01.204 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2024-04-24T06:55:01.204 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2024-04-24T06:55:01.204 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 0, 2024-04-24T06:55:01.204 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 0, 2024-04-24T06:55:01.204 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T06:55:01.204 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2024-04-24T06:55:01.204 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2024-04-24T06:55:01.205 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2024-04-24T06:55:01.205 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 928, 2024-04-24T06:55:01.205 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 2, 2024-04-24T06:55:01.205 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2024-04-24T06:55:01.205 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2024-04-24T06:55:01.205 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2024-04-24T06:55:01.206 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2024-04-24T06:55:01.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-24T06:55:01.206 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T06:55:01.206 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-04-24T06:55:01.206 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-04-24T06:55:01.206 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-04-24T06:55:01.206 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 239, 2024-04-24T06:55:01.206 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 1699072, 2024-04-24T06:55:01.206 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-04-24T06:55:01.207 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 239, 2024-04-24T06:55:01.207 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-04-24T06:55:01.207 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-04-24T06:55:01.207 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 239, 2024-04-24T06:55:01.207 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 1699072, 2024-04-24T06:55:01.207 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-04-24T06:55:01.207 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-04-24T06:55:01.207 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-04-24T06:55:01.208 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-04-24T06:55:01.208 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:55:01.208 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-24T06:55:01.208 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-24T06:55:01.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:01 smithi137 ceph-mon[33017]: pgmap v340: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:55:01.716 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:01 smithi005 ceph-mon[27798]: pgmap v340: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:55:01.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:01 smithi037 ceph-mon[32864]: pgmap v340: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:55:02.118 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-24T06:55:02.118 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-24T06:55:02.119 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-24T06:55:02.119 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-24T06:55:02.119 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-24T06:55:02.119 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-04-24T06:55:02.119 DEBUG:tasks.check_counter:Getting stats from b 2024-04-24T06:55:02.119 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-24T06:55:02.384 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T06:55:03.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:03 smithi137 ceph-mon[33017]: pgmap v341: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:55:03.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:03 smithi005 ceph-mon[27798]: pgmap v341: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:55:03.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:03 smithi037 ceph-mon[32864]: pgmap v341: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-24T06:55:04.354 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-24T06:55:04.354 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T06:55:04.354 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 16631, 2024-04-24T06:55:04.354 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 16604, 2024-04-24T06:55:04.355 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 4263564, 2024-04-24T06:55:04.355 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 32876408, 2024-04-24T06:55:04.355 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 157, 2024-04-24T06:55:04.355 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 8, 2024-04-24T06:55:04.355 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 1.077036271, 2024-04-24T06:55:04.355 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.379533324, 2024-04-24T06:55:04.356 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.277743258, 2024-04-24T06:55:04.356 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.334230822, 2024-04-24T06:55:04.356 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T06:55:04.356 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 16598, 2024-04-24T06:55:04.356 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.625561380, 2024-04-24T06:55:04.356 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000037688 2024-04-24T06:55:04.357 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.357 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-24T06:55:04.357 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 435, 2024-04-24T06:55:04.357 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000089076, 2024-04-24T06:55:04.357 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000204 2024-04-24T06:55:04.357 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.357 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 100896, 2024-04-24T06:55:04.358 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 362000 2024-04-24T06:55:04.358 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.358 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T06:55:04.358 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 20223, 2024-04-24T06:55:04.358 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 20106, 2024-04-24T06:55:04.359 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 7333386, 2024-04-24T06:55:04.359 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 20906180, 2024-04-24T06:55:04.359 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 8, 2024-04-24T06:55:04.359 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 5, 2024-04-24T06:55:04.359 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 1.134390113, 2024-04-24T06:55:04.359 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.349060192, 2024-04-24T06:55:04.360 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.351390946, 2024-04-24T06:55:04.360 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.382541212, 2024-04-24T06:55:04.360 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T06:55:04.360 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 20108, 2024-04-24T06:55:04.360 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.751325668, 2024-04-24T06:55:04.360 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000037364 2024-04-24T06:55:04.360 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.361 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-24T06:55:04.361 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 2193, 2024-04-24T06:55:04.361 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000511903, 2024-04-24T06:55:04.361 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000233 2024-04-24T06:55:04.361 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.361 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 1844416, 2024-04-24T06:55:04.361 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 171328 2024-04-24T06:55:04.362 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.362 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T06:55:04.362 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 37096, 2024-04-24T06:55:04.362 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 52118, 2024-04-24T06:55:04.362 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 11862074, 2024-04-24T06:55:04.362 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 31623812, 2024-04-24T06:55:04.362 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 23, 2024-04-24T06:55:04.363 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 7, 2024-04-24T06:55:04.363 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 2.293985693, 2024-04-24T06:55:04.363 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.872471914, 2024-04-24T06:55:04.363 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 1.531335877, 2024-04-24T06:55:04.363 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.353905416, 2024-04-24T06:55:04.363 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T06:55:04.363 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 52117, 2024-04-24T06:55:04.363 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 1.184839146, 2024-04-24T06:55:04.364 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000022734 2024-04-24T06:55:04.364 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.364 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-24T06:55:04.364 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 52430, 2024-04-24T06:55:04.364 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.008965199, 2024-04-24T06:55:04.364 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000170 2024-04-24T06:55:04.364 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.364 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 323008, 2024-04-24T06:55:04.365 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 313360 2024-04-24T06:55:04.365 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.365 INFO:teuthology.orchestra.run.smithi037.stdout: "cct": { 2024-04-24T06:55:04.365 INFO:teuthology.orchestra.run.smithi037.stdout: "total_workers": 1, 2024-04-24T06:55:04.365 INFO:teuthology.orchestra.run.smithi037.stdout: "unhealthy_workers": 0 2024-04-24T06:55:04.365 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.365 INFO:teuthology.orchestra.run.smithi037.stdout: "finisher-MDSRank": { 2024-04-24T06:55:04.365 INFO:teuthology.orchestra.run.smithi037.stdout: "queue_len": 0, 2024-04-24T06:55:04.366 INFO:teuthology.orchestra.run.smithi037.stdout: "complete_latency": { 2024-04-24T06:55:04.366 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 30988, 2024-04-24T06:55:04.366 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 18.421496705, 2024-04-24T06:55:04.366 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000594471 2024-04-24T06:55:04.366 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-24T06:55:04.366 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.366 INFO:teuthology.orchestra.run.smithi037.stdout: "finisher-PurgeQueue": { 2024-04-24T06:55:04.367 INFO:teuthology.orchestra.run.smithi037.stdout: "queue_len": 0, 2024-04-24T06:55:04.367 INFO:teuthology.orchestra.run.smithi037.stdout: "complete_latency": { 2024-04-24T06:55:04.367 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 4712, 2024-04-24T06:55:04.367 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 1.144460852, 2024-04-24T06:55:04.367 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000242882 2024-04-24T06:55:04.367 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-24T06:55:04.367 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.367 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-04-24T06:55:04.368 INFO:teuthology.orchestra.run.smithi037.stdout: "request": 10156, 2024-04-24T06:55:04.368 INFO:teuthology.orchestra.run.smithi037.stdout: "reply": 10104, 2024-04-24T06:55:04.368 INFO:teuthology.orchestra.run.smithi037.stdout: "reply_latency": { 2024-04-24T06:55:04.368 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 10104, 2024-04-24T06:55:04.368 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 25.580752896, 2024-04-24T06:55:04.368 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.002531745 2024-04-24T06:55:04.369 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.369 INFO:teuthology.orchestra.run.smithi037.stdout: "slow_reply": 0, 2024-04-24T06:55:04.369 INFO:teuthology.orchestra.run.smithi037.stdout: "forward": 52, 2024-04-24T06:55:04.369 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_fetch_complete": 0, 2024-04-24T06:55:04.369 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_fetch_keys": 0, 2024-04-24T06:55:04.369 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_commit": 220, 2024-04-24T06:55:04.370 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_split": 25, 2024-04-24T06:55:04.370 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_merge": 2, 2024-04-24T06:55:04.370 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes": 5270, 2024-04-24T06:55:04.370 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_top": 503, 2024-04-24T06:55:04.370 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_bottom": 4712, 2024-04-24T06:55:04.370 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_pin_tail": 55, 2024-04-24T06:55:04.371 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_pinned": 4551, 2024-04-24T06:55:04.371 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_expired": 833, 2024-04-24T06:55:04.371 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_with_caps": 2914, 2024-04-24T06:55:04.371 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": 2914, 2024-04-24T06:55:04.372 INFO:teuthology.orchestra.run.smithi037.stdout: "subtrees": 69, 2024-04-24T06:55:04.372 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse": 10432, 2024-04-24T06:55:04.372 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_hit": 9837, 2024-04-24T06:55:04.373 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_forward": 52, 2024-04-24T06:55:04.373 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_discover": 0, 2024-04-24T06:55:04.373 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_dir_fetch": 0, 2024-04-24T06:55:04.373 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_remote_ino": 0, 2024-04-24T06:55:04.374 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_lock": 3, 2024-04-24T06:55:04.374 INFO:teuthology.orchestra.run.smithi037.stdout: "load_cent": 0, 2024-04-24T06:55:04.374 INFO:teuthology.orchestra.run.smithi037.stdout: "q": 0, 2024-04-24T06:55:04.374 INFO:teuthology.orchestra.run.smithi037.stdout: "exported": 20, 2024-04-24T06:55:04.375 INFO:teuthology.orchestra.run.smithi037.stdout: "exported_inodes": 816, 2024-04-24T06:55:04.375 INFO:teuthology.orchestra.run.smithi037.stdout: "imported": 0, 2024-04-24T06:55:04.375 INFO:teuthology.orchestra.run.smithi037.stdout: "imported_inodes": 0, 2024-04-24T06:55:04.375 INFO:teuthology.orchestra.run.smithi037.stdout: "openino_dir_fetch": 0, 2024-04-24T06:55:04.376 INFO:teuthology.orchestra.run.smithi037.stdout: "openino_backtrace_fetch": 43832, 2024-04-24T06:55:04.376 INFO:teuthology.orchestra.run.smithi037.stdout: "openino_peer_discover": 0, 2024-04-24T06:55:04.376 INFO:teuthology.orchestra.run.smithi037.stdout: "root_rfiles": 3454, 2024-04-24T06:55:04.376 INFO:teuthology.orchestra.run.smithi037.stdout: "root_rbytes": 26624153, 2024-04-24T06:55:04.377 INFO:teuthology.orchestra.run.smithi037.stdout: "root_rsnaps": 6, 2024-04-24T06:55:04.377 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_backtrace_fetch": 43832, 2024-04-24T06:55:04.377 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_set_tag": 0, 2024-04-24T06:55:04.378 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T06:55:04.378 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_inotable_repaired": 0, 2024-04-24T06:55:04.378 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_dir_inodes": 898, 2024-04-24T06:55:04.378 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_dir_base_inodes": 19, 2024-04-24T06:55:04.379 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_dirfrag_rstats": 879, 2024-04-24T06:55:04.379 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_file_inodes": 42934, 2024-04-24T06:55:04.379 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_inode_file_caps": 2, 2024-04-24T06:55:04.379 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_cap_op_revoke": 216, 2024-04-24T06:55:04.380 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_cap_op_grant": 10427, 2024-04-24T06:55:04.380 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_cap_op_trunc": 6, 2024-04-24T06:55:04.380 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T06:55:04.380 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T06:55:04.380 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_client_caps": 9222, 2024-04-24T06:55:04.381 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_client_caps_dirty": 6230, 2024-04-24T06:55:04.381 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_client_cap_release": 1076, 2024-04-24T06:55:04.381 INFO:teuthology.orchestra.run.smithi037.stdout: "process_request_cap_release": 3072 2024-04-24T06:55:04.382 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.382 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_cache": { 2024-04-24T06:55:04.382 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_update": 0, 2024-04-24T06:55:04.382 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_update_receipt": 0, 2024-04-24T06:55:04.383 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_try_discover": 0, 2024-04-24T06:55:04.383 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_send_discover": 0, 2024-04-24T06:55:04.383 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_handle_discover": 19, 2024-04-24T06:55:04.383 INFO:teuthology.orchestra.run.smithi037.stdout: "num_strays": 2, 2024-04-24T06:55:04.384 INFO:teuthology.orchestra.run.smithi037.stdout: "num_strays_delayed": 0, 2024-04-24T06:55:04.384 INFO:teuthology.orchestra.run.smithi037.stdout: "num_strays_enqueuing": 0, 2024-04-24T06:55:04.384 INFO:teuthology.orchestra.run.smithi037.stdout: "strays_created": 2794, 2024-04-24T06:55:04.385 INFO:teuthology.orchestra.run.smithi037.stdout: "strays_enqueued": 2789, 2024-04-24T06:55:04.385 INFO:teuthology.orchestra.run.smithi037.stdout: "strays_reintegrated": 3, 2024-04-24T06:55:04.385 INFO:teuthology.orchestra.run.smithi037.stdout: "strays_migrated": 0, 2024-04-24T06:55:04.385 INFO:teuthology.orchestra.run.smithi037.stdout: "num_recovering_processing": 0, 2024-04-24T06:55:04.386 INFO:teuthology.orchestra.run.smithi037.stdout: "num_recovering_enqueued": 0, 2024-04-24T06:55:04.386 INFO:teuthology.orchestra.run.smithi037.stdout: "num_recovering_prioritized": 0, 2024-04-24T06:55:04.386 INFO:teuthology.orchestra.run.smithi037.stdout: "recovery_started": 0, 2024-04-24T06:55:04.386 INFO:teuthology.orchestra.run.smithi037.stdout: "recovery_completed": 0, 2024-04-24T06:55:04.387 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_quiesce_path": 0, 2024-04-24T06:55:04.387 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_quiesce_inode": 0, 2024-04-24T06:55:04.387 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_enqueue_scrub": 19, 2024-04-24T06:55:04.387 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_exportdir": 20, 2024-04-24T06:55:04.388 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_flush": 0, 2024-04-24T06:55:04.388 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_fragmentdir": 27, 2024-04-24T06:55:04.388 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_fragstats": 0, 2024-04-24T06:55:04.389 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_inodestats": 0 2024-04-24T06:55:04.389 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.389 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_log": { 2024-04-24T06:55:04.389 INFO:teuthology.orchestra.run.smithi037.stdout: "evlrg": 0, 2024-04-24T06:55:04.390 INFO:teuthology.orchestra.run.smithi037.stdout: "evadd": 21410, 2024-04-24T06:55:04.390 INFO:teuthology.orchestra.run.smithi037.stdout: "evex": 0, 2024-04-24T06:55:04.390 INFO:teuthology.orchestra.run.smithi037.stdout: "evtrm": 0, 2024-04-24T06:55:04.390 INFO:teuthology.orchestra.run.smithi037.stdout: "ev": 21410, 2024-04-24T06:55:04.391 INFO:teuthology.orchestra.run.smithi037.stdout: "evexg": 0, 2024-04-24T06:55:04.391 INFO:teuthology.orchestra.run.smithi037.stdout: "evexd": 0, 2024-04-24T06:55:04.391 INFO:teuthology.orchestra.run.smithi037.stdout: "segadd": 30, 2024-04-24T06:55:04.391 INFO:teuthology.orchestra.run.smithi037.stdout: "segex": 0, 2024-04-24T06:55:04.392 INFO:teuthology.orchestra.run.smithi037.stdout: "segtrm": 0, 2024-04-24T06:55:04.392 INFO:teuthology.orchestra.run.smithi037.stdout: "seg": 30, 2024-04-24T06:55:04.392 INFO:teuthology.orchestra.run.smithi037.stdout: "segmjr": 3, 2024-04-24T06:55:04.393 INFO:teuthology.orchestra.run.smithi037.stdout: "segexg": 0, 2024-04-24T06:55:04.393 INFO:teuthology.orchestra.run.smithi037.stdout: "segexd": 0, 2024-04-24T06:55:04.393 INFO:teuthology.orchestra.run.smithi037.stdout: "expos": 4194304, 2024-04-24T06:55:04.393 INFO:teuthology.orchestra.run.smithi037.stdout: "wrpos": 58152204, 2024-04-24T06:55:04.394 INFO:teuthology.orchestra.run.smithi037.stdout: "rdpos": 0, 2024-04-24T06:55:04.394 INFO:teuthology.orchestra.run.smithi037.stdout: "jlat": { 2024-04-24T06:55:04.394 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 475, 2024-04-24T06:55:04.394 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 14.884058919, 2024-04-24T06:55:04.395 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.031334860 2024-04-24T06:55:04.395 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.395 INFO:teuthology.orchestra.run.smithi037.stdout: "replayed": 0 2024-04-24T06:55:04.396 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.396 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_mem": { 2024-04-24T06:55:04.396 INFO:teuthology.orchestra.run.smithi037.stdout: "ino": 3632, 2024-04-24T06:55:04.396 INFO:teuthology.orchestra.run.smithi037.stdout: "ino+": 7228, 2024-04-24T06:55:04.397 INFO:teuthology.orchestra.run.smithi037.stdout: "ino-": 3596, 2024-04-24T06:55:04.397 INFO:teuthology.orchestra.run.smithi037.stdout: "dir": 298, 2024-04-24T06:55:04.397 INFO:teuthology.orchestra.run.smithi037.stdout: "dir+": 362, 2024-04-24T06:55:04.397 INFO:teuthology.orchestra.run.smithi037.stdout: "dir-": 64, 2024-04-24T06:55:04.398 INFO:teuthology.orchestra.run.smithi037.stdout: "dn": 5270, 2024-04-24T06:55:04.398 INFO:teuthology.orchestra.run.smithi037.stdout: "dn+": 8600, 2024-04-24T06:55:04.398 INFO:teuthology.orchestra.run.smithi037.stdout: "dn-": 3330, 2024-04-24T06:55:04.399 INFO:teuthology.orchestra.run.smithi037.stdout: "cap": 2914, 2024-04-24T06:55:04.399 INFO:teuthology.orchestra.run.smithi037.stdout: "cap+": 6517, 2024-04-24T06:55:04.399 INFO:teuthology.orchestra.run.smithi037.stdout: "cap-": 3603, 2024-04-24T06:55:04.399 INFO:teuthology.orchestra.run.smithi037.stdout: "rss": 104280, 2024-04-24T06:55:04.400 INFO:teuthology.orchestra.run.smithi037.stdout: "heap": 231800 2024-04-24T06:55:04.400 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.400 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_server": { 2024-04-24T06:55:04.400 INFO:teuthology.orchestra.run.smithi037.stdout: "dispatch_client_request": 10439, 2024-04-24T06:55:04.401 INFO:teuthology.orchestra.run.smithi037.stdout: "dispatch_server_request": 718, 2024-04-24T06:55:04.401 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_client_request": 10156, 2024-04-24T06:55:04.401 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_client_session": 54, 2024-04-24T06:55:04.401 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_peer_request": 1436, 2024-04-24T06:55:04.402 INFO:teuthology.orchestra.run.smithi037.stdout: "req_create_latency": { 2024-04-24T06:55:04.402 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 6421, 2024-04-24T06:55:04.402 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 14.966191304, 2024-04-24T06:55:04.402 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.002330819 2024-04-24T06:55:04.402 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.403 INFO:teuthology.orchestra.run.smithi037.stdout: "req_getattr_latency": { 2024-04-24T06:55:04.403 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 163, 2024-04-24T06:55:04.403 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.334912770, 2024-04-24T06:55:04.403 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.002054679 2024-04-24T06:55:04.403 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.404 INFO:teuthology.orchestra.run.smithi037.stdout: "req_getfilelock_latency": { 2024-04-24T06:55:04.404 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:04.404 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:04.404 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:04.404 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.405 INFO:teuthology.orchestra.run.smithi037.stdout: "req_link_latency": { 2024-04-24T06:55:04.405 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 3, 2024-04-24T06:55:04.405 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.005956454, 2024-04-24T06:55:04.405 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.001985484 2024-04-24T06:55:04.405 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.406 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookup_latency": { 2024-04-24T06:55:04.406 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 18, 2024-04-24T06:55:04.406 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.879144206, 2024-04-24T06:55:04.406 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.048841344 2024-04-24T06:55:04.406 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.406 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookuphash_latency": { 2024-04-24T06:55:04.406 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:04.407 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:04.407 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:04.407 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.407 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookupino_latency": { 2024-04-24T06:55:04.407 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 1, 2024-04-24T06:55:04.407 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000379904, 2024-04-24T06:55:04.407 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000379904 2024-04-24T06:55:04.408 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.408 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookupname_latency": { 2024-04-24T06:55:04.408 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:04.408 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:04.408 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:04.408 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.408 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookupparent_latency": { 2024-04-24T06:55:04.408 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:04.409 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:04.409 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:04.409 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.409 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookupsnap_latency": { 2024-04-24T06:55:04.409 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:04.409 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:04.409 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:04.410 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.411 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lssnap_latency": { 2024-04-24T06:55:04.411 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 11, 2024-04-24T06:55:04.411 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.003256951, 2024-04-24T06:55:04.411 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000296086 2024-04-24T06:55:04.411 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.411 INFO:teuthology.orchestra.run.smithi037.stdout: "req_mkdir_latency": { 2024-04-24T06:55:04.412 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 84, 2024-04-24T06:55:04.412 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.077398761, 2024-04-24T06:55:04.412 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000921413 2024-04-24T06:55:04.412 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.412 INFO:teuthology.orchestra.run.smithi037.stdout: "req_mknod_latency": { 2024-04-24T06:55:04.412 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:04.412 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:04.412 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:04.413 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.413 INFO:teuthology.orchestra.run.smithi037.stdout: "req_mksnap_latency": { 2024-04-24T06:55:04.413 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 11, 2024-04-24T06:55:04.413 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.442572481, 2024-04-24T06:55:04.413 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.040233861 2024-04-24T06:55:04.413 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.413 INFO:teuthology.orchestra.run.smithi037.stdout: "req_open_latency": { 2024-04-24T06:55:04.413 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 428, 2024-04-24T06:55:04.414 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.172559939, 2024-04-24T06:55:04.414 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000403177 2024-04-24T06:55:04.414 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.414 INFO:teuthology.orchestra.run.smithi037.stdout: "req_readdir_latency": { 2024-04-24T06:55:04.414 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 104, 2024-04-24T06:55:04.414 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 1.450597875, 2024-04-24T06:55:04.414 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.013948056 2024-04-24T06:55:04.414 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.415 INFO:teuthology.orchestra.run.smithi037.stdout: "req_rename_latency": { 2024-04-24T06:55:04.415 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 69, 2024-04-24T06:55:04.415 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.149288919, 2024-04-24T06:55:04.415 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.002163607 2024-04-24T06:55:04.415 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.415 INFO:teuthology.orchestra.run.smithi037.stdout: "req_renamesnap_latency": { 2024-04-24T06:55:04.415 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:04.416 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:04.416 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:04.416 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.416 INFO:teuthology.orchestra.run.smithi037.stdout: "req_snapdiff_latency": { 2024-04-24T06:55:04.416 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:04.416 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:04.416 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:04.416 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.417 INFO:teuthology.orchestra.run.smithi037.stdout: "req_rmdir_latency": { 2024-04-24T06:55:04.417 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 22, 2024-04-24T06:55:04.417 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.566962106, 2024-04-24T06:55:04.417 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.025771004 2024-04-24T06:55:04.417 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.417 INFO:teuthology.orchestra.run.smithi037.stdout: "req_rmsnap_latency": { 2024-04-24T06:55:04.417 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 5, 2024-04-24T06:55:04.417 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.039213187, 2024-04-24T06:55:04.418 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.007842637 2024-04-24T06:55:04.418 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.418 INFO:teuthology.orchestra.run.smithi037.stdout: "req_rmxattr_latency": { 2024-04-24T06:55:04.418 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:04.418 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:04.418 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:04.418 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.419 INFO:teuthology.orchestra.run.smithi037.stdout: "req_setattr_latency": { 2024-04-24T06:55:04.419 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 7, 2024-04-24T06:55:04.419 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.160668645, 2024-04-24T06:55:04.419 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.022952663 2024-04-24T06:55:04.419 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.419 INFO:teuthology.orchestra.run.smithi037.stdout: "req_setdirlayout_latency": { 2024-04-24T06:55:04.419 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:04.419 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:04.420 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:04.420 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.420 INFO:teuthology.orchestra.run.smithi037.stdout: "req_setfilelock_latency": { 2024-04-24T06:55:04.420 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 8, 2024-04-24T06:55:04.420 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.003793157, 2024-04-24T06:55:04.420 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000474144 2024-04-24T06:55:04.420 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.420 INFO:teuthology.orchestra.run.smithi037.stdout: "req_setlayout_latency": { 2024-04-24T06:55:04.421 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:04.421 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:04.421 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:04.421 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.421 INFO:teuthology.orchestra.run.smithi037.stdout: "req_setxattr_latency": { 2024-04-24T06:55:04.421 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 8, 2024-04-24T06:55:04.422 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.020968551, 2024-04-24T06:55:04.422 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.002621068 2024-04-24T06:55:04.422 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.422 INFO:teuthology.orchestra.run.smithi037.stdout: "req_symlink_latency": { 2024-04-24T06:55:04.422 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 6, 2024-04-24T06:55:04.422 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.005237650, 2024-04-24T06:55:04.422 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000872941 2024-04-24T06:55:04.422 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.423 INFO:teuthology.orchestra.run.smithi037.stdout: "req_unlink_latency": { 2024-04-24T06:55:04.423 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 2735, 2024-04-24T06:55:04.423 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 6.301650036, 2024-04-24T06:55:04.423 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.002304076 2024-04-24T06:55:04.423 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.423 INFO:teuthology.orchestra.run.smithi037.stdout: "cap_revoke_eviction": 0, 2024-04-24T06:55:04.423 INFO:teuthology.orchestra.run.smithi037.stdout: "cap_acquisition_throttle": 0, 2024-04-24T06:55:04.424 INFO:teuthology.orchestra.run.smithi037.stdout: "req_getvxattr_latency": { 2024-04-24T06:55:04.424 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:04.424 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:04.424 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:04.424 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-24T06:55:04.424 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.424 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_sessions": { 2024-04-24T06:55:04.424 INFO:teuthology.orchestra.run.smithi037.stdout: "session_count": 2, 2024-04-24T06:55:04.425 INFO:teuthology.orchestra.run.smithi037.stdout: "session_add": 5, 2024-04-24T06:55:04.425 INFO:teuthology.orchestra.run.smithi037.stdout: "session_remove": 3, 2024-04-24T06:55:04.425 INFO:teuthology.orchestra.run.smithi037.stdout: "sessions_open": 2, 2024-04-24T06:55:04.425 INFO:teuthology.orchestra.run.smithi037.stdout: "sessions_stale": 0, 2024-04-24T06:55:04.425 INFO:teuthology.orchestra.run.smithi037.stdout: "total_load": 50, 2024-04-24T06:55:04.425 INFO:teuthology.orchestra.run.smithi037.stdout: "average_load": 25, 2024-04-24T06:55:04.425 INFO:teuthology.orchestra.run.smithi037.stdout: "avg_session_uptime": 632, 2024-04-24T06:55:04.425 INFO:teuthology.orchestra.run.smithi037.stdout: "mdthresh_evicted": 0 2024-04-24T06:55:04.425 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.426 INFO:teuthology.orchestra.run.smithi037.stdout: "mempool": { 2024-04-24T06:55:04.426 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_bytes": 0, 2024-04-24T06:55:04.426 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_items": 0, 2024-04-24T06:55:04.426 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T06:55:04.426 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_items": 0, 2024-04-24T06:55:04.426 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T06:55:04.426 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_items": 0, 2024-04-24T06:55:04.427 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T06:55:04.427 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T06:55:04.427 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T06:55:04.427 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T06:55:04.427 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T06:55:04.427 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_items": 0, 2024-04-24T06:55:04.427 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T06:55:04.427 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T06:55:04.428 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_bytes": 0, 2024-04-24T06:55:04.428 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_items": 0, 2024-04-24T06:55:04.428 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_bytes": 0, 2024-04-24T06:55:04.428 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_items": 0, 2024-04-24T06:55:04.428 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T06:55:04.428 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T06:55:04.428 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T06:55:04.428 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T06:55:04.429 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T06:55:04.429 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_items": 0, 2024-04-24T06:55:04.429 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_bytes": 0, 2024-04-24T06:55:04.429 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_items": 0, 2024-04-24T06:55:04.429 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T06:55:04.429 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T06:55:04.429 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_bytes": 0, 2024-04-24T06:55:04.429 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_items": 0, 2024-04-24T06:55:04.430 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_bytes": 0, 2024-04-24T06:55:04.430 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_items": 0, 2024-04-24T06:55:04.430 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T06:55:04.430 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_items": 0, 2024-04-24T06:55:04.430 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T06:55:04.430 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_items": 0, 2024-04-24T06:55:04.430 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_bytes": 448092, 2024-04-24T06:55:04.430 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_items": 3402, 2024-04-24T06:55:04.431 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_bytes": 1496, 2024-04-24T06:55:04.431 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_items": 17, 2024-04-24T06:55:04.431 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_bytes": 0, 2024-04-24T06:55:04.431 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_items": 0, 2024-04-24T06:55:04.431 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_bytes": 0, 2024-04-24T06:55:04.431 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_items": 0, 2024-04-24T06:55:04.431 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_bytes": 0, 2024-04-24T06:55:04.431 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_items": 0, 2024-04-24T06:55:04.432 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_bytes": 8128, 2024-04-24T06:55:04.432 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_items": 169, 2024-04-24T06:55:04.432 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T06:55:04.432 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_items": 0, 2024-04-24T06:55:04.432 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_bytes": 0, 2024-04-24T06:55:04.432 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_items": 0, 2024-04-24T06:55:04.432 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_bytes": 13825261, 2024-04-24T06:55:04.433 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_items": 151629, 2024-04-24T06:55:04.433 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_bytes": 0, 2024-04-24T06:55:04.433 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_items": 0, 2024-04-24T06:55:04.433 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_bytes": 0, 2024-04-24T06:55:04.433 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_items": 0 2024-04-24T06:55:04.433 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.433 INFO:teuthology.orchestra.run.smithi037.stdout: "objecter": { 2024-04-24T06:55:04.433 INFO:teuthology.orchestra.run.smithi037.stdout: "op_active": 0, 2024-04-24T06:55:04.434 INFO:teuthology.orchestra.run.smithi037.stdout: "op_laggy": 0, 2024-04-24T06:55:04.434 INFO:teuthology.orchestra.run.smithi037.stdout: "op_send": 48117, 2024-04-24T06:55:04.434 INFO:teuthology.orchestra.run.smithi037.stdout: "op_send_bytes": 57961387, 2024-04-24T06:55:04.434 INFO:teuthology.orchestra.run.smithi037.stdout: "op_resend": 37, 2024-04-24T06:55:04.434 INFO:teuthology.orchestra.run.smithi037.stdout: "op_reply": 48117, 2024-04-24T06:55:04.434 INFO:teuthology.orchestra.run.smithi037.stdout: "op_latency": { 2024-04-24T06:55:04.435 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 48117, 2024-04-24T06:55:04.435 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 55.307263900, 2024-04-24T06:55:04.435 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.001149432 2024-04-24T06:55:04.435 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.435 INFO:teuthology.orchestra.run.smithi037.stdout: "op_inflight": 0, 2024-04-24T06:55:04.435 INFO:teuthology.orchestra.run.smithi037.stdout: "oplen_avg": { 2024-04-24T06:55:04.436 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 48117, 2024-04-24T06:55:04.436 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 48436 2024-04-24T06:55:04.436 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.436 INFO:teuthology.orchestra.run.smithi037.stdout: "op": 48117, 2024-04-24T06:55:04.436 INFO:teuthology.orchestra.run.smithi037.stdout: "op_r": 43901, 2024-04-24T06:55:04.436 INFO:teuthology.orchestra.run.smithi037.stdout: "op_w": 4216, 2024-04-24T06:55:04.436 INFO:teuthology.orchestra.run.smithi037.stdout: "op_rmw": 0, 2024-04-24T06:55:04.436 INFO:teuthology.orchestra.run.smithi037.stdout: "op_pg": 0, 2024-04-24T06:55:04.437 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_stat": 0, 2024-04-24T06:55:04.437 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_create": 11, 2024-04-24T06:55:04.437 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_read": 50, 2024-04-24T06:55:04.437 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_write": 506, 2024-04-24T06:55:04.437 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_writefull": 45, 2024-04-24T06:55:04.437 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_writesame": 0, 2024-04-24T06:55:04.437 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_append": 0, 2024-04-24T06:55:04.438 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_zero": 0, 2024-04-24T06:55:04.438 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_truncate": 9, 2024-04-24T06:55:04.438 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_delete": 3345, 2024-04-24T06:55:04.438 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_mapext": 0, 2024-04-24T06:55:04.438 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_sparse_read": 0, 2024-04-24T06:55:04.438 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_clonerange": 0, 2024-04-24T06:55:04.438 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_getxattr": 43851, 2024-04-24T06:55:04.439 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_setxattr": 22, 2024-04-24T06:55:04.439 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_cmpxattr": 0, 2024-04-24T06:55:04.439 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_rmxattr": 0, 2024-04-24T06:55:04.439 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_resetxattrs": 0, 2024-04-24T06:55:04.439 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_call": 0, 2024-04-24T06:55:04.439 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_watch": 0, 2024-04-24T06:55:04.439 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_notify": 0, 2024-04-24T06:55:04.440 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T06:55:04.440 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_pgls": 0, 2024-04-24T06:55:04.440 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_pgls_filter": 0, 2024-04-24T06:55:04.440 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_other": 6, 2024-04-24T06:55:04.440 INFO:teuthology.orchestra.run.smithi037.stdout: "linger_active": 0, 2024-04-24T06:55:04.440 INFO:teuthology.orchestra.run.smithi037.stdout: "linger_send": 0, 2024-04-24T06:55:04.441 INFO:teuthology.orchestra.run.smithi037.stdout: "linger_resend": 0, 2024-04-24T06:55:04.441 INFO:teuthology.orchestra.run.smithi037.stdout: "linger_ping": 0, 2024-04-24T06:55:04.441 INFO:teuthology.orchestra.run.smithi037.stdout: "poolop_active": 0, 2024-04-24T06:55:04.441 INFO:teuthology.orchestra.run.smithi037.stdout: "poolop_send": 0, 2024-04-24T06:55:04.441 INFO:teuthology.orchestra.run.smithi037.stdout: "poolop_resend": 0, 2024-04-24T06:55:04.441 INFO:teuthology.orchestra.run.smithi037.stdout: "poolstat_active": 0, 2024-04-24T06:55:04.441 INFO:teuthology.orchestra.run.smithi037.stdout: "poolstat_send": 0, 2024-04-24T06:55:04.442 INFO:teuthology.orchestra.run.smithi037.stdout: "poolstat_resend": 0, 2024-04-24T06:55:04.442 INFO:teuthology.orchestra.run.smithi037.stdout: "statfs_active": 0, 2024-04-24T06:55:04.442 INFO:teuthology.orchestra.run.smithi037.stdout: "statfs_send": 0, 2024-04-24T06:55:04.442 INFO:teuthology.orchestra.run.smithi037.stdout: "statfs_resend": 0, 2024-04-24T06:55:04.442 INFO:teuthology.orchestra.run.smithi037.stdout: "command_active": 0, 2024-04-24T06:55:04.443 INFO:teuthology.orchestra.run.smithi037.stdout: "command_send": 0, 2024-04-24T06:55:04.443 INFO:teuthology.orchestra.run.smithi037.stdout: "command_resend": 0, 2024-04-24T06:55:04.443 INFO:teuthology.orchestra.run.smithi037.stdout: "map_epoch": 97, 2024-04-24T06:55:04.443 INFO:teuthology.orchestra.run.smithi037.stdout: "map_full": 0, 2024-04-24T06:55:04.443 INFO:teuthology.orchestra.run.smithi037.stdout: "map_inc": 12, 2024-04-24T06:55:04.443 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_sessions": 12, 2024-04-24T06:55:04.443 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_session_open": 12, 2024-04-24T06:55:04.444 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_session_close": 0, 2024-04-24T06:55:04.444 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_laggy": 0, 2024-04-24T06:55:04.444 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_wr": 556, 2024-04-24T06:55:04.444 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_rd": 0, 2024-04-24T06:55:04.444 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_del": 35 2024-04-24T06:55:04.444 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.444 INFO:teuthology.orchestra.run.smithi037.stdout: "oft": { 2024-04-24T06:55:04.444 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_total_objs": 1, 2024-04-24T06:55:04.445 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_total_kv_pairs": 0, 2024-04-24T06:55:04.445 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_total_updates": 0, 2024-04-24T06:55:04.445 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_total_removes": 0 2024-04-24T06:55:04.445 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.445 INFO:teuthology.orchestra.run.smithi037.stdout: "osd-slow-ops": { 2024-04-24T06:55:04.445 INFO:teuthology.orchestra.run.smithi037.stdout: "slow_ops_count": 0 2024-04-24T06:55:04.445 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.446 INFO:teuthology.orchestra.run.smithi037.stdout: "purge_queue": { 2024-04-24T06:55:04.446 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executing_ops": 0, 2024-04-24T06:55:04.446 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executing_ops_high_water": 10, 2024-04-24T06:55:04.446 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executing": 0, 2024-04-24T06:55:04.446 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executing_high_water": 10, 2024-04-24T06:55:04.446 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executed_ops": 2789, 2024-04-24T06:55:04.446 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executed": 2789, 2024-04-24T06:55:04.447 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_item_in_journal": 0 2024-04-24T06:55:04.447 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.447 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T06:55:04.447 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-04-24T06:55:04.447 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-04-24T06:55:04.447 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-04-24T06:55:04.447 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 73946, 2024-04-24T06:55:04.447 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 16735259, 2024-04-24T06:55:04.448 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-04-24T06:55:04.448 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 73946, 2024-04-24T06:55:04.448 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-04-24T06:55:04.448 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-04-24T06:55:04.448 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 73946, 2024-04-24T06:55:04.448 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 16735259, 2024-04-24T06:55:04.448 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-04-24T06:55:04.449 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:04.449 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:04.449 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:04.449 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-24T06:55:04.449 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.449 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-objecter_bytes": { 2024-04-24T06:55:04.449 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-04-24T06:55:04.450 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-04-24T06:55:04.450 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-04-24T06:55:04.450 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 0, 2024-04-24T06:55:04.450 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 0, 2024-04-24T06:55:04.450 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-04-24T06:55:04.450 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 0, 2024-04-24T06:55:04.451 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 48117, 2024-04-24T06:55:04.451 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 58245740, 2024-04-24T06:55:04.451 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 48117, 2024-04-24T06:55:04.451 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 58245740, 2024-04-24T06:55:04.451 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-04-24T06:55:04.451 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:04.451 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:04.451 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:04.452 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-24T06:55:04.452 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.452 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-objecter_ops": { 2024-04-24T06:55:04.452 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-04-24T06:55:04.452 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 1024, 2024-04-24T06:55:04.452 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-04-24T06:55:04.452 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 0, 2024-04-24T06:55:04.452 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 0, 2024-04-24T06:55:04.453 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-04-24T06:55:04.453 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 0, 2024-04-24T06:55:04.453 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 48117, 2024-04-24T06:55:04.453 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 48117, 2024-04-24T06:55:04.453 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 48117, 2024-04-24T06:55:04.453 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 48117, 2024-04-24T06:55:04.453 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-04-24T06:55:04.453 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:04.454 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:04.454 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:04.454 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-24T06:55:04.454 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.454 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-write_buf_throttle": { 2024-04-24T06:55:04.454 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-04-24T06:55:04.454 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 3758096384, 2024-04-24T06:55:04.455 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-04-24T06:55:04.455 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 2789, 2024-04-24T06:55:04.455 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 284353, 2024-04-24T06:55:04.455 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-04-24T06:55:04.455 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 2789, 2024-04-24T06:55:04.455 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-04-24T06:55:04.456 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-04-24T06:55:04.456 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 31, 2024-04-24T06:55:04.456 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 284353, 2024-04-24T06:55:04.456 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-04-24T06:55:04.456 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:04.456 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:04.456 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:04.457 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-24T06:55:04.457 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:04.457 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-write_buf_throttle-0x561a8e5c6460": { 2024-04-24T06:55:04.457 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 60, 2024-04-24T06:55:04.457 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 3758096384, 2024-04-24T06:55:04.457 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-04-24T06:55:04.457 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 21410, 2024-04-24T06:55:04.457 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 53957900, 2024-04-24T06:55:04.458 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-04-24T06:55:04.458 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 21410, 2024-04-24T06:55:04.458 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-04-24T06:55:04.458 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-04-24T06:55:04.458 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 475, 2024-04-24T06:55:04.458 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 53957840, 2024-04-24T06:55:04.458 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-04-24T06:55:04.459 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:04.459 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:04.459 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:04.459 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-24T06:55:04.459 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-24T06:55:04.459 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-24T06:55:05.133 INFO:tasks.check_counter:Daemon mds.b mds.exported=20 2024-04-24T06:55:05.134 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-04-24T06:55:05.134 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-24T06:55:05.134 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2024-04-24T06:55:05.134 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=5 2024-04-24T06:55:05.134 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=25 2024-04-24T06:55:05.134 DEBUG:tasks.check_counter:Getting stats from e 2024-04-24T06:55:05.135 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-24T06:55:05.388 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T06:55:05.672 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:05 smithi037 ceph-mon[32864]: pgmap v342: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-24T06:55:05.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:05 smithi137 ceph-mon[33017]: pgmap v342: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-24T06:55:05.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:05 smithi005 ceph-mon[27798]: pgmap v342: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-24T06:55:07.155 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-24T06:55:07.156 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T06:55:07.156 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 237, 2024-04-24T06:55:07.156 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 225, 2024-04-24T06:55:07.156 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 1732496, 2024-04-24T06:55:07.156 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 138784, 2024-04-24T06:55:07.156 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 4, 2024-04-24T06:55:07.156 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 0, 2024-04-24T06:55:07.156 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.052120662, 2024-04-24T06:55:07.157 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.015033742, 2024-04-24T06:55:07.157 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.040136580, 2024-04-24T06:55:07.157 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.005849281, 2024-04-24T06:55:07.157 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T06:55:07.157 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 224, 2024-04-24T06:55:07.157 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.011915383, 2024-04-24T06:55:07.157 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000053193 2024-04-24T06:55:07.157 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:07.158 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-24T06:55:07.158 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:07.158 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:07.158 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:07.158 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:07.158 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 1732496, 2024-04-24T06:55:07.158 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 138784 2024-04-24T06:55:07.158 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:07.159 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T06:55:07.159 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 3, 2024-04-24T06:55:07.159 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 174, 2024-04-24T06:55:07.159 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 480, 2024-04-24T06:55:07.159 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 265120, 2024-04-24T06:55:07.159 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 6, 2024-04-24T06:55:07.159 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 3, 2024-04-24T06:55:07.159 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.012824233, 2024-04-24T06:55:07.159 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.008652031, 2024-04-24T06:55:07.160 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.001548452, 2024-04-24T06:55:07.160 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T06:55:07.160 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T06:55:07.160 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 170, 2024-04-24T06:55:07.160 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.026935131, 2024-04-24T06:55:07.160 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000158441 2024-04-24T06:55:07.160 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:07.160 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-24T06:55:07.160 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:07.161 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:07.161 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:07.161 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:07.161 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-24T06:55:07.161 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 265120 2024-04-24T06:55:07.161 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:07.161 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T06:55:07.161 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 4, 2024-04-24T06:55:07.161 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 4, 2024-04-24T06:55:07.162 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 3024, 2024-04-24T06:55:07.162 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 448, 2024-04-24T06:55:07.162 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 2, 2024-04-24T06:55:07.162 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 1, 2024-04-24T06:55:07.162 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.000763181, 2024-04-24T06:55:07.162 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.000042608, 2024-04-24T06:55:07.162 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.000688884, 2024-04-24T06:55:07.162 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T06:55:07.163 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T06:55:07.163 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 3, 2024-04-24T06:55:07.163 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000736015, 2024-04-24T06:55:07.163 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000245338 2024-04-24T06:55:07.163 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:07.163 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-24T06:55:07.163 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:07.163 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:07.163 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:07.164 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:07.164 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-04-24T06:55:07.164 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 448 2024-04-24T06:55:07.164 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:07.164 INFO:teuthology.orchestra.run.smithi037.stdout: "cct": { 2024-04-24T06:55:07.164 INFO:teuthology.orchestra.run.smithi037.stdout: "total_workers": 0, 2024-04-24T06:55:07.164 INFO:teuthology.orchestra.run.smithi037.stdout: "unhealthy_workers": 0 2024-04-24T06:55:07.164 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:07.165 INFO:teuthology.orchestra.run.smithi037.stdout: "mempool": { 2024-04-24T06:55:07.165 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_bytes": 0, 2024-04-24T06:55:07.165 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_items": 0, 2024-04-24T06:55:07.165 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T06:55:07.165 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_items": 0, 2024-04-24T06:55:07.165 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T06:55:07.165 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_items": 0, 2024-04-24T06:55:07.165 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T06:55:07.165 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T06:55:07.165 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T06:55:07.166 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T06:55:07.166 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T06:55:07.166 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_items": 0, 2024-04-24T06:55:07.166 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T06:55:07.166 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T06:55:07.166 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_bytes": 0, 2024-04-24T06:55:07.166 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_items": 0, 2024-04-24T06:55:07.167 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_bytes": 0, 2024-04-24T06:55:07.167 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_items": 0, 2024-04-24T06:55:07.167 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T06:55:07.167 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T06:55:07.167 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T06:55:07.167 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T06:55:07.167 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T06:55:07.167 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_items": 0, 2024-04-24T06:55:07.167 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_bytes": 0, 2024-04-24T06:55:07.168 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_items": 0, 2024-04-24T06:55:07.168 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T06:55:07.168 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T06:55:07.168 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_bytes": 0, 2024-04-24T06:55:07.168 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_items": 0, 2024-04-24T06:55:07.168 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_bytes": 0, 2024-04-24T06:55:07.168 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_items": 0, 2024-04-24T06:55:07.168 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T06:55:07.168 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_items": 0, 2024-04-24T06:55:07.169 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T06:55:07.169 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_items": 0, 2024-04-24T06:55:07.169 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_bytes": 148955, 2024-04-24T06:55:07.169 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_items": 30, 2024-04-24T06:55:07.169 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_bytes": 0, 2024-04-24T06:55:07.169 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_items": 0, 2024-04-24T06:55:07.169 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_bytes": 0, 2024-04-24T06:55:07.169 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_items": 0, 2024-04-24T06:55:07.170 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_bytes": 0, 2024-04-24T06:55:07.170 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_items": 0, 2024-04-24T06:55:07.170 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_bytes": 0, 2024-04-24T06:55:07.170 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_items": 0, 2024-04-24T06:55:07.170 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_bytes": 0, 2024-04-24T06:55:07.170 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_items": 0, 2024-04-24T06:55:07.170 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T06:55:07.170 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_items": 0, 2024-04-24T06:55:07.171 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_bytes": 0, 2024-04-24T06:55:07.171 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_items": 0, 2024-04-24T06:55:07.171 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_bytes": 928, 2024-04-24T06:55:07.171 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_items": 2, 2024-04-24T06:55:07.171 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_bytes": 0, 2024-04-24T06:55:07.171 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_items": 0, 2024-04-24T06:55:07.171 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_bytes": 0, 2024-04-24T06:55:07.171 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_items": 0 2024-04-24T06:55:07.172 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:07.172 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T06:55:07.172 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-04-24T06:55:07.172 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-04-24T06:55:07.172 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-04-24T06:55:07.172 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 240, 2024-04-24T06:55:07.172 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 1699099, 2024-04-24T06:55:07.172 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-04-24T06:55:07.172 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 240, 2024-04-24T06:55:07.173 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-04-24T06:55:07.173 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-04-24T06:55:07.173 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 240, 2024-04-24T06:55:07.173 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 1699099, 2024-04-24T06:55:07.173 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-04-24T06:55:07.173 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:07.173 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:07.173 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:07.173 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-24T06:55:07.174 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-24T06:55:07.174 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-24T06:55:07.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:07 smithi037 ceph-mon[32864]: pgmap v343: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-24T06:55:07.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:07 smithi137 ceph-mon[33017]: pgmap v343: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-24T06:55:07.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:07 smithi005 ceph-mon[27798]: pgmap v343: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-24T06:55:08.097 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2024-04-24T06:55:08.097 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2024-04-24T06:55:08.097 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-04-24T06:55:08.098 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-04-24T06:55:08.098 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-04-24T06:55:08.098 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.e 2024-04-24T06:55:08.098 DEBUG:tasks.check_counter:Getting stats from h 2024-04-24T06:55:08.098 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-24T06:55:08.353 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T06:55:09.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:09 smithi137 ceph-mon[33017]: pgmap v344: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-24T06:55:09.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:09 smithi137 ceph-mon[33017]: osdmap e98: 12 total, 12 up, 12 in 2024-04-24T06:55:09.742 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:09 smithi037 ceph-mon[32864]: pgmap v344: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-24T06:55:09.742 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:09 smithi037 ceph-mon[32864]: osdmap e98: 12 total, 12 up, 12 in 2024-04-24T06:55:09.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:09 smithi005 ceph-mon[27798]: pgmap v344: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-24T06:55:09.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:09 smithi005 ceph-mon[27798]: osdmap e98: 12 total, 12 up, 12 in 2024-04-24T06:55:10.017 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-24T06:55:10.018 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T06:55:10.018 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 834, 2024-04-24T06:55:10.018 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 836, 2024-04-24T06:55:10.018 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 1054692, 2024-04-24T06:55:10.018 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 265276, 2024-04-24T06:55:10.018 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 6, 2024-04-24T06:55:10.019 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 2, 2024-04-24T06:55:10.019 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.078255241, 2024-04-24T06:55:10.019 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.030989537, 2024-04-24T06:55:10.019 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.018419713, 2024-04-24T06:55:10.019 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.019714003, 2024-04-24T06:55:10.019 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T06:55:10.019 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 834, 2024-04-24T06:55:10.019 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.035421158, 2024-04-24T06:55:10.020 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000042471 2024-04-24T06:55:10.020 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.020 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-24T06:55:10.020 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:10.020 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:10.020 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:10.020 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.020 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T06:55:10.021 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T06:55:10.021 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.021 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T06:55:10.021 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 671, 2024-04-24T06:55:10.021 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 841, 2024-04-24T06:55:10.021 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 164830, 2024-04-24T06:55:10.021 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 754493, 2024-04-24T06:55:10.021 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 8, 2024-04-24T06:55:10.022 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 5, 2024-04-24T06:55:10.022 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.076203568, 2024-04-24T06:55:10.022 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.027621768, 2024-04-24T06:55:10.022 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.016444984, 2024-04-24T06:55:10.022 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.023432949, 2024-04-24T06:55:10.022 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T06:55:10.022 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 837, 2024-04-24T06:55:10.023 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.048936291, 2024-04-24T06:55:10.023 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000058466 2024-04-24T06:55:10.023 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.023 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-24T06:55:10.023 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:10.023 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:10.023 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:10.023 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.024 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-24T06:55:10.024 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 542064 2024-04-24T06:55:10.024 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.024 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T06:55:10.024 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 1070, 2024-04-24T06:55:10.024 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 1051, 2024-04-24T06:55:10.024 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 2049472, 2024-04-24T06:55:10.024 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 398480, 2024-04-24T06:55:10.024 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 3, 2024-04-24T06:55:10.025 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 2, 2024-04-24T06:55:10.025 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.151785418, 2024-04-24T06:55:10.025 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.046900916, 2024-04-24T06:55:10.025 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.065240196, 2024-04-24T06:55:10.025 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.039071357, 2024-04-24T06:55:10.025 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T06:55:10.026 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 1051, 2024-04-24T06:55:10.026 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.059363451, 2024-04-24T06:55:10.026 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000056482 2024-04-24T06:55:10.026 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.026 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-24T06:55:10.026 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:10.026 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:10.027 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:10.027 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.027 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 1777984, 2024-04-24T06:55:10.027 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 141536 2024-04-24T06:55:10.027 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.027 INFO:teuthology.orchestra.run.smithi037.stdout: "cct": { 2024-04-24T06:55:10.027 INFO:teuthology.orchestra.run.smithi037.stdout: "total_workers": 1, 2024-04-24T06:55:10.027 INFO:teuthology.orchestra.run.smithi037.stdout: "unhealthy_workers": 0 2024-04-24T06:55:10.028 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.028 INFO:teuthology.orchestra.run.smithi037.stdout: "finisher-MDSRank": { 2024-04-24T06:55:10.028 INFO:teuthology.orchestra.run.smithi037.stdout: "queue_len": 0, 2024-04-24T06:55:10.028 INFO:teuthology.orchestra.run.smithi037.stdout: "complete_latency": { 2024-04-24T06:55:10.028 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 3945, 2024-04-24T06:55:10.028 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.118391827, 2024-04-24T06:55:10.028 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000030010 2024-04-24T06:55:10.028 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-24T06:55:10.029 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.029 INFO:teuthology.orchestra.run.smithi037.stdout: "finisher-PurgeQueue": { 2024-04-24T06:55:10.029 INFO:teuthology.orchestra.run.smithi037.stdout: "queue_len": 0, 2024-04-24T06:55:10.029 INFO:teuthology.orchestra.run.smithi037.stdout: "complete_latency": { 2024-04-24T06:55:10.029 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:10.029 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:10.029 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:10.029 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-24T06:55:10.030 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.030 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-04-24T06:55:10.030 INFO:teuthology.orchestra.run.smithi037.stdout: "request": 0, 2024-04-24T06:55:10.030 INFO:teuthology.orchestra.run.smithi037.stdout: "reply": 0, 2024-04-24T06:55:10.030 INFO:teuthology.orchestra.run.smithi037.stdout: "reply_latency": { 2024-04-24T06:55:10.030 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:10.030 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:10.030 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:10.031 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.031 INFO:teuthology.orchestra.run.smithi037.stdout: "slow_reply": 0, 2024-04-24T06:55:10.031 INFO:teuthology.orchestra.run.smithi037.stdout: "forward": 0, 2024-04-24T06:55:10.031 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_fetch_complete": 0, 2024-04-24T06:55:10.031 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_fetch_keys": 0, 2024-04-24T06:55:10.031 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_commit": 0, 2024-04-24T06:55:10.031 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_split": 0, 2024-04-24T06:55:10.031 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_merge": 0, 2024-04-24T06:55:10.032 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes": 254, 2024-04-24T06:55:10.032 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_top": 11, 2024-04-24T06:55:10.032 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_bottom": 243, 2024-04-24T06:55:10.032 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_pin_tail": 0, 2024-04-24T06:55:10.032 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_pinned": 239, 2024-04-24T06:55:10.032 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_expired": 0, 2024-04-24T06:55:10.032 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_with_caps": 0, 2024-04-24T06:55:10.032 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": 0, 2024-04-24T06:55:10.033 INFO:teuthology.orchestra.run.smithi037.stdout: "subtrees": 5, 2024-04-24T06:55:10.033 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse": 0, 2024-04-24T06:55:10.033 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_hit": 0, 2024-04-24T06:55:10.033 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_forward": 0, 2024-04-24T06:55:10.033 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_discover": 0, 2024-04-24T06:55:10.033 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_dir_fetch": 0, 2024-04-24T06:55:10.033 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_remote_ino": 0, 2024-04-24T06:55:10.033 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_lock": 0, 2024-04-24T06:55:10.034 INFO:teuthology.orchestra.run.smithi037.stdout: "load_cent": 0, 2024-04-24T06:55:10.034 INFO:teuthology.orchestra.run.smithi037.stdout: "q": 0, 2024-04-24T06:55:10.034 INFO:teuthology.orchestra.run.smithi037.stdout: "exported": 0, 2024-04-24T06:55:10.034 INFO:teuthology.orchestra.run.smithi037.stdout: "exported_inodes": 0, 2024-04-24T06:55:10.034 INFO:teuthology.orchestra.run.smithi037.stdout: "imported": 0, 2024-04-24T06:55:10.034 INFO:teuthology.orchestra.run.smithi037.stdout: "imported_inodes": 0, 2024-04-24T06:55:10.034 INFO:teuthology.orchestra.run.smithi037.stdout: "openino_dir_fetch": 0, 2024-04-24T06:55:10.034 INFO:teuthology.orchestra.run.smithi037.stdout: "openino_backtrace_fetch": 0, 2024-04-24T06:55:10.034 INFO:teuthology.orchestra.run.smithi037.stdout: "openino_peer_discover": 0, 2024-04-24T06:55:10.035 INFO:teuthology.orchestra.run.smithi037.stdout: "root_rfiles": 0, 2024-04-24T06:55:10.035 INFO:teuthology.orchestra.run.smithi037.stdout: "root_rbytes": 0, 2024-04-24T06:55:10.035 INFO:teuthology.orchestra.run.smithi037.stdout: "root_rsnaps": 0, 2024-04-24T06:55:10.035 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T06:55:10.035 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_set_tag": 0, 2024-04-24T06:55:10.035 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T06:55:10.035 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_inotable_repaired": 0, 2024-04-24T06:55:10.035 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_dir_inodes": 0, 2024-04-24T06:55:10.036 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T06:55:10.036 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T06:55:10.036 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_file_inodes": 0, 2024-04-24T06:55:10.036 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_inode_file_caps": 0, 2024-04-24T06:55:10.036 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_cap_op_revoke": 0, 2024-04-24T06:55:10.036 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_cap_op_grant": 0, 2024-04-24T06:55:10.037 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T06:55:10.037 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T06:55:10.037 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T06:55:10.037 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_client_caps": 0, 2024-04-24T06:55:10.037 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_client_caps_dirty": 0, 2024-04-24T06:55:10.037 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_client_cap_release": 0, 2024-04-24T06:55:10.037 INFO:teuthology.orchestra.run.smithi037.stdout: "process_request_cap_release": 0 2024-04-24T06:55:10.037 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.038 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_cache": { 2024-04-24T06:55:10.038 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_update": 0, 2024-04-24T06:55:10.038 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_update_receipt": 0, 2024-04-24T06:55:10.038 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_try_discover": 0, 2024-04-24T06:55:10.038 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_send_discover": 0, 2024-04-24T06:55:10.038 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_handle_discover": 0, 2024-04-24T06:55:10.038 INFO:teuthology.orchestra.run.smithi037.stdout: "num_strays": 0, 2024-04-24T06:55:10.038 INFO:teuthology.orchestra.run.smithi037.stdout: "num_strays_delayed": 0, 2024-04-24T06:55:10.039 INFO:teuthology.orchestra.run.smithi037.stdout: "num_strays_enqueuing": 0, 2024-04-24T06:55:10.039 INFO:teuthology.orchestra.run.smithi037.stdout: "strays_created": 0, 2024-04-24T06:55:10.039 INFO:teuthology.orchestra.run.smithi037.stdout: "strays_enqueued": 0, 2024-04-24T06:55:10.039 INFO:teuthology.orchestra.run.smithi037.stdout: "strays_reintegrated": 0, 2024-04-24T06:55:10.039 INFO:teuthology.orchestra.run.smithi037.stdout: "strays_migrated": 0, 2024-04-24T06:55:10.039 INFO:teuthology.orchestra.run.smithi037.stdout: "num_recovering_processing": 0, 2024-04-24T06:55:10.039 INFO:teuthology.orchestra.run.smithi037.stdout: "num_recovering_enqueued": 0, 2024-04-24T06:55:10.039 INFO:teuthology.orchestra.run.smithi037.stdout: "num_recovering_prioritized": 0, 2024-04-24T06:55:10.040 INFO:teuthology.orchestra.run.smithi037.stdout: "recovery_started": 0, 2024-04-24T06:55:10.040 INFO:teuthology.orchestra.run.smithi037.stdout: "recovery_completed": 0, 2024-04-24T06:55:10.040 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_quiesce_path": 0, 2024-04-24T06:55:10.040 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_quiesce_inode": 0, 2024-04-24T06:55:10.040 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T06:55:10.040 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_exportdir": 0, 2024-04-24T06:55:10.040 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_flush": 0, 2024-04-24T06:55:10.040 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_fragmentdir": 0, 2024-04-24T06:55:10.041 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_fragstats": 0, 2024-04-24T06:55:10.041 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_inodestats": 0 2024-04-24T06:55:10.041 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.041 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_log": { 2024-04-24T06:55:10.041 INFO:teuthology.orchestra.run.smithi037.stdout: "evlrg": 0, 2024-04-24T06:55:10.041 INFO:teuthology.orchestra.run.smithi037.stdout: "evadd": 0, 2024-04-24T06:55:10.041 INFO:teuthology.orchestra.run.smithi037.stdout: "evex": 0, 2024-04-24T06:55:10.041 INFO:teuthology.orchestra.run.smithi037.stdout: "evtrm": 0, 2024-04-24T06:55:10.042 INFO:teuthology.orchestra.run.smithi037.stdout: "ev": 384, 2024-04-24T06:55:10.042 INFO:teuthology.orchestra.run.smithi037.stdout: "evexg": 0, 2024-04-24T06:55:10.042 INFO:teuthology.orchestra.run.smithi037.stdout: "evexd": 0, 2024-04-24T06:55:10.042 INFO:teuthology.orchestra.run.smithi037.stdout: "segadd": 0, 2024-04-24T06:55:10.042 INFO:teuthology.orchestra.run.smithi037.stdout: "segex": 0, 2024-04-24T06:55:10.042 INFO:teuthology.orchestra.run.smithi037.stdout: "segtrm": 0, 2024-04-24T06:55:10.042 INFO:teuthology.orchestra.run.smithi037.stdout: "seg": 2, 2024-04-24T06:55:10.042 INFO:teuthology.orchestra.run.smithi037.stdout: "segmjr": 2, 2024-04-24T06:55:10.043 INFO:teuthology.orchestra.run.smithi037.stdout: "segexg": 0, 2024-04-24T06:55:10.043 INFO:teuthology.orchestra.run.smithi037.stdout: "segexd": 0, 2024-04-24T06:55:10.043 INFO:teuthology.orchestra.run.smithi037.stdout: "expos": 4194304, 2024-04-24T06:55:10.043 INFO:teuthology.orchestra.run.smithi037.stdout: "wrpos": 5030217, 2024-04-24T06:55:10.043 INFO:teuthology.orchestra.run.smithi037.stdout: "rdpos": 5029643, 2024-04-24T06:55:10.043 INFO:teuthology.orchestra.run.smithi037.stdout: "jlat": { 2024-04-24T06:55:10.043 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:10.043 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:10.044 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:10.044 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.044 INFO:teuthology.orchestra.run.smithi037.stdout: "replayed": 384 2024-04-24T06:55:10.044 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.044 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_mem": { 2024-04-24T06:55:10.044 INFO:teuthology.orchestra.run.smithi037.stdout: "ino": 225, 2024-04-24T06:55:10.044 INFO:teuthology.orchestra.run.smithi037.stdout: "ino+": 227, 2024-04-24T06:55:10.044 INFO:teuthology.orchestra.run.smithi037.stdout: "ino-": 2, 2024-04-24T06:55:10.045 INFO:teuthology.orchestra.run.smithi037.stdout: "dir": 27, 2024-04-24T06:55:10.045 INFO:teuthology.orchestra.run.smithi037.stdout: "dir+": 27, 2024-04-24T06:55:10.045 INFO:teuthology.orchestra.run.smithi037.stdout: "dir-": 0, 2024-04-24T06:55:10.045 INFO:teuthology.orchestra.run.smithi037.stdout: "dn": 254, 2024-04-24T06:55:10.045 INFO:teuthology.orchestra.run.smithi037.stdout: "dn+": 254, 2024-04-24T06:55:10.045 INFO:teuthology.orchestra.run.smithi037.stdout: "dn-": 0, 2024-04-24T06:55:10.045 INFO:teuthology.orchestra.run.smithi037.stdout: "cap": 0, 2024-04-24T06:55:10.045 INFO:teuthology.orchestra.run.smithi037.stdout: "cap+": 0, 2024-04-24T06:55:10.045 INFO:teuthology.orchestra.run.smithi037.stdout: "cap-": 0, 2024-04-24T06:55:10.046 INFO:teuthology.orchestra.run.smithi037.stdout: "rss": 53792, 2024-04-24T06:55:10.046 INFO:teuthology.orchestra.run.smithi037.stdout: "heap": 215416 2024-04-24T06:55:10.046 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.046 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_server": { 2024-04-24T06:55:10.046 INFO:teuthology.orchestra.run.smithi037.stdout: "dispatch_client_request": 0, 2024-04-24T06:55:10.046 INFO:teuthology.orchestra.run.smithi037.stdout: "dispatch_server_request": 0, 2024-04-24T06:55:10.046 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_client_request": 0, 2024-04-24T06:55:10.046 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_client_session": 0, 2024-04-24T06:55:10.047 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_peer_request": 0, 2024-04-24T06:55:10.047 INFO:teuthology.orchestra.run.smithi037.stdout: "req_create_latency": { 2024-04-24T06:55:10.047 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:10.047 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:10.047 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:10.047 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.047 INFO:teuthology.orchestra.run.smithi037.stdout: "req_getattr_latency": { 2024-04-24T06:55:10.047 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:10.048 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:10.048 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:10.048 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.048 INFO:teuthology.orchestra.run.smithi037.stdout: "req_getfilelock_latency": { 2024-04-24T06:55:10.048 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:10.048 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:10.048 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:10.048 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.049 INFO:teuthology.orchestra.run.smithi037.stdout: "req_link_latency": { 2024-04-24T06:55:10.049 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:10.049 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:10.049 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:10.049 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.049 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookup_latency": { 2024-04-24T06:55:10.049 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:10.049 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:10.049 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:10.050 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.050 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookuphash_latency": { 2024-04-24T06:55:10.050 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:10.050 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:10.050 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:10.050 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.050 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookupino_latency": { 2024-04-24T06:55:10.050 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:10.051 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:10.051 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:10.052 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.052 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookupname_latency": { 2024-04-24T06:55:10.052 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:10.052 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:10.053 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:10.053 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.054 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookupparent_latency": { 2024-04-24T06:55:10.054 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:10.054 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:10.054 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:10.054 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.054 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookupsnap_latency": { 2024-04-24T06:55:10.054 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:10.055 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:10.055 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:10.055 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.055 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lssnap_latency": { 2024-04-24T06:55:10.055 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:10.055 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:10.055 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:10.055 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.056 INFO:teuthology.orchestra.run.smithi037.stdout: "req_mkdir_latency": { 2024-04-24T06:55:10.056 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:10.056 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:10.057 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:10.057 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.057 INFO:teuthology.orchestra.run.smithi037.stdout: "req_mknod_latency": { 2024-04-24T06:55:10.057 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:10.057 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:10.057 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:10.057 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.057 INFO:teuthology.orchestra.run.smithi037.stdout: "req_mksnap_latency": { 2024-04-24T06:55:10.058 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:10.058 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:10.058 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:10.058 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.058 INFO:teuthology.orchestra.run.smithi037.stdout: "req_open_latency": { 2024-04-24T06:55:10.058 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:10.058 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:10.059 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:10.059 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.059 INFO:teuthology.orchestra.run.smithi037.stdout: "req_readdir_latency": { 2024-04-24T06:55:10.059 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:10.059 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:10.059 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:10.059 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.059 INFO:teuthology.orchestra.run.smithi037.stdout: "req_rename_latency": { 2024-04-24T06:55:10.059 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:10.060 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:10.060 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:10.060 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.060 INFO:teuthology.orchestra.run.smithi037.stdout: "req_renamesnap_latency": { 2024-04-24T06:55:10.060 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:10.060 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:10.060 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:10.060 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.061 INFO:teuthology.orchestra.run.smithi037.stdout: "req_snapdiff_latency": { 2024-04-24T06:55:10.061 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:10.061 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:10.061 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:10.061 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.061 INFO:teuthology.orchestra.run.smithi037.stdout: "req_rmdir_latency": { 2024-04-24T06:55:10.061 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:10.062 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:10.062 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:10.062 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.062 INFO:teuthology.orchestra.run.smithi037.stdout: "req_rmsnap_latency": { 2024-04-24T06:55:10.062 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:10.062 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:10.062 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:10.062 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.063 INFO:teuthology.orchestra.run.smithi037.stdout: "req_rmxattr_latency": { 2024-04-24T06:55:10.063 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:10.063 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:10.063 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:10.063 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.063 INFO:teuthology.orchestra.run.smithi037.stdout: "req_setattr_latency": { 2024-04-24T06:55:10.063 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:10.063 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:10.064 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:10.064 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.064 INFO:teuthology.orchestra.run.smithi037.stdout: "req_setdirlayout_latency": { 2024-04-24T06:55:10.064 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:10.064 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:10.064 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:10.064 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.064 INFO:teuthology.orchestra.run.smithi037.stdout: "req_setfilelock_latency": { 2024-04-24T06:55:10.065 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:10.065 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:10.065 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:10.065 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.065 INFO:teuthology.orchestra.run.smithi037.stdout: "req_setlayout_latency": { 2024-04-24T06:55:10.065 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:10.065 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:10.065 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:10.066 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.066 INFO:teuthology.orchestra.run.smithi037.stdout: "req_setxattr_latency": { 2024-04-24T06:55:10.066 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:10.066 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:10.066 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:10.066 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.066 INFO:teuthology.orchestra.run.smithi037.stdout: "req_symlink_latency": { 2024-04-24T06:55:10.066 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:10.067 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:10.067 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:10.067 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.067 INFO:teuthology.orchestra.run.smithi037.stdout: "req_unlink_latency": { 2024-04-24T06:55:10.067 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:10.067 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:10.067 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:10.067 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.068 INFO:teuthology.orchestra.run.smithi037.stdout: "cap_revoke_eviction": 0, 2024-04-24T06:55:10.068 INFO:teuthology.orchestra.run.smithi037.stdout: "cap_acquisition_throttle": 0, 2024-04-24T06:55:10.068 INFO:teuthology.orchestra.run.smithi037.stdout: "req_getvxattr_latency": { 2024-04-24T06:55:10.068 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:10.068 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:10.068 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:10.068 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-24T06:55:10.068 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.068 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_sessions": { 2024-04-24T06:55:10.069 INFO:teuthology.orchestra.run.smithi037.stdout: "session_count": 2, 2024-04-24T06:55:10.069 INFO:teuthology.orchestra.run.smithi037.stdout: "session_add": 4, 2024-04-24T06:55:10.069 INFO:teuthology.orchestra.run.smithi037.stdout: "session_remove": 2, 2024-04-24T06:55:10.069 INFO:teuthology.orchestra.run.smithi037.stdout: "sessions_open": 4, 2024-04-24T06:55:10.069 INFO:teuthology.orchestra.run.smithi037.stdout: "sessions_stale": 0, 2024-04-24T06:55:10.069 INFO:teuthology.orchestra.run.smithi037.stdout: "total_load": 0, 2024-04-24T06:55:10.069 INFO:teuthology.orchestra.run.smithi037.stdout: "average_load": 0, 2024-04-24T06:55:10.069 INFO:teuthology.orchestra.run.smithi037.stdout: "avg_session_uptime": 3846, 2024-04-24T06:55:10.070 INFO:teuthology.orchestra.run.smithi037.stdout: "mdthresh_evicted": 0 2024-04-24T06:55:10.070 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.070 INFO:teuthology.orchestra.run.smithi037.stdout: "mempool": { 2024-04-24T06:55:10.070 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_bytes": 0, 2024-04-24T06:55:10.070 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_items": 0, 2024-04-24T06:55:10.070 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T06:55:10.070 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_items": 0, 2024-04-24T06:55:10.070 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T06:55:10.071 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_items": 0, 2024-04-24T06:55:10.071 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T06:55:10.071 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T06:55:10.071 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T06:55:10.071 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T06:55:10.071 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T06:55:10.071 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_items": 0, 2024-04-24T06:55:10.071 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T06:55:10.072 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T06:55:10.072 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_bytes": 0, 2024-04-24T06:55:10.072 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_items": 0, 2024-04-24T06:55:10.072 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_bytes": 0, 2024-04-24T06:55:10.072 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_items": 0, 2024-04-24T06:55:10.072 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T06:55:10.072 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T06:55:10.072 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T06:55:10.073 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T06:55:10.073 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T06:55:10.073 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_items": 0, 2024-04-24T06:55:10.073 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_bytes": 0, 2024-04-24T06:55:10.073 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_items": 0, 2024-04-24T06:55:10.073 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T06:55:10.073 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T06:55:10.073 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_bytes": 0, 2024-04-24T06:55:10.073 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_items": 0, 2024-04-24T06:55:10.074 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_bytes": 0, 2024-04-24T06:55:10.074 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_items": 0, 2024-04-24T06:55:10.074 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T06:55:10.074 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_items": 0, 2024-04-24T06:55:10.074 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T06:55:10.074 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_items": 0, 2024-04-24T06:55:10.074 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_bytes": 197625, 2024-04-24T06:55:10.074 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_items": 274, 2024-04-24T06:55:10.075 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_bytes": 0, 2024-04-24T06:55:10.075 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_items": 0, 2024-04-24T06:55:10.075 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_bytes": 0, 2024-04-24T06:55:10.075 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_items": 0, 2024-04-24T06:55:10.075 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_bytes": 0, 2024-04-24T06:55:10.075 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_items": 0, 2024-04-24T06:55:10.075 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_bytes": 0, 2024-04-24T06:55:10.075 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_items": 0, 2024-04-24T06:55:10.076 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_bytes": 8248, 2024-04-24T06:55:10.076 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_items": 172, 2024-04-24T06:55:10.076 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T06:55:10.076 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_items": 0, 2024-04-24T06:55:10.076 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_bytes": 0, 2024-04-24T06:55:10.076 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_items": 0, 2024-04-24T06:55:10.076 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_bytes": 741043, 2024-04-24T06:55:10.076 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_items": 6967, 2024-04-24T06:55:10.077 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_bytes": 0, 2024-04-24T06:55:10.077 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_items": 0, 2024-04-24T06:55:10.077 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_bytes": 0, 2024-04-24T06:55:10.077 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_items": 0 2024-04-24T06:55:10.077 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.077 INFO:teuthology.orchestra.run.smithi037.stdout: "objecter": { 2024-04-24T06:55:10.077 INFO:teuthology.orchestra.run.smithi037.stdout: "op_active": 0, 2024-04-24T06:55:10.077 INFO:teuthology.orchestra.run.smithi037.stdout: "op_laggy": 0, 2024-04-24T06:55:10.078 INFO:teuthology.orchestra.run.smithi037.stdout: "op_send": 2310, 2024-04-24T06:55:10.078 INFO:teuthology.orchestra.run.smithi037.stdout: "op_send_bytes": 21, 2024-04-24T06:55:10.078 INFO:teuthology.orchestra.run.smithi037.stdout: "op_resend": 3, 2024-04-24T06:55:10.078 INFO:teuthology.orchestra.run.smithi037.stdout: "op_reply": 2310, 2024-04-24T06:55:10.078 INFO:teuthology.orchestra.run.smithi037.stdout: "op_latency": { 2024-04-24T06:55:10.078 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 2310, 2024-04-24T06:55:10.078 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 2.574965905, 2024-04-24T06:55:10.078 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.001114703 2024-04-24T06:55:10.078 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.079 INFO:teuthology.orchestra.run.smithi037.stdout: "op_inflight": 0, 2024-04-24T06:55:10.079 INFO:teuthology.orchestra.run.smithi037.stdout: "oplen_avg": { 2024-04-24T06:55:10.079 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 2310, 2024-04-24T06:55:10.079 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 2311 2024-04-24T06:55:10.079 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.079 INFO:teuthology.orchestra.run.smithi037.stdout: "op": 2310, 2024-04-24T06:55:10.079 INFO:teuthology.orchestra.run.smithi037.stdout: "op_r": 2310, 2024-04-24T06:55:10.079 INFO:teuthology.orchestra.run.smithi037.stdout: "op_w": 0, 2024-04-24T06:55:10.080 INFO:teuthology.orchestra.run.smithi037.stdout: "op_rmw": 0, 2024-04-24T06:55:10.080 INFO:teuthology.orchestra.run.smithi037.stdout: "op_pg": 0, 2024-04-24T06:55:10.080 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_stat": 1474, 2024-04-24T06:55:10.080 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_create": 0, 2024-04-24T06:55:10.080 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_read": 834, 2024-04-24T06:55:10.080 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_write": 0, 2024-04-24T06:55:10.080 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_writefull": 0, 2024-04-24T06:55:10.080 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_writesame": 0, 2024-04-24T06:55:10.080 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_append": 0, 2024-04-24T06:55:10.081 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_zero": 0, 2024-04-24T06:55:10.081 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_truncate": 0, 2024-04-24T06:55:10.081 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_delete": 0, 2024-04-24T06:55:10.081 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_mapext": 0, 2024-04-24T06:55:10.081 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_sparse_read": 0, 2024-04-24T06:55:10.081 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_clonerange": 0, 2024-04-24T06:55:10.081 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_getxattr": 1, 2024-04-24T06:55:10.082 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_setxattr": 0, 2024-04-24T06:55:10.082 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_cmpxattr": 0, 2024-04-24T06:55:10.082 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_rmxattr": 0, 2024-04-24T06:55:10.082 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_resetxattrs": 0, 2024-04-24T06:55:10.082 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_call": 0, 2024-04-24T06:55:10.082 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_watch": 0, 2024-04-24T06:55:10.082 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_notify": 0, 2024-04-24T06:55:10.082 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T06:55:10.082 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_pgls": 0, 2024-04-24T06:55:10.083 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_pgls_filter": 0, 2024-04-24T06:55:10.083 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_other": 0, 2024-04-24T06:55:10.083 INFO:teuthology.orchestra.run.smithi037.stdout: "linger_active": 0, 2024-04-24T06:55:10.083 INFO:teuthology.orchestra.run.smithi037.stdout: "linger_send": 0, 2024-04-24T06:55:10.083 INFO:teuthology.orchestra.run.smithi037.stdout: "linger_resend": 0, 2024-04-24T06:55:10.083 INFO:teuthology.orchestra.run.smithi037.stdout: "linger_ping": 0, 2024-04-24T06:55:10.083 INFO:teuthology.orchestra.run.smithi037.stdout: "poolop_active": 0, 2024-04-24T06:55:10.083 INFO:teuthology.orchestra.run.smithi037.stdout: "poolop_send": 0, 2024-04-24T06:55:10.084 INFO:teuthology.orchestra.run.smithi037.stdout: "poolop_resend": 0, 2024-04-24T06:55:10.084 INFO:teuthology.orchestra.run.smithi037.stdout: "poolstat_active": 0, 2024-04-24T06:55:10.084 INFO:teuthology.orchestra.run.smithi037.stdout: "poolstat_send": 0, 2024-04-24T06:55:10.084 INFO:teuthology.orchestra.run.smithi037.stdout: "poolstat_resend": 0, 2024-04-24T06:55:10.084 INFO:teuthology.orchestra.run.smithi037.stdout: "statfs_active": 0, 2024-04-24T06:55:10.084 INFO:teuthology.orchestra.run.smithi037.stdout: "statfs_send": 0, 2024-04-24T06:55:10.084 INFO:teuthology.orchestra.run.smithi037.stdout: "statfs_resend": 0, 2024-04-24T06:55:10.084 INFO:teuthology.orchestra.run.smithi037.stdout: "command_active": 0, 2024-04-24T06:55:10.085 INFO:teuthology.orchestra.run.smithi037.stdout: "command_send": 0, 2024-04-24T06:55:10.085 INFO:teuthology.orchestra.run.smithi037.stdout: "command_resend": 0, 2024-04-24T06:55:10.085 INFO:teuthology.orchestra.run.smithi037.stdout: "map_epoch": 98, 2024-04-24T06:55:10.085 INFO:teuthology.orchestra.run.smithi037.stdout: "map_full": 0, 2024-04-24T06:55:10.085 INFO:teuthology.orchestra.run.smithi037.stdout: "map_inc": 12, 2024-04-24T06:55:10.085 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_sessions": 5, 2024-04-24T06:55:10.085 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_session_open": 5, 2024-04-24T06:55:10.085 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_session_close": 0, 2024-04-24T06:55:10.085 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_laggy": 0, 2024-04-24T06:55:10.086 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_wr": 0, 2024-04-24T06:55:10.086 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_rd": 2, 2024-04-24T06:55:10.086 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_del": 0 2024-04-24T06:55:10.086 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.086 INFO:teuthology.orchestra.run.smithi037.stdout: "oft": { 2024-04-24T06:55:10.086 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_total_objs": 0, 2024-04-24T06:55:10.086 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_total_kv_pairs": 0, 2024-04-24T06:55:10.086 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_total_updates": 0, 2024-04-24T06:55:10.087 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_total_removes": 0 2024-04-24T06:55:10.087 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.087 INFO:teuthology.orchestra.run.smithi037.stdout: "osd-slow-ops": { 2024-04-24T06:55:10.087 INFO:teuthology.orchestra.run.smithi037.stdout: "slow_ops_count": 0 2024-04-24T06:55:10.087 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.087 INFO:teuthology.orchestra.run.smithi037.stdout: "purge_queue": { 2024-04-24T06:55:10.087 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executing_ops": 0, 2024-04-24T06:55:10.087 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executing_ops_high_water": 0, 2024-04-24T06:55:10.087 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executing": 0, 2024-04-24T06:55:10.088 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executing_high_water": 0, 2024-04-24T06:55:10.088 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executed_ops": 0, 2024-04-24T06:55:10.088 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executed": 0, 2024-04-24T06:55:10.088 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_item_in_journal": 0 2024-04-24T06:55:10.088 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.088 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T06:55:10.088 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-04-24T06:55:10.088 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-04-24T06:55:10.089 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-04-24T06:55:10.089 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 2571, 2024-04-24T06:55:10.089 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 3021302, 2024-04-24T06:55:10.089 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-04-24T06:55:10.089 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 2571, 2024-04-24T06:55:10.089 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-04-24T06:55:10.089 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-04-24T06:55:10.089 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 2571, 2024-04-24T06:55:10.089 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 3021302, 2024-04-24T06:55:10.090 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-04-24T06:55:10.090 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:10.090 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:10.090 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:10.090 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-24T06:55:10.090 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.090 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-objecter_bytes": { 2024-04-24T06:55:10.090 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-04-24T06:55:10.091 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-04-24T06:55:10.091 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-04-24T06:55:10.091 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 0, 2024-04-24T06:55:10.091 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 0, 2024-04-24T06:55:10.091 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-04-24T06:55:10.091 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 0, 2024-04-24T06:55:10.091 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 2310, 2024-04-24T06:55:10.091 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 835918, 2024-04-24T06:55:10.091 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 2310, 2024-04-24T06:55:10.092 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 835918, 2024-04-24T06:55:10.092 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-04-24T06:55:10.092 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:10.092 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:10.092 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:10.092 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-24T06:55:10.092 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.092 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-objecter_ops": { 2024-04-24T06:55:10.093 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-04-24T06:55:10.093 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 1024, 2024-04-24T06:55:10.093 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-04-24T06:55:10.093 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 0, 2024-04-24T06:55:10.093 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 0, 2024-04-24T06:55:10.093 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-04-24T06:55:10.093 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 0, 2024-04-24T06:55:10.093 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 2310, 2024-04-24T06:55:10.094 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 2310, 2024-04-24T06:55:10.094 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 2310, 2024-04-24T06:55:10.094 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 2310, 2024-04-24T06:55:10.094 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-04-24T06:55:10.094 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:10.094 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:10.094 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:10.094 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-24T06:55:10.095 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.095 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-write_buf_throttle": { 2024-04-24T06:55:10.095 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-04-24T06:55:10.095 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 3758096384, 2024-04-24T06:55:10.095 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-04-24T06:55:10.095 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 0, 2024-04-24T06:55:10.095 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 0, 2024-04-24T06:55:10.095 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-04-24T06:55:10.095 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 0, 2024-04-24T06:55:10.096 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-04-24T06:55:10.096 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-04-24T06:55:10.096 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 0, 2024-04-24T06:55:10.096 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 0, 2024-04-24T06:55:10.096 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-04-24T06:55:10.096 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:10.096 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:10.096 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:10.097 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-24T06:55:10.097 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:10.097 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-write_buf_throttle-0x55eaffd006e0": { 2024-04-24T06:55:10.097 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-04-24T06:55:10.097 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 3758096384, 2024-04-24T06:55:10.097 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-04-24T06:55:10.097 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 0, 2024-04-24T06:55:10.097 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 0, 2024-04-24T06:55:10.097 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-04-24T06:55:10.098 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 0, 2024-04-24T06:55:10.098 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-04-24T06:55:10.098 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-04-24T06:55:10.098 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 0, 2024-04-24T06:55:10.098 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 0, 2024-04-24T06:55:10.098 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-04-24T06:55:10.098 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:10.098 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:10.099 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:10.099 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-24T06:55:10.099 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-24T06:55:10.099 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-24T06:55:10.771 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-24T06:55:10.771 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-24T06:55:10.771 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-04-24T06:55:10.771 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-24T06:55:10.771 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-24T06:55:10.771 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-04-24T06:55:10.771 DEBUG:tasks.check_counter:Getting stats from k 2024-04-24T06:55:10.772 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-24T06:55:11.021 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T06:55:11.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:11 smithi137 ceph-mon[33017]: pgmap v346: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-24T06:55:11.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:11 smithi005 ceph-mon[27798]: pgmap v346: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-24T06:55:11.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:11 smithi037 ceph-mon[32864]: pgmap v346: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-24T06:55:12.843 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-24T06:55:12.843 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T06:55:12.843 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 4, 2024-04-24T06:55:12.844 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 4, 2024-04-24T06:55:12.844 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 3024, 2024-04-24T06:55:12.844 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 448, 2024-04-24T06:55:12.844 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 4, 2024-04-24T06:55:12.844 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 0, 2024-04-24T06:55:12.844 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.001606924, 2024-04-24T06:55:12.844 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.000038502, 2024-04-24T06:55:12.845 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.000494936, 2024-04-24T06:55:12.845 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T06:55:12.845 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T06:55:12.845 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 3, 2024-04-24T06:55:12.845 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000657345, 2024-04-24T06:55:12.845 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000219115 2024-04-24T06:55:12.845 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:12.845 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-24T06:55:12.846 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:12.846 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:12.846 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:12.846 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:12.846 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-04-24T06:55:12.846 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 448 2024-04-24T06:55:12.846 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:12.847 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T06:55:12.847 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 2, 2024-04-24T06:55:12.847 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 175, 2024-04-24T06:55:12.847 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 320, 2024-04-24T06:55:12.847 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 187072, 2024-04-24T06:55:12.847 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 5, 2024-04-24T06:55:12.847 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 3, 2024-04-24T06:55:12.847 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.013181950, 2024-04-24T06:55:12.847 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.008998078, 2024-04-24T06:55:12.848 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.001541320, 2024-04-24T06:55:12.848 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T06:55:12.848 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T06:55:12.848 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 172, 2024-04-24T06:55:12.848 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.025317802, 2024-04-24T06:55:12.848 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000147196 2024-04-24T06:55:12.848 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:12.848 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-24T06:55:12.849 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:12.849 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:12.849 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:12.849 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:12.849 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-24T06:55:12.849 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 187072 2024-04-24T06:55:12.849 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:12.849 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T06:55:12.850 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 241, 2024-04-24T06:55:12.850 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 229, 2024-04-24T06:55:12.850 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 1733584, 2024-04-24T06:55:12.850 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 141344, 2024-04-24T06:55:12.850 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 2, 2024-04-24T06:55:12.850 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 1, 2024-04-24T06:55:12.850 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.050051396, 2024-04-24T06:55:12.850 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.015334106, 2024-04-24T06:55:12.851 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.037277875, 2024-04-24T06:55:12.851 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.005878180, 2024-04-24T06:55:12.851 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T06:55:12.851 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 228, 2024-04-24T06:55:12.851 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.012661903, 2024-04-24T06:55:12.851 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000055534 2024-04-24T06:55:12.851 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:12.851 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-24T06:55:12.851 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:12.852 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:12.852 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:12.852 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:12.852 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 1733584, 2024-04-24T06:55:12.852 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 141344 2024-04-24T06:55:12.852 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:12.852 INFO:teuthology.orchestra.run.smithi037.stdout: "cct": { 2024-04-24T06:55:12.852 INFO:teuthology.orchestra.run.smithi037.stdout: "total_workers": 0, 2024-04-24T06:55:12.853 INFO:teuthology.orchestra.run.smithi037.stdout: "unhealthy_workers": 0 2024-04-24T06:55:12.853 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:12.853 INFO:teuthology.orchestra.run.smithi037.stdout: "mempool": { 2024-04-24T06:55:12.853 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_bytes": 0, 2024-04-24T06:55:12.853 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_items": 0, 2024-04-24T06:55:12.853 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T06:55:12.853 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_items": 0, 2024-04-24T06:55:12.853 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T06:55:12.854 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_items": 0, 2024-04-24T06:55:12.854 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T06:55:12.854 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T06:55:12.854 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T06:55:12.854 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T06:55:12.854 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T06:55:12.854 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_items": 0, 2024-04-24T06:55:12.854 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T06:55:12.855 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T06:55:12.855 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_bytes": 0, 2024-04-24T06:55:12.855 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_items": 0, 2024-04-24T06:55:12.855 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_bytes": 0, 2024-04-24T06:55:12.855 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_items": 0, 2024-04-24T06:55:12.855 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T06:55:12.855 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T06:55:12.855 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T06:55:12.855 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T06:55:12.856 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T06:55:12.856 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_items": 0, 2024-04-24T06:55:12.856 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_bytes": 0, 2024-04-24T06:55:12.856 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_items": 0, 2024-04-24T06:55:12.856 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T06:55:12.856 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T06:55:12.856 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_bytes": 0, 2024-04-24T06:55:12.856 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_items": 0, 2024-04-24T06:55:12.857 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_bytes": 0, 2024-04-24T06:55:12.857 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_items": 0, 2024-04-24T06:55:12.857 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T06:55:12.857 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_items": 0, 2024-04-24T06:55:12.857 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T06:55:12.857 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_items": 0, 2024-04-24T06:55:12.857 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_bytes": 148714, 2024-04-24T06:55:12.857 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_items": 30, 2024-04-24T06:55:12.858 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_bytes": 0, 2024-04-24T06:55:12.858 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_items": 0, 2024-04-24T06:55:12.858 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_bytes": 0, 2024-04-24T06:55:12.858 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_items": 0, 2024-04-24T06:55:12.858 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_bytes": 0, 2024-04-24T06:55:12.858 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_items": 0, 2024-04-24T06:55:12.858 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_bytes": 0, 2024-04-24T06:55:12.858 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_items": 0, 2024-04-24T06:55:12.859 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_bytes": 0, 2024-04-24T06:55:12.859 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_items": 0, 2024-04-24T06:55:12.859 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T06:55:12.859 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_items": 0, 2024-04-24T06:55:12.859 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_bytes": 0, 2024-04-24T06:55:12.859 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_items": 0, 2024-04-24T06:55:12.859 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_bytes": 928, 2024-04-24T06:55:12.859 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_items": 2, 2024-04-24T06:55:12.860 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_bytes": 0, 2024-04-24T06:55:12.860 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_items": 0, 2024-04-24T06:55:12.860 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_bytes": 0, 2024-04-24T06:55:12.860 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_items": 0 2024-04-24T06:55:12.860 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-24T06:55:12.860 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T06:55:12.860 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-04-24T06:55:12.860 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-04-24T06:55:12.860 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-04-24T06:55:12.861 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 243, 2024-04-24T06:55:12.861 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 1699592, 2024-04-24T06:55:12.861 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-04-24T06:55:12.861 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 243, 2024-04-24T06:55:12.861 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-04-24T06:55:12.861 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-04-24T06:55:12.861 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 243, 2024-04-24T06:55:12.861 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 1699592, 2024-04-24T06:55:12.862 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-04-24T06:55:12.862 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-24T06:55:12.862 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-24T06:55:12.862 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-24T06:55:12.862 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-24T06:55:12.862 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-24T06:55:12.862 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-24T06:55:13.498 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:13 smithi037 ceph-mon[32864]: pgmap v347: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-24T06:55:13.533 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-04-24T06:55:13.533 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-04-24T06:55:13.533 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-04-24T06:55:13.533 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-04-24T06:55:13.534 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-04-24T06:55:13.534 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-04-24T06:55:13.534 DEBUG:tasks.check_counter:Getting stats from c 2024-04-24T06:55:13.534 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-24T06:55:13.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:13 smithi137 ceph-mon[33017]: pgmap v347: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-24T06:55:13.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:13 smithi005 ceph-mon[27798]: pgmap v347: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-24T06:55:13.785 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T06:55:15.583 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-24T06:55:15.583 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T06:55:15.583 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 1177, 2024-04-24T06:55:15.583 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 1179, 2024-04-24T06:55:15.584 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 21339771, 2024-04-24T06:55:15.584 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 374296, 2024-04-24T06:55:15.584 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 7, 2024-04-24T06:55:15.584 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 3, 2024-04-24T06:55:15.584 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.146354136, 2024-04-24T06:55:15.584 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.041411800, 2024-04-24T06:55:15.584 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.047723893, 2024-04-24T06:55:15.585 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.044008338, 2024-04-24T06:55:15.585 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T06:55:15.585 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1177, 2024-04-24T06:55:15.585 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.064187959, 2024-04-24T06:55:15.585 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000054535 2024-04-24T06:55:15.585 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.585 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-04-24T06:55:15.586 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:15.586 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:15.586 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:15.586 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.586 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T06:55:15.586 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T06:55:15.586 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.586 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T06:55:15.587 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 643, 2024-04-24T06:55:15.587 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 814, 2024-04-24T06:55:15.587 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 24772475, 2024-04-24T06:55:15.587 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 670976, 2024-04-24T06:55:15.587 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 8, 2024-04-24T06:55:15.587 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 6, 2024-04-24T06:55:15.587 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.078782016, 2024-04-24T06:55:15.588 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.031212411, 2024-04-24T06:55:15.588 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.025137288, 2024-04-24T06:55:15.588 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.014814680, 2024-04-24T06:55:15.588 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T06:55:15.588 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 811, 2024-04-24T06:55:15.588 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.061900887, 2024-04-24T06:55:15.588 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000076326 2024-04-24T06:55:15.589 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.589 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-04-24T06:55:15.589 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:15.589 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:15.589 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:15.589 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.589 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-24T06:55:15.589 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 467072 2024-04-24T06:55:15.589 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.590 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T06:55:15.590 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 1046, 2024-04-24T06:55:15.590 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 1027, 2024-04-24T06:55:15.590 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 10361563, 2024-04-24T06:55:15.590 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 391335, 2024-04-24T06:55:15.590 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 4, 2024-04-24T06:55:15.590 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 3, 2024-04-24T06:55:15.591 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.133097322, 2024-04-24T06:55:15.591 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.044222872, 2024-04-24T06:55:15.591 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.059645903, 2024-04-24T06:55:15.591 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.032249108, 2024-04-24T06:55:15.591 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T06:55:15.591 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1027, 2024-04-24T06:55:15.591 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.044970302, 2024-04-24T06:55:15.591 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000043788 2024-04-24T06:55:15.592 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.592 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-04-24T06:55:15.592 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:15.592 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:15.592 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:15.592 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.592 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 1778688, 2024-04-24T06:55:15.592 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 142976 2024-04-24T06:55:15.592 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.593 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2024-04-24T06:55:15.593 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 1, 2024-04-24T06:55:15.593 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2024-04-24T06:55:15.593 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.593 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-MDSRank": { 2024-04-24T06:55:15.593 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-04-24T06:55:15.593 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-04-24T06:55:15.593 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 4271, 2024-04-24T06:55:15.594 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.132218631, 2024-04-24T06:55:15.594 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000030957 2024-04-24T06:55:15.594 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-24T06:55:15.594 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.594 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-PurgeQueue": { 2024-04-24T06:55:15.594 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-04-24T06:55:15.594 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-04-24T06:55:15.595 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:15.595 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:15.595 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:15.595 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-24T06:55:15.595 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.595 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-04-24T06:55:15.595 INFO:teuthology.orchestra.run.smithi137.stdout: "request": 0, 2024-04-24T06:55:15.595 INFO:teuthology.orchestra.run.smithi137.stdout: "reply": 0, 2024-04-24T06:55:15.595 INFO:teuthology.orchestra.run.smithi137.stdout: "reply_latency": { 2024-04-24T06:55:15.596 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:15.596 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:15.596 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:15.596 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.596 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_reply": 0, 2024-04-24T06:55:15.596 INFO:teuthology.orchestra.run.smithi137.stdout: "forward": 0, 2024-04-24T06:55:15.596 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_complete": 0, 2024-04-24T06:55:15.597 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_keys": 0, 2024-04-24T06:55:15.597 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_commit": 0, 2024-04-24T06:55:15.597 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_split": 0, 2024-04-24T06:55:15.597 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_merge": 0, 2024-04-24T06:55:15.597 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes": 7338, 2024-04-24T06:55:15.597 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_top": 0, 2024-04-24T06:55:15.597 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_bottom": 7338, 2024-04-24T06:55:15.597 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pin_tail": 0, 2024-04-24T06:55:15.597 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pinned": 7338, 2024-04-24T06:55:15.598 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_expired": 0, 2024-04-24T06:55:15.598 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_with_caps": 0, 2024-04-24T06:55:15.598 INFO:teuthology.orchestra.run.smithi137.stdout: "caps": 0, 2024-04-24T06:55:15.598 INFO:teuthology.orchestra.run.smithi137.stdout: "subtrees": 69, 2024-04-24T06:55:15.598 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse": 0, 2024-04-24T06:55:15.598 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_hit": 0, 2024-04-24T06:55:15.598 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_forward": 0, 2024-04-24T06:55:15.599 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_discover": 0, 2024-04-24T06:55:15.599 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_dir_fetch": 0, 2024-04-24T06:55:15.599 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_remote_ino": 0, 2024-04-24T06:55:15.599 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_lock": 0, 2024-04-24T06:55:15.599 INFO:teuthology.orchestra.run.smithi137.stdout: "load_cent": 0, 2024-04-24T06:55:15.599 INFO:teuthology.orchestra.run.smithi137.stdout: "q": 0, 2024-04-24T06:55:15.599 INFO:teuthology.orchestra.run.smithi137.stdout: "exported": 0, 2024-04-24T06:55:15.600 INFO:teuthology.orchestra.run.smithi137.stdout: "exported_inodes": 0, 2024-04-24T06:55:15.600 INFO:teuthology.orchestra.run.smithi137.stdout: "imported": 0, 2024-04-24T06:55:15.600 INFO:teuthology.orchestra.run.smithi137.stdout: "imported_inodes": 0, 2024-04-24T06:55:15.600 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_dir_fetch": 0, 2024-04-24T06:55:15.600 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_backtrace_fetch": 0, 2024-04-24T06:55:15.600 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_peer_discover": 0, 2024-04-24T06:55:15.600 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rfiles": 3454, 2024-04-24T06:55:15.600 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rbytes": 26624153, 2024-04-24T06:55:15.601 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rsnaps": 6, 2024-04-24T06:55:15.601 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T06:55:15.601 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_set_tag": 0, 2024-04-24T06:55:15.601 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T06:55:15.601 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_inotable_repaired": 0, 2024-04-24T06:55:15.601 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_inodes": 0, 2024-04-24T06:55:15.601 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T06:55:15.601 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T06:55:15.602 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_file_inodes": 0, 2024-04-24T06:55:15.602 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_inode_file_caps": 0, 2024-04-24T06:55:15.602 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_revoke": 0, 2024-04-24T06:55:15.602 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_grant": 0, 2024-04-24T06:55:15.602 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T06:55:15.602 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T06:55:15.602 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T06:55:15.602 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps": 0, 2024-04-24T06:55:15.603 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps_dirty": 0, 2024-04-24T06:55:15.603 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_cap_release": 0, 2024-04-24T06:55:15.603 INFO:teuthology.orchestra.run.smithi137.stdout: "process_request_cap_release": 0 2024-04-24T06:55:15.603 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.603 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_cache": { 2024-04-24T06:55:15.603 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update": 0, 2024-04-24T06:55:15.603 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update_receipt": 0, 2024-04-24T06:55:15.603 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_try_discover": 0, 2024-04-24T06:55:15.604 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_send_discover": 0, 2024-04-24T06:55:15.604 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_handle_discover": 0, 2024-04-24T06:55:15.604 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays": 0, 2024-04-24T06:55:15.604 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_delayed": 0, 2024-04-24T06:55:15.604 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_enqueuing": 0, 2024-04-24T06:55:15.604 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_created": 0, 2024-04-24T06:55:15.604 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_enqueued": 0, 2024-04-24T06:55:15.604 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_reintegrated": 0, 2024-04-24T06:55:15.605 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_migrated": 0, 2024-04-24T06:55:15.605 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_processing": 0, 2024-04-24T06:55:15.605 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_enqueued": 0, 2024-04-24T06:55:15.605 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_prioritized": 0, 2024-04-24T06:55:15.605 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_started": 0, 2024-04-24T06:55:15.605 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_completed": 0, 2024-04-24T06:55:15.605 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_quiesce_path": 0, 2024-04-24T06:55:15.605 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_quiesce_inode": 0, 2024-04-24T06:55:15.606 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T06:55:15.606 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_exportdir": 0, 2024-04-24T06:55:15.606 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_flush": 0, 2024-04-24T06:55:15.606 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragmentdir": 0, 2024-04-24T06:55:15.606 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragstats": 0, 2024-04-24T06:55:15.606 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_inodestats": 0 2024-04-24T06:55:15.606 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.607 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_log": { 2024-04-24T06:55:15.607 INFO:teuthology.orchestra.run.smithi137.stdout: "evlrg": 0, 2024-04-24T06:55:15.607 INFO:teuthology.orchestra.run.smithi137.stdout: "evadd": 0, 2024-04-24T06:55:15.607 INFO:teuthology.orchestra.run.smithi137.stdout: "evex": 0, 2024-04-24T06:55:15.607 INFO:teuthology.orchestra.run.smithi137.stdout: "evtrm": 0, 2024-04-24T06:55:15.607 INFO:teuthology.orchestra.run.smithi137.stdout: "ev": 21411, 2024-04-24T06:55:15.607 INFO:teuthology.orchestra.run.smithi137.stdout: "evexg": 0, 2024-04-24T06:55:15.607 INFO:teuthology.orchestra.run.smithi137.stdout: "evexd": 0, 2024-04-24T06:55:15.608 INFO:teuthology.orchestra.run.smithi137.stdout: "segadd": 0, 2024-04-24T06:55:15.608 INFO:teuthology.orchestra.run.smithi137.stdout: "segex": 0, 2024-04-24T06:55:15.608 INFO:teuthology.orchestra.run.smithi137.stdout: "segtrm": 0, 2024-04-24T06:55:15.608 INFO:teuthology.orchestra.run.smithi137.stdout: "seg": 30, 2024-04-24T06:55:15.608 INFO:teuthology.orchestra.run.smithi137.stdout: "segmjr": 3, 2024-04-24T06:55:15.608 INFO:teuthology.orchestra.run.smithi137.stdout: "segexg": 0, 2024-04-24T06:55:15.608 INFO:teuthology.orchestra.run.smithi137.stdout: "segexd": 0, 2024-04-24T06:55:15.608 INFO:teuthology.orchestra.run.smithi137.stdout: "expos": 4194304, 2024-04-24T06:55:15.609 INFO:teuthology.orchestra.run.smithi137.stdout: "wrpos": 58152316, 2024-04-24T06:55:15.609 INFO:teuthology.orchestra.run.smithi137.stdout: "rdpos": 58152204, 2024-04-24T06:55:15.609 INFO:teuthology.orchestra.run.smithi137.stdout: "jlat": { 2024-04-24T06:55:15.609 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:15.609 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:15.609 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:15.609 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.609 INFO:teuthology.orchestra.run.smithi137.stdout: "replayed": 21411 2024-04-24T06:55:15.609 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.610 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_mem": { 2024-04-24T06:55:15.610 INFO:teuthology.orchestra.run.smithi137.stdout: "ino": 3621, 2024-04-24T06:55:15.610 INFO:teuthology.orchestra.run.smithi137.stdout: "ino+": 7209, 2024-04-24T06:55:15.610 INFO:teuthology.orchestra.run.smithi137.stdout: "ino-": 3588, 2024-04-24T06:55:15.610 INFO:teuthology.orchestra.run.smithi137.stdout: "dir": 288, 2024-04-24T06:55:15.610 INFO:teuthology.orchestra.run.smithi137.stdout: "dir+": 362, 2024-04-24T06:55:15.610 INFO:teuthology.orchestra.run.smithi137.stdout: "dir-": 74, 2024-04-24T06:55:15.611 INFO:teuthology.orchestra.run.smithi137.stdout: "dn": 7338, 2024-04-24T06:55:15.611 INFO:teuthology.orchestra.run.smithi137.stdout: "dn+": 8166, 2024-04-24T06:55:15.611 INFO:teuthology.orchestra.run.smithi137.stdout: "dn-": 828, 2024-04-24T06:55:15.611 INFO:teuthology.orchestra.run.smithi137.stdout: "cap": 0, 2024-04-24T06:55:15.611 INFO:teuthology.orchestra.run.smithi137.stdout: "cap+": 0, 2024-04-24T06:55:15.611 INFO:teuthology.orchestra.run.smithi137.stdout: "cap-": 0, 2024-04-24T06:55:15.611 INFO:teuthology.orchestra.run.smithi137.stdout: "rss": 74152, 2024-04-24T06:55:15.611 INFO:teuthology.orchestra.run.smithi137.stdout: "heap": 215416 2024-04-24T06:55:15.612 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.612 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_server": { 2024-04-24T06:55:15.612 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_client_request": 0, 2024-04-24T06:55:15.612 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_server_request": 0, 2024-04-24T06:55:15.612 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_request": 0, 2024-04-24T06:55:15.612 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_session": 0, 2024-04-24T06:55:15.612 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_peer_request": 0, 2024-04-24T06:55:15.612 INFO:teuthology.orchestra.run.smithi137.stdout: "req_create_latency": { 2024-04-24T06:55:15.612 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:15.613 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:15.613 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:15.613 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.613 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getattr_latency": { 2024-04-24T06:55:15.613 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:15.613 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:15.613 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:15.613 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.614 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getfilelock_latency": { 2024-04-24T06:55:15.614 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:15.614 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:15.614 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:15.614 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.614 INFO:teuthology.orchestra.run.smithi137.stdout: "req_link_latency": { 2024-04-24T06:55:15.614 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:15.614 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:15.615 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:15.615 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.615 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookup_latency": { 2024-04-24T06:55:15.615 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:15.615 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:15.615 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:15.615 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.615 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookuphash_latency": { 2024-04-24T06:55:15.616 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:15.616 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:15.616 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:15.616 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.616 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupino_latency": { 2024-04-24T06:55:15.616 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:15.616 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:15.616 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:15.617 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.617 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupname_latency": { 2024-04-24T06:55:15.617 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:15.617 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:15.617 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:15.617 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.617 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupparent_latency": { 2024-04-24T06:55:15.617 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:15.618 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:15.618 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:15.618 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.618 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupsnap_latency": { 2024-04-24T06:55:15.618 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:15.618 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:15.618 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:15.619 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.619 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lssnap_latency": { 2024-04-24T06:55:15.619 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:15.619 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:15.619 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:15.619 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.619 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mkdir_latency": { 2024-04-24T06:55:15.619 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:15.620 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:15.620 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:15.620 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.620 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mknod_latency": { 2024-04-24T06:55:15.620 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:15.620 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:15.620 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:15.620 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.621 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mksnap_latency": { 2024-04-24T06:55:15.621 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:15.621 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:15.621 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:15.621 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.621 INFO:teuthology.orchestra.run.smithi137.stdout: "req_open_latency": { 2024-04-24T06:55:15.621 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:15.621 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:15.622 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:15.622 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.622 INFO:teuthology.orchestra.run.smithi137.stdout: "req_readdir_latency": { 2024-04-24T06:55:15.622 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:15.622 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:15.622 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:15.622 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.622 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rename_latency": { 2024-04-24T06:55:15.623 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:15.623 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:15.623 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:15.623 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.623 INFO:teuthology.orchestra.run.smithi137.stdout: "req_renamesnap_latency": { 2024-04-24T06:55:15.623 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:15.623 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:15.623 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:15.624 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.624 INFO:teuthology.orchestra.run.smithi137.stdout: "req_snapdiff_latency": { 2024-04-24T06:55:15.624 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:15.624 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:15.624 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:15.625 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.625 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmdir_latency": { 2024-04-24T06:55:15.625 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:15.625 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:15.625 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:15.625 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.625 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmsnap_latency": { 2024-04-24T06:55:15.625 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:15.626 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:15.626 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:15.626 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.626 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmxattr_latency": { 2024-04-24T06:55:15.626 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:15.626 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:15.626 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:15.626 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.627 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setattr_latency": { 2024-04-24T06:55:15.627 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:15.627 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:15.627 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:15.627 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.627 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setdirlayout_latency": { 2024-04-24T06:55:15.627 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:15.627 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:15.628 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:15.628 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.628 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setfilelock_latency": { 2024-04-24T06:55:15.628 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:15.628 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:15.628 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:15.628 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.628 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setlayout_latency": { 2024-04-24T06:55:15.629 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:15.629 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:15.629 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:15.629 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.629 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setxattr_latency": { 2024-04-24T06:55:15.629 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:15.629 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:15.629 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:15.630 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.630 INFO:teuthology.orchestra.run.smithi137.stdout: "req_symlink_latency": { 2024-04-24T06:55:15.630 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:15.630 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:15.630 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:15.630 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.630 INFO:teuthology.orchestra.run.smithi137.stdout: "req_unlink_latency": { 2024-04-24T06:55:15.630 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:15.631 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:15.631 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:15.631 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.631 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_revoke_eviction": 0, 2024-04-24T06:55:15.631 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_acquisition_throttle": 0, 2024-04-24T06:55:15.631 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getvxattr_latency": { 2024-04-24T06:55:15.631 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:15.631 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:15.632 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:15.632 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-24T06:55:15.632 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.632 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_sessions": { 2024-04-24T06:55:15.632 INFO:teuthology.orchestra.run.smithi137.stdout: "session_count": 2, 2024-04-24T06:55:15.632 INFO:teuthology.orchestra.run.smithi137.stdout: "session_add": 5, 2024-04-24T06:55:15.632 INFO:teuthology.orchestra.run.smithi137.stdout: "session_remove": 3, 2024-04-24T06:55:15.633 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_open": 4, 2024-04-24T06:55:15.633 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_stale": 0, 2024-04-24T06:55:15.633 INFO:teuthology.orchestra.run.smithi137.stdout: "total_load": 0, 2024-04-24T06:55:15.633 INFO:teuthology.orchestra.run.smithi137.stdout: "average_load": 0, 2024-04-24T06:55:15.633 INFO:teuthology.orchestra.run.smithi137.stdout: "avg_session_uptime": 3660, 2024-04-24T06:55:15.633 INFO:teuthology.orchestra.run.smithi137.stdout: "mdthresh_evicted": 0 2024-04-24T06:55:15.633 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.633 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2024-04-24T06:55:15.633 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2024-04-24T06:55:15.634 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2024-04-24T06:55:15.634 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T06:55:15.634 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2024-04-24T06:55:15.634 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T06:55:15.634 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2024-04-24T06:55:15.634 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T06:55:15.634 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T06:55:15.635 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T06:55:15.635 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T06:55:15.635 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T06:55:15.635 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2024-04-24T06:55:15.635 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T06:55:15.635 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T06:55:15.635 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2024-04-24T06:55:15.635 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2024-04-24T06:55:15.636 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2024-04-24T06:55:15.636 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2024-04-24T06:55:15.636 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T06:55:15.636 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T06:55:15.636 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T06:55:15.636 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T06:55:15.636 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T06:55:15.636 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2024-04-24T06:55:15.637 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2024-04-24T06:55:15.637 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2024-04-24T06:55:15.637 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T06:55:15.637 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T06:55:15.637 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2024-04-24T06:55:15.637 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2024-04-24T06:55:15.637 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2024-04-24T06:55:15.638 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2024-04-24T06:55:15.638 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T06:55:15.638 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2024-04-24T06:55:15.638 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T06:55:15.638 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2024-04-24T06:55:15.638 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 349795, 2024-04-24T06:55:15.638 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 3725, 2024-04-24T06:55:15.639 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 0, 2024-04-24T06:55:15.639 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 0, 2024-04-24T06:55:15.639 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2024-04-24T06:55:15.639 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2024-04-24T06:55:15.639 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2024-04-24T06:55:15.639 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2024-04-24T06:55:15.639 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2024-04-24T06:55:15.639 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2024-04-24T06:55:15.640 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 8248, 2024-04-24T06:55:15.640 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 172, 2024-04-24T06:55:15.640 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T06:55:15.640 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2024-04-24T06:55:15.640 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2024-04-24T06:55:15.640 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2024-04-24T06:55:15.640 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 13912433, 2024-04-24T06:55:15.640 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 120790, 2024-04-24T06:55:15.641 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2024-04-24T06:55:15.641 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2024-04-24T06:55:15.641 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2024-04-24T06:55:15.641 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2024-04-24T06:55:15.641 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.641 INFO:teuthology.orchestra.run.smithi137.stdout: "objecter": { 2024-04-24T06:55:15.641 INFO:teuthology.orchestra.run.smithi137.stdout: "op_active": 0, 2024-04-24T06:55:15.641 INFO:teuthology.orchestra.run.smithi137.stdout: "op_laggy": 0, 2024-04-24T06:55:15.642 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send": 2599, 2024-04-24T06:55:15.642 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send_bytes": 26, 2024-04-24T06:55:15.642 INFO:teuthology.orchestra.run.smithi137.stdout: "op_resend": 4, 2024-04-24T06:55:15.642 INFO:teuthology.orchestra.run.smithi137.stdout: "op_reply": 2599, 2024-04-24T06:55:15.642 INFO:teuthology.orchestra.run.smithi137.stdout: "op_latency": { 2024-04-24T06:55:15.642 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2599, 2024-04-24T06:55:15.642 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 1.641970239, 2024-04-24T06:55:15.642 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000631770 2024-04-24T06:55:15.643 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.643 INFO:teuthology.orchestra.run.smithi137.stdout: "op_inflight": 0, 2024-04-24T06:55:15.643 INFO:teuthology.orchestra.run.smithi137.stdout: "oplen_avg": { 2024-04-24T06:55:15.643 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2599, 2024-04-24T06:55:15.643 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 2600 2024-04-24T06:55:15.643 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.643 INFO:teuthology.orchestra.run.smithi137.stdout: "op": 2599, 2024-04-24T06:55:15.643 INFO:teuthology.orchestra.run.smithi137.stdout: "op_r": 2599, 2024-04-24T06:55:15.644 INFO:teuthology.orchestra.run.smithi137.stdout: "op_w": 0, 2024-04-24T06:55:15.644 INFO:teuthology.orchestra.run.smithi137.stdout: "op_rmw": 0, 2024-04-24T06:55:15.644 INFO:teuthology.orchestra.run.smithi137.stdout: "op_pg": 0, 2024-04-24T06:55:15.644 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_stat": 1656, 2024-04-24T06:55:15.644 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_create": 0, 2024-04-24T06:55:15.644 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_read": 940, 2024-04-24T06:55:15.644 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_write": 0, 2024-04-24T06:55:15.644 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writefull": 0, 2024-04-24T06:55:15.645 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writesame": 0, 2024-04-24T06:55:15.645 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_append": 0, 2024-04-24T06:55:15.645 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_zero": 0, 2024-04-24T06:55:15.645 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_truncate": 0, 2024-04-24T06:55:15.645 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_delete": 0, 2024-04-24T06:55:15.645 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_mapext": 0, 2024-04-24T06:55:15.645 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_sparse_read": 0, 2024-04-24T06:55:15.645 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_clonerange": 0, 2024-04-24T06:55:15.646 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_getxattr": 2, 2024-04-24T06:55:15.646 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_setxattr": 0, 2024-04-24T06:55:15.646 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_cmpxattr": 0, 2024-04-24T06:55:15.646 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_rmxattr": 0, 2024-04-24T06:55:15.646 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_resetxattrs": 0, 2024-04-24T06:55:15.646 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_call": 0, 2024-04-24T06:55:15.646 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_watch": 0, 2024-04-24T06:55:15.646 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_notify": 0, 2024-04-24T06:55:15.647 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T06:55:15.647 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls": 0, 2024-04-24T06:55:15.647 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls_filter": 0, 2024-04-24T06:55:15.647 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_other": 0, 2024-04-24T06:55:15.647 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_active": 0, 2024-04-24T06:55:15.647 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_send": 0, 2024-04-24T06:55:15.647 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_resend": 0, 2024-04-24T06:55:15.647 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_ping": 0, 2024-04-24T06:55:15.648 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_active": 0, 2024-04-24T06:55:15.648 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_send": 0, 2024-04-24T06:55:15.648 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_resend": 0, 2024-04-24T06:55:15.648 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_active": 0, 2024-04-24T06:55:15.648 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_send": 0, 2024-04-24T06:55:15.648 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_resend": 0, 2024-04-24T06:55:15.648 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_active": 0, 2024-04-24T06:55:15.648 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_send": 0, 2024-04-24T06:55:15.648 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_resend": 0, 2024-04-24T06:55:15.649 INFO:teuthology.orchestra.run.smithi137.stdout: "command_active": 0, 2024-04-24T06:55:15.649 INFO:teuthology.orchestra.run.smithi137.stdout: "command_send": 0, 2024-04-24T06:55:15.649 INFO:teuthology.orchestra.run.smithi137.stdout: "command_resend": 0, 2024-04-24T06:55:15.649 INFO:teuthology.orchestra.run.smithi137.stdout: "map_epoch": 98, 2024-04-24T06:55:15.649 INFO:teuthology.orchestra.run.smithi137.stdout: "map_full": 0, 2024-04-24T06:55:15.649 INFO:teuthology.orchestra.run.smithi137.stdout: "map_inc": 12, 2024-04-24T06:55:15.649 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_sessions": 8, 2024-04-24T06:55:15.649 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_open": 8, 2024-04-24T06:55:15.650 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_close": 0, 2024-04-24T06:55:15.650 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_laggy": 0, 2024-04-24T06:55:15.650 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_wr": 0, 2024-04-24T06:55:15.650 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_rd": 2, 2024-04-24T06:55:15.650 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_del": 0 2024-04-24T06:55:15.650 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.650 INFO:teuthology.orchestra.run.smithi137.stdout: "oft": { 2024-04-24T06:55:15.651 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_objs": 0, 2024-04-24T06:55:15.651 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_kv_pairs": 0, 2024-04-24T06:55:15.651 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_updates": 0, 2024-04-24T06:55:15.651 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_removes": 0 2024-04-24T06:55:15.651 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.651 INFO:teuthology.orchestra.run.smithi137.stdout: "osd-slow-ops": { 2024-04-24T06:55:15.652 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_ops_count": 0 2024-04-24T06:55:15.652 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.652 INFO:teuthology.orchestra.run.smithi137.stdout: "purge_queue": { 2024-04-24T06:55:15.652 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops": 0, 2024-04-24T06:55:15.652 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops_high_water": 0, 2024-04-24T06:55:15.652 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing": 0, 2024-04-24T06:55:15.652 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_high_water": 0, 2024-04-24T06:55:15.652 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed_ops": 0, 2024-04-24T06:55:15.653 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed": 0, 2024-04-24T06:55:15.653 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_item_in_journal": 0 2024-04-24T06:55:15.653 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.653 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T06:55:15.653 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-24T06:55:15.653 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-04-24T06:55:15.653 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-24T06:55:15.653 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 2862, 2024-04-24T06:55:15.654 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 56199820, 2024-04-24T06:55:15.654 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-24T06:55:15.654 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 2862, 2024-04-24T06:55:15.654 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-04-24T06:55:15.654 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-04-24T06:55:15.654 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 2862, 2024-04-24T06:55:15.654 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 56199820, 2024-04-24T06:55:15.654 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-24T06:55:15.655 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:15.655 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:15.655 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:15.655 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-24T06:55:15.655 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.655 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_bytes": { 2024-04-24T06:55:15.655 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-24T06:55:15.655 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-04-24T06:55:15.655 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-24T06:55:15.656 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-04-24T06:55:15.656 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-04-24T06:55:15.656 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-24T06:55:15.656 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-04-24T06:55:15.656 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 2599, 2024-04-24T06:55:15.656 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 53966677, 2024-04-24T06:55:15.656 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 2599, 2024-04-24T06:55:15.656 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 53966677, 2024-04-24T06:55:15.657 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-24T06:55:15.657 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:15.657 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:15.657 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:15.657 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-24T06:55:15.657 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.657 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_ops": { 2024-04-24T06:55:15.657 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-24T06:55:15.658 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 1024, 2024-04-24T06:55:15.658 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-24T06:55:15.658 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-04-24T06:55:15.658 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-04-24T06:55:15.658 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-24T06:55:15.658 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-04-24T06:55:15.658 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 2599, 2024-04-24T06:55:15.658 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 2599, 2024-04-24T06:55:15.659 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 2599, 2024-04-24T06:55:15.659 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 2599, 2024-04-24T06:55:15.659 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-24T06:55:15.659 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:15.659 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:15.659 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:15.659 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-24T06:55:15.660 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.660 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle": { 2024-04-24T06:55:15.660 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-24T06:55:15.660 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-04-24T06:55:15.660 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-24T06:55:15.660 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-04-24T06:55:15.660 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-04-24T06:55:15.661 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-24T06:55:15.661 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-04-24T06:55:15.661 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-04-24T06:55:15.661 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-04-24T06:55:15.661 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 0, 2024-04-24T06:55:15.661 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 0, 2024-04-24T06:55:15.661 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-24T06:55:15.661 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:15.662 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:15.662 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:15.662 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-24T06:55:15.662 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:15.662 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle-0x55db51f966e0": { 2024-04-24T06:55:15.662 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-24T06:55:15.662 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-04-24T06:55:15.662 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-24T06:55:15.663 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-04-24T06:55:15.663 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-04-24T06:55:15.663 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-24T06:55:15.663 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-04-24T06:55:15.663 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-04-24T06:55:15.663 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-04-24T06:55:15.663 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 0, 2024-04-24T06:55:15.663 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 0, 2024-04-24T06:55:15.663 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-24T06:55:15.664 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:15.664 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:15.664 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:15.664 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-24T06:55:15.664 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-24T06:55:15.664 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-24T06:55:15.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:15 smithi137 ceph-mon[33017]: pgmap v348: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-24T06:55:15.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:15 smithi005 ceph-mon[27798]: pgmap v348: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-24T06:55:15.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:15 smithi037 ceph-mon[32864]: pgmap v348: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-24T06:55:16.301 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-24T06:55:16.302 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-24T06:55:16.302 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-04-24T06:55:16.302 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-24T06:55:16.302 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-24T06:55:16.302 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-04-24T06:55:16.302 DEBUG:tasks.check_counter:Getting stats from f 2024-04-24T06:55:16.303 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-24T06:55:16.549 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T06:55:17.388 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:17 smithi137 ceph-mon[33017]: osdmap e99: 12 total, 12 up, 12 in 2024-04-24T06:55:17.389 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:17 smithi137 ceph-mon[33017]: pgmap v350: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-24T06:55:17.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:17 smithi005 ceph-mon[27798]: osdmap e99: 12 total, 12 up, 12 in 2024-04-24T06:55:17.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:17 smithi005 ceph-mon[27798]: pgmap v350: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-24T06:55:17.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:17 smithi037 ceph-mon[32864]: osdmap e99: 12 total, 12 up, 12 in 2024-04-24T06:55:17.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:17 smithi037 ceph-mon[32864]: pgmap v350: 97 pgs: 97 active+clean; 85 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-24T06:55:18.235 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-24T06:55:18.236 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T06:55:18.236 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 2337, 2024-04-24T06:55:18.236 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 2339, 2024-04-24T06:55:18.236 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 6572770, 2024-04-24T06:55:18.236 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 743173, 2024-04-24T06:55:18.236 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 6, 2024-04-24T06:55:18.236 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 2, 2024-04-24T06:55:18.236 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.208553724, 2024-04-24T06:55:18.237 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.071292029, 2024-04-24T06:55:18.237 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.053236064, 2024-04-24T06:55:18.237 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.067508518, 2024-04-24T06:55:18.237 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T06:55:18.237 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2337, 2024-04-24T06:55:18.237 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.090198184, 2024-04-24T06:55:18.237 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000038595 2024-04-24T06:55:18.237 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.238 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-04-24T06:55:18.238 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:18.238 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:18.238 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:18.238 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.238 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T06:55:18.238 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T06:55:18.238 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.239 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T06:55:18.239 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 268, 2024-04-24T06:55:18.239 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 249, 2024-04-24T06:55:18.239 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 1779820, 2024-04-24T06:55:18.239 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 144330, 2024-04-24T06:55:18.239 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 3, 2024-04-24T06:55:18.239 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 2, 2024-04-24T06:55:18.239 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.051843257, 2024-04-24T06:55:18.239 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.015632820, 2024-04-24T06:55:18.240 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.040682873, 2024-04-24T06:55:18.240 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.005873047, 2024-04-24T06:55:18.240 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T06:55:18.240 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 249, 2024-04-24T06:55:18.240 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.014383050, 2024-04-24T06:55:18.240 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000057763 2024-04-24T06:55:18.240 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.240 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-04-24T06:55:18.241 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:18.241 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:18.241 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:18.241 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.241 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 1779264, 2024-04-24T06:55:18.241 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 143632 2024-04-24T06:55:18.241 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.241 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T06:55:18.242 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 3, 2024-04-24T06:55:18.242 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 176, 2024-04-24T06:55:18.242 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 588, 2024-04-24T06:55:18.242 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 546142, 2024-04-24T06:55:18.242 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 7, 2024-04-24T06:55:18.242 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 4, 2024-04-24T06:55:18.242 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.011004091, 2024-04-24T06:55:18.242 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.006847434, 2024-04-24T06:55:18.243 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.002166127, 2024-04-24T06:55:18.243 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000025377, 2024-04-24T06:55:18.243 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T06:55:18.243 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 173, 2024-04-24T06:55:18.243 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.038380506, 2024-04-24T06:55:18.243 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000221852 2024-04-24T06:55:18.243 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.243 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-04-24T06:55:18.243 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:18.244 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:18.244 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:18.244 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.244 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-24T06:55:18.244 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 545824 2024-04-24T06:55:18.244 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.244 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2024-04-24T06:55:18.244 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 1, 2024-04-24T06:55:18.245 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2024-04-24T06:55:18.245 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.245 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-MDSRank": { 2024-04-24T06:55:18.245 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-04-24T06:55:18.245 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-04-24T06:55:18.245 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 3984, 2024-04-24T06:55:18.245 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.114279710, 2024-04-24T06:55:18.245 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000028684 2024-04-24T06:55:18.245 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-24T06:55:18.246 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.246 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-PurgeQueue": { 2024-04-24T06:55:18.246 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-04-24T06:55:18.246 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-04-24T06:55:18.246 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:18.246 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:18.246 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:18.246 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-24T06:55:18.247 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.247 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-04-24T06:55:18.247 INFO:teuthology.orchestra.run.smithi137.stdout: "request": 0, 2024-04-24T06:55:18.247 INFO:teuthology.orchestra.run.smithi137.stdout: "reply": 0, 2024-04-24T06:55:18.247 INFO:teuthology.orchestra.run.smithi137.stdout: "reply_latency": { 2024-04-24T06:55:18.247 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:18.247 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:18.247 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:18.248 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.248 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_reply": 0, 2024-04-24T06:55:18.248 INFO:teuthology.orchestra.run.smithi137.stdout: "forward": 0, 2024-04-24T06:55:18.248 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_complete": 0, 2024-04-24T06:55:18.248 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_keys": 0, 2024-04-24T06:55:18.248 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_commit": 0, 2024-04-24T06:55:18.248 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_split": 0, 2024-04-24T06:55:18.248 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_merge": 0, 2024-04-24T06:55:18.248 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes": 1758, 2024-04-24T06:55:18.249 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_top": 2, 2024-04-24T06:55:18.249 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_bottom": 1756, 2024-04-24T06:55:18.249 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pin_tail": 0, 2024-04-24T06:55:18.249 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pinned": 1756, 2024-04-24T06:55:18.249 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_expired": 0, 2024-04-24T06:55:18.249 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_with_caps": 0, 2024-04-24T06:55:18.249 INFO:teuthology.orchestra.run.smithi137.stdout: "caps": 0, 2024-04-24T06:55:18.249 INFO:teuthology.orchestra.run.smithi137.stdout: "subtrees": 68, 2024-04-24T06:55:18.250 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse": 0, 2024-04-24T06:55:18.250 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_hit": 0, 2024-04-24T06:55:18.250 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_forward": 0, 2024-04-24T06:55:18.250 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_discover": 0, 2024-04-24T06:55:18.250 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_dir_fetch": 0, 2024-04-24T06:55:18.250 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_remote_ino": 0, 2024-04-24T06:55:18.250 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_lock": 0, 2024-04-24T06:55:18.250 INFO:teuthology.orchestra.run.smithi137.stdout: "load_cent": 0, 2024-04-24T06:55:18.251 INFO:teuthology.orchestra.run.smithi137.stdout: "q": 0, 2024-04-24T06:55:18.251 INFO:teuthology.orchestra.run.smithi137.stdout: "exported": 0, 2024-04-24T06:55:18.251 INFO:teuthology.orchestra.run.smithi137.stdout: "exported_inodes": 0, 2024-04-24T06:55:18.251 INFO:teuthology.orchestra.run.smithi137.stdout: "imported": 0, 2024-04-24T06:55:18.251 INFO:teuthology.orchestra.run.smithi137.stdout: "imported_inodes": 0, 2024-04-24T06:55:18.251 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_dir_fetch": 0, 2024-04-24T06:55:18.251 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_backtrace_fetch": 0, 2024-04-24T06:55:18.251 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_peer_discover": 0, 2024-04-24T06:55:18.251 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rfiles": 0, 2024-04-24T06:55:18.252 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rbytes": 0, 2024-04-24T06:55:18.252 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rsnaps": 0, 2024-04-24T06:55:18.252 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T06:55:18.252 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_set_tag": 0, 2024-04-24T06:55:18.252 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T06:55:18.252 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_inotable_repaired": 0, 2024-04-24T06:55:18.252 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_inodes": 0, 2024-04-24T06:55:18.252 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T06:55:18.253 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T06:55:18.253 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_file_inodes": 0, 2024-04-24T06:55:18.253 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_inode_file_caps": 0, 2024-04-24T06:55:18.253 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_revoke": 0, 2024-04-24T06:55:18.253 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_grant": 0, 2024-04-24T06:55:18.253 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T06:55:18.253 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T06:55:18.253 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T06:55:18.254 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps": 0, 2024-04-24T06:55:18.254 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps_dirty": 0, 2024-04-24T06:55:18.254 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_cap_release": 0, 2024-04-24T06:55:18.254 INFO:teuthology.orchestra.run.smithi137.stdout: "process_request_cap_release": 0 2024-04-24T06:55:18.254 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.254 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_cache": { 2024-04-24T06:55:18.254 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update": 0, 2024-04-24T06:55:18.254 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update_receipt": 0, 2024-04-24T06:55:18.255 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_try_discover": 0, 2024-04-24T06:55:18.255 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_send_discover": 0, 2024-04-24T06:55:18.255 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_handle_discover": 0, 2024-04-24T06:55:18.255 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays": 0, 2024-04-24T06:55:18.255 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_delayed": 0, 2024-04-24T06:55:18.255 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_enqueuing": 0, 2024-04-24T06:55:18.255 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_created": 0, 2024-04-24T06:55:18.255 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_enqueued": 0, 2024-04-24T06:55:18.255 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_reintegrated": 0, 2024-04-24T06:55:18.256 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_migrated": 0, 2024-04-24T06:55:18.256 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_processing": 0, 2024-04-24T06:55:18.256 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_enqueued": 0, 2024-04-24T06:55:18.256 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_prioritized": 0, 2024-04-24T06:55:18.256 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_started": 0, 2024-04-24T06:55:18.256 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_completed": 0, 2024-04-24T06:55:18.256 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_quiesce_path": 0, 2024-04-24T06:55:18.256 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_quiesce_inode": 0, 2024-04-24T06:55:18.257 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T06:55:18.257 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_exportdir": 0, 2024-04-24T06:55:18.257 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_flush": 0, 2024-04-24T06:55:18.257 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragmentdir": 0, 2024-04-24T06:55:18.257 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragstats": 0, 2024-04-24T06:55:18.257 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_inodestats": 0 2024-04-24T06:55:18.257 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.257 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_log": { 2024-04-24T06:55:18.257 INFO:teuthology.orchestra.run.smithi137.stdout: "evlrg": 0, 2024-04-24T06:55:18.258 INFO:teuthology.orchestra.run.smithi137.stdout: "evadd": 0, 2024-04-24T06:55:18.258 INFO:teuthology.orchestra.run.smithi137.stdout: "evex": 0, 2024-04-24T06:55:18.258 INFO:teuthology.orchestra.run.smithi137.stdout: "evtrm": 0, 2024-04-24T06:55:18.258 INFO:teuthology.orchestra.run.smithi137.stdout: "ev": 3349, 2024-04-24T06:55:18.258 INFO:teuthology.orchestra.run.smithi137.stdout: "evexg": 0, 2024-04-24T06:55:18.258 INFO:teuthology.orchestra.run.smithi137.stdout: "evexd": 0, 2024-04-24T06:55:18.258 INFO:teuthology.orchestra.run.smithi137.stdout: "segadd": 0, 2024-04-24T06:55:18.258 INFO:teuthology.orchestra.run.smithi137.stdout: "segex": 0, 2024-04-24T06:55:18.259 INFO:teuthology.orchestra.run.smithi137.stdout: "segtrm": 0, 2024-04-24T06:55:18.259 INFO:teuthology.orchestra.run.smithi137.stdout: "seg": 5, 2024-04-24T06:55:18.259 INFO:teuthology.orchestra.run.smithi137.stdout: "segmjr": 2, 2024-04-24T06:55:18.259 INFO:teuthology.orchestra.run.smithi137.stdout: "segexg": 0, 2024-04-24T06:55:18.259 INFO:teuthology.orchestra.run.smithi137.stdout: "segexd": 0, 2024-04-24T06:55:18.259 INFO:teuthology.orchestra.run.smithi137.stdout: "expos": 4194304, 2024-04-24T06:55:18.259 INFO:teuthology.orchestra.run.smithi137.stdout: "wrpos": 10103172, 2024-04-24T06:55:18.259 INFO:teuthology.orchestra.run.smithi137.stdout: "rdpos": 10102598, 2024-04-24T06:55:18.260 INFO:teuthology.orchestra.run.smithi137.stdout: "jlat": { 2024-04-24T06:55:18.260 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:18.260 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:18.260 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:18.260 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.260 INFO:teuthology.orchestra.run.smithi137.stdout: "replayed": 3349 2024-04-24T06:55:18.260 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.260 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_mem": { 2024-04-24T06:55:18.261 INFO:teuthology.orchestra.run.smithi137.stdout: "ino": 1072, 2024-04-24T06:55:18.261 INFO:teuthology.orchestra.run.smithi137.stdout: "ino+": 1523, 2024-04-24T06:55:18.261 INFO:teuthology.orchestra.run.smithi137.stdout: "ino-": 451, 2024-04-24T06:55:18.261 INFO:teuthology.orchestra.run.smithi137.stdout: "dir": 86, 2024-04-24T06:55:18.261 INFO:teuthology.orchestra.run.smithi137.stdout: "dir+": 94, 2024-04-24T06:55:18.261 INFO:teuthology.orchestra.run.smithi137.stdout: "dir-": 8, 2024-04-24T06:55:18.261 INFO:teuthology.orchestra.run.smithi137.stdout: "dn": 1758, 2024-04-24T06:55:18.261 INFO:teuthology.orchestra.run.smithi137.stdout: "dn+": 1758, 2024-04-24T06:55:18.261 INFO:teuthology.orchestra.run.smithi137.stdout: "dn-": 0, 2024-04-24T06:55:18.262 INFO:teuthology.orchestra.run.smithi137.stdout: "cap": 0, 2024-04-24T06:55:18.262 INFO:teuthology.orchestra.run.smithi137.stdout: "cap+": 0, 2024-04-24T06:55:18.262 INFO:teuthology.orchestra.run.smithi137.stdout: "cap-": 0, 2024-04-24T06:55:18.262 INFO:teuthology.orchestra.run.smithi137.stdout: "rss": 56688, 2024-04-24T06:55:18.262 INFO:teuthology.orchestra.run.smithi137.stdout: "heap": 215416 2024-04-24T06:55:18.262 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.262 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_server": { 2024-04-24T06:55:18.262 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_client_request": 0, 2024-04-24T06:55:18.263 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_server_request": 0, 2024-04-24T06:55:18.263 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_request": 0, 2024-04-24T06:55:18.263 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_session": 0, 2024-04-24T06:55:18.263 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_peer_request": 0, 2024-04-24T06:55:18.263 INFO:teuthology.orchestra.run.smithi137.stdout: "req_create_latency": { 2024-04-24T06:55:18.263 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:18.263 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:18.263 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:18.263 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.264 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getattr_latency": { 2024-04-24T06:55:18.264 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:18.264 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:18.264 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:18.264 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.264 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getfilelock_latency": { 2024-04-24T06:55:18.264 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:18.264 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:18.265 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:18.265 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.265 INFO:teuthology.orchestra.run.smithi137.stdout: "req_link_latency": { 2024-04-24T06:55:18.265 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:18.265 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:18.265 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:18.265 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.265 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookup_latency": { 2024-04-24T06:55:18.266 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:18.266 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:18.266 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:18.266 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.266 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookuphash_latency": { 2024-04-24T06:55:18.266 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:18.266 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:18.266 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:18.266 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.267 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupino_latency": { 2024-04-24T06:55:18.267 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:18.267 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:18.267 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:18.267 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.267 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupname_latency": { 2024-04-24T06:55:18.267 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:18.267 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:18.268 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:18.268 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.268 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupparent_latency": { 2024-04-24T06:55:18.268 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:18.268 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:18.268 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:18.268 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.268 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupsnap_latency": { 2024-04-24T06:55:18.269 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:18.269 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:18.269 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:18.269 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.269 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lssnap_latency": { 2024-04-24T06:55:18.269 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:18.269 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:18.269 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:18.269 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.270 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mkdir_latency": { 2024-04-24T06:55:18.270 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:18.270 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:18.271 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:18.271 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.271 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mknod_latency": { 2024-04-24T06:55:18.271 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:18.271 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:18.271 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:18.271 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.271 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mksnap_latency": { 2024-04-24T06:55:18.272 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:18.272 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:18.272 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:18.272 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.272 INFO:teuthology.orchestra.run.smithi137.stdout: "req_open_latency": { 2024-04-24T06:55:18.272 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:18.272 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:18.272 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:18.273 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.273 INFO:teuthology.orchestra.run.smithi137.stdout: "req_readdir_latency": { 2024-04-24T06:55:18.273 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:18.273 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:18.273 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:18.273 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.273 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rename_latency": { 2024-04-24T06:55:18.273 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:18.274 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:18.274 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:18.274 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.274 INFO:teuthology.orchestra.run.smithi137.stdout: "req_renamesnap_latency": { 2024-04-24T06:55:18.274 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:18.274 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:18.274 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:18.274 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.274 INFO:teuthology.orchestra.run.smithi137.stdout: "req_snapdiff_latency": { 2024-04-24T06:55:18.275 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:18.275 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:18.275 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:18.275 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.275 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmdir_latency": { 2024-04-24T06:55:18.275 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:18.275 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:18.275 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:18.276 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.276 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmsnap_latency": { 2024-04-24T06:55:18.276 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:18.276 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:18.276 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:18.276 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.276 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmxattr_latency": { 2024-04-24T06:55:18.276 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:18.276 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:18.277 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:18.277 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.277 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setattr_latency": { 2024-04-24T06:55:18.277 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:18.277 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:18.277 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:18.277 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.277 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setdirlayout_latency": { 2024-04-24T06:55:18.278 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:18.278 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:18.278 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:18.278 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.278 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setfilelock_latency": { 2024-04-24T06:55:18.278 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:18.278 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:18.278 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:18.279 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.279 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setlayout_latency": { 2024-04-24T06:55:18.279 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:18.279 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:18.279 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:18.279 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.279 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setxattr_latency": { 2024-04-24T06:55:18.279 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:18.280 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:18.280 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:18.280 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.280 INFO:teuthology.orchestra.run.smithi137.stdout: "req_symlink_latency": { 2024-04-24T06:55:18.280 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:18.280 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:18.280 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:18.280 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.280 INFO:teuthology.orchestra.run.smithi137.stdout: "req_unlink_latency": { 2024-04-24T06:55:18.281 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:18.281 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:18.281 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:18.281 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.281 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_revoke_eviction": 0, 2024-04-24T06:55:18.281 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_acquisition_throttle": 0, 2024-04-24T06:55:18.281 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getvxattr_latency": { 2024-04-24T06:55:18.281 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:18.282 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:18.282 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:18.282 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-24T06:55:18.282 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.282 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_sessions": { 2024-04-24T06:55:18.282 INFO:teuthology.orchestra.run.smithi137.stdout: "session_count": 2, 2024-04-24T06:55:18.282 INFO:teuthology.orchestra.run.smithi137.stdout: "session_add": 4, 2024-04-24T06:55:18.282 INFO:teuthology.orchestra.run.smithi137.stdout: "session_remove": 2, 2024-04-24T06:55:18.283 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_open": 2, 2024-04-24T06:55:18.283 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_stale": 0, 2024-04-24T06:55:18.283 INFO:teuthology.orchestra.run.smithi137.stdout: "total_load": 0, 2024-04-24T06:55:18.283 INFO:teuthology.orchestra.run.smithi137.stdout: "average_load": 0, 2024-04-24T06:55:18.283 INFO:teuthology.orchestra.run.smithi137.stdout: "avg_session_uptime": 5159, 2024-04-24T06:55:18.283 INFO:teuthology.orchestra.run.smithi137.stdout: "mdthresh_evicted": 0 2024-04-24T06:55:18.283 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.283 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2024-04-24T06:55:18.284 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2024-04-24T06:55:18.284 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2024-04-24T06:55:18.284 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T06:55:18.284 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2024-04-24T06:55:18.284 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T06:55:18.284 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2024-04-24T06:55:18.284 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T06:55:18.284 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T06:55:18.285 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T06:55:18.285 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T06:55:18.285 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T06:55:18.285 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2024-04-24T06:55:18.285 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T06:55:18.285 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T06:55:18.285 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2024-04-24T06:55:18.285 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2024-04-24T06:55:18.285 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2024-04-24T06:55:18.286 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2024-04-24T06:55:18.286 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T06:55:18.286 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T06:55:18.286 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T06:55:18.286 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T06:55:18.286 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T06:55:18.286 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2024-04-24T06:55:18.286 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2024-04-24T06:55:18.287 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2024-04-24T06:55:18.287 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T06:55:18.287 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T06:55:18.287 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2024-04-24T06:55:18.287 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2024-04-24T06:55:18.287 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2024-04-24T06:55:18.287 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2024-04-24T06:55:18.287 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T06:55:18.288 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2024-04-24T06:55:18.288 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T06:55:18.288 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2024-04-24T06:55:18.288 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 220847, 2024-04-24T06:55:18.288 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 1116, 2024-04-24T06:55:18.288 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 0, 2024-04-24T06:55:18.288 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 0, 2024-04-24T06:55:18.288 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2024-04-24T06:55:18.288 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2024-04-24T06:55:18.289 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2024-04-24T06:55:18.289 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2024-04-24T06:55:18.289 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2024-04-24T06:55:18.289 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2024-04-24T06:55:18.289 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 8248, 2024-04-24T06:55:18.289 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 172, 2024-04-24T06:55:18.289 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T06:55:18.289 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2024-04-24T06:55:18.290 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2024-04-24T06:55:18.290 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2024-04-24T06:55:18.290 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 3812217, 2024-04-24T06:55:18.290 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 31934, 2024-04-24T06:55:18.290 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2024-04-24T06:55:18.290 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2024-04-24T06:55:18.290 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2024-04-24T06:55:18.290 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2024-04-24T06:55:18.291 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.291 INFO:teuthology.orchestra.run.smithi137.stdout: "objecter": { 2024-04-24T06:55:18.291 INFO:teuthology.orchestra.run.smithi137.stdout: "op_active": 0, 2024-04-24T06:55:18.291 INFO:teuthology.orchestra.run.smithi137.stdout: "op_laggy": 0, 2024-04-24T06:55:18.291 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send": 2340, 2024-04-24T06:55:18.291 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send_bytes": 21, 2024-04-24T06:55:18.291 INFO:teuthology.orchestra.run.smithi137.stdout: "op_resend": 3, 2024-04-24T06:55:18.291 INFO:teuthology.orchestra.run.smithi137.stdout: "op_reply": 2340, 2024-04-24T06:55:18.291 INFO:teuthology.orchestra.run.smithi137.stdout: "op_latency": { 2024-04-24T06:55:18.292 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2340, 2024-04-24T06:55:18.292 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 1.423974188, 2024-04-24T06:55:18.292 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000608535 2024-04-24T06:55:18.292 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.292 INFO:teuthology.orchestra.run.smithi137.stdout: "op_inflight": 0, 2024-04-24T06:55:18.292 INFO:teuthology.orchestra.run.smithi137.stdout: "oplen_avg": { 2024-04-24T06:55:18.292 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2340, 2024-04-24T06:55:18.292 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 2341 2024-04-24T06:55:18.293 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.293 INFO:teuthology.orchestra.run.smithi137.stdout: "op": 2340, 2024-04-24T06:55:18.293 INFO:teuthology.orchestra.run.smithi137.stdout: "op_r": 2340, 2024-04-24T06:55:18.293 INFO:teuthology.orchestra.run.smithi137.stdout: "op_w": 0, 2024-04-24T06:55:18.293 INFO:teuthology.orchestra.run.smithi137.stdout: "op_rmw": 0, 2024-04-24T06:55:18.293 INFO:teuthology.orchestra.run.smithi137.stdout: "op_pg": 0, 2024-04-24T06:55:18.293 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_stat": 1492, 2024-04-24T06:55:18.293 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_create": 0, 2024-04-24T06:55:18.294 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_read": 846, 2024-04-24T06:55:18.294 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_write": 0, 2024-04-24T06:55:18.294 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writefull": 0, 2024-04-24T06:55:18.294 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writesame": 0, 2024-04-24T06:55:18.294 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_append": 0, 2024-04-24T06:55:18.294 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_zero": 0, 2024-04-24T06:55:18.294 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_truncate": 0, 2024-04-24T06:55:18.294 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_delete": 0, 2024-04-24T06:55:18.294 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_mapext": 0, 2024-04-24T06:55:18.295 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_sparse_read": 0, 2024-04-24T06:55:18.295 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_clonerange": 0, 2024-04-24T06:55:18.295 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_getxattr": 1, 2024-04-24T06:55:18.295 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_setxattr": 0, 2024-04-24T06:55:18.295 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_cmpxattr": 0, 2024-04-24T06:55:18.295 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_rmxattr": 0, 2024-04-24T06:55:18.295 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_resetxattrs": 0, 2024-04-24T06:55:18.295 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_call": 0, 2024-04-24T06:55:18.296 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_watch": 0, 2024-04-24T06:55:18.296 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_notify": 0, 2024-04-24T06:55:18.296 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T06:55:18.296 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls": 0, 2024-04-24T06:55:18.296 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls_filter": 0, 2024-04-24T06:55:18.296 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_other": 0, 2024-04-24T06:55:18.296 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_active": 0, 2024-04-24T06:55:18.296 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_send": 0, 2024-04-24T06:55:18.296 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_resend": 0, 2024-04-24T06:55:18.297 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_ping": 0, 2024-04-24T06:55:18.297 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_active": 0, 2024-04-24T06:55:18.297 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_send": 0, 2024-04-24T06:55:18.297 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_resend": 0, 2024-04-24T06:55:18.297 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_active": 0, 2024-04-24T06:55:18.297 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_send": 0, 2024-04-24T06:55:18.297 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_resend": 0, 2024-04-24T06:55:18.297 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_active": 0, 2024-04-24T06:55:18.298 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_send": 0, 2024-04-24T06:55:18.298 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_resend": 0, 2024-04-24T06:55:18.298 INFO:teuthology.orchestra.run.smithi137.stdout: "command_active": 0, 2024-04-24T06:55:18.298 INFO:teuthology.orchestra.run.smithi137.stdout: "command_send": 0, 2024-04-24T06:55:18.298 INFO:teuthology.orchestra.run.smithi137.stdout: "command_resend": 0, 2024-04-24T06:55:18.298 INFO:teuthology.orchestra.run.smithi137.stdout: "map_epoch": 99, 2024-04-24T06:55:18.298 INFO:teuthology.orchestra.run.smithi137.stdout: "map_full": 0, 2024-04-24T06:55:18.298 INFO:teuthology.orchestra.run.smithi137.stdout: "map_inc": 13, 2024-04-24T06:55:18.299 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_sessions": 4, 2024-04-24T06:55:18.299 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_open": 4, 2024-04-24T06:55:18.299 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_close": 0, 2024-04-24T06:55:18.299 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_laggy": 0, 2024-04-24T06:55:18.299 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_wr": 0, 2024-04-24T06:55:18.299 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_rd": 2, 2024-04-24T06:55:18.299 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_del": 0 2024-04-24T06:55:18.299 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.299 INFO:teuthology.orchestra.run.smithi137.stdout: "oft": { 2024-04-24T06:55:18.300 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_objs": 0, 2024-04-24T06:55:18.300 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_kv_pairs": 0, 2024-04-24T06:55:18.300 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_updates": 0, 2024-04-24T06:55:18.300 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_removes": 0 2024-04-24T06:55:18.300 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.300 INFO:teuthology.orchestra.run.smithi137.stdout: "osd-slow-ops": { 2024-04-24T06:55:18.300 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_ops_count": 0 2024-04-24T06:55:18.300 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.301 INFO:teuthology.orchestra.run.smithi137.stdout: "purge_queue": { 2024-04-24T06:55:18.301 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops": 0, 2024-04-24T06:55:18.301 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops_high_water": 0, 2024-04-24T06:55:18.301 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing": 0, 2024-04-24T06:55:18.301 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_high_water": 0, 2024-04-24T06:55:18.301 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed_ops": 0, 2024-04-24T06:55:18.301 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed": 0, 2024-04-24T06:55:18.301 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_item_in_journal": 0 2024-04-24T06:55:18.301 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.302 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T06:55:18.302 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-24T06:55:18.302 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-04-24T06:55:18.302 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-24T06:55:18.302 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 2604, 2024-04-24T06:55:18.302 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 8102361, 2024-04-24T06:55:18.302 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-24T06:55:18.302 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 2604, 2024-04-24T06:55:18.303 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-04-24T06:55:18.303 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-04-24T06:55:18.303 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 2604, 2024-04-24T06:55:18.303 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 8102361, 2024-04-24T06:55:18.303 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-24T06:55:18.303 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:18.303 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:18.303 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:18.304 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-24T06:55:18.304 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.304 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_bytes": { 2024-04-24T06:55:18.304 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-24T06:55:18.304 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-04-24T06:55:18.304 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-24T06:55:18.304 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-04-24T06:55:18.304 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-04-24T06:55:18.304 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-24T06:55:18.305 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-04-24T06:55:18.305 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 2340, 2024-04-24T06:55:18.305 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 5910568, 2024-04-24T06:55:18.305 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 2340, 2024-04-24T06:55:18.305 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 5910568, 2024-04-24T06:55:18.305 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-24T06:55:18.305 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:18.305 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:18.306 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:18.306 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-24T06:55:18.306 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.306 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_ops": { 2024-04-24T06:55:18.306 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-24T06:55:18.306 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 1024, 2024-04-24T06:55:18.306 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-24T06:55:18.306 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-04-24T06:55:18.307 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-04-24T06:55:18.307 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-24T06:55:18.307 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-04-24T06:55:18.307 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 2340, 2024-04-24T06:55:18.307 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 2340, 2024-04-24T06:55:18.307 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 2340, 2024-04-24T06:55:18.307 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 2340, 2024-04-24T06:55:18.307 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-24T06:55:18.308 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:18.308 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:18.308 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:18.308 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-24T06:55:18.308 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.308 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle": { 2024-04-24T06:55:18.308 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-24T06:55:18.308 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-04-24T06:55:18.309 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-24T06:55:18.309 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-04-24T06:55:18.309 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-04-24T06:55:18.309 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-24T06:55:18.309 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-04-24T06:55:18.309 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-04-24T06:55:18.309 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-04-24T06:55:18.309 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 0, 2024-04-24T06:55:18.309 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 0, 2024-04-24T06:55:18.310 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-24T06:55:18.310 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:18.310 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:18.310 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:18.310 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-24T06:55:18.310 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:18.310 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle-0x55e713c246e0": { 2024-04-24T06:55:18.310 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-24T06:55:18.311 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-04-24T06:55:18.311 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-24T06:55:18.311 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-04-24T06:55:18.311 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-04-24T06:55:18.311 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-24T06:55:18.311 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-04-24T06:55:18.311 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-04-24T06:55:18.311 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-04-24T06:55:18.311 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 0, 2024-04-24T06:55:18.312 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 0, 2024-04-24T06:55:18.312 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-24T06:55:18.312 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:18.312 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:18.312 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:18.312 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-24T06:55:18.312 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-24T06:55:18.312 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-24T06:55:19.025 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-24T06:55:19.025 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-24T06:55:19.026 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-04-24T06:55:19.026 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-24T06:55:19.026 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-24T06:55:19.026 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-04-24T06:55:19.026 DEBUG:tasks.check_counter:Getting stats from i 2024-04-24T06:55:19.026 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-24T06:55:19.265 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T06:55:19.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:19 smithi005 ceph-mon[27798]: pgmap v351: 97 pgs: 97 active+clean; 85 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 103 B/s wr, 6 op/s 2024-04-24T06:55:19.642 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:19 smithi137 ceph-mon[33017]: pgmap v351: 97 pgs: 97 active+clean; 85 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 103 B/s wr, 6 op/s 2024-04-24T06:55:19.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:19 smithi037 ceph-mon[32864]: pgmap v351: 97 pgs: 97 active+clean; 85 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 103 B/s wr, 6 op/s 2024-04-24T06:55:20.902 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-24T06:55:20.903 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T06:55:20.903 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 3698, 2024-04-24T06:55:20.903 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 4280, 2024-04-24T06:55:20.903 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 1908303, 2024-04-24T06:55:20.903 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 7784424, 2024-04-24T06:55:20.903 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 12, 2024-04-24T06:55:20.904 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 5, 2024-04-24T06:55:20.904 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.261689719, 2024-04-24T06:55:20.904 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.102921709, 2024-04-24T06:55:20.904 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.109251725, 2024-04-24T06:55:20.904 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.046359321, 2024-04-24T06:55:20.904 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T06:55:20.904 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 4279, 2024-04-24T06:55:20.904 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.124866525, 2024-04-24T06:55:20.904 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000029181 2024-04-24T06:55:20.905 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.905 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-04-24T06:55:20.905 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 3966, 2024-04-24T06:55:20.905 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000655897, 2024-04-24T06:55:20.905 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000165 2024-04-24T06:55:20.905 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.905 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 4752, 2024-04-24T06:55:20.905 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 6320 2024-04-24T06:55:20.905 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.906 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T06:55:20.906 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 7097, 2024-04-24T06:55:20.906 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 8827, 2024-04-24T06:55:20.906 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 2099072, 2024-04-24T06:55:20.906 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 5098210, 2024-04-24T06:55:20.906 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 51, 2024-04-24T06:55:20.906 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 8, 2024-04-24T06:55:20.906 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.344206716, 2024-04-24T06:55:20.907 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.132694774, 2024-04-24T06:55:20.907 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.198768079, 2024-04-24T06:55:20.907 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.053735239, 2024-04-24T06:55:20.907 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T06:55:20.907 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 8824, 2024-04-24T06:55:20.907 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.174032290, 2024-04-24T06:55:20.907 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000019722 2024-04-24T06:55:20.908 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.908 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-04-24T06:55:20.908 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 8257, 2024-04-24T06:55:20.908 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.001179482, 2024-04-24T06:55:20.908 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000142 2024-04-24T06:55:20.908 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.908 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 2160, 2024-04-24T06:55:20.908 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 472912 2024-04-24T06:55:20.909 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.909 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T06:55:20.909 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 2462, 2024-04-24T06:55:20.909 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 2398, 2024-04-24T06:55:20.909 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 2395405, 2024-04-24T06:55:20.909 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 1126846, 2024-04-24T06:55:20.909 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 8, 2024-04-24T06:55:20.909 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 6, 2024-04-24T06:55:20.910 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.180230337, 2024-04-24T06:55:20.910 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.062316704, 2024-04-24T06:55:20.910 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.086320594, 2024-04-24T06:55:20.910 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.046378351, 2024-04-24T06:55:20.910 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T06:55:20.910 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2397, 2024-04-24T06:55:20.911 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.065112851, 2024-04-24T06:55:20.911 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000027164 2024-04-24T06:55:20.911 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.911 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-04-24T06:55:20.911 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 160, 2024-04-24T06:55:20.911 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000024799, 2024-04-24T06:55:20.911 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000154 2024-04-24T06:55:20.911 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.912 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 1796944, 2024-04-24T06:55:20.912 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 172064 2024-04-24T06:55:20.912 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.912 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2024-04-24T06:55:20.912 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 1, 2024-04-24T06:55:20.912 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2024-04-24T06:55:20.912 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.913 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-MDSRank": { 2024-04-24T06:55:20.913 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-04-24T06:55:20.913 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-04-24T06:55:20.913 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 4950, 2024-04-24T06:55:20.913 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 1.755989333, 2024-04-24T06:55:20.913 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000354745 2024-04-24T06:55:20.913 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-24T06:55:20.913 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.914 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-PurgeQueue": { 2024-04-24T06:55:20.914 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-04-24T06:55:20.914 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-04-24T06:55:20.914 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 818, 2024-04-24T06:55:20.914 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.036291497, 2024-04-24T06:55:20.914 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000044366 2024-04-24T06:55:20.914 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-24T06:55:20.915 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.915 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-04-24T06:55:20.915 INFO:teuthology.orchestra.run.smithi137.stdout: "request": 1756, 2024-04-24T06:55:20.915 INFO:teuthology.orchestra.run.smithi137.stdout: "reply": 1749, 2024-04-24T06:55:20.915 INFO:teuthology.orchestra.run.smithi137.stdout: "reply_latency": { 2024-04-24T06:55:20.915 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1749, 2024-04-24T06:55:20.915 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 1.348690566, 2024-04-24T06:55:20.915 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000771120 2024-04-24T06:55:20.916 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.916 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_reply": 0, 2024-04-24T06:55:20.916 INFO:teuthology.orchestra.run.smithi137.stdout: "forward": 7, 2024-04-24T06:55:20.916 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_complete": 0, 2024-04-24T06:55:20.916 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_keys": 0, 2024-04-24T06:55:20.916 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_commit": 75, 2024-04-24T06:55:20.916 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_split": 8, 2024-04-24T06:55:20.916 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_merge": 0, 2024-04-24T06:55:20.917 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes": 1316, 2024-04-24T06:55:20.917 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_top": 139, 2024-04-24T06:55:20.917 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_bottom": 1177, 2024-04-24T06:55:20.917 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pin_tail": 0, 2024-04-24T06:55:20.917 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pinned": 520, 2024-04-24T06:55:20.917 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_expired": 1, 2024-04-24T06:55:20.917 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_with_caps": 6, 2024-04-24T06:55:20.917 INFO:teuthology.orchestra.run.smithi137.stdout: "caps": 6, 2024-04-24T06:55:20.918 INFO:teuthology.orchestra.run.smithi137.stdout: "subtrees": 69, 2024-04-24T06:55:20.918 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse": 3522, 2024-04-24T06:55:20.918 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_hit": 2821, 2024-04-24T06:55:20.918 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_forward": 7, 2024-04-24T06:55:20.918 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_discover": 12, 2024-04-24T06:55:20.918 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_dir_fetch": 0, 2024-04-24T06:55:20.918 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_remote_ino": 0, 2024-04-24T06:55:20.918 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_lock": 0, 2024-04-24T06:55:20.919 INFO:teuthology.orchestra.run.smithi137.stdout: "load_cent": 0, 2024-04-24T06:55:20.919 INFO:teuthology.orchestra.run.smithi137.stdout: "q": 0, 2024-04-24T06:55:20.919 INFO:teuthology.orchestra.run.smithi137.stdout: "exported": 0, 2024-04-24T06:55:20.919 INFO:teuthology.orchestra.run.smithi137.stdout: "exported_inodes": 0, 2024-04-24T06:55:20.919 INFO:teuthology.orchestra.run.smithi137.stdout: "imported": 10, 2024-04-24T06:55:20.919 INFO:teuthology.orchestra.run.smithi137.stdout: "imported_inodes": 647, 2024-04-24T06:55:20.919 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_dir_fetch": 0, 2024-04-24T06:55:20.919 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_backtrace_fetch": 6150, 2024-04-24T06:55:20.920 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_peer_discover": 0, 2024-04-24T06:55:20.920 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rfiles": 3454, 2024-04-24T06:55:20.920 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rbytes": 26624153, 2024-04-24T06:55:20.920 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rsnaps": 6, 2024-04-24T06:55:20.920 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_fetch": 6150, 2024-04-24T06:55:20.920 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_set_tag": 0, 2024-04-24T06:55:20.920 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T06:55:20.920 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_inotable_repaired": 0, 2024-04-24T06:55:20.921 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_inodes": 45, 2024-04-24T06:55:20.921 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T06:55:20.921 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dirfrag_rstats": 45, 2024-04-24T06:55:20.921 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_file_inodes": 6105, 2024-04-24T06:55:20.921 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_inode_file_caps": 0, 2024-04-24T06:55:20.922 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_revoke": 93, 2024-04-24T06:55:20.922 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_grant": 1122, 2024-04-24T06:55:20.922 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T06:55:20.922 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T06:55:20.922 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T06:55:20.922 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps": 2601, 2024-04-24T06:55:20.922 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps_dirty": 2057, 2024-04-24T06:55:20.923 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_cap_release": 456, 2024-04-24T06:55:20.923 INFO:teuthology.orchestra.run.smithi137.stdout: "process_request_cap_release": 903 2024-04-24T06:55:20.923 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.923 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_cache": { 2024-04-24T06:55:20.924 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update": 0, 2024-04-24T06:55:20.924 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update_receipt": 0, 2024-04-24T06:55:20.924 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_try_discover": 0, 2024-04-24T06:55:20.924 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_send_discover": 6, 2024-04-24T06:55:20.924 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_handle_discover": 1, 2024-04-24T06:55:20.924 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays": 0, 2024-04-24T06:55:20.924 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_delayed": 0, 2024-04-24T06:55:20.925 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_enqueuing": 0, 2024-04-24T06:55:20.925 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_created": 451, 2024-04-24T06:55:20.926 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_enqueued": 451, 2024-04-24T06:55:20.926 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_reintegrated": 0, 2024-04-24T06:55:20.926 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_migrated": 0, 2024-04-24T06:55:20.926 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_processing": 0, 2024-04-24T06:55:20.926 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_enqueued": 0, 2024-04-24T06:55:20.927 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_prioritized": 0, 2024-04-24T06:55:20.927 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_started": 0, 2024-04-24T06:55:20.927 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_completed": 0, 2024-04-24T06:55:20.927 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_quiesce_path": 0, 2024-04-24T06:55:20.927 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_quiesce_inode": 0, 2024-04-24T06:55:20.927 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T06:55:20.928 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_exportdir": 0, 2024-04-24T06:55:20.928 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_flush": 0, 2024-04-24T06:55:20.928 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragmentdir": 8, 2024-04-24T06:55:20.928 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragstats": 0, 2024-04-24T06:55:20.928 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_inodestats": 0 2024-04-24T06:55:20.929 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.929 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_log": { 2024-04-24T06:55:20.929 INFO:teuthology.orchestra.run.smithi137.stdout: "evlrg": 0, 2024-04-24T06:55:20.929 INFO:teuthology.orchestra.run.smithi137.stdout: "evadd": 3349, 2024-04-24T06:55:20.930 INFO:teuthology.orchestra.run.smithi137.stdout: "evex": 0, 2024-04-24T06:55:20.930 INFO:teuthology.orchestra.run.smithi137.stdout: "evtrm": 0, 2024-04-24T06:55:20.930 INFO:teuthology.orchestra.run.smithi137.stdout: "ev": 3349, 2024-04-24T06:55:20.930 INFO:teuthology.orchestra.run.smithi137.stdout: "evexg": 0, 2024-04-24T06:55:20.931 INFO:teuthology.orchestra.run.smithi137.stdout: "evexd": 0, 2024-04-24T06:55:20.931 INFO:teuthology.orchestra.run.smithi137.stdout: "segadd": 5, 2024-04-24T06:55:20.931 INFO:teuthology.orchestra.run.smithi137.stdout: "segex": 0, 2024-04-24T06:55:20.931 INFO:teuthology.orchestra.run.smithi137.stdout: "segtrm": 0, 2024-04-24T06:55:20.931 INFO:teuthology.orchestra.run.smithi137.stdout: "seg": 5, 2024-04-24T06:55:20.931 INFO:teuthology.orchestra.run.smithi137.stdout: "segmjr": 2, 2024-04-24T06:55:20.932 INFO:teuthology.orchestra.run.smithi137.stdout: "segexg": 0, 2024-04-24T06:55:20.932 INFO:teuthology.orchestra.run.smithi137.stdout: "segexd": 0, 2024-04-24T06:55:20.932 INFO:teuthology.orchestra.run.smithi137.stdout: "expos": 4194304, 2024-04-24T06:55:20.932 INFO:teuthology.orchestra.run.smithi137.stdout: "wrpos": 10103172, 2024-04-24T06:55:20.932 INFO:teuthology.orchestra.run.smithi137.stdout: "rdpos": 0, 2024-04-24T06:55:20.933 INFO:teuthology.orchestra.run.smithi137.stdout: "jlat": { 2024-04-24T06:55:20.933 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 45, 2024-04-24T06:55:20.933 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.418080483, 2024-04-24T06:55:20.933 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.009290677 2024-04-24T06:55:20.933 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.933 INFO:teuthology.orchestra.run.smithi137.stdout: "replayed": 0 2024-04-24T06:55:20.934 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.934 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_mem": { 2024-04-24T06:55:20.934 INFO:teuthology.orchestra.run.smithi137.stdout: "ino": 1080, 2024-04-24T06:55:20.934 INFO:teuthology.orchestra.run.smithi137.stdout: "ino+": 1531, 2024-04-24T06:55:20.934 INFO:teuthology.orchestra.run.smithi137.stdout: "ino-": 451, 2024-04-24T06:55:20.934 INFO:teuthology.orchestra.run.smithi137.stdout: "dir": 94, 2024-04-24T06:55:20.934 INFO:teuthology.orchestra.run.smithi137.stdout: "dir+": 103, 2024-04-24T06:55:20.934 INFO:teuthology.orchestra.run.smithi137.stdout: "dir-": 9, 2024-04-24T06:55:20.935 INFO:teuthology.orchestra.run.smithi137.stdout: "dn": 1316, 2024-04-24T06:55:20.935 INFO:teuthology.orchestra.run.smithi137.stdout: "dn+": 1768, 2024-04-24T06:55:20.935 INFO:teuthology.orchestra.run.smithi137.stdout: "dn-": 452, 2024-04-24T06:55:20.935 INFO:teuthology.orchestra.run.smithi137.stdout: "cap": 6, 2024-04-24T06:55:20.935 INFO:teuthology.orchestra.run.smithi137.stdout: "cap+": 864, 2024-04-24T06:55:20.935 INFO:teuthology.orchestra.run.smithi137.stdout: "cap-": 858, 2024-04-24T06:55:20.935 INFO:teuthology.orchestra.run.smithi137.stdout: "rss": 75284, 2024-04-24T06:55:20.935 INFO:teuthology.orchestra.run.smithi137.stdout: "heap": 223608 2024-04-24T06:55:20.936 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.936 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_server": { 2024-04-24T06:55:20.936 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_client_request": 2434, 2024-04-24T06:55:20.936 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_server_request": 0, 2024-04-24T06:55:20.936 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_request": 1756, 2024-04-24T06:55:20.936 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_session": 25, 2024-04-24T06:55:20.936 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_peer_request": 676, 2024-04-24T06:55:20.936 INFO:teuthology.orchestra.run.smithi137.stdout: "req_create_latency": { 2024-04-24T06:55:20.937 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 216, 2024-04-24T06:55:20.937 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.248197385, 2024-04-24T06:55:20.937 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001149061 2024-04-24T06:55:20.937 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.937 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getattr_latency": { 2024-04-24T06:55:20.937 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1, 2024-04-24T06:55:20.937 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.001776479, 2024-04-24T06:55:20.938 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001776479 2024-04-24T06:55:20.938 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.938 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getfilelock_latency": { 2024-04-24T06:55:20.938 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:20.938 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:20.938 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:20.938 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.938 INFO:teuthology.orchestra.run.smithi137.stdout: "req_link_latency": { 2024-04-24T06:55:20.939 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:20.939 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:20.939 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:20.939 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.939 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookup_latency": { 2024-04-24T06:55:20.939 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1080, 2024-04-24T06:55:20.939 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.307730808, 2024-04-24T06:55:20.939 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000284935 2024-04-24T06:55:20.940 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.940 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookuphash_latency": { 2024-04-24T06:55:20.940 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:20.940 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:20.940 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:20.940 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.940 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupino_latency": { 2024-04-24T06:55:20.940 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:20.941 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:20.941 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:20.941 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.941 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupname_latency": { 2024-04-24T06:55:20.941 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:20.941 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:20.941 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:20.942 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.942 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupparent_latency": { 2024-04-24T06:55:20.942 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:20.942 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:20.942 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:20.942 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.942 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupsnap_latency": { 2024-04-24T06:55:20.942 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:20.943 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:20.943 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:20.943 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.943 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lssnap_latency": { 2024-04-24T06:55:20.943 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:20.944 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:20.944 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:20.944 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.944 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mkdir_latency": { 2024-04-24T06:55:20.944 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:20.945 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:20.945 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:20.945 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.945 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mknod_latency": { 2024-04-24T06:55:20.945 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:20.945 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:20.945 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:20.946 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.946 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mksnap_latency": { 2024-04-24T06:55:20.946 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:20.946 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:20.946 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:20.946 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.946 INFO:teuthology.orchestra.run.smithi137.stdout: "req_open_latency": { 2024-04-24T06:55:20.946 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1, 2024-04-24T06:55:20.947 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000278526, 2024-04-24T06:55:20.947 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000278526 2024-04-24T06:55:20.947 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.947 INFO:teuthology.orchestra.run.smithi137.stdout: "req_readdir_latency": { 2024-04-24T06:55:20.947 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:20.947 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:20.947 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:20.947 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.948 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rename_latency": { 2024-04-24T06:55:20.948 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:20.948 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:20.948 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:20.948 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.948 INFO:teuthology.orchestra.run.smithi137.stdout: "req_renamesnap_latency": { 2024-04-24T06:55:20.948 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:20.948 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:20.949 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:20.949 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.949 INFO:teuthology.orchestra.run.smithi137.stdout: "req_snapdiff_latency": { 2024-04-24T06:55:20.949 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:20.949 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:20.949 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:20.949 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.949 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmdir_latency": { 2024-04-24T06:55:20.950 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:20.950 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:20.950 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:20.950 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.950 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmsnap_latency": { 2024-04-24T06:55:20.950 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:20.950 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:20.950 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:20.951 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.951 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmxattr_latency": { 2024-04-24T06:55:20.951 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:20.951 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:20.951 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:20.951 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.951 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setattr_latency": { 2024-04-24T06:55:20.951 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:20.951 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:20.952 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:20.952 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.952 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setdirlayout_latency": { 2024-04-24T06:55:20.952 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:20.952 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:20.953 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:20.953 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.953 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setfilelock_latency": { 2024-04-24T06:55:20.953 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:20.953 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:20.953 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:20.953 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.953 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setlayout_latency": { 2024-04-24T06:55:20.953 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:20.954 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:20.954 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:20.954 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.954 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setxattr_latency": { 2024-04-24T06:55:20.954 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:20.955 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:20.955 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:20.955 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.955 INFO:teuthology.orchestra.run.smithi137.stdout: "req_symlink_latency": { 2024-04-24T06:55:20.955 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:20.955 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:20.955 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:20.956 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.956 INFO:teuthology.orchestra.run.smithi137.stdout: "req_unlink_latency": { 2024-04-24T06:55:20.956 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 451, 2024-04-24T06:55:20.956 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.790707368, 2024-04-24T06:55:20.956 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001753231 2024-04-24T06:55:20.956 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.957 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_revoke_eviction": 0, 2024-04-24T06:55:20.957 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_acquisition_throttle": 0, 2024-04-24T06:55:20.957 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getvxattr_latency": { 2024-04-24T06:55:20.957 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:20.957 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:20.957 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:20.957 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-24T06:55:20.957 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.958 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_sessions": { 2024-04-24T06:55:20.958 INFO:teuthology.orchestra.run.smithi137.stdout: "session_count": 2, 2024-04-24T06:55:20.958 INFO:teuthology.orchestra.run.smithi137.stdout: "session_add": 4, 2024-04-24T06:55:20.958 INFO:teuthology.orchestra.run.smithi137.stdout: "session_remove": 2, 2024-04-24T06:55:20.958 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_open": 2, 2024-04-24T06:55:20.959 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_stale": 0, 2024-04-24T06:55:20.959 INFO:teuthology.orchestra.run.smithi137.stdout: "total_load": 1680, 2024-04-24T06:55:20.959 INFO:teuthology.orchestra.run.smithi137.stdout: "average_load": 840, 2024-04-24T06:55:20.959 INFO:teuthology.orchestra.run.smithi137.stdout: "avg_session_uptime": 616, 2024-04-24T06:55:20.959 INFO:teuthology.orchestra.run.smithi137.stdout: "mdthresh_evicted": 0 2024-04-24T06:55:20.959 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.959 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2024-04-24T06:55:20.959 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2024-04-24T06:55:20.960 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2024-04-24T06:55:20.960 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T06:55:20.960 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2024-04-24T06:55:20.960 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T06:55:20.960 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2024-04-24T06:55:20.961 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T06:55:20.961 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T06:55:20.961 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T06:55:20.961 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T06:55:20.961 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T06:55:20.961 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2024-04-24T06:55:20.961 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T06:55:20.962 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T06:55:20.962 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2024-04-24T06:55:20.962 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2024-04-24T06:55:20.962 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2024-04-24T06:55:20.962 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2024-04-24T06:55:20.963 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T06:55:20.963 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T06:55:20.963 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T06:55:20.963 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T06:55:20.963 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T06:55:20.963 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2024-04-24T06:55:20.964 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2024-04-24T06:55:20.964 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2024-04-24T06:55:20.964 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T06:55:20.964 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T06:55:20.964 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2024-04-24T06:55:20.965 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2024-04-24T06:55:20.965 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2024-04-24T06:55:20.965 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2024-04-24T06:55:20.965 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T06:55:20.965 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2024-04-24T06:55:20.966 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T06:55:20.966 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2024-04-24T06:55:20.966 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 331430, 2024-04-24T06:55:20.966 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 999, 2024-04-24T06:55:20.966 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 528, 2024-04-24T06:55:20.966 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 6, 2024-04-24T06:55:20.966 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2024-04-24T06:55:20.967 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2024-04-24T06:55:20.967 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2024-04-24T06:55:20.967 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2024-04-24T06:55:20.967 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2024-04-24T06:55:20.967 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2024-04-24T06:55:20.967 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 8128, 2024-04-24T06:55:20.968 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 169, 2024-04-24T06:55:20.968 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T06:55:20.968 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2024-04-24T06:55:20.968 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2024-04-24T06:55:20.968 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2024-04-24T06:55:20.968 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 3593971, 2024-04-24T06:55:20.968 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 38773, 2024-04-24T06:55:20.968 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2024-04-24T06:55:20.969 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2024-04-24T06:55:20.969 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2024-04-24T06:55:20.969 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2024-04-24T06:55:20.969 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.969 INFO:teuthology.orchestra.run.smithi137.stdout: "objecter": { 2024-04-24T06:55:20.969 INFO:teuthology.orchestra.run.smithi137.stdout: "op_active": 0, 2024-04-24T06:55:20.969 INFO:teuthology.orchestra.run.smithi137.stdout: "op_laggy": 0, 2024-04-24T06:55:20.969 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send": 6800, 2024-04-24T06:55:20.970 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send_bytes": 6702344, 2024-04-24T06:55:20.970 INFO:teuthology.orchestra.run.smithi137.stdout: "op_resend": 33, 2024-04-24T06:55:20.970 INFO:teuthology.orchestra.run.smithi137.stdout: "op_reply": 6800, 2024-04-24T06:55:20.970 INFO:teuthology.orchestra.run.smithi137.stdout: "op_latency": { 2024-04-24T06:55:20.970 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 6800, 2024-04-24T06:55:20.970 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 6.558881477, 2024-04-24T06:55:20.970 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000964541 2024-04-24T06:55:20.970 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.971 INFO:teuthology.orchestra.run.smithi137.stdout: "op_inflight": 0, 2024-04-24T06:55:20.971 INFO:teuthology.orchestra.run.smithi137.stdout: "oplen_avg": { 2024-04-24T06:55:20.971 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 6800, 2024-04-24T06:55:20.971 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 6907 2024-04-24T06:55:20.971 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.971 INFO:teuthology.orchestra.run.smithi137.stdout: "op": 6800, 2024-04-24T06:55:20.971 INFO:teuthology.orchestra.run.smithi137.stdout: "op_r": 6156, 2024-04-24T06:55:20.971 INFO:teuthology.orchestra.run.smithi137.stdout: "op_w": 644, 2024-04-24T06:55:20.972 INFO:teuthology.orchestra.run.smithi137.stdout: "op_rmw": 0, 2024-04-24T06:55:20.972 INFO:teuthology.orchestra.run.smithi137.stdout: "op_pg": 0, 2024-04-24T06:55:20.972 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_stat": 0, 2024-04-24T06:55:20.972 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_create": 10, 2024-04-24T06:55:20.972 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_read": 6, 2024-04-24T06:55:20.972 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_write": 51, 2024-04-24T06:55:20.972 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writefull": 15, 2024-04-24T06:55:20.972 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writesame": 0, 2024-04-24T06:55:20.973 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_append": 0, 2024-04-24T06:55:20.973 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_zero": 0, 2024-04-24T06:55:20.973 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_truncate": 0, 2024-04-24T06:55:20.973 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_delete": 472, 2024-04-24T06:55:20.973 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_mapext": 0, 2024-04-24T06:55:20.973 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_sparse_read": 0, 2024-04-24T06:55:20.973 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_clonerange": 0, 2024-04-24T06:55:20.973 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_getxattr": 6150, 2024-04-24T06:55:20.974 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_setxattr": 20, 2024-04-24T06:55:20.974 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_cmpxattr": 0, 2024-04-24T06:55:20.974 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_rmxattr": 0, 2024-04-24T06:55:20.974 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_resetxattrs": 0, 2024-04-24T06:55:20.974 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_call": 0, 2024-04-24T06:55:20.974 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_watch": 0, 2024-04-24T06:55:20.974 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_notify": 0, 2024-04-24T06:55:20.974 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T06:55:20.975 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls": 0, 2024-04-24T06:55:20.975 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls_filter": 0, 2024-04-24T06:55:20.975 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_other": 0, 2024-04-24T06:55:20.975 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_active": 0, 2024-04-24T06:55:20.975 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_send": 0, 2024-04-24T06:55:20.975 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_resend": 0, 2024-04-24T06:55:20.975 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_ping": 0, 2024-04-24T06:55:20.975 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_active": 0, 2024-04-24T06:55:20.976 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_send": 0, 2024-04-24T06:55:20.977 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_resend": 0, 2024-04-24T06:55:20.977 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_active": 0, 2024-04-24T06:55:20.977 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_send": 0, 2024-04-24T06:55:20.977 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_resend": 0, 2024-04-24T06:55:20.978 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_active": 0, 2024-04-24T06:55:20.978 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_send": 0, 2024-04-24T06:55:20.978 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_resend": 0, 2024-04-24T06:55:20.978 INFO:teuthology.orchestra.run.smithi137.stdout: "command_active": 0, 2024-04-24T06:55:20.978 INFO:teuthology.orchestra.run.smithi137.stdout: "command_send": 0, 2024-04-24T06:55:20.978 INFO:teuthology.orchestra.run.smithi137.stdout: "command_resend": 0, 2024-04-24T06:55:20.978 INFO:teuthology.orchestra.run.smithi137.stdout: "map_epoch": 99, 2024-04-24T06:55:20.978 INFO:teuthology.orchestra.run.smithi137.stdout: "map_full": 0, 2024-04-24T06:55:20.979 INFO:teuthology.orchestra.run.smithi137.stdout: "map_inc": 13, 2024-04-24T06:55:20.979 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_sessions": 12, 2024-04-24T06:55:20.979 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_open": 12, 2024-04-24T06:55:20.979 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_close": 0, 2024-04-24T06:55:20.979 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_laggy": 0, 2024-04-24T06:55:20.979 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_wr": 175, 2024-04-24T06:55:20.979 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_rd": 0, 2024-04-24T06:55:20.979 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_del": 8 2024-04-24T06:55:20.980 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.980 INFO:teuthology.orchestra.run.smithi137.stdout: "oft": { 2024-04-24T06:55:20.980 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_objs": 1, 2024-04-24T06:55:20.980 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_kv_pairs": 0, 2024-04-24T06:55:20.980 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_updates": 0, 2024-04-24T06:55:20.980 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_removes": 0 2024-04-24T06:55:20.980 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.980 INFO:teuthology.orchestra.run.smithi137.stdout: "osd-slow-ops": { 2024-04-24T06:55:20.981 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_ops_count": 0 2024-04-24T06:55:20.981 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.981 INFO:teuthology.orchestra.run.smithi137.stdout: "purge_queue": { 2024-04-24T06:55:20.981 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops": 0, 2024-04-24T06:55:20.981 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops_high_water": 10, 2024-04-24T06:55:20.981 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing": 0, 2024-04-24T06:55:20.981 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_high_water": 10, 2024-04-24T06:55:20.981 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed_ops": 451, 2024-04-24T06:55:20.982 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed": 451, 2024-04-24T06:55:20.982 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_item_in_journal": 0 2024-04-24T06:55:20.982 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.982 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T06:55:20.982 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-24T06:55:20.982 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-04-24T06:55:20.982 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-24T06:55:20.982 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 13253, 2024-04-24T06:55:20.982 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 5189592, 2024-04-24T06:55:20.983 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-24T06:55:20.983 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 13253, 2024-04-24T06:55:20.983 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-04-24T06:55:20.983 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-04-24T06:55:20.983 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 13253, 2024-04-24T06:55:20.983 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 5189592, 2024-04-24T06:55:20.983 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-24T06:55:20.983 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:20.983 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:20.984 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:20.984 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-24T06:55:20.984 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.984 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_bytes": { 2024-04-24T06:55:20.984 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-24T06:55:20.984 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-04-24T06:55:20.984 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-24T06:55:20.984 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-04-24T06:55:20.985 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-04-24T06:55:20.985 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-24T06:55:20.985 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-04-24T06:55:20.985 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 6800, 2024-04-24T06:55:20.985 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 6751503, 2024-04-24T06:55:20.985 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 6800, 2024-04-24T06:55:20.985 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 6751503, 2024-04-24T06:55:20.985 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-24T06:55:20.985 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:20.986 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:20.986 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:20.986 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-24T06:55:20.986 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.986 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_ops": { 2024-04-24T06:55:20.986 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-24T06:55:20.986 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 1024, 2024-04-24T06:55:20.987 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-24T06:55:20.987 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-04-24T06:55:20.987 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-04-24T06:55:20.987 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-24T06:55:20.987 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-04-24T06:55:20.987 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 6800, 2024-04-24T06:55:20.988 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 6800, 2024-04-24T06:55:20.988 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 6800, 2024-04-24T06:55:20.988 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 6800, 2024-04-24T06:55:20.988 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-24T06:55:20.988 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:20.988 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:20.988 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:20.989 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-24T06:55:20.989 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.989 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle": { 2024-04-24T06:55:20.989 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-24T06:55:20.989 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-04-24T06:55:20.989 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-24T06:55:20.990 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 451, 2024-04-24T06:55:20.990 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 49159, 2024-04-24T06:55:20.990 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-24T06:55:20.990 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 451, 2024-04-24T06:55:20.990 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-04-24T06:55:20.990 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-04-24T06:55:20.990 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 6, 2024-04-24T06:55:20.991 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 49159, 2024-04-24T06:55:20.991 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-24T06:55:20.991 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:20.991 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:20.991 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:20.991 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-24T06:55:20.991 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:20.992 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle-0x555c7151e460": { 2024-04-24T06:55:20.992 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-24T06:55:20.992 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-04-24T06:55:20.992 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-24T06:55:20.992 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 3349, 2024-04-24T06:55:20.992 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 5908868, 2024-04-24T06:55:20.992 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-24T06:55:20.992 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 3349, 2024-04-24T06:55:20.993 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-04-24T06:55:20.993 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-04-24T06:55:20.993 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 45, 2024-04-24T06:55:20.993 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 5908868, 2024-04-24T06:55:20.993 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-24T06:55:20.993 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:20.993 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:20.993 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:20.993 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-24T06:55:20.994 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-24T06:55:20.994 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-24T06:55:21.646 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:21 smithi137 ceph-mon[33017]: pgmap v352: 97 pgs: 97 active+clean; 85 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-24T06:55:21.673 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-24T06:55:21.673 INFO:tasks.check_counter:Daemon mds.i mds.imported=10 2024-04-24T06:55:21.673 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-04-24T06:55:21.673 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-24T06:55:21.674 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-24T06:55:21.674 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=8 2024-04-24T06:55:21.674 DEBUG:tasks.check_counter:Getting stats from l 2024-04-24T06:55:21.674 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411 shell --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-24T06:55:21.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:21 smithi005 ceph-mon[27798]: pgmap v352: 97 pgs: 97 active+clean; 85 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-24T06:55:21.767 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:21 smithi037 ceph-mon[32864]: pgmap v352: 97 pgs: 97 active+clean; 85 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-24T06:55:21.912 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T06:55:23.458 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:23 smithi137 ceph-mon[33017]: pgmap v353: 97 pgs: 97 active+clean; 85 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-24T06:55:23.458 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-24T06:55:23.459 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T06:55:23.459 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 248, 2024-04-24T06:55:23.459 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 234, 2024-04-24T06:55:23.459 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 1737424, 2024-04-24T06:55:23.459 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 143712, 2024-04-24T06:55:23.459 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 4, 2024-04-24T06:55:23.459 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 0, 2024-04-24T06:55:23.459 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.050531243, 2024-04-24T06:55:23.460 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.015105292, 2024-04-24T06:55:23.460 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.039040944, 2024-04-24T06:55:23.460 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.005761231, 2024-04-24T06:55:23.460 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T06:55:23.460 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 234, 2024-04-24T06:55:23.460 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.013191871, 2024-04-24T06:55:23.460 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000056375 2024-04-24T06:55:23.460 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:23.460 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-04-24T06:55:23.461 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:23.461 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:23.461 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:23.461 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:23.461 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 1737424, 2024-04-24T06:55:23.461 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 143712 2024-04-24T06:55:23.461 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:23.461 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T06:55:23.462 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 2, 2024-04-24T06:55:23.462 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 180, 2024-04-24T06:55:23.462 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 320, 2024-04-24T06:55:23.462 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 266352, 2024-04-24T06:55:23.462 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 6, 2024-04-24T06:55:23.462 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 3, 2024-04-24T06:55:23.462 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.012260167, 2024-04-24T06:55:23.462 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.008184775, 2024-04-24T06:55:23.463 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.001617654, 2024-04-24T06:55:23.463 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T06:55:23.463 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T06:55:23.463 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 177, 2024-04-24T06:55:23.463 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.028741876, 2024-04-24T06:55:23.463 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000162383 2024-04-24T06:55:23.463 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:23.463 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-04-24T06:55:23.464 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:23.464 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:23.464 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:23.464 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:23.464 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-24T06:55:23.464 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 266352 2024-04-24T06:55:23.464 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:23.464 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T06:55:23.465 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 0, 2024-04-24T06:55:23.465 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 2, 2024-04-24T06:55:23.465 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 0, 2024-04-24T06:55:23.466 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 0, 2024-04-24T06:55:23.466 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 2, 2024-04-24T06:55:23.466 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 1, 2024-04-24T06:55:23.466 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.000171780, 2024-04-24T06:55:23.466 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.000000000, 2024-04-24T06:55:23.466 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.000109706, 2024-04-24T06:55:23.466 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T06:55:23.467 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T06:55:23.467 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:23.467 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:23.467 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:23.467 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:23.468 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-04-24T06:55:23.468 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:23.468 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:23.468 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:23.468 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:23.468 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T06:55:23.468 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T06:55:23.468 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:23.469 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2024-04-24T06:55:23.469 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 0, 2024-04-24T06:55:23.469 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2024-04-24T06:55:23.469 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:23.469 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2024-04-24T06:55:23.469 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2024-04-24T06:55:23.470 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2024-04-24T06:55:23.470 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T06:55:23.470 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2024-04-24T06:55:23.470 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T06:55:23.470 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2024-04-24T06:55:23.470 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T06:55:23.471 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T06:55:23.471 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T06:55:23.471 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T06:55:23.471 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T06:55:23.471 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2024-04-24T06:55:23.471 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T06:55:23.471 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T06:55:23.472 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2024-04-24T06:55:23.472 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2024-04-24T06:55:23.472 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2024-04-24T06:55:23.472 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2024-04-24T06:55:23.472 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T06:55:23.472 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T06:55:23.472 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T06:55:23.473 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T06:55:23.473 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T06:55:23.473 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2024-04-24T06:55:23.473 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2024-04-24T06:55:23.473 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2024-04-24T06:55:23.473 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T06:55:23.474 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T06:55:23.474 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2024-04-24T06:55:23.474 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2024-04-24T06:55:23.474 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2024-04-24T06:55:23.474 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2024-04-24T06:55:23.474 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T06:55:23.475 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2024-04-24T06:55:23.475 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T06:55:23.475 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2024-04-24T06:55:23.475 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 148722, 2024-04-24T06:55:23.475 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 30, 2024-04-24T06:55:23.475 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 0, 2024-04-24T06:55:23.476 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 0, 2024-04-24T06:55:23.476 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2024-04-24T06:55:23.476 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2024-04-24T06:55:23.476 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2024-04-24T06:55:23.476 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2024-04-24T06:55:23.476 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2024-04-24T06:55:23.477 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2024-04-24T06:55:23.477 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 0, 2024-04-24T06:55:23.477 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 0, 2024-04-24T06:55:23.477 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T06:55:23.477 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2024-04-24T06:55:23.477 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2024-04-24T06:55:23.477 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2024-04-24T06:55:23.478 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 928, 2024-04-24T06:55:23.478 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 2, 2024-04-24T06:55:23.478 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2024-04-24T06:55:23.478 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2024-04-24T06:55:23.478 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2024-04-24T06:55:23.479 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2024-04-24T06:55:23.479 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-24T06:55:23.479 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T06:55:23.479 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-24T06:55:23.479 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-04-24T06:55:23.480 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-24T06:55:23.480 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 246, 2024-04-24T06:55:23.480 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 1699982, 2024-04-24T06:55:23.480 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-24T06:55:23.481 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 246, 2024-04-24T06:55:23.481 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-04-24T06:55:23.481 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-04-24T06:55:23.481 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 246, 2024-04-24T06:55:23.481 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 1699982, 2024-04-24T06:55:23.481 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-24T06:55:23.482 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-24T06:55:23.482 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-24T06:55:23.482 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-24T06:55:23.482 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-24T06:55:23.482 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-24T06:55:23.482 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-24T06:55:23.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:23 smithi005 ceph-mon[27798]: pgmap v353: 97 pgs: 97 active+clean; 85 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-24T06:55:23.767 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:23 smithi037 ceph-mon[32864]: pgmap v353: 97 pgs: 97 active+clean; 85 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-24T06:55:24.091 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-04-24T06:55:24.092 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-04-24T06:55:24.092 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-04-24T06:55:24.092 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-04-24T06:55:24.092 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-04-24T06:55:24.092 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-04-24T06:55:24.093 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-24T06:55:24.104 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-24T06:55:24.105 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-24T06:55:24.105 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:55:24.170 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-24T06:55:24.170 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T06:55:24.228 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:55:24.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.445+0000 7f37e6400640 1 -- 172.21.15.5:0/766845500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37e0104460 msgr2=0x7f37e0104840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:24.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.445+0000 7f37e6400640 1 --2- 172.21.15.5:0/766845500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37e0104460 0x7f37e0104840 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f37d0009900 tx=0x7f37d002a3f0 comp rx=0 tx=0).stop 2024-04-24T06:55:24.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.446+0000 7f37e6400640 1 -- 172.21.15.5:0/766845500 shutdown_connections 2024-04-24T06:55:24.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.446+0000 7f37e6400640 1 --2- 172.21.15.5:0/766845500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37e0109530 0x7f37e0111100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:24.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.446+0000 7f37e6400640 1 --2- 172.21.15.5:0/766845500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e0104e10 0x7f37e0108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:24.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.446+0000 7f37e6400640 1 --2- 172.21.15.5:0/766845500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37e0104460 0x7f37e0104840 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:24.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.446+0000 7f37e6400640 1 -- 172.21.15.5:0/766845500 >> 172.21.15.5:0/766845500 conn(0x7f37e00ffcc0 msgr2=0x7f37e01020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:55:24.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.446+0000 7f37e6400640 1 -- 172.21.15.5:0/766845500 shutdown_connections 2024-04-24T06:55:24.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.446+0000 7f37e6400640 1 -- 172.21.15.5:0/766845500 wait complete. 2024-04-24T06:55:24.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.447+0000 7f37e6400640 1 Processor -- start 2024-04-24T06:55:24.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.447+0000 7f37e6400640 1 -- start start 2024-04-24T06:55:24.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.447+0000 7f37e6400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37e0104460 0x7f37e01a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:24.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.447+0000 7f37e6400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37e0109530 0x7f37e01a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:24.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.447+0000 7f37e6400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e019a930 0x7f37e019ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:24.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.447+0000 7f37e6400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f37e0114610 con 0x7f37e0104460 2024-04-24T06:55:24.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.447+0000 7f37e6400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f37e0114490 con 0x7f37e019a930 2024-04-24T06:55:24.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.447+0000 7f37e6400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f37e0114790 con 0x7f37e0109530 2024-04-24T06:55:24.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.448+0000 7f37df400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37e0104460 0x7f37e01a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:24.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.448+0000 7f37df400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37e0104460 0x7f37e01a0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39476/0 (socket says 172.21.15.5:39476) 2024-04-24T06:55:24.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.448+0000 7f37df400640 1 -- 172.21.15.5:0/1219082279 learned_addr learned my addr 172.21.15.5:0/1219082279 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:55:24.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.448+0000 7f37dfe00640 1 --2- 172.21.15.5:0/1219082279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e019a930 0x7f37e019ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:24.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.448+0000 7f37dea00640 1 --2- 172.21.15.5:0/1219082279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37e0109530 0x7f37e01a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:24.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.448+0000 7f37dfe00640 1 -- 172.21.15.5:0/1219082279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37e0109530 msgr2=0x7f37e01a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:24.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.448+0000 7f37dfe00640 1 --2- 172.21.15.5:0/1219082279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37e0109530 0x7f37e01a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:24.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.448+0000 7f37dfe00640 1 -- 172.21.15.5:0/1219082279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37e0104460 msgr2=0x7f37e01a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:24.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.448+0000 7f37dfe00640 1 --2- 172.21.15.5:0/1219082279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37e0104460 0x7f37e01a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:24.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.448+0000 7f37dfe00640 1 -- 172.21.15.5:0/1219082279 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e019b450 con 0x7f37e019a930 2024-04-24T06:55:24.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.448+0000 7f37dfe00640 1 --2- 172.21.15.5:0/1219082279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e019a930 0x7f37e019ade0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f37d800e970 tx=0x7f37d800ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:55:24.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.449+0000 7f37d7e00640 1 -- 172.21.15.5:0/1219082279 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d800ccc0 con 0x7f37e019a930 2024-04-24T06:55:24.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.449+0000 7f37d7e00640 1 -- 172.21.15.5:0/1219082279 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37d8005020 con 0x7f37e019a930 2024-04-24T06:55:24.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.449+0000 7f37d7e00640 1 -- 172.21.15.5:0/1219082279 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d8004a00 con 0x7f37e019a930 2024-04-24T06:55:24.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.449+0000 7f37e6400640 1 -- 172.21.15.5:0/1219082279 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37e019b740 con 0x7f37e019a930 2024-04-24T06:55:24.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.449+0000 7f37e6400640 1 -- 172.21.15.5:0/1219082279 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37e01a7940 con 0x7f37e019a930 2024-04-24T06:55:24.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.450+0000 7f37e6400640 1 -- 172.21.15.5:0/1219082279 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37e0069300 con 0x7f37e019a930 2024-04-24T06:55:24.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.452+0000 7f37d7e00640 1 -- 172.21.15.5:0/1219082279 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f37d80051d0 con 0x7f37e019a930 2024-04-24T06:55:24.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.453+0000 7f37d7e00640 1 --2- 172.21.15.5:0/1219082279 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f37c00a3d20 0x7f37c00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:24.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.453+0000 7f37d7e00640 1 -- 172.21.15.5:0/1219082279 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f37d80d07c0 con 0x7f37e019a930 2024-04-24T06:55:24.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.454+0000 7f37df400640 1 --2- 172.21.15.5:0/1219082279 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f37c00a3d20 0x7f37c00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:24.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.454+0000 7f37df400640 1 --2- 172.21.15.5:0/1219082279 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f37c00a3d20 0x7f37c00a61e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f37d00096c0 tx=0x7f37d00046c0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:55:24.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.458+0000 7f37d7e00640 1 -- 172.21.15.5:0/1219082279 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f37d809cdf0 con 0x7f37e019a930 2024-04-24T06:55:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.668+0000 7f37e6400640 1 -- 172.21.15.5:0/1219082279 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f37e0104e10 con 0x7f37e019a930 2024-04-24T06:55:24.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.670+0000 7f37d7e00640 1 -- 172.21.15.5:0/1219082279 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+22974 (secure 0 0 0) 0x7f37d809c6f0 con 0x7f37e019a930 2024-04-24T06:55:24.671 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:55:24.671 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":99,"fsid":"43e9bad0-0204-11ef-bc93-c7b262605968","created":"2024-04-24T06:32:26.340555+0000","modified":"2024-04-24T06:55:16.104158+0000","last_up_change":"2024-04-24T06:40:36.790934+0000","last_in_change":"2024-04-24T06:40:17.803257+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:36:27.320342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:41:32.254502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:41:33.739067+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":98,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c2e974c-745c-4dd5-8c3d-8e8da2eb1ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6803","nonce":2155858515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6805","nonce":2155858515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6809","nonce":2155858515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6807","nonce":2155858515}]},"public_addr":"172.21.15.5:6803/2155858515","cluster_addr":"172.21.15.5:6805/2155858515","heartbeat_back_addr":"172.21.15.5:6809/2155858515","heartbeat_front_addr":"172.21.15.5:6807/2155858515","state":["exists","up"]},{"osd":1,"uuid":"50467744-db20-4721-bf2d-1c5db938297b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6801","nonce":1302362650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6803","nonce":1302362650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6807","nonce":1302362650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6805","nonce":1302362650}]},"public_addr":"172.21.15.37:6801/1302362650","cluster_addr":"172.21.15.37:6803/1302362650","heartbeat_back_addr":"172.21.15.37:6807/1302362650","heartbeat_front_addr":"172.21.15.37:6805/1302362650","state":["exists","up"]},{"osd":2,"uuid":"0e190b15-d04c-4288-8d48-ecf6664392fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6801","nonce":2885027256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6803","nonce":2885027256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6807","nonce":2885027256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6805","nonce":2885027256}]},"public_addr":"172.21.15.137:6801/2885027256","cluster_addr":"172.21.15.137:6803/2885027256","heartbeat_back_addr":"172.21.15.137:6807/2885027256","heartbeat_front_addr":"172.21.15.137:6805/2885027256","state":["exists","up"]},{"osd":3,"uuid":"e05592ba-d96a-472a-9f63-09a535d04dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6811","nonce":3701747260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6813","nonce":3701747260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6817","nonce":3701747260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6815","nonce":3701747260}]},"public_addr":"172.21.15.5:6811/3701747260","cluster_addr":"172.21.15.5:6813/3701747260","heartbeat_back_addr":"172.21.15.5:6817/3701747260","heartbeat_front_addr":"172.21.15.5:6815/3701747260","state":["exists","up"]},{"osd":4,"uuid":"c3f36e51-7a3b-4dd9-af4b-a10558830c4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6809","nonce":759253844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6811","nonce":759253844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6815","nonce":759253844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6813","nonce":759253844}]},"public_addr":"172.21.15.37:6809/759253844","cluster_addr":"172.21.15.37:6811/759253844","heartbeat_back_addr":"172.21.15.37:6815/759253844","heartbeat_front_addr":"172.21.15.37:6813/759253844","state":["exists","up"]},{"osd":5,"uuid":"97a0500c-cbb8-4ff5-b476-8634b26b3821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6809","nonce":3771921560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6811","nonce":3771921560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6815","nonce":3771921560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6813","nonce":3771921560}]},"public_addr":"172.21.15.137:6809/3771921560","cluster_addr":"172.21.15.137:6811/3771921560","heartbeat_back_addr":"172.21.15.137:6815/3771921560","heartbeat_front_addr":"172.21.15.137:6813/3771921560","state":["exists","up"]},{"osd":6,"uuid":"33a33d62-bbb3-4ed9-834e-19ae93a97670","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6819","nonce":2064636084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6821","nonce":2064636084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6825","nonce":2064636084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6823","nonce":2064636084}]},"public_addr":"172.21.15.5:6819/2064636084","cluster_addr":"172.21.15.5:6821/2064636084","heartbeat_back_addr":"172.21.15.5:6825/2064636084","heartbeat_front_addr":"172.21.15.5:6823/2064636084","state":["exists","up"]},{"osd":7,"uuid":"c0f4e036-4717-4514-bb59-ab11fbaab063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6817","nonce":2952750659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6819","nonce":2952750659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6823","nonce":2952750659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6821","nonce":2952750659}]},"public_addr":"172.21.15.37:6817/2952750659","cluster_addr":"172.21.15.37:6819/2952750659","heartbeat_back_addr":"172.21.15.37:6823/2952750659","heartbeat_front_addr":"172.21.15.37:6821/2952750659","state":["exists","up"]},{"osd":8,"uuid":"5391fd87-44ea-440f-80da-c54d0327c04c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6817","nonce":29724945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6819","nonce":29724945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6823","nonce":29724945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6821","nonce":29724945}]},"public_addr":"172.21.15.137:6817/29724945","cluster_addr":"172.21.15.137:6819/29724945","heartbeat_back_addr":"172.21.15.137:6823/29724945","heartbeat_front_addr":"172.21.15.137:6821/29724945","state":["exists","up"]},{"osd":9,"uuid":"d94c9dbc-f43d-4237-ac7f-47a6a5ede1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6827","nonce":2257102107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6829","nonce":2257102107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6833","nonce":2257102107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6831","nonce":2257102107}]},"public_addr":"172.21.15.5:6827/2257102107","cluster_addr":"172.21.15.5:6829/2257102107","heartbeat_back_addr":"172.21.15.5:6833/2257102107","heartbeat_front_addr":"172.21.15.5:6831/2257102107","state":["exists","up"]},{"osd":10,"uuid":"e2455332-2a5e-44b1-85d3-8e47da399f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6825","nonce":2451570637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6827","nonce":2451570637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6831","nonce":2451570637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6829","nonce":2451570637}]},"public_addr":"172.21.15.37:6825/2451570637","cluster_addr":"172.21.15.37:6827/2451570637","heartbeat_back_addr":"172.21.15.37:6831/2451570637","heartbeat_front_addr":"172.21.15.37:6829/2451570637","state":["exists","up"]},{"osd":11,"uuid":"49d9396b-704a-4cef-8210-75d30c28e924","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6825","nonce":90600596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6827","nonce":90600596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6831","nonce":90600596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6829","nonce":90600596}]},"public_addr":"172.21.15.137:6825/90600596","cluster_addr":"172.21.15.137:6827/90600596","heartbeat_back_addr":"172.21.15.137:6831/90600596","heartbeat_front_addr":"172.21.15.137:6829/90600596","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:35:35.591619+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:00.568817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:24.195643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:56.451724+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:22.109293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:46.161749+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:19.278305+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:45.620256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:10.041165+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:43.813697+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:10.736026+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:35.384377+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6800/4042936965":"2024-04-25T06:44:01.965881+0000","172.21.15.5:0/2912680052":"2024-04-25T06:44:01.965881+0000","172.21.15.5:6801/4042936965":"2024-04-25T06:44:01.965881+0000","172.21.15.5:0/3478347847":"2024-04-25T06:44:01.965881+0000","172.21.15.5:0/23307164":"2024-04-25T06:33:41.161852+0000","172.21.15.5:0/1012004002":"2024-04-25T06:44:01.965881+0000","172.21.15.5:6801/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:0/230785941":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/805658012":"2024-04-25T06:44:01.965881+0000","172.21.15.5:0/2940023597":"2024-04-25T06:33:07.040369+0000","172.21.15.5:6801/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3886613904":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/4199165437":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3459593499":"2024-04-25T06:33:41.161852+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:55:24.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.673+0000 7f37e6400640 1 -- 172.21.15.5:0/1219082279 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f37c00a3d20 msgr2=0x7f37c00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:24.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.673+0000 7f37e6400640 1 --2- 172.21.15.5:0/1219082279 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f37c00a3d20 0x7f37c00a61e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f37d00096c0 tx=0x7f37d00046c0 comp rx=0 tx=0).stop 2024-04-24T06:55:24.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.673+0000 7f37e6400640 1 -- 172.21.15.5:0/1219082279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e019a930 msgr2=0x7f37e019ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:24.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.673+0000 7f37e6400640 1 --2- 172.21.15.5:0/1219082279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e019a930 0x7f37e019ade0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f37d800e970 tx=0x7f37d800ee30 comp rx=0 tx=0).stop 2024-04-24T06:55:24.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.673+0000 7f37e6400640 1 -- 172.21.15.5:0/1219082279 shutdown_connections 2024-04-24T06:55:24.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.673+0000 7f37e6400640 1 --2- 172.21.15.5:0/1219082279 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f37c00a3d20 0x7f37c00a61e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:24.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.673+0000 7f37e6400640 1 --2- 172.21.15.5:0/1219082279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e019a930 0x7f37e019ade0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:24.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.673+0000 7f37e6400640 1 --2- 172.21.15.5:0/1219082279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37e0109530 0x7f37e01a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:24.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.673+0000 7f37e6400640 1 --2- 172.21.15.5:0/1219082279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37e0104460 0x7f37e01a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:24.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.673+0000 7f37e6400640 1 -- 172.21.15.5:0/1219082279 >> 172.21.15.5:0/1219082279 conn(0x7f37e00ffcc0 msgr2=0x7f37e0102010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:55:24.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.673+0000 7f37e6400640 1 -- 172.21.15.5:0/1219082279 shutdown_connections 2024-04-24T06:55:24.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.673+0000 7f37e6400640 1 -- 172.21.15.5:0/1219082279 wait complete. 2024-04-24T06:55:24.695 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T06:36:27.320342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T06:41:32.254502+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T06:41:33.739067+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 98, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T06:55:24.695 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T06:55:24.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.916+0000 7f15b1800640 1 -- 172.21.15.5:0/3320793876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15ac076530 msgr2=0x7f15ac076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:24.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.916+0000 7f15b1800640 1 --2- 172.21.15.5:0/3320793876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15ac076530 0x7f15ac076910 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f159c009890 tx=0x7f159c02a360 comp rx=0 tx=0).stop 2024-04-24T06:55:24.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.916+0000 7f15b1800640 1 -- 172.21.15.5:0/3320793876 shutdown_connections 2024-04-24T06:55:24.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.916+0000 7f15b1800640 1 --2- 172.21.15.5:0/3320793876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15ac112e70 0x7f15ac115260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:24.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.916+0000 7f15b1800640 1 --2- 172.21.15.5:0/3320793876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15ac076e50 0x7f15ac112930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:24.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.916+0000 7f15b1800640 1 --2- 172.21.15.5:0/3320793876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15ac076530 0x7f15ac076910 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:24.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.916+0000 7f15b1800640 1 -- 172.21.15.5:0/3320793876 >> 172.21.15.5:0/3320793876 conn(0x7f15ac0ffc80 msgr2=0x7f15ac1020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:55:24.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.916+0000 7f15b1800640 1 -- 172.21.15.5:0/3320793876 shutdown_connections 2024-04-24T06:55:24.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.917+0000 7f15b1800640 1 -- 172.21.15.5:0/3320793876 wait complete. 2024-04-24T06:55:24.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.918+0000 7f15b1800640 1 Processor -- start 2024-04-24T06:55:24.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.918+0000 7f15b1800640 1 -- start start 2024-04-24T06:55:24.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.919+0000 7f15b1800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15ac076530 0x7f15ac19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:24.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.919+0000 7f15b1800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15ac076e50 0x7f15ac19f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:24.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.919+0000 7f15b1800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15ac112e70 0x7f15ac1a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:24.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.919+0000 7f15b1800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f15ac1189d0 con 0x7f15ac112e70 2024-04-24T06:55:24.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.919+0000 7f15b1800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f15ac118850 con 0x7f15ac076530 2024-04-24T06:55:24.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.919+0000 7f15b1800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f15ac118b50 con 0x7f15ac076e50 2024-04-24T06:55:24.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.919+0000 7f15ab400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15ac112e70 0x7f15ac1a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:24.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.919+0000 7f15ab400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15ac112e70 0x7f15ac1a3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39498/0 (socket says 172.21.15.5:39498) 2024-04-24T06:55:24.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.919+0000 7f15ab400640 1 -- 172.21.15.5:0/2232950724 learned_addr learned my addr 172.21.15.5:0/2232950724 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:55:24.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.919+0000 7f15aa000640 1 --2- 172.21.15.5:0/2232950724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15ac076e50 0x7f15ac19f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:24.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.919+0000 7f15aaa00640 1 --2- 172.21.15.5:0/2232950724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15ac076530 0x7f15ac19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:24.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.919+0000 7f15ab400640 1 -- 172.21.15.5:0/2232950724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15ac076e50 msgr2=0x7f15ac19f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:24.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.919+0000 7f15ab400640 1 --2- 172.21.15.5:0/2232950724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15ac076e50 0x7f15ac19f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:24.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.919+0000 7f15ab400640 1 -- 172.21.15.5:0/2232950724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15ac076530 msgr2=0x7f15ac19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:24.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.919+0000 7f15ab400640 1 --2- 172.21.15.5:0/2232950724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15ac076530 0x7f15ac19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:24.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.920+0000 7f15ab400640 1 -- 172.21.15.5:0/2232950724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15ac1a3f30 con 0x7f15ac112e70 2024-04-24T06:55:24.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.920+0000 7f15aaa00640 1 --2- 172.21.15.5:0/2232950724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15ac076530 0x7f15ac19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:55:24.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.920+0000 7f15ab400640 1 --2- 172.21.15.5:0/2232950724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15ac112e70 0x7f15ac1a3800 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f15a400eb40 tx=0x7f15a400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:55:24.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.920+0000 7f15a3400640 1 -- 172.21.15.5:0/2232950724 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a4004b20 con 0x7f15ac112e70 2024-04-24T06:55:24.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.920+0000 7f15b1800640 1 -- 172.21.15.5:0/2232950724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15ac1a4220 con 0x7f15ac112e70 2024-04-24T06:55:24.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.920+0000 7f15a3400640 1 -- 172.21.15.5:0/2232950724 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15a4004cc0 con 0x7f15ac112e70 2024-04-24T06:55:24.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.920+0000 7f15a3400640 1 -- 172.21.15.5:0/2232950724 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a4010850 con 0x7f15ac112e70 2024-04-24T06:55:24.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.920+0000 7f15b1800640 1 -- 172.21.15.5:0/2232950724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15ac1abab0 con 0x7f15ac112e70 2024-04-24T06:55:24.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.922+0000 7f15b1800640 1 -- 172.21.15.5:0/2232950724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15ac069300 con 0x7f15ac112e70 2024-04-24T06:55:24.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.924+0000 7f15a3400640 1 -- 172.21.15.5:0/2232950724 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f15a4005020 con 0x7f15ac112e70 2024-04-24T06:55:24.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.925+0000 7f15a3400640 1 --2- 172.21.15.5:0/2232950724 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f15900a3da0 0x7f15900a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:24.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.925+0000 7f15a3400640 1 -- 172.21.15.5:0/2232950724 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f15a40d0f20 con 0x7f15ac112e70 2024-04-24T06:55:24.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.926+0000 7f15aaa00640 1 --2- 172.21.15.5:0/2232950724 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f15900a3da0 0x7f15900a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:24.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.927+0000 7f15aaa00640 1 --2- 172.21.15.5:0/2232950724 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f15900a3da0 0x7f15900a6260 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f159c035040 tx=0x7f159c0056d0 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:55:24.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:24.929+0000 7f15a3400640 1 -- 172.21.15.5:0/2232950724 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f15a409d550 con 0x7f15ac112e70 2024-04-24T06:55:25.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.134+0000 7f15b1800640 1 -- 172.21.15.5:0/2232950724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f15ac02d620 con 0x7f15ac112e70 2024-04-24T06:55:25.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.135+0000 7f15a3400640 1 -- 172.21.15.5:0/2232950724 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) ==== 93+0+10 (secure 0 0 0) 0x7f15a409ce50 con 0x7f15ac112e70 2024-04-24T06:55:25.136 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-24T06:55:25.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.138+0000 7f15b1800640 1 -- 172.21.15.5:0/2232950724 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f15900a3da0 msgr2=0x7f15900a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:25.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.138+0000 7f15b1800640 1 --2- 172.21.15.5:0/2232950724 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f15900a3da0 0x7f15900a6260 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f159c035040 tx=0x7f159c0056d0 comp rx=0 tx=0).stop 2024-04-24T06:55:25.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.138+0000 7f15b1800640 1 -- 172.21.15.5:0/2232950724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15ac112e70 msgr2=0x7f15ac1a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:25.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.138+0000 7f15b1800640 1 --2- 172.21.15.5:0/2232950724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15ac112e70 0x7f15ac1a3800 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f15a400eb40 tx=0x7f15a400c2d0 comp rx=0 tx=0).stop 2024-04-24T06:55:25.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.138+0000 7f15b1800640 1 -- 172.21.15.5:0/2232950724 shutdown_connections 2024-04-24T06:55:25.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.138+0000 7f15b1800640 1 --2- 172.21.15.5:0/2232950724 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f15900a3da0 0x7f15900a6260 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:25.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.138+0000 7f15b1800640 1 --2- 172.21.15.5:0/2232950724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15ac112e70 0x7f15ac1a3800 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:25.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.138+0000 7f15b1800640 1 --2- 172.21.15.5:0/2232950724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15ac076e50 0x7f15ac19f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:25.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.138+0000 7f15b1800640 1 --2- 172.21.15.5:0/2232950724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15ac076530 0x7f15ac19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:25.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.138+0000 7f15b1800640 1 -- 172.21.15.5:0/2232950724 >> 172.21.15.5:0/2232950724 conn(0x7f15ac0ffc80 msgr2=0x7f15ac114600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:55:25.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.138+0000 7f15b1800640 1 -- 172.21.15.5:0/2232950724 shutdown_connections 2024-04-24T06:55:25.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.138+0000 7f15b1800640 1 -- 172.21.15.5:0/2232950724 wait complete. 2024-04-24T06:55:25.160 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T06:55:25.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.379+0000 7f22f0400640 1 -- 172.21.15.5:0/2644669175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22e810a590 msgr2=0x7f22e810a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:25.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.379+0000 7f22f0400640 1 --2- 172.21.15.5:0/2644669175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22e810a590 0x7f22e810a970 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f22dc009900 tx=0x7f22dc02a480 comp rx=0 tx=0).stop 2024-04-24T06:55:25.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.379+0000 7f22f0400640 1 -- 172.21.15.5:0/2644669175 shutdown_connections 2024-04-24T06:55:25.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.379+0000 7f22f0400640 1 --2- 172.21.15.5:0/2644669175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22e81090c0 0x7f22e8115570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:25.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.379+0000 7f22f0400640 1 --2- 172.21.15.5:0/2644669175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22e8108720 0x7f22e8108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:25.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.379+0000 7f22f0400640 1 --2- 172.21.15.5:0/2644669175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22e810a590 0x7f22e810a970 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:25.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.379+0000 7f22f0400640 1 -- 172.21.15.5:0/2644669175 >> 172.21.15.5:0/2644669175 conn(0x7f22e8077e70 msgr2=0x7f22e8106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:55:25.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.380+0000 7f22f0400640 1 -- 172.21.15.5:0/2644669175 shutdown_connections 2024-04-24T06:55:25.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.380+0000 7f22f0400640 1 -- 172.21.15.5:0/2644669175 wait complete. 2024-04-24T06:55:25.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.381+0000 7f22f0400640 1 Processor -- start 2024-04-24T06:55:25.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.381+0000 7f22f0400640 1 -- start start 2024-04-24T06:55:25.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.381+0000 7f22f0400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22e8108720 0x7f22e819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:25.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.381+0000 7f22f0400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22e81090c0 0x7f22e819f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:25.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.381+0000 7f22f0400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22e810a590 0x7f22e81a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:25.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.381+0000 7f22f0400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f22e8118b40 con 0x7f22e810a590 2024-04-24T06:55:25.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.381+0000 7f22f0400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f22e81189c0 con 0x7f22e81090c0 2024-04-24T06:55:25.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.381+0000 7f22f0400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f22e8118cc0 con 0x7f22e8108720 2024-04-24T06:55:25.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.382+0000 7f22eda00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22e8108720 0x7f22e819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:25.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.382+0000 7f22eda00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22e8108720 0x7f22e819efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:33496/0 (socket says 172.21.15.5:33496) 2024-04-24T06:55:25.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.382+0000 7f22eda00640 1 -- 172.21.15.5:0/4040942499 learned_addr learned my addr 172.21.15.5:0/4040942499 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:55:25.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.382+0000 7f22ed000640 1 --2- 172.21.15.5:0/4040942499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22e81090c0 0x7f22e819f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:25.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.382+0000 7f22ee400640 1 --2- 172.21.15.5:0/4040942499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22e810a590 0x7f22e81a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:25.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.382+0000 7f22eda00640 1 -- 172.21.15.5:0/4040942499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22e81090c0 msgr2=0x7f22e819f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:25.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.382+0000 7f22eda00640 1 --2- 172.21.15.5:0/4040942499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22e81090c0 0x7f22e819f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:25.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.382+0000 7f22eda00640 1 -- 172.21.15.5:0/4040942499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22e810a590 msgr2=0x7f22e81a3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:25.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.382+0000 7f22eda00640 1 --2- 172.21.15.5:0/4040942499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22e810a590 0x7f22e81a3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:25.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.382+0000 7f22eda00640 1 -- 172.21.15.5:0/4040942499 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22e81a3f40 con 0x7f22e8108720 2024-04-24T06:55:25.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.382+0000 7f22ee400640 1 --2- 172.21.15.5:0/4040942499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22e810a590 0x7f22e81a3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T06:55:25.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.383+0000 7f22eda00640 1 --2- 172.21.15.5:0/4040942499 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22e8108720 0x7f22e819efa0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f22dc004770 tx=0x7f22dc003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:55:25.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.384+0000 7f22e6000640 1 -- 172.21.15.5:0/4040942499 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc03f070 con 0x7f22e8108720 2024-04-24T06:55:25.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.384+0000 7f22e6000640 1 -- 172.21.15.5:0/4040942499 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22dc031ca0 con 0x7f22e8108720 2024-04-24T06:55:25.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.385+0000 7f22f0400640 1 -- 172.21.15.5:0/4040942499 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22e81a41d0 con 0x7f22e8108720 2024-04-24T06:55:25.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.385+0000 7f22e6000640 1 -- 172.21.15.5:0/4040942499 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc035030 con 0x7f22e8108720 2024-04-24T06:55:25.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.385+0000 7f22f0400640 1 -- 172.21.15.5:0/4040942499 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22e81abac0 con 0x7f22e8108720 2024-04-24T06:55:25.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.386+0000 7f22f0400640 1 -- 172.21.15.5:0/4040942499 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22e8069300 con 0x7f22e8108720 2024-04-24T06:55:25.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.388+0000 7f22e6000640 1 -- 172.21.15.5:0/4040942499 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f22dc042070 con 0x7f22e8108720 2024-04-24T06:55:25.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.389+0000 7f22e6000640 1 --2- 172.21.15.5:0/4040942499 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f22d00a3da0 0x7f22d00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:25.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.389+0000 7f22e6000640 1 -- 172.21.15.5:0/4040942499 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f22dc0f0df0 con 0x7f22e8108720 2024-04-24T06:55:25.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.390+0000 7f22ed000640 1 --2- 172.21.15.5:0/4040942499 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f22d00a3da0 0x7f22d00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:25.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.390+0000 7f22ed000640 1 --2- 172.21.15.5:0/4040942499 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f22d00a3da0 0x7f22d00a6260 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f22e8108080 tx=0x7f22d4005f90 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:55:25.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.394+0000 7f22e6000640 1 -- 172.21.15.5:0/4040942499 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f22dc049030 con 0x7f22e8108720 2024-04-24T06:55:25.505 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:25 smithi005 ceph-mon[27798]: pgmap v354: 97 pgs: 97 active+clean; 85 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-24T06:55:25.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:25 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/1219082279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:55:25.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:25 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/2232950724' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T06:55:25.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.602+0000 7f22f0400640 1 -- 172.21.15.5:0/4040942499 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f22e8110bc0 con 0x7f22e8108720 2024-04-24T06:55:25.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.603+0000 7f22e6000640 1 -- 172.21.15.5:0/4040942499 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) ==== 104+0+11 (secure 0 0 0) 0x7f22dc0bd4e0 con 0x7f22e8108720 2024-04-24T06:55:25.603 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-24T06:55:25.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.605+0000 7f22f0400640 1 -- 172.21.15.5:0/4040942499 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f22d00a3da0 msgr2=0x7f22d00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:25.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.605+0000 7f22f0400640 1 --2- 172.21.15.5:0/4040942499 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f22d00a3da0 0x7f22d00a6260 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f22e8108080 tx=0x7f22d4005f90 comp rx=0 tx=0).stop 2024-04-24T06:55:25.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.605+0000 7f22f0400640 1 -- 172.21.15.5:0/4040942499 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22e8108720 msgr2=0x7f22e819efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:25.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.605+0000 7f22f0400640 1 --2- 172.21.15.5:0/4040942499 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22e8108720 0x7f22e819efa0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f22dc004770 tx=0x7f22dc003c10 comp rx=0 tx=0).stop 2024-04-24T06:55:25.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.605+0000 7f22f0400640 1 -- 172.21.15.5:0/4040942499 shutdown_connections 2024-04-24T06:55:25.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.605+0000 7f22f0400640 1 --2- 172.21.15.5:0/4040942499 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f22d00a3da0 0x7f22d00a6260 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:25.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.605+0000 7f22f0400640 1 --2- 172.21.15.5:0/4040942499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22e810a590 0x7f22e81a3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:25.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.605+0000 7f22f0400640 1 --2- 172.21.15.5:0/4040942499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22e81090c0 0x7f22e819f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:25.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.605+0000 7f22f0400640 1 --2- 172.21.15.5:0/4040942499 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22e8108720 0x7f22e819efa0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:25.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.605+0000 7f22f0400640 1 -- 172.21.15.5:0/4040942499 >> 172.21.15.5:0/4040942499 conn(0x7f22e8077e70 msgr2=0x7f22e80784d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:55:25.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.606+0000 7f22f0400640 1 -- 172.21.15.5:0/4040942499 shutdown_connections 2024-04-24T06:55:25.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.606+0000 7f22f0400640 1 -- 172.21.15.5:0/4040942499 wait complete. 2024-04-24T06:55:25.625 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T06:55:25.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:25 smithi137 ceph-mon[33017]: pgmap v354: 97 pgs: 97 active+clean; 85 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-24T06:55:25.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:25 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/1219082279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:55:25.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:25 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/2232950724' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T06:55:25.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:25 smithi037 ceph-mon[32864]: pgmap v354: 97 pgs: 97 active+clean; 85 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-24T06:55:25.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:25 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/1219082279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:55:25.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:25 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/2232950724' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T06:55:25.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.844+0000 7fe599e00640 1 -- 172.21.15.5:0/804626772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe594108540 msgr2=0x7fe5941089a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:25.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.844+0000 7fe599e00640 1 --2- 172.21.15.5:0/804626772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe594108540 0x7fe5941089a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe580009ca0 tx=0x7fe58002a950 comp rx=0 tx=0).stop 2024-04-24T06:55:25.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.844+0000 7fe599e00640 1 -- 172.21.15.5:0/804626772 shutdown_connections 2024-04-24T06:55:25.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.844+0000 7fe599e00640 1 --2- 172.21.15.5:0/804626772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe594108ee0 0x7fe5941152b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:25.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.844+0000 7fe599e00640 1 --2- 172.21.15.5:0/804626772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe594108540 0x7fe5941089a0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:25.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.844+0000 7fe599e00640 1 --2- 172.21.15.5:0/804626772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe59410e4f0 0x7fe59410e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:25.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.844+0000 7fe599e00640 1 -- 172.21.15.5:0/804626772 >> 172.21.15.5:0/804626772 conn(0x7fe594077e10 msgr2=0x7fe594106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:55:25.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.844+0000 7fe599e00640 1 -- 172.21.15.5:0/804626772 shutdown_connections 2024-04-24T06:55:25.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.844+0000 7fe599e00640 1 -- 172.21.15.5:0/804626772 wait complete. 2024-04-24T06:55:25.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.845+0000 7fe599e00640 1 Processor -- start 2024-04-24T06:55:25.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.845+0000 7fe599e00640 1 -- start start 2024-04-24T06:55:25.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.846+0000 7fe599e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe594108540 0x7fe59419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:25.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.846+0000 7fe599e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe594108ee0 0x7fe59419f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:25.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.846+0000 7fe599e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe59410e4f0 0x7fe5941a3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:25.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.846+0000 7fe599e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe594118810 con 0x7fe59410e4f0 2024-04-24T06:55:25.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.846+0000 7fe599e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe594118690 con 0x7fe594108540 2024-04-24T06:55:25.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.846+0000 7fe599e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe594118990 con 0x7fe594108ee0 2024-04-24T06:55:25.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.846+0000 7fe599400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe59410e4f0 0x7fe5941a3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:25.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.846+0000 7fe593e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe594108ee0 0x7fe59419f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:25.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.846+0000 7fe599400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe59410e4f0 0x7fe5941a3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39542/0 (socket says 172.21.15.5:39542) 2024-04-24T06:55:25.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.846+0000 7fe599400640 1 -- 172.21.15.5:0/2226926756 learned_addr learned my addr 172.21.15.5:0/2226926756 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:55:25.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.846+0000 7fe598a00640 1 --2- 172.21.15.5:0/2226926756 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe594108540 0x7fe59419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:25.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.846+0000 7fe593e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe594108ee0 0x7fe59419f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:33510/0 (socket says 172.21.15.5:33510) 2024-04-24T06:55:25.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.847+0000 7fe593e00640 1 -- 172.21.15.5:0/2226926756 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe594108540 msgr2=0x7fe59419ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:25.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.847+0000 7fe593e00640 1 --2- 172.21.15.5:0/2226926756 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe594108540 0x7fe59419ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:25.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.847+0000 7fe593e00640 1 -- 172.21.15.5:0/2226926756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe59410e4f0 msgr2=0x7fe5941a3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:25.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.847+0000 7fe593e00640 1 --2- 172.21.15.5:0/2226926756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe59410e4f0 0x7fe5941a3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:25.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.847+0000 7fe593e00640 1 -- 172.21.15.5:0/2226926756 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5941a3d40 con 0x7fe594108ee0 2024-04-24T06:55:25.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.847+0000 7fe598a00640 1 --2- 172.21.15.5:0/2226926756 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe594108540 0x7fe59419ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:55:25.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.847+0000 7fe593e00640 1 --2- 172.21.15.5:0/2226926756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe594108ee0 0x7fe59419f290 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe580005e30 tx=0x7fe580003c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:55:25.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.847+0000 7fe591600640 1 -- 172.21.15.5:0/2226926756 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe580034040 con 0x7fe594108ee0 2024-04-24T06:55:25.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.847+0000 7fe591600640 1 -- 172.21.15.5:0/2226926756 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe58002f930 con 0x7fe594108ee0 2024-04-24T06:55:25.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.847+0000 7fe599e00640 1 -- 172.21.15.5:0/2226926756 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5941a3fd0 con 0x7fe594108ee0 2024-04-24T06:55:25.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.848+0000 7fe591600640 1 -- 172.21.15.5:0/2226926756 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5800046e0 con 0x7fe594108ee0 2024-04-24T06:55:25.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.848+0000 7fe599e00640 1 -- 172.21.15.5:0/2226926756 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5941ab8c0 con 0x7fe594108ee0 2024-04-24T06:55:25.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.849+0000 7fe599e00640 1 -- 172.21.15.5:0/2226926756 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe59404fd90 con 0x7fe594108ee0 2024-04-24T06:55:25.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.851+0000 7fe591600640 1 -- 172.21.15.5:0/2226926756 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fe580037020 con 0x7fe594108ee0 2024-04-24T06:55:25.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.852+0000 7fe591600640 1 --2- 172.21.15.5:0/2226926756 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fe5780a3da0 0x7fe5780a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:25.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.852+0000 7fe591600640 1 -- 172.21.15.5:0/2226926756 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7fe5800f15d0 con 0x7fe594108ee0 2024-04-24T06:55:25.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.853+0000 7fe598a00640 1 --2- 172.21.15.5:0/2226926756 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fe5780a3da0 0x7fe5780a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:25.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.853+0000 7fe598a00640 1 --2- 172.21.15.5:0/2226926756 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fe5780a3da0 0x7fe5780a6260 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe584009f40 tx=0x7fe584005f90 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:55:25.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:25.855+0000 7fe591600640 1 -- 172.21.15.5:0/2226926756 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe5800bdd10 con 0x7fe594108ee0 2024-04-24T06:55:26.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.065+0000 7fe599e00640 1 -- 172.21.15.5:0/2226926756 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe594110370 con 0x7fe594108ee0 2024-04-24T06:55:26.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.066+0000 7fe591600640 1 -- 172.21.15.5:0/2226926756 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) ==== 100+0+11 (secure 0 0 0) 0x7fe5800bd610 con 0x7fe594108ee0 2024-04-24T06:55:26.066 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-24T06:55:26.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.069+0000 7fe599e00640 1 -- 172.21.15.5:0/2226926756 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fe5780a3da0 msgr2=0x7fe5780a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:26.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.069+0000 7fe599e00640 1 --2- 172.21.15.5:0/2226926756 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fe5780a3da0 0x7fe5780a6260 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe584009f40 tx=0x7fe584005f90 comp rx=0 tx=0).stop 2024-04-24T06:55:26.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.069+0000 7fe599e00640 1 -- 172.21.15.5:0/2226926756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe594108ee0 msgr2=0x7fe59419f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:26.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.069+0000 7fe599e00640 1 --2- 172.21.15.5:0/2226926756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe594108ee0 0x7fe59419f290 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe580005e30 tx=0x7fe580003c00 comp rx=0 tx=0).stop 2024-04-24T06:55:26.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.069+0000 7fe599e00640 1 -- 172.21.15.5:0/2226926756 shutdown_connections 2024-04-24T06:55:26.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.069+0000 7fe599e00640 1 --2- 172.21.15.5:0/2226926756 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fe5780a3da0 0x7fe5780a6260 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:26.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.069+0000 7fe599e00640 1 --2- 172.21.15.5:0/2226926756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe59410e4f0 0x7fe5941a3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:26.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.069+0000 7fe599e00640 1 --2- 172.21.15.5:0/2226926756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe594108ee0 0x7fe59419f290 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:26.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.069+0000 7fe599e00640 1 --2- 172.21.15.5:0/2226926756 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe594108540 0x7fe59419ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:26.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.069+0000 7fe599e00640 1 -- 172.21.15.5:0/2226926756 >> 172.21.15.5:0/2226926756 conn(0x7fe594077e10 msgr2=0x7fe594078530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:55:26.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.069+0000 7fe599e00640 1 -- 172.21.15.5:0/2226926756 shutdown_connections 2024-04-24T06:55:26.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.069+0000 7fe599e00640 1 -- 172.21.15.5:0/2226926756 wait complete. 2024-04-24T06:55:26.089 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T06:55:26.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.294+0000 7fefb2800640 1 -- 172.21.15.5:0/2862287404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fefac1090c0 msgr2=0x7fefac115570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:26.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.294+0000 7fefb2800640 1 --2- 172.21.15.5:0/2862287404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fefac1090c0 0x7fefac115570 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fefa4009ca0 tx=0x7fefa402a940 comp rx=0 tx=0).stop 2024-04-24T06:55:26.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.294+0000 7fefb2800640 1 -- 172.21.15.5:0/2862287404 shutdown_connections 2024-04-24T06:55:26.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.294+0000 7fefb2800640 1 --2- 172.21.15.5:0/2862287404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fefac1090c0 0x7fefac115570 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:26.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.294+0000 7fefb2800640 1 --2- 172.21.15.5:0/2862287404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefac108720 0x7fefac108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:26.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.294+0000 7fefb2800640 1 --2- 172.21.15.5:0/2862287404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefac10a590 0x7fefac10a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:26.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.294+0000 7fefb2800640 1 -- 172.21.15.5:0/2862287404 >> 172.21.15.5:0/2862287404 conn(0x7fefac077e70 msgr2=0x7fefac106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:55:26.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.294+0000 7fefb2800640 1 -- 172.21.15.5:0/2862287404 shutdown_connections 2024-04-24T06:55:26.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.294+0000 7fefb2800640 1 -- 172.21.15.5:0/2862287404 wait complete. 2024-04-24T06:55:26.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.295+0000 7fefb2800640 1 Processor -- start 2024-04-24T06:55:26.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.295+0000 7fefb2800640 1 -- start start 2024-04-24T06:55:26.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.295+0000 7fefb2800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefac108720 0x7fefac19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:26.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.295+0000 7fefb2800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefac1090c0 0x7fefac19f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:26.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.295+0000 7fefb2800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fefac10a590 0x7fefac1a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:26.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.295+0000 7fefb2800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fefac118b40 con 0x7fefac10a590 2024-04-24T06:55:26.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.295+0000 7fefb2800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fefac1189c0 con 0x7fefac108720 2024-04-24T06:55:26.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.295+0000 7fefb2800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fefac118cc0 con 0x7fefac1090c0 2024-04-24T06:55:26.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.296+0000 7fefabe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fefac10a590 0x7fefac1a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:26.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.296+0000 7fefabe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fefac10a590 0x7fefac1a37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39556/0 (socket says 172.21.15.5:39556) 2024-04-24T06:55:26.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.296+0000 7fefabe00640 1 -- 172.21.15.5:0/2251185443 learned_addr learned my addr 172.21.15.5:0/2251185443 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:55:26.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.296+0000 7fefa3e00640 1 --2- 172.21.15.5:0/2251185443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefac1090c0 0x7fefac19f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:26.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.296+0000 7fefab400640 1 --2- 172.21.15.5:0/2251185443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefac108720 0x7fefac19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:26.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.296+0000 7fefabe00640 1 -- 172.21.15.5:0/2251185443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefac1090c0 msgr2=0x7fefac19f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:26.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.296+0000 7fefabe00640 1 --2- 172.21.15.5:0/2251185443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefac1090c0 0x7fefac19f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:26.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.296+0000 7fefabe00640 1 -- 172.21.15.5:0/2251185443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefac108720 msgr2=0x7fefac19ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:26.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.296+0000 7fefabe00640 1 --2- 172.21.15.5:0/2251185443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefac108720 0x7fefac19ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:26.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.296+0000 7fefabe00640 1 -- 172.21.15.5:0/2251185443 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefac1a3ef0 con 0x7fefac10a590 2024-04-24T06:55:26.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.296+0000 7fefa3e00640 1 --2- 172.21.15.5:0/2251185443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefac1090c0 0x7fefac19f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:55:26.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.296+0000 7fefab400640 1 --2- 172.21.15.5:0/2251185443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefac108720 0x7fefac19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:55:26.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.297+0000 7fefabe00640 1 --2- 172.21.15.5:0/2251185443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fefac10a590 0x7fefac1a37c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fefa4005e30 tx=0x7fefa402f5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:55:26.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.297+0000 7fefa8c00640 1 -- 172.21.15.5:0/2251185443 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa4034040 con 0x7fefac10a590 2024-04-24T06:55:26.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.297+0000 7fefa8c00640 1 -- 172.21.15.5:0/2251185443 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefa402fe10 con 0x7fefac10a590 2024-04-24T06:55:26.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.297+0000 7fefb2800640 1 -- 172.21.15.5:0/2251185443 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefac1a4180 con 0x7fefac10a590 2024-04-24T06:55:26.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.297+0000 7fefa8c00640 1 -- 172.21.15.5:0/2251185443 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa4004510 con 0x7fefac10a590 2024-04-24T06:55:26.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.297+0000 7fefb2800640 1 -- 172.21.15.5:0/2251185443 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefac1aba70 con 0x7fefac10a590 2024-04-24T06:55:26.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.298+0000 7fefb2800640 1 -- 172.21.15.5:0/2251185443 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefac069300 con 0x7fefac10a590 2024-04-24T06:55:26.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.300+0000 7fefa8c00640 1 -- 172.21.15.5:0/2251185443 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fefa4037020 con 0x7fefac10a590 2024-04-24T06:55:26.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.302+0000 7fefa8c00640 1 --2- 172.21.15.5:0/2251185443 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fef900a3d20 0x7fef900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:26.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.302+0000 7fefa8c00640 1 -- 172.21.15.5:0/2251185443 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7fefa40f14b0 con 0x7fefac10a590 2024-04-24T06:55:26.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.302+0000 7fefab400640 1 --2- 172.21.15.5:0/2251185443 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fef900a3d20 0x7fef900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:26.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.303+0000 7fefab400640 1 --2- 172.21.15.5:0/2251185443 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fef900a3d20 0x7fef900a61e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fef9c009fc0 tx=0x7fef9c005f90 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:55:26.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.307+0000 7fefa8c00640 1 -- 172.21.15.5:0/2251185443 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fefa40bdbf0 con 0x7fefac10a590 2024-04-24T06:55:26.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.530+0000 7fefb2800640 1 -- 172.21.15.5:0/2251185443 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fefac1abc40 con 0x7fefac10a590 2024-04-24T06:55:26.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.531+0000 7fefa8c00640 1 -- 172.21.15.5:0/2251185443 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v99) ==== 81+0+926 (secure 0 0 0) 0x7fefa40bd4f0 con 0x7fefac10a590 2024-04-24T06:55:26.532 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/4042936965 2024-04-25T06:44:01.965881+0000 2024-04-24T06:55:26.532 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2912680052 2024-04-25T06:44:01.965881+0000 2024-04-24T06:55:26.532 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/4042936965 2024-04-25T06:44:01.965881+0000 2024-04-24T06:55:26.532 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/3478347847 2024-04-25T06:44:01.965881+0000 2024-04-24T06:55:26.533 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/23307164 2024-04-25T06:33:41.161852+0000 2024-04-24T06:55:26.533 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1012004002 2024-04-25T06:44:01.965881+0000 2024-04-24T06:55:26.533 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/2147029578 2024-04-25T06:33:41.161852+0000 2024-04-24T06:55:26.533 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/230785941 2024-04-25T06:33:41.161852+0000 2024-04-24T06:55:26.533 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/2147029578 2024-04-25T06:33:41.161852+0000 2024-04-24T06:55:26.533 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/2413496885 2024-04-25T06:33:07.040369+0000 2024-04-24T06:55:26.533 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/805658012 2024-04-25T06:44:01.965881+0000 2024-04-24T06:55:26.533 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2940023597 2024-04-25T06:33:07.040369+0000 2024-04-24T06:55:26.534 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/2413496885 2024-04-25T06:33:07.040369+0000 2024-04-24T06:55:26.534 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/3886613904 2024-04-25T06:33:07.040369+0000 2024-04-24T06:55:26.534 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/4199165437 2024-04-25T06:33:07.040369+0000 2024-04-24T06:55:26.534 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/3459593499 2024-04-25T06:33:41.161852+0000 2024-04-24T06:55:26.535 INFO:teuthology.orchestra.run.smithi005.stderr:listed 16 entries 2024-04-24T06:55:26.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.534+0000 7fefb2800640 1 -- 172.21.15.5:0/2251185443 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fef900a3d20 msgr2=0x7fef900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:26.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.534+0000 7fefb2800640 1 --2- 172.21.15.5:0/2251185443 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fef900a3d20 0x7fef900a61e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fef9c009fc0 tx=0x7fef9c005f90 comp rx=0 tx=0).stop 2024-04-24T06:55:26.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.534+0000 7fefb2800640 1 -- 172.21.15.5:0/2251185443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fefac10a590 msgr2=0x7fefac1a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:26.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.534+0000 7fefb2800640 1 --2- 172.21.15.5:0/2251185443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fefac10a590 0x7fefac1a37c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fefa4005e30 tx=0x7fefa402f5c0 comp rx=0 tx=0).stop 2024-04-24T06:55:26.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.534+0000 7fefb2800640 1 -- 172.21.15.5:0/2251185443 shutdown_connections 2024-04-24T06:55:26.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.534+0000 7fefb2800640 1 --2- 172.21.15.5:0/2251185443 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fef900a3d20 0x7fef900a61e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:26.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.534+0000 7fefb2800640 1 --2- 172.21.15.5:0/2251185443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fefac10a590 0x7fefac1a37c0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:26.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.534+0000 7fefb2800640 1 --2- 172.21.15.5:0/2251185443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefac1090c0 0x7fefac19f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:26.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.534+0000 7fefb2800640 1 --2- 172.21.15.5:0/2251185443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefac108720 0x7fefac19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:26.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.534+0000 7fefb2800640 1 -- 172.21.15.5:0/2251185443 >> 172.21.15.5:0/2251185443 conn(0x7fefac077e70 msgr2=0x7fefac078390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:55:26.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.534+0000 7fefb2800640 1 -- 172.21.15.5:0/2251185443 shutdown_connections 2024-04-24T06:55:26.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:26.535+0000 7fefb2800640 1 -- 172.21.15.5:0/2251185443 wait complete. 2024-04-24T06:55:26.556 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-24T06:55:26.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:55:26.556 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-24T06:55:26.643 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:26 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/4040942499' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T06:55:26.644 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:26 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/2226926756' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T06:55:26.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:26 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/4040942499' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T06:55:26.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:26 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/2226926756' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T06:55:26.761 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-04-24T06:55:26.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:55:26.761 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T06:55:26.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:26 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/4040942499' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T06:55:26.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:26 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/2226926756' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T06:55:26.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:55:26.846 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-04-24T06:55:26.888 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T06:55:26.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:55:26.888 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T06:55:26.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T06:55:26.994 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link delete ceph-brx 2024-04-24T06:55:27.091 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-24T06:55:27.104 INFO:tasks.cephadm:Teardown begin 2024-04-24T06:55:27.105 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-24T06:55:27.105 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:55:27.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.293+0000 7f1b3f000640 1 -- 172.21.15.5:0/1367161938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b38109530 msgr2=0x7f1b38111100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:27.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.293+0000 7f1b3f000640 1 --2- 172.21.15.5:0/1367161938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b38109530 0x7f1b38111100 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1b30009be0 tx=0x7f1b3002a6c0 comp rx=0 tx=0).stop 2024-04-24T06:55:27.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.293+0000 7f1b3f000640 1 -- 172.21.15.5:0/1367161938 shutdown_connections 2024-04-24T06:55:27.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.293+0000 7f1b3f000640 1 --2- 172.21.15.5:0/1367161938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b38109530 0x7f1b38111100 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:27.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.293+0000 7f1b3f000640 1 --2- 172.21.15.5:0/1367161938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b38104e10 0x7f1b38108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:27.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.293+0000 7f1b3f000640 1 --2- 172.21.15.5:0/1367161938 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b38104460 0x7f1b38104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:27.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.293+0000 7f1b3f000640 1 -- 172.21.15.5:0/1367161938 >> 172.21.15.5:0/1367161938 conn(0x7f1b380ffcc0 msgr2=0x7f1b381020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:55:27.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.294+0000 7f1b3f000640 1 -- 172.21.15.5:0/1367161938 shutdown_connections 2024-04-24T06:55:27.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.294+0000 7f1b3f000640 1 -- 172.21.15.5:0/1367161938 wait complete. 2024-04-24T06:55:27.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.295+0000 7f1b3f000640 1 Processor -- start 2024-04-24T06:55:27.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.295+0000 7f1b3f000640 1 -- start start 2024-04-24T06:55:27.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.295+0000 7f1b3f000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b38104460 0x7f1b381a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:27.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.295+0000 7f1b3f000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b38109530 0x7f1b381a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:27.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.295+0000 7f1b3f000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b3819a880 0x7f1b3819ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:27.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.295+0000 7f1b3f000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1b381145c0 con 0x7f1b3819a880 2024-04-24T06:55:27.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.295+0000 7f1b3f000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1b38114440 con 0x7f1b38104460 2024-04-24T06:55:27.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.295+0000 7f1b3f000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1b38114740 con 0x7f1b38109530 2024-04-24T06:55:27.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.295+0000 7f1b3d000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b3819a880 0x7f1b3819ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:27.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.295+0000 7f1b37e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b38104460 0x7f1b381a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:27.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.295+0000 7f1b2f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b38109530 0x7f1b381a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:27.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.295+0000 7f1b37e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b38104460 0x7f1b381a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:43630/0 (socket says 172.21.15.5:43630) 2024-04-24T06:55:27.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.295+0000 7f1b37e00640 1 -- 172.21.15.5:0/786062316 learned_addr learned my addr 172.21.15.5:0/786062316 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:55:27.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.296+0000 7f1b37e00640 1 -- 172.21.15.5:0/786062316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b38109530 msgr2=0x7f1b381a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:27.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.296+0000 7f1b37e00640 1 --2- 172.21.15.5:0/786062316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b38109530 0x7f1b381a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:27.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.296+0000 7f1b37e00640 1 -- 172.21.15.5:0/786062316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b3819a880 msgr2=0x7f1b3819ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:27.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.296+0000 7f1b37e00640 1 --2- 172.21.15.5:0/786062316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b3819a880 0x7f1b3819ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:27.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.296+0000 7f1b37e00640 1 -- 172.21.15.5:0/786062316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b3819b3a0 con 0x7f1b38104460 2024-04-24T06:55:27.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.296+0000 7f1b37e00640 1 --2- 172.21.15.5:0/786062316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b38104460 0x7f1b381a06d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800e9c0 tx=0x7f1b2800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:55:27.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.296+0000 7f1b35600640 1 -- 172.21.15.5:0/786062316 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2800ccc0 con 0x7f1b38104460 2024-04-24T06:55:27.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.297+0000 7f1b35600640 1 -- 172.21.15.5:0/786062316 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b28005500 con 0x7f1b38104460 2024-04-24T06:55:27.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.297+0000 7f1b35600640 1 -- 172.21.15.5:0/786062316 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b28010430 con 0x7f1b38104460 2024-04-24T06:55:27.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.297+0000 7f1b3f000640 1 -- 172.21.15.5:0/786062316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b3819b690 con 0x7f1b38104460 2024-04-24T06:55:27.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.297+0000 7f1b3f000640 1 -- 172.21.15.5:0/786062316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b381a7930 con 0x7f1b38104460 2024-04-24T06:55:27.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.298+0000 7f1b3f000640 1 -- 172.21.15.5:0/786062316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b38069300 con 0x7f1b38104460 2024-04-24T06:55:27.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.300+0000 7f1b35600640 1 -- 172.21.15.5:0/786062316 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f1b280046c0 con 0x7f1b38104460 2024-04-24T06:55:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.301+0000 7f1b35600640 1 --2- 172.21.15.5:0/786062316 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1b1c0a3da0 0x7f1b1c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.301+0000 7f1b35600640 1 -- 172.21.15.5:0/786062316 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f1b280d0aa0 con 0x7f1b38104460 2024-04-24T06:55:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.301+0000 7f1b2f400640 1 --2- 172.21.15.5:0/786062316 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1b1c0a3da0 0x7f1b1c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.302+0000 7f1b2f400640 1 --2- 172.21.15.5:0/786062316 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1b1c0a3da0 0x7f1b1c0a6260 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1b381033f0 tx=0x7f1b20005f90 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:55:27.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.303+0000 7f1b35600640 1 -- 172.21.15.5:0/786062316 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1b2809d0d0 con 0x7f1b38104460 2024-04-24T06:55:27.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.564+0000 7f1b3f000640 1 -- 172.21.15.5:0/786062316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b3819bdb0 con 0x7f1b38104460 2024-04-24T06:55:27.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.565+0000 7f1b35600640 1 -- 172.21.15.5:0/786062316 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11856 (secure 0 0 0) 0x7f1b2809c9d0 con 0x7f1b38104460 2024-04-24T06:55:27.566 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:55:27.566 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1735462750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1735462750},{"type":"v1","addr":"172.21.15.5:6837","nonce":1735462750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1499469370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1499469370},{"type":"v1","addr":"172.21.15.5:6841","nonce":1499469370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3705638847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3705638847},{"type":"v1","addr":"172.21.15.37:6835","nonce":3705638847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T06:41:34.764490+0000","modified":"2024-04-24T06:48:57.768410+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24550,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6837/1390103296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1390103296},{"type":"v1","addr":"172.21.15.37:6837","nonce":1390103296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3217754215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3217754215},{"type":"v1","addr":"172.21.15.137:6835","nonce":3217754215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6837/3044222976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3044222976},{"type":"v1","addr":"172.21.15.137:6837","nonce":3044222976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.5:6839/3173201459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3173201459},{"type":"v1","addr":"172.21.15.5:6839","nonce":3173201459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6839/1889774232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1889774232},{"type":"v1","addr":"172.21.15.137:6839","nonce":1889774232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6839/3536721753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":3536721753},{"type":"v1","addr":"172.21.15.37:6839","nonce":3536721753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24550,24569,24539]},"id":1}]} 2024-04-24T06:55:27.566 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-04-24T06:55:27.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.568+0000 7f1b3f000640 1 -- 172.21.15.5:0/786062316 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1b1c0a3da0 msgr2=0x7f1b1c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:27.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.568+0000 7f1b3f000640 1 --2- 172.21.15.5:0/786062316 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1b1c0a3da0 0x7f1b1c0a6260 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1b381033f0 tx=0x7f1b20005f90 comp rx=0 tx=0).stop 2024-04-24T06:55:27.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.568+0000 7f1b3f000640 1 -- 172.21.15.5:0/786062316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b38104460 msgr2=0x7f1b381a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:27.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.568+0000 7f1b3f000640 1 --2- 172.21.15.5:0/786062316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b38104460 0x7f1b381a06d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800e9c0 tx=0x7f1b2800ee80 comp rx=0 tx=0).stop 2024-04-24T06:55:27.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.568+0000 7f1b3f000640 1 -- 172.21.15.5:0/786062316 shutdown_connections 2024-04-24T06:55:27.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.568+0000 7f1b3f000640 1 --2- 172.21.15.5:0/786062316 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f1b1c0a3da0 0x7f1b1c0a6260 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:27.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.568+0000 7f1b3f000640 1 --2- 172.21.15.5:0/786062316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b3819a880 0x7f1b3819ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:27.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.568+0000 7f1b3f000640 1 --2- 172.21.15.5:0/786062316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b38109530 0x7f1b381a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:27.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.568+0000 7f1b3f000640 1 --2- 172.21.15.5:0/786062316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b38104460 0x7f1b381a06d0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:27.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.568+0000 7f1b3f000640 1 -- 172.21.15.5:0/786062316 >> 172.21.15.5:0/786062316 conn(0x7f1b380ffcc0 msgr2=0x7f1b38101fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:55:27.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.568+0000 7f1b3f000640 1 -- 172.21.15.5:0/786062316 shutdown_connections 2024-04-24T06:55:27.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.568+0000 7f1b3f000640 1 -- 172.21.15.5:0/786062316 wait complete. 2024-04-24T06:55:27.584 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:27 smithi005 ceph-mon[27798]: pgmap v355: 97 pgs: 97 active+clean; 85 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:55:27.584 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:27 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/2251185443' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T06:55:27.587 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T06:55:27.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:27 smithi137 ceph-mon[33017]: pgmap v355: 97 pgs: 97 active+clean; 85 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:55:27.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:27 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/2251185443' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T06:55:27.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:27 smithi037 ceph-mon[32864]: pgmap v355: 97 pgs: 97 active+clean; 85 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-24T06:55:27.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:27 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/2251185443' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T06:55:27.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.794+0000 7f1193e00640 1 -- 172.21.15.5:0/3940283545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1194102ac0 msgr2=0x7f1194106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:27.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.794+0000 7f1193e00640 1 --2- 172.21.15.5:0/3940283545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1194102ac0 0x7f1194106b00 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1180009900 tx=0x7f118002a3f0 comp rx=0 tx=0).stop 2024-04-24T06:55:27.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.794+0000 7f1193e00640 1 -- 172.21.15.5:0/3940283545 shutdown_connections 2024-04-24T06:55:27.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.794+0000 7f1193e00640 1 --2- 172.21.15.5:0/3940283545 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11941071e0 0x7f119410edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:27.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.794+0000 7f1193e00640 1 --2- 172.21.15.5:0/3940283545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1194102ac0 0x7f1194106b00 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:27.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.794+0000 7f1193e00640 1 --2- 172.21.15.5:0/3940283545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1194102110 0x7f11941024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:27.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.794+0000 7f1193e00640 1 -- 172.21.15.5:0/3940283545 >> 172.21.15.5:0/3940283545 conn(0x7f11940fda10 msgr2=0x7f11940ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:55:27.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.794+0000 7f1193e00640 1 -- 172.21.15.5:0/3940283545 shutdown_connections 2024-04-24T06:55:27.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.794+0000 7f1193e00640 1 -- 172.21.15.5:0/3940283545 wait complete. 2024-04-24T06:55:27.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.795+0000 7f1193e00640 1 Processor -- start 2024-04-24T06:55:27.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.795+0000 7f1193e00640 1 -- start start 2024-04-24T06:55:27.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.796+0000 7f1193e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1194102110 0x7f11941a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:27.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.796+0000 7f1193e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11941071e0 0x7f11941a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:27.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.796+0000 7f1193e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f119419a800 0x7f119419acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:27.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.796+0000 7f1193e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f11941122a0 con 0x7f1194102110 2024-04-24T06:55:27.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.796+0000 7f1193e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1194112120 con 0x7f11941071e0 2024-04-24T06:55:27.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.796+0000 7f1193e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1194112420 con 0x7f119419a800 2024-04-24T06:55:27.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.796+0000 7f1192a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1194102110 0x7f11941a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:27.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.796+0000 7f1192000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11941071e0 0x7f11941a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:27.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.796+0000 7f1192000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11941071e0 0x7f11941a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.5:43634/0 (socket says 172.21.15.5:43634) 2024-04-24T06:55:27.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.796+0000 7f1192000640 1 -- 172.21.15.5:0/3156430636 learned_addr learned my addr 172.21.15.5:0/3156430636 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:55:27.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.796+0000 7f1193400640 1 --2- 172.21.15.5:0/3156430636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f119419a800 0x7f119419acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:27.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.797+0000 7f1192000640 1 -- 172.21.15.5:0/3156430636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f119419a800 msgr2=0x7f119419acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:27.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.797+0000 7f1192000640 1 --2- 172.21.15.5:0/3156430636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f119419a800 0x7f119419acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:27.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.797+0000 7f1192000640 1 -- 172.21.15.5:0/3156430636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1194102110 msgr2=0x7f11941a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:27.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.797+0000 7f1192000640 1 --2- 172.21.15.5:0/3156430636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1194102110 0x7f11941a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:27.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.797+0000 7f1192000640 1 -- 172.21.15.5:0/3156430636 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f119419b390 con 0x7f11941071e0 2024-04-24T06:55:27.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.797+0000 7f1192a00640 1 --2- 172.21.15.5:0/3156430636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1194102110 0x7f11941a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:55:27.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.797+0000 7f1192000640 1 --2- 172.21.15.5:0/3156430636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11941071e0 0x7f11941a0bd0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f118002a910 tx=0x7f1180003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:55:27.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.797+0000 7f118b400640 1 -- 172.21.15.5:0/3156430636 <== mon.2 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118003f070 con 0x7f11941071e0 2024-04-24T06:55:27.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.797+0000 7f118b400640 1 -- 172.21.15.5:0/3156430636 <== mon.2 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1180031c90 con 0x7f11941071e0 2024-04-24T06:55:27.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.797+0000 7f118b400640 1 -- 172.21.15.5:0/3156430636 <== mon.2 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1180035030 con 0x7f11941071e0 2024-04-24T06:55:27.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.797+0000 7f1193e00640 1 -- 172.21.15.5:0/3156430636 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f119419b620 con 0x7f11941071e0 2024-04-24T06:55:27.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.797+0000 7f1193e00640 1 -- 172.21.15.5:0/3156430636 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11941a7800 con 0x7f11941071e0 2024-04-24T06:55:27.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.798+0000 7f1193e00640 1 -- 172.21.15.5:0/3156430636 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11941035e0 con 0x7f11941071e0 2024-04-24T06:55:27.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.801+0000 7f118b400640 1 -- 172.21.15.5:0/3156430636 <== mon.2 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f1180042070 con 0x7f11941071e0 2024-04-24T06:55:27.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.803+0000 7f118b400640 1 --2- 172.21.15.5:0/3156430636 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f11780a3d90 0x7f11780a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:27.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.803+0000 7f118b400640 1 -- 172.21.15.5:0/3156430636 <== mon.2 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f11800f15d0 con 0x7f11941071e0 2024-04-24T06:55:27.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.803+0000 7f1192a00640 1 --2- 172.21.15.5:0/3156430636 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f11780a3d90 0x7f11780a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:27.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.804+0000 7f1192a00640 1 --2- 172.21.15.5:0/3156430636 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f11780a3d90 0x7f11780a6250 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1184009fc0 tx=0x7f1184005f90 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:55:27.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:27.804+0000 7f118b400640 1 -- 172.21.15.5:0/3156430636 <== mon.2 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f11800bd5b0 con 0x7f11941071e0 2024-04-24T06:55:28.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.050+0000 7f1193e00640 1 -- 172.21.15.5:0/3156430636 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f119404fe00 con 0x7f11941071e0 2024-04-24T06:55:28.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.051+0000 7f118b400640 1 -- 172.21.15.5:0/3156430636 <== mon.2 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11856 (secure 0 0 0) 0x7f11800c18b0 con 0x7f11941071e0 2024-04-24T06:55:28.052 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:55:28.052 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1735462750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":1735462750},{"type":"v1","addr":"172.21.15.5:6837","nonce":1735462750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/1499469370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":1499469370},{"type":"v1","addr":"172.21.15.5:6841","nonce":1499469370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2228938014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2228938014},{"type":"v1","addr":"172.21.15.37:6833","nonce":2228938014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/681129825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":681129825},{"type":"v1","addr":"172.21.15.137:6833","nonce":681129825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2026364804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2026364804},{"type":"v1","addr":"172.21.15.5:6835","nonce":2026364804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3705638847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3705638847},{"type":"v1","addr":"172.21.15.37:6835","nonce":3705638847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T06:41:34.764490+0000","modified":"2024-04-24T06:48:57.768410+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24550,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6837/1390103296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1390103296},{"type":"v1","addr":"172.21.15.37:6837","nonce":1390103296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3217754215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3217754215},{"type":"v1","addr":"172.21.15.137:6835","nonce":3217754215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6837/3044222976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3044222976},{"type":"v1","addr":"172.21.15.137:6837","nonce":3044222976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.5:6839/3173201459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3173201459},{"type":"v1","addr":"172.21.15.5:6839","nonce":3173201459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6839/1889774232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1889774232},{"type":"v1","addr":"172.21.15.137:6839","nonce":1889774232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6839/3536721753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":3536721753},{"type":"v1","addr":"172.21.15.37:6839","nonce":3536721753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24550,24569,24539]},"id":1}]} 2024-04-24T06:55:28.053 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-04-24T06:55:28.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.054+0000 7f1193e00640 1 -- 172.21.15.5:0/3156430636 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f11780a3d90 msgr2=0x7f11780a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:28.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.054+0000 7f1193e00640 1 --2- 172.21.15.5:0/3156430636 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f11780a3d90 0x7f11780a6250 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1184009fc0 tx=0x7f1184005f90 comp rx=0 tx=0).stop 2024-04-24T06:55:28.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.054+0000 7f1193e00640 1 -- 172.21.15.5:0/3156430636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11941071e0 msgr2=0x7f11941a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:28.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.054+0000 7f1193e00640 1 --2- 172.21.15.5:0/3156430636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11941071e0 0x7f11941a0bd0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f118002a910 tx=0x7f1180003b90 comp rx=0 tx=0).stop 2024-04-24T06:55:28.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.054+0000 7f1193e00640 1 -- 172.21.15.5:0/3156430636 shutdown_connections 2024-04-24T06:55:28.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.054+0000 7f1193e00640 1 --2- 172.21.15.5:0/3156430636 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f11780a3d90 0x7f11780a6250 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:28.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.054+0000 7f1193e00640 1 --2- 172.21.15.5:0/3156430636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f119419a800 0x7f119419acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:28.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.054+0000 7f1193e00640 1 --2- 172.21.15.5:0/3156430636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11941071e0 0x7f11941a0bd0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:28.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.054+0000 7f1193e00640 1 --2- 172.21.15.5:0/3156430636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1194102110 0x7f11941a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:28.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.054+0000 7f1193e00640 1 -- 172.21.15.5:0/3156430636 >> 172.21.15.5:0/3156430636 conn(0x7f11940fda10 msgr2=0x7f11940ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:55:28.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.054+0000 7f1193e00640 1 -- 172.21.15.5:0/3156430636 shutdown_connections 2024-04-24T06:55:28.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.054+0000 7f1193e00640 1 -- 172.21.15.5:0/3156430636 wait complete. 2024-04-24T06:55:28.074 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T06:55:28.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.290+0000 7f4cbf200640 1 -- 172.21.15.5:0/2314806944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4cc010d8a0 msgr2=0x7f4cc0115420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:28.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.290+0000 7f4cbf200640 1 --2- 172.21.15.5:0/2314806944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4cc010d8a0 0x7f4cc0115420 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8009be0 tx=0x7f4cb802a6b0 comp rx=0 tx=0).stop 2024-04-24T06:55:28.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.290+0000 7f4cbf200640 1 -- 172.21.15.5:0/2314806944 shutdown_connections 2024-04-24T06:55:28.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.290+0000 7f4cbf200640 1 --2- 172.21.15.5:0/2314806944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4cc010d8a0 0x7f4cc0115420 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:28.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.290+0000 7f4cbf200640 1 --2- 172.21.15.5:0/2314806944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cc0109160 0x7f4cc010d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:28.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.290+0000 7f4cbf200640 1 --2- 172.21.15.5:0/2314806944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cc01087b0 0x7f4cc0108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:28.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.290+0000 7f4cbf200640 1 -- 172.21.15.5:0/2314806944 >> 172.21.15.5:0/2314806944 conn(0x7f4cc0077dd0 msgr2=0x7f4cc0106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:55:28.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.290+0000 7f4cbf200640 1 -- 172.21.15.5:0/2314806944 shutdown_connections 2024-04-24T06:55:28.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.290+0000 7f4cbf200640 1 -- 172.21.15.5:0/2314806944 wait complete. 2024-04-24T06:55:28.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.291+0000 7f4cbf200640 1 Processor -- start 2024-04-24T06:55:28.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.291+0000 7f4cbf200640 1 -- start start 2024-04-24T06:55:28.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.292+0000 7f4cbf200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4cc01087b0 0x7f4cc01a4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:28.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.292+0000 7f4cbf200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cc010d8a0 0x7f4cc01a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:28.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.292+0000 7f4cbf200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cc019eae0 0x7f4cc019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:28.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.292+0000 7f4cbf200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc01188e0 con 0x7f4cc019eae0 2024-04-24T06:55:28.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.292+0000 7f4cbf200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc0118760 con 0x7f4cc010d8a0 2024-04-24T06:55:28.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.292+0000 7f4cbf200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc0118a60 con 0x7f4cc01087b0 2024-04-24T06:55:28.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.292+0000 7f4cbe800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cc019eae0 0x7f4cc019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:28.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.292+0000 7f4cbde00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4cc01087b0 0x7f4cc01a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:28.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.292+0000 7f4cbe800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cc019eae0 0x7f4cc019ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39618/0 (socket says 172.21.15.5:39618) 2024-04-24T06:55:28.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.292+0000 7f4cbe800640 1 -- 172.21.15.5:0/3116391724 learned_addr learned my addr 172.21.15.5:0/3116391724 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:55:28.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.292+0000 7f4cb7e00640 1 --2- 172.21.15.5:0/3116391724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cc010d8a0 0x7f4cc01a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:28.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.292+0000 7f4cbde00640 1 -- 172.21.15.5:0/3116391724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cc010d8a0 msgr2=0x7f4cc01a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:28.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.292+0000 7f4cbde00640 1 --2- 172.21.15.5:0/3116391724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cc010d8a0 0x7f4cc01a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:28.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.292+0000 7f4cbde00640 1 -- 172.21.15.5:0/3116391724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cc019eae0 msgr2=0x7f4cc019ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:28.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.292+0000 7f4cbde00640 1 --2- 172.21.15.5:0/3116391724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cc019eae0 0x7f4cc019ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:28.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.292+0000 7f4cbde00640 1 -- 172.21.15.5:0/3116391724 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc019f600 con 0x7f4cc01087b0 2024-04-24T06:55:28.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.292+0000 7f4cbe800640 1 --2- 172.21.15.5:0/3116391724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cc019eae0 0x7f4cc019ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:55:28.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.293+0000 7f4cbde00640 1 --2- 172.21.15.5:0/3116391724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4cc01087b0 0x7f4cc01a4930 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4cb000e9f0 tx=0x7f4cb000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:55:28.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.293+0000 7f4cb6a00640 1 -- 172.21.15.5:0/3116391724 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb000ccc0 con 0x7f4cc01087b0 2024-04-24T06:55:28.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.293+0000 7f4cb6a00640 1 -- 172.21.15.5:0/3116391724 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cb0005020 con 0x7f4cc01087b0 2024-04-24T06:55:28.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.293+0000 7f4cb6a00640 1 -- 172.21.15.5:0/3116391724 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb00049e0 con 0x7f4cc01087b0 2024-04-24T06:55:28.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.293+0000 7f4cbf200640 1 -- 172.21.15.5:0/3116391724 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cc019f8f0 con 0x7f4cc01087b0 2024-04-24T06:55:28.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.293+0000 7f4cbf200640 1 -- 172.21.15.5:0/3116391724 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cc01abaf0 con 0x7f4cc01087b0 2024-04-24T06:55:28.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.294+0000 7f4cbf200640 1 -- 172.21.15.5:0/3116391724 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cc004fe00 con 0x7f4cc01087b0 2024-04-24T06:55:28.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.297+0000 7f4cb6a00640 1 -- 172.21.15.5:0/3116391724 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f4cb0005560 con 0x7f4cc01087b0 2024-04-24T06:55:28.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.298+0000 7f4cb6a00640 1 --2- 172.21.15.5:0/3116391724 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f4ca40a3d90 0x7f4ca40a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:28.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.298+0000 7f4cb6a00640 1 -- 172.21.15.5:0/3116391724 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f4cb00d0800 con 0x7f4cc01087b0 2024-04-24T06:55:28.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.299+0000 7f4cb7e00640 1 --2- 172.21.15.5:0/3116391724 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f4ca40a3d90 0x7f4ca40a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:28.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.299+0000 7f4cb7e00640 1 --2- 172.21.15.5:0/3116391724 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f4ca40a3d90 0x7f4ca40a6250 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0078bf0 tx=0x7f4ca8009260 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:55:28.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.301+0000 7f4cb6a00640 1 -- 172.21.15.5:0/3116391724 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4cb00051d0 con 0x7f4cc01087b0 2024-04-24T06:55:28.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.512+0000 7f4cbf200640 1 -- 172.21.15.5:0/3116391724 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4cc019ffd0 con 0x7f4cc01087b0 2024-04-24T06:55:28.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.514+0000 7f4cb6a00640 1 -- 172.21.15.5:0/3116391724 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+22974 (secure 0 0 0) 0x7f4cb009cec0 con 0x7f4cc01087b0 2024-04-24T06:55:28.515 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-24T06:55:28.515 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":99,"fsid":"43e9bad0-0204-11ef-bc93-c7b262605968","created":"2024-04-24T06:32:26.340555+0000","modified":"2024-04-24T06:55:16.104158+0000","last_up_change":"2024-04-24T06:40:36.790934+0000","last_in_change":"2024-04-24T06:40:17.803257+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T06:36:27.320342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T06:41:32.254502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T06:41:33.739067+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":98,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c2e974c-745c-4dd5-8c3d-8e8da2eb1ce0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6803","nonce":2155858515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6805","nonce":2155858515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6809","nonce":2155858515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2155858515},{"type":"v1","addr":"172.21.15.5:6807","nonce":2155858515}]},"public_addr":"172.21.15.5:6803/2155858515","cluster_addr":"172.21.15.5:6805/2155858515","heartbeat_back_addr":"172.21.15.5:6809/2155858515","heartbeat_front_addr":"172.21.15.5:6807/2155858515","state":["exists","up"]},{"osd":1,"uuid":"50467744-db20-4721-bf2d-1c5db938297b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6801","nonce":1302362650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6803","nonce":1302362650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6807","nonce":1302362650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1302362650},{"type":"v1","addr":"172.21.15.37:6805","nonce":1302362650}]},"public_addr":"172.21.15.37:6801/1302362650","cluster_addr":"172.21.15.37:6803/1302362650","heartbeat_back_addr":"172.21.15.37:6807/1302362650","heartbeat_front_addr":"172.21.15.37:6805/1302362650","state":["exists","up"]},{"osd":2,"uuid":"0e190b15-d04c-4288-8d48-ecf6664392fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6801","nonce":2885027256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6803","nonce":2885027256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6807","nonce":2885027256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":2885027256},{"type":"v1","addr":"172.21.15.137:6805","nonce":2885027256}]},"public_addr":"172.21.15.137:6801/2885027256","cluster_addr":"172.21.15.137:6803/2885027256","heartbeat_back_addr":"172.21.15.137:6807/2885027256","heartbeat_front_addr":"172.21.15.137:6805/2885027256","state":["exists","up"]},{"osd":3,"uuid":"e05592ba-d96a-472a-9f63-09a535d04dbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6811","nonce":3701747260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6813","nonce":3701747260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6817","nonce":3701747260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":3701747260},{"type":"v1","addr":"172.21.15.5:6815","nonce":3701747260}]},"public_addr":"172.21.15.5:6811/3701747260","cluster_addr":"172.21.15.5:6813/3701747260","heartbeat_back_addr":"172.21.15.5:6817/3701747260","heartbeat_front_addr":"172.21.15.5:6815/3701747260","state":["exists","up"]},{"osd":4,"uuid":"c3f36e51-7a3b-4dd9-af4b-a10558830c4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6809","nonce":759253844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6811","nonce":759253844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6815","nonce":759253844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":759253844},{"type":"v1","addr":"172.21.15.37:6813","nonce":759253844}]},"public_addr":"172.21.15.37:6809/759253844","cluster_addr":"172.21.15.37:6811/759253844","heartbeat_back_addr":"172.21.15.37:6815/759253844","heartbeat_front_addr":"172.21.15.37:6813/759253844","state":["exists","up"]},{"osd":5,"uuid":"97a0500c-cbb8-4ff5-b476-8634b26b3821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6809","nonce":3771921560}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6811","nonce":3771921560}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6815","nonce":3771921560}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3771921560},{"type":"v1","addr":"172.21.15.137:6813","nonce":3771921560}]},"public_addr":"172.21.15.137:6809/3771921560","cluster_addr":"172.21.15.137:6811/3771921560","heartbeat_back_addr":"172.21.15.137:6815/3771921560","heartbeat_front_addr":"172.21.15.137:6813/3771921560","state":["exists","up"]},{"osd":6,"uuid":"33a33d62-bbb3-4ed9-834e-19ae93a97670","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6819","nonce":2064636084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6821","nonce":2064636084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6825","nonce":2064636084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2064636084},{"type":"v1","addr":"172.21.15.5:6823","nonce":2064636084}]},"public_addr":"172.21.15.5:6819/2064636084","cluster_addr":"172.21.15.5:6821/2064636084","heartbeat_back_addr":"172.21.15.5:6825/2064636084","heartbeat_front_addr":"172.21.15.5:6823/2064636084","state":["exists","up"]},{"osd":7,"uuid":"c0f4e036-4717-4514-bb59-ab11fbaab063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6817","nonce":2952750659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6819","nonce":2952750659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6823","nonce":2952750659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2952750659},{"type":"v1","addr":"172.21.15.37:6821","nonce":2952750659}]},"public_addr":"172.21.15.37:6817/2952750659","cluster_addr":"172.21.15.37:6819/2952750659","heartbeat_back_addr":"172.21.15.37:6823/2952750659","heartbeat_front_addr":"172.21.15.37:6821/2952750659","state":["exists","up"]},{"osd":8,"uuid":"5391fd87-44ea-440f-80da-c54d0327c04c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6817","nonce":29724945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6819","nonce":29724945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6823","nonce":29724945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":29724945},{"type":"v1","addr":"172.21.15.137:6821","nonce":29724945}]},"public_addr":"172.21.15.137:6817/29724945","cluster_addr":"172.21.15.137:6819/29724945","heartbeat_back_addr":"172.21.15.137:6823/29724945","heartbeat_front_addr":"172.21.15.137:6821/29724945","state":["exists","up"]},{"osd":9,"uuid":"d94c9dbc-f43d-4237-ac7f-47a6a5ede1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6827","nonce":2257102107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6829","nonce":2257102107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6833","nonce":2257102107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2257102107},{"type":"v1","addr":"172.21.15.5:6831","nonce":2257102107}]},"public_addr":"172.21.15.5:6827/2257102107","cluster_addr":"172.21.15.5:6829/2257102107","heartbeat_back_addr":"172.21.15.5:6833/2257102107","heartbeat_front_addr":"172.21.15.5:6831/2257102107","state":["exists","up"]},{"osd":10,"uuid":"e2455332-2a5e-44b1-85d3-8e47da399f9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6825","nonce":2451570637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6827","nonce":2451570637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6831","nonce":2451570637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2451570637},{"type":"v1","addr":"172.21.15.37:6829","nonce":2451570637}]},"public_addr":"172.21.15.37:6825/2451570637","cluster_addr":"172.21.15.37:6827/2451570637","heartbeat_back_addr":"172.21.15.37:6831/2451570637","heartbeat_front_addr":"172.21.15.37:6829/2451570637","state":["exists","up"]},{"osd":11,"uuid":"49d9396b-704a-4cef-8210-75d30c28e924","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6825","nonce":90600596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6827","nonce":90600596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6831","nonce":90600596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":90600596},{"type":"v1","addr":"172.21.15.137:6829","nonce":90600596}]},"public_addr":"172.21.15.137:6825/90600596","cluster_addr":"172.21.15.137:6827/90600596","heartbeat_back_addr":"172.21.15.137:6831/90600596","heartbeat_front_addr":"172.21.15.137:6829/90600596","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:35:35.591619+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:00.568817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:24.195643+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:36:56.451724+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:22.109293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:37:46.161749+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:19.278305+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:38:45.620256+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:10.041165+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:39:43.813697+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:10.736026+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T06:40:35.384377+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6800/4042936965":"2024-04-25T06:44:01.965881+0000","172.21.15.5:0/2912680052":"2024-04-25T06:44:01.965881+0000","172.21.15.5:6801/4042936965":"2024-04-25T06:44:01.965881+0000","172.21.15.5:0/3478347847":"2024-04-25T06:44:01.965881+0000","172.21.15.5:0/23307164":"2024-04-25T06:33:41.161852+0000","172.21.15.5:0/1012004002":"2024-04-25T06:44:01.965881+0000","172.21.15.5:6801/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:0/230785941":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2147029578":"2024-04-25T06:33:41.161852+0000","172.21.15.5:6800/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/805658012":"2024-04-25T06:44:01.965881+0000","172.21.15.5:0/2940023597":"2024-04-25T06:33:07.040369+0000","172.21.15.5:6801/2413496885":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3886613904":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/4199165437":"2024-04-25T06:33:07.040369+0000","172.21.15.5:0/3459593499":"2024-04-25T06:33:41.161852+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T06:55:28.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.517+0000 7f4cbf200640 1 -- 172.21.15.5:0/3116391724 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f4ca40a3d90 msgr2=0x7f4ca40a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:28.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.517+0000 7f4cbf200640 1 --2- 172.21.15.5:0/3116391724 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f4ca40a3d90 0x7f4ca40a6250 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0078bf0 tx=0x7f4ca8009260 comp rx=0 tx=0).stop 2024-04-24T06:55:28.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.517+0000 7f4cbf200640 1 -- 172.21.15.5:0/3116391724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4cc01087b0 msgr2=0x7f4cc01a4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:28.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.517+0000 7f4cbf200640 1 --2- 172.21.15.5:0/3116391724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4cc01087b0 0x7f4cc01a4930 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4cb000e9f0 tx=0x7f4cb000eeb0 comp rx=0 tx=0).stop 2024-04-24T06:55:28.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.517+0000 7f4cbf200640 1 -- 172.21.15.5:0/3116391724 shutdown_connections 2024-04-24T06:55:28.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.517+0000 7f4cbf200640 1 --2- 172.21.15.5:0/3116391724 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f4ca40a3d90 0x7f4ca40a6250 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:28.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.517+0000 7f4cbf200640 1 --2- 172.21.15.5:0/3116391724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cc019eae0 0x7f4cc019ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:28.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.517+0000 7f4cbf200640 1 --2- 172.21.15.5:0/3116391724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cc010d8a0 0x7f4cc01a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:28.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.517+0000 7f4cbf200640 1 --2- 172.21.15.5:0/3116391724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4cc01087b0 0x7f4cc01a4930 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:28.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.517+0000 7f4cbf200640 1 -- 172.21.15.5:0/3116391724 >> 172.21.15.5:0/3116391724 conn(0x7f4cc0077dd0 msgr2=0x7f4cc010e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:55:28.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.517+0000 7f4cbf200640 1 -- 172.21.15.5:0/3116391724 shutdown_connections 2024-04-24T06:55:28.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.517+0000 7f4cbf200640 1 -- 172.21.15.5:0/3116391724 wait complete. 2024-04-24T06:55:28.536 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T06:55:28.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:28 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/786062316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:55:28.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:28 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/3156430636' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:55:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.748+0000 7fb4c4e00640 1 -- 172.21.15.5:0/179270723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4c0108ee0 msgr2=0x7fb4c01152b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.749+0000 7fb4c4e00640 1 --2- 172.21.15.5:0/179270723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4c0108ee0 0x7fb4c01152b0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8009ca0 tx=0x7fb4b802a950 comp rx=0 tx=0).stop 2024-04-24T06:55:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.749+0000 7fb4c4e00640 1 -- 172.21.15.5:0/179270723 shutdown_connections 2024-04-24T06:55:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.749+0000 7fb4c4e00640 1 --2- 172.21.15.5:0/179270723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4c0108ee0 0x7fb4c01152b0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.749+0000 7fb4c4e00640 1 --2- 172.21.15.5:0/179270723 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4c0108540 0x7fb4c01089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.749+0000 7fb4c4e00640 1 --2- 172.21.15.5:0/179270723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4c010e4f0 0x7fb4c010e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.749+0000 7fb4c4e00640 1 -- 172.21.15.5:0/179270723 >> 172.21.15.5:0/179270723 conn(0x7fb4c0077e10 msgr2=0x7fb4c0106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:55:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.749+0000 7fb4c4e00640 1 -- 172.21.15.5:0/179270723 shutdown_connections 2024-04-24T06:55:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.749+0000 7fb4c4e00640 1 -- 172.21.15.5:0/179270723 wait complete. 2024-04-24T06:55:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.750+0000 7fb4c4e00640 1 Processor -- start 2024-04-24T06:55:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.750+0000 7fb4c4e00640 1 -- start start 2024-04-24T06:55:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.750+0000 7fb4c4e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4c0108540 0x7fb4c01a48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:28.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.750+0000 7fb4c4e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4c0108ee0 0x7fb4c01a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:28.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.750+0000 7fb4c4e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4c010e4f0 0x7fb4c019ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:28.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.750+0000 7fb4c4e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c01187a0 con 0x7fb4c0108540 2024-04-24T06:55:28.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.750+0000 7fb4c4e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c0118620 con 0x7fb4c0108ee0 2024-04-24T06:55:28.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.750+0000 7fb4c4e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c0118920 con 0x7fb4c010e4f0 2024-04-24T06:55:28.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.750+0000 7fb4bf400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4c0108540 0x7fb4c01a48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:28.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.751+0000 7fb4bf400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4c0108540 0x7fb4c01a48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39624/0 (socket says 172.21.15.5:39624) 2024-04-24T06:55:28.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.751+0000 7fb4bf400640 1 -- 172.21.15.5:0/3558264947 learned_addr learned my addr 172.21.15.5:0/3558264947 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:55:28.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.751+0000 7fb4b7e00640 1 --2- 172.21.15.5:0/3558264947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4c0108ee0 0x7fb4c01a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:28.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.751+0000 7fb4bfe00640 1 --2- 172.21.15.5:0/3558264947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4c010e4f0 0x7fb4c019ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:28.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.751+0000 7fb4bf400640 1 -- 172.21.15.5:0/3558264947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4c010e4f0 msgr2=0x7fb4c019ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:28.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.751+0000 7fb4bf400640 1 --2- 172.21.15.5:0/3558264947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4c010e4f0 0x7fb4c019ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:28.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.751+0000 7fb4bf400640 1 -- 172.21.15.5:0/3558264947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4c0108ee0 msgr2=0x7fb4c01a4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:28.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.751+0000 7fb4bf400640 1 --2- 172.21.15.5:0/3558264947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4c0108ee0 0x7fb4c01a4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:28.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.751+0000 7fb4bf400640 1 -- 172.21.15.5:0/3558264947 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4c019f200 con 0x7fb4c0108540 2024-04-24T06:55:28.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.751+0000 7fb4bf400640 1 --2- 172.21.15.5:0/3558264947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4c0108540 0x7fb4c01a48d0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb4b000e9f0 tx=0x7fb4b000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:55:28.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.752+0000 7fb4bcc00640 1 -- 172.21.15.5:0/3558264947 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b000cd80 con 0x7fb4c0108540 2024-04-24T06:55:28.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.752+0000 7fb4bcc00640 1 -- 172.21.15.5:0/3558264947 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4b0010910 con 0x7fb4c0108540 2024-04-24T06:55:28.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.752+0000 7fb4c4e00640 1 -- 172.21.15.5:0/3558264947 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4c019f4f0 con 0x7fb4c0108540 2024-04-24T06:55:28.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.752+0000 7fb4bcc00640 1 -- 172.21.15.5:0/3558264947 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b0018990 con 0x7fb4c0108540 2024-04-24T06:55:28.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.752+0000 7fb4c4e00640 1 -- 172.21.15.5:0/3558264947 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4c01ab790 con 0x7fb4c0108540 2024-04-24T06:55:28.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.753+0000 7fb4c4e00640 1 -- 172.21.15.5:0/3558264947 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4c004fd90 con 0x7fb4c0108540 2024-04-24T06:55:28.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:28 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/786062316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:55:28.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:28 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/3156430636' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:55:28.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.755+0000 7fb4bcc00640 1 -- 172.21.15.5:0/3558264947 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fb4b0010ac0 con 0x7fb4c0108540 2024-04-24T06:55:28.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.757+0000 7fb4bcc00640 1 --2- 172.21.15.5:0/3558264947 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb4a40a3d20 0x7fb4a40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:28.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.757+0000 7fb4bcc00640 1 -- 172.21.15.5:0/3558264947 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7fb4b00d0ad0 con 0x7fb4c0108540 2024-04-24T06:55:28.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.758+0000 7fb4b7e00640 1 --2- 172.21.15.5:0/3558264947 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb4a40a3d20 0x7fb4a40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:28.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.758+0000 7fb4b7e00640 1 --2- 172.21.15.5:0/3558264947 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb4a40a3d20 0x7fb4a40a61e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0108090 tx=0x7fb4a8005f90 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:55:28.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:28.762+0000 7fb4bcc00640 1 -- 172.21.15.5:0/3558264947 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb4b009cab0 con 0x7fb4c0108540 2024-04-24T06:55:28.767 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:28 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/786062316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:55:28.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:28 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/3156430636' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T06:55:29.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.010+0000 7fb4c4e00640 1 -- 172.21.15.5:0/3558264947 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb4c0110900 con 0x7fb4c0108540 2024-04-24T06:55:29.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.014+0000 7fb4bcc00640 1 -- 172.21.15.5:0/3558264947 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) ==== 161+0+0 (secure 0 0 0) 0x7fb4b00a0db0 con 0x7fb4c0108540 2024-04-24T06:55:29.015 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T06:55:29.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.018+0000 7fb4c4e00640 1 -- 172.21.15.5:0/3558264947 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb4a40a3d20 msgr2=0x7fb4a40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:29.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.018+0000 7fb4c4e00640 1 --2- 172.21.15.5:0/3558264947 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb4a40a3d20 0x7fb4a40a61e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0108090 tx=0x7fb4a8005f90 comp rx=0 tx=0).stop 2024-04-24T06:55:29.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.018+0000 7fb4c4e00640 1 -- 172.21.15.5:0/3558264947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4c0108540 msgr2=0x7fb4c01a48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:29.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.018+0000 7fb4c4e00640 1 --2- 172.21.15.5:0/3558264947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4c0108540 0x7fb4c01a48d0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb4b000e9f0 tx=0x7fb4b000eeb0 comp rx=0 tx=0).stop 2024-04-24T06:55:29.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.018+0000 7fb4c4e00640 1 -- 172.21.15.5:0/3558264947 shutdown_connections 2024-04-24T06:55:29.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.018+0000 7fb4c4e00640 1 --2- 172.21.15.5:0/3558264947 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb4a40a3d20 0x7fb4a40a61e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:29.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.018+0000 7fb4c4e00640 1 --2- 172.21.15.5:0/3558264947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4c010e4f0 0x7fb4c019ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:29.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.018+0000 7fb4c4e00640 1 --2- 172.21.15.5:0/3558264947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4c0108ee0 0x7fb4c01a4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:29.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.018+0000 7fb4c4e00640 1 --2- 172.21.15.5:0/3558264947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4c0108540 0x7fb4c01a48d0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:29.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.018+0000 7fb4c4e00640 1 -- 172.21.15.5:0/3558264947 >> 172.21.15.5:0/3558264947 conn(0x7fb4c0077e10 msgr2=0x7fb4c0078430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:55:29.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.018+0000 7fb4c4e00640 1 -- 172.21.15.5:0/3558264947 shutdown_connections 2024-04-24T06:55:29.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.019+0000 7fb4c4e00640 1 -- 172.21.15.5:0/3558264947 wait complete. 2024-04-24T06:55:29.040 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-24T06:55:29.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.231+0000 7fa06a800640 1 -- 172.21.15.5:0/2494117631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa064109350 msgr2=0x7fa064110f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:29.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.231+0000 7fa06a800640 1 --2- 172.21.15.5:0/2494117631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa064109350 0x7fa064110f20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa05c009ca0 tx=0x7fa05c02a8c0 comp rx=0 tx=0).stop 2024-04-24T06:55:29.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.231+0000 7fa06a800640 1 -- 172.21.15.5:0/2494117631 shutdown_connections 2024-04-24T06:55:29.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.231+0000 7fa06a800640 1 --2- 172.21.15.5:0/2494117631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa064109350 0x7fa064110f20 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:29.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.231+0000 7fa06a800640 1 --2- 172.21.15.5:0/2494117631 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa064104c30 0x7fa064108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:29.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.231+0000 7fa06a800640 1 --2- 172.21.15.5:0/2494117631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa064104280 0x7fa064104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:29.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.231+0000 7fa06a800640 1 -- 172.21.15.5:0/2494117631 >> 172.21.15.5:0/2494117631 conn(0x7fa0640ffae0 msgr2=0x7fa064101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:55:29.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.231+0000 7fa06a800640 1 -- 172.21.15.5:0/2494117631 shutdown_connections 2024-04-24T06:55:29.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.231+0000 7fa06a800640 1 -- 172.21.15.5:0/2494117631 wait complete. 2024-04-24T06:55:29.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.232+0000 7fa06a800640 1 Processor -- start 2024-04-24T06:55:29.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.233+0000 7fa06a800640 1 -- start start 2024-04-24T06:55:29.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.233+0000 7fa06a800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa064104280 0x7fa0641a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:29.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.233+0000 7fa06a800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa064109350 0x7fa0641a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:29.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.233+0000 7fa06a800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa06419a780 0x7fa06419ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:29.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.233+0000 7fa06a800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa064114460 con 0x7fa064104280 2024-04-24T06:55:29.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.233+0000 7fa06a800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa0641142e0 con 0x7fa06419a780 2024-04-24T06:55:29.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.233+0000 7fa06a800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa0641145e0 con 0x7fa064109350 2024-04-24T06:55:29.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.233+0000 7fa069400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa064104280 0x7fa0641a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:29.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.233+0000 7fa068a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa064109350 0x7fa0641a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:29.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.233+0000 7fa068a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa064109350 0x7fa0641a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:33624/0 (socket says 172.21.15.5:33624) 2024-04-24T06:55:29.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.233+0000 7fa068a00640 1 -- 172.21.15.5:0/3696209554 learned_addr learned my addr 172.21.15.5:0/3696209554 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:55:29.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.233+0000 7fa069e00640 1 --2- 172.21.15.5:0/3696209554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa06419a780 0x7fa06419ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:29.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.234+0000 7fa068a00640 1 -- 172.21.15.5:0/3696209554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa06419a780 msgr2=0x7fa06419ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:29.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.234+0000 7fa068a00640 1 --2- 172.21.15.5:0/3696209554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa06419a780 0x7fa06419ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:29.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.234+0000 7fa068a00640 1 -- 172.21.15.5:0/3696209554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa064104280 msgr2=0x7fa0641a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:29.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.234+0000 7fa068a00640 1 --2- 172.21.15.5:0/3696209554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa064104280 0x7fa0641a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:29.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.234+0000 7fa068a00640 1 -- 172.21.15.5:0/3696209554 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa06419b380 con 0x7fa064109350 2024-04-24T06:55:29.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.234+0000 7fa068a00640 1 --2- 172.21.15.5:0/3696209554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa064109350 0x7fa0641a0bb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa05000e9f0 tx=0x7fa05000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:55:29.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.234+0000 7fa062000640 1 -- 172.21.15.5:0/3696209554 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05000ccc0 con 0x7fa064109350 2024-04-24T06:55:29.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.234+0000 7fa062000640 1 -- 172.21.15.5:0/3696209554 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa050005500 con 0x7fa064109350 2024-04-24T06:55:29.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.234+0000 7fa062000640 1 -- 172.21.15.5:0/3696209554 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050010690 con 0x7fa064109350 2024-04-24T06:55:29.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.234+0000 7fa06a800640 1 -- 172.21.15.5:0/3696209554 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0641a7410 con 0x7fa064109350 2024-04-24T06:55:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.235+0000 7fa06a800640 1 -- 172.21.15.5:0/3696209554 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0641a7810 con 0x7fa064109350 2024-04-24T06:55:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.236+0000 7fa06a800640 1 -- 172.21.15.5:0/3696209554 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa064105750 con 0x7fa064109350 2024-04-24T06:55:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.238+0000 7fa062000640 1 -- 172.21.15.5:0/3696209554 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fa0500046c0 con 0x7fa064109350 2024-04-24T06:55:29.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.239+0000 7fa062000640 1 --2- 172.21.15.5:0/3696209554 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fa04c0a3da0 0x7fa04c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:29.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.240+0000 7fa062000640 1 -- 172.21.15.5:0/3696209554 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8756+0+0 (secure 0 0 0) 0x7fa0500d0b10 con 0x7fa064109350 2024-04-24T06:55:29.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.240+0000 7fa069400640 1 --2- 172.21.15.5:0/3696209554 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fa04c0a3da0 0x7fa04c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:29.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.240+0000 7fa069400640 1 --2- 172.21.15.5:0/3696209554 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fa04c0a3da0 0x7fa04c0a6260 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa058009bd0 tx=0x7fa058005f90 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:55:29.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.241+0000 7fa062000640 1 -- 172.21.15.5:0/3696209554 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa05009d060 con 0x7fa064109350 2024-04-24T06:55:29.403 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:29 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mon-a[27794]: 2024-04-24T06:55:29.012+0000 7f8361e00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T06:55:29.404 INFO:journalctl@ceph.mds.b.smithi037.stdout:Apr 24 06:55:29 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-b[63330]: ignoring --setuser ceph since I am not root 2024-04-24T06:55:29.405 INFO:journalctl@ceph.mds.b.smithi037.stdout:Apr 24 06:55:29 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-b[63330]: ignoring --setgroup ceph since I am not root 2024-04-24T06:55:29.405 INFO:journalctl@ceph.mds.b.smithi037.stdout:Apr 24 06:55:29 smithi037 ceph-mds[63334]: -- 172.21.15.37:0/403163762 <== mon.2 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560269a4da40 con 0x560269a20c00 2024-04-24T06:55:29.405 INFO:journalctl@ceph.mds.b.smithi037.stdout:Apr 24 06:55:29 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-b[63330]: starting mds.b at 2024-04-24T06:55:29.405 INFO:journalctl@ceph.mds.c.smithi137.stdout:Apr 24 06:55:29 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-c[63762]: ignoring --setuser ceph since I am not root 2024-04-24T06:55:29.405 INFO:journalctl@ceph.mds.c.smithi137.stdout:Apr 24 06:55:29 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-c[63762]: ignoring --setgroup ceph since I am not root 2024-04-24T06:55:29.406 INFO:journalctl@ceph.mds.c.smithi137.stdout:Apr 24 06:55:29 smithi137 ceph-mds[63766]: -- 172.21.15.137:0/2043582626 <== mon.1 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563d4cf81a40 con 0x563d4cf55000 2024-04-24T06:55:29.406 INFO:journalctl@ceph.mds.c.smithi137.stdout:Apr 24 06:55:29 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-c[63762]: starting mds.c at 2024-04-24T06:55:29.406 INFO:journalctl@ceph.mds.g.smithi005.stdout:Apr 24 06:55:29 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-g[65603]: ignoring --setuser ceph since I am not root 2024-04-24T06:55:29.406 INFO:journalctl@ceph.mds.g.smithi005.stdout:Apr 24 06:55:29 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-g[65603]: ignoring --setgroup ceph since I am not root 2024-04-24T06:55:29.406 INFO:journalctl@ceph.mds.g.smithi005.stdout:Apr 24 06:55:29 smithi005 ceph-mds[65607]: -- 172.21.15.5:0/128191015 <== mon.2 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b927535a40 con 0x55b927508800 2024-04-24T06:55:29.407 INFO:journalctl@ceph.mds.g.smithi005.stdout:Apr 24 06:55:29 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-g[65603]: starting mds.g at 2024-04-24T06:55:29.407 INFO:journalctl@ceph.mds.h.smithi037.stdout:Apr 24 06:55:29 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-h[63026]: ignoring --setuser ceph since I am not root 2024-04-24T06:55:29.407 INFO:journalctl@ceph.mds.h.smithi037.stdout:Apr 24 06:55:29 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-h[63026]: ignoring --setgroup ceph since I am not root 2024-04-24T06:55:29.407 INFO:journalctl@ceph.mds.h.smithi037.stdout:Apr 24 06:55:29 smithi037 ceph-mds[63030]: -- 172.21.15.37:0/1325254016 <== mon.2 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5635383b9a40 con 0x56353838cc00 2024-04-24T06:55:29.407 INFO:journalctl@ceph.mds.h.smithi037.stdout:Apr 24 06:55:29 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-h[63026]: starting mds.h at 2024-04-24T06:55:29.407 INFO:journalctl@ceph.mds.f.smithi137.stdout:Apr 24 06:55:29 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-f[63455]: ignoring --setuser ceph since I am not root 2024-04-24T06:55:29.408 INFO:journalctl@ceph.mds.f.smithi137.stdout:Apr 24 06:55:29 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-f[63455]: ignoring --setgroup ceph since I am not root 2024-04-24T06:55:29.408 INFO:journalctl@ceph.mds.f.smithi137.stdout:Apr 24 06:55:29 smithi137 ceph-mds[63459]: -- 172.21.15.137:0/2889617770 <== mon.1 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562320f0da40 con 0x562320ee0c00 2024-04-24T06:55:29.408 INFO:journalctl@ceph.mds.f.smithi137.stdout:Apr 24 06:55:29 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-f[63455]: starting mds.f at 2024-04-24T06:55:29.408 INFO:journalctl@ceph.mds.i.smithi137.stdout:Apr 24 06:55:29 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-i[64073]: ignoring --setuser ceph since I am not root 2024-04-24T06:55:29.408 INFO:journalctl@ceph.mds.i.smithi137.stdout:Apr 24 06:55:29 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-i[64073]: ignoring --setgroup ceph since I am not root 2024-04-24T06:55:29.409 INFO:journalctl@ceph.mds.i.smithi137.stdout:Apr 24 06:55:29 smithi137 ceph-mds[64077]: -- 172.21.15.137:0/2913221792 <== mon.1 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557c2b7c9a40 con 0x557c2b79cc00 2024-04-24T06:55:29.409 INFO:journalctl@ceph.mds.i.smithi137.stdout:Apr 24 06:55:29 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-i[64073]: starting mds.i at 2024-04-24T06:55:29.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:29.493+0000 7fa06a800640 1 -- 172.21.15.5:0/3696209554 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa06404fd90 con 0x7fa064109350 2024-04-24T06:55:29.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:29 smithi137 ceph-mon[33017]: pgmap v356: 97 pgs: 97 active+clean; 85 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-24T06:55:29.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:29 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/3116391724' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:55:29.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:29 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/3558264947' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-24T06:55:29.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:29 smithi137 ceph-mon[33017]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T06:55:29.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:29 smithi137 ceph-mon[33017]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T06:55:29.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:29 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/3558264947' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-24T06:55:29.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:29 smithi137 ceph-mon[33017]: osdmap e100: 12 total, 12 up, 12 in 2024-04-24T06:55:29.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:29 smithi137 ceph-mon[33017]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-24T06:55:29.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:29 smithi005 ceph-mon[27798]: pgmap v356: 97 pgs: 97 active+clean; 85 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-24T06:55:29.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:29 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/3116391724' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:55:29.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:29 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/3558264947' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-24T06:55:29.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:29 smithi005 ceph-mon[27798]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T06:55:29.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:29 smithi005 ceph-mon[27798]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T06:55:29.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:29 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/3558264947' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-24T06:55:29.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:29 smithi005 ceph-mon[27798]: osdmap e100: 12 total, 12 up, 12 in 2024-04-24T06:55:29.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:29 smithi005 ceph-mon[27798]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-24T06:55:29.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:29 smithi037 ceph-mon[32864]: pgmap v356: 97 pgs: 97 active+clean; 85 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-24T06:55:29.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:29 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/3116391724' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T06:55:29.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:29 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/3558264947' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-24T06:55:29.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:29 smithi037 ceph-mon[32864]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T06:55:29.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:29 smithi037 ceph-mon[32864]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T06:55:29.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:29 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/3558264947' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-24T06:55:29.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:29 smithi037 ceph-mon[32864]: osdmap e100: 12 total, 12 up, 12 in 2024-04-24T06:55:29.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:29 smithi037 ceph-mon[32864]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-24T06:55:30.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.021+0000 7fa062000640 1 -- 172.21.15.5:0/3696209554 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7fa05009c960 con 0x7fa064109350 2024-04-24T06:55:30.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.025+0000 7fa06a800640 1 -- 172.21.15.5:0/3696209554 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fa04c0a3da0 msgr2=0x7fa04c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:30.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.025+0000 7fa06a800640 1 --2- 172.21.15.5:0/3696209554 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fa04c0a3da0 0x7fa04c0a6260 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa058009bd0 tx=0x7fa058005f90 comp rx=0 tx=0).stop 2024-04-24T06:55:30.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.025+0000 7fa06a800640 1 -- 172.21.15.5:0/3696209554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa064109350 msgr2=0x7fa0641a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:30.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.025+0000 7fa06a800640 1 --2- 172.21.15.5:0/3696209554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa064109350 0x7fa0641a0bb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa05000e9f0 tx=0x7fa05000eeb0 comp rx=0 tx=0).stop 2024-04-24T06:55:30.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.026+0000 7fa06a800640 1 -- 172.21.15.5:0/3696209554 shutdown_connections 2024-04-24T06:55:30.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.026+0000 7fa06a800640 1 --2- 172.21.15.5:0/3696209554 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fa04c0a3da0 0x7fa04c0a6260 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:30.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.026+0000 7fa06a800640 1 --2- 172.21.15.5:0/3696209554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa06419a780 0x7fa06419ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:30.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.026+0000 7fa06a800640 1 --2- 172.21.15.5:0/3696209554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa064109350 0x7fa0641a0bb0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:30.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.026+0000 7fa06a800640 1 --2- 172.21.15.5:0/3696209554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa064104280 0x7fa0641a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:30.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.026+0000 7fa06a800640 1 -- 172.21.15.5:0/3696209554 >> 172.21.15.5:0/3696209554 conn(0x7fa0640ffae0 msgr2=0x7fa064101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:55:30.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.026+0000 7fa06a800640 1 -- 172.21.15.5:0/3696209554 shutdown_connections 2024-04-24T06:55:30.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.026+0000 7fa06a800640 1 -- 172.21.15.5:0/3696209554 wait complete. 2024-04-24T06:55:30.047 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-24T06:55:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.262+0000 7fb849e00640 1 -- 172.21.15.5:0/606947363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb844108a10 msgr2=0x7fb844108df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.262+0000 7fb849e00640 1 --2- 172.21.15.5:0/606947363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb844108a10 0x7fb844108df0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb834009890 tx=0x7fb83402a3f0 comp rx=0 tx=0).stop 2024-04-24T06:55:30.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.262+0000 7fb849e00640 1 -- 172.21.15.5:0/606947363 shutdown_connections 2024-04-24T06:55:30.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.262+0000 7fb849e00640 1 --2- 172.21.15.5:0/606947363 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb84410ab70 0x7fb8441155d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:30.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.262+0000 7fb849e00640 1 --2- 172.21.15.5:0/606947363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb84410a1d0 0x7fb84410a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:30.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.262+0000 7fb849e00640 1 --2- 172.21.15.5:0/606947363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb844108a10 0x7fb844108df0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:30.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.262+0000 7fb849e00640 1 -- 172.21.15.5:0/606947363 >> 172.21.15.5:0/606947363 conn(0x7fb844077df0 msgr2=0x7fb844106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:55:30.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.262+0000 7fb849e00640 1 -- 172.21.15.5:0/606947363 shutdown_connections 2024-04-24T06:55:30.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.262+0000 7fb849e00640 1 -- 172.21.15.5:0/606947363 wait complete. 2024-04-24T06:55:30.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.263+0000 7fb849e00640 1 Processor -- start 2024-04-24T06:55:30.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.264+0000 7fb849e00640 1 -- start start 2024-04-24T06:55:30.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.264+0000 7fb849e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb844108a10 0x7fb84419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:30.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.264+0000 7fb849e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb84410a1d0 0x7fb84419f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:30.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.264+0000 7fb849e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb84410ab70 0x7fb8441a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:30.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.264+0000 7fb849e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb844118b00 con 0x7fb84410ab70 2024-04-24T06:55:30.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.264+0000 7fb849e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb844118980 con 0x7fb84410a1d0 2024-04-24T06:55:30.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.264+0000 7fb849e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb844118c80 con 0x7fb844108a10 2024-04-24T06:55:30.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.265+0000 7fb843400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb84410ab70 0x7fb8441a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:30.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.265+0000 7fb842a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb844108a10 0x7fb84419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:30.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.265+0000 7fb83be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb84410a1d0 0x7fb84419f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:30.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.265+0000 7fb842a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb844108a10 0x7fb84419ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.5:33644/0 (socket says 172.21.15.5:33644) 2024-04-24T06:55:30.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.265+0000 7fb843400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb84410ab70 0x7fb8441a37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39656/0 (socket says 172.21.15.5:39656) 2024-04-24T06:55:30.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.265+0000 7fb842a00640 1 -- 172.21.15.5:0/2714054579 learned_addr learned my addr 172.21.15.5:0/2714054579 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:55:30.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.265+0000 7fb842a00640 1 -- 172.21.15.5:0/2714054579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb84410a1d0 msgr2=0x7fb84419f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:30.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.265+0000 7fb842a00640 1 --2- 172.21.15.5:0/2714054579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb84410a1d0 0x7fb84419f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:30.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.265+0000 7fb842a00640 1 -- 172.21.15.5:0/2714054579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb84410ab70 msgr2=0x7fb8441a37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:30.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.265+0000 7fb842a00640 1 --2- 172.21.15.5:0/2714054579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb84410ab70 0x7fb8441a37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:30.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.265+0000 7fb842a00640 1 -- 172.21.15.5:0/2714054579 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8441a3f20 con 0x7fb844108a10 2024-04-24T06:55:30.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.265+0000 7fb842a00640 1 --2- 172.21.15.5:0/2714054579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb844108a10 0x7fb84419ef80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb834000c00 tx=0x7fb834030490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:55:30.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.266+0000 7fb848a00640 1 -- 172.21.15.5:0/2714054579 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83403f030 con 0x7fb844108a10 2024-04-24T06:55:30.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.266+0000 7fb849e00640 1 -- 172.21.15.5:0/2714054579 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8441a41b0 con 0x7fb844108a10 2024-04-24T06:55:30.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.266+0000 7fb848a00640 1 -- 172.21.15.5:0/2714054579 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb834005020 con 0x7fb844108a10 2024-04-24T06:55:30.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.266+0000 7fb848a00640 1 -- 172.21.15.5:0/2714054579 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb834037040 con 0x7fb844108a10 2024-04-24T06:55:30.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.266+0000 7fb849e00640 1 -- 172.21.15.5:0/2714054579 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8441abaa0 con 0x7fb844108a10 2024-04-24T06:55:30.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.267+0000 7fb849e00640 1 -- 172.21.15.5:0/2714054579 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb844069300 con 0x7fb844108a10 2024-04-24T06:55:30.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.269+0000 7fb848a00640 1 -- 172.21.15.5:0/2714054579 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fb834042070 con 0x7fb844108a10 2024-04-24T06:55:30.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.270+0000 7fb848a00640 1 --2- 172.21.15.5:0/2714054579 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb8280a3d20 0x7fb8280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:30.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.271+0000 7fb848a00640 1 -- 172.21.15.5:0/2714054579 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8756+0+0 (secure 0 0 0) 0x7fb8340f0750 con 0x7fb844108a10 2024-04-24T06:55:30.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.271+0000 7fb83be00640 1 --2- 172.21.15.5:0/2714054579 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb8280a3d20 0x7fb8280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:30.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.272+0000 7fb83be00640 1 --2- 172.21.15.5:0/2714054579 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb8280a3d20 0x7fb8280a61e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb844079cd0 tx=0x7fb82c009260 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:55:30.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.273+0000 7fb848a00640 1 -- 172.21.15.5:0/2714054579 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb83406f020 con 0x7fb844108a10 2024-04-24T06:55:30.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:30.473+0000 7fb849e00640 1 -- 172.21.15.5:0/2714054579 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb8441a0280 con 0x7fb844108a10 2024-04-24T06:55:30.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:30 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/3696209554' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T06:55:30.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:30 smithi137 ceph-mon[33017]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T06:55:30.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:30 smithi137 ceph-mon[33017]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T06:55:30.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:30 smithi137 ceph-mon[33017]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T06:55:30.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:30 smithi137 ceph-mon[33017]: Cluster is now healthy 2024-04-24T06:55:30.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:30 smithi137 ceph-mon[33017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-24T06:55:30.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:30 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T06:55:30.696 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:30 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T06:55:30.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:30 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T06:55:30.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:30 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T06:55:30.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:30 smithi137 ceph-mon[33017]: mds.? [v2:172.21.15.37:6836/2563856039,v1:172.21.15.37:6837/2563856039] up:boot 2024-04-24T06:55:30.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:30 smithi137 ceph-mon[33017]: mds.? [v2:172.21.15.37:6838/2215079734,v1:172.21.15.37:6839/2215079734] up:boot 2024-04-24T06:55:30.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:30 smithi137 ceph-mon[33017]: mds.? [v2:172.21.15.137:6835/3457733815,v1:172.21.15.137:6836/3457733815] up:boot 2024-04-24T06:55:30.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:30 smithi137 ceph-mon[33017]: mds.? [v2:172.21.15.137:6834/3083082667,v1:172.21.15.137:6837/3083082667] up:boot 2024-04-24T06:55:30.697 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:30 smithi137 ceph-mon[33017]: mds.? [v2:172.21.15.137:6838/2709404597,v1:172.21.15.137:6839/2709404597] up:boot 2024-04-24T06:55:30.698 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:30 smithi137 ceph-mon[33017]: mds.? [v2:172.21.15.5:6838/791807619,v1:172.21.15.5:6839/791807619] up:boot 2024-04-24T06:55:30.698 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:30 smithi137 ceph-mon[33017]: fsmap 12 up:standby 2024-04-24T06:55:30.698 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:30 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T06:55:30.698 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:30 smithi137 ceph-mon[33017]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T06:55:30.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:30 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/3696209554' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T06:55:30.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:30 smithi005 ceph-mon[27798]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T06:55:30.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:30 smithi005 ceph-mon[27798]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T06:55:30.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:30 smithi005 ceph-mon[27798]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T06:55:30.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:30 smithi005 ceph-mon[27798]: Cluster is now healthy 2024-04-24T06:55:30.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:30 smithi005 ceph-mon[27798]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-24T06:55:30.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:30 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T06:55:30.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:30 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T06:55:30.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:30 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T06:55:30.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:30 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T06:55:30.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:30 smithi005 ceph-mon[27798]: mds.? [v2:172.21.15.37:6836/2563856039,v1:172.21.15.37:6837/2563856039] up:boot 2024-04-24T06:55:30.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:30 smithi005 ceph-mon[27798]: mds.? [v2:172.21.15.37:6838/2215079734,v1:172.21.15.37:6839/2215079734] up:boot 2024-04-24T06:55:30.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:30 smithi005 ceph-mon[27798]: mds.? [v2:172.21.15.137:6835/3457733815,v1:172.21.15.137:6836/3457733815] up:boot 2024-04-24T06:55:30.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:30 smithi005 ceph-mon[27798]: mds.? [v2:172.21.15.137:6834/3083082667,v1:172.21.15.137:6837/3083082667] up:boot 2024-04-24T06:55:30.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:30 smithi005 ceph-mon[27798]: mds.? [v2:172.21.15.137:6838/2709404597,v1:172.21.15.137:6839/2709404597] up:boot 2024-04-24T06:55:30.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:30 smithi005 ceph-mon[27798]: mds.? [v2:172.21.15.5:6838/791807619,v1:172.21.15.5:6839/791807619] up:boot 2024-04-24T06:55:30.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:30 smithi005 ceph-mon[27798]: fsmap 12 up:standby 2024-04-24T06:55:30.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:30 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T06:55:30.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:30 smithi005 ceph-mon[27798]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T06:55:30.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:30 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/3696209554' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T06:55:30.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:30 smithi037 ceph-mon[32864]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T06:55:30.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:30 smithi037 ceph-mon[32864]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T06:55:30.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:30 smithi037 ceph-mon[32864]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T06:55:30.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:30 smithi037 ceph-mon[32864]: Cluster is now healthy 2024-04-24T06:55:30.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:30 smithi037 ceph-mon[32864]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-24T06:55:30.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:30 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T06:55:30.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:30 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T06:55:30.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:30 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T06:55:30.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:30 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T06:55:30.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:30 smithi037 ceph-mon[32864]: mds.? [v2:172.21.15.37:6836/2563856039,v1:172.21.15.37:6837/2563856039] up:boot 2024-04-24T06:55:30.769 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:30 smithi037 ceph-mon[32864]: mds.? [v2:172.21.15.37:6838/2215079734,v1:172.21.15.37:6839/2215079734] up:boot 2024-04-24T06:55:30.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:30 smithi037 ceph-mon[32864]: mds.? [v2:172.21.15.137:6835/3457733815,v1:172.21.15.137:6836/3457733815] up:boot 2024-04-24T06:55:30.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:30 smithi037 ceph-mon[32864]: mds.? [v2:172.21.15.137:6834/3083082667,v1:172.21.15.137:6837/3083082667] up:boot 2024-04-24T06:55:30.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:30 smithi037 ceph-mon[32864]: mds.? [v2:172.21.15.137:6838/2709404597,v1:172.21.15.137:6839/2709404597] up:boot 2024-04-24T06:55:30.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:30 smithi037 ceph-mon[32864]: mds.? [v2:172.21.15.5:6838/791807619,v1:172.21.15.5:6839/791807619] up:boot 2024-04-24T06:55:30.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:30 smithi037 ceph-mon[32864]: fsmap 12 up:standby 2024-04-24T06:55:30.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:30 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T06:55:30.770 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:30 smithi037 ceph-mon[32864]: from='mgr.25202 172.21.15.5:0/793423964' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T06:55:31.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.426+0000 7fb848a00640 1 -- 172.21.15.5:0/2714054579 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v101) ==== 181+0+0 (secure 0 0 0) 0x7fb8441a0280 con 0x7fb844108a10 2024-04-24T06:55:31.430 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' removed 2024-04-24T06:55:31.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.436+0000 7fb849e00640 1 -- 172.21.15.5:0/2714054579 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb8280a3d20 msgr2=0x7fb8280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:31.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.436+0000 7fb849e00640 1 --2- 172.21.15.5:0/2714054579 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb8280a3d20 0x7fb8280a61e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb844079cd0 tx=0x7fb82c009260 comp rx=0 tx=0).stop 2024-04-24T06:55:31.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.436+0000 7fb849e00640 1 -- 172.21.15.5:0/2714054579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb844108a10 msgr2=0x7fb84419ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:31.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.436+0000 7fb849e00640 1 --2- 172.21.15.5:0/2714054579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb844108a10 0x7fb84419ef80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb834000c00 tx=0x7fb834030490 comp rx=0 tx=0).stop 2024-04-24T06:55:31.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.437+0000 7fb849e00640 1 -- 172.21.15.5:0/2714054579 shutdown_connections 2024-04-24T06:55:31.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.437+0000 7fb849e00640 1 --2- 172.21.15.5:0/2714054579 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7fb8280a3d20 0x7fb8280a61e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:31.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.437+0000 7fb849e00640 1 --2- 172.21.15.5:0/2714054579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb84410ab70 0x7fb8441a37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:31.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.437+0000 7fb849e00640 1 --2- 172.21.15.5:0/2714054579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb84410a1d0 0x7fb84419f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:31.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.437+0000 7fb849e00640 1 --2- 172.21.15.5:0/2714054579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb844108a10 0x7fb84419ef80 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:31.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.437+0000 7fb849e00640 1 -- 172.21.15.5:0/2714054579 >> 172.21.15.5:0/2714054579 conn(0x7fb844077df0 msgr2=0x7fb8440783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:55:31.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.437+0000 7fb849e00640 1 -- 172.21.15.5:0/2714054579 shutdown_connections 2024-04-24T06:55:31.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.437+0000 7fb849e00640 1 -- 172.21.15.5:0/2714054579 wait complete. 2024-04-24T06:55:31.462 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-24T06:55:31.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.660+0000 7f157b200640 1 -- 172.21.15.5:0/1275267770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f157c108dc0 msgr2=0x7f157c1152c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:31.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.660+0000 7f157b200640 1 --2- 172.21.15.5:0/1275267770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f157c108dc0 0x7f157c1152c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1574009c10 tx=0x7f157402a6e0 comp rx=0 tx=0).stop 2024-04-24T06:55:31.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.660+0000 7f157b200640 1 -- 172.21.15.5:0/1275267770 shutdown_connections 2024-04-24T06:55:31.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.660+0000 7f157b200640 1 --2- 172.21.15.5:0/1275267770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f157c108dc0 0x7f157c1152c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:31.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.660+0000 7f157b200640 1 --2- 172.21.15.5:0/1275267770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f157c108400 0x7f157c108880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:31.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.660+0000 7f157b200640 1 --2- 172.21.15.5:0/1275267770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f157c10e490 0x7f157c10e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:31.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.660+0000 7f157b200640 1 -- 172.21.15.5:0/1275267770 >> 172.21.15.5:0/1275267770 conn(0x7f157c077b80 msgr2=0x7f157c079fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:55:31.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.660+0000 7f157b200640 1 -- 172.21.15.5:0/1275267770 shutdown_connections 2024-04-24T06:55:31.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.660+0000 7f157b200640 1 -- 172.21.15.5:0/1275267770 wait complete. 2024-04-24T06:55:31.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.661+0000 7f157b200640 1 Processor -- start 2024-04-24T06:55:31.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.661+0000 7f157b200640 1 -- start start 2024-04-24T06:55:31.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.662+0000 7f157b200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f157c108400 0x7f157c1a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:31.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.662+0000 7f157b200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f157c108dc0 0x7f157c1a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:31.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.662+0000 7f157b200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f157c10e490 0x7f157c19ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:31.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.662+0000 7f157b200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f157c1187a0 con 0x7f157c108400 2024-04-24T06:55:31.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.662+0000 7f157b200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f157c118620 con 0x7f157c108dc0 2024-04-24T06:55:31.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.662+0000 7f157b200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f157c118920 con 0x7f157c10e490 2024-04-24T06:55:31.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.662+0000 7f1579e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f157c108400 0x7f157c1a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:31.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.662+0000 7f157a800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f157c10e490 0x7f157c19ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:31.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.662+0000 7f1579e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f157c108400 0x7f157c1a48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39680/0 (socket says 172.21.15.5:39680) 2024-04-24T06:55:31.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.662+0000 7f1579e00640 1 -- 172.21.15.5:0/4180193482 learned_addr learned my addr 172.21.15.5:0/4180193482 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-24T06:55:31.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.662+0000 7f1579400640 1 --2- 172.21.15.5:0/4180193482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f157c108dc0 0x7f157c1a4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:31.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.662+0000 7f1579e00640 1 -- 172.21.15.5:0/4180193482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f157c10e490 msgr2=0x7f157c19ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:31.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.662+0000 7f1579e00640 1 --2- 172.21.15.5:0/4180193482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f157c10e490 0x7f157c19ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:31.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.662+0000 7f1579e00640 1 -- 172.21.15.5:0/4180193482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f157c108dc0 msgr2=0x7f157c1a4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:31.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.662+0000 7f1579e00640 1 --2- 172.21.15.5:0/4180193482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f157c108dc0 0x7f157c1a4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:31.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.662+0000 7f1579e00640 1 -- 172.21.15.5:0/4180193482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f157c19f1e0 con 0x7f157c108400 2024-04-24T06:55:31.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.662+0000 7f157a800640 1 --2- 172.21.15.5:0/4180193482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f157c10e490 0x7f157c19ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T06:55:31.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.663+0000 7f1579e00640 1 --2- 172.21.15.5:0/4180193482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f157c108400 0x7f157c1a48b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f156800e9f0 tx=0x7f156800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:55:31.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.663+0000 7f1572a00640 1 -- 172.21.15.5:0/4180193482 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156800cd80 con 0x7f157c108400 2024-04-24T06:55:31.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.663+0000 7f1572a00640 1 -- 172.21.15.5:0/4180193482 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1568010910 con 0x7f157c108400 2024-04-24T06:55:31.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.663+0000 7f157b200640 1 -- 172.21.15.5:0/4180193482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f157c19f4d0 con 0x7f157c108400 2024-04-24T06:55:31.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.663+0000 7f1572a00640 1 -- 172.21.15.5:0/4180193482 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1568018990 con 0x7f157c108400 2024-04-24T06:55:31.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.663+0000 7f157b200640 1 -- 172.21.15.5:0/4180193482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f157c1ab770 con 0x7f157c108400 2024-04-24T06:55:31.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.664+0000 7f157b200640 1 -- 172.21.15.5:0/4180193482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f157c04fd90 con 0x7f157c108400 2024-04-24T06:55:31.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.667+0000 7f1572a00640 1 -- 172.21.15.5:0/4180193482 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f1568010ac0 con 0x7f157c108400 2024-04-24T06:55:31.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.668+0000 7f1572a00640 1 --2- 172.21.15.5:0/4180193482 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f15600a3d90 0x7f15600a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T06:55:31.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.669+0000 7f1572a00640 1 -- 172.21.15.5:0/4180193482 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8231+0+0 (secure 0 0 0) 0x7f15680d0920 con 0x7f157c108400 2024-04-24T06:55:31.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.669+0000 7f1579400640 1 --2- 172.21.15.5:0/4180193482 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f15600a3d90 0x7f15600a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T06:55:31.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.670+0000 7f1579400640 1 --2- 172.21.15.5:0/4180193482 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f15600a3d90 0x7f15600a6250 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f157c10d9b0 tx=0x7f156c009260 comp rx=0 tx=0).ready entity=mgr.25202 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T06:55:31.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.673+0000 7f1572a00640 1 -- 172.21.15.5:0/4180193482 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f156809ca30 con 0x7f157c108400 2024-04-24T06:55:31.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:31 smithi137 ceph-mon[33017]: pgmap v358: 97 pgs: 97 active+clean; 85 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 921 B/s wr, 7 op/s 2024-04-24T06:55:31.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:31 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/2714054579' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T06:55:31.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:31 smithi137 ceph-mon[33017]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T06:55:31.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:31 smithi005 ceph-mon[27798]: pgmap v358: 97 pgs: 97 active+clean; 85 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 921 B/s wr, 7 op/s 2024-04-24T06:55:31.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:31 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/2714054579' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T06:55:31.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:31 smithi005 ceph-mon[27798]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T06:55:31.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:31 smithi037 ceph-mon[32864]: pgmap v358: 97 pgs: 97 active+clean; 85 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 921 B/s wr, 7 op/s 2024-04-24T06:55:31.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:31 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/2714054579' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T06:55:31.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:31 smithi037 ceph-mon[32864]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T06:55:31.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:31.880+0000 7f157b200640 1 -- 172.21.15.5:0/4180193482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f157c02d620 con 0x7f157c108400 2024-04-24T06:55:32.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:32.433+0000 7f1572a00640 1 -- 172.21.15.5:0/4180193482 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v102) ==== 169+0+0 (secure 0 0 0) 0x7f15680a0d30 con 0x7f157c108400 2024-04-24T06:55:32.434 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' removed 2024-04-24T06:55:32.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:32.438+0000 7f157b200640 1 -- 172.21.15.5:0/4180193482 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f15600a3d90 msgr2=0x7f15600a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:32.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:32.438+0000 7f157b200640 1 --2- 172.21.15.5:0/4180193482 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f15600a3d90 0x7f15600a6250 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f157c10d9b0 tx=0x7f156c009260 comp rx=0 tx=0).stop 2024-04-24T06:55:32.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:32.438+0000 7f157b200640 1 -- 172.21.15.5:0/4180193482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f157c108400 msgr2=0x7f157c1a48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T06:55:32.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:32.438+0000 7f157b200640 1 --2- 172.21.15.5:0/4180193482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f157c108400 0x7f157c1a48b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f156800e9f0 tx=0x7f156800eeb0 comp rx=0 tx=0).stop 2024-04-24T06:55:32.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:32.439+0000 7f157b200640 1 -- 172.21.15.5:0/4180193482 shutdown_connections 2024-04-24T06:55:32.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:32.439+0000 7f157b200640 1 --2- 172.21.15.5:0/4180193482 >> [v2:172.21.15.5:6800/3154624651,v1:172.21.15.5:6801/3154624651] conn(0x7f15600a3d90 0x7f15600a6250 secure :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f157c10d9b0 tx=0x7f156c009260 comp rx=0 tx=0).stop 2024-04-24T06:55:32.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:32.439+0000 7f157b200640 1 --2- 172.21.15.5:0/4180193482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f157c10e490 0x7f157c19ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:32.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:32.439+0000 7f157b200640 1 --2- 172.21.15.5:0/4180193482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f157c108dc0 0x7f157c1a4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T06:55:32.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:32.439+0000 7f157b200640 1 --2- 172.21.15.5:0/4180193482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f157c108400 0x7f157c1a48b0 secure :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f156800e9f0 tx=0x7f156800eeb0 comp rx=0 tx=0).stop 2024-04-24T06:55:32.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:32.439+0000 7f157b200640 1 -- 172.21.15.5:0/4180193482 >> 172.21.15.5:0/4180193482 conn(0x7f157c077b80 msgr2=0x7f157c078300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T06:55:32.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:32.441+0000 7f157b200640 1 -- 172.21.15.5:0/4180193482 shutdown_connections 2024-04-24T06:55:32.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-24T06:55:32.441+0000 7f157b200640 1 -- 172.21.15.5:0/4180193482 wait complete. 2024-04-24T06:55:32.474 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T06:55:32.548 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T06:55:32.582 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T06:55:32.613 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-24T06:55:32.613 DEBUG:teuthology.orchestra.run.smithi005:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-24T06:55:32.666 DEBUG:teuthology.orchestra.run.smithi037:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-24T06:55:32.683 DEBUG:teuthology.orchestra.run.smithi137:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-24T06:55:32.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:32 smithi137 ceph-mon[33017]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-24T06:55:32.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:32 smithi137 ceph-mon[33017]: osdmap e101: 12 total, 12 up, 12 in 2024-04-24T06:55:32.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:32 smithi137 ceph-mon[33017]: from='client.? 172.21.15.5:0/4180193482' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T06:55:32.698 INFO:tasks.cephadm:Stopping all daemons... 2024-04-24T06:55:32.699 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-24T06:55:32.699 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mon.a 2024-04-24T06:55:32.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:32 smithi005 ceph-mon[27798]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-24T06:55:32.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:32 smithi005 ceph-mon[27798]: osdmap e101: 12 total, 12 up, 12 in 2024-04-24T06:55:32.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:32 smithi005 ceph-mon[27798]: from='client.? 172.21.15.5:0/4180193482' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T06:55:32.767 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:32 smithi037 ceph-mon[32864]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-24T06:55:32.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:32 smithi037 ceph-mon[32864]: osdmap e101: 12 total, 12 up, 12 in 2024-04-24T06:55:32.768 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:32 smithi037 ceph-mon[32864]: from='client.? 172.21.15.5:0/4180193482' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T06:55:33.209 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:32 smithi005 systemd[1]: Stopping Ceph mon.a for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:55:33.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:33 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mon-a[27794]: 2024-04-24T06:55:33.207+0000 7f8368c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T06:55:33.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:33 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mon-a[27794]: 2024-04-24T06:55:33.207+0000 7f8368c00700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-24T06:55:33.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:33 smithi005 podman[97098]: 2024-04-24 06:55:33.261262804 +0000 UTC m=+0.195213000 container died a5810a88074d7113bae166a1dc2a8c1cbfafa4aa077ee7a5fe566495cd6d1c44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22) 2024-04-24T06:55:33.848 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:33 smithi005 podman[97098]: 2024-04-24 06:55:33.630537342 +0000 UTC m=+0.564487592 container cleanup a5810a88074d7113bae166a1dc2a8c1cbfafa4aa077ee7a5fe566495cd6d1c44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-24T06:55:33.849 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:33 smithi005 bash[97098]: ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mon-a 2024-04-24T06:55:34.255 INFO:journalctl@ceph.mon.a.smithi005.stdout:Apr 24 06:55:33 smithi005 podman[97112]: 2024-04-24 06:55:33.848471098 +0000 UTC m=+0.575495998 container remove a5810a88074d7113bae166a1dc2a8c1cbfafa4aa077ee7a5fe566495cd6d1c44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True) 2024-04-24T06:55:34.371 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mon.a.service' 2024-04-24T06:55:34.508 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T06:55:34.508 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-24T06:55:34.508 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-24T06:55:34.509 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mon.b 2024-04-24T06:55:34.819 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:34 smithi037 systemd[1]: Stopping Ceph mon.b for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:55:35.153 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:34 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mon-b[32859]: 2024-04-24T06:55:34.818+0000 7f6ce67f4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T06:55:35.154 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:34 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mon-b[32859]: 2024-04-24T06:55:34.818+0000 7f6ce67f4700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-24T06:55:35.154 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:34 smithi037 podman[68828]: 2024-04-24 06:55:34.88040996 +0000 UTC m=+0.178335662 container died 68e4b241a9f40057a2400d4e4e92af04fb8330239ae7776805f34b5edf0a0a6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mon-b, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True) 2024-04-24T06:55:35.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:35 smithi037 podman[68828]: 2024-04-24 06:55:35.361320063 +0000 UTC m=+0.659245765 container cleanup 68e4b241a9f40057a2400d4e4e92af04fb8330239ae7776805f34b5edf0a0a6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-04-24T06:55:35.518 INFO:journalctl@ceph.mon.b.smithi037.stdout:Apr 24 06:55:35 smithi037 bash[68828]: ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mon-b 2024-04-24T06:55:35.745 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mon.b.service' 2024-04-24T06:55:35.794 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T06:55:35.795 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-24T06:55:35.795 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-24T06:55:35.795 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mon.c 2024-04-24T06:55:36.072 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:35 smithi137 systemd[1]: Stopping Ceph mon.c for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:55:36.432 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:36 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mon-c[33000]: 2024-04-24T06:55:36.071+0000 7fa078fbf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T06:55:36.432 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:36 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mon-c[33000]: 2024-04-24T06:55:36.071+0000 7fa078fbf700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-24T06:55:36.432 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:36 smithi137 podman[69735]: 2024-04-24 06:55:36.204428232 +0000 UTC m=+0.212223825 container died 15c232db705d9b32bb64c22cae9e32439bbd1e994337c7498b9c14abbf619bd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mon-c, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-24T06:55:36.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:36 smithi137 podman[69735]: 2024-04-24 06:55:36.613694168 +0000 UTC m=+0.621489776 container cleanup 15c232db705d9b32bb64c22cae9e32439bbd1e994337c7498b9c14abbf619bd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mon-c, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-24T06:55:36.695 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:36 smithi137 bash[69735]: ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mon-c 2024-04-24T06:55:36.951 INFO:journalctl@ceph.mon.c.smithi137.stdout:Apr 24 06:55:36 smithi137 podman[69749]: 2024-04-24 06:55:36.848854722 +0000 UTC m=+0.642802200 container remove 15c232db705d9b32bb64c22cae9e32439bbd1e994337c7498b9c14abbf619bd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mon-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True) 2024-04-24T06:55:36.968 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mon.c.service' 2024-04-24T06:55:37.012 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T06:55:37.012 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-24T06:55:37.012 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-24T06:55:37.013 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mgr.x 2024-04-24T06:55:37.505 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:55:37 smithi005 systemd[1]: Stopping Ceph mgr.x for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:55:37.983 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:55:37 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:55:37.617+0000 7fd6f6400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T06:55:37.983 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:55:37 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x[28026]: 2024-04-24T06:55:37.617+0000 7fd6f6400700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-24T06:55:37.983 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:55:37 smithi005 podman[97249]: 2024-04-24 06:55:37.710076356 +0000 UTC m=+0.238923508 container died 96ec027a3df56bded49d5e581d53c11a5c83dccfdebab4f2a4cc463b5da72e02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416) 2024-04-24T06:55:38.256 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:55:38 smithi005 podman[97249]: 2024-04-24 06:55:38.143337402 +0000 UTC m=+0.672184575 container cleanup 96ec027a3df56bded49d5e581d53c11a5c83dccfdebab4f2a4cc463b5da72e02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-24T06:55:38.256 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:55:38 smithi005 bash[97249]: ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x 2024-04-24T06:55:38.755 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Apr 24 06:55:38 smithi005 podman[97263]: 2024-04-24 06:55:38.361168053 +0000 UTC m=+0.633520785 container remove 96ec027a3df56bded49d5e581d53c11a5c83dccfdebab4f2a4cc463b5da72e02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a3c8a22bd5efe18ab230a8e2d072db8f381a0411, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-24T06:55:38.895 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-24T06:55:39.008 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T06:55:39.008 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-24T06:55:39.008 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-24T06:55:39.008 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mgr.y 2024-04-24T06:55:39.362 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:55:39 smithi037 systemd[1]: Stopping Ceph mgr.y for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:55:39.642 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:55:39 smithi037 podman[68961]: 2024-04-24 06:55:39.361888315 +0000 UTC m=+0.156511766 container died 9a3b3e8361ba8db73b3abfdf55057c37b36ccc670d8421b7344c1b87744b7370 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-24T06:55:40.018 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:55:39 smithi037 podman[68961]: 2024-04-24 06:55:39.700025229 +0000 UTC m=+0.494648685 container cleanup 9a3b3e8361ba8db73b3abfdf55057c37b36ccc670d8421b7344c1b87744b7370 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T06:55:40.018 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:55:39 smithi037 bash[68961]: ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y 2024-04-24T06:55:40.382 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-24T06:55:40.404 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:55:40 smithi037 podman[68975]: 2024-04-24 06:55:40.142171843 +0000 UTC m=+0.775410248 container remove 9a3b3e8361ba8db73b3abfdf55057c37b36ccc670d8421b7344c1b87744b7370 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-y, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True) 2024-04-24T06:55:40.404 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:55:40 smithi037 systemd[1]: ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-04-24T06:55:40.404 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:55:40 smithi037 systemd[1]: ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-04-24T06:55:40.404 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:55:40 smithi037 systemd[1]: Stopped Ceph mgr.y for 43e9bad0-0204-11ef-bc93-c7b262605968. 2024-04-24T06:55:40.404 INFO:journalctl@ceph.mgr.y.smithi037.stdout:Apr 24 06:55:40 smithi037 systemd[1]: ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mgr.y.service: Consumed 21.545s CPU time. 2024-04-24T06:55:40.418 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T06:55:40.418 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-24T06:55:40.418 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-24T06:55:40.418 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mgr.z 2024-04-24T06:55:40.797 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:55:40 smithi137 systemd[1]: Stopping Ceph mgr.z for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:55:41.092 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:55:40 smithi137 podman[69869]: 2024-04-24 06:55:40.797302994 +0000 UTC m=+0.185017456 container died 1b2a03d1f584c2d2fdeb021d51c5faabd6730240e0e1d7a991189a3f2ff8e9e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-24T06:55:41.444 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:55:41 smithi137 podman[69869]: 2024-04-24 06:55:41.093021753 +0000 UTC m=+0.480736223 container cleanup 1b2a03d1f584c2d2fdeb021d51c5faabd6730240e0e1d7a991189a3f2ff8e9e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22) 2024-04-24T06:55:41.444 INFO:journalctl@ceph.mgr.z.smithi137.stdout:Apr 24 06:55:41 smithi137 bash[69869]: ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mgr-z 2024-04-24T06:55:41.572 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-24T06:55:41.605 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T06:55:41.606 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-24T06:55:41.606 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-24T06:55:41.606 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-43e9bad0-0204-11ef-bc93-c7b262605968@osd.0 2024-04-24T06:55:42.006 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 24 06:55:41 smithi005 systemd[1]: Stopping Ceph osd.0 for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:55:42.506 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 24 06:55:42 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-0[39877]: 2024-04-24T06:55:42.244+0000 7f9fa8200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T06:55:42.506 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 24 06:55:42 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-0[39877]: 2024-04-24T06:55:42.244+0000 7f9fa8200700 -1 osd.0 102 *** Got signal Terminated *** 2024-04-24T06:55:42.507 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 24 06:55:42 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-0[39877]: 2024-04-24T06:55:42.244+0000 7f9fa8200700 -1 osd.0 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T06:55:47.967 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 24 06:55:47 smithi005 podman[97370]: 2024-04-24 06:55:47.70447014 +0000 UTC m=+5.624770860 container died 7e96d6f9ef113a4ec1e8ecf462aff6c3aecb660c25ba96cb3e6ef99adbe7d4d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-0, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T06:55:48.582 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 24 06:55:48 smithi005 podman[97370]: 2024-04-24 06:55:48.305775591 +0000 UTC m=+6.226076347 container cleanup 7e96d6f9ef113a4ec1e8ecf462aff6c3aecb660c25ba96cb3e6ef99adbe7d4d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True) 2024-04-24T06:55:48.582 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 24 06:55:48 smithi005 bash[97370]: ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-0 2024-04-24T06:55:49.005 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 24 06:55:48 smithi005 podman[97390]: 2024-04-24 06:55:48.581727195 +0000 UTC m=+0.868850936 container remove 7e96d6f9ef113a4ec1e8ecf462aff6c3aecb660c25ba96cb3e6ef99adbe7d4d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-24T06:55:49.382 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 24 06:55:49 smithi005 podman[97490]: 2024-04-24 06:55:49.034198135 +0000 UTC m=+0.036698083 image pull b5c3183062a6ce19f361681cc58de6e3ac1de1a54c80e6094ec5695237cd98d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b 2024-04-24T06:55:49.382 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 24 06:55:49 smithi005 podman[97490]: 2024-04-24 06:55:49.232637673 +0000 UTC m=+0.235137635 container create 121b9bd308e92c159a755b679a51a435a53b8e9180d050808b269191fb5ceec3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22) 2024-04-24T06:55:49.654 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 24 06:55:49 smithi005 podman[97490]: 2024-04-24 06:55:49.494652852 +0000 UTC m=+0.497152821 container init 121b9bd308e92c159a755b679a51a435a53b8e9180d050808b269191fb5ceec3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-24T06:55:49.658 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 24 06:55:49 smithi005 podman[97490]: 2024-04-24 06:55:49.504894537 +0000 UTC m=+0.507394470 container start 121b9bd308e92c159a755b679a51a435a53b8e9180d050808b269191fb5ceec3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-0-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-24T06:55:49.658 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 24 06:55:49 smithi005 podman[97490]: 2024-04-24 06:55:49.592956852 +0000 UTC m=+0.595456801 container attach 121b9bd308e92c159a755b679a51a435a53b8e9180d050808b269191fb5ceec3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-24T06:55:50.256 INFO:journalctl@ceph.osd.0.smithi005.stdout:Apr 24 06:55:49 smithi005 podman[97490]: 2024-04-24 06:55:49.970041247 +0000 UTC m=+0.972541246 container died 121b9bd308e92c159a755b679a51a435a53b8e9180d050808b269191fb5ceec3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T06:55:50.737 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@osd.0.service' 2024-04-24T06:55:50.861 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T06:55:50.861 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-24T06:55:50.862 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-24T06:55:50.862 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-43e9bad0-0204-11ef-bc93-c7b262605968@osd.1 2024-04-24T06:55:51.174 INFO:journalctl@ceph.osd.1.smithi037.stdout:Apr 24 06:55:50 smithi037 systemd[1]: Stopping Ceph osd.1 for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:55:51.518 INFO:journalctl@ceph.osd.1.smithi037.stdout:Apr 24 06:55:51 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-1[37455]: 2024-04-24T06:55:51.173+0000 7fecbf318700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T06:55:51.518 INFO:journalctl@ceph.osd.1.smithi037.stdout:Apr 24 06:55:51 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-1[37455]: 2024-04-24T06:55:51.173+0000 7fecbf318700 -1 osd.1 102 *** Got signal Terminated *** 2024-04-24T06:55:51.518 INFO:journalctl@ceph.osd.1.smithi037.stdout:Apr 24 06:55:51 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-1[37455]: 2024-04-24T06:55:51.173+0000 7fecbf318700 -1 osd.1 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T06:55:56.865 INFO:journalctl@ceph.osd.1.smithi037.stdout:Apr 24 06:55:56 smithi037 podman[69088]: 2024-04-24 06:55:56.570799403 +0000 UTC m=+5.508883112 container died a365fa714acfc2f4f1f73bf1102382e2c8c76efd2aa71abd4714d83794c134b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-1, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-24T06:55:57.191 INFO:journalctl@ceph.osd.1.smithi037.stdout:Apr 24 06:55:56 smithi037 podman[69088]: 2024-04-24 06:55:56.865200922 +0000 UTC m=+5.803284635 container cleanup a365fa714acfc2f4f1f73bf1102382e2c8c76efd2aa71abd4714d83794c134b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-1, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-24T06:55:57.191 INFO:journalctl@ceph.osd.1.smithi037.stdout:Apr 24 06:55:56 smithi037 bash[69088]: ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-1 2024-04-24T06:55:57.518 INFO:journalctl@ceph.osd.1.smithi037.stdout:Apr 24 06:55:57 smithi037 podman[69105]: 2024-04-24 06:55:57.191363498 +0000 UTC m=+0.614520049 container remove a365fa714acfc2f4f1f73bf1102382e2c8c76efd2aa71abd4714d83794c134b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416) 2024-04-24T06:55:57.998 INFO:journalctl@ceph.osd.1.smithi037.stdout:Apr 24 06:55:57 smithi037 podman[69194]: 2024-04-24 06:55:57.549647095 +0000 UTC m=+0.018122003 image pull b5c3183062a6ce19f361681cc58de6e3ac1de1a54c80e6094ec5695237cd98d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b 2024-04-24T06:55:57.998 INFO:journalctl@ceph.osd.1.smithi037.stdout:Apr 24 06:55:57 smithi037 podman[69194]: 2024-04-24 06:55:57.684052064 +0000 UTC m=+0.152527012 container create 428a40724678acb0459b150e9e91fcc79426f5152343af26005ebceb79a2d629 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T06:55:57.998 INFO:journalctl@ceph.osd.1.smithi037.stdout:Apr 24 06:55:57 smithi037 podman[69194]: 2024-04-24 06:55:57.887233399 +0000 UTC m=+0.355708301 container init 428a40724678acb0459b150e9e91fcc79426f5152343af26005ebceb79a2d629 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T06:55:57.998 INFO:journalctl@ceph.osd.1.smithi037.stdout:Apr 24 06:55:57 smithi037 podman[69194]: 2024-04-24 06:55:57.890442359 +0000 UTC m=+0.358917259 container start 428a40724678acb0459b150e9e91fcc79426f5152343af26005ebceb79a2d629 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-1-deactivate, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T06:55:57.998 INFO:journalctl@ceph.osd.1.smithi037.stdout:Apr 24 06:55:57 smithi037 podman[69194]: 2024-04-24 06:55:57.997717462 +0000 UTC m=+0.466192371 container attach 428a40724678acb0459b150e9e91fcc79426f5152343af26005ebceb79a2d629 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-1-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-24T06:55:58.268 INFO:journalctl@ceph.osd.1.smithi037.stdout:Apr 24 06:55:58 smithi037 podman[69194]: 2024-04-24 06:55:58.195402497 +0000 UTC m=+0.663877390 container died 428a40724678acb0459b150e9e91fcc79426f5152343af26005ebceb79a2d629 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-24T06:55:58.807 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@osd.1.service' 2024-04-24T06:55:58.847 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T06:55:58.847 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-24T06:55:58.847 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-24T06:55:58.848 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-43e9bad0-0204-11ef-bc93-c7b262605968@osd.2 2024-04-24T06:55:59.138 INFO:journalctl@ceph.osd.2.smithi137.stdout:Apr 24 06:55:58 smithi137 systemd[1]: Stopping Ceph osd.2 for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:55:59.445 INFO:journalctl@ceph.osd.2.smithi137.stdout:Apr 24 06:55:59 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-2[38168]: 2024-04-24T06:55:59.136+0000 7f72be23d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T06:55:59.445 INFO:journalctl@ceph.osd.2.smithi137.stdout:Apr 24 06:55:59 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-2[38168]: 2024-04-24T06:55:59.136+0000 7f72be23d700 -1 osd.2 102 *** Got signal Terminated *** 2024-04-24T06:55:59.446 INFO:journalctl@ceph.osd.2.smithi137.stdout:Apr 24 06:55:59 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-2[38168]: 2024-04-24T06:55:59.136+0000 7f72be23d700 -1 osd.2 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T06:56:04.830 INFO:journalctl@ceph.osd.2.smithi137.stdout:Apr 24 06:56:04 smithi137 podman[69995]: 2024-04-24 06:56:04.560183221 +0000 UTC m=+5.514276193 container died 959703a0b51f3c13200804ff0e1e9638c6f9faba48214ff896cba9135c56304e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-2, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-24T06:56:05.195 INFO:journalctl@ceph.osd.2.smithi137.stdout:Apr 24 06:56:04 smithi137 podman[69995]: 2024-04-24 06:56:04.929474513 +0000 UTC m=+5.883567485 container cleanup 959703a0b51f3c13200804ff0e1e9638c6f9faba48214ff896cba9135c56304e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T06:56:05.195 INFO:journalctl@ceph.osd.2.smithi137.stdout:Apr 24 06:56:04 smithi137 bash[69995]: ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-2 2024-04-24T06:56:05.542 INFO:journalctl@ceph.osd.2.smithi137.stdout:Apr 24 06:56:05 smithi137 podman[70013]: 2024-04-24 06:56:05.19721683 +0000 UTC m=+0.637578600 container remove 959703a0b51f3c13200804ff0e1e9638c6f9faba48214ff896cba9135c56304e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-2, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416) 2024-04-24T06:56:05.842 INFO:journalctl@ceph.osd.2.smithi137.stdout:Apr 24 06:56:05 smithi137 podman[70104]: 2024-04-24 06:56:05.445978837 +0000 UTC m=+0.018263046 image pull b5c3183062a6ce19f361681cc58de6e3ac1de1a54c80e6094ec5695237cd98d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b 2024-04-24T06:56:05.842 INFO:journalctl@ceph.osd.2.smithi137.stdout:Apr 24 06:56:05 smithi137 podman[70104]: 2024-04-24 06:56:05.556517444 +0000 UTC m=+0.128801645 container create 5ccae3f0e11ac6da50a88e7378163239cdc3500d95f295584558fffeb2922cb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-24T06:56:05.842 INFO:journalctl@ceph.osd.2.smithi137.stdout:Apr 24 06:56:05 smithi137 podman[70104]: 2024-04-24 06:56:05.767312715 +0000 UTC m=+0.339596916 container init 5ccae3f0e11ac6da50a88e7378163239cdc3500d95f295584558fffeb2922cb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-24T06:56:05.843 INFO:journalctl@ceph.osd.2.smithi137.stdout:Apr 24 06:56:05 smithi137 podman[70104]: 2024-04-24 06:56:05.770440846 +0000 UTC m=+0.342725047 container start 5ccae3f0e11ac6da50a88e7378163239cdc3500d95f295584558fffeb2922cb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.vendor=CentOS) 2024-04-24T06:56:05.843 INFO:journalctl@ceph.osd.2.smithi137.stdout:Apr 24 06:56:05 smithi137 podman[70104]: 2024-04-24 06:56:05.842061947 +0000 UTC m=+0.414346174 container attach 5ccae3f0e11ac6da50a88e7378163239cdc3500d95f295584558fffeb2922cb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-24T06:56:06.092 INFO:journalctl@ceph.osd.2.smithi137.stdout:Apr 24 06:56:06 smithi137 podman[70104]: 2024-04-24 06:56:06.06546514 +0000 UTC m=+0.637749333 container died 5ccae3f0e11ac6da50a88e7378163239cdc3500d95f295584558fffeb2922cb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.vendor=CentOS) 2024-04-24T06:56:06.641 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@osd.2.service' 2024-04-24T06:56:06.678 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T06:56:06.679 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-24T06:56:06.679 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-24T06:56:06.679 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-43e9bad0-0204-11ef-bc93-c7b262605968@osd.3 2024-04-24T06:56:07.256 INFO:journalctl@ceph.osd.3.smithi005.stdout:Apr 24 06:56:06 smithi005 systemd[1]: Stopping Ceph osd.3 for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:56:07.756 INFO:journalctl@ceph.osd.3.smithi005.stdout:Apr 24 06:56:07 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-3[46366]: 2024-04-24T06:56:07.296+0000 7f89e2400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T06:56:07.756 INFO:journalctl@ceph.osd.3.smithi005.stdout:Apr 24 06:56:07 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-3[46366]: 2024-04-24T06:56:07.296+0000 7f89e2400700 -1 osd.3 102 *** Got signal Terminated *** 2024-04-24T06:56:07.756 INFO:journalctl@ceph.osd.3.smithi005.stdout:Apr 24 06:56:07 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-3[46366]: 2024-04-24T06:56:07.296+0000 7f89e2400700 -1 osd.3 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T06:56:08.756 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:08 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:08.300+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:43.854855+0000 front 2024-04-24T06:55:43.854927+0000 (oldest deadline 2024-04-24T06:56:07.955068+0000) 2024-04-24T06:56:09.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:08 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:08.866+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:09.505 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:09 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:09.313+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:43.854855+0000 front 2024-04-24T06:55:43.854927+0000 (oldest deadline 2024-04-24T06:56:07.955068+0000) 2024-04-24T06:56:09.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:09 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:09.182+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:10.005 INFO:journalctl@ceph.osd.3.smithi005.stdout:Apr 24 06:56:09 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-3[46366]: 2024-04-24T06:56:09.656+0000 7f89ddc00700 -1 osd.3 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:43.793729+0000 front 2024-04-24T06:55:43.793237+0000 (oldest deadline 2024-04-24T06:56:09.093418+0000) 2024-04-24T06:56:10.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:09 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:09.901+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:10.445 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:10 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:10.128+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:10.505 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:10 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:10.264+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:43.854855+0000 front 2024-04-24T06:55:43.854927+0000 (oldest deadline 2024-04-24T06:56:07.955068+0000) 2024-04-24T06:56:10.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:10 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:10.139+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:11.005 INFO:journalctl@ceph.osd.3.smithi005.stdout:Apr 24 06:56:10 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-3[46366]: 2024-04-24T06:56:10.678+0000 7f89ddc00700 -1 osd.3 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:43.793729+0000 front 2024-04-24T06:55:43.793237+0000 (oldest deadline 2024-04-24T06:56:09.093418+0000) 2024-04-24T06:56:11.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:10 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:10.872+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:11.445 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:11 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:11.134+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:11.505 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:11 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:11.268+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:43.854855+0000 front 2024-04-24T06:55:43.854927+0000 (oldest deadline 2024-04-24T06:56:07.955068+0000) 2024-04-24T06:56:11.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:11 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:11.122+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:12.005 INFO:journalctl@ceph.osd.3.smithi005.stdout:Apr 24 06:56:11 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-3[46366]: 2024-04-24T06:56:11.634+0000 7f89ddc00700 -1 osd.3 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:43.793729+0000 front 2024-04-24T06:55:43.793237+0000 (oldest deadline 2024-04-24T06:56:09.093418+0000) 2024-04-24T06:56:12.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:11 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:11.876+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:12.445 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:12 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:12.115+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:12.445 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:12 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5[45127]: 2024-04-24T06:56:12.231+0000 7f0a316e1700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:46.549209+0000 front 2024-04-24T06:55:46.549265+0000 (oldest deadline 2024-04-24T06:56:11.849276+0000) 2024-04-24T06:56:12.505 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:12 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:12.271+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:43.854855+0000 front 2024-04-24T06:55:43.854927+0000 (oldest deadline 2024-04-24T06:56:07.955068+0000) 2024-04-24T06:56:12.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:12 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:12.081+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:12.961 INFO:journalctl@ceph.osd.3.smithi005.stdout:Apr 24 06:56:12 smithi005 podman[97595]: 2024-04-24 06:56:12.677473962 +0000 UTC m=+5.556054378 container died fdcb99b527f21fbd5dc3066ef7120489333e689147c8fce2804fc64f45fba2b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, maintainer=Guillaume Abrioux ) 2024-04-24T06:56:13.226 INFO:journalctl@ceph.osd.3.smithi005.stdout:Apr 24 06:56:13 smithi005 podman[97595]: 2024-04-24 06:56:13.072576543 +0000 UTC m=+5.951156997 container cleanup fdcb99b527f21fbd5dc3066ef7120489333e689147c8fce2804fc64f45fba2b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.build-date=20240416) 2024-04-24T06:56:13.227 INFO:journalctl@ceph.osd.3.smithi005.stdout:Apr 24 06:56:13 smithi005 bash[97595]: ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-3 2024-04-24T06:56:13.227 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:13 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:13.109+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:13.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:12 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:12.906+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:13.445 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:13 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5[45127]: 2024-04-24T06:56:13.211+0000 7f0a316e1700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:46.549209+0000 front 2024-04-24T06:55:46.549265+0000 (oldest deadline 2024-04-24T06:56:11.849276+0000) 2024-04-24T06:56:13.445 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:13 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:13.147+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:13.505 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:13 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:13.225+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:43.854855+0000 front 2024-04-24T06:55:43.854927+0000 (oldest deadline 2024-04-24T06:56:07.955068+0000) 2024-04-24T06:56:13.506 INFO:journalctl@ceph.osd.3.smithi005.stdout:Apr 24 06:56:13 smithi005 podman[97613]: 2024-04-24 06:56:13.256903096 +0000 UTC m=+0.565821541 container remove fdcb99b527f21fbd5dc3066ef7120489333e689147c8fce2804fc64f45fba2b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-24T06:56:14.267 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:13 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:13.861+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:14.349 INFO:journalctl@ceph.osd.3.smithi005.stdout:Apr 24 06:56:14 smithi005 podman[97698]: 2024-04-24 06:56:14.063593883 +0000 UTC m=+0.040244311 image pull b5c3183062a6ce19f361681cc58de6e3ac1de1a54c80e6094ec5695237cd98d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b 2024-04-24T06:56:14.350 INFO:journalctl@ceph.osd.3.smithi005.stdout:Apr 24 06:56:14 smithi005 podman[97698]: 2024-04-24 06:56:14.190977513 +0000 UTC m=+0.167627970 container create 554486a4d08bfc8c6d740aa8eddd40b22e8f5992cb03bdf44637e3f3cc505cea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T06:56:14.350 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:14 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:14.263+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:43.854855+0000 front 2024-04-24T06:55:43.854927+0000 (oldest deadline 2024-04-24T06:56:07.955068+0000) 2024-04-24T06:56:14.350 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:14 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:14.068+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:14.445 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:14 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:14.127+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:14.445 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:14 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5[45127]: 2024-04-24T06:56:14.194+0000 7f0a316e1700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:46.549209+0000 front 2024-04-24T06:55:46.549265+0000 (oldest deadline 2024-04-24T06:56:11.849276+0000) 2024-04-24T06:56:14.965 INFO:journalctl@ceph.osd.3.smithi005.stdout:Apr 24 06:56:14 smithi005 podman[97698]: 2024-04-24 06:56:14.602502168 +0000 UTC m=+0.579152602 container init 554486a4d08bfc8c6d740aa8eddd40b22e8f5992cb03bdf44637e3f3cc505cea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-24T06:56:14.965 INFO:journalctl@ceph.osd.3.smithi005.stdout:Apr 24 06:56:14 smithi005 podman[97698]: 2024-04-24 06:56:14.614377975 +0000 UTC m=+0.591028401 container start 554486a4d08bfc8c6d740aa8eddd40b22e8f5992cb03bdf44637e3f3cc505cea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416) 2024-04-24T06:56:14.965 INFO:journalctl@ceph.osd.3.smithi005.stdout:Apr 24 06:56:14 smithi005 podman[97698]: 2024-04-24 06:56:14.710023218 +0000 UTC m=+0.686673643 container attach 554486a4d08bfc8c6d740aa8eddd40b22e8f5992cb03bdf44637e3f3cc505cea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=) 2024-04-24T06:56:15.255 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:15 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:15.109+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:15.256 INFO:journalctl@ceph.osd.3.smithi005.stdout:Apr 24 06:56:15 smithi005 podman[97698]: 2024-04-24 06:56:15.105481814 +0000 UTC m=+1.082132233 container died 554486a4d08bfc8c6d740aa8eddd40b22e8f5992cb03bdf44637e3f3cc505cea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-24T06:56:15.267 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:14 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:14.875+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:15.445 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:15 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5[45127]: 2024-04-24T06:56:15.147+0000 7f0a316e1700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:46.549209+0000 front 2024-04-24T06:55:46.549265+0000 (oldest deadline 2024-04-24T06:56:11.849276+0000) 2024-04-24T06:56:15.445 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:15 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:15.101+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:15.755 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:15 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:15.288+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:43.854855+0000 front 2024-04-24T06:55:43.854927+0000 (oldest deadline 2024-04-24T06:56:07.955068+0000) 2024-04-24T06:56:15.820 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@osd.3.service' 2024-04-24T06:56:15.924 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T06:56:15.924 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-24T06:56:15.924 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-24T06:56:15.924 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-43e9bad0-0204-11ef-bc93-c7b262605968@osd.4 2024-04-24T06:56:16.252 INFO:journalctl@ceph.osd.4.smithi037.stdout:Apr 24 06:56:15 smithi037 systemd[1]: Stopping Ceph osd.4 for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:56:16.252 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:15 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:15.876+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:16.445 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:16 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:16.142+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:16.446 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:16 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5[45127]: 2024-04-24T06:56:16.154+0000 7f0a316e1700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:46.549209+0000 front 2024-04-24T06:55:46.549265+0000 (oldest deadline 2024-04-24T06:56:11.849276+0000) 2024-04-24T06:56:16.506 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:16 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:16.249+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:43.854855+0000 front 2024-04-24T06:55:43.854927+0000 (oldest deadline 2024-04-24T06:56:07.955068+0000) 2024-04-24T06:56:16.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:16 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:16.106+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:16.517 INFO:journalctl@ceph.osd.4.smithi037.stdout:Apr 24 06:56:16 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-4[44150]: 2024-04-24T06:56:16.251+0000 7f7c763c8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T06:56:16.518 INFO:journalctl@ceph.osd.4.smithi037.stdout:Apr 24 06:56:16 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-4[44150]: 2024-04-24T06:56:16.251+0000 7f7c763c8700 -1 osd.4 102 *** Got signal Terminated *** 2024-04-24T06:56:16.518 INFO:journalctl@ceph.osd.4.smithi037.stdout:Apr 24 06:56:16 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-4[44150]: 2024-04-24T06:56:16.251+0000 7f7c763c8700 -1 osd.4 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T06:56:17.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:16 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:16.882+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:17.408 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:17 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:17.296+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:43.854855+0000 front 2024-04-24T06:55:43.854927+0000 (oldest deadline 2024-04-24T06:56:07.955068+0000) 2024-04-24T06:56:17.409 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:17 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:17.116+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:17.445 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:17 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:17.136+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:17.445 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:17 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5[45127]: 2024-04-24T06:56:17.199+0000 7f0a316e1700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:46.549209+0000 front 2024-04-24T06:55:46.549265+0000 (oldest deadline 2024-04-24T06:56:11.849276+0000) 2024-04-24T06:56:17.445 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:17 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5[45127]: 2024-04-24T06:56:17.199+0000 7f0a316e1700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:55.950328+0000 front 2024-04-24T06:55:55.950407+0000 (oldest deadline 2024-04-24T06:56:17.050314+0000) 2024-04-24T06:56:17.896 INFO:journalctl@ceph.osd.4.smithi037.stdout:Apr 24 06:56:17 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-4[44150]: 2024-04-24T06:56:17.602+0000 7f7c729bc700 -1 osd.4 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:52.556950+0000 front 2024-04-24T06:55:52.556546+0000 (oldest deadline 2024-04-24T06:56:16.656924+0000) 2024-04-24T06:56:18.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:17 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:17.896+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:18.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:17 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:17.896+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:18.426 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:18 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:18.267+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:43.854855+0000 front 2024-04-24T06:55:43.854927+0000 (oldest deadline 2024-04-24T06:56:07.955068+0000) 2024-04-24T06:56:18.427 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:18 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:18.101+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:18.445 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:18 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:18.137+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:18.445 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:18 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5[45127]: 2024-04-24T06:56:18.189+0000 7f0a316e1700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:46.549209+0000 front 2024-04-24T06:55:46.549265+0000 (oldest deadline 2024-04-24T06:56:11.849276+0000) 2024-04-24T06:56:18.445 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:18 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5[45127]: 2024-04-24T06:56:18.189+0000 7f0a316e1700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:55.950328+0000 front 2024-04-24T06:55:55.950407+0000 (oldest deadline 2024-04-24T06:56:17.050314+0000) 2024-04-24T06:56:18.768 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:18 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:18.328+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.624953+0000 front 2024-04-24T06:55:53.625107+0000 (oldest deadline 2024-04-24T06:56:17.725121+0000) 2024-04-24T06:56:18.769 INFO:journalctl@ceph.osd.4.smithi037.stdout:Apr 24 06:56:18 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-4[44150]: 2024-04-24T06:56:18.562+0000 7f7c729bc700 -1 osd.4 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:52.556950+0000 front 2024-04-24T06:55:52.556546+0000 (oldest deadline 2024-04-24T06:56:16.656924+0000) 2024-04-24T06:56:19.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:18 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:18.860+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:19.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:18 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:18.860+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:19.445 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:19 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5[45127]: 2024-04-24T06:56:19.186+0000 7f0a316e1700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:46.549209+0000 front 2024-04-24T06:55:46.549265+0000 (oldest deadline 2024-04-24T06:56:11.849276+0000) 2024-04-24T06:56:19.445 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:19 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5[45127]: 2024-04-24T06:56:19.186+0000 7f0a316e1700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:55.950328+0000 front 2024-04-24T06:55:55.950407+0000 (oldest deadline 2024-04-24T06:56:17.050314+0000) 2024-04-24T06:56:19.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:19 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:19.169+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:56.064734+0000 front 2024-04-24T06:55:56.064528+0000 (oldest deadline 2024-04-24T06:56:18.364453+0000) 2024-04-24T06:56:19.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:19 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:19.164+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:19.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:19 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:19.164+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:56:19.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:19 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:19.140+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:19.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:19 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:19.140+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:56:19.507 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:19 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:19.292+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:43.854855+0000 front 2024-04-24T06:55:43.854927+0000 (oldest deadline 2024-04-24T06:56:07.955068+0000) 2024-04-24T06:56:19.507 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:19 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:19.292+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.958304+0000 front 2024-04-24T06:55:54.958228+0000 (oldest deadline 2024-04-24T06:56:18.458488+0000) 2024-04-24T06:56:19.768 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:19 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:19.351+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.624953+0000 front 2024-04-24T06:55:53.625107+0000 (oldest deadline 2024-04-24T06:56:17.725121+0000) 2024-04-24T06:56:19.768 INFO:journalctl@ceph.osd.4.smithi037.stdout:Apr 24 06:56:19 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-4[44150]: 2024-04-24T06:56:19.588+0000 7f7c729bc700 -1 osd.4 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:52.556950+0000 front 2024-04-24T06:55:52.556546+0000 (oldest deadline 2024-04-24T06:56:16.656924+0000) 2024-04-24T06:56:20.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:19 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:19.868+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:20.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:19 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:19.868+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:20.445 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:20 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:20.158+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:56.064734+0000 front 2024-04-24T06:55:56.064528+0000 (oldest deadline 2024-04-24T06:56:18.364453+0000) 2024-04-24T06:56:20.446 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:20 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5[45127]: 2024-04-24T06:56:20.170+0000 7f0a316e1700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:46.549209+0000 front 2024-04-24T06:55:46.549265+0000 (oldest deadline 2024-04-24T06:56:11.849276+0000) 2024-04-24T06:56:20.446 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:20 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5[45127]: 2024-04-24T06:56:20.170+0000 7f0a316e1700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:55.950328+0000 front 2024-04-24T06:55:55.950407+0000 (oldest deadline 2024-04-24T06:56:17.050314+0000) 2024-04-24T06:56:20.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:20 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:20.162+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:20.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:20 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:20.162+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:56:20.506 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:20 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:20.291+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:43.854855+0000 front 2024-04-24T06:55:43.854927+0000 (oldest deadline 2024-04-24T06:56:07.955068+0000) 2024-04-24T06:56:20.506 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:20 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:20.291+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.958304+0000 front 2024-04-24T06:55:54.958228+0000 (oldest deadline 2024-04-24T06:56:18.458488+0000) 2024-04-24T06:56:20.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:20 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:20.150+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:20.507 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:20 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:20.150+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:56:20.768 INFO:journalctl@ceph.osd.4.smithi037.stdout:Apr 24 06:56:20 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-4[44150]: 2024-04-24T06:56:20.559+0000 7f7c729bc700 -1 osd.4 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:52.556950+0000 front 2024-04-24T06:55:52.556546+0000 (oldest deadline 2024-04-24T06:56:16.656924+0000) 2024-04-24T06:56:20.768 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:20 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:20.319+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.624953+0000 front 2024-04-24T06:55:53.625107+0000 (oldest deadline 2024-04-24T06:56:17.725121+0000) 2024-04-24T06:56:21.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:20 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:20.853+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:21.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:20 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:20.853+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:21.445 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:21 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:21.172+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:56.064734+0000 front 2024-04-24T06:55:56.064528+0000 (oldest deadline 2024-04-24T06:56:18.364453+0000) 2024-04-24T06:56:21.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:21 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:21.193+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:21.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:21 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:21.193+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:56:21.446 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:21 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5[45127]: 2024-04-24T06:56:21.179+0000 7f0a316e1700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:46.549209+0000 front 2024-04-24T06:55:46.549265+0000 (oldest deadline 2024-04-24T06:56:11.849276+0000) 2024-04-24T06:56:21.446 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:21 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5[45127]: 2024-04-24T06:56:21.179+0000 7f0a316e1700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:55.950328+0000 front 2024-04-24T06:55:55.950407+0000 (oldest deadline 2024-04-24T06:56:17.050314+0000) 2024-04-24T06:56:21.506 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:21 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:21.247+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:43.854855+0000 front 2024-04-24T06:55:43.854927+0000 (oldest deadline 2024-04-24T06:56:07.955068+0000) 2024-04-24T06:56:21.506 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:21 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:21.247+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.958304+0000 front 2024-04-24T06:55:54.958228+0000 (oldest deadline 2024-04-24T06:56:18.458488+0000) 2024-04-24T06:56:21.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:21 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:21.113+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:21.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:21 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:21.113+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:56:21.623 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:21 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:21.298+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.624953+0000 front 2024-04-24T06:55:53.625107+0000 (oldest deadline 2024-04-24T06:56:17.725121+0000) 2024-04-24T06:56:21.887 INFO:journalctl@ceph.osd.4.smithi037.stdout:Apr 24 06:56:21 smithi037 podman[69311]: 2024-04-24 06:56:21.623406394 +0000 UTC m=+5.500711575 container died e3c22d9f3672814e1bf720dd88b8da381110c68e387a312cc2006f7f3e5df5dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-24T06:56:21.888 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:21 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:21.886+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:21.888 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:21 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:21.886+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:22.252 INFO:journalctl@ceph.osd.4.smithi037.stdout:Apr 24 06:56:22 smithi037 podman[69311]: 2024-04-24 06:56:22.018292307 +0000 UTC m=+5.895597486 container cleanup e3c22d9f3672814e1bf720dd88b8da381110c68e387a312cc2006f7f3e5df5dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-4, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-24T06:56:22.253 INFO:journalctl@ceph.osd.4.smithi037.stdout:Apr 24 06:56:22 smithi037 bash[69311]: ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-4 2024-04-24T06:56:22.445 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:22 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:22.168+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:56.064734+0000 front 2024-04-24T06:55:56.064528+0000 (oldest deadline 2024-04-24T06:56:18.364453+0000) 2024-04-24T06:56:22.446 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:22 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5[45127]: 2024-04-24T06:56:22.211+0000 7f0a316e1700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:46.549209+0000 front 2024-04-24T06:55:46.549265+0000 (oldest deadline 2024-04-24T06:56:11.849276+0000) 2024-04-24T06:56:22.446 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:22 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5[45127]: 2024-04-24T06:56:22.211+0000 7f0a316e1700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:55.950328+0000 front 2024-04-24T06:55:55.950407+0000 (oldest deadline 2024-04-24T06:56:17.050314+0000) 2024-04-24T06:56:22.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:22 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:22.201+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:22.447 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:22 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:22.201+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:56:22.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:22 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:22.065+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:22.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:22 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:22.065+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:56:22.506 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:22 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:22.222+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:43.854855+0000 front 2024-04-24T06:55:43.854927+0000 (oldest deadline 2024-04-24T06:56:07.955068+0000) 2024-04-24T06:56:22.506 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:22 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:22.222+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.958304+0000 front 2024-04-24T06:55:54.958228+0000 (oldest deadline 2024-04-24T06:56:18.458488+0000) 2024-04-24T06:56:22.518 INFO:journalctl@ceph.osd.4.smithi037.stdout:Apr 24 06:56:22 smithi037 podman[69328]: 2024-04-24 06:56:22.252722208 +0000 UTC m=+0.630942271 container remove e3c22d9f3672814e1bf720dd88b8da381110c68e387a312cc2006f7f3e5df5dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-4, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-24T06:56:22.518 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:22 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:22.315+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.624953+0000 front 2024-04-24T06:55:53.625107+0000 (oldest deadline 2024-04-24T06:56:17.725121+0000) 2024-04-24T06:56:22.897 INFO:journalctl@ceph.osd.4.smithi037.stdout:Apr 24 06:56:22 smithi037 podman[69420]: 2024-04-24 06:56:22.496932985 +0000 UTC m=+0.018838557 image pull b5c3183062a6ce19f361681cc58de6e3ac1de1a54c80e6094ec5695237cd98d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b 2024-04-24T06:56:22.897 INFO:journalctl@ceph.osd.4.smithi037.stdout:Apr 24 06:56:22 smithi037 podman[69420]: 2024-04-24 06:56:22.611878496 +0000 UTC m=+0.133784049 container create 68de360bfffcfaf4511ae481de9b1d99468f5c93faaf900d448e1388b8d72eb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T06:56:22.897 INFO:journalctl@ceph.osd.4.smithi037.stdout:Apr 24 06:56:22 smithi037 podman[69420]: 2024-04-24 06:56:22.822952331 +0000 UTC m=+0.344857895 container init 68de360bfffcfaf4511ae481de9b1d99468f5c93faaf900d448e1388b8d72eb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-4-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.vendor=CentOS) 2024-04-24T06:56:22.898 INFO:journalctl@ceph.osd.4.smithi037.stdout:Apr 24 06:56:22 smithi037 podman[69420]: 2024-04-24 06:56:22.826545012 +0000 UTC m=+0.348450563 container start 68de360bfffcfaf4511ae481de9b1d99468f5c93faaf900d448e1388b8d72eb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T06:56:22.898 INFO:journalctl@ceph.osd.4.smithi037.stdout:Apr 24 06:56:22 smithi037 podman[69420]: 2024-04-24 06:56:22.897309507 +0000 UTC m=+0.419215069 container attach 68de360bfffcfaf4511ae481de9b1d99468f5c93faaf900d448e1388b8d72eb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-24T06:56:23.268 INFO:journalctl@ceph.osd.4.smithi037.stdout:Apr 24 06:56:23 smithi037 podman[69420]: 2024-04-24 06:56:23.13819425 +0000 UTC m=+0.660099797 container died 68de360bfffcfaf4511ae481de9b1d99468f5c93faaf900d448e1388b8d72eb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T06:56:23.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:22 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:22.933+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:23.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:22 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:22.933+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:23.445 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:23 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:23.136+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:56.064734+0000 front 2024-04-24T06:55:56.064528+0000 (oldest deadline 2024-04-24T06:56:18.364453+0000) 2024-04-24T06:56:23.446 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:23 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5[45127]: 2024-04-24T06:56:23.234+0000 7f0a316e1700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:46.549209+0000 front 2024-04-24T06:55:46.549265+0000 (oldest deadline 2024-04-24T06:56:11.849276+0000) 2024-04-24T06:56:23.446 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:23 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5[45127]: 2024-04-24T06:56:23.234+0000 7f0a316e1700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:55.950328+0000 front 2024-04-24T06:55:55.950407+0000 (oldest deadline 2024-04-24T06:56:17.050314+0000) 2024-04-24T06:56:23.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:23 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:23.203+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:23.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:23 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:23.203+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:56:23.505 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:23 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:23.087+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:23.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:23 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:23.087+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:56:23.506 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:23 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:23.270+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:43.854855+0000 front 2024-04-24T06:55:43.854927+0000 (oldest deadline 2024-04-24T06:56:07.955068+0000) 2024-04-24T06:56:23.506 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:23 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:23.270+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.958304+0000 front 2024-04-24T06:55:54.958228+0000 (oldest deadline 2024-04-24T06:56:18.458488+0000) 2024-04-24T06:56:23.684 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:23 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:23.310+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.624953+0000 front 2024-04-24T06:55:53.625107+0000 (oldest deadline 2024-04-24T06:56:17.725121+0000) 2024-04-24T06:56:23.704 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@osd.4.service' 2024-04-24T06:56:23.745 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T06:56:23.746 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-24T06:56:23.746 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-24T06:56:23.746 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-43e9bad0-0204-11ef-bc93-c7b262605968@osd.5 2024-04-24T06:56:24.018 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:23 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:23.913+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:24.018 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:23 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:23.913+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:24.066 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:23 smithi137 systemd[1]: Stopping Ceph osd.5 for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:56:24.066 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:24 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5[45127]: 2024-04-24T06:56:24.064+0000 7f0a390f5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T06:56:24.066 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:24 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5[45127]: 2024-04-24T06:56:24.064+0000 7f0a390f5700 -1 osd.5 102 *** Got signal Terminated *** 2024-04-24T06:56:24.066 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:24 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5[45127]: 2024-04-24T06:56:24.064+0000 7f0a390f5700 -1 osd.5 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T06:56:24.445 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:24 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:24.128+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:56.064734+0000 front 2024-04-24T06:55:56.064528+0000 (oldest deadline 2024-04-24T06:56:18.364453+0000) 2024-04-24T06:56:24.446 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:24 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5[45127]: 2024-04-24T06:56:24.256+0000 7f0a316e1700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:46.549209+0000 front 2024-04-24T06:55:46.549265+0000 (oldest deadline 2024-04-24T06:56:11.849276+0000) 2024-04-24T06:56:24.446 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:24 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5[45127]: 2024-04-24T06:56:24.256+0000 7f0a316e1700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:55.950328+0000 front 2024-04-24T06:55:55.950407+0000 (oldest deadline 2024-04-24T06:56:17.050314+0000) 2024-04-24T06:56:24.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:24 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:24.215+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:24.447 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:24 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:24.215+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:56:24.505 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:24 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:24.268+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:43.854855+0000 front 2024-04-24T06:55:43.854927+0000 (oldest deadline 2024-04-24T06:56:07.955068+0000) 2024-04-24T06:56:24.506 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:24 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:24.268+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.958304+0000 front 2024-04-24T06:55:54.958228+0000 (oldest deadline 2024-04-24T06:56:18.458488+0000) 2024-04-24T06:56:24.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:24 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:24.137+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:24.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:24 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:24.137+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:56:24.768 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:24 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:24.269+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.624953+0000 front 2024-04-24T06:55:53.625107+0000 (oldest deadline 2024-04-24T06:56:17.725121+0000) 2024-04-24T06:56:25.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:24 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:24.936+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:25.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:24 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:24.936+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:25.445 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:25 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:25.167+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:56.064734+0000 front 2024-04-24T06:55:56.064528+0000 (oldest deadline 2024-04-24T06:56:18.364453+0000) 2024-04-24T06:56:25.445 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:25 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:25.261+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:25.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:25 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:25.261+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:56:25.446 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:25 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5[45127]: 2024-04-24T06:56:25.248+0000 7f0a316e1700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:46.549209+0000 front 2024-04-24T06:55:46.549265+0000 (oldest deadline 2024-04-24T06:56:11.849276+0000) 2024-04-24T06:56:25.446 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:25 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5[45127]: 2024-04-24T06:56:25.248+0000 7f0a316e1700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:55.950328+0000 front 2024-04-24T06:55:55.950407+0000 (oldest deadline 2024-04-24T06:56:17.050314+0000) 2024-04-24T06:56:25.505 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:25 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:25.151+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:25.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:25 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:25.151+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:56:25.506 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:25 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:25.234+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:43.854855+0000 front 2024-04-24T06:55:43.854927+0000 (oldest deadline 2024-04-24T06:56:07.955068+0000) 2024-04-24T06:56:25.506 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:25 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:25.234+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.958304+0000 front 2024-04-24T06:55:54.958228+0000 (oldest deadline 2024-04-24T06:56:18.458488+0000) 2024-04-24T06:56:25.767 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:25 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:25.280+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.624953+0000 front 2024-04-24T06:55:53.625107+0000 (oldest deadline 2024-04-24T06:56:17.725121+0000) 2024-04-24T06:56:26.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:25 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:25.944+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:26.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:25 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:25.944+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:26.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:25 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:25.944+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:56:26.445 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:26 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5[45127]: 2024-04-24T06:56:26.297+0000 7f0a316e1700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:46.549209+0000 front 2024-04-24T06:55:46.549265+0000 (oldest deadline 2024-04-24T06:56:11.849276+0000) 2024-04-24T06:56:26.445 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:26 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5[45127]: 2024-04-24T06:56:26.297+0000 7f0a316e1700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:55.950328+0000 front 2024-04-24T06:55:55.950407+0000 (oldest deadline 2024-04-24T06:56:17.050314+0000) 2024-04-24T06:56:26.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:26 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:26.194+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:56.064734+0000 front 2024-04-24T06:55:56.064528+0000 (oldest deadline 2024-04-24T06:56:18.364453+0000) 2024-04-24T06:56:26.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:26 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:26.194+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.265368+0000 front 2024-04-24T06:56:01.265402+0000 (oldest deadline 2024-04-24T06:56:25.965496+0000) 2024-04-24T06:56:26.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:26 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:26.280+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:26.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:26 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:26.280+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:56:26.505 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:26 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:26.212+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:43.854855+0000 front 2024-04-24T06:55:43.854927+0000 (oldest deadline 2024-04-24T06:56:07.955068+0000) 2024-04-24T06:56:26.506 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:26 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:26.212+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.958304+0000 front 2024-04-24T06:55:54.958228+0000 (oldest deadline 2024-04-24T06:56:18.458488+0000) 2024-04-24T06:56:26.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:26 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:26.200+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:26.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:26 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:26.200+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:56:26.768 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:26 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:26.288+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.624953+0000 front 2024-04-24T06:55:53.625107+0000 (oldest deadline 2024-04-24T06:56:17.725121+0000) 2024-04-24T06:56:26.768 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:26 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:26.288+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.826231+0000 front 2024-04-24T06:56:01.826096+0000 (oldest deadline 2024-04-24T06:56:25.326172+0000) 2024-04-24T06:56:27.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:26 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:26.949+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:27.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:26 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:26.949+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:27.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:26 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:26.949+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:56:27.506 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:27 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:27.205+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:43.854855+0000 front 2024-04-24T06:55:43.854927+0000 (oldest deadline 2024-04-24T06:56:07.955068+0000) 2024-04-24T06:56:27.506 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:27 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:27.205+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.958304+0000 front 2024-04-24T06:55:54.958228+0000 (oldest deadline 2024-04-24T06:56:18.458488+0000) 2024-04-24T06:56:27.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:27 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:27.248+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:27.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:27 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:27.248+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:56:27.695 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:27 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5[45127]: 2024-04-24T06:56:27.340+0000 7f0a316e1700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:46.549209+0000 front 2024-04-24T06:55:46.549265+0000 (oldest deadline 2024-04-24T06:56:11.849276+0000) 2024-04-24T06:56:27.695 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:27 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5[45127]: 2024-04-24T06:56:27.340+0000 7f0a316e1700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:55.950328+0000 front 2024-04-24T06:55:55.950407+0000 (oldest deadline 2024-04-24T06:56:17.050314+0000) 2024-04-24T06:56:27.696 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:27 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:27.223+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:56.064734+0000 front 2024-04-24T06:55:56.064528+0000 (oldest deadline 2024-04-24T06:56:18.364453+0000) 2024-04-24T06:56:27.696 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:27 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:27.223+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.265368+0000 front 2024-04-24T06:56:01.265402+0000 (oldest deadline 2024-04-24T06:56:25.965496+0000) 2024-04-24T06:56:27.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:27 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:27.276+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:27.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:27 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:27.276+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:56:27.697 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:27 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:27.276+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:56:27.768 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:27 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:27.333+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.624953+0000 front 2024-04-24T06:55:53.625107+0000 (oldest deadline 2024-04-24T06:56:17.725121+0000) 2024-04-24T06:56:27.768 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:27 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:27.333+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.826231+0000 front 2024-04-24T06:56:01.826096+0000 (oldest deadline 2024-04-24T06:56:25.326172+0000) 2024-04-24T06:56:28.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:27 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:27.915+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:28.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:27 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:27.915+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:28.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:27 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:27.915+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:56:28.445 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:28 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5[45127]: 2024-04-24T06:56:28.317+0000 7f0a316e1700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:46.549209+0000 front 2024-04-24T06:55:46.549265+0000 (oldest deadline 2024-04-24T06:56:11.849276+0000) 2024-04-24T06:56:28.445 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:28 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5[45127]: 2024-04-24T06:56:28.317+0000 7f0a316e1700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:55.950328+0000 front 2024-04-24T06:55:55.950407+0000 (oldest deadline 2024-04-24T06:56:17.050314+0000) 2024-04-24T06:56:28.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:28 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:28.182+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:56.064734+0000 front 2024-04-24T06:55:56.064528+0000 (oldest deadline 2024-04-24T06:56:18.364453+0000) 2024-04-24T06:56:28.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:28 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:28.182+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.265368+0000 front 2024-04-24T06:56:01.265402+0000 (oldest deadline 2024-04-24T06:56:25.965496+0000) 2024-04-24T06:56:28.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:28 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:28.287+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:28.447 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:28 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:28.287+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:56:28.447 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:28 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:28.287+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:56:28.505 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:28 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:28.246+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:43.854855+0000 front 2024-04-24T06:55:43.854927+0000 (oldest deadline 2024-04-24T06:56:07.955068+0000) 2024-04-24T06:56:28.506 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:28 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:28.246+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.958304+0000 front 2024-04-24T06:55:54.958228+0000 (oldest deadline 2024-04-24T06:56:18.458488+0000) 2024-04-24T06:56:28.506 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:28 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:28.246+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.160560+0000 front 2024-04-24T06:56:03.160633+0000 (oldest deadline 2024-04-24T06:56:27.860782+0000) 2024-04-24T06:56:28.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:28 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:28.232+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:28.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:28 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:28.232+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:56:28.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:28 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:28.232+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.915218+0000 front 2024-04-24T06:56:03.914687+0000 (oldest deadline 2024-04-24T06:56:28.014915+0000) 2024-04-24T06:56:28.768 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:28 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:28.301+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.624953+0000 front 2024-04-24T06:55:53.625107+0000 (oldest deadline 2024-04-24T06:56:17.725121+0000) 2024-04-24T06:56:28.768 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:28 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:28.301+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.826231+0000 front 2024-04-24T06:56:01.826096+0000 (oldest deadline 2024-04-24T06:56:25.326172+0000) 2024-04-24T06:56:29.265 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:28 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:28.906+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:29.265 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:28 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:28.906+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:29.266 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:28 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:28.906+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:56:29.445 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:29 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:29.150+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:56.064734+0000 front 2024-04-24T06:55:56.064528+0000 (oldest deadline 2024-04-24T06:56:18.364453+0000) 2024-04-24T06:56:29.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:29 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:29.150+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.265368+0000 front 2024-04-24T06:56:01.265402+0000 (oldest deadline 2024-04-24T06:56:25.965496+0000) 2024-04-24T06:56:29.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:29 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:29.315+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:29.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:29 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:29.315+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:56:29.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:29 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:29.315+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:56:29.518 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:29 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:29.265+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.624953+0000 front 2024-04-24T06:55:53.625107+0000 (oldest deadline 2024-04-24T06:56:17.725121+0000) 2024-04-24T06:56:29.518 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:29 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:29.265+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.826231+0000 front 2024-04-24T06:56:01.826096+0000 (oldest deadline 2024-04-24T06:56:25.326172+0000) 2024-04-24T06:56:29.732 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:29 smithi137 podman[70224]: 2024-04-24 06:56:29.47705926 +0000 UTC m=+5.534413275 container died 1b0e660e6aca4747b75c6a9127bba56f095b760491449354f66fa387d26c632f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-24T06:56:29.755 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:29 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:29.266+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:29.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:29 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:29.266+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:56:29.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:29 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:29.266+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.915218+0000 front 2024-04-24T06:56:03.914687+0000 (oldest deadline 2024-04-24T06:56:28.014915+0000) 2024-04-24T06:56:29.756 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:29 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:29.268+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:43.854855+0000 front 2024-04-24T06:55:43.854927+0000 (oldest deadline 2024-04-24T06:56:07.955068+0000) 2024-04-24T06:56:29.756 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:29 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:29.268+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.958304+0000 front 2024-04-24T06:55:54.958228+0000 (oldest deadline 2024-04-24T06:56:18.458488+0000) 2024-04-24T06:56:29.756 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:29 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:29.268+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.160560+0000 front 2024-04-24T06:56:03.160633+0000 (oldest deadline 2024-04-24T06:56:27.860782+0000) 2024-04-24T06:56:30.081 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:29 smithi137 podman[70224]: 2024-04-24 06:56:29.839198932 +0000 UTC m=+5.896552941 container cleanup 1b0e660e6aca4747b75c6a9127bba56f095b760491449354f66fa387d26c632f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5) 2024-04-24T06:56:30.082 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:29 smithi137 bash[70224]: ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5 2024-04-24T06:56:30.267 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:29 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:29.938+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:30.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:29 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:29.938+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:30.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:29 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:29.938+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:56:30.407 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:30 smithi137 podman[70242]: 2024-04-24 06:56:30.082122024 +0000 UTC m=+0.608000525 container remove 1b0e660e6aca4747b75c6a9127bba56f095b760491449354f66fa387d26c632f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T06:56:30.407 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:30 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:30.307+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:30.407 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:30 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:30.308+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:56:30.407 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:30 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:30.308+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:56:30.408 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:30 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:30.188+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:56.064734+0000 front 2024-04-24T06:55:56.064528+0000 (oldest deadline 2024-04-24T06:56:18.364453+0000) 2024-04-24T06:56:30.408 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:30 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:30.188+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.265368+0000 front 2024-04-24T06:56:01.265402+0000 (oldest deadline 2024-04-24T06:56:25.965496+0000) 2024-04-24T06:56:30.505 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:30 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:30.226+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:30.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:30 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:30.226+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:56:30.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:30 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:30.226+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.915218+0000 front 2024-04-24T06:56:03.914687+0000 (oldest deadline 2024-04-24T06:56:28.014915+0000) 2024-04-24T06:56:30.506 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:30 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:30.300+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:43.854855+0000 front 2024-04-24T06:55:43.854927+0000 (oldest deadline 2024-04-24T06:56:07.955068+0000) 2024-04-24T06:56:30.506 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:30 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:30.300+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.958304+0000 front 2024-04-24T06:55:54.958228+0000 (oldest deadline 2024-04-24T06:56:18.458488+0000) 2024-04-24T06:56:30.507 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:30 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:30.300+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.160560+0000 front 2024-04-24T06:56:03.160633+0000 (oldest deadline 2024-04-24T06:56:27.860782+0000) 2024-04-24T06:56:30.695 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:30 smithi137 podman[70331]: 2024-04-24 06:56:30.310010629 +0000 UTC m=+0.017906782 image pull b5c3183062a6ce19f361681cc58de6e3ac1de1a54c80e6094ec5695237cd98d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b 2024-04-24T06:56:30.696 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:30 smithi137 podman[70331]: 2024-04-24 06:56:30.432986948 +0000 UTC m=+0.140883100 container create 1ce2c27805490b708c5fb0138e93cca71a4fa1de3517ebc8954abe70f06b8d80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-24T06:56:30.696 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:30 smithi137 podman[70331]: 2024-04-24 06:56:30.635514803 +0000 UTC m=+0.343410954 container init 1ce2c27805490b708c5fb0138e93cca71a4fa1de3517ebc8954abe70f06b8d80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-24T06:56:30.696 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:30 smithi137 podman[70331]: 2024-04-24 06:56:30.638566084 +0000 UTC m=+0.346462237 container start 1ce2c27805490b708c5fb0138e93cca71a4fa1de3517ebc8954abe70f06b8d80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-24T06:56:30.768 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:30 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:30.294+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.624953+0000 front 2024-04-24T06:55:53.625107+0000 (oldest deadline 2024-04-24T06:56:17.725121+0000) 2024-04-24T06:56:30.768 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:30 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:30.294+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.826231+0000 front 2024-04-24T06:56:01.826096+0000 (oldest deadline 2024-04-24T06:56:25.326172+0000) 2024-04-24T06:56:31.158 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:30 smithi137 podman[70331]: 2024-04-24 06:56:30.714182567 +0000 UTC m=+0.422078722 container attach 1ce2c27805490b708c5fb0138e93cca71a4fa1de3517ebc8954abe70f06b8d80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T06:56:31.159 INFO:journalctl@ceph.osd.5.smithi137.stdout:Apr 24 06:56:30 smithi137 podman[70331]: 2024-04-24 06:56:30.929612303 +0000 UTC m=+0.637508458 container died 1ce2c27805490b708c5fb0138e93cca71a4fa1de3517ebc8954abe70f06b8d80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-24T06:56:31.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:30 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:30.910+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:31.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:30 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:30.910+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:31.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:30 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:30.910+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:56:31.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:31 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:31.157+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:56.064734+0000 front 2024-04-24T06:55:56.064528+0000 (oldest deadline 2024-04-24T06:56:18.364453+0000) 2024-04-24T06:56:31.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:31 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:31.157+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.265368+0000 front 2024-04-24T06:56:01.265402+0000 (oldest deadline 2024-04-24T06:56:25.965496+0000) 2024-04-24T06:56:31.447 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:31 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:31.334+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:31.447 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:31 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:31.334+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:56:31.447 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:31 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:31.334+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:56:31.506 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:31 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:31.346+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:43.854855+0000 front 2024-04-24T06:55:43.854927+0000 (oldest deadline 2024-04-24T06:56:07.955068+0000) 2024-04-24T06:56:31.506 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:31 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:31.346+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.958304+0000 front 2024-04-24T06:55:54.958228+0000 (oldest deadline 2024-04-24T06:56:18.458488+0000) 2024-04-24T06:56:31.506 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:31 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:31.346+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.160560+0000 front 2024-04-24T06:56:03.160633+0000 (oldest deadline 2024-04-24T06:56:27.860782+0000) 2024-04-24T06:56:31.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:31 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:31.244+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:31.507 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:31 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:31.244+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:56:31.507 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:31 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:31.244+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.915218+0000 front 2024-04-24T06:56:03.914687+0000 (oldest deadline 2024-04-24T06:56:28.014915+0000) 2024-04-24T06:56:31.524 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@osd.5.service' 2024-04-24T06:56:31.562 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T06:56:31.563 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-24T06:56:31.563 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-24T06:56:31.564 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-43e9bad0-0204-11ef-bc93-c7b262605968@osd.6 2024-04-24T06:56:31.767 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:31 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:31.328+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.624953+0000 front 2024-04-24T06:55:53.625107+0000 (oldest deadline 2024-04-24T06:56:17.725121+0000) 2024-04-24T06:56:31.768 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:31 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:31.328+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.826231+0000 front 2024-04-24T06:56:01.826096+0000 (oldest deadline 2024-04-24T06:56:25.326172+0000) 2024-04-24T06:56:32.005 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:31 smithi005 systemd[1]: Stopping Ceph osd.6 for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:56:32.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:31 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:31.901+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:32.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:31 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:31.901+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:32.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:31 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:31.901+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:56:32.445 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:32 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:32.188+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:56.064734+0000 front 2024-04-24T06:55:56.064528+0000 (oldest deadline 2024-04-24T06:56:18.364453+0000) 2024-04-24T06:56:32.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:32 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:32.188+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.265368+0000 front 2024-04-24T06:56:01.265402+0000 (oldest deadline 2024-04-24T06:56:25.965496+0000) 2024-04-24T06:56:32.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:32 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:32.291+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:32.447 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:32 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:32.291+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:56:32.447 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:32 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:32.291+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:56:32.506 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:32 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:32.162+0000 7fe6df000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T06:56:32.506 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:32 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:32.162+0000 7fe6df000700 -1 osd.6 102 *** Got signal Terminated *** 2024-04-24T06:56:32.507 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:32 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:32.162+0000 7fe6df000700 -1 osd.6 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T06:56:32.507 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:32 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:32.330+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:43.854855+0000 front 2024-04-24T06:55:43.854927+0000 (oldest deadline 2024-04-24T06:56:07.955068+0000) 2024-04-24T06:56:32.507 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:32 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:32.330+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.958304+0000 front 2024-04-24T06:55:54.958228+0000 (oldest deadline 2024-04-24T06:56:18.458488+0000) 2024-04-24T06:56:32.507 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:32 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:32.330+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.160560+0000 front 2024-04-24T06:56:03.160633+0000 (oldest deadline 2024-04-24T06:56:27.860782+0000) 2024-04-24T06:56:32.507 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:32 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:32.255+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:32.507 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:32 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:32.255+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:56:32.508 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:32 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:32.255+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.915218+0000 front 2024-04-24T06:56:03.914687+0000 (oldest deadline 2024-04-24T06:56:28.014915+0000) 2024-04-24T06:56:32.768 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:32 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:32.320+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.624953+0000 front 2024-04-24T06:55:53.625107+0000 (oldest deadline 2024-04-24T06:56:17.725121+0000) 2024-04-24T06:56:32.768 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:32 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:32.320+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.826231+0000 front 2024-04-24T06:56:01.826096+0000 (oldest deadline 2024-04-24T06:56:25.326172+0000) 2024-04-24T06:56:33.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:32 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:32.898+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:33.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:32 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:32.898+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:33.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:32 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:32.898+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:56:33.445 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:33 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:33.167+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:56.064734+0000 front 2024-04-24T06:55:56.064528+0000 (oldest deadline 2024-04-24T06:56:18.364453+0000) 2024-04-24T06:56:33.445 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:33 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:33.167+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.265368+0000 front 2024-04-24T06:56:01.265402+0000 (oldest deadline 2024-04-24T06:56:25.965496+0000) 2024-04-24T06:56:33.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:33 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:33.253+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:33.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:33 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:33.253+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:56:33.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:33 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:33.253+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:56:33.506 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:33 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:33.290+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:43.854855+0000 front 2024-04-24T06:55:43.854927+0000 (oldest deadline 2024-04-24T06:56:07.955068+0000) 2024-04-24T06:56:33.506 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:33 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:33.290+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.958304+0000 front 2024-04-24T06:55:54.958228+0000 (oldest deadline 2024-04-24T06:56:18.458488+0000) 2024-04-24T06:56:33.506 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:33 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:33.290+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.160560+0000 front 2024-04-24T06:56:03.160633+0000 (oldest deadline 2024-04-24T06:56:27.860782+0000) 2024-04-24T06:56:33.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:33 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:33.228+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:33.507 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:33 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:33.228+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:56:33.507 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:33 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:33.228+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.915218+0000 front 2024-04-24T06:56:03.914687+0000 (oldest deadline 2024-04-24T06:56:28.014915+0000) 2024-04-24T06:56:33.768 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:33 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:33.300+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.624953+0000 front 2024-04-24T06:55:53.625107+0000 (oldest deadline 2024-04-24T06:56:17.725121+0000) 2024-04-24T06:56:33.768 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:33 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:33.300+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.826231+0000 front 2024-04-24T06:56:01.826096+0000 (oldest deadline 2024-04-24T06:56:25.326172+0000) 2024-04-24T06:56:34.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:33 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:33.931+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:34.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:33 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:33.931+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:34.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:33 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:33.931+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:56:34.445 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:34 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:34.148+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:56.064734+0000 front 2024-04-24T06:55:56.064528+0000 (oldest deadline 2024-04-24T06:56:18.364453+0000) 2024-04-24T06:56:34.445 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:34 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:34.148+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.265368+0000 front 2024-04-24T06:56:01.265402+0000 (oldest deadline 2024-04-24T06:56:25.965496+0000) 2024-04-24T06:56:34.445 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:34 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:34.235+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:34.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:34 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:34.235+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:56:34.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:34 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:34.235+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:56:34.506 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:34 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:34.281+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:43.854855+0000 front 2024-04-24T06:55:43.854927+0000 (oldest deadline 2024-04-24T06:56:07.955068+0000) 2024-04-24T06:56:34.506 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:34 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:34.281+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.958304+0000 front 2024-04-24T06:55:54.958228+0000 (oldest deadline 2024-04-24T06:56:18.458488+0000) 2024-04-24T06:56:34.506 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:34 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:34.281+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.160560+0000 front 2024-04-24T06:56:03.160633+0000 (oldest deadline 2024-04-24T06:56:27.860782+0000) 2024-04-24T06:56:34.506 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:34 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:34.281+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:07.862406+0000 front 2024-04-24T06:56:07.861917+0000 (oldest deadline 2024-04-24T06:56:33.762018+0000) 2024-04-24T06:56:34.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:34 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:34.210+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:34.507 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:34 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:34.210+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:56:34.507 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:34 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:34.210+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.915218+0000 front 2024-04-24T06:56:03.914687+0000 (oldest deadline 2024-04-24T06:56:28.014915+0000) 2024-04-24T06:56:34.507 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:34 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:34.210+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.517851+0000 front 2024-04-24T06:56:08.518044+0000 (oldest deadline 2024-04-24T06:56:33.817722+0000) 2024-04-24T06:56:34.768 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:34 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:34.334+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.624953+0000 front 2024-04-24T06:55:53.625107+0000 (oldest deadline 2024-04-24T06:56:17.725121+0000) 2024-04-24T06:56:34.768 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:34 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:34.334+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.826231+0000 front 2024-04-24T06:56:01.826096+0000 (oldest deadline 2024-04-24T06:56:25.326172+0000) 2024-04-24T06:56:35.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:34 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:34.942+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:35.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:34 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:34.942+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:35.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:34 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:34.942+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:56:35.445 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:35 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:35.167+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:56.064734+0000 front 2024-04-24T06:55:56.064528+0000 (oldest deadline 2024-04-24T06:56:18.364453+0000) 2024-04-24T06:56:35.445 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:35 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:35.167+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.265368+0000 front 2024-04-24T06:56:01.265402+0000 (oldest deadline 2024-04-24T06:56:25.965496+0000) 2024-04-24T06:56:35.445 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:35 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:35.167+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.266701+0000 front 2024-04-24T06:56:08.266799+0000 (oldest deadline 2024-04-24T06:56:34.666972+0000) 2024-04-24T06:56:35.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:35 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:35.232+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:35.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:35 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:35.232+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:56:35.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:35 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:35.232+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:56:35.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:35 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:35.232+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:56:35.505 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:35 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:35.228+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:35.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:35 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:35.228+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:56:35.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:35 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:35.228+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.915218+0000 front 2024-04-24T06:56:03.914687+0000 (oldest deadline 2024-04-24T06:56:28.014915+0000) 2024-04-24T06:56:35.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:35 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:35.228+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.517851+0000 front 2024-04-24T06:56:08.518044+0000 (oldest deadline 2024-04-24T06:56:33.817722+0000) 2024-04-24T06:56:35.506 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:35 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:35.236+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:43.854855+0000 front 2024-04-24T06:55:43.854927+0000 (oldest deadline 2024-04-24T06:56:07.955068+0000) 2024-04-24T06:56:35.507 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:35 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:35.236+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.958304+0000 front 2024-04-24T06:55:54.958228+0000 (oldest deadline 2024-04-24T06:56:18.458488+0000) 2024-04-24T06:56:35.507 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:35 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:35.236+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.160560+0000 front 2024-04-24T06:56:03.160633+0000 (oldest deadline 2024-04-24T06:56:27.860782+0000) 2024-04-24T06:56:35.507 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:35 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:35.236+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:07.862406+0000 front 2024-04-24T06:56:07.861917+0000 (oldest deadline 2024-04-24T06:56:33.762018+0000) 2024-04-24T06:56:35.768 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:35 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:35.313+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.624953+0000 front 2024-04-24T06:55:53.625107+0000 (oldest deadline 2024-04-24T06:56:17.725121+0000) 2024-04-24T06:56:35.768 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:35 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:35.313+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.826231+0000 front 2024-04-24T06:56:01.826096+0000 (oldest deadline 2024-04-24T06:56:25.326172+0000) 2024-04-24T06:56:35.768 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:35 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:35.313+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:11.728126+0000 front 2024-04-24T06:56:11.728356+0000 (oldest deadline 2024-04-24T06:56:34.627958+0000) 2024-04-24T06:56:36.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:35 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:35.981+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:36.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:35 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:35.981+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:36.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:35 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:35.981+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:56:36.445 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:36 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:36.136+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:56.064734+0000 front 2024-04-24T06:55:56.064528+0000 (oldest deadline 2024-04-24T06:56:18.364453+0000) 2024-04-24T06:56:36.445 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:36 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:36.136+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.265368+0000 front 2024-04-24T06:56:01.265402+0000 (oldest deadline 2024-04-24T06:56:25.965496+0000) 2024-04-24T06:56:36.445 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:36 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:36.136+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.266701+0000 front 2024-04-24T06:56:08.266799+0000 (oldest deadline 2024-04-24T06:56:34.666972+0000) 2024-04-24T06:56:36.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:36 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:36.214+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:36.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:36 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:36.214+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:56:36.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:36 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:36.214+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:56:36.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:36 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:36.214+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:56:36.506 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:36 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:36.235+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:43.854855+0000 front 2024-04-24T06:55:43.854927+0000 (oldest deadline 2024-04-24T06:56:07.955068+0000) 2024-04-24T06:56:36.506 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:36 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:36.235+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.958304+0000 front 2024-04-24T06:55:54.958228+0000 (oldest deadline 2024-04-24T06:56:18.458488+0000) 2024-04-24T06:56:36.507 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:36 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:36.235+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.160560+0000 front 2024-04-24T06:56:03.160633+0000 (oldest deadline 2024-04-24T06:56:27.860782+0000) 2024-04-24T06:56:36.507 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:36 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6[52891]: 2024-04-24T06:56:36.235+0000 7fe6da800700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:07.862406+0000 front 2024-04-24T06:56:07.861917+0000 (oldest deadline 2024-04-24T06:56:33.762018+0000) 2024-04-24T06:56:36.507 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:36 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:36.246+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:36.507 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:36 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:36.246+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:56:36.508 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:36 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:36.246+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.915218+0000 front 2024-04-24T06:56:03.914687+0000 (oldest deadline 2024-04-24T06:56:28.014915+0000) 2024-04-24T06:56:36.508 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:36 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:36.246+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.517851+0000 front 2024-04-24T06:56:08.518044+0000 (oldest deadline 2024-04-24T06:56:33.817722+0000) 2024-04-24T06:56:36.770 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:36 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:36.305+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.624953+0000 front 2024-04-24T06:55:53.625107+0000 (oldest deadline 2024-04-24T06:56:17.725121+0000) 2024-04-24T06:56:36.770 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:36 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:36.305+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.826231+0000 front 2024-04-24T06:56:01.826096+0000 (oldest deadline 2024-04-24T06:56:25.326172+0000) 2024-04-24T06:56:36.770 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:36 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:36.305+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:11.728126+0000 front 2024-04-24T06:56:11.728356+0000 (oldest deadline 2024-04-24T06:56:34.627958+0000) 2024-04-24T06:56:37.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:36 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:36.978+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:37.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:36 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:36.978+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:37.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:36 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:36.978+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:56:37.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:36 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:36.978+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:10.875826+0000 front 2024-04-24T06:56:10.875743+0000 (oldest deadline 2024-04-24T06:56:36.175699+0000) 2024-04-24T06:56:37.445 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:37 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:37.202+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:37.445 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:37 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:37.202+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:56:37.445 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:37 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:37.202+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:56:37.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:37 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:37.202+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:56:37.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:37 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:37.120+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:56.064734+0000 front 2024-04-24T06:55:56.064528+0000 (oldest deadline 2024-04-24T06:56:18.364453+0000) 2024-04-24T06:56:37.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:37 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:37.120+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.265368+0000 front 2024-04-24T06:56:01.265402+0000 (oldest deadline 2024-04-24T06:56:25.965496+0000) 2024-04-24T06:56:37.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:37 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:37.120+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.266701+0000 front 2024-04-24T06:56:08.266799+0000 (oldest deadline 2024-04-24T06:56:34.666972+0000) 2024-04-24T06:56:37.589 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:37 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:37.258+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:37.590 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:37 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:37.258+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:56:37.590 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:37 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:37.258+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.915218+0000 front 2024-04-24T06:56:03.914687+0000 (oldest deadline 2024-04-24T06:56:28.014915+0000) 2024-04-24T06:56:37.590 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:37 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:37.258+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.517851+0000 front 2024-04-24T06:56:08.518044+0000 (oldest deadline 2024-04-24T06:56:33.817722+0000) 2024-04-24T06:56:37.768 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:37 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:37.327+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.624953+0000 front 2024-04-24T06:55:53.625107+0000 (oldest deadline 2024-04-24T06:56:17.725121+0000) 2024-04-24T06:56:37.768 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:37 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:37.327+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.826231+0000 front 2024-04-24T06:56:01.826096+0000 (oldest deadline 2024-04-24T06:56:25.326172+0000) 2024-04-24T06:56:37.768 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:37 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:37.327+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:11.728126+0000 front 2024-04-24T06:56:11.728356+0000 (oldest deadline 2024-04-24T06:56:34.627958+0000) 2024-04-24T06:56:38.005 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:37 smithi005 podman[97815]: 2024-04-24 06:56:37.588706941 +0000 UTC m=+5.546054044 container died 56254981f47985578c32e7c2313623c6d37d1d2dc0eeef46dc7aca637bb6e858 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T06:56:38.256 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:38 smithi005 podman[97815]: 2024-04-24 06:56:38.148367718 +0000 UTC m=+6.105714799 container cleanup 56254981f47985578c32e7c2313623c6d37d1d2dc0eeef46dc7aca637bb6e858 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True) 2024-04-24T06:56:38.256 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:38 smithi005 bash[97815]: ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6 2024-04-24T06:56:38.256 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:38 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:38.239+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:38.256 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:38 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:38.239+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:56:38.257 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:38 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:38.239+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.915218+0000 front 2024-04-24T06:56:03.914687+0000 (oldest deadline 2024-04-24T06:56:28.014915+0000) 2024-04-24T06:56:38.257 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:38 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:38.239+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.517851+0000 front 2024-04-24T06:56:08.518044+0000 (oldest deadline 2024-04-24T06:56:33.817722+0000) 2024-04-24T06:56:38.353 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:38 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:38.018+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:38.353 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:38 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:38.018+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:38.353 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:38 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:38.018+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:56:38.353 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:38 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:38.018+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:10.875826+0000 front 2024-04-24T06:56:10.875743+0000 (oldest deadline 2024-04-24T06:56:36.175699+0000) 2024-04-24T06:56:38.445 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:38 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:38.168+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:56.064734+0000 front 2024-04-24T06:55:56.064528+0000 (oldest deadline 2024-04-24T06:56:18.364453+0000) 2024-04-24T06:56:38.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:38 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:38.168+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.265368+0000 front 2024-04-24T06:56:01.265402+0000 (oldest deadline 2024-04-24T06:56:25.965496+0000) 2024-04-24T06:56:38.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:38 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:38.168+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.266701+0000 front 2024-04-24T06:56:08.266799+0000 (oldest deadline 2024-04-24T06:56:34.666972+0000) 2024-04-24T06:56:38.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:38 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:38.251+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:38.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:38 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:38.251+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:56:38.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:38 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:38.251+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:56:38.447 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:38 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:38.251+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:56:38.755 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:38 smithi005 podman[97835]: 2024-04-24 06:56:38.399366874 +0000 UTC m=+0.792856795 container remove 56254981f47985578c32e7c2313623c6d37d1d2dc0eeef46dc7aca637bb6e858 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416) 2024-04-24T06:56:38.767 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:38 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:38.353+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.624953+0000 front 2024-04-24T06:55:53.625107+0000 (oldest deadline 2024-04-24T06:56:17.725121+0000) 2024-04-24T06:56:38.768 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:38 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:38.353+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.826231+0000 front 2024-04-24T06:56:01.826096+0000 (oldest deadline 2024-04-24T06:56:25.326172+0000) 2024-04-24T06:56:38.768 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:38 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:38.353+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:11.728126+0000 front 2024-04-24T06:56:11.728356+0000 (oldest deadline 2024-04-24T06:56:34.627958+0000) 2024-04-24T06:56:39.392 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:39 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:39.036+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:39.393 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:39 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:39.036+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:39.393 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:39 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:39.036+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:56:39.393 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:39 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:39.036+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:10.875826+0000 front 2024-04-24T06:56:10.875743+0000 (oldest deadline 2024-04-24T06:56:36.175699+0000) 2024-04-24T06:56:39.445 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:39 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:39.167+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:56.064734+0000 front 2024-04-24T06:55:56.064528+0000 (oldest deadline 2024-04-24T06:56:18.364453+0000) 2024-04-24T06:56:39.445 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:39 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:39.167+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.265368+0000 front 2024-04-24T06:56:01.265402+0000 (oldest deadline 2024-04-24T06:56:25.965496+0000) 2024-04-24T06:56:39.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:39 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:39.167+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.266701+0000 front 2024-04-24T06:56:08.266799+0000 (oldest deadline 2024-04-24T06:56:34.666972+0000) 2024-04-24T06:56:39.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:39 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:39.237+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:39.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:39 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:39.237+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:56:39.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:39 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:39.237+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:56:39.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:39 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:39.237+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:56:39.503 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:39 smithi005 podman[97922]: 2024-04-24 06:56:39.113508321 +0000 UTC m=+0.033656728 image pull b5c3183062a6ce19f361681cc58de6e3ac1de1a54c80e6094ec5695237cd98d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b 2024-04-24T06:56:39.504 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:39 smithi005 podman[97922]: 2024-04-24 06:56:39.341927732 +0000 UTC m=+0.262076174 container create 3f8808d56aa5756a67f8a715d8646a4b45915b5788f166a9bf56d5714c1f4bcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, maintainer=Guillaume Abrioux ) 2024-04-24T06:56:39.504 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:39 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:39.201+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:39.504 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:39 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:39.201+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:56:39.504 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:39 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:39.201+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.915218+0000 front 2024-04-24T06:56:03.914687+0000 (oldest deadline 2024-04-24T06:56:28.014915+0000) 2024-04-24T06:56:39.504 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:39 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:39.201+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.517851+0000 front 2024-04-24T06:56:08.518044+0000 (oldest deadline 2024-04-24T06:56:33.817722+0000) 2024-04-24T06:56:39.679 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:39 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:39.392+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.624953+0000 front 2024-04-24T06:55:53.625107+0000 (oldest deadline 2024-04-24T06:56:17.725121+0000) 2024-04-24T06:56:39.679 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:39 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:39.392+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.826231+0000 front 2024-04-24T06:56:01.826096+0000 (oldest deadline 2024-04-24T06:56:25.326172+0000) 2024-04-24T06:56:39.679 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:39 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:39.392+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:11.728126+0000 front 2024-04-24T06:56:11.728356+0000 (oldest deadline 2024-04-24T06:56:34.627958+0000) 2024-04-24T06:56:39.756 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:39 smithi005 podman[97922]: 2024-04-24 06:56:39.606174841 +0000 UTC m=+0.526323290 container init 3f8808d56aa5756a67f8a715d8646a4b45915b5788f166a9bf56d5714c1f4bcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T06:56:39.756 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:39 smithi005 podman[97922]: 2024-04-24 06:56:39.61801311 +0000 UTC m=+0.538161559 container start 3f8808d56aa5756a67f8a715d8646a4b45915b5788f166a9bf56d5714c1f4bcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-24T06:56:39.756 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:39 smithi005 podman[97922]: 2024-04-24 06:56:39.68557703 +0000 UTC m=+0.605725539 container attach 3f8808d56aa5756a67f8a715d8646a4b45915b5788f166a9bf56d5714c1f4bcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-24T06:56:40.384 INFO:journalctl@ceph.osd.6.smithi005.stdout:Apr 24 06:56:40 smithi005 podman[97922]: 2024-04-24 06:56:40.089449943 +0000 UTC m=+1.009598391 container died 3f8808d56aa5756a67f8a715d8646a4b45915b5788f166a9bf56d5714c1f4bcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.schema-version=1.0) 2024-04-24T06:56:40.384 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:40 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:40.241+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:40.384 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:40 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:40.241+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:56:40.385 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:40 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:40.241+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.915218+0000 front 2024-04-24T06:56:03.914687+0000 (oldest deadline 2024-04-24T06:56:28.014915+0000) 2024-04-24T06:56:40.385 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:40 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:40.241+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.517851+0000 front 2024-04-24T06:56:08.518044+0000 (oldest deadline 2024-04-24T06:56:33.817722+0000) 2024-04-24T06:56:40.386 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:40 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:40.061+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:40.386 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:40 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:40.061+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:40.387 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:40 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:40.061+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:56:40.387 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:40 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:40.061+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:10.875826+0000 front 2024-04-24T06:56:10.875743+0000 (oldest deadline 2024-04-24T06:56:36.175699+0000) 2024-04-24T06:56:40.445 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:40 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:40.220+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:40.445 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:40 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:40.220+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:56:40.445 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:40 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:40.220+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:56:40.445 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:40 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:40.220+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:56:40.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:40 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:40.168+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:56.064734+0000 front 2024-04-24T06:55:56.064528+0000 (oldest deadline 2024-04-24T06:56:18.364453+0000) 2024-04-24T06:56:40.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:40 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:40.168+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.265368+0000 front 2024-04-24T06:56:01.265402+0000 (oldest deadline 2024-04-24T06:56:25.965496+0000) 2024-04-24T06:56:40.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:40 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:40.168+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.266701+0000 front 2024-04-24T06:56:08.266799+0000 (oldest deadline 2024-04-24T06:56:34.666972+0000) 2024-04-24T06:56:40.649 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:40 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:40.385+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.624953+0000 front 2024-04-24T06:55:53.625107+0000 (oldest deadline 2024-04-24T06:56:17.725121+0000) 2024-04-24T06:56:40.649 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:40 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:40.385+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.826231+0000 front 2024-04-24T06:56:01.826096+0000 (oldest deadline 2024-04-24T06:56:25.326172+0000) 2024-04-24T06:56:40.649 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:40 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:40.385+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:11.728126+0000 front 2024-04-24T06:56:11.728356+0000 (oldest deadline 2024-04-24T06:56:34.627958+0000) 2024-04-24T06:56:40.840 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@osd.6.service' 2024-04-24T06:56:40.957 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T06:56:40.958 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-24T06:56:40.958 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-24T06:56:40.958 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-43e9bad0-0204-11ef-bc93-c7b262605968@osd.7 2024-04-24T06:56:41.254 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:41 smithi037 systemd[1]: Stopping Ceph osd.7 for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:56:41.254 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:41 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:41.087+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:41.254 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:41 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:41.087+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:41.255 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:41 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:41.087+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:56:41.255 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:41 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:41.087+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:10.875826+0000 front 2024-04-24T06:56:10.875743+0000 (oldest deadline 2024-04-24T06:56:36.175699+0000) 2024-04-24T06:56:41.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:41 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:41.123+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:56.064734+0000 front 2024-04-24T06:55:56.064528+0000 (oldest deadline 2024-04-24T06:56:18.364453+0000) 2024-04-24T06:56:41.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:41 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:41.123+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.265368+0000 front 2024-04-24T06:56:01.265402+0000 (oldest deadline 2024-04-24T06:56:25.965496+0000) 2024-04-24T06:56:41.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:41 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:41.123+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.266701+0000 front 2024-04-24T06:56:08.266799+0000 (oldest deadline 2024-04-24T06:56:34.666972+0000) 2024-04-24T06:56:41.447 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:41 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:41.254+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:41.447 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:41 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:41.254+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:56:41.447 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:41 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:41.254+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:56:41.448 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:41 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:41.254+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:56:41.518 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:41 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:41.253+0000 7f993c71f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T06:56:41.518 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:41 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:41.253+0000 7f993c71f700 -1 osd.7 102 *** Got signal Terminated *** 2024-04-24T06:56:41.518 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:41 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:41.253+0000 7f993c71f700 -1 osd.7 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T06:56:41.519 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:41 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:41.341+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.624953+0000 front 2024-04-24T06:55:53.625107+0000 (oldest deadline 2024-04-24T06:56:17.725121+0000) 2024-04-24T06:56:41.519 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:41 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:41.341+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.826231+0000 front 2024-04-24T06:56:01.826096+0000 (oldest deadline 2024-04-24T06:56:25.326172+0000) 2024-04-24T06:56:41.519 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:41 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:41.341+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:11.728126+0000 front 2024-04-24T06:56:11.728356+0000 (oldest deadline 2024-04-24T06:56:34.627958+0000) 2024-04-24T06:56:41.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:41 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:41.269+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:41.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:41 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:41.269+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:56:41.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:41 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:41.269+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.915218+0000 front 2024-04-24T06:56:03.914687+0000 (oldest deadline 2024-04-24T06:56:28.014915+0000) 2024-04-24T06:56:41.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:41 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:41.269+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.517851+0000 front 2024-04-24T06:56:08.518044+0000 (oldest deadline 2024-04-24T06:56:33.817722+0000) 2024-04-24T06:56:42.334 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:42 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:42.049+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:42.334 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:42 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:42.049+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:42.334 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:42 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:42.049+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:56:42.334 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:42 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:42.049+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:10.875826+0000 front 2024-04-24T06:56:10.875743+0000 (oldest deadline 2024-04-24T06:56:36.175699+0000) 2024-04-24T06:56:42.445 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:42 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:42.111+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:56.064734+0000 front 2024-04-24T06:55:56.064528+0000 (oldest deadline 2024-04-24T06:56:18.364453+0000) 2024-04-24T06:56:42.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:42 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:42.111+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.265368+0000 front 2024-04-24T06:56:01.265402+0000 (oldest deadline 2024-04-24T06:56:25.965496+0000) 2024-04-24T06:56:42.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:42 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:42.111+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.266701+0000 front 2024-04-24T06:56:08.266799+0000 (oldest deadline 2024-04-24T06:56:34.666972+0000) 2024-04-24T06:56:42.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:42 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:42.258+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:42.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:42 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:42.258+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:56:42.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:42 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:42.258+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:56:42.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:42 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:42.258+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:56:42.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:42 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:42.258+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:18.541399+0000 front 2024-04-24T06:56:18.541315+0000 (oldest deadline 2024-04-24T06:56:42.041243+0000) 2024-04-24T06:56:42.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:42 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:42.319+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:42.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:42 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:42.319+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:56:42.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:42 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:42.319+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.915218+0000 front 2024-04-24T06:56:03.914687+0000 (oldest deadline 2024-04-24T06:56:28.014915+0000) 2024-04-24T06:56:42.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:42 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:42.319+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.517851+0000 front 2024-04-24T06:56:08.518044+0000 (oldest deadline 2024-04-24T06:56:33.817722+0000) 2024-04-24T06:56:42.767 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:42 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:42.333+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.624953+0000 front 2024-04-24T06:55:53.625107+0000 (oldest deadline 2024-04-24T06:56:17.725121+0000) 2024-04-24T06:56:42.768 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:42 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:42.333+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.826231+0000 front 2024-04-24T06:56:01.826096+0000 (oldest deadline 2024-04-24T06:56:25.326172+0000) 2024-04-24T06:56:42.768 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:42 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:42.333+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:11.728126+0000 front 2024-04-24T06:56:11.728356+0000 (oldest deadline 2024-04-24T06:56:34.627958+0000) 2024-04-24T06:56:42.768 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:42 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:42.333+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:19.928965+0000 front 2024-04-24T06:56:19.929019+0000 (oldest deadline 2024-04-24T06:56:41.629085+0000) 2024-04-24T06:56:43.327 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:43 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:43.071+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:43.328 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:43 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:43.071+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:43.328 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:43 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:43.071+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:56:43.328 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:43 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:43.071+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:10.875826+0000 front 2024-04-24T06:56:10.875743+0000 (oldest deadline 2024-04-24T06:56:36.175699+0000) 2024-04-24T06:56:43.445 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:43 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:43.068+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:56.064734+0000 front 2024-04-24T06:55:56.064528+0000 (oldest deadline 2024-04-24T06:56:18.364453+0000) 2024-04-24T06:56:43.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:43 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:43.068+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.265368+0000 front 2024-04-24T06:56:01.265402+0000 (oldest deadline 2024-04-24T06:56:25.965496+0000) 2024-04-24T06:56:43.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:43 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:43.068+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.266701+0000 front 2024-04-24T06:56:08.266799+0000 (oldest deadline 2024-04-24T06:56:34.666972+0000) 2024-04-24T06:56:43.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:43 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:43.068+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:20.969376+0000 front 2024-04-24T06:56:20.969408+0000 (oldest deadline 2024-04-24T06:56:42.669350+0000) 2024-04-24T06:56:43.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:43 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:43.262+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:43.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:43 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:43.262+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:56:43.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:43 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:43.262+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:56:43.447 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:43 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:43.262+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:56:43.447 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:43 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:43.262+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:18.541399+0000 front 2024-04-24T06:56:18.541315+0000 (oldest deadline 2024-04-24T06:56:42.041243+0000) 2024-04-24T06:56:43.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:43 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:43.332+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:43.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:43 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:43.332+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:56:43.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:43 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:43.332+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.915218+0000 front 2024-04-24T06:56:03.914687+0000 (oldest deadline 2024-04-24T06:56:28.014915+0000) 2024-04-24T06:56:43.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:43 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:43.332+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.517851+0000 front 2024-04-24T06:56:08.518044+0000 (oldest deadline 2024-04-24T06:56:33.817722+0000) 2024-04-24T06:56:43.768 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:43 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:43.327+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.624953+0000 front 2024-04-24T06:55:53.625107+0000 (oldest deadline 2024-04-24T06:56:17.725121+0000) 2024-04-24T06:56:43.768 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:43 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:43.327+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.826231+0000 front 2024-04-24T06:56:01.826096+0000 (oldest deadline 2024-04-24T06:56:25.326172+0000) 2024-04-24T06:56:43.768 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:43 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:43.327+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:11.728126+0000 front 2024-04-24T06:56:11.728356+0000 (oldest deadline 2024-04-24T06:56:34.627958+0000) 2024-04-24T06:56:43.768 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:43 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:43.327+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:19.928965+0000 front 2024-04-24T06:56:19.929019+0000 (oldest deadline 2024-04-24T06:56:41.629085+0000) 2024-04-24T06:56:44.445 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:44 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:44.081+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:56.064734+0000 front 2024-04-24T06:55:56.064528+0000 (oldest deadline 2024-04-24T06:56:18.364453+0000) 2024-04-24T06:56:44.445 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:44 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:44.081+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.265368+0000 front 2024-04-24T06:56:01.265402+0000 (oldest deadline 2024-04-24T06:56:25.965496+0000) 2024-04-24T06:56:44.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:44 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:44.081+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.266701+0000 front 2024-04-24T06:56:08.266799+0000 (oldest deadline 2024-04-24T06:56:34.666972+0000) 2024-04-24T06:56:44.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:44 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:44.081+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:20.969376+0000 front 2024-04-24T06:56:20.969408+0000 (oldest deadline 2024-04-24T06:56:42.669350+0000) 2024-04-24T06:56:44.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:44 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:44.295+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:44.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:44 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:44.295+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:56:44.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:44 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:44.295+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:56:44.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:44 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:44.295+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:56:44.447 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:44 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:44.295+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:18.541399+0000 front 2024-04-24T06:56:18.541315+0000 (oldest deadline 2024-04-24T06:56:42.041243+0000) 2024-04-24T06:56:44.518 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:44 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:44.063+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:44.518 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:44 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:44.063+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:44.518 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:44 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:44.063+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:56:44.518 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:44 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:44.063+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:10.875826+0000 front 2024-04-24T06:56:10.875743+0000 (oldest deadline 2024-04-24T06:56:36.175699+0000) 2024-04-24T06:56:44.519 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:44 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:44.300+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.624953+0000 front 2024-04-24T06:55:53.625107+0000 (oldest deadline 2024-04-24T06:56:17.725121+0000) 2024-04-24T06:56:44.519 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:44 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:44.300+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.826231+0000 front 2024-04-24T06:56:01.826096+0000 (oldest deadline 2024-04-24T06:56:25.326172+0000) 2024-04-24T06:56:44.519 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:44 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:44.300+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:11.728126+0000 front 2024-04-24T06:56:11.728356+0000 (oldest deadline 2024-04-24T06:56:34.627958+0000) 2024-04-24T06:56:44.519 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:44 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:44.300+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:19.928965+0000 front 2024-04-24T06:56:19.929019+0000 (oldest deadline 2024-04-24T06:56:41.629085+0000) 2024-04-24T06:56:44.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:44 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:44.295+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:44.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:44 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:44.295+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:56:44.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:44 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:44.295+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.915218+0000 front 2024-04-24T06:56:03.914687+0000 (oldest deadline 2024-04-24T06:56:28.014915+0000) 2024-04-24T06:56:44.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:44 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:44.295+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.517851+0000 front 2024-04-24T06:56:08.518044+0000 (oldest deadline 2024-04-24T06:56:33.817722+0000) 2024-04-24T06:56:45.313 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:45 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:45.062+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:56.064734+0000 front 2024-04-24T06:55:56.064528+0000 (oldest deadline 2024-04-24T06:56:18.364453+0000) 2024-04-24T06:56:45.314 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:45 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:45.062+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.265368+0000 front 2024-04-24T06:56:01.265402+0000 (oldest deadline 2024-04-24T06:56:25.965496+0000) 2024-04-24T06:56:45.314 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:45 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:45.062+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.266701+0000 front 2024-04-24T06:56:08.266799+0000 (oldest deadline 2024-04-24T06:56:34.666972+0000) 2024-04-24T06:56:45.314 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:45 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:45.062+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:20.969376+0000 front 2024-04-24T06:56:20.969408+0000 (oldest deadline 2024-04-24T06:56:42.669350+0000) 2024-04-24T06:56:45.322 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:45 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:45.023+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:45.323 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:45 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:45.023+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:45.323 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:45 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:45.023+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:56:45.323 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:45 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:45.023+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:10.875826+0000 front 2024-04-24T06:56:10.875743+0000 (oldest deadline 2024-04-24T06:56:36.175699+0000) 2024-04-24T06:56:45.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:45 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:45.313+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:45.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:45 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:45.313+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:56:45.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:45 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:45.313+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:56:45.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:45 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:45.313+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:56:45.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:45 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:45.313+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:18.541399+0000 front 2024-04-24T06:56:18.541315+0000 (oldest deadline 2024-04-24T06:56:42.041243+0000) 2024-04-24T06:56:45.755 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:45 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:45.281+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:45.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:45 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:45.281+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:56:45.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:45 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:45.281+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.915218+0000 front 2024-04-24T06:56:03.914687+0000 (oldest deadline 2024-04-24T06:56:28.014915+0000) 2024-04-24T06:56:45.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:45 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:45.281+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.517851+0000 front 2024-04-24T06:56:08.518044+0000 (oldest deadline 2024-04-24T06:56:33.817722+0000) 2024-04-24T06:56:45.768 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:45 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:45.322+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.624953+0000 front 2024-04-24T06:55:53.625107+0000 (oldest deadline 2024-04-24T06:56:17.725121+0000) 2024-04-24T06:56:45.768 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:45 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:45.322+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.826231+0000 front 2024-04-24T06:56:01.826096+0000 (oldest deadline 2024-04-24T06:56:25.326172+0000) 2024-04-24T06:56:45.768 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:45 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:45.322+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:11.728126+0000 front 2024-04-24T06:56:11.728356+0000 (oldest deadline 2024-04-24T06:56:34.627958+0000) 2024-04-24T06:56:45.768 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:45 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7[50880]: 2024-04-24T06:56:45.322+0000 7f9938d13700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:19.928965+0000 front 2024-04-24T06:56:19.929019+0000 (oldest deadline 2024-04-24T06:56:41.629085+0000) 2024-04-24T06:56:46.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:45 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:45.987+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:46.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:45 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:45.987+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:46.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:45 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:45.987+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:56:46.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:45 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:45.987+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:10.875826+0000 front 2024-04-24T06:56:10.875743+0000 (oldest deadline 2024-04-24T06:56:36.175699+0000) 2024-04-24T06:56:46.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:45 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:45.987+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:20.276664+0000 front 2024-04-24T06:56:20.276689+0000 (oldest deadline 2024-04-24T06:56:45.576810+0000) 2024-04-24T06:56:46.290 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:46 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:46.032+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:56.064734+0000 front 2024-04-24T06:55:56.064528+0000 (oldest deadline 2024-04-24T06:56:18.364453+0000) 2024-04-24T06:56:46.290 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:46 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:46.032+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.265368+0000 front 2024-04-24T06:56:01.265402+0000 (oldest deadline 2024-04-24T06:56:25.965496+0000) 2024-04-24T06:56:46.290 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:46 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:46.032+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.266701+0000 front 2024-04-24T06:56:08.266799+0000 (oldest deadline 2024-04-24T06:56:34.666972+0000) 2024-04-24T06:56:46.291 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:46 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:46.032+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:20.969376+0000 front 2024-04-24T06:56:20.969408+0000 (oldest deadline 2024-04-24T06:56:42.669350+0000) 2024-04-24T06:56:46.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:46 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:46.246+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:46.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:46 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:46.246+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:56:46.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:46 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:46.246+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.915218+0000 front 2024-04-24T06:56:03.914687+0000 (oldest deadline 2024-04-24T06:56:28.014915+0000) 2024-04-24T06:56:46.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:46 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:46.246+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.517851+0000 front 2024-04-24T06:56:08.518044+0000 (oldest deadline 2024-04-24T06:56:33.817722+0000) 2024-04-24T06:56:46.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:46 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:46.290+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:46.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:46 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:46.290+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:56:46.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:46 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:46.290+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:56:46.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:46 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:46.290+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:56:46.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:46 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:46.290+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:18.541399+0000 front 2024-04-24T06:56:18.541315+0000 (oldest deadline 2024-04-24T06:56:42.041243+0000) 2024-04-24T06:56:46.904 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:46 smithi037 podman[69538]: 2024-04-24 06:56:46.651687384 +0000 UTC m=+5.504952058 container died 689aa2f5855ecd2a4e4881d164e7379bb89dcd057b7b81a1fe7ac5ad6e587008 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T06:56:47.268 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:47 smithi037 podman[69538]: 2024-04-24 06:56:47.003648371 +0000 UTC m=+5.856913053 container cleanup 689aa2f5855ecd2a4e4881d164e7379bb89dcd057b7b81a1fe7ac5ad6e587008 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-24T06:56:47.268 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:47 smithi037 bash[69538]: ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7 2024-04-24T06:56:47.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:47 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:47.009+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:47.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:47 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:47.009+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:47.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:47 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:47.009+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:56:47.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:47 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:47.009+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:10.875826+0000 front 2024-04-24T06:56:10.875743+0000 (oldest deadline 2024-04-24T06:56:36.175699+0000) 2024-04-24T06:56:47.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:47 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:47.009+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:20.276664+0000 front 2024-04-24T06:56:20.276689+0000 (oldest deadline 2024-04-24T06:56:45.576810+0000) 2024-04-24T06:56:47.339 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:47 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:47.049+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:56.064734+0000 front 2024-04-24T06:55:56.064528+0000 (oldest deadline 2024-04-24T06:56:18.364453+0000) 2024-04-24T06:56:47.339 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:47 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:47.049+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.265368+0000 front 2024-04-24T06:56:01.265402+0000 (oldest deadline 2024-04-24T06:56:25.965496+0000) 2024-04-24T06:56:47.339 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:47 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:47.049+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.266701+0000 front 2024-04-24T06:56:08.266799+0000 (oldest deadline 2024-04-24T06:56:34.666972+0000) 2024-04-24T06:56:47.339 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:47 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:47.049+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:20.969376+0000 front 2024-04-24T06:56:20.969408+0000 (oldest deadline 2024-04-24T06:56:42.669350+0000) 2024-04-24T06:56:47.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:47 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:47.242+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:47.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:47 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:47.242+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:56:47.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:47 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:47.242+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.915218+0000 front 2024-04-24T06:56:03.914687+0000 (oldest deadline 2024-04-24T06:56:28.014915+0000) 2024-04-24T06:56:47.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:47 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:47.242+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.517851+0000 front 2024-04-24T06:56:08.518044+0000 (oldest deadline 2024-04-24T06:56:33.817722+0000) 2024-04-24T06:56:47.614 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:47 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:47.338+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:47.614 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:47 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:47.338+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:56:47.614 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:47 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:47.338+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:56:47.614 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:47 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:47.338+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:56:47.615 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:47 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:47.338+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:18.541399+0000 front 2024-04-24T06:56:18.541315+0000 (oldest deadline 2024-04-24T06:56:42.041243+0000) 2024-04-24T06:56:47.768 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:47 smithi037 podman[69554]: 2024-04-24 06:56:47.379936964 +0000 UTC m=+0.730172363 container remove 689aa2f5855ecd2a4e4881d164e7379bb89dcd057b7b81a1fe7ac5ad6e587008 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-24T06:56:48.049 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:47 smithi037 podman[69647]: 2024-04-24 06:56:47.681841014 +0000 UTC m=+0.018828359 image pull b5c3183062a6ce19f361681cc58de6e3ac1de1a54c80e6094ec5695237cd98d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b 2024-04-24T06:56:48.049 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:47 smithi037 podman[69647]: 2024-04-24 06:56:47.780700857 +0000 UTC m=+0.117688267 container create d61840ca65f70b58f3cf38fc6803e9735f9f114fe558d9a605b86d2ba8ee9163 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T06:56:48.049 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:47 smithi037 podman[69647]: 2024-04-24 06:56:47.991210276 +0000 UTC m=+0.328197621 container init d61840ca65f70b58f3cf38fc6803e9735f9f114fe558d9a605b86d2ba8ee9163 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-24T06:56:48.049 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:47 smithi037 podman[69647]: 2024-04-24 06:56:47.99439729 +0000 UTC m=+0.331384635 container start d61840ca65f70b58f3cf38fc6803e9735f9f114fe558d9a605b86d2ba8ee9163 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T06:56:48.050 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:48 smithi037 podman[69647]: 2024-04-24 06:56:48.049474257 +0000 UTC m=+0.386461611 container attach d61840ca65f70b58f3cf38fc6803e9735f9f114fe558d9a605b86d2ba8ee9163 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=) 2024-04-24T06:56:48.050 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:47 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:47.992+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:48.050 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:47 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:47.992+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:48.050 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:47 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:47.992+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:56:48.050 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:47 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:47.992+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:10.875826+0000 front 2024-04-24T06:56:10.875743+0000 (oldest deadline 2024-04-24T06:56:36.175699+0000) 2024-04-24T06:56:48.050 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:47 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:47.992+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:20.276664+0000 front 2024-04-24T06:56:20.276689+0000 (oldest deadline 2024-04-24T06:56:45.576810+0000) 2024-04-24T06:56:48.344 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:48 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:48.042+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:56.064734+0000 front 2024-04-24T06:55:56.064528+0000 (oldest deadline 2024-04-24T06:56:18.364453+0000) 2024-04-24T06:56:48.345 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:48 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:48.042+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.265368+0000 front 2024-04-24T06:56:01.265402+0000 (oldest deadline 2024-04-24T06:56:25.965496+0000) 2024-04-24T06:56:48.345 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:48 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:48.042+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.266701+0000 front 2024-04-24T06:56:08.266799+0000 (oldest deadline 2024-04-24T06:56:34.666972+0000) 2024-04-24T06:56:48.345 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:48 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:48.042+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:20.969376+0000 front 2024-04-24T06:56:20.969408+0000 (oldest deadline 2024-04-24T06:56:42.669350+0000) 2024-04-24T06:56:48.505 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:48 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:48.278+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:48.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:48 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:48.278+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:56:48.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:48 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:48.278+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.915218+0000 front 2024-04-24T06:56:03.914687+0000 (oldest deadline 2024-04-24T06:56:28.014915+0000) 2024-04-24T06:56:48.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:48 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:48.278+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.517851+0000 front 2024-04-24T06:56:08.518044+0000 (oldest deadline 2024-04-24T06:56:33.817722+0000) 2024-04-24T06:56:48.610 INFO:journalctl@ceph.osd.7.smithi037.stdout:Apr 24 06:56:48 smithi037 podman[69647]: 2024-04-24 06:56:48.316904658 +0000 UTC m=+0.653891998 container died d61840ca65f70b58f3cf38fc6803e9735f9f114fe558d9a605b86d2ba8ee9163 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-24T06:56:48.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:48 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:48.344+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:48.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:48 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:48.344+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:56:48.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:48 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:48.344+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:56:48.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:48 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:48.344+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:56:48.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:48 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:48.344+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:18.541399+0000 front 2024-04-24T06:56:18.541315+0000 (oldest deadline 2024-04-24T06:56:42.041243+0000) 2024-04-24T06:56:48.890 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@osd.7.service' 2024-04-24T06:56:48.931 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T06:56:48.931 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-24T06:56:48.931 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-24T06:56:48.932 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-43e9bad0-0204-11ef-bc93-c7b262605968@osd.8 2024-04-24T06:56:49.241 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:48 smithi137 systemd[1]: Stopping Ceph osd.8 for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:56:49.242 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:49 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:49.075+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:56.064734+0000 front 2024-04-24T06:55:56.064528+0000 (oldest deadline 2024-04-24T06:56:18.364453+0000) 2024-04-24T06:56:49.242 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:49 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:49.075+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.265368+0000 front 2024-04-24T06:56:01.265402+0000 (oldest deadline 2024-04-24T06:56:25.965496+0000) 2024-04-24T06:56:49.242 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:49 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:49.075+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.266701+0000 front 2024-04-24T06:56:08.266799+0000 (oldest deadline 2024-04-24T06:56:34.666972+0000) 2024-04-24T06:56:49.242 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:49 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:49.075+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:20.969376+0000 front 2024-04-24T06:56:20.969408+0000 (oldest deadline 2024-04-24T06:56:42.669350+0000) 2024-04-24T06:56:49.242 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:49 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:49.240+0000 7f7e5cfbb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T06:56:49.242 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:49 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:49.240+0000 7f7e5cfbb700 -1 osd.8 102 *** Got signal Terminated *** 2024-04-24T06:56:49.242 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:49 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:49.240+0000 7f7e5cfbb700 -1 osd.8 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T06:56:49.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:48 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:48.969+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:49.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:48 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:48.969+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:49.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:48 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:48.969+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:56:49.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:48 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:48.969+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:10.875826+0000 front 2024-04-24T06:56:10.875743+0000 (oldest deadline 2024-04-24T06:56:36.175699+0000) 2024-04-24T06:56:49.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:48 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:48.969+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:20.276664+0000 front 2024-04-24T06:56:20.276689+0000 (oldest deadline 2024-04-24T06:56:45.576810+0000) 2024-04-24T06:56:49.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:49 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:49.306+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:49.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:49 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:49.306+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:56:49.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:49 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:49.306+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.915218+0000 front 2024-04-24T06:56:03.914687+0000 (oldest deadline 2024-04-24T06:56:28.014915+0000) 2024-04-24T06:56:49.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:49 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:49.306+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.517851+0000 front 2024-04-24T06:56:08.518044+0000 (oldest deadline 2024-04-24T06:56:33.817722+0000) 2024-04-24T06:56:49.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:49 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:49.309+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:49.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:49 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:49.309+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:56:49.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:49 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:49.309+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:56:49.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:49 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:49.309+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:56:49.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:49 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:49.309+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:18.541399+0000 front 2024-04-24T06:56:18.541315+0000 (oldest deadline 2024-04-24T06:56:42.041243+0000) 2024-04-24T06:56:50.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:49 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:49.927+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:50.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:49 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:49.927+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:50.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:49 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:49.927+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:56:50.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:49 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:49.927+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:10.875826+0000 front 2024-04-24T06:56:10.875743+0000 (oldest deadline 2024-04-24T06:56:36.175699+0000) 2024-04-24T06:56:50.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:49 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:49.927+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:20.276664+0000 front 2024-04-24T06:56:20.276689+0000 (oldest deadline 2024-04-24T06:56:45.576810+0000) 2024-04-24T06:56:50.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:49 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:49.927+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:25.577454+0000 front 2024-04-24T06:56:25.577435+0000 (oldest deadline 2024-04-24T06:56:49.677340+0000) 2024-04-24T06:56:50.445 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:50 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:50.076+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:56.064734+0000 front 2024-04-24T06:55:56.064528+0000 (oldest deadline 2024-04-24T06:56:18.364453+0000) 2024-04-24T06:56:50.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:50 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:50.076+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.265368+0000 front 2024-04-24T06:56:01.265402+0000 (oldest deadline 2024-04-24T06:56:25.965496+0000) 2024-04-24T06:56:50.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:50 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:50.076+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.266701+0000 front 2024-04-24T06:56:08.266799+0000 (oldest deadline 2024-04-24T06:56:34.666972+0000) 2024-04-24T06:56:50.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:50 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:50.076+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:20.969376+0000 front 2024-04-24T06:56:20.969408+0000 (oldest deadline 2024-04-24T06:56:42.669350+0000) 2024-04-24T06:56:50.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:50 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:50.281+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:50.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:50 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:50.281+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:56:50.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:50 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:50.281+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:56:50.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:50 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:50.281+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:56:50.447 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:50 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:50.281+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:18.541399+0000 front 2024-04-24T06:56:18.541315+0000 (oldest deadline 2024-04-24T06:56:42.041243+0000) 2024-04-24T06:56:50.755 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:50 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:50.263+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:50.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:50 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:50.263+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:56:50.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:50 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:50.263+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.915218+0000 front 2024-04-24T06:56:03.914687+0000 (oldest deadline 2024-04-24T06:56:28.014915+0000) 2024-04-24T06:56:50.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:50 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:50.263+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.517851+0000 front 2024-04-24T06:56:08.518044+0000 (oldest deadline 2024-04-24T06:56:33.817722+0000) 2024-04-24T06:56:51.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:50 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:50.956+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:51.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:50 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:50.956+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:51.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:50 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:50.956+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:56:51.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:50 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:50.956+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:10.875826+0000 front 2024-04-24T06:56:10.875743+0000 (oldest deadline 2024-04-24T06:56:36.175699+0000) 2024-04-24T06:56:51.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:50 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:50.956+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:20.276664+0000 front 2024-04-24T06:56:20.276689+0000 (oldest deadline 2024-04-24T06:56:45.576810+0000) 2024-04-24T06:56:51.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:50 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:50.956+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:25.577454+0000 front 2024-04-24T06:56:25.577435+0000 (oldest deadline 2024-04-24T06:56:49.677340+0000) 2024-04-24T06:56:51.445 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:51 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:51.119+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:56.064734+0000 front 2024-04-24T06:55:56.064528+0000 (oldest deadline 2024-04-24T06:56:18.364453+0000) 2024-04-24T06:56:51.445 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:51 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:51.119+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.265368+0000 front 2024-04-24T06:56:01.265402+0000 (oldest deadline 2024-04-24T06:56:25.965496+0000) 2024-04-24T06:56:51.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:51 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:51.119+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.266701+0000 front 2024-04-24T06:56:08.266799+0000 (oldest deadline 2024-04-24T06:56:34.666972+0000) 2024-04-24T06:56:51.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:51 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:51.119+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:20.969376+0000 front 2024-04-24T06:56:20.969408+0000 (oldest deadline 2024-04-24T06:56:42.669350+0000) 2024-04-24T06:56:51.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:51 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:51.302+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:51.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:51 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:51.302+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:56:51.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:51 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:51.302+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:56:51.447 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:51 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:51.302+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:56:51.447 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:51 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:51.302+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:18.541399+0000 front 2024-04-24T06:56:18.541315+0000 (oldest deadline 2024-04-24T06:56:42.041243+0000) 2024-04-24T06:56:51.755 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:51 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:51.263+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:51.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:51 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:51.263+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:56:51.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:51 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:51.263+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.915218+0000 front 2024-04-24T06:56:03.914687+0000 (oldest deadline 2024-04-24T06:56:28.014915+0000) 2024-04-24T06:56:51.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:51 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:51.263+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.517851+0000 front 2024-04-24T06:56:08.518044+0000 (oldest deadline 2024-04-24T06:56:33.817722+0000) 2024-04-24T06:56:52.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:51 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:51.972+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:52.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:51 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:51.972+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:52.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:51 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:51.972+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:56:52.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:51 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:51.972+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:10.875826+0000 front 2024-04-24T06:56:10.875743+0000 (oldest deadline 2024-04-24T06:56:36.175699+0000) 2024-04-24T06:56:52.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:51 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:51.972+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:20.276664+0000 front 2024-04-24T06:56:20.276689+0000 (oldest deadline 2024-04-24T06:56:45.576810+0000) 2024-04-24T06:56:52.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:51 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:51.972+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:25.577454+0000 front 2024-04-24T06:56:25.577435+0000 (oldest deadline 2024-04-24T06:56:49.677340+0000) 2024-04-24T06:56:52.445 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:52 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:52.125+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:56.064734+0000 front 2024-04-24T06:55:56.064528+0000 (oldest deadline 2024-04-24T06:56:18.364453+0000) 2024-04-24T06:56:52.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:52 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:52.125+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.265368+0000 front 2024-04-24T06:56:01.265402+0000 (oldest deadline 2024-04-24T06:56:25.965496+0000) 2024-04-24T06:56:52.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:52 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:52.125+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.266701+0000 front 2024-04-24T06:56:08.266799+0000 (oldest deadline 2024-04-24T06:56:34.666972+0000) 2024-04-24T06:56:52.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:52 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:52.125+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:20.969376+0000 front 2024-04-24T06:56:20.969408+0000 (oldest deadline 2024-04-24T06:56:42.669350+0000) 2024-04-24T06:56:52.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:52 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:52.125+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:26.170259+0000 front 2024-04-24T06:56:26.170327+0000 (oldest deadline 2024-04-24T06:56:52.070412+0000) 2024-04-24T06:56:52.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:52 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:52.261+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:52.447 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:52 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:52.261+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:56:52.447 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:52 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:52.261+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:56:52.447 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:52 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:52.261+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:56:52.447 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:52 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:52.261+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:18.541399+0000 front 2024-04-24T06:56:18.541315+0000 (oldest deadline 2024-04-24T06:56:42.041243+0000) 2024-04-24T06:56:52.755 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:52 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:52.269+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:52.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:52 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:52.269+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:56:52.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:52 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:52.269+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.915218+0000 front 2024-04-24T06:56:03.914687+0000 (oldest deadline 2024-04-24T06:56:28.014915+0000) 2024-04-24T06:56:52.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:52 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:52.269+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.517851+0000 front 2024-04-24T06:56:08.518044+0000 (oldest deadline 2024-04-24T06:56:33.817722+0000) 2024-04-24T06:56:53.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:52 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:52.995+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:53.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:52 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:52.995+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:53.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:52 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:52.995+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:56:53.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:52 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:52.995+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:10.875826+0000 front 2024-04-24T06:56:10.875743+0000 (oldest deadline 2024-04-24T06:56:36.175699+0000) 2024-04-24T06:56:53.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:52 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:52.995+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:20.276664+0000 front 2024-04-24T06:56:20.276689+0000 (oldest deadline 2024-04-24T06:56:45.576810+0000) 2024-04-24T06:56:53.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:52 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:52.995+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:25.577454+0000 front 2024-04-24T06:56:25.577435+0000 (oldest deadline 2024-04-24T06:56:49.677340+0000) 2024-04-24T06:56:53.445 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:53 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:53.146+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:56.064734+0000 front 2024-04-24T06:55:56.064528+0000 (oldest deadline 2024-04-24T06:56:18.364453+0000) 2024-04-24T06:56:53.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:53 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:53.146+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.265368+0000 front 2024-04-24T06:56:01.265402+0000 (oldest deadline 2024-04-24T06:56:25.965496+0000) 2024-04-24T06:56:53.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:53 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:53.146+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.266701+0000 front 2024-04-24T06:56:08.266799+0000 (oldest deadline 2024-04-24T06:56:34.666972+0000) 2024-04-24T06:56:53.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:53 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:53.146+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:20.969376+0000 front 2024-04-24T06:56:20.969408+0000 (oldest deadline 2024-04-24T06:56:42.669350+0000) 2024-04-24T06:56:53.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:53 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:53.146+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:26.170259+0000 front 2024-04-24T06:56:26.170327+0000 (oldest deadline 2024-04-24T06:56:52.070412+0000) 2024-04-24T06:56:53.447 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:53 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:53.271+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:53.447 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:53 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:53.271+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:56:53.447 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:53 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:53.271+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:56:53.447 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:53 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:53.271+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:56:53.447 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:53 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:53.271+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:18.541399+0000 front 2024-04-24T06:56:18.541315+0000 (oldest deadline 2024-04-24T06:56:42.041243+0000) 2024-04-24T06:56:53.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:53 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:53.290+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:53.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:53 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:53.290+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:56:53.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:53 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:53.290+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.915218+0000 front 2024-04-24T06:56:03.914687+0000 (oldest deadline 2024-04-24T06:56:28.014915+0000) 2024-04-24T06:56:53.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:53 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:53.290+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.517851+0000 front 2024-04-24T06:56:08.518044+0000 (oldest deadline 2024-04-24T06:56:33.817722+0000) 2024-04-24T06:56:53.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:53 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:53.290+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:27.922412+0000 front 2024-04-24T06:56:27.922326+0000 (oldest deadline 2024-04-24T06:56:52.622583+0000) 2024-04-24T06:56:54.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:53 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:53.980+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:54.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:53 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:53.980+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:54.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:53 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:53.980+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:56:54.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:53 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:53.980+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:10.875826+0000 front 2024-04-24T06:56:10.875743+0000 (oldest deadline 2024-04-24T06:56:36.175699+0000) 2024-04-24T06:56:54.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:53 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:53.980+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:20.276664+0000 front 2024-04-24T06:56:20.276689+0000 (oldest deadline 2024-04-24T06:56:45.576810+0000) 2024-04-24T06:56:54.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:53 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:53.980+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:25.577454+0000 front 2024-04-24T06:56:25.577435+0000 (oldest deadline 2024-04-24T06:56:49.677340+0000) 2024-04-24T06:56:54.445 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:54 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:54.146+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:56.064734+0000 front 2024-04-24T06:55:56.064528+0000 (oldest deadline 2024-04-24T06:56:18.364453+0000) 2024-04-24T06:56:54.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:54 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:54.146+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:01.265368+0000 front 2024-04-24T06:56:01.265402+0000 (oldest deadline 2024-04-24T06:56:25.965496+0000) 2024-04-24T06:56:54.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:54 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:54.146+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.266701+0000 front 2024-04-24T06:56:08.266799+0000 (oldest deadline 2024-04-24T06:56:34.666972+0000) 2024-04-24T06:56:54.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:54 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:54.146+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:20.969376+0000 front 2024-04-24T06:56:20.969408+0000 (oldest deadline 2024-04-24T06:56:42.669350+0000) 2024-04-24T06:56:54.446 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:54 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8[51852]: 2024-04-24T06:56:54.146+0000 7f7e595af700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:26.170259+0000 front 2024-04-24T06:56:26.170327+0000 (oldest deadline 2024-04-24T06:56:52.070412+0000) 2024-04-24T06:56:54.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:54 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:54.319+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:54.447 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:54 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:54.319+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:56:54.447 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:54 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:54.319+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:56:54.447 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:54 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:54.319+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:56:54.447 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:54 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:54.319+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:18.541399+0000 front 2024-04-24T06:56:18.541315+0000 (oldest deadline 2024-04-24T06:56:42.041243+0000) 2024-04-24T06:56:54.447 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:54 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:54.319+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:28.144247+0000 front 2024-04-24T06:56:28.144309+0000 (oldest deadline 2024-04-24T06:56:54.044306+0000) 2024-04-24T06:56:54.755 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:54 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:54.272+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:54.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:54 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:54.272+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:56:54.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:54 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:54.272+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.915218+0000 front 2024-04-24T06:56:03.914687+0000 (oldest deadline 2024-04-24T06:56:28.014915+0000) 2024-04-24T06:56:54.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:54 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:54.272+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.517851+0000 front 2024-04-24T06:56:08.518044+0000 (oldest deadline 2024-04-24T06:56:33.817722+0000) 2024-04-24T06:56:54.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:54 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:54.272+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:27.922412+0000 front 2024-04-24T06:56:27.922326+0000 (oldest deadline 2024-04-24T06:56:52.622583+0000) 2024-04-24T06:56:55.016 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:54 smithi137 podman[70451]: 2024-04-24 06:56:54.746104933 +0000 UTC m=+5.615359066 container died 875574a3714de355f846ba67729b46e0cbd87e4c5875c7a801d70e377a443b0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-24T06:56:55.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:54 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:54.932+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:55.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:54 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:54.932+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:55.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:54 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:54.932+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:56:55.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:54 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:54.932+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:10.875826+0000 front 2024-04-24T06:56:10.875743+0000 (oldest deadline 2024-04-24T06:56:36.175699+0000) 2024-04-24T06:56:55.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:54 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:54.932+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:20.276664+0000 front 2024-04-24T06:56:20.276689+0000 (oldest deadline 2024-04-24T06:56:45.576810+0000) 2024-04-24T06:56:55.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:54 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:54.932+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:25.577454+0000 front 2024-04-24T06:56:25.577435+0000 (oldest deadline 2024-04-24T06:56:49.677340+0000) 2024-04-24T06:56:55.274 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:55 smithi137 podman[70451]: 2024-04-24 06:56:55.116261158 +0000 UTC m=+5.985515272 container cleanup 875574a3714de355f846ba67729b46e0cbd87e4c5875c7a801d70e377a443b0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, ceph=True) 2024-04-24T06:56:55.274 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:55 smithi137 bash[70451]: ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8 2024-04-24T06:56:55.677 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:55 smithi137 podman[70469]: 2024-04-24 06:56:55.359633543 +0000 UTC m=+0.610713609 container remove 875574a3714de355f846ba67729b46e0cbd87e4c5875c7a801d70e377a443b0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-24T06:56:55.678 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:55 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:55.273+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:55.678 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:55 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:55.273+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:56:55.678 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:55 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:55.273+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:56:55.678 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:55 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:55.273+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:56:55.678 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:55 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:55.273+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:18.541399+0000 front 2024-04-24T06:56:18.541315+0000 (oldest deadline 2024-04-24T06:56:42.041243+0000) 2024-04-24T06:56:55.678 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:55 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:55.273+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:28.144247+0000 front 2024-04-24T06:56:28.144309+0000 (oldest deadline 2024-04-24T06:56:54.044306+0000) 2024-04-24T06:56:55.755 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:55 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:55.306+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:55.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:55 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:55.306+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:56:55.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:55 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:55.306+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.915218+0000 front 2024-04-24T06:56:03.914687+0000 (oldest deadline 2024-04-24T06:56:28.014915+0000) 2024-04-24T06:56:55.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:55 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:55.306+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.517851+0000 front 2024-04-24T06:56:08.518044+0000 (oldest deadline 2024-04-24T06:56:33.817722+0000) 2024-04-24T06:56:55.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:55 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:55.306+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:27.922412+0000 front 2024-04-24T06:56:27.922326+0000 (oldest deadline 2024-04-24T06:56:52.622583+0000) 2024-04-24T06:56:55.945 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:55 smithi137 podman[70565]: 2024-04-24 06:56:55.677703849 +0000 UTC m=+0.113911435 container create a7e6d2f5b948be849796e1f7b34cd67c9a8708075689aab4a1f67f80f8dc0f70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-24T06:56:55.945 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:55 smithi137 podman[70565]: 2024-04-24 06:56:55.582315961 +0000 UTC m=+0.018523554 image pull b5c3183062a6ce19f361681cc58de6e3ac1de1a54c80e6094ec5695237cd98d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b 2024-04-24T06:56:55.946 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:55 smithi137 podman[70565]: 2024-04-24 06:56:55.896906423 +0000 UTC m=+0.333114018 container init a7e6d2f5b948be849796e1f7b34cd67c9a8708075689aab4a1f67f80f8dc0f70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.schema-version=1.0, ceph=True) 2024-04-24T06:56:55.946 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:55 smithi137 podman[70565]: 2024-04-24 06:56:55.899959941 +0000 UTC m=+0.336167529 container start a7e6d2f5b948be849796e1f7b34cd67c9a8708075689aab4a1f67f80f8dc0f70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True) 2024-04-24T06:56:56.213 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:55 smithi137 podman[70565]: 2024-04-24 06:56:55.971919787 +0000 UTC m=+0.408127393 container attach a7e6d2f5b948be849796e1f7b34cd67c9a8708075689aab4a1f67f80f8dc0f70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-24T06:56:56.214 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:56 smithi137 podman[70565]: 2024-04-24 06:56:56.19063181 +0000 UTC m=+0.626839398 container died a7e6d2f5b948be849796e1f7b34cd67c9a8708075689aab4a1f67f80f8dc0f70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-24T06:56:56.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:55 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:55.905+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:56.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:55 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:55.905+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:56.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:55 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:55.905+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:56:56.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:55 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:55.905+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:10.875826+0000 front 2024-04-24T06:56:10.875743+0000 (oldest deadline 2024-04-24T06:56:36.175699+0000) 2024-04-24T06:56:56.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:55 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:55.905+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:20.276664+0000 front 2024-04-24T06:56:20.276689+0000 (oldest deadline 2024-04-24T06:56:45.576810+0000) 2024-04-24T06:56:56.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:55 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:55.905+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:25.577454+0000 front 2024-04-24T06:56:25.577435+0000 (oldest deadline 2024-04-24T06:56:49.677340+0000) 2024-04-24T06:56:56.467 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:56 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:56.293+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:56.467 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:56 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:56.293+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:56:56.467 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:56 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:56.293+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:56:56.467 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:56 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:56.293+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:56:56.468 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:56 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:56.293+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:18.541399+0000 front 2024-04-24T06:56:18.541315+0000 (oldest deadline 2024-04-24T06:56:42.041243+0000) 2024-04-24T06:56:56.468 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:56 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:56.293+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:28.144247+0000 front 2024-04-24T06:56:28.144309+0000 (oldest deadline 2024-04-24T06:56:54.044306+0000) 2024-04-24T06:56:56.719 INFO:journalctl@ceph.osd.8.smithi137.stdout:Apr 24 06:56:56 smithi137 podman[70591]: 2024-04-24 06:56:56.717158089 +0000 UTC m=+0.521445797 container remove a7e6d2f5b948be849796e1f7b34cd67c9a8708075689aab4a1f67f80f8dc0f70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-24T06:56:56.734 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@osd.8.service' 2024-04-24T06:56:56.755 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:56 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:56.354+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:56.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:56 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:56.354+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:56:56.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:56 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:56.354+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.915218+0000 front 2024-04-24T06:56:03.914687+0000 (oldest deadline 2024-04-24T06:56:28.014915+0000) 2024-04-24T06:56:56.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:56 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:56.354+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.517851+0000 front 2024-04-24T06:56:08.518044+0000 (oldest deadline 2024-04-24T06:56:33.817722+0000) 2024-04-24T06:56:56.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:56 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:56.354+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:27.922412+0000 front 2024-04-24T06:56:27.922326+0000 (oldest deadline 2024-04-24T06:56:52.622583+0000) 2024-04-24T06:56:56.771 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T06:56:56.771 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-24T06:56:56.772 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-24T06:56:56.772 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-43e9bad0-0204-11ef-bc93-c7b262605968@osd.9 2024-04-24T06:56:57.255 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:56 smithi005 systemd[1]: Stopping Ceph osd.9 for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:56:57.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:56 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:56.860+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:57.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:56 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:56.860+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:57.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:56 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:56.860+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:56:57.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:56 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:56.860+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:10.875826+0000 front 2024-04-24T06:56:10.875743+0000 (oldest deadline 2024-04-24T06:56:36.175699+0000) 2024-04-24T06:56:57.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:56 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:56.860+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:20.276664+0000 front 2024-04-24T06:56:20.276689+0000 (oldest deadline 2024-04-24T06:56:45.576810+0000) 2024-04-24T06:56:57.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:56 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:56.860+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:25.577454+0000 front 2024-04-24T06:56:25.577435+0000 (oldest deadline 2024-04-24T06:56:49.677340+0000) 2024-04-24T06:56:57.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:57 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:57.262+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:57.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:57 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:57.262+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:56:57.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:57 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:57.262+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:56:57.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:57 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:57.262+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:56:57.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:57 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:57.262+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:18.541399+0000 front 2024-04-24T06:56:18.541315+0000 (oldest deadline 2024-04-24T06:56:42.041243+0000) 2024-04-24T06:56:57.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:57 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:57.262+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:28.144247+0000 front 2024-04-24T06:56:28.144309+0000 (oldest deadline 2024-04-24T06:56:54.044306+0000) 2024-04-24T06:56:57.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:57 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:57.351+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:57.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:57 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:57.351+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:56:57.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:57 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:57.351+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.915218+0000 front 2024-04-24T06:56:03.914687+0000 (oldest deadline 2024-04-24T06:56:28.014915+0000) 2024-04-24T06:56:57.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:57 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:57.351+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.517851+0000 front 2024-04-24T06:56:08.518044+0000 (oldest deadline 2024-04-24T06:56:33.817722+0000) 2024-04-24T06:56:57.757 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:57 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:57.351+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:27.922412+0000 front 2024-04-24T06:56:27.922326+0000 (oldest deadline 2024-04-24T06:56:52.622583+0000) 2024-04-24T06:56:57.757 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:57 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:57.389+0000 7f711f200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T06:56:57.757 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:57 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:57.389+0000 7f711f200700 -1 osd.9 102 *** Got signal Terminated *** 2024-04-24T06:56:57.757 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:57 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:57.389+0000 7f711f200700 -1 osd.9 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T06:56:58.136 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:57 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:57.867+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:58.137 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:57 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:57.867+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:58.137 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:57 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:57.867+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:56:58.137 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:57 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:57.867+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:10.875826+0000 front 2024-04-24T06:56:10.875743+0000 (oldest deadline 2024-04-24T06:56:36.175699+0000) 2024-04-24T06:56:58.137 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:57 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:57.867+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:20.276664+0000 front 2024-04-24T06:56:20.276689+0000 (oldest deadline 2024-04-24T06:56:45.576810+0000) 2024-04-24T06:56:58.137 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:57 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:57.867+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:25.577454+0000 front 2024-04-24T06:56:25.577435+0000 (oldest deadline 2024-04-24T06:56:49.677340+0000) 2024-04-24T06:56:58.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:58 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:58.257+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:58.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:58 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:58.257+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:56:58.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:58 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:58.257+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:56:58.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:58 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:58.257+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:56:58.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:58 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:58.257+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:18.541399+0000 front 2024-04-24T06:56:18.541315+0000 (oldest deadline 2024-04-24T06:56:42.041243+0000) 2024-04-24T06:56:58.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:58 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:58.257+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:28.144247+0000 front 2024-04-24T06:56:28.144309+0000 (oldest deadline 2024-04-24T06:56:54.044306+0000) 2024-04-24T06:56:58.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:58 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:58.359+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:58.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:58 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:58.359+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:56:58.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:58 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:58.359+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.915218+0000 front 2024-04-24T06:56:03.914687+0000 (oldest deadline 2024-04-24T06:56:28.014915+0000) 2024-04-24T06:56:58.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:58 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:58.359+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.517851+0000 front 2024-04-24T06:56:08.518044+0000 (oldest deadline 2024-04-24T06:56:33.817722+0000) 2024-04-24T06:56:58.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:58 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:58.359+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:27.922412+0000 front 2024-04-24T06:56:27.922326+0000 (oldest deadline 2024-04-24T06:56:52.622583+0000) 2024-04-24T06:56:58.757 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:58 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:58.359+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-04-24T06:56:34.924411+0000 front 2024-04-24T06:56:34.924768+0000 (oldest deadline 2024-04-24T06:56:57.824258+0000) 2024-04-24T06:56:59.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:58 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:58.892+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:56:59.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:58 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:58.893+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:56:59.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:58 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:58.893+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:56:59.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:58 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:58.893+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:10.875826+0000 front 2024-04-24T06:56:10.875743+0000 (oldest deadline 2024-04-24T06:56:36.175699+0000) 2024-04-24T06:56:59.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:58 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:58.893+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:20.276664+0000 front 2024-04-24T06:56:20.276689+0000 (oldest deadline 2024-04-24T06:56:45.576810+0000) 2024-04-24T06:56:59.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:58 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:58.893+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:25.577454+0000 front 2024-04-24T06:56:25.577435+0000 (oldest deadline 2024-04-24T06:56:49.677340+0000) 2024-04-24T06:56:59.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:59 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:59.399+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:56:59.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:59 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:59.399+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:56:59.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:59 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:59.399+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.915218+0000 front 2024-04-24T06:56:03.914687+0000 (oldest deadline 2024-04-24T06:56:28.014915+0000) 2024-04-24T06:56:59.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:59 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:59.399+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.517851+0000 front 2024-04-24T06:56:08.518044+0000 (oldest deadline 2024-04-24T06:56:33.817722+0000) 2024-04-24T06:56:59.506 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:59 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:59.399+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:27.922412+0000 front 2024-04-24T06:56:27.922326+0000 (oldest deadline 2024-04-24T06:56:52.622583+0000) 2024-04-24T06:56:59.507 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:56:59 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:56:59.399+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-04-24T06:56:34.924411+0000 front 2024-04-24T06:56:34.924768+0000 (oldest deadline 2024-04-24T06:56:57.824258+0000) 2024-04-24T06:56:59.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:59 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:59.302+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:56:59.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:59 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:59.302+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:56:59.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:59 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:59.302+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:56:59.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:59 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:59.302+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:56:59.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:59 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:59.302+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:18.541399+0000 front 2024-04-24T06:56:18.541315+0000 (oldest deadline 2024-04-24T06:56:42.041243+0000) 2024-04-24T06:56:59.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:56:59 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:56:59.302+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:28.144247+0000 front 2024-04-24T06:56:28.144309+0000 (oldest deadline 2024-04-24T06:56:54.044306+0000) 2024-04-24T06:57:00.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:59 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:59.937+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:57:00.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:59 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:59.937+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:57:00.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:59 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:59.937+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:57:00.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:59 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:59.937+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:10.875826+0000 front 2024-04-24T06:56:10.875743+0000 (oldest deadline 2024-04-24T06:56:36.175699+0000) 2024-04-24T06:57:00.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:59 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:59.937+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:20.276664+0000 front 2024-04-24T06:56:20.276689+0000 (oldest deadline 2024-04-24T06:56:45.576810+0000) 2024-04-24T06:57:00.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:56:59 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:56:59.937+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:25.577454+0000 front 2024-04-24T06:56:25.577435+0000 (oldest deadline 2024-04-24T06:56:49.677340+0000) 2024-04-24T06:57:00.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:00 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:00.284+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:57:00.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:00 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:00.284+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:57:00.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:00 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:00.284+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:57:00.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:00 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:00.284+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:57:00.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:00 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:00.284+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:18.541399+0000 front 2024-04-24T06:56:18.541315+0000 (oldest deadline 2024-04-24T06:56:42.041243+0000) 2024-04-24T06:57:00.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:00 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:00.284+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:28.144247+0000 front 2024-04-24T06:56:28.144309+0000 (oldest deadline 2024-04-24T06:56:54.044306+0000) 2024-04-24T06:57:00.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:57:00 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:57:00.429+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:57:00.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:57:00 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:57:00.429+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:57:00.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:57:00 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:57:00.429+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.915218+0000 front 2024-04-24T06:56:03.914687+0000 (oldest deadline 2024-04-24T06:56:28.014915+0000) 2024-04-24T06:57:00.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:57:00 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:57:00.429+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.517851+0000 front 2024-04-24T06:56:08.518044+0000 (oldest deadline 2024-04-24T06:56:33.817722+0000) 2024-04-24T06:57:00.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:57:00 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:57:00.429+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:27.922412+0000 front 2024-04-24T06:56:27.922326+0000 (oldest deadline 2024-04-24T06:56:52.622583+0000) 2024-04-24T06:57:00.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:57:00 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:57:00.429+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-04-24T06:56:34.924411+0000 front 2024-04-24T06:56:34.924768+0000 (oldest deadline 2024-04-24T06:56:57.824258+0000) 2024-04-24T06:57:01.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:00 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:00.901+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:57:01.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:00 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:00.901+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:57:01.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:00 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:00.901+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:57:01.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:00 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:00.901+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:10.875826+0000 front 2024-04-24T06:56:10.875743+0000 (oldest deadline 2024-04-24T06:56:36.175699+0000) 2024-04-24T06:57:01.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:00 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:00.901+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:20.276664+0000 front 2024-04-24T06:56:20.276689+0000 (oldest deadline 2024-04-24T06:56:45.576810+0000) 2024-04-24T06:57:01.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:00 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:00.901+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:25.577454+0000 front 2024-04-24T06:56:25.577435+0000 (oldest deadline 2024-04-24T06:56:49.677340+0000) 2024-04-24T06:57:01.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:00 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:00.901+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-04-24T06:56:36.679058+0000 front 2024-04-24T06:56:36.679157+0000 (oldest deadline 2024-04-24T06:57:00.778817+0000) 2024-04-24T06:57:01.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:01 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:01.244+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:57:01.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:01 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:01.244+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:57:01.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:01 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:01.244+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:57:01.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:01 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:01.244+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:57:01.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:01 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:01.244+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:18.541399+0000 front 2024-04-24T06:56:18.541315+0000 (oldest deadline 2024-04-24T06:56:42.041243+0000) 2024-04-24T06:57:01.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:01 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:01.244+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:28.144247+0000 front 2024-04-24T06:56:28.144309+0000 (oldest deadline 2024-04-24T06:56:54.044306+0000) 2024-04-24T06:57:01.755 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:57:01 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:57:01.433+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:47.206979+0000 front 2024-04-24T06:55:47.206915+0000 (oldest deadline 2024-04-24T06:56:08.906877+0000) 2024-04-24T06:57:01.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:57:01 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:57:01.433+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:53.410691+0000 front 2024-04-24T06:55:53.410575+0000 (oldest deadline 2024-04-24T06:56:18.110770+0000) 2024-04-24T06:57:01.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:57:01 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:57:01.433+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:03.915218+0000 front 2024-04-24T06:56:03.914687+0000 (oldest deadline 2024-04-24T06:56:28.014915+0000) 2024-04-24T06:57:01.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:57:01 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:57:01.433+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:08.517851+0000 front 2024-04-24T06:56:08.518044+0000 (oldest deadline 2024-04-24T06:56:33.817722+0000) 2024-04-24T06:57:01.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:57:01 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:57:01.433+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:27.922412+0000 front 2024-04-24T06:56:27.922326+0000 (oldest deadline 2024-04-24T06:56:52.622583+0000) 2024-04-24T06:57:01.756 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:57:01 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9[59478]: 2024-04-24T06:57:01.433+0000 7f7115a00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-04-24T06:56:34.924411+0000 front 2024-04-24T06:56:34.924768+0000 (oldest deadline 2024-04-24T06:56:57.824258+0000) 2024-04-24T06:57:02.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:01 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:01.935+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:57:02.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:01 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:01.935+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:57:02.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:01 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:01.935+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:57:02.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:01 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:01.935+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:10.875826+0000 front 2024-04-24T06:56:10.875743+0000 (oldest deadline 2024-04-24T06:56:36.175699+0000) 2024-04-24T06:57:02.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:01 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:01.935+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:20.276664+0000 front 2024-04-24T06:56:20.276689+0000 (oldest deadline 2024-04-24T06:56:45.576810+0000) 2024-04-24T06:57:02.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:01 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:01.935+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:25.577454+0000 front 2024-04-24T06:56:25.577435+0000 (oldest deadline 2024-04-24T06:56:49.677340+0000) 2024-04-24T06:57:02.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:01 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:01.935+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-04-24T06:56:36.679058+0000 front 2024-04-24T06:56:36.679157+0000 (oldest deadline 2024-04-24T06:57:00.778817+0000) 2024-04-24T06:57:02.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:02 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:02.198+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:57:02.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:02 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:02.198+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:57:02.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:02 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:02.198+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:57:02.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:02 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:02.198+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:57:02.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:02 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:02.198+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:18.541399+0000 front 2024-04-24T06:56:18.541315+0000 (oldest deadline 2024-04-24T06:56:42.041243+0000) 2024-04-24T06:57:02.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:02 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:02.198+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:28.144247+0000 front 2024-04-24T06:56:28.144309+0000 (oldest deadline 2024-04-24T06:56:54.044306+0000) 2024-04-24T06:57:03.128 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:57:02 smithi005 podman[98040]: 2024-04-24 06:57:02.849185943 +0000 UTC m=+5.619792228 container died fb4abfa84440083b3d7147ca5eb9e82c6a9a0d4dd777f6288b9baad622397e52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-24T06:57:03.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:02 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:02.908+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:57:03.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:02 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:02.908+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:57:03.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:02 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:02.908+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:57:03.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:02 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:02.908+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:10.875826+0000 front 2024-04-24T06:56:10.875743+0000 (oldest deadline 2024-04-24T06:56:36.175699+0000) 2024-04-24T06:57:03.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:02 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:02.908+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:20.276664+0000 front 2024-04-24T06:56:20.276689+0000 (oldest deadline 2024-04-24T06:56:45.576810+0000) 2024-04-24T06:57:03.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:02 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:02.908+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:25.577454+0000 front 2024-04-24T06:56:25.577435+0000 (oldest deadline 2024-04-24T06:56:49.677340+0000) 2024-04-24T06:57:03.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:02 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:02.908+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-04-24T06:56:36.679058+0000 front 2024-04-24T06:56:36.679157+0000 (oldest deadline 2024-04-24T06:57:00.778817+0000) 2024-04-24T06:57:03.458 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:57:03 smithi005 podman[98040]: 2024-04-24 06:57:03.281363687 +0000 UTC m=+6.051970002 container cleanup fb4abfa84440083b3d7147ca5eb9e82c6a9a0d4dd777f6288b9baad622397e52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True) 2024-04-24T06:57:03.458 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:57:03 smithi005 bash[98040]: ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9 2024-04-24T06:57:03.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:03 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:03.227+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:57:03.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:03 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:03.227+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:57:03.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:03 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:03.227+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:57:03.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:03 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:03.227+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:57:03.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:03 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:03.227+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:18.541399+0000 front 2024-04-24T06:56:18.541315+0000 (oldest deadline 2024-04-24T06:56:42.041243+0000) 2024-04-24T06:57:03.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:03 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:03.227+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:28.144247+0000 front 2024-04-24T06:56:28.144309+0000 (oldest deadline 2024-04-24T06:56:54.044306+0000) 2024-04-24T06:57:03.755 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:57:03 smithi005 podman[98058]: 2024-04-24 06:57:03.457269848 +0000 UTC m=+0.590072524 container remove fb4abfa84440083b3d7147ca5eb9e82c6a9a0d4dd777f6288b9baad622397e52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-24T06:57:04.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:03 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:03.919+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:57:04.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:03 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:03.919+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:57:04.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:03 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:03.919+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:57:04.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:03 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:03.919+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:10.875826+0000 front 2024-04-24T06:56:10.875743+0000 (oldest deadline 2024-04-24T06:56:36.175699+0000) 2024-04-24T06:57:04.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:03 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:03.919+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:20.276664+0000 front 2024-04-24T06:56:20.276689+0000 (oldest deadline 2024-04-24T06:56:45.576810+0000) 2024-04-24T06:57:04.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:03 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:03.919+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:25.577454+0000 front 2024-04-24T06:56:25.577435+0000 (oldest deadline 2024-04-24T06:56:49.677340+0000) 2024-04-24T06:57:04.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:03 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:03.919+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-04-24T06:56:36.679058+0000 front 2024-04-24T06:56:36.679157+0000 (oldest deadline 2024-04-24T06:57:00.778817+0000) 2024-04-24T06:57:04.562 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:57:04 smithi005 podman[98146]: 2024-04-24 06:57:04.212604204 +0000 UTC m=+0.037142560 image pull b5c3183062a6ce19f361681cc58de6e3ac1de1a54c80e6094ec5695237cd98d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b 2024-04-24T06:57:04.562 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:57:04 smithi005 podman[98146]: 2024-04-24 06:57:04.308040548 +0000 UTC m=+0.132578930 container create 3843abe4b64103306b0d7481af1913e1cc6fa4300bddbb85589c6d43e480d169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416) 2024-04-24T06:57:04.562 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:57:04 smithi005 podman[98146]: 2024-04-24 06:57:04.561345971 +0000 UTC m=+0.385884365 container init 3843abe4b64103306b0d7481af1913e1cc6fa4300bddbb85589c6d43e480d169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-24T06:57:04.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:04 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:04.269+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:57:04.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:04 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:04.269+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:57:04.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:04 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:04.269+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:57:04.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:04 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:04.269+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:57:04.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:04 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:04.269+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:18.541399+0000 front 2024-04-24T06:56:18.541315+0000 (oldest deadline 2024-04-24T06:56:42.041243+0000) 2024-04-24T06:57:04.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:04 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:04.269+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:28.144247+0000 front 2024-04-24T06:56:28.144309+0000 (oldest deadline 2024-04-24T06:56:54.044306+0000) 2024-04-24T06:57:05.005 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:57:04 smithi005 podman[98146]: 2024-04-24 06:57:04.574093482 +0000 UTC m=+0.398631851 container start 3843abe4b64103306b0d7481af1913e1cc6fa4300bddbb85589c6d43e480d169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416) 2024-04-24T06:57:05.006 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:57:04 smithi005 podman[98146]: 2024-04-24 06:57:04.643441432 +0000 UTC m=+0.467979802 container attach 3843abe4b64103306b0d7481af1913e1cc6fa4300bddbb85589c6d43e480d169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416) 2024-04-24T06:57:05.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:04 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:04.885+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:57:05.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:04 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:04.885+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:57:05.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:04 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:04.885+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:57:05.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:04 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:04.885+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:10.875826+0000 front 2024-04-24T06:56:10.875743+0000 (oldest deadline 2024-04-24T06:56:36.175699+0000) 2024-04-24T06:57:05.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:04 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:04.885+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:20.276664+0000 front 2024-04-24T06:56:20.276689+0000 (oldest deadline 2024-04-24T06:56:45.576810+0000) 2024-04-24T06:57:05.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:04 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:04.885+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:25.577454+0000 front 2024-04-24T06:56:25.577435+0000 (oldest deadline 2024-04-24T06:56:49.677340+0000) 2024-04-24T06:57:05.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:04 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:04.885+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-04-24T06:56:36.679058+0000 front 2024-04-24T06:56:36.679157+0000 (oldest deadline 2024-04-24T06:57:00.778817+0000) 2024-04-24T06:57:05.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Apr 24 06:57:05 smithi005 podman[98146]: 2024-04-24 06:57:05.043708471 +0000 UTC m=+0.868246909 container died 3843abe4b64103306b0d7481af1913e1cc6fa4300bddbb85589c6d43e480d169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_CLEAN=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-24T06:57:05.680 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@osd.9.service' 2024-04-24T06:57:05.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:05 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:05.224+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:57:05.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:05 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:05.224+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:57:05.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:05 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:05.224+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:57:05.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:05 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:05.224+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:57:05.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:05 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:05.224+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:18.541399+0000 front 2024-04-24T06:56:18.541315+0000 (oldest deadline 2024-04-24T06:56:42.041243+0000) 2024-04-24T06:57:05.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:05 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:05.224+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:28.144247+0000 front 2024-04-24T06:56:28.144309+0000 (oldest deadline 2024-04-24T06:56:54.044306+0000) 2024-04-24T06:57:05.780 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T06:57:05.780 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-24T06:57:05.781 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-24T06:57:05.781 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-43e9bad0-0204-11ef-bc93-c7b262605968@osd.10 2024-04-24T06:57:06.089 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:05 smithi037 systemd[1]: Stopping Ceph osd.10 for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:57:06.089 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:05 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:05.854+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:57:06.089 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:05 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:05.854+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:57:06.089 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:05 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:05.854+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:57:06.089 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:05 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:05.854+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:10.875826+0000 front 2024-04-24T06:56:10.875743+0000 (oldest deadline 2024-04-24T06:56:36.175699+0000) 2024-04-24T06:57:06.090 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:05 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:05.854+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:20.276664+0000 front 2024-04-24T06:56:20.276689+0000 (oldest deadline 2024-04-24T06:56:45.576810+0000) 2024-04-24T06:57:06.090 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:05 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:05.854+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:25.577454+0000 front 2024-04-24T06:56:25.577435+0000 (oldest deadline 2024-04-24T06:56:49.677340+0000) 2024-04-24T06:57:06.090 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:05 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:05.854+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-04-24T06:56:36.679058+0000 front 2024-04-24T06:56:36.679157+0000 (oldest deadline 2024-04-24T06:57:00.778817+0000) 2024-04-24T06:57:06.090 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:06 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:06.088+0000 7fdbff70e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T06:57:06.090 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:06 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:06.088+0000 7fdbff70e700 -1 osd.10 102 *** Got signal Terminated *** 2024-04-24T06:57:06.090 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:06 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:06.088+0000 7fdbff70e700 -1 osd.10 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T06:57:06.445 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:06 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:06.181+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:57:06.445 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:06 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:06.181+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:57:06.445 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:06 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:06.181+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:57:06.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:06 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:06.181+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:57:06.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:06 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:06.181+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:18.541399+0000 front 2024-04-24T06:56:18.541315+0000 (oldest deadline 2024-04-24T06:56:42.041243+0000) 2024-04-24T06:57:06.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:06 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:06.181+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:28.144247+0000 front 2024-04-24T06:56:28.144309+0000 (oldest deadline 2024-04-24T06:56:54.044306+0000) 2024-04-24T06:57:07.195 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:07 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:07.154+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:57:07.195 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:07 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:07.154+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:57:07.195 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:07 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:07.154+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:57:07.196 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:07 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:07.154+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:57:07.196 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:07 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:07.154+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:18.541399+0000 front 2024-04-24T06:56:18.541315+0000 (oldest deadline 2024-04-24T06:56:42.041243+0000) 2024-04-24T06:57:07.196 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:07 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:07.154+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:28.144247+0000 front 2024-04-24T06:56:28.144309+0000 (oldest deadline 2024-04-24T06:56:54.044306+0000) 2024-04-24T06:57:07.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:06 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:06.880+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:57:07.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:06 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:06.880+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:57:07.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:06 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:06.880+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:57:07.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:06 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:06.880+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:10.875826+0000 front 2024-04-24T06:56:10.875743+0000 (oldest deadline 2024-04-24T06:56:36.175699+0000) 2024-04-24T06:57:07.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:06 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:06.880+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:20.276664+0000 front 2024-04-24T06:56:20.276689+0000 (oldest deadline 2024-04-24T06:56:45.576810+0000) 2024-04-24T06:57:07.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:06 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:06.880+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:25.577454+0000 front 2024-04-24T06:56:25.577435+0000 (oldest deadline 2024-04-24T06:56:49.677340+0000) 2024-04-24T06:57:07.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:06 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:06.880+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-04-24T06:56:36.679058+0000 front 2024-04-24T06:56:36.679157+0000 (oldest deadline 2024-04-24T06:57:00.778817+0000) 2024-04-24T06:57:08.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:07 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:07.865+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:57:08.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:07 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:07.865+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:57:08.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:07 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:07.865+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:57:08.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:07 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:07.865+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:10.875826+0000 front 2024-04-24T06:56:10.875743+0000 (oldest deadline 2024-04-24T06:56:36.175699+0000) 2024-04-24T06:57:08.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:07 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:07.865+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:20.276664+0000 front 2024-04-24T06:56:20.276689+0000 (oldest deadline 2024-04-24T06:56:45.576810+0000) 2024-04-24T06:57:08.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:07 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:07.865+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:25.577454+0000 front 2024-04-24T06:56:25.577435+0000 (oldest deadline 2024-04-24T06:56:49.677340+0000) 2024-04-24T06:57:08.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:07 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:07.865+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-04-24T06:56:36.679058+0000 front 2024-04-24T06:56:36.679157+0000 (oldest deadline 2024-04-24T06:57:00.778817+0000) 2024-04-24T06:57:08.445 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:08 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:08.127+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:57:08.445 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:08 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:08.127+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:57:08.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:08 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:08.127+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:57:08.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:08 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:08.127+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:57:08.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:08 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:08.127+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:18.541399+0000 front 2024-04-24T06:56:18.541315+0000 (oldest deadline 2024-04-24T06:56:42.041243+0000) 2024-04-24T06:57:08.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:08 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:08.127+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:28.144247+0000 front 2024-04-24T06:56:28.144309+0000 (oldest deadline 2024-04-24T06:56:54.044306+0000) 2024-04-24T06:57:09.169 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:08 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:08.879+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:57:09.169 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:08 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:08.879+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:57:09.170 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:08 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:08.879+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:57:09.170 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:08 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:08.879+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:10.875826+0000 front 2024-04-24T06:56:10.875743+0000 (oldest deadline 2024-04-24T06:56:36.175699+0000) 2024-04-24T06:57:09.170 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:08 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:08.879+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:20.276664+0000 front 2024-04-24T06:56:20.276689+0000 (oldest deadline 2024-04-24T06:56:45.576810+0000) 2024-04-24T06:57:09.170 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:08 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:08.879+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:25.577454+0000 front 2024-04-24T06:56:25.577435+0000 (oldest deadline 2024-04-24T06:56:49.677340+0000) 2024-04-24T06:57:09.170 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:08 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:08.879+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-04-24T06:56:36.679058+0000 front 2024-04-24T06:56:36.679157+0000 (oldest deadline 2024-04-24T06:57:00.778817+0000) 2024-04-24T06:57:09.445 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:09 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:09.144+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:57:09.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:09 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:09.144+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:57:09.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:09 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:09.144+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:57:09.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:09 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:09.144+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:57:09.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:09 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:09.144+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:18.541399+0000 front 2024-04-24T06:56:18.541315+0000 (oldest deadline 2024-04-24T06:56:42.041243+0000) 2024-04-24T06:57:09.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:09 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:09.144+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:28.144247+0000 front 2024-04-24T06:56:28.144309+0000 (oldest deadline 2024-04-24T06:56:54.044306+0000) 2024-04-24T06:57:10.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:09 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:09.921+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:57:10.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:09 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:09.921+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:57:10.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:09 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:09.921+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:57:10.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:09 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:09.921+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:10.875826+0000 front 2024-04-24T06:56:10.875743+0000 (oldest deadline 2024-04-24T06:56:36.175699+0000) 2024-04-24T06:57:10.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:09 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:09.921+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:20.276664+0000 front 2024-04-24T06:56:20.276689+0000 (oldest deadline 2024-04-24T06:56:45.576810+0000) 2024-04-24T06:57:10.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:09 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:09.921+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:25.577454+0000 front 2024-04-24T06:56:25.577435+0000 (oldest deadline 2024-04-24T06:56:49.677340+0000) 2024-04-24T06:57:10.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:09 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:09.921+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-04-24T06:56:36.679058+0000 front 2024-04-24T06:56:36.679157+0000 (oldest deadline 2024-04-24T06:57:00.778817+0000) 2024-04-24T06:57:10.445 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:10 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:10.156+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:57:10.445 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:10 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:10.156+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:57:10.445 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:10 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:10.156+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:57:10.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:10 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:10.156+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:57:10.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:10 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:10.156+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:18.541399+0000 front 2024-04-24T06:56:18.541315+0000 (oldest deadline 2024-04-24T06:56:42.041243+0000) 2024-04-24T06:57:10.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:10 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:10.156+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:28.144247+0000 front 2024-04-24T06:56:28.144309+0000 (oldest deadline 2024-04-24T06:56:54.044306+0000) 2024-04-24T06:57:10.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:10 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:10.156+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6820 osd.7 since back 2024-04-24T06:56:44.446303+0000 front 2024-04-24T06:56:44.446374+0000 (oldest deadline 2024-04-24T06:57:09.746300+0000) 2024-04-24T06:57:11.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:10 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:10.933+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:42.470421+0000 front 2024-04-24T06:55:42.470360+0000 (oldest deadline 2024-04-24T06:56:08.369880+0000) 2024-04-24T06:57:11.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:10 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:10.933+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:51.672192+0000 front 2024-04-24T06:55:51.672349+0000 (oldest deadline 2024-04-24T06:56:16.972288+0000) 2024-04-24T06:57:11.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:10 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:10.933+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:02.773479+0000 front 2024-04-24T06:56:02.773494+0000 (oldest deadline 2024-04-24T06:56:25.673653+0000) 2024-04-24T06:57:11.268 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:10 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:10.933+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:10.875826+0000 front 2024-04-24T06:56:10.875743+0000 (oldest deadline 2024-04-24T06:56:36.175699+0000) 2024-04-24T06:57:11.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:10 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:10.933+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:20.276664+0000 front 2024-04-24T06:56:20.276689+0000 (oldest deadline 2024-04-24T06:56:45.576810+0000) 2024-04-24T06:57:11.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:10 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:10.933+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:25.577454+0000 front 2024-04-24T06:56:25.577435+0000 (oldest deadline 2024-04-24T06:56:49.677340+0000) 2024-04-24T06:57:11.269 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:10 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10[57971]: 2024-04-24T06:57:10.933+0000 7fdbfbd02700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-04-24T06:56:36.679058+0000 front 2024-04-24T06:56:36.679157+0000 (oldest deadline 2024-04-24T06:57:00.778817+0000) 2024-04-24T06:57:11.445 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:11 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:11.147+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:57:11.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:11 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:11.147+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:57:11.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:11 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:11.147+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:57:11.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:11 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:11.147+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:57:11.447 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:11 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:11.147+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:18.541399+0000 front 2024-04-24T06:56:18.541315+0000 (oldest deadline 2024-04-24T06:56:42.041243+0000) 2024-04-24T06:57:11.447 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:11 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:11.147+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:28.144247+0000 front 2024-04-24T06:56:28.144309+0000 (oldest deadline 2024-04-24T06:56:54.044306+0000) 2024-04-24T06:57:11.447 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:11 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:11.147+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6820 osd.7 since back 2024-04-24T06:56:44.446303+0000 front 2024-04-24T06:56:44.446374+0000 (oldest deadline 2024-04-24T06:57:09.746300+0000) 2024-04-24T06:57:11.755 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:11 smithi037 podman[69753]: 2024-04-24 06:57:11.492572305 +0000 UTC m=+5.517368678 container died 4321db2367ce9dfd1d4989461c52fc47410403156837ed22cdc681b2aeda2163 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T06:57:12.018 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:11 smithi037 podman[69753]: 2024-04-24 06:57:11.854912509 +0000 UTC m=+5.879708883 container cleanup 4321db2367ce9dfd1d4989461c52fc47410403156837ed22cdc681b2aeda2163 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-24T06:57:12.018 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:11 smithi037 bash[69753]: ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10 2024-04-24T06:57:12.445 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:12 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:12.099+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:57:12.445 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:12 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:12.099+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:57:12.445 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:12 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:12.099+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:57:12.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:12 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:12.099+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:57:12.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:12 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:12.099+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:18.541399+0000 front 2024-04-24T06:56:18.541315+0000 (oldest deadline 2024-04-24T06:56:42.041243+0000) 2024-04-24T06:57:12.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:12 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:12.099+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:28.144247+0000 front 2024-04-24T06:56:28.144309+0000 (oldest deadline 2024-04-24T06:56:54.044306+0000) 2024-04-24T06:57:12.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:12 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:12.099+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6820 osd.7 since back 2024-04-24T06:56:44.446303+0000 front 2024-04-24T06:56:44.446374+0000 (oldest deadline 2024-04-24T06:57:09.746300+0000) 2024-04-24T06:57:12.481 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:12 smithi037 podman[69782]: 2024-04-24 06:57:12.12274336 +0000 UTC m=+0.623769527 container remove 4321db2367ce9dfd1d4989461c52fc47410403156837ed22cdc681b2aeda2163 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True) 2024-04-24T06:57:12.768 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:12 smithi037 podman[69873]: 2024-04-24 06:57:12.481923995 +0000 UTC m=+0.113635717 container create 7b12d5180d38d3663c476b580c85dd56c9778889b3533617d7bf58112de90bd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True) 2024-04-24T06:57:12.768 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:12 smithi037 podman[69873]: 2024-04-24 06:57:12.386550188 +0000 UTC m=+0.018261864 image pull b5c3183062a6ce19f361681cc58de6e3ac1de1a54c80e6094ec5695237cd98d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b 2024-04-24T06:57:12.768 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:12 smithi037 podman[69873]: 2024-04-24 06:57:12.684768483 +0000 UTC m=+0.316480160 container init 7b12d5180d38d3663c476b580c85dd56c9778889b3533617d7bf58112de90bd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-24T06:57:12.768 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:12 smithi037 podman[69873]: 2024-04-24 06:57:12.688105487 +0000 UTC m=+0.319817167 container start 7b12d5180d38d3663c476b580c85dd56c9778889b3533617d7bf58112de90bd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-24T06:57:12.768 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:12 smithi037 podman[69873]: 2024-04-24 06:57:12.767459591 +0000 UTC m=+0.399171278 container attach 7b12d5180d38d3663c476b580c85dd56c9778889b3533617d7bf58112de90bd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-24T06:57:13.017 INFO:journalctl@ceph.osd.10.smithi037.stdout:Apr 24 06:57:12 smithi037 podman[69873]: 2024-04-24 06:57:12.988774184 +0000 UTC m=+0.620485864 container died 7b12d5180d38d3663c476b580c85dd56c9778889b3533617d7bf58112de90bd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-24T06:57:13.445 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:13 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:13.135+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:57:13.445 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:13 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:13.135+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:57:13.445 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:13 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:13.135+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:57:13.445 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:13 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:13.135+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:57:13.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:13 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:13.135+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:18.541399+0000 front 2024-04-24T06:56:18.541315+0000 (oldest deadline 2024-04-24T06:56:42.041243+0000) 2024-04-24T06:57:13.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:13 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:13.135+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:28.144247+0000 front 2024-04-24T06:56:28.144309+0000 (oldest deadline 2024-04-24T06:56:54.044306+0000) 2024-04-24T06:57:13.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:13 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:13.135+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6820 osd.7 since back 2024-04-24T06:56:44.446303+0000 front 2024-04-24T06:56:44.446374+0000 (oldest deadline 2024-04-24T06:57:09.746300+0000) 2024-04-24T06:57:13.561 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@osd.10.service' 2024-04-24T06:57:13.602 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T06:57:13.602 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-24T06:57:13.603 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-24T06:57:13.603 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-43e9bad0-0204-11ef-bc93-c7b262605968@osd.11 2024-04-24T06:57:13.911 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:13 smithi137 systemd[1]: Stopping Ceph osd.11 for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:57:14.195 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:13 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:13.910+0000 7f3479eb6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T06:57:14.196 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:13 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:13.910+0000 7f3479eb6700 -1 osd.11 102 *** Got signal Terminated *** 2024-04-24T06:57:14.196 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:13 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:13.910+0000 7f3479eb6700 -1 osd.11 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T06:57:14.196 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:14 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:14.155+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:57:14.196 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:14 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:14.155+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:57:14.196 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:14 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:14.155+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:57:14.196 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:14 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:14.155+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:57:14.196 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:14 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:14.155+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:18.541399+0000 front 2024-04-24T06:56:18.541315+0000 (oldest deadline 2024-04-24T06:56:42.041243+0000) 2024-04-24T06:57:14.196 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:14 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:14.155+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:28.144247+0000 front 2024-04-24T06:56:28.144309+0000 (oldest deadline 2024-04-24T06:56:54.044306+0000) 2024-04-24T06:57:14.197 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:14 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:14.155+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6820 osd.7 since back 2024-04-24T06:56:44.446303+0000 front 2024-04-24T06:56:44.446374+0000 (oldest deadline 2024-04-24T06:57:09.746300+0000) 2024-04-24T06:57:15.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:15 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:15.197+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:57:15.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:15 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:15.197+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:57:15.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:15 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:15.197+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:57:15.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:15 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:15.197+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:57:15.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:15 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:15.197+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:18.541399+0000 front 2024-04-24T06:56:18.541315+0000 (oldest deadline 2024-04-24T06:56:42.041243+0000) 2024-04-24T06:57:15.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:15 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:15.197+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:28.144247+0000 front 2024-04-24T06:56:28.144309+0000 (oldest deadline 2024-04-24T06:56:54.044306+0000) 2024-04-24T06:57:15.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:15 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:15.197+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6820 osd.7 since back 2024-04-24T06:56:44.446303+0000 front 2024-04-24T06:56:44.446374+0000 (oldest deadline 2024-04-24T06:57:09.746300+0000) 2024-04-24T06:57:16.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:16 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:16.211+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:57:16.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:16 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:16.211+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:57:16.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:16 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:16.211+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:57:16.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:16 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:16.211+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:57:16.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:16 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:16.211+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:18.541399+0000 front 2024-04-24T06:56:18.541315+0000 (oldest deadline 2024-04-24T06:56:42.041243+0000) 2024-04-24T06:57:16.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:16 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:16.211+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:28.144247+0000 front 2024-04-24T06:56:28.144309+0000 (oldest deadline 2024-04-24T06:56:54.044306+0000) 2024-04-24T06:57:16.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:16 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:16.211+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6820 osd.7 since back 2024-04-24T06:56:44.446303+0000 front 2024-04-24T06:56:44.446374+0000 (oldest deadline 2024-04-24T06:57:09.746300+0000) 2024-04-24T06:57:17.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:17 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:17.204+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:57:17.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:17 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:17.204+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:57:17.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:17 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:17.204+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:57:17.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:17 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:17.204+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:57:17.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:17 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:17.204+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:18.541399+0000 front 2024-04-24T06:56:18.541315+0000 (oldest deadline 2024-04-24T06:56:42.041243+0000) 2024-04-24T06:57:17.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:17 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:17.204+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:28.144247+0000 front 2024-04-24T06:56:28.144309+0000 (oldest deadline 2024-04-24T06:56:54.044306+0000) 2024-04-24T06:57:17.696 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:17 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:17.204+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6820 osd.7 since back 2024-04-24T06:56:44.446303+0000 front 2024-04-24T06:56:44.446374+0000 (oldest deadline 2024-04-24T06:57:09.746300+0000) 2024-04-24T06:57:18.445 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:18 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:18.155+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-04-24T06:55:44.334819+0000 front 2024-04-24T06:55:44.334682+0000 (oldest deadline 2024-04-24T06:56:09.634659+0000) 2024-04-24T06:57:18.445 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:18 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:18.155+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6804 osd.1 since back 2024-04-24T06:55:54.336015+0000 front 2024-04-24T06:55:54.336052+0000 (oldest deadline 2024-04-24T06:56:18.435994+0000) 2024-04-24T06:57:18.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:18 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:18.155+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6804 osd.2 since back 2024-04-24T06:56:04.038723+0000 front 2024-04-24T06:56:04.038772+0000 (oldest deadline 2024-04-24T06:56:26.338842+0000) 2024-04-24T06:57:18.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:18 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:18.155+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-04-24T06:56:12.140433+0000 front 2024-04-24T06:56:12.140421+0000 (oldest deadline 2024-04-24T06:56:35.040170+0000) 2024-04-24T06:57:18.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:18 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:18.155+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6812 osd.4 since back 2024-04-24T06:56:18.541399+0000 front 2024-04-24T06:56:18.541315+0000 (oldest deadline 2024-04-24T06:56:42.041243+0000) 2024-04-24T06:57:18.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:18 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:18.155+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6812 osd.5 since back 2024-04-24T06:56:28.144247+0000 front 2024-04-24T06:56:28.144309+0000 (oldest deadline 2024-04-24T06:56:54.044306+0000) 2024-04-24T06:57:18.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:18 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:18.155+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.37:6820 osd.7 since back 2024-04-24T06:56:44.446303+0000 front 2024-04-24T06:56:44.446374+0000 (oldest deadline 2024-04-24T06:57:09.746300+0000) 2024-04-24T06:57:18.446 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:18 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11[58613]: 2024-04-24T06:57:18.155+0000 7f34764aa700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.137:6820 osd.8 since back 2024-04-24T06:56:52.047061+0000 front 2024-04-24T06:56:52.047101+0000 (oldest deadline 2024-04-24T06:57:17.347064+0000) 2024-04-24T06:57:19.520 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:19 smithi137 podman[70687]: 2024-04-24 06:57:19.234736442 +0000 UTC m=+5.427195672 container died 74c111b171bc2b7c905c72ab0ed62ef33a37e7de70e66322c917f8125254cb00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T06:57:19.827 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:19 smithi137 podman[70687]: 2024-04-24 06:57:19.594110761 +0000 UTC m=+5.786569995 container cleanup 74c111b171bc2b7c905c72ab0ed62ef33a37e7de70e66322c917f8125254cb00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-24T06:57:19.828 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:19 smithi137 bash[70687]: ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11 2024-04-24T06:57:20.153 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:19 smithi137 podman[70704]: 2024-04-24 06:57:19.828258819 +0000 UTC m=+0.596095897 container remove 74c111b171bc2b7c905c72ab0ed62ef33a37e7de70e66322c917f8125254cb00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-24T06:57:20.423 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:20 smithi137 podman[70792]: 2024-04-24 06:57:20.057144981 +0000 UTC m=+0.018222836 image pull b5c3183062a6ce19f361681cc58de6e3ac1de1a54c80e6094ec5695237cd98d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b 2024-04-24T06:57:20.423 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:20 smithi137 podman[70792]: 2024-04-24 06:57:20.171001989 +0000 UTC m=+0.132079829 container create 0eec45abc277a1d34993d0e75103464e913036f773f59bf47bfd8b56757150c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-24T06:57:20.424 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:20 smithi137 podman[70792]: 2024-04-24 06:57:20.348314561 +0000 UTC m=+0.309392401 container init 0eec45abc277a1d34993d0e75103464e913036f773f59bf47bfd8b56757150c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T06:57:20.424 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:20 smithi137 podman[70792]: 2024-04-24 06:57:20.35135074 +0000 UTC m=+0.312428584 container start 0eec45abc277a1d34993d0e75103464e913036f773f59bf47bfd8b56757150c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, io.buildah.version=1.33.5) 2024-04-24T06:57:20.424 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:20 smithi137 podman[70792]: 2024-04-24 06:57:20.42336065 +0000 UTC m=+0.384438502 container attach 0eec45abc277a1d34993d0e75103464e913036f773f59bf47bfd8b56757150c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T06:57:20.695 INFO:journalctl@ceph.osd.11.smithi137.stdout:Apr 24 06:57:20 smithi137 podman[70792]: 2024-04-24 06:57:20.642011172 +0000 UTC m=+0.603089014 container died 0eec45abc277a1d34993d0e75103464e913036f773f59bf47bfd8b56757150c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-osd-11-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-24T06:57:21.212 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@osd.11.service' 2024-04-24T06:57:21.253 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T06:57:21.253 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-24T06:57:21.254 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-24T06:57:21.254 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mds.a 2024-04-24T06:57:21.755 INFO:journalctl@ceph.mds.a.smithi005.stdout:Apr 24 06:57:21 smithi005 systemd[1]: Stopping Ceph mds.a for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:57:22.255 INFO:journalctl@ceph.mds.a.smithi005.stdout:Apr 24 06:57:21 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-a[65951]: 2024-04-24T06:57:21.846+0000 7f58a5e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T06:57:22.256 INFO:journalctl@ceph.mds.a.smithi005.stdout:Apr 24 06:57:21 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-a[65951]: 2024-04-24T06:57:21.846+0000 7f58a5e00700 -1 mds.a *** got signal Terminated *** 2024-04-24T06:57:25.255 INFO:journalctl@ceph.mds.a.smithi005.stdout:Apr 24 06:57:24 smithi005 podman[98262]: 2024-04-24 06:57:24.952675528 +0000 UTC m=+3.240799466 container died 0acdc9dffbd4732a386dd7d5199d155cd4c529cc7b30f359b684d33f617b93c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-a, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-24T06:57:25.505 INFO:journalctl@ceph.mds.a.smithi005.stdout:Apr 24 06:57:25 smithi005 podman[98262]: 2024-04-24 06:57:25.364188127 +0000 UTC m=+3.652312085 container cleanup 0acdc9dffbd4732a386dd7d5199d155cd4c529cc7b30f359b684d33f617b93c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T06:57:25.506 INFO:journalctl@ceph.mds.a.smithi005.stdout:Apr 24 06:57:25 smithi005 bash[98262]: ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-a 2024-04-24T06:57:26.005 INFO:journalctl@ceph.mds.a.smithi005.stdout:Apr 24 06:57:25 smithi005 podman[98276]: 2024-04-24 06:57:25.615077516 +0000 UTC m=+0.639612225 container remove 0acdc9dffbd4732a386dd7d5199d155cd4c529cc7b30f359b684d33f617b93c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-a, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-24T06:57:26.116 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mds.a.service' 2024-04-24T06:57:26.233 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T06:57:26.233 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-24T06:57:26.233 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-24T06:57:26.233 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mds.d 2024-04-24T06:57:26.755 INFO:journalctl@ceph.mds.d.smithi005.stdout:Apr 24 06:57:26 smithi005 systemd[1]: Stopping Ceph mds.d for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:57:27.255 INFO:journalctl@ceph.mds.d.smithi005.stdout:Apr 24 06:57:26 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-d[64986]: 2024-04-24T06:57:26.925+0000 7f5c73600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T06:57:27.256 INFO:journalctl@ceph.mds.d.smithi005.stdout:Apr 24 06:57:26 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-d[64986]: 2024-04-24T06:57:26.925+0000 7f5c73600700 -1 mds.d *** got signal Terminated *** 2024-04-24T06:57:31.005 INFO:journalctl@ceph.mds.d.smithi005.stdout:Apr 24 06:57:30 smithi005 podman[98380]: 2024-04-24 06:57:30.706392343 +0000 UTC m=+3.984923987 container died 0b6768e60a1130572350dca189046fa20a3709a408071061e4041165a95897fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-24T06:57:31.505 INFO:journalctl@ceph.mds.d.smithi005.stdout:Apr 24 06:57:31 smithi005 podman[98380]: 2024-04-24 06:57:31.250897008 +0000 UTC m=+4.529428680 container cleanup 0b6768e60a1130572350dca189046fa20a3709a408071061e4041165a95897fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-24T06:57:31.506 INFO:journalctl@ceph.mds.d.smithi005.stdout:Apr 24 06:57:31 smithi005 bash[98380]: ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-d 2024-04-24T06:57:32.005 INFO:journalctl@ceph.mds.d.smithi005.stdout:Apr 24 06:57:31 smithi005 podman[98394]: 2024-04-24 06:57:31.536187551 +0000 UTC m=+0.812184514 container remove 0b6768e60a1130572350dca189046fa20a3709a408071061e4041165a95897fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-24T06:57:32.132 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mds.d.service' 2024-04-24T06:57:32.246 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T06:57:32.246 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-24T06:57:32.246 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-24T06:57:32.247 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mds.g 2024-04-24T06:57:32.755 INFO:journalctl@ceph.mds.g.smithi005.stdout:Apr 24 06:57:32 smithi005 systemd[1]: Stopping Ceph mds.g for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:57:33.255 INFO:journalctl@ceph.mds.g.smithi005.stdout:Apr 24 06:57:32 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-g[65603]: 2024-04-24T06:57:32.843+0000 7f059f400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T06:57:33.256 INFO:journalctl@ceph.mds.g.smithi005.stdout:Apr 24 06:57:32 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-g[65603]: 2024-04-24T06:57:32.843+0000 7f059f400700 -1 mds.g *** got signal Terminated *** 2024-04-24T06:57:37.252 INFO:journalctl@ceph.mds.g.smithi005.stdout:Apr 24 06:57:36 smithi005 podman[98501]: 2024-04-24 06:57:36.861957052 +0000 UTC m=+4.149996978 container died fa9159430b1334b7e5c85b6b942065a4c505748ec48fcb97fe125d3de3edef0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-24T06:57:37.505 INFO:journalctl@ceph.mds.g.smithi005.stdout:Apr 24 06:57:37 smithi005 podman[98501]: 2024-04-24 06:57:37.361889148 +0000 UTC m=+4.649929067 container cleanup fa9159430b1334b7e5c85b6b942065a4c505748ec48fcb97fe125d3de3edef0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-g, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.schema-version=1.0) 2024-04-24T06:57:37.505 INFO:journalctl@ceph.mds.g.smithi005.stdout:Apr 24 06:57:37 smithi005 bash[98501]: ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-g 2024-04-24T06:57:37.916 INFO:journalctl@ceph.mds.g.smithi005.stdout:Apr 24 06:57:37 smithi005 podman[98515]: 2024-04-24 06:57:37.679517854 +0000 UTC m=+0.800507994 container remove fa9159430b1334b7e5c85b6b942065a4c505748ec48fcb97fe125d3de3edef0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-g, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22) 2024-04-24T06:57:37.956 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mds.g.service' 2024-04-24T06:57:38.069 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T06:57:38.069 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-24T06:57:38.069 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-24T06:57:38.069 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mds.j 2024-04-24T06:57:38.505 INFO:journalctl@ceph.mds.j.smithi005.stdout:Apr 24 06:57:38 smithi005 systemd[1]: Stopping Ceph mds.j for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:57:39.005 INFO:journalctl@ceph.mds.j.smithi005.stdout:Apr 24 06:57:38 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-j[64623]: 2024-04-24T06:57:38.729+0000 7fc66c000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T06:57:39.006 INFO:journalctl@ceph.mds.j.smithi005.stdout:Apr 24 06:57:38 smithi005 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-j[64623]: 2024-04-24T06:57:38.729+0000 7fc66c000700 -1 mds.j *** got signal Terminated *** 2024-04-24T06:57:42.878 INFO:journalctl@ceph.mds.j.smithi005.stdout:Apr 24 06:57:42 smithi005 podman[98620]: 2024-04-24 06:57:42.519038504 +0000 UTC m=+3.915545753 container died 708d1ca029d015e739334056bcaf017f109dc3a5a96c37bf451378fad15c6011 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-j, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T06:57:43.157 INFO:journalctl@ceph.mds.j.smithi005.stdout:Apr 24 06:57:42 smithi005 podman[98620]: 2024-04-24 06:57:42.990285065 +0000 UTC m=+4.386792307 container cleanup 708d1ca029d015e739334056bcaf017f109dc3a5a96c37bf451378fad15c6011 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-j, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-24T06:57:43.158 INFO:journalctl@ceph.mds.j.smithi005.stdout:Apr 24 06:57:42 smithi005 bash[98620]: ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-j 2024-04-24T06:57:43.505 INFO:journalctl@ceph.mds.j.smithi005.stdout:Apr 24 06:57:43 smithi005 podman[98646]: 2024-04-24 06:57:43.157463031 +0000 UTC m=+0.619852285 container remove 708d1ca029d015e739334056bcaf017f109dc3a5a96c37bf451378fad15c6011 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-j, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-24T06:57:43.714 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mds.j.service' 2024-04-24T06:57:43.831 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T06:57:43.832 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-24T06:57:43.832 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-24T06:57:43.832 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mds.b 2024-04-24T06:57:44.138 INFO:journalctl@ceph.mds.b.smithi037.stdout:Apr 24 06:57:43 smithi037 systemd[1]: Stopping Ceph mds.b for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:57:44.412 INFO:journalctl@ceph.mds.b.smithi037.stdout:Apr 24 06:57:44 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-b[63330]: 2024-04-24T06:57:44.137+0000 7f482e73c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T06:57:44.412 INFO:journalctl@ceph.mds.b.smithi037.stdout:Apr 24 06:57:44 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-b[63330]: 2024-04-24T06:57:44.137+0000 7f482e73c700 -1 mds.b *** got signal Terminated *** 2024-04-24T06:57:48.470 INFO:journalctl@ceph.mds.b.smithi037.stdout:Apr 24 06:57:48 smithi037 podman[69978]: 2024-04-24 06:57:48.151189298 +0000 UTC m=+4.111899061 container died fd4020104cab49cbb86a2c65378dc7b1f489fdddd822a068f201358fc1138541 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-24T06:57:48.470 INFO:journalctl@ceph.mds.b.smithi037.stdout:Apr 24 06:57:48 smithi037 podman[69978]: 2024-04-24 06:57:48.470549975 +0000 UTC m=+4.431259740 container cleanup fd4020104cab49cbb86a2c65378dc7b1f489fdddd822a068f201358fc1138541 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-b, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-24T06:57:48.767 INFO:journalctl@ceph.mds.b.smithi037.stdout:Apr 24 06:57:48 smithi037 bash[69978]: ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-b 2024-04-24T06:57:49.055 INFO:journalctl@ceph.mds.b.smithi037.stdout:Apr 24 06:57:48 smithi037 podman[70004]: 2024-04-24 06:57:48.805097996 +0000 UTC m=+0.654158370 container remove fd4020104cab49cbb86a2c65378dc7b1f489fdddd822a068f201358fc1138541 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-24T06:57:49.059 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mds.b.service' 2024-04-24T06:57:49.096 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T06:57:49.096 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-24T06:57:49.096 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-24T06:57:49.097 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mds.e 2024-04-24T06:57:49.433 INFO:journalctl@ceph.mds.e.smithi037.stdout:Apr 24 06:57:49 smithi037 systemd[1]: Stopping Ceph mds.e for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:57:49.768 INFO:journalctl@ceph.mds.e.smithi037.stdout:Apr 24 06:57:49 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-e[62718]: 2024-04-24T06:57:49.433+0000 7fb4ec851700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T06:57:49.768 INFO:journalctl@ceph.mds.e.smithi037.stdout:Apr 24 06:57:49 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-e[62718]: 2024-04-24T06:57:49.433+0000 7fb4ec851700 -1 mds.e *** got signal Terminated *** 2024-04-24T06:57:53.768 INFO:journalctl@ceph.mds.e.smithi037.stdout:Apr 24 06:57:53 smithi037 podman[70111]: 2024-04-24 06:57:53.445130372 +0000 UTC m=+4.127943375 container died c97d3e315a8d120e55ec98250377cd69c9c49437bee9cb24ebbf33712baaa82b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-e, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T06:57:54.166 INFO:journalctl@ceph.mds.e.smithi037.stdout:Apr 24 06:57:53 smithi037 podman[70111]: 2024-04-24 06:57:53.907586018 +0000 UTC m=+4.590399020 container cleanup c97d3e315a8d120e55ec98250377cd69c9c49437bee9cb24ebbf33712baaa82b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-e, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22) 2024-04-24T06:57:54.167 INFO:journalctl@ceph.mds.e.smithi037.stdout:Apr 24 06:57:53 smithi037 bash[70111]: ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-e 2024-04-24T06:57:54.346 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mds.e.service' 2024-04-24T06:57:54.379 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T06:57:54.379 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-24T06:57:54.380 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-24T06:57:54.380 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mds.h 2024-04-24T06:57:54.768 INFO:journalctl@ceph.mds.h.smithi037.stdout:Apr 24 06:57:54 smithi037 systemd[1]: Stopping Ceph mds.h for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:57:54.768 INFO:journalctl@ceph.mds.h.smithi037.stdout:Apr 24 06:57:54 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-h[63026]: 2024-04-24T06:57:54.685+0000 7fc08deb2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T06:57:54.768 INFO:journalctl@ceph.mds.h.smithi037.stdout:Apr 24 06:57:54 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-h[63026]: 2024-04-24T06:57:54.685+0000 7fc08deb2700 -1 mds.h *** got signal Terminated *** 2024-04-24T06:57:57.453 INFO:journalctl@ceph.mds.h.smithi037.stdout:Apr 24 06:57:57 smithi037 podman[70231]: 2024-04-24 06:57:57.082734021 +0000 UTC m=+2.487426241 container died 3aaa91ad501856d64c05aa1dff7ed8447b6bb4a0d11259c63183bc608bb1e7a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-h, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-24T06:57:57.768 INFO:journalctl@ceph.mds.h.smithi037.stdout:Apr 24 06:57:57 smithi037 podman[70231]: 2024-04-24 06:57:57.544329075 +0000 UTC m=+2.949021311 container cleanup 3aaa91ad501856d64c05aa1dff7ed8447b6bb4a0d11259c63183bc608bb1e7a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-h, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T06:57:57.768 INFO:journalctl@ceph.mds.h.smithi037.stdout:Apr 24 06:57:57 smithi037 bash[70231]: ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-h 2024-04-24T06:57:57.930 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mds.h.service' 2024-04-24T06:57:57.962 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T06:57:57.963 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-24T06:57:57.963 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-24T06:57:57.963 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mds.k 2024-04-24T06:57:58.268 INFO:journalctl@ceph.mds.k.smithi037.stdout:Apr 24 06:57:58 smithi037 systemd[1]: Stopping Ceph mds.k for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:57:58.768 INFO:journalctl@ceph.mds.k.smithi037.stdout:Apr 24 06:57:58 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-k[62397]: 2024-04-24T06:57:58.314+0000 7f70a25af700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T06:57:58.768 INFO:journalctl@ceph.mds.k.smithi037.stdout:Apr 24 06:57:58 smithi037 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-k[62397]: 2024-04-24T06:57:58.314+0000 7f70a25af700 -1 mds.k *** got signal Terminated *** 2024-04-24T06:58:02.598 INFO:journalctl@ceph.mds.k.smithi037.stdout:Apr 24 06:58:02 smithi037 podman[70351]: 2024-04-24 06:58:02.323438585 +0000 UTC m=+4.137066625 container died 2d48a66b627449c18f74d743cb990ee408062e5b8fdc9497d95fccd9a1150bc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-k, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22) 2024-04-24T06:58:02.957 INFO:journalctl@ceph.mds.k.smithi037.stdout:Apr 24 06:58:02 smithi037 podman[70351]: 2024-04-24 06:58:02.68141776 +0000 UTC m=+4.495045792 container cleanup 2d48a66b627449c18f74d743cb990ee408062e5b8fdc9497d95fccd9a1150bc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-k, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T06:58:02.957 INFO:journalctl@ceph.mds.k.smithi037.stdout:Apr 24 06:58:02 smithi037 bash[70351]: ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-k 2024-04-24T06:58:03.115 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mds.k.service' 2024-04-24T06:58:03.147 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T06:58:03.148 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-24T06:58:03.148 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-24T06:58:03.148 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mds.c 2024-04-24T06:58:03.442 INFO:journalctl@ceph.mds.c.smithi137.stdout:Apr 24 06:58:03 smithi137 systemd[1]: Stopping Ceph mds.c for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:58:03.695 INFO:journalctl@ceph.mds.c.smithi137.stdout:Apr 24 06:58:03 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-c[63762]: 2024-04-24T06:58:03.441+0000 7f7d3e6fd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T06:58:03.695 INFO:journalctl@ceph.mds.c.smithi137.stdout:Apr 24 06:58:03 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-c[63762]: 2024-04-24T06:58:03.441+0000 7f7d3e6fd700 -1 mds.c *** got signal Terminated *** 2024-04-24T06:58:05.445 INFO:journalctl@ceph.mds.c.smithi137.stdout:Apr 24 06:58:05 smithi137 podman[70897]: 2024-04-24 06:58:05.100175869 +0000 UTC m=+1.762669786 container died 2a46530d84e88b5e61af5ee6d1dadae4a3809f4476b403f81b1c4422658bec37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-c, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-24T06:58:05.945 INFO:journalctl@ceph.mds.c.smithi137.stdout:Apr 24 06:58:05 smithi137 podman[70897]: 2024-04-24 06:58:05.675119575 +0000 UTC m=+2.337613535 container cleanup 2a46530d84e88b5e61af5ee6d1dadae4a3809f4476b403f81b1c4422658bec37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-c, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.build-date=20240416) 2024-04-24T06:58:05.945 INFO:journalctl@ceph.mds.c.smithi137.stdout:Apr 24 06:58:05 smithi137 bash[70897]: ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-c 2024-04-24T06:58:06.101 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mds.c.service' 2024-04-24T06:58:06.137 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T06:58:06.137 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-24T06:58:06.138 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-24T06:58:06.138 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mds.f 2024-04-24T06:58:06.695 INFO:journalctl@ceph.mds.f.smithi137.stdout:Apr 24 06:58:06 smithi137 systemd[1]: Stopping Ceph mds.f for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:58:06.696 INFO:journalctl@ceph.mds.f.smithi137.stdout:Apr 24 06:58:06 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-f[63455]: 2024-04-24T06:58:06.445+0000 7fc0df241700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T06:58:06.696 INFO:journalctl@ceph.mds.f.smithi137.stdout:Apr 24 06:58:06 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-f[63455]: 2024-04-24T06:58:06.445+0000 7fc0df241700 -1 mds.f *** got signal Terminated *** 2024-04-24T06:58:09.362 INFO:journalctl@ceph.mds.f.smithi137.stdout:Apr 24 06:58:09 smithi137 podman[71029]: 2024-04-24 06:58:09.089178204 +0000 UTC m=+2.738199674 container died 8a32b4df1c7fbdbbde4a59fa66064e7bad0177a0e1fe0a7f38dbbbc858413493 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-f, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-24T06:58:09.695 INFO:journalctl@ceph.mds.f.smithi137.stdout:Apr 24 06:58:09 smithi137 podman[71029]: 2024-04-24 06:58:09.453613671 +0000 UTC m=+3.102635144 container cleanup 8a32b4df1c7fbdbbde4a59fa66064e7bad0177a0e1fe0a7f38dbbbc858413493 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-f, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.vendor=CentOS) 2024-04-24T06:58:09.695 INFO:journalctl@ceph.mds.f.smithi137.stdout:Apr 24 06:58:09 smithi137 bash[71029]: ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-f 2024-04-24T06:58:09.874 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mds.f.service' 2024-04-24T06:58:09.906 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T06:58:09.906 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-24T06:58:09.906 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-24T06:58:09.906 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mds.i 2024-04-24T06:58:10.445 INFO:journalctl@ceph.mds.i.smithi137.stdout:Apr 24 06:58:09 smithi137 systemd[1]: Stopping Ceph mds.i for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:58:10.445 INFO:journalctl@ceph.mds.i.smithi137.stdout:Apr 24 06:58:10 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-i[64073]: 2024-04-24T06:58:10.215+0000 7fdc8f908700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T06:58:10.445 INFO:journalctl@ceph.mds.i.smithi137.stdout:Apr 24 06:58:10 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-i[64073]: 2024-04-24T06:58:10.215+0000 7fdc8f908700 -1 mds.i *** got signal Terminated *** 2024-04-24T06:58:13.357 INFO:journalctl@ceph.mds.i.smithi137.stdout:Apr 24 06:58:13 smithi137 podman[71150]: 2024-04-24 06:58:13.096252901 +0000 UTC m=+2.977093862 container died 65c868131cb6f58b1c9a8151791d0cfc06299b5bdb93ac6312e7824e3f6d6e29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-i, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T06:58:13.617 INFO:journalctl@ceph.mds.i.smithi137.stdout:Apr 24 06:58:13 smithi137 podman[71150]: 2024-04-24 06:58:13.424037401 +0000 UTC m=+3.304878362 container cleanup 65c868131cb6f58b1c9a8151791d0cfc06299b5bdb93ac6312e7824e3f6d6e29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-i, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-24T06:58:13.617 INFO:journalctl@ceph.mds.i.smithi137.stdout:Apr 24 06:58:13 smithi137 bash[71150]: ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-i 2024-04-24T06:58:13.847 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mds.i.service' 2024-04-24T06:58:13.867 INFO:journalctl@ceph.mds.i.smithi137.stdout:Apr 24 06:58:13 smithi137 podman[71164]: 2024-04-24 06:58:13.617532304 +0000 UTC m=+0.519162191 container remove 65c868131cb6f58b1c9a8151791d0cfc06299b5bdb93ac6312e7824e3f6d6e29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-i, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-24T06:58:13.867 INFO:journalctl@ceph.mds.i.smithi137.stdout:Apr 24 06:58:13 smithi137 systemd[1]: ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mds.i.service: Deactivated successfully. 2024-04-24T06:58:13.868 INFO:journalctl@ceph.mds.i.smithi137.stdout:Apr 24 06:58:13 smithi137 systemd[1]: Stopped Ceph mds.i for 43e9bad0-0204-11ef-bc93-c7b262605968. 2024-04-24T06:58:13.868 INFO:journalctl@ceph.mds.i.smithi137.stdout:Apr 24 06:58:13 smithi137 systemd[1]: ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mds.i.service: Consumed 9.481s CPU time. 2024-04-24T06:58:13.878 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T06:58:13.878 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-24T06:58:13.878 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-24T06:58:13.879 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mds.l 2024-04-24T06:58:14.130 INFO:journalctl@ceph.mds.l.smithi137.stdout:Apr 24 06:58:13 smithi137 systemd[1]: Stopping Ceph mds.l for 43e9bad0-0204-11ef-bc93-c7b262605968... 2024-04-24T06:58:14.445 INFO:journalctl@ceph.mds.l.smithi137.stdout:Apr 24 06:58:14 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-l[63142]: 2024-04-24T06:58:14.202+0000 7f551a55f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T06:58:14.445 INFO:journalctl@ceph.mds.l.smithi137.stdout:Apr 24 06:58:14 smithi137 ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-l[63142]: 2024-04-24T06:58:14.202+0000 7f551a55f700 -1 mds.l *** got signal Terminated *** 2024-04-24T06:58:15.761 INFO:journalctl@ceph.mds.l.smithi137.stdout:Apr 24 06:58:15 smithi137 podman[71271]: 2024-04-24 06:58:15.509944981 +0000 UTC m=+1.422062087 container died 1efb8334f844a2db7cce32fb5355cac7655458b103a29a53c92cd07aa1314a4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-l, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.vendor=CentOS) 2024-04-24T06:58:16.020 INFO:journalctl@ceph.mds.l.smithi137.stdout:Apr 24 06:58:15 smithi137 podman[71271]: 2024-04-24 06:58:15.844767714 +0000 UTC m=+1.756884822 container cleanup 1efb8334f844a2db7cce32fb5355cac7655458b103a29a53c92cd07aa1314a4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T06:58:16.020 INFO:journalctl@ceph.mds.l.smithi137.stdout:Apr 24 06:58:15 smithi137 bash[71271]: ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-l 2024-04-24T06:58:16.270 INFO:journalctl@ceph.mds.l.smithi137.stdout:Apr 24 06:58:16 smithi137 podman[71285]: 2024-04-24 06:58:16.020820959 +0000 UTC m=+0.506057473 container remove 1efb8334f844a2db7cce32fb5355cac7655458b103a29a53c92cd07aa1314a4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1636720154c5f9def661695c77cfc42ee0cd621ee831ea25094b371055ce928b, name=ceph-43e9bad0-0204-11ef-bc93-c7b262605968-mds-l, RELEASE=wip-rishabh-testing-20240416.193735-5-a3c8a22, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416) 2024-04-24T06:58:16.270 INFO:journalctl@ceph.mds.l.smithi137.stdout:Apr 24 06:58:16 smithi137 systemd[1]: ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mds.l.service: Deactivated successfully. 2024-04-24T06:58:16.273 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-43e9bad0-0204-11ef-bc93-c7b262605968@mds.l.service' 2024-04-24T06:58:16.303 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T06:58:16.303 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-24T06:58:16.304 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-24T06:58:16.639 INFO:teuthology.orchestra.run.smithi005.stdout:Deleting cluster with fsid: 43e9bad0-0204-11ef-bc93-c7b262605968 2024-04-24T06:58:21.751 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-24T06:58:21.950 INFO:teuthology.orchestra.run.smithi037.stdout:Deleting cluster with fsid: 43e9bad0-0204-11ef-bc93-c7b262605968 2024-04-24T06:58:24.459 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-24T06:58:24.650 INFO:teuthology.orchestra.run.smithi137.stdout:Deleting cluster with fsid: 43e9bad0-0204-11ef-bc93-c7b262605968 2024-04-24T06:58:26.965 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T06:58:27.065 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T06:58:27.098 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T06:58:27.123 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-24T06:58:27.125 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi/7671165/remote/smithi005/crash 2024-04-24T06:58:27.127 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/crash -- . 2024-04-24T06:58:27.230 INFO:teuthology.orchestra.run.smithi005.stderr:tar: /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-24T06:58:27.231 INFO:teuthology.orchestra.run.smithi005.stderr:tar: Error is not recoverable: exiting now 2024-04-24T06:58:27.235 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi/7671165/remote/smithi037/crash 2024-04-24T06:58:27.236 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/crash -- . 2024-04-24T06:58:27.275 INFO:teuthology.orchestra.run.smithi037.stderr:tar: /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-24T06:58:27.275 INFO:teuthology.orchestra.run.smithi037.stderr:tar: Error is not recoverable: exiting now 2024-04-24T06:58:27.277 DEBUG:teuthology.misc:Transferring archived files from smithi137:/var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi/7671165/remote/smithi137/crash 2024-04-24T06:58:27.278 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/crash -- . 2024-04-24T06:58:27.330 INFO:teuthology.orchestra.run.smithi137.stderr:tar: /var/lib/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-24T06:58:27.331 INFO:teuthology.orchestra.run.smithi137.stderr:tar: Error is not recoverable: exiting now 2024-04-24T06:58:27.332 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-24T06:58:27.332 DEBUG:teuthology.orchestra.run.smithi005:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-24T06:58:27.442 INFO:tasks.cephadm:Compressing logs... 2024-04-24T06:58:27.442 DEBUG:teuthology.orchestra.run.smithi005:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T06:58:27.446 DEBUG:teuthology.orchestra.run.smithi037:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T06:58:27.449 DEBUG:teuthology.orchestra.run.smithi137:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T06:58:27.472 INFO:teuthology.orchestra.run.smithi137.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-24T06:58:27.473 INFO:teuthology.orchestra.run.smithi137.stderr:: No such file or directory 2024-04-24T06:58:27.473 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph.log 2024-04-24T06:58:27.473 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-24T06:58:27.473 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/cephadm.log: gzip/var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph.log: -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-24T06:58:27.473 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-24T06:58:27.473 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-24T06:58:27.475 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mds.f.log: /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-24T06:58:27.475 INFO:teuthology.orchestra.run.smithi137.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-24T06:58:27.475 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-24T06:58:27.480 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-24T06:58:27.481 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-24T06:58:27.481 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mgr.z.log: 87.5% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-24T06:58:27.481 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-24T06:58:27.481 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mds.i.log: 90.3% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-24T06:58:27.485 INFO:teuthology.orchestra.run.smithi037.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-24T06:58:27.485 INFO:teuthology.orchestra.run.smithi037.stderr:: No such file or directory 2024-04-24T06:58:27.485 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph.log 2024-04-24T06:58:27.485 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-24T06:58:27.485 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-24T06:58:27.485 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-24T06:58:27.486 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-24T06:58:27.486 INFO:teuthology.orchestra.run.smithi037.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-24T06:58:27.486 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-24T06:58:27.486 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-24T06:58:27.486 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-24T06:58:27.486 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-24T06:58:27.486 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-24T06:58:27.486 INFO:teuthology.orchestra.run.smithi037.stderr: 90.3% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-24T06:58:27.487 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-24T06:58:27.487 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-24T06:58:27.487 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.5% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-24T06:58:27.487 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-24T06:58:27.489 INFO:teuthology.orchestra.run.smithi137.stderr: 92.0% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-24T06:58:27.489 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-24T06:58:27.489 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-24T06:58:27.490 INFO:teuthology.orchestra.run.smithi137.stderr: 83.5% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-24T06:58:27.492 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mds.e.log: 94.4% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-24T06:58:27.492 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-24T06:58:27.492 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-volume.log: 91.9% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-24T06:58:27.492 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-24T06:58:27.499 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-24T06:58:27.501 INFO:teuthology.orchestra.run.smithi005.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-24T06:58:27.501 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-24T06:58:27.502 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph.log 2024-04-24T06:58:27.503 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-24T06:58:27.504 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mon.b.log: /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mds.k.log: 91.0% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-24T06:58:27.504 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-24T06:58:27.505 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-24T06:58:27.506 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-24T06:58:27.507 INFO:teuthology.orchestra.run.smithi005.stderr: 87.5% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-24T06:58:27.507 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-24T06:58:27.508 INFO:teuthology.orchestra.run.smithi005.stderr: 90.2% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-24T06:58:27.508 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-24T06:58:27.510 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-24T06:58:27.511 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-24T06:58:27.512 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mgr.x.log: 93.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-24T06:58:27.512 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-24T06:58:27.513 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-24T06:58:27.513 INFO:teuthology.orchestra.run.smithi005.stderr: 94.3% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-24T06:58:27.514 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-24T06:58:27.515 INFO:teuthology.orchestra.run.smithi037.stderr: 94.5% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-24T06:58:27.516 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph.cephadm.log: 84.0% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-24T06:58:27.518 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-24T06:58:27.519 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-24T06:58:27.521 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-24T06:58:27.531 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mds.d.log: /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-osd.9.log: 94.4% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-24T06:58:27.532 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mds.l.log: 94.5% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-24T06:58:27.532 INFO:teuthology.orchestra.run.smithi005.stderr: 94.4% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-24T06:58:27.536 INFO:teuthology.orchestra.run.smithi005.stderr: 90.9% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-24T06:58:27.546 INFO:teuthology.orchestra.run.smithi137.stderr: 91.0% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-24T06:58:27.581 INFO:teuthology.orchestra.run.smithi037.stderr: 94.3% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-24T06:58:27.739 INFO:teuthology.orchestra.run.smithi137.stderr: 95.0% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-24T06:58:27.923 INFO:teuthology.orchestra.run.smithi005.stderr: 90.1% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-24T06:58:27.986 INFO:teuthology.orchestra.run.smithi137.stderr: 91.8% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-24T06:58:28.124 INFO:teuthology.orchestra.run.smithi037.stderr: 91.6% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-24T06:58:28.136 INFO:teuthology.orchestra.run.smithi005.stderr: 94.3% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-24T06:58:28.372 INFO:teuthology.orchestra.run.smithi037.stderr: 93.6% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-24T06:58:28.633 INFO:teuthology.orchestra.run.smithi005.stderr: 93.7% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-24T06:58:28.664 INFO:teuthology.orchestra.run.smithi137.stderr: 95.3% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-24T06:58:28.798 INFO:teuthology.orchestra.run.smithi005.stderr: 90.5% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-24T06:58:28.968 INFO:teuthology.orchestra.run.smithi037.stderr: 94.2% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-24T06:58:29.021 INFO:teuthology.orchestra.run.smithi137.stderr: 93.9% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-24T06:58:29.021 INFO:teuthology.orchestra.run.smithi037.stderr: 93.7% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-24T06:58:29.030 INFO:teuthology.orchestra.run.smithi137.stderr: 93.8% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-24T06:58:29.043 INFO:teuthology.orchestra.run.smithi037.stderr: 94.0% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-24T06:58:29.068 INFO:teuthology.orchestra.run.smithi005.stderr: 93.9% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-24T06:58:29.101 INFO:teuthology.orchestra.run.smithi137.stderr: 94.1% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-24T06:58:29.122 INFO:teuthology.orchestra.run.smithi005.stderr: 93.9% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-24T06:58:29.188 INFO:teuthology.orchestra.run.smithi137.stderr: 93.7% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-24T06:58:29.294 INFO:teuthology.orchestra.run.smithi005.stderr: 94.0% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-24T06:58:29.298 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-04-24T06:58:29.298 INFO:teuthology.orchestra.run.smithi005.stderr:real 0m1.820s 2024-04-24T06:58:29.298 INFO:teuthology.orchestra.run.smithi005.stderr:user 0m7.849s 2024-04-24T06:58:29.298 INFO:teuthology.orchestra.run.smithi005.stderr:sys 0m0.665s 2024-04-24T06:58:29.835 INFO:teuthology.orchestra.run.smithi137.stderr: 94.2% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-24T06:58:29.837 INFO:teuthology.orchestra.run.smithi137.stderr: 2024-04-24T06:58:29.837 INFO:teuthology.orchestra.run.smithi137.stderr:real 0m2.374s 2024-04-24T06:58:29.837 INFO:teuthology.orchestra.run.smithi137.stderr:user 0m10.390s 2024-04-24T06:58:29.837 INFO:teuthology.orchestra.run.smithi137.stderr:sys 0m0.367s 2024-04-24T06:58:44.471 INFO:teuthology.orchestra.run.smithi037.stderr: 93.6% -- replaced with /var/log/ceph/43e9bad0-0204-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-24T06:58:44.473 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-24T06:58:44.473 INFO:teuthology.orchestra.run.smithi037.stderr:real 0m17.008s 2024-04-24T06:58:44.473 INFO:teuthology.orchestra.run.smithi037.stderr:user 0m22.522s 2024-04-24T06:58:44.474 INFO:teuthology.orchestra.run.smithi037.stderr:sys 0m0.727s 2024-04-24T06:58:44.474 INFO:tasks.cephadm:Archiving logs... 2024-04-24T06:58:44.475 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi/7671165/remote/smithi005/log 2024-04-24T06:58:44.476 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T06:58:45.548 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi/7671165/remote/smithi037/log 2024-04-24T06:58:45.549 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T06:58:48.723 DEBUG:teuthology.misc:Transferring archived files from smithi137:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi/7671165/remote/smithi137/log 2024-04-24T06:58:48.724 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T06:58:50.008 INFO:tasks.cephadm:Removing cluster... 2024-04-24T06:58:50.008 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 --force 2024-04-24T06:58:50.344 INFO:teuthology.orchestra.run.smithi005.stdout:Deleting cluster with fsid: 43e9bad0-0204-11ef-bc93-c7b262605968 2024-04-24T06:58:50.986 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 --force 2024-04-24T06:58:51.186 INFO:teuthology.orchestra.run.smithi037.stdout:Deleting cluster with fsid: 43e9bad0-0204-11ef-bc93-c7b262605968 2024-04-24T06:58:51.473 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 43e9bad0-0204-11ef-bc93-c7b262605968 --force 2024-04-24T06:58:51.664 INFO:teuthology.orchestra.run.smithi137.stdout:Deleting cluster with fsid: 43e9bad0-0204-11ef-bc93-c7b262605968 2024-04-24T06:58:51.921 INFO:tasks.cephadm:Removing cephadm ... 2024-04-24T06:58:51.922 DEBUG:teuthology.orchestra.run.smithi005:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-24T06:58:51.978 DEBUG:teuthology.orchestra.run.smithi037:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-24T06:58:51.996 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-24T06:58:52.011 INFO:tasks.cephadm:Teardown complete 2024-04-24T06:58:52.012 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-24T06:58:52.031 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-24T06:58:52.031 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T06:58:52.040 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T06:58:52.042 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T06:58:52.186 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-24T06:58:52.197 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-24T06:58:52.197 DEBUG:teuthology.orchestra.run.smithi005:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T06:58:52.205 DEBUG:teuthology.orchestra.run.smithi037:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T06:58:52.207 DEBUG:teuthology.orchestra.run.smithi137:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T06:58:52.221 INFO:teuthology.orchestra.run.smithi137.stderr:bash: line 1: ntpq: command not found 2024-04-24T06:58:52.222 INFO:teuthology.orchestra.run.smithi037.stderr:bash: line 1: ntpq: command not found 2024-04-24T06:58:52.227 INFO:teuthology.orchestra.run.smithi037.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T06:58:52.227 INFO:teuthology.orchestra.run.smithi037.stdout:=============================================================================== 2024-04-24T06:58:52.227 INFO:teuthology.orchestra.run.smithi037.stdout:^- hv01.front.sepia.ceph.com 2 6 377 52 +503us[ +512us] +/- 579ms 2024-04-24T06:58:52.227 INFO:teuthology.orchestra.run.smithi037.stdout:^* hv02.front.sepia.ceph.com 3 6 377 47 -314us[ -304us] +/- 60ms 2024-04-24T06:58:52.227 INFO:teuthology.orchestra.run.smithi037.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 +414us[ +423us] +/- 79ms 2024-04-24T06:58:52.227 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T06:58:52.228 INFO:teuthology.orchestra.run.smithi137.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T06:58:52.228 INFO:teuthology.orchestra.run.smithi137.stdout:=============================================================================== 2024-04-24T06:58:52.228 INFO:teuthology.orchestra.run.smithi137.stdout:^- hv01.front.sepia.ceph.com 2 6 377 48 +497us[ +497us] +/- 579ms 2024-04-24T06:58:52.228 INFO:teuthology.orchestra.run.smithi137.stdout:^* hv02.front.sepia.ceph.com 3 6 377 52 -328us[ -320us] +/- 60ms 2024-04-24T06:58:52.228 INFO:teuthology.orchestra.run.smithi137.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 48 +417us[ +417us] +/- 79ms 2024-04-24T06:58:52.228 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T06:58:52.253 INFO:teuthology.orchestra.run.smithi005.stderr:bash: line 1: ntpq: command not found 2024-04-24T06:58:52.264 INFO:teuthology.orchestra.run.smithi005.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T06:58:52.264 INFO:teuthology.orchestra.run.smithi005.stdout:=============================================================================== 2024-04-24T06:58:52.264 INFO:teuthology.orchestra.run.smithi005.stdout:^- hv01.front.sepia.ceph.com 2 6 377 48 +499us[ +499us] +/- 579ms 2024-04-24T06:58:52.264 INFO:teuthology.orchestra.run.smithi005.stdout:^* hv02.front.sepia.ceph.com 3 6 377 49 -309us[ -296us] +/- 60ms 2024-04-24T06:58:52.264 INFO:teuthology.orchestra.run.smithi005.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 50 +431us[ +444us] +/- 79ms 2024-04-24T06:58:52.265 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T06:58:52.266 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-24T06:58:52.279 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-24T06:58:52.280 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-24T06:58:52.292 DEBUG:teuthology.orchestra.run.smithi005:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T06:58:52.498 DEBUG:teuthology.orchestra.run.smithi037:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T06:58:52.572 DEBUG:teuthology.orchestra.run.smithi137:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T06:58:52.637 DEBUG:teuthology.orchestra.run.smithi005:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T06:58:52.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:58:52.737 DEBUG:teuthology.orchestra.run.smithi037:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T06:58:52.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:58:52.768 DEBUG:teuthology.orchestra.run.smithi137:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T06:58:52.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:58:52.795 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-24T06:58:52.806 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-24T06:58:52.818 INFO:teuthology.task.internal:Duration was 2117.897934 seconds 2024-04-24T06:58:52.819 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-24T06:58:52.830 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-24T06:58:52.831 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T06:58:52.838 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T06:58:52.841 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T06:58:52.877 INFO:teuthology.orchestra.run.smithi137.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T06:58:52.881 INFO:teuthology.orchestra.run.smithi037.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T06:58:52.988 INFO:teuthology.orchestra.run.smithi005.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T06:58:53.595 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-24T06:58:53.595 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi005.front.sepia.ceph.com 2024-04-24T06:58:53.595 DEBUG:teuthology.orchestra.run.smithi005:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T06:58:53.663 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi037.front.sepia.ceph.com 2024-04-24T06:58:53.663 DEBUG:teuthology.orchestra.run.smithi037:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T06:58:53.694 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi137.front.sepia.ceph.com 2024-04-24T06:58:53.695 DEBUG:teuthology.orchestra.run.smithi137:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T06:58:53.723 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-24T06:58:53.723 DEBUG:teuthology.orchestra.run.smithi005:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T06:58:53.731 DEBUG:teuthology.orchestra.run.smithi037:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T06:58:53.736 DEBUG:teuthology.orchestra.run.smithi137:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T06:58:53.817 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-24T06:58:53.817 DEBUG:teuthology.orchestra.run.smithi005:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T06:58:53.823 DEBUG:teuthology.orchestra.run.smithi037:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T06:58:53.825 DEBUG:teuthology.orchestra.run.smithi137:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T06:58:54.329 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-24T06:58:54.341 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-24T06:58:54.342 DEBUG:teuthology.orchestra.run.smithi005:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T06:58:54.433 DEBUG:teuthology.orchestra.run.smithi037:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T06:58:54.457 DEBUG:teuthology.orchestra.run.smithi137:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T06:58:54.483 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-24T06:58:54.495 DEBUG:teuthology.orchestra.run.smithi005:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T06:58:54.503 DEBUG:teuthology.orchestra.run.smithi037:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T06:58:54.505 DEBUG:teuthology.orchestra.run.smithi137:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T06:58:54.527 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern = core 2024-04-24T06:58:54.546 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern = core 2024-04-24T06:58:54.600 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern = core 2024-04-24T06:58:54.645 DEBUG:teuthology.orchestra.run.smithi005:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T06:58:54.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:58:54.697 DEBUG:teuthology.orchestra.run.smithi037:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T06:58:54.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:58:54.712 DEBUG:teuthology.orchestra.run.smithi137:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T06:58:54.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T06:58:54.726 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-24T06:58:54.739 INFO:teuthology.task.internal:Transferring archived files... 2024-04-24T06:58:54.740 DEBUG:teuthology.misc:Transferring archived files from smithi005:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi/7671165/remote/smithi005 2024-04-24T06:58:54.740 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T06:58:54.845 DEBUG:teuthology.misc:Transferring archived files from smithi037:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi/7671165/remote/smithi037 2024-04-24T06:58:54.845 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T06:58:54.883 DEBUG:teuthology.misc:Transferring archived files from smithi137:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi/7671165/remote/smithi137 2024-04-24T06:58:54.883 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T06:58:54.918 INFO:teuthology.task.internal:Removing archive directory... 2024-04-24T06:58:54.919 DEBUG:teuthology.orchestra.run.smithi005:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T06:58:54.926 DEBUG:teuthology.orchestra.run.smithi037:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T06:58:54.929 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T06:58:54.981 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-24T06:58:54.996 INFO:teuthology.task.internal:Not uploading archives. 2024-04-24T06:58:54.996 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-24T06:58:55.011 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-24T06:58:55.011 DEBUG:teuthology.orchestra.run.smithi005:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T06:58:55.017 DEBUG:teuthology.orchestra.run.smithi037:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T06:58:55.020 DEBUG:teuthology.orchestra.run.smithi137:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T06:58:55.032 INFO:teuthology.orchestra.run.smithi037.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 24 06:58 /home/ubuntu/cephtest 2024-04-24T06:58:55.034 INFO:teuthology.orchestra.run.smithi137.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 24 06:58 /home/ubuntu/cephtest 2024-04-24T06:58:55.067 INFO:teuthology.orchestra.run.smithi005.stdout: 393249 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 24 06:58 /home/ubuntu/cephtest 2024-04-24T06:58:55.071 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-24T06:58:55.086 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-24T06:58:55.119 INFO:teuthology.nuke:Checking targets against current locks 2024-04-24T06:58:55.144 DEBUG:teuthology.nuke:shortname: smithi005 2024-04-24T06:58:55.145 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T06:58:55.170 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi005.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi/7671165', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 06:13:56.896466', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKj7GNK3Wtmq8arxDItaws5Fl+9upi47CoNcM8UMyv5nceA/3uz7NO6nwS5IYipSqdXOoBlY8t46ktjkKGOy78I='} 2024-04-24T06:58:55.174 DEBUG:teuthology.nuke:shortname: smithi037 2024-04-24T06:58:55.175 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T06:58:55.202 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi037.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi/7671165', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 06:13:56.895443', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFopbXrlRFxdN2HulQsNhehpuLGUo6Q2jl3w3Fvt0lxFADrcnB34mbgljrKTNngb6CEPVB4DXGLsF1NTEtdwAa8='} 2024-04-24T06:58:55.203 DEBUG:teuthology.nuke:shortname: smithi137 2024-04-24T06:58:55.203 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T06:58:55.229 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi137.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-24_05:22:11-fs-wip-rishabh-testing-20240416.193735-5-testing-default-smithi/7671165', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 06:13:56.894206', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJl4eVXNbwEo5ONmUM8Y0T1gP+7S6UE6oT+9urDJzP5PYsoEmXEhTzARVOlzvelG1eOvuEqctqarOnf0ZkgHpss='} 2024-04-24T06:58:55.238 INFO:teuthology.orchestra.console.smithi005:Power off 2024-04-24T06:58:55.239 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-24T06:58:55.256 INFO:teuthology.orchestra.console.smithi037:Power off 2024-04-24T06:58:55.257 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-24T06:58:55.272 DEBUG:teuthology.orchestra.console.smithi005:power off output: Chassis Power Control: Down/Off 2024-04-24T06:58:55.272 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T06:58:55.283 DEBUG:teuthology.orchestra.console.smithi037:power off output: Chassis Power Control: Down/Off 2024-04-24T06:58:55.284 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T06:58:55.297 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is on 2024-04-24T06:58:55.308 INFO:teuthology.orchestra.console.smithi137:Power off 2024-04-24T06:58:55.309 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-24T06:58:55.319 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is on 2024-04-24T06:58:55.333 DEBUG:teuthology.orchestra.console.smithi137:power off output: Chassis Power Control: Down/Off 2024-04-24T06:58:55.333 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T06:58:55.357 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is on 2024-04-24T06:58:59.299 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T06:58:59.320 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T06:58:59.358 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T06:58:59.410 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is on 2024-04-24T06:58:59.430 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is on 2024-04-24T06:58:59.469 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is on 2024-04-24T06:59:03.415 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T06:59:03.431 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T06:59:03.470 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T06:59:03.526 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is off 2024-04-24T06:59:03.541 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is off 2024-04-24T06:59:03.580 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is off 2024-04-24T06:59:03.628 INFO:teuthology.orchestra.console.smithi005:Power off completed 2024-04-24T06:59:03.641 INFO:teuthology.orchestra.console.smithi037:Power off completed 2024-04-24T06:59:03.681 INFO:teuthology.orchestra.console.smithi137:Power off completed 2024-04-24T06:59:03.788 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/fsx}} duration: 2117.897933959961 failure_reason: 'Command failed (workunit test suites/fsx.sh) on smithi005 with status 2: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a3c8a22bd5efe18ab230a8e2d072db8f381a0411 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh''' flavor: default owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=01cb0e02457d45849ab5f39f1f15e2e1 status: fail success: false 2024-04-24T06:59:03.788 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T06:59:03.882 INFO:teuthology.run:FAIL